- Program evaluation is a systematic method
- There is no reason to collect and analyze information if the results are not going to be used to make changes.
- Be mindful of creating and sustaining an effective feedback loop
- Program evaluation can help to determine the value, effectiveness and impact of educational programs and identify program areas for improvement, replacement or further development
- Formative evaluation, also known as process evaluation provides information on how well an individual or program is progressing toward a goal. This information is then "fed back" to improve/enhance progress and can allow for just in time corrections
- Summative evaluation is used to determine if pre-defined outcomes have been met. This information is often used to make a "final decision".
- Include evaluation processes in the initial design of any program
What is Program Evaluation? A Brief Introduction. Arbor Educational & Clinical Consulting Inc. 5 minutes.
Robert Fischer Discusses Program Evaluation. FC Cleveland. 7 minutes.
♥Cook DA. Twelve tips for evaluating educational programs. Med Teach. 2010;32(4):296-301. doi: 10.3109/01421590903480121. PubMed PMID: 20353325.
Schuwirth LW, van der Vleuten CP. General overview of the theories used in assessment: AMEE Guide No. 57. Med Teach. 2011;33(10):783-97. doi: 10.3109/0142159X.2011.611022. Review. PubMed PMID: 21942477.
Musick DW. A conceptual model for program evaluation in graduate medical education. Acad Med. 2006 Aug;81(8):759-65. Review. PubMed PMID: 16868435.
Simpson D, Lypson M. The Year is Over, Now What? The Annual Program Evaluation. J Grad Med Educ. 2011 Sep;3(3):435-7. doi: 10.4300/JGME-D-11-00150.1. PubMed PMID: 22942982; PubMed Central PMCID: PMC3179219.
Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach. 2012;34(5):e288-99. doi: 10.3109/0142159X.2012.668637. Review. PubMed PMID: 22515309.
Vassar M, Wheeler DL, Davison M, Franklin J. Program Evaluation in Medical Education: An Overview of the Utilization-focused Approach. J Educ Eval Health Prof. 2010 Jun 15;7:1. doi: 10.3352/jeehp.2010.7.1. PubMed PMID: 20559515; PubMed Central PMCID: PMC2886237.
Sleight D. Evaluating Educational Programs: A Planning Tool. MedEdPORTAL Publications; 2007. This is an introductory level web-based tutorial that guides users through the six steps to developing an evaluation plan for a program or curriculum.
Enhancing Program Performance with Logic Models. UW Extension. This free online course provides a holistic approach to planning and evaluating education and outreach programs. It helps program practitioners use and apply logic models - a framework and way of thinking to help us improve our work and be accountable for results. You will learn what a logic model is and how to use one for planning, implementation, evaluation or communicating about your program.
Program Evaluation. UC Irvine OCW. 3 hours.
Case Studies in Program Monitoring & Evaluation. Vanderbilt University. 1 hour and 14 minutes.
Fink AG. Evaluation Fundamentals: Insights into Program Effectiveness, Quality, and Value. Third Edition. 2014. 304 pages. $52.74
Fitzpatrick JL. Program Evaluation: Alternative Approaches and Practical Guidelines, 2nd Edition. 558 pages. $44.92
Barrett NF. Program Evaluation: A Step-by-Step Guide. 84 page Kindle book. $1.99
♥Evaluating Educational Programs: A Planning Tool. Office of Medical Education Research and Development, Michigan State University. Web-based tutorial.
Fundamentals of Program Evaluation. Johns Hopkins Bloomberg School of Public Health. Open Education Consortium.
ED-46: A medical school must collect and use a variety of outcome data, including national norms of accomplishment, to demonstrate the extent to which its educational program objectives are being met.
ED-47: In assessing program quality, schools must consider student evaluations of their courses and teachers, as well as a variety of other measures.