Clarifying Six Confusing Types of Evaluation
JAMES D. RUSSELL
VISITING PROFESSOR FROM PURDUE UNIVERSITY
As I talk with students and professionals in the field, I often detect confusion about the various types of evaluation. The same is true for writers in our field. Many are inconsistent in their use of the terms formative, summative, and confirmative evaluation.
I think what confuses them is that there are three basic types of evaluation: formative, summative, and confirmative. In addition each can be applied to both learners and materials, so there are really six different types of evaluation:
- Formative evaluation of learners
- Formative evaluation of materials
- Summative evaluation of learners
- Summative evaluation of materials/li>
- Confirmative evaluation of learners
- Confirmative evaluation of materials
Consequently, one should not use formative, summative, and confirmative without stating or implying whether it applies to learners or materials. I am using learners to be synonymous with students or trainees. Any evaluation of learners takes place after development of the materials. Materials refer to printed materials, computer-based instruction, Web-based training, etc. Each of the six types of evaluation are very different in their purposes and procedures. As you read this article bear in mind that evaluation of materials is s part of instructional design. Even though the instruments and procedures to evaluation learners are developed as part of instructional design, they are implemented as a part of performance evaluation. Let's briefly look at each of them.
Formative evaluation of learners
Formative evaluation of learners diagnosis learning problems and if necessary prescribes enrichment or remediation. It occurs during instruction and uses self-checks, progress tests, non-graded quizzes, or even instructor observation.
Formative evaluation of materials.
Formative evaluation of materials is done during the development of the materials in order to improve them. Often this type of evaluation is called "learner tryout and revision." This type of evaluation is done while the materials are still in draft form. Often it is done with individual learners or small groups.
Summative evaluation of learners
Summative evaluation of learners is conducted at the end of instruction to give a grade or to certify competency. After a "final exam" is administered to measure students achievement. These evaluations can be paper and pencil or performance.
Summative evaluation of materials
Summative evaluation of materials is designed to determine the effectiveness of the materials. Often this process is called validation or verification. It is conducted after the materials have been developed and have been used by the learners. If the materials are then revised, the process is by definition "formative evaluation of materials", not summative evaluation of materials.
Confirmative evaluation of learners
Sometime, usually months or years, after completion of instruction, learners are re-tested to determine if they have maintained their competence demonstrated during summative evaluation of learners immediately after completing the instruction.
Confirmative evaluation of materials
Similar to confirmative evaluation of learners, confirmative evaluation of materials is conducted some time, again months or years, after initial implementation of the materials. The purpose is to determine if the materials are still enabling learners to meet the original objectives. The process may determine that the content has changed since the original development of the material or that the characteristics of the target audience have changed. In either case, the materials need to be revised.
Conclusion
I hope this helps to alleviate in your mind some of the confusion about the various types of evaluation. If not, please email me.
References
Hellebrandt, Josef, & Russell, James D. Confirmative Evaluation of Instructional Materials and Learners. Performance & Instruction. July 1993.
Russell, James D. & Blake, Bonnie L. Formative and Summative Evaluation of Instructional Products and Learners. Educational Technology. September 1988.
Author
Since 1996 Jim has taught in the Instructional Systems Program at Florida State University and worked at the Learning Systems Institute during the Spring Semesters. The rest of the year, he is Professor of Educational Technology at Purdue University and a consultant with Purdue's Center for Instructional Excellence. His publications include Instructional Media and Technologies for Learning (co-authored with Heinich, Molenda, and Smaldino) and Instructional Technology for Teaching and Learning (co-authored with Newby, Lehman, and Stepich). Jim can be reached year round at jrussell@purdue.edu