ISU COE Title
ISU COE Logo Analyze Phase Design Phase Develop Phase Implement Phase Evaluate Phase
Spires
Evaluate Phase Description
Sample Evaluation Plan
Practical Evaluation Problems
Blank Spacer
ADDIE Home Page
The Oregon Trail design team has completed the prototype of a multimedia product. They are three weeks behind the completion of the design phase. Two members of the design team have completed one on one evaluations with three students. The individual students collectively were able to follow 85% of the instructions and wrote down approximately 15 words that were unclear or confusing. The descriptive data indicates that students are somewhat motivated but were having some difficulty operating the program independently. Encouraged by the initial results of the one on one evaluations, the design team decided to cancel the small group evaluation and field test to make the production deadline. The team made this decision partially because the production coordinator insistence on complete production on time and the design team anticipated that they could partially complete the small group and field test during the final performance-context evaluation. The design team worked on the Oregon Trail project an additional week to clear up some of the ambiguous instructions and develop a few more questions relevant to a forth grade students. At the end of the week, the design team felt very confident that they had an effective product and were ready to conduct a performance context evaluation.

The design teams used the students from the one on one evaluations and twenty additional students from the local forth graders classroom to begin the performance context evaluation. The student and teacher incorporate the Oregon Trail multimedia program into the Idaho history, math and regional geography curriculum. At the end of three weeks, the design team returns to interview 5 students and the teacher. During the interview, the teacher confessed to her disappointment with the program. The students required one on one supervision to operate the program, 68 % of the students had difficulty understanding the instructions and 30% refused to use the Oregon Trail. The initial data confirmed the teacher’s impression, 43% of the math problems and 57 % of geography problems did not transfer to other areas in the curriculum. Discouraged by the results yet aware of a production meeting with the project coordinator and grant supervisor, the team reassessed the evaluation data. The design realized the evaluation was limited and it was difficult to tease out all the problems discovered during performance context evaluation.

Help the design team to re-evaluate the formative evaluation process. Determine the errors in the formative evaluation process. How would you correct the plan and what formative evaluation process would you use? Include information like what is being evaluated, what evaluation methods, what procedures will be used, and what are the expected outcomes. Also, address the criteria used to evaluate clarity of instruction, impact on the learner, feasibility, interactive process and instructional strategies and the learner’s setting.
Idaho State University College of Education
Science, Math, & Technology Education
A. W. Strickland, Ph.D.
Campus Box 8059, Pocatello ID 83209-8059
(208) 282-3680 | (208) 282-2244 Fax