Section 3: Evaluating, Implementing and Managing Instructional Programs and Projects

Chapter 10 discusses evaluation in instructional design and provides you with two evaluation models, the CIPP and Kirkpatrick models for evaluation. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction. 

Goal Free Evaluation
This model was created by Nfichael Scriven in the 70's.  The purpose of using this model is to capture intended as well as unintended outcomes of a project.  As the title indicates, the idea is not to consider the goals or objectives of the program so that all aspects of the program can be considered.  The focus will be on the effects of the program and not the goals so the ability to profile the needs of the "target population" is required and if there is positive impact on those needs, it will produce a postive evaluation.

It requires the evalutor have access to participants, data and have time to conduct the evaluation.   It is qualitative and relies on interviews in an unstructured manner.   The evalutor should have good knowledge of the subject and if they are not competent, it will be a major factor.  The evaluator also needs to be free of bias so they can interpret the data for what it is.

I can see this model being helpful to evaluate my instruction with a perspective of the overall value of the content or process.   There will be benefits that could be substantial that would go unnoticed and unappreciated if a evaluation of this type were not considered.   While I am typically a numbers type of person, a qualitative evaluation model will help tell the rest of the story that justifies the use of the instruction design or at least certain elements that could be coupled with other design that produces better "numbers"

Tyler Model
In contrast to Goal Free Evaluation, the Tyler Model is all about the objectives.  The goals need to be measurable for this method to be helpful. The main function of the evaluator is to identify a process to measure the goals and will design a way for the data to be collected.  The information is given to the client and the evaluator is no longer in the process.  The client will make their own judgments as to what the collected data tells them.

The ability to identify goals is critical and the process to collect data should be tested well.  The output is only as good as the correct goals are expressed.

I can see this model as extremely helpful to know if the instructional design is meeting the "numbers".  Perhaps there could be two evaluation models utilized to capture the quantitative and qualitative value of the design.   I probably lean more to this model as it might be easier to place measuring devices and collect data to tell the story.  However, the data could be interpreted in many different ways, and perhaps the qualitative method above would bring into focus how or why the objectives were met.

Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. What else would be useful to know?

Cost and Time would be some factors to consider.  In many of the evaluation models, the evaluators need to be experts or unbiased, which means not necessarily connected to the current situation.   Time is money and expertise is money.  
Who will review or assess the evaluation.   Much like the Tyler Model, the interpretation is to be performed by the client.  Is there support from management or administration to give due diligence to understand the data and take necessary actions.

What have been the outcomes of similar studies or evaluations?  What elements have made them successful?

Chapter's 12 & 13 focus on project management and how to manage projects when resources are scarce. You have been assigned to develop a series of professional development sessions focusing on technology use in the classroom for teachers during a time of economic decline. How will you use Situational Leadership to facilitate this project and manage scarce resources? 

Situational Leadership is a very practical way to manage and lead projects and people. The key behavior for a situational leader is to adapt to each of their team members according to what they need.  What they need depends on their motivation and their competency.  If they are Low Competent and Low Motivation, it will require more direction and follow up as well as investigation as to what will help the team member buy into the project.  If if they have high motivation and low competency, then it is a matter of training and feedback to bring them up to speed.   On the other side of the spectrum, we have high competency and low motivation, once it is understood what it will take to bring that person back into the tent, they will be very productive members of the team.  Obviously the high motivation, high competency will be more of a monitoring interfacing role that the situational leader will use.     Not all members of the team will stay in any given quadrant and it requires a strong knowledge of people skills and assessment skills to identify what actions to take.

Understanding each team members role is also critical to the programs success.  Knowledge of the job and/or specific tasks is information that the leader needs to know.

Regarding scarcity of resources to develop a series of professional development courses, I would need to fully understand the scope and amount of resources that are available.  I would assume that the need or demand is somewhat fixed, however, one can assume that we are talking about teachers who can be picky and critical and refuse to use something that is not up to their expectations.   I would definitely try to understand what gaps or desires the teachers have and try to capture a large percentage of potential participants.

I would balance the scope, time frame and amount of work with the number of resources available.  Negotiation of the deliverables would be required so that we do not over promise and under deliver.  Once we had our goals and objectives, I would set up some clear channels of communication via meetings and mailing lists.  It would be clear where to store documents and we would establish and celebrate milestones.