|Route map: HomePublicationsEvaluation CookbookWhy evaluate|
The Costs of Evaluating
Let's start with students, since often they are an afterthought. What are the concerns of the group you're targeting? There are some obvious areas which interest them, from gaining a better education, through issues of accessibility, to the passing of the coming exam. As with all the stakeholders, don't plunge into constructing the evaluation without talking to them and exploring their concerns around the educational intervention on which you're focusing. Then the resulting evaluation will be centred on discovering how the intervention can be improved to satisfy the real aims of the target audience rather than what you decided they should be.
Evaluating for developers is more straightforward. Given that the content is appropriate, the developer is interested in how easy or difficult the user found it to access the material. Were there any bugs? Was the navigation instinctive? Was the text in a suitable font and was it presented in appropriate volume? Was the feedback provided at the right place and did it satisfy the user? And so on.
Lecturers want to know about learning gains and efficiency. Was this a better way of presenting the material than the tutorial or the lecture? Did it free up time for more effective contact with the student or to conduct research? Are there additions such as handouts which need to be considered to improve the effectiveness of the intervention?
Management need evidence that the time spent on development has led to greater efficiency while maintaining at least the same quality. Does the product justify the costs? Have the students welcomed the change and will the course continue to attract recruits? Have the exam results remained acceptable? Will it help with the TQA? Can fewer resources be devoted to the course than before?
There are usually other stakeholders who have an wider interest in the results of evaluations, especially of computer assisted learning. The Funding Councils, for example, wish to consider whether money is well spent in this area, and though a large external evaluation will usually be conducted to provide the answer, the sum of small local evaluations feed into the decision.
Will it be Worth it?
© All rights reserved LTDI and content authors.
Last modified: 26 March 1999.