Evaluation Cookbook

cookbook logo
Route map: Home=>Publications=>Evaluation Cookbook=>Trials
 Summary   Process   Variations   Links page   References  Add comment 


1. Identify your innovation
The first stage is to recognise that you are planning a change or development that needs some feedback from others. Explicitly defining what it is that you are building and the scale of innovation involved is important. This is when you need to judge what you want to achieve with your students. Hint

2. Prepare a schedule
Work out a development plan. This could be your chance to try a project planner, but a simple idea of the tasks involved and content needed, matched to when you might tackle them, is all that is needed. If you make "use with your students" the only deadline in this plan, you are running a risk that it will not be ready and it will not be suitable. A series of small deadlines is easier to meet than one big one. Hint

3. Pick your time(s)
From your plan, look for an early point when you should have something available that can be used by students. You might have issues of what tools are needed and how to present information, as well as the choices about the content and teaching approach. It could be possible to address some of these questions separately and in ways that can be simply organised. However, you should also be looking for a trial of a more or less complete system before your final deadline. Note the time or times for these trials and treat them as commitments. Just identifying the trial can make a change in how people proceed with their work and the direction of effort in a project. Hint

4. Find your users
Having decided what it is you would like to find out and when you want to do it, you need to decide who with. Trials do create work for everyone involved, so you have to value your own and your students' time. Small scale trials with friendly users can answer some initial questions, but the best groups of users will be those who are closest to the eventual users - real students. It can be worth paying a small fee to get people to spend enough time using your system, thereby giving good feedback. In early trials, numbers are less important than the quality of information. Hint

5. Carry out the evaluation
The other sections of this book will guide you towards ways in which you can perform the evaluation itself. Picking the right evaluation method will certainly help get good quality information but there is also value in the less formal feedback from how the trial went - how difficult it was to organise, the stress levels in keeping things running, the relief when it is over. Hint

6. Use the results
Carrying out trials is not an end in itself. The feedback you have from your users needs to be used in your development process. Make some initial judgements about how well it went consider if there needs to be significant changes made to your plans; there might be some radical suggestions that can be identified. More likely you will have some generally positive information together with some complaints or weaknesses that have been identified. Try not to let these dishearten you, but instead see that you now have a clear basis for action to address these. You might need to weigh up some insightful comments against consistent patterns across your users. If the trial was local you can always ask people to clarify their points. Hint

7. Do it all again!
Trials are part of the process of formative evaluation; this means that you are making changes to what you are offering and, of course, to validate those changes you really should carry out some trials and evaluation.

Avoid cancelling trials

Keeping track of developments

versions for printing are available

[ Summary | Process | Variations | Links page | References | Add comment ]
[ Printable version ]
[ Contents | Recipes ]

© All rights reserved LTDI and content authors.
Last modified: 25 March 1999.