to LTDI

Evaluation Cookbook

cookbook logo
Route map: Home=>Publications=>Evaluation Cookbook=>Recipe Index

Recipes


Cookbook
>Contents
>Recipes
>Information
>Preparation
>Presentation 
>Exemplars


Also Available:
Index of recipes
classified by use

Alaphabetic list of recipes
Click on a title to go directly to that recipe, or scroll down the page for the summaries

Checklists
Concept maps
Confidence logs
Cost-effectiveness
Designing experiments
Ethnography
Focus groups
Interviews
Nominal group technique
Pre and post testing
Questionnaires
Resource questionnaires
Split screen video
Supplemental observation
System log data
Trials
LTDI chocolate cake recipe

Summaries

Checklists
A quick way of getting a lot of information about an implementation: data gathering is efficient and collection and analysis can be automated. Although the data gathered is rather low level there can be lots of it.

Concept maps
A concept map or a mind map is a visual representation of the links or associations between different concepts or pieces of information.

Confidence logs
These are self assessment measures which are used to gauge a student's confidence level in a particular part of a course.

Cost-effectiveness
Any educational intervention has an outcome and a cost. A measure of the cost effectiveness is obtained by measuring costs against outcomes.

Designing experiments
A controlled experiment is a way of teasing out the details of just which aspects of your innovation are influencing the outcomes you are considering and bringing about the changes you observe.

Ethnography
The essence of an ethnographic approach is 'in situ' observation. The events being observed should be as little disturbed and as authentic as possible.

Focus groups
Focus groups are moderated meetings of 'involved' people discussing their experience of an educational intervention. They are a useful tool for formative/developmental or summative/retrospective evaluation and can serve as a single, self-contained method or link to other evaluation activities.

Interviews
"Interviews are conversations where the outcome is a coproduction of the interviewer and the interviewee" Kvale (1996).

Nominal group technique
An interesting method which can yield surprising results. Advantages are that the views of everyone are considered, and the time for all stages is low. It is a group in name only as the views of all participants are collected separately.

Pre and post testing
This recipe describes a test methodology currently being used to test a group of postgraduate students undertaking a programme of instruction in bibliographic classification, but which has broad application possibilities (see the overview in the information page).

Questionnaires
A collection of open or closed questions which can be paper-based or online.

Resource questionnaires
However good a CAL package is, it will have no positive effect on student learning if it isn't used. Resource questionnaires seek to identify which resources students are using.

Split screen video
Involves video recording the student and the software in use.

Supplemental observation

This involves watching how your students make use of a piece of software.

System log data
System log data is a step-by-step recording of user interaction with a software program. The level of detail recorded is determined by a purpose-built program and can include mouse clicks, menu calls, and all operations on objects. Modelled on a 'record' facility, log data is a useful way of tracking user preferences and navigational choices.

Trials
Planning trials allows you to pace the introduction of new techniques with your students. You can decide at what stages you need to test your developments, and who can give you the right sort of feedback so that improvements can be made. Once trials are in your plan, they also become a deadline to work to; and can make all the difference between something that is really used and something that looks nice, but never gets put in front of any student.

LTDI chocolate cake recipe
Our favourite!

>Contacts
>Legal
>About