Evaluation Cookbook

1. Define issues for focus
Start with broad themes which can be made explicit to the groups, keep a checklist of individual points of concern to prompt for, if they don't emerge naturally.

2. Identify participants from relevant population
Try to make these representative of various types of 'user', i.e. different motivations, different entry levels, different learning environments ... Hint

3. Design the sessions
Set group size - between 6 and 12 is recommended.

Decide whether mixed groups or contrastive groups will best serve your need, comparing similar groups to check agreement or distinct groups to establish perspectives on issues.

Decide on structuring strategy - one or two broad topics, or a guided programme for discussion within allocated timeslots? Let conversation flow, if purpose is exploration. Eliciting sequenced 'rounds' of input if there is already a strong agenda to the study, if extremes within, align according to need. Hint

Define required analysis level - qualitative, 'ethnographic' or systematic content coding, or a combination of these, depending on goals and resourcing. Hint

Decide on recording options - notes? audio recorded? video-recorded?

4. Stage the sessions!
The most important thing is to be both confident, and relaxed - then they will be too. Hint

5. Transcribe records
Verbatim, expressing as written text, or noting against pre-defined criteria whilst listening to/watching tape.

6. (Code and) analyse transcripts
Start with two, and examine closely to establish most useful breakdown of detail in terms of evaluation targets. Broad possibilities are by topic theme, or by participant type. When procedure agreed, test it against remaining transcripts - does it cover the data?

7. Interpret findings
Integrate with other outcomes from other methods used. Hint

Report your recommendations.