Evaluation Cookbook

1. Define your population sample
Who (and how many) will you interview? Hint

2. Identify the target interview issues
List them by 'theme' if possible.

3. Design your study
Think of how to approach interviewees, the conduct of the interview itself and allow time for post-interview feedback.

4. Produce an interview script
Broad cues for a conversational approach, guide-list for a structured interview, questions for a standardised 'open interview'.

There are several sorts of questions that can usefully be asked, and Patton (1996) suggests that these should if possible be sequenced in the following way: behaviour/experience questions; opinion/value questions; feeling questions; knowledge questions; sensory questions; demographic/background questions.

5. Pilot the interviews
Use a small subset (at least two) representatives of the different populations involved. This is an often skipped but absolutely essential step!

6. Conduct the interviews
Write up a short note after each episode, including anything that stood out, went amiss, etc. just logging general procedure.

Hint

7. Transcribe responses and organise in best way for analysis
This can mean tagging 'rich' text responses to secure context information (respondent details, intervention details, schedule details) and storing responses under issue/question headings for whatever level of content analysis is appropriate. It can mean providing for 'response profiles' from individual participants. It is quite likely that both orientations will be necessary.

Typing each interview response into a spreadsheet with the individual case as a row with columns holding demographic information, respondent factors etc. having one column for each open question is a good way of holding the data safe. Printouts of individual columns with all the text responses can then be coded and this information entered into a duplicate file. Excel 7 allows word-wrap for quite lengthy responses!

8. Analyse outcomes
Remember that this technique gives qualitative data! Even if it has been possible to code responses such that descriptive statistics are possible and useful, any further analysis has to be very carefully conducted, using 'non-parametric' tests.

9. Interpret findings
Integrate with other outcomes from other methods used.

Hint

Report your recommendations.