Evaluation Cookbook

When you are carrying out interviews, focus groups and discussions it is important to keep a record of what was said for future transcription, reference or analysis. The method you use will depend on how much detail you require.

Keeping written notes:
Taking notes as you go along can be a useful way of gauging general opinion and the time taken to stop and take notes can be a useful way of giving breathing space during the discussion. However, trying to keep the discussion going and keep an accurate record of everything said can prove to be an almost impossible task.

Taping your session:
The easiest way to keep a record of a discussion, is to tape your session. However, it is important to make sure that you have your participants' permission to tape their interview. In addition, let them know exactly why you are carrying out this exercise and what you are planning to do with the information. You might also like to let the participants see any transcriptions that you are planning to use.

Sometimes students can feel awkward about the idea of being taped so you might start by having some general conversation at the beginning of your session in order to get them accustomed to talking with a tape recorder running.

Hiding a tape-recorder and recording secretly might seem to be a good tactic to help keep your participants relaxed, but apart from the obvious ethical problems this raises, it also means that legitimately, you can't make use of any of information you collect.

Setting up the taping equipment:
Make sure that you are familiar with using the equipment and that everything is working OK before you start. It can be very disruptive to a session, if you have to spend the first 15 minutes trying to get the tape recorder to work. Batteries in the tape recorder or tapes running out can also mean that you lose important sections of a discussion.

Having a good microphone is probably more important than having a good tape recorder. Constant hissing on a tape or just having indistinct mumbling can cause all sorts of problems when transcribing. Placing a microphone on a soft surface such as foam or a piece of carpet can help to improve sound quality and a flat microphone can also appear less intrusive.

Most microphones built into recorders have a restricted range and if you are working with a group of people, you could find that you miss one or two people's contributions because they are just out of range. If you are working with a group, try to make sure that they are an equal distance from the microphone and if you can't get hold of an omni-directional microphone try and use a second one.

Recording progress:
It is often a good idea to try and give your groups a feel of their progress during the session. This can be done through using a spray diagram or flip chart to record ideas as you go along. Visual methods can be more effective than a list of points. They also have the benefit of focusing discussion and ensuring you are accurately interpreting what is being said as any misconceptions can be corrected there and then. In addition, when an idea is represented in this way it becomes separated from the individual who put it forward and therefore it becomes easier for the other group members to criticise or disagree with its content. As part of your exercise, you could also ask your groups to summarise the main points of the discussion.

Working with recordings:
It is a good idea to try and transcribe your own tapes as you can remember more clearly who said what. A speed play option on your recorder can assist when scanning through material for particular sections and foot pedal attachments are available, whereby you can control your tape recorder being switched on and off. These are particularly useful when you are transcribing data.

If you don't want to go through the process above, you might like to use software (e.g. CODE-A-TEXT) which is now available and which works from digitised sound files. The coding, memoing and analysis can be done in the software while listening to the sound file. This software can also work for transcribed data primarily as a content analysis program.

Identifying who said what:
Which ever method you are using it is a good idea to ask someone to keep a note of the first few words said by each participant. Reference each participant by their initials or give them each a reference number. If you don't have a scribe to do this, then think about mentioning the participant's name in the conversation after or before they speak, to make it easier for a transcription. It is important to try and do this in as natural a way as possible and in a way which doesn't affect the flow of the conversation.

Focus groups can sometimes comprise students/ staff from different disciplines which can make it more important to identify who says what. You might consider using speakers identifiers which look something like this : BSC-1-8, MA-3-6, HND-1-2 etc. representing a first year student from focus group 8, a third year student from group 6 and first year HND student from group 2. This also means that you can carry out automated text searches on the basis of the speaker's identifier if you are using data analysis software at a later stage.

Jen Harvey
Implementation Support Consultant,
LTDI, Heriot-Watt University.