Evaluation Cookbook

A toolkit for embedding C&IT into the curriculum
A wide variety of communications and information technologies (C&IT) are now available, and these offer education a broad range of potential benefits. However, the uptake and use of these resources is patchy at best. This mismatch between potential and use has been seen as increasingly important. However, although there are numerous incentives to increase the use of C&IT, concerns have been voiced that these may ignore issues concerning the appropriate use of the resources. This section outlines a pedagogically sound methodology for integrating C&IT into the curriculum. This builds on existing frameworks for designing courses involving C&IT and provides a context for evaluating their impact.

The pedagogic toolkit
The toolkit allows practitioners to integrate C&IT into their teaching in an appropriate manner. It is based on a number of assumptions, the most important of which is that no single teaching media is the best for all situations and styles of teaching. It is also assumed that if practitioners can adequately describe and compare teaching media, they will be able to make informed decisions about which are best suited to their particular needs.

The toolkit consists of the following key steps that guide practitioners through the process of redesigning a course:

1. Review the existing course to identify its strengths and weaknesses

2. Identify additional suitable teaching media

3. Use an elimination table to select the most appropriate teaching media for the course, considering:
 the preparation time required (both in staff time and other resources)
 the flexibility of delivery (in terms of the time and location of staff and students)
 the educational interactions supported
 the local factors that affect media use (opportunities or hindrances to the integration of C&IT)

4. Link the distinct components resulting from the steps above to form an integrated course

It should be noted that the term “teaching media” is used to cover traditional methods of teaching, such as lectures and seminars, as well as mediated teaching such as web pages or open learning texts.

Tables are used to structure the descriptions and comparisons required by the toolkit. Each table describes one particular aspect of the media being considered, and relates to one of the elements covered by step three of the frame-work above. Practitioners using these tables are required to edit the entries for each media in order to reflect the way that they actually make use of them, rather than simply assuming that all teachers use resources in the same way.

Once comparisons have been made in terms of preparation time, flexibility and educational interactions, the resultant short-list is compared in terms of the ‘local factors’. X-opportunities or hinderances that will affect how successful the media is. This ensures that the resultant course design is sensitive to the pragmatic concerns of practitioners. A fifth and final table is then used to collect the outputs from each of the comparisons. This allows the progressive elimination of unsuitable media, and is referred to as the elimination table.

In a sense the toolkit involves decomposing and reconstructing a course. The final step in the toolkit emphasises this in the form of a ‘before’ and ‘after’ learning plan for the course. This representation has two advantages: it provides a structure for the decomposition and reintegration of the course, and it enables a quick comparison of the courses in terms of the educational interactions supported.

Conclusion
Feedback on the toolkit has highlighted its role in Quality Assurance. Inherent in its design is that the process of integrating C&IT should be structured, with justifications of decisions that have been taken. The elimination table, described above, is the clearest example of this.

The step by step guidance of the toolkit provides a supportive method of transforming and enhancing courses. Rather than being prescriptive, it seeks to highlight issues for the tutor to consider, and provides tools that can be used to address

Grainne Conole & Martin Oliver LaTID, The Learning Centre, University of North London