1. Decide on the composition of the test group
Ideally, to perform pre and post testing, you should aim to achieve a student test group of at least 30 students. You will find that should you try to use this technique on very large student populations the work involved in determination of student profiles and analysis of test results becomes unmanageable (unless you are putting considerable staffing resources into performing the survey). Hint
2. Create your student profiles
When creating your student profiles, try to take into account the various external factors which can influence student learning. Profiles should therefore be created to elicit detail of:
personal factors (derived from interview),
educational factors (including previous experience and use of computers and CAL),
3. Familiarisation with CAL package
Although a package might be simple to use it is important to ensure that students are familiar with all aspects of how to use the various features – such as taking notes online, activating the glossary, navigating through various pages and beginning an online test session.
You could consider organising a familiarisation session prior to your evaluation. Hint
As part of the evaluation session, students could be asked to complete a questionnaire which includes questions relating to their degree of comfort and confidence in being able to use the CAL package. Hint
4. Pre and post testing Hint
A. work around the CAL package
Decide when you want to pre and post test your students. Think about how much of the subject content they need to know before a pre test. It is advisable to post test immediately after they have completed their study of the material in the CAL package.
B selection of groups for two alternative modes of learning
One option might be to use the data derived from student profiles and responses to stage 3 questions so that the cohort can be divided into two. One group can use the CAL package as a substitute for lectures (on at least 2 occasions). The second group can follow the standard lecture programme. Both groups should undertake pre and post tests.
C work around the lecture
This is the reverse of Stage A. At this stage all students take the CAL unit prior to the delivery of the lecture in the topic. The pre and post testing is delivered immediately prior to and immediately after the lecture. These tests could be online or paper-based.
5. Analysis of results
The various tests will provide a huge amount of data – some of it will be raw numeric data that can be analysed using standard statistical tests. Hint
The advantage of having taken such an exhaustive approach to examining all the factors which might affect student learning (when building up the student profiles) is that, as was said earlier there are a large number of variables which can potentially affect student learning. At the analysis stage you will be able to take account of all of these variables.
Amount of effort required