to LTDI homepage LEARNING
TECHNOLOGY
DISSEMINATION
INITIATIVE

Case Studies

to contents page
Route map: Home => Publications =>Case Studies=>Assessment Group

8. Life Sciences Assessment Group

Jen Harvey, LTDI
to contents
to next page
to previous page

Also available
as a pdf file.

Summary

This case study describes the formation and activities of a cross-institutional group, who meet in person and also electronically to discuss ideas and issues of common interest - in this case assessment within the life sciences.
Key points are that the network:
  • Promotes cross-institutional collaboration.
  • Provides a supportive environment for sharing common problems and experiences.
  • Has benefited from a central administrator, willing to stimulate discussion.

Aims and objectives

One of the common requests for implementation support has been from biology departments who have been wishing to develop computer based assessments, primarily of an objective type, in multiple choice format. This has been, in part, initiated by ever increasing student numbers and the desire of academic staff to provide feedback to students through formative assessment methods. The use of learning technology has a number of additional advantages in enabling student responses to be both marked and analysed with relative ease and speed.

Multiple choice questions have been used already as a method of summatively assessing principally first and second year students for many years. However, the translation of questions from paper to computer based exercises has seemingly elicited a much higher level of scrutiny of objective questions and consideration of student responses. This could have been as a result of the Teaching Quality Assessment Exercise which was then about to be carried out within biology departments, but it also grew in part from the increased range of facilities available from computer software which encouraged an in depth analysis of test items. For example, the time consuming calculations of discrimination indices and facility values of hundreds of questions from examination papers has now become almost immediately available by selecting an option from a menu bar.

Focusing in on computer based assessments

Over the last two years, LTDI has been supporting academic staff wishing to develop computer based assessments in a number of ways. Two national conferences concerned with Learning Technology in assessment were held in 1994 and 1995. A range of relevant and available software was presented which could be used to assess students and in addition, speakers addressed the different ways in which these had been used within academic departments. Subsequently, several workshops were organised for staff within departments and implementation support was provided to individuals working to develop their own test items. These involved advising staff on the construction of objective tests, outlining the implications of different software presentation formats on student learning and discussing issues of software security, question validity, reliability and the ability of test items to discriminate between different groups of students.

Life Sciences Assessment Group

To complement these existing activities but from a subject based perspective, a Scottish Life Sciences Assessment Group was set up by LTDI in January 1996. This was primarily established to enable academic staff from several different institutions to collaborate, exchange ideas and to share their experiences in developing computer based assessments. This type of group has a number of advantages, in that its members :

Setting up the group

Although the majority of members had already met each other prior to the establishment of the assessment group, an inaugural meeting was set up to help formalise the group's main aims and objectives and the level at which it might function over the following months. As an original interest in collaboration had been expressed by individuals from Aberdeen, Glasgow and Edinburgh, it was decided to hold the first meeting approximately midway between these; in Dundee.

Rather than solely having a business type format, it was felt that the initial meeting might be more productive if a speaker was invited to give a presentation relating to their work in developing computer based assessments in order to trigger discussion within the group. In addition, a pre-release copy of Question Mark Designer Version 3 for the PC (Question Mark Computing) was made available by the software company for the group to evaluate during the that session.

The proposed structure and function of the group

It was decided, during this first meeting, that the Life Sciences assessment group could function most usefully in two different ways :

A. Email discussion group

An email discussion group would be set up which would be based at Heriot-Watt University. This would function as a group mailing system which would enable individuals to send mail to the collective group and also to see any ongoing discussions between group members. At present, this email group comprises 24 members from 14 different Scottish HE institutions, in addition to the CTI Biology, with each member acting almost as a representative for their department. Discussions have included the use of negative marking in objective tests, scaling multiple choice questions and the possibilities of setting up biology question banks.

B. Group meetings

There were 2 further meetings of the group during the 1995-96 session.

The second meeting focused on the use of Optical Mark Readers to analyse student responses and a demonstration of a reader by a commercial company was given. Ways in which students' responses might be analysed were outlined. These initial discussions were followed up via the email group.

The third meeting focused on alternative ways of assessing students using Learning Technology, for example, concept mapping systems, Modified Essay Questions and student constructed portfolios. In addition, ways in which group based activities might be assessed were outlined which could be used if, for example, students were set a group task using a computer based laboratory simulation.

The future

Another meeting is planned to air a number of issues raised within the email discussion group. This is to be in the form of an informal debate. In addition, a 'SWAP Shop' forum is to be used to exchange ideas between group members and to share personal experiences in developing and using a range of different computer based modes of assessment.

Conclusions

This group has worked well for what seem to be two reasons : firstly, there has been a central administrator willing to set up meetings and help initiate discussions and secondly, the members have been readily willing to contribute to the group. The level and quality of the contributions towards the group's activities have been impressively high with several people being prepared to share their experiences and knowledge in the field of assessment by, for example, citing useful references and outlining their own particular successes and failures.

In addition, a combination of meetings and email discussion group has worked well. Face to face meetings are useful, for example, for demonstrations of software and re-establishing contacts with those from other institutions. However, having too many meetings can prove problematic within the constraints of academic timetables. Providing a complementary email group also allows discussion to continue and those not able to attend the meetings can keep in touch with the group's activities.

The Life Sciences Assessment Group members were originally selected for either their existing or future planned activity in developing computer based assessments for biology students. They might be considered, therefore, to have an interest in mutually supporting each other. Although one or two individuals might take on the responsibility of setting up a group, even at an institutional level, a group's power is ultimately in its members and their enthusiasm to maintain the group.

to previous page to contents to next page

Link to:
Publications
Publications
Home
Home
Contact details
Contact details

To contact the maintainers - mail (ltdi@icbl.hw.ac.uk)
HTML by Phil Barker
© All rights reserved LTDI.
Last modified: 07 December 1999 (formatting),
First Published May 1997.