LEARNING
TECHNOLOGY
DISSEMINATION
INITIATIVE

Evaluation Studies

Route map: Home => Publications =>Evaluation Studies=>Leading New Teachers ...

Leading New Teachers to Learning Technology

Jan Drysdale & Linda Creanor, Glasgow Caledonian University
to contents
to next page
to previous page

Also available
as a pdf file.

Abstract

Providing an introductory course in learning and teaching has long been a core activity in staff and educational development at Glasgow Caledonian University. However, a fundamental re-appraisal of course delivery was required. The new approach, piloted in semester A 1997/98, incorporated several key features:

The web site was designed to provide a flexible, easy to use and attractive delivery structure with easy navigation between sections, with links to the course outline, direct e-mail contact with course tutors and online registration. To introduce the concept of computer mediated communication (CMC) to new staff, the hypernews discussion group has been set up as a forum for discussing the issues in teaching and learning which arise during the course.

Because this was a pilot, evaluation of both content and process was important. Criteria used were attendance and completion rates, quality of assessments submitted and qualitative feedback from discussion with the course participants. Responses were mixed. Few course participants completed all of the assessed elements but the self-assessments and the feedback from those who did were encouraging. Staff involved in delivering the course learned a number of valuable lessons which have been incorporated into course revisions for Semester B.

Introduction

At Glasgow Caledonian University, responsibility for providing staff development opportunities in learning and teaching lies with the Department of Learning and Educational Development. There are two main approaches to staff development: the Postgraduate Certificate in Tertiary Level Teaching Methods (TLTM) which uses a mentor-supported work-based learning model where the course participants work at their own pace to achieve personal and professional development in learning and teaching and an Introductory Course which is viewed more as a 'survival skills' training course and was, until 1997 delivered over a three day period.

Course redesign - the key questions

Why?

Evaluation of the Introductory Course over a two year period had shown that the biggest problems were:

Clearly a fundamental re-appraisal was needed. Additional stimuli for change included:

Thus, the remit in revising the induction course was to integrate it into the CPD framework, make it more accessible to its target audience and more innovative in its delivery. One explicit function of the redesigned course was to ensure that new staff gained hands-on experience of using learning technology and the opportunity to reflect on whether and how it could be incorporated into their own teaching.

What?

Believing in the systems approach to curriculum design as we do, the first thing was to clarify the training need and to interpret that into a set of objectives and a delivery structure for the course. Because the course participants come from such diverse backgrounds, the new course had to incorporate a degree of flexibility so that participants did not need to spend a lot of time on topics of only marginal relevance to them. Nevertheless most, if not all, of them would be preparing materials to use in their teaching or to give out to students, delivering lectures, and working with small groups in tutorials, seminars or practicals. Some would also be involved in assessing student learning. The delivery model would be required to integrate with the TLTM modules so a work-based learning approach was adopted and the following series of objectives, based on those used in the previous course, drawn up:

At the end of this course you should be able to:

1 describe, in broad terms, the operation of the university's academic structures and procedures;
2 prepare high quality materials for learning and teaching which are appropriate to your teaching;
3 prepare and deliver effective lectures;
4 select and use appropriate group methods to enhance students' learning;
5 appraise the validity, reliability and practicability of instruments of assessment.

Two other objectives were added to indicate that this was only the start of CPD in learning and teaching and to ensure that the course participants were at least aware of current possibilities in learning technology:

6 produce an action plan for your personal and professional development;
7 as a student, use learning technology with confidence.

How?

Immediately we were looking at something which would take place over a much longer time scale than the previous three day Introductory Course so it seemed logical to extend the course over a semester and let course participants work their way through the various parts of the course as the demands of their teaching commitment dictated, much the way people work through TLTM but TLTM course participants have a mentor. How could we support these individuals through the semester?

One of the best features of the previous Introductory Course was the opportunity it provided for interactions not only with staff from LED but also with other new teachers from a range of disciplines and to share their concerns and classroom experiences. In an attempt to retain these benefits we opted for a multiple media delivery model emphasising learning and professional development through four types of activity:

But how would we know if they really had done it? We took the radical step of introducing an element of assessment into the course. This fulfils several functions. It tells us that the course participants have worked through the required elements of the course. It also enables us to award a certificate to those who successfully complete the course and this counts for CPD as well as providing credit towards some of the TLTM modules.

The next stage was to develop the materials for the web site. These fell into three types namely information pages outlining the requirements of the course and its delivery model, reference pages on professional development issues, structures and procedures and the study guides themselves.

Online delivery

The task of converting a more traditional paper-based course to an online delivery mode requires careful planning. As a starting point, it is often advisable to take a step back and re-think what is to be achieved through this alternative delivery medium. What 'added value' can the Web offer to deliverers of course material and their students?

In this case, the main aims were to increase the flexibility of delivery and to decrease the amount of time hard-pressed academic staff had to devote to attending face-to-face tutorials. The secondary aims were to harness the capability of the Internet to enhance communication and to raise awareness among the course participants of its potential for teaching and learning.

In order to achieve these aims, and to provide a pedagogical model which was both simple and effective, we developed a frame-based layout which allows a consistent interface throughout and helps restrict navigation to within the course materials. These consist of the four Study Guides:

Each Study Guide leads the course participants through the recommended readings, followed by a variety of work-based activities which they are expected to carry out as part of the assessment for the course, and finally they also provide access to an online discussion. Within each section there are also a number of links to other relevant web sites such as information on the Scottish Higher Education Funding Council, the ubiquitous Dearing Report and several interesting articles on teaching and learning from the Times Higher Online. In addition to these are two reference sections containing information on Glasgow Caledonian University's Structures and Procedures, and some advice and current thinking on Professional Development.

To encourage awareness of the Scottish Metropolitan Area Networks (MANs) which are impinging on all areas of learning and teaching in HE, we decided to link directly to Clyde Virtual University (CVU), a collaborative project on the ClydeNET MAN in which Glasgow Caledonian University is a participant. We made use of CVU's facility for hosting Hypernews discussions for the local universities, and hoped through this medium to encourage informal discussion on a variety of the course issues. In order to encourage participation it was decided to make contributing to the discussion a small part of the overall assessment for the course, and direct links were made to it from each of the four Study Guides. For these pilot sessions the discussion has been restricted to course participants, but there is the potential to share both this and the course materials with other local universities, and invite well-known 'experts' in the field to join in a debate on some of the issues.

Finally, thought had to be given as to how the online course materials would be accessed by the users. In order to give the course a high profile two main links were created from the home page of Glasgow Caledonian's Centre for Learning and Teaching Innovation. The first leads to a widely available information section which gives details of the target audience and an outline of the course content. It also includes an online application form for prospective participants which is automatically sent to the course tutor on completion. The second link gives access to the actual course materials which for copyright reasons are password-protected. Staff are issued with an individual username and password when they have enrolled for the course. This does have implications for administration, but as the numbers on the course are restricted to around twenty, it is not an excessive burden. The advantage is that tutors have full control over who can access the course materials.

Evaluation and Feedback

Semester A 1997/98

The revised Introductory Course was piloted during Semester A of the 1997/98 academic session. The course is obviously still in its formative stages so evaluation of both content and process was considered highly important for future development of the model of learning and teaching adopted. Criteria used were attendance at tutorials, completion rates, quality of assessments submitted and qualitative feedback from discussion with course participants.

Responses were mixed. Still, some observations and inferences can be made. The course began in September 1997 with 25 people registered as participants. They included lecturers, researchers and research students from all parts of the university, although one department had a fairly large 'block booking'.

Attendance at the three half day tutorials has been disappointing. It had been hoped that, by reducing the mandatory attendance and by spreading it over a longer time period, more participants would attend. In fact, of the 25 course participants registered, 17 attended the first tutorial. This number had dwindled to 6 by the third tutorial. Some of the absentees expressed valid reasons for non-attendance and concern at having 'missed out'. Others could not be contacted at all. It would seem therefore that we still have a motivational problem.

At the second tutorial course participants were asked to outline their progress to date through the course materials and activities. The responses were very disappointing but when it was stated that the model clearly was not working, this view was firmly rebutted. Course participants argued that the due date for assessments was nearly two months off and therefore the activity had not been accorded the priority treatment that an assessment date would confer.

Some course participants have reported that they have problems accessing the materials on the web site. This is a particularly common complaint among the research students on the course who do not have PCs in their offices, although they are available in open access labs and libraries. Similarly this group is the one most likely to comment that the assessments are not appropriate to their situation because of the restricted range of their teaching. No-one has approached the staff associated with delivering the course to negotiate a way round this difficulty.

Six course participants, mainly the more mature members of the group, completed the assessments by the due date. The work submitted is generally of very high quality. In fact, it has been necessary in the feedback on these self-assessments to remind these individuals that a balanced self-assessment includes the good points as well as the not so good and the plans for improvement.

Qualitative feedback indicated that, in general, participants on the course did appreciate the flexibility which the web site provided. There were some inevitable problems with the technology which will have to be take into consideration for future courses. These included difficulties with access to computers, an unexpected 'firewall' between one of the smaller campuses and Strathclyde University in whose domain the Clyde Virtual University server resides, and the well-documented issue of the disappointing quantity of contributions to the HyperNews discussion group.

When asked, the reasons given for minimal or even non-participation in the online discussion were those of lack of time and lack of access. Heads of Departments may have encouraged new staff to attend the Introductory Course for Learning and Teaching, but often they did not support them by allowing them any extra time to complete course assignments or to contribute to the online discussion. Difficulty in gaining access to a suitable computer was a particular problem for part-time or research staff who did not have their own desktop machine. There is no immediate solution to either of these issues, but if departments wish their staff to participate in courses such as these, they will have to be made aware of the support they should offer.

The online course was successful in demonstrating to new staff a simple model of using learning technology, and giving them first hand experience of its use. It succeeded also in raising their awareness of current practices in this area, which they were encouraged to consider in relation to their own teaching. From this point of view it has certainly been a worthwhile exercise, and one from which both participants and course deliverers have benefited.

Changes for Semester B

As a result of our experiences in Semester A changes were made in course administration, the structure of the tutorials, the web site and the evaluation methodology.

The administrative changes were designed to target the course more effectively to staff who are at least relatively new to teaching within the university and who have a varied teaching commitment. There is general agreement that a different type of staff development activity should be organised for lab demonstrators who do not benefit from this Introductory Course. The memo to Heads of Department and the course participant registration form were rewritten to reflect the emphasis of the course and the necessity of course participants being given time and access to the IT facilities required to undertake the course.

The series of tutorials was restructured around an action planning and review cycle with course participants being required to submit by e-mail their action plans for development in learning and teaching during the semester.

For the second run of the course in Semester B a few additions and alterations were made to the web site. The main one has been the integration of online personal portfolios into which each participant can 'post' their outline development plans, reflections on their activities and final course assessments. This is done via online forms, the content of which is written to password-protected personal web pages accessible only to the individual student and their tutor. These portfolios build up to provide a useful record of an individual's professional development.

Another addition is an online evaluation form which participants are asked to complete when they have finished the course. The content of the form is e-mailed directly to the course tutors. Questions cover all aspects of the course from the organisation of the half-day tutorials and relevance of the course materials to the usefulness of the web site itself. This should provide some useful feedback which will allow refinements to be made to future courses.

Initial Feedback from semester B and issues for 1998/99

Despite the tightening up of administrative procedures out of ten course participants registered for semester B, two subsequently withdrew, one because he did not have a teaching commitment at the time and the other because he had 15 years teaching experience in other universities. Two other members of teaching staff with some limited teaching experience chose, after some dialogue with the course tutor, not to register. Obviously there is room for more improvement in course administration.

The opportunity to discuss their experiences informally was well received and set the tone for frank and open discussions in all of the tutorials. Participants initially responded well to the action planning and review cycle. All but one of the nine active course participants did submit an action plan. Inevitably target dates in the action plans were not met but participants were encouraged to continue attending tutorials and working at their own pace throughout the semester. Attendance at tutorials was again a bit disappointing, reducing from seven of the active participants to four. Interestingly, though, apologies and reasons were given for absence in nearly all cases.

Qualitative feedback at tutorials has been very encouraging with all course participants indicating that they have found the experience to date positive and supportive. Use of the personal portfolios has been mixed. Some participants have submitted detailed entries explaining what they were seeking to achieve, why they decided on the approaches they did and how successful they found these approaches in practice. Others have adopted a much more superficial approach providing only outline answers. Two course participants have not used the personal portfolios at all. Clearly there is scope for further discussion and development here.

The Dearing and Garrick Reports place considerable emphasis on the importance of communications and information technology. Indeed the committee believes that,

"for the majority of students, over the next ten years the delivery of some course materials and much of the organisation and communication of course arrangements will be conducted by computer".

It was hoped that by allowing staff first hand experience of online learning, they would be encouraged to think about using technology in their own teaching. They would also become aware of the issues involved and the implications for their own students. Some of these issues have certainly come to the fore during these pilot runs of the Introductory Course and have been highlighted in feedback from participants. These include access to computers, inadequate technical support, time management in a more flexible environment, and not least the effort required to participate in an online discussion.

The results of the online evaluation and assessments are awaited with interest!

Conclusions and ideas for further development

Our aim is to encourage course participants to become reflective practitioners. To do this we have encouraged course participants to evaluate their own experiences and share those experiences with the others on the course. The security blanket of the 'right' answers is not available, but access to a sympathetic audience, whether tutor or fellow course participant, either face to face or through electronic communication, is. The motivation levels among course participants appeared high in semester B. Still, there is always room for improvement. Some issues which need further work for 1998/99 are:

References:

Centre For Teaching and Learning Innovation HomePage, Glasgow Caledonian University. Available online at: http://led.gcal.ac.uk/clti/

The Dearing Report. Available online at: http://www.leeds.ac.uk/educol/ncihe/

Lee S., Groves P., Stephens C. (1996) Internet Teaching: Existing Tools and Projects for Online Teaching. JTAP Report, Oxford University Computing Service. Available online at: http://info.ox.ac.uk/jtap/reports/teaching/

Mayes T. (1997) Dialogue with a Dumb Terminal, Times Higher Education Supplement, Oct 10th. Available online at: http://led.gcal.ac.uk/clti/papers/thes.html

to previous page to contents to next page

To contact the maintainers - mail (ltdi@icbl.hw.ac.uk)
HTML by Phil Barker
© All rights reserved LTDI.
Last modified: 29 September 1998.