You are in :
  [HEFCE logo] [SHEFC logo] [HEFCW logo] [DENI logo]

Report 98/47

An evaluation of the Computers in Teaching Initiative and Teaching and Learning Technology Support Network

September 1998

The electronic version of this publication contains Sections 1 to 4 only. The complete printed document including the appendices is available from the HEFCE.

Table of contents

Foreword
Summary and recommendations
Introduction
Section 1: The Computers in Teaching Initiative
Section 2: The Teaching and Learning Technology Support Network
Section 3: Current experience of institutions
Section 4: Recommendations for the future
Appendix A: Criteria for possible Subject Centres
Table 1:CTI Centres and funding
Table 2: Subject Centres and possible funding options
Appendix B: Membership of the Review Group and terms of reference
Appendix C: Results of heads of institutions survey
Questionnaire to heads of institutions
Appendix D: CTI users report
Appendix E: TLTSN users report
Appendix F: Institutional visits report
Appendix G: CTI visits report
Appendix H: TLTSN visits report
Appendix I: Abbreviations and acronyms

Foreword

On behalf of the four higher education funding bodies, I welcome this report and its proposals. Communications and information technology (C&IT) and computer-aided learning have proved to be successful and innovative features of learning and teaching in many higher education institutions. These new approaches have complemented other developmental work undertaken in subject-based activity. We need to integrate more effectively existing good practice and experience, so that learning and teaching are enhanced. The development of subject centres and subject networks will provide an important means of facilitating change and improvement in learning and teaching practice.

The report offers a thorough and balanced review of the UK-wide Computers in Teaching Initiative (CTI) and the Teaching and Learning Technology Support Network (TLTSN) within the larger picture. It looks at the complexity of meeting C&IT requirements in subject areas and the best way to provide support. The funding bodies will use this report to help decide on arrangements for the future. In doing so, they will build on the experience and successes of the CTI and TLTSN initiatives, and ensure that there are opportunities to retain the strengths of the present arrangements.

I am extremely grateful to the Review Group for their enthusiasm and commitment to this review. Their report provides a valuable insight into the role of C&IT in learning and teaching in higher education today.

Brian Fender CMG
Chief Executive


Summary and recommendations

Background to the review

1. The Review was established by the three Higher Education Funding Councils for England, Scotland and Wales, and the Department of Education Northern Ireland (DENI). It ran from November 1997 to June 1998. The Review had three aims. The first was to determine the extent to which the Computers in Teaching Initiative (CTI) and the Teaching and Learning Technology Support Network (TLTSN) had fulfilled their terms of reference. The second aim was to capture some of the lessons learned during the lifetime of these two initiatives. The third was to gather views and make recommendations about what should happen to these programmes at the end of their current period of funding, in the context of other programmes and other developments in higher education (HE).

2. The Review therefore took stock of the inter-relationships between the CTI, the TLTSN, the Teaching and Learning Technology Programme (TLTP), the projects of the Joint Information Systems Committee (JISC), and the Fund for the Development of Teaching and Learning (FDTL). Review Group members were also aware of the debates about the roles of the Institute for Learning and Teaching in Higher Education (ILTHE) and of the Quality Assurance Agency for Higher Education (QAA).

3. While all four funding bodies were involved in CTI and TLTSN they also had related initiatives of their own. There were, for example, important differences between England and Scotland in the history of the projects for staff development and for area networks. Consequently, a higher degree of collaboration between institutions had been achieved in Scotland. The Review was also able to make use of the findings of the 'Evaluation of the Learning Technology Dissemination Initiative' (LTDI) reported by SHEFC in 1997, and to build on it where appropriate.

4. The context in which the Review Group conducted its study was complex. On the one hand, there was evidence of considerable commitment to, and investment in, the use of computer-assisted learning (CAL) packages such as those developed under TLTP, and in the exploitation of information and communication technologies (ICT) such as the World Wide Web (WWW), to support learning and teaching.

5. As an agenda, CAL and ICT were clearly gaining in importance as institutions developed formal learning and teaching strategies and as many sought new markets through distance, open, work-based or lifelong learning initiatives. The great majority of students were thought to have access to a personal computer, and in the near future to expect CAL and ICT to be integrated into their programmes of study; an expectation believed to be shared by their current and future employers. Consequently, CAL and ICT were being seen as part of an essential set of tools to be used by all staff with responsibility for mainstream learning, rather than as a specialised and essentially minority interest. This implied that all academic staff now needed to be IT literate - as is recognised for other sectors of education. The drive was no longer for experimentation and awareness-raising, but for effective integration, appropriate use, and convincing evidence of enhanced learning.

6. On the other hand, it was also clear to the Review Group that under-utilisation of CAL and ICT remained a continuing, major problem for virtually all higher education institutions (HEIs) participating in this study. Many existing academic staff development programmes were relatively ineffective in supporting the use of CAL and ICT; and many academic staff were reluctant to use off-the-shelf materials, but did not have the time, skill or incentive to develop and customise their own. Further, while few any longer believed that use of CAL and ICT would reduce the cost of teaching, many were concerned by the scale of investment in connectivity, hardware, software, licences and recurrent user charges that lay ahead. Nor was it just a question of resources. There was unanimity about the need for change in attitude as well, especially on the part of senior and middle managers, and agreement on the levers which could be used to effect it. As one TLTSN client commented with regard to his own institution:

'What is required [here] is not more money or more technology, but more staff development. The momentum is such that one will be using learning technology, but not at the rate people expect: there is a need for education within a culture of resistance.'

7. It is against this background that the key findings from the Review need to be interpreted.

Key findings from the study

The current position

8. The CTI has fulfilled its terms of reference and in the eyes of its direct users has provided a good and valued service. There is no doubt that its subject orientation is the source of its strength and success. It has been less successful at reaching, systematically, heads of department or degree programme directors, and at working jointly with academic-related staff who have an institutional brief to support the use of CAL and ICT.

9. While it has proved difficult to find hard measures of impact, the TLTSN has, as intended, provided useful generic and specialist support to those institutions it has worked with. Given funding constraints and its short life, it has not worked with a large number of institutions, and has tended to be most effective in the smaller colleges and monotechnics. TLTSN Centres need to work at senior levels within institutions, but their staff have not always possessed the necessary experience or confidence in their own consultancy skills to be fully credible. However, the TLTSN has successfully built itself into an effective, self-managed network, which has partly offset the problems of communication and status.

10. The vast majority of respondents believe that CAL and ICT can enhance some aspects of learning. The main benefit perceived is the delivery of supported and collaborative learning to students at a distance, and greater flexibility for students in where, when and how they study.

11. Under-utilisation of CAL and ICT is seen as a significant problem by one-third of the institutions, and by the remaining two-thirds as a serious problem that has only recently begun to improve. In spite of this trend, HEIs do not, in general, feel that they have sufficient internal capability to deal with the problem of under-utilisation unaided. They still perceive a need for external sources of information, expertise and assistance in order to develop good learning and teaching strategies which incorporate CAL and ICT.

12. The need for long-term external help is of two kinds: subject-specific support, and advice on (new) generic technologies.

13. New categories of learning support staff such as learning technology officers and web editors are emerging with institution-wide responsibilities to help embed CAL and ICT. Sometimes they are seconded academics. They can be found in computing services, staff development units, CAL units, or attached to particular schools and faculties. The functional boundaries between them and other staff may currently seem blurred, but nevertheless, to be effective, they need access both to subject-specific and to generic technological expertise.

14. In addition, senior institutional managers responsible for teaching and learning strategies may need access in the short term to highly experienced, relevant and tailored advice. However, there are considerable differences among institutions of higher education in the nature and extent of the 'consultancy' that is required and desired. Management styles differ widely: approaches that might work well in a small monotechnic would be inappropriate in a large, multi-discipline and multi-campus institution.

15. There is a need for more research-based evidence of the use of CAL and ICT in higher education. This could be met through the recently announced programme of research into effective teaching and learning, funded by the HEFCE and designed, developed and directed by the Economic and Social Research Council (ESRC).

16. From these and related findings it seemed clear to the Review Group that the funding bodies now need to rethink their support for greater use of CAL and ICT in learning and teaching. This view was strengthened by the consistent and trenchant criticisms, coming from all parts of the study, about the way that the current set of centrally funded programmes has evolved historically over the last 10 years.

Criticisms and issues

17. There are now too many different projects and programmes in the use of ICT, funded jointly and severally by the Funding Councils and DENI (and indeed by other central agencies such as the DfEE). At Funding Council level the situation looks particularly messy, with variable membership of different programmes (and stages of programmes), overlaps in remit, and inconsistencies in budgetary mechanisms.

18. The plethora of initiatives is widely seen as confusing to end-users - a perception strongly confirmed in the user surveys conducted for this study. One consequence is that there is no 'one-stop shop' for information and assistance on a subject basis, nor on a generic technology basis. Meanwhile, staff in these different initiatives may themselves have only a partially accurate idea of the experience of other centrally funded projects and may find it difficult to refer enquirers to appropriate expertise.

19. While some diversity is probably needed, resources are being wasted through unnecessary duplication across different programmes (for example, in video-conferencing or in using the Internet for teaching). Many respondents were of the view that a large number of different programmes funded on a short-term and limited basis was not the best way of responding to the complexity of the ICT issues in the sector.

20. Although understandable historically, the absence of strong, integrated management of the different initiatives, including JISC, at Funding Council level is now seen as frustrating by the directors of project centres and by programme co-ordinators. They want better dialogue. They are supported in this view by other groups in this study who feel that the funding bodies are not getting as much useful collaboration between programmes as they could, and do not appear to be using the lessons learned by existing programmes when formulating new policy initiatives.

21. There are also significant gaps in coverage. Several subject areas (such as business studies, hospitality and leisure) have no specific CTI Centre and there is a perception that existing centres and indeed subjects are skewed towards the 'old' universities. Shifting boundaries between disciplines and the reconceptualisation of some subjects have not been adequately addressed by existing programmes.

22. Engagement with the crucial, operational level of middle managers, such as heads of department and degree programme directors, seems to be a particular challenge to existing programmes, while new roles such as those of the learning technology officers mentioned above are not being well supported either.

23. There is a major concern that existing initiatives and programmes do not generally recognise HE work franchised to further education (FE) or, indeed, FE-level work being taught in HE institutions - in spite of their growing importance. (The Review Group acknowledges that the funding of HE in FE is handled differently in Scotland and Northern Ireland.) Similarly, it appears potentially wasteful for ICT initiatives being funded centrally in HE not to be collaborating, at the very least, with similar initiatives in the FE sector. As HE and FE institutions increasingly share ICT infrastructure on a regional basis, there appears to be a real risk of wasteful duplication of effort between the two sectors.

24. In summary, projects originally designated as short-term have crept into long-term existence; new programmes have been introduced without sufficient regard to their impact on, and relationship to, existing programmes; inadequate co-ordination at the centre has resulted in wasteful duplication; and too little attention has been given to the difficult question: 'How will we know when top-sliced support should be ended?'. To meet these criticisms, and to support their requirements as institutions, respondents were consistent and clear about what was now needed.

Meeting needs in the future

25. There is substantial support across the sector for taking forward a reconceptualised programme on a subject basis. CAL and ICT would be an important but not exclusive focus of the new programme, which should embrace innovatory and good practice in learning and teaching in the subject comprehensively. Virtually all respondents believe that academic staff identify most readily with their subject and that, to be successful, implementation and integration of ICT have to be tackled from a subject perspective. Further, while acknowledging the special case of Welsh language instruction, respondents overwhelmingly want a UK-wide programme for subject support. The vast majority of participants in this study believed that to be effective this subject programme should be funded by a central top-slice.

26. There is equally a need for comprehensive advice on implementation and use of (new) learning technologies, and on the integration of pedagogic and management information systems (MIS). Respondents wish this also to be available from a single, albeit distributed, expert source. The current list of generic technologies includes: tools for customising materials on the WWW, video-conferencing, authoring software packages, conversion of stand-alone materials for distribution over a network, improving connectivity on and to campus, electronic management of the learning environment, and equipment and resources to support students with particular learning disabilities. Again, there is much to be gained from establishing such a resource on a UK-wide basis rather than having three or four separate national centres each trying to keep abreast of international trends. Respondents would strongly prefer funding for this resource also to be provided by a central top-slice.

27. Given the heterogeneity of insititutional missions, and the importance of building internal capability to implement teaching and learning strategies, there seemed to be little support for a semi-permanent UK-wide programme as successor to the TLTSN. Rather, short-term, ring-fenced funding which could be used flexibly by institutions (for example on benchmarking or on staff training) seemed better suited now to the complex and diverse developments within institutions.

28. It was suggested that (some) senior institutional managers need external support in handling the issues surrounding learning and teaching strategies. The four funding bodies might wish to respond to this differentially, building on their previous distinctive initiatives, and linking to other change-management agendas with a generic component. Alternatively, a UK-wide brokerage service for high-level consultancy in this area could be established. However, given the status issues involved and the scepticism with which management training can be viewed by senior academics, the consultancy service would need to be conceptualised very differently from the TLTSN if it is to be credible. For these reasons, the Review Group felt that it would be unwise at this stage to include senior management development in the remits of the centres and the unit it was recommending for the future. More work needs to be done on the kinds of support that would be valued and on where ownership of any service should lie.

Recommendations

29. The main recommendations of the Review Group can be summarised as follows:

a. Rationalise the existing set of programmes and initiatives on a UK-wide basis and establish a standing committee of the four funding bodies to achieve strategic co-ordination between them.

b. Wind up the TLTSN, and the CTI in its present form.

c. Establish new, broad-based Subject Centres as comprehensive one-stop shops and information gateways.

d. Ensure that all subject areas are covered on a UK-wide basis.

e. Establish a single Generic Technology Centre on a UK-wide basis.

f. Establish one Central Unit to manage all the programmes.

g. Provide earmarked support to institutions to implement their learning and teaching strategies.

h. Ensure that research into the effectiveness of CAL and ICT is conducted as part of the new ESRC programme on teaching and learning.

The arguments for the first two and last two points have already been covered. Fuller treatment of the rest is set out below.

Subject Centres

30. The Funding Councils and DENI should establish new, broad-based and reconceptualised Subject Centres to support the sharing of innovation and good practice, including the integration of ICT in learning and teaching. Subject Centres would be the one-stop shop and information gateway for academic staff, heads of department or equivalent, and degree programme directors. The Subject Centres must provide comprehensive coverage of subjects and disciplines offered in UK higher education.

31. Determining the appropriate size and number of Subject Centres is a matter of judgement. On the one hand, the centres need to have a critical mass of staff who can work as a team, avoiding over-dependency on one key individual (as has happened in some CTI Centres), and who can be proactive in the sector as well as responsive to enquiries. Efficiencies in clerical/administrative and technical support can be achieved with larger centres, and the final number of centres has to be manageable for the Director of the Central Unit. Larger centres also look more attractive to potential bidders.

32. On the other hand, to be successful, the new centres must reflect the way academic staff see themselves and the way they carry out their teaching. Most academic staff, heads of department and degree programme directors see themselves as working within clear subject or discipline boundaries. This is unlikely to change radically in the near future - particularly if the QAA endorses a lengthy list of discrete subjects as the basis for subject review. Further, if the centres cover too disparate a cluster of specialisms, it is likely that some will feel relatively disadvantaged in spite of the best efforts of the centre's staff. This has been the experience in some of the CTI Centres which span a particularly heterogeneous subject. Finally, involvement of subject associations and professional and statutory bodies is seen as crucial to the effectiveness of the new Subject Centres. These bodies need to map reasonably well onto the structure which emerges. Taking these and related arguments into account, the Review Group proposes a list of 20+ Subject Centres. Some comprise a single broad field such as biology, but others combine two closely related subjects, such as sociology and politics (see Appendix A). The Review Group acknowledges that no list will command universal support and that further fine tuning may well be necessary.

33. The functions of Subject Centres would be as follows:

a. Acting as knowledge brokers, promoting up-to-date concepts of good and innovatory practice based on evaluation and research in their subject area. Over time this should extend to European and international sources, and to comprehensive coverage of pedagogy and instructional design.

b. Facilitating the sharing of practical experience in the implementation and integration of innovatory and good practice.

c. Managing a network of departmental users.

d. Reviewing new teaching/learning materials which come onto the market and the application of new technologies in the subject context.

e. Disseminating outcomes of other subject-based initiatives.

34. The Subject Centres would operate as follows:

a. There would be a Subject Centre for each major subject area or small group of cognate specialisms recognised in HE. Each centre would be staffed according to the size of its 'constituency'.

b. The centres would be required to involve and work closely with discipline networks, subject associations and professional and statutory bodies in their field.

c. They would draw on the expertise of the Generic Technology Centre (see below), but would be expected to know where the generic technologies were being implemented successfully in their own subject.

d. Subject Centres should be based in institutions of higher education so that their staff remain in daily contact with academics and are attuned to the concerns and priorities on the ground.

e. Their performance should be judged on effectiveness of outcomes rather than by quantity of outputs, and contractual monitoring arrangements should be sufficiently flexible to take account of differences in the characteristics of their fields.

f. They would be managed by, and accountable to, the Central Unit (see below).

35. The funding of the Subject Centres would be handled as follows:

a. Subject Centres would be funded by a central top-slice to offer a standard service across the UK.

b. The standard service would be essentially free to individual users, but departments would be expected to pay a small subscription for membership of the network to ensure that they had some direct influence on the centres' work.

c. Departments/institutions would be expected to make a contribution towards any 'consultancy' which was specifically tailored to their needs beyond the standard service.

d. The funding of Subject Centres would be linked to the size of their constituency, including HE in FE, and should be given for five years in the first instance to enable appropriate staff appointments to be made.

Generic Technology Centre

36. The Funding Councils and DENI should establish a Generic Technology Centre, managed by a Director, to advise the sector strategically on the use of new technologies to support learning and teaching. It should draw on international developments as well as the UK experience.

37. The centre must be primarily responsive to the needs of users rather than to the leading-edge functionalities of new technologies for their own sake. Its relationship to JISC would need to be carefully defined and managed. One possibility is that JISC would be responsible for the early stages of experimentation with new technologies, including test projects. But once these stages were passed, and there appeared to be a good case for widespread use, the Generic Technology Centre would take over responsibility for dissemination and implementation, working closely with the Subject Centres. The Generic Technology Centre could also be responsible for identifying gaps in generic materials and tools from a user perspective, and for commissioning the necessary development, potentially in partnership with the private sector.

38. The functions of the Generic Technology Centre would be as follows:

a. Knowledge management and brokerage.

b. Maintenance of a demonstration and 'showcase' facility, both static and mobile.

c. Review, evaluation and strategic advice to the sector on the use of (new and emerging) learning technologies.

d. Dissemination of good practice and of experience of implementation from funded test, pilot and demonstrator projects.

e. Support of Subject Centres.

f. Commissioning of new or repurposed generic tools and materials.

39. The Generic Technology Centre would operate as follows:

a. Its clients would include the Subject Centres, senior staff in HEIs with strategic responsibility for learning and teaching, learning technology officers, and other academic-related support staff in universities and colleges who have a staff development role.

b. It would work closely with relevant professional associations such as the Universities and Colleges Information Systems Association (UCISA), the Association for Learning Technology (ALT) and with any similar initiative established by the Further Education Funding Councils of England and Wales (FEFCs).

c. It would manage its operation from one location based in an HEI - though its expertise would be distributed to make the most effective use of existing sources in all parts of the UK.

d. It would be managed directly by the Central Unit (see below), but in consultation with JISC.

40. The funding of the Generic Technology Centre would be handled as follows:

a. It would be funded from a central top-slice to offer a standard service on a UK-wide basis. It might, however, be expected to secure some external funding in cash or in kind from the private sector, albeit on a non-exclusive basis to protect its reputation for impartial advice.

b. Institutions wanting tailored 'consultancy' beyond the standard service would be expected to pay for

c. Funding should be for a fixed period, renewable subject to review.

Central Unit

41. The Funding Councils and DENI should establish one Central Unit, under a Director, to manage the Subject Centres, the Generic Technology Centre, and any other programmes funded specifically to improve learning and teaching, such as the FDTL and the TLTP. This would mean ending the existing arrangements whereby each major programme and initiative has its own support and co-ordination team. The Review Group acknowledges that differences in contractual arrangements between existing support services would mean a short period of messy overlap, but the advantages and efficiencies to be obtained are considerable.

42. The functions of the Central Unit would be as follows:

a. To manage all the programmes, including procedures for the appointment of key staff.

b. To identify and resource professional development and training for staff in the various centres.

c. To implement target setting, evaluation and feedback processes for centres.

d. To provide advice to centres on marketing and public relations in the widest sense, while ensuring that users receive a co-ordinated and efficient set of communications.

e. To negotiate publishing, web-site management, and other service contracts on behalf of the centres using (software) publishing houses as appropriate.

f. To network between the Subject Centres, the Generic Technology Centre, JISC and the funding bodies, maintaining two-way dialogue between those developing policy for learning and teaching at Funding Council level and those with experience of user concerns and requirements. The unit should be the integrated source of information and advice, and the single point of engagement, for the funding bodies.

g. To liaise with the FEFCs, and with relevant programmes funded by other central agencies, to ensure that mutually beneficial collaboration occurs where possible.

h. To source briefing papers and seminars periodically for the sector which summarise and analyse emergent trends, good practice and state-of-the-art implementation.

43. The Central Unit would operate as follows:

a. Responsibility for funding and monitoring the work of the unit would rest with the funding bodies. However, it is unlikely that the funding bodies would wish themselves to have day-to-day management responsibility for the work of the unit. The Review Group therefore recommends that the unit be made the subject of a UK-wide tender. This could permit the possibility of public/private partnership if the funding bodies so wished. Ultimately, the unit could be moved to the ILTHE. However, for the medium-term, the ILTHE is seen as necessarily preoccupied with issues of accreditation and registration.

b. It would be located in one physical place. This could either be at one of the funding bodies' sites to ensure maximum dialogue, or in a neutral venue elsewhere.

c. It would work closely with the JISC Secretariat.

d. It would report regularly to the joint standing committee of the funding bodies and their respective national committees on trends in user requirements and expectations; on the need for new initiatives or for R&D in response to new technologies; and on opportunities for collaboration between HE and FE to achieve greater effectiveness on the ground.

Earmarked funding

44. Within the block grant to institutions, monies need to be identified to support the implementation of learning and teaching strategies, including the use of ICT. Institutions would wish these monies to be additional to the existing level of grant (achieved, for example, through reduction of top-sliced items elsewhere). There is a strong case for such monies to be earmarked, and allocated on receipt of evidence as to how their use relates to the institution's own plans for strategy implementation. Within these general constraints, the Review Group recommends that institutions should be free to use such monies according to their own perceptions of need and priority.

Tenders

45. The Review Group strongly recommends that the new Subject Centres are established from a competitive UK-wide tender open to all higher education institutions, and possibly to subject associations jointly with them. The funding bodies will want to ensure that Subject Centres are located in fertile teaching and learning environments. This could be evidenced in the bids in a variety of ways, including innovatory practice which is not restricted to ICT and CAL.

46. The Generic Technology Unit should also be the subject of a UK-wide tender. Bids could be made collaboratively or as a consortium between different HEIs and could include plans for private sector involvement. The funding bodies would want to see evidence of a well-resourced environment in which advanced technology had been brought to the service of pedagogy.

47. The Central Unit should be put out to tender if the funding bodies do not themselves wish to assume day-to-day management responsibility for it. Again, there may be advantages in permitting public/private partnerships to bid.

Timetable and migration path

48. The Funding Councils' Boards and DENI would need to agree the extra resources in 1998. The invitation-to-tender documents for the Subject Centres, the Generic Technology Centre and the Central Unit could then be prepared in the autumn and a decision taken by the end of 1998.

49. The new Central Unit would start work in January 1999, with the new Subject Centres and the Generic Technology Centre coming into being on 1 April 1999.

50. The funding bodies will wish to consider extending the funding of the TLTSN Centres for a few more months. The current funding of the CTI runs out in the summer of 1999 and would not need to be extended further.


Introduction

51. This study was established to review two UK-wide programmes which are funded by the three Higher Education Funding Councils (HEFCE, SHEFC, HEFCW) and by the Department of Education Northern Ireland. The first of these programmes is the Computers in Teaching Initiative which was established in 1989. The CTI currently consists of 24 subject-based centres. It also has its own support service (CTISS). The total annual budget from the funding bodies is £1.25 million, and the host institutions are also required to meet some additional costs. Under their terms of reference the CTI Centres are expected to:

a. Advise academic departments on the availability and suitability of computer learning within the context of institutional guidelines.

b. Provide detailed information on the materials available in a subject, a resource guide and regular reviews of teaching materials.

c. Visit departments at regular intervals or hold workshops for departmental staff.

d. Assist in disseminating products arising from the funding bodies' Teaching and Learning Technology Programme.

52. The second programme to be reviewed was the Teaching and Learning Technology Support Network which was set up in January 1996. This followed the funding bodies' decision to allocate continuation funding to several of the TLTP institutional projects. The total annual budget for the TLTSN is £400,000. The host institutions are also required to meet some of the additional costs. There are currently nine TLTSN Centres. The aim of the TLTSN is to:

a. Apply the experience of the institutional projects in the TLTP.

b. Support and encourage UK higher education institutions seeking to integrate technology into mainstream teaching and learning.

c. Enhance the quality of provision for student learning.

53. The Review had three aims. The first was to determine the extent to which both programmes had fulfilled their terms of reference. The second aim was to capture some of the lessons learned during the lifetime of these two initiatives, and to contribute to the debate about how to maximise the effectiveness of programmes of this kind. The third aim was prospective in nature: it was to gather views and make recommendations about what should happen to these programmes at the end of their current period of funding, in the context of other programmes and other developments in higher education.

54. The formal terms of reference and membership of the Review Group are given in Appendix B.

55. The Review was conducted between November 1997 and June 1998. Evidence was gathered from several sources:

  • the CTI Centres and CTISS
  • the TLTSN Centres and their Co-ordinator
  • surveys of users and clients
  • visits to a stratified sample of 14 HEIs
  • a questionnaire sent to all heads of institutions
  • a 24-hour residential meeting (with representatives from different HEIs and the funding bodies, as well as key individuals in the field)
  • an electronic discussion forum established for the purpose.

56. In addition, the Review Group read samples of the literature produced by the CTI and TLTSN, and documents prepared by them especially for this Review. The group also received a few letters from subject associations in support of particular CTI Centres.

57. Fuller details of the methodology and a set of findings are presented in Appendices C to H. Readers should note that there were practical difficulties in the survey of CTI users which may have affected the response rate. It should also be pointed out that the sample of TLTSN clients surveyed was nominated by the TLTSN Centres themselves, and was not selected by the Review Group on a random and independent basis.


Section 1: The Computers in Teaching Initiative

Roles and objectives

58. CTI Centres themselves identified four main roles, the balance between them depending on the 'maturity' of use of ICT in their respective constituencies. These roles were:

  • being an information point
  • giving advice on implementing aspects of ICT in teaching/learning the subject
  • providing a link between old and new technologies
  • enabling change in the teaching/learning of their subject.

59. The last of these was the role most frequently endorsed. The majority of centres believe that over the last three years their emphasis has been changing towards the use of more general ICT tools and techniques to enhance learning, particularly in relation to use of the World Wide Web, and away from the delivery of subject-specific course material by computer ('courseware'). They also reported a greater sophistication in the technical and pedagogic level of questions received, and an increase in the number of enquiries relating to staff development. There is a clear trend away from rather simple awareness-raising towards the more complex issues of integration and implementation of ICT. This trend has been assisted by centres' web pages, which are now designed to deal with routine requests for information and to answer the most commonly asked questions.

60. Not surprisingly, the centres felt they worked most effectively when their constituency was IT literate and enthusiastic about computer-based learning. A few of the centres felt they had successfully engaged with cross-disciplinary issues, but most preferred working with well-defined subject boundaries. On balance, they found that the 'new' post-1992 universities were more receptive than the 'old', especially where departments had devolved budgets and could make decisions about resources. Active, positive links with subject associations were important for credibility, for access (for example, to heads of department), and for leverage of effort. CTI Centres varied in their perception of the activities which yielded the most value; this seemed to depend primarily on the characteristics of their subject area. Thus face-to-face events, open days, road shows, hard-copy newsletters, the web pages, and networking across the sector all received endorsement.

61. The survey of users confirmed these perceptions of roles and trends. Users join CTI mailing lists and attend their workshops principally for the up-to-date information obtained, and for the opportunity to gain hands-on experience of computer-based learning materials and ICT-based approaches to teaching.

Effectiveness of the CTI Centres

62. CTI Centres themselves believe they have been effective. All are now putting monitoring and evaluation plans in place to provide better information on the outcomes of their work. Evidence of effectiveness included the following: numbers attending workshops and other events; repeat attendance or requests for return visits; recommendations and referrals; recognition by the subject associations; results of user surveys; number of 'hits' on their web sites; and evaluation of feedback sheets handed out at events.

63. The CTI users who responded to the survey were consistently pleased with the quality of the service they had received from 'their' CTI Centre. Over three-quarters of mailing list users and key contacts were satisfied with the literature they were receiving; only a very small proportion (4.5 per cent) wanted more reviews of specific software or other material. Two-thirds of workshop attendees felt that nothing extra could reasonably have been expected from the workshop. Overall, 88 per cent of users described the service they had received as good or excellent, and only 1.8 per cent regarded it as poor. Just over half of all users said that they had made changes to their teaching - usually by extending existing activities - as a result of attending a workshop or reading CTI literature. Of these respondents, 90 per cent felt that the change they had made would be sustained. There was some variation between centres in the level of endorsement received, but overall in the eyes of its immediate and direct users the CTI has clearly fulfilled the objectives it set itself since the last evaluation in 1994. There was also no doubt that the users wanted some form of CTI to continue.

64. Most CTI Centres felt they had been less successful in reaching middle managers, such as heads of department, or academic-related staff in computer support services and staff development units. This perception was confirmed in the institutional visits; in 12 out of 14 universities and colleges, middle and senior managers did not know whether or not the CTI had been active in their institution. So although there was widespread recognition of the CTI as a subject-focused initiative, no use had been made of it for strategic planning, for technical advice, or for high-level staff development. Indeed, interviewees recorded mixed views about the value of the CTI (and observed that some centres were better than others); there were concerns about its inaccessibility to the inexperienced practitioner, alongside praise for openness and user-friendliness.

65. Compared with the direct users, there was also less enthusiastic endorsement of the CTI recorded by those who completed the institutional questionnaire. It has to be remembered that those with an institution-wide management responsibility may be unaware of the extent of CTI involvement at individual academic level. Certainly, most respondents skipped over those items which called for a more detailed judgement on the factors contributing to the success or failure of the CTI. With this caveat in mind, it should be noted that less than half of institutional respondents felt that the CTI had been even partially successful in raising awareness, and only a little over a quarter felt that it had been successful in supporting the uptake of CAL. Perhaps most surprisingly, a third of respondents felt that the CTI's impact in providing subject-specific support had been negligible, and just under half rated it as only 'fair'. Nevertheless, there was support for continuation of a top-slice to sustain a CTI-type, subject-based programme.

66. Consistent threads running through all the sources are that any future programme will need to be proactive, to focus pragmatically on implementation and integration, and to have a subject-specific identity. Ways have to be found of meeting the needs and interests not just of a minority of enthusiastic academics, but also of departments and their managers more broadly within the institutional context.

Management of the CTI

67. Many of the centres praised the current Co-ordinator and the support provided by CTISS. The regular forums, where centre staff were able to exchange views and experiences as well as obtain briefings, were particularly welcomed. There were some concerns expressed about the underlying tensions in the Co-ordinator's role, and the difficulty of meeting the diversity of support now needed across the centres. There was substantial criticism of the way in which the funding bodies had handled the CTI; these points are taken up later in the report.


Section 2: The Teaching and Learning Technology Support Network

Roles and objectives

68. The TLTSN Centres felt they were working with two main roles: assisting senior managers in institutions predominantly but not exclusively in their region; and acting as a source of specialist advice on a national basis, reaching a wider spectrum of staff. These roles are directly in line with the original aims for the TLTSN when it was established in 1996. On balance, the centres felt that the former role resulted in more change. This meant that they saw their job primarily as that of facilitating change in accordance with the needs of a particular client HEI. The consultancy in which they were engaged embraced awareness-raising, especially to catch the attention of senior managers; advice on drafting policy documents; assistance with writing bids, such as those for TLTP 3; sharing their own institution's experience of implementation; and disseminating good practice from across the sector.

69. Although they organised and participated in a number of one-off events such as speaking at conferences, and raised the profile of the TLTSN through publishing articles, the main aim was to establish long-term relationships with HEIs. This was because centres felt they were at their most effective when helping senior staff to develop strategies, or at least raise appropriate questions, and when they were building confidence in the possibility of change by providing case studies and success stories. Rather than a quick 'in and out' approach, they preferred to provide on-going assistance as an institution evolved its teaching and learning strategy through several stages.

70. Given its level of resourcing, this meant that the TLTSN was limited in the number of institutions it could reasonably handle in two years. The centres also seemed to be most effective with smaller (and newer) universities that were 'ready for change' and that wished to catch up with reasonable expectations about ICT use in larger and older institutions. These smaller institutions were comparatively straightforward as clients because they had a fairly simple management structure, with a top-down implementation and resourcing policy. The right key staff could be identified, and both formal and informal processes of decision making were easier to work with. By contrast, the centres reported that large HEIs were a much more complex proposition, with distributed decision structures and opaque internal politics that made them less receptive to external assistance.

71. The direct users who were surveyed confirmed that the roles identified by the TLTSN Centres had been the reason for getting the TLTSN involved in their institution. Clients had multiple objectives of their own when approaching the TLTSN for assistance. Just under half had approached it with a very clear and specific need, such as a one-day symposium for staff, or help with designing web-based learning materials for students with disabilities. A third of the users brought in the TLTSN to raise awareness of the potential of teaching and learning technologies generally, and to get staff across the institution talking to each other and alerted to new developments in the sector. Half had approached the TLTSN to identify and form links with relevant projects and people in other institutions, either to help their own staff directly, or to provide market intelligence on what was going on elsewhere. But the majority of clients had used the TLTSN to reach senior management, for example through sessions on funding or on barriers to institutional change and strategies for overcoming them.

Effectiveness of the TLTSN

72. The TLTSN Centres and their clients had difficulty in determining the precise extent to which the TLTSN's intervention had been effective. With the exception of one bad experience, clients in the user survey praised the process of engagement: TLTSN staff had been hard-working, helpful and well organised. The staff presented well and possessed good public relations skills. The information they disseminated was seen as relevant, up to date, and attuned to the needs of the client. They were also regarded as effective because they could use their own networks of contacts to refer a client with a specific problem to an appropriate source of expertise, and because they came from outside the institution and so conveyed a sense of well-informed objectivity.

73. The survey confirmed that those who had attended TLTSN sessions were able to feed back useful information to their institution, had learned about new teaching and learning technologies, and were more aware of the issues in managing change of this kind. TLTSN staff had also helped to put TLTP 3 bids together and assisted with writing other proposals.

74. Beyond such immediate feedback, however, clients felt it was hard to quantify impact. New ideas for using teaching and learning technologies have a long incubation period, and successful implementation depends more on institutional factors such as resourcing than on the input from the TLTSN, however good. This view was shared by the TLTSN staff, who felt that it was difficult to measure the medium or long-term effect of their involvement with an institution. This difficulty existed for a number of reasons: their input was just one among several factors contributing to change at any one time; change was very slow; and changes in people's attitudes are not particularly tangible.

75. One major difficulty reported by the TLTSN Centres was the lack of seniority of many of their own staff. This was not just an impediment to direct access to senior management teams. It also limited the effectiveness of the consultancy because they had little technical knowledge of institutional planning and resourcing, and limited first-hand experience of decision-making processes at senior management level on which to draw. Nor had many TLTSN staff had much experience of implementing change on an institutional scale. When the TLTSN was originally conceived, there appears to have been an expectation that staff who had been directing the TLTP institutional projects would move across into the TLTSN. This did not always happen and new, more junior staff were frequently recruited. Alternatively, institutions that hosted TLTSN Centres created hybrid posts by pooling TLTSN and other sources of funding, with inevitable blurring of responsibilities and time commitments.

76. Although training for TLTSN staff was organised early in the network's history, it was unable to provide the kind of insight and skills needed for acting as (sophisticated) change agents. Indeed, in the circumstances it would have been hard to do so. Subsequently, the centres themselves made an excellent job of providing mutual support, going out to clients in pairs and exchanging information about particular institutions.

77. Given the small number of institutions with which the TLTSN was able to work, it is not particularly surprising that a predominantly negative view of its activities emerged from the institutional survey and visits in this Review. The majority of the questionnaire respondents omitted items asking for a more detailed judgement of the factors which had contributed to the success or failure of the network, which raises the issue of the extent of their knowledge of the TLTSN. Nevertheless, two-thirds of the survey respondents felt that the TLTSN had limited or no impact on supporting the integration of ICT; a third felt that it had only negligible impact at senior management level; while a third said their senior managers had no contact with the network.

78. Ambivalence about the TLTSN is discernible in the views expressed during the institutional visits. Of the 14 institutions visited, two had no contact with the TLTSN, five expressed some criticism (for example, of their lack of experience or partiality to particular products), three made no evaluative comment, and four had used the TLTSN and found it useful at a strategic level. With one exception, the institutions visited had not taken advice from the TLTSN on their staff development policy, or approached the network for technical advice.

79. However, in spite of these criticisms, half the respondents thought that it would be useful or very useful to have external consultancy support at senior level. Although agreement to continue the TLTSN funding was weaker than for the CTI, it was still positive.

Management of the TLTSN

80. All the centres praised the current Co-ordinator. However, they would have benefited from more closely targeted arrangements for staff development. Additionally, the uncertainty of year-on-year funding had made it difficult to justify spending money on staff training. Until very recently, the network itself had not taken on responsibility for organising staff development in anything other than an informal 'on the job' basis.

81. The TLTSN was strikingly effective as a self-managing network, which is an achievement that deserves praise. Furthermore, part of its dissemination strategy had been to publish case studies of the experience of client institutions. However, it was less clear to the Review Group that the TLTSN had given quite as much thought to the benefits of networking its clients directly to each other, so that they could exchange experience and discuss common obstacles together.


Section 3: Current experience of institutions

Strategies for learning and teaching

82. Many respondents in this study believe that the majority of HEIs are moving towards a learning and teaching strategy which incorporates CAL and ICT. The form and nature of these learning and teaching strategies vary widely, although there is widespread agreement that they should not be conceived separately from institutions' information systems (IS) strategies. It is also worth noting that while 35 per cent of respondents to the institutional questionnaire believed that under-utilisation of ICT was a significant problem for them, 63 per cent saw it as a problem, but one that had recently improved, and by implication would continue to do so. Only a handful of institutions claimed to have already implemented a learning and teaching strategy. For most respondents, implementation was seen as a considerably more complex and difficult task than generating the strategy in the first place. This may well account for the overwhelming endorsement (by 95 per cent of respondents) of the need for continued external sources of expertise and assistance in ICT.

Perceptions of the benefits of ICT and CAL

83. In this study there was considerable agreement across all sources as to the benefits derived from incorporating CAL and ICT more fully into learning and teaching. Chief among these were the delivery of supported and collaborative learning to students at a distance, and greater flexibility for students in where, when and how they studied. No one doubted that open, distance and lifelong learning will continue to be important drivers for ICT, and generally students are thought to welcome opportunities for collaborative and networked learning.

84. Another perceived benefit was the greater employability of students by virtue of their ICT skills and their familiarity with the World Wide Web. Indeed, where ICT is used in the daily work of professionals in a particular field, that in itself is a strong driver for incorporation of ICT into initial training programmes.

85. In general, the assumption that appropriate use of ICT could enhance aspects of student learning was supported. This was provided that students were trained to use the technology appropriately, and provided that the learning materials they worked on were of a high standard - in terms of both the subject content and the pedagogical design. Without the application of sophisticated skills and understanding, the information accessed by students would not be structured and transformed into useful knowledge. Nevertheless, respondents in all parts of this study felt that the case had not yet been made convincingly that CAL or ICT produced enhanced learning. Indeed, both the CTI and the TLTSN Centres regarded the lack of applied research in this area to be a barrier to the credibility of their work.

86. The costs of resourcing an appropriate learning and teaching strategy were seen to be high in terms of hardware, infrastructure, development or purchase of learning materials (with attendant licences), recurrent user charges, and crucially in staff time. However, only one institution visited in this study had tried to apply a methodology for determining the cost-benefit ratio of different types of investment. The remainder of the institutions felt it was too difficult or too early to do so. The vast majority of respondents from all sources doubted that use of CAL or ICT had cut the financial costs of teaching and learning, or would do so in the future. The respondents did not expect to see crude efficiency gains accruing to the institution from implementing a teaching and learning strategy that incorporated ICT.

87. Where efficiency gains had been identified, they lay in subject areas which could make good use of stable data sets, texts or records, and in remedial/service teaching of basic subjects such as mathematics to a non-advanced level. There was also a strong view that collaboration across HEIs was needed in the production of new courseware, to reduce the high development costs. However, given the paucity of data on costs of teaching and learning using traditional methods, such views and assumptions must be largely speculative.

88. Interestingly, institutions in this study did not seem to have worked out a funding methodology for CAL and ICT. Unlike provision of books and journals in the library, purchase of CAL packages and ICT tools was often borne, erratically, by departmental budgets rather than by an institutional top-slice, and this applied even to universities with converged learning support services.

Barriers to fuller use of CAL and ICT

89. Drawing on all the sources in this study it was possible to analyse the current view of the barriers to greater utilisation of CAL and ICT.

90. Barriers fell into three distinct groups: money, materials and mindset.

Money

91. More resources had to be found for the following: computer workstations housed in suitable premises offering good access to students; greater network bandwidth and better connectivity on campus, and between campus and residences, digs, homes and places of work; improved compatibility between platforms; more support staff who have both technical and subject expertise; more help-desk facilities; more courseware and associated licences and upgrades; and more training for staff and students.

Materials

92. The problem here lay in a widespread perception that in many subject areas there was insufficient relevant, high-quality courseware available. Where it did exist, it could seem expensive to purchase. 'Productivity' tools which would allow material to be repurposed, customised and integrated more easily into existing courses were still the preserve of enthusiasts. Many lecturers did not have the appropriate skills in instructional design to use authoring tools well. Since lecturers were thought to take professional pride in developing and delivering their own materials in ways suited to their particular students, the lack of these skills was a major disincentive to get involved in CAL.

93. Some respondents thought that academic staff were beginning to realise that they would have to accept ready-made material if they wanted high-quality multi-media resources. However, the lack of applied research into the effectiveness of different uses of ICT, or into specific CAL packages, and also the absence of authoritative advice on implementation, were further disincentives to engagement. It takes far longer to 'open up' a multi-media package and check its suitability as a learning resource than it does to 'gut' a textbook.

94. There were divided views on whether the involvement of commercial publishers in the multi-media market would help to overcome the problems of shortage in the longer term. Many respondents were sceptical that commercial publishers would be interested in developing learning materials that did not have a global or at least a North American mass-market appeal. However, collaboration between institutions in the production of new materials was seen as highly desirable, especially when the software provided a framework into which case-study material specific to each institution could be inserted.

Mindset

95. Across the study, technophobic or otherwise hostile senior and middle managers, including heads of department, were seen as a major barrier to further progress. Without senior and middle-level champions, supported by appropriate committee ownership, the under-resourcing and under-utilisation of ICT would continue. And without such commitment it was likely that the institution would lack a clear sense of direction for its learning and teaching strategy. The institution would implement the strategy badly, would be unduly put off by technical failures, and would continue to exhibit versions of the 'not invented here' syndrome. This would make the sharing of experience, products and outcomes across HEIs considerably harder. There would also be adverse consequences for staff development programmes, many of which were currently regarded as inadequate to support the implementation of ICT on a subject basis.

96. A second aspect of mindset concerns the relative valuing of teaching and research, and the perceived absence of rewards and incentives for using new technologies. The problem manifests itself in the lack of time that academic staff are prepared to spend on developing and implementing ICT and CAL in their own teaching, or in the programmes of study for which they are responsible. This issue has been widely rehearsed in the Dearing Report, in the funding bodies' consultation on their next Research Assessment Exercise (RAE), and elsewhere. It is not surprising, therefore, that across the study respondents felt that academic staff were giving first priority to their research, and did not have the time to innovate and integrate new approaches into their teaching. They were unlikely to be willing to spend a great deal of time in staff development sessions acquiring the advanced IT skills needed to customise existing CAL or World Wide Web materials.

97. It was widely stated that the design and development of new CAL materials was not regarded in the last RAE as equivalent in prestige to more conventional forms of research, thus denying recognition to those who had invested a considerable proportion of their time in TLTP and similar projects. It was also believed that, with the exception of the Education Unit of Assessment in the RAE, articles on studies on learning and teaching attracted less weight than research that contributed to the substantive knowledge base of the subject. Therefore, while respondents recognised that many HEIs had recently tried to give greater emphasis to learning and teaching in internal promotion criteria, lecturers and their heads of department were still thought to be unwilling to risk a substantial increase in the time devoted to implementing ICT in modules and programmes of study.

Levers to fuller use of CAL and ICT

98. When identifying levers or drivers for greater utilisation of CAL and ICT, many items given were the reverse of the barriers already described above. (For example, that it would be helpful if the next RAE permitted staff to be entered both for their subject and for the Education Unit of Assessment if some of their publications focused on learning and teaching.) The levers can be categorised under staffing, strategies and standards.

Staffing

99. Commitment from the senior and middle managers in the institution was seen as essential. Although the energy of enthusiasts remained a powerful lever, it was also helpful if a critical mass of staff were computer literate and were generally familiar with the WWW. This made for more receptive attitudes to change. It also became easier to target staff development at implementation within subjects rather than limiting it to basic, generic IT skills. Finally, the gradual replacement of less positive staff by newly appointed staff who took ICT very much for granted, and who might even have it listed in their job descriptions, was mentioned by several respondents as a useful lever.

Strategies

100. Commitment was essential, but it needed to be translated into effective institutional strategies if maximum leverage was to be obtained. Institutional policy which combines IS and ICT elements, a committee structure dedicated to the enhancement of learning and teaching, and support staff with clear ownership and responsibility for implementation of ICT and CAL, are all vital levers. Patience, and recognition that staff need to be supported over time if they are expected to re-skill themselves or make fundamental changes in their practice and outlook, are important elements of effective strategies. Few doubted the potency of adequate resources prioritised from the block grant, supplemented where possible by external income from industry or from IT-related projects. A proportion of this investment had to be handled centrally to ensure compatibility of equipment and adequate investment in infrastructure. However, it was also regarded as important for individual departments or programmes to be able to purchase software and associated licences for learning materials specifically designed for their field.

Standards

101. A powerful lever was thought to be networks that provided an institution with up-to-date information on what was being done elsewhere in the sector, and what was emerging as good practice. Demonstrator projects, consortium projects for courseware development, and the sharing of experience through case studies and support groups were all seen as the start of benchmarking. By contrast, however, kitemarking of CAL products by a central agency was not supported because it was felt to be too rigid: materials are used in many different ways and for different purposes. Meanwhile, one of the real challenges in the future would be to move towards an agreed common specification for the development, use and management of an ICT learning environment. This would include the handling of student attainment records, assessment methods, and tools for rapid customising and repurposing. In this connection, mention was made of the US National Learning Infrastructure Initiative; this has an Instructional Management System Project focused on this area, of which JISC is a 'partner' member.


Section 4: Recommendations for the future

102. There were consistent, widespread and strongly endorsed views from all sources that contributed to this study on what now needs to be done at a national level to give further support to the use of CAL and ICT. These views can be categorised into:

  • rationalisation of initiatives
  • new management structures
  • need for research
  • links to accreditation and quality assessment regimes
  • funding models.

Rationalisation of initiatives and programmes

103. The overwhelming majority of respondents in all parts of this study said that there were now too many projects supporting the use of ICT in higher education. They felt that this situation was very confusing to end-users, as well as wasteful in terms of duplication of effort by the projects themselves. While some diversity and redundancy were probably still required in a field characterised by a young technology, the present plethora of projects was far too great. Paradoxically, it was making it harder for people with common interests to work together. Thus, 70 per cent of respondents to the institutional survey wanted either rationalisation or better co-ordination; a further 20 per cent favoured reduction to a single initiative. Only 12 per cent favoured continuation of the current diversity. Over two-thirds of CTI users could not say how the remits of the most prominent programmes differed; even some staff in CTI and TLTSN Centres felt a little at a loss to say precisely where the boundaries between programmes now lay.

104. Confusion was reported between all of the following: the CTI and TLTSN; the CTI and TLTP; the CTI and particular JISC-funded projects such as eLib, JTAP and Netskills; between the TLTSN and JISC-ASSIST; between the TLTSN and the role of CTISS; between CTI Centres and some of the technology-using projects in FDTL; and between initiatives north and south of the Scottish border. Interviewees in the institutional visits were also uncertain as to whether the CTI and TLTSN were driven by considerations of pedagogy or technology.

105. It was not clear that individual projects or centres were well networked across programmes. Overall, acknowledgment of each other was relatively limited. Referrals of clients and users between the TLTSN and CTI appeared to be predominantly in one direction, and unsystematic at best between the CTI and FDTL, and between the CTI and JISC. While both the CTI and TLTSN were, in effect, in competition with each other to develop as the one-stop shop, neither had achieved this ambition.

106. The problem seemed to lie in the co-ordination and support structures which had been established within programmes and initiatives - but not across them. Arguably this has provided co-ordination on the wrong axes. Users wanting comprehensive information on best practice in their subject, or on how best to use a generic technology such as video-conferencing still have no single information gateway: they have to hunt around between several projects in different programmes and initiatives. And these are precisely the needs that users have. By contrast, they are less likely to want to know the details of all CTI Centres or all FDTL projects.

107. Similarly, while users recognise and value the expertise of staff in the centres, they also wish to talk to each other to share actual experience of change, implementation and integration. This does not seem to be happening systematically. As one client said:

'We need a hands-on speaker who has overcome the problems involved in setting the timetable, monitoring participation, and identifying those students lost in cyberspace...'

108. Following these arguments, it is possible to identify a set of criteria for rationalising the current plethora of initiatives:

a. Focus on the information and knowledge needs of the real end-users. This means retention of a subject focus, but one that goes beyond ICT to be comprehensive and inclusive of good practice sector-wide. There needs to be a one-stop shop for each subject.

b. Rationalise around provision of expert help in evaluating and implementing new generic technologies, including the development and use of learning environments.

c. Introduce brokerage between those who have personal experience of integration and/or implementation, at subject or institutional level, and those who wish to learn from that first-hand experience. Users should be able to share and collaborate with each other and not just with the staff in particular project centres.

d. Ensure that co-ordination supports users' needs and is not structured on a historical basis programme by programme.

e. Pave the way for better co-ordination between the funding bodies, and between the funding bodies and other central agencies currently acting separately in this area, for example the FEFCs, the Scottish Office Education and Industry Department (SOEID) and the DfEE.

109. Having been critical of the current situation, it must be repeated that only a small minority of respondents were in favour of ending top-sliced provision of support for ICT use. The vast majority of respondents in all parts of the study wanted an external source through which to access up-to-date information, advice and expertise, and to share experience of integration, implementation and change. This source was needed by different groups within institutions at two levels: the subject-specific/departmental, and the generic/institutional.

New management structures

110. In order to effect the rationalisation and co-ordination urgently desired, the majority of respondents in all parts of the study made recommendations on how the funding bodies should discharge their management responsibilities for top-sliced provision.

111. First and foremost there has to be stronger central direction and control exercised by the funding bodies over whatever programmes are established in the future. This needs to happen at two levels. At a strategic level there needs to be a joint forum of the funding bodies, such as a standing committee. In that forum, a common position on teaching and learning priorities, funding contributions, and a policy framework could be hammered out, to avoid the confusing inconsistencies of the present situation. Second, at an operational level, there needs to be a single source of hands-on management of teaching and learning programmes and centres. The relationship between this unit and JISC has also to be determined and monitored. At present, the absence of a single source of direction leads to unhelpful and wasteful overlap in scope, remit and ambition. It also perpetuates divisions between pedagogy and technology at a time when they have to be combined by institutions into strategies for learning and teaching.

112. Better two-way dialogue between the funding bodies and whatever programmes are to be established in the future was a major concern for all those with experience of the current co-ordinating arrangements. Both CTI and TLTSN staff felt that they had inadequate contact with officers at the funding bodies, particularly those concerned with the development of policy for learning and teaching. They also felt inadequately briefed on new initiatives as they emerged. As a corollary, the CTI and TLTSN staff felt that the funding bodies had failed to use the extensive knowledge and experience of the programmes as a resource, for example when designing and launching TLTP 3. For their part, senior officers of the funding bodies appeared to be subject to considerable lobbying and advocacy from each of the major programmes and initiatives individually. The resulting politics of influence are understandable, but not necessarily the most effective way for the funding bodies to acquire coherent briefings or disinterested advice.

113. Contact between the existing programme co-ordinators has in the main come about informally through their own efforts, rather than those of the funding bodies. This may have slowed down effective collaboration. Further, the informal meetings have revealed, among other things, duplication of effort in the training of staff in the respective projects, with less sharing of experience than might be considered sensible. (This includes experience in, for example, the skills of designing a successful workshop, or strategies for disseminating outcomes to a wider audience, or designing and managing web pages.) Once again it is not clear that the sector is benefiting from having several support services, each with its own co-ordinator, when so many of their functions are similar: target setting and monitoring of performance, staff training, printing and publishing, liaison and networking.

114. The question of whether any future programme should serve regional generic needs or offer a specialism on a national basis proved to be a difficult issue for the Review to resolve. Respondents to the institutional questionnaire were evenly balanced: 27 per cent preferred the regional approach; 26 per cent opted for the national specialist role. However, those interviewed in the institutional visits were much more positive about the national role.

115. Several respondents suggested that it was possible to achieve both roles if the specialist centres were part of a geographically distributed network. This was of course the thinking behind the TLTSN, but the Review Group concluded that it was a far from satisfactory outcome. It turned out that three of the centres lay close to each other in the East Midlands, with little coverage being offered to the North-East, London, or the far South-West. Through its location, the TLTSN Centre in Wales had better access to HEIs in England than to those in South Wales, yet had quite properly a concern for Welsh language instruction; and the centre in Northern Ireland had (currently) only two institutions to serve. The position in Scotland was complicated by the existence of the Learning Technology Dissemination Initiative, which had been set up as a one-stop shop and where there appeared to be overlap in activities with the TLTSN.

116. On balance, therefore, it seemed to the Review Group that the advantage lay with selecting the best possible national specialist centres, relying on continuing improvements in ICT to make it easier to communicate and collaborate over a distance. The group was also persuaded by the arguments of interviewees in the institutional visits that regional collaboration was increasing in importance, but was being driven by other processes (such as the Regional Development Agencies, the University for Industry, and bids to European funds). They implied that it would be foolish to cut across these other dynamics.

117. Whatever the view on the regional-national issue, there was a strong case made for future subject centres to be based in academic departments in HEIs rather than brought together under one central but semi-detached roof. The advantages of remaining in daily contact with members of the client group outweigh the disadvantages of having less informal networking between centres. Institutions which are currently host to a centre were happy to continue with an arrangement which requires a contribution in cash and kind from them, although not all centres felt that this institutional support had been adequate or willingly given.

118. Interestingly, one criticism of the existing arrangements for the CTI and TLTSN is that they overly give advantage to the host institution to the detriment of users in other places. The CTI in particular, as it was established before 1992, has centres located predominantly in the old universities. Further, although three additional centres have been added recently serving the built environment, nursing/midwifery, and art and design, many subjects offered predominantly in the 'new' university sector do not have their own CTI centre. Examples include business studies, sports science, and leisure and tourism. These points may account for a further criticism of 'the tang of elitism' of the CTI; there has been no re-tendering since 1989 and thus no opportunity for the new universities and colleges to enter the field.

119. On a more practical note, the Review Group would also advise the funding bodies to consider the following improvements on the existing management arrangements of the CTI and TLTSN:

a. Flexible monitoring and evaluation by outcome, not output.

b. Involvement in the appointment of centre staff by those managing the programmes in the Central Unit.

c. Provision for re-tendering a centre within the five-year period if it fails to meet reasonable expectations.

d. Funding to take account of salary increases and the likelihood that excellent staff will reasonably expect promotion - and agreed mechanisms for achieving this.

e. Registration of users/clients under the Data Protection regulations in such a way that the funding bodies, and any future evaluators, can obtain direct and independent access to their views on the service provided by the centres.

The need for research and evaluation

120. Consistently across all parts of the study, respondents identified a need for more and better applied research into the (cost) effectiveness of the use of CAL and ICT. The current paucity of evidence is seen as particularly unhelpful when trying to establish the case for change to a sceptical group of academics. As an immediate step, the funding bodies might want to consider whether the ITATL (Information Technology Assisted Teaching and Learning in UK Higher Education) Report commissioned by them for the Dearing Committee could be updated and widely disseminated. In the medium-term, the case for a funded research initiative was made by several respondents, but with the proviso that it should be operated independently of staff in whatever centres or programmes then existed. (This contrasts with the view of staff in some CTI Centres who believe that subject centres could and should be funded to carry out research in their area.)

121. In the final weeks of this study, the ESRC announced a major research programme, funded by the HEFCE, into effective teaching and learning. The programme is to cover all sectors of education, but technology and learning is identified as one of the issues likely to be addressed. It is to be hoped that the opportunity will be taken to fund empirical work of a high standard on CAL and ICT in higher education.

122. Concurrently, there seems to be a continuing need to commission and disseminate systematic case studies of innovatory or good practice in integrating the use of CAL and ICT into learning. Such case studies need to be context rich and include negative as well as positive outcomes.

Link to the QAA and the ILTHE

123. Respondents were in little doubt that if the QAA were to require more evidence of integration of CAL and ICT in the learning programmes of students it would prompt appropriate changes in departments and institutions. However, there was little enthusiasm for assigning a more prescriptive role to the QAA, thus eroding the autonomy of very diverse institutions to determine their own way of delivering learning. Views were accordingly mixed on the desirability of making the use of ICT a stronger criterion of excellence than it now is. However, greater evidence of reasons for integration or non-integration might well be acceptable and sufficient.

124. Similarly, there was considerable support for seeing skills in ICT as an integral part of the training of staff at (Associate) Member level of the proposed Institute for Learning and Teaching in Higher Education. However, giving the ILTHE the responsibility for rationalising, managing and co-ordinating all ICT and associated programmes seemed premature, since its first priority was seen as establishing an accreditation framework for staff and for staff development courses. Accordingly, a large majority of respondents were of the view that funding and management responsibility had to remain with the existing HE funding bodies for the short-term at least. Nor did they want rationalisation and other changes to be put on hold for several years while the sector sorted out the precise scope and funding basis of the proposed ILTHE.

Funding models

125. Respondents in virtually all parts of the study were asked for their views on contrasting approaches to funding the use of ICT in teaching and learning. The options offered were: continuation of a top-slice; earmarked funding for all institutions; or the replacement of a top-slice by a voluntary subscription. Views were remarkably consistent. There was very little support for ending the top-slice and thereby forcing existing or future programmes to rely on a more commercial or subscription basis. Only 16 per cent of respondents to the institutional questionnaire favoured the wind-up option; 73 per cent favoured continuation of the top-slice in some form.

126. Top-slicing was seen as efficient for commissioning R&D on behalf of the whole sector, as necessary 'venture capital' to take some risks with new and immature technologies, and as a useful seed corn around which other sources of funding could gather. By contrast, setting up services on a fully commercial basis was viewed as contrary to the ethos of higher education. It was seen as a 'tax on change', and a disproportionate blow to the poorer institutions which were often the ones who needed the external services most in order to 'catch up' with the rest of the field.

127. More negatively, TLTSN and CTI staff thought that neither institutions nor departments would be willing to prioritise money for subscriptions and services against other demands - particularly when the jury was still out on the cost-benefit ratio of investing in ICT. Centre staff felt vulnerable at the thought of competing head on for change-management business with the established consultancy firms, and also feared they would lose their neutrality if they partnered commercial publishers. However, other respondents could see a case for departments making some contribution to consultancy that was provided over and above a standard service. There was also support for a small subscription to be levied, not least to ensure that centres would pay attention to the needs of all their users and not give disproportionate attention to just one section.

128. Continued top-slicing should not, however, be seen as endorsement of the existing levels of financial support and funding arrangements in the TLTSN or CTI. Neither programme regarded itself as adequately funded to do the job it was meant to undertake. For the CTI, the current levels did not really reflect differences in the size of the constituency of subjects, and without co-funding from other sources such as the TLTP it was difficult to be as proactive as users wanted and expected. The respondents to the institutional survey endorsed these views. Further, the emphasis on annual 'contracting' by outputs - such as the number of workshops held or publications completed - had done little to encourage self-evaluation of effectiveness from the users' perspective.

129. For the TLTSN, low levels of funding had meant the loss of senior staff who were the very people who would have brought the necessary credibility to the consultancy role. It also severely limited the number of institutions which could be supported through change, on an on-going basis, and thus curbed staff's desire to open up too many possible requests for help. Further, the uncertainty brought about by yearly funding made it difficult to justify expenditure on staff development for those employed in TLTSN Centres, and this made the posts relatively unattractive. Alternatively, staff were carrying other responsibilities in addition to TLTSN work, which had an adverse effect on the time they could commit to it.

130. Even though respondents wanted top-slicing to be continued, there was also strong support for adding supplementary monies to institutions' block grant which would be earmarked specifically for implementing their teaching and learning strategy. Some 64 per cent of respondents to the institutional questionnaire preferred this option; it was also the option most often endorsed in the institutional visits. This approach was seen as best meeting the needs of a diverse set of strategies and very different levels of current investment. It receives endorsement from the experience of the Staff Development Initiative in Scotland, funded by SHEFC, and the results of a national survey conducted in Scotland in 1997.

131. Finally, respondents were asked whether the funding bodies should give priority to the development of new CAL packages, or to the development of productivity tools which enable lecturers to customise their own learning resources from material on the WWW and elsewhere. New expenditure on both would be welcomed, but if forced to choose, then respondents would opt for the latter. Although this was not asked directly, responses to other questions suggested that development and testing of electronic learning environments which bring together ICT and MIS might also be supported.

132. There is no doubt that courseware development will remain a substantive issue for the funding bodies, and one which may need partnership with the private sector if it is to be tackled comprehensively. Such approaches were beyond the remit of this Review. However, if any new programme of this kind were to be funded, respondents wanted the funding bodies to heed the lessons from past programmes (including the TLTP) and ensure that what was produced could be scaled up and supported over time. New learning materials, environments and tools should be developed collaboratively to ensure reasonably wide take-up, and the teams should be multi-skilled. The funding bodies could also promote greater collaboration in the purchase and use of relevant equipment and facilities.