Impacts

1.    Most participants considered that the benefits of participation exceeded the costs they had incurred.  However, although a high proportion of projects were concerned with product or process development there has been little commercialisation of project outputs since completion.  Of the 41 projects covered:

• five led to cost savings in production processes.
  
• a further six had been incorporated into products in some form or another.
  

2.    In total these represent over 25% of the sample but the size of the commercial returns are in most cases very small, typically amounting to a few thousand pounds per annum.  Technical success rates in the project were high with over 80% of participants having achieved all, or most, of their objectives.  This was not therefore generally a reason for the low rates of commercialisation which instead reflected:

• for 17 projects, the overriding aim was to generate technical information or scientific knowledge and direct commercialisation was never an objective
  
• five projects which although technically successful did not lead to commercialisation because of corporate changes within a member of the consortium, and
  
• a range of factors including, the fact that the only UK participants were HEIs, insufficient technical progress was made during the project, continuing development work is still underway and the project was overtaken by technological or market developments.
  

3.    The fact that participants were satisfied with their participation despite the low levels of commercialisation reflects the other benefits they derived.  Over three-quarters of those interviewed cited enhanced scientific or technical knowledge as a key benefit of participation.  This encompassed a number of more specific benefits as follows:

• the generation of new scientific knowledge on the basis of fundamental research.  This impact was largely confined to projects involving HEIs
  
• the acquisition of technical knowledge which could in principle be applied by companies
  
• the acquisition of technical capabilities which is distinct from knowledge in that the former relates mainly to human resources and know-how and the latter to codified information.  Only eight per cent cited technical capabilities, in this sense, as a benefit arising from FP participation
  
• just over one third of those interviewed felt that improved access to technical networks had arisen from participation in the project.  The majority of links were not, however, new, but built on and consolidated existing networks
  
• fifteen per cent of those interviewed felt that participation had raised their profiles with customers.  These chiefly relate to projects where a customer (actual or potential) is involved in the consortium.

4.    HEIs are in a special position in relation to FP.  For most of the HEIs we interviewed, FP funding represents just one element of their research portfolios and is considered almost a perfect substitute for research council funding.  FP projects have supported postdoctoral and PhD students and, in some cases, have also fed directly into undergraduate programmes.  Staff were also able to publish in the normal academic journals.  The research was of interest in part because of the involvement of companies which adds an extra dimension to the work.  However, for most of those we interviewed, their contribution to the project was not significantly more applied than the sort of work they would undertake under a research council grant.  Many would in fact refuse to participate in projects which they viewed to be too applied.

Project Characteristics and Experience

5.    Each project was assessed against a range of criteria with the following results:

• in four projects a process or product was developed (though not necessarily commercialised) during the life of the project and in three cases some market testing was also undertaken.  There was no involvement from the science base in these projects
  
• six projects which again developed processes or products.  In these cases there was a significant science base involvement but so far as we could identify no papers were published and dissemination of results is therefore likely to have been minimal.  In four of these cases participants had previously undertaken R&D in this area and the FP project represented a continuation of this work
  
• four projects which also developed products or processes but in these cases academic papers resulted suggesting a wider dissemination of results
  
• five projects which primarily generated technical knowledge for the participants although there was limited dissemination of this knowledge.  These have been placed after the previous cluster because the projects did not lead to product or process development during the project
  
• ten projects, primarily from the Biomed programme.  These almost all comprised HEIs and RTOs and were concerned with quite fundamental research with, so far, limited commercial applications.  In all cases participants had undertaken previous research in the area
  
• twelve projects which have been clustered together because they each involved one or more competitors.  A number of these were classic examples of pre-competitive research in that they involved several competitors exchanging information and results.  However, the research was in several cases fairly routine involving testing, analysis and data collection and did not give rise to academic publications.
  

6.    In summary, over a quarter of the projects involved competing businesses and a further quarter were generating fairly fundamental knowledge disseminated through scientific publications.  The remainder could be considered quite close to market with many of the benefits being internalised by the participants.

Project Structures

7.    Most projects were structured to minimise interdependencies between partners.  The aim, which was generally fulfilled, was to minimise the risks of the project failing because of a failure on the part of one or more partners to deliver.  However, a side effect is to reduce interactions between partners and in many cases the result was single bilateral relationships for most partners, with the exception of the co-ordinator.  This necessarily reduces the scope for knowledge and technology transfer between partners.

Funding and Additionality

8.    The main impacts of FP funding, which are often related have been:

• wider scope.  This reflects the fact that FP projects are typically many times larger than in other programmes.  For many participants this was one of their major advantages.  It means that there is the opportunity to explore a greater range of approaches than would normally be possible and therefore a greater chance of finding the optimum approach.  The research is not necessarily more speculative in nature but simply wider in scope
  
• collaborative.  Participants have entered into partnership with others rather than undertaking, more limited, work individually.  As discussed in Chapter 4, twelve projects involved competing businesses and while some of these consortia would have gone ahead without FP funding the number of partners would generally have been smaller
  
• more partners.  In six cases the availability of funding led to increased numbers of partners even though a collaborative project would probably have gone ahead in the absence of FP funding.  This generally reflects the addition of foreign partners.
  

9.    The main impact of funding was to generate projects with wider aims and scope than would otherwise have been the case.  Closely related to this, some companies were encouraged to collaborate on research, which they might otherwise have undertaken in-house, and on a smaller scale.  In a few cases the number of partners has been increased by the availability of funding and in most of these cases the additional partner(s) were foreign.

10.   Thirteen respondents considered the European dimension to be essential and a crucial element of success and a further 40 participants felt that the foreign partner(s) conferred significant benefit to the project.  This implies that 64% considered the European dimension to be either essential or of real value.  More generally, foreign partners are valued because of the skills, experience or facilities they bring to the project.  In some cases, the foreign contribution was of the utmost importance and this tended to be where the other countries were acknowledged to have a substantial lead over the UK.  The perceived benefits of transnational collaborations are, however, restricted to research.  Very few respondents considered the European dimension to have been useful in commercialising results.

Conclusions

11.   The key conclusion is that interviewees overwhelmingly considered their participation to have represented value for the money they had contributed and many were developing plans to participate in FP5 programmes.  The benefits they derive are not, primarily, directly commercial and include in particular the acquisition of technical knowledge.  European funding is generating additional research activity but, unsurprisingly, this mainly relates to widening the aims and scope of projects rather than anything approaching 'full' additionality.  Our view is that the levels of additionality generated by FP funding are in line with those generated by national programmes.  The contribution of foreign partners is valued by UK participants, but for most the availability of a subsidy is an important factor given widespread recognition of the costs which transnational collaborations give rise to.

12.   Many, but not all, of the projects in our sample shared a number of other common characteristics.  First, few projects were undertaking highly speculative research.  In fact 80% achieved all or most of their technical objectives.  Second, relatively few projects were concerned with pre-competitive research in the normal sense.  Only twelve of the 41 consortia included business competitors and a significant number were aiming for some commercial development soon after project completion.  Third, there was a tendency to rely on existing networks and previous collaborations rather than generate new alliances.  Seventy per cent of UK-UK collaborations and 62% of UK-foreign collaborations had a previous history of links.  In a number of cases they had also received FP funding previously for related research.

13.   These tendencies in the main reflected two related points.  First, bidders felt their proposals needed to demonstrate good prospects of eventual exploitation, both to the Commission but also internally in the case of business organisations.  Second, the difficulties and associated costs of managing transnational collaborations are well recognised by virtually all those we interviewed.  As a result, efforts were made to structure projects so as to minimise risk where possible.

Comparisons with Previous Studies

14.   A great deal of literature has been produced concerning the impact of Framework programmes, a review of much of this literature is given in a recent article by Terttu Luukkonen¹.  The author points out that there are two main sources for recent information analysing the impacts of Framework Programmes.  The first are Programme Evaluations which are traditionally carried out on a five year basis and bridge different Framework Programmes.  The second are so called "Impact Studies," commissioned by member states to assess the effects of EC Programme participation on national organisations.

15.   Luukkonen asserts that despite all of these studies, there is no direct evidence that EU funded research has advanced European competitiveness.  This may be due to the inherent difficulties of measuring competitiveness and also the problems associated with linking EU participation with any perceived increase in performance of participating companies.  However, it may also reflect two other issues.  First, the relatively small size of FP funding compared with the resources that organisations, and national governments, devote to research.  Second, the fact that all public programmes are seeking to generate impacts at the margin rather than transform sectors.

16.   Our study showed that EU projects do occasionally seem to have effects on the performance of individual companies, but this is seldom a linear relationship and other factors usually play a role.  It is also often difficult to make an accurate assessment of what would have happened in the absence of the project.  Our study also confirms Luukkonen's assertion that EU participation does not effect the research strategy of large firms, indeed they are unlikely to participate in an area of research that has not already been previously defined as part of their corporate strategy.

17.   The most widespread, and perhaps underrated, impact is that participation in EU programmes enhances existing skills and broadens the knowledge base of participating organisations.  The results of our study confirm this.  Such impacts though intangible could have far-reaching and permanent effects on organisations.  Some previous studies have suggested that such impacts may be particularly important in academic departments with the establishment of new research groups with associated international networks being formed as a direct consequence of participation in an EC project.  There was only one example of this in our study but we suspect the sample may be unrepresentative in this sense since it was heavily biased towards leading research universities.  However, more generally we found several examples of consortia that had remained substantially the same through several sequential projects.  This may indicate that this "networking effect" is becoming less important today.  Studies of the evolution and spin-offs associated with such networks through time could be one way of attempting to trace the efficiency of such activity².

18.   There have been several impact studies in the UK.  In the early 1990's there was an attempt to conduct a co-ordinated series of such impact studies across a number of members states using broadly similar methodologies.  The UK study found at the time that the UK had the largest number of collaborations of any member state (over 17,000)³, and the majority of those contacted felt that the experience had been beneficial.  The results of the study are broadly in line with that of the current study with the majority of respondents feeling that participation was worthwhile.

19.   One noticeable difference from the current study is that 50% of companies cited new products and processes as outputs of the project, which is significantly higher than the current study revealed.  In the current study and others we have undertaken recently (see IMT project below), we have found a much lower level of impact in this area.  This inconsistency may be explained by differences in methodology and definitions of what is meant by "product" and "process" developments.  In the current study we sought to restrict such impacts to cases where there had been a direct and tangible contribution to product or process development during the project or since its completion.  Other studies may have adopted a wider definition.  It could also reflect changes in the way in which proposals for FP funding are assessed.  We think it is fair to say that during FP3 and FP4 there was some tightening up of the allocation process to exclude projects considered too close to market.

20.   Our study also showed very few problems with IPR arrangements which was a feature of the 1993 study.  This probably represents a generally greater awareness within the UK of IPR issues, some of this at least can be attributable to previous FP participation.  It is interesting to note that a recent Finnish study shows that IPR is still perceived to be a problem by many Finnish participants who are likely to have had less experience of FP participation⁴.

21.   The 1993 impact study found that although EU funding only accounted for 4% of research funding for the university sector it was believed to have an effect much larger than its absolute size would suggest.  At the time of the study EU funding was a relatively new phenomenon and therefore less influenced by historical precedents.  It follows that relatively small amounts of funding going to "non-traditional" departments could have a multiplier effect by increasing the department's rating in the Research Assessment Exercises.  It is doubtful that such an impact happens today, in fact our study has shown several instances of EU funding repeatedly going to the same university departments.

22.   Our study confirms the view expressed in the PREST study that, though the majority of participants from all types of organisations felt that the benefits outweighed the costs of participation in FP projects, the universities were generally more enthusiastic than industry.  We also found that participation was sometimes less popular with administrators than with those actually involved with the research, reflecting the low overheads FP funding brings and the specific difficulties of funding equipment which will be depreciated over a longer period of time than the project.

23.   Another area in which our study agrees with the PREST study is that all organisations found the cost of proposal preparation in terms of manpower a real drain on the organisation.  This is particularly onerous for SMEs and small university departments who both find it difficult to create the necessary "slack" to permit adequate proposal preparation.

24.   Both studies also found that impacts could be identified in terms of the gaining of new managerial skills.  Though this can be very time consuming (particularly for co-ordinators), several interviewees noted that they found managing an EC project a useful part of their career development.  Even experienced managers often found it interesting (and sometimes frustrating) to have first hand experience of management styles in other member states.

25.   Finally we concur with the PREST report that bringing together partners from different members states added value to the research.  There were many examples of complementarity of skills, ability to use novel equipment in a member state or projects that relied on gathering data from across Europe or of involving a wide range of users.

26.   A significant difference between our study and the PREST study was in terms of "Europeanisation."  The PREST report states "Probably the most important and sustainable effect has been to re-orient the research community to a point where it regards itself as part of a emergent European scientific community."  This was a factor which was rarely raised at our interviews with most participants treating European participation as a matter of course.  The differences between the two studies probably reflect the periods they were undertaken with the UK now having significantly greater experience of FP participation than in 1993.

27.   Many of these studies indicate the importance of less tangible impacts on participants such as net gains and losses in the organisation's knowledge base.  This is notoriously difficult to track as such interchanges tend not to be linear but rather have number of different dimensions and connections to other organisations and/or lines of research.  This lack of control over flows of information acts as a significant disincentive to many firms.  The second Finnish study quoted above highlighted the reluctance of European pharmaceutical companies to participate in Framework for this very reason, which repeats a finding of the 1993 PREST study.

28.   SQW is also involved in the current study of the Brite/Euram programme. This has been commissioned by DG XII and is being led by Bureau van Dijk who are responsible for the study design and analysis.  This is designed to examine the impact of completed Brite/Euram projects several years after they had been completed⁵.  At the time of writing the project team has just completed the second impact assessment report which involved interviews with 210 partners in Brite/Euram projects which finished in 1994, and covering all member states⁶.  This project indicates a much higher level of commercial exploitation than in the DTI FP project.  Some economic impact was indicated in 40% of those interviewed but in many cases this was small or not solely attributable to the results of the specific project considered.  Economic impact in this study also includes environmental improvements as well as commercial exploitation.

29.   It should also be stressed that this study differs from the current one in that the methodology was designed to identify projects which had some level of commercial or employment benefit from which the sample for interview was selected.  There was an initial screening by postal questionnaire (around 1,500 partners) which identified projects with some level of commercial benefit.  Also, many Brite/Euram projects are applied in nature and the overall potential for commercial exploitation is higher than in many other programmes.  In other respects the results from the interviews in the two studies are similar.

¹Luukkonen, T., The Difficulties in Assessing the Impact of EU Framework Programmes, Research Policy, Vol 16 (6), 1998, pp 599-6103.
² This issue is returned to in Annex A.
³ Georghiou, L., Cameron, H., Stein, J.A., Nevada, M., Janes, M., Yates, J., Piper, M., Boden, M., Senker, J., The Impact of EC Policies for Research and Technological Development upon Science and Technology in the UK, 1993, HMSO, London.
⁴ Luukkonen, T., Niskansen, P., Learning Through Collaboration, Finnish Participation in EU Framework Programmes, 1998, VTT, Group for Technological Studies, Espoo, 176p.
⁵ Further information is given in Annex A.
⁶ Impact Assessment of Brite/Euram Projects Finished in 1994, Brussels, 30th June 1999, 36p.