LEONARDO DA VINCI PROJECT “REBASING”
Interim report
October 2011
Foreword
This is the interim report for the Leonardo da Vinci transfer of innovation project
REBASING. It has been written in compliance with the quality plan that the
implementing partnership discussed and approved during project inception. This
document is produced at half of the project life cycle and applies indicators to project
performance and outputs. The information included in the report has been regularly
collected during project meetings and partners’ activities, by using the monitoring tools
agreed upon in the quality plan. Moreover, the report considers the outcomes of the
satisfaction questionnaires, which were delivered after the seminars organised in Mestre
and Bergen.
The report is articulated in four parts. The first sums up the main characteristics of the
evaluation design as was presented in the quality plan. The second is about the
application of the indicators to seize project performance. The third is about the
conclusions.
Section one: The evaluation design for REBASING.
As the Quality management for REBASING points out, policy priorities (the Lifelong
learning Programme and the EU policy to make of Europe the knowledge based society
most dynamic in the world), Programme characteristics (the Lifelong learning
Programme and Leonardo da Vinci objectives) and stakeholders’ priorities (in REBASING
case, mainly entrepreneurs Associations and Universities but also local institutions and
development actors), have all to be taken into account to identify objectives for the
evaluation. Moreover, the terms of reference for internal evaluation have to be
considered, taking them from the approved application form. The information on
priorities and on the TOR is useful to identify the objects to evaluate (evaluands) and
formulate evaluation questions. The final stage is the definition of evaluation
methodology and techniques, which is influenced by available resources and availability
of data.
As it was agreed in the kick off meeting held in Venezia Mestre on January 2011, the
REBASING project has designed a comprehensive policy for monitoring and evaluation,
which is part of the quality securing policy. This action has several objectives: first of
all, by collecting elements on the project implementation, monitoring and evaluation
may suggest to responsible staff the adoption of changes in outputs and results. In fact,
a fully integrated evaluation process can provide for a key contribution to the way
projects are managed and carried out. By analysing monitoring data and investigating
underlying causes of difficulties encountered, evaluation can provide feedback to
project management and support on going corrections even at an early stage. In fact,
Leonardo da Vinci projects normally produce early outputs, especially when there is a
well-specified implementation chain and logic model.
Besides that, REBASING has chosen as an evaluation strategy a mixed approach:
evaluation is verification of outcomes since it considers project’s outputs and actions as
processes aiming at creating an impact on problems and challenges identified by the
project. And it is evaluation as research since it reconstructs the process of results
production, considering the modalities of working, the organisational conditions, and
involved actors, trying to link this to the conceptual framework that generated the
project idea. Within this strategic approach, monitoring has an informative finality,
3
aiming to verify efficiency and correctness in meeting the project objectives, the results
production process according to plans, the support to continuous improvement of
project activities, the realisation of evaluation by feeding a constant flow of quality
information, the analysis of the valorisation plan and of its realisation.
On the other hand, the evaluation activity is structured to verify the conformity and
consistency of results and outputs with reference to expectations and plans and to verify
the impact of innovation. Moreover, evaluation consists in the verification of the
effectiveness of the project implementation and (since REBASING is a TOI project) of the
real transfer of innovation in recipient contexts and territories. Monitoring and
evaluation are realised with a participatory approach that considers all project actions
and that verifies the application of transferred innovation by involving all interested
actors (including final beneficiaries).
From this mixed strategy stem three main dimensions that are considered in the
monitoring and evaluation plan: the internal dimension that is about the realisation of
activities, the differences with respect to plans, the production of results. The second
dimension is related to transfer and is about the meeting of transfer objectives and the
quality of results. The third dimension is about the impact that project outcomes will
have on partner organisations. The impact dimension examines feedback from
participants as well; in fact, the reactions and suggestions of participants to events and
workshops are a powerful source of information referring to project implementation.
Methodologies for quality securing in REBASING
The finality of the quality securing activity consists of sharing with project partners
procedures and tools to implement the intervention according to high quality levels. This
encompasses activities of different nature:

the definition of procedures and of documentation - which aims at defining from
the very beginning how to feed archives and collect the documents needed for
managing and budgeting the project.

the monitoring and evaluation actions aim at organising the informative and
decisional activity of the partnership.

the quality assurance activity aims at verifying how the project action can satisfy
the expectations of all involved actors.
4
From a general point of view the quality securing activity aims at several interrelated
objectives: to guarantee the correct management of documentation, to ease the
collection of analytical information on the project in order to formulate an evaluation
judgement of the capacity of the partnership to meet its objectives.
The leading partner for this activity - Confindustria Veneto SIAV - normally works under
an internal quality standard system: part of this framework will be used in REBASING.
The methodologies that the partnership has adopted for quality assurance and
evaluation partly refer to project management procedures that have already been used
for planning the project. More in detail, for defining the procedures, the documentation
and the archive management, tools such as the work breakdown structure (WBS) and
GANTT diagrams will be used. All meetings are expected to produce minutes, as a
support to task accomplishment and progress of the activities verification. A database
has been implemented for the financial management, in order to collect data from
partners useful for interacting with the National Agency.
Methodologies for monitoring and evaluation
As far as monitoring and evaluation are regarded, the overall approach is participatory
and is getting benefit from the cooperation of all partners involved. In the very early
stage of the project, and especially during the kick off meeting, the quality plan has
been discussed with partners who approved it.
The project dimensions that are under consideration by monitoring and evaluation are:

Activities realisation and progress;

Gaps and differences referring to the original planning;

Transfer of innovation;

Appreciation of participants and impact.
In particular, the monitoring activity is structured to verify the correct realisation of
work programme, to support the continuous improvement of realised activities, to allow
project steering committee to adopt corrective decision when necessary and to analyse
the valorisation actions.
Both monitoring and evaluation regularly check opinions and reactions of final
beneficiaries and of stakeholders, using questionnaires that are delivered after each
5
event. During each steering committee, information on quality securing activity is
provided, to update all partners on project progress and to manage eventual delays.
Having summed up the main characteristics of project evaluation strategy, let us now
consider the state of the art of the project at half its life cycle.
Activities realisation and progress.
Project inception went as planned: project procedures have been set up encompassing
communication, quality management, financial management, work activities, tasks and
roles. All this was done during the kick off meeting organised in Veneto in January 2011.
During the start up phase a change in the partnership occurred (see beyond: Gap and
differences) and it was necessary to emend the Contract with the National Agency.
Meetings have been realised according to schedule and have proved effective in sharing
knowledge on the Norwegian approach to brokering. In particular, two partnership
meetings and three workshops have been realised1. The structure of the workshops
(seminars open to local territories key actors coming from the productive environment,
from human resource developers, and from the research and university sector) proved to
be a fertile ground for discussing and sharing ideas.
In particular, the first event realised in Marghera was attended by 62 key actors
encompassing: entrepreneurs, VET professionals, University staff and students, research
centres staff, Public authorities representatives, brokers, representative associations,
journalists, consultancy firms. The second event, held in Bergen, aimed to enhance the
transfer action, by sharing quality information on the brokering approach and on the
concrete applications it had in Norway. It was attended by 23 participants
encompassing: university people, enterprises, consultancy firms, and representative
associations, research centres.
Partners have completed the mapping of subjects relevant for the brokering activity in
their territory; as a consequence of the information exchanged during the meetings and
via the project website, the definition of the profile of innovation broker is on going and
soon will start its declination in a ECVET proof format, which is foreseen for the second
half of the project.
1
As for events, only Mestre and Bergen are considered here, since the workshop in Lugano took place in
late November.
6
Dissemination activity started with the launching conference in Marghera and with the
elaboration of dissemination plans by all partners (which is part of the quality
assurance). The web site was available in the first weeks of the project and is now used
as a medium for communication and a place for storing and exchanging documents.
Within this context the Swiss silent partner organised in late November a visibility event
in Lugano, which was not foreseen in the project.
Gaps and differences related to the original planning.
The most relevant change is related to the disappearance of one of the planned
partners: CEFORALP. The French partner was closed off around the time REBASING
started: although there was no money transfer to be recovered, it was necessary to
agree an amendment with the Italian National Agency and redefine the work plan, as
any attempt to find a substitute in France was unsuccessful. As a consequence, major
changes are: in WP 3 Transfer, the analysis of subjects relevant for the brokering
activity in the Rhone Alps area was not realised. In compensation, the analysis in
Campania was broadened. In WP 5 (Supervising the transfer) the number of identified
open innovation networks is now up to 4 and adapted profiles for local context is equally
elevated to 4. There will be 10 innovation audits. In WP 6 - Sustainability - of which
CEFORALP was leader, the lead partner SIAV designed the dissemination strategy, in
cooperation with all partners. The final Conference, originally foreseen as online, will
be a conference hosted in Campania. Moreover, the Bergen University will produce a
scientific document in English (with abstracts in partner languages) on the innovation
broker methodology.
Transfer of innovation.
During the first and the second meetings, the Norwegian partner shared with all
consortium members and with workshop participants the approach, methodology and
tools for the preparation and activation of the innovation broker at local level.
Companies and local authorities were involved as testimonials in the transfer process.
From a practical point of view, transfer was realised as follows: partners identified and
mapped strong and weak points of the research based competence broker as outlined by
the Norwegian experience. Moreover, all partners investigated relations between triple
7
helix actors (Public bodies, Universities, Companies) in their territories to identify levels
and modalities of cooperation. The results of the analyses (that have been formalised in
two analyses of local performances) are the basis for the development of the innovation
broker profile at local level, which is one of the outstanding outputs of transfer. At
Italian level, DIEG and DIMEG discussed on the operative features and activities of the
broker: results of the discussion have been presented during the dissemination
conference in Lugano.
All in all, the conceptualisation effort at the base of the transfer activity is completed:
from now on, after the formalisation of the ECVET-proof profile, test phase will be
possible.
Appreciation of participant and impact.
As a methodological choice of the quality plan, the appreciation of participants to
meeting and events is regularly checked. In the first half of the project attendees’
feedback was collected twice, through delivering of a questionnaire. The questionnaires
are divided in three sections: the first proposes eight topics against which participants
have to express their opinion using a scale from “very positive” to “very negative”.
Moreover, they have the possibility to identify what they liked best or what could be
changed and improved.
The second section questions whether attendees would recommend participation to the
meeting to other people and if they consider to have improved their competences in the
field covered by the workshops. Finally, participants have a free text field to express
comments and recommendations.
The first questionnaire was used to analyse the feedbacks of participants to the first
conference of the project, held in Venice on January the 18 th 2011 on “Industry and
research: network and performance”. In particular, the idea was to appreciate the
reactions to the two parallel workshops organised. The two workshops were dedicated
to: “The broker: role and competence” and “The broker and the innovation processes
and contexts”. However, it has to be kept in mind that some of the respondents made
comments on the conference as a whole, appreciating elements not encompassed by the
workshops. In the case of the first questionnaire, 26 persons answered.
8
As for results of the investigation2, first of all, appreciation on the conference and on
the two workshops was high and with conviction. Finalities of the project are clear and
the first impact has been very positive. All items considered in the questionnaire have
met attendees’ appraisal while negative remarks are statistically unimportant.
Second of all, the general subject dealt with by the conference is of widespread interest
and the necessity to cope with it to improve territories strength is in the minds of most
participants. This is a first good result for the project strategy, as the necessity of the
intervention seems to be justified.
Third of all, motivation in participants seems to be very high and convinced, as the
visibility of the benefits that participants may gain for their work practice was evident
from the very first meeting. The perception of a solid partnership, well-assorted and
committed to produce stated results was shared during the whole conference.
Finally, there seem to be only a few things to change: probably the most important is
related to time management and keeping during the workshops. A proper balance
between the different activities carried out is something to care for in a very accurate
way.
As for the second questionnaire, it was used to analyse the feedbacks of participants to
the second workshop of the project, held in Bergen on May the 10 th 2011 on
“Competence brokering in Regional R&D and innovation - VRI”. In particular, the
questionnaire was delivered after the two-days meeting which encompassed the second
project meeting and the second workshop. The workshop involved - beside project staff
and experts - Norwegian actors who were active in implementing the methodology to
link university and companies, such as professionals, entrepreneurs, human resources
developers, and local public administrators. 17 people responded to the questionnaire.
As in the first case, appreciation on the open seminar was high and with conviction. The
characteristics of the approach to brokering knowledge were clearer and the potential
for transfer seemed remarkable. All items considered in the questionnaire have met
attendees’ appraisal while negative remarks are statistically unimportant.
Second of all, the general subject dealt with by the seminar is of widespread interest
and the necessity to cope with it to improve territories strength is in the minds of most
participants. This confirms the opinion widespread in the Marghera event and the
necessity of the intervention seems to be justified.
2
The complete analyses of the two questionnaires are available in two notes that have been uploaded in
the project web site.
9
Third of all, motivation in participants is still very high and convinced, as the visibility of
the benefits that participants may gain for their work practice was evident from most of
participants. Finally, there seem to be only a few things to change: probably the most
important is related to the availability of working documents that should be circulated
before the event.
10
Section 2: Project outputs
Monitoring indicators
As far as indicators are regarded, it has to be remembered that some of them were
already mentioned in the application form. In fact, the procedure for project applying
required to identify some short and long term target groups, territories and sectors for
their application. Table one lists them by typology. Please note that France is still
included in the areas of application, although CEFORALP withdrew.
Table 1: Short and long term impact target groups
Short term
Long term
Target
groups
and/or
sectors
Enterprises
and
their
workers
Universities and research
centres
VET operators
Practitioners working with
companies.
Territories
Enterprises
and
their
workers
Universities and research
centres
Developers
of
tertiary
education curricula
Human
resources
developers in companies
VET operators
Practitioners working with
companies
Industrial
representative
associations
Veneto
Campania
Saxony
Rhone Alps
Estonia
Veneto
Campania
Saxony
Rhone Alps
Estonia
These are the target groups listed in the approved project. In order to have high quality
information on project outcomes, the quality assurance plan identified indicators,
relating them to other dimensions. These, as Table 2 shows, stem from the results that
REBASING intends to produce and which are partly realised. In Table 3, indicators are
11
applied against 6 dimensions: result, added value, innovation, transferability,
sustainability, impact.
Table 2: project planned results and thresholds
Result
Related
Work
Package
Threshold (if the case)
1 Tools for managing 1
the project
Realised outputs
2
Quality 2
management plan
3 Quality interim 2
report
4 Quality final report 2
1
Documentation
archive.
Communication
and
work procedures.
Steering committee.
1
1
1
1
5 Minutes of the
meetings
6 Financial database
7 Appreciation
questionnaires
1 per meeting
At the end of the
project
2
1
1 per meeting
1
2
8 Workshop 1
All project
lifespan
2
2
+
all
project
lifespan
3
9
Performance 3
analysis
4
10 Workshop 2
11
Competence 4
broker
profile
structured according
EQF
and
ECVET
Frameworks
12 Training needs and 4
competence analysis
13 Identification of 5
Open
Innovation
Networks
14 Innovation audits
5
15
Competence 5
broker
profiles
adapted
to
local
At
least
participants
10
–
20 62 attendees
2
At
least
participants
1
10
–
20 23 attendees
Available
in
the
second half of the
project
Available
for
the
Lugano dissemination
meeting
At
least
5
(Veneto, Available
in
the
Campania, Estonia, Rhone second half of the
Alps, Saxony)
project
At least 10
Available
in
the
second half of the
project
5
Available
in
the
second half of the
project
12
context
16 Workshop 3
5
At
least
participants
17 Final conference
5
1
18
Dissemination 6
products
(plus
dissemination plan)
19 Project progress 6
updating (via web
and newsletter)
10
–
20 Foreseen
for
second half of
project
Foreseen
for
second half of
project
Project
Logo
Leaflet
the
the
the
the
and
2 newsletter + 6 articles Currently on going
on the press.
13
Table 3: Grid of indicators.
Indicator typology
Result
Indicator description
1. Coherence between outputs and
planned results;
2. Coherence
between
produced
outcomes and Leonardo da Vinci
objectives and priorities.
3. Number of workshops realized (min 3);
4. Number of subjects involved in
workshops (10/20) and in the final
conference;
5. Number of stakeholders and key actors
involved.
6. Respect of planned schedules;
7. Number of performance analysis
realized;
8. Realization of the profile for
competence broker;
9. Open innovation networks identified;
10. Innovation audits realized;
11. Dissemination products realized;
12. Efficiency
of
organizational
mechanisms.
Performance
1. According to monitoring activity and feedback
appreciation, coherence is high.
2. Outcomes are in line with LdV priorities (establish
links between the labour market and VET
environment.
3. 2 workshops have been organized plus one
conference in Swizerland
4. 62 participants in the first event and 23 in the
second one.
5. The two events involved about 60 stakeholders
belonging to public authorities, companies,
universities, research centers, consultancy and
service firms.
6. There are no relevant delays in the
implementation process.
7. Available in the second half of the project
8. Available in the second half of the project
9. Available in the second half of the project
10. Available in the second half of the project
11. Web site is on line; logo and leaflet.
12. Management ok, communication improvable.
Added value
13. Presence of a quality plan
14. Level
of
satisfaction
amongst
workshops participants;
15. Level
of
satisfaction
amongst
conference participants;
16. Perceived added value in staff and
participants.
13. The quality plan has been produced and discussed
with partners.
14. The level of satisfaction appreciated during the
first two questionnaires is high.
15. To be appreciated after the conference.
16. The added value has been explicitly indicated by
staff and participants in the high utility of the
Norwegian approach to the cooperation between
university and enterprises and in its coherence
with the work tasks of partners.
Innovation
17. Typology of innovation (output,
methodology, end users);
18. Finality of innovation;
19. Context of innovation (territory,
sectors, company size);
20. Level of access and re use of results
from partners.
17. The typology of innovation is mainly related to
methodology;
18. The finality of innovation is the sharing of the
Norwegian approach to the innovation broker
profile and its adaptation to recipient contexts;
19. The context of innovation is represented by
partner countries (Veneto, Campania, Germany,
Estonia). The project addresses SMEs of different
sectors.
20. So far the level of access and re use of results
from partners is limited to the analysis of local
relations between enterprises and university and
to the analysis on the components of the broker
profile.
21. Presence of a detailed
plan;
22. Typology and richness of
activities;
23. Degree of involvement of:
institutions, representing
dissemination
and
21. In the first phase of the project partners have
indicated the local target groups and the
modalities for reaching them. This allowed the
lead partner to design a valorisation plan that
details the information contained in the
application form;
22. Valorisation activities have already been started
Sustainability
valorization
valorization
authorities,
bodies, in
valorization
15
Transferability
Impact
activity;
24. Number of articles on the press;
25. Number of newsletter.
from project inception. They encompass online
actions with events at presence and targeted
communication;
23. Around 6o stakeholder have been already
involved; these include: authorities, institutions,
representing bodies, companies, universities,
research centres, professionals;
24. Number of articles on the press (?);
25. Newsletters will be issued in the second half of
the project.
26. Number of organizations wishful to
transfer the experiences;
27. Number of agreements between the
partnership
and
organization
interested in products;
28. Partner organizations that adopted the
products.
29. Number of partners which adopted the
competence broker profile
30. Open innovation networks identified;
31. Innovation audits realized;
26. The transfer of the experiences will be finalized in
the second half of the project;
27. Agreements between the partnership and
organization interested in products will be
finalized in the second half of the project;
28. This indicator will be applied at the end of the
project.
29. This indicator will be applied at the end of the
project.
30. This indicator will be applied at the end of the
project.
31. This indicator will be applied at the end of the
project.
16
Section three: Conclusions
The information coming from the monitoring activity, from the participation to project
events and from the appreciation of feedback questionnaires allow the evaluator to
formulate the following statements:
From a general point of view, the project implementation process is on track; the
constitution of the partnership was regular and the withdrawal of the French
organization proved to be without critical consequences on the capacity of the network
to carry out agreed tasks. All foreseen structural mechanisms were set up (quality plan,
financial database, functioning procedures and on line tools) and all planned meetings
were realized according to schedule.
From a quality point of view, realized actions were able to satisfy the expectations of
participants and stakeholders. Feedback from meetings and workshops attendees testify
that the Norwegian methodology for the innovation broker is a concrete answer for the
development of relations between university and companies at local level. There is need
to increase the level of involvement of public institutions (at least in Italy) in order to
“read” the Rebasing experience as a practical way to realize the Triple Helix Model.
The partnership has proved to have all the skills to realize planned goals and results.
This confirms the positive expectations towards a complete realization of scheduled
results and outputs. The cooperation that has been set up is characterized by a friendly
atmosphere and by clarity of intents. This is possible thanks to the contributions of all
partners and especially thanks to the availability and sensitiveness of the Norwegian
organization.
On the other hand, there are a few points of attention and issues for improvement. First
of all, communication between partners is fragmented and uneven. Although
communication procedures and tools have been available from project inception, during
the periods between meetings, contacts between network members have been somehow
occasional, despite the efforts of the coordinating partner. Some of the partners have
worked in couples, but there is need to share in a complete way the results produced
and the knowledge acquired.
Second of all, there is need to increase the coordination efforts towards the final
products production. So far, outputs have been according to plans; however, they are
results of the preliminary stage. The finalization of products for their use by final
beneficiaries requires reinforced cooperation and commitment, especially as far as the
ECVET - proof profile definition and Open innovation networks identification are
regarded.
18