Measuring R&D:
Challenges Faced by Developing Countries
SUB-REGIONAL HANDS-ON TRAINING ON SCIENCE, TECHNOLOGY AND INNOVATION INDICATORS
Damascus, Syria
18-20 September 2010
www.uis.unesco.org
Products
www.uis.unesco.org
Outline
 The problem
 The process
 Contents of the Technical Guide
 Thinking ahead
www.uis.unesco.org
R&D statistics in developing countries (1)
 Recognition, meeting targets, evidence-based
S&T policy, but:
• lack of interest at the level of policy makers (low policyrelevance?)
• S&T is still not properly represented in economic/social
public policies. lack of resources devoted to statistics in
S&T
• lack of technical knowledge for the production of crossnationally comparable R&D statistics
• weak statistical institutions
• difficulties in applying FM concepts and methods
www.uis.unesco.org
R&D statistics in developing countries (2)
 Particular characteristics of R&D activities to be taken into
account:
• R&D performers function within the specific context of a national,
cultural, political, financial and economic system
• different structures in terms of government, innovation system,
higher education system, statistical system
• particular ‘culture of information’
• Users of R&D stat: Gov, analysts. + international donor agencies
 S&T indicators
• adapted to particular policy needs
• provide answers to actual policy questions
 However, international comparability is foremost
www.uis.unesco.org
The process (1)
 Experience acquired through the UIS work, in particular
through direct contact with S&T statisticians in numerous
workshops and other meetings around the developing
world.
 Advisory Meeting to the UIS S&T Statistics Programme
held in Montreal, Canada, December 2007.
 Papers commissioned by UIS to Jacques Gaillard (IRD,
Paris), Michael Kahn et al (HSRC, South Africa), and
Gustavo Arber et al (RICYT, Argentina).
 Proposal for an annex to the Frascati Manual on
measuring R&D in developing countries was presented at
the OECD 2008 and 2009 NESTI meeting.
www.uis.unesco.org
The process (2)
 Expert Meeting on Measuring R&D in Developing
Countries in Windhoek, Namibia, 14 to 16
September 2009
 Consultant has drafted:
• Technical Guide on Measuring R&D: Challenges for
Developing Countries
• Proposed Annex to the Frascati Manual
 Both to be released in 2010
 Some of the issues might also present
measurement challenges for a future revision of
the Frascati Manual
www.uis.unesco.org
Contents of the Technical Guide
1. Introduction
2. The nature of R&D activity in developing countries
3. R&D expenditure
4. Internal and international mobility of the R&D workforce
5. Specific fields of R&D activity
6. Foreign and internationally controlled entities
7. Strengthening R&D statistical systems
8. Thinking ahead
www.uis.unesco.org
Chapter 2: The nature of R&D activity in
developing countries
 The growing importance of R&D
• More ‘R’ than ‘D’ in developing countries.
• Strong presence of the government and
higher education sectors in the
performance of R&D. Lower emphasis on
R&D in business sector.
• Occasional R&D / Informal R&D
• Special types of R&D
www.uis.unesco.org
Chapter 2: The nature of R&D activity in
developing countries
cont…
 Heterogeneity and concentration
•
Developing countries are a heterogeneous
group:
» Group A: countries with consolidated R&D systems and
developed S&T statistics systems  no major difficulties in
applying Frascati Manual concepts.
» Group B: countries with consolidated R&D systems and
less developed S&T statistics systems  need specific
guidance on how to establish and consolidate sound R&D
statistics systems.
» Group C: countries with incipient R&D systems  need
specific guidelines on how to start creating a regular R&D
statistical collection.
•
High degree of concentration (in group of
countries, in particular institutions, in major
projects, etc)  lead to volatility and inconsistencies in
statistics.
www.uis.unesco.org
Chapter 3: R&D expenditure
 Use of secondary data from national budget
 New sources of funds emerging
 Discrepancy between voted and allocated budget
 Budgetary commitments are not followed up
 Mixing of budgetary records and annual reports from performing units
 Definition of S&T / R&D budgets
 Identifying R&D components in the national budget
 State-owned enterprises, university-owned companies and national
scientific academies
 Private universities
 Fiscal year vs. calendar year
 Information systems in government and higher education inadequate
for statistics
www.uis.unesco.org
Chapter 4: Internal and international
mobility of the R&D workforce
Underestimation of researchers
 Unpaid research
 Informal research
 Research outside of the normal work setting with
external funding
 Multiple part time positions not taken into account
or undercounted
 Master’s research
www.uis.unesco.org
Counting researchers
Overestimation of researchers
 Counting the contract instead of the real effort
 Multiple full-time research positions
www.uis.unesco.org
Counting researchers
Special cases
 FTE calculation >1 and FTE>HC
 R&D in times of crisis
 Visiting researchers
 Brain circulation
www.uis.unesco.org
Counting researchers
Recommendations
 Peer interviews of researchers
 Include a module on barriers
 Use secondary sources
• Publication databases, both national and international
• STMIS and other databases of researchers
• Databases and registers of clinical trials
• Databases and registers of the main foreign donors
involved in funding R&D in the countries
• University accreditation databases
www.uis.unesco.org
Chapter 5: Specific fields of R&D activity
 Traditional knowledge
 Clinical trials
 Industrial activities
 Other activities
www.uis.unesco.org
Special types of R&D - Traditional
knowledge
Traditional knowledge (TK)
A cumulative body of knowledge, know-how,
practices and representations maintained and
developed by peoples with extended histories of
interaction with the natural environment.
These sophisticated sets of understandings,
interpretations and meanings are part and parcel of
a cultural complex that encompasses language,
naming and classification systems, resource use
practices, ritual, spirituality and worldview.
www.uis.unesco.org
Special types of R&D - Traditional
knowledge
Dichotomy between traditional and scientific
knowledge systems
 substantive grounds – because of differences in
the subject matter and characteristics of traditional
and scientific knowledge
 methodological and epistemological grounds –
because the two forms of knowledge employ
different methods to investigate reality
 contextual grounds – because traditional
knowledge is more deeply rooted in its
environment
www.uis.unesco.org
Special types of R&D - Traditional
knowledge
Links between traditional and scientific
knowledge systems
 Scientific approach to TK (in ethno-botany, ethnopedology, ethno-forestry, ethno-veterinary medicine, ethnoecology, etc).
 The application of scientific methods to TK,
converting it into a source of scientific information.
(in biodiversity science or nature conservation; traditional
health and pharmacopeia).
 Interaction between scientists and communities in
participatory technology development
www.uis.unesco.org
Special types of R&D - Traditional
knowledge
Measurement issues and recommendations
 Establish the boundaries for TK (what qualify as
R&D)
 The activities establishing an interface between
traditional knowledge and R&D
 Some fields of activities in TK are transdisciplinary (e.g. ethno-botany), making them
extremely difficult to map into the current
classification’s structure.
www.uis.unesco.org
Special types of R&D - Clinical trials
Clinical trials
 (Can) involve a significant amount of R&D
 Need to be conducted on a wide population
 Growth area for developing countries
www.uis.unesco.org
Special types of R&D - Clinical trials
Measurement of clinical trials
 Registers of clinical trials available, e.g. WHO but also
national
 Funding often from abroad
 Performance various possibilities
• a local branch of the foreign main sponsor
• universities and university hospitals
• individual researchers
• local medical clinics
• locally registered PNPs
• international PNPs
www.uis.unesco.org
Special types of R&D - Clinical trials
Measurement issues and recommendations
 Occupation category of local staff
• Medical doctors and other professionals with at least
ISCED 5A degrees should be considered as
researchers
• Nurses and other staff with qualifications below ISCED
5A should be accounted for as technicians
 FTE calculation is important (often part-time)
 Attribution of sector of performance must be done
with care to avoid double counting
www.uis.unesco.org
Special types of R&D - Industrial activities
 Reverse engineering: understanding the
structure and functioning of an object (in order to
make a new device or program creates a similar
object in a different way), copying it, or improving
it.
 Recommendation: If reverse engineering is
carried out in the framework of an R&D project to
develop a new (and different) product, it should be
considered as R&D.
www.uis.unesco.org
Special types of R&D - Other
activities
 Community development and other social
projects
• R&D only in development and testing phase 
experimental development (most probably in the field of
social sciences)
 Religious research
• part of humanities,
• should be included in R&D surveys.
 This (religious research) will not be a recommendation
www.uis.unesco.org
Chapter 6: Foreign and internationally
controlled entities
 Foreign antennas
 Foreign company’s R&D labs
 International organizations operating in the
country
 Foreign universities based and conducting
R&D in campuses set up in the country
www.uis.unesco.org
The foreign institutions sector
Recommendation
 Create a “foreign institutions” (FI) sector as a separate
sector of performance
 Funding flowing from this sector to other sectors should be
considered from “Abroad” as stated in the main body of the
Frascati Manual
What is included?
• Foreign antennas
• International organizations
• Foreign company’s R&D labs  (remains in the business sector)
• Foreign universities  (remains in the HE sector)
www.uis.unesco.org
The foreign institutions sector
The principal sector sub-classification
 Business enterprises
 Government
 Higher Education
 Private non-profit
 International organizations
www.uis.unesco.org
Chapter 7: Strategies for setting up S&T
statistics systems in developing countries
 Institutionalizing S&T statistics
 Establishing registers
 Structural issues in the private sector and the
private not-for-profit sector
 User-producer networks
 Science & Technology Management Information
Systems and other secondary sources
 Survey procedures and estimation
www.uis.unesco.org
Institutionalization of S&T statistics
 Political support
 Infrastructure and sustained staff
training/capacity building
 Involvement of NSOs: “Official statistics”
status for R&D surveys.
 Adequate legal framework
www.uis.unesco.org
Establishing registers
 R&D in developing countries tends to be very much the
purview of public bodies
Recommendations:
 Establishing a database of public sector R&D projects
• include human and financial resources; align with national policies.
• design could reflect the R&D statistical reporting/definitions.
• source for evaluation of such projects.
 Establishing STMIS
• provide overview of research system.
• framework for establishing complete registers as sample frames
for R&D surveys.
www.uis.unesco.org
Establishing registers
 Other sources
• associations (trade, academic).
• learned societies.
• registers or databases of scientists and engineers.
• database of research grants.
• databases of scientific publications.
• patents and other IP documents.
• business registers.
www.uis.unesco.org
Structural issues in the private sector and
the PNP sector
 Publicly-owned businesses play a major role in
R&D in some developing countries
Recommendations:
• should consider issuing data for ‘publicly-owned
businesses’ separately from the ‘fully private enterprise
sector’.
• private enterprises could also be disaggregated by
ownership, in particular the various degrees of foreign
ownership.
www.uis.unesco.org
Structural issues in the private sector and
the PNP sector
 Business enterprise R&D is presumed to be generally weak in
developing countries when compared to industrial countries.
Recommendations:
• take into account when conducting sample surveys, perhaps by
over-sampling, especially amongst larger companies.
• big companies should not be missed out as it might imply
significant error.
• invest time in interviewing key firms to understand their R&D
function and obtain a clear picture of their activity.
 Private-non-profit sector: make a significant contribution to R&D
in developing countries, but the sector tends to be very volatile.
www.uis.unesco.org
User-producer networks
Recommendations:
• user-producer networks and other forms of stakeholder consultation
should be instituted.
• establishing national S&T statistics groups.
• involve multiple actors.
• coordinating/networking among institutions/databases.
• partnering with business associations.
• conducting face-to-face visits by statisticians and project leaders.
• exploit pre-existing personnel ties.
• get NSO involved; to deal with privacy of information.
• training of interviewers/primary data producers.
www.uis.unesco.org
Science and Technology Management
Information System and other secondary
sources
 STMIS (e.g. database of scientists, research grants, etc):
frequent source for the production of R&D statistics.
Recommendations:
• need close integration between the statistical system and the
STMIS.
• need adjustments to produce comparable statistics, taking into
account issues of definitions and coverage.
• need a balanced approach using both STMIS and surveys.
• need different approach to Private sector organizations as they are
frequently not covered by these systems.
 Combined R&D and innovation surveys
Recommendations:
• the relative rarity of occurrence of R&D in businesses needs to be
taken into account.
www.uis.unesco.org
Survey procedure and estimation
Recommendations:
• attention needs to be paid to questionnaire design.
• frequency of survey.
• prioritize area of work; accompanied by step-by-step approach.
• use of survey questionnaires of other countries for inspiration: need
adaptations to local situation.
• get expertise from the NSO, in conducting survey, in sampling ….
• different questionnaires might be designed for different sectors
based on stakeholder consultations. “One size does not fit all”.
• procedures need to be developed for estimating missing data.
www.uis.unesco.org
Chapter 8: Thinking ahead: Other products
– beyond R&D
 Redefine the concepts of scientific and
technological education and training at broadly the
third level (STET), Scientific and technological
services (STS) and S&T activities (STA)
 Better integrate education statistics with R&D
statistics
 Hands on guidance
 Metadata
 Model questionnaire
www.uis.unesco.org
Thank you!
http://www.uis.unesco.org
r.pathirage@uis.unesco.org
www.uis.unesco.org