The Measurement of Scientific, Technological and Innovation Activities

OSLO MANUAL 2018:

GUIDELINES FOR COLLECTING,
REPORTING AND USING DATA
ON INNOVATION
OECD-NBS International Training Workshop on Innovation Statistics
16-18 July 2019 Xi’an, China

Fernando Galindo-Rueda
OECD Directorate for Science, Technology and Innovation
 The origins of the Oslo Manual

• OECD statistical guidelines for

measuring innovation
– A common language to discuss and
measure innovation
– Not legally binding document
– Consensus view on best practice
• Two distinctive features:
– Concerned with the measurement of 1992
innovation within the business
enterprise sector
– Provides guidelines for survey-based
measurement of innovation
 A manual in evolution

1992 1997 2005 2018

Oslo 2nd edition Oslo 3rd edition
Oslo 1st edition
Technological Product, process,
Technological
innovation;
innovation; organisational, Oslo 4th ed.
Manufacturing and marketing
Manufacturing
most market services innovations.

Published jointly by OECD and Eurostat since 1997

 Third revision of the Oslo Manual:
2016-2018
• Responsibility of the OECD Working
Party of National Experts on Science and
Technology Indicators (NESTI), together
with ESTAT STI Working Group.
• Extensive preparatory work before
revision
– Cognitive testing of concepts
– Ad hoc studies on measurement of
Kick off workshop, Research Council,
innovation and services, design, innovation
Oslo, December 2015
policies
– Distributed microdata work (Innovation in
firms).
• Engagement of 40+ countries and
international organisations
– International steering group.
• Expert and stakeholder workshops
– OECD and ESTAT committees
– Workshops hosted by academics and national
Final workshop and approval, governments
FECYT/Ministry of the Economy,
Madrid, December 2017
4
The 2018 edition
Part I. Introduction to the measurement of
innovation
1. Introduction to innovation statistics and the Oslo Manual
2. Concepts for measuring innovation

Part II. Framework and guidelines for measuring

business innovation
3. Concepts and definitions for measuring business innovation
4. Measuring business innovation activities
5. Measuring business capabilities for innovation
6. Business innovation and knowledge flows
7. Measuring external factors influencing innovation in firms
8. Objectives and outcomes of business innovation

Part III. Methods for collecting, analysing and

reporting statistics on business innovation
9. Methods for collecting data on business innovation
10. The object method for innovation measurement
11. Use of innovation data for statistical indicators and
analysis
Glossary of terms
5
 Conceptualising innovation

• Common use of term “innovation” refers to either

– Notion of process of innovation (what is done by a subject)
– Notion of outcome (what comes out) = an innovation
• Both are relevant and need to be measured.
– “Outcome” view conceptualised in this manual as: “Innovation(s)”
– “Process” view conceptualised as “Innovation activity(ies)”
• Innovation
– More than ideas, implementation required
– Not necessarily R&D-based
– At least new to the subject
– Success is aimed at, but neither guaranteed nor required

 The general concept of innovation:

Implementation of something new to meet a given objective.

6
 General definition of innovation for
measurement purposes

“An innovation is
• a new or improved product or process (or
combination thereof)
• that differs significantly from the unit’s previous
products or processes and
• that has been made available to potential users
(product) or brought into use by the unit (process).”

The generic term ‘unit’ describes the actor responsible

for innovations. It refers to any institutional unit in
any sector, including households and their individual
members
Sectoral scope of practical guidance in
this edition
OM3=OM4 scope

Non profit
Rest of institutions
serving
the Linkages households
world
All business
enterprises
(public and
private)
Government
Households sector

Informal
economy
OM2018 Chapter 2.
Concepts for measuring innovation

Why? Measurement How?

User needs
strategies
for innovation
(sources,
statistics
responsibilities,
etc...)
Concept of
innovation

Efforts to
Relevant Sectoral measure
Phenomena scope innovation
What? for
beyond
measurement
business

Business Focus General

Future
of OM2018 Definition of measurement
Innovation
guidelines agenda

9
 Central part of OM 2018:
Covered in New in OM
Framework for measuring OM 2005 2018

business innovation

Business
Innovation Business
innovation
Business innovation
objectives and
innovation
active firms activities (ch3) outcomes
(ch4) (ch8)

Innovation
and
knowledge
flows (ch6)

All firms Business

innovation
External
factors
capabilities influencing
(ch5) business inno
(ch7)

Internal dimension External dimension 10

 Revised innovation definitions for the
business sector

A business innovation is a new or improved

Business product or business process (or combination
thereof) that differs significantly from the
innovation
(ch3)

firm's previous products or business processes

and that has been introduced on the market or
brought into use by the firm.
Innovation activities include all
developmental, financial and commercial
activities undertaken by a firm that are intended
to result in an innovation for the firm.

11
 Types of business innovation

• A product innovation is a new or improved good

or service that differs significantly from the firm’s
previous goods or services and that has been
introduced on the market.
-> Goods and services. This edition clarifies role of
knowledge products that share features with both.

• A business process innovation is a new or

improved business process for one or more business
functions that differs significantly from the firm’s
previous business processes and that has been
brought into use by the firm.

12
 Taxonomy of business processes
• Based on the full array of business functions
• All functions potentially subject to improvements
that qualify as innovations
Business functions devoted to
1. Production of goods and services
2. Distribution and logistics
3. Marketing and sales
4. Information and communication technology (ICT)
5. Administration and management
6. Product and business process development

• Differs only slightly from OM2005 but has

more solid conceptual and practical basis 13
 Innovation activities
Business Innovation activities include all developmental, financial and
innovation
activities commercial activities undertaken by a firm that are intended to
(ch4) result in an innovation for the firm.

A range of knowledge-based activities highlighted for identification as

potential loci of innovation activity:
• Research and experimental development (R&D) activities
• Engineering, design and other creative work activities
• Marketing and brand equity activities
• Intellectual property (IP) related activities
• Employee training activities
• Software development and database activities
• Activities relating to the acquisition or lease of tangible assets
• Innovation management activities

Identify also when activity carried out by another party.

Identify expenditures associated to those items – total (KBC estimates) and

separating the amounts incurred in pursuit of innovation.
14
 Innovation within the innovation system:
Innovation and knowledge flows
Innovation
and
The chapter about “open innovation”
knowledge
flows (ch6) and “knowledge exchange”
• Outbound dimension of innovation
complementing traditional inbound
focus
• Sources of knowledge, collaboration
partners
• Recommended focus on knowledge
exchange channels with PRIs and
HEs
15
 Understanding internal and external factors
shaping innovation in the firm and outcomes
• Management
Business
innovation
capabilities
• IP
(ch5) • Human resources
• Technology in broad sense (technical, design, user
engagement, digital)

External
factors • Markets
influencing
business – Position in value chain, supply, demand, platforms
innovation
(ch7) • Public policy – including government support
• Society and environmental aspects

Business • Objectives – outcomes

innovation
objectives and
outcomes
• Impacts on market
(ch8)
• Quantitative measures
16
 Collection and use of Covered in New in OM
OM 2005 2018
data on business innovation

Business
innovation
Aimed at data producers: Recommend convergence in
survey Lifecycle perspective on survey country practices based on
methodology
(ch9) data collection and integration recent evidence

Object-based Complementary focus on most

approaches to
measurement
important innovation (or change) of
(ch10) the firm

Chapter intended for users of innovation data

Data use:
indicators and • Types of innovation indicators
analysis (ch11)
• Construction of indicators
• Analysis using innovation data

17
 Cross cutting issues

• Digital-proofing
– Digital products, ICT business processes, platforms, digital
capabilities, digital tools for measurement.
• Globalisation
– Global markets
– Cross boundary links within MNEs
– Location of business process functions, aligned to business
process innovation questions.
• Contribution to capacity building
– Guidance on developing countries in previous annex
mainstreamed into core document (esp. chapters on
internal and external drivers)
– Detailed methodology guidance for producers and users

18
 Policy and broader relevance of the OM

Demonstrated • Source of guidance for data collection

but in need of • Source of definitions and concepts
strengthening used in policy documents at OECD,
EU and country level
• Relevance to academics/researchers
• Policy relevance of statistical data
collected under OM guidelines
• Relevance to business
To be
demonstrated – Value to respondents?
through – Innovation management
implementation • Broader statistical relevance
– Better measuring the changing economy
 OECD Innovation data resources

• The Oslo Manual website

– http://oe.cd/oslomanual
• OECD Innovation Statistics and Indicators
– http://www.oecd.org/innovation/inno/inno-
stats.htm
• Community of practice on innovation data
– Password protected, for practitioners:
https://community.oecd.org/community/oslo
-review

20
 Other data resources using Oslo
Manual guidelines
Eurostat Community Innovation Survey (CIS) indicator database
Innovation indicators from the CIS for selected member states of the European Statistical
System (ESS): http://ec.europa.eu/eurostat/web/science-technology-
innovation/data/database.

Ibero-American/Inter-American Network of Science and

Technology Indicators (RICYT)
Innovation indicators for manufacturing and service industries for selected Ibero-
American countries: www.ricyt.org/indicadores.

UNESCO Institute for Statistics (UIS) Innovation Data

Global database of innovation statistics focused on manufacturing industries:
http://uis.unesco.org/en/topic/innovation-data.

The NEPAD (New Partnership for Africa’s Development) for the

African Union is also active in promoting the use of comparable indicators in
Africa. Online links to this manual will provide up-to-date links to international
and national sources of statistical data and indicators on innovation.
21
Beyond the 2018 edition

The 2108 Oslo Manual as

Platform for extending
measurement of
innovation following the
2016 Blue Sky Agenda

22
FOR MORE INFORMATION, EMAIL: [email protected]

FOLLOW US ON TWITTER: #OSLOMANUAL; @OECDINNOVATION 23

ADDITIONAL MATERIAL

24
ANNEX ON CHAPTER 9

25
Sections
• 1.2 Population and other basic characteristics
for a survey
– Reporting unit, main economic activity, unit size,
frequency of data collection, observation and
reference periods
– Recommendation that the observation period
should not exceed the frequency of data
collection

26
Sections
• 1.3 Question & questionnaire design
– Question design: translation, comprehension and
response categories
– Questionnaire design: filters, question order,
combining innovation with other business surveys
– Questionnaire testing: cognitive testing and pilot
surveys
– Recommendation to conduct cognitive testing

27
Sections
• 1.4 Sampling
– Survey frame, census versus sample, stratified
sampling
– Longitudinal panel data and cross-sectional
surveys
– Recommendations on sampling fractions, cells
defined by industry and size classes

28
Sections
• 1.5 Data collection methods
– Postal, online, telephone, face-to-face, combined
survey methods
• 1.6 Survey protocol
– Support for respondents, mandatory and
voluntary surveys, non-response follow-up,
conducting non-response surveys
– Recommendations for when to conduct a non-
response survey: response rates < 70%

29
Sections
• 1.7 Post-survey data processing
– Error checks, imputation of missing data,
calculating weights
• 1.8 Publication and dissemination of results
– Metadata and quality reports, data access
– Recommend improving data access if NSOs lack
resources for in-house analyses

30
ANNEX ON CHAPTER 11

31
 Why this new chapter?

• Need to consider both the production and

use of business innovation data
– Relevance and use as quality criteria
– Variety of users and potential users
– Address paradox and challenge of innovation
indicators without OM-based innovation data
• Capitalise and further promote good use of
data as indicators and analysis based on all
types of innovation data (macro and
microdata )
 Data and indicators on business
innovation

• Data, statistical data, stat data about

innovation
• Statistical indicators-> Innovation indicators
• Desirable properties of innovation indicators–
a reminder of quality requirements
• Basic principles
– International comparisons
• Sources for international resources
 Methodologies for business innovation
indicators
• Aggregation and simplification
– From microdata to macro. Relevant uses.
– Reducing complexity – composite indicators and
classification schemes
• Indicator development and presentation for
international comparisons
• Innovation headline indicators, dashboards,
scoreboards, composite indexes.
• Advantages and limitations.
• Firm-level data
– Micro-level scoreboards. Some health warnings.
 A blueprint for producing indicators on
business innovation

1. User needs
2. Choice of indicators
3. Choice of data sources
4. Interpretation
Broad thematic areas – matching the
manual’s part 2
 Other indicator issues

• Official vs non official sources

• The role of survey data
• Direct and indirect measurement
• Change vs position indicators
• Simple -> Complex indicators
• Limits of indicators
 Using data on innovation to analyse
innovation performance, policies and impacts
• Descriptive multivariate analysis –
examples
• Inference of causal effects in innovation
analysis
 Why measuring innovation impact is
difficult

• Direct and indirect methods.

• The CDM framework
 Policy analysis

• Specific implications for data collection

and analysis
• Procedures
• Coordinated analysis of innovation
microdata across countries