Identifying knowledge value measurement in a company - june 2006

Methodologies for identifying knowledge value measurement indicators
in a company
Paolo Petrucciani – Epistema – Rome - Italy

July 2006

Summary
This paper exposes a working methodology utilized in a real client case (Petrucciani,
2005), and subsequently generalizable, to identify a ‘knowledge management value’
measurement system in a company.
At the root of study-project was the top management need to measure and evaluate
company capability: a) to identify critical knowledge present in its population, with
reference to its specialistic service sector, b) to render explicit some tacit knowledge
present in its tech professionals, and c) to facilitate and optimize knowledge exchange
about systems and sw applications matters/issues, between various professionals,
prevalently for company’s institutional clients benefit, from the point of view of: a)
‘efficiency of existing knowledge asset governance processes’, b) ‘capitalization of
acquired experiences’ and c) ‘solutions engineering’.

1.1

Purpose of the project and the client case

The project, named “Knowledge Management Indicators”, started in the client “X” during
summer 2003, pushed by the Managing Director to investigate and better comprehend: 1)
how much, its own professionals, “know”, “know how to do” and “how many knowledge
and solutions share” to facilitate either productive internal efficiency and capitalization of
experiences acquired on the field, and 2) how to set and orientate internal collaboration
themes on exchange, feeding and diffusion of internal know-how, both technical and
application-oriented, to advantage internal growth and service to institutional clients.
Client “X” is a large public information technology company that creates and
implements sw management applications mainly based on mainframes and on clusters of
servers (more than one thousand) and PC, with either central and local-territorial data
processing and telematics, following its own customers requests and requirements,
second scope of the company is assistance and customer service (contact centers, web
1
Methodologies for identifying knowledge value measurement indicators in a company
july 2006 - Paolo Petrucciani

channels, knowledge base, etc.) to a large mass of users utilizing its applications. The
company employs about 1600 people in various functions, technical staff are about 1200
professional resources and individuals directly involved in the project team were roughly
15-20.

1.2

Identification of knowledge management value according to Balanced
Scorecard methodology

First activities of the project were addressed to an internal survey to identify which should
be considered critical success factors (CSF) (Rockart, 1979) with direct impact on
company services towards core clients, having in mind, as “principal driver for
investigation”, both company performance profile and internal/external know-how
circulation.
The methodology utilized to identify these critical factors, subsequently transformed
in KM indicators, is the Balanced Scorecard (BS) (Kaplan and Norton, 1996).
The objective of this first phase (recognition) of the project was to select rapidly
some factors-indicators able to measure “knowledge value” for the company, both under
economic, professional and specialistic profile.
As illustrative but not exhaustive note, some of principal critical factors (CF)
identified in this part of the survey (overall the project team identified about 20 critical
factors), also starting from quality system indicators of the company (certification ISO UNI
EN 9001: 2002, productive processes) and following 4 Balanced Scorecard perspectives
logic, were:


economic-financial perspective: company value generation linked to innovative
contracts content with existing clients and to traditional/innovative contracts with new
clients (where innovative contracts content stays for new or more efficient technology
solution offered);



customer perspective: applications effectiveness [defectiveness of sw applications
under guarantee or not], average delay of intervention for maintenance of sw
applications running; no. of successful solutions vs customer claims inbound calls to
first level call center; no. of internal users served by company knowledge base;



internal business processes perspective: time to market [speed to release new sw
products/applications]; tools for sw quality measurement;



learning and growth perspective: quantity of existing technical documentation on
running applications; overall active participative level to specialistic forum (under
2

intranet); amount of investments in specific education/training seminars finalized to
increase specialistic and technologies-methodologies knowledge.
A preliminary measurement system and measurement periodicity was then identified
for each factor-indicator.

1.3

From critical factors recognition to KM indicators, sharing their relative
importance and key-criteria for subsequent measurement in time

In the second phase (general setting) the project team transformed the identified CSF in
KM indicators and produced a system to validate, evaluate and graduate (rank) selected
factors-indicators by means of ‘democratic sharing mechanisms’, following ‘paired
comparison’ methodology (David, 1988) that permits to vote relative internal importance
between them.
Finded some key-criteria that should have permitted to facilitate the task to collect
and measure in time these indicators, with the same ‘paired comparison’ methodology, the
project team produced evaluations and graduations.
Below are shown the two final templates (Table 1, Table 2) that report votesgraduations realized by five client internal experts.
Parallel, as auxiliary operative tool, the project team build up a “collaboration matrix”
to track internal demand/offer process, in other words, whose technical offices were
involved in internal specialistic knowledge/consultancies requests (customers) and offers
(suppliers), based on work needs. This task was done first at cartaceous level (e.g.
recording and filing of e-mail requests), subsequently with electronic automation.

1.4

From

KM

factors-indicators

importance

to

creation

of

attributes/characteristics linked to their value
To complete second phase (general setting) the project team realized a special format
to clarify more precisely motivation of the choice of one KM factor-indicators vs others and
principal benefits derived by its measurement; see example below in succession (Table
3).
These activity was completed for six KM factors-indicators, of the eight selected,
considered particularly critical for their company value and impact.

3

Table 1
Final Ranking - KM Selected Factors

4

Table 2
Final Ranking - Key-Criteria for measuring KM Indicators

5

Table 3
Special Format - Attributes/Characteristics of KM Indicators

6

1.5

A performance management system for knowledge management

Finished first two phases (recognition and general setting), the project rapidly addressed
individuation of more analytical series of indicators, which, opportunely linked to company
productive processes, should have permitted to measure periodically either experiences
and knowledge exchange, and progressive know-how capitalization by means of data
base feeding (Petrucciani, 1986) and other internal expert systems appropriately prepared
(Petrucciani, 1988).
In this third phase (specific performance setting) the project followed mainly a
typical Performance Management methodology, oriented to identify KPI (Key Performance
Indicators) (Kaydos, 1999).
The client, with the support of a consultancy firm, specialized in this sector, build up
some basic assumptions and target representation templates showing logic framework of
final phases of the project.
The whole setting of work realized is presented in succession, linking the 4 Balanced
Scorecard

(BS)

Perspectives,

Performance

Management

(PM)

and

Knowledge

Management (KM) assumptions and initial objectives (Templates 1 and 2).
Template 1
BS-PM-KM basic assumptions

Assumption 1

Assumption 2

KM is decisive to improve
services performance for
institutional clients (quality,
satisfaction, realibility)

KM is as much more
effective as much
internal/external users may
benefit from its utilization

Assumption 4

Assumption 3

KM is decisive to acquire and
accumulate new knowledge,
coherently with company
strategic priorities

KM is as much more efficient
as its measurement tools are
pervasive in core processes of
the organization and
intensively utilized

7

Template 2
Initial objectives of company BS-KM Performance Management system

Objective 1

Objective 2

Find and evaluate KM
indicators to measure services
performance for institutional
clients

indicators to measure
internal/external users
usability (utilization,
benefits)

Objective 4

Objective 3

indicators to measure
acquisition, formalization and
exchange rate of new strategic
company knowledge

indicators to measure access,
diffusion, share and re-use of
existing knowledge in core
processes of the organization

Starting from illustrated whole design, the project team carried out in the following
weeks the fourth phase (analytical knowledge mapping), producing subsequently,
several analyses: a) existing/necessary knowledge typologies/maps to sustain specific
company core processes (e.g. institutional clients contract satisfaction, overall services
satisfaction, etc.), b) existing/necessary tools/instruments/calculations to measure
knowledge increase (e.g. ratio, data, surveys, statistics, trends, etc.), c) tentative initial
standard targets, expressed as specific ‘unit of measurement’ for each KM indicator,
mapping them all on existing company core productive business processes and taking into
account the exposed general phases of KM governance processes.
An illustration of utilized methodology it is shown below, where the macro-matrix
synthetizes in depth analyses realized by the project team (Template 3). Every box of the
matrix was initially fulfilled - as for point a) exposes above - from 1 to 4 items (for a an total
of about 50 item), that represented first idea of final KM indicators.
8

Template 3
KM processes – Core company business processes Macro-matrix

1.6

Knowledge mapping linked to core company business processes: core
knowledge

During the same fourth phase (analytical knowledge mapping), the project team
realized a complete reconnaissance of principal core competencies associated to
company business processes, starting from existing knowledge classification included in
company skill inventory and other techniques (McGraw, Harbison-Briggs, 1989).
In this occasion the project team produced a specific format to allow all managers of
technical

Organizational

Units

(architect,

project,

realization,

data

processing

management, systems and networks, assistance, education and training) to highlight
those they considered the ‘key-knowledge necessary to possess for a specific
business process’, then, in this way, associable to professionals of that specific
Organizational Unit (Phillips, Stone, Phillips, 2001).
In the next table (Table 4) it is shown the provided format for “Systems and data
processing management” business process – segment/profession: “Central and territorialperipheral applications processing”.
9

Table 4
Format - Core competencies in a company business process

Process:

Profession:
Central and territorial-peripheral applications
processing

Required level (knowledge requirements)
1

2

3

4

Notes

1 Processing Mainframe

2 Processing PC
3 Processor operation systems –
MVS/OS/390
4 Processor operation systems Windows XP
5 Access methods – VSAM

6 DBMS relational – DB2

7 OLTP TSO/ISPF

8 OLTP CICS

Previously not required in skill
inventory professional sheet
competence

9 JCL language

10 SAS language
11 Support tool - Endevor
Datawarehouse support tools
(Warehouse Manager)
12 Data Management

Previously not required in skill
inventory professional sheet
competence

Compilation notes:
Select a maximum of 10-12 competencies for specific profession, extracting them by specific segment of company skill
inventory (“Technology knowledge, Processes and methods, Professional experience, Foreign languages,
Institutional clients organizational/technological environment knowledge, Application typologies knowledge”),
adapting them to own organizational-professional environment and specific business process.
Concentrate on fundamental and strategic competencies to fulfil effectively profession indicated, or those
competencies that, if not present, may cause a possible drop in job performance, under know-how/expertise profile.

10

1.7

Executive dashboard, second and third level company KM indicators

In the

the project

team rationalized overall obtained results during previous weeks and months:
a) final identification of objectives for Balanced Scorecard-Performance Management
system for measuring company KM value;
b) analytical building up of architecture of company KM and scope. The structure of
the architecture represents the three organizational observations/dimensions of
company KM: Overall company, Core production business processes, Tools and
systems;
c) model/framework for company KM indicators, segmented in 3 layers levels: first
level-dashboard (6 items), second level-synthetic (18 items) and third level-detailed (40
items).
d) final fine tuning of the executive dashboard and other levels KM indicators, for
their evaluation and monitoring in time;
e) setting of data model (for subsequent recovering and calculation);
f) procedures for collecting and processing statistical data relative to each KM
indicator;
g) setting and creation of calculation algorithms for each KM indicator.

In the following pages are presented, respectively, point a) (Template 4), point b)
with explanation of the KM architecture (Template 5), point c) with explanation of
normalized and weighed sums (Template 6), and an example of a third level-detailed KM
indicator for Overall company dimension (Template 7), and, finally, for point d), the
illustration of the executive dashboard overview (Template 8) and the complete showing of
one the 6 KM indicators of the executive dashboard, related to Tools and systems
dimension (Template 9), including the cascading of first, second and third level KM
indicators.
In the last Template 9, the numbers included in green and red boxes express
complements to 100% (delta %) in case of not achieving (red) or achieving/over-achieving
(green) vs established targets, as result of calculation formula for each relative KM
indicator.

11

Template 4
Final objectives of company BS-KM Performance Management system

Safeguard and enhance client relationships
Improve client environment knowledge
Create value for clients
Improve cost-effective utilization of instrumental
and human resources
o Support clients in strategic technology choices

o
o
o
o

o Generate new knowledge
o Accumulate existing and new knowledge
o Rationalize, formalize and standardize
knowledge

for user friendliness and easiness (knowledge
base, information support systems, education,
training, competencies centers, web channels,
etc,.)
o Encourage KM tools utilization

o Evaluate degree of knowledge diffusion
o Improve organizational solutions for
knowledge access and diffusion

o Select and develop critical knowledge for the
company

o Destroy/eliminate old, redundant or obsolete
knowledge

o Verify internal/external users satisfaction
o Propose and improve organizational solutions

o Monitor knowledge disposability to improve
its sharing and exchange
o Monitor knowledge access frequency to
evaluate internal practices to re-use or reinvent

12

Template 5
Architecture and dimensions of KM indicators

FOCUS ON OVERALL COMPANY
(coherence between company strategies,
policies, objectives and
operative decisions)

KM
MM

OVERALL COMPANY
KM Indicators

CORE PRODUCTION
BUSINESS PROCESSES
KM Indicators

TOOLS AND SYSTEMS
KM indicators

CORE PRODUCTION
BUSINESS PROCESSES

FOCUS ON

(organizational know-how memories and
practices)

TOOLS AND
SYSTEMS

FOCUS ON

(organizational levers and solutions
supporting company KM)
13


Template 6
Model/framework of KM indicators

Overall
company

Core
production
business
processes

Tools and
instruments

First level –
executive
dashboard

Second levelSecond levelSecond level –
– synthetic –
KM indicators

o
o

Third level – detailed
KM indicators

Third level –

o

14

Overall
company

Template 7
A third level-detailed KM indicator (Overall company dimension)

First level –
dashboard

Final objective 2 “Make knowledge usable”

Second level synthetic

Executive dashboard – FIRST LEVEL
“Make knowledge usable – at Company level”

Third level detailed

Synthetic KM Indicator – SECOND LEVEL
3.2 Degree of encouraging KM tools utilization
INDICATOR TITLE/NAME + FORMULA

INDICATOR DESCRIPTION AND RATIONALE

3.2.2 KM accessibility degree (1)
This indicator expresses the percentage of company employees
which is allowed complete access to all “KM drivers” in the
company (education, documentation system, internet, intranet,
extranet, forum, press review) vs total employees.

Width of intersection subset:
employees allowed to any specific
KM driver (with a min of education:
5 days/year)
X 100

total company employees

elementary data
NUMERATOR
• employees registered
allowed to any specific
service (KM driver) with a
minimum of 5 days of
education in the year
DENOMINATOR
• all company employees

Increase in this indicator stays for a larger accessibility to KM
tools within company employees.

organizational zoom

• all company
• first layer departments

feeding source
• HR data
• internet, intranet, extranet

survey periodicity

• half-yearly

users list

• documentation system users
list

• employees activities sheets
• press review users list

15

Template 8
Executive dashboard for company KM indicators (overview)

EXECUTIVE DASHBOARD OVERVIEW – 6 KM Indicators – FIRST LEVEL

First level –
dashboard

Second level synthetic

Third level - detailed

CORE PRODUCTION
BUSINESS
PROCESSES
“Improve capability to satisfy
institutional clients and
perform better services”

1. Improve capability to
fulfil company mission
and relative coherences

2. Make knowledge usable
– at Company level

“Make knowledge usable”

3. Make knowledge usable
– at core production
business processes level

4. Evaluate organizational
levers and solutions to
support KM diffusion and
re-use

“Create new processes to
enhance existing knowledge
diffusion and exchange”

“Manage the knowledge life
cycle”

5. Generate, accumulate
and develop knowledge –
at Company level

6. Generate, accumulate
and develop knowledge –
at core production
business processes level

16

Template 9
Executive dashboard of KM indicators (KM indicator no. 4 – Tools and systems)
Final
O bjective 3

TO O LS AN D S Y S TE M S D IM E N S IO N - K M IN D IC ATO R S (K P I)

E X E C U T IV E

DASHBOARD -

F IR S T L E V E L

S Y N T H E T IC K M

IN D IC A T O R S -

SECOND LEVEL

"C reate n ew p ro cesses to en h an ce existin g kn o w led g e d iffu sio n
an d exch an g e"
code

K M IN D IC A TO R TITLE /N A M E

AC H IE V E D

4

E valuate organizational levers and solutions to support K M
diffusion and re-use

50%

50%

code

K M IN D IC ATO R TITLE /N A M E

A C H IE V E D

25%

4.1

D iffu sio n o f co m p an y
d o cu m en tatio n system

38%

39%

25%

4.2

D iffu sio n o f co m p an y
in tran et system

27%

24%

code

K M IN D IC ATO R TITLE /N A M E

A C H IE V E D

TA R G E T

15%

4.1.1

P ublic publishing of new
docum ents (into
docum entation system )

20%

30%

20%

4.1.2

P ublic publishing of new
docum ents vs all docum ents
(in docum entation system )

75%

30%

4.1.3

D egree of utilization of
com pany docum entation
system

25%

4.1.4

D

TA R G E T

W E IG H T %

D E T A IL E D K M IN D IC A T O R S - T H IR D L E V E L

W E IG H T %

10%

D

TAR G E T

A verage of docum ents
downloaded by
docum entation system (sam e
departm ent)
A verage of docum ents
downloaded by
4.1.5
docum entation system
(different departm ent)

0%

D

W E IG H T %

code

K M IN D IC ATO R TITLE /N AM E

AC H IE V E D

TAR G E T

-1%

25%

4.3

D iffu sio n o f in tern an
ed u catio n an d train in g

75%

77%

-2%

3%

25%

4.4

D iffu sio n o f co m p eten cies
cen ters co m p an y

63%

63%

0%

W E IG H T %

code

K M IN D IC ATO R TITLE /N AM E

AC H IE V E D

TAR G E T

-10%

40%

4.3.1

A verage intense
education/training (m inim um of 5
days/year)

76%

80%

-4%

75%

0%

60%

4.3.2

A verage education/training
days/year for em ployee

75%

75%

0%

40%

35%

5%

50%

4.4.1

40%

50%

-10%

10%

10%

0%

50%

4.4.2

85%

75%

10%

50%

60%

-10%

D

55%

4.2.1

D egree of utilization of
com pany intranet system

40%

35%

5%

45%

4.2.2

A verage pages downloaded
from com pany intranet
system s

10%

10%

D egree of utilization of com pany
com petencies centers
organizational units by other
units
B illing days by com pany
com petencies centers
organizational units for
institutional clients

D

0%

17

KM indicators measurement activities started in the company during winter 2004.
During that period the project team helped to find initial standard to measure and confront
in time, with established periodicity. In this sense any subsequent measurement served
for monitoring improvement/worsening in any KM indicator (whichever dimension) and
taking necessary corrective actions, including calculation formulae.
Subsequently, in spring 2004, the knowledge management project team produced a
survey based on a perceptive questionnaire about the state of the art of KM in the
company, in which the participants made clear points of improvement of the KM
measurement system designed.

1.8

Conclusions

Working methodologies presented in the paper are based on collaborative and shared
identification criteria of a knowledge management indicators system, for their subsequent
measuring, evaluating and monitoring. In this sense the ‘value of KM for the company’ is
evolutionary and semi-stable and may depend strongly (Sveiby, 2006) by the part of
company involved in these topical decision: top management, experts and/or
professionals that address their own open perception of a quantifiable dimension linked to
knowledge (e.g. revenue, profit, quality, culture, organizational alignment, etc.).
The tools and indicators identified enable to measure periodically exchange of
experiences and knowledge that happens in a company (Petrucciani, 1990) and permit, at
the same time, to consider their trend/situation in time to intervene prevalently with: 1)
cultural, educational and training actions, 2) direct organizational interventions to facilitate
knowledge exchange on the job, 3) creation of communities of practices (CoP) and
working group (Dalkir, 2005), even semi-permanent, based on critical knowledge-sharing.
The project highlighted that KM indicators selected are flexible, evolutionary and
open to dynamic adaptation, following technological and application development for
institutional clients reached by the company.

18

References (last publishing date first)

Karl-Erik Sveiby, Soft assets; measuring the immeasurable (chapter 10, pgg. 91-105), in
Next Generation Knowledge Management, by Jerry Ash, Association of knowledgework AOL, Ark Group, London, 2006
Paolo Petrucciani, Metodologie per l’individuazione di indicatori di misurazione della
conoscenza in azienda, 10° Knowledge Management Forum”, organized by Jekpot srl,
Siena 24-25 november 2005
Kimir Dalkir, Knowledge Management in theory and practice, Elsevier, 2005
Jack J. Phillips, Ron D. Stone, Patricia Pulliam Phillips, The Human Resources
Scorecard – measuring the return on investment, Butterworth-Heinemann, Boston, USA,
2001
Will Kaydos, Operational Performance Measurement, St. Lucie Press, New York, USA,
1999
Robert S.Kaplan, David P. Norton, The Balanced Scorecard – translating strategy into
action, Harvard Business School Press, Boston, Massachussets, 1996
Paolo Petrucciani, Verso il duemila: incertezze e valori, Tempo economico-Rivista di
management, F.lli Pini Editori Srl, july-august 1990, Anno XXVII, n.299-300, pagg.54-62
Karen L. Mcgraw, Karan Harbison-Briggs, Knowledge Acquisition: Principles and
Guidelines, Prentice-Hall, Englewood Cliffs, New Jersey, USA, 1989
Paolo Petrucciani, L’impatto organizzativo dei sistemi a supporto delle decisioni,
Informatica e Direzione aziendale, Cedis Editrice, june 1988, Anno 3, n.6, pagg.8-21
David. H.A., The method of paired comparisons, second edition, Chapman and Hall,
London, 1988
Paolo Petrucciani, Produttivita’, management e Apprendimento- 3a parte: nuove frontiere
nell’apprendimento, Impresa e Societa’, Cedis Editrice, 15 april 1986, Anno XVI, n.7,
pagg.6-19
John F. Rockart, Chief Executives define their own data need, Harvard Business Review,
March-April 1979

19

Identifying knowledge value measurement in a company - june 2006

More Related Content

What's hot

Viewers also liked

Similar to Identifying knowledge value measurement in a company - june 2006

More from Epistema

Recently uploaded

Identifying knowledge value measurement in a company - june 2006