Criteria for Screening

New Industrial Products

Robert G. Cooper
Ulrike de Brentanil
Screening new industrial product ideas-the initial COIN0
GO decision in the new product process-is a critical deci-
sion. This article reports the results of an extensive investiga-
tion into what criteria managers use in their screening deci-
sions, and how these criteria are weighted and combined.
INTRODUCTION
The screening of new product proposals is the first
critical evaluation in the new product process. While
many normative approaches have been proposed for
screening new products, there is little evidence that these
methods have seen widespread application, nor that the
screening decision is undertaken in a proficient manner.
A need exists, therefore, to probe how managers screen
new products in order to gain a better insight into this
vital decision area. The purpose of the research reported
in this article was to identify the new product screening
criteria used by managers, and to determine how these
Results presented in this article are based on the Ph.D. dissertation of
Ulrike de Brentani, Evaluation of Industrial New Product Ideas: An Empirical
Study, McGill University, Faculty of Management, March 1983.
Address Correspondence to: Robert G. Cooper, Faculty of Business,
McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada
L86-4M4.
criteria are weighted and combined in order to yield a
GO/NO GO decision.
New product screening is the initial decision to com-
mit resources to a new product project. It is a critical
decision, and also a difficult and complex one. At the
screening stage, relatively little reliable information is
available on the proposed products market, it costs, and
the nature of the investment required [ 1, 131 .Screening
then, for many projects, amounts to an investment deci-
sion, but made in the absence of concrete financial data
[6]. The lack of data highlights the major conceptual and
methodological differences faced in new product screen-
ing decisions versus traditional investment-and-return
decisions [ 11.
A number of normative approaches to new product
screening have been proposed. These include: portfolio
models, based on linear, integer or dynamic program-
ming; benefit contribution methods, such as check lists,
scoring models, and paired comparisons; financial return
models, both deterministic and probabilistic; and market
research approaches [2, 141. But of all approaches, only
benefit contribution, notably check lists and scoring
models, is regularly used by managers [3, 41. Indeed,
check list and scoring models are deemed most appropri-
ate for screening, where only rough distinctions among
projects are required [ 121.
Industrial Marketing Management 13, 149- 156 (1984)
0 Elsevier Science Publishing Co., Inc., 1984
52 Vanderbilt Ave., New York, New York 10017
149
0019-8501/X4/$03.00
The fact that few normative approaches see wide- * attribute elicitation: Where managers were directly
spread application, together with the questionable predic- asked to indicate what screening criteria they used.
tive abilities of firms screening decisions raises serious 9 mod$ed repertory grid: Respondents were asked to
questions about the nature of the screening decision in compare several recently screened proposals, and to
firms. The research reported in this article was designed indicate how each proposal differed from the others in
to probe managers screening decision behavior and terms of the reasons for the GO or KILL decision. By
sought answers to the following questions: shifting the focus from screening criteria to actual
the areas of product and market opportunity
were important
l What are the screening criteria or screening variables
used by managers?
l Can these be reduced to a subset of screening factors-
a multidimensional representation of the screening
decision?
l What is the relative impact of these criteria or screen-
ing dimensions on the GO/NO GO decision?
THE RESEARCH
A two phase research study was designed. The objec-
tive of Phase 1 was to develop a comprehensive set of
screening criteria that managers indicate they employ.
Phase 2 focused on actual screening decisions and related
the GO/KILL decisions made on actual new product
projects to the ratings of these projects on the set of
screening criteria.
Phase 1
Personal interviews were arranged with managers
closest to the screening decision in a sample of 45 firms
known to be active in new product development. Three
separate approaches were used during the lengthy inter-
view to elicit the set of screening criteria used by the
managers:
__
ROBERT G. COOPER IS Professor of Marketing at the Faculty
of Business, McMaster University, and Director of Research
of the federally funded Canadian Industrial Innovation Centre
(Waterloo).
ULRIKE DE BRENTANI is Associate Professor of Marketing,
Faculty of Commerce and Admlnistratton, Concordla
University
150
projects, a number of additional criteria were un-
covered.
l list completion: A comprehensive list of screening
criteria, developed from the literature. was shown
respondents. Respondents were asked to check off or
add criteria that they used.
Content analysis yielded a list of 86 screening items.
Phase 2
In phase 2, the sample was expanded to 63 industrial
product firms and 243 managers in these companies, who
were regularly involved in screening decisions and
agreed to participate in the study. The unit of analysis
became individual new product proposals: each manager
was asked to identify two new product proposals, one an
accept and the other, a reject. Respondents rated
the proposals on each of the 86 items identified in phase
1 using seven point Likert scales. A degree of project
acceptability was also measured: minus five to plus five,
strong accept to strong reject. A total of 192
managers completed the questionnaires (response rate of
58.1%), yielding data on 370 projects.
RESULTS: SCREENING DIMENSIONS
An analysis of the ratings of each of the 370 projects
studies showed that many of the 86 screening criteria
were strongly correlated with each other. Therefore fac-
tor analysis of the screening variables, as suggested by
Schocker, Gensch, and Simon [ 111, was undertaken
(principal component analysis, varimax rotation). The
appropriate number of factors was determined using the
scree test which identified 9 and I 1 factor solutions [ 5 1.
The 11 factor solution was chosen based on interpretabil-
ity of factors. The factor solution was validated by cal-
culating Cronbach alphas (internal consistency) for each
of the 11 factors [9]; the results were positive (YS ranging
from 0.46 to 0.90.
The 11 factors proved to be easily interpreted, featured
strong variable loadings, and together explained 49.1%
of the variance of the original 86 variables. Table 1
presents these factors and the more important variable
loadings.
Three of the screening factors capture the magnitude
of the product opportunity: the products expected ad-
vantage (F-l), its financial potential (F-5), and its proba-
ble life (F-6). The first, Product Differential Advantage
(F-l) denotes the likely advantage the product will enjoy
in the market largely because of its innovativeness and
technology. Products strong on this dimension were the
first of their kind in the market, represented a revolution-
ary innovation, and were clearly differentiated from
competitors products. Moreover, the product presented
an opportunity for the firm to become a technological
leader in the market, while the product itself had tech-
nological strength in the market.
The second product opportunity factor gauges the ex-
pected financial potential for the product (F-5). Product
ideas rated high on this dimension had a high profit and
sales growth potential, a large expected market share, a
strong likelihood of success, and were aimed at high
growth markets.
The final product opportunity factor, Product Life
(F-9), describes the expected life and stability of the
.
rational market (F-10): The customers use objective
decision criteria, have a sound understanding of the
product, and are commercial (versus institutional)
customers.
l domestic market (F-l 1): The market is a domestic
one, and competition is primarily domestic.
Three screening dimensions capture the fit or synergy
of the project with the firms existing resource base, and
denote the ease with which the firm can undertake the
project utilizing existing resources. The first synergy
factor is Corporate Synergy (F-2): a good fit in terms of
distribution channels and salesforce, business, custom-
ers, marketing research resources, organization, and
management capabilities. The second dimension is Tech-
nological and Production Synergy (F-3): a fit in terms of
production resources engineering and design skills, and
in-house availability of component parts. The final di-
mension that describes the firms ability to undertake the
project is a financial one, Project Financing (F-4). This
factor describes projects where outside funding is both
required and available for the project; e.g., from custom-
ers or government agencies.
The final two screening dimensions uncovered in this
analysis deal with product strategy. Both factors describe
the role of the proposed product when viewed in the light
of the firms new product mission or overall new product
strategy. They include:
. diversification (F-7): the product is the key to entering
a new product class, a new market, or a new tech-
nology to the firm.
How do managers screen new products?
proposed product. If introduced, such products were not
expected to change for a long time, and the product was
thought to have a long life. Moreover, the future devel-
opment pattern for the product was clear and predicable
(Table 1).
Another three factors portray a second facet of the
proposed product, namely its market opportunity. These
are :
l size of market (F-6): The market is a mass market, has
high dollar volume, is broad geographically and has a
large long term potential.
l market maintenance (F-8): the product is important to
defending a market, to surviving in the business, or
replaces or updates a current company product.
These four major categories of screening dimensions,
namely Magnitude of Product Opportunity, Market Op-
portunity, Synergy and Product Strategy, closely parallel
screening dimensions suggested in the literature. For
example, OMearas model speaks of Marketability, Du-
rability, Productive Ability, and Potential [lo]. Similar-
ly, Coopers NewProduct screening model, based on the
analysis of actual successes and failures, identified three
151
TABLE 1
Factor Analysis Results: Screening Dimensions
Factor Name Variables Loading On Factor
FI
Product Differential Advantage
(13.4% cy = 0.90)
F2
Corporate Synergy
(lo.V%; (Y = 0.89)
F3
Technological and Production Synergy
(4.5%; u = 0.79)
F4
Project Financing
(3.7%; cy = 0.69)
~~_.
FS
Financial Potential
(3.3%; cy = 0.80)
F6
Si7c of Market
(3.0%~: LY = 0.65)
F7
Diversification Strategy
(2.6%; u = 0.65)
F8
Market Maintenance Strategy
(2.3%; u = 0.69)
F9
Product Life
(2.0%; (Y = 0.49)
FIO
Rational Market
(2.0%: 01 = 0.46)
FI I
Domestic Market
(1.9%; u = 0.47)
First to introduce product to market
Opportunity to become technolo@d leader in market
A revolutionary innovation
Clearly differentiated from competitors products
Achieves an important technological strength
Makes the firm a major entity in the market
Involves the application of a different technology to a problem
(Not) Gmilar to competitive offerings
Product is patentable
Product is of higher quality than what is on the market
Technologically delivered product-in house
Requires considerable technological expertise
Produces important savings for customer
Uses firms distribution and salesforce
Fit5 firms present busmess
Aimed at firms current customers
Uses firm\ marketing rcsearch rcwurcc~
Fit5 the fnma organizational set-up
Fits the firms managerial capabilities
Prospective competitors are known/understood
Fit\ top managements preferences
Fits into firms corporate strategy
Use\ firms engineeringidesign resources
Fits production facilitie\
Firm knows production method5
Product 15 comprised of current product5 or materials
Outside funding is required
Major customer investment is required
Complex financing is required
Expected ROIiprofit potential i\ hiph
Expected sales growth is high
Expcctcd market growth is high
Expcctcd market share is high
Likelihood of success i\ high
Market is il nws market
Dollar market potential is large
Ma&et is broad feoer;lphically
Future market potential i\ high
Product has variety of applications
ApgreGve competition
A key to entering 3 new product class to firm
M;rrket ha\ room for :! new competitor
A key to entering :i new market to firm
A key to cntcring ;i new technology to firm
A defensive (mamtain share) product
A wrvivvl strategy
Rcplaceh present products
Rcprescnts 2, technological updatc/enhancemcnt
Product will not change for long time
Future development pattern is clear and predicable
Ha5 lonp cxpccted life
Cu\tomcrs use quantitative/objective decision criteria
Competition is oligopolistic, not monopolistic
Cu\tomer~ understand product
Customer\ iirc commercial ;ib opposed to institutional u\cr$
Market is il domestic one
Competition oh domc\tic
Serves need previously poorly satisfied
Factor Loading
0.7x3
0.777
0.724
0.703
0.690
0.60.5
0.604
-0.582
0.572
0.564
0.564
0.553
0.528
0.827
0.771
0.761
0.737
0.708
0.700
0.592
0.571
0.570
0.716
0.704
0.702
0.5x4
0.720
0.68 I
0.647
0.673
0. SW
0.551
0.4x I
0.4x0
0.666
0.57Y
0.537
0.4)s
0.475
0.426
0.669
0.5.53
0.552
0.482
0.657
0.632
0.518
0.467
0.649
0.547
0.484
0.5.59
-0.48 I
0.452
0.409
0.657
o.ssu
0.410
Pcrccnt\ indicate percent of variance explained. prior to rotatmn. ;md add to 39. I%. The Cronbach (Y is alw indicated.
152
major categories: Marketability, Product/Company Fit,
and Market Opportunity [6].
IMPACT OF SCREENING DIMENSIONS
The 86 screening items elicited from managers were
reduced to an 11 factor representation of the screening
domain (above). The next question concerns the impact
or relative importance of each of these dimensions to the
GO/NO GO decision; that is, what factors are most
important to the decision to move ahead with a project,
and what are their relative weightings?
A generalized screening model-one which portrays
the screening decisions for the average project-was
developed by relating these 11 screening dimensions to
actual screening decision outcomes. Two-group discrim-
inant analysis was used to relate the outcome of the
screening decision, GO or NO GO, to the projects factor
scores on each of the 1 1 dimensions. Multiple regression
was also used, where the criterion variable was the de-
gree of acceptance or rejection of the project, a continu-
ous variable.
The results of both analyses were virtually identical. A
total of nine statistically significant factors were identi-
fied (ar I 0.01 for the discriminant solution; o 5 0.05
for the regression). The regression equation explained
56.4% of the variance in the degree of project accep-
tance, while the discriminant analysis correctly classified
86.2% of the cases (jackknife method of validation).
Both solutions were strongly significant (cx I 0.0001).
Only results of the regression analysis are shown (Table
2).
TABLE 2
Regression Analysis Results: Degree of Acceptance/Rejection
Versus Screening Dimensions
Standardized
Screening Factor Regression F Significance
In Solution Coefficient Value of F
F5 Financial Potential 1.398 158.2 0.0001
F2 Corporate Synergy 1.285 134.2 0.0001
F3 Technological and Production 0.932 69.9 0.0001
Synergy
Fl Product Differential 0.881 62.7 0.0001
Advantage
F9 Product Life 0.576 26.7 0.0001
F8 Market Maintenance Strategy 0.425 14.6 0.0002
F6 Size of Market 0.384 12.0 0.0006
F7 Diversification Strategy 0.270 5.8 0.0161
FI I Domestic Market 0.223 4.1 0.0450
Ad,justcd R? = 0.564; F of relationship = 54.3. significant at the 0.0001
level.
The Dominant Criteria
The most important single screening dimension was
the Financial Potential (F-5). Products where the ex-
pected profit and sales growth, the market share expecta-
tion, the likelihood of success, and the market growth
were all high were strongly favored in the initial GO/NO
GO decision. Not only were these measures of market
and financial performance strongly related to each other,
but taken together, they were most influential in the
decision to move ahead with a project. Quantifiable per-
formance measures thus dominated the project evalua-
tion, even at this early screening stage despite the limited
reliability of such measures at this early stage.
The next two most critical screening criteria both de-
scribe synergy or fit. Corporate Synergy (F-2), where
there existed a good fit in terms of distribution channels
and salesforce, the firms current business, the firms
customers, etc. , is the more important dimension. Next
in importance is Technological and Production Synergy
(F-4): a fit in terms of engineering and design skills and
resources, and production facilities and knowledge. The
key role of these two synergy factors is strong evidence
of managements desire to select projects which can
make use of the firms existing resources at marginal
cost, and which exploit the firms previous experiences
and skills.
One more factor was found particularly important in
the screening decision: Product Differential Advantage
(F-l). Products that were first to market, promised a
leadership position, were innovative, and had a tech-
nological edge were most often accepted at the screening
stage. The fact that a differential advantage is a strong
positive factor in the acceptance of a new product project
is not surprising. Perhaps of greater concern is that man-
agement rates two synergy dimensions, both internal
resource criteria, ahead of differential advantage, a prox-
ie for potential success and payoffs.
These four dimensions dominated the screening deci-
sion, together accounting for 86.5% of the variance
eventually explained by the regression model. To a large
extent, therefore, criteria that describe financial poten-
tial, synergy and product differential advantage deter-
mine whether or not firms move ahead with a new prod-
uct project.
Secondary Criteria
Another five screening factors were found related to
project GO/NO GO decisions, but in a less dramatic
way. Product Life (F-9) was positively related to the GO
decision. Products which were expected to remain un-
153
changed, to have a long life, and to have a clear and
predicable development pattern tended to be evaluated
more positively in the screening decision. Management
appears to be seeking projects where a predicable and
long life are possible, to diminish uncertainty and to
ensure a steady stream of future returns. Note that all
three factors describing the product potential, Financial
Potential (F-5), Product Differential Advantage (F- 1 ),
and Product Life (F-9), were important criteria in the
screening evaluation.
Both strategy criteria were tied to screening decisions
and in a positive fashion. The rnarket maintenance crite-
rion is the more important of the two.
ing decision. The result is surprising when one considers
that the screening variables, hence dimensions, were in
fact derived from what managers indicated they consid-
ered in their decisions. But actual decision behavior was
not tied to two dimensions in a consistent fashion. The
least significant factor was Project Financing (F-4). Al-
though often mentioned by managers, it appears that
whether a project requires and/or receives outside fund-
ing from government or customers has little bearing on
whether the project moves ahead or not.
The second factor with little effect on screening out-
comes was the Rational Market dimension (F- 10).
Whether potential customers are rational buyers or not,
the most important screening criteria was
the financial potential.
l Projects that promised to maintain markets (F-8),
defend a market, survive in a business, or replace or
update a current product, were more positively rated.
l Projects that created opportunities for diversification
(F-7)-a new product class to the firm, a new product
or a new technology-fared better in screening.
Those dimensions that describe the nature of the mar-
ketplace, surprisingly had a moderate-to-low impact on
the screening decision. None of these dimensions are
among the dominant criteria. Moreover, the factors, Size
of Market (F-6) and Domestic Market (F-l I), entered the
regression relationship as the seventh and ninth most
important criteria. Projects tended to fare better at the
screening evaluation when they scored high on:
l sizr of market (F-6): a mass market, large dollar
volume, large long term potential, and broad geo-
graphically; and
l domestic market (F-l 1): a domestic market with do-
mestic competition
Note, however, that certain market items, namely ex-
pected market share and market growth, were included in
the dominant screening factor, Product Potential (F-5).
Dimensions With No Impact
Only two of the 11 screening dimensions appear to
have no consistent or significant influence on the screen-
154
whether they understand the product, and whether indus-
trial users versus institutional, had little effect on the
GO/NO GO decision.
IMPLICATIONS
Screening decisions appear to be dominated by four
main considerations. Two of these describe the magni-
tude of the product opportunity itself, namely Financial
Potential and Product Differential Advantage; the other
two are synergy factors: Corporate Synergy and Tech-
nological Synergy. None portray the nature of the market
at which the new product is aimed.
These results are in part reassuring. The key criteria
identified in this study of decision behavior parallel, to a
certain extent, the criteria obtained from studies of actual
product outcomes. Screening dimensions that emphasize
synergy and differential advantage of the product were
found important in the current research, and coincide
with normative screening models and with studies on
actual product outcomes:
Dominant Screening Criteria From This Research (Mana-
gerial Decision Behavior)
1.
2.
3.
4.
Financial Potential.
Corporate Synergy.
Technological Synergy.
Product Differential Advantage.
Factors Found Important To New Product Successess [6,
71
1. Product Uniqueness and Superiority (Differential
Advantage).
2. Project/Company Resource Compatability.
3. Market Need, Growth and Size.
4. Economic Advantage of the Product to the User.
The fact that these four criteria are so dominant, ex-
plaining the great majority of screening decisions, also
raises provocative questions. First, the results suggest
that many managers may oversimplify the screening de-
cision by reducing it to a handful of evaluative criteria.
That is, a comprehensive and complete set of criteria, as
is suggested in the literature, appears lacking in this
critical evaluation. Second, the fact that almost no mar-
ket criteria are present in these dominant dimensions
points to an unbalanced screening approach; perhaps
there exists too much emphasis on internal issues, for
example, synergy questions, and not sufficient consid-
eration of external market considerations. One might
speculate that managers are either not aware of the im-
portance of market factors in project evaluation, or else
are unable to assess market variables and to build these
assessments into their screening decisions. In a similar
vein, the products differential advantage, of first impor-
tance in product success/failure studies was a lower pri-
ority item in managers decisions. The relative emphasis
placed by managers on screening criteria must be ques-
tioned. Third, the fact that the most critical criterion,
namely Financial Potential (F-5), is comprised largely of
financial, sales and market gauges raises concerns, par-
ticularly when one realizes that such quantifiable gauges
are likely to be highly unreliable at the point of early
screening. It could be that management is placing too
much faith in traditional, simple, and quantifiable eval-
uative criteria when in fact such measures are of ques-
tionable reliability.
A surprising finding is the lack of impact of the factor,
Project Financing. The importance of outside funding for
projects, either from customers or government agencies,
was often mentioned by managers during the phase 1
interviews. Yet an analysis of projects that passed versus
failed screening shows no consistent pattern in terms of
outside financing. In practice, whether a project requires
or receives outside funding simply has little impact on
the GO/NO GO decision. This finding has obvious im-
plications to public policy, where the assumption in
many countries is that corporations require government
support of R&D projects. Our results suggest that such
funding probably has little effect on the decision to move
ahead with a project.
A final conclusion concerns the use of the current
research findings as a guide to managerial practice. In the
first place, a comprehensive list of screening items and
dimensions has been identified in the research. Such a
list provides a useful starting point for a manager in the
development of his or her own firms screening model.
Second, a model based on the actual decision behavior of
managers may indeed be useful as a normative guide to
future decisions. Considerable evidence exists that sup-
ports the notion of replacing a manager with a model of
his or her actual behavior [8]; and that an average model
based on behavior is fairly close to an optimal decision
model. If this is true, then the results of the current
research can provide a useful guide to future screening
decisions, both in terms of which items are important to
consider, and the relative importance of these.
New product screening at best will remain a spec-
ulative decision. But effective screening is critical to the
overall performance of a firms new product program.
Improved screening approaches and models, in part
based on the findings of this research, have considerable
potential to heighten the effectiveness of the screening
decision.
ACKNOWLEDGEMENTS. This research was supported
by a grant from the Canadian Government, Dept. of
Regional Industrial Expansion, Office of Industrial
Innovation.
REFERENCES
1.
2.
3
4.
5.
6.
7.
8.
Albala, A. Stage Approach for the Evaluation and Selection of R&D
Projects, IEEE Transuctions on Er~~inecvin~ Marqymrnt. EM-22,
153-164, (1975).
Baker, N. R.. R&D Project Selection Models: An Assessment. IEEE
Transtrcrions o,? En~irzrerinl: Manu,ymmt, EM-21, I6Sp I7 I ( 1974).
Baker. N. R. and James Freeland (1975). Recent Advances in R&D
Benefit Measurement and Project Selection Methods. Munugemern Sci-
r,~ce. 21, 1164-1175 (1975).
Bradbury, F. R., W. M. Gallagher and C. W. Suckling (1973).
Qualitative Aspects of the Evaluation and Control of R&D Projccta.
R&D Manapnc,~t. 3, 49-57 (1973).
Cattell. R. B. The Scree Test for the Number of Factors. Mulriruricrrc
Brhuviorul Rcsecrrch. 1, 245-276 (1966).
Cooper, Robert G. (1981) An Empirically Derived New Product Project
Selection Model, IEEE Trmsuc~tion.s on Enginrerin~ Munupvmwr.
EM-28, 54-61 (1981).
Cooper, Robert G., The Dimensions of New Industrial Product Success
and Failure, Journd of Marketing, 43, 93-103 (1979).
Goldberg, L. R., Man Versus Model of Man: A Rationale Plus Some
Evidence for a Method of Improving Clinical Inferences, Pswholqical
Bulldn. 73, 422-432 (1970).
155
9. Horn, John L., On the Internal Consistency Reliability of Factors.
Multiwrrrrrc Behaviorul Rescwch. 2, I IS- 125 ( 1969).
10. OMeara, J. T. Jr. Selecting Profitable Products, Hmw~rd B~~.~itw.ss
Revim~. X3-89. (January/February 1961).
I I. Schocker. A. D., D. Gensch and L. S. Simon (19691, Toward the
Improvement of New Prouct Search and Screening, in 1969 I;u// Con-
~fercnw Prowdings of the Americm Murkrtiq Associcrrior~, (Ed .) by P.
R. McDonald. 168~175 (1969).
12. Soudcr. W. E. A Scoring Methodology for Assessing the Suitability of
Management Science Models, Marza~em~vrr Science, 18, 8526-543
(1972).
1.3. Souder. W. E. (19781, A System for using R&D Project Evaluation
Methods. Rrsmrc~h Mana,qe~net~t. 21, 29-37 (September 19781.
14. Urban, G. L. and J. R. Hauser. Desrgn and Markering ofNew Produc.ts.
Englewood Cliffs. N.J.: Prentice-Hall Inc., Chapters 9 and IO (1980).
156