Expert Systems with Applications 39 (2012) 9868–9885

Contents lists available at SciVerse ScienceDirect

Expert Systems with Applications

journal homepage: www.elsevier.com/locate/eswa

Managing uncertainty to improve decision-making in NPD portfolio

management with a fuzzy expert system
Jeongsu Oh a, Jeongsam Yang b,⇑, Sungjoo Lee b
a
PDEM Development Group, Samsung SDS, Republic of Korea
b
Department of Industrial & Information Systems Engineering, Ajou University, San 5, Wonchun-dong, Yeongtong-gu, Suwon 443-749, Republic of Korea

a r t i c l e i n f o a b s t r a c t

Keywords: The importance of new product development (NPD) for a company’s growth and prosperity is empha-
Portfolio management sized and a number of methods have been suggested to help decision-making for NPD project portfolio
New product development (NPD) management. In spite of their utilities, however, little attention was paid to develop a supporting system
Project selection for portfolio management that can help quick but careful decision-makings under uncertainties. There-
Uncertainty
fore, this research proposes a decision-making framework that uses a fuzzy expert system in portfolio
Fuzzy expert system
management for dealing with the uncertainty of the fuzzy front-end of product development. For the pur-
pose of developing the framework, we adopted the three tools – strategic bucket for strategic resource
allocation, scoring models for evaluating projects and portfolio matrixes for balancing projects – to find
an optimal set of projects in the portfolio. In particular, this research established fuzzy inference-based
models for evaluation criteria which are too ambiguous to be numerically described. Also, based on
the evaluation results, the final selection of projects is made by an expert system, which can encompass
the operational knowledge and company strategy in the rule-based system. The suggested framework
was applied to the portfolio analysis in an electronics firm in Korea and verified its feasibility.
Ó 2012 Elsevier Ltd. All rights reserved.

1. Introduction most feasible NPD candidates for the demands. We therefore need
an effective decision-making process: a process that can accurately
As a management strategy, new product development (NPD) is evaluate numerous NPD projects with limited resources and make
critical for the survival and growth of companies in a rapidly a sound selection of the optimum set of products. Further, the
changing market. Successful NPD can provide increased sales, prof- necessity of such a systematic and judicious decision-making pro-
its, and competitive advantage for most companies: many leading cess is highlighted particularly for NPD projects, which are often
high-tech companies, for example, have found that more than 50% hard to stop once initiated.
of their current sales come from new products (Balbontin, Yazdani, NPD strategies can be realized by implementing an objective
Cooper, & Souder, 2000). Therefore it is apparent that a firm’s NPD decision-making process through successful portfolio manage-
strategy is a primary determinant of performance. Particularly ment, which includes the development of product and technology
these days, consumer markets are becoming more versatile and roadmaps that link business strategy and technology planning.
technology is changing more rapidly than ever. Thus, new products Through portfolio management, companies can make various NPD
need to be adapted to variable and dynamically changing markets. decisions in association with both long-term and short-term strat-
These uncertainties have forced a firm to dedicate much effort to egies; they can also make decisions on strategic investment and re-
NPD, being attentive to the needs of customers and the vicissitudes source allocation to achieve business goals. Building a strategic NPD
of modern technology for maintaining a market share. portfolio while giving due consideration to business goals and con-
However, although companies are continuously striving to de- straints is an important and challenging task. As a result, various
velop new products using much resource, NPD pressure is exacer- methods including multi-criteria decision-making tools (e.g.
bating the risk factor and causing abnormally high failure rates in analytic hierarchy process (AHP)) and optimization techniques
the early stages of development. NPD success depends on the abil- (e.g. linear programming), have been proposed to help evaluate
ity to predict potential demands in the market and to select the the characteristics of NPD projects and build optimal portfolios. In
spite of the meaningful contributions of these methods to portfolio
management, most of the existing methods fail to reflect the uncer-
⇑ Corresponding author. Tel.: +82 31 219 1879; fax: +82 31 219 1610. tainty of portfolio decision-making. Moreover, the potential use of
E-mail addresses: [email protected] (J. Oh), [email protected] (J. Yang), sungjoo these methods as practical management models is limited because
@ajou.ac.kr (S. Lee). they apply the same evaluation criteria to all projects during the

0957-4174/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved.
doi:10.1016/j.eswa.2012.02.164
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9869

decision-making process, even though various selection criteria can within the constraints of resources, project logic and dynamics,
be chosen to match the project characteristics. technology, and project-related strategies. They include a range
To overcome this limitation, we propose a decision-making of methods, such as linear, nonlinear, integer, dynamic, goal, and
method that uses a fuzzy expert system in portfolio management stochastic mathematical programming methods (Heidenberger &
for dealing with the uncertainty of the fuzzy front end of product Stummer, 1999). The mathematical optimization method is the
development. Depending on the market environment, marketing best method from a theoretical perspective and a number of tech-
evaluations in the planning phase of a new product are often con- niques have been suggested to model practical portfolio selection
ducted in an uncertain or ambiguous state, particularly in cases process, considering partial funding, and the interrelation of pro-
involving the expected sales and profits. This paper establishes fuz- jects and their periods (Beaujon, Marin, & McDonald, 2001;
zy inference-based portfolio evaluation models for items which are Dickinson, Thornton, & Graves, 2001; Kester, Hultink, & Lauche,
too ambiguous to be numerically evaluated. With this evaluation 2009). The drawback of these methods is the unreliability of the
model, major NPD projects that should be planned under uncer- results, a problem attributed to the paucity of correct input data
tainty in a firm can be evaluated and prioritized. For this purpose, for calculating the optimized values.
we developed a portfolio expert system which facilitates the selec- The last category involves a strategic management approach.
tion of right projects to develop balanced investment R&D pro- This approach overcomes the limitations of the prioritization ap-
grams and satisfy the goal of portfolio management. The research proach and ensures a balanced portfolio. Examples include a bub-
results are expected to support effective and efficient decision- ble diagram, a portfolio map, and a strategic bucket method
making in association with NPD strategy, especially in environ- (Balbontin et al., 2000; Wang & Hwang, 2007). It also enhances
ments where markets and technology are changing rapidly. the relationships between the NPD projects and strategy. Recent
research suggests that differences between the most innovative
companies and less innovative companies depend on how well
2. Related works
they define and utilize a strategic bucket (Barczak, Griffin, & Kahn,
2009). Therefore, in this paper, we suggest a portfolio selection
A project portfolio is a set of projects conducted under the man-
method that achieves the maximization value proposed by Cooper,
agement of a specific company. Project portfolio selection is the
Edgett, and Kleinschmidt (2004a, 2004b, 2004c), particularly with
periodic activity to select a portfolio from project candidates (both
regard to the selection of balanced projects and strategy alignment.
available project proposals and projects currently underway) to
Specifically, we use a strategic bucket and a portfolio map that sup-
meet the organization’s stated objectives in a desirable manner
plement each other; we also take advantage of the prioritization
within available resources or without violating other constraints
approach by using a scoring model and a financial analysis method.
(Archer & Ghasemzadeh, 1999). Therefore, decision-making needs
The outcome of these efforts is the development of a fuzzy-based
to be based on the features of individual projects as well as on the
expert system.
overall portfolio and strategic goals. Cooper (1994) suggested the
following three drivers for effective portfolio management:
2.2. Methods of decision-making in conditions of uncertainty
 Maximization of value: projects with high profitability and high
chance for success should be selected; Portfolio decision-making deals with a long-term vision based
 Balanced projects: an appropriate balance of projects in terms on uncertainty. Uncertainty means insufficient information
of long-term and short-term objectives, degree of risks, and (Spender, 1993); it may lead to unreliable decision-making. Several
diversity of markets and technologies; recently suggested methods have therefore attempted to reflect
 Strategy alignment: resource allocation strategy and corporate uncertainty in the evaluation and selection of projects.
investment strategy should be reflected in the portfolio strategy. One popular method involves the use of fuzzy logic (Buyukozkan
& Feyzioglu, 2004). The evaluation of NPD projects can be more or
Various methods of analyzing portfolios, such as the AHP, have less ambiguous, particularly with regard to the numerical expres-
been proposed as a means of satisfying the goals of the dynamic sion of evaluation criteria or evaluation results. A number of stud-
and multi-criteria decision-making process. ies have demonstrated the usefulness of fuzzy logic for the
selection of NPD projects. Kuchta (2001) used fuzzy numbers to
show the NPV and resource leveling of each project. Wei and Chang
2.1. Methods of NPD portfolio management
(2011) used a fuzzy multi-criteria approach for the selection of
portfolios. In a similar context, Buyukozkan and Feyzioglu (2004)
The literature features more than a hundred methods of analyz-
and Chen, Lee, and Tong (2006) adopted a fuzzy-AHP method,
ing project portfolios, which can be classified into three main cat-
while Wang and Hwang (2007) used a fuzzy set and option pricing
egories. The first category is a prioritization approach, in which
in their proposed R&D portfolio selection methods. In addition, Lin
expected project outcomes are evaluated and projects are priori-
and Hsieh (2004) proposed a framework with a fuzzy decision-
tized based on them. This category includes comparative methods,
making system for strategic portfolio management. In this paper,
such as a scoring method (Martino, 1995), Q-sort (Souder &
we develop a fuzzy-based expert system for selecting a portfolio
Mandakovic, 1986), and AHP (Brenner, 1994), as well as financial
of NPD projects under uncertainty.
analysis methods, such as the net present value (NPV) method
(Chun, 1994), the return on investment method (Martino, 1995),
and option pricing theory (Perlitz, Peske, & Schrank, 1999). These 3. NPD portfolio framework
methods are simple and useful but are limited in the way they
manage the portfolio balance. According to Cooper, Edgett, and 3.1. Concept of the framework
Kleinschmidt (2004a, 2004b, 2004c), companies that rely solely
on a financial method in NPD portfolio selection tend to have the Portfolio management and project prioritization involve deci-
worst outcomes. Nevertheless, the financial method is still the sion-makings on the assignment of limited resources to the right
most widely used method. project. Since these processes determine the future products and
The second category involves a mathematical optimization ap- markets for a firm, a rapid and sound decision-making system as
proach. These methods try to optimize various objective functions well as sharing the same information across the firm is necessary.
9870 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Fig. 1. Fuzzy inference-based portfolio framework.

Fig. 2. Example of project prioritization with the strategic bucket method.

Such a system should enable to prioritize projects considering the the scoring model and matrix analysis. The final phase, phase 4,
optimal balance of profit versus risk, of short-term versus long- aims to review the evaluation results of the previous phase. Also
term projects, and of growth versus revenue. As a result, it will a project that is excluded from a portfolio according to the evalu-
be possible to guarantee a proper balance among the selected ation results but must be conducted is examined at this stage.
NPD projects and effective use of limited resources. To support a There are some projects that rarely contribute to sales and thus
rapid and reliable decision-making process, we propose a fuzzy are likely to be excluded in the portfolio, but are essential to
inference-based portfolio framework, which consists of the four achieve a strategic goal of a firm. Considering such projects, those
phases as shown in Fig. 1. who are in charge of portfolio management will determine the fi-
In phase 1, marketing managers submit a proposal for NPD nal set of NPD projects using a simulation function of web
based on the concepts and requirements for new products to be applications.
sold next year, where target goals of each NPD project are set. In
phase 2, the portfolio is executed to achieve the three portfolio 3.2. Executing portfolio
management goals as a result of three successive analyses: namely
the strategic bucket, the scoring model, and the matrix analysis. After completing an evaluation report on the target goals and
The projects are then evaluated in batches by the portfolio expert characteristics of the NPD projects, the marketing department
system using the target goals suggested by marketing managers feeds the report data into the portfolio expert system. In the port-
in phase 1. In phase 3, fuzzy inference is used in the overall port- folio execution phase, the NPD projects are prioritized by means of
folio analysis; the inference is based on the evaluation scores of the strategic bucket, the scoring model, and the matrix analysis.
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9871

Table 1
Priority assessment criteria for an NPD project.

Category Criteria
Financial contribution NPV (30%)
(40%) Cost (15%)
Revenue (30%)
Sales (20%)
Quantity (5%)
Strategic importance (15%) Fit with key initiatives and priorities (40%)
Innovation relative to market (30%)
Core competency development (30%)
Commercial potential Base NPV (40%)
(15%) Gross profit margin (1 year) (30%)
User base growth (1 year) (30%)
Technical risk (15%) Proof of concept (40%)
Product, process, and clinical development Fig. 4. A risk-reward matrix.
(30%)
Intellectual property (30%)
Commercial risk (15%) Competitive positioning at launch (40%)
Customer preferences (30%) Table 2
Operational leverage (30%) Standard criteria for the matrix evaluation.

⁄
The number in parenthesis is the assessment weight. GE multifactor matrix Y-axis X-axis
Strategic importance Commercial potential
Risk-reward matrix Y-axis X-axis
Technical risk Commercial potential

assigned one of the bucket types, and each bucket represents a pro-
ject portfolio. The strategic priority is based on the TBB of each
bucket. For example, if the new product bucket has a TBB of $8.7
million, projects A, D, G, and J are selected because their aggregate
value is within the assigned budget. In the case of the maintenance
bucket, the combined value of the NPD projects is less than the TBB
value of $18.5 million. The maintenance bucket needs more NPD
proposals from the relevant departments or else the bucket alloca-
tions need to be adjusted following discussions with the finance
department. In this way, the strategic bucket reflects the strategic
priorities of the company. This method is an effective way of con-
firming the budget allocations and the distribution of resources for
Fig. 3. GE multifactor business portfolio matrix.
the development of new products.
In this paper, we use three types of strategic buckets to evaluate
The prioritization, which is based on goal-setting information pro- the NPD projects in the review stage:
vided by the marketing department, is finalized in the develop-
ment planning stage.  Architecture (high innovativeness projects): acquisition of core
technology and platform development projects (New-to-the
3.2.1. Strategic bucket firm type products or processes, platform or next-generation
The strategic bucket is related to the level of innovation. Finan- products, and breakthrough products development).
cial resources are allocated in accordance with the needs and stra-  New products (medium innovativeness projects): NPD projects
tegic priority of each NPD project. Each candidate NPD project is in general (Major revisions to existing products or processes,
represented by a bucket. The priority of each project is based on and line extensions).
the target-based budget (TBB) of each bucket. An important strate-  Maintenance (low innovativeness projects): maintenance and
gic consideration is how well the development projects can be har- repair projects for technical supports or improvement of prod-
monized and balanced. According to benchmarking research of the uct functions (Incremental improvements to products or pro-
American Productivity and Quality Center, non-innovative projects cesses including changes for cost reduction).
generally make up about 64% of projects whereas innovative pro-
jects make up only 10.2% (Cooper, Edgett, & Kleinschmidt, 3.2.2. Design of the scoring model
2002a). The goals of a company are not limited to new and innova- Various models, including a financial model and a balanced
tive projects. NPD projects that focus on improving the existing score card model, have been designed to concentrate the resources
products to satisfy customer requirements are also necessary; nev- of a company on competitive projects while keeping the portfolio
ertheless, innovative projects are likely to contribute enormously values maximized, balanced and harmonized. In this paper, a
to the overall success of a company. Therefore, the strategic bucket weighted scoring model was adopted to evaluate projects, which
model needs to be organized very carefully and prudently to pre- were prioritized according to evaluation results. Evaluation criteria
serve a suitable balance. in the scoring model, as shown in Table 1, can be based on the pro-
Fig. 2 illustrates a decision-making process based on strategic ject score table proposed by Cooper (2005). However, the criteria
bucket. There are three buckets: one for a new product, another and their weights can be modified depending on business and stra-
for maintenance, and one for cost savings. Each NPD project is tegic goals in a company.
9872 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Fig. 5. Membership functions for fuzzification.

Fig. 6. Example of representing project evaluation rules as IF–THEN.

Fig. 7. Evaluation of fuzzy rules.

Fig. 8. Aggregation of rule consequents.

The financial contribution category consists of five criteria: NPV, evaluation criteria table can be used, which transforms absolute val-
cost, revenue, sales and quantity. It can therefore be used to evaluate ues for financial contributions to a proper range of scores (see Table
the economic value of NPD projects. The scores for these five criteria A.1). The evaluation of the other four categories is conducted by
are determined by the target goals of each project submitted by the managers, particularly with regard to their perceptions and experi-
relevant departments. Here, to assign scores for each criterion, an ences of what makes an NPD project successful. Again, an evaluation
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9873

Fig. 9. Defuzzification results of the centroid technique.

Fig. 10. Fuzzy inference model for portfolio evaluation.

criteria table can be used to assign appropriate scores with increas- 3.2.3. Analysis of the portfolio matrix
ing objectivity (see Table A.2). The evaluation scores of each category Some leading consulting firms in the early 1970s and 1980s
are calculated by weighted sum of relevant criteria and the final proposed the concept of a portfolio matrix to support decision-
evaluation scores are gained by weighted sum of scores for the five making for competitive advantage and resource allocation (Cooper,
categories, which are used to prioritize the candidate NPD projects. Scott, & Kleinschmidt, 2002b). A portfolio matrix is based on
9874 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Fig. 11. Graphs for the membership functions.

consideration of the necessary resources and business strategies attractiveness involves external factors (beyond the control of the
relating to NPD plans. company), whereas business strength involves internal factors
Though various portfolio matrices have been suggested, one of (which can be controlled by the company). Many companies use this
the most popular ones is the GE multifactor portfolio matrix matrix extensively, depending on their standards and characteris-
co-developed by General Electric and McKinsey, which is shown in tics. The external factors are generally related to market profitability,
Fig. 3 (Hax & Majluf, 1983). This portfolio matrix sheds light on the particularly time, cost, quality and functionality. Companies use the
competitive position of a company. The X-axis represents industry portfolio matrix to analyze these factors in relation to their internal
attractiveness, and the Y-axis represents business strength. Industry resources, such as labor, capital and information technology.
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9875

Another commonly used matrix is a risk-reward matrix, which be applied to each appropriate fuzzy set via the membership func-
is a BCG portfolio matrix on a quadrant plane. It helps managers tions. As shown in Fig. 5, the crisp input data, x1 and y1, refer to the
understand whether the project investments are balanced. For specific quantity or numerical values of X and Y, respectively. Once
example, the type of project selection that relies heavily on profit the universe of discourse is determined by an expert’s decision, it is
and loss might lead to a serious loss in the initial phase of a project; matched to numerical evaluation criteria for NPD projects. For
risky projects might rarely be successful but, when successful, the example, a manager is asked to suggest the input values between
profit margin is expected to be high. Senior managers can use the 0 and 100% for investment money and human resources as a part
risk-reward matrix to disperse the risks in a portfolio. In Fig. 4, the of the risk evaluation criteria.
X-axis connects the NPV evaluation criteria, and the Y-axis shows Membership functions A1, A2 and A3 are defined as three lin-
the likelihood of technical success. The pearl quadrant indicates a guistic variables ‘High’, ‘Medium’ and ‘Low’ to easily understand
high probability of market success and a high expectation of big re- investment money and membership functions B1 and B2 are
wards. The oyster quadrant also has a high expectation of big re- defined as ‘Many’ and ‘Small’ respectively. If the crisp input value
wards but a low probability of technical success. The bread and is 30% for x1 (investment money) and 60% for y1 (human
butter quadrant indicates a high chance of technical success but resources), the membership function of the investment money is
a low reward level. Finally, the white elephants quadrant indicates converted to membership function values of 0.5 for A3 (Low) and
a low level in both categories of technical success and rewards. 0.2 for A2 (Medium). In addition, the membership function of
In this paper, we derived indices of the five categories in the human resources is converted to membership function values of
scoring model for each axis of the GE multifactor matrix and the 0.7 for B1 (Many) and 0.17 for B2 (Small).
risk-reward matrix; the categories were described earlier for the
evaluation of the NPD projects. Finally, as a way of using the port- 3.3.2. Step 2: rule evaluation
folio matrix to evaluate NPD projects, we combined the indices to The rule evaluation step takes the fuzzy inputs of l(x = A3) = 0.5
form standard criteria for the matrix evaluation (see Table 2). (or l(x = A2) = 0.2, l(y = B1) = 0.7, l(y = B2) = 0.1) and applies it to
the antecedent ‘x is A3 (or x is A2, y is B1, y is B2)’ of the fuzzy
3.3. Fuzzy inferencing IF–THEN rule. The knowledge in the expert system is represented
by a set of rules for creating an IF–THEN sentence; the set is then
The reliance on human empirical knowledge for NPD decision- compared to the crisp input values. As shown in Fig. 6, for example,
making can undermine objectivity and exactitude in a market the rules for the project investment money and human resources
environment with a high level of uncertainty. We therefore use a (which are evaluation rules for the NPD projects) are defined as
fuzzy inference method to determine a criterion threshold value an IF–THEN rule.
for analyzing portfolios. Accordingly, our evaluation of NPD pro- Corporate business rules that depend on the on-site knowledge
jects is based on human linguistics instead of numerical theory. of experts are addressed through working-level questionnaires.
We found fuzzy logic to be useful because the evaluation criteria These rules can be implemented as basic rules. First, a number rep-
and their results are too ambiguous to be expressed in numerical resenting the evaluation results is acquired by applying an ante-
terms. Fuzzy inference is a max-min composition of facts and rules cedent to the defined rules. As shown in Fig. 7, this number is
based on linguistic variables that map the input fuzzy variables to then applied to the membership function of the consequent, which
the output. Examples of fuzzy inference include the Mamdani is part of the ‘z is C2’ term defined after the application of the
method (Mamdani & Asilian, 1975) and the Sugeno method IF–THEN rule.
(Sugeno, 1985). We use a three-step Mamdani-style fuzzy infer- In this case, the OR fuzzy operation is used to evaluate the dis-
ence method for our evaluation process. junction of the rule antecedents for overlapped area of the mem-
bership functions (see Fig. 7). Fuzzy expert systems typically
3.3.1. Step 1: fuzzification make use of the following classical fuzzy operation union:
Fuzzification determines the input and output variables for each
membership function. It also determines how much value should lA[B ¼ max½lA ðxÞ; lB ðxÞ ð1Þ

Fig. 12. A decision tree for the evaluation of the GE matrix.

9876 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Fig. 13. A decision tree for the evaluation of the risk-reward matrix.

Fig. 14. A decision tree for the portfolio evaluation.

Fig. 15. Architecture of the portfolio expert system.

where lA[B is the union of the output values from membership output number after all the rule consequents obtained by applying
functions of lA(x) and lB(x); lA(x) the output value of the member- the crisp in the input phase of the previous step have been aggre-
ship function of x on A and lB(x) the output value of the member- gated into a fuzzy set. We use a centroid technique, which is one of
ship function of x on B. the most common defuzzification methods as shown in Eq. (2).
Pb
lA ðxÞx
3.3.3. Step 3: defuzzification COG ¼ Px¼a
b
; ð2Þ
x¼a lA ðxÞ
The third and final step of the fuzzy inference process is defuzz-
ification. Fuzziness can help evaluate the rules, but the final output where x is the degree of membership and lA(x) is the output value
of a fuzzy system must be a crisp number. The input for the defuzz- of the membership function of x on A.
ification process is the aggregate output fuzzy set and the output is Let us take a look at a previous example of the NPD project.
a single number. Fig. 8 shows a procedure that determines a single After x1 (Investment money) and y1 (Human resource) are fuzz-
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9877

Fig. 16. Portfolio execution scenario of electronics company A.

Fig. 17. Project lists loaded onto the expert system.

ificated at 30% and 60%, respectively, x1 is fuzzificated at A3 fuzzification process on Z (Project risks) and the membership
(Low) with a value of 0.5 and at A2 (Medium) with a value of function value is then calculated in terms of C2 (0.1) and C1
0.2, while y1 is fuzzificated at B1 (Many) with a value of 0.7 (0.5). Lastly, as shown in Fig. 9, using the centroid technique to
and at B2 (Small) with a value of 0.1. Next, the OR fuzzy opera- calculate the presumed defuzzification, we get a risk factor of
tion is applied to the output results of the rule evaluation. The 75.71%.
9878 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

3.4. A fuzzy inference model for portfolio evaluation variables of all the evaluation criteria are extracted by means of the
fuzzification process and delivered to the relevant managers. This
We formulated an NPD Mamdani-style fuzzy inference model step is followed by the construction of the evaluation rules and
based on the numerical data of each phase evaluated in the portfo- decision trees for the NPD projects. The antecedents and the conse-
lio execution phase as shown in Fig. 10. quents of the IF–THEN rule are connected by inference chains
First, a scoring model is used to calculate scores from zero to 100 for based on a decision-tree. In this study, we used a forward chaining
an NPD project. These scores are then converted into fuzzy linguistic inference, which is a type of data-driven inference.
Ò
variables of ‘High’, ‘Medium’ and ‘Low’ (Fig. 11(a)). Next, a member- Figs. 12 and 13 show a part of decision tree made by XpertRule ,
ship function graph of the Y-axis (Fig. 11(b)) and the X-axis a knowledge-based system (KBS) of XpertRule Software Ltd. The
(Fig. 11(c)) is used to configure the GE matrix for the fuzzification of tree is used to build an inference chain for the evaluation rules of
three input variables. At this time, the membership function of the the GE matrix and the risk-reward matrix. For example, when
Y-axis is converted to the fuzzy linguistic variables of ‘High’, the Y-axis of the GE matrix is ‘High’ and the X-axis is ‘Weak’, the
‘Medium’ and ‘Low’, while the membership function of the X-axis is outcome of the inference chain decision-making process will be
converted to the fuzzy linguistic variables of ‘Strong’, ‘Medium’ and the linguistic variable ‘Selectivity’. This term denotes the probabi-
‘Low’. Then, the matrix evaluation is applied to the crisp input vari- listic numerical information on the linguistic variable based on a
ables for the X-axis and Y-axis of the GE matrix. A business rule table fuzzy membership having a value between zero and 1. The conver-
of a company is used in this process. As shown in Fig. 11(d), the numer- sion process is based on a graph of the evaluation results of the
ical results of the matrix evaluation are converted to fuzzy linguistic membership function of the GE matrix shown in Fig. 11.
variables by means of the membership functions. The fuzzy linguistic The decision tree in Fig. 14 represents the fuzzy inference model
variables are then converted to the following categories: ‘Investment for the evaluation of a portfolio of NPD projects. This decision tree
and growth’; ‘Selective growth’; ‘Selectivity’; and ‘Harvest/Divest’. is used to evaluate each NPD project in terms of five linguistic rat-
As with the GE matrix procedure, the risk-reward matrix con- ings: ‘Very high’, ‘High’, ‘Medium’, ‘Low’, and ‘Very low’. For exam-
verts data into fuzzy linguistic variables through the membership ple, if the scoring model is evaluated as ‘High’ and the evaluation
graph (Fig. 11(e)). The membership functions for the evaluation re- result of the risk-reward matrix is ’Pearl’, the project in the fuzzy
sults of the risk-reward matrix are converted into four fuzzy lin- inference process is expected to have a ‘Very high’ likelihood of
guistic variables: ‘Pearls’, ‘Oysters’, ‘Bread and butter’, and ‘White success. This rule can be expressed as follows:
elephants’. Fig. 11(g) shows the definition of the membership func-
IF evaluation of the scoring model is high
tion for the final results of the fuzzy inference of the NPD project.
AND evaluation of the risk-reward is the pearl category
The evaluation results of the NPD projects can be corrected by
THEN the likelihood of success for the corresponding project is
those responsible for the projects. Experience dictates that when
very high.
a linear graph of the membership function is triangular or trape-
zoid in the neighboring region the overlap of the midpoint to the However, this method of formulating rules for business pro-
edge of the neighboring fuzzy regions averages between 25% and cesses has a weakness. If there are too many business rules, the
50% of its bottom lines. Furthermore, this process enhances the rules may conflict with each other or become unstable. Accord-
accuracy of the fuzzy inference. Table A.3 shows the numerical ingly, it is necessary to improve the reliability of the evaluation
range of the linguistic variables of the membership function. system by continually adjusting the portfolio fuzzy inference mod-
The business rules of the company must then be formulated for el by means of the XpertRule KBS after the portfolio expert system
the decision-making process. For this purpose, the fuzzy linguistic has been implemented.

Table 3
Result of project’s priority orders after portfolio execution.

Project Score of the scoring model Score of the GE matrix Score of the risk-reward matrix Total score of portfolio
evaluation
Linguistic Score Linguistic Score Linguistic Score
1 UC6U M 56 SG 77.1 Pearls 89.17 79.16
2 UC6V M 50 SG 65 Bread and butter 59.17 64.68
3 POUSSIN M 63 SG 65 Bread and butter 49.44 64.56
4 PC5G M 55 SG 61.67 Bread and butter 45.66 61.09
5 MCKINLEY H 67 S 45.61 Pearls 54.02 59.29
6 UC7W L 30 IG 86.77 Pearls 70.14 58.42
7 LB3F M 37 S 35 Bread and butter 56.1 55
8 HAYDN M 66 S 41.43 Oysters 54.52 54.53
9 COBALT M 50 S 43.18 Pearls 60 54
10 UC4Q M 47 SG 54.08 Oysters 61.56 53.59
11 30-MFM M 48 S 36.28 Bread and butter 62.5 52.44
12 LB6T M 64 S 25.38 Bread and butter 53.98 51.17
13 LC6M M 51 SG 61.92 White elephants 30.17 47.95
14 TULIP M 57 S 32.95 White elephants 45.92 46.55
15 LC6P M 60 HD 10 Bread and butter 52.72 45
16 LC5F M 58 S 42.5 White elephants 28.55 41.43
17 LB5K L 29 S 35.36 Bread and butter 52.82 41.11
18 MATISSE M 51 S 35 Oysters 44.93 38.75
19 LC4E M 57 HD 19.43 Bread and butter 44.81 33.62
20 CARNELIAN L 33 S 35 White elephants 23.5 30
21 PLUM M 55 HD 17.35 Bread and butter 35 22.94
22 UC5R L 25 HD 12.74 Bread and butter 52.5 21.1
23 COBALT-SMPLE M 46 HD 10 White elephants 18.5 20

Scoring model: H (High), M (Medium), L (Low).

GE matrix: IG (Investment and growth), SG (Selective growth), S (Selectivity), HD (Harvest/Divest).
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9879

Fig. 18. Analysis result of the portfolio evaluation.

3.5. Portfolio review 4. System development and experiment

Our evaluation model relies on fuzzy logic. We applied it to a 4.1. Overview of the system
portfolio, focusing on various aspects such as the financial re-
sources and strategic alignment. The model is useful to apply but The portfolio expert system was designed to run in conjunction
there are sometimes projects that should be conducted regardless with the three layers shown in Fig. 15: the presentation layer, the
of evaluation results. For example, a firm may need to start project business layer and the data layer. The presentation layer, which was
Ò
because it is strategically important to cope with changes in the developed with Visual C# and ASP.NET of Microsoft , provides a user
market place, though its short-term financial performance is low interface that enables a user to evaluate a portfolio of NPD projects in a
and relevant technical capability in a firm is weak. Another exam- Web environment. The business layer can interface with a fuzzy inter-
ple includes a case where there is a necessity to respond urgently face engine of the XpertRule KBS and the presentation layer by means
to the competitive products of other companies. Due to these of the asynchronous JavaScript and an XML (AJAX) module. It is also
market uncertainties, flexible approach will be helpful. Here, if designed to output the evaluation results onto a Web page. Therefore,
an NPD project needs to be added or deleted from a portfolio, the the business layer uses the inference-based rule model to analyze the
review function can cope with the necessary changes. That is, the strategic bucket, the scoring model and the matrix for the purpose of
reviewing function continually manages the portfolio by monitor- evaluating NPD projects. It then feeds the results into the presentation
ing the overall portfolio plans. This process ensures the projects are layer. The data layer is a service module. It systematically facilitates
balanced in the event of an urgent NPD project, particularly with the management of the system operation data and analyzes and then
regard to the resource assignment and strategic bucket. stores the output data in an internal ODBC database.
9880 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Fig. 19. Sample of the portfolio dashboard.

4.2. Experiments on the portfolio application 4.2.2. Execution of a portfolio

Several NPD projects proposed by the marketing department
To test the system in a portfolio analysis and to verify the feasi- are loaded into the portfolio expert system for a particular portfolio
bility of the portfolio, we used sample data from 88 NPD projects of analysis scenario. The project lists, as shown in Fig. 17, are used to
Korean electronics company A (see Table A.4). conduct frequent portfolio analyses until a decision maker con-
firms final projects in September. In this case, the history of every
4.2.1. Portfolio analysis scenario update of the NPD projects proposed by the marketing department
Fig. 16 shows portfolio analysis scenarios of company A. The needs to be managed carefully. Using the versions feature of the
marketing department usually launches several NPD projects in portfolio expert system enables to trace the changes whenever
June, so we used our portfolio expert system to evaluate projects they are made during the portfolio analysis. As shown in Fig. 17,
in the proposed project list. The results go through the portfolio re- a marketing manager registers a statement titled ‘‘2011. 1Q Pro-
view phase via the portfolio dashboard, and the goals of the NPD jects’’ as a version of information into the system; this step is fol-
projects are adjusted in light of the review results and the relevant lowed by the next two phases, the execution phase and the
discussion between the development department and marketing portfolio reviewing phase.
department. The marketing department also gets feedback on the
discussion. Lastly, the final portfolio is confirmed by management 4.2.3. Review of a portfolio
in September following the repetition of the proposal, execution, The portfolio execution was used to narrow down the 88 NPD
and review phases. projects proposed by the marketing department to just 23 projects
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9881

(Table A.4). First, in the portfolio execution phase mentioned in the ments. It is also flexible. Suitable decision-making rules can be
previous section, we made the following selections within the tar- added or deleted. In addition, its ability to visualize a portfolio
get budget: eight projects of the ‘Architecture’ type in the strategic map enhances the convenience of users.
bucket, four projects of the ‘New product’ type, and 11 projects of
the ‘Maintenance’ type. For the prioritization of the NPD projects,
we applied the evaluation of the scoring model and the portfolio 5. Conclusions
matrix analysis to the 23 projects. Table 3 shows the prioritization
results from the portfolio execution. When new markets are appearing, the product life cycle is get-
It is interesting to compare LB3F (ranked 7th) with HAYDN ting shorter and the costs of innovation are getting higher, compa-
(ranked 8th). Although the scoring model produced the scores of nies have to keep introduce new products to the market and
66 for the HAYDN project, which is higher than those of the LB3F strategic new product planning is essential to success. Accordingly,
project, HAYDN was ranked 8th. When the evaluation results of most companies analyze their project portfolio each year for effi-
the GE matrix were considered, the HAYDN project scored 41.43, cient resource allocations and alignment the projects with corpo-
which was also higher than those of the LB3F project. However, rate strategies. Prudent analysis is required because portfolio
the LB3F project was in the ‘Bread and butter’ quadrant while the decision-making significantly affects not only annual sales and
HAYDN project was in the ‘Oyster’ quadrant. Because the business profits but also long-term growth of the company. To help make
rules applied to the portfolio expert system in company A were de- NPD decision-makings, this research suggests an expert system
signed to give a higher score to projects with high technical feasi- for portfolio analysis based on the work by Cooper (1994). The sug-
bility but low rewards than projects with low technical feasibility gested system uses fuzzy inferencing to analyze the matrices of the
but high rewards. Of course, these rules in the expert system can scoring model and manages portfolios to ensure that the projects
be tailored to the characteristics of the company strategy through are well balanced in terms of periods, risks and profits. The system
consultation with top-level managers. framework is flexible and extendable. It easily applies the portfolio
The 23rd ranked COBALT-SMPLE project achieved results of evaluation rules to comply with the corporate strategies and
‘Harvest/divest’ (10) and ‘White elephants’ (18.5) from the scores directions in changing markets and can be accessed on Web
of the matrix evaluation (excluding the scoring model). Deemed applications.
a very high risk, this project is expected to be regarded in the eval- In spite of these meaningful contributions, this research is sub-
uation phase of the strategic bucket as a high priority in terms of ject to a few limitations and further research is needed. First, the
the target budget of NPD projects. However, this assessment might suggested expert system-based approach focuses more on evalua-
be erroneous when the head of the marketing department inputs tion process than evaluation criteria and decision-making rules.
the target goals for each project. To prevent human errors, our sys- The system needs to be improved by elaborating a scoring method
tem has a dashboard-style window that enables users to analyze and decision-making rules. Second, we use the same if-then rule
the portfolio (Fig. 18). After the relevant project is excluded from regardless of the project types. However, because of the variety
the project master list, other projects can be added. The system of project types in a strategic bucket, we need to apply different
also intuitively provides useful information on the status quo of selection criteria for each type of project. Third, we focus on two
NPD projects. The bar charts in Fig. 18 provide information on common types of portfolios. In the future, we need to design a
new products (for next year’s market) that have failed to achieve portfolio that reflects more versatile standards for balancing all
the targeted NPV. These results can be adjusted following discus- the aspects of a project. Last, further research is needed on the
sions with relevant departments. Moreover, as shown in Fig. 19, prioritizing of projects, particularly with regard to old and new
the portfolio dashboard helps users preview NPD projects selected models based on the research results of our portfolio expert
through the portfolio evaluation, particularly with regard to the system.
market performance of past products.
Acknowledgement
4.3. Discussion
This work was supported by National Research Foundation of
Our fuzzy-based portfolio expert system has three main
Korea Grant funded by the Korean Government (2009-0074748).
advantages over other portfolio selection methods. First, it can
reflect a variety of decision-making criteria in times of need.
The system can add meaningful projects to a portfolio by assign- Appendix A
ing IF-THEN rules based on the characteristics of past successful
projects and the heuristic experience of project managers. This
capability of flexibility is expected to overcome the limitations
of conventional evaluation methods, which prioritize projects
simply by aggregating the results of the evaluation criteria. Table A.1
Evaluation basis table for five indices in financial contribution.
Our system combines a strategic bucket and portfolio analysis.
The NPD projects for the next year can be considered after users Score NPV Cost Revenuea Salesa Quantityb
have defined the ratio of project types to be included in the final 1 1 300 5 10 1
portfolio and developed suitable decision-making rules for those 2 1–10 200–300 5–10 10–50 1–2
types of projects. 3 10–20 100–200 10–30 50–100 2–3
4 20–30 50–100 30–50 100–200 3–4
Second, our portfolio expert system is effective in conditions of
5 30–40 40–50 50–70 200–300 4–5
uncertainty and can therefore provide useful support for fuzzy 6 40–50 30–40 70–90 300–400 5–10
front-end decision-making in the early stage of NPD projects. 7 50–60 20–30 90–110 400–500 10–20
Third, it can be implemented as a computer-based system and 8 60–70 10–20 110–120 500–700 20–30
therefore supports a dynamic decision-making process in portfolio 9 70–80 5–10 120–140 700–900 30–40
10 90 5 150 900 40
management. Many projects are created and then discarded con-
a
tinuously for short-period of time these days. The proposed system Cost/revenue/sales: one million.
b
supports quick objective evaluations in rapidly changing environ- Quantity: 10 million.
 9882
Table A.2
Evaluation basis table for the other categories: strategic importance, commercial potential, technical risk, and commercial risk.

Category Index 0 1–3 4–7 8–10

Strategic importance A1. Fit with key No clear fit Weak fit Consistent with strategic direction Strong fit and addresses targeted
initiatives and growth areas
priorities
A2. Innovation No significant advance Incremental Substantial Transformational
relative to market

J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

A3. Core competency No contribution to development of Adds to development of strategically Significant contribution to development of Lead program for developing
development strategically identified core competency for identified core competency for future strategically identified core competency for strategically identified core competency
future future for future
Commercial potential B1. Base NPV <$10 million $10–100 million $100–200 million >$200 million
B2. Gross profit <60% 60–75% 75–85% >85%
margin (3rd year
post-launch)
B3. User base growth No growth in user base Small increase (>0–2%) in user base or Moderate increase (>2–5%) in user base Substantial increase (>5%) in user base
(3rd year post- may halt decline in user base
launch)
Technical risk C1. Proof of concept High risk; concept not yet proven, and path Feasibility not yet demonstrated, but Feasibility not fully demonstrated, but there are Feasibility is fully demonstrated
forward not clear path forward is clear no anticipated challenges
C2. Product, process, High risk; new technology, new skill sets, Moderately high risk; multiple Controllable risk factors; access to required Existing product/process technologies
and clinical and uncertain clinical path uncertainties talent and clear path to achieving targeted and certain clinical outcome
development clinical outcome
C3. Intellectual IP presents substantial obstacle; difficult to Complicated IP field, but may be Full freedom to operate within known IP domain Dominant IP position and ability to
property license/acquire possible to establish footing or and secure desired position block competitors
license/acquire
Commercial risk D1. Competitive Weak Parity with competition or minor Moderate performance, feature, and/or cost Very strong positioning that will result
positioning at launch advantage advantage that could significantly affect buying in clear market leadership
behavior
D2. Customer Customer needs and product performance Matches some know customer needs; Matches most customer needs; product Product performance matches known
preferences not know product performance not well know performance mostly know customer needs (may be an existing
category)
D3. Operational Requires building new competencies and Some operational leverage, but Strong operational leverage; modest business Full leverage; no new processes or
leverage infrastructure(e.g., manufacturing customer requires significant new processes process and skill set impact competencies required
service) and skill sets
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9883

Table A.3
Numerical range for linguistic variables of the membership function.

Linguistic value Symbol Numerical range

Linguistic variables: scores of the scoring model
High H [60, 100]
Medium M [25, 75]
Low L [0, 40]
Linguistic variables: scores on Y-axis of the GE matrix
High H [60, 100]
Medium M [25, 75]
Low L [0, 40]
Linguistic variables: scores on X-axis of the GE matrix
Strong H [60, 100]
Medium M [25, 75]
Weak L [0, 40]
Linguistic variables: analysis ratings of the GE matrix
Investment and growth VH [70, 100]
Selective growth H [40, 90]
Selectivity M [10, 60]
Harvest/divest L [0, 30]
Linguistic variables: scores on Y-axis of the risk-reward matrix
High H [20, 100]
Low L [0, 80]
Linguistic variables: scores on X-axis of the risk-reward matrix
High H [20, 100]
Low L [0, 80]
Linguistic variables: analysis ratings of the risk-reward matrix
Peals VH [70, 100]
Oysters H [45, 85]
Bread and butter M [20, 60]
White elephants L [0, 35]
Linguistic variables: analysis ratings of NPD project evaluation
Very high VH [60, 100]
High H [55, 85]
Medium M [30, 70]
Low L [15, 45]
Very low VL [0, 35]

Table A.4
A NPD project list of company A on next year for evaluation of the portfolio expert system.

Project NPV Costa Revenuea Salesa Quantityb SIc CPc TRc CRc
Mandarin50 0.08 0.4492 0.2246 0.3346 0.12 68 18 83 51
HIGHBIRD 96.4 13.656 6.828 27.1071 1.36 53 57 51 72
NOVA 1.49 1.802 0.901 1.3854 0.1435 89 84 71 46
SPINO 52.9 3.24 1.62 5.7024 1.135 52 56 65 54
BORDEAUX 4.47 4.1168 2.0584 2.9912 0.1016 76 67 41 42
MANASLU2 65 7.452 3.726 18.6672 1.88 65 47 39 8
Negotiator 0.6 21.364 10.682 38.4765 1.176 25 63 83 73
DONGSARY 3 6.616 3.308 12.0742 1.2 40 30 98 53
CARNELIAN 1.8 63.936 31.968 130.8411 1.4214 84 7 45 44
BIZET 7 1.9328 0.9664 1.5675 0.186 64 70 50 70
MONDRIAN 25.7 38.24 19.12 238.6914 5.6169 59 54 44 76
LC6P 60.5 111.244 55.622 405.5296 3.0204 18 47 63 56
NEXINO 9 10.416 5.208 94.9418 8.49 50 35 69 60
EVOLUTION 97.8 9.616 4.808 19.28 1.38 63 74 42 52
LAPFIT 4.96 0.8392 0.4196 0.6353 0.132 17 83 62 60
RUBY 7 5.088 2.544 3.7598 0.1139 50 39 30 34
NEXUS 74.7 19.064 9.532 66.3789 2.8569 52 78 62 64
Goay II 23 38.276 19.138 129.1049 2.748 47 34 37 75
LB5K 7 12.5348 6.2674 13.8728 0.4817 67 41 78 44
GOYA 0.98 1.8076 0.9038 1.3511 0.1225 51 85 26 78
CREAM 5.4 4.804 2.402 33.4026 6.3281 70 90 46 67
WARHOL 5.36 1.2436 0.6218 1.8657 0.8753 75 56 37 47
Kaiser Combo 26.5 12.66 6.33 49.6651 3.298 35 34 16 54
T 32.5 35.56 17.78 149.8142 3.588 47 45 35 46
LB6A 4.98 7.5604 3.7802 6.4182 0.2239 42 82 79 70
CREAM-MFM 85.7 22.124 11.062 61.9029 2.173 64 60 64 56
HAYDN 66.5 92.252 46.126 1264.1752 13.0785 51 68 34 70
AQUA 5.92 4.0904 2.0452 4.375 0.4446 42 48 37 50
MAXINO 7 2.572 1.286 5.378 1.466 64 50 19 102
ELVIS 19 7.568 3.784 17.3534 1.668 93 44 87 67

(continued on next page)

9884 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885

Table A.4 (continued)

Project NPV Costa Revenuea Salesa Quantityb SIc CPc TRc CRc
Khan2 84.9 4.128 2.064 9.8163 1.753 54 51 20 96
CB1J 15 1.3256 0.6628 1.1698 0.2575 36 82 63 14
GREEN 7.2 10.12 5.06 16.6635 1.0216 89 81 27 93
Gold Rush Project 87.9 20.828 10.414 60.7969 2.294 60 52 41 86
BRINO 0.26 0.648 0.324 0.6729 0.4134 87 56 73 56
LEOHIGHLIGHT 46.6 7.06 3.53 13.6611 1.31 22 53 40 26
UC7W 8.08 20.6084 10.3042 15.6472 0.1342 78 84 63 51
UC5R 1.49 8.5864 4.2932 7.581 0.2579 27 52 51 54
LC4H 93.6 30.764 15.382 65.521 1.5047 48 56 63 67
LIBRA 83.9 16.796 8.398 92.9154 4.907 52 27 40 52
CB5K 61.3 6.88 3.44 12.556 1.2 52 43 37 79
JADE 7.2 31.324 15.662 74.4391 1.7514 72 66 84 46
TULIP 96.2 83.296 41.648 186.7246 1.6167 39 48 43 24
MENDEL 12.2 42.456 21.228 211.6091 4.3592 77 59 99 53
LB4C 49.1 10.38 5.19 76.973 6.7905 28 27 43 64
CORONA 7.2 8.812 4.406 23.7831 2.0739 55 63 93 35
MIDAS 12.8 8.452 4.226 36.2168 3.66 23 68 95 88
AURORA 51.2 5.12 2.56 13.998 2.15 52 81 50 52
1C04 99.1 5.4 2.7 10.935 1.4 48 32 78 40
LB3F 16.7 32.728 16.364 121.933 3.1006 48 57 72 67
T-MFM 35.4 21.372 10.686 48.7067 1.654 56 44 41 55
PEBBBLE 8.84 2.5036 1.2518 2.4924 0.3705 82 91 19 67
LC6M 80.9 55.472 27.736 131.523 1.7459 74 46 19 7
BDP-C5500 5.06 1.8628 0.9314 1.7725 0.3265 52 43 80 68
LC5F 62.7 90.608 45.304 414.2199 3.9465 92 35 32 70
VATICAN 70.6 21.804 10.902 39.7923 1.2 26 59 69 41
SCORPIUS 20 6.528 3.264 14.9817 1.67 46 30 78 64
PLUM 54.4 57.504 28.752 532.0385 8.6272 66 10 70 19
LB7U 80.6 45.792 22.896 75.4887 1.0235 98 60 64 47
COBALT-SMPLE 0.8 100.416 50.208 1840.4164 17.7029 47 7 38 60
POUSSIN 99.8 24.176 12.088 264.5229 10.3165 88 47 53 62
LB6T 51.8 141.316 70.658 748.7705 4.6735 35 51 70 95
PYROPE 63.5 77.66 38.83 163.2021 1.4764 59 51 37 71
ZEUS 48.1 21.268 10.634 143.4526 6.12 91 32 59 66
BLADE 28.9 11.736 5.868 29.3047 1.872 86 23 58 32
CB5H 83.5 5.24 2.62 10.2546 1.332 66 83 84 67
ECOFIT-MFM 89.2 26.02 13.01 64.8158 1.866 43 29 49 47
LAVENDER 74.4 24.912 12.456 102.1641 3.476 56 8 31 75
ANDES 3 2.1992 1.0996 1.9462 0.26 25 74 24 59
LIME 89.4 22.464 11.232 55.486 1.845 91 83 70 64
MCKINLEY 19.2 293.016 146.508 11818.44874 39.7088 67 60 58 48
BX50 10.8 51.736 25.868 280.616 4.799 29 24 71 86
MATISSE 42.3 69.32 34.66 387.984 4.972 44 61 22 37
K2 70.8 26.724 13.362 80.0678 2.3711 68 35 19 64
CC1D 85.6 5.36 2.68 13.0516 1.81 44 32 25 37
PC5G 94 36.736 18.368 61.22 1.0414 76 47 48 67
30-MFM 58.7 24.272 12.136 70.7122 2.2883 61 50 83 52
UC4Q 58.3 40.38 20.19 103.9251 1.9486 61 80 36 58
KINGCOBRA 9.8 1.7596 0.8798 1.3109 0.12 72 31 24 68
BDP-C6500 0.5 1.0376 0.5188 0.8134 0.159 4 27 71 42
ET 1.1 2.7276 1.3638 2.1642 0.1684 34 58 55 71
LC4E 63.4 66.56 33.28 341.7244 4.509 36 31 76 71
Cetus 1 45 22.776 11.388 136.7698 5.38 49 51 69 72
Leo 9 3.1208 1.5604 4.766 0.9022 58 20 45 79
UC6V 10.8 160.816 80.408 669.3023 3.5369 76 47 74 44
COBALT 40.7 46.652 23.326 315.9627 6.1477 44 75 59 52
TRINO Project 23.5 7.612 3.806 29.4584 3.245 27 80 56 73
UC6U 86.2 85.236 42.618 215.0359 1.8978 69 82 78 70
a
Cost/revenue/sales: one million.
b
Quantity: 10 million.
c
SI (strategic importance), CP (commercial potential), TR (technical risk), and CR (commercial risk): scores between zero and 100.

References Brenner, M. S. (1994). Practical R&D project prioritization. Research & Technology
Management, 37(5), 38–42.
Buyukozkan, G., & Feyzioglu, O. (2004). A fuzzy-logic-based decision-making
Archer, N. P., & Ghasemzadeh, F. (1999). An integrated framework for project
approach for new product development. International Journal of Production
portfolio selection. International Journal of Project Management, 17(4), 207–216.
Economics, 90, 27–45.
Balbontin, A., Yazdani, B. B., Cooper, R., & Souder, W. E. (2000). New product
Chen, H. H., Lee, A. H. I., & Tong, Y. (2006). Analysis of new product mix selection at
development practices in American and British firms. Technovation, 20,
TFT–LCD technological conglomerate network under uncertainty. Technovation,
257–274.
26, 1210–1221.
Barczak, G., Griffin, A., & Kahn, K. B. (2009). Perspective: trends and drivers of
Chun, Y. H. (1994). Sequential decisions under uncertainty in the R&D project
success in NPD practices: results of the 2003 PDMA best practices study. Journal
selection problem. IEEE Transactions on Engineering Management, 40, 404–413.
of Product Innovation Management, 26(1), 3–23.
Cooper, R. G. (1994). Perspective: Third generation new product processes. Journal
Beaujon, G. J., Marin, S. P., & McDonald, G. C. (2001). Balancing and optimizing a
of Product Innovation Management, 11, 3–14.
portfolio of R&D projects. Naval Research Logistics, 48(1), 18–40.
 J. Oh et al. / Expert Systems with Applications 39 (2012) 9868–9885 9885

Cooper, R. G. (2005). Product leadership: Pathways to profitable innovation. New York, Kester, L., Hultink, E. J., & Lauche, K. (2009). Portfolio decision-making genres: A
NY, USA: Perseus Publishing. case study. Journal of Engineering and Technology Management, 26, 327–341.
Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2002a). New product development Kuchta, D. (2001). A fuzzy mode for R&D project selection with benefit, outcome
best practices study: What distinguishes the top performers. Houston, USA: and resource. The Engineering Economist, 46(3), 164–180.
American Productivity & Quality Center. Lin, C., & Hsieh, P. (2004). A fuzzy decision support system for strategic portfolio
Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2004a). Benchmarking best NPD management. Decision Support Systems, 38, 383–398.
practices-I. Research-Technology Management, 47(1), 31–43. Mamdani, E. H., & Asilian, S. (1975). An experiment in linguistic synthesis with a
Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2004b). Benchmarking best NPD fuzzy logic controller. International Journal of Man–Machine Studies, 7(1), 1–13.
practices-II. Research-Technology Management, 47(3), 50–59. Martino, J. P. (1995). R&D project selection. New York, NY, USA: Wiley.
Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2004c). Benchmarking best NPD Perlitz, M., Peske, T., & Schrank, R. (1999). Real options valuation: new frontier in
practices-III. Research-Technology Management, 47(6), 43–55. R&D project evaluation. R&D Management, 29(3), 255–269.
Cooper, R. G., Scott, J. E., & Kleinschmidt, E. J. (2002b). Optimizing the stage–gate Souder, W. E., & Mandakovic, T. (1986). R&D project selection models. Research
process: what best-practice companies do II. Research-Technology Management, Management, 29(4), 36–42.
45(6), 43–49. Spender, J. (1993). Some frontier activities around strategy theorizing. Journal of
Dickinson, M. W., Thornton, A. C., & Graves, S. (2001). Technology portfolio Management Studies, 30(1), 11–30.
management: Optimizing interdependent projects over multiple time periods. Sugeno, M. (1985). Industrial applications of fuzzy control. New York, NY, USA:
IEEE Transactions on Engineering Management, 48(4), 518–527. Elsevier Science Inc..
Hax, A. C., & Majluf, N. S. (1983). The use of the industry attractiveness–business Wang, J., & Hwang, W. L. (2007). A fuzzy set approach for R&D portfolio selection
strength matrix in strategic planning. Interfaces, 13(2), 54–71. using a real options valuation model. Omega, 35(3), 247–257.
Heidenberger, K., & Stummer, C. (1999). Research and development project Wei, C., & Chang, H. (2011). A new approach for selecting portfolio of new product
selection and resource allocation – a review of quantitative modelling development projects. Expert Systems with Applications, 38, 429–434.
approaches. International Journal of Management Review, 1, 197–224.