EmbeddedMarket

forecasters

The Economics of Embedded

Development, Testing, Deployment and
Support
Helping Senior Management Make Better Investment Decisions
And Gain Financial Control of the Product Development and Deployment
Cycle

Jerry Krasner, Ph.D., MBA

February 2020

Embedded Market Forecasters

www.embeddedforecast.com
American Technology International, Inc.
About EMF: www.embeddedforecast.com 508-740-3673

EMF is the premier market intelligence and advisory firm in the embedded technology industry.
Embedded technology refers to the ubiquitous class of products which use some type of processor as a
controller. These products include guided missiles, radars, and avionics as well as robots, automobiles,
telecom gear, and medical electronics.

Embedded Market Forecasters (EMF) is the market research division of American Technology
International, Inc. EMF clients range from startups to Global 100 companies worldwide. Founded by
Dr. Jerry Krasner, a recognized authority on electronics markets, product development and channel
distribution, EMF is headquartered in Framingham, Mass.

About the author:

Jerry Krasner, Ph.D., MBA is Vice President of Embedded Market Forecasters and its parent company,
American Technology International. A recognized authority with over 30 years of embedded industry
experience, Dr. Krasner was formerly Chairman of Biomedical Engineering at Boston University, and
Chairman of Electrical and Computer Engineering at Wentworth Institute of Technology and Bunker Hill
Community College. In addition to his academic appointments, Dr. Krasner served as President of
Biocybernetics, Inc. and CLINCO, Inc., Executive Vice President of Plasmedics, Inc. and Clinical
Development Corporation, and Director of Medical Sciences for the Carnegie-Mellon Institute of Research.
Earlier, he was Senior Engineer at the MIT Instrumentation Laboratory. Dr. Krasner earned BSEE and
MSEE degrees from Washington University, a Ph.D. in Medical Physiology / Biophysics from Boston
University and an MBA from Nichols College. He is a visiting professor at the Universidad de Las Palmas
(Spain) where he is recognized for his work in neurosciences and computer technology.

Copyright 2020 by Embedded Market Forecasters, a division of American Technology International, Inc
218 Villager Rd, Chester NH 03036. All rights reserved. No part of this document covered by copyright
hereon may be reproduced or copied without expressed permission. Every effort has been made to
provide accurate data. To the best of the editor’s knowledge, data is reliable and complete, but no
warranty is made for this
Executive Overview

Economic considerations of embedded developments have been largely absent by

corporate senior managers, CEOs and CFOs. They are certainly not alone. The design
process is not particularly transparent to those who monitor costs and to those who
make upfront decisions committing the development to one or another tool sets and
design processes.

In the majority of cases, the investment in the tool sets used is the dominant factor. The
down-side risk considerations are seldom integrated into the decision process.

So, in order to gain a more accurate financial perspective, senior managers must take
into consideration the following direct costs, associative costs and extrinsic cost
considerations:

 On time shipment of product – saves on engineering costs

 Maintaining the expected performance, systems functionality and features and
schedule of the development
 Controlling development expenses
 Number of software and systems engineers on project
 Cost of hardware and testing engineers
 Cost of being behind schedule
 Cost of cancellations
 Choice of components – using appropriate OSes and interfaces
 Choice of tools
 Choice of testing methodologies
 Achieving market windows of opportunity
 Cost of in-field support

In addition, the financial officer must also take into consideration additional “risk factors”,
including:

 Cost of maintenance (depends on the complexity of the design and the

management of the design process)
 Cost of upgrading software to meet changes in underlying hardware (requires
upgrading legacy software)
 Cost of recalls (cost includes actual cost of retrieving products, lost revenue, cost
of reimbursement and the loss of reputation)
 Cost of providing code reuse (can save enormously on cost of development for
system/product upgrades)
 Cost of removing product features (failure to adequately meet pre-design
expectations)
 Cost of being late to market (depending on application, can result in the loss of
30%-90% of potential market – example would be consumer electronic products)

These risk-related costs can reach orders-of-magnitude greater than the upfront cost of
development tools and the direct cost of development.

Bringing these considerations back into the broader embedded world, this paper is
aimed at helping senior management address its product return on investment in a
manner that can be effectively applied to their everyday management practices.

3
In this report, EMF presents a methodology for calculating the ROI for a development
based on the direct cost of development, associative costs, extrinsic costs and a process
for estimating risk. This report includes a telecom infrastructure development example
comparing the ROI for developments using Model Based Development (MBD) and those
not using MBD.

The results show the benefits of MBD throughout the development cycle. MBD enables
the importation of legacy software for upgrades and the ability to reuse developed
software when the underlying hardware has changed, or component availability has
made further production of a product or continued support of a product in the field
impossible or impractical.

Based on these real-world data, EMF strongly suggests that all ROI calculations
(including tool acquisition) be made with design outcome and go-to-market factors taken
into consideration.

4
l. Introduction

This report is concerned with whether the use of Model Based Development (MBD)
tools for embedded designs creates a significant savings over traditional design
implementations as measured by an ROI calculation. A real world example is
presented to illustrate the ROI calculation.

This paper will illustrate that tool acquisition costs are a small fraction of the total costs a
company will experience with development costs. In addition, we suggest that company
executives weigh the high risk-related costs of product recall and of missing limited
windows of opportunity. We will show that the savings one achieves is predicated on the
use of full MBD tools.

Note that EMF excludes from this discussion UML Model-Based Drawing Tools,
which are inexpensive tools that lack the capabilities of full MBD tools. There have
been recent trends for more frugal development managers to purchase hundred dollar
UML based drawing tools in lieu of full MBD tools that can cost into the thousands of
dollars. Many people assume they can get the same benefits for a fraction of the price.
This assumption is incorrect for the following key reasons:

o No early validation because you cannot simulate the design

o Little reduction in programming errors because you have to write the
majority of the code manually and the code is at best loosely tied to the
model
o No reduction in testing time because you have to write all the tests
manually rather than generating them from the model
o Weak requirements traceability because it only extends to the model but
not to the code and to the tests which validate the requirements
o No support (or very limited support) for industry specific solutions such as
Autosar, DoDAF, Net Centric Operations, Telecom Handsets, Medical
FDA certification support, etc
o No linkage with embedded operating systems so it is not possible to
validate the design on the target

EMF data shows significant significantly higher ROI when developing embedded
software products using MBD tools, in comparison to both model-based drawing tools
and development approaches that rely on manual development.

5
II. Managing ROI for Embedded Developments

Year-over-year, extensive EMF surveys of embedded developers clearly show that

embedded designs are becoming more complex while time-to-market requirements are
shrinking the time available to both complete designs on-time and meet pre-design
expectations. Financial control considerations are frequently at odds with the needs of
developers to get a product to market. The savings in development costs, and the ability
to amortize the cost of a new methodology over many years of development use, are not
necessarily intuitive. The purpose of this paper is to address the financial considerations
of senior management to alternative strategies by removing “tekkie speak” from the
analysis.

There is a happy middle ground that provides developers the ability to effectively
manage complex design processes, bring products to market on-time and with full
feature packages, and give financial oversight to the process. This technology is known
as Model Based Development (MBD). The emphasis of this paper is to provide
measurable development data to illustrate the advantages that MBD brings to a
company.

Far removed from the developer’s/manager’s selection of tools and design process, an
organization needs to manage its return on its investment (ROI). For the larger
percentage of embedded developments, the investment is usually considered to be the
“up front costs” that would include the cost of licenses purchased, and possibly a royalty
on shipped product. Most military acquisitions are predicated on “getting the best deal up
front”.

EMF data has clearly shown that the total cost of development can be orders of
magnitude greater than acquisition costs as seen in the following analysis:

 On time shipment of product – MBD developments ship faster than comparable

developments not using MBD
 Maintaining the expected performance, systems functionality and features and
schedule of the development – final design results are closer to pre-design
expectations for MBD developments
 Achieving market windows of opportunity – MBD developments not only get to
market faster, but they can be easily upgraded to meet new market opportunities
by integrating legacy software into new designs and automatically generating and
deploying new code
 Cost of re-design necessitated by changes in available hardware is minimal with
MBD.
 Cost of in-field support is more effective because support personnel can more
clearly understand design models and code versus just source code MBD
minimizes the possibility of product recalls

6
III. Calculating the ROI for the Development Cycle – Establishing Cost

The typical development process includes:

 Product Specification
 Engineering Specification
 Requirements Engineering
 Hardware/Software design considerations and tradeoffs
(what should be done in software and what should be done in hardware)
 Software Conceptual Design
 Detailed Software Design
 Software Coding
 Software Module Testing
 System Software Simulation and testing
 System Rapid Prototyping
 Hardware/Software Integration and Test
 Final Release

Understandably CFOs would want to break out each item and assign numbers of
developers/engineers and time on job to get highly granular results – but this level of
detail obfuscates issues of importance. For purposes of discussion and calculation, we
will break our analysis into software design processes and support processes.

Design team size is affected by the design cycle and the complexity of the design itself.
In a typical embedded design, a project manager will assign parts of the design to
different groups within the design team. One group may focus on the high level
application; one group the user interface design; another on device drivers and
integration to the hardware. The development environment and the team’s familiarity
with the tools, the target operating systems and hardware platform, and the application
itself all play a role in a software project manager’s decision regarding team size.

A project manager may be faced with having to include training for part of his team in
order for them to be productive. It is important to have the appropriate resources
available to bring the team up to speed quickly in order to minimize impact on the design
cycle.

A simpler method for rapidly calculating ROI and comparative/alternative outcomes does
not need the detailed enumeration listed above. If problems do occur, senior
management can then do the detailed breakout to determine where in the design
process problems are originating.

Classical ROI can be calculated in different forms including:

I. ROI = Revenue – Development Costs

Subtracting costs from benefits provides a basic comparison which leaves off G&A.

II. ROI = Revenue

Development Costs
Ratios like Return on Investment, Return on Assets and Return on Sales calculations are
common methods for evaluation

7
III. ROI = Revenue – Development Costs
Revenue

Formula III is very similar to the calculation of Gross Margin.

CEOs and CFOs need to comparatively evaluate alternative design processes to

determine which will provide the best financial outcomes in terms of total development
costs, costs associated with cancellations and behind schedule completions, the cost of
tools and the cost of being late to market.

Unless one can measure revenue loss associated with being late to market, one can
assume that anticipated revenue is the same regardless of the development process
used. Hence senior management can use total direct costs associated with development
plus the cost of cancellations and behind schedule completions as a measuring stick to
compare alternative tool sets.

Hence the use of ROI for comparative evaluations can be effectively reduced to “Total
Cost of Development” and comparative design outcome advantages.

The EMF ROI software development framework is designed to enable a CFO, project
manager or embedded developer to identify and estimate common embedded software
development costs while encompassing additional risk elements for which they do not
frequently plan. Total Cost of Development includes the direct expenditure of software
development resources from design initiation to product shipment. It includes Associated
Costs (including the cost of development tools, maintenance and support) and includes
Extrinsic Costs (including a consideration of the cost due to poor design outcomes,
royalty costs, if any, etc.). Risk costs include the cost of product recall and missed
windows of opportunity

The Total Cost of Development consists of the following calculations:

 Software Development Costs

 Support Development Costs (hardware, testing and program management)
 Cost of Cancellation
 Direct Cost of Late Design Completion
 Associated Costs
 Extrinsic Costs
 Estimated Risk Costs

Software Development Costs

 Time-to-Market (TTM), the time from design start to product shipment

 Average # of Software Engineers and embedded software developers, but
excluding testers, QA and program managers (SWE)
 Man months of design effort (MM) = (TTM) * (SWE)
 Cost/Software Engineer (Cost), e.g., salary + overhead per month
 Software development costs = (MM) * (Cost)

8
Support Development costs

 Time-to-Market (TTM), the time from design start to product shipment

 Average # testers, hardware developers and program managers (Eng Support)
 Man months of design effort (MM) = (TTM) * (Eng Support)
 Cost/Support Engineers (Cost), e.g., salary + overhead per month
 Support Development Costs = (MM) * (Cost)

Total Direct Cost of Application Software Development (TDC)

 TDC = Software Development costs + Support Development costs

Cost of Cancellation

 Cancellation Costs = (TDC) * (percent of designs cancelled) * (months project

runs before cancellation)

Direct Cost of Late Design Completion

Direct Cost of Late Design completion = (TDC) * (percent of designs behind schedule) *
(average number of months project runs behind schedule)

Associated Costs (AC)

 AC takes into consideration development tool availability and amortized tool

acquisition cost as well as the cost of ongoing maintenance and support over the
duration of the product life cycle. Required training costs can be calculated here.

Extrinsic Costs

 Royalty costs if any

 Costs associated with the failure of final design results to approximate pre-
design expectations within 20%. There is a cost associated with imperfect
designs that result in extensive delays or removal of product features.

Estimated Risk Costs

 Cost of recalls
 Cost of Missing Target Window

The Total Cost of Development is directly measurable based on 3 factors: time-to-

market, number of software engineers and support team, and the employment cost of
those engineers. Our analysis includes the cost of cancellation and the cost of delivering
behind schedule, as well as Associated Costs. Extrinsic and Risk Costs vary from
company to company, and must be estimated.

9
Associated Costs are often variable and are influenced by development team size, the
skill level of the software engineering team, product life cycle, vendor negotiated terms,
and product appropriateness or quality for a particular embedded systems design.

Extrinsic costs are predicated on the frequency and resulting cost of deficiencies in
design processes. Risk costs reflect the cost of missed opportunities and the actual cost
of recalls. This is a variable that should be included in a trade off analysis between
design options (e.g., the use of MBD tools versus the use of non-MBD tools).

10
IV. Comparative Analysis of Expected ROI from Alternative Design Methodologies

Example: Comparative Telecom Equipment Development ROI Analysis

A real world example of a comparative ROI evaluation is presented for

Telecommunications equipment developments based on EMF’s 2019 detailed survey of
embedded developers (2455 respondents).

Following the ROI breakout in Section III, the 2019 EMF Survey data can be used to
compare direct costs between MBD-based telecom developers and non-MBD telecom
developers (developers using traditional hand coding). EMF conducts statistically
accurate surveys of embedded developers, who answer more than 70 detailed questions
that provide insights to what they use, how long it takes them to get a product to market,
how close to their final design is to their pre-design expectation, etc. Our data presented
herein is predicated on the responses of 2455 developers.

Table IV-1 presents:

 Total number of lines of code – including written, reuse and open source
 Number of software developers on design – and number of other developers and
support staff.

Telecom and Telecom

Not MBD and MBD

Total Project Lines of Code x1000 458.9 464.4

Ave number of SW Developers 13.7 12.4
Ave Project Support Staff 16.3 16.5

Table IV-1

As chance would have it, developers using MBD match up very closely for total lines of
code, number of developers and support staff. Given the common baseline between
these groups of MBD users and non-users, we can objectively calculate the respective
associated costs.

Given the common baseline between alternatives using these parameters, we can then
look to the following for comparative insights.

 Time taken from design start to actual shipment date

 Percent of Designs Cancelled
 Average number of months before cancellation
 Percent of designs completed behind schedule
 Average number of months behind schedule

Then a more detailed and comprehensive analysis can be made from the survey data in
which developers were asked “How close was your final design result to your pre-design

11
expectation for performance, systems functionality, and features and Schedule?” EMF
uses the best outcome (within 20% of expectation) for comparison.

If there is a compelling reason to replace standard telecom design methodologies with

MBD, we should be able to see significant improvement in the closeness of the final
design outcome to the pre-design expectation.

Table IV-2 presents a baseline comparison between MBD and non-MBD designs.

Telecom and Telecom and Improvement

Not MBD MBD with MBD

Total lines of Code - Project 458.9 464.4 Same

Months - start to shipment 11.6 9.4 23.4%
Designs Cancelled 14.0% 7.2% 94.4%
Months until Cancellation 3.6 3.7 -2.7%
Designs behind schedule 36.4% 19.7% 84.8%
Months behind schedule 2.3 1.8 27.8%

Table IV-2

Assuming that the costs associated with software developers (including overhead) is
$10,000/month and the costs of support staff is $8500/month we can calculate the
respective direct costs of development using MBD versus not. These results are
presented in Table IV-3.

Percent Improvement
Average Telecom Project Cost Telecom and Not MBD Telecom and MBD with MBD

Cost of Application Software $1,589,200 $1,165,600 36.3%

Cost of Support Staff $1,607,180 $1,318,350 21.9%
Ave Total Direct Cost of
Development $3,196,380 $2,483,950 28.7%

Ave Cancellation costs $138,877 $70,396 97.3%

Ave Late completion costs $230,690 $93,703 146.2%

Total Costs of Software

Development $3,565,947 $2,648,049 31.8%

Table IV-3

12
The calculations are straight forward. The Cost of Application Software equals the
number of project months from start to shipment multiplied by the number of software
developers multiplied by the monthly cost per developer ($10,000).

The calculation for cancellation costs and late completion costs equals the number of
software developers multiplied by the monthly cost/developer plus the number of support
staff multiplied by the monthly cost/support personnel. This number is multiplied by the
number of months it takes before a project is cancelled (or for late completions the
number of months the project is late) multiplied by the percentage of project
cancellations (or late completions).

The comparison between MBD use and no MBD use is significant – and the fact that the
projects matched up nearly exactly makes the conclusion more significant.

Extrinsic Costs:

Table IV-4 presents the relationship between pre-design expectations and final design
outcomes. Design Table IV-4 presents the percentage of final designs that are within
20% of the pre-design expectation.

Telecom and Telecom Improvement

Not MBD and MBD with MBD

Performance 60.0% 73.3% 22.2%

Systems Functionality 55.0% 66.7% 21.3%
Features & Schedule 65.0% 66.7% 2.6%

Table IV-4

Year-over-year analysis of design outcomes focus on several factors:

 Time-to-market
 The percent of designs completed behind schedule – and the average number of
months behind
 The percentage of designs cancelled – and the number of months until
cancellation
 Design outcomes as measured by the closeness of final design outcomes to pre-
design expectations.

In each measure, MBD was shown to outperform non-MBD factors. These

considerations must be included in a comparative ROI.

Although controlling direct costs is important, other issues must also be taken into
consideration. In addition to direct costs, one must place an additional dollar value on
the following which we call “Risk Costs”:

13
Risk Costs

 Cost of maintenance (depends on the complexity of the design and the

management of the design process)
 Cost of upgrading software to meet changes in underlying hardware (requires
upgrading legacy software)
 Cost of recalls (cost includes actual cost of retrieving products, lost revenue, cost
of reimbursement and the loss of reputation)
 Cost of providing code reuse (can save enormously on cost of development for
system/product upgrades)
 Cost of removing product features (failure to adequately meet pre-design
expectations)
 Cost of being late to market (depending on application, can result in the loss of
30%-90% of potential market – example would be consumer electronic product)

These risk costs can reach orders-of-magnitude greater than the upfront cost of
development tools and the direct cost of development.

It is up to the CFO/CEO to consider the potential cost of each risk item in light of its
probability of occurrence – and weigh that against the cost of protecting against the risk
factors happening.

In the case of MBD, protection against these risk items are built into the MBD process
and don’t require additional expenditures for non-MBD processes. The cost of a product
recall is dependent upon the nature of the product and the reputation of the company.
The recall of a medical product could be devastating. Imagine the actual cost of bringing
products back from the field, having to reimburse purchasers, having the company’s
reputation at stake and the cost of reapplying to the CDRH (FDA) for Class II 510k
compliance.

14
V. Summary and Conclusions

The intent of this report was to familiarize senior management with the benefits of a
design methodology based on MBD and the significant impact it can have on improving
ROI.

The model presented was intended to be transparent so that the reality of the design
process is not clouded over with numbers that lose touch with reality.

An actual example was presented from EMF data to illustrate how ROI can be used to
decide which methodologies to use for company developments.

In the case of Telecom equipment developers, MBD not only saved significant
development dollars but also provided significant improvements in time-to-market, fewer
cancellations, fewer designs being completed behind schedule, and better design
outcomes.

EMF suggests that senior managers pay attention to considerations to which it is difficult
to assign numbers – but are of distinct importance. The cost of recalls, for example,
might be prohibitive. Senior managers can estimate potential losses due to risk by
estimating the probable cost of recall or a missed window of opportunity multiplied by the
probability of such an occurrence. This would provide a calculated assumption of risk –
and would vary according to the specifics of an organization.

15