Advanced 3D HMI/SCADA

Visualization
Research Brief
 Advanced 3D HMI/SCADA Visualization

Russ Agrusa, Member, IEEE, Valeria G. Mazza, Roberto Penso, Alexander Pinkham
ICONICS Inc.

Using simple two-dimensional (2D) graphics for

Abstract — Today’s manufacturing and building
automation of process control has been the most common
operations are faced with the need to reduce cost and be
approach for some time. With the introduction and
more competitive with the fewest of resources.
optimization of three-dimensional (3D) graphics into
Connectivity to different infrastructures for data gathering
process control applications, many companies are seeing
and the need to analyze and visualize data in real time is
increased value in upgrade existing systems. Modern
ever more important in today’s economic environment.
visualization applications are leading the way to
Recently, visualization systems have taken a giant step
incorporating 3D graphics technology into a vast array of
forward incorporating advanced hardware accelerated 3D
industrial solutions that can meet real customer needs in
graphics into standard off-the shelf commercial products.
multiple industries, including:
As a result, providing compelling 3D graphics applications
and having access to plant data is the key to having a • Automotive
competitive advantage. • Building Automation
This paper presents an overview of a powerful approach to • Oil, Gas & Petrochemical
visualization of manufacturing that greatly reduces the • Food & Pharmaceutical
learning curve for developing Human Machine Interface • Water and Wastewater
(HMI) applications. among many others.
The inclusion of 3D graphics in to HMI displays has
Keywords — 3D Graphics, HMI, SCADA, WPF, XAML, resulted in faster learning of process control systems and a
AutomationML. better understanding of operations based on the idea that
3D graphics are better at representing our environment
I. INTRODUCTION
then 2D solutions can be. Overlaying process data on to

M ANUFACTURING systems are comprised of products,

equipment, people, information systems, control
and support functions for the competitive development,
3D models can enhance the contextual nature of HMI data
showing not only what is happening, but precisely where.

production, delivery and total life cycle of products. II. STATE OF THE ART: EVOLUTION OF 3D INDUSTRY
Company goals often consist of satisfying market demand
Several companies today offer 3D software tools and
while lowering the cost of production. Industrial and
plug-ins, pre-built 3D data, and services to a growing
automotive manufacturers are facing numerous challenges
number of 3D content creators. A global community of
across all phases of the product development process –
hundreds of thousands of 3D enthusiasts as well as
shortened development times, managing global supply
professional engineers and programmers utilize 3D
chains, fierce competition, and increasingly complex
technology for product development, prototyping,
products. They are in need of solutions that can help them
engineering analysis.
get to market faster, cheaper and with greater
What are the factors affecting the continued evolution
functionality.
of this 3D ecosystem? There are three fundamental
The best strategy to achieve these goals is to adopt open
changes at work: the rising popularity and consistency of
industrial standards for complete integration of multiple
new 3D software, the growing global influence reduced
systems at a hardware level providing freedom from heavy
costs through prototyping, and the growing availability of
dependency on single vendors as well as to choose the
pre-made 3D content.
latest software tools that are built on an architecture that
An increasingly rich array of software for creating 3D
can be scaled and remain relevant for at least five to ten
content is now available. Previously, large, multi-purpose
years.
3D software packages such as Softimage’s XSI and
ICONICS Inc.- 100 Foxborough Blvd, Foxboro, 02035 MA (USA)
Autodesk’s 3ds Max and Maya dominated the commercial
Tel: +1 508 543 8600 ; Fax: +1 508 543 1503 tools market. Today, also other stand-alone, focused 3D
products offer solutions for many functions and
Russ Agrusa, ICONICS President and CEO, [email protected] technologies specially optimized for particular needs.
Valeria G. Mazza, 3D Graphic Designer, [email protected]
Roberto Penso, South Europe Sales Manager, [email protected] Specific software has also gained traction in specific
Alexander Pinkham, Product Marketing Engineer, [email protected] industries allowing standardization within vertical
markets. As the number of standards grows, file formats time data driving on-screen values and animations.
like COLLADA or 3DS have emerged as ways to transfer HMI/SCADA software of the today and the future will not
models between products, eliminating a frustration in 3D only provide control of a process and visualization of data,
collaboration. but will show real-time, realistic machine assets with 3D
Since people started using 3D software, the desire to modelling.
share the experience with other 3D users has been strong.
Today, the interaction among 3D engineers is occurring on III. 3D ENHANCES REAL-TIME VISUALIZATION
a much larger scale via the Internet, where very large 3D visualization can benefit HMI applications by
online communities have formed. In dozens of online providing true to life shapes, forms, processes and other
communities, 24 hours a day, 3D works in progress are details that otherwise are lost when viewed in a traditional
presented and critiqued, technology advances are 2D manner. Having a 3D model, set of images or
discussed, and techniques are shared. animation allows better interactivity with the idea, process
The increasing availability of 3D content in today’s or product. When an idea needs to be presented to the
university curriculum and professional offices has created potential client, most times the idea is best received in a
a powerful gateway to introduce a large number of people 3D form. We live in a 3D world; our eyes and brains are
to the hands-on use of 3D technology. This hands-on used to seeing and perceiving depth, perspective and form.
experience is beyond the passive exposure people receive
from 3D rendered pictures. Multiple studies indicate that
3D elements of graphical image are perceived better by
human operators than 2D process drawings. That is why
many companies have been developing industrial 3D
solutions:
• Autodesk Inventor: is an extension of the
framework behind AutoCAD to provide 3D
prototyping allowing users to shave material and
money off of existing products at little to no
researching overhead.
• Hoops 3D Application Framework® by Spatial
Corp [1]: provides a core graphics infrastructure and
functionality for 3D applications and full Application
Programming Interface (API) access to a powerful Fig. 1. 2D Automotive HMI graphic (ICONICS[3])
underlying suite of integrated components
• DeskArtes ViewExpert®: offers tools for viewing, Products come alive when they can be delivered in a 3D
verifying, measuring and communicating 3D CAD format. Process and technologies can be improved, both in
data files, such as STL, VRML.ZPR and IGES function and visualization, by showing them in a 3D
formats environment, especially when the environment can be
• Google 3D Warehouse: Offering DAE and Sketch made to resemble the actual working conditions of the
Up models from Google’s 3D modelling program. [2] process – rather than a simplified 2D drawing.
and many others solutions are available to meet the Technically, a 3D image is the best way to handle
ever growing market needs. visualization of a large number data and to keep a global
For professional applications, rapid prototyping or vision on a system. It is indeed the most natural way to
virtual prototyping has become the industry standard for represent a large number of heterogeneous information.
product and process research. For real-time operations
graphics HMI/SCADA software has taken a note from
visual simulation industries to concentrate on embedding
efficient rendering algorithms for 3D geometry loaded
from a file store. These algorithms allow HMI/SCADA
software to implement features such as a polygonal
reduction of prototyping models to optimize them for real-
time viewing. With efficient models for rendering current
desktop and web visualization technologies HMI/SCADA
software can benefit from the 3D capability of displaying
not just process diagrams and maps of operations, but
rather the real world view of the operation. Operators can
view all assets from any angle or have events like alarms
in their system bring information and 3D views to the
operator.
With the abilities of SCADA software to bring data to
life, 3D HMIs can further contextualize data by bringing
Fig. 2. 3D Train HMI graphic (GENESIS64 Suite[4])
an interactive view of operations to operators with real
 level, exemplifying the next generation of standard-based
HMI displays can take advantages of the following communication.
benefits: Today, industrial and manufacturing solutions can be
• Reduce production costs and time-to-market developed taking maximum advantage of state-of-the-art
• Expedite complicated decision making processes graphic hardware acceleration through DirectX10,
• Review product in concept and each stage of integrated with Windows Presentation Foundation (WPF)
design and development for rich 2D and 3D HMI applications.
• Decrease dependency on the 2D drawing WPF [6] is a graphical subsystem in .NET Framework
interpretation to avoid production mistakes 3.0, which uses a mark-up language known as XAML
• Operate from easy to understandable visual plans. (eXtensible Application Mark-up Language) [7], for rich
• Showcase industrial environment and products user interface development.
features from different angles before full-scale Innovative HMI applications benefit of the following
production. WPF features:
• Supports vector-based graphics
Using 3D displays greatly improves design quality • Supports 3D model rendering and interaction in
because it is a more complete process than 2D design. As a 2D applications
result, many human errors that can occur with traditional • Interactive 2D content overlaid on 3D surfaces
2D design methods are avoided. • Offload some graphics tasks to the Graphics
This makes 3D HMI solutions a powerful business tool: Processing Unit found on the computer’s graphics
it can communicate complex subject quickly, reduce costs card
and errors rate, and make collaboration more efficient and
flexible.

IV. LEVERAGING ADVANCED TECHNOLOGY

With the convergence of 64-bit, multi-core, multi-
processor computing; the introduction of new 64-bit only
operating systems; and demand for high performance 64-
bit native applications, modern visualization solutions
provide customers with the greatest saleability, reliability
and flexibility.
Designed from the ground up and taking advantage of
the OPC-UA communications standard, .NET managed
code and SharePoint® technology, the newest HMI
solutions allow for connectivity from plant floor and
business facilities to corporate business systems.
OPC Unified Architecture (OPC-UA) [5] is a robust,
secure and scalable expansion of the highly successful
basic COM/DCOM-based OPC standard communication
Fig. 4. Microsoft .Net 3.0 Stack
protocol. OPC-UA allows the interoperability of best-of-
breed real-time, alarm management and historian systems.
The specific advantage that XAML brings to WPF is
that XAML is a completely declarative language. As a
result, the developer (or designer) describes the behaviour
and integration of components without the use of
procedural programming. This allows someone with little
or no traditional programming experience to create an
entire working application with no programming.

Fig. 3. OPC-UA Specification Layer

This allows for a standard model of plant floor

integration with the enterprise. Any industrial or facility
control system that is currently using OPC
communications can easily add the latest OPC-UA
applications to their existing system, giving them the
added value of Web services that allow for more enterprise
connectivity. The next generation of OPC Data Access
(DA), OPC Alarm and Events (A&E) and OPC
Historical Data Access (HDA) allows for secure, open
connectivity from plants and facilities to the enterprise
 Fig. 7. Engineering chain

Fig. 5. 3D Facility Control WPF and XAML [8] This is the innovation brought by AutomationMLTM
(Automation Markup Language) to achieve reliability,
Although it is rare that an entire application will be built availability of knowledge, independence and cost
completely in XAML, the introduction of XAML allows effectiveness, by combining well accepted standardized
application designers to more effectively contribute to the formats already deeply used in the market.
application development cycle. Using XAML to develop
user interfaces also allows for separation of model and VI. THE GLUE FOR SEAMLESS AUTOMATION
view; which is considered a good architectural principle. ENGINEERING: <AUTOMATIONML/>
The engineering cost of an automation project is
typically 60% of the total project cost. In the past
optimization strategies have addressed the bought-in parts
well; a wide area to increase efficiency is still in
engineering, for example:

• Factory plans are manually redrawn in other tools

for production line planning
• Companies suffer from heterogeneous CAD
(Computer-Aided Design) tool where CAD
systems do not collaborate
Fig. 6. XAML between designers and developers • Conveyor Sequences are developed with office
tools and are not reusable with PLC
V. BRIDGING THE GAPS IN YOUR DESIGN AND programming.
MANUFACTURING WORKFLOW
Compared to several years ago, the automotive industry AutomationML [9], as an open intermediate format,
has been very rapidly changing. New business workflows, improves automation engineering, reducing costs
processes, procedures and manufacturing techniques must associated with that. AutomationML is a neutral data
support environments that inevitably reduce the format based on XML for the storage and exchange of
manufacturing design and preparation time in developing a plant engineering information. The goal is to interconnect
new automobile. This requires engineering systems that the heterogeneous automation CAD tools of modern
improve collaboration, driven by the asset utilization of engineering production planning of the different
factory resources and substantial reduction in costs. By disciplines, e.g. mechanical plant engineering, electrical
using this new approach and system, savings in time and design, visualization development, PLC, robot control.
cost of process and material planning are possible, and the AutomationML describes real plant components as
reliability of the plan result is greatly improved. objects encapsulating the different aspects of the plant
As customer demands diversify, product lifecycle is operation. An object can consist out of other sub-objects,
shortened and global competition among companies and can itself be part of a bigger composition. It can
becomes fiercer, automotive companies strive to discover describe a screw, a claw, a robot or a complete
new paradigm shifts and technologies for rapid and cost manufacturing cell in different levels of detail.
effective ways of developing new products.
As a result, the goal consists of adopting several
specialized standards under one umbrella to support as
many aspects of the engineering chain as possible.

Fig. 8. Combined kinematics system

 Fig. 9. Production cell used in different CAD tools increasing design change time and cost, and longer time to
market.
Typical objects in plant automation comprise In today’s global market, as manufacturers work to
information about topology, geometry, kinematics and reduce design cycle and cost margins, industry experts are
logic, where logic comprises sequencing, behaviour and championing Digital Prototyping as a way to cost-
control. AutomationML incorporates different standards effectively validate design ideas and accelerate the
through strongly typed links across the formats: development of competitive products.
Digital Prototyping gives manufacturers the ability to
• Topology implemented with CAEX (IEC 62424): virtually explore a complete product before it is built and
properties and relations of objects in their put into production. This is done so they can create,
hierarchical structure validate, optimize, and manage designs from the
• Geometry implemented with COLLADA of the conceptual design phase through the manufacturing phase
Khronos Group: graphical attributes and 3D of product development. By using digital prototyping,
information manufacturers can boost design efficiency and innovation
by visualizing and simulating the real-world performance
• Kinematics implemented with COLLADA:
and characteristic of a specific design, and save time and
connections and dependencies among objects to
money by reducing the number of physical prototypes that
support motion planning are built.
Autodesk® Inventor™ [10] is a solution that is
redefining traditional CAD workflows by helping
• Logics implemented with PLCopen XML: engineers focus on the functional requirements of a design
sequences of actions, internal behaviour of to drive the automatic creation of 3D models (i.e. steel
objects and I/O connection tools frames, rotating machinery, tube and pipe runs etc.) that
can be used to realize compelling 3D HMI applications.
Reducing the time spent on geometry allows engineers
to spend more time innovating design and catch errors
before they reach production.

Fig. 10. AutomationML standard

VII. DIGITAL PROTOTYPING: BEYOND 3D DESIGN

Over the past couple of decades, there have been several
dedicated software applications available to engineers to
create HMI applications. Major drawbacks of such
dedicated software include the advanced knowledge
required although their lack of integration with 3D Fig. 11. Digital Prototyping with Autodesk® Inventor™
computer-aided-design (CAD) software. Consequently,
designers and engineers have been working in isolation
from one another resulting in duplication of work,
 Fig.14. Polygons reduction algorithm applied to turbine blades

At the same time, this approach helps manufacturers to IX. LOW-POLY 3D MODELS FOR INDUSTRIAL AND
realize the benefits of Digital Prototyping with minimal MANUFACTURING SYSTEMS
disruption to existing workflows. This provides the most 3D HMI applications are often confronted with either
straightforward path to creating and maintaining a single very dense and over-sampled surfaces or models. This is
digital model in a multidisciplinary engineering design and especially relevant to companies using models imported
prototyping environment. from internal files which often hold very exact information
with low tolerances. The result is 3D HMIs which are far
VIII. LEVERAGING 3D IN HMI APPLICATIONS too complex for real-time viewing and operation.
Manufacturing and Industrial systems with existing 3D As with many applications in computer graphics and
models or who have the need to create 3D content for HMI related fields, HMI applications can benefit from
visualization historically have used third party systems to automatic simplification of complex polygonal surface
develop graphics. “3D” graphics were then brought into models, usually coming from 3D CAD drawings.
HMI applications as various still images. With the vast Recently, much research has gone into this subject in order
number of 3D modeling programs available there has been to develop the most effective polygons reduction
little integration to SCADA systems due to the issues that algorithm. The goal simply, consists of retaining the
arise with file formats and import/export capabilities. quality, fidelity and the appearance of the original 3D
While companies have been hesitant to require 3D drawings while reducing the overall number of polygons.
graphics in HMI, SCADA software vendors had
historically avoided 3D based on the capabilities of
computer graphics and perceived value to customers.

With more and more 3D content available via online

sources or from other divisions of the companies
themselves, 3D integration to HMI applications has seen
an increase recently as the time of development has
decreased enough to allow for companies to realize the
benefits of 3D without incurring the cost in development
time that once hindered the inclusion of 3D graphics.
Complex 3D displays can be made in minutes with
imported 3D models.
Fig. 14. 3D drawings imported in ICONICS HMI
To further aid development, 3D models have become
commonplace in a number of industries, resulting in some The core of the polygon reduction algorithm is based on
websites creating offerings for 3D content built by users the “Edges Contraction” technique [11]-[12]. Edge
and published for other users to include in their own work. contraction simply means that the two end vertices of a
As discussed earlier there are a number of different ways model edge are replaced by a single new vertex. This
for manufacturing companies to incorporate existing 3D target vertex is usually somewhere in between the other
graphics into their HMI applications. But for companies two, in a place where it best approximates the original
that do not have the capability, infrastructure or manpower model. This edge contraction step removes a vertex and
to create 3D graphics specific to their application, websites one, two, or more faces from the model, depending on the
like Google’s 3D Warehouse allow users to download free mesh neighbourhood.
models to use in their applications. Most models can be When this step is repeated several times, it results in a
downloaded in Google’s own format or in the COLLADA simpler model, which is an approximation to the original.
(.DAE) format for inclusion into HMI applications.
 X. CONCLUSIONS: ELEVATE HMIS TO NEW LEVELS
The goal of the next generation in industrial automation
software consists of giving the user the power to quickly
and efficiently create HMI integrating graphics, real-time
manufacturing data and business information.
Fig. 17 and Fig. 18 are few examples of real HMI
Fig. 15. “Edge Contraction” technique visualization applications that users are able to create with
the powerful tools and the innovative technologies
Since data is removed from the original model the available today.
resulting model will only be a very close approximation; In this paper we walked you through 3D graphics
each of these steps is associated with a given increase in planning, creation and data connectivity, step by step,
error (cost of contraction) that is recalculated at each step. ultimately demonstrating the industry standards available
Then the lowest cost contraction is performed again until today to reduce engineering, development costs and total
the desired target face count is reached. Simplification can cost of ownership. We also described the main features to
also be stopped when the lowest error contraction is above achieve this ultimate goal:
a certain error threshold. • Hardware Accelerated 3D Graphics
• 2D and 3D XAML, WPF Visualization
• OPC-UA for real-time data connectivity
• AutomationML, Digital Prototyping
Fig. 16. The edge cost formula • 3D file Format Conversion and Polygon
Reduction
Fig.16 shows the edge cost formula where Tu is the set In order to take advantage of the advanced 3D graphics
of triangles that contain u and Tuv is the set of triangles of today’s HMI, manufacturing and facility control
that contain both u and v. operators who want to extend and branch out of the
The effectiveness of a polygon reduction algorithm is traditional 2D graphics may need some guidance, training
best demonstrated by showing a model before and after it and orientation to learn these new powerful technologies.
has been simplified (Fig.12). Most research papers Through the inclusion of 3D graphics into real-time
demonstrate their results using highly tessellated models in control HMI displays the gap between design and
the neighbourhood of 1 million polygons, reducing them manufacturing workflow will ultimately be reduced.
to 50,000 polygons with similar visual results after Built on the foundation of high-performance graphics
rendering. hardware subsystems, Human-Machine-Interface (HMIs)
systems in the future will require a higher level of
sophistication with respect to configuration and overall
operation. The promise of 3D graphics brings HMI
visualization applications to a new level of awareness for
today’s manufacturing and facility controls designers.
What is needed now are automated software tools that
make designing, prototyping and automated generation of
HMI visualizations systems an easier process.

Fig. 18. 3D GENESIS64 automotive factory

Fig. 17. 3D oil and petrochemical display

 REFERENCES [7] Januszewski K., Rodriguez J.: “The new iteration: how XAML
transforms the collaboration between designers and developers in
[1] Spatial Corp: “Growth opportunities for 3D manufacturing Windows Presentation Foundation” , Microsoft Corporation, 2007.
application providers in pressed-metal tooling”, 2009 [8] http://iconics.com/products/pdfs/GENESIS64_Flyer.pdf
[2] http://sketchup.google.com/3dwarehouse/ [9] Drath R., ABB AG, Corporate Research Ladenburg.:
[3] http://www.iconics.com/products/genesis32.asp “AutomationML top-level architecture document”, April 2008.
Available: http://www.automationml.org
[4] http://www.iconics.com/products/genesis64.asp
[5] http://www.opcfoundation.org/ [10] Autodesk,Inc: “ Benefits of using Autodesk Inventor simulation
http://www.opcconnect.com/ua.php suite in engineering design”.
Available: http://www.autodesk.com
Hannelius, T. Salmenpera, M. Kuikka, S. : “Roadmap to
[11] Garland M.: “Quadric-based polygonal surface simplification”,
adopting OPC UA”, presented at the 6th IEEE Int. Conf. on
Industrial Informatics, Daejeon, July 2008. Ph.D. dissertation, School of Computer Science, Carnegie Mellon
[6] Andrade C., Livermore S., Meyers M., Van Vliet S.: “Professional University, Pittsburgh, May 1999.
WPF programming: .NET development with the Windows® [12] Melax S.: “A simple, fast, and effective polygon reduction
Presentation Foundation”, 2007. algorithm”, Game Developer Magazine, November 1998.
 Founded in 1986, ICONICS is an award-winning independent software developer
offering real-time visualization, HMI/SCADA, energy, fault detection, manufacturing
intelligence, MES and a suite of analytics solutions for operational excellence.
ICONICS solutions are installed in 70% of the Fortune 500 companies around the
world, helping customers to be more profitable, agile and efficient, to improve
quality and be more sustainable.

ICONICS is leading the way in cloud-based solutions with its HMI/SCADA, analytics,
mobile and data historian to help its customers embrace the Internet of Things (IoT).
ICONICS products are used in manufacturing, building automation, oil & gas,
renewable energy, utilities, water/wastewater, pharmaceuticals, automotive and
many other industries. ICONICS’ advanced visualization, productivity, and
sustainability solutions are built on its flagship products: GENESIS64™ HMI/SCADA,
Hyper Historian™ plant historian, AnalytiX® solution suite and MobileHMI™ mobile
apps. Delivering information anytime, anywhere, ICONICS’ solutions scale from the
smallest standalone embedded projects to the largest enterprise applications.

ICONICS promotes an international culture of innovation, creativity and excellence in

product design, development, technical support, training, sales and consulting
services for end users, systems integrators, OEMs and Channel Partners. ICONICS has
over 300,000 applications installed in multiple industries worldwide.

World Headquarters European Headquarters

100 Foxborough Blvd. Netherlands
Foxborough, MA, USA, 02035 Tel: 31 252 228 588
Tel: 508 543 8600 Email: [email protected]
Email: [email protected]
Web: www.iconics.com

Czech Republic France China

Tel: 420 377 183 420 Tel: 33 4 50 19 11 80 Tel: 86 10 8494 2570
Email: [email protected] Email: [email protected] Email: [email protected]

Italy UK India
Tel: 39 010 46 0626 Tel: 44 1384 246 700 Tel: 91 22 67291029
Email: [email protected] Email: [email protected] Email: [email protected]

Germany Australia Middle East

Tel: 49 2241 16 508 0 Tel: 61 2 9605 1333 Tel: 966 540 881 264
Email: [email protected] Email: [email protected] Email: [email protected]

www.iconics.com
© 2015 ICONICS, Inc. All rights reserved. Specifications are subject to change without notice. AnalytiX and its respective modules are
registered trademarks of ICONICS, Inc. GENESIS64, GENESIS32, Hyper Historian, BizViz, PortalWorX, MobileHMI and their respective
modules, OPC-To-The-Core, and Visualize Your Enterprise are trademarks of ICONICS, Inc. Other product and company names mentioned
herein may be trademarks of their respective owners.