—ABB CORPORATE RESEARCH CENTER GERMANY
Bottleneck Identification and Performance Modeling
of OPC UA Communication Models
Andreas Burger, Heiko Koziolek, Julius Rückert, Marie Platenius-Mohr, Gösta Stomberg
PUBLIC
—
Motivation
Towards an Industrial Internet of Things (IIoT)
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 2
Industrial
controllers
Servers Operator
workstations
Temperature
sensor
Pump Motor &
drive
Field
instrument
Flow
meter
Field comm.
interface
Cloud
Smart
sensor
IIoT
14 BUSD
annual
market
> 70.000
control
systems
70 million
connected
devices
—
Open Platform Communications Unified Architecture (OPC UA)
– Evolves towards universal communication standard in industrial automation
– Recently communication model added for field level communication
Motivation
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 3
OPC UA: Standardizing M2M Communication
Industrial
controllers
Servers Operator
workstations
Temperature
sensor
Pump Motor &
drive
Field
instrument
Flow
meter
Field comm.
interface
*Picture source: Eckhardt et al.: An Evaluation of the Applicability of OPC UA Publish Subscribe on Factory Automation
use Cases. In IEEE Conference on Emerging Technologies and Factory Automation (ETFA), 2018.
OPC UA
OPC UA
Fieldbus
protocols
OPC UA
OPC UA
S/P
OPC UA
S+P
OPC UA
S+P
OPC UA
S/P
OPC UA
S+P
Server
+
Client
Server
+
Client
Server
+
Client
OPC
UA
Client
New OPC UA
Communication
Model *
—
Motivation
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 4
Client/server Publish/subscribe (pub/sub)
So far, missing experience and guidelines on use of OPC UA pub/sub in industrial applications
OPC UA Communication Models
OPC UA
Server
OPC UA
Client
OPC UA
Publisher
Subscriber
Subscriber
SubscriberTCP/IP connection
IP
Multicast
OPC UA
Client
Acyclic & cyclic
data exchange
OPC UA
Publisher
Subscriber
Subscriber
Subscriber
Broker
Cyclic
data exchange
Broker-basedBroker-less
Session
Session
—
Motivation
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 5
Client/server Publish/subscribe (pub/sub)
Contribution: Measurement study on bottlenecks & proposal of performance model
Potential Bottlenecks
OPC UA
Server
OPC UA Client
OPC UA Client
OPC UA Client
OPC UA Client
CPU Mem
Network
OPC UA
Publisher
Subscriber
Subscriber
Subscriber
Subscriber
CPU Mem
CPU Mem
CPU Mem
CPU Mem
Network
Potential bottleneck
IP MulticastTCP/IP
CPU Mem
—
Measurement Study
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 6
Testbed Setup and Configuration
Industrial-grade
managed Ethernet
switches, with
IP multicast support
Monitor &
control laptop
Server / Publisher
Client /
Subscriber
(2x Raspberry Pi 3, Model B,
Quad Core 1.2GHz 64bit
CPU, 1GB RAM, RTLinux)
(Raspberry Pi Zero,
1GHz single-core CPU,
512MB RAM, RTLinux)
Hirschmann switch
RSP35
(2x Raspberry Pi 3, Model B,
Quad Core 1.2GHz 64bit
CPU, 1GB RAM, RTLinux)
Client /
Subscriber
Client /
Subscriber
Client /
Subscriber
Industrial
controllers
Servers Operator
workstations
Temperature
sensor
Pump Motor &
drive
Field
instrument
Flow
meter
Field Comm.
Interface
—
Measurement Study
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 7
Two sets of software clients developed Performance measurements
Implementation and Instrumentation
OPC UA
Server
OPC UA
Client
OPC UA
Publisher
OPC UA
Subscriber
OPC UA SDK OPC UA SDK
Sensor
values
Server/
Publisher
Client/
Subscriber
Memory
NetworkCPU
Network
traffic
Memory
CPU
Network
traffic
Measurements
For measurements:
Randomly generated
signal values
OPC UA C++ SDK by
Unified Automation
+ pub/sub extension
OPC UA C++ SDK by
Unified Automation
+ pub/sub extension
—
Performance Measurements
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 8
CPU Utilization
Pub/sub:
40,000 signals/s
Client/server (1 client):
25,000 signals/s
Client/server (30 clients):
10,000 signals/sExemplary control use case
– 6,000 I/O points with 100ms updates (= 60,000 signals/s)
– 2 controllers share 8 field communication interfaces (FCIs)
Industrial
controllers
Sensors &
actuators
Field comm.
interfaces
… …
………
… …
—
Performance Measurements
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 9
Memory & Network Utilization
Overall <4% of Raspberry Pi
Zero’s main memory
Overall <1% of switch
capacity and <10% of
network interface capacity
Pub/sub: lower bandwidth
and independent of clients
—
Performance Measurements
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 10
Client/server Publish/subscribe (pub/sub)
Result: Server/publisher CPU utilization main bottleneck
Bottleneck Identification
OPC UA
Server
OPC UA Client
OPC UA Client
OPC UA Client
OPC UA Client
CPU Mem
Network
OPC UA
Publisher
Subscriber
Subscriber
Subscriber
Subscriber
CPU Mem
CPU Mem
CPU Mem
CPU Mem
Network
Potential bottleneck
(future work)
IP MulticastTCP/IP
CPU Mem
No bottleneck Identified bottleneck
—
Performance Model
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 11
Vision for Model Application
Performance
Model
(Control platform
specific)
New System
Design
(E.g. planned
control system)
Performance
Prediction
(Expected
CPU utilization)
Target Control Platform
(Hardware & Software)
Assess feasibility,
choose comm. model,
size hardware
Receives/defines
requirements for new
control system
—
Performance Model
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 12
Model Construction
Performance
Model
Predicted
CPU utilization
(bottleneck resource)
CPU
Model parameters (features)
ns: number of signals
spr: signal publishing rate
nc: number of clients
feature
transformation
weight vector feature vector
predicted
CPU utilization
Linear regression
w/ Maximum Likelihood estimation
—
Model calibration specific to target hardware/software platform
Performance Model
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 13
Model considerations Feature transformation
Transform feature vector 𝑥 using nonlinear
feature transformation
– Feature engineering
• Domain knowledge
• Comparison of transformations
– Considers linear dependencies
• Result of iteratively tuning
feature contributions
Training the model
Weight vector 𝑤 calibrates model
– Inferred from experiment data
– Maximum likelihood linear regression
• [Φ] 𝑇
: Gram matrix of transformed feature
vectors as columns
• 𝑡 : vector of measured CPU load values
Model Construction for Client/Server Model
feature transformation
weight vector feature vector
predicted
CPU utilization
Model parameters (features)
ns: number of signals
spr: signal publishing rate
nc: number of clients
—
Performance Model
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 14
Trained vs. predicted (sanity check)
– Model trained with 10 client/server
connections & publishing rate of 100ms
 Model closely reflects training data
Applied to 5 client connections
– Prediction for 5 connections
 Good match to measured data
Applied to 20 client connections
– Prediction for 20 connections
 Measurements shows step, but still
considered useful for predictions
Validation of Model Accuracy
—
Use case system parameters
– 6,000 I/O points with 100ms updates, 2 process controllers
– Controllers share 8 field communication interfaces (FCIs)
 Can each controller handle 3,000 I/O points via 4 connections?
 Can each FCI handle 750 I/O points via 2 connections?
Performance Model
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 15
Application to Hardware Sizing Use Case
Limit for RP Zero
Industrial
controllers
Sensors &
actuators
Field comm.
interfaces
… …
………
… …
750 3000
—
Conclusion
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 16
OPC UA Measurement and Modeling
Quantified
benefit of
OPC UA
pub/sub
Proposed
practical
performance
model
Gained
experience w/
pub/sub
model
Showed
feasibility
for practical
use cases
Identified
potential for
future work
IIoT using
OPC UA
Pub/Sub
Contribution
towards
IIoT vision
Bottleneck Identification and Performance Modeling of OPC UA Communication Models
—
References
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 18
[8] W. Mahnke, S.-H. Leitner, and M. Damm. 2009. OPC Unified Architecture.
Springer Science & Business Media.
[9] OPC Foundation. 2018. OPC Unified Architecture Specification Part 14:
PubSub.
[10] J. Pfrommer, A. Ebner, S. Ravikumar, and B. Karunakaran. 2018. Open
Source OPC UA PubSub over TSN for Realtime Industrial Communication. In IEEE
Emerging Technologies in Factory Automation (ETFA). 1–4.
[11] M. Silveira Rocha, G. Serpa Sestito, A. Luis Dias, A. Celso Turcato, and D.
Brandão. 2018. Performance Comparison between OPC UA and MQTT for Data
Exchange. In 2018 Workshop on Metrology for Industry 4.0 and IoT. 175–179.
https://doi.org/10.1109/METROI4.2018.8428342
[12] P. Runeson and M. Höst. 2009. Guidelines for Conducting and Reporting Case
Study Research in Software Engineering. Empirical Software Engineering 14, 2
(2009), 131.
[13] M. Schleipen, S.-S. Gilani, T. Bischoff, and J. Pfrommer. 2016. OPC UA &
Industrie 4.0 – Enabling Technology with High Diversity and Variability. Procedia
CIRP 57, 1 (2016), 315–320.
[1] C. M. Bishop. 2006. Pattern Recognition and Machine Learning (Information
Science and Statistics). Springer-Verlag New York.
[2] D. Bruckner, R. Blair, M-P. Stanica, A. Ademaj, W. Skeffington, and D. Kutscher.
2018. OPC UA TSN A new Solution for Industrial Communication. Technical
Report. Shapers Group White Paper. 1–10 pages.
https://www.automationworld.com/opc-ua-tsn-new-solution-industrial-
communication
[3] B. Cain, S. Deering, I. Kouvelas, B. Fenner, and A. Thyagarajan. 2002. Internet
Group Management Protocol, Version 3. RFC 3376.
https://tools.ietf.org/html/rfc3376
[4] S. Cavalieri and F. Chiacchio. 2013. Analysis of OPC UA Performances.
Computer Standards & Interfaces 36, 1 (2013), 165–177.
[5] M. Fojcik and K. Folkert. 2012. Introduction to OPC UA Performance. In
International Conference on Computer Networks. Springer, 261–270.
[6] S. Grüner, J. Pfrommer, and F. Palm. 2016. RESTful Industrial Communication
with OPC UA. IEEE Transactions on Industrial Informatics 12, 5 (2016), 1832–
1841.
[7] H. Koziolek, A. Burger, and J. Doppelhamer. 2018. Self-Commissioning
Industrial IoT-Systems in Process Automation: A Reference Architecture. In 2018
IEEE Internat. Conf. on Software Architecture (ICSA). IEEE, 196–19609.
—
Author Focus
Mahnke et al. [8] Rudimentary performance analysis focused on round-trip times, comparing classic OPC, UA Binary, SOAP/XML
Cavalieri et al. [4] OPC UA round-trip times for different message sizes, delay for subscriptions with short update intervals, round-trip time overhead caused by
encryption
Fojcik et al. [5] Discuss basic performance parameters for OPC , performance measurements using Prosys OPC UA SDK on powerful PCs
Gruener et al. [6] Propose RESTful extension for OPC UA and measured round-trip times for different number of read requests per session
Rocha et al. [11] Performance measurements of MQTT and OPC UA in cloud-to-client setting
“Shaper group” [2] Measurements regarding time synchronization between nodes in a time-sensitive network setting; theoretical calculation on minimum
achievable cycle times (relevant for OPC UA Pub/Sub although not discussed)
Pfrommer et al. [10] Jitter measurements on OPC UA Pub/Sub using open62541 SDK
Koziolek et al [7] Performance measurements for OPC UA Pub/Sub with the focus on encoding overhead
Related Work
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 19
Overview
So far, no systematic performance measurements to guide use of OPC UA pub/sub in industrial setting
—
Performance Measurements
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 20
Bottleneck Identification
CPU utilization main bottleneck for client/server and pub/sub model
Max. CPU utilization: 80-90% Max. memory usage: < 4% Max. network utilization: < 1%
—
Performance Measurements
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 21
– Implemented multi-
threaded signal
updating using POSIX
threads
– For Raspberry Pi Zero,
no performance gain
was possible
– For Raspberry Pi 3, a
35% performance gain
was possible, enabling
higher signal rates
– Evaluated security
impact only for client/
server model
– Using state-of-the-art
encryption policy
Basic256Sha256
– Highest overhead of
about 10% for higher
number of clients
– Security policy should
be always considered
Security overhead and multithreading
RQ5: How much overhead in terms of CPU utilization is caused by
security policies?
RQ6: What is the impact of multithreading on the server
performance?
—
Potential Threats to Validity
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 22
Construct validity
– Raspberry Pi devices as substitutes for
industrial embedded systems
– Debian Linux with the RT PREEMPT patch
we used a soft real-time operating system
(RTOS) instead of commercial RTOS
– Chosen OPC UASDK from Unified
Automation
– Used update intervals of 50ms or higher in
our experiments
– Focus on IGMP-based multicast for pub/sub
model
Internal validity
– Beta-status of the chosen OPC UA SDK
(not performance optimized)
– Additional bottlenecks could hide in the
network for different client information need
profiles
– Chosen simple maximum likelihood linear
regression model
External validity
– Representativeness of results for real
control application profiles
– Missing comparison of pub/sub
performance results to other SDKs or M2M
middlewares
Overview of considered threads
—
Overall, our results show how performance models can guide and improve the design of real control systems
Conclusion
ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 23
Performance measurements
– CPU utilization identified as performance
bottleneck for OPC UA client/server and
pub/sub model
– Memory and network overhead observed to
be negligible and not a bottleneck
– Low overhead added by security policies
– Implementation can benefit from
multithreading
Performance model
– Linear regression model: simple model but
already provided interesting insights and
valuable for hardware sizing
– Proposed models considered useful for
practitioners and researchers designing
industrial automation systems
– Considered especially useful for industrial
Internet-of-Things applications with many
OPC UA communication partners
Potential future work
– Extend measurements study to deepen
understanding on OPC UA performance
• Different hardware platforms
• Additional application profiles
– Improve performance model and make it
applicable in even more situations
• Queuing network
• Neural network
Summary, Take Away, and Future Work

More Related Content

PPTX
OpenPnP: a Plug-and-Produce Architecture for the Industrial Internet of Things
PDF
Software Architecture in Process Automation: UML & the "Smart Factory"
PPTX
Self-commissioning Industrial IoT Systems
PPTX
Tool-Driven Technology Transfer in Software Engineering
PPTX
Plug-and-Produce based on Standardized Industrie 4.0 Asset Admin Shells
PPTX
IoT challenges for Smart Manufacturing
PPTX
Building IoT Mashups for Industry 4.0 with Eclipse Kura and Kura Wires
PDF
OPC UA: Ready for realtime
OpenPnP: a Plug-and-Produce Architecture for the Industrial Internet of Things
Software Architecture in Process Automation: UML & the "Smart Factory"
Self-commissioning Industrial IoT Systems
Tool-Driven Technology Transfer in Software Engineering
Plug-and-Produce based on Standardized Industrie 4.0 Asset Admin Shells
IoT challenges for Smart Manufacturing
Building IoT Mashups for Industry 4.0 with Eclipse Kura and Kura Wires
OPC UA: Ready for realtime

What's hot (20)

PDF
View Page Update Presentation Close Internet of Things Cologne 2015: OPC Uni...
PDF
Manufacturing IoT - OPC UA Information Revolution
PPTX
Intelligent Energy Systems
PDF
RA TechED 2019 - CL05 Reduce Waste with Logixai
PDF
Eclipse SCADA
PDF
COMPACT PLC WITH HIGH EXPANDABILITY: NX-ERA XPRESS
PDF
Building IoT Mashups for Industry 4.0 with Eclipse Kura and Kura Wires
PDF
RA TechED 2019 - SS14 - Electronic Lockout Tagout Management Systems
PDF
Observations of can Bus Control System
PDF
Industry 4.0 - Advantech Solutions
PDF
Digital Future with OPC UA over TSN
DOC
IEEE 2015 Projects for M.Tech & B.Tech VLSI
PPTX
Automation of calibration of instruments using labview - Project PPT
PPT
Automatizacion de procesos PlantStruxure
PDF
An end-to-end standard oneM2M infrastructure for the Smart Home - Andre Bottaro
PDF
Eclipse Kura Shoot a-pi
PDF
Creating a Java Internet of Things Gateway
PDF
ME490B - SMART BOARD FINAL REPORT
PPTX
OPC PPT
PDF
Catalogo modicon m580 2014
View Page Update Presentation Close Internet of Things Cologne 2015: OPC Uni...
Manufacturing IoT - OPC UA Information Revolution
Intelligent Energy Systems
RA TechED 2019 - CL05 Reduce Waste with Logixai
Eclipse SCADA
COMPACT PLC WITH HIGH EXPANDABILITY: NX-ERA XPRESS
Building IoT Mashups for Industry 4.0 with Eclipse Kura and Kura Wires
RA TechED 2019 - SS14 - Electronic Lockout Tagout Management Systems
Observations of can Bus Control System
Industry 4.0 - Advantech Solutions
Digital Future with OPC UA over TSN
IEEE 2015 Projects for M.Tech & B.Tech VLSI
Automation of calibration of instruments using labview - Project PPT
Automatizacion de procesos PlantStruxure
An end-to-end standard oneM2M infrastructure for the Smart Home - Andre Bottaro
Eclipse Kura Shoot a-pi
Creating a Java Internet of Things Gateway
ME490B - SMART BOARD FINAL REPORT
OPC PPT
Catalogo modicon m580 2014
Ad

Similar to Bottleneck Identification and Performance Modeling of OPC UA Communication Models (20)

PDF
Opc e book_2021_3rd_edition_lay06
PDF
OPC UA Inside Out, Part 1 - Introduction and Playing Field
PDF
OPC UA TSN - A new Solution for Industrial Communication | White Paper
PDF
Transforming IIoT Data Interoperability with OPC UA
PDF
Transforming IIoT Data Interoperability with OPC UA
PDF
OPC UA Inside Out Part 3 - Edge Devices
PDF
OPC UA for Embedded & Constrained Devices
PPTX
OPC UA Inside Out Part 6 - Brownfield and Greenfield Webinar
PPTX
CWIN17 Toulouse / Opc ua, the de facto interoperability standard for industry...
PDF
OPC UA Inside Out Part 4 - OPC Tunneller
PDF
Transition from PROFIBUS to PROFINET Network | Webinar
PPTX
Hannover Messe 2017 - Systems Federation in industrie 4.0
PDF
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
PDF
OPC UA Open Platform Communications.pdf
PDF
Platform independent secure data exchange not only for RFID
PDF
OPC UA Connectivity with InduSoft and the OPC Foundation
PDF
practical-guide-to-opcua.pdf
PDF
A Converged Approach to Standards for Industrial Automation
PDF
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
PDF
OPC UA Inside Out Part 5 - Cloud Connectivity
Opc e book_2021_3rd_edition_lay06
OPC UA Inside Out, Part 1 - Introduction and Playing Field
OPC UA TSN - A new Solution for Industrial Communication | White Paper
Transforming IIoT Data Interoperability with OPC UA
Transforming IIoT Data Interoperability with OPC UA
OPC UA Inside Out Part 3 - Edge Devices
OPC UA for Embedded & Constrained Devices
OPC UA Inside Out Part 6 - Brownfield and Greenfield Webinar
CWIN17 Toulouse / Opc ua, the de facto interoperability standard for industry...
OPC UA Inside Out Part 4 - OPC Tunneller
Transition from PROFIBUS to PROFINET Network | Webinar
Hannover Messe 2017 - Systems Federation in industrie 4.0
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
OPC UA Open Platform Communications.pdf
Platform independent secure data exchange not only for RFID
OPC UA Connectivity with InduSoft and the OPC Foundation
practical-guide-to-opcua.pdf
A Converged Approach to Standards for Industrial Automation
DEVELOPMENT AND IMPLEMENTATION OF LOW COST IIOT GATEWAY WITH EDGE COMPUTING F...
OPC UA Inside Out Part 5 - Cloud Connectivity
Ad

More from Heiko Koziolek (17)

PPTX
Architectural Decision Forces at Work: Experiences in an Industrial Consultan...
PPTX
6 Years of Performance Modeling at ABB
PPTX
Towards the Automation Cloud: Architectural Challenges for a Novel Smart Ecos...
PPTX
Rapid Performance Modeling by transforming Use Case Maps to Palladio Componen...
PPTX
MORPHOSIS: A Case Study on Lightweight Architecture Sustainability Analysis
PDF
Sustainability Evaluation of Software Architectures: A Systematic Review
PPTX
The SPOSAD Architectural Style for Multi-tenant Software Applications
PPTX
2011 05-27-icse
PPTX
ICSE 2011: Q-ImPrESS - An Industrial Case Study on Quality Impact Prediction
PPTX
Towards Software Sustainability Guides for Industrial Software Systems
PPTX
Q-ImPrESS
PPTX
A Large-Scale Industrial Case Study on Architecture-based Software Reliabilit...
PPTX
Towards an Architectural Style for Multi-tenant Software Applications
PPSX
PerOpteryx
PPTX
Evolving Industrial Software Architectures into a Software Product Line: A Ca...
PPTX
A Model Transformation from the Palladio Component Model to Layered Queueing ...
PPTX
Parameter Dependencies for Component Reliability Specifications
Architectural Decision Forces at Work: Experiences in an Industrial Consultan...
6 Years of Performance Modeling at ABB
Towards the Automation Cloud: Architectural Challenges for a Novel Smart Ecos...
Rapid Performance Modeling by transforming Use Case Maps to Palladio Componen...
MORPHOSIS: A Case Study on Lightweight Architecture Sustainability Analysis
Sustainability Evaluation of Software Architectures: A Systematic Review
The SPOSAD Architectural Style for Multi-tenant Software Applications
2011 05-27-icse
ICSE 2011: Q-ImPrESS - An Industrial Case Study on Quality Impact Prediction
Towards Software Sustainability Guides for Industrial Software Systems
Q-ImPrESS
A Large-Scale Industrial Case Study on Architecture-based Software Reliabilit...
Towards an Architectural Style for Multi-tenant Software Applications
PerOpteryx
Evolving Industrial Software Architectures into a Software Product Line: A Ca...
A Model Transformation from the Palladio Component Model to Layered Queueing ...
Parameter Dependencies for Component Reliability Specifications

Recently uploaded (20)

PDF
Architecture types and enterprise applications.pdf
PDF
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
PDF
Taming the Chaos: How to Turn Unstructured Data into Decisions
PPT
What is a Computer? Input Devices /output devices
PPTX
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
PPTX
Chapter 5: Probability Theory and Statistics
PPT
Geologic Time for studying geology for geologist
PDF
A review of recent deep learning applications in wood surface defect identifi...
PDF
Unlock new opportunities with location data.pdf
PDF
Five Habits of High-Impact Board Members
PDF
sustainability-14-14877-v2.pddhzftheheeeee
PPTX
Web Crawler for Trend Tracking Gen Z Insights.pptx
PPTX
The various Industrial Revolutions .pptx
PDF
Developing a website for English-speaking practice to English as a foreign la...
PPTX
Modernising the Digital Integration Hub
PDF
A comparative study of natural language inference in Swahili using monolingua...
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDF
Enhancing emotion recognition model for a student engagement use case through...
PDF
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Architecture types and enterprise applications.pdf
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
Taming the Chaos: How to Turn Unstructured Data into Decisions
What is a Computer? Input Devices /output devices
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
Chapter 5: Probability Theory and Statistics
Geologic Time for studying geology for geologist
A review of recent deep learning applications in wood surface defect identifi...
Unlock new opportunities with location data.pdf
Five Habits of High-Impact Board Members
sustainability-14-14877-v2.pddhzftheheeeee
Web Crawler for Trend Tracking Gen Z Insights.pptx
The various Industrial Revolutions .pptx
Developing a website for English-speaking practice to English as a foreign la...
Modernising the Digital Integration Hub
A comparative study of natural language inference in Swahili using monolingua...
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
Enhancing emotion recognition model for a student engagement use case through...
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Assigned Numbers - 2025 - Bluetooth® Document

Bottleneck Identification and Performance Modeling of OPC UA Communication Models

  • 1. —ABB CORPORATE RESEARCH CENTER GERMANY Bottleneck Identification and Performance Modeling of OPC UA Communication Models Andreas Burger, Heiko Koziolek, Julius Rückert, Marie Platenius-Mohr, Gösta Stomberg PUBLIC
  • 2. — Motivation Towards an Industrial Internet of Things (IIoT) ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 2 Industrial controllers Servers Operator workstations Temperature sensor Pump Motor & drive Field instrument Flow meter Field comm. interface Cloud Smart sensor IIoT 14 BUSD annual market > 70.000 control systems 70 million connected devices
  • 3. — Open Platform Communications Unified Architecture (OPC UA) – Evolves towards universal communication standard in industrial automation – Recently communication model added for field level communication Motivation ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 3 OPC UA: Standardizing M2M Communication Industrial controllers Servers Operator workstations Temperature sensor Pump Motor & drive Field instrument Flow meter Field comm. interface *Picture source: Eckhardt et al.: An Evaluation of the Applicability of OPC UA Publish Subscribe on Factory Automation use Cases. In IEEE Conference on Emerging Technologies and Factory Automation (ETFA), 2018. OPC UA OPC UA Fieldbus protocols OPC UA OPC UA S/P OPC UA S+P OPC UA S+P OPC UA S/P OPC UA S+P Server + Client Server + Client Server + Client OPC UA Client New OPC UA Communication Model *
  • 4. — Motivation ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 4 Client/server Publish/subscribe (pub/sub) So far, missing experience and guidelines on use of OPC UA pub/sub in industrial applications OPC UA Communication Models OPC UA Server OPC UA Client OPC UA Publisher Subscriber Subscriber SubscriberTCP/IP connection IP Multicast OPC UA Client Acyclic & cyclic data exchange OPC UA Publisher Subscriber Subscriber Subscriber Broker Cyclic data exchange Broker-basedBroker-less Session Session
  • 5. — Motivation ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 5 Client/server Publish/subscribe (pub/sub) Contribution: Measurement study on bottlenecks & proposal of performance model Potential Bottlenecks OPC UA Server OPC UA Client OPC UA Client OPC UA Client OPC UA Client CPU Mem Network OPC UA Publisher Subscriber Subscriber Subscriber Subscriber CPU Mem CPU Mem CPU Mem CPU Mem Network Potential bottleneck IP MulticastTCP/IP CPU Mem
  • 6. — Measurement Study ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 6 Testbed Setup and Configuration Industrial-grade managed Ethernet switches, with IP multicast support Monitor & control laptop Server / Publisher Client / Subscriber (2x Raspberry Pi 3, Model B, Quad Core 1.2GHz 64bit CPU, 1GB RAM, RTLinux) (Raspberry Pi Zero, 1GHz single-core CPU, 512MB RAM, RTLinux) Hirschmann switch RSP35 (2x Raspberry Pi 3, Model B, Quad Core 1.2GHz 64bit CPU, 1GB RAM, RTLinux) Client / Subscriber Client / Subscriber Client / Subscriber Industrial controllers Servers Operator workstations Temperature sensor Pump Motor & drive Field instrument Flow meter Field Comm. Interface
  • 7. — Measurement Study ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 7 Two sets of software clients developed Performance measurements Implementation and Instrumentation OPC UA Server OPC UA Client OPC UA Publisher OPC UA Subscriber OPC UA SDK OPC UA SDK Sensor values Server/ Publisher Client/ Subscriber Memory NetworkCPU Network traffic Memory CPU Network traffic Measurements For measurements: Randomly generated signal values OPC UA C++ SDK by Unified Automation + pub/sub extension OPC UA C++ SDK by Unified Automation + pub/sub extension
  • 8. — Performance Measurements ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 8 CPU Utilization Pub/sub: 40,000 signals/s Client/server (1 client): 25,000 signals/s Client/server (30 clients): 10,000 signals/sExemplary control use case – 6,000 I/O points with 100ms updates (= 60,000 signals/s) – 2 controllers share 8 field communication interfaces (FCIs) Industrial controllers Sensors & actuators Field comm. interfaces … … ……… … …
  • 9. — Performance Measurements ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 9 Memory & Network Utilization Overall <4% of Raspberry Pi Zero’s main memory Overall <1% of switch capacity and <10% of network interface capacity Pub/sub: lower bandwidth and independent of clients
  • 10. — Performance Measurements ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 10 Client/server Publish/subscribe (pub/sub) Result: Server/publisher CPU utilization main bottleneck Bottleneck Identification OPC UA Server OPC UA Client OPC UA Client OPC UA Client OPC UA Client CPU Mem Network OPC UA Publisher Subscriber Subscriber Subscriber Subscriber CPU Mem CPU Mem CPU Mem CPU Mem Network Potential bottleneck (future work) IP MulticastTCP/IP CPU Mem No bottleneck Identified bottleneck
  • 11. — Performance Model ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 11 Vision for Model Application Performance Model (Control platform specific) New System Design (E.g. planned control system) Performance Prediction (Expected CPU utilization) Target Control Platform (Hardware & Software) Assess feasibility, choose comm. model, size hardware Receives/defines requirements for new control system
  • 12. — Performance Model ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 12 Model Construction Performance Model Predicted CPU utilization (bottleneck resource) CPU Model parameters (features) ns: number of signals spr: signal publishing rate nc: number of clients feature transformation weight vector feature vector predicted CPU utilization Linear regression w/ Maximum Likelihood estimation
  • 13. — Model calibration specific to target hardware/software platform Performance Model ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 13 Model considerations Feature transformation Transform feature vector 𝑥 using nonlinear feature transformation – Feature engineering • Domain knowledge • Comparison of transformations – Considers linear dependencies • Result of iteratively tuning feature contributions Training the model Weight vector 𝑤 calibrates model – Inferred from experiment data – Maximum likelihood linear regression • [Φ] 𝑇 : Gram matrix of transformed feature vectors as columns • 𝑡 : vector of measured CPU load values Model Construction for Client/Server Model feature transformation weight vector feature vector predicted CPU utilization Model parameters (features) ns: number of signals spr: signal publishing rate nc: number of clients
  • 14. — Performance Model ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 14 Trained vs. predicted (sanity check) – Model trained with 10 client/server connections & publishing rate of 100ms  Model closely reflects training data Applied to 5 client connections – Prediction for 5 connections  Good match to measured data Applied to 20 client connections – Prediction for 20 connections  Measurements shows step, but still considered useful for predictions Validation of Model Accuracy
  • 15. — Use case system parameters – 6,000 I/O points with 100ms updates, 2 process controllers – Controllers share 8 field communication interfaces (FCIs)  Can each controller handle 3,000 I/O points via 4 connections?  Can each FCI handle 750 I/O points via 2 connections? Performance Model ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 15 Application to Hardware Sizing Use Case Limit for RP Zero Industrial controllers Sensors & actuators Field comm. interfaces … … ……… … … 750 3000
  • 16. — Conclusion ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 16 OPC UA Measurement and Modeling Quantified benefit of OPC UA pub/sub Proposed practical performance model Gained experience w/ pub/sub model Showed feasibility for practical use cases Identified potential for future work IIoT using OPC UA Pub/Sub Contribution towards IIoT vision
  • 18. — References ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 18 [8] W. Mahnke, S.-H. Leitner, and M. Damm. 2009. OPC Unified Architecture. Springer Science & Business Media. [9] OPC Foundation. 2018. OPC Unified Architecture Specification Part 14: PubSub. [10] J. Pfrommer, A. Ebner, S. Ravikumar, and B. Karunakaran. 2018. Open Source OPC UA PubSub over TSN for Realtime Industrial Communication. In IEEE Emerging Technologies in Factory Automation (ETFA). 1–4. [11] M. Silveira Rocha, G. Serpa Sestito, A. Luis Dias, A. Celso Turcato, and D. Brandão. 2018. Performance Comparison between OPC UA and MQTT for Data Exchange. In 2018 Workshop on Metrology for Industry 4.0 and IoT. 175–179. https://doi.org/10.1109/METROI4.2018.8428342 [12] P. Runeson and M. Höst. 2009. Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empirical Software Engineering 14, 2 (2009), 131. [13] M. Schleipen, S.-S. Gilani, T. Bischoff, and J. Pfrommer. 2016. OPC UA & Industrie 4.0 – Enabling Technology with High Diversity and Variability. Procedia CIRP 57, 1 (2016), 315–320. [1] C. M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York. [2] D. Bruckner, R. Blair, M-P. Stanica, A. Ademaj, W. Skeffington, and D. Kutscher. 2018. OPC UA TSN A new Solution for Industrial Communication. Technical Report. Shapers Group White Paper. 1–10 pages. https://www.automationworld.com/opc-ua-tsn-new-solution-industrial- communication [3] B. Cain, S. Deering, I. Kouvelas, B. Fenner, and A. Thyagarajan. 2002. Internet Group Management Protocol, Version 3. RFC 3376. https://tools.ietf.org/html/rfc3376 [4] S. Cavalieri and F. Chiacchio. 2013. Analysis of OPC UA Performances. Computer Standards & Interfaces 36, 1 (2013), 165–177. [5] M. Fojcik and K. Folkert. 2012. Introduction to OPC UA Performance. In International Conference on Computer Networks. Springer, 261–270. [6] S. Grüner, J. Pfrommer, and F. Palm. 2016. RESTful Industrial Communication with OPC UA. IEEE Transactions on Industrial Informatics 12, 5 (2016), 1832– 1841. [7] H. Koziolek, A. Burger, and J. Doppelhamer. 2018. Self-Commissioning Industrial IoT-Systems in Process Automation: A Reference Architecture. In 2018 IEEE Internat. Conf. on Software Architecture (ICSA). IEEE, 196–19609.
  • 19. — Author Focus Mahnke et al. [8] Rudimentary performance analysis focused on round-trip times, comparing classic OPC, UA Binary, SOAP/XML Cavalieri et al. [4] OPC UA round-trip times for different message sizes, delay for subscriptions with short update intervals, round-trip time overhead caused by encryption Fojcik et al. [5] Discuss basic performance parameters for OPC , performance measurements using Prosys OPC UA SDK on powerful PCs Gruener et al. [6] Propose RESTful extension for OPC UA and measured round-trip times for different number of read requests per session Rocha et al. [11] Performance measurements of MQTT and OPC UA in cloud-to-client setting “Shaper group” [2] Measurements regarding time synchronization between nodes in a time-sensitive network setting; theoretical calculation on minimum achievable cycle times (relevant for OPC UA Pub/Sub although not discussed) Pfrommer et al. [10] Jitter measurements on OPC UA Pub/Sub using open62541 SDK Koziolek et al [7] Performance measurements for OPC UA Pub/Sub with the focus on encoding overhead Related Work ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 19 Overview So far, no systematic performance measurements to guide use of OPC UA pub/sub in industrial setting
  • 20. — Performance Measurements ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 20 Bottleneck Identification CPU utilization main bottleneck for client/server and pub/sub model Max. CPU utilization: 80-90% Max. memory usage: < 4% Max. network utilization: < 1%
  • 21. — Performance Measurements ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 21 – Implemented multi- threaded signal updating using POSIX threads – For Raspberry Pi Zero, no performance gain was possible – For Raspberry Pi 3, a 35% performance gain was possible, enabling higher signal rates – Evaluated security impact only for client/ server model – Using state-of-the-art encryption policy Basic256Sha256 – Highest overhead of about 10% for higher number of clients – Security policy should be always considered Security overhead and multithreading RQ5: How much overhead in terms of CPU utilization is caused by security policies? RQ6: What is the impact of multithreading on the server performance?
  • 22. — Potential Threats to Validity ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 22 Construct validity – Raspberry Pi devices as substitutes for industrial embedded systems – Debian Linux with the RT PREEMPT patch we used a soft real-time operating system (RTOS) instead of commercial RTOS – Chosen OPC UASDK from Unified Automation – Used update intervals of 50ms or higher in our experiments – Focus on IGMP-based multicast for pub/sub model Internal validity – Beta-status of the chosen OPC UA SDK (not performance optimized) – Additional bottlenecks could hide in the network for different client information need profiles – Chosen simple maximum likelihood linear regression model External validity – Representativeness of results for real control application profiles – Missing comparison of pub/sub performance results to other SDKs or M2M middlewares Overview of considered threads
  • 23. — Overall, our results show how performance models can guide and improve the design of real control systems Conclusion ICPE 2019 - Bottleneck Identification and Performance Modeling of OPC UA Communication ModelsSlide 23 Performance measurements – CPU utilization identified as performance bottleneck for OPC UA client/server and pub/sub model – Memory and network overhead observed to be negligible and not a bottleneck – Low overhead added by security policies – Implementation can benefit from multithreading Performance model – Linear regression model: simple model but already provided interesting insights and valuable for hardware sizing – Proposed models considered useful for practitioners and researchers designing industrial automation systems – Considered especially useful for industrial Internet-of-Things applications with many OPC UA communication partners Potential future work – Extend measurements study to deepen understanding on OPC UA performance • Different hardware platforms • Additional application profiles – Improve performance model and make it applicable in even more situations • Queuing network • Neural network Summary, Take Away, and Future Work

Editor's Notes

  • #3: 14 BUSD annual market and growing ABB install base: > 70 mio. connected devices > 70.000 control systems Source: https://new.abb.com/news/detail/245/abb-and-microsoft-partner-to-drive-digital-industrial-transformation
  • #4: Initially designed for server-to-workstation communication Recently enhanced for field device communication with short cycle times on resource-constraint devices*
  • #9: Recorded measurements from the first twenty seconds of each experiment are discarded to limit the analysis to the stable phase and ignore transient effects. All experiments were repeated at least five times to rule out random effects caused by the testbed setup and, e.g., operating system processes. We report results in the form of average values over the individual measurement repetitions. Configuration: 100ms target publishing rate C/S: More than 30 client sessions led to error messages issued by the server, therefore, these measurements could not be finalized
  • #10: The Raspberry Pi Zero has 512MB of RAM Same scenarios as for the CPU utilization (i.e., 100 ms update interval) Observations for both communication models Memory consumption of OPC UA server observed to be between 11 and 15 MB in all scenarios Slight increase with increasing signal rates Larger signals could cause higher consumption but case considered rare for industrial automation applications Setting results in perspective The Raspberry Pi Zero device has 512MB of main memory Observed memory usage is less than 4% of available memory  Memory was not the bottleneck in any experiment Higher network utilizations may occur at the client side if multiple servers send as much data as they can to single clients. For pub/sub communication, dynamic multicast filtering assures that packets are only sent to other nodes if they have subscribed to the respective multicast addresses. Otherwise, the packets are not forwarded by the network switch, therefore reducing total network traffic. Publish/subscribe model Pub/sub uses lowest amount of server bandwidth (IP multicast) Bandwidth usage independent of number of clients and slightly above 1 Mbps for high signal rates Client/server model Steeper increase (per-client sessions) but overall below 6 Mbps Setting results in perspective For used 1 Gbps network switch, bandwidth usage of less than 1% or less than 10% for 100 Mbps network interfaces  Network bandwidth not the bottleneck in any experiment
  • #11: Bottleneck shown for RP Zero, where it was visible the best but also applies to other tested platforms.
  • #12: Going beyond individual measurements Derive performance model based on measurements Predict performance for untested system configurations Control platform specific model Specific to control hardware/software measured Calibration of model for other platforms based on set of additional measurements Benefit: Integration of model in system development Check feasibility of design assumptions Select communication model Derive hardware requirements
  • #13: Model considerations Considered continuous output variable from parameters Model selection: Mainly linear trends in measurement results Selected linear regression with Maximum Likelihood estimation as starting point
  • #14: Note feature transformation: Shown here: feature transformation for client/server model Influence of the number of subscribing clients was stepwise subdued using an exponent which greatly improved the prediction accuracy. A model for pub/sub communication could be derived in a similar manner but is not presented here. Training the model Maximum likelihood linear regression (= least squares applied to the feature transformations)
  • #16: In this example: Load for controller considered too high with 3,000 signals and 4 connections each FCIs could handle use case load
  • #23: See paper for discussion