Intelligent Human Systems Integration 2020
Uploaded by
Tatiana DianaIntelligent Human Systems Integration 2020
Uploaded by
Tatiana DianaAdvances in Intelligent Systems and Computing 1131
Tareq Ahram · Waldemar Karwowski ·
Alberto Vergnano · Francesco Leali ·
Redha Taiar Editors
Intelligent
Human Systems
Integration 2020
Proceedings of the 3rd International
Conference on Intelligent Human
Systems Integration (IHSI 2020):
Integrating People and Intelligent
Systems, February 19–21, 2020,
Modena, Italy
Advances in Intelligent Systems and Computing
Volume 1131
Series Editor
Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences,
Warsaw, Poland
Advisory Editors
Nikhil R. Pal, Indian Statistical Institute, Kolkata, India
Rafael Bello Perez, Faculty of Mathematics, Physics and Computing,
Universidad Central de Las Villas, Santa Clara, Cuba
Emilio S. Corchado, University of Salamanca, Salamanca, Spain
Hani Hagras, School of Computer Science and Electronic Engineering,
University of Essex, Colchester, UK
László T. Kóczy, Department of Automation, Széchenyi István University,
Gyor, Hungary
Vladik Kreinovich, Department of Computer Science, University of Texas
at El Paso, El Paso, TX, USA
Chin-Teng Lin, Department of Electrical Engineering, National Chiao
Tung University, Hsinchu, Taiwan
Jie Lu, Faculty of Engineering and Information Technology,
University of Technology Sydney, Sydney, NSW, Australia
Patricia Melin, Graduate Program of Computer Science, Tijuana Institute
of Technology, Tijuana, Mexico
Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro,
Rio de Janeiro, Brazil
Ngoc Thanh Nguyen , Faculty of Computer Science and Management,
Wrocław University of Technology, Wrocław, Poland
Jun Wang, Department of Mechanical and Automation Engineering,
The Chinese University of Hong Kong, Shatin, Hong Kong
The series “Advances in Intelligent Systems and Computing” contains publications
on theory, applications, and design methods of Intelligent Systems and Intelligent
Computing. Virtually all disciplines such as engineering, natural sciences, computer
and information science, ICT, economics, business, e-commerce, environment,
healthcare, life science are covered. The list of topics spans all the areas of modern
intelligent systems and computing such as: computational intelligence, soft comput-
ing including neural networks, fuzzy systems, evolutionary computing and the fusion
of these paradigms, social intelligence, ambient intelligence, computational neuro-
science, artificial life, virtual worlds and society, cognitive science and systems,
Perception and Vision, DNA and immune based systems, self-organizing and
adaptive systems, e-Learning and teaching, human-centered and human-centric
computing, recommender systems, intelligent control, robotics and mechatronics
including human-machine teaming, knowledge-based paradigms, learning para-
digms, machine ethics, intelligent data analysis, knowledge management, intelligent
agents, intelligent decision making and support, intelligent network security, trust
management, interactive entertainment, Web intelligence and multimedia.
The publications within “Advances in Intelligent Systems and Computing” are
primarily proceedings of important conferences, symposia and congresses. They
cover significant recent developments in the field, both of a foundational and
applicable character. An important characteristic feature of the series is the short
publication time and world-wide distribution. This permits a rapid and broad
dissemination of research results.
** Indexing: The books of this series are submitted to ISI Proceedings,
EI-Compendex, DBLP, SCOPUS, Google Scholar and Springerlink **
More information about this series at http://www.springer.com/series/11156
Tareq Ahram Waldemar Karwowski
• •
Alberto Vergnano Francesco Leali
• •
Redha Taiar
Editors
Intelligent Human Systems
Integration 2020
Proceedings of the 3rd International
Conference on Intelligent Human Systems
Integration (IHSI 2020): Integrating People
and Intelligent Systems, February 19–21, 2020,
Modena, Italy
123
Editors
Tareq Ahram Waldemar Karwowski
Institute for Advanced University of Central Florida
Systems Engineering Orlando, FL, USA
University of Central Florida
Orlando, FL, USA Francesco Leali
Department of Mechanical
Alberto Vergnano and Civil Engineering
Dipartimento Di Ingegneria, Edificio 25 University of Modena
Università di Modena e Reggio Emilia Modena, Modena, Italy
Modena, Modena, Italy
Redha Taiar
GRESPI
Université de Reims Champagne-Ardenne
Reims Cedex 2, France
ISSN 2194-5357 ISSN 2194-5365 (electronic)
Advances in Intelligent Systems and Computing
ISBN 978-3-030-39511-7 ISBN 978-3-030-39512-4 (eBook)
https://doi.org/10.1007/978-3-030-39512-4
© Springer Nature Switzerland AG 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
This volume, entitled Intelligent Human Systems Integration 2020, aims to provide
a global forum for introducing and discussing novel approaches, design tools,
methodologies, techniques, and solutions for integrating people with intelligent
technologies, automation, and artificial cognitive systems in all areas of human
endeavor in industry, economy, government, and education. Some of the notable
areas of application include, but are not limited to, energy, transportation, urban-
ization and infrastructure development, digital manufacturing, social development,
human health, sustainability, a new generation of service systems, as well as
developments in safety, risk assurance, and cybersecurity in both civilian and
military contexts. Indeed, rapid progress in developments in the ambient intelli-
gence, including cognitive computing, modeling, and simulation, as well as smart
sensor technology, weaves together the human and artificial intelligence and will
have a profound effect on the nature of their collaboration at both the individual and
societal levels in the near future.
As applications of artificial intelligence and cognitive computing become more
prevalent in our daily lives, they also bring new social and economic challenges and
opportunities that must be addressed at all levels of contemporary society. Many
of the traditional human jobs that require high levels of physical or cognitive
abilities, including human motor skills, reasoning, and decision-making abilities, as
well as training capacity, are now being automated. While such trends might boost
economic efficiency, they can also negatively impact the user experience and bring
about many unintended social consequences and ethical concerns.
The intelligent human systems integration is, to a large extent, affected by the
forces shaping the nature of future computing and artificial system development.
This book discusses the needs and requirements for the symbiotic collaboration
between humans and artificially intelligent systems, with due consideration of the
software and hardware characteristics allowing for such cooperation from the
societal and human-centered design perspectives, with the focus on the design of
intelligent products, systems, and services that will revolutionize future human–
technology interactions. This book also presents many innovative studies of
ambient artificial technology and its applications, including the human–machine
v
vi Preface
interfaces with a particular emphasis on infusing intelligence into the development
of technology throughout the lifecycle development process, with due consideration
of user experience and the design of interfaces for virtual, augmented, and mixed
reality applications of artificial intelligence.
Reflecting on the above-outlined perspective, the papers contained in this vol-
ume are organized into seven main sections, including:
1. Automotive design and transportation engineering
2. Humans and artificial cognitive systems
3. Intelligence, technology, and analytics
4. Computational modeling and simulation
5. Humans and artificial systems complexity
6. Materials and inclusive human systems
7. Human–autonomy teaming
8. Applications and future trends
We would like to extend our sincere thanks to Axel Schulte, Stefania Campione,
and Marinella Ferrara, for leading a part of the technical program that focuses on
human–autonomy teaming and smart materials and inclusive human systems. Our
appreciation also goes to the members of Scientific Program Advisory Board who
have reviewed the accepted papers that are presented in this volume, including the
following individuals:
D. Băilă, Romania
H. Blaschke, Germany
S. Campione, Italy
J. Chen, USA
G. Coppin, France
M. Draper, USA
A. Ebert, Germany
M. Ferrara, Italy
M. Hou, Canada
M. Jipp, Germany
E. Karana, The Netherlands
A. Kluge, Germany
D. Lange, USA
S. Lucibello, Italy
E. Macioszek, Poland
M. Neerincx, The Netherlands
R. Philipsen, Germany
J. Platts, UK
D. Popov, USA
A. Ratti, Italy
R. Rodriquez, Italy
V. Rognoli, Italy
U. Schmid, Germany
Preface vii
A. Schulte, Germany
N. Stanton, UK
E. Suhir, USA
We hope that this book, which presents the current state of the art in intelligent
human systems integration, will be a valuable source of both theoretical and applied
knowledge, enabling the design and applications of a variety of intelligent products,
services, and systems for their safe, effective, and pleasurable collaboration with
people.
Contents
Automotive Design and Transportation Engineering
User-Centered Design Within the Context of Automated Driving
in Trucks – Guideline and Methods for Future Conceptualization
of Automated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Paula Laßmann, Florian Reichelt, Dominique Stimm, and Thomas Maier
Towards Probabilistic Analysis of Human-System Integration
in Automated Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Ephraim Suhir, Gunther Paul, and Hermann Kaindl
Trust Provisioning in the Transport Infrastructure . . . . . . . . . . . . . . . . 15
Scott Cadzow
Drivers’ Interaction with, and Perception Toward Semi-autonomous
Vehicles in Naturalistic Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Jisun Kim, Kirsten Revell, Pat Langdon, Mike Bradley, Ioannis Politis,
Simon Thompson, Lee Skrypchuk, Jim O-Donoghue, Joy Richardson,
Jed Clark, Aaron Roberts, Alex Mouzakitis, and Neville A. Stanton
Are Autonomous Vehicles the Solution to Drowsy Driving? . . . . . . . . . . 27
Daniel Grunstein and Ron Grunstein
Exploring New Concepts to Create Natural and Trustful Dialogue
Between Humans and Intelligent Autonomous Vehicles . . . . . . . . . . . . . 34
Andrea Di Salvo and Andrea Arcoraci
Integrating Human Acceptable Morality in Autonomous Vehicles . . . . . 41
Giorgio M. Grasso, Chiara Lucifora, Pietro Perconti, and Alessio Plebe
The Future of User Experience Design in the Interior
of Autonomous Car Driven by AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Laura Giraldi
ix
x Contents
Measuring Driver Discomfort in Autonomous Vehicles . . . . . . . . . . . . . 52
Dario Niermann and Andreas Lüdtke
Human-Centered Design for Automotive Styling Design:
Conceptualizing a Car from QFD to ViP . . . . . . . . . . . . . . . . . . . . . . . . 59
Gian Andrea Giacobone and Giuseppe Mincolelli
Enriching the User Experience of a Connected Car
with Quantified Self . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Maurizio Caon, Marc Demierre, Omar Abou Khaled, Elena Mugellini,
and Pierre Delaigue
Constructing a Mental Model of Automation Levels
in the Area of Vehicle Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Larissa Zacherl, Jonas Radlmayr, and Klaus Bengler
Effect of Phone Interface Modality on Drivers’ Task Load Index
in Conventional and Semi-Automated Vehicles . . . . . . . . . . . . . . . . . . . 80
Kristina Davtyan and Francesca Favaro
Software Failure Mode and Effects Analysis . . . . . . . . . . . . . . . . . . . . . 86
Palak Talwar
A Validated Failure Behavior Model for Driver Behavior Models
for Generating Skid-Scenarios on Motorways . . . . . . . . . . . . . . . . . . . . 92
Bernd Huber, Paul Schmidl, Christoph Sippl, and Anatoli Djanatliev
Human-Machine Interface Research of Autonomous Vehicles Based
on Cognitive Work Analysis Framework . . . . . . . . . . . . . . . . . . . . . . . . 99
Chi Zhang, Goudong Yin, and Zheng Wu
Mercury: A Vision-Based Framework for Driver Monitoring . . . . . . . . 104
Guido Borghi, Stefano Pini, Roberto Vezzani, and Rita Cucchiara
Investigating the Impact of Time-Lagged End-to-End Control
in Autonomous Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Haruna Asai, Yoshihiro Hashimoto, and Giuseppe Lisi
The Car as a Transformer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Jeremy Aston and Rui Pedro Freire
Unmanned Small Shared Electric Vehicle . . . . . . . . . . . . . . . . . . . . . . . 124
Binhong Zhai, Guodong Yin, and Zhen Wu
A Forward Train Detection Method Based on Convolutional
Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Zhangyu Wang, Tony Lee, Michael Leung, Simon Tang, Qiang Zhang,
Zining Yang, and Virginia Cheung
Contents xi
Styling Research of DFAC-6851H4E City Bus Based
on Fuzzy Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Huajie Wang and Jianxin Cheng
Object Detection to Evaluate Image-to-Image Translation
on Different Road Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Fumiya Sudo, Yoshihiro Hashimoto, and Giuseppe Lisi
Humans and Artificial Cognitive Systems
Modelling Proxemics for Human-Technology-Interaction
in Decentralized Social-Robot-Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Thomas Kirks, Jana Jost, Jan Finke, and Sebastian Hoose
Category Learning as a Use Case for Anticipating Individual
Human Decision Making by Intelligent Systems . . . . . . . . . . . . . . . . . . . 159
Marcel Lommerzheim, Sabine Prezenski, Nele Russwinkel,
and André Brechmann
System Architecture of a Human Biosensing and Monitoring
Suite with Adaptive Task Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Brandon Cuffie and Lucas Stephane
The Role of Artificial Intelligence in Contemporary Medicine . . . . . . . . 172
Larisa Hambardzumyan, Viktoria Ter-Sargisova,
and Aleksandr Baghramyan
Improving Policy-Capturing with Active Learning
for Real-Time Decision Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Bénédicte Chatelais, Daniel Lafond, Alexandre Hains,
and Christian Gagné
Task Measures for Air Traffic Display Operations . . . . . . . . . . . . . . . . . 183
Shi Yin Tan, Chun Hsien Chen, Sun Woh Lye, and Fan Li
Identifying People Based on Machine Learning Classification of Foods
Consumed in Order to Offer Tailored Healthier Food Options . . . . . . . 190
Jenna Kim, Shuhao Lin, Giannina Ferrara, Jenna Hua, and Edmund Seto
On the Perception of Disharmony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Stijn Verwulgen, Thomas Peeters, Sander Van Goethem,
and Sofia Scataglini
Mobile Real-Time Eye-Tracking for Gaze-Aware Security
Surveillance Support Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Alexandre Marois, Daniel Lafond, François Vachon, Eric R. Harvey,
Bruno Martin, and Sébastien Tremblay
xii Contents
Detecting Impulsive Behavior Through Agent-Based Games . . . . . . . . . 208
Alia El Bolock, Ahmed Ghonaim, Cornelia Herbert,
and Slim Abdennadher
Visual and Motor Capabilities of Future Car Drivers . . . . . . . . . . . . . . 214
Ferdinando Tripi, Rita Toni, Angela Lucia Calogero,
Pasqualino Maietta Latessa, Antonio Tempesta, Stefania Toselli,
Alessia Grigoletto, Davide Varotti, Francesco Campa, Luigi Manzoni,
and Alberto Vergnano
A Fixation-Click Count Signature as a Visual Monitoring
Enhancement Feature for Air Traffic Controllers . . . . . . . . . . . . . . . . . 221
Hong Jie Wee, Sun Woh Lye, and Jean-Philippe Pinheiro
Digital Transformation in Product Service System for Kids.
Design Tools for Emerging Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Benedetta Terenzi and Arianna Vignati
A Novel Heuristic Mechanism to Formalize Online Behavior
Through Search Engine Credibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Debora Di Caprio and Francisco J. Santos-Arteaga
Caterina, Alexa and the Others . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Elisabetta Benelli and Jurji Filieri
Event-Related Potential Study on Military Icon Based
on Composition-Semantic Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Xian Li, Haiyan Wang, and Junkai Shao
Ekybot: Framework Proposal for Chatbot in Financial Enterprises . . . 254
Maritzol Tenemaza, Sergio Luján-Mora, Angélica de Antonio,
Jaime Ramírez, and Omar Zarabia
Alpha and Beta EEG Desynchronizations Anticipate Steering Actions
in a Driving Simulation Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
Giovanni Vecchiato, Maria Del Vecchio, Sergey Antopolskiy,
Andrea Bellotti, Alessia Colucciello, Anna Marchenkova,
Jonas Ambeck-Madsen, Luca Ascari, and Pietro Avanzini
The Quantitative Evaluation of Permanent Disability in Forensic
Medicine Through Stereo Photogrammetric Technology . . . . . . . . . . . . 266
Claudia Trignano, Andrea Castelli, Vittorio Dell‘Orfano,
and Elena Mazzeo
A Unified Framework for Symbol Grounding
in Human-Machine Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Dingzhou Fei
Contents xiii
Improving Machine Translation Output of German Compound
and Multiword Financial Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Christina Valavani, Christina Alexandris, and George Mikros
Self-adjusted Data-Driven System for Prediction
of Human Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Oleksandr Burov, Evgeniy Lavrov, Nadiia Pasko, Olena Hlazunova,
Olga Lavrova, Vasyl Kyzenko, and Yana Dolgikh
Human Factor and Cognitive Methods in the Design of Products
and Production Systems of Mechanical Engineering in the Framework
of NBIC Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Evgeny Kolbachev, Elena Sidorova, and Polina Vaneeva
Automatic Assessment System of Operators’ Risk in Order Picking
Process for Task Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Yangxu Li, Bach Q. Ho, Tatsunori Hara, and Jun Ota
TARS Mobile App with Deep Fingertip Detector
for the Visually Impaired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Tetsushi Miwa, Youichi Hosokawa, Yoshihiro Hashimoto,
and Giuseppe Lisi
Analysis Process of Exploratory Research Represented
in a Coordinate System XYZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
Olga Popova, Boris Popov, Vladimir Karandey, and Vladimir Afanasyev
A Variety of Visual-Speech Matching ERP Studies
in Quiet-Noise Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Lingling Hu, Chengqi Xue, and Junkai Shao
Research on Color Stratification in Dynamic Environment:
Frequency Domain Analysis of Delta, SMR and Theta
EEG Rhythms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
Cheng Guan, Lei Zhou, Tongtong Zhang, and Xiang Zeng
Human AI Symbiosis: The Role of Artificial Intelligence
in Stratifying High-Risk Outpatient Senior Citizen Fall Events
in a Non-connected Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Chandrasekar Vuppalapati, Anitha Ilapakurti, Sharat Kedari,
Rajasekar Vuppalapati, Jayashankar Vuppalapati, and Santosh Kedari
Intelligence, Technology and Analytics
Distinguishing a Human or Machine Cyberattacker . . . . . . . . . . . . . . . 335
Wayne Patterson, Acklyn Murray, and Lorraine Fleming
Using Eye Tracking to Assess User Behavior in Virtual Training . . . . . 341
Mina Fahimipirehgalin, Frieder Loch, and Birgit Vogel-Heuser
xiv Contents
Democratization of AI to Small Scale Farmers, Albeit Food
Harvesting Citizen Data Scientists, that Are at the Bottom
of the Economic Pyramid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
Chandrasekar Vuppalapati, Anitha Ilapakurti, Sharat Kedari,
Rajasekar Vuppalapati, Jayashankar Vuppalapati, and Santosh Kedari
Cybersecurity in Educational Networks . . . . . . . . . . . . . . . . . . . . . . . . . 359
Oleksandr Burov, Svitlana Lytvynova, Evgeniy Lavrov,
Yuliya Krylova-Grek, Olena Orliyk, Sergiy Petrenko,
Svitlana Shevchenko, and Oleksii M. Tkachenko
The Problem of Tracking the Center of Attention
in Eye Tracking Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
Marina Boronenko, Vladimir Zelensky, Oksana Isaeva,
and Elizaveta Kiseleva
Health Risk Assessment Matrix for Back Pain Prediction
Among Call Center Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
Sunisa Chaiklieng and Pornnapa Suggaravetsiri
Towards Conceptual New Product Development Framework
for Latvian ICT Sector Companies and Startups . . . . . . . . . . . . . . . . . . 379
Didzis Rutitis and Tatjana Volkova
A Liveness Detection Method for Palmprint Authentication . . . . . . . . . 385
Ayaka Sugimoto, Yuya Shiomi, Akira Baba, Norihiro Okui, Tetushi Ohki,
Yutaka Miyake, and Masakatsu Nishigaki
Procedure for the Implementation of the Manufacturing Module
of an ERP System in MSME. Applied Case: Textile “Tendencias”
Enterprise, UDA ERP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
Pedro Mogrovejo, Juan Manuel Maldonado-Maldonado,
Esteban Crespo-Martínez, and Catalina Astudillo
Model of Emotionally Stained Pupillogram Plot . . . . . . . . . . . . . . . . . . . 398
Marina Boronenko, Yurii Boronenko, Oksana Isaeva,
and Elizaveta Kiseleva
Cost-Informed Water Decision-Making Technology
for Smarter Farming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
Joanne Tingey-Holyoak, John Dean Pisaniello, Peter Buss,
and Ben Wiersma
A Review on the Role of Embodiment in Improving Human-Vehicle
Interaction: A Proposal for Further Development
of Embodied Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
Hamid Naghdbishi and Alireza Ajdari
Contents xv
Analysis of Topological Relationships of Human . . . . . . . . . . . . . . . . . . 415
Jia Zhou, Xuebo Chen, and Zhigang Li
Impact of Technological Innovation on the Productivity
of Manufacturing Companies in Peru . . . . . . . . . . . . . . . . . . . . . . . . . . 421
Julio César Ortíz Berrú, Cristhian Aldana Yarlequé,
and Lucio Leo Verástegui Huanca
Parametric Urban Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
Rongrong Gu and Wuzhong Zhou
Narrative Review of the Role of Wearable Devices in Promoting
Health Behavior: Based on Health Belief Model . . . . . . . . . . . . . . . . . . 433
Dingzhou Fei and Xia Wang
Competitiveness of Higher Education System as a Sector of Economy:
Conceptual Model of Analysis with Application to Ukraine . . . . . . . . . . 439
Olha Hrynkevych, Oleg Sorochak, Olena Panukhnyk, Nazariy Popadynets,
Rostyslav Bilyk, Iryna Khymych, and Yazina Viktoriia
Application of Classification Algorithms in the Generation
of a Network Intrusion Detection Model Using
the KDDCUP99 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
Jairo Hidalgo and Marco Yandún
Vulnerability Discovery in Network Systems Based
on Human-Machine Collective Intelligence . . . . . . . . . . . . . . . . . . . . . . . 453
Ye Han, Jianfeng Chen, Zhihong Rao, Yifan Wang, and Jie Liu
Computational Modeling and Simulation
Supporting Decisions in Production Line Processes by Combining
Process Mining and System Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 461
Mahsa Pourbafrani, Sebastiaan J. van Zelst, and Wil M. P. van der Aalst
Using Real Sensors Data to Calibrate a Traffic Model
for the City of Modena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
Chiara Bachechi, Federica Rollo, Federico Desimoni, and Laura Po
Logistic Regression for Criteria Weight Elicitation
in PROMETHEE-Based Ranking Methods . . . . . . . . . . . . . . . . . . . . . . 474
Elia Balugani, Francesco Lolli, Maria Angela Butturi, Alessio Ishizaka,
and Miguel Afonso Sellitto
3D CAD Design of Jewelry Accessories, Determination
of Geometrical Features and Characteristics of the Used Material
of Precious Metals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
Tihomir Dovramadjiev, Mariana Stoeva, Violeta Bozhikova,
and Rozalina Dimova
xvi Contents
Discovering and Mapping LMS Course Usage Patterns
to Learning Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
Darko Etinger
Drug Recommendation System for Geriatric Patients Based
on Bayesian Networks and Evolutionary Computation . . . . . . . . . . . . . 492
Lourdes Montalvo and Edwin Villanueva
Software for the Determination of the Time and the F Value
in the Thermal Processing of Packaged Foods Using the Modified
Ball Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
William Rolando Miranda Zamora, Manuel Jesus Sanchez Chero,
and Jose Antonio Sanchez Chero
Communication Protocol Between Humans and Bank Server
Secure Against Man-in-the-Browser Attacks . . . . . . . . . . . . . . . . . . . . . 503
Koki Mukaihira, Yasuyoshi Jinno, Takashi Tsuchiya, Tetsushi Ohki,
Kenta Takahashi, Wakaha Ogata, and Masakatsu Nishigaki
Development of a Solution Model for Timetabling Problems
Through a Binary Integer Linear Programming Approach . . . . . . . . . . 510
Juan Manuel Maldonado-Matute, María José González Calle,
and Rosana María Celi Costa
Machine, Discourse and Power: From Machine Learning
in Construction of 3D Face to Art and Creativity . . . . . . . . . . . . . . . . . 517
Man Lai-man Tin
Modelling Alzheimer’s People Brain Using Augmented Reality
for Medical Diagnosis Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Ramalakshmi Ramar, Swashi Muthammal, Tamilselvi Dhamodharan,
and Gopi Krishnan Rajendran
Software Vulnerability Mining Based
on the Human-Computer Coordination . . . . . . . . . . . . . . . . . . . . . . . . . 532
Jie Liu, Da He, Yifan Wang, Jianfeng Chen, and Zhihong Rao
Design and Verification Method for Flammable Fluid Drainage
of Civil Aircraft Based on DMU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
Yu Chen
Low-Income Dwelling Bioclimatic Design with CAD Technologies.
A Case Study in Monte Sinahí, Ecuador . . . . . . . . . . . . . . . . . . . . . . . . 546
Jesús Rafael Hechavarría Hernández, Boris Forero,
Robinson Vega Jaramillo, Katherine Naranjo, Fernanda Sánchez,
Billy Soto, and Félix Jaramillo
Contents xvii
Virtual Reduction and Interaction of Chinese Traditional
Furniture and Its Usage Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
Dehua Yu
Humans and Artificial Systems Complexity
Your Voice Assistant Will See You Now: Reducing Complexity
in Human and Artificial System Collaboration Using Voice
as an Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
Viraj Patwardhan, Neil Gomes, and Maia Ottenstein
Pre-emptive Culture Mapping: Exploring a System of Language
to Better Understand the Abstract Traits of Human Interaction . . . . . . 567
Timothy J. Stock and Marie Lena Tupot
''Meanings'' Based Human Centered Design of Systems . . . . . . . . . . . . . 573
Santosh Basapur and Keiichi Sato
A Systematic Review of Sociotechnical System Methods Between
1951 and 2019 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580
Amangul A. Imanghaliyeva
Designing a Safety Confirmation System that Utilizes Human
Behavior in Disaster Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588
Masayuki Ihara, Hiroshi Nakajima, Goro Inomae, and Hiroshi Watanabe
Designing Ethical AI in the Shadow of Hume’s Guillotine . . . . . . . . . . . 594
Pertti Saariluoma and Jaana Leikas
A Counterattack of Misinformation: How the Information Influence
to Human Being . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 600
Subin Lee and Ken Nah
Effects of Increased Cognitive Load on Field of View in Multi-task
Operations Involving Surveillance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
Seng Yuen Marcus Goh, Ka Lon Sou, Sun Woh Lye, and Hong Xu
Investigating Human Factors in the Hand-Held Gaming Interface
of a Telerehabilitation Robotic System . . . . . . . . . . . . . . . . . . . . . . . . . . 612
S. M. Mizanoor Rahman
Procedure of Mining Relevant Examples of Armed Conflicts to Define
Plausibility Based on Numerical Assessment of Similarity of Situations
and Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619
Ahto Kuuseok
Human Digital Twins: Two-Layer Machine Learning Architecture
for Intelligent Human-Machine Collaboration . . . . . . . . . . . . . . . . . . . . 627
Wael Hafez
xviii Contents
Semantic Network Analysis of Korean Virtual Assistants’
Review Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633
Hyewon Lim, Xu Li, Harim Yeo, and Hyesun Hwang
Design Collaboration Mode of Man–Computer Symbiosis
in the Age of Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
Jinjing Liu and Ken Nah
User Experience over Time with Personal Assistants of Mobile
Banking Application in Turkey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646
Hatice Merve Demirci and Mehmet Berberoğlu
Human-Automation Interaction Through Shared and Traded
Control Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653
Mauricio Marcano, Sergio Díaz, Joshue Pérez, Andrea Castellano,
Elisa Landini, Fabio Tango, and Paolo Burgio
Alignment of Management by Processes and Quality Tools
and Lean to Reduce Unfilled Orders of Fabrics for Export:
A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660
Z. Bardales, P. Tito, F. Maradiegue, Carlos Raymundo-Ibañez,
and Luis Rivera
Detection and Prevention of Criminal Attacks in Cloud
Computing Using a Hybrid Intrusion Detection Systems . . . . . . . . . . . . 667
Thierry Nsabimana, Christian Ildegard Bimenyimana, Victor Odumuyiwa,
and Joël Toyigbé Hounsou
Development of Tutoring Assistance Framework Using Machine
Learning Technology for Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677
Satoshi Togawa, Akiko Kondo, and Kazuhide Kanenishi
Replenishment System Using Inventory Models with Continuous
Review and Quantitative Forecasting to Reduce Stock-Outs
in a Commercial Company . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
Carlos Malca-Ramirez, Luis Nuñez-Salome, Ernesto Altamirano,
and José Alvarez-Merino
Applying SLP in a Lean Manufacturing Model to Improve
Productivity of Furniture SME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690
Zhelenn Farfan-Quintanilla, Manuel Caira-Jimenez,
Fernando Sotelo-Raffo, Carlos Raymundo-Ibañez, and Moises Perez
Collaborative Model Based on ARIMA Forecasting for Reducing
Inventory Costs at Footwear SMEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697
Alejandra Angulo-Baca, Michael Bernal-Bazalar, Juan Sotelo-Raffo,
Carlos Raymundo-Ibañez, and Moises Perez
Contents xix
A Framework of Quality Control Matrix in Paprika Chain Value:
An Empirical Investigation in Peru . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
Diana Garcia-Montero, Luz Roman-Ramirez, Fernando Sotelo-Raffo,
and Edgar Ramos-Palomino
Inventory Optimization Model Applying the Holt-Winters Method
to Improve Stock Levels in SMEs in the Sports Retail Sector . . . . . . . . 711
Diego Amasifén-Pacheco, Angela Garay-Osorio, Maribel Perez-Paredes,
Carlos Raymundo-Ibañez, and Luis Rivera
Recruitment and Training Model for Retaining and Improving
the Reputation of Medical Specialists to Increase Revenue
of a Private Healthcare SME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719
Audy Castro-Blancas, Carlos Rivas-Zavaleta, Carlos Cespedes-Blanco,
Carlos Raymundo, and Luis Rivera
Research on Disabled People’s Museum Visit Experience
from the Perspective of Actor-Network Theory . . . . . . . . . . . . . . . . . . . 726
Shifeng Zhao and Jie Shen
Production Management Model to Balance Assembly Lines
Focused on Worker Autonomy to Increase the Efficiency
of Garment Manufacturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
Valeria Sosa-Perez, Jose Palomino-Moya, Claudia Leon-Chavarril,
Carlos Raymundo-Ibañez, and Moises Perez
Rural Ecotourism Associative Model to Optimize the Development
of the High Andean Tourism Sector in Peru . . . . . . . . . . . . . . . . . . . . . 740
Oscar Galvez-Acevedo, Jose Martinez-Castañon, Mercedes Cano-Lazarte,
Carlos Raymundo-Ibañez, and Moises Perez
Picking Management Model with a Focus on Change Management
to Reduce the Deterioration of Finished Products in Mass
Consumption Distribution Centers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746
Lourdes Canales-Ramos, Arelis Velasquez-Vargas, Pedro Chavez-Soriano,
Carlos Raymundo-Ibañez, and Moises Perez
Risk Factors Associated with Work-Related Low Back Pain
Among Home-Based Garment Workers . . . . . . . . . . . . . . . . . . . . . . . . . 753
Sunisa Chaiklieng, Pornnapa Suggaravetsiri, and Sari Andajani
Demand Management Model Based on Quantitative Forecasting
Methods and Continuous Improvement to Increase Production
Planning Efficiencies of SMEs Bakeries . . . . . . . . . . . . . . . . . . . . . . . . . 760
Denilson Contreras-Choccata, Juan Sotelo-Raffo,
Carlos Raymundo-Ibañez, and Luis Rivera
xx Contents
Study on Key Elements of Shopping App Design for the Elderly . . . . . . 766
Wenfeng Liu, Fenghong Wang, and Yiyan Chen
Shopping Website Accessibility Study Based
on Users’ Mental Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773
Zhen Wu, Chengqi Xue, Yanfei Zhu, Binhong Zhai, and Chi Zhang
HIRAC-Based Risk Management Model with POKA–YOKE
and TPM Continuity to Control and Mitigate Emergency Scenarios
in Hydrocarbon Sector Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780
Jose Echevarria-Cahuas, Maria Quispe-Huapaya, Cesar Ramirez-Valdivia,
Carlos Raymundo, and Luis Rivera
Materials and Inclusive Human Systems
CAD-Based Risk Assessment Approach for Safe Scheduling of HRC
Operations for Parts Produced by Laser Powder Bed Fusion . . . . . . . . 789
Fabio Pini, Enrico Dalpadulo, and Francesco Leali
Photogrammetry and Additive Manufacturing Based Methodology
for Decentralized Spare Part Production in Automotive Industry . . . . . 796
Antonio Bacciaglia, Alessandro Ceruti, and Alfredo Liverani
Improved Heat Sink for Thermoelectric Energy Harvesting Systems . . . 803
Alessandro Bertacchini, Silvia Barbi, and Monia Montorsi
A Framework Designing for Story Sharing of the Elderly:
From Design Opportunities to Concept Selection . . . . . . . . . . . . . . . . . . 810
Cun Li, Jun Hu, Bart Hengeveld, and Caroline Hummels
A Methodological Approach for the Design of Inclusive
Assistive Devices by Integrating Co-design and Additive
Manufacturing Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816
Francesco Gherardini, Andrea Petruccioli, Enrico Dalpadulo,
Valentina Bettelli, Maria Teresa Mascia, and Francesco Leali
New Collaborative Version of the Quality Function Deployment:
Practical Application to the HABITAT Project . . . . . . . . . . . . . . . . . . . 823
Giuseppe Mincolelli, Gian Andrea Giacobone, Michele Marchi,
and Silvia Imbesi
Human Centered Methodologies for the Development
of Multidisciplinary Design Research in the Field of IOT Systems:
Project Habitat and Pleinair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 829
Giuseppe Mincolelli, Silvia Imbesi, Gian Andrea Giacobone,
and Michele Marchi
Contents xxi
Design of an Innovative Furniture System: Improving Acoustic
Comfort in Coworking Workplaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
Viola Geniola, Stefania Camplone, Antonio Marano, and Emilio Rossi
Modeling of Subcutaneous Implantable Microchip Intention of Use . . . 842
Mona A. Mohamed
A Brief Analysis of the Status Quo and Trend of Wearable
Smart Jewellery Devices Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 848
Jing Liu and Ken Nah
Accessibility Evaluation of Video Games for Users
with Cognitive Disabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853
Luis Salvador-Ullauri, Patricia Acosta-Vargas, and Sergio Luján-Mora
Design of Smart Devices for Older People: A User Centered Approach
for the Collection of Users’ Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 860
Silvia Imbesi and Giuseppe Mincolelli
Examining Feedback of Apple Watch Users in Korea Using
Textmining Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865
Yu Lim Lee, Minji Jung, In-Hyoung Park, Ahyoung Kim,
and Jae-Eun Chung
Structural Testing of Laminated Prosthetic Sockets:
Comparison of Philippine Pineapple Fabric and Fiberglass . . . . . . . . . . 871
Glenn Alkuino, Ervin Fandialan, and Marvin Medina
Challenges and Improvements in Website Accessibility
for Health Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875
Patricia Acosta-Vargas, Paula Hidalgo, Gloria Acosta-Vargas,
Mario Gonzalez, Javier Guaña-Moya, and Belén Salvador-Acosta
Providing Comprehensive Navigational Cues Through the Driving
Seat to Reduce Visual Distraction in Current Generation
of Semi-autonomous Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 882
Ahmed Farooq, Grigori Evreinov, and Roope Raisamo
Ensuring the Sustainability of Inclusive Projects Through
Strategic Addressing Supported by Process Management:
Case Applied to Aquamarinna Handmade Soap . . . . . . . . . . . . . . . . . . 889
Diego S. Suarez, Esteban Crespo-Martínez, and Pedro Mogrovejo
A New Model to Bionic Hand Prosthesis with Individual
Fingers Actuators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
Marcelo H. Stoppa, Guilherme F. Neto, and Danillo A. de S. Dunck
A Predictive Model of Users’ Behavior and Values
of Smart Energy Meters Using PLS-SEM . . . . . . . . . . . . . . . . . . . . . . . 903
Ahmed Shuhaiber
xxii Contents
UltraSurfaces: A New Material Design Vision . . . . . . . . . . . . . . . . . . . . 909
Marinella Ferrara and Chiara Pasetti
The Hybrid Dimension of Material Design: Two Case Studies
of a Do-It-Yourself Approach for the Development of Interactive,
Connected, and Smart Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916
Stefano Parisi, Markus Holzbach, and Valentina Rognoli
Human-Autonomy Teaming
Goal Directed Design of Rewards and Training Features for Self-
learning Agents in a Human-Autonomy-Teaming Environment . . . . . . . 925
Simon Schwerd, Sebastian Lindner, and Axel Schulte
Facial Expressions as Indicator for Discomfort
in Automated Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932
Matthias Beggiato, Nadine Rauh, and Josef Krems
Can We Talk? – The Impact of Conversational Interfaces
on Human Autonomy Teaming Perception, Performance
and Situation Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 938
Adam Bogg, Andrew Parkes, and Mike Bromfield
Driver’s Situational Awareness and Impact of Phone Interface
Modality in Conventional and Semi-autonomous Vehicles . . . . . . . . . . . 945
Syeda Rizvi, Francesca Favaro, and Nazanin Nader
Concept of an Adaptive Cockpit to Maintain the Workflow
of the Cockpit Crew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 952
Juliane Müller and Axel Schulte
A Conceptual Augmentation of a Pilot Assistant System
with Physiological Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 959
Dennis Mund, Evgeni Pavlidis, Matthew Masters, and Axel Schulte
Implementation of Teaming Behavior in Unmanned Aerial Vehicles . . . 966
Marius Dudek, Sebastian Lindner, and Axel Schulte
Behavioral Analysis of Information Exchange Digitalization
in the Context of Demand Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973
Tim Lauer and Katharina Franke
Signs Symbols & Displays in Automated Vehicles:
A Focus Group Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 980
Joy Richardson, Kirsten Revell, Jisun Kim, and Neville A. Stanton
Beauty Attracts the Eye but Character Captures the Heart:
Why Personality Matters in Chat Bot Design . . . . . . . . . . . . . . . . . . . . . 986
Helen Muncie
Contents xxiii
Integration of Humans in the Fallback Process by a Machine in Fully
Automated Railway Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 992
Bilal Üyümez and Andreas Oetting
Analysis of Facial Expressions Explain Affective State
and Trust-Based Decisions During Interaction with Autonomy . . . . . . . 999
Catherine Neubauer, Gregory Gremillion, Brandon S. Perelman,
Claire La Fleur, Jason S. Metcalfe, and Kristin E. Schaefer
Let’s Get in Touch Again: Tangible AI and Tangible XR
for a More Tangible, Balanced Human Systems Integration . . . . . . . . . 1007
Frank Flemisch, Konrad Bielecki, Daniel López Hernández,
Ronald Meyer, Ralph Baier, Nicolas Daniel Herzberger,
and Joscha Wasser
Time Line Based Tasking Concept for MUM-T Mission Planning
with Multiple Delegation Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1014
Felix Heilemann and Axel Schulte
Towards Cognitive Assistance and Teaming in Aviation by Inferring
Pilot’s Mental State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1021
Nele Russwinkel, Christoph Vernaleken, and Oliver W. Klaproth
Evaluating the Impact of Phone Interface Modality on Response
Times to Stimuli in Conventional and Semi-automated Vehicles . . . . . . 1028
Sky O. Eurich, Shivangi Agarwal, and Francesca Favaro
Design and Evaluation of Human-Friendly Hand-Held Gaming
Interface for Robot-Assisted Intuitive Telerehabilitation . . . . . . . . . . . . 1034
S. M. Mizanoor Rahman
Capture of Intruders by Cooperative Multiple Robots
Using Mobile Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1041
Yasushi Kambayashi, Taichi Sekido, and Munehiro Takimoto
Automation as Driver Companion: Findings of AutoMate Project . . . . . 1048
Andrea Castellano, Massimo Fossanetti, Elisa Landini, Fabio Tango,
and Roberto Montanari
Applications and Future Trends
Development of a Human System Integration Program
in Military Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1057
Jari Laarni and Marja Ylönen
Beyond Confluence, Integration and Symbiosis: Creating
More Aware Relationships in Smart Cities . . . . . . . . . . . . . . . . . . . . . . 1063
H. Patricia McKenna
xxiv Contents
A Potential Analysis of Cognitive Assistance Systems
in Production Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1069
Jessica Klapper, Bastian Pokorni, and Moritz Hämmerle
Identifying and Analysing Risk Factors from a Sociotechnical
System Perspective: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074
Amangul A. Imanghaliyeva
Experimental Learning for a Basic Technology Acquisition
of Moving Images Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1082
Akiko Kondo and Satoshi Togawa
Mechanical Fatigue Evaluation by Image Recognition . . . . . . . . . . . . . . 1088
Massimo Milani, Luca Montorsi, Luca Fontanili, Gabriele Storchi,
and Gabriele Muzzioli
Universal Access and Inclusive Dwelling Design for a Family
in Monte Sinahí, Guayaquil, Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . 1094
Jesús Rafael Hechavarría Hernández, Boris Forero,
and Robinson Vega Jaramillo
Integrated Safety Risk Assessment Between Enterprises,
Industries and Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1101
Lu Zhang, Yun Luo, and Rui Liao
Comparison Between ARIMA and LSTM-RNN
for VN-Index Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107
Nguyen Trong Co, Hoang Huu Son, Ngo Thanh Hoang,
Tran Thi Phuong Lien, and Trinh Minh Ngoc
E-material Formatting Application Prototype 2.0 Development
Through Usability Testing of Prototype 1.0 . . . . . . . . . . . . . . . . . . . . . . 1113
Kristine Mackare, Anita Jansone, and Raivo Mackars
Use of CAD-CAM Technologies in the Production of Furniture
for Natural Disaster Areas in Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . 1119
Francesco Giuseppe Magnone, Víctor Gustavo Gómez Rodríguez,
Yoenia Portilla Castell, and Jesús Rafael Hechavarría Hernández
A Five-Factor KMS Success Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126
Gabriel Nyame and Zhiguang Qin
A Study on Understanding of Visitor Needs in Art Museum:
Based on Analysis of Visual Perception Through Eye-Tracking . . . . . . . 1132
Taeha Yi, Mi Chang, Sukjoo Hong, Meereh Kim, and Ji-Hyun Lee
Analysis of Art Museums’ Visitor Behavior and Eye Movements
for Mobile Guide App Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138
Mi Chang, Taeha Yi, Po Yan Lai, Jun Hee Lee, and Ji-Hyun Lee
Contents xxv
Al-Maqta Canal of Abu Dhabi, UAE: A Study of Waterfront
Landscapes and Flow in Manmade Canals . . . . . . . . . . . . . . . . . . . . . . 1145
Mohamed El Amrousi and Mohamed Elhakeem
A Discussion of User Experience on a Panoramic Scooter Riding
Video Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1152
Fei-Hui Huang
Application Trend of Interactive Multimedia in Art Museums . . . . . . . 1159
Yongbin Wang and Jian Yu
Design Criteria in Vernacular Architecture as a Proposal
for Low-Income Dwelling for Urban Parishes of the Babahoyo
Canton, Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164
Julio Franco Puga, Bryan Colorado Pástor,
Jesús Rafael Hechavarría Hernández, and Maikel Leyva
Consumer Experience of a Disruptive Technology:
An O2O Food Delivery App Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1171
Jaehye Suk, Yeon Ji Yang, Yun Jik Jeong, Muzi Xiang, and Kee Ok Kim
A Group Travel Recommender System Based on Collaborative
Filtering and Group Approximate Constraint Satisfaction . . . . . . . . . . . 1178
JinLu He, IlYoung Choi, and JaeKyeong Kim
Consumer’s Information Privacy and Security Concerns
and Use of Intelligent Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1184
Seonglim Lee, Jaehye Suk, Hee Ra Ha, Xiao Xi Song,
and YuanZhou Deng
PEST Analysis Based on Fuzzy Decision Maps for the Ordering of
Risk Factors in Territorial Planning of the Vinces Canton, Ecuador . . . 1190
Carlos Luis Valero Fajardo and Jesús Rafael Hechavarría Hernández
Model for Urban Consolidation of Informal Human Settlements
Based on Cooperation Systems and Human Participation
in Guayaquil, Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1195
María Milagros Fois, Karen Sellan, Karla Moscoso, and Maria Ruiz
Systemic Approach to the Territorial Planning of the Urban Parish
La Aurora, Daule, Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1201
Cyntia Alava Portugal, Jesús Rafael Hechavarría Hernández,
and Milagros Fois Lugo
Systemic Approach to Strategic Tourism Planning in the Cantonal
Capital of Bahía de Caráquez, Sucre, Ecuador . . . . . . . . . . . . . . . . . . . 1206
Milton Zambrano and Jesús Rafael Hechavarría Hernández
xxvi Contents
Cognitive Rehabilitation for Autism Children Mental Status
Observation Using Virtual Reality Based Interactive Environment . . . . 1213
Tamilselvi Dhamodharan, Manju Thomas, Sathiyaprakash Ramdoss,
Karthikeyan JothiKumar, SaiNaveenaSri SaravanaSundharam,
BhavaniDevi Muthuramalingam, NilofarNisa Hussainalikhan,
Sugirtha Ravichandran, VaibhavaShivani Vadivel, Pavika Suresh,
Sasikumar Buddhan, and Ajith Madhusudanan
Eye Control System Development and Research of Effects of Color
of Icons on Visual Search Performance Based on the System . . . . . . . . 1219
Jiaqi Cui, Yafeng Niu, Chengqi Xue, Xijiang Cai, Yi Xie, Bingzheng Shi,
and Lincun Qiu
How to Improve Manufacturing Process Implementing 5S Practices:
A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1225
Beata Mrugalska, Monika Konieczna, and Magdalena K. Wyrwicka
The Initial Stage of Development of a New Computer Program
for the Processing of Psychophysiological Tests . . . . . . . . . . . . . . . . . . . 1233
Jelena Turlisova and Anita Jansone
Experimental Study on Dynamic Map Information Layout Based
on Eye Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1238
Jiapei Ren, Haiyan Wang, and Junkai Shao
Research on Readability of Adaptive Foreground
in Dynamic Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1244
Maoping Chi and Lei Zhou
Research on Interaction Design of Children’s Companion Robot
Based on Cognitive Psychology Theory . . . . . . . . . . . . . . . . . . . . . . . . . 1250
Tianmai Zhang and Wencheng Tang
Strategies for Accessibility to the Teodoro Maldonado Hospital
in Guayaquil. A Design Proposal Focused on the Human Being . . . . . . 1256
Josefina Avila Beneras, Milagros Fois Lugo,
and Jesús Rafael Hechavarría Hernández
Fatigue Measurement of Task: Based on Multiple Eye-Tracking
Parameters and Task Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1263
Hanyang Xu, Xiaozhou Zhou, and Chengqi Xue
Emotional Data Visualization for Well-Being,
Based on HRV Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1270
Akane Matsumae, Ruiyao Luo, Yun Wang, Eigo Nishimura,
and Yuki Motomura
Contents xxvii
A Consumer-Centric Approach to Understand
User’s Digital Experiences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1277
Yeon Ji Yang, Jaehye Suk, Kee Ok Kim, Hyesun Hwang, Hyewon Lim,
and Muzi Xiang
Research on Design Skills for Personnel Evaluation Systems
and Educational Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284
Toshiya Sasaki
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1289
Automotive Design and Transportation
Engineering
User-Centered Design Within the Context
of Automated Driving in Trucks – Guideline
and Methods for Future Conceptualization
of Automated Systems
Paula Laßmann1(&), Florian Reichelt1, Dominique Stimm2,
and Thomas Maier1
1
Institute for Engineering Design and Industrial Design (IKTD),
Department of Industrial Design Engineering, University of Stuttgart,
Pfaffenwaldring 9, 70569 Stuttgart, Germany
{Paula.Lassmann,Florian.Reichelt,
Thomas.Maier}@iktd.uni-stuttgart.de
2
Institute for Mobility and Digital Innovation (MoDI),
Stuttgart Media University, Nobelstraße 10, 70569 Stuttgart, Germany
[email protected]
Abstract. The monotony of moving in queues enhances truckers´ sleepiness
and the danger of micro sleep, which put a serious risk on traffic safety. The
research project TANGO aims at overcoming these risks by developing an
Attention and Activity Assistance system (AAA). The AAA shall enable truck
drivers to engage in non-driving-related activities during automated driving
phases (up to SAE level 3), but keep them alert for tasks (monitoring or take-
over) nevertheless. Throughout the project, numerous user studies have led to a
great gain of knowledge about the users’ needs and design requirements con-
cerning an automated system. For the purpose of an easier accessibility of this
knowledge for future research, a guideline will be developed that covers both
design guidance as well as a review of applied methods. The main scope of this
paper focuses on the methodical process of developing the guideline.
Keywords: Automated driving User-centered design Automotive design
Guideline Methods
1 Objective and Significance
The monotony during driving a truck, mainly caused by moving in queues, increases the
danger of micro sleep and therefore causes a serious risk on traffic safety [1]. The aim of
TANGO (German abbreviation for Technologie für automatisiertes Fahren nutzerg-
erecht optimiert, English equivalent Technology for automated driving, optimized to
user needs) is a gain of user experience and acceptance of automated driving in trucks.
The project is developing a new technology that helps keeping the driver at the optimal
level of strain with consideration of the driver comfort at the same time: the Attention
and Activity Assistance system (AAA). The AAA shall enable truck drivers to engage in
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 3–8, 2020.
https://doi.org/10.1007/978-3-030-39512-4_1
4 P. Laßmann et al.
non-driving-related activities during automated driving phases (SAE level 2 and 3 [2]).
This will be achieved with the help of a driver observer system as well as a user-friendly
human machine interface (HMI) that is constantly evaluated and improved throughout
the development process. The development of the AAA follows the user-centered
design process according to DIN EN ISO 9241-210 [3], which integrates users’ con-
cerns in iterative steps of conceptualization and testing. Within this process, the project
consortium gained a vast knowledge about the users’ concerns and safety while inter-
acting with an automated truck system like the AAA. In addition, within this devel-
opment process the consortium has applied a broad range of scientific methods. For the
purpose of an easier accessibility of this knowledge for HMI designers and future
research projects, the consortium will derive user concerns, top findings, and recom-
mendations (summarized in a chapter design guidance) as well as a review of methods
and document them in a guideline.
For manual driving, several guidelines exist, among other things a project docu-
mentation by the project HARDIE [4], that covers recommendations and method-
ological advice. Other guidelines focus on certain interface elements like vibro-tactile
displays [5], high-priority warning signals [6] or in-vehicle information systems [7]. In
terms of guidelines for automated driving in the context of SAE level 2 and 3 there are
few guidelines providing actual guidance on designing and developing automated
systems. The National Highway Traffic Safety Administration (NHTSA) published a
guidance for Human Factors Design for those levels in 2018 [8]. The guidance pro-
vides much information and aid on designing interfaces for automated vehicles. In
addition, Naujoks and colleagues [9] give advice by listing design recommendations
and including an extended checklist for automated interface design.
The main goal of the guideline presented in this paper is to provide knowledge,
gained within the TANGO project, to support developers concerned with similar
research questions within the future development and design of automated systems.
Therefore, the guideline focusses not only on recommendations concerning the HMI
design and the design of the AAA in general but also provides possibilities for the
methodological approach while developing automated systems which is not yet cov-
ered in other guidelines.
2 Methodological Development of the Guideline
A comprehensive project documentation serves as a basis for the guideline. This
documentation contains all data and facts gathered throughout the iterative develop-
ment of the AAA. Since the project documentation is very detailed and will not be
published, it is not particularly usable for other projects and developers in general. The
guideline, in contrary, should provide easy access to the knowledge gained within
TANGO. For differentiation of design and methodological approach, the TANGO
guideline is divided into two parts: design guidance and a review of methods. An
example of what could be addressed in this guideline is the question of how the drivers
could occupy themselves during automated driving phases: what would be best from
the perspective of the user (e.g. [10]) and what would be best in terms of traffic safety
(e.g. [11])? The provided information of the two segments design guidance and review
User-Centered Design Within the Context of Automated Driving in Trucks – Guideline 5
of methods is extracted by using the knowledge derived from each part of the project
documentation. Whereas the focus of design guidance is on top findings, recommen-
dations and user concerns regarding the design of the AAA, the review of methods
provides a detailed overview of the methods applied within TANGO that can be used
for both development and evaluation of an automated system like the AAA. Figure 1
shows the development process and structure of the guideline as well as the connec-
tions to the detailed project documentation.
Fig. 1. Structure of the guideline, connections to the project documentation, and development
steps.
2.1 Design Guidance
The design guidance should provide advices concerning the future development and
design of automated systems. Major findings listed in the project documentation are
discussed and categorized in workshops within the project consortium.
Due to the fact that not every finding can be used as a valid basis for deriving an
appropriate recommendation, e.g. if the finding results only from one study (in contrary
to multiple studies), a consequent clustering and categorization was conducted in
workshops with special regards to importance and scientific validity. Therefore, the
findings are going to be clustered within three core categories: user concerns, top
findings and recommendations. This categorization is important due to the fact, that the
TANGO project included multiple qualitative user studies. The users had various
concerns that the consortium can neither recommend including in the TANGO system
nor in the design of an automated system level 2 or 3 in general, e.g. the wish to sleep
while automated driving. Nevertheless, these user concerns could be interesting for
another project, for example regarding higher automation levels. Top findings contain
all findings resulting from the studies guaranteeing a scientific validity and can be used
as a basis for recommendations. An example for a top finding is the fact, that during
level 2, an auditory cognitive non-driving related task leads to a good monitoring
6 P. Laßmann et al.
performance in terms of a visual detection task [11]. The recommendations are going to
include design suggestions derived from the study results of the project as well as a
discussion with references to other projects.
2.2 Review of Methods
The methods applied within the various studies (qualitative user research as well as
simulator studies) are gathered and evaluated via checklists. These checklists contain a
description of each method as well as a critical methodical reflection in order to give
advice for future user-centered design projects. Therefore, all methods applied within
the TANGO project are categorized into macro and micro methods, whereas a macro
method is the superordinate method that includes one or more micro methods. For
instance, within a simulator study, which would be the macro method, any question-
naire or objective method used would be a micro method.
Each macro and micro method was critically reflected with the help of the quality
criteria of scientific tests (objectivity, reliability and validity) as well as strengths and
weaknesses of the method. Each project partner filled the checklists for the methods
applied in their studies. Furthermore, for a better methodical overview each method
was rated via a three color system, comparable to a traffic light in the colors green,
yellow and red. Red means, that the method did not work well in the context of the
project TANGO and is not recommended for similar application. A method that is rated
yellow could be used, but adaptation is suggested. If the method worked well in this
application, it is rated green. Each method assessment comes with an explanation of the
Table 1. Example of the rating of several micro methods according to the gathered experience
in the specific context of TANGO.
Macro method Title and objective micro Traffic Justifications/Remarks
methods light
status
Ethnographical Observation of truck Green Achievement in the
Interview: User- drivers: Discovery of discovery of relevant
centered research to potentially critical positive and negative
identify driver´s needs moments in terms of occurrences during the
[10] driver experience drive
Driving simulator Sign Yellow Method to measure visual
study at the University Detection distraction, hit rate too
of Stuttgart [11] Task (SDT) high, targets should be
displayed shorter
Driving simulator Electrodermal Red No differences of EDA
study at MAN Truck activity (EDA) could be shown during
& Bus SE different levels of strain
Red: The method did not work well in the context of the project TANGO and is not
recommended for similar application
Yellow: Adaptation of the method is suggested in the context of the project TANGO
Green: The method worked well in the context of the project TANGO
User-Centered Design Within the Context of Automated Driving in Trucks – Guideline 7
rating by the operator. It must be clear, that the rating cannot be applied to any other
project right away without consideration of the context. Publications of the results of
the TANGO studies will be linked to each method for further consultation, if available.
Reviews of three methods that have been used within the project are displayed in
Table 1. In conclusion, the developer will have a list with all methods that were
conducted in TANGO. In this list, additional information about the context as well as a
(non) recommendation will be provided.
3 Conclusion and Outlook
This guideline, which includes both design recommendations and methods, sums up
the findings of TANGO in an accessible format. It aims at being an assistance for future
projects dealing with automated vehicle design, in particular regarding the design of an
HMI. The main scope of this paper focuses on the methodical process of developing
the guideline regarding both the recommendations and methods. The guideline will
presumably be published after the TANGO project ends and might be a handy tool for
developers of automated systems. In addition, the process description of the devel-
opment of the guideline aims at being an example for how other research projects could
summarize and document their findings in order to make them accessible for other
scientists. Especially in terms of a method review, no examples have been found. The
goal is to start a discourse about a best practice in this regard.
Acknowledgments. The German Federal Ministry of Economic Affairs and Energy fund the
presented research. The goal of this 3.5 years research project is to develop an Attention and
Activity Assistance system (AAA) to overcome monotony and truckers’ sleepiness. The project
consortium consists of Robert Bosch GmbH, Volkswagen Aktiengesellschaft, MAN Truck &
Bus SE, Stuttgart Media University and University of Stuttgart.
References
1. National Transportation Safety Board. Factors that affect fatigue in heavy truck accidents.
Safety Study NTSB/SS-95/01 and NTSB/SS-95/02. Washington, DC (1995)
2. SAE international: Taxonomy and definition for terms related to driving automation systems
for on-road motor vehicles (2018)
3. DIN EN ISO 9241-210 Ergonomics of human-system interaction – Part 210: Human-
centered design for interactive systems (2010)
4. Ross, T., Midtland, K., Fuchs, M., Pauzié, A., Engert, A., Duncan, B., Vaughan, G., Vernet,
M., Peters, H., Burnett, G.E., May, A.J.: HARDIE Design Guidelines Handbook. Human
Factors Guidelines for Information Presentation by ATT Systems, vol. 530 (1996)
5. Van Erp, J.B.: Guidelines for the use of vibro-tactile displays in human computer interaction.
Paper Presented at the Proceedings of Eurohaptics (2002)
6. United Nations Economic Commission for Europe. Guidelines on establishing requirements
for high-priority warning signals (2009)
7. Transport Research Laboratory. A checklist for the assessment of in-vehicle information
systems (IVIS) Wokingham: TRL (2011)
8 P. Laßmann et al.
8. Campbell, J.L., Brown, J.L., Graving, J.S., Richard, C.M., Lichty, M.G., Bacon, L.P.,
Sanquist, T.: Human factors design guidance for level 2 and level 3 automated driving
concepts (Report No. DOT HS 812 555). Washington, DC: National Highway Traffic Safety
Administration (2018)
9. Frederik, N., et al.: Towards guidelines and verification methods for automated vehicle
HMIs. Transp. Res. Part F: Traffic Psychol. Behav. 60, 121–136 (2019)
10. Ruppert, M., Engeln, A., Michel, B., Stimm, D.: The human factor. working to ensure
acceptance of autonomous systems among truck drivers. Fut. Transp. Rev. 2018, S118–123,
(2018). London
11. Lassmann, P., Fischer, M. S., Bieg, H.-J., Jenke, M., Reichelt, F., Tüzün, G.-J., Maier, T.:
Keeping the balance between overload and underload during partly automated driving:
relevant secondary tasks. Submitted to 17. ATZ-Fachtagung. Springer (2019)
Towards Probabilistic Analysis
of Human-System Integration
in Automated Driving
Ephraim Suhir1,2,3,4(&), Gunther Paul2, and Hermann Kaindl3
1
Portland State University, Portland, OR, USA
[email protected]
2
James Cook University, Townsville, QLD, Australia
[email protected]
3
TU Wien, Vienna, Austria
[email protected]
4
ERS Co., Los Altos, CA, USA
Abstract. According to the Automated Driving Roadmap ERTRAC 17, only
vehicles of Level 5 may not need human interference. The current adaptive
cruise control system or more advanced automated driving solutions below
Level 5 require, therefore, that a human driver takes over, if an extraordinary
situation occurs. A critical safety problem may be caused by the very short time
span available to the driver. It has been recently demonstrated, mostly in
application to the aerospace domain, how probabilistic analytical modeling
(PAM) could effectively complement computational simulation techniques in
various human-in-the-loop (HITL) related missions and off-normal situations,
when the reliability of the equipment (instrumentation), both hard- and software,
and the performance of the human contribute jointly to their likely outcome. Our
objective is to extend this approach, with appropriate modifications, to safety
analyses of automated driving applications.
Keywords: Human factors Human-systems integration Automated driving
1 Introduction
It has been recently demonstrated, mostly in application to the aerospace domain [1–3],
how PAM [4, 5] could effectively complement computational simulations (see, e.g., [6,
7]) or model checking (see, e.g., [8, 9]), when the reliability of the equipment (instru-
mentation), both hard- and software [10], and human performance contribute jointly to
the outcome of a mission or an extraordinary situation. In the analysis of this paper, we
show, using an example of emergency stopping, how this approach could be brought
“down to earth”, i.e., modified for its application to automotive automated driving. The
application of the PAM concept can improve the understanding of and accounting for
the human performance in various vehicular missions and situations [11, 12].
It is noteworthy that the automotive vehicle environment may be even less for-
giving than for an aircraft: slight deviations in aircraft altitude, speed, or human actions
are often tolerable without immediate consequences, while an automotive vehicle is
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 9–14, 2020.
https://doi.org/10.1007/978-3-030-39512-4_2
10 E. Suhir et al.
likely to have tighter control requirements for avoiding collision than an aircraft [13].
The existing Automated Driving Roadmap ERTRAC 17 [14] indicates that only
vehicles of Level 5 may not need human interference. Hence, using any of the current
adaptive cruise control (ACC) or more advanced autopilot solutions requires a human
driver to take over, if something extraordinary occurs, or if there is an autopilot failure.
A key safety problem is likely to be caused particularly by the short time span
available to the driver to react properly. While this rather obvious problem is fairly well
understood qualitatively, we are not aware of any publicly available quantitative
evaluation.
In order to address the key problem, we present and analyze here a first example,
where the automated system all of a sudden hands over to the driver in the face of an
obstacle. In fact, this is a scenario where the automated system may directly and
automatically initiate emergency braking [15] instead. We think that this will even-
tually be the better approach with respect to safety in such a situation, but there may
even be legal problems involved for not implementing this and handing over to the
driver. Our analysis provides some quantitative evidence, how bad this is in terms of
safety because of the time needed by the human driver to take over and to initiate the
breaking himself. We conjecture that the basic modeling approach and the probabilistic
analysis based on it will be amenable to a more general study of HITL in the context of
automated driving.
The remainder of this paper is organized in the following manner. First, we present
the example problem chosen for this initial study. Then this example is analyzed both
in a deterministic and a probabilistic approach, where the latter analysis is the core
contribution of this paper. Finally, we draw a few conclusions and sketch future work
on our way towards probabilistic analysis of human-system integration in automated
driving.
2 An Example Problem
The context of this example is using ACC or even a more advanced autopilot. Let us
assume that there is fog ahead, and hidden through it there was already an accident
happening, so that the road is blocked by one or more obstacles that do not move (any
more). In such a situation, the automated system may all of a sudden realize that and
hand over to the human driver.
The role of the human factor could be assessed by comparing the distance covered
by the vehicle from the moment of time, when an obstacle has been detected (pre-
deceleration time), to the distance covered from the end of this ‘undesirable’ time to the
moment of time when the ‘useful’ constant-deceleration time ends, see Fig. 1. The
accident could be avoided, if the probability that the total operational distance covered
by the moving vehicle during these two random periods of time exceeds the available
sight distance (automatically detected, e.g., by radar) is sufficiently low.
The pre-deceleration time includes three major time periods:
(1) the decision-making time, i.e., the time that the system and/or the driver need to
decide that/if the driver has to intervene, including time to take over control of the
Towards Probabilistic Analysis of Human-System Integration 11
Fig. 1. Overview of the model.
vehicle (this important distinction still has to be determined and decided upon
[16], and this critical effort and the associated time and distance is beyond the
scope of this analysis);
(2) the time that the driver needs to make his decision on pushing the brakes (pre-
braking time), including biomechanical motion time, and
(3) the time that the driver needs to adjust the brakes (brake-adjusting time) when
interacting with the vehicle’s anti-lock (anti-skid) braking system. Although this
third period of time is affected by both human and vehicle performance, in this
analysis we conservatively assume that it is part of the pre-deceleration time.
A safe outcome of the situation in question is likely, if the probability that the
(random) sum of the (random) distances corresponding to the (also random)
sequential pre-deceleration and deceleration times exceeds the ‘available’ (also
random) sight distance is sufficiently low.
3 Analysis
3.1 Deterministic Approach
A deterministic approach distinguishes two periods of time in a possible collision sit-
uation: (1) The pre-deceleration time T0 from the initial moment of time, when the radar
detects the obstacle, until the time, when the vehicle starts to decelerate; during this time,
that depends on the driver’s experience, age, fatigue and, perhaps, his/her other human
capacity factors (HCFs), as well as the identification-reaction-communication perfor-
mance of the automated vehicle system [1–4, 12]; it could be assumed that the vehicle
keeps moving with its initial speed V0 during this time, and it is this time that charac-
terizes the performance of the driver and automation systems; if, e.g., the car’s initial
speed is V0 ¼ 13:41 m=s; the pre-deceleration time may be T0 ¼ 3:0 s (the corre-
sponding distance is S0 ¼ V0 T0 ¼ 13:41 3 ¼ 40:23 mÞ and (2) The deceleration
time T1 ¼ 2S V0 ¼ a obtained assuming constant and immediate deceleration a, i.e.,
1 V0
immediate effective braking with no phase of brake force build–up; where S1 is the
stopping distance during the deceleration time. If, e.g., a ¼ 4 m=s2 (it is this deceleration
force that characterizes the vehicle’s ability to decelerate, when necessary), and the
initial velocity is V0 ¼ 13:41 m s ; then T1 ¼ a ¼ 4 ¼ 3:35 s; is the deceleration time,
V0 13:41
and S1 ¼ V02T1 ¼ 13:41 2 3:35 ¼ 22:46 m is the corresponding braking distance. The con-
tributions of the two main constituents of the total stopping distance S ¼ S0 þ S1 ¼
40:23 þ 22:46 ¼ 62:69 m in this example are at a magnitude 2:1. It follows also from
the formula S ¼ V0 T0 þ T21 for the total stopping distance that the pre-deceleration
12 E. Suhir et al.
time T0 that reflects the role of the human factor, is even more critical than the decel-
eration time T1 affected by the vehicle and its brakes, and that the total stopping time is
simply proportional to the initial velocity that should be low enough to avoid an accident
and allow the driver to make a decision in a timely fashion. If the stopping distance S is
smaller than the available obstacle distance S from the location at which the autopilot
and/or the driver detected the obstacle in sight, to the location of the obstacle, than the
collision could possibly be avoided. In the above example, the detected obstacle should
not be closer than S ¼ 63 m (available sight distance) to the location, at which the
vehicle and/or driver detected it, otherwise a collision becomes unavoidable.
3.2 Probabilistic Approach
A probabilistic approach considers that, in effect, none of the above times and the
corresponding distances are known, or could be, or even will ever be, evaluated with
sufficient certainty. Therefore, a probabilistic approach should be employed to assess
the probability of an accident.
Here is a possible rationale behind such an approach to the problem above. To
some extent, it is similar to the “convolution approach” applied in the helicopter-
landing-ship situation [4], where, however, random times, not random distances, were
considered. Here is the rationale behind the PAM concept used in this analysis: if the
probability PðS S Þ that the random sum S ¼ S0 þ S1 of the two random distances S0
and S1 is above the ‘available’ (and also random) distance S to the obstacle is low,
then there is a good chance and a good reason to believe that collision will be avoided.
It is natural to assume that both random times T0 and T1 , as well as the corre-
sponding distances, are distributed in accordance with the Rayleigh law (see, e.g. [11]).
Indeed, both these random times cannot be zero, but cannot be very long either. In such
an emergency situation, short time values are more likely than long time values, and,
because of that, their probability density distribution functions should be heavily
skewed in the direction of short time values. The Rayleigh distribution f ðtÞ ¼
2
r2 exp 2r2 possesses all these physically important properties, and is used, therefore,
t t
for our analysis. Here r is the mode (most likely value) of the random variable T; t ¼
ppffiffi pffiffiffiffiffi q4p
ffiffiffiffiffiffiffi
2 r is its mean value and D t ¼ 2 r is its standard deviation. Using Rayleigh law
and treating the distances S0 and S1 as non-random functions
of the random variables
s20 s2
T0 and T1 , we use the expressions f0 ðs0 Þ ¼ r2 exp 2r2 and f1 ðs1 Þ ¼ rs12 exp 2r12 as
s0
0 0 1 1
their probability density functions. Here, r0 ¼ V0 T0 and r1 ¼ V20 T1 :
It is also natural
h to assume
i that the available distance S is normally distributed:
ðssÞ2
fs ðsÞ ¼ pffiffiffiffi exp 2r2 ; where rs 4, s is the mean value of the random distance S
1
2prs s
s
and rs is its standard deviation. The “safety factor” rss in this distribution should be large
enough, so that the normal law could be used as an acceptable approximation of the
random times and distances that cannot be negative. For large rss ratios, the small negative
values, although they exist, are ‘suppressed’ by the large positive values of the distri-
bution. The Rayleigh law cannot be employed for the available sight distance and the
Towards Probabilistic Analysis of Human-System Integration 13
corresponding time, because this distance and time, unlike the random times T0 and
T1 , should be symmetric with respect to their mean values. The probability, PS that
the sum S ¼ s0 þ s1 of the random variables b
2 S0 and2 S
1 exceeds a certain level S is
R S^ s0 ðSs0 Þ
^
ds0 . If, e.g., ^
s
[1, 11] PS ¼ 1 0 r2 exp 2r02 1 exp 2r2 S ¼ 85:2 m, r0 ¼
0 0 1
40:23 m; and r1 ¼ 44:97 m; then PS ¼ 0:6832. The probability that the normally
R ^S
distributed ‘available’ distance S is below the b
S level, is P ¼ pffiffiffiffi 1
s
2p r s 1
h i
ðssÞ2 1
exp 2r2 ds ¼ 12 1 erf pffiffi2r^Ss .
s ^
S
If, e.g., the mean value s of the available distance is s ¼ 66:5 m; and its standard
deviation is rs ¼ 13:5 m; then, with ^S ¼ 85:2 m; the formula (2) yields: P ¼ 0:0829;
and the probability of the collision is PC ¼ PS P ¼ 0:6832 0:0829 ¼ 0:0566: This
probability should be evaluated for different ^S values.
After a low enough probability PC is established (agreed upon), the suggested
methodology can be used to quantify, on the probabilistic basis, the role of the human
factor and the initial speed of the vehicle on the likelihood of a successful outcome of
the addressed situation.
4 Conclusion and Future Work
This example problem and especially its probabilistic analysis intend to show how such
an approach can provide quantitative analyses of the more general issues of automated
driving with human drivers in the loop. While this example is confined to a special
situation, it points already to the potential of such an approach for mathematically
founded analyses of problems to be urgently solved for a safe adoption of automated
driving (below Level 5), which is currently widely judged as important for both
industry and society.
Hence, PAM may provide an effective means for reducing vehicular casualties. We
conjecture that it will be able to improve dramatically the state-of-the-art in under-
standing and accounting for the human performance in various vehicular missions and
off-normal situations, and in particular in the pressing issue of analyzing the role of the
time span for the human to take over from the system.
Future work should include implementation of the suggested methodology, as well
as the development of practical recommendations, considering that the probability of an
accident, although never zero, could and should be predicted, adjusted to a particular
vehicle, autopilot, driver and environment, and made sufficiently low. This work
should include also considerable effort, both theoretical (analytical and computer-
aided) and experimental, in the addressed and in many other problems associated with
the upcoming era of automated driving.
14 E. Suhir et al.
References
1. Suhir, E.: Human-in-the-Loop: Probabilistic Modeling of an Aerospace Mission Outcome.
CRC Press, Boca Raton (2018)
2. Suhir, E.: Assessment of the required human capacity factor (HCF) using flight simulator as
an appropriate accelerated test vehicle. IJHFMS 6(1), 71–74 (2019)
3. Suhir, E.: Probabilistic Risk Analysis (PRA) in Aerospace Human-in-the-Loop (HITL)
Tasks, plenary lecture, IHSI 2019, Biarritz, France, 11–13 September 2019
4. Suhir, E.: Helicopter-landing-ship: undercarriage strength and the role of the human factor.
ASME OMAE J. 132(1), 011603 (2009)
5. Suhir, E.: Analytical modeling occupies a special place in the modeling effort. Short Comm.
J. Phys. Math. 7(1), 2 (2016)
6. Luckender, C., Rathmair, M., Kaindl, H.: Investigating and coordinating safety-critical
feature interactions in automotive systems using simulation. In: 50th Hawaii International
Conference on System Sciences, pp. 6151–6160 (2017)
7. Rathmair, M., Luckender, C., Kaindl, H., Radojicic, C.: Semi-symbolic simulation and
analysis of deviation propagation of feature coordination in cyber-physical systems, best
paper award. In: 51st Hawaii International Conference on System Sciences (HICSS 2018),
pp. 5655–5664 (2018)
8. Rathmair, M., Luckender, C., Kaindl, H.: Minimalist qualitative models for model checking
cyber-physical feature coordination. In: 23rd Asia-Pacific Software Engineering Conference
(APSEC 2016). IEEE Press (2016)
9. Luckender, C., Kaindl, H.: Systematic top-down design of cyber-physical models with
integrated validation and formal verification. In: 40th International Conference on Software
Engineering: Companion. ACM/IEEE (2018)
10. Suhir, E.: Probabilistic design for reliability of electronic materials, assemblies, packages
and systems: attributes, challenges, pitfalls, plenary lecture. In: MMCTSE 2017, Murray
Edwards College, Cambridge, UK, 25 February 2017
11. Suhir, E.: Applied Probability for Engineers and Scientists. McGraw Hill, New York (1997)
12. Suhir, E.: Adequate trust, human-capacity-factor, probability-distribution-function of human
Non-failure and its entropy. IJHFMS 6(1), 75–83 (2019)
13. Gerstenmaier, W.H.: NASA Headquarters, private communication, September 2019
14. ERTRAC 17, Automated Driving Roadmap, Version 7.0, May (2017)
15. Kudarauskas, N.: Analysis of emergency braking of a vehicle. Transport 22(3), 154–159
(2007)
16. Sirkin, D.M.: Stanford University, private communication, September 2019
Trust Provisioning in the Transport
Infrastructure
Scott Cadzow(&)
C3L for Cryptalabs, London, England, UK
[email protected]
Abstract. In any security process the role of trust anchors and trust roots have a
complex interaction and applying them to the transport domain where many
different forms of trust relationship have to work together is going to be one of
greatest of the problems to surmount. ITS and CAV are data centric information
systems in which the provenance of the data in the system is key to the success
of the system. Provenance and integrity are underpinnings of trust, but in CAV
and ITS there is no a priori knowledge of the source and value of data. This
means that the provenance of a signal or message from the infrastructure is
unlikely to be tested in advance and the verification of trust needs to be applied
on demand for each message. Determining trust distribution in the transport
infrastructure is one of the biggest unanswered questions regarding the viability
of CAV.
Keywords: Integrity Trust Provenance Connected and Autonomous
Vehicles (CAV) Intelligent Transport Systems (ITS) Trust networks Smart
infrastructure
1 Introduction
In the days before we had any concept of Intelligent Transport Systems (ITS) there was
a simple model of trust in most transport networks. The roads would be broadly
serviceable to the traffic they were expected to handle. At source and destination of any
journey there would be somewhere to park and load or unload. If the distances were
beyond the reach of a single fueling you could reasonably count on being able to refuel
without major changes of route. If restrictions were placed on your journey by the use
of speed limits, or barring certain routes, you saw the notifications and acted on them in
good faith. The trust model was implicit and reinforced by education and experience
over the many hundreds of years that mankind has needed to move themselves and
goods from A to B. Roads, vehicles and all the infrastructure of transport was marked
by steady evolution between revolutionary leaps. Our concept of a connected and
autonomous or automated vehicle belonged to the pages of science fiction novels but
instinctively is also evolution of our existing transport modes. However, this evolution
has more stakeholders than ever before and there is a danger that without taking a
holistic view to how we engage, and trust, with the new environment that there may be
an existential threat to this particular evolutionary step.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 15–19, 2020.
https://doi.org/10.1007/978-3-030-39512-4_3
16 S. Cadzow
A key assertion is that the future of transport is that our understanding of transport
moves from a mostly physical infrastructure to a pair of parallel and interworking
infrastructures, one being the existing physical infrastructure and the other being
additional data infrastructure that enables ITS and the associated world of Connected
and Autonomous Vehicles (CAV).
2 The Role of Trust
If we take one of the more common definitions of trust as “firm belief in the reliability,
truth, or ability of someone or something” as it applies to transport and transport
infrastructure, and consider the complexity of that domain, then it should be clear that
the question of who to trust is a difficult one to answer.
Trust is not absolute, it is often not rational, and it is often not transferable, but it is
fundamental to making decisions based on data received. For any security process the
role of trust anchors and trust roots have a complex interaction and when in place serve
to answer, at least in part, the question of “who do I trust?”.
A major concern in security in the transport domain is that many different forms of
trust relationship have to work together. Managing trust is one of the more complex of
many security and privacy issues that have to be considered in moving to an auton-
omous vehicle world that intersects to the smart city. Smart cities, smart highways,
smart junctions, in addition to smart vehicles will need to exchange data. Every player
will need to know with a very high level of assurance if the data they are receiving and
expected to act on can be trusted. This requires verifying the integrity, or more com-
pletely the provenance, of data in the system. The methods available to achieve this
kind of active trust verification are complex to install and to manage. One of the major
problems is that there is no a priori knowledge of the forms of interaction a vehicle will
have with the infrastructure. This means that the provenance of a signal or message
from the infrastructure is unlikely to be tested in advance and the normal cryptographic
means of verifying trust need to be supplied on the fly. How such trust is distributed in
the transport infrastructure is one of the biggest unanswered questions regarding the
viability of large scale CAV.
A by-word of security is to anchor things in hardware to then spread roots of trust
through the system. The grand engineering question is where these hardware trust
anchors fit into the built infrastructure. So what are the engineering challenges in
building a strongly anchored, but highly mutable, security system that have to serve
10 s or even 100 s of millions of users exchanging billions of messages a year?
The set of stakeholders in the trusted CAV and smart-entity system that enables it is
considerable and includes, amongst others, the civil engineering industry, the vehicle
manufacturing industry, government, public safety organizations with all of their
implied blue light groups, and the ICT industry amongst many in making a success of
smart transport infrastructure. If this is addressed and trust is endemic to the system
then the promise of an intelligent integrated and society supporting transport system
can begin to be achieved. What is the promise of evolving from our vague notions of
ITS towards the intelligent integrated and society supporting transport system can
achieve?
Trust Provisioning in the Transport Infrastructure 17
Conventional trust hierarchies such as that shown in Fig. 1 have been enthusias-
tically adopted as the norm for the management of cryptographic keys but aren’t all that
accurate when mapped to the real-life trust structures of society.
Root of the
trust
network
First trusted First trusted
level - level -
branch A branch B
Alice Bob Charles
Fig. 1. A conventional view of trust hierarchies
The trust architecture of Fig. 1 suggests that Alice and Bob trust each other by a
shared association to the node above them, but have to go back to the root in order to
trust Charles. Whilst this fits to the cryptographic models of Public Key Infrastructures
it tends to fall down in real scenarios. Thus it is essential to consider characteristics of
trust and to model from there to what we need in ITS and CAV and the smart-city (see
also contributions to standardization made by Cadzow [3–5]):
• Trust is highly dynamic and contextual, and may be described in assurance levels
based on specific measures that identify when and how a relationship or transaction
can be relied upon.
• Trust measures can combine a variety of assurance elements that include identity,
attribution, attestation and non-repudiation.
• An entity A has no need to have a direct trust relationship with another entity B if
B’s operation has no direct impact on A. It may be that entity C is affected by entity
B’s operations, and that entity A relies on entity C, but this does not affect entity A
directly, and therefore the trust relationship can be considered separate.
• The core requirement related to trust in any ICT system is the identification of the
“root of trust”.
• For each element protected within a trust relationship it is necessary to identify
both the root of trust and the path from the protected element to the root of trust.
It is strongly recommended (non-negotiable in some jurisdictions) that the root
of trust is a trusted hardware module (that is able to store keys for example in
tamper-resistant hardware).
18 S. Cadzow
• Having a secured communications channel with another entity is never sufficient
reason to trust that entity, even if you trust the underlying security primitives on
which that communications channel is based.
• Trust is not a binary operation. There may be various levels of trust that an entity
has for another.
• Trust may be relative, not absolute.
• Entity A may trust Entity C more than Entity B, without trusting either absolutely.
• Trust is rarely symmetric. Entity A may trust Entity B completely, whereas the
amount of trust that B has for A may be very low.
• This does not always matter: a schoolchild may trust a schoolteacher, for
instance, without any requirement for that trust to be reciprocated.
• One of the axes for trust is time, and the trust relationship between two entities may
be highly dynamic over time.
• Just because a certain level of trust was established at point T, it does not mean
that that level will be maintained at time T + s, as it can increase and decrease.
3 Allocation of Trust Roots
Roots and anchors are what we need to be concerned about in the transport network’s
data infrastructure. Every stakeholder in the trust network has to be anchored – the
anchor is akin to that of the underpinning of a building and has to be in hardware. If the
data is cryptographically protected it is essential that the keys that enable protection are
themselves firmly rooted in hardware.
In the specific domain of Co-operative ITS in which equipped vehicles send fre-
quent updates of their location to enable receiving entities to make calculations of such
things as collision likelihood, but also to enable situational awareness (i.e. knowing
where vehicles are with respect to the receiver), it is essential that the provenance of the
messages containing this data is strongly asserted. The security model identified in
ETSI’s TS 102 940 [1] and TS 102 941 [2] require that each message is digitally signed
by a pseudonymous identity that has been asserted as valid by a trusted authority. The
unstated requirement is that key pairs are generated in a trusted environment and at a
suitably performant rate – this strongly suggests that only specialized crypto hardware
will fit and this offers the concrete root of trust. Similarly as each key pair requires
attachment to a 3rd party authority this trusted authority, and there may be many
depending on the assertions contained in each message, has to have a concrete
attachment to a trust anchor.
A first step in the allocation of trust anchors is recognition of stakeholder’s role in a
trusted transport infrastructure. From this every stakeholder has to be ready to docu-
ment the form of assertions they make, or rely on, and to identify how they would
enable proof of that assertion. In this regard, proof includes provenance of the
assertion.
In everyday terms the aim is to give the same level of confidence in ephemeral data
that there is in something that is tangible. Thus being able to validate that data from,
say, a Mercedes really comes from a Mercedes and that adopts or inherits the trust
Trust Provisioning in the Transport Infrastructure 19
associated to that marque is key to giving credence to the data. If the data is intended to
alter behavior then this provenance is essential. Hence every stakeholder has to be
ready and able to build such proof into the data they offer.
4 Summary and Conclusion
Every stakeholder in the transport system that has to be trusted when making an
assertion that has to be acted on has to be in a position to verify the provenance of that
assertion or claim. An assertion that is proven without a tangible hardware anchor may
be less trusted than one that is strongly anchored and rooted in hardware.
In moving towards the data centric virtualized infrastructure the concrete of built
infrastructure has to move too. This offers a path towards trust in the infrastructure by
having anchors to the trust relationships, and strong roots building from those anchors.
This is still an evolving area and the benefits of this particular evolution of our transport
world needs considerable investment in time and intellectual effort. This paper has
barely scratched the surface other than to assert that trust has to be anchored in
hardware and the roots drawn from that hardware to the relying parties. The challenges
remaining are considerable to ensure that the future data centric transport infrastructure
has the same strength and resilience as the centuries old built infrastructure we have
relied on for so long.
Acknowledgments. The author thanks CryptaLabs staff, in particular Joe Luong, Justin
Roberts and Alison Roberts, for support in review and discussions the have inspired this paper.
References
1. ETSI TS 102 940: Intelligent Transport Systems (ITS); Security; ITS communications
security architecture and security management
2. ETSI TS 102 941: Intelligent Transport Systems (ITS); Security; Trust and Privacy
Management
3. Cadzow, S.: Security and Privacy Issues Arising from ITS Integration with Smart
Infrastructure, from ETSI ITS Workshop 2019
4. Cadzow, S.: Ethics in AI, from ETSI Security Workshop 2019
5. Cadzow, A.: Are we designing cybersecurity to protect people from malicious actors? In:
Proceedings of the 1st International Conference on Human Systems Engineering and Design
(IHSED2018): Future Trends and Applications, CHU-Université de Reims Champagne-
Ardenne, France, 25–27 October 2018
Drivers’ Interaction with, and Perception
Toward Semi-autonomous Vehicles
in Naturalistic Settings
Jisun Kim1(&), Kirsten Revell1, Pat Langdon2, Mike Bradley2,
Ioannis Politis2, Simon Thompson3, Lee Skrypchuk3,
Jim O-Donoghue3, Joy Richardson1, Jed Clark1, Aaron Roberts1,
Alex Mouzakitis3, and Neville A. Stanton1
1
University of Southampton, Southampton, UK
{J.Kim,K.M.Revell,Joy.Richardson,jrc1g15,apr1c13,
N.Stanton}@soton.ac.uk
2
University of Cambridge, Cambridge, UK
[email protected], [email protected],
[email protected]
3
Jaguar Land Rover, Coventry, UK
{sthom261,lskrypch,jodonog1,
amouzak1}@jaguarlandrover.com
Abstract. Partially automated vehicles are in actual use, and vehicles with
higher levels of automation are under development. Given that highly automated
vehicles (AVs) still require drivers’ intervention in certain conditions, effective
collaboration between the driver and vehicle seems essential for driving safety.
Having a clear understanding about drivers’ interactions with the current tech-
nologies is key to enhance them. Additionally, comprehending drivers’ per-
ceptions toward AVs investigated in naturalistic settings seems important. This
study particularly focuses on usability, workload, and acceptance of AVs as they
are key indicators of drivers’ perceptions. Eight drivers conducted manual and
automated driving in urban and highway environments. Their interactions and
verbal descriptions were recorded, and perceptions were measured after each
drive. Instances that may have negatively affected the perceptions were identi-
fied. The results showed that workload was higher, usability and acceptance
were lower in automated driving in general. Findings show what should be
considered to improve driver-autonomous vehicle interaction, in turn to help
reduce workload, enhance usability, and acceptance.
Keywords: Autonomous vehicles Human-machine interaction Human
factors Workload Usability Acceptance
1 Introduction
Semi-autonomous vehicles are in actual use, and highly automated vehicles are actively
under development. This has been motivated by the major benefit of automated driving
technologies that is insusceptible to human errors [1]. Thus, an interest in the tech-
nology has increased, but concerns have also been expressed regarding system failure,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 20–26, 2020.
https://doi.org/10.1007/978-3-030-39512-4_4
Drivers’ Interaction with, and Perception Toward Semi-autonomous Vehicles 21
overreliance, and insufficient situation awareness [2]. The main concerns on issues
about driver-vehicle interaction, where undesirable consequences may result from
unsatisfactory interaction between the operator and the machine [3]. However, highly
automated vehicles still require drivers’ intervention in certain conditions [4], thus
creating effective HMI will remain as challenging [5]. Therefore, the investigation of
driver-vehicle interaction with existing technologies on actual road conditions would
benefit further enhancement. Additionally, it would be beneficial to improve our
understanding about drivers’ perceptions toward AVs influenced by the interaction.
Acceptance of AVs can be explained by workload, and usability of the system [6–8].
Whilst simulations have widely been selected as a method for investigating driver-
vehicle interaction [9], they might not reflect real-world situations [10], thus the dri-
vers’ responses may be limited. A number of studies conducted on public roads are
documented in academic literature. Revell et al. [11] investigates driver-vehicle
interaction to understand how drivers interact with designs of semi-autonomous vehicle
interaction in a naturalistic setting. Wei et al. [12] introduces an autonomous vehicle
research platform widely tested on public roads, that is enabled to conduct everyday
driving. Wang et al. [13] develop a platform that allows testing driver-vehicle inter-
action on the road without obtaining an actual autonomous vehicle. Endsley [10] runs a
naturalistic driving study about automation features of the Tesla S. Her own 6-month
driving experience is recorded focusing on situation awareness, and problems of
automation in realistic conditions. In summary, a need to investigate driver-
autonomous vehicle interaction in real-world settings is clear, and relevant attempts
have been made. However, few discussions have been implemented to interpret the
dynamics between the interaction and subjective evaluation of workload, usability, and
acceptance. Therefore, this study aims to investigate driver-autonomous vehicle
interaction in naturalistic settings in lights of perceived workload, usability, and
acceptance. This study compares drivers’ perceptions in automated and manual driving,
and uses video and verbal protocol data to understand in greater depth how interaction
incidents explain the results found. Insights into what needs to be considered to
improve driver-vehicle interaction are discussed.
2 Method
The study was designed to enable explorations and comparisons of driver-autonomous
vehicle interaction, and perceptions in naturalistic settings. Participants conducted
driving in urban and highway environments; manual and automated modes; in the
2016 T Model S. It was equipped with Adaptive Cruise Control (ACC); Steering Assist
(S/A); Stop & Go; and Auto Lane Change. The routes were located on the M40 near
Coventry, and public roads in Coventry area. The experiment procedure was as fol-
lows. The participants reviewed an information sheet, and a consent form, signed the
form as an expression to agree to participate. They were given training sessions on the
Jaguar Land Rover test track. In the experiments, they conducted driving on the
planned routes being asked to engage autonomy features as much as they could, but
only when it was safe to do so. For safety reason, a safety driver was ready to take
control whenever circumstances required it. During the drives, think-aloud protocol
22 J. Kim et al.
was used to capture the drivers’ thoughts, and actions. Video and audio data was
collected to record the drivers’ actions, verbal protocols; vehicles’ interfaces; and road
ahead. After each drive, participants completed questionnaires, and had a break.
The NASA Task Load Index [14], the System Usability Scale [15], and the acceptance
of advanced transport telematics scale [16] were used. Mean scores for the first scale,
and overall scores for the last two scales calculated according to the scoring methods
were used for analysis. Box plots were generated for descriptive analysis. For inter-
pretation, median was mainly used considering upper, lower hinges, and extremes.
Video recordings were reviewed to discern instances in which the perceptions may
have been negatively affected. For the analysis, cases were selected that showed
biggest/bigger differences in workload, usability, and acceptance between manual and
automated driving compared to other participants, thus the instances during the drive
warranted reviewing. Healthy adults having a full UK driving license, including 1
female and 7 males, aged between 36 and 59 participated. They included 2 drivers with
more experience in automated driving, and 6 novice drivers of AVs.
3 Results
3.1 Comparisons Between Automated and Manual Driving
Higher workload, and lower usability and acceptance were scored for automated
driving than manual driving both in highway and urban environments considering the
general tendency (see Figs. 1 and 2).
Highway Urban Highway Urban Highway Urban Highway Urban
Manual Manual Automated Automated Manual Manual Automated Automated
Fig. 1. Workload (Left), and Usability (Right) scores for driving in each condition
Drivers’ Interaction with, and Perception Toward Semi-autonomous Vehicles 23
Highway Highway Urban Urban Highway Highway Urban Urban
Manual Manual Manual Manual Automated Automated Automated Automated
Usefulness Satisfying Usefulness Satisfying Usefulness Satisfying Usefulness Satisfying
Fig. 2. Acceptance (usefulness and satisfying) scores for driving in each condition
3.2 Instances Linked to Higher Workload, Lower Satisfaction,
and Acceptance in Automated Driving
In order to identify how workload, usability, and acceptance may have been negatively
affected in automated driving, qualitative analysis was carried out. The cases of
automated driving in highway and urban environments will be discussed. First, par-
ticipant 4’s case presented the biggest differences in workload, usability, and second
biggest difference in satisfying (part of acceptance) score. Main issues were: (1) the
driver indicated when intending to adjusted the ACC speed; (2) the vehicle could not
detect the car behind in the targeted lane, but the vehicle approached the lane when the
auto lane change was on, and this required manual steering input; (3) the vehicle surged
to reach the ACC speed when the driver did not know the reason. Second, participant
6’s case showed the biggest differences in workload, usability, and second biggest
difference in acceptance. Main issues were: (1) the vehicle did not seem to position in
the middle of the lane when only one side of the lane marking, or the vehicle in front
was used for the S/A, and this kind of instances were recorded 5 times, thus the driver
put manual steering input 4 times including 1 sudden input; (2) the vehicle did not stop
at the traffic light because the car did not react to the stationary car in front.
4 Discussion
This section discusses comparison between manual and autonomous driving, and sup-
porting explanation about higher workload, lower usability, and acceptance observed in
automated driving.
Differences between manual and automated driving may have resulted from fre-
quent drivers’ monitoring of the road ahead, and the system; active projections; and
interventions. The processes included checking the vehicles’ automation status, and
behaviour: to see if the vehicle detected the car in front, and lane markings; if the
vehicle reacted accordingly; if not, to intervene. This tended to increase mental demand
[17]. The heightened workload in automated driving does not seem to be consistent
with the most of the previous findings that workload was lower in automated mode [9].
24 J. Kim et al.
The discrepancy may be explained the majority of the studies adopted simulation that
may not have reflected the real world situations [10]. Further, in this study, the drivers
were asked to pay attention to the road conditions all the time, and place their hands
close to the wheel to intervene when needed. Hence, workload was increased, usability,
and acceptance were lower than manual driving that did not require the monitoring, and
immediate intervention when the automation reached the limits.
Instances that could have a negative impact on the measured perceptions are
summarised as follows. First, positions of the ACC and indicator stalks seemed to
cause confusion. They were located close to each other on the left hand side of the
wheel, and this layout seemed to require careful selection. However, the drivers tended
to get used to it as time passed. Second, the failure of detecting the vehicle behind led
to the driver immediately adjusting the steering angle not to depart from the current
lane. The situation required monitoring of the car behind, and the host vehicle’s
behaviour before taking control. Third, the acceleration of the vehicle was a normal
behaviour, but the driver could not understand the intention behind it. Hence, this may
have been perceived as an automation surprise. Relevant knowledge could be gained
through appropriate training to develop more accurate driver’s mental model of the
system [18]. Based on the observations of the first and third cases, more experienced
drivers more familiar with usage of the autonomy features, and having more accurate
mental model would have perceived the instances less demanding. Fourth, the S/A did
not seem to provide stable support, thus the instances were found that the car deviated
from the centre of the lane. This may have resulted from the manufacture’s intention to
allow a slight more lateral movement to replicate human driving (S Thompson 2019,
personal communication), but in this experiment, the instances hardly perceived as
positive experiences. Rather, the situations seemed to require the driver’s interventions.
Fifth, the driver needed to remember limitations of the automation: the vehicle detected
stationary car in front, however, it did not slow down, as the host vehicle did not
recognise the stationary car as significant information for the ACC. As seen in the
second, fourth, and fifth cases, the technical instability led to the circumstances that the
driver had to keep monitoring. If the vehicle did not steer accordingly, the driver had to
take control back. As previously described, these instances could increase workload,
and lower usability, and acceptance.
5 Conclusion
This study was conducted to investigate driver-autonomous vehicle interaction through
assessment of the drivers’ workload, usability, and acceptance in naturalistic settings.
The experiment was run in manual and automated modes, highway and urban envi-
ronments. The result showed that the drivers’ workload was higher, usability and
acceptance were lower in automated driving. Active monitoring of the road, the system,
and the vehicle’s behaviour seemed to increase workload. Thus, the drivers did not
seem to perceive the driving easier, simpler or more effective to use than manual
driving. A number of instances were identified which may have negatively affected the
perceptions. They were the cases of novice drivers’ of AVs. They were summarised as
layout of the stalks which required cautious selection that may cause mistake and
Drivers’ Interaction with, and Perception Toward Semi-autonomous Vehicles 25
confusion; insufficient understanding about the autonomous function that led to an
automation surprise; and technical instability that required the drivers’ intervention.
Improvement of interface design to enable more intuitive usage; appropriate driver
training; and advancement in technology to reduce the occurrence of incorrect actions
of AVs seem necessary to help reduce workload, and improve usability and acceptance.
Acknowledgments. This work was supported by Jaguar Land Rover and the UK-EPSRC grant
EP/N011899/1 as part of the jointly funded Towards Autonomy: Smart and Connected Control
(TASCC) Programme. The authors thank the funders for their support.
References
1. Van Brummelen, J., O’Brien, M., Gruyer, D., Najjaran, H.: Autonomous vehicle perception:
The technology of today and tomorrow. Transp. Res. Part C: Emerg. Technol. 89, 384–406
(2018)
2. Aria, E., Olstam, J., Schwietering, C.: Investigation of automated vehicle effects on driver’s
behavior and traffic performance. Transp. Res. Proc. 15, 761–770 (2016)
3. Degani, A., Heymann, M.: Formal verification of human-automation interaction. Hum.
Factors 44(1), 28–43 (2002)
4. SAE International 2018, SAE International Releases Updated Visual Chart for Its “Levels of
Driving Automation” Standard for Self-Driving Vehicles
5. Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., Nass, C.: Why did my car just do that?
Explaining semi-autonomous driving actions to improve driver understanding, trust, and
performance. Int. J. Interact. Des. Manuf. (IJIDeM) 9(4), 269–275 (2015)
6. Nees, M.A.: Acceptance of self-driving cars: an examination of idealized versus realistic
portrayals with a self-driving car acceptance scale. In: Proceedings of the Human Factors and
Ergonomics Society Annual Meeting, p. 1449. SAGE Publications Sage CA, Los Angeles
(2016)
7. Cummings, M., Clare, A.: Holistic modelling for human-autonomous system interaction.
Theor. Issues Ergon. Sci. 16(3), 214–231 (2015)
8. Jordan, P.W.: An Introduction to Usability. CRC Press, Boca Raton (1998)
9. De Winter, J.C., Happee, R., Martens, M.H., Stanton, N.A.: Effects of adaptive cruise
control and highly automated driving on workload and situation awareness: a review of the
empirical evidence. Transp. Res. Part F: Traffic Psychol. Behav. 27, 196–217 (2014)
10. Endsley, M.R.: Autonomous driving systems: a preliminary naturalistic study of the Tesla
model S. J. Cogn. Eng. Decis. Making 11(3), 225–238 (2017)
11. Revell, K., Richardson, J., Langdon, P., Bradley, M., Politis, I., Thompson, S., Skrypchuk,
L., O’Donoghue, J., Mouzakitis, A., Stanton, N.: “That was scary…” exploring driver-
autonomous vehicle interaction using the perceptual cycle model
12. Wei, J., Snider, J.M., Kim, J., Dolan, J.M., Rajkumar, R., Litkouhi, B.: Towards a viable
autonomous driving research platform. In: 2013 IEEE Intelligent Vehicles Symposium (IV),
p. 763. IEEE (2013)
13. Wang, P., Sibi, S., Mok, B., Ju, W.: Marionette: enabling on-road wizard-of-oz autonomous
driving studies. In: 2017 12th ACM/IEEE International Conference on Human-Robot
Interaction (HRIIEEE), p. 234 (2017)
14. Hart, S.G.: NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the
Human Factors and Ergonomics Society Annual meeting, p. 904. sage publications
Sage CA, Los Angeles (2006)
26 J. Kim et al.
15. Brooke, J.: SUS-a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)
16. Der Laan, Van, Jinke, D., Heino, A., De Waard, D.: A simple procedure for the assessment
of acceptance of advanced transport telematics. Transp. Res. Part C: Emerg. Technol. 5(1),
1–10 (1997)
17. Stapel, J., Mullakkal-Babu, F.A., Happee, R.: Automated driving reduces perceived
workload, but monitoring causes higher cognitive load than manual driving. Transp. Res.
Part F: Traffic Psychol. Behav. 60, 590–605 (2019)
18. Endsley, M.R.: From here to autonomy: lessons learned from human–automation research.
Hum. Factors 59(1), 5–27 (2017)
Are Autonomous Vehicles the Solution
to Drowsy Driving?
Daniel Grunstein1(&) and Ron Grunstein2
1
Kellogg College, University of Oxford, Oxford, UK
[email protected]
2
Woolcock Institute of Medical Research, University of Sydney, Sydney,
Australia
Abstract. The estimated cost of drowsy driving exceeds $100 billion annually
in the US alone. Initial Autonomous Vehicle (AV) literature discusses an
opportunity to partially salvage financial and societal burdens in 93% of acci-
dents which are caused by human error. It is often assumed that AVs are capable
of continuous safe operation during cases of fall-asleep driving. However, the
Dynamic Driving Task (DDT) rather than being suddenly eliminated, is incre-
mentally simplified over time and fall-asleep episodes invariably increase
alongside. This paper suggests some potential consequences for sleep-related
road accidents in the context of AVs and identifies potential future areas of
collaborative research between social and sleep scientists. Its concluding
hypothesis is that whilst full-autonomy is likely to decrease the risk of fatigue-
induced accidents, interim incremental innovations may increase this risk given
the lethal combination of passive sleepiness and driver over-confidence.
Keywords: Autonomous Vehicles Drowsy driving Road accidents
1 Introduction
It is estimated that approximately 40% of fatal crashes are primarily caused by sub-
stance abuse, distraction and/or fatigue [1]. Full vehicular autonomy presents an
opportunity to offset or possibly eliminate medical, economic, social and infrastructural
issues resulting from drowsy driving and startling evidence indicative of its increasing
incidence and severity. Should AVs be capable of operation without human inter-
vention then the drowsy driver becomes a drowsy passenger who can sleep safely. This
would not only reduce vehicle crashes attributed to fall-asleep and micro-sleep episodes
and impaired judgement during drowsiness, but also see substantial awake commuting
hours substituted for rest hours leading to a less sleep-deprived society.
Consideration of this scenario does not imply exact knowledge of the likelihood or
a timeline to reaching this goal. Trying to predict the nature, timing and adoption of
new inventions is an extremely uncertain science. Thus, this paper makes no prediction
as to the AV technology timeline. Rather it works with facts on the ground, namely that
the most advanced vehicle currently being tested is semi-autonomous not fully
autonomous. Therefore, research concerning implications for drowsy drivers should
prioritise the semi-autonomous context over fully autonomous scenarios.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 27–33, 2020.
https://doi.org/10.1007/978-3-030-39512-4_5
28 D. Grunstein and R. Grunstein
2 The Taxonomy of Autonomous Vehicles
Many adjectives are used to describe the extent of human operation of motor vehicles;
e.g. automatic, autonomic, automated, cooperative, and autonomous.
Automatic transmission describes a vehicle which automatically changes gears
without the need for a gearbox and clutch (i.e. manual transmission). Autonomic is
used to describe an autonomous feature in a vehicle that itself may not necessarily be
autonomous, usually the vehicle is classified as ‘automated’ but could also simply be
automatic/manual. It is often used in patents for specific component technologies e.g.
the Autonomic vehicle safety system [2, 3]. Additionally, the Society of Automotive
Engineers (SAE) International’s J3016, the golden standard for vehicular autonomy
classification, holds that if self-driving depends on communication with outside entities
like a control room, the vehicle should be classified as cooperative instead of auton-
omous [4]. Autonomous systems are capable of making decisions independently of
human intervention, while automated systems require intervention in situations for
which they have not been programmed [5]. In its original January 2014 publication, the
SAE delineated six levels of automotive autonomy, according to this classification,
vehicles are autonomous at Level 4 and fully autonomous at Level 5.
3 What Is Drowsy Driving?
Drowsy driving implies controlling a motor vehicle under conditions of increasing
sleepiness or an incipient fall-asleep episode. Sleepiness may be the result of acute or
chronic reduction in sleep duration in individuals without sleep disorders driven by
both homeostatic and circadian factors [6]. Generally, the longer a person has gone
without sleep, the sleepier they are (homeostatic process) but this relationship is also
impacted by presence or absence of a circadian alerting drive, making night time
driving more vulnerable to drowsiness. Other diverse factors related to a higher inci-
dence of drowsy driving include duration of the drive, age of individual (younger more
affected), co-existing use of central nervous system depressants such as alcohol and
presence of sleep-fragmenting disorders including obstructive sleep apnea (OSA) [7].
Many medical conditions, both primary sleep disorders (such as OSA and insomnia)
and other chronic medical conditions that impact sleep quality (such as lung disease,
asthma, renal disease, depression, and anxiety) can lead to sleep-related cognitive
decrements that increase crash risk [8]. Sleepiness can be increased by monotony with
drowsy driving more common when there is less environmental stimulation.
The exact impact of drowsy driving is unknown. This depends on crash reports
from police who often have limited expertise in drowsy driving or assign causes
erroneously (e.g. a crash may be blamed on excessive speed when the driver may have
not slept for 24h and exhibited increased risk taking and poor judgement). The annual
economic cost of drowsy driving has been estimated at $109 billion in the USA [9].
Are Autonomous Vehicles the Solution to Drowsy Driving? 29
4 Drowsy Driving and Autonomous Vehicles
Research addressing drowsy driving in the context of AVs is understandably limited
given that the technology is only in pilot-stage and is not expected to reach the market
for several more years. The gradual ‘autonomization’ of passenger vehicles has been
occurring for over two decades beginning with automatic transmission and later fea-
tures such as Automatic Cruise Control (ACC) and Highly Automated Driving (HAD –
automation of both longitudinal and lateral motion) [10]. Additionally, innovation in
products that are complementary to automobiles themselves such as Satellite Naviga-
tion, reversing sensors, lane-keeping assistance and E-payment for toll-roads have also
contributed to the simplification of the DDT in recent decades. There is ample evidence
to suggest that increased simplification of DDT increases the likelihood of driver
drowsiness – for every innovation that simplifies the task, the likelihood of giving in to
drowsiness progressively increases [11].
Full autonomy (Level 5) might relieve the major societal burden of sleep lost to
commuting commitments by allowing the driver to safely sleep in a moving vehicle.
However, AV technology has and is expected to continue along a path of incremental
innovation and therefore technology, and its regulation, are likely to only initially
facilitate on-road AVs with a driver, or in actuality ‘a passenger who knows how to
drive’ – ready to take control in cases of disengagement (Level 4). Under these cir-
cumstances, the driver needs to be alert, not drowsy and certainly not asleep. The
incremental innovation over past decades and drowsy driver research alongside would
predict that under Level 4 autonomy drowsy driver incidence might increase signifi-
cantly given the task simplification. Under Level 5 autonomy, if the technology reaches
that point, incidence will be even higher, but the negative implications for society will
be allegedly eliminated.
Due to the hypercompetitive landscape that consists of traditional automobile
OEMs (e.g. Daimler, GM, Ford etc.), Technology Companies (e.g. Google, Apple,
Intel, Nvidia etc.) and Ridesharing Companies (e.g. Gett, Uber, Yandex and Lyft), a
culture of trade secrets is pervasive and it is difficult to get an accurate picture of where
the technology is at. Nevertheless, it is known that the vehicles being tested on-road
today are Level 4. There have already been cases of accidents caused by human error in
these pilot journeys, and indeed the one recorded fatal accident was a result of driver
distraction [12]. There has been no inquiry as to the role of drowsiness in these
incidents, and this should be rectified as regulatory oversight is formalized.
The state of the market would suggest that Level 4 semi-autonomy is forthcoming
as a mass market reality whereas the exact pipeline and social/regulatory shifts required
to bring Level 5 to a similar state are difficult to determine. Driver fatigue has been
cited as a key motivator behind the race to perfecting vehicular autonomy [13]. If the
DDT is too simple in semi-autonomous vehicles the passengers designated as emer-
gency drivers in cases of disengagement may experience passive fatigue despite the
potential need for intervention in extreme situations. This inference is drawn from
experience with tasks of low cognitive load where subjects invariably have direct
control over the task at hand [14]. Domain-specific research has demonstrated the
inverse relationship between passive fatigue and driver performance [15]. Conversely,
30 D. Grunstein and R. Grunstein
boredom may also proliferate from low workload in periods of automated driving
causing drivers to seek more entertaining tasks. This has serious consequences for
situations when the driver must intervene during a disengagement [16].
5 Drowsy Driving and the Current State of the AV Market
5.1 Ridesharing/Ride-Hailing
Some of the most advanced players in the AV market are ride hailing companies. The
long-term AV landscape is also dominated by discussion of a shared vehicle network as
opposed to today’s largely owner-based model. In order to get to a fully autonomous
ride hailing network, paid drivers will likely operate interim AV prototypes (Level 4) as
some of the most frequent ride hailers (the youth, elderly, disabled, and inebriated) are
not able to safely take control in event of a disengagement. A trained driver is also
needed in case of disengagement between journeys.
The largest impediment to achieving enough sleep is the number of hours worked
[17]. Many drivers in the ridesharing industry view this type of work as an additional
income stream. Drivers often work a primary job and drive in their “off” time, such a
schedule may lead to driving after extended periods of wakefulness or during nights.
Working long and irregular hours (especially at night), or having multiple jobs, sub-
stantially increases the risk of work-related accidents and automobile crashes [18]. In
the near future while AV technology remains at Level 4 or below, ridesharing vehicles
will be staffed by a driver who is at increasing risk of harmful drowsiness.
5.2 Autonomous Heavy (Freight) Vehicles
Freight vehicles, if totally autonomous are only carrying goods not people, and are
therefore considered a popular initial use-case for full-autonomy (Level 5). Though the
risk might be lowered by conducting pilots in freight vehicles, these AVs would share
the road with smaller passenger vehicles and any malfunction resulting in a collision
could be fatal given the size of the freight truck. Proponents argue that trucks drive on
highways which are more spacious and have less obstacles for the AV to consider, and
that freight road transport is often overnight when there are less passenger cars on the
roads. A reasonable conclusion is that testing Level 5 technology in freight vehicles
should be considered risk minimizing rather than risk mitigating.
Testing (currently ongoing) [19] and eventual roll-out of Level 4 vehicles may also
see heavy freight vehicles become an attractive starting point for similar reasons. In this
case, the driver ready to intervene will likely be a regular truck driver. These drivers are
significantly overrepresented in road deaths internationally, often due to lifestyle fac-
tors making them susceptible to drowsy driving [20]. For example, deaths related to
heavy-vehicle crashes in the United States comprise 8% of all road deaths, despite
accounting for only 1% of vehicle registrations [21]. One reason may be that OSA is
common in heavy vehicle drivers [22]. Also significant is the challenging work con-
ditions involving long distances and driving durations coupled with tight delivery
schedules. Case-control research comparing drivers involved in non-fatal, non-serious
Are Autonomous Vehicles the Solution to Drowsy Driving? 31
crashes show night time driving and work hours to be strongly related to crash risk
[21]. Further simplification of a task performed by an at-risk population with dispro-
portionally high rates of drowsiness will possibly increase likelihood of accession to
that drowsiness. It could be a dangerous combination during roll-out of Level 4
autonomy in heavy vehicles.
5.3 Encouraged Adoption of Collision Avoidance Technologies
Human error or poor decision making was the primary causal factor in 94% of motor
crashes, according to a NHTSA report [23]. The US DoT is working to accelerate the
spread of crash-avoidance technologies that have the potential to prevent the thousands
of deaths caused by human error. The Department is also working to make vehicle-to-
vehicle safety communications a part of future fleets and to identify and address
potential obstacles to safety innovations within existing regulations [24]. While safety
innovations based on vehicle automation may be able to prevent some drowsy driving
crashes, intermediate stages of automation will require human oversight. Such inter-
actions between driver and vehicle will likely be impacted by drowsiness. Some of
these technologies may lead to greater levels of drowsiness as less driver involvement
is required due to increasing levels of automation and may constitute an emerging
safety concern that should be monitored as these systems become more ubiquitous.
Finally, there is a notable exception here and that is with respect to technologies that
specifically address drowsiness. For example, Mobileye sounds a loud beep when the
car leaves its lane, approaches a pedestrian or moves too close to a vehicle in-front.
A driver who has fallen into micro-sleep will be jolted awake by audio and visual
stimuli [25]. The mitigating role of driver drowsiness detection technologies in the
pathway to Level 5 is not limited to car-based devices like Mobileye but also eye
closure or other drowsiness detection methods.
6 Conclusions and Discussion of Topics for Further Research
Research in this emerging domain will need to expand upon realms of both ergonomics
and sleep science, and ideally both concurrently. There is a need to expand on the line
of inquiry taken in this paper and also to explore plenty more ergonomic features of
increasingly autonomous/automated driving technologies which have a strong rela-
tionship with drowsy driving. Subjects of further research in this domain might include
overreliance and trust of the system (the AV), skill degradation and others. On the sleep
medicine side, there is a need to evaluate some existing assumptions and future
assumptions through credible trials, surveying and data analysis. Given the strong
evidence supporting significance between lifestyle factors and cases of drowsy driving
we determine a good starting point would be to research potential changes in time use
in the context of vehicles that increasingly operate autonomously. This informs our
next line of inquiry on this subject and will put to the test some of the hypotheses
outlined above.
32 D. Grunstein and R. Grunstein
References
1. Fagnant, D., et al.: Preparing a nation for autonomous vehicles: opportunities, barriers and
policy recommendations. Transp. Res. Part A: Policy Pract. 77, 169 (2015)
2. Uchida, T., et al.: Autonomic traveling apparatus for a vehicle. US Patent 8,548,664 (2013)
3. Mays, W.: Autonomic vehicle safety system. US Patent 12/366,521 (2010)
4. SAE International: Taxonomy and Definitions for Terms Related to Driving Automation
Systems for On-Road Motor Vehicles, SAE International (2014)
5. Brodsky, J.: Autonomous vehicle regulation: how an uncertain legal landscape may hit the
brakes on self-driving cars. Berkeley Technol. Law J. 31(2), 851–878 (2016)
6. Smolensky, M., Di Milia, L., Ohayon, M., Philip, P.: Sleep disorders, medical conditions,
and road accident risk. Accid. Anal. Prev. 43(2), 533–548 (2011)
7. Tregear, S., et al.: Obstructive sleep apnea and risk of motor vehicle crash: systematic review
and meta-analysis. J. Clin. Sleep Med. 5(6), 573–581 (2009)
8. Howard, M., et al.: Sleepiness, sleep-disordered breathing, and accident risk factors in
commercial vehicle drivers. Am. J. Respir. Crit. Care Med. 170(9), 1014–1021 (2004)
9. Higgins, J., et al: Asleep at the wheel-the road to addressing drowsy driving 40 (2017)
10. De Winter, J., et al.: Effects of adaptive cruise control and highly automated driving on
workload and situation awareness: a review of the empirical evidence. Transp. Res. Part F:
Traffic Psychol. Behav. 27, 196–217 (2014)
11. McDowell, K., Nunez, P., Hutchins, S., Metcalfe, J.: Secure mobility and the autonomous
driver. IEE Trans. Robot. 24, 688–697 (2008)
12. BBC News: Uber not criminally liable’ for self-driving death, 6 March 2019. https://www.
bbc.co.uk/news/technology-47468391. Accessed 26 Sept 2019
13. Oron-Gilad, T., Ronen, A.: Road characteristics and driver fatigue: a simulator study. Traffic
Inj. Prev. 8(3), 281–289 (2007)
14. Desmond, P., Hancock, P.: Active and Passive Fatigue States, in Stress, Workload, and
Fatigue, pp. 455–465. Lawrence Erlbaum, Mahwah (2001)
15. Neubauer, C., Matthews, G., Langheim, L., Saxby, D.: Fatigue and voluntary utilization of
automation in simulated driving. Hum. Factors 54(5), 734–746 (2012)
16. Stanton, N., Young, M., et al.: Drive-by-wire: the case of driver workload and reclaiming
control with adaptive cruise control. Saf. Sci. 27(2–3), 149–159 (1997)
17. Basner, M., Fomberstein, K., Razavi, F., et al.: American time use survey: sleep time and its
relationship to waking activities. Sleep 30(9), 1085–1095 (2007)
18. Stutts, J., Wilkins, J., Scott, O., Vaughn, B.: Driver risk factors for sleep-related crashes.
Accid. Anal. Prev. 35(3), 321–331 (2003)
19. Anmar, F.: Self-driving trucks are being tested on public roads in Virginia, 10 September
2019. https://www.cnbc.com/2019/09/10/self-driving-trucks-are-being-tested-on-public-
roads-in-virginia.html. Accessed 30 Sept 2019
20. Australian Bureau of Statistics, Motor Vehicle Census, ABS, Canberra, Australia (2011)
21. Stevenson, M., et al.: The role of sleepiness, sleep disorders, and the work environment on
heavy-vehicle crashes in 2 Australian states. Am. J. Epidemiol. 179(5), 594–601 (2014)
22. Sharwood, L., Elkington, J., Stevenson, M., Grunstein, R., et al.: Assessing sleepiness and
sleep disorders in Australian long-distance commercial vehicle drivers: self-report versus an
at home monitoring device. Sleep 35(4), 469–475 (2012)
23. National Highway Traffic Safety Administration, Drowsy driving. https://www.nhtsa.gov/
risky-driving/drowsy-driving. Accessed 20 Mar 2018
Are Autonomous Vehicles the Solution to Drowsy Driving? 33
24. National Transpiration Safety Board, NTSB 2017–2018 Most Wanted List of Transport-
ation Safety Improvements. https://www.ntsb.gov/safety/mwl/Pages/mwl6-2017-18.aspx.
Accessed 20 Mar 2018
25. Mobileye, Wake Up for Lane Departure Warnings, 24 October 2018. https://www.mobileye.
com/us/fleets/blog/wake-up-for-lane-departure-warnings/. Accessed 29 Sept 2019
Exploring New Concepts to Create Natural
and Trustful Dialogue Between Humans
and Intelligent Autonomous Vehicles
Andrea Di Salvo(&) and Andrea Arcoraci
DAD - Department of Architecture and Design, Politecnico di Torino,
Viale Mattioli, 39, 10125 Turin, Italy
{andrea.disalvo,andrea.arcoraci}@polito.it
Abstract. In recent years, research on autonomous cars had an ever-increasing
spread, exploring many of the technological elements, the intelligence necessary
to build hardware and connections, the limits linked to legislation and ethical
aspects. The automotive field is rich of interior style solutions, but the topics
linked to interaction design and digital user experience are relatively young and
need a different approach. In fact, if the higher limits were considered the
cognitive workload, the attention and the reaction times required by a still semi-
automatic drive, what are the methods to follow in order to obtain an innovative
user experience, that also considers the aspects of complexity and of personal
digital connections? The presented case study aims to describe the method-
ological approach and the first concepts developed for the research project that
studies an innovative interface system for an autonomous vehicle, able to travel
both on the ground and in flight.
Keywords: Interaction design User experience Design concept
1 Introduction
Autonomous driving is a research field that is continuously developing since many
years, the focus is based on the ambition to completely automate the driving process,
integrating the use of sensors and actuators connected to both an internal car software
system and to the mobility system. The primary goals of these projects are: a sub-
stantial increase in safety [1, 2], let the car passengers free from driving tasks in order
to allow them to engage in other activities [2], to increase the sustainability of the
mobility system considering cars as a part of an integrated service based on sharing [3,
4]. To deal with such a complex challenge, car manufacturers and research institutions
are following two paths, the first specifically concerns with the safety of the car, the
second imagines new on-board configurations. The latter task is usually managed by
the style centers that design solutions based on formal languages, while the interaction
modes are still conditioned by the reached transition state. The presence of driving
controls in semi-autonomous driving cars impacts not only on the structure of the
interior but on interaction and experience. In a semi-autonomous vehicle, in fact, the
responsibility of the vehicle is still of the driver and the other activities are strongly
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 34–40, 2020.
https://doi.org/10.1007/978-3-030-39512-4_6
Exploring New Concepts to Create Natural and Trustful Dialogue 35
limited by the response times that the driver must maintain to return in the loop. Then,
the topics linked to interaction design (IxD) and digital user experience are still rela-
tively young because they are still closely related to the driving activity. In fact,
considering the perspective of a fully autonomous car, interaction designers are sup-
posed not to consider only actual primary limits coming from cognitive ergonomics,
such as the cognitive workload. On the contrary they should focus on other aspects,
such as the critical shift in the division of driving task, that needs to reconsider a
communication between users and cars. The users will move from performing the
driving to managing several elements (e.g. setting direction and choosing driving style)
and letting the system execute the action as it knows best [5]. The user will continu-
ously evaluate the performance of the vehicle to build acceptance and trust [6], if these
don’t meet the expectations, the potential risk is to decrease user trust. With a low trust
level, the user may not only dismiss the particular vehicle but dismiss autonomous
vehicles altogether [5]. In order to be accepted, autonomous vehicles have to be per-
ceived as useful, as well as safe and competent [7]. Interaction designers, in conclusion,
have the responsibility to focus on approaches and methods to obtain a completely
different UX, also considering the aspects of complexity and of personal digital
connections.
2 Scenario
The 5 automation levels of a vehicle [8] represent a roadmap, however, that document
does not suggest which other activities could be carried out on board besides driving.
Furthermore, the level 5 of automation appears technologically achievable today, but
the lack of connection to the mobility system is evident, at least as far as cars are
concerned. Trucks manufacturers worked on different bases and they are putting efforts
to create a shared system to test the platooning techniques i.e. a semi-autonomous
mode that requires an interoperable system capable of coordinating many vehicles.
There are many differences between trucks and cars, but the common factor concerns
interaction [9] and on-board experience. Despite users are free from the driving task,
they still interact with the system, but it will happen differently [10]. The users will
have a seemingly more passive role inside the vehicle as their ability to send direct
input to the system through commands (e.g. pedals, steering) decreases or disappears at
all. Instead, the emphasis will be more on output information that user receives from
the system [5]; this information will be crucial to increasing the perception of pre-
dictability and then building user’s trust [11]. As the way of communicating between
the users and system will change, the traditional interface is no longer suitable in the
new configuration of fully autonomous vehicles and the new one should be also
designed to convey new type of information such as intentions and awareness [5].
Furthermore, the interface placement itself will be different and users will be involved
in much more activity and will experience the cabin in a totally different way [12].
These revolutionary factors open the possibility to reconfigure the vehicle under many
aspects covering both sides, physical and digital. Today, tests in real contexts focus on
the relations between the number of disengagements [13] and the possible causes, and
on the reaction time that the driver had in front of an unexpected event. The driver’s
36 A. Di Salvo and A. Arcoraci
monitoring phase is, therefore, relegated to safety critical situations or to keep the
driver in the loop by assessing the level of attention or fatigue [10]. Much work,
instead, has been implemented to evaluate other aspects that can be considered as part
of UX. First of all, the level of acceptance and the consequent trustability perceived not
only by those aboard autonomous vehicles, but also by other actors in the mobility
system. A trend that has a significant impact on the UX concerns the detection of
emotions on board and data potentially useful to create passengers’ profiles [14]. This
can be already done technologically through personal devices, like wearables, and on-
board sensors, but it allows in perspective to connect the user’s emotions to the car’s
behavior and to the on-board interfaces, making the cabin more relational. This
research field can also be faced during the metadesign phase through customer journeys
that design sentiment and emotional reactions. This approach can be considered as
speculative and broadens the design horizon, usually linked to the concept of HMI to
the dimension of mobility as a service.
3 Design Approaches and Methods
The methodological approach intersects IxD, UX and design fiction [15, 16] in order to
build all the design elements based on a plausible story and, at the same time, with a
wider overview that these projects need. Consequently, HCD is shifted in a more
speculative approach. Although the project was initially structured on a megacity, we
decided to work on the city of Turin, the location of the design course of study, so as to
be able to carry out a detailed exploration of the territory. The process started with a
user research phase, integrating a desk research based on mobility reports and data
released by public institution, to an in-depth study of new interactive technologies
released on the market or in embryonic stage. This phase was complemented by a
hands-on research in which the involved students tested all the sharing and pooling
services available in Turin. Enlarging the research field also to bike sharing, cargo or to
pooling services allowed to shift the focus more on the service, in order to structure the
customer journey without focusing on the only HMI which has, however, been ana-
lyzed in detail. Only after this phase, the research project was presented, which already
included a prototype and an interface that were in turn explored hands-on. Following
this phase Personas were built. Although data came from user research and from further
demographic analyses obtained through experts, the imaginative effort is attributable to
that carried out by design fiction. Creating narrative and filmic characters able to
distinguish themselves from the usual businessman that appears in the initial narration
of the project. The next step has been the creation of the interface concepts. Even in this
case, the methodology has drawn on the typical techniques of filmic narration, iden-
tifying the interface as another character with a specific personality. This approach
proved to be very useful for defining the behaviors and interactions of interfaces. As a
result of the concept, a Customer Journey was realized for each Persona and therefore
all the interactive and graphic languages. The project ended with a communication
video able to show the UX on board.
Exploring New Concepts to Create Natural and Trustful Dialogue 37
4 The Research Project
This project is part of a research program named and involves a company of the Turin
territory which developed a concept of fully electric and fully autonomous modular
system. One of the peculiarities is its ability to reconfigure according to the user needs,
it can move inside and outside the urban context in both ways, by ground and by flight.
Relevant factors as the rise of urban density and the increase of megacities across the
world expected for the next fifty years led the transportation market to explore new
concept and new ways of thinking about urban movements. Another big trend is the
dismission of private vehicles in favor of the spread of shared vehicles [17], this shift
should make the whole mobility system more efficient and sustainable, mitigating
problem as congestion and pollution. Therefore, the vehicle has been designed as a
service. This characteristic, together with the double mode of movement and the fully
automation of the vehicle, will dramatically change the experience of city moving. The
role of this research project is thus to understand the future needs of and desires of the
actors within the new mobility system, imagine and rethink the possible UXs within
autonomous drive vehicle, and exploring possible applications in terms of innovative
technologies.
5 The Concepts
The high-level interaction concepts that are presented are categorized according to the
grammar of the interaction and the experience on board. They have been developed by
8 groups of students during the master’s degree course. In particular, the concepts
explore the way in which: interfaces and artificial intelligence will recognize and
answer to the user’s specific needs; personal and on-board devices will handle emer-
gencies, delays, malfunctions. Particular attention has been dedicated to the emotional
aspects of the user and the way of monitoring them, in order to design the system’s
behaviors and the constant dialogue through the interfaces. The interaction modes are
shown in the figure and in the description that follows, leaving out the different services
that vary a lot from project to project (Fig. 1).
Fig. 1. The 8 high level concepts
38 A. Di Salvo and A. Arcoraci
The extended personal is based on a hologram projected by the personal device,
connected to the car, which answers to the needs expressed by the user. It is not limited
to answering but rather, given its connection to the user’s personal data, it is able to
anticipate requests and reshape the interactive elements of the cabin according the
detected emotions. The interface becomes multisensory, the main dialogue is based on
the voice, but the connections makes it really personal. In this case AI is an extension
the personal device, capable of communicating with the on-board system. Relaxed
Monitoring conceives the car as part of a stand-alone delivery service. The interface is
therefore useful for constantly programming and monitoring the journey from the
user’s personal device. It assumes the dimensions of a current app with the aim of
reassuring both the sender and the receiver. Pick up needs interface is based on the
evolution of direct manipulation, using three-dimensional holographic projections able
to provide tactile feedback. The user can interact with the on-board systems using his
hands. The grammar of interaction is radically different and is initially based on a 6-
sided cube, each side allows entering a specific section of the system. In addition to the
usual infotainment the interface allows, by pinching the corners of the cards, to explore
the contents more and more in depth. The whole surrounding context is reconstructed
and visualized in 3d allowing also a punctual interaction with the city. 360° 360°
interaction is based on free touch gestures, while the interface is developed on all the
glass surfaces of the cabin. This allows not only an interaction with the context through
augmented reality but amplifies the possibilities of vision, projecting for example
immersive 360° contents on treated surfaces; the system agrees with the user a series of
activities on board connected to the motivation of the journey, like going to a concert or
to an event. Continuous Communication’s interface is designed to continuously
communicate with users through a tactile interface, specifically designed for visual
disabilities, but scalable to all users. For continuous communication we do not intend to
leave the channel always open, as happens for voice assistants, but to trigger a con-
tinuous and natural dialogue that reassures the user during all phases of the journey.
Playful Grammar changes interaction grammar into visceral, referring to the Norman
categories, gestures are not learned by the user but developed by the user according to
his needs and expectations. In this way, users can feel engaged by a new and peculiar
language, also aimed at amazing the other passenger. Body touchless interaction with
the contents is no longer tied to an object or display, but it shapes itself on the body of
the user while remaining invisible to others. Based on the need for privacy and hygiene
on board a shared vehicle, the user can, for example, write tapping on his legs using a
virtual keyboard projected onto his device. The movement of his fingers is detected,
and the user can easily look directly to the device. Although lacking in a tangible tactile
interface, the concept leaves user’s arms and hands in a relaxed position. My favorite
instrument turns the car into an extension of users’ very professional instruments in a
disruptive way, the windows and the windscreen together with a voluminous control
console can become the studio of a photographer working directly during the flight, or
a recording studio for a musician, or a reporter.
Exploring New Concepts to Create Natural and Trustful Dialogue 39
6 Conclusion and Future Works
The presented high-level interaction concepts are characterized by a futuristic vision of
technologies and a paradigm shift in the grammar of interaction. This is a speculative
work that figures out a scenario in which autonomous-shared mobility becomes crucial
in the city environment, highlighting the fundamental role of the UX and the IxD. At
present, concepts have been tested in some vertical aspect, presented and discussed
with the other project partners. Despite the critical issues related to the technological
feasibility of some concepts, this research phase underlined how wide the margin of
innovation is and what is the role of IxD and UX in the design, not only of the
interfaces but also of the behaviors that the AI should play within the dialogue with
users. The next steps undoubtedly concern an intensive phase of tests that consider,
above all, the other parameters that contribute to the perception of comfort such as the
flight context and the time duration.
References
1. Andersen, J., et al.: Autonomous Vehicle Technology. RAND, Santa Monica (2016)
2. Fagnant, D., Kockelman, K.: Preparing a nation for autonomous vehicles: opportunities,
barriers and policy recommendations. Transp. Res. Part A: Policy Pract. 77, 167–181 (2015)
3. Pavone, M., et al.: Robotic load balancing for mobility-on-demand systems. Int. J. Robot.
Res. 31(7), 839–854 (2012)
4. Spieser, K., et al.: Toward a systematic approach to the design and evaluation of automated
mobility-on-demand systems: a case study in Singapore. In: Road Vehicle Automation,
pp. 229–245 (2014)
5. Strömberg, H., et al.: HMI of autonomous vehicles - more than meets the eye. In: Advances
in Intelligent Systems and Computing, pp. 359–368 (2018)
6. Ghazizadeh, M., et al.: Extending the technology acceptance model to assess automation.
Cogn. Technol. Work 14(1), 39–49 (2011)
7. Choi, J., Ji, Y.: Investigating the importance of trust on adopting an autonomous vehicle. Int.
J. Hum.-Comput. Interact. 31(10), 692–702 (2015)
8. SAE International: Taxonomy and definitions for terms related to driving automation
systems for on-road motor vehicles. SAE International, (J3016) (2016)
9. Ramm, S., et al.: A first approach to understanding and measuring naturalness in driver-car
interaction. In: Proceedings of the 6th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications - AutomotiveUI 2014 (2014)
10. Helgath, J., et al.: Investigating the effect of different autonomy levels on user acceptance
and user experience in self-driving cars with a VR driving simulator. In: Design, User
Experience, and Usability: Users, Contexts and Case Studies, pp. 247–256 (2018)
11. Ekman, F., et al.: Exploring automated vehicle driving styles as a source of trust information.
Transp. Res. Part F: Traffic Psychol. Behav. 65, 268–279 (2019)
12. Payre, W., et al.: Intention to use a fully automated car: attitudes and a priori acceptability.
Transp. Res. Part F: Traffic Psychol. Behav. 27, 252–263 (2014)
13. Dixit, V., et al.: Autonomous vehicles: disengagements, accidents and reaction times.
PLoS ONE 11(12), e0168054 (2016)
14. Amazon Technologies Inc.: Passenger profiles for autonomous vehicles (2018)
40 A. Di Salvo and A. Arcoraci
15. Dunne, A., Raby, F.: Speculative Everything: Design, Fiction, and Social Dreaming.
The MIT Press, Cambridge (2013)
16. Bleecker, J.: Design Fiction: A Short Essay on Design, Science, Fact and Fiction, p. 29. Near
Future Laboratory, Los Angeles (2009)
17. Kim, S., et al.: Autonomous taxi service design and user experience. Int. J. Hum.–Comput.
Interact. 1–20 (2019)
Integrating Human Acceptable Morality
in Autonomous Vehicles
Giorgio M. Grasso, Chiara Lucifora(&), Pietro Perconti,
and Alessio Plebe
Department of Cognitive Science, University of Messina, Messina, Italy
{gmgrasso,clucifora,perconti,aplebe}@unime.it
Abstract. Our study aims at progressing the assessment of the moral behaviour of
human drivers. It is a felicitous coincidence that psychological and philosophical
research into human morality has been dominated by thought experiments,
resembling vehicles facing emergency situations. These thought experiments
involve a running trolley, and have been used to contrast different moral principles,
especially deontology versus utilitarianism. We designed an ecologically valid
trolley-like dilemma with the help of virtual reality, aimed to understand the moral
behavior of human subjects when facing a car accident situation. We report and
comment on early results of our first tests.
Keywords: Moral dilemma Virtual reality Autonomous driving system
1 Introduction
Recent advances in deep neural algorithms and in sensor technologies are pushing the
deployment of autonomous vehicles close to reality [11, 15]. It is time to reflect on the
social integration of fleets of autonomously driven cars and trucks with humans. Within
this perspective, some of the most compelling challenges concern moral issues, like
those arising when a vehicle is facing emergency situations [3, 5]. This issue cannot be
addressed from a pure engineering standpoint only, because it typically involves the
comparison of different choices with different predictable damages. Ranking different
possible harms pertains to human morality. In principle, the engineering solution
should integrate in the vehicle design a decision logic yielding choices as close as
possible to those humans would have chosen in the same situation. However, it is far
from being clear how human drivers behave in emergency situations requiring moral
decisions.
In our opinion, a remarkable advantage of still using trolley-like dilemmas is the
careful critical analysis undertaken in the last decades on their legitimacy as experi-
mental stimuli. There are known limitations and drawbacks, and there are prescriptions
for exploiting at best this experimental paradigm.
For example, Himmelreich [9] argues that the moral dilemma is not relevant for the
autonomous vehicle, because the hypothesized situations rarely occur in the real world,
and when it happens the speed of the autonomous vehicle is not sufficient to make a
choice; Nyholm and Smids [14] remark that moral dilemmas ignore some proprieties of
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 41–45, 2020.
https://doi.org/10.1007/978-3-030-39512-4_7
42 G. M. Grasso et al.
the real world, such as who is the responsible for the road accident, and ignore
information about special obligations, that in the real world are very important to
understand the moral permissibility. Even more serious is the criticism that moral
studies using trolley-like dilemmas as stimuli lack of ecological validity [4, 13]: moral
mental processes used in real situations perform differently when tested in an unusual
conceptual context. Therefore, we took great care on this issue. We designed a driving
simulator in realistic city traffic scenarios, in which subjects are immersed wearing a
virtual reality helmet, driving using car-like steering wheel and pedals. Moreover, even
if our stimuli are highly ecological, we try further to distinguish moral judgments from
moral behavior in the driving context. We administered two different tasks to the
subjects: a first one recording in real time the choice actually made in the critical
situation by driving, and a second one halting the driving simulation, and allowing the
subjects enough time to reflect before deciding their action, and we ask subjects to
report the motivations of their choices.
Abundant evidence in the literature indicates that the reported motivations of
subjects may diverge from their unconscious decision process [10] including moral
decisions. Therefore, it is useful to understand if there are systematic discrepancies
between what human subjects deem is morally correct when driving, and the moral
choice they make automatically in emergency driving maneuvers.
2 Our Research
In our approach, we think that it is possible to correlate ethics with engineering,
although ethics has mainly taken a top-down approach and engineering uses more often
a bottom-up approach, in which the actions are not guided by explicit rules. In fact,
there are many studies that use a top-down approach to build the moral artificial agent.
For example, Wienfield et al. (2014) uses a clearly top-down approach to construct
robots capable to implementing Asimov’s first law, and Anderson et al. [2] built a robot
based on a specific ethical theory, namely principalism.
So, we must distinguish between an implicit ethical agency to an explicit ethical
agency [6]. Based on the first, it is possible to build agents acting on ethically relevant
consideration, but this does not mean that the agent should rely on such considerations
to explain or justify his choices, so it is possible to speak about “functional morality”
[1, 16] in which the system simply act within acceptable standard. As stated in the
Introduction, a distinctive objective of our study is to assess possible discrepancies
between moral conduct in ecological conditions, and post-hoc moral judgments.
2.1 Experiment
In our experiment we put the subjects in a virtual driving situation and study their
behaviour, and in order to fulfill our just described aims, we created two different
scenarios. In the strictly ecological choice, while the user drives in an unknown city, he
breaks into a sudden car accident due to the crossing of a child who does not take care
of the danger. In this case, three alternatives are shown to the user: A. Invest the child;
Integrating Human Acceptable Morality in Autonomous Vehicles 43
B. turn right and kill three passers-by (like a family) on the sidewalk unaware of the
danger; C. turn left and kill two road construction workers on the roadway.
The other – let call it “cold” - conceptual condition differs in halting the simulation
before the subject breaks into the road accident again. Here, the alternatives of choice
are visible to the user, and clarified orally by the experimenter, and the subject makes
her choice and report a motivation. Our experimental setup include a virtual reality
helmet that allows the subject to feel immersed in the road environment, and a setup
consisting of a real driving seat positioned on a rigid frame, a steering wheel with force
feedback, pedals and gearshift used to provide a complete driving experience during
the simulation [8].
3 Preliminary Results
To date, experiments have been performed on a sample of 84 subjects, composed by 10
males and 74 females, with a mean age of 22.3 years. These first results show a
dramatic difference in the moral behavior of the subjects between the strict ecological
condition and the “cold” conceptual condition. While in the strictly ecological choice,
92% of subjects chose A, in the cold choice only 35% confirms this decision, 61%
chose C, and 4% chose B (Fig. 1).
Histogram of multiple variables
Hot vs Cold
80
70
60
50
40
30
20
10
Hot
0 Cold
a c b
Fig. 1. Bar chart comparison hot-cold choice. Hot: A 92.85% B 3.57% C 3.57%. Cold: A
35.71% B 3.57% C 60.71%
We have classified the reported motivations for the cold choice into 4 categories:
Deontological reasons (protection for family and/or child); Utilitarian reasons (oriented
more to quantity than to quality); Normative reasons (related to compliance with the
highway code); Other.
The choice A (kill 1 child) made by 34% of the subjects, is due to 46% utilitarian
reasons and 54% normative reasons. The choice B (to kill 3 people/family) is due, in
100% of the cases to motivations of “other origin” that have no statistical significance
(only 3 subjects of 76). The choice C (kill 2 road workers) made by 61% of the subjects
44 G. M. Grasso et al.
is due for 58% to deontological reasons; 17% to utilitarian reasons; 4% to normative
reasons and 21% of other reasons.
This classification of the moral preferences in the “cold” situation confirm the
relative prevalence of the utilitarian arguments (27.38%), already assessed by previous
studies. Bonnefon et al. [3] reported that 76% of the subjects in their study choose to
kill the least number of people even at the cost of their own lives, and their moral
utilitarian attitude increases with the number of lives that can be saved.
Also Faulhaber et al. [7] report in their study a greater preference for the subjects in
minimizing the number of deaths, independently on age or context.
However, our results show that there are other factors (15.48%) that influence the
moral choice, for example, social and economic factors [12] and a particular preference
for the deontological code (35.71%) and normative code (21.43%) – see Fig. 2.
Fig. 2. Pie chart about motivations of cold choice: 35.71% Deontology; 27.38% Utilitarianism;
21.43% Normative; 15.48% Other
4 Conclusion
The main result of our study is that there is a significant discrepancy between the
mental moral processes in act when people are immersed in a situation similar to
driving, and the “cold” moral judgments that subjects report in the less ecological and
more conceptual evaluation of the same critical situation. Specifically, in the strict
ecological condition subjects appear much more deontological than utilitarian in their
moral choices, reverting the proportions of moralities in their verbal reports after “cold”
situations. If confirmed on a larger sample of subjects, this result raises worries on the
possibility of conceptual trolley-like stimuli in grasping a correct evaluation of the
common morality of human drivers.
Integrating Human Acceptable Morality in Autonomous Vehicles 45
References
1. Allen, C., Smit, I., Wallach, W.: Artificial morality: top-down, bottom-up, and hybrid
approaches. Ethics Inf. Technol. 7, 149–155 (2005)
2. Anderson, M., Anderson, S.L.: Machine ethics: creating an ethical intelligent agent. AI Mag.
28(4), 15 (2007)
3. Bonnefon, J.F., Shariff, A., Rahwan, I.: The social dilemma of autonomous vehicles. Science
352, 1573–1576 (2016)
4. Casebeer, W.D.: Moral cognition and its neural constituents. Nat. Rev. Neurosci. 4(10), 840
(2003)
5. Coca-Vila, I.: Self-driving cars in dilemmatic situations: an approach based on the theory of
justification in criminal law. Crim. Law Philos. 12(1), 59–82 (2018)
6. Danaher, J.: Should we create artificial moral agents? A Critical Analysis. Philosophical
Disquisitions (2019)
7. Faulhaber, A.K., Dittmer, A., Blind, F., Wächter, M.A., Timm, S., Sütfeld, L.R., König, P.:
Human decisions in moral dilemmas are largely described by utilitarianism: virtual car
driving study provides guidelines for autonomous driving vehicles. Sci. Eng. Ethics 25, 399–
418 (2019)
8. Grasso, G., Lucifora, C., Perconti, P., Plebe, A.: Evaluating mentalization during driving. In:
Proceedings of the 5th International Conference on Vehicle Technology and Intelligent
Transport Systems (VEHITS 2019), pp. 536–541 (2019)
9. Himmelreich, J.: Never mind the trolley: the ethics of autonomous vehicles in mundane
situations. Ethical Theor. Moral Pract. 21, 669–684 (2018)
10. Jack, A., Roepstorff, A.: Trusting the subject? Part 2 (2004)
11. Li, J., Cheng, H., Guo, H., Qiu, S.: Survey on artificial intelligence for vehicles. Automot.
Innov. 1(1), 2–14 (2018)
12. Maxmen, A.: Self-driving car dilemmas reveal that moral choices are not universal. Nature
562, 469–470 (2018)
13. Moll, J., Zahn, R., de Oliveira-Souza, R., Krueger, F., Grafman, J.: The neural basis of
human moral cognition. Nat. Rev. Neurosci. 6(10), 799 (2005)
14. Nyholm, S., Smids, J.: The ethics of accident-algorithms for self-driving cars: an applied
trolley problem? Ethical Theory Moral Pract. 19, 1275–1289 (2016)
15. Takács, Á., Drexler, D.A., Galambos, P., Rudas, I.J., Haidegger, T.: Assessment and
standardization of autonomous vehicles. In: 2018 IEEE 22nd International Conference on
Intelligent Engineering Systems (INES), pp. 000185–000192, June 2018
16. Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right from Wrong. Oxford
University Press, Oxford (2008)
The Future of User Experience Design
in the Interior of Autonomous Car Driven
by AI
Laura Giraldi(&)
Department Dida, University of Florence, Florence, Italy
[email protected]
Abstract. Today it is not easy to imagine what would be the future of user
experience (UX) design in the world of Artificial Intelligence (AI). Referring to
the sector of autonomous vehicles the paper aims to explore the changes that
will be brought by artificial intelligence to the innovative sector of the self-
driving car. According to these transformation car interiors and passengers
experiences will become very different from the actual one. The aim of the
research is, starting from different kind of passengers needs (Human Centered
Design approach) individuating essential factors to design autonomous cars
interiors and in particular to design innovative interfaces for a better commu-
nication with the passengers and a high level of living experience for different
kind of users.
Keywords: Human factors Autonomous car UE/UI Communication
design Product design AI
1 Introduction: Context of Reference
The availability of ubiquitous network access to shared data (now called big-data),
through different kind of devices such as smartphone, tablet, computer, the diffusion of
IoT (Internet of Things) is bringing the application of Artificial Intelligence (AI) in
many areas of our daily activities. Besides, the imminent arrival of the new wireless
broadband standard 5G in a large number of popular products of daily use will allow
the access AI and data analytics on the move.
AI is drastically changing our world and our habits. The application of these
technologies to the automotive business as a key building block for the design of the
autonomous car generates great interest to solve transportation problems like road
traffic and address safety issues.
Today it is not easy to imagine what would be the future of User Experience
(UX) and User Interaction (UI) design in the world of Artificial Intelligence (AI). It is a
time when the human-AI connection is so deep that some experts say there will be “no
interface.” Currently, UX/UI often depends on an interface with screens that are often
inadequate and not friendly.
Specialists of transportation technologies, software and telecommunications engi-
neers, are working to release autonomous vehicles in the very next future but this fact is
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 46–51, 2020.
https://doi.org/10.1007/978-3-030-39512-4_8
The Future of User Experience Design in the Interior of Autonomous Car 47
not sufficient for their success and diffusion because it is necessary to understand if
users are ready to use this kind of vehicle. For this reason it is very important their
opinion, expectations benefits and needs. Indeed, the success of this technology will
depend largely on the public opinion and will be strictly related to the society’s ben-
efits, concerns, and by the level of adoption by the people.
Studies dedicated to the interaction of the human being with the self-driving car
demonstrates that the above factors change in relation to important users differences:
gender, age range, study level, culture, employment status, etc. [1].
Until now, in general, to improve user engagement in their products, UX designers
have turned to tools and metrics such as usability tests, heat maps and usage data. By
using AI, it will be possible to observe users’ behaviours at 360° to provide tips on how
they can improve their user experience, eventually leading to a better and more
effective one [2]. AI could be used to design according to each user’s needs (practical
and emotional), based on the analysis of the collected data. All this is achieved through
the application of machine and deep learning algorithms that combines large data sets
to make inferences. Additionally, these systems can learn from the data and adjust their
behaviour accordingly, in real time. This revolution is currently on going and some
popular examples are the Siri and Alexa digital personal assistants and Netflix’s highly
predictive algorithms, which took the market by storm. For that regards the automotive
market, some examples can be found in the Toyota’s artificial intelligent buddy named
Yui, an AI personality, embedded into the vehicle’s architecture, able to help users to
navigate, communicate and even contributes to enhance their conversations.
Today, it is globally recognized that technology will permeate the car of the future.
However, in our view, this new kind of vehicles, instead of being a cold and weird
flying robot car, will have to go completely in the opposite direction. Although it is
definitely futuristic, it will have a friendly and welcome demeanor. The focus will be
on the importance of the human that is using it.
Indeed, a car digital assistant like Yui has to support driver and passengers ever
since they approach the vehicle. In this way, the rear of the vehicle can display
messages to communicate about upcoming turns or warn about a potential hazard. The
front of the vehicle could be able to communicate whether the car is in automated or
manual drive.
In the future, human being will have to increasingly answer the questions of what
will be the relationship between the vehicles of the future, and the people who drive
them and whether technology can be warm and friendly, engaging, and immersive.
More often cars have become our home on wheels. We connect with our cars, and we
build an emotional relationship. What was once an unspoken bond will now progress
on grounds of mutual affection.
2 The Aim of the Research
Nowadays there are a lot researches about autonomous car and the effect of this possible
innovation in daily life of people. At first the studies deepened, for the most, technical
and security aspects and many engineers are working hard to solve all the related
problems also involving other disciplines. Mostly the product and communication
48 L. Giraldi
designers point of view is interested to study how the car could change in terms of
morphologies, interactions and user experiences. As said above, according to the pos-
sibilities offered by new technologies, probably, the new autonomous vehicles should be
so different from the previous one that it could be very hard to recognize them in relation
to the archetype of cars of the collective imaginary. Consequently both the outside and
the interior could completely change in comparison with the current vehicles.
The present work focuses on the possible changes of autonomous car AI driven in
order to consider possible strengths and weakness to design new interiors also
according to users need and behaviours to introduced innovations able to be under-
standable and easy usable by people generating a pleasant UX.
The aim of the paper is to explore the changes that will be brought by artificial
intelligence to the innovative sector of the self-driving car in order to design autono-
mous cars interiors. In particular the intention is to find some factors useful to design
innovative interfaces for a better communication with the passengers and a high level
of living experience for different kind of users.
3 Multidisciplinary Approach
Passengers of a self-driving car have very different experience in comparison with
passengers in a car with human driver. The interaction is so different according to many
factors, for this reason it is important to study their behaviors and needs in both
scenarios. Consequently, it is necessary to define a design method to analyses these
factors according to the skills, behaviors and inclinations of different kind of passen-
gers. In this way it is possible to design a system really able to interact and entertain
passengers in a pleasant way using strategic solutions while they travelling within a
self-driving car.
The methodological approach to design passengers experience refers to the Human
Centered Design approach (HCD) in order to study in detail different passengers fac-
tors. As a matter of fact, from the birth to older age people, as possible passengers, they
have a continuous development of physical and psychical abilities, which are necessary
to analyze in order to design new solutions.
The research refers to a multi-disciplinary and holistic approach involving different
disciplines. The involved experts are not only engineers and specialist in vehicle
technologies but also psychologists, anthropologists, smart material specialists, inter-
action designers, user experience designers, communication designers, product
designers. This main method put users at the center of the entire design process [2]. At
the same time all the individuated factors are to be approved and tested by all the
specialists. The contributions of each discipline represent the indispensable basic
knowledge necessary to summarize all the features to design an appropriate and sat-
isfactory interaction and passengers experiences.
The Future of User Experience Design in the Interior of Autonomous Car 49
4 Analysis for Future Scenarios
Autonomous vehicles and in particular cars represent a very important development
able to improve our daily life [3]. In particular these kind of “cars” may help different
kind of people in mobility, offering them different experience according to their needs
and use opportunity. Anyhow human behavior in self-driving cars is not linear and
sometimes unforeseeable; users perceptions of service experiences have always been
important to the success of product/service. The relationship between quality percep-
tions and satisfaction is the key for the innovation acceptance by users but this relation
can change according to the kind of user and his/her gender. The factors able to
influence all the users (both male and female) satisfactions are tangibility, reliability
and responsiveness of a product/service [4].
According to recent studies passengers prefer to interact with the car using the
agent interlocutor. The voice command is able to attract the highest level of trust
improving the pleasure on their experience [5]. In autonomous car the interaction
between the user and the vehicle is very different, Users inside autonomous cars have
free time because their attention is not for driving. They could use their time to work to
study to play, to converse to navigate, to shop online, to organize their trip, and so on.
It is possible to say that, in public autonomous driven cars, as taxi of the next
future, users needs depend on a series of elements according to the kind of users (i.e.
children, older people, disables,...) their habits and culture, the aim of the trip (a tourist
trip, a business one,…).
Following these factors and specific user needs the interior could change adjusting
in real time to offer different living solutions and giving personalized information [6].
To design the interior of a AV is essential to know different users necessities but it
is not sufficient because they could change according to chance of use, the context, and
finally to the travel time.
Recent researches on this issue underline that in six years cars will be smarter and
more functional, with high-efficiency engines and lighter materials [7]. Due to the
diffusion of fifth generation network (5G) there will be a real revolution on connec-
tivity. This network will allow the car to communicate in real time with the surrounding
environment, making possible the increasingly advanced autonomous driving systems.
It is possible to foresee that the future self-driving cars will have swivel and
adjustable seats to self create a open space inside the car, advanced head-up display
screens, biometric systems to interact, to open and lock the vehicle in safety. Moreover
commands will be activated by gestures and voice control, and a smart centre of the
car’s infotainment system able to connect to smart public and private sensors and
devices, and of course the on-board computer will be the only driver of the car itself.
The whole cycle of user travel, from the autonomous car’s call to the final desti-
nation, could be characterized by the main actions as follow:
1. User call the autonomous car communicating his/her destination. This action is
possible using the relative app with the smartphone.
2. The autonomous car app communicates the time to arriving and the price.
(These two above points happen also today calling for instance Ubertaxi service.)
50 L. Giraldi
3. The autonomous car app asks the passengers different information necessary to
customize the interior of the autonomous car according to:
– number of passengers;
– kind of passengers (gender, adult, older people, younger, children, babies, if
able-bodies or kind of disability) if there are animals (pet or other kind…);
– kind of personal effects: (computer, shopping bags…);
– kind of luggage (trolley, bag, stroller, baby car seat…);
– aim and time of travel (for business, tourism, free time, personal necessities,…).
Analysing these above data, the system proposes different solutions also asking
specific questions to passengers. Then the autonomous car sets its interior customized
according to data and users needs, before taking users on board.
The use of Autonomous vehicles introduces interesting change in human habits and
behaviours giving new opportunities to live innovative experiences during a trip. In fact
the absence of the human driver allows all the passengers to do different activities
saving their time. For the most, the user interaction is with voice command and a
virtual “assistant” is at disposal to satisfy passengers needs, giving information using,
for instance, augmented reality and smart screens. Thanks to the AI the vehicle become
friendly and very personal even if it is a share vehicle.
According to Mike Ramsey, research director of Automotive and Smart Mobility,
when on board passengers communicate to the car using the voice command and the
car will communicate with the roads, other cars and infrastructure not only to calculate
the best route for the passenger but also for any other passengers that it is possible to
“collect” during the journey. Once the passengers have been taken to their destination,
the car will automatically withdraw to the nearest charging station, waiting to be called
to bring passengers back or to continue its journey to take other passengers.
We suppose that the future autonomous car will be not only a car but a partner, it
will be able to connect with the passengers on a level that current UX doesn’t reach and
by using functions over and over, users end up establishing an interdependent rela-
tionship with the system immediately.
In this way the AI’s deep learning algorithms can collect, by following a continuous
cycle, data and use them to learn and adapt the human interface to the real need and
behaviour of the passengers. Users will not be aware about it and before they will know
it, they will be deeply connected.
AI could become another complementary method to help designers in designing
products and services with user interfaces able to involve user senses satisfying
material and intangible needs and emotions. The success of self-driving cars depends to
the users opinions on benefit, concern, and on the use of these technologies. The
findings can help supporting the design and the development of vehicle agent-based
voice interfaces to enhance trust and user experience in autonomous cars.
Acknowledgments. The research was carried out thanks to the consulting of Gianluca Mando’
Head of Research, Technology & Innovation at THALES ITALIA S.P.A.
The Future of User Experience Design in the Interior of Autonomous Car 51
References
1. Marti-Belda, A., Boso, P., Lijarci, I.: Beliefs and expectations of driving learners about
autonomous driving. ToTS, Palacky University in Olomouc, 10(X) (2019)
2. Norman, D.A.: Emotional Design. Apogeo, Milano (2013)
3. Gebresenbet, Bayyou, D.: Artificially-intelligent-self-driving-vehicle-technologies–benefits-
and-challenges. Int. J. Emerg. Technol. Comput. Sci. Electron. 26(3), 5–13 (2019)
4. Mokhlis, S.: The influence of service quality on satisfaction: a gender comparison. Public
Adm. Res. 1(1), 103 (2012)
5. Large, D.R., Harrington, K.J., Burnett, G.E., Bennet, P.: To please in a pod: employing an
anthropomorphic agent-interlocutor to enhance trust and user experience in an autonomous,
self-driving vehicle. In: Automotive User Interfaces and Interactive Vehicular Applications
Conference, Utrecht, Nederland, pp. 49–59 (2019)
6. Cianfanelli, E., Tufarelli, M.: Smart vehicles. a design contribution for the changing urban
mobility. In: Cumulus Conference Proceedings Wuxi, DTDO, Diffused Transition and Design
Opportunities (2018)
7. Keeney, T.: Mobility as a service: why self-driving cars could change eveything. ARK Invest
Research (2017)
Measuring Driver Discomfort
in Autonomous Vehicles
Dario Niermann(&) and Andreas Lüdtke
OFFIS e.V., Human Centered Design, Escherweg 2, 26121 Oldenburg, Germany
{Dario.Niermann,Andreas.Luedtke}@offis.de
Abstract. Autonomous driving is getting more common and easily accessible
with rapid improvements in technology. Prospective buyers of autonomous
vehicles need to adapt to this technology equally rapidly to feel comfortable in
them. However, this is not always the case, since taking away control from the
user often correlates with loss of comfort. Detecting uncomfortable and stressful
situations while driving could improve driving quality and overall acceptance of
autonomous vehicles through adaption of driving style, interface and other
methods. In this paper, we test a range of methods, which measure the dis-
comfort of a user of an autonomous vehicle in real-time. We propose a portable
set of sensors that measure heart rate, skin conductance, sitting position, g-forces
and subjective discomfort. Preliminary results will be examined and next steps
will be discussed.
Keywords: Autonomous driving Discomfort Acceptance On-the-road
Human factors Human-machine-interaction
1 Introduction
We start with the hypothesis that drivers of (autonomous) vehicles have moments of
stress or discomfort while driving through traffic, due to always occurring slightly
dangerous or unpredictable traffic conditions. Especially in autonomous vehicles
(AVs), stress might occur more frequently, since loss of direct control often reduces
trust in the system. Therefore, it is desired to develop methods that lower discomfort
and stress, thus increasing the acceptance of AVs. Besides static methods, like always
driving slowly, the detection of the exact situations where discomfort or stress is
building up is an approach to individualize the needs of the driver and to adapt only
when adaption is needed.
Measuring uncomfortable or stressful situations can be done using two different
streams of information: The physiological indicators of the driver and the information
from surrounding traffic. A combination of both validations is a promising way to
evaluate these situations. Since traffic information will necessarily be recorded and
interpreted by AVs, we investigate the usefulness of physiological sensor data while
driving through diverse traffic conditions. A more precise definition of ‘stress’ and what
precisely we measure will be explained in Sect. 2. In the following, we explain the
overall concept of our idea.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 52–58, 2020.
https://doi.org/10.1007/978-3-030-39512-4_9
Measuring Driver Discomfort in Autonomous Vehicles 53
The concept of combining the above-mentioned streams of information and the
measurement of stressful situations is displayed in Fig. 1. On the left are multiple
sensors that will be used to measure physiological responses (see Sects. 2 and 3.1).
These will be combined in real time and give input to a classification model. This
model analyses the given sensor values and outputs a likelihood on how uncomfortable
or stressed the user currently is.
The traffic context is the current situation of the vehicle and its surroundings. This
considers the current driving style, maneuver, road and weather conditions, time of day
and so on and should be handled by the board computer of the AV.
The maneuver model combines both outputs from the stress classifier and traffic
context and assigns the stress likelihood to the current maneuver. For example, if the
vehicle is currently overtaking another vehicle, on a wet road in the dark and the user
feels highly stressed in the same moment, the maneuver model assigns and saves the
current maneuver to high stress. If necessary, the maneuver model can collect multiple
equal maneuvers and collect an average stress likelihood. If the maneuver model has a
precise measure on the stress likelihood for the currently planned maneuver, the
maneuver can be adapted towards a more relaxing one (Fig. 1).
Fig. 1. Process chart of our concept to reduce stress. The grey marked nodes are our focus in the
paper. Greyed out parts are only necessary for training of the model and can later be removed.
Training the classification model is done by using self-reported subjective stress
while driving (see Sect. 2). This self-reported stress acts as ground truth and allows for
supervised training strategies of many machine-learning algorithms.
2 Stress and Physiological Sensors
The word stress is a difficult term, which is not well defined. We will use it as a
collection of terms all concerning discomfort, risk perception, tension, anxiety and so
on. Hence, every situation concerning the traffic situation and driving style of the
vehicle that induces negative physiological and psychological effects. We collect these
situations simply by letting the test subjects tell us when they felt those negative
emotions, by pressing a continuous handset button (see Sect. 3.1). This way, we
capture situations that induce subjective ‘stress’ and assign the intensity of the feeling
to a value between zero (no stress) and one (high stress). From now on, we will refer to
these subjective feelings as ‘stress’ to make the paper more easily readable.
54 D. Niermann and A. Lüdtke
Measuring stress is a popular objective in many scientific fields. Different methods
and possible bio signals emerged over the last decades, most notably measuring the
skin conductance (also called galvanic skin response, GSR) and heart rate (HR).
Coming from the medical field, [1] proposed that GSR is an indicator of stress. Later,
GSR signals were used to develop a stress detector with reported success rate of 78%
[2]. Various other signals were also analyzed; [4] used EEG, ECG and EMG combined
with GSR and [5] used skin temperature. Another approach analyzed facial expressions
to detect stress [7].
In above-mentioned submissions, most measurements are done in very controlled
environments and therefore, the used methods will most likely not yield good results in
driving vehicles. A more related submission from Healey and Picard [6] measured
ECG, EMG, GSR, heart rate and respiration rate while the test subjects were driving
through traffic. They assigned different traffic conditions to different stress levels (e.g.
city driving = high stress) with a resolution of five-minute intervals and reported a 97%
accuracy for stress level classification. However, they reported this high performance is
only possible in these five-minute intervals and not on a real-time resolution. In
addition, the stress levels of the subjects where assigned by the researchers, not the
subjects themselves, thus could be a strong mismatch between the assigned stress and
real physiological stress. They also analyzed real time correlations by assigning stress
levels every second based on video recording of the drive. However, they did not use
this data to create a classification model.
One result of their analysis showed that GSR and HR correlated best with stress
levels of the driver, also considering the real time data. Since HR and GSR mea-
surements have an overall good reported correlation to stress levels and the corre-
sponding sensors are also very unobtrusive, we decided to use these two bio signals for
our measurements.
We found no released research project that accomplished a real time stress detection
in real traffic conditions.
3 Methology
3.1 Hardware Setup
To measure the stress level of the subjects on a real time resolution, we use a handset,
which is pressed during stressful or uncomfortable situations. With this method, we
measure the real time, self-reported emotions and do not need to assign the (possibly
wrong) values by someone else. The handset can be pressed continuously strong to
reflect the amount of stress or discomfort the subject is experiencing (light press =
slight discomfort, strong press = high stress).
In addition to above sensors, we also implemented pressure sensors into a portable
seat mat with three pressure zones (see Fig. 2). With these, we measure the positional
changes of the subject’s body and the g-forces acting on them, which could also
correlate to the feeling of discomfort or stress. They work by attaching long silicon
tubes to piezoelectric pressure sensors on one end. The other end of the tube is sealed.
If a tube is compressed somewhere along its length, the air pressure presses against the
Measuring Driver Discomfort in Autonomous Vehicles 55
pressure sensor. Therefore, the tube defines an area on which pressure is measured. For
the areas P3 two tubes were connected to one sensor, thus the sensor is collecting data
on both sides of the backrest. Areas P1 and P2 use folded tubes to cover a broader area.
All sensors connect to a Raspberry Pi 3 Model B via wired connections. The GSR1
and (optical) HR2 sensors were bought from Seeedstudio3 and connect to and Arduino
Nano, which forwards the signals via USB B to the Raspberry Pi. The Handset is a
common handset from Carrera, also connected to the Arduino Nano. Pressure sensors
are connected via a custom made circuit board4 to the Raspberry Pi. The setup is shown
in Fig. 2.
Fig. 2. Display of the portable hardware on an office chair. The blue marked pressure areas
show where the silicon tubes are located
3.2 Study Setup
While driving, the handset is held in one hand of the test subject and the GSR sensor is
strapped on two fingers of the other hand to remove any correlations that could occur
because of finger movement. The HR sensor is clipped to the ear and measures the
blood flow optically. Pressure sensors are attached under a seat mat and are not
noticeable.
The test subject is in the passenger seat and is driven through traffic by a driver. The
test subject should concentrate on the traffic and cannot control the car, therefore has a
similar experience as in an autonomously driven vehicle. An additional person on the
back seats monitors the sensor data, can talk to the test subjects and answer questions.
1
GSR Sensor from Seeedstudio, http://wiki.seeedstudio.com/Grove-GSR_Sensor.
2
HR Sensor from Seeedstudio, http://wiki.seeedstudio.com/Grove-Ear-clip_Heart_Rate_Sensor/.
3
Seeedstudio Grove system, http://wiki.seeedstudio.com/Grove_System.
4
Developed by OFFIS Health.
56 D. Niermann and A. Lüdtke
The traffic is uncontrolled, which is not a problem but rather an advantage, since
many different situations lead to stress and create a diverse set of measurements, which
can also improve machine-learning models.
4 Preliminary Results
To test our system and get first impressions about the measured data, we measured
three internal colleagues each over a one-hour tour through city, highways and rural
roads. The analysis of the collected data is presented as a preliminary result in this
chapter.
First, we want to look at the measured self-reported stress situations to prove our
initial hypothesis. Over one hour of normal driving, all test subjects reported multiple
situations where they felt at least uncomfortable. For example, the stress value of over
0.5 was reached between 10 and 15 times in this hour. Smaller values occurred roughly
every 2 min. This is corresponding well with our hypothesis.
The duration of a stressful situation is typically very short, with only a few seconds,
rarely surpassing four seconds. This emphasizes how important a quick classification
is. However, since the physiological responses need time to relax again, the handset
value was edited afterwards, such that it decays exponentially to create larger time
windows and fits physiological relax times.
While the test subjects report stress, the physiological sensors need to measure
some abnormalities such that an automatic classification is possible. If we look only at
moments of medium to high stress, defined by handset values of higher than 0.5, we
can recognize abnormalities and average trends of the measurements. In Fig. 3(a), such
an analysis is displayed for one test subject. Trends like rising heart rate and lowering
GSR can be seen clearly.
Fig. 3. (a) Time series plots of multiple sensors while the test subjects reported medium to high
stress (see handset value higher than 0.5). (b) Heart rate distributions over three test subjects
divided into relaxed and stressed (handset values > 0.5) intervals.
This makes clear that the subjective self-reported stress is indeed correlated with
some physiological responses.
Measuring Driver Discomfort in Autonomous Vehicles 57
However, Fig. 3(b) makes also clear, that not all test subjects react equally and that
the overall distributions of values while stressed and relaxed are very similar. We need
to further investigate the differences of users in further studies.
As a small preview, we trained a neural network to classify stress based on our
sensor data. This neural network model with simple sigmoid units and one hidden layer
provided best results in comparison with simpler models. We did not yet develop a
good performance measure, since it is very difficult to detect and compare peaks
reasonably, especially with maneuver based knowledge. As a way to show the per-
formance of our model, we overlaid the measured, self-reported stress with the model
prediction in Fig. 4. Here, it is visible that the model predicts most of the peaks, but
also misses around 20%. This is a random cut out from the drive of proband zero. Other
subjects were more difficult to model, with a peak detection success rate of around
50%. As mentioned, this was done without any information about the traffic, so this
classification could improve when this information is also considered.
Fig. 4. Comparison of the self-reported stress (‘Desired’) and the model prediction over time.
Both curves should be identical for a perfect model.
5 Discussion and Outlook
In future work we will proceed with a thorough study, including at least 10 participants
of different age and different driving experience. This will ensure a diverse data set and
give insights about the differences of users. We will analyze differences between test
subjects and performance of multiple machine learning models. Another step will be to
analyze how well a general model for all persons can be used and if a person specific
model is necessary. In addition, we will investigate the usefulness of every used sensor
and which one correlated most with self-reported stress.
References
1. Storm, H., Myre, K., Rostrup, M., Stokland, O., Lien, M.D., Raeder, J.C.: Skin conductance
correlates with perioperative stress. Acta Anaesthesiol. Scand. 46(7), 887–895 (2002)
2. Villarejo, M.V., Zapirain, B.G., Zorrilla, A.M.: A stress sensor based on Galvanic Skin
Response (GSR) controlled by ZigBee. Sensors 12(5), 6075–6101 (2012)
3. Minguillon, J., Perez, E., Lopez-Gordo, M., Pelayo, F., Sanchez-Carrion, M.: Portable system
for real-time detection of stress level. Sensors 18(8), 2504 (2018)
58 D. Niermann and A. Lüdtke
4. Yamakoshi, T., Yamakoshi, K.I., Tanaka, S., Nogawa, M., Shibata, M., Sawada, Y., Rolfe,
P., Hirose, Y.: A preliminary study on driver’s stress index using a new method based on
differential skin temperature measurement. In: 29th Annual International Conference of the
IEEE Engineering in Medicine and Biology Society, pp. 722–725, August 2007
5. Healey, J., Picard, R.W.: Detecting stress during real-world driving tasks using physiological
sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005)
6. Gao, H., Yüce, A., Thiran, J.P.: Detecting emotional stress from facial expressions for driving
safety. In: IEEE International Conference on Image Processing (ICIP), pp. 5961–5965,
October 2014
Human-Centered Design for Automotive
Styling Design: Conceptualizing a Car
from QFD to ViP
Gian Andrea Giacobone(&) and Giuseppe Mincolelli
Department of Architecture, University of Ferrara, Via Ghiara 36,
44121 Ferrara, Italy
{gianandrea.giacobone,giuseppe.mincolelli}@unife.it
Abstract. Designing a vehicle is a harmonic balancing between engineering
functions, technologies, and designing activities. However, the design process is
laborious because the convergence between styling and engineering needs to
find constant compromises. Using a human-centered design strategy improves
the management of the process. Considering this, the paper reports an auto-
motive design process that is a part of a multidisciplinary project funded by
Emilia Romagna Region, Italy. The main goal of the overall project is to design
the concept of a hybrid sport vehicle. The peculiarity of this specific research
concerns the combination of two different approaches that rely on the impor-
tance of human-centered design. In particular, The process initially uses Quality
Function Deployment (QFD) and it concludes the styling process through
Vision in Product Design (ViP). The result converges both outcomes to come up
with a novel and integrated concept solution.
Keywords: Human-centered design Automotive design Research methods
Quality Function Deployment Vision in Product
1 Introduction
The vehicle is a very intricate amalgam of technological, functional and styling
properties. Its design is the result of holistic thinking determined by a harmonic bal-
ancing between engineering functions and designing activities that make the vehicle
itself unique [1]. The entire design process is laborious because the convergence
between styling and engineering needs to find constant compromises [2]. This hard task
is generally undertaken by automotive design or automotive styling, namely the dis-
cipline that is implicated in giving a form to the car, encompassing package design and
negotiating engineering constraints [3]. Good management of those activities ensures
for a company the successful development of the vehicle.
However, given the complexity of the automotive product, sometimes many dif-
ficulties in managing the entire design may occur during the process. If all aspects of a
car are not faced up with proper solutions, they could create a real gap between the
level of satisfaction desired by end-users and the development management utilized by
the process to achieve the requested goals of the company [4]. For these reasons,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 59–65, 2020.
https://doi.org/10.1007/978-3-030-39512-4_10
60 G. A. Giacobone and G. Mincolelli
developing an optimal strategy at the early stages of automotive design helps a com-
pany to achieve the high score of customer and avoid negative outcomes. One of the
potential and successful strategies is adopting a human-centered design
(HCD) methodology, because it is capable to identify users’ perspective, which can be
subsequently used throughout the process to lead the entire project. Every need is
immediately analyzed and consequently incorporated into research as decision-making
tool for designing the expected vehicle. The outcomes derived from this practice can
improve not only the engineering properties of the car, but also the styling aspects, in
terms of appearance, functions, and operation [5]. Together, all these factors can match
user desires determining, at the same time, the acceptance of total product experience.
2 Objective and Methodology
This research is integrated within a wider multidisciplinary project called “Automotive
Academy: learning by doing project for innovation engineering automotive” which is
financed by the Emilia Romagna Region, Italy. The overall project aims to develop
several engineering and designing solutions in different expert areas that can be applied
for developing the future concept of a sustainable sport car.
Regarding this specific research, the main objective is to give a proper vision and
shape to the overall project through defining a successful design strategy to develop the
concept vehicle. The paper proposes the development of the conceptual design phase
using a human-centered methodology that is focused on two different approaches:
Quality Function Deployment and Vision in Product Design. Both models are selected
because they are perfectly suitable in automotive for structuring idea-generating
throughout the styling process. They support also design team in taking significant
decisions for automotive product development.
Quality Function Deployment (QFD) is a human-centered tool and inclusive that is
able to orient the product development process toward the real exigencies of end-users
[6, 7]. The core of QDF is based on a bi-dimensional mathematical matrix, called
House of Quality. It represents graphically the correlation between the most important
needs of users and the product characteristics that are considered more significant to
develop suitable solutions for satisfying costumer demand [8]. The result is a clear
assessment of the most important product specifications that are utilized as guidelines
both to design the final product and achieve the desired quality requested by users.
Vision in Product Design (ViP) is an innovative strategic method that enables
innovation through envisioning new ideas for the future [9]. Instead of simply solve
present-day problems, ViP tries to discover new design opportunities that can be
designed in the future ahead. It establishes the raison d’êtré of the final design, namely
the intrinsic and anthropologic meaning of the artefact, which justify its existence in the
world [10]. To do this, ViP model starts from carefully understanding the current
evolution of socio-cultural models [11], to successively have a strong basis to envision
the future context. ViP is a step-by-step design process and is divided into two areas:
the left side represents the process of “deconstruction” and it analyses the current state
of the art; the right side is connected to the process of “designing”, in which the vision
is realized in eight steps. Each area is divided into three main levels of study, which are
Human-Centered Design for Automotive Styling Design 61
based on product, interaction and context analysis. In the end, the outcomes provided
by the vision are represented by a final abstract model that is called advance concept.
3 Activities and Results
In the initial phase of design process, QFD was utilized to perform user analysis and
interpreter the most important needs in its referred context. Next, ViP envisioned the
concept vehicle in a future context in which the expected model will be inserted. In the
end, the outcomes of both methods were converged together in a third and integrated
activity in which the final concept was designed.
To facilitate the research, the exploratory design process selected an existing car as
a case study to develop the final concept: KTM X-Bow. This because the model
coincides perfectly to the requirements of the project but also because user analysis has
been conducted during a specific race event sponsored by KTM and dedicated to
X-Bow series [12]. From this activity, about 200 needs were gathered and successively
processed through QFD.
To better manage QFD process, data were grouped into four systems of the vehicle
that have been considered the areas in which more interactions occur between users and
vehicle. The areas are: (a) vehicle dynamics; (b) powertrain system; (c) passenger
package; (d) vehicle interface. About 20–24 needs for each area has been selected and
processed within its specific QFD matrix. Next, about 20–28 technical requirements for
each QFD matrix were chosen in order to find the most important product specifica-
tions that can be considered useful for satisfying user expectations. Data of each QFD
matrix were correlated together revealing the most important product specifications that
should be included in the styling process to develop the concept vehicle.
The considerations about QFD assessment are: (a) the powertrain must provide
quick acceleration and different engine mappings in order to have many driving per-
formances in base of a particular context or a specific purpose (for instance urban or
race use). The powertrain must also produce sound, which in turn suggests the research
to maintain the traditional experience provided by internal combustion engine; (b) re-
garding vehicle dynamics, the car must be rigid, permeable and must provide adequate
downforce to guarantee high stability and performance. The vehicle must maintain low
height, ground clearance and its center of mass to increase handling; (c) bout interior
package, the vehicle must guarantee safety both at the belt and head heights so that
airflow cannot stress the driver or affect negatively its driving performance. The cockpit
must provide also a not-intrusive steering wheel hub, in order to provide comfortable
ingress/egress to all drivers; (d) concerning driving interface, the vehicle must provide
a usable configuration of its driving controls. The interface must clear and under-
standable in showing only the essential information based on the context of use. The
controls must be immediately recognizable and reachable by hands.
After QFD process, ViP model envisioned the advanced concept in a future con-
text. KTM’s car was used again as a case study for creating the final solution. Several
context factors were gathered to define the vision and successively grouped in a four-
square matrix. Each square indicated a statement, namely a potential way in which the
advanced concept would be designed. Only one of them was chosen and it was: “I want
62 G. A. Giacobone and G. Mincolelli
people to experience mobility that expresses themselves as an act of resistance”. The
sentence operated as the main goal to take a position in the world of tomorrow. This
choice affected the entire design of the advanced concept [10]. In this case, the
statement led the process in defining the characters of the expected vehicle utilizing
analogies to better explain the purpose of the statement. The selected analogy was “It
feels like making graffiti on the wall”, which consequently delineated two main product
characters: secretive and adversative. The properties defined all features of the
advanced concept in terms of use, shape and technology (Fig. 1).
Fig. 1. Visual representation of the advanced concept developed with ViP process.
The vision developed an extreme and interesting scenario in which a hidden and
silent car can be driven only in the night or darkness as resistance against daylight
routine of an efficient and technological mobility. For this reason, the character of
secretive delineated a fully electric and four-wheel drive vehicle. The ingress/egress is
located in the rear of the car and is hidden by coachwork. This configuration allows the
vehicle to eliminate traditional outlines of the doors and hence it obscures the vehicle’s
driving orientation. The cockpit covers also driver’s identity by eliminating windshield
and every window from the car. Consequently, the interface proposes to drive the
vehicle only through digital screens, improving driver’s perception through night
vision. The character of adversative produced a three-seat configuration. The driver’s
position is in the middle while the other ones are slightly behind it so that the rebel
property can imprint to the pilot more activism in the act of driving. This experience is
also accentuated by maintaining the traditional steering hub, which is able to transfer
every vehicle’s behavior directly to the driver’s hands.
Once the two methods were performed, it has been chosen to converge the out-
comes of both methods in a single and unique solution. The final results produced an
integrated vehicle, which was the combination of the two approaches.
ViP inserted the specific personality of the vision into the final solution. For
instance, the character of secretive influences the aerodynamics of the vehicle. Two air
intakes in the front deviate airflow, which conveys along the lateral sides of the car to
cool down both engine and batteries, without deliberately showing the outlines on
Human-Centered Design for Automotive Styling Design 63
coachwork. Instead, data from QFD reduced the frontal-section area and improved its
permeability to reduce aerodynamic drag but also to increase downforce.
Again, since the advanced concept was full electric, the same personality of secretive
mediated with QFD a parallel hybrid system. This particular configuration equips the
vehicle with a downsized engine and three lightweight electric motors – two in the front
and one integrated into the rear transmission as KERS (Kinetic Energy Recovery
System) – in order to have different driving settings, and improve the efficiency of fuel
consumption and carbon emission. For example, fully electric mode permits to drive a
silence car (according to the vision), while sport mode allows drivers to experience the
sound of the engine without losing the feeling of traditional driving (Fig. 2).
Fig. 2. On the left the integrated solution and its active seating. On the right the parallel hybrid
system (with the rear mid-engine layout) and the aerodynamic set up.
ViP and QFD data were combined also for passenger package and its corre-
sponding driving interface. The character of adversative configured an active driving
seat, which can be slid from the lateral location to the central racing position. The
dynamic configuration guarantees much more activism in driving providing two dif-
ferent experiences. Instead, QFD served to set the height of the lateral sides at shoulder
height, in order to improve safety and decrease drive exposure against airflow. Con-
cluding, the character of adversative provided to the vehicle a racing steering wheel, in
which the main controls are reachable by hands as expected by QFD outputs. Con-
cluding, the character of adversative provided to the vehicle a racing steering wheel, in
which the main controls are reachable by hands as expected by QFD outputs.
64 G. A. Giacobone and G. Mincolelli
4 Conclusions
In conclusion, QFD and VIP supported each other in producing many design outputs,
which consequently were used as guidelines to define every technical and aesthetic
property of the final vehicle. The combined approached produced an inedited design
strategy, which supported automotive design to come up with a novel and inspiring
concept vehicle. ViP equips the vehicle with a character, which can delineate the
styling properties of the car throughout the process. While QFD qualitatively matches
user desires underlining many constraints that are utilized to define technical specifi-
cations of the car according to their wants. This approach is capable of making the
design strategy wider, as well. The two approaches are based on two different timelines,
which together offer new and long-term automotive product management. QFD is a
mid-term approach because it investigates the near context starting from user needs.
However, people tend to express their wants about the present without pushing their
wishes further on. Therefore, since ViP is a long-term strategy, it supports the design
strategy to push its boundaries in the future anticipating new coming trends, which are
subsequently used by ViP to drive every decision of the process. Finally, QFD refines
the vision making the final concept more feasible and tangible.
References
1. Chandra, S.: Evaluation of clay modelling and surfacing cycles from designers perspective.
In: Weber, C., Husung, S., Cascini, G., Cantamessa, M., Marjanovic, D., and Rotini, F.
(eds.) DS 80-5 Proceedings of the 20th International Conference on Engineering Design
(ICED 15). Design Methods and Tools - Part 1, Milan, Italy, 27–30 July 2015, vol 5,
pp. 215–224. Lightning Source, Milton Keynes (2015)
2. Macey, S., Wardle, G.: H-point: The Fundamentals of Car Design & Packaging. Design
Studio Press, Culver City (2014)
3. Van Grondelle, E., Brand de Groot, S.: Discovering the meaning of form by exploded
sketching. In: Bohemia, E., Buck, L., Eriksen, K., Kovacevic, A., Ovesen, N., and
Tollestrup, C. (eds.) Proceedings of the 18th International Conference on Engineering and
Product Design Education (E&PDE16), Design Education: Collaboration and Cross-
Disciplinarity, Aalborg, Denmark, 8th-9th September 2016, pp. 374–379. The Design
Society, Glasgow (2016)
4. Bhise, V.D.: Automotive Product Development: A Systems Engineering Implementation.
CRC Press, Boca Raton (2017). https://doi.org/10.1201/9781315119502
5. Hekkert, P., Mostert, M., Stompff, G.: Dancing with a machine: a case of experience-driven
design. In: DPPI 2003 Proceedings of the 2003 International Conference on Designing
Pleasurable Products and Interfaces, pp. 114–119. ACM, New York (2003). https://doi.org/
10.1145/782896.782925
6. Akao, Y.: Quality Function Deployment (QFD) Integrating Customer Requirements into
Product Design. Productivity Press, Portland (1990)
7. Mincolelli, G.: Customer/user centered design. Analisi di un caso applicativo. Maggioli
Editore, Santarcangelo (2008)
8. Franceschini, F.: Advanced Quality Function Deployment. St. Lucie Press, Boca Raton
(2001)
Human-Centered Design for Automotive Styling Design 65
9. Van Boeijen, A., Daalhuizen, J., Zijlstra, J., Van Der Schoor, R.: Delft Design Guide. Bis
Publisher, Amsterdam (2014). https://doi.org/10.1017/CBO9781107415324.004
10. Hekkert, P., Van Dijk, M.: VIP Vision in Design: A Guidebook for Innovators. Bis
Publisher, Amsterdam (2011)
11. Verganti, R.: Design, meanings and radical innovation: a meta-model and a research agenda.
J. Prod. Innov. Manag. 25, 436–456 (2008). https://doi.org/10.1111/j.1540-5885.2008.
00313.x
12. Giacobone, G.A., Mincolelli, G.: Human-centered design and quality function deployment:
understanding needs on a multidisciplinary automotive research. In: Bucchianico, G.D. (ed.)
Advances in Intelligent Systems and Computing, pp. 57–68. Springer, Heidelberg (2020).
https://doi.org/10.1007/978-3-030-20444-0_6
Enriching the User Experience of a Connected
Car with Quantified Self
Maurizio Caon1(&), Marc Demierre1, Omar Abou Khaled1,
Elena Mugellini1, and Pierre Delaigue2
1
University of Applied Sciences and Arts Western Switzerland (HES-SO),
Fribourg, Switzerland
{maurizio.caon,omar.aboukhaled,
elena.mugellini}@hes-so.ch
2
Renault S.A., Boulogne-Billancourt, France
[email protected]
Abstract. This paper presents the R’SENS project, the design for a system
leveraging the Internet of Things to connect the dots between the user, her car
and her environment through the integration of tools belonging to different
domains, such as the connected car, the quantified self and web services. The
aim of this study is to test a concept through a prototype to explore the possi-
bility of enhancing the user experience of a connected car through the opportune
and ubiquitous display of relevant information not only about the car status and
the environment, but also the physical status of the driver. The integration of
three different user interfaces (i.e., wearable, smartphone and infotainment
system) to be used in different moments played a crucial role in the preliminary
evaluation of the perceived usefulness and usability of this proof-of-concept.
Keywords: Personal informatics Virtual companion Wearable Connected
car
1 Introduction
The automotive world is currently changing fast and connected technology started to be
integrated in commercial cars. The possibilities of interaction [1] and for the creation of
smart services [2] grows with the sources of data that are available for the user. For
instance, when the connected car (i.e., a vehicle that is able to send information about
its status to remote servers and to download information from the Internet) meets the
quantified self (i.e., the possibility to measure physiological signals about the person’s
status) new opportunities can arise [3]. In particular, the R’SENS project aimed at
exploring driver monitoring and interaction with the vehicle and its environment
outside of the driving situation. Its goal was to design and develop an always-on multi-
modal companion system for the driver, to provide her with complete feedback about
the driving conditions. This system provides a moment of self-reflection on the current
conditions of (1) her physiological status, (2) the car status and (3) the surrounding area
(including weather and traffic). For instance, if the wearable device recorded that the
user did not sleep much during the night, this information will be shown on one of the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 66–72, 2020.
https://doi.org/10.1007/978-3-030-39512-4_11
Enriching the User Experience of a Connected Car with Quantified Self 67
system interfaces to make the user aware of the not ideal conditions before a
trip. Moreover, the wearable device is used as a personal wireless key to open the car’s
doors, increasing the personal relation with the vehicle and also inducing the user to
look at the wearable, which shows a summary of the three domains conditions, before
entering into the car (Fig. 1). This concept and its prototype investigate how the user
experience of owning or just using a car is not limited to driving in itself, but it implies
other activities of our daily living. Cars play a crucial role in our current society;
therefore, it is important to design its experience in a comprehensive manner that
encompasses all the aspects of their users’ complex lives, which go way beyond the
time actually spent in the vehicle.
Fig. 1. Scenario depicting the concept of the use of a wearable device connected to the R’SENS
system.
2 User Experience and System Design
Car applications for smartphones and connected watches are not new [4]. Connected
cars already incorporate technology that enables owners to communicate remotely with
their vehicle. But none of these solutions collects data from a wearable’s body sensors.
R’SENS stands apart by delivering real-time feedback concerning the driver, the
vehicle and the environment for a seamless experience. Moreover, what makes this
technology unique is that it breaks the traditional frontiers by making this information
available on different platforms, applications and devices: on the Microsoft Band
wearable device for ubiquitous interaction [5], on the Android smartphone for detailed
information and on the R-Link infotainment system embedded in the car [6]. Last but
not least, R’SENS is truly user-friendly and boasts clear graphics aiming at generating a
moment of reflection on the conditions for a safe drive aiming at improving the user
experience (Fig. 2). Connected to a smartphone via Bluetooth, the wearable embeds
sensors to collect a range of biometric information about the user, including body
temperature, heart rate, sleeping time, calories spent, etc. The smartphone then
68 M. Caon et al.
transmits these data to a cloud-based server. The server is in turn connected to the
vehicle, from which it receives data such as tire pressures, remaining fuel or battery
charge and service intervals. Completing the triangle, the server receives real-time
information about the vehicle’s environment (e.g., weather and traffic). This informa-
tion is then made available under three menus respectively called “Me,” “My car” and
“My world” via the wearable, a smartphone application, or the in-car display (Fig. 2).
Fig. 2. The interfaces of R’SENS; (a) the wearable device showing a summary; (b) the
infotainment system of the connected car presenting all the details; the smartphone application
with the global view (c), and the detailed “Me” tab (d).
The central point of this system is a wearable device replacing the traditional car
key and giving a simple feedback in the form of icons or lights, as shown in Fig. 2(a).
There is also a smartphone application giving detailed feedback, Fig. 2(c) and (d), and
an application running in the R-Link car’s infotainment system to give feedback when
starting the car, as depicted in Fig. 2(b). The wearable interface has been designed as a
convenient interface to be glanced at just before entering into the car to check if
everything is fine. In case the user wants to read the details of each domain, she can do
so via the mobile application or the R-Link interface. The combination of these three
modalities offers a complete and integrated experience to the user, allowing her to
always be informed about the situation in a natural way. To determine if the conditions
are good for driving, the application gathers and analyzes data from 3 different sources:
the driver (via physiological sensors placed in a wearable device); the car (via Renault
R-Link); the driving environment (via external web services). The wearable gives a
simplified feedback to the user, where each of these 3 elements has an associated
colored icon (green, amber or red), making each icon similar to a traffic light. To get
more information, the driver can use the smartphone app (when outside the car or not
yet driving). In her car, she can get detailed information on the car’s infotainment
Enriching the User Experience of a Connected Car with Quantified Self 69
system’s R’SENS app. The wearable also acts as a car key to lock and unlock the
doors, replacing the traditional card key.
Figure 3 shows the general architecture of the project. Its main components are the
following:
• MS Band: The Microsoft Band, which gathers physiological data, shows the 3 icons
feedback and allows the user to access the car key features. It is connected to the
smartphone via BLE.
• Key module: Arduino-based modified card key, which receives commands from the
smartphone application via BLE and relays them to the car.
• Smartphone application: R’SENS Android application, which contains the logic to
talk with the key module, receive and analyze the physiological data from the
wearable and push it to the Collabox platform, send periodic GPS updates to the
Collabox platform, and receive the data from the car and environment from
Collabox.
• R-Link application: Embedded application in the R-Link car infotainment system
and connects to Collabox to push the car diagnostics and to receive the health and
environment data. It communicates with Collabox using the MQTT protocol.
• Third-party APIs: Third-party web services for weather (Forecast.io), traffic (Bing)
and health data (Microsoft Health).
• Collabox (Loop) Platform: The Collabox platform acts as a central bridge and
storage point to process, store and push data between the different applications. The
Collabox platform gets data from the traffic and weather services when a new GPS
location is received. It acts as a bridge for the Microsoft Health API.
Fig. 3. R’SENS System architecture diagram.
70 M. Caon et al.
3 Evaluation
The goal of this first user testing is to get a first feedback of the usefulness and usability
of the system, and more generally on the ideas of the project. It is also a good moment
to receive suggestions for changes or new features. For these tests, the users get the
application freshly installed on their device. They complete the first launch setup
wizard, and then use the system for a certain duration. The car part of the system was
simulated by injecting messages from Loop Triggers, as it was not possible to provide
users with a connected car. A few warnings about the car’s battery and tire pressure
were simulated at different moments to ensure that the status changed. Two types of
user tests have been done: long tests (more than 16 h, including a night); short tests
(about an hour). After the tests, the users were asked to fill a computer-based ques-
tionnaire (on Google Forms) to give their feedback. This questionnaire had three parts:
1. A System Usability Scale (SUS), which is a standard test used to measure the
usability of a system. 2. Some questions about their experience (what warnings they
received, if they noticed any bug/crash, etc.). This is useful to see if they noticed the
simulated warnings, and what other things happened during their test. 3. Some ques-
tions to collect their general feedback on the project and its ideas, what they
liked/didn’t like and additional remarks. In total, 5 people took part in the testing.
3 people did the long test, and 2 people did the short test. They have each been given a
participant number, so it would be possible to distinguish between the two groups.
Table 1 shows the scoring of the SUS. The results were computed for all users
together, and also for the long tests and short tests participants to see if there are
differences.
Table 1. Results of the SUS questionnaires filled out by the test participants.
Type of test SUS scores (0–100)
Average Minimum Maximum
All (5 participants) 82.0 75 95.0
Long test (3 participants) 85.8 75 95.0
Short test (2 participants) 76.3 75 77.5
The overall usability scores are quite good (a score of 80/100 corresponds to an A
grade). It was expected, because the system is relatively simple and there is not much
user input needed outside the initial setup process. The global minimal score is 75/100,
which is still above the value considered average (68) and corresponds to a “Good”
interface [7]. An interesting result is that the score for the short tests is lower than for
the long tests. This could suggest that as time passes the user becomes more com-
fortable with the system. The low number of users is of course a limitation.
The results of the first questions about noticing the changes in status show that the
user noticed at least one of the warnings. For the users in the long tests, there were
additional red and amber status for external reasons (mostly traffic incidents). One user
also had warnings about lack of sleep, which were correct.
Enriching the User Experience of a Connected Car with Quantified Self 71
In the general feedback part of the questionnaire, all users said that they thought
that R’SENS would be useful for drivers and that the separation in three domains (me,
my car, my world) made sense. However, there were some differences in the more
precise questions. One of the 5 users thought that including the user’s health/fitness
was not useful. At the question “Do you think having a simple feedback on the
wearable is useful?”, the results were mixed. The three users having done the long tests
answered “yes”, but the 2 users of the short tests answered “no”. These answers could
implicate that using the wearable in the long term is useful, but that the short test
settings did not reflect it. At the question “Do you think that replacing the car key with
a wearable device makes the interaction with the car more convenient?”, 3 users out of
5 answered no. One user also commented that the feature is similar to remote entry, so
it does not make interaction more convenient. These results mean that the advantage of
having the key always on the wrist is not enough to make it more useful. Other features
(such as virtual keys or increased security) would probably be needed. When asked
about irrelevant/useless features or information, one user thought that the wind con-
ditions seemed of limited use and another said he didn’t see a use for the heart rate in
his test. Winds can have effect on driving, but instant heart rate may not be really useful
as is. It’s worth exploring what other insights could be extracted from the heart rate.
When asked what they liked about the system in general, users mentioned the well-
integrated interface, the simplicity of the system, the additional information offered by
the “Me” and “My World” reports and the centralization of the information. When
asked what they disliked about the system, remarks about the lack of detailed infor-
mation and the fact that green should be hidden were again voiced by the same user.
4 Conclusion
This paper presented a first study aiming at exploring the concept of integrating per-
sonal relevant information through wearable sensors for the enhancement of the user
experience in an automotive context. Such experience should not be limited to the
driving activity but should put together many aspects of the user’s daily life. The
R’SENS system was developed to test this concept, including three interfaces designed
to provide this relevant information to the user in different manners, always adapted to
the context of use. The test showed that the system was appreciated and perceived as
useful although a number of modifications can be applied in order to improve it.
References
1. Angelini, L., Baumgartner, J., Carrino, F., Carrino, S., Caon, M., Khaled, O.A., Sauer, J.,
Lalanne, D., Mugellini, E., Sonderegger, A.: A comparison of three interaction modalities in
the car: gestures, voice and touch. In: Actes de la 28ième conférence francophone sur
l’Interaction Homme-Machine, pp. 188–196. ACM, October, 2016
2. Coppola, R., Morisio, M.: Connected car: technologies, issues, future trends. ACM Comput.
Surv. (CSUR) 49(3) (2016). Article No. 46
72 M. Caon et al.
3. Swan, M.: Connected car: quantified self becomes quantified car. J. Sens. Actuator Netw. 4
(1), 2–29 (2015)
4. Caon, M., Tagliabue, M., Angelini, L., Perego, P., Mugellini, E., Andreoni, G.: Wearable
technologies for automotive user interfaces: danger or opportunity? In: Adjunct Proceedings
of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular
Applications, pp. 1–5. ACM, September, 2014
5. Rawassizadeh, R., Price, B.A., Petre, M.: Wearables: has the age of smartwatches finally
arrived? Commun. ACM 58(1), 45–47 (2015)
6. R-Link store. https://fi.rlinkstore.com/. Accessed 27 September 2019
7. Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale.
Int. J. Hum.-Comput. Interact. 24(6), 574–594 (2008)
Constructing a Mental Model of Automation
Levels in the Area of Vehicle Guidance
Larissa Zacherl(&), Jonas Radlmayr(&), and Klaus Bengler(&)
Chair of Ergonomics, Technical University of Munich, Boltzmannstr. 15,
85748 Garching, Germany
[email protected],
{jonas.radlmayr,bengler}@tum.de
Abstract. The aim of this work was to construct a mental model of automation
in the area of vehicle guidance. The users’ understanding and mental model
affect the safety and user experience. The qualitative method Card Sorting was
applied with 25 participants. The task was to categorize cards on driving
automation according to the participants’ own understanding. Analysis showed
that the mental model of automation in the area of vehicle guidance is made up
of three levels: the level “Information and Driver” incorporates functions, which
do not influence lateral and longitudinal guidance of the vehicle. Systems that
interfere or control the lateral and longitudinal guidance of the vehicle are
included in the level “Assisted to Automated Driving”. The level “Autonomous
Driving” specifies systems that operate the vehicle independently while no
driver has to be present. Findings indicate a mismatch between the mental model
of users and well-known taxonomies of automated driving.
Keywords: Automated driving Levels of automation Mental models
1 Introduction
Driving Automation. Alot of different assisting and automated systems are being
researched and exist already where the drivers take over different roles such as oper-
ating versus monitoring. Hence, a clear understanding and categorization of these
systems is necessary. Different organizations like the Society of Automotive Engineers
(SAE), the National Highway Traffic Safety Administration (NHTSA) or the German
Federal Highway Research Institute BASt have defined taxonomies of driving
automation. The SAE taxonomy includes six levels of automation [1]. The taxonomies
published by NHTSA and the BASt include five levels of automation [2].
Automated driving functions change the interactions of driver, vehicle and other
road users [3]. Due to these changes, it is necessary to find out how humans perceive
and understand different levels of driving automation. In the expert community, a clear
understanding of different levels of automation is based on the existing taxonomies,
whereas the understanding of the users might be different. This paper tries to answer
the following research question: How is automation in the area of vehicle guidance
represented in the users’ perspective?
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 73–79, 2020.
https://doi.org/10.1007/978-3-030-39512-4_12
74 L. Zacherl et al.
Mental Models. According to Durso et al., a mental model of a system is the mental
representation of the functionalities of the system with all including aspects [4].
Problems in the area of human-computer-interaction can be tackled with the help of
mental models, as studies about how users perceive a device and how the users’ view
affects the handling can be conducted [5]. Conforming conclusions about the con-
ception and design of a device can be gathered. Hence, mental models are important
when researching the successful handling of devices.
Card Sorting. When applying the user-centered method open card sorting, labelled
cards are being sorted into categories created by each participant [6]. Participants are
restricted as little as possible and often have the option of renaming, reusing, removing
and adding cards [6]. Due to these non-restrictive options, card sorting can be used to
generate mental models and offer valuable clues to existing and non-existing products [6].
2 Method
Sample and Experimental Setup. The cards for the following card sorting study were
developed in an iterative process incorporating both expert and novice feedback (semi-
standard interviews with 10 participants) based on descriptions of existing and future
assisted and automated driving functions. In addition, some cards featured system
limits or restrictions of the operational design domain of these functions. The terms
have its origin in literature, as well as definitions and descriptions of automobile
manufacturers like Tesla and BMW [7, 8].
The analysis of the interviews was conducted according to the qualitative content
analysis of Mayring [9]. To avoid influencing participants specifically with specific
levels of driving automation, none of the established levels of driving automation were
included. 39 terms have arisen from the interviews including different aspects and
functions of driving automation.
The card sorting study was conducted with 25 participants (11 female and 14 male
participants). The mean age was M = 40.1 years (SD = 19.5) and the range was 18–89.
The participants had different educational and occupational backgrounds, and it was
ensured that no experts were included in the experiment. German (23 participants),
Spanish (one participant) and English (one participant) was specified as their native
language and the card sorting was always carried out in German. Participants had to
provide a self-assessment of their respective knowledge about driving automation:
Eight participants estimated their knowledge as “low,” ten as “moderate” and five as
“very high.” Only one participant estimated their knowledge as “very low” respectively
“high.” The knowledge of the participants regarding driving automation did not stem
from an occupational background but was rather based on experience or media. All
participants took part voluntarily and did not receive any compensation.
Procedure and Measures. In addition to preparing the cards, blank cards and a pen to
label these were provided to the participants. To document the study, a camera was
used, filming the hands and recording the voice of the participants as each participant
Constructing a Mental Model of Automation Levels in the Area of Vehicle Guidance 75
was asked to think aloud. At the beginning, the 39 cards were distributed randomly on
a large table area. Thus, no order of the cards has been conveyed to the participants.
To begin with, each participant had to fill in a consent form. The task was described
to sort the cards into categories, which are defined by the participant using blank cards.
Participants were granted a high amount of autonomy, by allowing categorizing the
same card into different categories, determining the number of categories and naming
the categories with a name or description. After finishing the card sorting, participants
were asked to fill in a demographic questionnaire. The study lasted between 30 and
60 min per participant and was conducted between the March 26, 2017 and the April
23, 2017.
To analyze the card sorting study, a spreadsheet according to the model of Lamantia
was generated [10]. The raw data (created categories and assigned cards) were docu-
mented for each participant. The chosen categories were matched with the categories
from the other participants. If two categories of different participants would match
regarding the overall content, these categories were combined and identified as one by
adding the names of both categories and by adapting the number of each assigned term.
If a category did not match any of the existing categories, it was added as a new
category. The specific decision of either combining categories or having new ones was
reached by an expert decision from the authors. This procedure was repeated for all
participants. In this process, 23 different categories were generated. For each card in
each category, the absolute occurrence was recorded.
Afterwards, agreement weights were calculated for each category and each term
according to Paul [6]. Each category with an agreement weight of more than 50% was
analyzed further. In addition, an agreement weight for each term was calculated. Only
the terms that were put into a specific category by more than 50% of the participants
creating this category remain therein [6]. In summary, the remaining categories were
assigned general names considering the titles and descriptions provided by the
participants.
agreement weight (category) in % = number of participants generating this
category/total number of participants.
agreement weight (term) in % = absolute occurrence of term in this
category/number of participants generating this category.
3 Results
Each participant generated three to seven different categories; the mean is M = 4.2. The
participants created the categories according to different patterns, which they explained
aloud. The most frequent one being the arbitration of how much the driver or the
system are involved into controlling the vehicle. After applying the analysis, three
categories emerged:
Category 1 (C1): “Information and Driver” (agreement weight: 64%),
Category 2 (C2): “Assisted to Automated Driving” (agreement weight: 88%)
Category 3 (C3): “Autonomous Driving” (agreement weight: 88%).
76 L. Zacherl et al.
Table 1 shows the result of the card sorting study. Note: The card sorting was
originally conducted in the German language; this is a translated version that was not
used for the study but lectured by professional translators.
Table 1. Card sorting results, the agreement weight is shown per term and category in %.
Terms/cards with agreement weights under 50% are neglected.
Term/Card C1 C2 C3
The driver steers the vehicle 81.3 4.5
The driver accelerates the vehicle 81.3 9.1
The driver brakes the vehicle 81.3 4.5
The driver is not allowed to carry out any side tasks 81.3 4.5
Parking Sensor: The system warns the driver about potential 68.8 36.4
barriers when parking by giving an acoustic signal. The system does
not interfere with the lateral or longitudinal guidance of the vehicle
Blind Spot Monitor: The system warns the driver about potential 62.5 50.0 9.1
collisions when changing lane by a display in the side mirror. The
system does not interfere with the lateral or longitudinal guidance of
the vehicle
Lane Departure Warning System: The car warns the driver by 62.5 63.6
vibration on the steering wheel as soon as it unintentionally leaves
the lane. The car drives and does not steer itself but gives only
feedback by vibration
Collision Warning: The car warns the driver of possible frontal 62.5 63.6
collisions by an acoustic signal. The driver has to brake on his own
Cruise Control: The driver sets a speed that is maintained by the car. 18.8 95.5
The driver has to steer and, if necessary, adjust speed and brake
Adaptive Cruise Control: The driver enters a maximum speed and a 6.3 95.5
minimum distance to the vehicle ahead, which must not be
undershot. If the vehicle approaches another vehicle on the identical
lane, the car automatically brakes and accelerates again with free
travel to the maximum entered speed. The driver has to steer and
monitor the system
Traffic Jam Assistant: The car brakes, accelerates and steers in 90.9 12.5
traffic jams (up to approx. 60 km/h) - without lane change. The driver
must monitor the system permanently
Lane Departure Assist: The system prevents the vehicle from 6.3 90.9
unintentionally leaving the lane by interfering with lateral control
Park Steering Assist: The car controls the steering wheel when 90.9
parking – while the driver has to accelerate and brake. The car steers
itself and gives instructions for acceleration and braking
Collision Avoidance Assist: An (emergency) braking maneuver will 77.3 31.8
be executed by the car in critical situations
(continued)
Constructing a Mental Model of Automation Levels in the Area of Vehicle Guidance 77
Table 1. (continued)
Term/Card C1 C2 C3
Traffic Jam Pilot: The car brakes, accelerates and steers in traffic 68.2 31.8
jams (up to 60 km/h) itself - without lane change. The driver does not
need to monitor the system permanently
Emergency Stop Assist: If the car notices that the driver is unable to 63.3 40.9
drive the vehicle, the system takes control and brings the vehicle
safely to a halt
Autobahn/Highway Pilot: Enables automated driving on highways 63.6 40.9
up to approx. 130 km/h from the entrance to the exit, including lane
changes. The driver must turn on the system and then no longer
monitor it
Can be applied on autobahn/highway (without approach and 12.5 59.1 22.7
departure)
Robot Taxi: Allows autonomous driving from start to finish – 95.5
without restrictions: a driver is not necessary
Parking Garage Pilot: Allows autonomous parking in and out of a 13.6 81.8
parking garage including parking space search and provision of the
vehicle. The driver can move away and does not have to control the
parking process
The system steers the vehicle 18.2 81.8
Monitoring the driving environment is in charge of the system 6.3 81.8
The system accelerates the vehicle 36.4 72.7
The system brakes the vehicle 31.8 77.3
The driver does not have to be prepared to take over the driving task 77.3
completely
The driver is allowed to carry out the following side tasks: sleep 68.2
The system is able to react to weaker road users like pedestrians or 31.8 68.2
cyclists
The driver is allowed to carry out the following side tasks: play a 9.1 63.6
game on the smartphone
The driver is allowed to carry out the following side tasks: read the 22.7 59.1
news on the display at the center console
Can be applied in bright sunshine 25.0 36.4 59.1
Can be applied in the case of road construction 25.0 45.5 59.1
4 Discussion
Despite no limitations on the number of categories and categorizing according to the
participants’ own understanding, the mental model emerged to consist of three cate-
gories describing to which extend the driver and the system arbitrate vehicle guidance
with respect to different assisted or automated driving functions:
“Information and Driver” (C1): This category included the four warning functions,
which do not interfere with controlling the vehicle. This showed that the participants
78 L. Zacherl et al.
clearly separated warning functions and assisting or automated functions. The
remaining terms of this category described the responsibility of the driver, e.g. the
driver has to steer the vehicle.
“Assisted to Automated Driving” (C2): Two of the warning functions attributed to
C1 were also matched to C2. Additionally, this category incorporated all the given
assisting and automated driving functions where drivers still have the option to engage
into the driving task (level 1–3 following SAE/NHTSA). It can be concluded that the
participants had difficulties to sort the variety of different driving functions.
“Autonomous Driving” (C3): This category contained the two functions where the
driver becomes solely a passenger. Additionally, different terms describing the category
were included, e.g. the responsibility of the system taking over the steering task. These
clear and detailed descriptions highlighted that the participants did have a clear
imagination of this category.
SAE’s and NHTSA’s definition consist of five or six levels [1, 2], whereas the
mental model constructed in this study only contains three levels. C1 corresponds to
level 0 of SAE and NHTSA. C2 is rather unspecific and includes level 1, 2 and 3 of
SAE’s and NHTSA’s definition. Level 4 and 5 of SAE’s definition and level 4 of
NHTSA’s definition can be attributed to C3. Hence, the levels of automation of the
mental model of users appear to differ significantly from the taxonomies of the SAE
and the NHTSA. The comparison is shown in Table 2.
Table 2. Comparison of the mental model of driving automation constructed in this study and
the taxonomies of SAE and NHTSA.
Mental Information Assisted to Automated Driving Autonomous Driving
model and Driver
SAE Level 0: Level 1: Level 2: Level 3: Level 4: Level 5:
[1] No Driver Partial Conditional High Full
Automation Assisted Automation Automation Automation Automation
NHTSA Level 0: Level 1: Level 2: Level 3: Level 4:
[2] No Function- Combined Limited Full Self-Driving
Automation Specific Function Self-Driving Automation
Automation Automation Automation
Mental models are crucial to gather conclusions about the conception and design of
a function and hence, to allow a successful handling of the function. There appears to
be a mismatch between the arbitration of responsibility represented in users’ mental
models and existing taxonomies concerning specific functions. The successful intro-
duction of higher levels of automation is likely based on a correct mental model of
users and should be considered when designing automated driving functions. Con-
sidering the mental model contributes to developing systems and functions which are
understandable, intuitive and easy to use.
Constructing a Mental Model of Automation Levels in the Area of Vehicle Guidance 79
References
1. Society of Automotive Engineers: J3016 - Taxonomy and Definitions for Terms Related to
on-road Motor Vehicle Automated Driving Systems. Surface Vehicle Recommended
Practice (2018)
2. National Highway Traffic Safety Administration. Preliminary Statement of Policy Concern-
ing Automated Vehicles (2013). http://www.nhtsa.gov/staticfiles/rulemaking/pdf/
3. Meschtscherjakov, A., Tscheligi, M., Szostak, D., et al.: HCI and autonomous vehicles:
contextual experience informs design. In: CHI EA 2016 Proceedings of the 2016 CHI
Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3542–3549.
ACM (2016). https://doi.org/10.1145/2851581.2856489
4. Durso, F.T., Gronlund, S.D.: Situation awareness. In: Durso, F.T., Nickerson, R.S.,
Schvaneveldt, R.W., Dumais, S.T., Lindsay, D.S., Chi, M.T.H. (eds.) Handbook of Applied
Cognition, vol. 10, pp. 283–314. Wiley (1999)
5. Payne, S.J.: Mental models in human-computer interaction. In: The Human-Computer
Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications,
vol. 3, pp. 39–52, Second Edn. Taylor and Francis (2007). ISBN: 978-1-4200-8881-6
6. Paul, C.L.: A modified delphi approach to a new card sorting methodology. J. Usability
Stud. 4(1), 7–30 (2008). Usability Professionals’ Association
7. Tesla, I.: Autopilot: Full Self-Driving Hardware on All Cars (2018). https://www.tesla.com/
autopilot
8. BMW AG, Autonomous Driving (2018). https://www.bmwgroup.com/en/innovation/
technologies-and-mobility/autonomes-fahren.html
9. Mayring, P.: Qualitative content analysis. Advance online publication (2000). https://doi.org/
10.17169/FQS-1.2.1089. Flick, U., von Kardoff, E., Keupp, H., von Rosenstiel, L., Wolff, S.
(eds.)
10. Lamantia, J.: Analyzing Card Sort Results with a Spreadsheet Template (2003). http://
boxesandarrows.com/analyzing-card-sort-results-with-a-spreadsheet-template/
Effect of Phone Interface Modality on Drivers’
Task Load Index in Conventional
and Semi-Automated Vehicles
Kristina Davtyan1 and Francesca Favaro1,2(&)
1
RiSAS Research Center, San Jose State University, San Jose, CA, USA
{kristina.davtyan,francesca.favaro}@sjsu.edu
2
Department of Aviation and Technology, San Jose State University, San Jose,
CA, USA
Abstract. This study tested how two different interaction modalities with a
smartphone (manual texting and vocal replies) affected task loads perceived by
drivers places in various driving scenarios. The study employed human-in-the-
loop simulation, and involved both a city environment (driven in conventional
manual mode) and a highway environment (driven first in automated mode and
then requiring takeover from the driver). The interface was tested in relation to
following GPS instructions while being prompted to reply to texts at specific
times in the simulation. The results compare the overall mental load scores of
the manual texting condition for both city and highway scenarios to the vocal
reply condition. Unweighted NASA TLX data shows that the vocal interface led
to lower scores; however, statistical significance was shown only for the city
scenarios for physical load and effort calculations.
Keywords: Texting task load Human-in-the-loop simulation Smartphone
interaction
1 Introduction
Distracted driving is the main cause of fatalities on US roads [1]. The National
Highway Traffic Safety Administration estimates that, regardless of the regulation in
place in a specific state, over 60% of the drivers’ population engage in cell-phone based
activities while driving at least once per day [2]. Currently, there is no national ban on
texting or using a wireless phone while driving, but a number of U.S. states have
passed laws banning texting or wireless phones or requiring hands-free use of wireless
phones while driving [3]. Among researchers, there is a consensus that research is
needed to establish a clear and quantitative effect of smartphone usage on driving
performance [4]. A substantial body of literature from 2000 to 2018 shows mixed
results [5, 6], with manual texting clearly correlated to impairment of normal driving
activities, but also pointing to the existence of circumstances under which the use of
phones enhances alertness due to more expected threats. Moreover, in conjunction with
automation, research has shown that drivers of highly automated vehicles are more
likely to engage in secondary activities while driving [7].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 80–85, 2020.
https://doi.org/10.1007/978-3-030-39512-4_13
Effect of Phone Interface Modality on Drivers’ Task Load Index 81
The present work focuses on how such interaction with secondary tasks is carried
out, by comparing two different modes of interaction between drivers and smartphones:
touch-based commands vs. voice-to-text commands. 32 subjects participated in a
human-in-the-loop simulation scenario with an integrated-car simulator study. We
assessed the NASA Task Load Index (TLX) for both smartphone interaction modality
(touch-based vs. voice-to-text) during two driving scenarios (city vs. highway).
The structure of the paper is as follows: Sect. 2 provides an overview of the
experimental setup; Sect. 3 presents the study’s main results; Sect. 4 concludes the
paper.
2 Methods
2.1 Participants
32 participants (16 men and 16 women, M = 26.8 years of age, SD = 6.5 years of age)
from Silicon Valley, CA volunteered to participate in this driving experiment. All
participants were screened prior to participation to ensure normal or corrected-to-
normal vision. All participants were active CA-licensed drivers with no medical con-
dition that could impair their driving. Participants reported sending 2 text messages on
average per drive (SD = 1.24). The texting frequency of our population was compa-
rable to literature on the topic.
2.2 Apparatus and Stimuli
Driving behavior was assessed in a human-in-the-loop static simulator, consisting of a
real BMW series 6 rented from FKA Silicon Valley. The simulation environment used
the Linux-based simulation framework Virtual Test Drive (VTD) by Vires Simula-
tionstechnologie GmbH in version 2.1.0. Open standards (OpenDRIVE® and Open-
SCENARIO) were used for road and scenario creation. The simulated driving task was
displayed on a 220° surround projection screen, with a resolution of 1080 1920 and a
refresh rate of 60 Hz. The simulator showed road and traffic information projected in
front of the car and through a rear-view mirror behind the vehicle. A three-way split
rear-projection wall provided the projection for side and rear-view mirrors. A 4.3′′
Nexus 6 touch-screen smartphone running Android OS Version 5 was used for the
texting task, and to navigate the participants through the simulated city and highway
environments. The buttons on the keyboard of the smartphone were arranged in a
QWERTY layout.
2.3 Experiment Tasks
Driving Task. Participants were instructed to follow the GPS instructions on the
mounted Android phone during the driving scenarios. During manual city scenarios,
participants had to complete a series of left and right turns, and stop at stop signs and a red
light while they maintained posted speed limit indicated in the navigation interface. The
city scenario was driven in conventional manual mode. During the fully-autonomous
82 K. Davtyan and F. Favaro
highway scenarios, participants were instructed to continue monitoring the outside
environment also while the vehicle was driven autonomously, and to be ready to take over
control of the vehicle. A structured disengagement event would then take place, with GPS
indicating to take the next exit while observing all traffic rules. Vehicle dynamics,
including lane position, steering wheel position, speed, and braking distance were
recorded during the execution of the driving scenario. The simulated driving scenario
consisted of a 1 mile long single lane suburban streets grid (city scenario) and three-lane
highway with straight and curved portions (highway scenario). For the highway envi-
ronment, traffic density was kept constant for all tests, for a total of 50 vehicles distributed
within a 400-m diameter from the test vehicle.
Texting Task. A texting task was used to compare the distracting effects of manual
versus vocal texting on driving performance. In the city scenario, two texts were
introduced as secondary non-driving task. The first text prodded the driver to provide
his/her ETA (displayed on the GPS app), and was introduced in conjunction with a
traffic light turning red. While replying to the text, the participant would need to resume
driving after the light became green, while a second text message would be received,
asking whether the participant would prefer a cheese or pepperoni pizza for dinner. In
the highway scenario, a single text was employed. The text, which again inquired about
the ETA, was triggered right after the notification to the driver to “prepare to takeover”
for the vehicle movement. 15 s after the “priming” of the participant, the actual dis-
engagement would be triggered, with the driver supposed to correctly take the highway
exit and the text awaiting reply.
NASA TLX. NASA TLX has been cited in over 4,400 studies. It is considered a
useful measure for giving insight into the cognitive workload of a given task. A paper-
pen NASA TLX was administered immediately after the end of each driving scenario.
The TLX test consisted of six subjective subscales. In each section, the participant was
required to rate on a 20-points range for: mental demand; physical demand; temporal
demand; performance; effort; and frustration. Descriptions were provided for each item,
and each participant completed a total of 4 TLX surveys, one for each repetition of
scenario and combination of texting interface. Participants were not asked to rate each
subscale for perceived importance; hence, we use Raw TLX scores for analysis.
Test Procedure. The test comprised a total of five sections. Four of those were given
by the counterbalanced conditions from the variables tested. These included two dual-
task conditions in which participants operated the car in manual mode during the city
scenario while texting using manual input modality (manual + city) and voice-to-text
input modality (voice-to-text + city), as well as in autonomous car mode during a
highway scenario (manual + highway; voice-to-text + highway). A fifth section was
included at the beginning of each test, to allow participants to get accustomed to
simulated driving. The practice phase lasted roughly 7 min and was concluded only
after the participant expressed comfort with the simulator driving performance. The
tests had an average duration of 5 min per condition. The entire session lasted about
60 min (including pre- and post-test surveys), and drivers were informed that they
could withdraw from participation at any time. Figure 1 reports a schematic structure of
the test, where each intermediate drive would also feature an in-between tests survey.
Effect of Phone Interface Modality on Drivers’ Task Load Index 83
Fig. 1. Schematic representation of the test structure
Statistical Analysis. For the purposes of this study, NASA-TLX workload scores
from each of the 4 four test-conditions were analyzed and compared using effect-size
for a repeated measures. Cohen’s effect size is useful for estimating statistical power
and sample size, especially if there is low variance. Overall workload ratings were
derived through the use of unweighted “raw” combination of TLX subscales.
3 Analysis of Results
Looking at all the subscales across all conditions and participants, physical (M = 6.57,
SD = 4.10) and performance (M = 6.79, SD = 4.42) subscales total mean scores were
the lowest in the set (see Table 1). The highest recorded mean scores were for effort
(M = 8.65, SD = 4.57), mental (M = 8, SD = 4.60), temporal (M = 7.90, SD = 4.89), and
frustration (M = 7.79, SD = 4.85) sub-scales.
Table 1. Descriptive statistics summary of TLX sub-scores (mental demand; physical demand;
temporal demand; performance; effort; and frustration) across four test conditions (city manual -
Cm, city voice - Cv, hwy manual - Hm, hwy voice - Hv)
Mode Mental Physical Temporal Performance Effort Frustration Total
Cm M 9.03 8.50 7.78 5.91 9.91 7.88 49
SD 4.27 4.53 4.38 3.92 4.21 4.72 18.06
Cv M 8.13 7.28 7.14 5.52 8.17 7.67 44.17
SD 5.01 3.98 4.92 3.73 4.76 4.69 20.04
Hm M 7.97 5.44 8.78 8.41 8.44 7.91 46.94
SD 4.60 3.76 5.49 4.97 4.59 5.04 21.45
Hv M 6.88 5.06 7.63 7.34 8.09 7.69 42.69
SD 4.46 3.24 4.82 4.54 4.68 5.17 20.42
Looking at the total TLX mean scores by modality (manual vs voice-to-text) across
both city and highway scenarios, mean scores for city manual and highway manual
(M = 49, SD = 18.06, M = 46.94, SD = 21.45) appeared to be higher on average
compared to vocal replies for both the city or highway scenario (M = 44.17,
84 K. Davtyan and F. Favaro
SD = 20.04, M = 42.69, SD = 20.42). Figure 2 provides a visual depiction of the
average sub-scores for the investigated conditions.
Fig. 2. Graph showing the mean values for each of the 6 different sub-scores of NASA TLX
(mental demand; physical demand; temporal demand; performance; effort; and frustration) as
well as a total NASA TLX workload score
Cohen’s d effect size (d = .25) for paired samples within-group comparison design
analysis revealed that the main effect of cell-phone use modality (manual vs voice-to-
text) did not exceed Cohen’s convention for a large effect, where d = 0.2 is considered
to be a small effect size. These results indicate that participants in the voice-to-text
secondary task condition did not do better in the manual texting condition when
compared across both city and highway scenario. Two-tailed t-tests were executed.
Statistical significance was shown only when comparing manual and vocal interfaces
for physical demand (p = 0.039) and effort (p = 0.024) for city scenario. None of the
TLX sub-scores showed statistical significance for the highway scenario.
4 Conclusions
Data from our study suggests that when it comes to driver’s task load as measured
across mental, physical, temporal, effort, performance, frustration ratings, participants
who engaged in texting using vocal input have slightly lower scores compared to
conditions where they used traditional manual features, regardless if they engaged in
manual city or autonomous highway primary task scenarios. Statistical analysis
revealed however, that these differences have a low effect size. This has implications
for the current state driving regulations in U.S. overwhelmingly enforce a texting ban
while making no stipulations regarding the use of voice-to-text technology. These
conclusions are limited to measuring mental workload only and is not meant to
advocate for a ban of voice-to-text messaging during driving. However, it does cast
doubt to popular claim that leveraging voice commands technology represents a “safety
improvement”.
Effect of Phone Interface Modality on Drivers’ Task Load Index 85
Acknowledgments. Funding for this research was provided by the US Department of Trans-
portation (grant 69A3551747127 managed by the Mineta Transportation Institute of San Jose,
CA).
References
1. National Highway Traffic Safety Administration, NHTSA – Distracted Driving 2015 –
DOT HS 812 381 (2015). https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812381
2. Zendrive, 2018 Distracted driving snapshot (2018). http://blog.zendrive.com/distracted-
driving-is-far-worse/
3. US Federal Communications Commissions, The Dangers of Distracted Driving (2018).
https://www.fcc.gov/consumers/guides/dangers-texting-while-driving
4. Caird, J.K., Willness, C.R., Steel, P., Scialfa, C.: A meta-analysis of the effects of cell phones
on driver performance. Accid. Anal. Prev. 40(4), 1282–1293 (2008)
5. Fitch, G.M., Soccolich, S.A., Guo, F., McClafferty, J., Fang, Y., Olson, R.L., Perez, M.A.,
Hanowski, R.J., Hankey, J.M., Dingus, T.A.: The impact of hand-held and hands-free cell
phone use on driving performance and safety-critical event risk (No. DOT HS 811 757)
(2013)
6. Neubauer, C., Matthews, G., Saxby, D.: The effects of cell phone use and automation on
driver performance and subjective state in simulated driving. In: Proceedings of the Human
Factors and Ergonomics Society Annual Meeting, vol. 56, no. 1, pp. 1987–1991 (2012)
7. Llaneras, R.E., Salinger, J., Green, C.A.: Human factors issues associated with limited ability
autonomous driving systems. In: Proceedings of the 7th International Driving Symposium on
Human Factors in Driver Assessment, Training and Vehicle Design, Bolton Landing,
New York, 17–20 June, vol. 2013, pp. 92–98. Public Policy Center, University of Iowa, Iowa
City (2013)
Software Failure Mode and Effects Analysis
Palak Talwar(&)
Senior Safety Engineer, Lyft Level 5, Palo Alto, CA, USA
[email protected]
Abstract. Failure Mode and Effects Analysis (FMEA) is a key safety assess-
ment analysis that determine failure modes at system, hardware and software
level. Overlooking Failure Modes can often cause system or functionality failure
which directly impacts a systems safety performance, reliability and quality.
FMEA is a bottom-up approach which has four key phases - identification of
fault, assessment of impact, determination of potential causes and their resolu-
tions, and finally testing and documentation of analysis. FMEA addresses the
effect of failures at the system, software and hardware level. The outcome of the
analysis helps us identify gaps in safety requirements specification and provides
input for component testing, integration testing and system level testing. This
paper describes the application of Failure Mode and Effects Analysis (FMEA) to
software modules.
Keywords: Control systems faults Failure modes Safety assessment
Software failure analysis
1 Failure Mode and Effects Analysis (FMEA)
Software Failure Mode and Effects Analysis (FMEA) is a bottom-up analysis technique
to identify the consequences of possible software failure modes on the software system.
An example below outlines the application of Software FMEA to Brake ECU (Elec-
tronic Control Unit).
As depicted in Fig. 1 below, Brake ECU receives brake pedal sensor input from the
driver as an analog signal and vehicle speed information from another ECU via CAN which
in turn outputs brake torque request and brake module status to other ECUs over CAN.
Fig. 1. Brake ECU and its interfaces
FMEA starts with identifying different software failure modes that can influence the
subsystem or system. The four phases (mentioned above) is one potential approach to
perform FMEA. A brief expansion of these phases are:
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 86–91, 2020.
https://doi.org/10.1007/978-3-030-39512-4_14
Software Failure Mode and Effects Analysis 87
• Look at the system functionality holistically and identify a comprehensive list of
potential failure modes
• For each identified failure mode in step 1, assess the implications of failure on
connected software or hardware system and also on the overall performance of the
system
• Once we know the overall impact, we isolate potential causes for failure. Once the
causes are identified the system design needs to be enhanced to adequately prevent
future failures
• Once the design change is made, we retest the failure mode to ensure that the
system appropriately handles the failure before release. Then, the necessary docu-
mentation in done
Now that we have a brief understanding of the approach, let’s follow these steps to
perform software FMEA on Brake ECU depicted in Fig. 1 above.
1.1 Step 1
For the example above, let’s start by listing individual components including interfaces,
the function they provide and their failure modes.
The only component that is of interest here is Brake ECU with inputs and outputs.
The function can be defined as:
1. transmitting brake torque request, based off inputs: brake pedal sensor and vehicle
speed, to other vehicle modules.
2. sending brake module fault status to other vehicle modules.
The failure modes for interfaces and the component (Brake ECU) can be defined as
follows (Table 1):
Table 1. Potential failure modes.
Component/Interface Function Potential failure mode
Brake pedal sensor analog No signal
voltage input Signal voltage out of range
Vehicle speed Message corruption
Message loss
Message timeout
Brake ECU Transmits brake NO brake torque request
torque request DELAYED brake torque request
INVALID brake torque request
Sends brake module NO brake module status
fault status to other DELAYED brake module status
vehicle modules INVALID brake module status
88 P. Talwar
1.2 Step 2
Once we have listed the failure modes, let’s determine the effect(s) of the failure on
other system components and on the overall system for each failure mode.
For the example above, we determine the effect(s) of receiving invalid or delayed
vehicle speed, brake pedal analog voltage out of range, or not receiving anything at all
and ask this question:
– What if the brake pedal input requested by the driver is not received for a certain
period of time?
– What if we receive corrupted vehicle speed over CAN? Are we okay with 1*
corrupted message or not?
– Does the failure impact vehicle behavior resulting in high severity?
The table below lists the potential effect(s) of failure which might or might not
impact vehicle behavior (Table 2).
Table 2. Potential effects of failure.
Component/Interface Function Potential Potential effect(s) of failure
failure mode
Brake pedal sensor No signal
analog voltage input Signal
voltage out
of range
Vehicle speed Message
corruption
Message
loss
Message
timeout
Brake ECU Transmits brake NO brake No brake command issued to the
torque request torque vehicle actuator when requested
request by the driver
DELAYED Brake command issued too late to
brake torque the vehicle actuator when
request requested by the driver
INVALID Invalid brake command issued to
brake torque the vehicle actuator when
request requested by the driver which
might cause overbraking
Sends brake module NO brake No brake module status issued to
fault status to other module other vehicle modules in order to
vehicle modules status notify brake ECU failure
(continued)
Software Failure Mode and Effects Analysis 89
Table 2. (continued)
Component/Interface Function Potential Potential effect(s) of failure
failure mode
DELAYED Brake module status issued too
brake late to other vehicle modules in
module order to notify brake ECU failure
status
INVALID Invalid brake module status issued
brake to other vehicle modules in order
module to notify brake ECU failure
status
1.3 Step 3
After we are done defining the failure modes and potential effect(s) of failure, the next
step is to determine potential cause(s) of failure. For each failure mode, we determine
all possible causes, including both hardware and software. Listing potential cause(s) of
failure helps us figure out which design controls prevention technique to be imple-
mented in order to mitigate these failures. We can have the mitigation strategy defined
only in hardware or software or both (Table 3).
Table 3. Potential causes of failure.
Component/Interface Function Potential Potential effect(s) of Potential cause
failure mode failure (s) of failure
Brake pedal sensor No signal
analog voltage input Signal
voltage out
of range
Vehicle speed Message
corruption
Message
loss
Message
timeout
Brake ECU Transmits NO brake No brake command [brake pedal
brake torque torque issued to the vehicle sensor analog
request request actuator when voltage input]
requested by the driver No signal
[vehicle speed]
Message loss
No power
supply
(continued)
90 P. Talwar
Table 3. (continued)
Component/Interface Function Potential Potential effect(s) of Potential cause
failure mode failure (s) of failure
DELAYED Brake command [vehicle speed]
brake torque issued too late to the Message timeout
request vehicle actuator when [Brake ECU]
requested by the driver Internal fault
INVALID Invalid brake [brake pedal
brake torque command issued to the sensor analog
request vehicle actuator when voltage input]
requested by the driver Signal voltage
which might cause out of range
overbraking [vehicle speed]
Message
corruption
[Brake ECU]
Internal fault
Sends brake NO brake No brake module [Brake ECU]
module fault module status issued to other Internal fault
status to status vehicle modules in No power
other vehicle order to notify brake supply
modules ECU failure
DELAYED Brake module status [Brake ECU]
brake issued too late to other Internal fault
module vehicle modules in
status order to notify brake
ECU failure
INVALID Invalid brake module [Brake ECU]
brake status issued to other Internal fault
module vehicle modules in
status order to notify brake
ECU failure
1.4 Steps 4 and 5
After we are done identifying potential failure modes and causes of failure with the
severity of failure captured under potential effects column, we list down current design
controls prevention and recommend action(s) to mitigate these failures if already not in
place. For example:
– To mitigate brake pedal sensor failure, we can add a redundant sensor to fall back
on in case the primary sensor fails. Also, we can add plausibility check which reads
both the sensor voltages and compare against each other and set a fault if the
difference between the two increases by some value for some period of time.
Software Failure Mode and Effects Analysis 91
– To check for CAN message corruption, we can verify CRC (Cyclic Redundancy
Check), parity bit, etc. added to a field of CAN messages, on the receiver side and
set a fault flag if invalid CRCs exceed a threshold.
– To check for CAN message drop or loss, we can verify MC (Message Counter),
sequence number, etc. added to a field of CAN messages, on the receiver side
and/or check for timeout.
– To mitigate risk, we can add sensor fault detection strategy in hardware, like what
happens if the power supply to the sensor goes off, what is the sensor malfunctions,
what if there is a register failure, etc. and what actions to take.
2 Conclusion
Bottoms up FMEA analysis approach helps in functionality level failure modes iden-
tification, assessment of severity and the impact on the overall system. If there is no
impact, then we can be fairly confident that the system design is robust. If there is some
impact, then preventive measures need to be initiated as highlighted in this paper.
A Validated Failure Behavior Model
for Driver Behavior Models for Generating
Skid-Scenarios on Motorways
Bernd Huber1(&), Paul Schmidl2, Christoph Sippl1,
and Anatoli Djanatliev3
1
AUDI AG, Simulation Electric/Electronic, 85045 Ingolstadt, Germany
{bernd2.huber,christoph.sippl}@audi.de
2
Technical University of Munich, Chair of Ergonomics, Boltzmannstr. 15,
85748 Garching b. München, Germany
[email protected]
3
Computer Networks and Communication Systems, Friedrich-Alexander
Universität Erlangen, Martensstr. 3, 91058 Erlangen, Germany
[email protected]
Abstract. The automation of the driving task will gain importance in future
mobility solutions for private transport. However, the sufficient validation of
automated driving functions poses enormous challenges for academia and
industry. This contribution proposes a failure behavior model for driver models
for generating skid-scenarios on motorways. The model is based on results of
the five-step-method provided by accident researchers. The failure behavior
model is implemented using a neural network, which is trained utilizing a
reinforcement learning algorithm. Hereby, the aim of the neuronal network is to
maximize the vehicle’s side slip angle to initiate skidding of the vehicle. Con-
cluding, the failure behavior model is validated by reconstructing a real accident
in a traffic simulation using the failure behavior model.
Keywords: Simulation Human-Modeling Automated driving
1 Introduction
In future concepts of individual mobility, the automation of the driving task is
becoming an important requirement. Automated Driving Functions (ADFs) open up
new sales opportunities for car manufacturers [1]. In addition, ADFs allows traffic
safety to be significantly increased, as over 90% of traffic accidents can be linked to
driver failures [2]. In order to satisfy these objectives, the ADF must perform as error-
free as possible in any potential situation in its working domain. The resulting com-
plexity of ADFs impedes the formulation of detailed and complete requirements.
Consequently, the verification and validation of ADFs is partial and therefore involves
risks [3]. Bergenhem et al. defines this problem as “sematic gap” [4].
Various approaches exist to validate the mode of operation of ADFs. On the one
hand, function tests can be carried out on public roads with the aid of test drives. On the
other hand, simulations allow functions tests to be conducted in the laboratory.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 92–98, 2020.
https://doi.org/10.1007/978-3-030-39512-4_15
A Validated Failure Behavior Model for Driver Behavior Models 93
However, performing real test drives to validate ADFs is not feasible due to economic
constraints, technological restrictions or reasons of safety [5]. Along the product
development process, so-called X-in-the-loop (XIL) methods are deployed to evaluate
vehicle functions in a virtual environment [6]. Ulbrich et al. [7] describe the systematic
application of such simulation methods for testing an automated lane changing function
along the development phase. Here, the function developers modelled traffic simulation
scenarios for the validation of the ADF. This is time-consuming and complex as the
behavior of each entity must be described individually. Therefore, researchers proposed
different approaches for automating the scenario generation process [8, 9]. Alterna-
tively, behavior models such as driver behavior models are capable of defining the
behavior of entities in the simulation. Numerous driver behavior models for various
purposes and different levels of detail have been presented in the literature [10].
2 Concept
We have already published the concept [11] of a modular and validated failure behavior
model in driver behavior models for testing ADFs. The concept is based on findings by
an interdisciplinary team of the Audi Accident Research Unit (AARU)1. Especially the
conclusions of the five-step-method [12] are valuable for the modeling of failure
behavior for driver models. Among other aspects, this method enables the AARU to
define and categorize the human cause factor of an accident. The human cause factors
of the five-step-method are derived from human decision-making models. This infor-
mation is used to model driver’s failure behavior. Therefore, a driver model is
mandatory, which also reflects the levels of the human decision-making process. This
enables the implementation of modular human failure behaviors in corresponding
levels of the driver behavior model. These are categorized as motivational driver
behavior models [10]. Furthermore, the modeled misbehavior is validated by recon-
structing real accidents from the AARU accident database in a respective traffic sim-
ulation. We proved the effectiveness of the method by modelling an information
processing failure in the decision-making process of the driver. Thereafter, we vali-
dated the model by successfully reconstructing a real accident, the cause of which
according to the findings of the AARU corresponded to the same class of human failure
behavior [11].
In this contribution, we present a failure behavior model for generating skid-
scenarios on motorways. Skidding situations, if they occur, lead in many cases to
severe injuries or casualties. The number of accidents of this type has decreased sig-
nificantly due to the introduction of Electronic Stability Control (ESC), but such sit-
uations still exist on motorways [13]. Nevertheless, it is necessary to investigate the
behavior of the ADF in such rare events.
We utilized the driver behavior model from the Human Factors Consult (HFC)2.
The model consists of four consecutive layers: perception, situation awareness,
1
Audi Accident Research Unit (AARU): https://www.aaru.de/.
2
Human Factors Consult (HFC): https://human-factors-consult.de/en/.
94 B. Huber et al.
decision-making and tasks-execution [14]. These layers implement the human
decision-making process. Furthermore, our vehicle model is controlled by the HFC
driver behavior model. Additionally, a two-track model is employed to simulate the
vehicle dynamics of our vehicle model [15]. Thus it is possible to generate over- or
understeer based on the control commands of the driver behavior model.
3 Implementation
We utilized a neural network to model the failure behavior to generate skidding, since it
was not possible for us to identify a clear human failure behavior from the AARU
database. The neural network is trained by applying the reinforcement learning algo-
rithm Proximal Policy Optimization [16]. The algorithm is composed of two neural
networks with common parameters. The first network calculates the next state, while
the second network represents the value function.
_
LPPO
t ðhÞ ¼ Et ½LCLIP
t ðhÞ c1 LVF
t ðhÞ þ c2 S½ph st : ð1Þ
Where h denotes the parameters of the neural networks, ph is defined by the policy
_
and Et represents the expected value over multiple calculation steps. c1 and c2 refer to
constants and S describes the entropy in time step st . The deviation of the neural
network of the value function is explained by LVF t ðhÞ. Lt ðhÞ determines the clipped
CLIP
main objective. This parameter defines the maximum change of the neural network
within a calculation step. Here, the modification is generated on the basis of the
introduced policy. In addition to the policy, a corresponding reward (R) is calculated,
which is dependent on the current state of the environment. In the training phase of the
neural network, the policy is optimized to obtain a maximum reward. For a more
detailed explanation of the functionality of the algorithm, see [16]. The training process
of our failure behavior model is illustrated in the following pseudo-code:
while not done do
for t=1,2,…,N do
Execute Policy
Save ,
end for
calculate for
optimize
end while
In our setup, the steering wheel position is defined by the neural network. As input
values, the current distance of the vehicle to the edge of the road, the speed of the
vehicle and the heading angle relative to the road are provided to the neural network.
Subsequently, the corresponding reward is computed based on two conditions. First the
reward is increased when the vehicle is on the road, otherwise it is drastically reduced.
A Validated Failure Behavior Model for Driver Behavior Models 95
Secondly, the current slip angle [17] of the vehicle is computed. A nonzero slip angle
signifies a positive reward. A termination condition of the training process is not
implemented.
On completion of the training, the neural net is integrated into the task-execution
layer of the driver behavior model. Whenever necessary, the neural network may be
activated to produce an increased slip angle according to the current situation of the
vehicle model and its environment.
4 Validation
A reconstructed skidding accident from the AARU database is simulated in a sub-
microscopic traffic simulation with our driver behavior model to validate the failure
behavior model. Hereby, the objective is to substantiate the model’s ability to reflect
human failure behavior in skidding accidents. Therefore the database for the validation
of the failure behavior is explained. Afterwards, the simulation results are compared
with the reconstruction of the real accident.
4.1 Database
The database for the validation consists of the results of the AARU on an analyzed
skidding accident. The reconstruction of the collisions of the skidding accident is
illustrated in Fig. 1. Since the reconstruction is focused on the collisions of the vehi-
cles, the trajectory of the skidding vehicle was extrapolated for this purpose.
Fig. 1. Reconstruction of the collisions of a skidding accident provided by the AARU. The
trajectory of the skidding vehicle was extrapolated by the authors, represented by the blue
vehicles.
In this accident, vehicle A (blue vehicle in Fig. 1) was driving on the right-hand
lane of a two-lane motorway. The driver performed a lane-change maneuver to the left
lane. Vehicle B (vehicle on the left lane in Fig. 1) was approaching from behind. As the
driver of vehicle A recognized vehicle B the driver of vehicle A abruptly steers back to
the right lane. Thereby, the driver lost control of the vehicle. This is followed by
vehicle A and vehicle B colliding. Furthermore, the AARU estimated the velocity of
vehicle A to 110 km/h, while the estimation of vehicle B was 145 km/h.
96 B. Huber et al.
The illustrated states of vehicle A were determined according to the roadway
geometry and the roadway markings. In addition, the measured values were used as
interpolations points for the states of the vehicle. Thus, the trajectory as well as the slip
angle can be used for the validation. The results are visualized in Fig. 2 by the blue
line. For a more accurate representation of the trajectory, the image is inverted.
4.2 Simulations Results
The driver behavior model with integrated failure behavior model is embedded in our
traffic simulation and controls vehicle A. The virtual motorway in the simulation sce-
nario is modeled on the basis of the reconstruction image. Vehicle B is simulated by the
traffic simulation and controlled respective. Additionally, another vehicle is placed on
the right lane in our scenario. It forces vehicle A to perform the lane change maneuver to
the left lane of the motorway. At the same time, the simulated vehicle B is approaching
from behind. As soon as vehicle B is approximately 50 m behind vehicle A, the failure
behavior model is activated. The results are show in Fig. 2 by the orange line.
Fig. 2. The graphs compare the simulations results with the reconstruction. The left graphs
describes the trajectory of vehicle A. The right graph visualizes the respective slip angle.
Referring to the left part of Fig. 2, the steering back to the right lane matches the
data from the reconstruction. However, the reconstructed trajectory reaches is maxi-
mum earlier, which also shows a lower amplitude. This deviation may be caused by
different factors. On the one hand, it is likely that the neural networks react too slowly
to the oversteering vehicle, since it is optimized to maximize the slip angle. As pictured
in the right part of Fig. 2, the slip angle in the simulation achieves a higher maximum
earlier. On the other hand, it is important to note that the vehicle parameters are
approximated and therefore the behavior of vehicle A may be not perfectly modeled.
A Validated Failure Behavior Model for Driver Behavior Models 97
5 Discussion and Future Work
In this contribution we use a reinforcement learning algorithm to models the failure
behavior of drivers in skidding situations. Subsequently, the model was validated on
the basis of a real accident, which was analyzed and reconstructed by the AARU. The
validation shows that the neural network is able to approximate the behavior of the
accident driver. Since we consider it impossible to generalize the driving behavior of
human drivers, the deviation nevertheless indicates human behavior.
In our future work we will utilize the failure behavior models to generate multi-
causal accidents or near crash situations in our traffic simulation for testing ADFs.
References
1. McKinsey & Company: Automotive revolution – perspective towards 2030: How the
convergence of disruptive technology-driven trends could transform the auto industry.
Report, Advanced Industries (2016)
2. Singh, S.: Critical reasons for crashes investigated in the national motor vehicle crash
causation survey. Report, Traffic Safety Facts – Crash (2015)
3. Ardelt, M., Coester, C., Kaempchen, N.: Highly automated driving on freeways in real traffic
using a probabilistic framework. In: 2012 IEEE Intelligent Transportation Systems (ITSC),
pp. 1576–1585 (2012)
4. Bergenhem, C., Johansson, R., Soederberg, A., Nilsson, J., Tryggvesson, J., Toerngrne, S.,
Ursing, S.: How to reach complete safety requirement refinement for autonomous vehicles.
In: CARS 2015 – Critical Automotive Applications (2015)
5. Winner, H., Wachenfeld, W., Junietz, P.: Validation and introduction of automated driving.
In: Winner, H., Prokop, G., Maurer, M. (eds.) Automotive Systems Engineering II, pp. 177–
196. Springer, Cham (2018)
6. Shokry, H., Hinchey, M.: Model-based verification of embedded software (2009)
7. Ulbrich, S., Schuldt, F., Homeier, K., Steinhoff, M., Menzel, T., Krause, J., Maurer, M.:
Testing and validating tactical lane change behavior planning for automated driving. In:
Watzenig, D., Horn, M. (eds.) Automated Driving, pp. 451–471. Springer, Cham (2017)
8. Schuldt, F., Reschka, A., Maurer, M.: A method for an efficient, systematic test case
generation for advanced driver assistance systems in virtual environments. In: Winner, H.,
Prokop, G., Mauerer, M. (eds.) Automotive Systems Engineering II, pp. 147–175. Springer,
Cham (2017)
9. Menzel, T., Bagschick, G., Isensee, L., Schomburg, A., Maurer, M.: From functional to
logical scenarios: detailing a keyword-based scenario description for execution in a
simulation environment. Technical report, IEEE Intelligent Vehicles Symposium (2019)
10. Michon, J.A.: A critical view of driver behavior models: what do we know, what should we
do? In: Evans, L., Schwing, R.C. (eds.) Human Behavior and Traffic Safety, pp. 485–524.
Springer, Boston (1985)
11. Huber, B., Sippl, C., German, R., Djanatliev, A.: A validated failure behavior model for
driver models to test automated driving functions. In: Ahram, T., Taiar, R., Colson, S.,
Choplin, A. (eds.) Human Interaction and Emerging Technologies, IHIET 2019. Advances
in Intelligent Systems and Computing, vol. 1018, Springer, Cham (2020)
98 B. Huber et al.
12. Weber, S., Ernstberger, A., Donner, E., Kiss, M.: Learning from accidents: using technical
and subjective information to identify accident mechanisms and to develop driver assistance
systems. In: Dorn, L., Sullman, M. (eds.) Driver Behaviour and Training, vol. VI, pp. 223–
230. Ashgate, Surrey (2013)
13. Winkle, T.: Safety benefits of automated vehicles: extended findings from accident research
for development, validation and testing. In: Maurer, M., Gerdes, J., Lenz, B., Winner, H.
(eds.) Autonomous Driving. Springer, Heidelberg (2016)
14. Human Factor Consult Driver Modeling. https://human-factors-consult.de/en/services-and-
products/driver-modeling/
15. Schramm, D., Hiller, M., Bardini, R.: Zweispurmodelle. In: Modellbildung und Simulation
der Dynamik von Kraftfahrzeugen. Springer, Heidelberg (2018)
16. Schulmann, J., Wolksi, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy
optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
17. Von Vietinghoff, A.: Nichtlineare Regelung von Kraftfahrzeugen in querdynamischen
Fahrsituationen. Insitut fuer Industrielle Informationstechnik (2008)
Human-Machine Interface Research
of Autonomous Vehicles Based on Cognitive
Work Analysis Framework
Chi Zhang, Goudong Yin(&), and Zheng Wu
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], [email protected], [email protected]
Abstract. This article deals with human-machine (HMI) interface design of
autonomous vehicles. Due to the limitations of the current autonomous driving
technologies, drivers are also required to participate in some driving tasks,
which leads to a lot of challenges in the human-machine interface (HMI) design
of autonomous vehicles, especially when drivers are using in-vehicle informa-
tion systems (IVIS) or mobile devices. This paper presents a methodology
aiming to solve these challenges. The method relies on the Cognitive Work
Analysis framework and mainly using the example of vehicle to driver handover
phase to identify and to organize information that should be displayed to min-
imize distraction and maximize safety benefits. Finally, some practical HMI
design principles of autonomous vehicles are provided.
Keywords: Human factors Interaction design Systems engineering
1 Introduction
In recent years, many functions of vehicles have become automated, including some
main driving tasks, autonomous driving technology is more and more powerful, the
latest advanced automated driving system can already be in full control of the car under
the condition of limited road, and drivers can delegate control to automated systems
when they wish. The current SAE (the Society of Automotive Engineers) taxonomy
provide a clear classification which ranges from level 0 to level 5, literally from no
driving automation to full driving automation [1].
This article deals with human-machine (HMI) interface design of autonomous
vehicles. Under the SAE lv3, the system is responsible for monitoring the environment,
drivers are expected to respond and control the car if there is a request to intervene.
Wiener and Curry indicates that increased automation can lead to errors [2]. This is
probably caused by the lack of situational awareness, drivers think the system can
control the whole driving process so they don’t need to pay attention. In the first part,
we present the research context and the methodology used. Then we mainly examines a
specific handover situations called non-scheduled system initiated handover by using
Cognitive Work Analysis framework. Finally, we provided some design principles to
guide the interface design of autonomous vehicles.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 99–103, 2020.
https://doi.org/10.1007/978-3-030-39512-4_16
100 C. Zhang et al.
2 Context and Methodology
Many researches have conducted studies in the handover situations of autonomous
vehicle. In SAE level3, driver is necessary, but is not required to monitor the envi-
ronment. Therefore, the driver can perform some other activities but also needs to
understand the inside and outside of the car in order to drive manually when needed. It
is necessary to convey this information through the appropriate human-machine
interface, so that the driver can know these situations, namely, to establish accurate
situational awareness. If the driver cannot get enough information through the inter-
active interface, it will not only cause the accident due to the lack of situational
awareness, but also make the driver express his fear of driverless and resist driverless
technology [3].
The handover process has three specific phases:
1. Automated driving phase, in this phase the autonomous driving system controls the
vehicle.
2. Handover phase from the system to the driver. This process can be divided into
scheduled handover and non - scheduled handover [4].
3. The transition from manual driving to autonomous driving system, human drivers
believe that their driving task has been completed and are ready to hand over control
to system again.
This paper specifically discusses how to maintain situational awareness in the non-
scheduled system initiated handover, because this condition is likely to lead to acci-
dents. To support this, we refer to Endsley’s model, which divides situational aware-
ness into three different parts: perceive elements within an environment; Understanding
the meanings of task goals and prediction of specific future actions. Then we use CWA
methodology to extract the information required by drivers. Based on this information,
some design principles are established to guide the interface design of autonomous
vehicles.
3 Cognitive Work Analysis
Cognitive Work Analysis proposed by Rasmussen [5]. This research method comes
from Rasmussen’s analysis of industrial accident. Vicente developed five stages of
cognitive work analysis [6]: Work Domain Analysis, Control Task Analysis, Strategies
Analysis, Social Organization and Cooperation Analysis, and Worker Competencies
Analysis. In the five main parts of CWA, this paper mainly uses two methods: Work
Domain Analysis (WDA) and Control Task Analysis (ConTA).
3.1 Work Domain Analysis
The implementation hierarchy supports five levels of abstraction to describe the work
domain: functional purposes (the goal of the work domain), abstract functions, general
functions, physical processes and activities, and physical objects. The five levels of
implementation hierarchy are connected step by step to form the system connection
Human-Machine Interface Research of Autonomous Vehicles 101
architecture of means-ends. From top to bottom, it can express the realization of means
of system functions, while from bottom to top, it can explain the purpose of the
existence of a specific facility and process in the system.
Functional purposes
Abstract function
Ends
Means
General function
Physical function
Physical objects
Fig. 1. Means-ends
Work domain analysis of Lv3 autopilot has been done by Debernard [7], so I will not
finish all the stages of WDA here. Based on the cognitive work of level3 autonomous
vehicles conducted by s. Debernard, we can know that main purposes of a level 3
autonomous vehicle are to realize safe and efficient driving process. In addition, I would
like to make some specific supplementary information refer to handover process. In the
abstract function layer: in order to achieve controllability, the condition inside and
outside the vehicle should be clearly displayed and make sure the driver understands this
information. In the physical function layer: the driver’s ability to get information should
be taken into account. In the physical object layer: in addition to the Vehicle’s senses,
the driver’s sensory organs should also be consideration.
Based on the informational basis provided by abstraction hierarchy method, control
task analysis could be conducted.
3.2 Control Task Analysis
Control Task Analysis (ConTA) is related to activities needed to meet specific system
goals. In the handover scenario, as a result of the existence of people, there will be a lot
of observation, decision-making and analysis process. In CWA framework this process
is called ConTA. ConTA can take the form of decision ladder [6] to represent the
control task and transform the specific activities in WDA analysis into the process of
information processing and decision making. In this form, we use black and white box
instead traditional represents to maximize the use of the page size, the black box
represents information processing and the white ones represents the cognitive state
generated by these activities.
Figure 1 shows that the different stages of the handover situation performed by the
system and identifies what needs to be done in each stage according to functional goals.
In ConTA, observe is the phase in which the driver receives information. After
receiving the observed information, the driver will judge the current state, which is the
Diagnose state in the table. Skilled driver will perform tasks directly after understanding
the status, but novice driver will continue to process some information (Fig. 2).
102 C. Zhang et al.
GOALS: Transition from manual to automated driving
Evaluate performance
OPTIONS: Do I need to take over the
car quickly? Do I need to change GOAL: Manually control the
lanes after taking over control? Do I vehicle out of road conditions
need to change speed after taking that the system cannot handle
over control? Do I need to brake after and return control to the tech-
taking over control? Can I continue nical agent.
driving after taking over control.
Predict consequence
STATE: What is preventing the auto-
pilot from continuing to drive? On
TARGET: Fast and secure
what roads? What are the current road
handover process.
conditions? Is there much traffic
around? Do I bump into obstacles?
Diagnose state Definition of task
INFORMATION: What's the current
speed? What's the speed of the cars TASK: The transition of con-
around? Where is the current vehicle? trol from the technical agent to
What is the road condition around the the driver's manual
vehicles? Are there any cars moving? Is control.
there an obstacle near me? What are the
surrounding obstacles? What's the current
weather like? Planning of procedure
Observe PROCEDURE: Maintain situa-
tional awareness. Gradually
ALERT: what's wrong with the system? take over control of the car.
Why the sudden transfer of control? Is the Slow down, change lane or pull
current situation dangerous? Should I take over. Hand over control to
over the car immediately? Is it dangerous system when the operation is
over
not to take over the car?
Execute
Activition
Fig. 2. Handover decision ladder
Human-Machine Interface Research of Autonomous Vehicles 103
4 Conclusion
Through the cognitive work analysis methodology, we have obtained a lot of infor-
mation needed by the driver. Based on four main stages of decision ladder, we can
summarize some design principles and requirements.
Information acquisition:
Principle 1: in the alert phase, there should be a clear reminder to let the driver
know that the system is no longer able to continue the driving task.
Principle 2: sufficient time should be allowed for the driver to observe the envi-
ronment during the switching process. If this cannot be done, the system should
gradually transfer control and take deceleration actions to ensure safety.
Analysis information:
Principle 3: when the driver is observing the environment, the system can provide
necessary assistance, enabling the driver to find out the cause of the failure of the
automatic driving system faster so as to quickly maintain the situational aware-ness of
the environment.
Select tasks:
Principle 4: the system should alert the driver to possible dangerous conditions to
help the driver make correct prediction.
Implementation of tasks:
Principle 5: the system provides certain guidelines when a driver performs a driving
task.
References
1. Automated Vehicles for Safety – NHTSA (2018). https://www.nhtsa.gov/technology-
innovation/automated-vehicles-safety
2. Wiener, E.L., Curry, R.E.: Flight-deck automation: promises and problems. Ergonomics 23
(10), 995–1011 (1980)
3. Bennett, R., Vijaygopal, R., Kottasz, R.: Willingness of people with mental health disabilities
to travel in driverless vehicles. J. Transp. Health 12, 1–12 (2019)
4. McCall, R., McGee, F., Mirnig, A., et al.: A taxonomy of autonomous vehicle handover
situations. Transp. Res. Part A Policy Pract. 124, 507–522 (2018)
5. Rasmussen, J.: Information Processing and Human-Machine Interaction: An Approach to
Cognitive Engineering. North-Holland, New York (1986)
6. Vicente, K.J.: Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-
Based Work. CRC Press, Boca Raton (1999)
7. Debernard, S., Chauvin, C.: Designing Human-Machine Interface for Autonomous Vehicles.
IFAC-PapersOnLine (2016)
Mercury: A Vision-Based Framework
for Driver Monitoring
Guido Borghi2(&), Stefano Pini1, Roberto Vezzani1,
and Rita Cucchiara1
1
Dipartimento di Ingegneria “Enzo Ferrari”,
Università degli Studi di Modena e Reggio Emilia, Modena, Italy
{s.pini,roberto.vezzani,rita.cucchiara}@unimore.it
2
Centro di Ricerca Interdipartimentale Softech-ICT, Università degli Studi di
Modena e Reggio Emilia, 41125 Modena, Italy
[email protected]
Abstract. In this paper, we propose a complete framework, namely Mercury,
that combines Computer Vision and Deep Learning algorithms to continuously
monitor the driver during the driving activity. The proposed solution complies to
the requirements imposed by the challenging automotive context: the light
invariance, in order to have a system able to work regardless of the time of day
and the weather conditions. Therefore, infrared-based images, i.e. depth maps
(in which each pixel corresponds to the distance between the sensor and that
point in the scene), have been exploited in conjunction with traditional intensity
images. Second, the non-invasivity of the system is required, since driver’s
movements must not be impeded during the driving activity: in this context, the
use of cameras and vision-based algorithms is one of the best solutions. Finally,
real-time performance is needed since a monitoring system must immediately
react as soon as a situation of potential danger is detected.
Keywords: Driver Monitoring Human-Car Interaction Computer Vision
Deep Learning Convolutional neural networks Depth maps
1 Introduction
The loss of vehicle control is a common problem, due to driving distractions, also
linked to driver’s stress, fatigue and poor psycho-physical conditions. Indeed, humans
are easily distractible, struggling to keep a constant concentration level and showing
signs of drowsiness just after a few hours of driving [1].
Furthermore, the future arrival of (semi-)autonomous driving cars and the necessary
transition period, characterized by the coexistence of traditional and autonomous
vehicles, is going to increase the already-high interest about driver attention monitoring
systems, since for legal, moral and ethical implications driver must be ready to take the
control of the (semi-)autonomous car [2].
Therefore, in this paper, a driver monitoring system, here referred as Mercury,
based on Computer Vision and Deep Learning algorithms, is proposed.
From a technical point of view, Mercury framework is composed of different
stages.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 104–110, 2020.
https://doi.org/10.1007/978-3-030-39512-4_17
Mercury: A Vision-Based Framework for Driver Monitoring 105
Given an input depth map, in the first step the head center is automatically detected,
according to which a bounding box containing the driver’s head and a minor part of the
background is extracted.
Then, this crop is fed into a system that estimates the 3D pose of the head, in terms
of yaw, pitch and roll angles. On the same head crop, a facial landmark localization
algorithm is applied, in order to identify the salient areas that belong to the face and are
visible in both the depth and the intensity frame.
We believe that head pose information could be useful in terms of driver moni-
toring, to understand where the driver is looking (e.g. inside or outside the car cabin)
and for the infotainment systems, in order to make new systems for the Human-Car
Interaction more intuitive and user-friendly and to improve the velocity of operations
conducted by the driver inside the cockpit [3].
Finally, driver’s eyes, detected through the facial landmark system mentioned
above, are analyzed to estimate the level of driver drowsiness. Specifically, PERCLOS,
a drowsiness detection measure, is exploited.
2 Mercury
The whole architecture of the system is represented in Fig. 1. The system has been set
up in a modular way, since it is able to work only with RGB data, depth data or both.
The functions available for each type of data are different and are described in the
following paragraphs.
Fig. 1. Overall architecture of the proposed framework. Blue blocks represent algorithms
applied on RGB or gray-level frames, while in the green ones are reported the solutions for the
depth maps. Finally, orange boxes represent the computed measures for the Driver Monitoring
task.
2.1 Acquisition Device
The Microsoft Kinect One sensor has been exploited as acquisition device. Although
more recent depth sensors are available on the market, with more suitable factor forms
for the automotive field, this device has been preferred due to the availability and
relative simplicity of the proprietary SDK that is compatible with the Python
106 G. Borghi et al.
programming language1, as well as the simultaneous presence of an excellent color
camera and a depth sensor with a good spatial resolution and a low noise level.
Moreover, the minimum range for data acquisition allows its use on the automotive
context, since in our case the device is placed in front of the driver, near the car
dashboard.
In the following, technical specifications of the device are reported:
• Full HD RGB camera: the spatial resolution is 1920 1080. The device is able to
acquire up to 30 frame per second;
• Depth sensor: the spatial resolution is 512 424. This sensor provides depth
information as a two-dimensional array of pixels, namely depth map, similar to a
gray-level image. Since each pixel represents the distance in millimeters from the
camera, depth images are coded in 16 bits;
• Infrared emitter: this device is based on the Time-of-Flight (ToF) technology and its
range starts from 0.5 up to 7 m;
• Microphone array: in this device is present an array of microphones. Audio data are
sampled at 16 kHz and coded with 16 bits. It is used to pick up the voice com-
mands, for the calibration of the peripheral and the suppression of white noise.
Microphones are not used in this project.
2.2 Head Extraction
The first step of the framework is the automatic head extraction from the acquired
frames. It is assumed that within the acquired scene there is no more than one person,
i.e. the driver, sitting in front of the acquisition device, about 1 m away.
For the part of head detection on RGB frames, the well-known object detection
framework of Viola & Jones [4] has been exploited: despite the presence of more recent
deep learning-based algorithms, is still a valid choice for its effectiveness and good
speed performance. Moreover, in this regard, we note that many car companies still use
derivative versions of the algorithm in question.
Head detection on depth maps is conducted through the Fully Convolutional
Network proposed in [5]: the limited depth of the network preserves and balances
detection accuracy and speed performance.
The output of RGB and depth head detection modules consists of the head crop,
which is a bounding box containing the driver’s face with minimal background
portions.
2.3 Head Pose Estimation
The obtained head crop is used to produce three different inputs for the Head Pose
Estimation module. The first input is represented by the raw depth map, while the
second is a Motion Image, obtained running the Optical Flow algorithm (Farneback
implementation) on a sequence of depth images. The third input is generated by a
network called Face-from-Depth [6], that is able to reconstructs gray-level face images
1
https://github.com/Kinect/PyKinect2.
Mercury: A Vision-Based Framework for Driver Monitoring 107
starting from the related depth images. All these three inputs are then processed by a
regressive Convolutional Neural Network (CNN) [7] that finally outputs the value of
the yaw, pitch and roll 3D angles expressed as continuous values.
2.4 Facial Landmark Estimation
The goal of this module is a reliable estimation of the facial landmark coordinates, i.e.
salient regions of the face, like eyes, eyebrows, mouth, nose and jawline. Due to the
limited spatial resolution of available depth images, we focus on a selection of five
facial landmarks: eye pupils, mouth corners and the nose tip. Accordingly, the system
outputs 10 image coordinates, i.e., the x and y values for each facial landmark.
The core of the method is a CNN [8] that works in regression and receives a stream
of depth images as input. The ground truth annotation of the landmark positions is
required during the network training step and is used as a comparison during the test.
The developed system has real-time performance and it is more reliable than state-
of- art competitors in presence of poor illumination and light changes, thanks to the use
of depth images as input. The extraction of the facial landmarks allows obtaining the
bounding box of the driver’s right and left eye: these images are then analyzed by a
CNN to determine if the eye is open or closed.
2.5 Driver Monitoring
The analysis of the attention and the level of fatigue of the driver is done through the
indicator called PERCLOS (PERcentage of eyelid CLOSure), introduced in 1994 in [9].
This measure aims to express the percentage of time in a minute in which the eye
remains closed from 80% to 100%.
In general, the computation of the PERCLOS measurement yields a numerical
value that can be analyzed to determine the driver’s fatigue level. Blinking beats,
identifiable as almost instantaneous closures of the eye, are excluded in its computa-
tion, since only the prolonged and slow closures, usually called droops, are maintained.
The level of driver attention is then classified through three thresholds:
• Alert (PERCLOS < 0:3): good level of attention, optimal driver conditions;
• Drowsy (0.3 < PERCLOS < 0.7): first signs of carelessness and fatigue, the driver
is not in optimal conditions and that introduces some risks;
• Unfocused (PERCLOS > 0.7): complete lack of attention of the driver who is in
poor physical condition or is sleeping;
3 Implementation
In the following paragraphs, details about the hardware and the software implemen-
tation of Mercury are reported and discussed.
108 G. Borghi et al.
3.1 Hardware Implementation
The Mercury framework has been implemented and tested on a computer equipped
with an Intel Core i7-7700 K processor, 32 GB of RAM and a Nvidia 1080Ti GPU.
The various components belonging to the deep learning tools have been imple-
mented with the Keras2 framework with Tensorflow3 as backend.
Finally, a graphic interface has been created through the Qt Libraries4 and it is
detailed in the following paragraph.
Being aware that the aforementioned hardware equipment is not suitable for the
automotive context, due to reasons related to the costs and power consumption, we
have started to implement these systems on embedded boards. The Nvidia TX2 board
has been chosen, since it is equipped with an embedded GPU suitable for deep
learning-based algorithms.
3.2 Graphical User Interface
The Graphical User Interface (GUI), shown in Fig. 2, has been created using the Qt
libraries 4 and Python programming language. It has been divided into four vertical
sections: 1. Input Visualization, 2. Head Detection, 3. Head Pose Estimation and Facial
Landmark Detection and 4. Driver Attention Analysis.
From the left, the first part contains the RGB and depth frames captured by the
Kinect One device, for visualization and debugging purposes. The side column is
dedicated to display the output of the head detection modules. In particular, the
localized bounding boxes are shown in green on the frames.
The central part is dedicated to the driver’s face analysis, and it is divided into
several horizontal sections. Starting from the top, the head pose is displayed either
through a 3D cube or with bars, centered on the zero angle. The yaw angle is shown in
blue, the roll angle in green and the pitch in red.
Further down, the facial landmarks found on the RGB frame are shown through red
dots. Thanks to these facial landmarks, the bounding boxes containing the right eye, the
left eye, all displayed at the bottom, are extracted to compute the PERCLOS measure.
At the extreme right, the information regarding the analysis of driver attention are
reported. The PERCLOS measure is indicated by its percentage value and, according to
the established thresholds, the term alert, drowsy or unfocused is highlighted. The head
pose is instead indicated with the representation of the possible positions of the gaze on
a given object.
An image of a generic car interior has been then used to graphically convey the
concept of Human-Car Interaction, highlighting with the red color the object inside the
passenger compartment. Some common areas have been chosen as target for the
interaction: the right, central and left rear mirrors, the area of the steering wheel, the
gearbox and the part of the infotainment.
2
https://keras.io/.
3
https://www.tensorflow.org/.
4
https://www.qt.io/.
Mercury: A Vision-Based Framework for Driver Monitoring 109
Finally, in the bottom part of the GUI, the buttons that allow the user to select the stream
of data on which the framework must work, i.e. RGB, depth or both streams, are placed.
Moreover, the measure of the frame per second for each part of the system is shown.
Fig. 2. The graphical user interface developed for the Mercury framework.
4 Conclusions
In this paper, Mercury, a framework for automatic Driver Monitoring, has been
introduced. Input data is represented by both RGB images and depth maps, in order to
improve the light invariance of the system. Future work will include the implemen-
tation of the framework in real embedded boards, suitable for the automotive context.
References
1. Young, K., Regan, M., Hammer, M.: Driver distraction: a review of the literature. In: Faulks,
I.J., Regan, M., Stevenson, M., Brown, J., Porter, A., Irwin, J.D. (eds.) Distracted Driving,
pp. 379–405. Australasian College of Road Safety, Sydney (2007)
2. Venturelli, M., Borghi, G., Vezzani, R., Cucchiara, R.: Deep head pose estimation from depth
data for in-car automotive applications. In: International Workshop on Understanding Human
Activities through 3D Sensors, pp. 74–85 (2016)
3. Nawaz, T., Mian, M.S., Habib, H.A.: Infotainment devices control by eye gaze and gesture
recognition fusion. IEEE Trans. Consum. Electron. 54(2), 277–282 (2008)
4. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154
(2004)
5. Ballotta, D., Borghi, G., Vezzani, R., Cucchiara, R.: Fully convolutional network for head
detection with depth images. In: 24th International Conference on Pattern Recognition
(ICPR), pp. 752–757 (2018)
110 G. Borghi et al.
6. Borghi, G., Fabbri, M., Vezzani, R., Cucchiara, R.: Face-from-depth for head pose estimation
on depth images. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
7. Borghi, G., Venturelli, M., Vezzani, R., Cucchiara, R.: POSEidon: face-from-depth for driver
pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 4661–4670 (2017)
8. Frigieri, E., Borghi, G., Vezzani, R., Cucchiara, R.: Fast and accurate facial landmark
localization in depth images for in-car applications. In: International Conference on Image
Analysis and Processing, pp. 539–549 (2017)
9. Wierwille, W.W., Wreggit, S.S., Kirn, C.L., Ellsworth, L.A., Fairbanks, R.J.: Research on
vehicle-based driver status/performance monitoring; development, validation, and refinement
of algorithms for detection of driver drowsiness (1994)
Investigating the Impact of Time-Lagged
End-to-End Control in Autonomous Driving
Haruna Asai(&), Yoshihiro Hashimoto, and Giuseppe Lisi
Nagoya Institute of Technology, Nagoya, Japan
[email protected], [email protected]
Abstract. End-to-end training, the strategy by which deep neural networks
learn to map raw pixels from front-facing cameras directly to steering com-
mands have recently gained attention in the autonomous driving field. Here, we
investigate the possibility of extending this approach with a time-lagged pro-
cedure, by training the system to map raw pixels at time T to steering commands
at time T+Lag. We are interested in evaluating such an approach for two main
applications: (1) time-lagged end-to-end control towards an artificial driving
instructor (ADI) that recommends future control actions to novice human dri-
vers; (2) time-domain data augmentation to improve the performance of stan-
dard non-lagged end-to-end control. Our results show that time-lagged end-to-
end training is not appropriate for time-lagged control, but using it for data
augmentation leads to a smoother output in standard non-lagged end-to-end
control. This suggests that time-lagged training improves the anticipatory ability
of end-to-end control and augmentation reduces overfitting.
Keywords: Artificial driving instructor ADI Autonomous driving Data
augmentation Time-lag End-to-end control Deep learning
1 Introduction
Deep neural networks that map raw pixels from front-facing cameras directly to
steering commands have recently gained attention since the work by Bojarski et al. [1].
Such end-to-end systems promise to be more compact and performant compared with
long pipelines of handcrafted feature extractors (e.g. lane, obstacle detectors). Here, we
investigate the possibility of extending this approach with time-lagged end-to-end
control, by training the system to map raw pixels at time T to steering commands at
time T+Lag. We are interested in evaluating such approach for two main applications:
(1) prototyping an artificial driving instructor (ADI) that recommends future control
actions to novice human drivers, given the current camera pixels; (2) time-domain data
augmentation to improve the performance of standard end-to-end control.
Artificial Driving Instructor (ADI) – Experiment 1. In this paper we propose the
ADI, a type of advanced driver-assistance system (ADAS) [2] that teaches novice
human drivers to drive, by recommending control actions (Fig. 1). Implementing ADI
in a controlled environment, such as a driving test tracks, has numerous advantages:
(1) novice drivers learn from the ADI and vice versa, in a co-adaptive manner, (2) test
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 111–117, 2020.
https://doi.org/10.1007/978-3-030-39512-4_18
112 H. Asai et al.
autonomous driving algorithms, while collecting data, (3) provide a service such as an
automated driving school, (4) minimize the risk of accidents. Given its characteristics, a
successful implementation of ADI is a necessary condition for the mass adoption of
autonomous driving. In this framework, end-to-end training based on deep learning
allows to easily learn a controller based on a new test track by simply collecting data
with a proficient human driver, eliminating the need of re-programming complex track-
specific algorithms. Subsequently, the neural network can adapt itself based on the
incoming successful data from the novice driver. Here, we simulate the system sug-
gestion by storing the output produced at time T and using it as an actual control
command at time T+Lag.
Recommends actions
Artificial Novice
Machine
Driving Human
Learning
Instructor Driver ”Please prepare to steer right
in 3 seconds”
Provides feedback
for improvement
a b
Fig. 1. Artificial Driving Instructor (ADI). (a) co-adaptation in ADI. (b) in a controlled
environment, such as a driving test tracks, the ADI recommends future control actions to novice
human drivers.
Time-Domain Data Augmentation - Experiment 2. Data augmentation is the
strategy that allows to increase the diversity of data available in the training set via
domain-specific transformations. For image data, commonly used transformations
include random cropping, random perturbation of brightness, saturation, hue and
contrast [3]. In this work, not only we perform data augmentation on the camera pixels,
but also in the time domain: we train the system to map the pixels at time T to steering
commands chosen randomly between time T and T+Lag.
2 Methods
In the next two sections we describe in detail the two experiments we conducted,
targeting the time-lagged end-to-end control for ADI and the time-domain data aug-
mentation, respectively. Both experiments are implemented in Python, Keras, and a
simulator provided by Udacity. Deep neural networks (DNN) with the same archi-
tecture as in Bojarski et al. [1] are employed. Standard data augmentation commonly
employed in the Udacity context is performed in both experiments, such as randomly
picking from the frontal, left and right camera and correcting the angle accordingly [1],
Investigating the Impact of Time-Lagged End-to-End Control in Autonomous Driving 113
brightness changes, small normal noise on the steering value, discarding a portion of
the samples associated with steering angle 0 to avoid unbalance in the data (steering
angle close to 0 is the majority).
In both experiments, training and testing are performed on the same track (i.e. track
1 of the Udacity software). Training was done on data from a 8 min run on the track,
for a total of about 4 laps (batch size: 512, steps per epoch: 125, epochs: 50). Per-
formance is assessed by making the DNN drive for maximum 120 s around the track,
and computing the steering angle absolute deviation and the time to crash. The former
represents the maximum angle of the steering wheel at each turn, and smaller values
represent a less shaky and better drive. It is defined as d ð pÞ ¼ jhð pÞj, where h repre-
sents the steering angle, and p is a local positive or negative peak of h, found using
the argrelextrema function from SciPy. For each run all the local peaks are found and
then all the d ð pÞ are computed; then the distributions of d ð pÞ are compared across
DNNs, using one-way ANOVA, Tuckey Multiple Comparison Test and Cohen’d effect
size. With respect to the time-to-crash, a value of 120 s indicates that the car was able to
complete a run without crashing.
Steering(T+Lag)
Steering(T+0.5)
Steering(T)
At time T
Fig. 2. Time-lagged end-to-end training strategy. For each time-lag a deep neural network
(DNN) is trained to predict the steering control at time T+Lag given an image at time T. With the
data augmentation strategy, a neural network is trained by randomly selecting a different time-lag
for each sample.
2.1 Time-Lagged End-to-End Training: Experiment 1
In this experiment, a DNN is trained to map raw pixels at time T to steering commands
at time T+Lag (Fig. 2). In total we trained 4 DNNs, each with a different lag time: 0.5,
1, 3, 5, 10 s. Moreover, we trained a DNN with no lag as a baseline model.
Experiment 1-1. The trained networks are evaluated by storing the output produced at
time T and using it as an actual control command at time T+Lag. This is represented as
Experiment 1-1 in Fig. 3.
114 H. Asai et al.
Experiment 1-2. Moreover, we tested the same networks by applying the output
immediately at time T (i.e. non-lagged control). This is labeled as Experiment 1-2 in
Fig. 3. For each lag level, we repeated this experiment 10 times.
2.2 Time-Domain Data Augmentation: Experiment 2
A neural network is trained by performing data augmentation using different lag times
at random. Specifically, based on the observations of Experiment 1, we trained two
different neural networks with lag augmentation range between 0–3 s and 0–10 s,
respectively. The trained neural network is evaluated in the same manner as in
Experiment 1-2, by using the raw pixels at time T to produce the steering command and
use it immediately, without lag. This is labeled as Experiment 2 in Fig. 3. For each lag
level, we repeated this experiment 10 times.
At time T
Augmentation DNN Lag DNN
(0-3s, 0-10s) (0s, 0.5s, 1s, 3s, 5s, 10s)
Steering at time T + Lag Steering at time T + Lag
Experiment 2 Experiment Experiment
1-2 1-1
time (s)
T T+Lag
Fig. 3. Evaluation strategies in the two experiments
3 Results
3.1 Time-Lagged End-to-End Training: Results of Experiment 1
In Experiment 1-1, the time-lagged end-to-end control is clearly challenging at every
lag level, and the car crashes out of track after a few seconds of driving. The time-to-
crash (TTC) of each lag level in Experiment 1-1 is represented in Table 1.
Table 1. Experiment 1-1. Time-to-crash (TTC) of time-lagged control (max. 120 s).
Time lag 0 0.5 1 3 5 10
TTC (s) 120 32 17 17 17 54
Investigating the Impact of Time-Lagged End-to-End Control in Autonomous Driving 115
However, to our surprise, in Experiment 1-2, if the control command of the net-
works trained with lag is used immediately, instead of storing the output and waiting
for the respective lag, the car is able to drive an entire lap without crashing, up to 3 s of
lag. The time-to-crash (TTC) of each lag level in Experiment 1-2 is represented in
Table 2.
Table 2. Experiment 1-2. Time-to-crash (TTC) of non-lagged control (max. 120 s). Mean and
standard deviation across 10 trials.
Time lag 0 0.5 1 3 5 10
Crash time (s) 120 0 120 0 84.60 16.24 115.17 15.25 57.10 7.37 55.91 9.42
3.2 Time-Domain Data Augmentation: Results of Experiment 2
In Table 3 we observe that data augmentation between 0–3 s does not crash in all the
10 trials, while 0–10 s crashes two times.
Table 3. Time-to-crash of non-lagged control using data augmentation (max. 120 s). Time-lag 0
is included for comparison
Time lag 0 0–3 0–10
Crash time (s) 120 120 110.46 20.97
3.3 Comparing Steering Angle Absolute Deviation
We analyzed the steering angle absolute deviation only for the runs where there was no
crash. Therefore, the comparison includes only a subset of models (Fig. 4). There are
statistically significant differences between group means as determined by one-way
ANOVA (F(4,5554) = 232.1, p = 1:41 10184 ). The results of multiple comparisons
and effect sizes are reported in Table 4 and Fig. 4. Notably, Lag: 0.5 and Aug: 0–3
have a lower deviation than Lag: 0, with differences of 2:15 and 1:56 , respectively. In
both cases, effect sizes close to a medium level (i.e. 0.5 [4]). It should be considered the
deviations are computed in absolute terms, therefore the reported differences impact
both the left and right steering, effectively doubling their effect (e.g. 2:15 ! 4:30 ).
116 H. Asai et al.
Fig. 4. Boxplot representing the steering angle absolute deviation of each model.
4 Discussion
Results of Experiment 1-1 indicate that the time-lagged end-to-end control makes the
car crash almost immediately. Indeed, the car behaves like an unstable oscillator and
after an initial mistake cannot recover due to the lag. In general, a DNN trained with an
end-to-end strategy learns to follow the borders of the track, making it reactive, and not
proactive. This suggests that memory-based control by recurrent neural networks may
be more appropriate. In general, we observed that end-to-end training produces strong
oscillations, that feel unnatural. Therefore, for ADI combining deep learning with
control theory (e.g. model predictive contouring control, MPCC) may be the way to
pursue [5].
Table 4. Multiple comparisons of steering angle deviation across models that do not crash
Group 1 Group 2 Mean diff p-value Cohen’s d
Aug: 0–10 Aug: 0–3 −0.96 0.0 −0.27
Aug: 0–10 Lag: 0 0.59 0.0002 0.16
Aug: 0–10 Lag: 0.5 −1.56 0.0 −0.49
Aug: 0–10 Lag: 3 2.53 0.0 0.60
Aug: 0–3 Lag: 0 1.56 0.0 0.45
Aug: 0–3 Lag: 0.5 −0.59 0.0 −0.19
Aug: 0–3 Lag: 3 3.50 0.0 0.87
Lag: 0 Lag: 0.5 −2.15 0.0 −0.68
Lag: 0 Lag: 3 1.94 0.0 0.48
Lag: 0.5 Lag: 3 4.09 0.0 1.09
Experiment 1-2 and Experiment 2 indicate that time-lagged training improves non-
lagged control. This may be due to the ability of anticipating an action, and achieving a
better response time. Specifically, compared to Lag: 0, Lag: 0.5 and Aug: 0–3 achieve a
Investigating the Impact of Time-Lagged End-to-End Control in Autonomous Driving 117
smoother drive. The time-domain data augmentation strategy (Aug: 0–3), may be more
general, since it includes several time-lags. Indeed, randomized time-lagged training
reduces the effect of overfitting and increases generalizability, producing a smoother
output. Future studies may address this point by testing this method on different tracks
and road conditions using an advanced simulator such as CARLA [6].
References
1. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D.,
Monfort, M., Muller, U., Zhang, J., Zhang, X.: End to end learning for self-driving cars. arXiv
preprint arXiv:1604.07316 (2016)
2. Geronimo, D., Lopez, A.M., Sappa, A.D., Graf, T.: Survey of pedestrian detection for
advanced driver assistance systems. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1239–
1258 (2009)
3. Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning
requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
4. Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Routledge, London (1988).
ISBN 978-1-134-74270-7
5. Kabzan, J., Valls, M.D.L.I., Reijgwart, V., Hendrikx, H.F.C., Ehmke, C., Prajapat, M.,
Bühler, A., Gosala, N., Gupta, M., Sivanesan, R., Dhall, A.: AMZ Driverless: The Full
Autonomous Racing System. arXiv preprint arXiv:1905.05150 (2019)
6. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: An open urban
driving simulator. arXiv preprint arXiv:1711.03938 (2017)
The Car as a Transformer
Jeremy Aston(&) and Rui Pedro Freire
ESAD, College of Art & Design; ESAD Idea (Investigation in Design and Art),
Av. Calouste Gulbenkian 1011, Sra. da Hora, 4460-268 Porto, Portugal
{jeremyaston,ruipedro}@esad.pt
Abstract. Vehicle design is at the forefront of innovation as we enter the next
technological era (industry 4.0). This transition from traditional to alternative
vehicles concerns not only the engine configuration, but the whole car as well.
Electrical optimisation, autonomous driving and ‘drive-by-wire’, are the three
main trends that are currently fostering within the automotive industry [1].
These changes provide designers with new opportunities to reimagine cars, as
well as their own contributions for design [2]. ESAD College provided creative
activities to stimulate the students through the design process by creating per-
sonas, scenarios and goals [5]. As a result, five different vehicle concepts were
proposed, addressing ‘modularity in use’ (MIU).
Keywords: Modularity in use Vehicle design Personas in design thinking
Alternative vehicles Autonomous driving Intelligent mobility Industry 4.0
1 Introduction
The article written by Martin Luccarelli ‘Technological challenges and the new design
opportunities’, published in Auto & Design magazine [2] was used to introduce the
project theme ‘The car as a transformer’ and acted as an inspirational tool for this
academic project.
Article. Car manufacturers are currently investing billions in energy storage systems to
retain technical advantages over their competitors. However, there is still some
uncertainty regarding the future orientation of sustainable passenger mobility [3]. In
this regard, modularity is an appealing method to cope with new vehicle concepts, and
the need for producing them using the current production systems. This methodology
can be applied to the product ‘modularity in design’ (MID), to its production process
‘modularity in production’ (MIP), and to its use ‘modularity in use’ (MIU).
While examples of MID (e.g. the Drive module and the Life module of BMW i3
and i8 models) and MIP (e.g. the MQB Platform of Volkswagen allowing the pro-
duction of 27 models for three brands) are available in the present car market, recent
examples of MIU are missing in current automotive design. ‘Modularity in use’ may
offer designers new opportunities to reimagine vehicles as well as their own contri-
bution on car design, by shifting their focus from car styling to product usability [2].
Apart from showing the technological challenges in car design, this article highlights
the importance of a multiple-disciplinary approach to design, i.e. multi-disciplinary, inter-
disciplinary and trans-disciplinary [4]. The objective was to demonstrate combining
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 118–123, 2020.
https://doi.org/10.1007/978-3-030-39512-4_19
The Car as a Transformer 119
disciplines (i.e. design and engineering) and collaborating to improve the overall result,
especially in the design of complex products like the car (see article conference video:
vimeo.com/350296012).
2 Method
ESAD students from various design backgrounds, were invited to participate. The
project lasted 8 weeks and was divided into 3 assignments: briefing; workshop;
visualisation (see video links).
The briefing was formed from the article, technical issues considered fundamental
for the project’s direction: ‘skateboard’ powertrain platform; high-customisation; low
centre of gravity; good vehicle package; MIU.
Students worked in teams to discuss ideas and concepts, and personas were created
to help understand the users’ needs, experiences, behaviours and goals. Developing a
different persona for each team made the design task less complex, guided their
decision making and orientated a good user experience for the targeted user group [5]
(see workshop video: vimeo.com/350451944).
3 Results
From the workshop, 5 vehicle concepts emerged, displaying different interpretations of
the project theme and opinions about alternative mobility for future lifestyles. The
concepts revolved around multi-tasking, sharing, connectivity and autonomy.
3.1 Concept 01, iBudd
A subcompact car for surfers (see pitch video: vimeo.com/350452664).
Technical Proposal. A mono-volume vehicle with high driving position and large
interior space. Voice controlled with artificial intelligence, learning about the driver’s
surroundings and auto-adjusting the comfort for different road conditions.
Persona. Francisco is a male student nurse, living in Porto with his parents. In the
evenings and weekends, he mets with friends to surf, often changing clothes in the car
and carrying wet equipment.
Justification. In 2012, the International Surfing Association (ISA) claimed 35 million
surfers worldwide [6] and was expected to increase to 60 million by 2018 [7]. ISA’s
statistics show 81% being male and 40% 24 years old and younger. Therefore, Fran-
cisco potentially represents 20 million users worldwide (40% to 45% American) with a
similar persona profile.
3.2 Concept 02, India
A station wagon/van for market gardeners (see pitch video: vimeo.com/350452793).
120 J. Aston and R. P. Freire
Technical Proposal. A ‘skate-board’ platform with interchangeable carriages for
transporting people or merchandise. With autonomous levels 4 and 5, the users manage
the vehicle through smart devices, synchronising agendas and operating needs.
Persona. Joaquim is a market gardener for 45 years and lives in Portugal. He is
married to a school teacher with three grown-up children. They live and work in the
countryside transporting workers and produce between fields and farm. On market
days, Joaquim uses the vehicle as a mobile market stall.
Justification. A market garden is a small scale production of fruits, vegetables and
flowers (also known as truck or organic farms), who sell directly to the consumer, small
traders or restaurants. In Portugal, organic farming is increasing, from 73 producers in
1993, to 1500 in 2005 [8]. However, on a global scale this is more impressive, with 2.9
million organic growers worldwide and 69.8 million hectares, 50% in Australia alone
[9].
3.3 Concept 03, Jensen
A patrol truck for breakdown assistance. (see pitch video: vimeo.com/350453136).
Technical Proposal. A pickup truck/van configuration with interchangeable carriages
for roadside car repair and autonomous recovery. This vehicle is connected to a
management system to optimise distribution, human resources and reduce waiting
times.
Persona. John Jenson is 30 and works as a breakdown patrol driver and mechanic. He
lives in California with his wife and newborn baby. John manages his patrol fleet in
Los Angeles, coordinating 200 drivers and assisting 300,000 vehicle incidents a year.
Justification. Statistically, it is unclear to identify accurate breakdown patrol figures
for American roads. According Popular roadside assistance services (i.e. Allstate Motor
Club, AAA, Good Sam and AARP), they claim 52 state coverage [10] and subcontract
to smaller regional companies, which makes it difficult to quote an accurate patrol fleet
number. However, in the UK patrol fleet numbers are more accurate and the 6 most
popular roadside rescue services (i.e. RAC, AA plc, GEM Motoring Assit, Green Flag,
Britannica Rescue and Auto Aid) [11], together represent 20 to 25 thousand patrols
operating nationwide.
3.4 Concept 04, Picos
A snowmobile for ski tourist groups. (see pitch video: vimeo.com/350453214).
Technical Proposal. An electric-powered snowmobile with passenger trailer. When
coupled, this vehicle provides sight-seeing, security, comfort, audio and visual infor-
mation, and ample luggage/equipment space. Separately, the snowmobile gives rapid
assistance to tourists in the snow. Meanwhile, the trailer becomes a shelter.
Persona. Fernando Martinez is 20 and works as a ranger for snow-seeking tourists of
all ages. Based in the Spanish Cantabrian Mountains, about 50 kms from Bilbao,
The Car as a Transformer 121
Fernando specialises in skiing and mountaineering. In the winter he transports people
and equipment to the mountain slopes for sight-seeing, walking, sledging and skiing.
Justification. The Cantabrian mountains is part of the 430 km Pyrenees mountain
range, between Spain, France and Andorra. In 2015, there were 3.1 million visitors on
the Spanish side of the boarder [12]. Snowmobiles are popular on these mountainous
regions, for their hill-climbing capability, speed and agility. Snow Coaches, Snowcats
and Terra Buses are commonly found in Canada, Greenland, Iceland and Norway. The
‘Picos’ concept combines these two snow vehicle types.
3.5 Concept 05, Seniors
An autonomous micro car for the elderly (see pitch video: vimeo.com/350453263).
Technical Proposal. A micro car with a spacious interior, elevator luggage access,
comfortable high-swivel seats and large windows with interior digital interface. This
vehicle is fully autonomous and promotes ‘drive-on-demand’ for elderly users.
Persona. John and Marie are retired and enjoy the freedom to go shopping, theatre,
garden centres, and visit friends and family around London. Recently, John has started
to lose his hearing and is experiencing slow reactions.
Justification. In Britain, there are 5.3 million over 70 s with full driving licences and
there is no legal age for seniors to stop driving. They can decide when to stop as long as
they do not have any medical conditions that affect their driving [13]. There are laws to
eventually retain a senior’s driving licence, but governments rely on the driver’s good
sense to notice slower reactions, driving anxiety or worsening eyesight.
4 Conclusions
All 5 vehicle concepts expressed some interesting perspectives on how student
designers interpret the latest technological and social trends. The ideas considered
industry 4.0 and its impact on many aspects of our lives in the future, in particular how
we could move around more effectively in new forms of intelligent mobility.
Niche. Some concepts could be considered ‘niche’ based on their limited persona
profiles. However, if the data research considered broader persona characteristics, then
statistics might prove otherwise.
If concept 01 (iBudd) included similar activities to surfing (i.e. camping, skiing and
fishing), this could increase user potential by forming persona ‘clusters’.
Concept 02 (India) improved x2000 when research moved from Portuguese data to
global organic farmer statistics.
Concept 03 (Jensen) had unclear patrol vehicle data in America. However,
according to AA plc (within the top 6 breakdown services in the UK), they introduced
500 new patrol vehicles to their 2000 fleet in 2015 [14]. This could be used as an
indicator that 25% of patrol vehicles are replaced annually within this business.
122 J. Aston and R. P. Freire
Concept 04 (Picos) was aimed at a multi-million dollar global industry with an
estimated 300 to 350 million visitors worldwide [15]. Unfortunately, this could be
affected by climate change.
Concept 05 (Seniors) is potentially a huge market with 42 million licensed drivers
aged 65 and older in the United States [16], almost x10 greater than the UK.
Gender. Interestingly, all concepts showed male persona bias, except for concepts 2
and 5, which introduced female partners with shared vehicle scenarios. Considering
that the majority (58%) of student participants in this project were female, they might
have considered more influential female personas. However, referring to published
statistics about driver dominant gender, women travel more often than men, but men
drive greater distances and with more diverse modes of transport [17], which justifies
certain persona choices within each concept.
On the Market. Since the project’s completion in May 2017, all 5 concepts have in
part, appeared on the market in similar forms.
Bosch puts a voice (an interactive assistant called “Hey Casey”) into the car with
multi language recognition [18] (Jan 2018).
Rinspeed released the ‘Snap’ concept at the Geneva Show and focusses on MIU
concept [19] (March 20018).
Scania introduced an autonomous truck without a cab. This takes a significant step
towards smart transport systems of the future [20] (Sept 2019).
Venturi Automobiles developed the first electric Snow Rover, designed for short
scientific missions in the polar ice [21] (Dec, 2018).
Waymo developed an autonomous car that helps the elderly, as well as individuals
with disabilities [22] (Dec 2018).
Final Observation. Based on this evidence, elements from all 5 concepts were able to
forecast similar ideas currently on the market. This emphasises the importance of
investing in a multidisciplinary approach to vehicle design.
Acknowledgments. Professors: Magri L; Gomes M.; Ferreira J.L.; guest professor Luc-
carelli M. (Reutlingen University). Students: Cunha A.; Dias D.; Correia E.; Kunsler J.;
Rodrigues M.; Gomes N.; Legoinha C.; Ribeiro C.; Silva M.; Neves P.; Alves S.; Ribeiro C.;
Coelho A.; Teixeira B.; Vieira S.; Etkin N.; Kulovesi A.; Dybicki K.; Nowoszynska D.
References
1. Luccarelli, M., Matt, D.T., Spena, P.R.: Modular architectures for future alternative vehicles.
Int. J. Veh. Des. 67(4), 368–387 (2015)
2. Luccarelli, M.: Technological Challenges and New Design Opportunities. Auto Des. Torino.
(222), 2–7 (2017)
3. Rossini, M., Matt, D.T., Ciarapica, F.E., Russo Spena, P., Luccarelli, M.: Electric vehicles
market penetration forecasts and scenarios: a review and outlook. Int. J. Oper. Quant.
Manage. 20(3), 153–192 (2014)
4. Luccarelli, M.: Interdisciplinarity in product design education. Balancing disciplines to foster
creativity. In: 4th International Conference on Design Creativity, Atlanta, GA (2016)
The Car as a Transformer 123
5. Personas - A Simple Introduction (Interaction Design Foundation, Est. 2002) (2019).
interaction-design.org/literature/article/personas-why-and-how-you-should-use-them
6. Surferstoday: How many surfer are there in the world? (2019). surfertoday.com/surfing/how-
many-surfers-are-there-in-the-world
7. How many surfer are there in the world? medium.com/lipchain/how-many-surfers-are-there-
in-the-world-c43d04b93442
8. Organic farming (2019). en.wikipedia.org/wiki/Agriculture_in_Portugal
9. Organic Agriculture (2019). orgprints.org/33355/5/lernoud-willer-2019-global-stats.pdf
10. Best roadside assistance (2019). toptenreviews.com/best-roadside-assistance-services
11. Best breakdown cover (2019). https://www.autoexpress.co.uk/car-news/driver-power/92413/
best-breakdown-cover-2019
12. Number of visitors to ski resort in the Pyrenees from 2012 to 2015 (2019). statista.com/
statistics/768160/number-visitors-station-ski-pyrenees-by-country/
13. Deciding when to stop driving (2019). https://www.nidirect.gov.uk/articles/older-drivers-
deciding-when-stop-driving
14. AA plc. (2019). en.wikipedia.org/wiki/AA_plc
15. A critical review of climate change risk for the ski tourism (2019). https://www.tandfonline.
com/doi/pdf/10.1080/13683500.2017.1410110
16. Older Adult Drivers (2019). cdc.gov/motorvehiclesafety/older_adult_drivers/index.html
17. Driver gender (2019). brake.org.uk/facts-resources/1593-driver-gender
18. Car, we have to talk (2019). bosch-presse.de/pressportal/de/en/car-we-have-to-talk-bosch-
puts-the-voice-assistant-behind-the-wheel-137856.html
19. Rinspeed Snap (2019). rinspeed.eu/en/Snap_48_concept-car.html
20. Scania AXL (2019). scania.com/group/en/a-new-cabless-concept-revealing-scania-axl/
21. Venturi Antarctica all-electric exploration rover (2019). newatlas.com/venturi-electric-
antarctica-polar-vehicle/57622/
22. Designing self-driving cars for the elderly (2019). ecnmag.com/article/2018/12/designing-
self-driving-cars-elderly
Unmanned Small Shared Electric Vehicle
Binhong Zhai, Guodong Yin(&), and Zhen Wu
School of Mechanical Engineering,
Southeast University, Nanjing 211189, China
[email protected], [email protected]
Abstract. Nowadays, shared electric cars are becoming a new form of trans-
portation. In the future, unmanned small shared electric cars will be the main-
stream direction of the market, which is very different from traditional cars in
terms of usage mode, usage scenario and interaction mode, which will promote
the fundamental transformation of the relationship between people and cars.
Based on the analysis of the advantages and disadvantages of the current
development of shared cars, this paper points out that in the future, driverless
vehicles will be the transformation direction of shared electric cars, which needs
to be developed in the direction of service design, and then introduces the design
concept of “service-oriented”. Under the guidance of this concept, the rela-
tionship between cars and people will be reconsidered to provide better travel
services and experience for users.
Keywords: Unmanned small shared electric vehicle Sharing economy User
experience and service Service-oriented User-centered
1 Introduction
With the advent of the sharing economy, it has an important impact on social devel-
opment [1]. Shared electric cars are becoming a new form of transportation. A large
number of small electric cars that share transportation are suddenly popular in the
market. But there are also many challenges in the process of its development, such as
car clean, quality of service, supply shortage during peak time and so on have hindered
its better development. In today’s society, driverless technology is improving con-
stantly, the development of driverless small shared electric cars is an important
opportunity that many companies want to seize, which is very different from traditional
cars in terms of usage mode, usage scenario and interaction mode. In the future, only
by carrying out service reform, aiming at the needs of users in the era of sharing
economy, and finding out the new positioning of unmanned shared electric cars, can
cars stand out in the industry and produce products that meet the needs of users.
Therefore, it is necessary to firmly grasp the future automobile development mode,
consider the future design of shared electric cars from a multi-dimensional perspective,
break through the constraints of traditional car design, and explore various possibilities
of design, so as to improve people’s short-distance travel problems faster and better.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 124–128, 2020.
https://doi.org/10.1007/978-3-030-39512-4_20
Unmanned Small Shared Electric Vehicle 125
2 Analysis of Advantages and Disadvantages of Shared Car
Development
2.1 Advantages
Although most families in China now have a car, it can indeed bring convenience to
some extent, but the high gasoline cost and maintenance cost of cars make most
families choose public transportation to go out. The emergence of shared cars greatly
solves this problem, enabling most people to use cars without spending insurance and
maintenance costs. Most families can choose to share cars when they go out for play,
which can save costs.
The development of shared car can bring great convenience to people. It is different
from the traditional method of car renting, users can reserve car online at anytime and
anywhere, and the corresponding software will have detailed cost description, the
follow-up matters of the car are taken care of by the shared car company, it has a
relatively simple procedures and more convenient for users.
2.2 Disadvantages
When the company operates the shared car, it is necessary to improve the relevant
facilities for users to provide greater convenience. However, the development of shared
cars is slow in China, and the construction of related facilities is not perfect in the
process of development. For example, the number of parking lots of shared cars is
small, and it is difficult to find parking spaces when using shared cars, which reduces
the user experience to some extent. Secondly, the distribution of charging piles is
uneven, and the distance between charging piles is relatively long, which makes it
inconvenient for users to use. In the end, users will no longer use shared cars, and the
development of shared cars is limited.
When users use the shared car, there is a problem of non-standard use. For someone
who just got a driver’s license but has never driven a car, shared car is definitely the
first choice. However, due to lack of experience of novices, if the accident on the road
caused damage to the car, then the responsibility of attribution becomes a big problem.
3 Future Prospects - Driverless Small Shared Electric
Vehicles
At present, sharing electric cars offers a lot of convenience for people, but there are also
a lot of difficulties that hinder its better development. In the future era of sharing
economy, driverless vehicles will greatly promote the development of shared cars,
bring more benefits to enterprises, and promote the fundamental transformation of the
relationship between people and cars, bringing better services and experience to people.
In this case, service design will be an important tool to provide innovation and comfort
[3]. Next, the development idea of driverless small shared electric vehicles will be
introduced in the direction of service design.
126 B. Zhai et al.
3.1 Service Design Research Methodology
In general, service design orchestrates programs, technologies, and interactions to co-
create valuable solutions in complex systems and integrate their stakeholders. It is
different from traditional design disciplines such as industrial design and product
design, the final result of service design is intangible: a service. Therefore, the product
of service design is not a physical object. Rather, it is the service experience itself that
occurs at the moment of service delivery. The purpose of a service designer is to serve
the end users of a service and create value for the service provider simultaneously.
3.2 Unmanned Small Shared Electric Vehicles Need to Comply
with Service Design Methods
Service design pays attention to each stakeholder through a comprehensive perspective
[2]. To achieve comprehensive development of driverless small shared electric vehi-
cles, service process needs to be paid attention to, and service design processes need to
be better integrated into it.
The service design process is mainly divided into four parts, discover, define,
design and deliver, which belong to parallel design. It is that the whole product life
cycle is considered from the beginning of the product, including the generation of the
concept, how the product is put into the market and the abandonment of the product.
Service design comes into play when problems arise, or when shared concepts occur.
The traditional design process includes serial design process and parallel design
process. The serial design process mainly includes the following stages: user demand
analysis, program design, technical design, test and production design. In this phase,
the processes connect to each other, and the previous process completes and passes to
the next. However, the disadvantage is that each department is independent of each
other, and there will be many problems left over from the previous stage in the
production process, which makes it difficult for products to be put into production
smoothly, resulting in a long production cycle and increased production costs. In the
case of driverless small shared cars, this is very bad, because the links are often
connected and the processes are not independent. Therefore, it is necessary to follow
the service design process for driverless small shared cars.
3.3 Unmanned Small Shared Electric Vehicles Should Follow Service
Design Principles
(1) User-centered problem analysis and planning phase
This process can also be called requirements exploration and discovery stage. In the
creation stage, a requirement must clearly dissect the current service process and
conduct user-centered research on the users of the service and the real situation when
the service is used, so as to determine the service positioning. The user-centered
research mentioned here applies different methods and analysis tools to study users’
psychology, physiology and various behaviors, then establish user role model [5]. It is
also a key step for enterprises to seize service opportunities in the early stage of service.
Unmanned Small Shared Electric Vehicle 127
(2) Design and development phase based on user participation design
This stage can also be called prototype creation stage. Through the establishment of
service prototype in the early stage and user test, the corresponding service contacts are
determined to further improve the service process system. In this stage, enterprises
should let real users participate in creative activities and design products for them-
selves, in order to enhance their sense of ownership, mobilize the enthusiasm of users,
then users could provide their own insights to achieve user participatory design [4].
Designers and users should work together to complete the preliminary prototype design
of products, and modify and improve the service design system constantly.
(3) The design evaluation of multiple scale increases the confidence of enterprises
This stage is also the last stage of service design, and it is also the stage to promote
the better development of service design. In the process of design evaluation, the
usability of service design is tested through the subjective feedback and objective
evaluation of users, and the design evaluation is made to correct and iterate the
problems in the design process, so as to design unmanned small shared electric vehicles
that better meets the needs of users and increase the credibility of the enterprise.
4 The Revolution of Automobile Travel Under Service Design
Based on the development direction of driverless small shared electric vehicles under
the guidance of service design, the design concept of “service-oriented” has been
formed. It aims to guide future driverless small shared electric vehicles with the goal of
improving the travelers’ experience, such as reducing travel waiting time, increase
transfer connection reliability, so as to improve service accessibility, diversity, and
improve the quality of service.
4.1 Travelers Can Enjoy Better Travel Experience
The driverless small shared electric vehicles provide the whole process of “door to
door” service based on “service-oriented”. Users only need to book the departure time,
starting point, destination and other special travel preferences before travel. After the
system plans the trip, users can obtain the specific information of all travel segments,
including the destination related information [6]. During the trip, the travel platform
will provide users with the data analysis results based on historical and dynamic
changes, so that users can obtain real-time trip information “anytime and anywhere”,
such as traffic information, weather information, the possibility of connecting the next
stage of the trip. Then users facilitate the re-arrangement of the trip according to
relevant information to make the whole travel process more reliable.
4.2 Travel Structure and Characteristics Will Be Changed
Since driverless small shared electric vehicles can make the connection between dif-
ferent transportation systems closer, the travel efficiency of people will be greatly
128 B. Zhai et al.
improved, and the travel distance will also increase. The spatial distance will no longer
be the major obstacle to travel, and the proportion of cross-district or even cross-city
travel will also increase significantly [7]. Driverless small shared electric cars will pay
more attention to improving the service quality for passengers, customized services will
also be provided for passengers with special needs, such as the elderly and the disabled.
The composition of travel chain of the whole society will also be more flexible, so that
people can more easily enter more places to have leisure, entertainment and social
contact [8].
5 Conclusion
This paper mainly discusses driverless small shared electric vehicles. Firstly, the cur-
rent development status of shared cars was introduced, and then it was proposed that
driverless cars would promote the development of shared cars. Next, the development
idea of driverless small shared electric vehicle is introduced in the direction of service
design, and the design concept of “service-oriented” is put forward. Under the guidance
of this concept, the product will be more responsive to user needs, better insight into
user behavior, and better understand that the car is not only a simple transportation tool,
but also an emotional partner for people to travel. It is accompanied by people’s good
expectations for travel time after time, as well as their desire for convenience, and
safety to go home.
Acknowledgments. I want to express my gratitude to my tutor and some good classmates who
helped me during writing the paper.
References
1. Giana, M.E., Mark, B.H., Baojun, J., Cait, L., Aric, R., Georgios, Z.: Marketing in the sharing
economy. J. Mark. 83(5), 5–27 (2019)
2. Turetken, O., Grefen, P., Gilsing, R., Adali, O.E.: Service-dominant business model design
for digital innovation in smart mobility. Bus. Inf. Syst. Eng. 61(1), 9–29 (2019)
3. Qian, S., Carolyn, R.: Is service design in demand? Acad. Des. Manag. Conf. Inflection Point
11, 67–78 (2016)
4. Kimbell, L.: Designing for service as one way of designing services. Int. J. Mark. 5(2), 41–52
(2011)
5. Alam, I., Perry, C.: A customer-oriented new service development process. J. Serv. Mark. 16
(6), 515–534 (2002)
6. Paundra, J., Rook, L., Van Dalen, J., Ketter, W.: Preferences for car sharing services: effects
of instrumental attributes and psychological ownership. J. Environ. Psychol. 53, 121–130
(2017)
7. Susan, S., Adam, C., Ismail, Z.: Shared mobility: current practices and guiding principles
(2016)
8. Mattia, G., Roberta, G.M., Ludovica, P.: Shared mobility as a driver for sustainable
consumptions: the intention to re-use free-floating car sharing (2019)
A Forward Train Detection Method Based
on Convolutional Neural Network
Zhangyu Wang1(&), Tony Lee2, Michael Leung2, Simon Tang2,
Qiang Zhang3, Zining Yang4, and Virginia Cheung4
1
Beijing Aiforrail Technology Co. Ltd., Beijing, China
[email protected]
2
MTR Corporation Ltd., Kowloon Bay, Hong Kong, China
{tkylee,mileung,simtang}@mtr.com.hk
3
Traffic Control Technology Co. Ltd., Beijing, China
[email protected]
4
Claremont Graduate University, Claremont, CA, USA
{zining.yang,virginia.cheung}@cgu.edu
Abstract. Forward train detection is of great significance to improve train
safety. In this paper, a vision-based method is proposed for forward train
detection. The proposed method based on CenterNet, a novel object detection
network, to realize accurate and fast forward train detection. The forward train
detection network is divided into two stages: downsampling stage and center
points generation stage. In downsampling stage, a full convolution network is
performed to downsample the image, meanwhile to extract the feature of the
image. In the center points generation stage, three branch networks are used to
predict the bounding box of the forward train, including heatmaps generation
branch, center offset regression branch and forward train bounding box size
regression branch. Experiments results show that the proposed method can
detect the forward train well and achieve 30.7% Average Precision (AP) with a
47.6 Frames Per Second (FPS).
Keywords: Train detection Railway safety Convolutional neural network
1 Introduction
In all transportation systems, especially in rail transit, safety is the most important [1].
However, collision incidents still exist in rail transit at the present stage. In the last
decade in China, there were 7 serious collisions which caused 786 injuries [2].
According to the study [3], 95% of incidents are related to human factors and 70% of
these are directly related to human errors. Therefore, it is essential to improve the train
operation safety by adopting forward train detection for reducing collision related
incidents caused by human factors.
There are various studies in the past few decades to improve the train safety by
forward train detection or forward obstacle detection. For example, Karaduman used
the image processing and laser meter to detect the obstacle in the railway [4]; García
et al., proposed a multisensory system to detect the obstacles, the system consists two
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 129–135, 2020.
https://doi.org/10.1007/978-3-030-39512-4_21
130 Z. Wang et al.
emitting and receiving barriers, which are placed on opposing sides of the railway
respectively, and the system uses infrared and ultrasonic sensors, establishing different
optical and acoustic links between them [5]; Yao et al. proposed three vision-based
algorithms for obstacle detection, mainly include method based on Gray Level His-
togram, method based on the Proportion of Black and White Pixels, and method based
on Integrity of Rails and Sleepers [6].
In this paper, convolutional neural network is applied for forward train detection.
The proposed method mainly based on CenterNet [7], a novel object detection net-
work, to realize accurate and fast forward train detection. The proposed method
transforms the forward train detection into the detection of critical points of the forward
train, and thus enhances the performance of the detection function. The remaining part
of this paper is structured as follows. Section 2 illustrates the methodological details of
the proposed framework for forward train detection. Section 3 shows the experiment
results. Finally, a conclusion is presented in Sect. 4.
2 Forward Train Detection
The accuracy and speed of the forward train detection are the most important factors in
practical application. In this paper we apply the Centernet [7], a novel object detection
framework for forward train detection.
2.1 Overview of the Framework
Instead of detecting the forward train as a series of bounding boxes which is widely
used in two-stage detectors [8] or one-stage detectors [9], the forward train detection
image is converted for a key point estimation, i.e. get the center points of the forward
train.
Fig. 1. Network architecture
A Forward Train Detection Method Based on Convolutional Neural Network 131
As illustrated in Fig. 1, forward train detection network is divided into two stages:
downsampling stage and center points generation stage. In downsampling stage, a full
convolution network, i.e. DLA-34 [10] is performed to downsample the image,
meanwhile to extract the features of the image and to get a 4 times lower sampled
feature map. Based on the downsampled feature map, the center points of the forward
train are then generated. In the center points generation stage, three branch networks
are used to predict the bounding box of the forward train, including heatmaps gener-
ation branch, center offset regression branch and forward train bounding box size
regression branch. The heatmaps generation branch is used to generate the center points
of the forward train. The number of channels of the heat map is equal to the number of
forward train categories to be detected, and in this paper, the number of the channels is
1. The center offset regression branch is used to correct the deviation between true train
center points and predicted train points which is generated from the first branch. The
train bounding box size regression branch is used to predict the width and height of the
forward train.
2.2 Detection of Forward Train Points
The forward train detection is taken as the center points prediction of the forward train.
Therefore, behind the full convolution network, there are three independent forward
train center points regression branches to detect the center points of the forward train.
Heatmaps Generation
In heatmap generation branch, two-layer convolution is applied to obtain the heatmaps
of the feature map, and then the 3 3 max pooling operation is applied to get the
peaks of the forward train center. The first 100 peaks are taken as the target center
points extracted by the network, and then a threshold is set for screening to obtain the
final forward train heatmaps.
b 2 ½0; 1 R R C , where the W and H are the
W H
The heatmaps can be expressed as Y
width and height of the input image respectively, the R is the downsampling factor, i.e.
4 in this paper, and the C is category of the detected forward train, i.e. 1 in this paper. In
the heatmaps, if the prediction value Y bx;y ¼ 1, it means the point in the coordinate xy is
a forward train, and if the prediction value Y bx;y ¼ 0, it means the point in the coordinate
xy is not a train.
In the training stage, a gaussian kernel is applied to map the key points to the
2
ðxep x Þ þ ðyep y Þ
2
feature map, i.e. Yxyc ¼ exp 2r2 , where e p x and e
p y are the forward train
p
center in ground truth, and the r is the standard deviation relative to the size of the
forward train. The loss function of the heatmaps can be expressed as:
8 a
1 X < 1Y bxyc log Ybxyc if Yxyc ¼ 1
Lk ¼ a ð1Þ
N xyc
: 1 Yxyc b Ybxyc log 1 Y bxyc otherwise
132 Z. Wang et al.
Where the a and b are the hyper-parameter of the focal loss, N is the number of key
points.
Offset Regression
In the heatmap generation branch, the feature map is sampled by 4 times lower;
therefore, error in accuracy will be caused when the feature map is remapped to the
original image and an additional local offset is used for each center point. In the offset
regression branch, two-layer convolution is applied to obtain the offset feature
map. The feature map can be expressed as O b 2 RWR HR2 , and the two feature maps
predicts the deviation of x and y respectively. In the offset regression branch, the loss
function can be expressed as:
1X b p
Loff ¼ j Oep e
p ð2Þ
N p R
Where the O b is the offset predicted, and the p e
p is the offset between the center
ep R
point in the original image and the center point of the downsampled feature maps.
Bounding Box Size Regression
In the forward train bounding box size regression branch, two-layer convolution is
applied to obtain the boxes size feature map. The feature map can be expressed as
b
S 2 R R R 2 , and the two feature maps predicted the width and height of the forward
W H
train respectively. The loss function can be expressed as:
1X N
Lsize ¼ jb
S pk sk j ð3Þ
N k¼1
Where the bS pk is the boxes size predicted, and the sk is the boxes size that in the
ground truth.
In the proposed network, the whole loss function in the training stage can expre-
ssed as:
Lsize ¼ Lk þ coff Loff þ csize Lsize ð4Þ
Where the coff and csize are the hyper-parameter of the whole loss.
3 Experiment
To evaluate the effectiveness of the proposed forward train detection method, a large
amount of real train running video with resolution of 1280 720 are collected. And we
made a dataset based on the video. The dataset is labeled in MS Coco’s [11] format and
contains 6,779 annotated samples (5811 for training and 968 for test respectively). All
experiments were based on Titan Xp GPU, Pytorch 1.1.0, CUDA 9.0, and CUDNN 7.1.
A Forward Train Detection Method Based on Convolutional Neural Network 133
The Adam [12] is used to optimize the overall objective, and the loss function curve is
shown in Fig. 2.
Fig. 2. Loss curve during training
Figure 2 shows the model can converge quickly. Besides, the accuracy of forward
train detection is evaluated in detail by Average Precision (AP) [11] and Frames Per
Second (FPS). The speed and accuracy of the forward train detection method is shown
in Table 1.
Table 1. Speed and accuracy of the forward train detection method.
AP AP50 AP75 APS APM APL FPS
0.307 0.456 0.331 0.323 0.348 0.382 47.6
Table 1 shows that the method can achieve 30.7% AP with 47.6 FPS. This shows
that our forward train method achieves a good balance between speed and accuracy.
This achieved AP and FPS are under stringent requirements adopted in the calculation
algorithm of this method. The FPS is higher than most of typical calculation algorithm.
Therefore, high accuracy of detection can be expected for practical application in
railway environment. In addition, in our practical application, by combining the
detection of railway, we can obtain 0.007 of leak detected rate (LDR) and 0.000478 of
the mistakenly detected rate (MDR). Some samples are shown in Fig. 3 to demonstrate
the forward train detection results.
Fig. 3. Experimental results
134 Z. Wang et al.
4 Conclusion
In this paper, a convolutional neural network based forward train detection method is
proposed. The proposed method mainly divided into two stages: downsampling stage
and center points generation stage. Downsampling stage mainly used for image sam-
pling and feature extraction. The center points generation stage used three branches to
predict the bounding box of the forward train. The experiments show that the proposed
forward train detection method can achieve 30.7% AP with 47.6 FPS. The achieved AP
and FPS are under stringent requirements adopted in the calculation algorithm and the
overall performance in accuracy and speed is superior than most of typical calculation
algorithm. Therefore, high performance of forward train detection can be expected for
practical application in railway environment. As image recognitions are susceptible to
changing of environmental lighting, application of the fusion of lidar and image will be
adopted in further study in order to further enhance the accuracy of forward train
detection.
Acknowledgments. This work is partially supported by the Beijing Municipal Science and
Technology Project under Grant # Z181100008918003. The MTR Corporation Ltd. in Hong Kong
has provided the testing field in co-researching the proposed forward train detection method and
technology. The authors would also like to thank the insightful and constructive comments from
anonymous reviewers.
References
1. García, J.J., Urena, J., Mazo, M., et al.: Sensory system for obstacle detection on high-speed
lines. Transp. Res. Part C Emerg. Technol. 18(4), 536–553 (2010)
2. Li, S., Cai, B., Liu, J., et al.: Collision risk analysis based train collision early warning
strategy. Accid. Anal. Prev. 112, 94–104 (2018)
3. Lüy, M., Çam, E., Ulamış, F., et al.: Initial results of testing a multilayer laser scanner in a
collision avoidance system for light rail vehicles. Appl. Sci. 8(4), 475 (2018)
4. Karaduman, M.: Image processing based obstacle detection with laser measurement in
railways. In: 2017 10th International Conference on Electrical and Electronics Engineering
(ELECO), pp. 899–903. IEEE (2017)
5. Garcia, J.J., Ureña, J., Hernandez, A., et al.: Efficient multisensory barrier for obstacle
detection on railways. IEEE Trans. Intell. Transp. Syst. 11(3), 702–713 (2010)
6. Yao, T., Dai, S., Wang, P., et al.: Image based obstacle detection for automatic train
supervision. In: 2012 5th International Congress on Image and Signal Processing, pp. 1267–
1270. IEEE (2012)
7. Zhou, X., et al.: Objects as points. arXiv preprint arXiv:1904.07850v2
8. Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with
region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–
99 (2015)
A Forward Train Detection Method Based on Convolutional Neural Network 135
9. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.
02767 (2018)
10. Yu, F., Wang, D., Shelhamer, E., et al.: Deep layer aggregation. In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp. 2403–2412 (2018)
11. Lin, T.Y., Maire, M., Belongie, S., et al.: Microsoft COCO: common objects in context. In:
Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) European Conference on Computer
Vision, pp. 740–755. Springer, Cham (2014)
12. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:
1412.6980 (2014)
Styling Research of DFAC-6851H4E City Bus
Based on Fuzzy Evaluation
Huajie Wang1,2(&) and Jianxin Cheng1
1
East China University of Science and Technology,
Shanghai 200237, People’s Republic of China
[email protected], [email protected]
2
Shanghai Art and Design Academy,
Shanghai 201808, People’s Republic of China
Abstract. In the process of design decision-making, in order to establish a
more scientific design evaluation method, help enterprises determine the best
city bus design plan and guarantee the best economic, social and cultural ben-
efits of design results. In this paper, the analytic hierarchy process is used to
analyze the form of the city bus, and the importance proportion of the appear-
ance of the product is studied. The design project of DFAC(DFAC: It is the
abbreviation of Dongfeng Automobile Company, which is the very famous Auto
company in China.)-6851H4E is taken as the research object, and the established
evaluation model is verified by practice. These results show that the scientific
design evaluation model is constructed, and the progress of the design is
effectively promoted in the process of product design, which avoids the
uncertainty of design, reduces the risk of design, and brings good economic
benefits to the enterprise.
Keywords: City bus styling Analytic hierarchy process Fuzzy evaluation
method Design decision
1 Introduction
In the development process of city bus, the styling design is an important stage. A good
modeling scheme is an important factor to win the market. In the process of selecting
the best scheme, product appearance styling design scheme will be optimized and
selected through design evaluation.
Ma [1] proposed the scheme evaluation compass, Yang proposed to establish a
perceptual evaluation index system based on crutches for the aged [2]. However, for the
design of the city bus styling scheme evaluation has not had mature case, this thesis
city buses DFAC - 6851H4E as the research object, through the use of certain quan-
titative analysis means to evaluate the product design, the design scheme of the per-
ceptual factors of quantitative analysis, using the evaluation model for evaluation,
increase the product outward appearance modeling design scheme decision-making
objectivity.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 136–142, 2020.
https://doi.org/10.1007/978-3-030-39512-4_22
Styling Research of DFAC-6851H4E City Bus Based on Fuzzy Evaluation 137
2 Process of Bus Modeling and Design
In bus enterprises, the design process of bus products is usually shown in Fig. 1.
Design evaluation is very critical, especially in the styling phase of the program
evaluation is an important link to determine the product design results.
Fig. 1. Flow chart of bus modeling design
3 Analysis of the Status Bus Industry Modeling Program
Review
In bus enterprises, the product modeling design scheme is generally determined by the
boss of the enterprise. This evaluation mode generally has the following problems:
(1) Incomplete consideration of evaluation objectives. It is not comprehensive to
consider which aspects of the design plan need to be evaluated,
(2) The weight of evaluation factors is not quantitative. There is no quantitative dis-
tinction between the evaluation objectives, which easily leads to the inversion of
priority and priority.
4 Fuzzy Evaluation Method
Fuzzy evaluation method is an effective comprehensive evaluation method for things
affected by multiple factors. It breaks through the logic and language of precise
mathematics [3], emphasizes the fuzziness in influencing factors to express the nature
of information in natural language, and processes evaluation information in numerical
calculation.
4.1 Determine the Evaluation Index Set
Due to the complexity of the evaluation target, it is necessary to decompose the
evaluation target into multi-level indexes and establish the hierarchical structure chart
of the evaluation index system. After the structure diagram is established, the evalu-
ation target can be regarded as a fuzzy set composed of multiple factors. There are n
138 H. Wang and J. Cheng
evaluation indicators, and there are n subsets. If the evaluation indicator set is U, then
U = {U1, U2, U3… Ui,… Un}, type in the Ui (i = 1, 2, 3… n) is each evaluation index.
4.2 Determination of Evaluation Set (Evaluation Level)
After the evaluation index set is determined, corresponding evaluation results should be
made for it. Each indicator is evaluated with abstract evaluation level to form a fuzzy
set of comments (called evaluation set). The grade of comments is generally expressed
by very good, better, general, poor, poorer or excellent, good, medium, poor and other
comments. If the evaluation set is set as V and m evaluation levels are set, then the
evaluation set V = {V1, V2… The Vm}.
4.3 Establish Fuzzy Relation Matrix R1
After the index set and evaluation set are determined, each index has certain quanti-
tative relation with any evaluation, which can be represented by matrix R. R is the
fuzzy relation synthesized by U and V, which is a (n m) fuzzy matrix [4], namely:
23 2 3
R1 r11 r12 r1j r1m
6 R2 7 6 r21 r22 r2j r2m 7
6 7 6 7
R ¼ U V ¼ 6 .. 7 ¼ 6 . .. .. .. .. .. 7 ð1Þ
4 . 5 4 .. . . . . . 5
Rm rn1 rn2 rnj rnm
rij represents the corresponding evaluation factor Ui, and its evaluation level of the
evaluation object is j.
4.4 Membership
Membership is the degree to which an element belongs to a set. Membership degree
can be represented by a value on the closed interval of 0–1. When the membership
degree is stronger, the value is closer to 1; when the membership degree is weaker, the
value is closer to 0. For example, when the membership degree is 0.9, it means that
90% of element x belongs to set A.
Fuzzy matrix R is a collection of the degree of attribution of each evaluation index
to each evaluation level. Membership of a single qualitative evaluation index Ui to Vi
can be expressed by the following formula (2):
di
ri ¼ ð2Þ
d
ri is an element in the fuzzy matrix, d represents the number of experts participating in
the evaluation, and di refers to the number of experts making Vi evaluation on the
evaluation index Ui.
Styling Research of DFAC-6851H4E City Bus Based on Fuzzy Evaluation 139
4.5 Determine the Weight of Each Evaluation Index
Considering that each evaluation index has different importance degrees, corresponding
weight coefficients should be given to it. This paper uses the common expert scoring
method to determine. The principle and formula are shown in the membership section.
If the weight set is set as A, then A = {a1, a2, a3…… an}.
4.6 Fuzzy Comprehensive Evaluation
The problem of fuzzy comprehensive evaluation is to transform A fuzzy set A in
the evaluation index set U into B in the evaluation index set V through fuzzy relation
R. Therefore, a mathematical model of fuzzy comprehensive evaluation can be
established [5]:
2 3
r11 r12 r1j r1m
6 r21 r22 r2j r2m 7
6 7
B = A R ¼ ða1 ; a2 . . .an Þ 6 . .. .. .. .. .. 7 ð3Þ
4 .. . . . . . 5
rn1 rn2 rnj rnm
In this model, the evaluation result B is a fuzzy subset of the domain V, and also a
fuzzy vector, or a 1 m fuzzy matrix, B = (b1, b2, b3… bm), A is the weight set, and R
is the fuzzy matrix synthesized by U (evaluation index) and V (evaluation grade).
5 Establishment of Evaluation and Quantification Model
of Bus Modeling Design Scheme
5.1 Bus Modeling Evaluation Index System
According to the fuzzy evaluation method and the components of product modeling
design, mainly from the perspective of aesthetics, the preliminary determination of bus
modeling evaluation index system is based on 13 evaluation indexes. As shown in
Fig. 2.
5.2 Set the Rating Level
For the evaluation of each index in the evaluation target, the grade judgment is gen-
erally carried out by means of excellent, good, medium and poor. Therefore, the
comment set V for evaluation index U is: V = (excellent, good, medium, poor) = (V1,
V2, V3, V4).
5.3 Weight Calculation
After the evaluation index of bus model is determined, the weight of each index should
be calculated. The methods are as follows: 40 experts (including major departments of
140 H. Wang and J. Cheng
Fig. 2. Bus model quality evaluation index system
the company) are selected to evaluate the importance of each indicator to the overall
modeling, and vote on which indicator is important.
Primary index can be obtained by the weight is: A = (a1, a2, a3, a4, a5) = (8/40,
25/40, 2/40 40, 1/4/40) = (0.2, 0.625, 0.025, 0.1). Similarly, the weights of secondary
indexes can be calculated.
6 Practical Case Verification
6.1 Preliminary Selection of Design Scheme
Now take DFA6851H4E city bus design scheme evaluation as an example. As shown
in Fig. 3, among multiple design schemes, the decision-makers of the enterprise first
select a design scheme, which is more in line with the design concept of Dongfeng
brand.
6.2 Use Fuzzy Evaluation to Evaluate the Design Scheme
In order to further verify whether the proposal is the best design proposal or not, the
fuzzy evaluation method is adopted for further evaluation, and a 15-person evaluation
team including engineering, technology, planning, marketing and sales is organized to
vote and evaluate. Vote on the primary plan. According to the statistical data, the fuzzy
evaluation matrix of evaluation indexes is calculated.
By multiplying each index weight coefficient A with the above fuzzy matrix R, the
fuzzy comprehensive evaluation of this design scheme can be obtained: B = A
R = (0.44, 0.38, 0.10, 0.03). Learned from this, the design of optimal membership
Styling Research of DFAC-6851H4E City Bus Based on Fuzzy Evaluation 141
Fig. 3. Best design scheme selected from multiple modeling design schemes
degree is 0.44, the good membership degree was 0.38, the middle membership degree
is 0.10, and the bad membership degree is 0.03, according to the principle of maximum
membership, the indicators of the evaluation objects, item by item, into a relative rank
score, can give the evaluation results with the corresponding score [6]. If the four
grades of excellent, good, medium and poor in the evaluation grade are successively
converted into 10, 8, 6 and 4 points, then the evaluation score of the modeling design
scheme is: X = 0.4433 10 + 0.3833 8 + 0.095 6 + 0.0283 4 = 8.1826.
Thus, it can be determined that the comprehensive evaluation of the modeling
design scheme is excellent.
6.3 Engineering and Mass Production of the Final Design Scheme
After nearly 8 months of product verification and trial-production, the design and
manufacturing task has been completed, and the implementation of the design scheme
has been achieved (as shown in Fig. 4).
Fig. 4. Mass production process of the optimal design scheme
142 H. Wang and J. Cheng
7 Conclusion
The fuzzy evaluation method can be used to evaluate the design scheme of bus
appearance quantitatively. The evaluation quantitative model not only provides the
evaluation basis for the development of products with market competitiveness, but also
effectively reduces the market risk of the strategic decision of product development.
The program selected through fuzzy evaluation has performed well in the market sales,
with the sales of single product exceeding 8 billion yuan (RMB), which has won the
approval of customers.
References
1. Ma, Y.: Research on SUV side shape evaluation based on fuzzy evaluation method. Jilin
University (2018)
2. Yang, C., Cheng, J.: Study on the sensibility evaluation method of old conceptual products
based on fuzzy analysis. Packag. Eng. 39(10), 128–132 (2018)
3. Chen, M.: Analysis and research on gesture interaction design of smart touch-screen mobile
phones. Wuhan University of Technology (2013)
4. Wang, L., Du, W.: Application of ASP in fuzzy comprehensive evaluation. Inner
Mongolia TV Univ. J. (01), 22–23 (2007)
5. Wang, H.: Design of Suzhou King Long B92V series city bus. Creat. Des. Sources (06), 42–
47 (2010)
6. Cheng, X.: Research on product aesthetic evaluation based on fuzzy evaluation method.
Design (02), 170–171 (2013)
Object Detection to Evaluate Image-to-Image
Translation on Different Road Conditions
Fumiya Sudo(&), Yoshihiro Hashimoto, and Giuseppe Lisi
Nagoya Institute of Technology, Nagoya, Japan
[email protected],
{hashimoto,lisi.giuseppe}@nitech.ac.jp
Abstract. In vision-based navigation, image-to-image translation systems have
been recently proposed to remove the effect of variable road conditions from
images, in order to normalize the available data towards a common distribution
(e.g. sunny condition). In this context, previous works have proposed image de-
raining using generative adversarial networks (GAN) and evaluated perfor-
mance based on pixel-to-pixel similarity measures that do not account for
semantic content relevant for driving. Here, we propose to evaluate the per-
formance of a de-raining GAN using an object detection neural network pre-
trained to find cars and pedestrians. We conducted experiments on the CARLA
simulator to collect training and evaluation data under several weather condi-
tions. Results indicate that GAN de-rained images achieve a high object
detection performance in some conditions, but on average lower than object
detection on the original rainy images. Future work will concentrate on
improving semantic reconstruction and detection of other road elements (e.g.
lanes, signs).
Keywords: Image-to-image translation Generative adversarial networks
Autonomous driving Image de-raining Object detection
1 Introduction
Road safety is one of the biggest obstacles to the adoption of autonomous driving. The
varying road conditions (e.g. rain, snow) affect car sensors in often unpredictable ways.
For example, the performance of LIDAR sensors significantly decreases under rain [1].
Similarly, deep learning models trained on camera images, may be strongly affected by
changing road conditions, resulting in poor performance. Indeed, most of the images
used to train deep neural networks come from sunny data collection sessions, while
much fewer samples are available for different road conditions. To mitigate this
problem, image-to-image translation systems, mapping from an input image to an
output image, have been proposed for de-raining [2]: removing the effects of rain from
images (e.g. rain drops, reflective road). Similarly, these systems could be used to
generate images under different road conditions, for data augmentation [3]. Such de-
raining systems have been only evaluated using methods that are typically used in the
image-to-image translation literature, such as Peak Signal to Noise Ratio, Structural
Similarity, Universal Quality Index and Visual Information Fidelity [2], which require
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 143–149, 2020.
https://doi.org/10.1007/978-3-030-39512-4_23
144 F. Sudo et al.
ground-truth frames and omit semantic factors that are relevant for driving. Here, we
propose to evaluate the translated images using an object detection neural network at
each frame of a driving session (Fig. 1). This allows for a quantitative evaluation of the
translated image that is based on its semantic content, similar to the FCN score pro-
posed in [4], and does not require ground-truth frames.
2 Methods
We conducted experiments on the CARLA simulator [5], where a self-driving car
autonomously drives around a simulated city, collecting frames under sunny and rainy
road conditions. An image-to-image translator is trained to map rainy to sunny images,
using a Cycle Generative Adversarial Network (Cycle GAN).
Fig. 1. Flowchart of the proposed evaluation strategy. The Cycle GAN image-to-image
translator takes a rainy image (leftmost) as input and converts it to a sunny image (center).
The SSD object detection finds objects (rightmost image) using the output of the Cycle GAN.
Surprisingly, Cycle GAN translates correctly not only the cars, but also the traffic signs, light
poles, trees, and even brands the truck. However, we observe some artifacts on the sidewalk.
The advantage of Cycle GAN is that a mapping from a source domain X to a target
domain Y is learned in the absence of paired examples [6]. Subsequently, a SSD [7]
object detector, pre-trained to detect cars and pedestrians, is applied to the original and
to the de-rained frames, respectively.
2.1 Data Collection
We evaluate the performance and generalization ability under several experimental
conditions: 3 weather conditions (i.e. sunny noon, mid-rain noon, hard-rain noon;
Fig. 2), 2 resolution levels (i.e. 320 240 and 280 200) of the Cycle GAN input
(leftmost in Fig. 1), and 2 towns (i.e. experimental environment in CARLA).
For this purpose, we collected all the data necessary for training the Cycle GAN
and for evaluating the object detection performance, including images from the frontal
camera, the coordinates of all the vehicles and pedestrians in the environment, and the
relative timestamps. The autonomous driving agent available in CARLA was executed
Object Detection to Evaluate Image-to-Image Translation on Different Road Conditions 145
Fig. 2. An example of image data collected using Carla
for about 13 min, saving data with a time step of about 0.2 s, producing 3650 samples,
for each condition. In all the runs, the CARLA environment contained 60 pedestrians
and 30 cars, moving around the town.
2.2 Training and Evaluation Strategy
Only using samples from Town 01 (i.e. training town), a Cycle GAN is trained to de-
rain images separately for each image resolution level and for different rain levels: mid-
rain, hard-rain and mixed samples from mid- and hard- rain, for a total of 6 different
neural networks. The training of Cycle GANs is done as in the original paper with
batch size 1 and 200 epochs.
In order to generate de-rained images and evaluate the performance, each model is
then validated on samples from its respective image resolution, at Town 02 (i.e. test
town) and Town 01 (i.e. validation town), under the mid-rain and hard-rain conditions.
The de-rained images are then fed to the SSD object detection in order to evaluate how
much semantic content is left in the de-rained images. In order to build a baseline, SSD
is also executed on the original image (i.e. no_change). The SSD neural network is
trained to detect pedestrians and cars.
The last step of the validation procedure is to evaluate the accuracy of the SSD object
detection. For this purpose, we used the vehicles and pedestrian coordinates to generate
the ground truth. Specifically, the area within a 20 m radius and within a 90° angle of view,
was considered as the detection area (Fig. 3). For each frame, the output of the SSD
detector was then compared with the number of pedestrians and cars in the detection area,
in order to compute the following measures: true positives (TP), false positive (FP), false
negatives (FN), true positive rate (TPR = TP/(TP + FN)) and the false discovery rate
(FDR = FP/(FP + TP)). Examples of TP, FP and FN are visualized in Fig. 4.
Fig. 3. Detection area for object detection
146 F. Sudo et al.
3 Result
The summary of the performance grouped by no_change and Cycle GAN averaged
models is shown in Table 1, while the separate results of all the experimental condi-
tions can be found from Tables 2, 3, 4 and 5. Model no_change represents the situation
where SSD is applied directly to the original image, i.e. without Cycle GAN. More-
over, the model called mid_hard_rain represents the Cycle GAN trained with a mix of
samples from the mid-rain and hard-rain conditions. The weather column of the tables
represents the condition of the road when the model was executed.
Fig. 4. Examples of no_change images containing at least a true positive (TP), false positive
(FP), false negative (FN)
Table 1 indicates that, in general, the model no_change achieves the best TPR and
FDR for car and pedestrian detection, suggesting that the Cycle GAN removes a large
portion of semantic content (i.e. lower TPR) and introduces artifacts that fool the SSD
into hallucinating objects (i.e. higher FDR).
However, in some instances, we observe only a 10% difference in TPR between
no_change and model hardrain (Town 02, hard-rain condition), and a 11% difference
in Town 01 and mid-rain condition. This is a quite impressive performance of the
Cycle GAN that is able to remove rain while keeping the features of big objects.
On the other hand, with respect to pedestrian detection, TPR is low for all the
models, including no_change, due to the fact that pedestrians entering the detection
area are often too far for SSD to work properly, and they rarely come close enough to
the car. Accordingly, the FDR is highly variable due to the low number of TP and FP
counts. Therefore, with the current method, it is hard to make a conclusive evaluation
of the Cycle GAN performance with small objects.
Remarkably, we find no performance difference between Town 01 (i.e. training and
validation environment) and Town 02 (i.e. test environment), suggesting that the neural
network learned at Town 01 is able to generalize to Town 02 with a different layout and
characteristics, and that overfitting was not a problem. Moreover, image resolution does
not have an impact on the performance of the Cycle GAN + SSD pipeline object
detection.
Object Detection to Evaluate Image-to-Image Translation on Different Road Conditions 147
Table 1. Average performance
Car Pedestrian
Model TPR FDR TPR FDR
no_change 50.9% 1.0% 2.4% 1.9%
Cycle GAN avg. 26.6% 4.4% 0.3% 41.1%
Table 2. Town 01, resolution = (200, 280)
Car Pedestrian
Weather Model TPR FDR TPR FDR
midrain no_change 47% 1% 0.96% 9.1%
midrain midrain 15% 3% 0.29% 25.0%
midrain hardrain 36% 1% 0.29% 25.0%
midrain mid_hard_rain 19% 4% 0.19% 33.3%
hardrain no_change 49% 2% 2.49% 0.0%
hardrain midrain 33% 2% 0.00% 100%
hardrain hardrain 24% 3% 0.28% 20.0%
hardrain mid_hard_rain 18% 5% 0.48% 12.5%
Table 3. Town 01, resolution = (240, 320)
Car Pedestrian
Weather Model TPR FDR TPR FDR
midrain no_change 47% 1% 1.54% 5.9%
midrain midrain 16% 2% 0.38% 63.6%
midrain hardrain 30% 2% 0.10% 66.7%
midrain mid_hard_rain 21% 3% 0.38% 42.9%
hardrain no_change 51% 2% 2.42% 0.0%
hardrain midrain 32% 3% 0.14% 33.3%
hardrain hardrain 27% 2% 0.42% 64.7%
hardrain mid_hard_rain 23% 5% 1.54% 5.9%
Table 4. Town 02, resolution = (200, 280)
Car Pedestrian
Weather Model TPR FDR TPR FDR
midrain no_change 56% 0% 4.99% 0.0%
midrain midrain 22% 7% 0.21% 40.0%
midrain hardrain 25% 8% 0.21% 0.0%
midrain mid_hard_rain 29% 10% 0.00% 100%
(continued)
148 F. Sudo et al.
Table 4. (continued)
Car Pedestrian
Weather Model TPR FDR TPR FDR
hardrain no_change 50% 1% 0.80% 0.0%
hardrain midrain 12% 5% 0.11% 33.3%
hardrain hardrain 40% 4% 0.06% 85.7%
hardrain mid_hard_rain 18% 5% 0.48% 12.5%
Table 5. Town 02, resolution = (240, 320)
Car Pedestrian
Weather Model TPR FDR TPR FDR
midrain no_change 57% 0% 4.99% 0.0%
midrain midrain 20% 6% 0.14% 71.4%
midrain hardrain 31% 7% 0.07% 50.0%
midrain mid_hard_rain 39% 3% 0.00% 100%
hardrain no_change 50% 1% 1.32% 0.0%
hardrain midrain 30% 6% 0.23% 0.0%
hardrain hardrain 40% 4% 0.17% 0.0%
hardrain mid_hard_rain 38% 5% 0.11% 0.0%
4 Discussion
In general, Cycle GAN drastically reduces object detection performance, suggesting
that the semantic reconstruction after de-raining is not perfect. However, in some
experimental conditions, TPR on de-rained images is surprisingly high, indicating that
Cycle GAN is able to reconstruct semantically valid images, especially with big
objects, such as cars. With small objects, such as pedestrians, the FDR increases
drastically, suggesting that the Cycle GAN introduces artifacts that make the SSD
hallucinate. With respect to the various experimental conditions, we find that
Cycle GAN generalizes well from training in Town 01 to testing in Town 02. More-
over, image resolution does not have a particular effect, which may due to the fact that
the resolution levels tested are quite similar. However, testing larger resolutions is
prohibitive from an hardware and memory perspective.
In future, we will improve the detection area for pedestrians, and expand the
proposed technique to different objects on the road, such as lanes, traffic lights.
Moreover, we will test different types of GANs such as the Semantic GAN (SemGAN),
which produces semantically consistent output.
Image-to-image translation generates surprisingly realistic sunny images; however,
more research is required to generate more semantically coherent images.
Object Detection to Evaluate Image-to-Image Translation on Different Road Conditions 149
References
1. Rasshofer, R.H., Gresser, K.G.: Automotive radar and lidar systems for next generation driver
assistance functions. Adv. Radio Sci. 3(B. 4), 205–209 (2005)
2. Wang, C., Xu, C., Wang, C., Tao, D.: Perceptual adversarial networks for image-to-image
transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018)
3. Uricár, M., Krízek, P., Hurych, D., Sobh, I., Yogamani, S.: Yes, we GAN: applying
adversarial techniques for autonomous driving. arXiv preprint arXiv:1902.03442 (2019)
4. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional
adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition (2017)
5. Dosovitskiy, A., Ros, G., Codevilla, F., López, A., Koltun, V.: CARLA: an open urban
driving simulator. arXiv preprint arXiv:1711.03938 (2017)
6. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-
consistent adversarial networks. In: Proceedings of the IEEE International Conference on
Computer Vision (2017)
7. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.: SSD: single shot multibox detector.
In: European Conference on Computer Vision. Springer, Cham (2016)
Humans and Artificial Cognitive
Systems
Modelling Proxemics
for Human-Technology-Interaction
in Decentralized Social-Robot-Systems
Thomas Kirks(&), Jana Jost, Jan Finke, and Sebastian Hoose
Fraunhofer Institute for Material Flow and Logistics,
J.-v.-F. Str. 2-4, 44227 Dortmund, Germany
{thomas.kirks,jana.jost,jan.finke,
sebastian.hoose}@iml.fraunhofer.de
Abstract. Automation in production and logistics facilities has drastically been
increased in the past decades in a way that human workers are continually
confronted with robots in their working environment. In this paper, we present
the approach to model interactions between workers and our socially interacting
AGV EMILI by using elliptic, worker dependent zones in free navigation so that
it is possible to navigate around humans while respecting their individual dis-
tances. We use the human agent to save and update the information about his or
her personal zones. An AGV, which needs to pass by the human, communicates
with his or her human agent to retrieve this information and then calculates a
path, which respects the persons preferred distance.
Keywords: Human factors Human-technology interaction Social
decentralized systems
1 Introduction
Automation in production and logistics facilities has been increased in the past decades
in a way that human workers are continually confronted with robots in their working
environment. Especially, automated guided vehicles (AGVs) are supporting human
workers. To increase the acceptance of workers, those robots need to interact with
humans in a safe and emphatic way. To design a social and interactive robot, the
AGV EMILI has been developed which can interact with humans through multiple
interfaces. Previous work has shown that each human keeps individual distances
around him/her. Depending on the counterpart one interacts with, different distances are
kept e.g. family members can enter a closer zone than friends. Contrary, people feel
uncomfortable if an unfamiliar person enters a private zone. The field of proxemics
investigates these spaces and their effects on interaction among humans. Studies have
shown that these zones are also applicable to robots as well as that they are not always
of circular shapes. For a better acceptance towards robots, one has to use this infor-
mation when designing the path planning process of an AGV. In this paper, we present
the approach to model interactions between workers and our socially interacting
AGV EMILI by using elliptic, worker dependent zones in free navigation so that it is
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 153–158, 2020.
https://doi.org/10.1007/978-3-030-39512-4_24
154 T. Kirks et al.
possible to navigate around humans while respecting their individual distances. We use
the human agent to save and update the information about his/her personal zones.
An AGV retrieves this information from the human agent and then calculates a path
respecting the persons distance.
2 Background
2.1 State of the Art in Human-Robot Interaction
Today, robots are used in diverse use cases in industry and private areas. When robots
were heavy industrial robots decades ago, nowadays their appearance changed to more
smaller, flexible and cognitive robots [1] along with their abilities. These robots interact
directly with humans and partially do not need safety devices. Hence, they can be used
in new fields of application e.g. surgery [2]. Although the usage of robots in industrial
applications is increasing, the human worker will still play a major role in production
and logistics. This can only be achieved by the development of new intuitive inter-
action methods for humans and robots. Human-robot collaboration is one interaction
form besides cooperation and co-existence in human-robot interaction (HRI) [3],
For HRI we need also to consider psychological issues like cognitive ergonomics,
which aims on the reduction of mental stress by reducing complexity, enabling to
create intuitive interaction modalities and thereby raising acceptance of robotic
systems.
2.2 Proxemics
Since nowadays humans and robots collaborate, the results of decades-long research in
the field of human-human interaction such as the theory of proxemics by Hall [4] can
be used to ensure a human-friendly work environment. According to Hall each person
has four unique zones surrounding him/her where each zone has its own distance and is
defined by the kind of people entering it e.g. close friends are allowed to enter the
second closest, the personal zone, whereas in the closest, the intimate zone, partners are
positioned. In contrast to that, people feel uncomfortable if an unfamiliar person enters
a private zone. Robots working close to humans have to react according to its related
zone. Various studies have focused on factors, which have an impact on the distance
between robot and human. Humans feel anxious with taller robots and keep greater
distances [5]. The distance kept is proportional to the robots velocity [6] and the
general attitude towards robots influence the way people approach robots [7]. Further,
gender and age as well as robot experience influence the distance [8]. In addition, Jost
et al. [9] showed that humans approaching robots are keeping a smaller distance in
contrast to being approached. Besides, they found out that for robot unexperienced
persons the facial expressions of the robot – showing emotions – has an impact on the
distance kept.
Modelling Proxemics for Human-Technology-Interaction 155
2.3 Navigation for Mobile Robots
One option for the software development of robots is the Robot Operating System
(ROS). It offers the so called Navigationstack, a ROS node, for navigating robots. It
needs range sensor data, a model of a robot, odometry data, a navigation goal and a
map of the environment as input. Using range sensor data and the given map, a
costmap is generated on which the path planning algorithm relies on. The calculated
path is transformed into translational and rotational velocities to move the robot from
one point to a goal [10]. The costmap generation is split up in layers of which each adds
up more information to the costmap. The first layer adds the general layout, while
following layers add data e.g. from range sensors. Each layer is independently
implemented in a plugin. The data in the costmap is represented by a character array,
which can be reinterpreted as a one-channel image. Each pixel represents an area
depending on the resolution and the corresponding cost [11]. Using costmap layers, the
behavior of the navigation can be directly. Hence, obstacles referring to humans can be
manipulated in the costmap to fit the needs of workers. An example is provided by the
social_plugin which adds a Gaussian on a given position with the purpose to imple-
ment a proxemics zone [11]. Unfortunately, this implementation does not allow an
individualization of the zones depending on the referenced person and its needs, as
described in Sect. 2.2.
3 Modelling Proxemics for Social-Robot-Systems
3.1 Requirements for the Model
In this paper, we present a model for the proxemics zone which is referring to the
various needs of a worker. To model those needs different requirements have been
identified. First, we want to cover different proxemics zones. The intimate zone ðhð x; yÞÞ
should not be entered by any robot. Further, for interaction with the robot the personal
zone is needed. Depending on the individual worker and his/her experience with robots
as well as individual and daily specifically characteristics e.g. inattentiveness and
mental workload, the size of the zones may vary in both directions (y0 : back and forth as
well as x0 : left to right). Further, the distance kept in front of a person and the one behind
a person may vary ðrY Þ. Due to secure feelings, people tend to keep a shorter distance if
something is frontal approaching while coming from behind may surprise them. In
addition, sinistral and dextral people keep different distances to their sides ðrX Þ. Finally,
other human characteristics e.g. visual impairment or transverse posture as well as
process specific requirements have to be met ðHÞ while modeling the personal zone.
Previous work has shown how to integrate the human worker into a multi-agent system
by representing him/her via a human agent which is running on his/her human interface
device e.g. smartphone and can provide the needed parameter information.
156 T. Kirks et al.
3.2 Concept and Implementation
To implement proxemics zones as a plugin for the ROS Navigationstack, a Gaussian
function [12] has been implemented (see equation below) which uses the standard
deviation to dynamically model the curvature of the proxemics as described in
Sect. 3.1. Accordingly, the possibility of shifting ðx0 ; y0 Þ the zones related to the
person’s position is given as well as the option to rotate the proxemics depending on
the personal needs (H). Also, two different areas (intimate hð x; yÞ and personal zone)
can be used, by defining each value of the Gaussian higher than a certain, person
dependence ðlÞ, value as a critical obstacle, such that routing algorithms would not plan
to move the robot through this, inner area. The outer area still is marked as an obstacle
but not as a critical one such that navigation algorithms tend to plan paths around this
area, but do move the robot through this area if necessary.
cos2 H 2 sin2 H 2
þ sin 2H ðxx0 Þ2 þ 2 sin2H
þ sin2H ðxx0 Þðyy0 Þ þ þ cos 2H ðyy0 Þ2
gðx, yÞ ¼ e 2r2
X
2r
Y
4r2
X
4r2
Y
2r2
X
2r
Y
gð x; yÞ; if gð x; yÞ l
hð x; yÞ ¼
253; else
The parameters as well as the position of a person is known by the proxemic_layer
by subscribing to the people message, which contains information of all workers. The
definition of this message, the source code for the proxemic_layer as well as a testing
node and usage description can be found in our public source code repository [15]. An
example of the results of the proxemics layer is shown in Fig. 1.
Fig. 1. Robot navigating around interacting human using proxemics. Left - photo of the
situation. Right - screenshot of the corresponding proxemics and planned route.
Modelling Proxemics for Human-Technology-Interaction 157
4 Conclusion and Outlook
The developed plugin for the ROS Navigationstack fulfills the needs of individual
workers while interacting with mobile robots by taking into account the theory of
proxemics. By modeling the zones via a Gaussian function all necessary characteristics
of a person e.g. sinistrality or visual impairment as well as process specific spatial
requirements. The proxemics plugin ensures that the human worker feels comfortable
while working with robots and is one important step towards human-robot
collaboration.
Future work will include case studies with participants in industrial environments.
Here we will evaluate the feasibility and the adequacy of the use of proxemics distances
with mobile robots. The evaluation will be based on specific use cases and methods like
“Negative Attitude Towards Robots Scale” [7] or User Experience Questionnaire [13].
Furthermore, we want to investigate on spatially extended influences like aerial robots
and therefore need to implement other approaches of proxemics e.g. different shapes.
Additionally, we extend the people message to include more features and abilities of
the workers. A topic we are already working on is to have the robots learn from and
adapt to the human worker’s needs and behaviors over time. In this process, we intend
to alter or sharpen the parameters influencing the proxemics shapes based on method of
machine learning. In this manner, we gain dynamically changing profiles for each
individual, which is considered private data and has to be stored in a save manner [14].
Acknowledgments. This research is supported by the Center of Excellence for Logistics and IT
and has been partially funded by the European Union’s Horizon 2020 research and innovation
program under grant agreement No. 688117 (SafeLog).
References
1. Weber, M.: Mensch-Roboter-Kollaboration (2001). https://www.arbeitswissenschaft.net/
fileadmin/Downloads/Angebote_und_Produkte/Zahlen_Daten_Fakten/ifaa_Zahlen_Daten_
Fakten_MRK.pdf
2. Onnasch, L., Maier, X., Jürgensohn, T.: Mensch-Roboter-Interaktion - Eine Taxonomie für
alle Anwendungsfälle, vol. 1. Fokus, Baua (2016)
3. International Organization for Standardization: ISO 8373:2012-03 Robots and robotic
devices - Vocabulary (2012)
4. Hall, E.: The Hidden Dimension. Doubleday Anchor Books, Doubleday (1966)
5. Hiroi, Y., Ito, A.: Are bigger robots scary? - the relationship between robot size and
psychological threat. In: 2008 IEEE/ASME International Conference on Advanced
Intelligent Mechatronics, pp. 546–551, July 2008
6. Sakai, T., Nakajima, H., Nishimura, D., Uematsu, H., Kitano, Y.: Autonomous mobile robot
system for delivery in hospital. Tech. Rep. Matsushita Electr. Works 53(2), 62–67 (2005)
7. Syrdal, D.S., Dautenhahn, K., Koay, K.L., Walters, M.L.: The negative attitudes towards
robots scale and reactions to robot behaviour in a live human-robot interaction study (2009)
8. Butler, J.T., Agah, A.: Psychological effects of behavior patterns of a mobile personal robot.
Auton. Robots 10(2), 185–202 (2001)
158 T. Kirks et al.
9. Jost, J., Kirks, T., Chapman, S., Rinkenauer, G.: Examining the effects of height, velocity
and emotional representation of a socital transport robot and human factors in human-robot
collaboration. In: Proceedings of the Human-Computer Interaction – INTERACT 2019 17th
IFIP TC 13 International Conference, Paphos, Cyprus, 2–6 September 2019, Part II (2019)
10. Martinez, A., Fernández, E.: Learning ROS for Robotics Programming. Packt Publishing
Ltd., Birmingham (2013)
11. Lu, D.V., Hershberger, D., Smart, W.D.: Layered costmaps for context-sensitive navigation.
In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (2014)
12. Bronstein, I.N., et al.: Taschenbuch der Mathematik, vol. 1. Springer, Heidelberg (2012)
13. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience
questionnaire. In: Proceedings of the 4th Symposium of the Workgroup Human-Computer
Interaction and Usability Engineering of the Austrian Computer Society on HCI and
Usability for Education and Work, USAB 2008, pp. 63–76. Springer, Heidelberg (2008)
14. Kirks, T., Uhlott, T., Jost, J.: The use of blockchain technology for private data handling for
mobile agents in human-technology interaction. In: 2019 IEEE International Conference on
Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation
and Mechatronics (RAM) (2019, in press)
15. Hoose, S.: GitHub repository of the implementation of the proxemic_layer. https://github.
com/iml130/proxemic_layer. Accessed 19 Sept 2019
Category Learning as a Use Case
for Anticipating Individual Human Decision
Making by Intelligent Systems
Marcel Lommerzheim1(&), Sabine Prezenski2, Nele Russwinkel2,
and André Brechmann1
1
Leibniz-Institute for Neurobiology,
Brenneckestrasse 6, 39118 Magdeburg, Germany
{marcel.lommerzheim,andre.brechmann}@lin-magdeburg.de
2
Technische Universität Berlin, Marchstrasse 23, 10587 Berlin, Germany
{sabine.prezenski,nele.russwinkel}@tu-berlin.de
Abstract. Anticipating human behavior requires a model of the rationale how
humans acquire knowledge while solving a problem. The rational aspects of
decision making needs to be taken into consideration for improving computa-
tional models that currently fail to fully explain behavioral data in rule-based
category learning. Compared to reinforcement learning models that assume
gradual learning, cognitive modelling allows to implement selection rules and
instance based learning for decision making to allow more flexible behavior.
Here we use ACT-R to model behavioral data of auditory category learning. By
systematically changing the probabilities of rule selection, capturing individual
preferences of auditory features, we first improve the original model regarding
the average learning curves of subjects. The aim is then to generate a version of
the model that explains learning performance of individual subjects. Neu-
roimaging data will allow to test the predictions of the model by analyzing the
dynamics of activation of brain areas linked to the model processes.
Keywords: ACT-R Category learning Decision making fMRI
1 Introduction
To achieve effective intelligent human systems integration, it is necessary to understand
intentions and to anticipate possible evolving mistakes of humans in order to provide
the right support. When working on problems, humans inevitably learn about causal
relations between objects and actions resulting in explanatory models used as a basis
for further decisions. A prime cognitive function that has been the focus of several
models for understanding human learning is categorization. However, due to the
assumption of a gradual learning process, such models cannot fully explain human
category learning and it has been claimed that different modeling accounts are required
that can capture the process of model building in humans [1, 2]. Accordingly, we have
used the cognitive architecture ACT-R towards understanding the dynamics and
individuality of category learning. ACT-R allows for an algorithmic implementation of
several human core abilities and their specific characteristics such as resource
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 159–164, 2020.
https://doi.org/10.1007/978-3-030-39512-4_25
160 M. Lommerzheim et al.
limitations e.g. in visual attention. A vast number of cognitive tasks have been
implemented that predict human behavior. Especially task models using memory and
learning mechanisms have shown good correspondence to human behavior.
In category learning participants form the categories at different speed with tran-
sitions between learning phases occurring at different points in time. Thus, category
learning is not well represented by group-average learning curves [3]. Reasons for such
variability may lie in differences in the learners’ prior experience or strategies applied
for solving a given category learning task. Moreover, the task design itself may
introduce stimulus features with different salience or require motor responses for which
participants have different preferences and thus increase the variance of resulting
behavioral data. Thus, predicting the dynamics of decision making in individual par-
ticipants requires models than can consider the influence of such factors.
In the present study we analyze behavioral data of an auditory category learning
experiment in which participants must learn the conjunction of two category-defining
rules by trial and error. In the middle of the experiment the assignment of the target
button was changed so that participants had to adapt to the changed situation. We used
ACT-R to model this task based on previous work [4] and expanded the model by
adding different preferences of the sound features in order to better explain the
behavioral results.
2 Methods
2.1 Experiment
55 subjects participated in the experiment that took place inside a 3 T MR scanner (27
female, 28 male, age range between 21 and 30 years, all right handed, with normal
hearing). All subjects gave written informed consent to the study, which was approved
by the ethics committee of the University of Magdeburg, Germany.
A set of 160 different frequency-modulated tones served as stimuli for the cate-
gorization task. The tones had five different features, with one of two possible cate-
gorical values: duration (short, 400 ms, vs. long, 800 ms), direction of frequency
modulation (rising vs. falling), volume (low, 76–81 dB, vs. high, 86–91 dB), fre-
quency range (five low frequencies, 500–831 Hz, vs. five high frequencies, 1630–
2639 Hz), and speed of modulation (slow, 0.25 octaves/s, vs. fast, 0.5 octaves/s). The
task relevant features were the direction of frequency modulation and sound duration,
resulting in four tone categories: short/rising, short/falling, long/rising, and long/falling.
For each participant, one of these categories constituted the target sounds (25%), while
the other three categories served as non-targets (75%).
As feedback stimuli, we used naturally spoken utterances (e.g., ja, “yes”; nein,
“no”) as well as one time-out utterance (zu spät, “too late”) taken from the evaluated
prosodic corpus MOTI [5, 6].
The experiment lasted about 33 min in which a large variety of frequency-
modulated tones were presented in 240 trials in pseudo-randomized order and with a
jittered inter-trial interval of six, eight, or ten seconds. The participants were instructed
to indicate via button-press whether they considered the tone in each trial to be a target
Category Learning as a Use Case for Anticipating Individual Human Decision Making 161
(right index finger) or a non-target (right middle finger). They were not informed about
the target category but had to learn by trial and error. Correct responses were followed
by positive feedback, incorrect responses by negative feedback. If participants failed to
respond within two seconds following the onset of the tone, the time-out feedback was
presented.
After 120 trials, a break of 20 s was introduced. From the next trial on the con-
tingencies were reversed such that the target stimulus required a push of the right
instead of the left button. The participants were informed in advance about a resting
period after finishing the first half of the experiment but they were not told about the
contingency reversal.
2.2 Model
Modelling Method. The experimental task was modeled with the ACT-R framework,
a cognitive architecture that provides a set of different cognitive functionalities called
modules that interact with each other via interfaces called buffers. We will first describe
the general modelling approach as recently implemented [2] and then outline the
changes that we made to include individual preferences in tone features.
Our model uses the motor, the declarative, the imaginal, the goal, the aural, and the
procedural module of ACT-R. The motor module is responsible for the motor output of
the model. The declarative module represents the long-term memory of ACT-R in
which all representation units (chunks) are stored and retrieved. The imaginal module
acts as the working memory that holds and modifies the current problem state. The goal
module represents the control states. The aural module is responsible for processing
auditory information. The procedural module is central for ACT-R as it coordinates the
other processing units by selecting production rules (representing procedural knowl-
edge) based on the current state of the modules.
Original Task Model. In order to specify a model in ACT-R the production rules and
the chunks (representing background knowledge) need to be defined. Chunks are the
smallest units of information and can be exchanged between Buffers. Production rules
or productions have a condition and an action part. They are selected sequentially.
Thus, only one production can be selected simultaneously. They are only selected if the
condition part of the production matches the state of the modules. Subsequently, the
action part modifies the chunks in the modules. In the case that more than one pro-
duction matches the state of the modules, a subsymbolic production selection process
chooses which production is selected. Another subsymbolic process of ACT-R is the
activation of a chunk. The activation of a chunk determines whether a chunk can be
retrieved from memory and how long this takes. A chunk’s activation value is deter-
mined by its past usefulness (base-level activation), its relevance in the current context
(associative activation) and a noise parameter.
The model is equipped with two different types of chunks: strategy chunks and
control chunks. Strategy chunks represent the strategies in form of examples of feature-
value pairs (i.e. duration is long and volume is loud) and responses (i.e. left or right
button). These strategy chunks are stored in and retrieved from long-term memory
162 M. Lommerzheim et al.
(declarative module). The currently pursued strategy is stored in working memory
(imaginal module). A strategy chunk contains the following information: Which feature
(s) (i.e. volume) and what corresponding categorical value(s) (i.e. loud) are relevant,
the proposed response, and the degree of complexity. The degree of complexity
determines whether the model attends to just one feature (one-feature strategy) or to
two features (two-feature strategy). Furthermore, it contains a metacognitive process
that includes noting whether and how often a strategy was successful. All of the
different possible strategy chunks are stored in declarative memory right from the start.
Control chunks represent other metacognitive aspects of the model. They are stored
in the goal buffer of the model. They include first the level of rule complexity used,
second whether or not a long-time successful strategy caused an error, and third
whether external changes occurred that require a new search for a strategy.
The production rules used for defining the task are described now in detail with the
specific names in parentheses. When a tone is presented to the model it enters the aural
location buffer (listen). Subsequently it is encoded in the aural buffer (encode). This
results in a chunk with all audio information necessary (duration, direction of pitch
change, volume, and frequency range) stored in the aural buffer. This audio chunk is
then compared to the strategy chunk in the imaginal buffer (compare). If the specific
features of the strategy chunk match those of the audio chunk, the response is chosen
according to the strategy chunk (react-same), if not, the opposite response is selected
(react-different). The model then listens to the feedback and holds it in the aural-
location buffer (listen-feedback). Subsequently it is encoded in the aural buffer (encode-
feedback). In case of positive feedback, the current strategy is maintained and the
count-slot is updated (feedback-correct). In case of negative feedback, the strategy
usually is altered depending on previous experiences (feedback-wrong).
Next we will describe how this strategy updating is implemented. In case that a one-
feature strategy fails in the first attempt, a different motor response is selected for this
feature-value pair. Otherwise, the feature-value pair is changed while the response is
retained. When a one-feature strategy was successful often and then fails once, it is not
directly exchanged, but re-evaluated and it is noted that the strategy has caused an
error. In two circumstances a switch from a one-feature strategy to a two-feature
strategy occurs: Either no successful one-feature strategy is left or an often successful
one-feature strategy fails repeatedly. For switches of two-feature strategies the fol-
lowing rules apply: If the first attempt of a two-feature strategy fails, any other two-
feature strategy is used. In the case a two-feature strategy that was initially successful
fails, a new strategy that retains one of the feature-value pairs and the response is
transferred to the imaginal buffer. When an environmental change is detected, an often
successful two-feature strategy will fail and a retrieval of another two-feature strategy
takes place.
Individual Preference Model. In order to improve the models performance and to
better fit the human experimental data, we included preferences to special strategies.
While the probability of selecting a strategy in the initial model was the same, we
changed the model by increasing the probability (by changing the utility of the pro-
duction) of being selected for different strategy chunks. We did so by setting the
activation values of certain strategy chunks to 1.0. In one model we increased the
Category Learning as a Use Case for Anticipating Individual Human Decision Making 163
activation values of strategies that use duration and/or direction of frequency modu-
lation as acoustic features. Thus, the strategy chunks that use these sound features
receive a higher probability of being retrieved as active strategy. In another model the
activation level of strategies using an irrelevant feature, i.e. volume was increased.
3 Results and Discussion
Fig. 1. Average performance of the participants and the two ACT-R models. Error bars depict
standard errors.
Figure 1 depicts the performance of the participants and three different model versions:
The initial model with without feature preference, a model with preference for the
relevant sound features duration and direction and a model with preference for the
irrelevant sound feature volume.
As expected the model with increased attention to volume performs even worse
than the initial model. In contrast, the model with increased attention to duration and
direction of frequency modulation performs nearly as good as the participants in blocks
4–6. A qualitative difference between the models and participants performance can be
seen after the change of response-contingencies (after block 6). While the participants
adopt rather fast to the change and increase performance already in block 8, all models
require more time to recover their performance.
Thus, while introducing preferences for task relevant sound features improved the
performance of the model, it performs still significantly different to the participants in
most blocks. There are two things that could be implemented to further improve it:
Firstly, response preferences could be included so that one response button is favored
over the other. This may improve the initial performance of the model. Secondly, the
164 M. Lommerzheim et al.
way the model learns after the reversal can be changed. Instead of randomly switching
to another strategy with a different feature-value pair as target tone, the model could
first try out to choose a different response to the same target tone. This is a necessary
next step to generate a model that can explain the average learning curves of humans.
Then certain simulations of the model can be run, that generated the exact same
performance as individual subjects during the learning experiment. Since we acquired
fMRI data during the participants’ learning, the third step would then include to
compare the activity of the different ACT-R modules to the neural activation in the
brain regions that have been suggested as corresponding brain regions [7].
Acknowledgments. The work was supported by EU-EFRE (ZS/2017/10/88783) and by DFG
BR 2267/9-1.
References
1. Smith, J., Ell, S.: One giant leap for categorizers: one small step for categorization theory.
PLoS One 10, e0137334 (2015)
2. Jarvers, C., Brosch, T., Brechmann, A., Woldeit, M.L., Schulz, A.L., Ohl, F.W.,
Lommerzheim, M., Neumann, H.: Reversal learning in humans and gerbils: dynamic control
network facilitates learning. Front. Neurosci. 10, 535 (2016)
3. Gallistel, C.R., Fairhurst, S., Balsam, P.: The learning curve: implications of a quantitative
analysis. Proc. Natl. Acad. Sci. 101, 13124–13131 (2004)
4. Prezenski, S., Brechmann, A., Wolff, S., Russwinkel, N.: A cognitive modeling approach to
strategy formation in dynamic decision making. Front. Psychol. 8, 1335 (2017)
5. Wolff, S., Brechmann, A.: MOTI: a motivational prosody corpus for speech-based tutorial
systems. In: Proceedings of the 10th ITG Conference on Speech Communication, pp. 1–4
(2012)
6. Wolff, S., Brechmann, A.: Carrot and stick 2.0: the benefits of natural and motivational
prosody in computer-assisted learning. Comput. Hum. Behav. 43, 76–84 (2015)
7. Borst, J.P., Anderson, J.R.: A step-by-step tutorial on using the cognitive architecture ACT-R
in combination with fMRI data. J. Math. Psychol. 76, 94–103 (2017)
System Architecture of a Human Biosensing
and Monitoring Suite with Adaptive Task
Allocation
Brandon Cuffie(&) and Lucas Stephane(&)
Florida Institute of Technology, Melbourne, FL 32901, USA
[email protected], [email protected]
Abstract. Future space missions require the integration of enhanced physical
and physiological monitoring systems for supporting missions of both short and
long duration. This human biosensing and monitoring suite will specify and
implement a proof of concept of a non-invasive sensor system for crew moni-
toring in space missions, by integrating cost-cutting equipment. HuBAM is
composed of: (1) astronaut wearable suit(s) (AWS), combined with (2) an
Automated Equipping station (AES) that can be deployed as needed within
space habitats and/or space transportation vehicles. The design and evaluation
methods employed will be systematically mapped onto each Design Thinking
stage for this first iteration. Based on results and findings, the paper will deliver
recommendations and tasks for further developments, and future iterations using
the Design Thinking process.
Keywords: Design Thinking Biosensing Monitoring Usability Human-
system integration
1 Introduction
As astronauts explore the deep space environment, where support from earth is not
feasible, mission success will depend on keeping the crew healthy and performing at
high levels, while making mission specific tasks easier, adaptable and optimal for the
crew [1]. This paper provides a system architecture design for a human biosensing and
monitoring suite (HuBAM) with adaptive task optimization using the Design Thinking
approach. This human biosensing and monitoring suite will specify and implement a
proof of concept of a non-invasive sensor system for crew monitoring in space mis-
sions, by integrating cost-cutting equipment. HuBAM is composed of an astronaut
wearable suit(s) (AWS) with physical and physiological sensors, integrated with an
Automated Equipping station (AES) that can be deployed as needed within space
habitats and/or space transportation vehicles. The astronaut-centered AWS leverages
state-of-the-art requirements inclusive of physical and physiological sensors. The AES
is leveraging requirements to equip the astronaut with his extra-vehicular activity
(EVA) suit, hardware and software solutions for both sensing and managing sensor
information and automated equipping tools and accessories necessary for specific EVA
missions.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 165–171, 2020.
https://doi.org/10.1007/978-3-030-39512-4_26
166 B. Cuffie and L. Stephane
Using the iterative and incremental five stages of Design Thinking (i.e. (1) problem
statement, (2) need-finding/empathizing, (3) ideation, (4) prototyping, (5) evaluation)
[2] to the design of the suite, will allow to systematically eliciting useful knowledge for
optimal design requirements and formative evaluation feedback from astronauts and
space experts. The design and evaluation methods employed will be systematically
mapped onto each Design Thinking stage (e.g. interviews with astronauts mapped onto
need-finding, design rationale and scenario-based design mapped onto ideation, fea-
sibility related to detailed technical hardware and software tools mapped onto proto-
typing, and testing protocols mapped onto the evaluation stage) for this first iteration.
2 Design Thinking: First Iteration
(See Fig. 1).
Problem
Evaluation Need inding
Prototpe Ideation
Fig. 1. Design Thinking cycle applied to the first iteration. This shows that this iteration is
cyclic.
2.1 Problem Statement
For this first iteration, the aim is to design the HuBAM process that will take place for
the astronaut as he goes through the phases of monitoring to mission equipping, in a
human-centered manner.
In the development of the process, we had to evaluate possible automated alter-
natives for the task-specific equipping of astronauts for the completion of a dynami-
cally assignable, planetary task. Within the context of this study, the planetary task is
defined as a nominal Mars rover support and maintenance scenario in which the
mission specialist must monitor and assist in the deployment of an automated rover.
The specialist will also support the rover in the collection of surface samples, and
monitor and assist in the rover’s recovery.
System Architecture of a Human Biosensing and Monitoring Suite 167
2.2 Need-Finding/Empathizing
In exploring the problem, it first required an intuitive understanding of the topic of
spacesuits. A combination of very in depth research, task analysis and state of the art
questions options and criteria (QOC) [3], shown in Fig. 2, achieved such. It also
included many articles, documentaries and even a manual on operating a spacesuit,
which aided in understanding use cases and scenarios about the problem. As part of this
type of participatory design, we conducted informal and formal interviews with subject
matter experts (SMEs) such as a space scientist, a biomedical expert and a retired
astronaut. The astronaut gave a precise account of his time in space and his recom-
mendations to improve his spacesuit.
Fig. 2. Design Space Analysis for Mission Based Task Allocation. This is a QOC showing the
various requirements needed for successful working concept.
2.3 Ideation
“What is a possible solution?” “What to test?” These crucial questions were answered
based on the synthesis of empathizing and the problem question. Sequence based
diagrams were used to depict and describe the solutions [4] (Fig. 3).
168 B. Cuffie and L. Stephane
Fig. 3. Sequence based diagrams for the mission “Rover Maintenance and Retrieval” (Adapted)
For modeling and simulation, we used Signavio Process Modeling [5]. Unlike
physical modeling, such as making a scale copy of the AES, this computer-based tool,
provided an environment while they are running, and having the possibility to view
them in 2D or 3D. The ability to analyze the model as it runs sets simulation modeling
apart from other methods, such as those using Excel or linear programming. This
model captured more detail than an analytical model, providing increased accuracy and
more precise forecasting. It was less expensive, took less time than real experiments
with real assets, and efficiently represented uncertainty in operations’ time and out-
come. As such, we compressed or expanded time in the simulation to speed up or slow
down the process in the AES.
2.4 Prototype
Heads up Displays (HUD), in the aerospace field, improves human perception in the
control of complex machines and refines skills in the training of astronauts for extra-
vehicular activity (EVA) and robotics operations [6]. They participate and contribute to
the astronaut’s actions and workload in space, and will be a key component in the
equipping of the astronaut to the procession to the specified mission. All necessary
information required for the smooth operation, such as mission information and mis-
sion tools will be displayed. Information from physical and physiological sensors such
as electrocardiography (EKG), heart rate variability (HRV), maximum oxygen con-
sumption (VO2), pulse oximetry (POx), electrodermal activity (EDA), electroen-
cephalography (EEG), and electromyography (EMG) will also be displayed..
The software program Adobe XD was used to create the prototype (Fig. 4).
System Architecture of a Human Biosensing and Monitoring Suite 169
Fig. 4. Selected screenshots of the prototype for the HUD at different stages: starting screen
(left), physiological monitor screen (top right) and equipped tools (bottom right).
2.5 Evaluation
Among the various methods available for usability assessment, and those that were
shortlisted, the Self-Assessment Manikin (SAM) and the Cognitive Walkthrough
methods evaluated the prototype. The SAM nine-pointer scale was used to directly
measure the pleasure, arousal, and dominance in response to, and related to the use and
the procedure process of the prototype HUD view. Figure 5 below shows the SAM
figure with the respective scales. For the cognitive walkthrough, we had limited
availability of expert users with knowledge and experience of the spacesuits used on
previous missions whose feedback carries weight. We followed through each pre-
defined tasks and procedure that the user is expected to go through and identify specific
patterns or required skills that could emerge as being critical.
All the participants had normal vision. The participants had never seen the video
before. The experiment was conducted in the evening in a closed room, and all were
placed in a comfortable chair. Whenever they felt the need, all participants had access
to the prototype and played with it multiple times.
170 B. Cuffie and L. Stephane
Fig. 5. SAM nine-point scale used to assess pleasure, arousal and control of the prototype
3 Conclusion and Future Work
It should be noted that a number of the features tested were done in the context of a fully
functioning automated system but tested using a laptop-based simulation of a software
prototype. A such, it was not possible to test the usability of interactive gestures, auto-
mated selection of mission tasks or displays for equipment located on the suit. Analysis
of the process of astronaut equipping by way of a sequence diagram (Fig. 2) illustrates
the inherent operational complexity of the task. From a design perspective, the cognitive
walkthrough and self-assessment manikin demonstrate the learnability and user expe-
rience characteristics of our prototype. Findings concerning the cognitive walkthrough
suggested required improvements in system observability. Per the internal focus group
findings of the SAM, the initial prototype did, however, support pleasurable and arousing
usage while promoting the perception that the user remained in control of the sequence.
Future goals of this project include the refinement of the HUD interface, the
automation of mission selection, the addition of salient sequence progress displays, the
addition of user position and orientation demonstration videos, and the provision of an
augmented reality display which depicts the location of equipment in real-time prior to
the astronaut’s departure from the station. These improvements should maintain
acceptable levels of excitement and arousal and promote the perception of user control
in the equipping process.
System Architecture of a Human Biosensing and Monitoring Suite 171
Acknowledgements. This project was supported by the National Aeronautics and Space
Administration, through the University of Central Florida’s NASA Florida Space Grant Con-
sortium, and SPACE FLORIDA. The authors wish to acknowledge and thank in particularly
Captain Winston E. Scott, USN (ret.) for his astronautics expertise as a NASA astronaut.
References
1. Furr, P.A., Monson, C.B., Santoro, R.L.: Extravehicular activities limitations study. National
Aeronautics and Space Administration (1988)
2. Meinl, C., Leifer, L.: Design Thinking: Understand, Improve, Apply. Springer, Heidelberg
(2011)
3. MacLean, A., Young, R.M., Bellotti, V.M., Moran, T.P.: Questions, options and criteria:
elements of design space analysis. Hum. Comput. Interact. 6, 201–250 (1991)
4. Cuffie, B., Bernard, T., Mehta, Y., Kaya, M., Scott, W.E., Stephane, L.: Proposed architecture
of a sensory enhanced suit for space applications. In: 2018 AIAA SPACE and Astronautics
Forum and Exposition (2018)
5. Signavio Process Manager: User Guide (12.6.0) (2018)
6. Aukstakalnis, S.: Practical Augmented Reality: A Guide to the Technologies, Applications,
and Human Factors for AR and VR. Addison-Wesley, Boston (2016)
7. Wharton, C., Rieman, J., Lewis, C., Polson, P.: The cognitive walkthrough method: a
practitioner’s guide. In: Nielson, J., Mack, R.L. (eds.) Usability Inspection Methods, pp. 105–
140. Wiley, Canada (1994)
The Role of Artificial Intelligence
in Contemporary Medicine
Larisa Hambardzumyan, Viktoria Ter-Sargisova(&),
and Aleksandr Baghramyan
Medical University named after St. Tereza, 54a Mashtots Ave,
0010 Yerevan, Armenia
[email protected]
Abstract. The use of artificial intelligence (AI) in the sphere of medicine is
aimed at primary diagnostics of human body and finding treatment approaches
and solutions. While some scientists consider that AI is able to detect many
complicated diseases at early stages thus saving lives of many patients, human
reasoning methods in the field of medicine cannot be fully replaced by
machines. This paper aims to find out the optimal balance between human and
AI participation in diagnostic and treatment decision processes. With this pur-
pose a simulation of diseases based on 100 real-case anonymous data was
generated to be diagnosed through independent and simultaneous AI and human
analysis. As a result, the quantitative and qualitative outcomes were summarized
and used to suggest a solution to combine AI and human participation, aimed at
elimination disadvantages and most effective use of advantages of both sides.
Keywords: Artificial intelligence Machine Learning Decision tree Human
factor Medicine Diagnostics Decision Management Automation Big data
1 Introduction
With the implementation of elements of computation to all aspects of our lives, in the
era of globalization it is becoming crucial to process large volumes of data and
information in a more efficient and resultative manner than it has been done before,
consequently technologies underlying the maintenance of AI (Speech Recognition,
Computer Vision, Media Processing, ML (Machine Learning), Decision Management,
Natural Speech/Language Synthesis, Full Automation) are becoming irreplaceable. The
field of medicine and healthcare is one of the most sensitive and challenging spheres
where the practitioners have to keep up will all the latest developments, in particular
those in the sphere of technological solutions.
The ability to make reasonable assumptions by analyzing a given set of data is a
fundamental attribute of intelligence, which, in turn, is an irreplaceable feature needed
to diagnose, treat and even prevent a wide range of diseases.
The purpose of this paper is to analyze the behavior of human medical practitioners
and artificial intelligence in artificially created terms where diagnosis and consecutive
treatment prescription are required, to evaluate the level of possible treats and benefits
of AI implementation in real hospital conditions [4].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 172–176, 2020.
https://doi.org/10.1007/978-3-030-39512-4_27
The Role of Artificial Intelligence in Contemporary Medicine 173
2 Methodology
With the purpose of performing analysis of AI and human behavior in terms of
diagnostics and treatment of potential patients, several medical archives of clinical case
reports amounting to more than 3,000 medical files were used for random selection of
150 clinical files for further research.
Three trial groups took part in the research. Group 1 consisted of 10 medical
specialists representing different areas of medicine (2 specialists per each of 5 selected
spheres: pediatrics, obstetrics, general surgery, cardiology, oncology), Group 2 repre-
sented the computer software controlled by 1 computer software specialist proficient in
AI performance control, and Group 3 consisted of 5 medical specialists representing
same 5 spheres of medicine as in Group 1, each of them working in collaboration with
the computer software applied in Group 2. Each of the 3 groups were given 50 clinical
files to work on, within a time period which was recorded.
The purpose of this research was to identify and evaluate the impact of usage of the
computer software on the efficiency of medical diagnosis [5].
The computer software was developed to include diagnostic models of over 50
diseases, starting from easily diagnosed ones as well as such diseases as tuberculosis
and depression which can be also misdiagnosed. The software enabled the operating
specialist to analyze and classify all available symptoms to give an exact diagnosis. The
process can be represented as a flowchart in a form of a decision tree [1, 2].
Sample decision tree is presented below (Fig. 1):
Fig. 1. The decision tree for hepatitis B predictions [6]
The whole process of analysis by all three groups was performed based on the real-
case patient symptoms through paper evidence only, with no involvement of real
patients. It is important to mention that the selected medical cases had high level of
complication to be able to consider more factors and functionalities of each tested
group and evaluate the results from different angles.
174 L. Hambardzumyan et al.
The major task of the tested groups was to form a collegial decision on analysis for
each patient case, followed by the respective treatment decision. The parameters taken
into consideration were as follows: number of correct diagnoses, number of correct
treatment solutions, and average duration of a single patient case analysis [3].
3 Findings
The comparison of diagnosis and treatment simulation results have shown that the
collaborative work of human medical practitioners and artificial intelligence has con-
tributed the most to the number of correct diagnosis cases in trial groups, as well as to
improved results of selected treatment options (Fig. 2).
Fig. 2. AI impact on the efficiency of diagnosis and treatment
The figure above shows that Group 1 with the participation of a group of medical
practitioners has shown the lowest results by number of correct diagnoses and number
of correct treatments, whereas the average duration of a single analysis was the longest
across three groups. This shows that taken relatively low ratios (although with a
marginal difference) of decision correctness, the timeframe of pure human activity with
no AI participation can considerably affect the treatment outcomes in cases when speed
is crucial for saving the life of a patient.
The results of Group 2 testing (AI participation only) have shown average results
by number of correct diagnoses and number of correct treatments, whereas the average
duration of a single analysis was the shortest across three groups with a considerable
difference (8 min in Group 2 vs. 38 and 26 min in Groups 1 and 3 respectively).
The Role of Artificial Intelligence in Contemporary Medicine 175
The results of Group 3 testing (collaboration of AI and human medical practitioners
only) have shown the highest results by number of correct diagnoses and number of
correct treatments, whereas the average duration of a single analysis was average,
though closer in duration to the results of Group 1.
Taking into consideration that the rate of correct diagnosis across 3 groups varied
from 84% to 94%, whereas the rate of correct treatment varied from 80% to 92%, the
role of AI and AI/Human collaboration in Groups 2 and 3 respectively was of medium
significance, while in case of the speed of decision-making the role of AI component
was major.
While relating the percentages of correct diagnoses and treatment decisions to the
average duration of decision-making process by groups, the results have shown that the
highest efficiency has been achieved in Group 3 (Fig. 3).
Fig. 3. Diagnostic efficiency
This finding supports the suggested thesis stating that the collaboration between
human medical practitioners and computer software representing AI can help to
achieve the highest patient saving rate due to quicker and more accurate decisions.
4 Conclusion
Based on the research results presented above, it can be concluded that human medical
workers have an important role in the situations where an artificial intelligence device
cannot substitute human advice and charity which can help a medical practitioner to
relieve patient’s anxiety or depression. The machine is also unable to cope with disease
cases with new components or mutations which have never occurred before, thus AI
176 L. Hambardzumyan et al.
cannot suggest new ways of treatment. At the same time AI helps to eliminate human
factor which might lead to routine errors, as well as it can help to lead the diagnostic
processes to a new level of speed and automation, whereas speed and correctness of
considering all the underlying factors can be often decisive to save a patient’s life. AI
“doctors” are also capable of constant learning of new medical knowledge, experience
and diagnosis process. Computer software can have a big database consisting of tens of
millions of clinical cases, designed to be frequently updated with new cases and
symptoms matrices.
Therefore, this paper suggests that further analysis needs to be performed aimed at
creating more detailed matrices of a synergy between human and artificial intelligence.
References
1. Murthy, S.K.: Automatic construction of decision trees from data: a multi-disciplinary survey.
Siemens Corporate Research, Princeton, USA (n.d.). https://cs.nyu.edu/*roweis/csc2515-
2006/readings/murthy_dt.pdf
2. Quinlan, J.R.: Industry of Decision Trees. Kluwer Academic Publishers (1985). https://link.
springer.com/content/pdf/10.1007/BF00116251.pdf
3. Henriksen, K., Brady, J.: The pursuit of better diagnostic performance: a human factors
perspective. BMJ Qual. Saf. (2013). https://qualitysafety.bmj.com/content/qhc/22/Suppl_2/
ii1.full.pdf
4. Karimi, K., Hamilton, H.J.: Generation and interpretation of temporal decision rules.
Department of Computer Science, University of Regina, Regina, Saskatchewan, Canada (n.
d.). https://arxiv.org/pdf/1004.3334.pdf
5. Quinlan, J.R.: Semi-autonomous acquisition of pattern-based knowledge. Baser Department
of Computer Science, University of Sydney (n.d.). http://citeseerx.ist.psu.edu/viewdoc/
download?doi=10.1.1.472.7028&rep=rep1&type=pdf
6. Albu, A.: From logical inference to decision trees in medical diagnosis. In: E-Health and
Bioengineering Conference (EHB), pp. 65–68 (2017)
Improving Policy-Capturing with Active
Learning for Real-Time Decision Support
Bénédicte Chatelais1(&), Daniel Lafond1, Alexandre Hains2,
and Christian Gagné2
1
Thales Research and Technology Canada, Quebec City, Canada
{Benedicte.Chatelais,
Daniel.Lafond}@ca.thalesgroup.com
2
Dép. Génie électrique et Génie Informatique, Université Laval,
Quebec City, Canada
[email protected],
[email protected]
Abstract. Thales Research and Technology Canada is developing a decision
support system consisting in multiple classification models trained simultane-
ously online to capture experts’ decision policies based on their previous
decisions. The system learns decision patterns from examples annotated by a
human expert during a training phase of knowledge capture. Because of the
small volume of labeled data, we investigated a machine learning technique
called active learning that copes with the dilemma of learning with minimal
resources and aims at requesting the most informative samples in a pool given
the current models. The current study evaluates the impact of using active
learning over an uninformed strategy (e.g., random sampling) in the context of
policy capturing to reduce the annotation cost during the knowledge capture
phase. This work shows that active learning has potential over random sampling
for capturing human decision policies with minimal amount of examples and for
reducing annotation cost significantly.
Keywords: Active learning Machine learning Policy capturing Decision
support Operator assistance Human-Machine collaboration Judgmental
analysis Expert system Cognitive modeling Classification
1 Introduction
Policy capturing refers to methods that use statistical models or machine learning
algorithms to model the decision-making patterns of humans [1]. As experts might not
be aware of their decision rules, or might not be able to fully explain in words their
professional domain knowledge [2, 3], the use of policy capturing systems is promising
for improving human-machine teaming. Thales Research and Technology Canada has
developed a cognitive modeling and decision support system called Cognitive Shadow,
based on artificial intelligence techniques. The solution aims at learning decision
patterns of experts in real-time in order to raise advisory warnings to notify operators in
case of mismatch between the expected decision inferred by models and the actual
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 177–182, 2020.
https://doi.org/10.1007/978-3-030-39512-4_28
178 B. Chatelais et al.
observed decision. The goal is to prevent human errors that could happen under
stressful or time pressure situations, mental overload, fatigue, stress or distraction. The
Cognitive Shadow consists in multiple classification models trained simultaneously to
capture experts’ decision policies based on their previous decisions. As decision pat-
terns are variable from one human to another, there is not one specific model that is
suited for all experts. Therefore, the key point of the Cognitive Shadow lies in training
a pool of different algorithms, allowing for a dynamic selection of the best model over
time [4–6]. Prior to real-time judgment analysis and actual decision support, the
Cognitive Shadow must learn decision patterns from a few examples annotated by
human experts during a short one-time training phase of knowledge capture. Impor-
tantly, the system does not require learning from any pre-existing dataset or ground
truth.
The current study aims at supporting operators working in maritime patrol aircrafts
(MPA) for classifying vessels into three categories (allied, neutral, sensitive) given a
list of variables of interest. The goal of the project is to improve human-machine
collaboration by taking advantage of the judgmental bootstrapping effect that
demonstrated a reduction in human error and, more interestingly, an increase in the
individual performances of the human-machine team [7, 8]. In order for Cognitive
Shadow to raise reliable warnings, the algorithms must be trained with data having a
good coverage of the domain of interest. Since each individual cannot annotate data
from every possible situation, they should at least annotate few examples from all
classes in order to pre-train the models before allowing the Cognitive Shadow system
to actively display warnings. Once trained during the knowledge capture phase, the
models keep on learning and updating over time for every new decision. Hundreds of
instances represent a large volume of data from the users’ perspective who has to
annotate them manually (time-consuming), while it is not sufficient for training
machine learning algorithms efficiently. Thus, it is essential to carefully select the most
informative data from which the models will learn. Currently, the system presents to
the operator a set of randomly sampled instances without any guarantee that it was
informative enough to cover the whole space of interest (passive learning).
Unlike passive learning using random examples, Active Learning (AL) can be used
to reduce the quantity of required data and improve model accuracy by automatically
selecting the most informative instances to be labeled. For instance, AL can be used
during the initial knowledge capture phase to allow algorithms to request data from
which they want to learn based on past queries and responses (labels) from those
queries. The current study aims at evaluating the impact of such use of AL over random
sampling during the initial knowledge capture in the context of automatic policy
capturing. With this approach, we aim at minimizing the labeling time required from
the experts using AL techniques, and as such improving Cognitive Shadow’s efficiency
at modelling an expert’s decision policy, while maximizing the models accuracy. The
experiment of the paper is conducted using a fixed number of training examples, but a
reduction in the number of instances presented during the knowledge capture phase can
also be considered. The main goal is to accelerate the learning process so that the
Cognitive Shadow system could be useful for decision support purposes as quick as
possible – when a satisfactory reliability level is reached – and selecting a good set of
Improving Policy-Capturing with Active Learning 179
instances to label that are covering the various situations that can be encountered,
including rare instances seldom observed in real operations.
Section 2 of this paper describes the methods used for applying AL in this context
and the datasets used for this study. Section 3 presents the results of two experiments.
Section 4 summarizes the main conclusions and proposes future research perspectives.
2 Method
2.1 Active Learning
AL refers to semi-supervised machine learning methods aiming at actively selecting the
input instance from which the model may learn the most. The instance is selected
among a pool of unlabeled data based on a measure of utility in order to maximize the
learning phase by focusing the training on the most informative data. The chosen
instance is presented to an oracle to be annotated; in our use-case, the oracle is the
human expert. The model(s) will learn from this new-labeled data and will then request
a new instance from the unlabeled pool to improve its training.
For the present purposes, two different instance selection methods from the liter-
ature were applicable for the project: pool-based sampling and stream-based sampling
[9]. In short, pool-based sampling evaluates and ranks the entire instances in a large
collection of unlabeled data, also called the pool, before selecting the most informative
one. Stream-based sampling scans sequentially every data in the pool one-by-one and
decides to either query or discard it based on its estimated information value. Although
pool-based sampling is computationally more expensive given the ranking phase
(which needs continuous reassessment as models learn), we focused on this method in
our study to allow the selection of the most informative data in the pool.
Different query strategies (or heuristics) are used in the literature to evaluate the
information level of instances in a pool [9]. The query strategy is a function that
measures a score for every instance in order to select the most informative one (i.e., the
one involving the most uncertainty). As the Cognitive Shadow consists in training
simultaneously different models on the same data because expert’s labeling time is
limited, the selected instances have to take into consideration all models and should be
informative for all, or at least the majority of them (depending of the chosen strategy).
The query-by-committee strategy allows each model in a committee to vote on the
labeling of instances. The one that generates the biggest disagreement is selected as the
query of interest. Here we used consensus entropy sampling consisting in calculating
the average of the class probabilities for each model of the committee. The instance
with the largest entropy is then selected and presented to the operator [9].
2.2 Datasets
We evaluated the impact of AL in two phases. The first phase aims at evaluating AL
strategies on a benchmark dataset. The goal is to evaluate technical choices, to
demonstrate the advantage of AL over random sampling independently of the classi-
fication task of interest, and to validate if we can achieve performances equivalent to
180 B. Chatelais et al.
the literature with a reduced number of queries. The second phase consists in validating
conclusions of the first phase on a synthetic dataset representative of the classification
problem of interest simulated on the high-fidelity simulator called Thales Airborne
Surveillance Mission System (AMASCOS).
Benchmark Dataset: Mushroom dataset is an openly available benchmark dataset1
for which the task consists in classifying mushrooms as poisonous or not. The dataset is
composed of 8124 instances characterized by 22 variables, taken from 23 different
species. We used one-hot encoding on categorical features, then split the dataset into
train and test subsets.
Experimental Dataset: The experimental dataset refers to the AMASCOS maritime
patrol task (radar contact classification by a tactical coordinator). Surface vessels are
characterized by fourteen features and the goal is to classify each vessel as “Allied”,
“Neutral”, or “Suspect”. The features are a mix of categorical and numerical attributes,
which include: platform type, speed, speed change, stationary, length, friend list, AIS
(on-off), nearest track distance, cluster size, sea/coastal proximity, interception, sea lane
deviation, heading change and nationality. We used one-hot encoding on the cate-
gorical variables. We generated a synthetic dataset of AMASCOS representative of all
possible combinations of features in which unrealistic combinations were excluded. In
total, this represents a pool of more than 3 million possible examples. For experimental
purposes, we simulated an expert policy (based on domain expert feedbacks) and
generated the labels for each vessel. Class distribution was unbalanced (approximately
5% of vessels classified as allied, 12% as neutral, and 83% as sensitive). Because of the
computational time necessary for ranking every instance in the pool and selecting the
most informative one, we reduced the size of the pool by selecting a subset of 500 000
combinations, with an equally balanced representation of each class.
2.3 Metrics
To evaluate the advantage in using AL over passive learning (i.e., the random sampling
baseline), we calculated the accuracy of the committee on the test dataset (a holdout
labeled dataset not used for training or optimizing models) after every training iteration.
Practically, it allows comparing the two sampling strategies and the number of queries
required to reach the target accuracy.
2.4 Training Process
The committee is composed of six heterogeneous classifiers (models implemented into
the Cognitive Shadow aiming at learning the experts’ decision pattern): a decision tree,
k-nearest neighbors (KNN), support vector classifier (SVC), a neural network (MLP), a
naïve Bayes classifier and logistic regression. Since it is impractical to start AL without
data from all classes, we are using a seed sample (n = 4 for the benchmark dataset and
n = 30 for AMASCOS dataset) with balanced stratification of every class. Data are
1
https://archive.ics.uci.edu/ml/datasets/mushroom.
Improving Policy-Capturing with Active Learning 181
selected randomly in the pool. Following every querying phase, queried instances are
removed from the unlabeled pool and are added to the labeled training dataset on which
models are learning. Because the labeled pool is relatively small, models are trained
and optimized using k-fold cross validation (k = 2 for the benchmark dataset and
k = 10 for AMASCOS dataset). We evaluate models’ performance between active and
passive learning by measuring the accuracy on the test dataset. The knowledge capture
ends after a predefined number of queried instances.
3 Results
Figure 1 illustrates the accuracy of the committee achieved with AL (green curve) and
passive learning (blue curve), respectively measured on Mushroom (Fig. 1, left) and
AMASCOS (Fig. 1, right) test datasets. We first assessed the performance achieved on
the Mushroom test dataset on 200 iterations of training after every new query. AL
outperformed random sampling by 153 queries to reach an accuracy of 99.9% and
achieved perfect prediction with less than 40 labeled examples while the committee’s
accuracy using random sampling kept oscillating around 98.8% after 200 queries.
Next, we evaluated AL on the AMASCOS synthetic dataset. The 500 iterations of
query selection were followed by a training phase for each performed query. Results
are showing that AL reached an accuracy of 95.0% after 47 examples, compared to
random sampling that required 225 examples to reach that level. It represents a
reduction of 178 annotations for the human expert. After 500 iterations, the final
random sampling reached an accuracy of 96.6% while AL sampling reached a near
perfect performance of 99.7%, representing an absolute gain of 3.1%.
Fig. 1. (left) Accuracy of the committee measured on the Mushroom test dataset for active
(green curve) and passive (blue curve) learning. (right) Accuracy of the committee achieved on
the AMASCOS test dataset for active (green curve) and passive (blue curve) learning.
182 B. Chatelais et al.
4 Discussion and Perspectives
This study shows that AL outperforms random sampling and reduces annotation time
for knowledge capture. In fact, the quantity of annotations required to reach an
accuracy of 99% on the benchmark dataset was reduced by 86%. On the AMASCOS
test dataset, it represents a reduction of 79% to reach an accuracy of 95%. Although
these results are demonstrating the potential of using AL to reduce the annotation cost,
further work is required to evaluate the potential of other query strategies. Furthermore,
it is possible that a homogenous committee will converge faster than a heterogeneous
committee. Therefore, we will evaluate AL on a Random Forest classifier (a
homogenous committee of decision trees) and compare the performances with the
current heterogeneous committee. We also plan to evaluate a different type of sampling
strategy from the field of design of experiments that could provide a more challenging
baseline. Finally, human-in-the-loop experiments must be performed to validate the
benefit of using AL in real-world situations, where the “oracle” involves human
variability; potentially introducing noise is the herein investigated process.
Acknowledgments. We are thankful to Frédéric Morin and his team for software development
support and integration of the current work into the Cognitive Shadow system. This work was
supported by MITACS and PROMPT-Québec R&D partnership grant awarded to Christian
Gagné, and the financial support from Thales Canada.
References
1. Cooksey, R.W.: Judgment Analysis: Theory, Methods, and Applications. Academic Press,
San Diego (1996)
2. Berry, D.C.: The problem of implicit knowledge. Expert Syst. 4, 144–151 (1987). https://doi.
org/10.1111/j.1468-0394.1987.tb00138.x
3. Ericsson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data. MIT Press,
Cambridge (1984)
4. Lafond, D., Tremblay, S., Banbury, S.: Cognitive shadow: a policy capturing tool to support
naturalistic decision making. https://doi.org/10.1109/cogsima.2013.6523837 (2013)
5. Lafond, D., Labonté, K., Hunter, A., Neyedli, H., Tremblay, S.: Judgment analysis for real-
time decision support using the cognitive shadow policy-capturing system. In: Ahram, T.,
Taiar, R., Colson, S., Choplin, A. (eds.) IHIET 2019. Advances in Intelligent Systems and
Computing, vol 1018, pp. 78–83. Springer, Cham (2020)
6. Lafond, D., Vallières, B.R., Vachon, F., Tremblay, S.: Judgment analysis in a dynamic multi-
task environment: capturing non-linear policies using decision trees. J. Cogn. Eng. Decis.
Mak. 11, 122–135 (2017)
7. Armstrong, J. S.: Judgmental Bootstrapping: Inferring Experts’ Rules for Forecasting.
Principles of Forecasting: A Handbook for Researchers and Practitioners, pp. 171–192.
Norwood, Kluwer. (2001)
8. Karelaia, N., Hogarth, R.: Determinants of linear judgment: a meta-analysis of lens model
studies. Psychol. Bull. 134(3), 404–426 (2008)
9. Settles, B.: Active Learning Literature Survey. Technical report, University of Wisconsin
Madison (2010)
Task Measures for Air Traffic Display
Operations
Shi Yin Tan1(&), Chun Hsien Chen1, Sun Woh Lye1, and Fan Li2(&)
1
School of Mechanical and Aerospace Engineering, Nanyang Technological
University, 50 Nanyang Ave, Block N3, Singapore 639798, Singapore
[email protected], {MCHchen,MSWLYE}@ntu.edu.sg
2
Fraunhofer IDM Centre, Nanyang Technological University, 50 Nanyang Ave,
Block NS1, Singapore 639798, Singapore
[email protected]
Abstract. With rising growth in air traffic globally, advanced technologies are
being developed to aid ATCOs in the managing and control of a foreseeable
denser airspace. The need to perform holding stack management, a potential
challenge to ATCO, especially during heavy traffic congestion owing to weather
and runway conditions is expected to be more frequent. To mitigate this chal-
lenge, the use of 3D displays was suggested. This paper examines the perfor-
mance impacts resulting from the adoption of 3D instead of 2D radar displays
with regards to visual search and relative vertical positioning identification.
Observations relating perceived increased in stress and workload by the par-
ticipants are also made.
Keywords: Human performance Task measures Stress and workload
Perspective changes in dimensional representation
1 Introduction
Affordable commercial air travel has sparked rapid growth in the aviation industry in
recent years [1]. Managing and controlling air traffic effectively and efficiently is
becoming more important than ever before. Given that the primary role of an air traffic
controller (ATCO) is to guide and direct air traffic flow, ATCO is presented with a vast
amount of flight data daily. From these data, ATCO would then process them and make
appropriate decisions. This is because their workload is dependent on the number of
flights within their jurisdiction [2]. Thus, several advanced technologies have been
implemented to reduce the workload of ATCOs, such as conflict resolution advisory
aids [3], 3D radar display [4], and adaptive fatigue management [5]. Nevertheless,
implementing these novel advanced technologies would substantially change the
interactions between human and systems. In other words, the tasks and operational
procedures of ATCOs will change with the introduction of advanced technologies.
In a conventional ATC setup, screens are made to display a plan view of the
airspace with vertical separation of aircrafts denoted by altitude values next to the
aircraft markers. In recent years, there have been various studies on the use of 3D radar
displays that suggest evidence that a change in perspective can help to improve the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 183–189, 2020.
https://doi.org/10.1007/978-3-030-39512-4_29
184 S. Y. Tan et al.
ATCOs spatial awareness of traffic and terrain thereby better navigate aircrafts in the
vertical plane [6]. Interface congestion or cluttering is also a problem with 2D radar
displays when operations such as holding stack management is utilized [7]. In holding
stack management, aircrafts are tasked to fly in a preassigned route while waiting for
clearance to land. This preassigned route normally takes the form of an oval circuit
with multiple aircrafts following the same route separated vertically. The close hori-
zontal proximity on the screen of these aircrafts while carrying out a holding procedure
poses problems to the ATCO viewing on 2D radar displays due to the overlapping of
flight labels and markers.
It has been suggested that the use of 3D radar displays can help to reduce the
cognitive loads of the ATCOs via the reduction of interpretative task as better and
clearer mental picture can be derived from such displays. This is especially so in
managing congested airspace or in holding stack management [6]. Furthermore, the
alleviation of congestion of markers in the display using 3D radar displays can help
ATCOs make better judgements on vertical arrangements. However, little is reported
on the effects of 3D displays on ATCO’s acceptance and performance. This paper seeks
to investigate into performance measures relating to accuracy and task completion time
between 2D and 3D display interfaces in an ATC setting.
2 Methodology
2.1 Experimental Design
A laboratory-based experiment composing of 16 university-level student subjects was
conducted to study into the two measures of accuracy and completion times when
performing various scenario tasks using 2D and 3D displays. Each exercise consists of
a set of eight scenarios. On accuracy, this is based on the number of correct answers
provided by the participants when queried. For task completion time, this is based on
the recorded timestamps that captures the start of task till the final mouse click that
signals the end of the exercise. Besides this, NASA-TLX questionnaires were also used
where the participants are required to complete the study based on a Likert-type rating
scale of (0–9) aimed to measure their perceived mental workload and stress when
undertaking the various scenario tasks.
The experiment was conducted using a real-time simulator, the NLR ATM
Research SIMulator (NARSIM), to generate and simulate air traffic scenarios in real
time. The simulated scenarios were displayed on two display monitors. Scenarios used
in this study are based on holding stack management, a scenario which occurs when
multiple aircrafts upon ATCO’s instructions are required to fly in a holding pattern at
separate altitudes. 2D and 3D visual interfaces are used in this experiment as shown in
Fig. 1.
2.2 Experimental Procedure
Figure 2 shows the experimental flow where participants would first be required to
undergo several calibration procedures in relation to the EEG headset and eye tracker
Task Measures for Air Traffic Display Operations 185
Fig. 1. 2D visual orientation (Left), 3D visual orientation (Right)
that would be used in this exercise. This is then followed by a training period in which
participants familiarize with the experimental environment by running through several
simulated scenarios.
Fig. 2. Experimental flow of exercise to be conducted
Once familiar, the actual experiment exercise commences which comprises 8 test
scenarios of varying intensity and dimensional information representation as high-
lighted in Fig. 3.
Fig. 3. A set of eight scenarios to be performed
The procedure within each scenario consists of two phases as follows:
(a) Memory Phase: Participants were first asked to memorize key flight details such
as the callsigns and altitude of the aircraft markers that are presented in the left
186 S. Y. Tan et al.
display for 10 s before the screen will be turned off. For example in Scenario 1,
the left display would present a particular airspace with three dots denoting the
flights with its callsigns and flight levels highlighted in 2D orientation.
Test Phase: The right display is then activated, displaying the same airspace and
flight activity and information. Participants are then required to carry out two
tasks namely:
T1: A visual search task where one is to locate the aircraft markers in alphabetical
order based on their callsigns.
T2: Relative Position Identification when one is to select an aircraft at a higher
altitude out of 2 highlighted flight markers.
(b) The participant is required to give his/her answer for T1 and T2. Each (Memory
and Test) phase is repeated 15 times per scenario per participant.
(c) At the end of the scenario, the participant is required to fill up a set of NASA-TLX
questionnaires used to measure the perceived mental workload and stress for that
scenario.
(d) A 5 min break would then be instituted before the commencement of the next
scenario. This process repeats itself until a participant completes all the eight
scenarios.
It is to be noted that in the first 4 scenarios, different combinations of 2D and 3D
orientation representation were adopted. For scenarios 5–8, they are similar to scenarios
1–4 except that it functions at a higher intensity level where the number of aircraft
markers are increased from 3 to 4. These intensity values were chosen as these values
represent the limits of spatial working memory of young healthy adults. It was reported
that a young adult is found to be only capable of maintaining 3 to 4 simple visual
objects in working memory [8, 9].
3 Results and Discussion
3.1 Response Accuracy
Accuracy in response was based on the answers provided by the participant for the 2
test tasks (T1 and T2) in each of the scenarios. For each test task, 240 responses were
derived from all the participants. Figure 4 shows the mean or average percentage of
correct responses over the total number of participants’ responses for each test task in
the various scenarios. To evaluate situations having low and high markers, scenarios 1
and 5 of 2D > 2D are used as reference. 2-tailed paired t-test was then conducted
between these references with the other 3 display combinations (3D > 2D, 3D > 3D
and 2D > 3D) to find the significance of the values. A p score <0.05 is deemed to be
significant whereas score >0.05 is regarded as null.
In visual search tasks (T1), t-test results showed that there were significant dif-
ferences in accuracy registered for 3D > 2D scenarios (2 & 6) with respect to 2D > 2D
scenarios (p = 0.0205 and p = 0.0007 for the low (3 dots/markers) and high intensity
(4 dots/markers) levels respectively. Based on the mean of Scenario 2 (0.917) and
Scenario 1 (0.996), there is a significant drop in accuracy in performing visual search
tasks. This is also seen at higher intensity levels where lower accuracy and 0.842 for
Task Measures for Air Traffic Display Operations 187
Fig. 4. Average task accuracy
scenario 6 were registered compared to of 0.946 for scenario 5. One possible reason is
that in scenario 2 and 6, the change in visual projection may have forced the partici-
pants to reorient the memorized airspace information which may be a factor for the
decrease in accuracy. Furthermore, participants performed the worse in terms of
accuracy when the 3D orientation is used for the memory phase. This coincides with a
study that found significant detrimental effects of using 3D orientation whilst per-
forming visual search tasks [10].
In identifying vertical relative positioning tasks (T2), all other scenarios register
significant difference except for 3D > 2D at low intensity level (p = 4.506). Accuracy
for T2 was higher in scenarios where the dimensional representation is either upgraded
from 2D > 3D or at 3D > 3D orientation (Scenarios 3, 4, 7, 8). It is observed that
introduction of a depth cue in the 3D orientation allowed participants to visualize
altitude separation of aircraft better thereby greatly enhanced performance in T2.
Participants engaged in Scenario 6 (3D > 2D) fared the worst on average. This would
be due to a combination of perspective reorientation and engagement of a more
complicated 3D display in the memory phase.
3.2 Task Completion Time
On task completion time, the measure is from the start of the test stage to the final click
recorded in the test stage. The timings were recorded in milliseconds. Like the previous
exercise on response accuracy, average values were obtained over 240 responses across
all participants for T1 and T2 tasks. and represented in Fig. 5. These average values are
based on times where the participants have completed their tasks correctly.
Fig. 5. Average completion time
In visual search tasks (T1), values for 3D > 3D scenarios (scenarios 3 and 7) were
found to be insignificant (p = 0.7638 and p = 0.068) This could be owing to the single
orientation setup in these scenarios. For scenarios involving dual orientation displays
188 S. Y. Tan et al.
(Scenarios 2, 4, 6, 8), participants completed the tasks on average slower than a single
orientation setup for both low and high intensity levels.
For vertical relative positioning identification (T2), values for all other scenarios are
found to be significant except for Scenario 2 (p = 0.3468). Scenarios which utilized the
3D orientations for the test phase (3, 4, 7, 8) registered faster completion time on
average as 3D orientation display provides a better representation in terms of relative
vertical positioning. However, in scenario 6, a longer completion time is observed
when compared to scenario 5 (2151.981 > 1905.397). This may be attributed to the
higher complexities of the 3D representation during the memory phase.
3.3 Self-task Evaluation
For the self-evaluation section by the participants, NASA-TLX questionnaires were
used. Each participant is required to evaluate the perceived stress and workload at the
end of each scenario. Mean scores across all participants for each scenario along with
the respective 2-tailed t-test p-values relative to the 2D > 2D scenarios for each
respective intensity level are shown in Fig. 6.
Fig. 6. Participants’ questionnaire scores on perceived stress and workload
From Fig. 6, a significant higher mean is registered for scenarios 2 and 6 when
compared to 2D > 2D scenarios indicating that a higher level of stress was perceived
by the participants. Noticeable p scores of p = 0.0278 and p = 0.0032 for scenarios 2
and 6 respectively were also registered which indicated a significant difference between
the two compared scenarios. On workload, the mean scores of the participants show
significantly difference for scenarios 2, (6, 7) when compared with scenarios 1 and
(5) respectively. This is also reflected in the p scores (<0.05) for these scenarios.
When the increase in perceived stress and workloads in scenarios 2 and 6 are
compared with response accuracy, a lower accuracy and slower completion time were
observed in visual search tasks. For relative vertical positioning identification, signif-
icant increment in task completion times were also observed. One could postulate that
higher perceived stress and workload can result in lower response accuracy and longer
task completion times.
Task Measures for Air Traffic Display Operations 189
4 Conclusion
Visual search and relative vertical positioning tasks were conducted to study the per-
formance effects on 2D and 3D displays. It was found that response accuracy and
completion time could be used to identify the difficulties of various scenarios as
experienced by the participants.
Significant differences were observed in terms of lower response accuracy and
longer task completion time when using dual displays from 3D to 2D dimensional
representations. On relative vertical positioning tasks, upgrade of dimensional repre-
sentation from 2D to 3D displays or use of 3D display can help to improve accuracy
and completion time due to the reduction of interface cluttering.
The reduction in response accuracy and speed of completion could be used as
measures to gauge the stress and workload experienced by the subjects.
References
1. I.A.T.A. J.P. Release: IATA forecasts passenger demand to double over 20 Years (2016)
2. Endsley, M., Rodgers, M.: Situation awareness information requirements analysis for en
route air traffic control. In: Proceedings of the Human Factors and Ergonomics Society
Annual Meeting, vol. 38, pp. 71–75 (1994)
3. Trapsilawati, F., Wickens, C., Qu, X., Chen, C.: Benefits of imperfect conflict resolution
advisory aids for future air traffic control. Hum. Factors J. Hum. Factors Ergon. Soc. 58,
1007–1019 (2016)
4. Liu, Y., Trapsilawati, F., Lan, Z., Sourina, O., Johan, H., Li, F., Chen, C., Mueller-Wittig,
W.: Human factors evaluation of ATC operational procedures in relation to use of 3D
display. In: Advances in Human Factors of Transportation, pp. 715–726 (2019)
5. Li, F., Xu, G., Li, F., Chen, C.H., Khoo, L.P.: Contextual information-based human fatigue
prediction for integrated traffic control. In: Proceedings of International Conference on
Computers and Industrial Engineering (2018)
6. Wong, B., Rozzi, S., Boccalatte, A., Gaukrodger, S., Amaldi, P., Fields, B., Loomes, M.,
Martin, P.: 3D-in-2D displays for ATC. In: 6th EUROCONTROL Innovative Research
Workshop & Exhibition, pp. 47–62. EUROCONTROL Experimental Centre, Bretigny
(2007)
7. Masotti, N., Persiani, F.: On the history and prospects of three-dimensional human–
computer interfaces for the provision of air traffic control services. CEAS Aeronaut. J. 7(2),
149–166 (2016)
8. Cowan, N.: The magical number 4 in short-term memory: a reconsideration of mental
storage capacity. Behav. Brain Sci. 24, 87–114 (2001)
9. Fukuda, K., Awh, E., Vogel, E.: Discrete capacity limits in visual working memory. Curr.
Opin. Neurobiol. 20, 177–182 (2010)
10. Popelka, S., Doležalová, J.: Non-photorealistic 3D visualization in city maps: an eye-
tracking study. In: Brus, J., Vondrakova, A., Vozenilek, V. (eds.) Modern Trends in
Cartography. Lecture Notes in Geoinformation and Cartography, pp. 357–367. Springer,
Cham (2014)
Identifying People Based on Machine Learning
Classification of Foods Consumed in Order
to Offer Tailored Healthier Food Options
Jenna Kim1, Shuhao Lin2, Giannina Ferrara3, Jenna Hua4,
and Edmund Seto5(&)
1
Computer Science Department, University of Washington,
Seattle, WA 98195, USA
[email protected]
2
Department of Kinesiology and Nutrition, University of Illinois,
Chicago, IL 60612, USA
[email protected]
3
Institute for Health Metrics and Evaluation, Seattle, WA 98121, USA
[email protected]
4
Stanford Prevention Research Center, Stanford University,
Stanford, CA 94305, USA
[email protected]
5
School of Public Health, University of Washington, Seattle, WA 98195, USA
[email protected]
Abstract. Using food recall data collected from over 8000 Americans, we used
Machine Learning to classify a person’s demographic characteristics (age,
gender, race, and income) based on the foods they consumed in a 24-h period.
The best-performing models predicted gender correctly 61%, race 44%, and age
43% of the time on independent validation data. The model was subsequently
used to provide tailored recommendations for healthier food options based on
low-calorie and low-fat options for specific food groups that are typically
consumed by other Americans that match the person’s demographics. This
system is part of a larger Smart Human-Centered System that assists users in
recording the foods they consume, recognizes nutritional content of the food,
offers tailored recommendations for consuming healthier foods, and tracking
behavior change over time.
Keywords: Intelligent interfaces for Human-Artificial systems Artificial
intelligence Diet and nutrition Health
1 Introduction
Risk factors for chronic disease include obesity and poor diet. In the United States it is
estimated that approximately 40% of adults (93 million persons) are obese [1].
Increasingly, technologies such as smartphone apps are assisting individuals in tracking
their dietary intake patterns [2, 3]. Our group’s prior work includes the development of
such apps for research studies that require dietary assessments. We have leveraged
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 190–194, 2020.
https://doi.org/10.1007/978-3-030-39512-4_30
Identifying People Based on Machine Learning Classification 191
human-machine user interfaces such as camera-based recognition of food items in order
to make it easier to input and classify foods that users consume, and have shown that
with quality images, the foods that are most frequently consumed by Americans can be
recognized and classified with over 80% accuracy [4]. However, more research on
developing intelligent agents that recognize a person’s eating habits, and that can
suggest healthier options is still needed. Tailored dietary interventions have typically
involved interactions with human nutrition experts and intervenists [5]. The objective
of this research paper is to describe the use of Machine Learning to classify the
demographic characteristics of a person, who has begun to record data on the foods
they consume. By recognizing key demographics (age, gender, race, income, and other
characteristics), the system can begin to offer tailored recommendations for healthier
food options. A challenge with recommending food substitutions is that such recom-
mendations should ideally be sensitive the person’s demographic context [6]. For
example, for those at risk of cardiovascular disease, low-fat and low-sodium options
within specific food groups that are consumed by people with similar demographics
might for the basis for reasonable recommendations. Moreover, the demographic
context ideally should be intelligently inferred, rather than having to explicitly ask
users to complete a burdensome questionnaire.
2 Methods
We used data from the 2015-16 National Health and Nutrition Examination Survey
(NHANES), in which 8,506 individuals of all ages were sampled for the What We Eat
in America (WWEIA) Survey which obtained 24-h dietary recalls using in-person
interviews conducted by US Department of Agriculture (USDA) researchers [7]. Over
a 24-h period, all foods and beverages consumed were assessed, and categorized
according to USDA’s Food and Nutrient Database. Machine Learning using Python
Scikit-Learn v0.21.3 was used to train various models to predict individual demo-
graphic characteristics including age, gender, race, income, and educational attainment
using 80% of the data for model training, and the remaining 20% for validation. Input
variables include total caloric, protein, fat, carbohydrate, caffeine, alcohol intake, and
intake specific to breakfast, lunch, dinner vs snack occasions, and intake specifically for
individual food groups (e.g., dairy, fruits, grains, etc). Confusion matrices (proportion
of the reference class i observations that was predicted as class j) were computed for
each of the best models to evaluate misclassifications.
3 Results
In all but one case, Random Forest models tended to perform better than others in
predicting the demographic characteristics in persons in the WWEIA survey based on
the foods they consumed (Table 1). The models that performed the best, predicted
gender correctly 61%, race 44%, and age 43% of the time on independent validation
data. The confusion matrix for gender illustrates that females tend to be classified better
than males (Table 2). The model for race/ethnicity (Table 3) illustrates that most races
192 J. Kim et al.
tend to be correctly classified, except that Hispanics and Other/multi-race groups tend
to be misclassified as White. The model for age (Table 4) shows that the 18–29, 50–69,
and 70+ year old groups tends to be misclassified into one class younger.
Table 1. Best performing demographic classification models.
Demographic Machine learning model Classification score
Age Random Forest 0.433
Gender Random Forest 0.610
Race/Ethnicity Gradient Boosting Classifier 0.439
Income Random Forest 0.328
Education Random Forest 0.368
Table 2. Classification of Gender showing proportion of reference class.
Prediction
Reference Male Female
Male 0.51 0.49
Female 0.30 0.70
Table 3. Classification of Race/Ethnicity showing proportion of reference class.
Prediction
Other race or
Reference Hispanic White Black Asian multi-racial
Hispanic 0.30 0.43 0.21 0.05 0.01
White 0.14 0.66 0.15 0.04 0.01
Black 0.21 0.36 0.40 0.03 0.00
Asian 0.17 0.30 0.12 0.39 0.02
Other race or 0.10 0.58 0.24 0.07 0.01
multi-racial
Table 4. Classification of Age (years) showing proportion of reference class.
Prediction
Reference 7-17 18-29 30-49 50-69 70+
7-17 0.81 0.03 0.10 0.06 0.00
18-29 0.43 0.09 0.28 0.16 0.03
30-49 0.28 0.08 0.37 0.26 0.01
50-69 0.25 0.05 0.21 0.44 0.05
70+ 0.20 0.01 0.15 0.52 0.13
Identifying People Based on Machine Learning Classification 193
Although the models consider many nutrient intake variables, boxplots showing the
total daily caloric intake of males vs females illustrates the overlap between genders,
making classification difficult (Fig. 1).
Fig. 1. Boxplots comparing calories consumed by males vs females.
4 Discussion
This work applies machine learning to 24-h. dietary recall data in order to predict
demographic characteristics that can be used to tailor recommendations for food sub-
stitutions that may be healthier (e.g., lower in fat, sodium, etc.). The univariate dis-
tributions of particular nutrients between demographic groups (e.g., caloric intake of
males vs females in Fig. 1) have a great deal of overlap, making simple classification
criteria difficult. While multivariate machine learning models have promise for dis-
criminating between demographic groups, the mediocre classification scores of our
models reflect the complexity and variability in diets within and between demographic
categories. For future work, the additional of other variables, such as more detailed
timing and location of meals, as well as brand names of packaged foods might improve
classification performance.
To apply these models to recommend food substitutions, the demographic models
are run in parallel to predict the likely demographic characteristics of the user. The
prediction may be run repeatedly over time for each 24-h. period to refine the pre-
diction. As a user enters a food item into an app (e.g., using text, barcode, or image
recognition), the food category is identified. Within the WWEIA data, there are 51 food
categories (e.g., beef, eggs, pasta, fruits, etc.). For the predicted combination of
demographics of the user {age, gender, race, income, education}, we can query the set
of food items within the food group that are consumed by other individuals with
matching demographics. From this set, we search for items that are healthier with
respect to some pre-specified criteria (e.g., for hypertension interventions, we may
search for items with lower sodium/g; for cardiovascular health interventions, we may
194 J. Kim et al.
search for items with lower fat/g). The results of the search are the set of items that may
be recommended for tailored intervention. To avoid recommending the same items,
certain demographic variables could be occasionally dropped from the search (e.g.,
matching on all but age) to create a larger set of possible recommended food items.
Moreover, it is possible to track whether recommendations are adopted, so as to further
refine the set of most likely food substitutions for different demographic groups. This
forms the basis for how the classification results can be used with the WWEIA database
in order to automate recommendations for healthier food options.
There are limitations associated with the WWEIA data. Some researchers have
commented on the general problem of relying on self-recall data [8]. Moreover, even
though the WWEIA database consists of data from a large number of subjects, when
these subjects are partitioned by different demographics, the resulting numbers in each
category become small. However, if the tailored recommendation system is deployed in
apps across large numbers of users, the recommendation system could incorporate new
user data over time, which could better reflect the changing dietary intake patterns of
more people over time.
Acknowledgments. Research supported by NIH grant U54 MD011240 and NIH grant 5P30
ES007033-23. JH is supported by NIH NHLBI T32 HL007034.
References
1. Centers for Disease Control and Prevention. https://www.cdc.gov/obesity/data/adult.html
2. Seto, E., Hua, J., Wu, L., Shia, V., Eom, S., Wang, M., Li, Y.: Models of individual dietary
behavior based on smartphone data: the influence of routine, physical activity, emotion, and
food environment. PLoS ONE 11(4), e0153085 (2016)
3. Boushey, C.J., Kerr, D.A., Wright, J., Lutes, K.D., Ebert, D.S., Delp, E.J.: Use of technology
in children’s dietary assessment. Eur. J. Clin. Nutr. 63(S1), S50 (2009)
4. Kim, J., Lin, S., Ferrara, G., Hua, J., Seto, E.: Validation of a food image classifier based on
the top 50 consumed foods from the what we eat in America survey. In: IEEE Biomedical
Health Informatics 2019 Conference, Chicago, 19–22 May (2019)
5. Heo, S., Moser, D.K., Lennie, T.A., Payne-Emerson, H., Welch, J.L., Weaver, M.:
Development and testing of the feasibility and acceptability of a tailored dietary intervention
in patients with heart failure. J. Cardiovasc. Nurs. 30(3), 213–221 (2015)
6. Gans, K.M., Risica, P.M., Strolla, L.O., Fournier, L., Kirtania, U., Upegui, D., Zhao, J.,
George, T., Acharyya, S.: Effectiveness of different methods for delivering tailored nutrition
education to low income, ethnically diverse adults. Int. J. Behav. Nutr. Phys. Act. 6(1), 24
(2009)
7. Dwyer, J., Picciano, M.F., Raiten, D.J.: Collection of Food and Dietary Supplement Intake
Data: What We Eat in America–NHANES. J. Nutr. 133, 590S–600S (2003)
8. Archer, E., Pavela, G., Lavie, C.J.: The inadmissibility of what we eat in America and
NHANES dietary data in nutrition and obesity research and the scientific formulation of
national dietary guidelines. Mayo Clin. Proc. 90(7), 911–926 (2015)
On the Perception of Disharmony
Stijn Verwulgen(&), Thomas Peeters, Sander Van Goethem,
and Sofia Scataglini
Department of Product Development, Faculty of Design Science,
University of Antwerp, Antwerp, Belgium
{stijn.verwulgen,thomas.peeters2,sander.vangoethem,
sofia.scataglini}@uantwerpen.be
Abstract. Our hearing system works roughly like a calculator for Fourier
series, decomposing a sound waves in basic frequencies and then activating
respective nerve channels. This relatively easy architecture of our sound per-
ceiving system accounts for the fact that we can filter different sources and thus
also that we can focus on a single sound of interest. Consequently, our brain is
trained to assess an incoming stem of frequencies (1/n) to a single source, as if
the input source is produced by a periodic sound wave. This understanding of
our sound system is successfully applied in the development of what is one of
the first and well known artificial cognition system: a cochlear implant,
bypassing the system physical detection of sound. Two or more sound pitches
that have relative fractional frequencies, are considered pleasing or “harmonic”.
Although music is a cultural product, evoking individual emotions, perception
of harmonics is largely a collective and individual phenomena. This also wit-
nesses the peculiar and intriguing architecture of our sound system. So the
question arises whether fractional frequencies is systematically necessary for
perception of harmony; are two sound pitches played at substantially irrational
(i.e. non fractional frequencies) perceived as non-harmonic? The golden ratio is
the most irrational number in the sense that it has the most slowly converging
continued fraction expansion. In this work, we present a hypothesis to generate
non-harmonics. Insights can be used to enhance inputs in artificial hearing, by
displaying different sound pitches at clearly distinguishable frequencies.
Keywords: Golden ratio Artificial hearing systems Disharmony
1 Introduction
Sound is generated by variations in air pressure that behave and propagate like waves.
This means that the magnitude of pressure uðx; y; z; tÞ at a given point ðx; y; zÞ at a given
time t satisfies the wave equation:
2
@2u @ u @2u @2u
¼ c þ þ ð1Þ
@t2 @x2 @y2 @z2
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 195–200, 2020.
https://doi.org/10.1007/978-3-030-39512-4_31
196 S. Verwulgen et al.
This wave equation is linear: the sum of two sound waves u1 and u2, defined as
ðu1 þ u2 Þðx; y; z; tÞ ¼ u1 ðx; y; z; tÞ þ u2 ðx; y; z; tÞ;
also satisfies Eq. 1.
The wave equation accurately represents the behavior of waves in various media, in
1D, 2D and 3D shapes, and it has entailed has many applications. For example, the
study of sound waves inspired Graham Bell to convert sound into electricity and back
[1], further implemented and improved by Thomas Edison [3]. The resulting transfer of
sound occurs along the following steps:
Step 1. A sound source s produces a sound wave u according to Eq. 1.
Step 2. The air pressure changes are directly, one-to-one in a linear order, converted
to electric field fluctuations.
Step 3. The electric field is transmitted through a conductive cable, e.g. a telegrapher
line. This transmission is also modelled by a linear differential equation. Transmission
through a cable drastically reduces energy required for sending information from one
place to another, as the signals are converted from a 3D to a 1D medium.
Step 4. The electrical signal is reconverted into air pressure waves.
Step 5. The air pressure is picked up by our auditory senses and we perceive the
sound source, as if it was nearby, surprisingly, although it was suppressed under way in
a single cable.
In the entire process, all conversions preserve linearity. This means in particular
that two sources s1 and s2 that produce sound waves u1 respectively u2, simultaneously
producing a sound wave u1 + u2, are also perceived as two separated sound waves after
transfer. Hence the transfer of sound is faithful with regard to our auditory perception.
As such, a telephone organized along the above steps, can be considered as the very
first artificial hearing system. We further explain how our hearing system can dis-
criminate between different sound sources and how this has led to the development of
bionic ears. Then we formulate a hypothesis on how one might optimize separation of
sound in such an artificial hearing system.
2 Physiology of Sound Perception
The fact that we can distinguish different sound sources is directly related to the physi-
ological architecture of our hearing system [4, 5]. The ability to perceive basic compo-
nents in a composed onset, is fundamentally different from e.g. our vision system; we
cannot see how a color is mixed by individual components. For example, a single orange
color can be the result of yellow-red mixtures in a variety of grading, but we are not able to
determine those original components. Also, mixing yellow, green and blue results in a
brown-green color, in which the original compounds seems not present anymore.
In auditory perception, we immediately recognize the individual sources, even if
there are more than two. The physiology of our auditory system also entails that we can
identify different sound sources and focus, for example on a single conversation of
interest in a very noisy environment. The transfer of sound from a source to our brains
is organized in analogue to the same five steps outlined above. Sound is converted from
air pressure variations to bone vibrations by our eardrum. These vibrations are
On the Perception of Disharmony 197
transferred via our ossicles to our cochlea. The cochlea converses these vibrations to
neurophysiological signals by which we perceive sound, comparable with step 4 of the
sound transfer in the previous section. Interestingly, this step is substantially linear too.
The cochlear decomposes an incoming wave into its different pitches and activates
nerves directly related to a specific place. As there is a direct relation between the pitch
and the place, we are able to discriminate sound pitches. A mathematical analogue of
this decomposition is through Fourier analysis. Each periodic signal uðtÞ can be written
as a composition of basic pitch l and harmonics pitches nl, say:
X
1
uð t Þ ¼ a0 þ ðan cos nlt þ bn sin nltÞ; ð2Þ
n¼1
with an and bn real numbers related to the intensity of the sound, and in particular the
intensity of the n-th pitch. The cochlea thus activates a specific series of nerves in
response to an incoming sound uðtÞ with specific frequency. Each nerve region is
activated with a specific intensity, related to an and bn in Eq. (2). This explains the
timbre of an instrument, a voice, or other sound producing sources: an activation
pattern in our pars cochlearis that directly relates to a Fourier decomposition of the
sound wave. As Fourier decomposition is a linear operation, which occurs at the last
step of a sequence of other linear operations in the transfer of sound from source to
perception, this also explains why we are able to distinguish different sound sources.
We can also focus on specific sounds, or filter it e.g. irrelevant background sources and
noise during a conversation in a loud environment.
As this filtering capacity diminish along age and is sometimes impossible due to
dysfunctioning of auditory system, there is an increasing need for assistive listening
devices and hearing systems.
3 Perception of Harmony
Plugging a string with to ends fixed produces a periodic sound, according to Eq. (1)
applied in 1D. With basic pitch l, the overtone pitches are nl, where n runs through the
natural numbers, according to the Fourier decomposition (Eq. 2). These basic har-
monics are produced by the string by fixating it at a fractional part of its length. For
example, take a string represented by the interval ½1; 1, with a standing wave
vibration mode sin px (the second harmonic of the string). Then the standing wave
sin 3px is also a harmonic, and occurs on fixating the string at fractions with nominator
3, as shown in Fig. 1.
So the activation pattern produced by the cochlea by a periodic sound signal
producing a frequency l, like in a piece of music, follows the harmonic modes of a
string, with basic harmonic frequency l and overtones nl. Also, two frequencies l1
and l2 are perceived as ‘harmonic’ if they are relatively fractional, that is if ll1 is a
2
rational number. If both frequencies are produced on a string of length l1 repectively l2 ,
this means that ll12 is rational.
198 S. Verwulgen et al.
Fig. 1. An example of two harmonics sine waves on a string. The nods are fractional parts of the
string.
A possible explanation that we perceive relative rationals as harmony, is that the
higher harmonics (high n- values in Eq. 2) coincide, so that we recognize a common
activation pattern in our pars cochlearis. From that hypothesis, the question arises at
what frequency ration we perceive two sounds as disharmonic, and if then, somehow,
all overtones are maximally dispersed in our pars cochlearis. If this is the case, then two
disharmonic sounds should be maximally distinguishable.
4 Golden Ratio as an Expression of Disharmony
The golden ratio is constructed by the division of a line [0, a] into a big part [0,b] and a
small part [b, a], such that the ratio whole length to the big part, equals the ratio of the
big to the small (Fig. 2).
Fig. 2. Division of a line in the golden ratio
This means that the ratio x ¼ ab satisfies the quatratic equation
x2 x 1 ¼ 0 ð3Þ
pffiffi
The positive solution U ¼ 521 ¼ 1:618. . . is called the golden ratio, or golden
section. This number has some remarquable mathematical properties it occurs in
physics and approximately in nature [6]. It is also a popular source of inspiration for
fine arts and architecture however the claimed universal properties for aesthetics have
no proven ground [7, 8].
On the Perception of Disharmony 199
Equation (2) can be rewritten as
1 1 1
x ¼ 1þ ¼ 1þ ¼ 1þ
x 1 þ 1x 1 þ 1 þ1 1
x
This suggests the continued fraction expansion
1
W ¼ 1þ ð4Þ
1þ 1
1 þ 1 þ1
which indeed holds [7]. Continued fraction expansion is mathematical technique to
approximate an irrational number, say p, naturally, by a ‘best fitting’ rational number
1
p ¼ a1 þ
1þ 1
a2 þ a 1
3 þ
where each ai is a naturall number. The bigger ai , the more accurate the i-th rational
approximates the number p. As each ai in the approximation of W equals 1 in Eq. (3),
the golden section is somethimes informally described as ‘the most irrational number of
all irrationals’. This quotation is also supported by the observation of the golden ratio in
phyllotaxis.
Phyllotaxis refers to the configuration of botanic elements (parastiches) around a
central stem. The angle between subsequent parastitchis is often fixed approximately,
and very often by the radial division of the golden ratio (Fig. 3).
Fig. 3. Phyllotaxis: configuration of botanic elements around a central support structure, often
approximates the golden ratio [7].
As the golden section is no rational number, this configuration entails in no leafs or
branches overlapping, thus avoiding occlusions and blocking from sunlight. In a per-
spective of botany, this ‘most irrational’ number also seems the most optimal angle,
admitting maximal radial coverage. This long standing open problem in botany was
indeed solved in the early’90, [9].
200 S. Verwulgen et al.
5 Discussion and Conclusion
We propose that two frequencies along the golden ratio are most clearly discordant.
This proposition is supported by (1) the observation that the continued fraction of the
golden ratio only contains the number 1, which is the smallest possible, and hence the
golden ratio has the least rational approximation, (2) the idea of entailed resulting
maximal dispersion is extensively observed in literature and successfully proven as a
self-organizing dynamical system in botany, (3) if two sound pitches have relative
rational frequencies, then a co-sound is perceived as harmonic, (4) by contraposition, if
we perceive a disharmonic co-sound, this implies that frequencies are relatively irra-
tional, (5) rational frequencies are one-to-one produced by strings with relative rational
lengths, (6) the most irrational ratio of two string lengths is given by the golden ration,
thus producing the most disharmonic co-sound. This co-sound is then maximally
dispersed in the pars-cochlear.
This paper provides a grounded hypothesis that two sounds displayed at pitches
that are in relative the golden ratio, are optimally distinguished from each-other. These
hypotheses should be further tested experimentally. E.g., one could organize auditory
experiments to assess the noise level subjects are able to comprehend a conversation
between two third parties, at different voice frequencies. Another possible approach
could be to evoke a noise of a basic frequency, and then assess the level under which
subjects are able to perceive a co-sound, at varying frequencies.
Finally, we would like to address that the hypothesis we present here, counteracts
the popular viewpoints on the golden ratio as ‘harmonic’, where we propose that it is
maximally ‘disharmonic’.
References
1. Gorman, M.E., Carlson, W.B.: Interpreting invention as a cognitive process: the case of
Alexander Graham Bell, Thomas Edison, and the telephone. Sci. Technol. Hum. Values 15
(2), 131–164 (1990)
2. Carlson, W.B., Gorman, M.E.: A cognitive framework to understand technological creativity:
Bell, Edison, and the telephone. Inventive minds: Creativity in technology, pp. 48–79 (1992)
3. Edison, T.A.: Improvement in speaking-telephone. 1878, Google Patents (1878)
4. French, N.R.: The Spiral of the Human Cochlea. J. Acoust. Soc. Am. 20(4), 591 (1948)
5. Warren, R.M.: Auditory Perception. Cambridge University Press, Cambridge, 19 June 2008
6. Livio, M.: The Golden Ratio: The Story of Phi, the World’s Most Astonishing Number.
Broadway Books, New York (2008)
7. Verwulgen, S., Cornelis, G.: Universele esthetiek! De gulden snede? Over zin en onzin
omtrent een merkwaardig getal. University Press Antwerp (2018)
8. Verwulgen, S., Van Goethem, S., Cornelis, G., Verlinden, J., Coppens, T.: Appreciation of
proportion in architecture: a comparison between facades primed in virtual reality and on
paper. In: Advances in Intelligent Systems and Computing, pp. 305–314, (2019)
9. Douady, S., Couder, Y.: Phyllotaxis as a physical self-organized growth process. Phys. Rev.
Lett. 68(13), 2098 (1992)
Mobile Real-Time Eye-Tracking for Gaze-
Aware Security Surveillance Support Systems
Alexandre Marois1(&), Daniel Lafond2, François Vachon1,
Eric R. Harvey3, Bruno Martin3, and Sébastien Tremblay1
1
Université Laval, Quebec City, Canada
[email protected], {francois.vachon,
sebastien.tremblay}@psy.ulaval.ca
2
Thales Research and Technology, Quebec City, Canada
[email protected]
3
Centre en imagerie numérique et médias interactifs, Quebec City, Canada
{eharvey,bmartin}@cimmi.ca
Abstract. The validity of two commercial-off-the-shelf eye trackers, including
a fixed system and a mobile device, was compared in the context of a gaze-based
assistant tool implemented to aid security surveillance. Participants performed a
security surveillance simulation that required them to detect incidents across
multiple video feeds. Aid from the Scantracker system—pinpointing neglected
cameras while detecting and correcting attentional tunneling and vigilance
decrease—was provided using gaze measures. Despite the lower amount of
valid data for the mobile system, both devices computed similar measures of
surveillance performance. Findings speak to the feasibility of using mobile
robust-to-movement decision-support systems based on real-time gaze data to
help improve incident detection in security surveillance environments.
Keywords: Security surveillance Eye movements Attention Real-time
measures Computer vision Mobile devices
1 Introduction
Surveillance is critical for the protection of key infrastructures and public places against
threats such as terrorism and criminal activity. Security surveillance is part of the
security strategy privileged by many domains, adopted both in military and public
fields [1, 2]. Several security surveillance systems rely on closed-circuit television
(CCTV), i.e. on a system of several video cameras that send images to a single
surveillance unit. Operators whose job is to monitor CCTV systems are however faced
with many cognitive challenges [3]. Such challenges need to be addressed to increase
the effectiveness of security surveillance teams and, ultimately, to help increase public
safety [4]. Because security surveillance mainly relies upon visual search, a promising
avenue for performance optimization in this context is the development of user-
centered intelligent support systems that analyze eye movements.
Hitherto, a few real-time intelligent support systems have been developed to assess
operators’ eyes behavior in contexts related to visual search (see [5, 6]). For example,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 201–207, 2020.
https://doi.org/10.1007/978-3-030-39512-4_32
202 A. Marois et al.
[7] developed a gaze-aware support tool in the context of CCTV security surveillance.
Implemented for multi-screen surveillance, the Scantracker—which relied upon real-
time eye-tracking measures—aimed at orienting participants toward neglected camera
feeds and at moving away their gaze in situations of attentional tunneling. The authors
showed that scan coverage among camera feeds was significantly improved for subjects
who could rely on the Scantracker. Live gaze monitoring, as part of intelligent support
systems, thus seems useful for security surveillance. Post-experiment analyses of eye-
tracking measures can also be useful in security contexts (e.g., to infer the visual scan
strategies privileged by surveillance operators; see [8]).
Two types of eye-tracking systems can be used in such situations: fixed or mobile
devices. Fixed eye trackers usually comprise a set of infrared cameras that are placed
on a table in front of the participant or within a built-in device (e.g., a computer
monitor). They are virtually noninvasive, and 3D-integration of the environment can be
easily performed as long as the visual environment remains static and the person stays
within the zone captured by the cameras. Mobile eye-tracking devices are rather worn
as glasses and are thus highly robust to one’s movements. This type of technology may
however feel more intrusive and uncomfortable if worn for long periods. Integration of
the 3D environment may also represent a challenge as the visual field constantly
changes, so is the position of the objects in that visual environment. To associate one’s
gaze to a specific object or place in the environment, a detection system based on image
analysis to detect fixed markers must be developed. No study has yet examined the
capability of implementing such mobile system for multi-screen eye tracking.
Hence, the current study aims at testing two types of eye tracker, that is a fixed and
a mobile commercial-off-the-shelf device, in the integration of an intelligent decision-
support system relying on the live processing of ocular data. Seventy participants took
part in a simulated surveillance task. Both eye trackers were compared in the imple-
mentation of the Scantracker tool which provided live visual countermeasures of
neglect and attentional tunneling [7], while also presenting scan-based vigilance-
heightening audio alerts in the form of task interruptions (see [9]). It was hypothesized
that, because wearable eye-tracking devices may be more robust to movement, using
the mobile eye tracker should entail more valid data than using a fixed device. Still,
both technologies should lead to similar behavioral and gaze measures.
2 Method
2.1 Participants
Seventy adults took part in a single 90-to-120-min experimental session in return of a
small honorarium. All reported normal or corrected-to-normal vision and hearing.
2.2 Apparatus and Material
Surveillance Testbed. A realistic and dynamic monitoring testbed referred to as the
“Cognitive Solutions for Surveillance in Security” (CSSS; [10]) microworld was used
to reproduce a security surveillance environment (see Fig. 1). This state-of-the-art
Mobile Real-Time Eye-Tracking 203
system is set up as a CCTV control room, with an operator workstation and a large
video display wall. The CSSS testbed contains eight cameras but can only display six at
the same time. Simulated scenarios presenting crowd scenes at different city locations
during a music festival were presented and comprised a total of 29 incidents.
Fig. 1. Experimental setup of the CSSS microworld (left) and integration of the eye gaze (green
dot) of the subjects and of the objects within CSSS by the mobile marker-based system (right).
Eye-Tracking Devices. Validity and reliability of the fixed Smart Eye Pro 3D system
(Smart Eye AB) and the Tobii Pro Glasses 2 mobile system (Tobii Technology) were
tested. The Smart Eye Pro 3D system is a device mounted onto the workstation of the
surveillance operator. It comprises four cameras recording at 120 Hz and five infrared
flashes. This device is deemed highly reliable in terms of calibration and of valid gaze
data [11]. It possesses a built-in component allowing 3D-modeling of the environment
according to the position of the cameras and subject detection within the environment.
The Tobii Pro Glasses 2 device is a head-mounted eye tracker using four 50-Hz
infrared cameras. This system sometimes lacks precision but is very simple to set up
and to use [12]. Contrary to the Smart Eye system, this device is not equipped with a
built-in module modeling the 3D environment. Hence, a computer vision-based
algorithm was used to infer gaze position. Based on Open CV, a computer vision
solution for image processing, the algorithm allowed detecting within the visual
environment the presence of 2D target markers (hexagonal or octagonal; see Fig. 1)
serving for geo-spatial positioning at a 12.5-Hz refreshing rate with the system’s frontal
camera. As long as a visual marker was detected, the system could infer the 3D pose of
the objects and superimpose the gaze while accounting for scene depth.
2.3 Procedure
Participants took part in an interactive tutorial to familiarize with the task. Following
calibration of the eye-tracking system, they performed four 16-min scenarios in which
they were asked to find incidents among eight simulated CCTV camera feeds. Based on
their live gaze data, the Scantracker could trigger neglect and tunneling visual alerts
(when detecting the lack or the excess of a certain amount of gaze data on a given video
feed, respectively; see [7]), and vigilance audio alerts when inspections on the camera
204 A. Marois et al.
feeds deviated from a within-subject baseline nominal pattern. Half of the participants
wore Tobii Glasses while the other half were monitored by the Smart Eye system. For
purposes outside the scope of the study, alarms were hidden for 50% of the subjects.
Here, we aimed at analyzing the impacts of the eye trackers on the behavioral metrics
captured by the Scantracker. At the end of each scenario, the NASA-TLX self-report
questionnaire was administered to assess perceived workload.
3 Results
3.1 Validity of the Gaze Measures
Table 1 presents the measures of valid gaze data (in %), the number of inspections and
the inspection duration (in s) for both groups of eye tracker. A mixed ANOVA per-
formed on the amount of valid gaze data uncovered a significant effect of Eye tracker
(F = 114.96, p < .001), of Block (F = 3.30, p = .030), and interaction (F = 6.26,
p < .001). Further analysis showed that the Tobii system led to fewer valid oculometric
data and to a decrease in validity from blocks 1 to 4. More valid inspections were also
detected by the Smart Eye system as supported by a significant effect of Eye tracker
(F = 37.11, p < .001). No effect of Block nor any interaction arose. This relationship
flipped regarding inspection duration as longer inspections were detected by the mobile
device with a significant effect of Eye tracker (F = 24.55, p < .001).
Table 1. Validity of the gaze measures as a function of the type of eye tracker
Surveillance measures Fixed Mobile
Valid gaze data (%) 83.34 (15.51) 47.92 (11.68)
Number of inspections 316.63 (59.01) 229.17 (57.14)
Inspection duration (s) 1.28 (0.28) 1.67 (0.32)
3.2 Measures of Surveillance
Despite the difference in valid data, no difference arose in terms of the surveillance
measures. Mixed ANOVAs performed on incident detection accuracy and time, on
perceived workload, on the scanlag (the lag elapsed since the last scan on a given feed;
[7]) and on cycle time (the average time taken to scan all cameras) all led to non-
significant main effects (Fs < 1.53, ps > .05). Furthermore, regardless of the eye
tracker, cycle time was negatively correlated with incident detection performance
(r = −.28, p = .042) and strongly associated to scanlag (r = .86, p < .001).
4 Discussion
The present study aimed at comparing two eye tracker systems integrated in the context
of the Scantracker gaze-based intelligent support system (see [7]). Contrary to our
hypothesis, the mobile system led to poorer gaze detection and fewer measures of
Mobile Real-Time Eye-Tracking 205
inspection. Nonetheless, measures of surveillance did not significantly vary across the
two systems. Measures of cycle time collected by both eye trackers were also related to
detection performance, thus supporting the operational principle of the Scantracker
(see Table 2).
Table 2. Measures of surveillance as a function of the type of eye tracker
Surveillance measures Fixed Mobile
Detection accuracy (%) 19.55 (11.70) 23.09 (10.59)
Detection time (s) 27.18 (9.44) 27.81 (8.07)
Perceived workload 5.42 (0.91) 5.22 (1.08)
Scanlag (10-s time bins) 3.91 (2.20) 4.05 (1.86)
Cycle time (s) 145.14 (102.06) 122.24 (63.30)
Differences in valid gaze data between both types of device might be mainly caused
by situations in which environmental markers were undetected (see the marker
unrecognized on the upper right of Fig. 1). Because integration of the 3D environment
was based on these markers, the decision-support tool could not always infer the gaze
position within the visual scene, which led to invalid eye data. Still, our results suggest
that implementation of a decision-support tool that relies on the live processing of eye
measures can be performed despite a significant amount of missing data. The gaze-
based measure of cycle time computed with the Scantracker can even be used to
predict incident detection accuracy (longer time being related to poorer performance).
Implementing wearable systems could be an asset for researchers interested in post-
hoc eye movement analysis and for practitioners aiming at implementing gaze-based
decision support tools for security surveillance [7, 8]. First, mobile eye trackers do not
impede operators’ movements. Second, integration of the 3D environment can be done
quite easily using cost-free open source computer vision solutions. Third, mobile
devices are often cheaper than fixed ones (for multi-screen contexts) and, despite their
accuracy limits [12], represent cost-efficient tools. Finally, once the gaze detection
algorithm developed, implementation of this system can be performed easily as long as
visual markers are detectable [11]. Still, using a mobile tool may incur some draw-
backs. First, head-mounted devices are sometimes deemed intrusive and uncomfort-
able. Because security surveillance operators often work for long shifts [13], wearing
eye-tracking glasses for many hours may become tedious. Second, because the system
highly depends on the detection of visual markers, images of the markers must be clear
and completely visible to be detected. Low performance (in resolution and integration
time capture) mobile systems may collect blurry images, which may mitigate detection
from the computer vision algorithm.
From an NSEEV (noticing - salience, expectancy, effort, value) model perspective
[14], visual alerts provided by the Scantracker tool may help to overcome the effort
required to move one’s gaze towards camera feeds in which incidents are scarce by
increasing saliency [7]. Moreover, according to [5], gaze-based decision-support sys-
tems deployed in contexts related to visual search can decrease cognitive load as they
206 A. Marois et al.
provide cues that alleviate demands on their user’s memory system. Since those
benefits are contingent on the decrease in effort (and cognitive load), using a fixed
device that may impose operators to limit their movements could mitigate the positive
impact of the decision-support system as further efforts may be required to stay in a
specific zone. Privileging mobile technologies could allow surveillance operators to
freely move and concentrate on their task, without having to think about their position.
Although no impact was observed on the measures of surveillance, the mobile
technology used in the current study still led to poorer data validity. Future studies
could aim at implementing this mobile gaze-aware system using augmented reality
tools equipped with an eye-tracking capability (e.g., the new Microsoft HoloLens 2) as
it may remove the necessity of environmental markers and of interfacing with the
surveillance system. This would ensure better and easier environment detection while
still allowing the user to move freely without adding any cost in cognitive effort.
Acknowledgments. This project was supported by a grant from the Innovation for Defence
Excellence and Security (IDEaS) program and by financial support from Mitacs Canada. We are
grateful to all the students involved in the data collection and to the financial and in-kind
contributions of Thales Research and Technology Canada.
References
1. Chérif, L., Wood, V., Marois, A., Labonté, K., Vachon, F.: Multitasking in the military:
cognitive consequences and potential solutions. Appl. Cogn. Psychol. 32, 429–439 (2017)
2. Piza, E.L., Welsh, B.C., Farrington, D.P., Thomas, A.L.: CCTV surveillance for crime
prevention: a 40-year systematic review with meta-analysis. Criminol. Public Policy 18,
135–159 (2019)
3. Hodgetts, H.M., Vachon, F., Chamberland, C., Tremblay, S.: See no evil: cognitive
challenges of security surveillance and monitoring. J. Appl. Res. Mem. Cogn. 6, 230–243
(2017)
4. van Voorthuijsen, G., van Hoof, H., Klima, M., Roubik, K., Bernas, M., Pata, P.: CCTV
Effectiveness Study. In: Proceedings of Carnahan Conference Security, pp. 105–108 (2005)
5. Taylor, P., Bilgrien, N., He, Z., Siegelmann, H.T.: Eyeframe: real-time memory aid
improves human multitasking via domain-general eye tracking procedures. Front. ICT 2, 17
(2015)
6. Morozkin, P., Swynghedauw, M., Trocan, M.: An Image compression for embedded eye-
tracking applications. In: IEEE International Symposium INISTA (2016)
7. Tremblay, S., Lafond, D., Chamberland, C., Hodgetts, H., Vachon, F.: Gaze-aware cognitive
assistant for multiscreen surveillance. In: International Conference on Intelligent Human
Systems Integration, pp. 230–236. Springer, Cham (2018)
8. Hodgetts, H., Chamberland, C., Latulippe-Thériault, J.-D., Vachon, F., Tremblay, S.:
Priority or parity? scanning strategies and detection performance of novice operators in
urban surveillance. In: Proceedings of Human Factors and Society, vol. 62, pp. 1113–1117
(2018)
9. St-John, M., Risser, M.R.: Sustaining vigilance by activating a secondary task when
inattention is detected. In: Proceedings of Human Factor and Erogonomics Society, vol. 23,
pp. 155–159 (2009)
Mobile Real-Time Eye-Tracking 207
10. Vachon, F., Vallières, B.R., Suss, J., Thériault, J.-D., Tremblay, S.: The CSSS microworld: a
gateway to understanding and improving CCTV security surveillance. In: Proceedings of
Human Factor and Erogonomics Society, vol. 60, pp. 265–269 (2016)
11. Funke, G., Greenlee, E., Carter, M., Dukes, A., Brown, R., Menke, L.: Which eye tracker is
right for your research? Performance evaluation of several cost variant eye trackers. In:
Proceedings of the Human Factors and Erogonomics Society, vol. 60, pp. 1240–1244 (2016)
12. Macinnes, J.J., Iqbal, S., Pearson, J., Johnson, E.N.: Wearable eye-tracking for research:
automated dynamic gaze mapping and accuracy/precision comparisons across devices, 28
June 2018, Unpublished manuscript
13. Keval, H., Sasse, M.A.: “Not the usual suspects”: a study of factors reducing the
effectiveness of CCTV. Secur. J. 23, 134–154 (2010)
14. Steelman, K.S., McCarley, J.S., Wickens, C.D.: Modeling the control of attention in visual
workspaces. Hum. Factors 53, 142–153 (2011)
Detecting Impulsive Behavior Through
Agent-Based Games
Alia El Bolock1,2(&), Ahmed Ghonaim1, Cornelia Herbert2,
and Slim Abdennadher1
1
German University in Cairo, Cairo 11432, Egypt
[email protected]
2
Department Applied Emotion and Motivation Psychology,
Ulm University, 89081 Ulm, Germany
Abstract. Impulsiveness is a trait that guides human behavior. People who
score high on self-report measures of impulsiveness are characterized by
unplanned risky behavior and acting without foresight. Character Computing
and Automatic personality recognition aim at detecting characteristics and traits
from different cues without relying on self-assessment measures. An individ-
ual’s behavior during agent-based games is one such cue. In this work, we
aimed to detect impulsiveness through two agent-based games: Tic Tac Toe and
Colored Trails. The developed agent estimated impulsiveness using Pearson
correlation and linear regression with an accuracy of 77% in comparison to self-
report questionnaires.
Keywords: Impulsiveness Personality Character Computing Agent-based
games Tic Tac Toe Colored Trails Regression
1 Introduction
Impulsiveness is a trait that strongly drives behavior. It is one of the risk factors that
contribute to impairments in mental health [12] and to disorders such as addictive
behavior [3] and obesity [13]. People who score high on self- report measures of
impulsiveness often have problems in decision making and in planning of behavior in
general. In addition, they show decreased sensitivity to negative consequences of
behavior, often acting without foresight (see [10]). Like all other personality traits,
impulsiveness is measured via self-report questionnaires. The TCI [10] is one such
measure which is not exclusively related to impulsiveness but measures it as one of the
facets belonging to a hierarchy of “top-level” traits. Self-report measures however are
challenged by a number of problems including the self-report bias [14]. Automatic
Personality Recognition (APR)) [16] and Character Computing [2, 7, 19, 20] tackle this
problem by investigating different ways of recognizing the personality traits and other
defining factors of an individual by analyzing individual behavior in different situations
(e.g. assessing an individual’s social media behavior. The aim of this paper is to
propose an APR technique by presenting different agent-based gaming scenarios for
detecting impulsiveness. We investigate whether the construct of impulsiveness can be
automatically detected from the interactions of an individual within certain gaming
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 208–213, 2020.
https://doi.org/10.1007/978-3-030-39512-4_33
Detecting Impulsive Behavior Through Agent-Based Games 209
scenarios. We implement two games (Colored Trails and Tic Tac Toe) within the
environment aimed at investigating impulsive behavior in the player’s interactions.
Both games have been used in previous studies to predict certain personality traits
[1, 15]. Akin to previous studies, in the present study the behavior during both games
was recorded and analyzed by the agent to form an estimate of the player’s impul-
siveness. The resulting value was then compared to the player’s measured impul-
siveness from self-report questionnaires. The estimated impulsiveness was found to be
highly correlated with the measured one and the corresponding linear regression model.
In the following, the design and results of the present study will be described in
detail (Sects. 3 and 4) on the basis of related previous work (Sect. 2).
2 Related Work
There has been a lot of work towards automatic personality recognition. One popular
approach is mapping behavioral characteristics extracted from smartphone data or
social media to self-reported personality traits [17]. One heavily investigated indicator
of personality traits is the behavior in games. For starters, the motivation behind online
game playing and personality are investigated in [11]. The relation between personality
and physiological effects on an individual whilst playing computer games is discussed
in [9]. In [4], the relationship between personality traits and satisfaction in on-line game
players using the big five model has been investigated. Of particular relevance to this
paper is predicting personality traits from Tic Tac Toe and CT. [15] is an example of
formative and psychological factor assessment of students through Tic Tac Toe game
playing. In [8], CT was used to find correlations between behavior and reward
dependence. Different ways to analyze strategies and behaviors of players in CT are
presented in [5] while unsupervised learning is used in [1] to map big five personality
traits to the behavior of different agent models in CT. However, the impulsiveness trait
has not been investigated using these means before.
3 Aim and Approach
Interactions in games are very telling of a person’s behavior and thus characteristics.
Impulsiveness can be very apparent when playing a game. As a proof of concept we
implemented two basic agent-based games: Tic Tac Toe and CT. Tic Tac Toe was
chosen as it is a very basic intuitive game that everyone is usually familiar with. It does
not introduce any high mental load and can thus show the intuitive reactions of the
players. CT was chosen as it is a relatively simple agent-based game that still provides
enough complexity to allow for measuring the difference in behavior between indi-
viduals. The interactions of the players with the game were automatically recorded and
analyzed by the game agents. This data was used by the agent to present an estimated
value of the player’s impulsiveness. In the following, we will briefly explain each
game, how its agent was implemented and how the agent calculated the player’s
impulsiveness.
210 A. E. Bolock et al.
3.1 Tic Tac Toe
The game is played on a 3X3 grid, where each player would place their mark (x or o).
A player wins upon having 3 vertical, horizontal or diagonal marks. Tic Tac Toe is a
zero-sum game, i.e. if both players are playing with an optimal strategy; every game
will end in a tie. The actions of the agent during the game were based on a variation of
the Minimax algorithm [6] where each player targets the move with the maximum
score in their turn. This is done by recursively calculating and comparing the score of
all possible moves from the moves tree. However, to avoid having an unbeatable agent,
a few losing strategies are incorporated into the agent and the agent alternates between
them, depending on the strategy of the player. The following was collected by the agent
during gameplay. (1) List of all the positions for player and agent. (2) Effect of each
position on player and agent (3) Time taken by the player for each move. (4) Winning
status: player, agent, tie.
3.2 Colored Trails Game
The Colored Trails Game [11] (CT) is a test-bed designed to learn about the cooper-
ativeness of an individual in their decision making. The board of the round-based game
is a 4 4 grid of colored squares. Each player starts each round at a specific position
and with a specific set of colored chips with the aim of reaching the goal square. The
player must own a chip of the same color of an adjacent square to move to it. Despite
being competitive, the players have to cooperate by negotiating chip exchange. The
communication phase is divided into three phases; offer, evaluation, and offer response.
Each round one player is the proposer and the other is the responder, alternatingly. The
initial chipsets of the player and the agent are generated and shuffled, depending on
their position and the board, using 4 variations based on the Manhattan distance to
calculate the route. The scoring function is
points ¼ 125 25 ðNumberOfSquaresÞ þ 10 ðUnusedChipsÞ: ð1Þ
For computing the best route, we traverse the whole board from the starting position
to the goal. We consider all possible routes based on the given chipset. The strategy of
the agent is determined by a complete search algorithm, where the agent tries to reach
the best scenario for both itself and the user. The agent tries to create a chipset for the
agent and the user where both of them reach the goal. If that is not possible, the agent
considers all the available chips and creates a win situation for itself without considering
the users state if the offer was accepted. Thus, the agent first analyzes the intentions of
the opponent and accordingly adapts its moves. The agent starts the game with a higher
probability of not helping the user or being selfish. After each round, the agent starts
analyzing the intentions of the user and then changes the probability of its actions. The
change made to the actions of the agent depends mainly on the favorability of the offer
made i.e. whether the offer was of benefit or did not intend to make the agent lose the
round. The favorability of the offer is determined by Δp.
Detecting Impulsive Behavior Through Agent-Based Games 211
Dp ¼ 125 25 ðNumberOfSquaresÞ þ 10 ðsumðchipSetÞ lenðRouteÞÞ: ð2Þ
The following data is automatically collected during gameplay (in reference to the
chipset at the beginning of a round). (1) Status of the agent/players before the offer (can
the goal be reached?). (2) Points of the agent/player if the offer is rejected/accepted.
(3) Response of the agent/player to the current offer. (4) Status of the agent/player if the
offer is accepted (can the goal be reached?). (5) Time taken for the player to make an
offer or respond to one. (6) The offered chipset of both the player and the agent.
3.3 Estimating Impulsiveness
The data collected by each game was then analyzed by the agent to calculate an estimate
for the current player’s impulsiveness. As inspired by [15], an average time for each
round was calculated based on all participants. Accordingly, a percentile calculation was
performed and the mean and standard deviation for the seconds per round was calcu-
lated. This was calculated separately for each game. This data was used to segment the
players into different groups by average time per game, as shown in Table 1. We also
separate the set of played rounds U into two sets: UF and UL. UF consists of the useful
offers and responses of CT and the useful moves of Tic Tac Toe. Accordingly, UL = U
\UF consists of all the useless actions. Given the function cat(r) which gives the speed
categorization of round r (Table 1), and the calculated impulsiveness constants per
speed category a, b, the estimated impulsiveness per user is calculated as
P P
r 2UF acatðrÞ r 2UL bcatðr Þ
: ð3Þ
rounds
Table 1. Speed categorization per round
No. Time Qualitative
Group 1 20 s/round Very Fast
Group 2 >20 s & 40 s Fast
Group 3 >40 s & 60 s Slightly Fast
Group 4 >60 s & 80 s Moderate
Group 5 >80 s & 100 s Slightly Slow
Group 6 >100 s & 120 Slow
Group 7 >120 s Very Slow
The indicators we chose and focus on are in line with previous findings from
psychological studies which suggest that impulsive individuals show shorter reaction
times and make more errors than less impulsive individuals in reaction time. However,
other (gaming) tasks and behavioral indicators may also prove worthwhile to differ-
entiate high from low impulsive individuals (e.g., measures of inhibitory control). To
validate the developed framework, we conducted an experiment, with the main aim of
comparing the estimated impulsiveness of the implemented agent to that of validated
212 A. E. Bolock et al.
self-report questionnaires. The participants consisted of 43 volunteer students recruited
from the university’s participant pool (23 males, 20 females, µage = 22.3). After signing
the consent form, the participants played 15 rounds of each game to record their
playing data and then took the TCI questionnaire as a measure for their impulsiveness.
This provided a record for each participant containing both, the estimated and the
reported impulsiveness. The Pearson Correlation between the estimated and the
reported impulsiveness was calculated. The correlation coefficient showed a high
correlation of 0.77 (p 0.01). As behavior can depend on the participants’ gender
[18], the correlation analysis was repeated separately for female and male participants.
It was found that the correlation was slightly higher for females (0.88, p 0.01) as
opposed to men (0.74, p 0.01). This shows that the impulsiveness measures should
be differently estimated for each gender, as it could affect the ktime, for example. Thus,
the grouping shown in Table 1 should be separately calculated for each gender. We
applied Linear Regression (LR) on the recorded values to predict the measured
impulsiveness and test whether the recorded data could reproduce the measured results.
We applied LR with 10-fold cross validation, which yielded r2 = 0.63 and mean
square error = 8.2 (p < 0.01). This is high, considering human behavior inherently has
a greater amount of unexplainable variation. Thus, the developed model is on the right
track but needs more training data. When adding the gender as a parameter the
accuracy increased to 0.66. Although the increase is not substantial, it shows that
adding more components defining the players is needed to accurately predict impul-
siveness from behavior cues.
4 Conclusion and Future Work
We presented two agent-based games (CT and Tic Tac Toe) for predicting impulsive
behavior in players. We conducted an experiment to collect data from 43 participants
and investigate the accuracy of the developed agents. For the personality measure of
impulsiveness, prediction accuracy is close to the test-retest accuracy of the used
impulsiveness assessment questionnaire (TCI). Further large scale testing should be
conducted while adding more parameters distinguishing between individuals, e.g. age,
occupation, affect and IQ [2, 17, 19, 20]. The agent model for estimating impulsiveness
should be adapted according to the different distinction measures between individuals,
as shown for gender. Estimation models for further traits should be added to the agents.
Finally, more gaming environments, e.g., Virtual Reality can be investigated.
References
1. Ahrndt, S., Albayrak, S.: Learning about human personalities. In: German Conference on
Multiagent System Technologies, pp. 1–18. Springer, Cham (2017)
2. Bolock, A.E., Abdelrahman, Y., Abdennadher, S. (eds.) Character Computing. Human
Computer Interaction Series. Springer, Switzerland (2020)
Detecting Impulsive Behavior Through Agent-Based Games 213
3. Brooks, S.J., Lochner, C., Shoptaw, S., Stein, D.J.: Using the research domain criteria
(RDoC) to conceptualize impulsivity and compulsivity in relation to addiction. In: Progress
in brain research, vol. 235, pp. 177–218. Elsevier (2017)
4. Chen, L.S.L., Tu, H.H.J., Wang, E.S.T.: Personality traits and life satisfaction among online
game players. CyberPsychol. Behav. 11(2), 145–149 (2008)
5. De Jong, S., Hennes, D., Tuyls, K., Gal, Y.K.: Metastrategies in the colored trails game. In:
The 10th Int. Conference on Autonomous Agents and Multi-agent Systems-Volume 2,
pp. 551–558. International Foundation for Autonomous Agents and Multiagent Systems
(2011)
6. Du, D.Z., Pardalos, P.M.: Minimax and applications, vol. 4. Springer, US (2013)
7. El Bolock, A.: Defining character computing from the perspective of computer science and
psychology. In: Proceedings of the 17th International Conference on Mobile and Ubiquitous
Multimedia, pp. 567–572. ACM (2018)
8. Gal, Y., Grosz, B.J., Kraus, S., Pfeffer, A., Shieber, S.: Colored trails: a formalism for
investigating decision-making in strategic environments. In: Proceedings of the IJCAI
Workshop on Reasoning, Representation, and Learning in Computer Games, pp. 25–30
(2005)
9. Griffiths, M.D., Dancaster, I.: The effect of type a personality on physiological arousal while
playing computer games. Addict. Behav. 20(4), 543–548 (1995)
10. Gutierrez-Zotes, J.A., et al.: Temperament and character inventory-revised (tci-r). Standard-
ization and normative data in a general population sample. Actas españolas de psiquiatría 32
(1), 8–15 (2004)
11. Jeng, S.P., Teng, C.I.: Personality and motivations for playing online games. Soc. Behav.
Pers. Int. J. 36(8), 1053–1060 (2008)
12. Kisa, C., Yildirim, S.G., Göka, E.: Impulsivity and mental disorders. Turkish J. Psychiatry
16(1), 46–54 (2005)
13. Mobbs, O., Crépin, C., Thiéry, C., Golay, A., Van der Linden, M.: Obesity and the four
facets of impulsivity. Patient Educ. Couns. 79(3), 372–377 (2010)
14. Van de Mortel, T.F., et al.: Faking it: social desirability response bias in self-report research.
Australian J. Adv. Nurs. 25(4), 40 (2008)
15. Prakash, V.C., et al.: Assessing the intelligence of a student through tic-tac-toe game for
career guidance. Int. J. Pure Appl. Math. 117(16), 565–572 (2017)
16. Vinciarelli, A., Mohammadi, G.: A survey of personality computing. IEEE Trans. Affect.
Comput. 5(3), 273–291 (2014)
17. El Bolock, A., Amr, R., Abdennadher, S.: Non-obtrusive sleep detection for character
computing profiling. In: International Conference on Intelligent Human Systems Integration,
pp. 249–254. Springer, Cham (2018)
18. El Bolock, A., El Kady, A., Herbert, C. Abdennadher, S.: Towards a character-based meta
recommender for movies. In: Computational Science and Technology, pp. 627–638.
Springer, Singapore (2020)
19. El Bolock, A., Salah, J., Abdelrahman, Y., Herbert, C., Abdennadher, S.: Character
computing: computer science meets psychology. In: 17th International Conference on
Mobile and Ubiquitous Multimedia, pp. 557–562. ACM (2018)
20. El Bolock, A., Salah, J., Abdennadher, S., Abdelrahman, Y.: Character computing:
challenges and opportunities. In: Proceedings of the 16th International Conference on
Mobile and Ubiquitous Multimedia. pp. 555–559. ACM (2017)
Visual and Motor Capabilities of Future Car
Drivers
Ferdinando Tripi1, Rita Toni2, Angela Lucia Calogero3,
Pasqualino Maietta Latessa4, Antonio Tempesta5, Stefania Toselli6,
Alessia Grigoletto6, Davide Varotti6, Francesco Campa6,
Luigi Manzoni7, and Alberto Vergnano8(&)
1
Ass. Volontariato “Insieme si Può”, via Canalino 67, 41123 Modena, Italy
[email protected]
2
Poliambulatorio Chirurgico Modenese, via Arquà 5, 41125 Modena, Italy
[email protected]
3
ASD inMo.To., via De’ Roberti 34, 41124 Modena, Italy
[email protected]
4
Department for Life Quality Studies, University of Bologna,
corso D’Augusto 237, Rimini, Italy
[email protected]
5
Automobile Club Modena, viale G. Verdi 7, 41121 Modena, Italy
[email protected]
6
Department of Biomedical and Neuromotor Sciences, University of Bologna,
via Ugo Foscolo 7, 40123 Bologna, Italy
{stefania.toselli,francesco.campa3}@unibo.it,
{alessia.grigoletto,davide.varotti}@studio.unibo.it
7
Poliambulatorio CFT Città di Vignola, viale Giuseppe Mazzini 5/2,
41058 Vignola, Italy
[email protected]
8
Department of Engineering Enzo Ferrari, University of Modena and Reggio
Emilia, via P. Vivarelli 10, 41125 Modena, Italy
[email protected]
Abstract. Driving safety is recognized as critical for young people by insti-
tutions, insurances and research. The ability to manage such a complex activity
as driving is still developing through adolescence and in early adulthood. The
present research investigates the human factors in the driver-car interaction. The
experimental method assesses the visual-motor coordination capabilities of
future drivers, also in relation to their life styles. The results show that a frequent
but good quality physical activity improves visual-motor coordination.
Keywords: Visual-Motor coordination Distance Rock test Peripheral Wall
Chart Broc String Motor Efficiency Test Human factors
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 214–220, 2020.
https://doi.org/10.1007/978-3-030-39512-4_34
Visual and Motor Capabilities of Future Car Drivers 215
1 Introduction
Road accidents are the leading cause of injury and death among people aged 15 to 29,
[1]. Literature identifies greater risks when driving for the elderly, [2] and for young
people, [3]. The driving styles of elderly are influenced by the decline in cognitive
abilities, neck stiffness and loss of vision. On the other hand, young people present
deficiencies in driving ability for reasons related to the maturation of the frontal lobe
through adolescence and in early adulthood, [4, 5]. The ability of an individual to
manage such a complex activity as driving is called Executive Function (EF), subdi-
vided into working memory, inhibition and set-shifting, [3]. Working memory is the
ability to monitor, update and manipulate information with readiness, [6]. Inhibitory
control is the ability to filter and resist to distractors and noise, [5]. Finally, set-shifting
enables to adapt to changes from the outside context or in the planned goal of driving
action, [5]. In a danger situation, the instinct of self-preservation pushes the brain to
process visual information as quickly as possible in order to command the body to react
to ensure survival. It happens that a car accident is not directly caused by lack of danger
perception, but more by the incapacity of adequate motor reaction.
Playing videogames can improve the adolescent EF, [7], but also challenges them
with an intense activity in an obscured room and focused on a monitor, without focus
changes or external stimuli. For this reason, young people of school age are increas-
ingly lacking in perception and coordination skills. The same phenomenon was also
observed in the ophthalmologic field: the ability to focus on near and/or distant ele-
ments in a rapid and accurate manner, as well as the ability to locate a physical object in
space, decreased. During growth, the link between EF and individual differences,
personality traits, social and emotional context is particularly important. The goal of the
present research work is to investigate the visual-motor coordination of future drivers.
The developed method of investigation would be useful both for the educational system
and for assessing the human factors in the car-driver interaction.
The paper is organized as follows. Section 2 introduces the research method,
Sect. 3 reports and discusses the results while conclusions are drawn in last Sect. 4.
2 Method of Investigation
The experimental method is conceived to link the three skills identified as fundamental
for driving (perception, elaboration and response) to the lifestyle of each individual.
Each test session consists of three phases, as described hereafter. The tests are per-
formed in primary and secondary school, for collecting data about future drivers which
are still developing their EF.
2.1 Individual Experiences
The psychophysical potential of individuals is positively or negatively exploited with a
leverage effect from the attitude and driving behavior. Being aware of the surrounding
environment, assessing the situation and responding with timely and appropriate
actions are all skills that can be developed through adequate and constant motor
216 F. Tripi et al.
activity. Data on physical activity of young people are collected with the Physical
Activity Questionnaire for Children (PAQ-C) and for Adolescents (PAC-A), [8].
PAQ-C and PAQ-A have been designed to monitor physical activity in the 7 days
before the test.
2.2 Visual Abilities
A driver tends to stare straight ahead in the travel direction, towards the position the
vehicle will soon take, [9]. The driver will then rapidly scan left and right. There is a
direct link between eye movements and attention. For many experiments on the field,
rapid standardized tests indicative of the ability of visual perception have been elab-
orated, [10, 11], and here adopted.
First, the Distance Rock test (DR) evaluates the ability of the subject to make rapid
and accurate focus shifts from far to near and back. The test assesses the flexibility of
accommodation and vergences. The test requires two tables of letters, as shown in
Fig. 1a. The subject must stand still at a predetermined distance from the larger table,
supporting the smaller one with hands. The test consists in reading the letters as fast as
possible from far to near and back (one cycle), and again for 30 s. The results of the test
are the (largest possible) numbers of cycles without errors to near (DR20/25) or far (DR
20/80) tables.
Fig. 1. Visual ability tests: (a) Distance Rock, (b) Peripheral Wall Chart, (c) Broc String.
Peripheral vision is particularly important for detecting potential dangers in a
poorly structured context, such as driving in traffic. The Peripheral Wall
Chart (PWC) test assesses the peripheral perception. The subject must stand still at a
distance of 40 cm from the chart shown in Fig. 1b. He/she must therefore read the
letters arranged in circles while keeping the sight fixed on the central point.
The Broc String test investigates the ability to locate an object in space. One end of
a rope is held by the subject under the nose tip, the other end by the researcher. A 1 cm
diameter ball stands on the rope between them, as shown in Fig. 1c. The test is based
on the physiological diplopia. By fixing the ball at 3 m from the subject, the objects
placed before or after are perceived as double, so that the two strings form an X. If the
cross occurs first, it indicates esoforia, if after exoforia.
Visual and Motor Capabilities of Future Car Drivers 217
2.3 Physical Efficiency
As in driving, observing space, assessing the situation and responding quickly and
appropriately with a movement of the body, arms or legs, are peculiar demands of sport
activities. So, the physical and coordination capabilities of the response will be eval-
uated with the Motor Efficiency Test (MET). MET has been developed by the Institute
of Sports Medicine and Science of the Italian National Olympic Committee (CONI),
[12]. MET consists of a circuit, as shown in Fig. 2, with 4 stages to be reached with 4
different gaits. The subject has to perform a coordination exercise in each stage. In the
end, the subjective perception of effort is assessed through the Borg scale, [13]. The
physical ability of a subject results both from speed and coordination.
Fig. 2. MET circuit and stages.
3 Experiment Results
The introduced method was tested with 647 students from Primary School (PS), Lower
Secondary School (LSS) and Upper Secondary School (USS). The scores achieved in
PS are significantly lower than in LSS and USS, as shown in Table 1. Then, for female
students, there is no clear difference between the scores from LSS to USS, except for
the DR tests. On the contrary, the scores of male students significantly increase in each
school level. At LSS females perform better than males, but in USS the males overtake
females. In fact, that 12–13 years age involves the adolescent shooting for girls, fol-
lowed only two years later by the male one.
The students are divided into five groups based on the Amount of physical activity
practiced, as shown in Table 2: GA0 = no physical activity, GA1 = 1–2 times a week,
GA2 = 3–4 times a week, GA3 = 5–6 times a week, GA4 = 7+ times a week. A sig-
nificant difference is observed between students do not practicing physical activity,
218 F. Tripi et al.
Table 1. ANOVA test for comparison of Visual Abilities and Physical Efficiency tests for
female and male students of different grade schools.
Test PS LSS USS F P-value
Female students DR20/80 9.35 ± 2.59 11.89 ± 3.27 13.93 ± 2.60 62.65 <0.001
DR20/25 7.80 ± 2.55 10.27 ± 2.93 11.51 ± 2.79 42.79 <0.001
PWC 3.70 ± 1.20 4.32 ± 0.93 4.30 ± 0.93 9.09 <0.001
MET 13.80 ± 4.43 17.16 ± 2.61 17.09 ± 3.42 21.29 <0.001
Male students DR20/80 9.08 ± 2.20 11.58 ± 3.06 13.69 ± 2.86 77.65 <0.001
DR20/25 7.28 ± 2.26 9.84 ± 2.65 10.81 ± 2.83 49.95 <0.001
PWC 3.61 ± 1.28 3.88 ± 1.24 4.10 ± 0.97 4.33 0.014
MET 13.76 ± 3.89 16.93 ± 3.52 19.19 ± 3.30 57.36 <0.001
Table 2. ANOVA test for evaluation of Visual Abilities and Physical Efficiency tests against the
amount of physical activity practiced.
Test GA0 GA1 GA2 GA3 GA4 F P-value
PS DR20/80 8.29 ± 2.10 9.25 ± 2.46 9.28 ± 2.64 9.79 ± 1.96 8.71 ± 2.41 1.33 0.260
DR20/25 6.81 ± 2.23 7.30 ± 2.81 7.80 ± 2.20 8.29 ± 2.20 7.71 ± 2.33 1.36 0.249
PWC 3.48 ± 1.29 3.46 ± 1.29 3.78 ± 1.04 4.29 ± 1.04 4.10 ± 1.04 3.03 0.019
MET 11.86 ± 4.28 13.03 ± 4.60 13.26 ± 3.26 15.13 ± 3.49 15.14 ± 3.58 3.11 0.017
LSS DR20/80 10.00 ± 0.00 12.50 ± 3.12 10.63 ± 2.00 12.20 ± 4.60 14.33 ± 1.53 1.17 0.352
DR20/25 6.50 ± 0.71 10.38 ± 3.16 9.13 ± 1.73 10.60 ± 2.41 10.67 ± 1.16 1.50 0.238
PWC 2.50 ± 0.71 3.75 ± 1.39 5.00 ± 0.93 3.80 ± 1.64 4.67 ± 0.58 2.32 0.090
MET 19.50 ± 2.12 16.63 ± 2.88 17.38 ± 1.85 17.40 ± 2.88 17.67 ± 0.58 0.60 0.666
USS DR20/80 13.79 ± 3.16 13.43 ± 2.83 14.47 ± 2.93 14.30 ± 2.79 13.73 ± 2.45 1.02 0.396
DR20/25 11.21 ± 3.19 10.75 ± 2.40 11.28 ± 3.13 11.67 ± 2.57 10.67 ± 2.98 0.74 0.563
PWC 4.18 ± 0.95 4.16 ± 0.99 4.30 ± 0.87 4.07 ± 1.14 4.20 ± 0.96 0.29 0.888
MET 16.09 ± 3.73 17.82 ± 3.64 18.09 ± 3.76 18.74 ± 2.75 18.63 ± 3.44 3.11 0.016
Table 3. ANOVA test for evaluation of Visual Abilities and Physical Efficiency tests against the
type of physical activity practiced.
Test GT0 GT1 GT2 GT3 F P-value
PS DR20/80 8.29 ± 2.10 9.71 ± 2.40 9.19 ± 2.44 9.08 ± 2.48 1.61 0.189
DR20/25 6.81 ± 2.23 7.58 ± 2.40 7.83 ± 2.79 7.57 ± 2.21 0.90 0.445
PWC 3.48 ± 1.29 3.82 ± 1.11 3.49 ± 1.21 4.02 ± 1.13 2.46 0.064
MET 11.86 ± 4.28 13.63 ± 3.00 13.69 ± 4.65 13.75 ± 3.84 1.30 0.275
LSS DR20/80 10.00 ± 0.00 11.67 ± 3.60 12.38 ± 2.72 12.50 ± 2.65 0.38 0.770
DR20/25 6.50 ± 0.71 10.17 ± 2.44 9.75 ± 2.61 10.25 ± 2.06 1.40 0.268
PWC 2.50 ± 0.71 4.42 ± 1.44 3.88 ± 1.36 4.75 ± 0.50 1.65 0.206
MET 19.50 ± 2.12 16.42 ± 2.94 18.13 ± 0.84 17.50 ± 1.29 1.65 0.207
USS DR20/80 13.79 ± 3.16 13.94 ± 2.86 14.21 ± 2.71 13.59 ± 2.75 0.49 0.692
DR20/25 11.21 ± 3.19 10.85 ± 2.73 11.45 ± 2.76 10.55 ± 3.04 1.06 0.369
PWC 4.18 ± 0.95 4.11 ± 1.02 4.23 ± 0.98 4.27 ± 0.87 0.26 0.855
MET 16.09 ± 3.73 19.31 ± 3.04 17.41 ± 3.48 18.48 ± 3.71 6.91 <0.001
Visual and Motor Capabilities of Future Car Drivers 219
GA0, and those who practice intense sports, GA3 and GA4. This difference was not
detected when comparing GA0 with those practicing moderate physical activity, GA1
and GA2.
The same students are also divided into groups based on the Type of physical
activity performed, as shown in Table 3: GT0 = no physical activity, GT1 = team
sports, GT2 = individual sports, G3 = both team and individual sports practice. Con-
sidering the PWC, GA3 performs better than GA0, GA1 and GA2, while GT3 perform
better than GT0, GT1 and GT2. The best results, for students practicing both team and
individual sports, indicate how the practice of different sports can better develop of
peripheral vision, probably as an adaptation to the various movement scenarios.
4 Conclusions
The purpose of the present study is to investigate the visual and motor capabilities of
future car drivers, thus 6 to 19 years students, in relation with age, sex, amount and type
of physical activity practiced. The results indicate a clear relationship between great
motor skills and practice of physical activity during childhood and adolescence.
About visual performances, the students practicing team sports turn out to be better
performing than the ones not practicing physical activity or the ones practicing indi-
vidual sports, probably because the team sports include the perception of the teammate
positions together with a moving ball. Finally, the results of simple and multiple
regressions about visual tests and motor testing associations show significant correla-
tion between the motor test with the DR20/80 and DR20/25, indicating that in some
cases the best motor performances are accompanied by good visual performances.
The study verified that frequent but good quality physical activity improves the
manual-eye coordination of future drivers. Further research will study methods and
technologies to train the visual capabilities with sport vision methodologies.
Acknowledgments. This work is financed by Fondazione Cassa di Risparmio di Modena. The
authors gratefully acknowledge the principals and teachers of the primary and secondary schools
involved: La Carovana, IPSIA Ferrari, IPSSCA Cattaneo-Deledda, ITIS Meucci, ITIS Levi,
Liceo Wiligelmo.
References
1. World Health Organization. Road traffic injuries. Fact sheet. http://www.who.int/
mediacentre/factsheets/fs358/en/. Accessed April 2018
2. Choi, J., Tay, R., Kim, S., Jeong, S.: Turning movements, vehicle offsets and ageing drivers
driving behaviour at channelized and unchannelized intersections. Accid. Anal. Prev. 108,
227–233 (2017)
3. Walshe, E.A., McIntosh, C.W., Romer, D., Winston, F.K.: Executive function capacities,
negative driving behavior and crashes in young drivers. Int. J. Environ. Res. Public Health
14(11), 1314 (2017)
220 F. Tripi et al.
4. Huizinga, M., Dolan, C.V., Van der Molen, M.W.: Age-related change in executive
function: developmental trends and a latent variable analysis. Neuropsychologia 44(11),
2017–2036 (2006)
5. Diamond, A.: Executive functions. Annu. Rev. Psychol. 64, 135–168 (2013)
6. Baddeley, A.: Working memory: theories, models, and controversies. Annu. Rev. Psychol.
63, 1–29 (2012)
7. Moisala, M., Salmela, V., Hietajärvi, L., Carlson, S., Vuontela, V., Lonka, K., Hakkarainen,
K., Salmela-Aro, K., Alho, K.: Gaming is related to enhanced working memory performance
and task-related cortical activity. Brain Res. 1655, 204–215 (2017)
8. Kowalski, K.C., Crocker, P.R., Donen, R.M.: The physical activity questionnaire for older
children (PAQ-C) and adolescents (PAQ-A) manual. Coll. Kinesiol. Univ. Saskatchewan
87(1), 1–38 (2004)
9. Underwood, G., Chapman, P., Brocklehurst, N., Underwood, J., Crundall, D.: Visual
attention while driving: Sequences of eye fixations made by experienced and novice drivers.
Ergonomics 46, 629–646 (2003)
10. Roncagli, V.: Sports Vision. Accademia Europea di Sports Vision (1991)
11. Teig, D.S.: High Performance Vision. Square One (2015)
12. Motor Efficiency Test. www.coni.it. Accessed April 2018
13. Eston, R.G., Davies, B.L., Williams, J.G.: Use of perceived effort ratings to control exercise
intensity in young healthy adults. Eur. J. Appl. Physiol. Occup. Physiol. 56(2), 222–224
(1987)
A Fixation-Click Count Signature as a Visual
Monitoring Enhancement Feature for Air
Traffic Controllers
Hong Jie Wee1,2(&), Sun Woh Lye1, and Jean-Philippe Pinheiro3
1
School of Mechanical and Aerospace Engineering, Nanyang Technological
University, 50 Nanyang Avenue, Block N3, Singapore 639798, Singapore
[email protected]
2
Thales Solutions Asia Pte Ltd.,
21 Changi North Rise, Singapore 498788, Singapore
3
Thales LAS France SAS, 3, Avenue Charles Lindbergh, 94628 Rungis, France
Abstract. This paper presents an approach to capture the deliberate perception
and comprehension behavior of Air Traffic Controllers (ATCOs) by utilizing
their eyeball and hand movement. Hence, a new fixation-click count signature,
involving ATCOs fixating on aircraft’s track or label, before performing
respective commands within a specified operational duration, is derived. 88 one-
hour real time simulations involving a remote eye tracker and scenarios mim-
icking actual air traffic were conducted for participants from three expertise
levels. Results show that this signature is common amongst all participants for
both incoming and outbound aircraft. Furthermore, addition visual attention
span is required with increasing commands issued, as a higher fixation count is
exhibited with more clicks performed for the one-hour duration. This signature
could also be used to differentiate the expertise levels of ATCOs, as experts
were found to exhibit the most fixation count across all click counts, followed by
intermediates and novices.
Keywords: Tactical Air Traffic Control Eye tracking signature Smart
human-centered systems
1 Introduction
New concepts of operations like the adoption of 4-Dimensional Trajectory Based
Operations (4D-TBO), where the flight trajectory of aircraft can be predicted in a
timely, accurate and consistent manner, and the increasing use of automation brought
about dynamic changes in Air Traffic Control (ATC) in recent years. As a result,
aircrafts are freer to fly on their own desired trajectories, creating a more complex air
route structure with tighter separation standards [1, 2].
Future Air Traffic Controller (ATCO) tasks would likely be dedicated to per-
forming more tactical monitoring activities presented on radar displays with lesser
flight crew communication, thereby changing the way ATCOs operate [3]. This brings
about a new set of challenges, as there is a lack of knowledge extraction and repre-
sentation in present ATC systems, to determine how ATCOs monitor the dynamic
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 221–227, 2020.
https://doi.org/10.1007/978-3-030-39512-4_35
222 H. J. Wee et al.
flight objects on the radar display in order to maintain awareness of the air traffic
situation. Studies also show that radar display error constitutes 60% of ATCO opera-
tional error [4], while perception and vigilance in planning and radar monitoring is the
most significant cognitive error of ATCOs [4, 5].
With a more complex air route design predicted in future, it is envisioned that an
ATCO’s operational performance during tactical radar monitoring sessions would be
impacted. Therefore, there is a need to conduct studies to capture and evaluate an
ATCO’s perception and comprehension level, for current and future ATC operations
[1, 6].
2 Fixation-Click Count Signature
To capture and evaluate the perception and comprehension level of ATCOs for their
tactical monitoring activities, a fixation-click count signature is proposed. This signa-
ture seeks to capture the deliberate perception and comprehension behavior of ATCOs
while performing tactical monitoring activities on the different flight objects on the
radar display.
In this signature, the eyeball and hand movement of an ATCO is utilised, which
involves the ATCO fixating on the track or label of an aircraft, before performing
respective commands within a specified operational duration. Examples of these
commands are taking jurisdiction of incoming aircraft (UCO), handing over of out-
bound aircraft to adjacent sector (TOC) and many more. Hence, if an ATCO would
have perceived and comprehended an aircraft during the tactical monitoring activities,
he/she would have at least fixated once on the aircraft (criteria 1) and clicked on the
appropriate command (criteria 2), within a specific duration. Figure 1 shows a flow-
chart, depicting the process of how this signature works. Using these 2 criteria, a binary
signature can then be created and evaluated across the 3 expertise levels for aircraft
requiring a frequency change and across the one-hour duration.
Fig. 1. Flowchart of fixation-click count signature
A Fixation-Click Count Signature as a Visual Monitoring Enhancement Feature 223
3 Concept Test and Validation
For the experiment, a real-time simulator, the NLR ATM Research SIMulator
(NARSIM), was used to generate and simulate air traffic scenarios in real time, with
every radar update at 9.8 s. Pseudo-pilots were used to facilitate the flying of aircraft,
by following the same scenario script, so that events in the same scenario occurred at
the same instant. A standard Controller Working Position (CWP), consisting of a 2 K
radar screen (50 cm by 50 cm) was used to display air traffic data in a given airspace
sector, as seen in Fig. 2.
Fig. 2. Standard controller working position (CWP)
A remote eye tracker, Tobii X2-30 was used to capture the eye data of the par-
ticipants in a non-intrusive manner. Prior to this study, a separate study was conducted
to determine the accuracy and precision of the eye tracker without a chinrest in this
condition, which is found to be 3.5° and 1.1° respectively [7, 8]. Raw eyeball move-
ment and radar data are extracted from the Tobii eye tracker and NARSIM respectively
via an intelligent real-time post processing server that has been developed, which
ensures that the two data streams are synchronous in time. These extracted data were
then relayed to the Thales TopSky-HF software to record the data of the different test
participants [9, 10].
88 one-hour experimental sessions of data were collected among male participants
whose ages varied between 24 and 41 (M = 29, SD = 5.34). These sessions comprised
24 novice sessions (zero ATC experience, only ATC knowledge), 20 intermediate
sessions (more than 50 h of simulator ATC training and ATC knowledge, yet to handle
live traffic) and 44 expert sessions (licensed ATCOs with more than three months of
live traffic experience) [11].
Prior to the experiment, each participant involved in a session underwent an initial
9-point eye-tracker calibration exercise. Calibration was repeated until a satisfactory
calibration was obtained that refers to Tobii Studio’s qualitative calibration report.
Real time simulations were conducted for 88 one-hour experimental sessions, with
different scenarios consisting of 60 aircraft each, that mimic operational air traffic,
which is approximately to 1.2 times of that of the regular traffic volume. In each
224 H. J. Wee et al.
scenario, there is a mixture of climbing and cruising flight along the different airways
(flight path). The airspace of interest in this real time study was based on an existing
airspace sector, with information referenced from the publicly available SkyVector
website [12].
4 Discussion of Results
4.1 Aircraft Frequency Change
For the activity of aircraft frequency change, the fixation-click count signature is
evaluated both incoming and outbound aircraft for the ‘UCO’ and ‘TOC’ commands
respectively, for a specified operational duration of 2 min. Such is performed from a
total of 45 selected aircraft in the scenarios, comprising of 270, 225 and 495 novice,
intermediate and expert instances across the 88 experimental sessions respectively.
The number of participants across all 3 expertise groups, who have exhibited the
aircraft frequency switch signature for incoming and outgoing aircraft by fulfilling the
two criteria for all sessions, are seen in Tables 1 and 2 respectively. In addition, both
Tables 1 and 2 also showcase the proportion of participants who have fulfilled only
criteria 1 and criteria 2 for all sessions respectively.
Table 1. Number of participants fulfilling fixation-click count signature criteria for incoming
aircraft
Incoming aircraft (UCO command) Novice Intermediate Expert
Number of participants who fulfilled both 235/270 212/225 455/495
Criteria 1 and 2 (0.87) (0.94) (0.91)
Number of participants who fulfilled only 235/270 212/225 455/495
Criteria 1 (0.87) (0.94) (0.91)
Number of participants who fulfilled only 270/270 225/225 495/495
Criteria 2 (1) (1) (1)
From Tables 1 and 2, it is observed that participants from all expertise groups can
exhibit the fixation-click count signature, as an average of 91% of the participants
fulfilled both Criteria 1 and 2 for ‘UCO’ command, while an average of 86% of the
participants fulfilled both Criteria 1 and 2 for ‘TOC’ command. These observations
showed that the two criteria forming the fixation-click count signature are noticeable
actions for all participants for both ‘UCO’ and ‘TOC’ commands. Such meant that
fixating on the aircraft and clicking on the respective commands on the aircraft’s label,
whenever there is a need for a frequency switch, is a common action amongst par-
ticipants. Therefore, the new fixation-click count signature could be used as a suitable
measure to identify if a participant has perceived and comprehended an aircraft during
this activity.
A Fixation-Click Count Signature as a Visual Monitoring Enhancement Feature 225
Table 2. Number of participants fulfilling the fixation-click count signature criteria for
outbound aircraft
Outbound aircraft (TOC command) Novice Intermediate Expert
Number of participants who fulfilled both 231/270 196/225 425/495
Criteria 1 and 2 (0.86) (0.87) (0.86)
Number of participants who fulfilled only 231/270 196/225 425/495
Criteria 1 (0.86) (0.87) (0.86)
Number of participants who fulfilled only 258/270 215/225 470/495
Criteria 2 (0.96) (0.96) (0.95)
4.2 One-Hour Duration
During the one-hour duration, the total number of fixation-click count signature
exhibited was computed, together with its fixation count for each click. Figure 3 shows
the distribution of the number of click count and fixation count for every radar update
frame of 9.8 s each.
Fig. 3. Distribution of fixation and click count across all frames
Results show that within every radar update frame, a higher click count results in a
higher fixation count on the flight objects. Such indicates that as more clicks (com-
mands) were issued in a frame, more attention is needed on the flight objects visually,
thereby giving rise to a higher fixation count.
Additionally, experts are found to exhibit 15% and 8% more fixation count than
novices and intermediates respectively, whereas intermediates exhibited 6% more
fixation count than novices, across all click counts. Such highlights that with more
experience and expertise, participants tend to exhibit a higher fixation count regardless
of click count. This further indicates a more focused attention span by participants with
226 H. J. Wee et al.
a higher level of expertise. This aspect to differentiate an ATCO’s expertise level using
the fixation-click count signature is an interesting phenomenon worth further study.
5 Conclusion
A fixation-click count signature involving the ATCO’s fixation on the track or label,
before clicking on respective commands within a specified duration based on opera-
tional requirements, is presented.
Using 88 one-hour real time simulations with a remote eye tracker, results show
that this signature is a common action amongst participants across all expertise levels
on aircraft requiring frequency switch (91% incoming and 86% outgoing aircraft). For
one-hour duration, results indicate that for all participants, a higher fixation count is
exhibited whenever there are more clicks performed. Such shows that more visual
attention span is required whenever more commands are issued. Furthermore, experts
are found to exhibit the most fixation count across all click counts, followed by
intermediates and novices. This shows that a higher level of expertise would likely give
rise to a more focused attention span. As a result, this signature could be further
explored and used to differentiate the expertise levels of ATCOs.
Operational standards based on the fixation-click count signature can also be
derived and applied in future ATC systems in the form of alerts. Instructors can also
better evaluate the awareness level of prospective ATCOs, paving the way for a hybrid
cognitive system to be incorporated into future ATC systems.
Acknowledgments. This research is funded by Thales Solutions Asia Pte Ltd, under the Eco-
nomic Development Board, Industrial Postgraduate Programme, with Nanyang Technological
University, Singapore. The authors would like to acknowledge and thank the staff at Thales LAS
France SAS, Thales Solutions Singapore, Air Traffic Management Research Institute of Nanyang
Technological University, Singapore and the participants in this study for their contributions and
support towards this work.
References
1. Wickens, C.D., et al.: The Future of Air Traffic Control: Human Operators and Automation.
National Academies Press (1998)
2. Bowen, D.: The SESAR concept and i4D. In: Educational Workshop in ATM Global (2014)
3. Dittmann, A., Kallus, K., Van Damme, D.: Integrated Task and Job Analysis of Air Traffic
Controllers-Phase 3-Baseline Reference of Air Traffic Controller Tasks and Cognitive
Processes in the ECAC Area. European Organisation Safety Air Navigation (2000)
4. Pounds, J., Isaac, A.: Validation of the JANUS technique: causal factors of human error in
operational errors. DTIC Document (2003)
5. Pounds, J., Isaac, A.: Preliminary test of two models for human factors analysis of
operational errors. In: Proceedings of the 11th International Symposium on Aviation
Psychology, Columbus, Ohio (2001)
A Fixation-Click Count Signature as a Visual Monitoring Enhancement Feature 227
6. Metzger, U., Parasuraman, R.: The role of the air traffic controller in future air traffic
management: an empirical study of active control versus passive monitoring. Hum. Factors:
J. Hum. Factors Ergon. Soc. 43(4), 519–528 (2001)
7. Clemotte, A., et al.: Accuracy and precision of the Tobii X2-30 eye-tracking under non ideal
conditions. In: Proceedings of the 2nd International Congress on Neurotechnology,
Electronics and Informatics - Volume 1: NEUROTECHNIX, p. 2 (2014)
8. Holmqvist, K., Nystrom, M., Mulvey, F.: Eye tracker data quality: what it is and how to
measure it. In: Proceedings of the Symposium on Eye Tracking Research and Applications,
Santa Barbara, California, pp. 45–52. ACM (2012)
9. Wee, H.J., Lye, S.W., Pinheiro, J.-P.: Real time eye tracking interface for visual monitoring
of radar controllers. In: AIAA Modeling and Simulation Technologies Conference.
American Institute of Aeronautics and Astronautics (2017)
10. Wee, H.J., et al.: Real time bio signal interface for visual monitoring of radar controllers. In:
Transdisciplinary Engineering: A Paradigm Shift: Proceedings of the 24th ISPE Inc.
International Conference on Transdisciplinary Engineering, 10–14 July 2017. IOS Press
(2017)
11. ICAO: Annex 1, Personnel licensing. In: International Standards and Recommended
Practices, Montreal, Canada (2011)
12. SkyVector. SkyVector Aeronautical Charts (2006/2018). https://skyvector.com/
Digital Transformation in Product
Service System for Kids. Design Tools
for Emerging Needs
Benedetta Terenzi1(&) and Arianna Vignati2
1
Environmental and Civil Engineering Department,
University of Perugia, Perugia, Italy
[email protected]
2
Department of Design, Politecnico di Milano, Milan, Italy
Abstract. This paper presents findings and discussion of the research con-
ducted in 2018 by the Observatory Kids&Toys of Politecnico of Milan with the
aim to understand the problems and emerging needs in the design sector linked
to children in the digital transformation era. The main focus is to highlight the
impact that the human-centered design approach can have in the creation of new
scenarios and innovative use of digital technologies for kids. Consequently, to
design new Product Service System more in line with the needs and the new
behaviours of kids and parents. In this sense, the research has identified how
much is changing the dialogue between companies in toys sector and their
customers (adults and children).
Keywords: Design for kids Human-centered design Product service
system Digital transformation Children needs
1 The Design Practices in Kids&Toys Sector
Making sense means for companies to make decisions, create changes that affect
business strategies. For designers make sense is a creative action that starts from the
user observation, the environment and the identification of an area of innovation able to
generate something new for everyone (companies, consumers, etc.) Designers bring out
meanings and lifestyle from the reality: they observe users, they understand their needs
and their relation with products and services and they translate these elements in
characteristics for new products and services. Between apprehension and opportunities
we are living through a sense of confusion and uncertainty on how children of the
future will engage with products and services [1].
Design has become more and more pervasive for businesses. The innovation
generated by a design-oriented approach (or design inspired) has enabled companies to
extend the boundaries of relation between the inside dimension (know–how, tech-
nologies, processes, human capital) and the outside dimension (the market, the cus-
tomers, the socio-political context, the environment, etc.) [2].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 228–234, 2020.
https://doi.org/10.1007/978-3-030-39512-4_36
Digital Transformation in Product Service System for Kids 229
Through a qualitative analysis, the research has identified digital transformation’s
social effects, user’s emerging needs and how much is changing the dialogue between
companies in toys sector and their customers (adults and children).
2 The Digital Technologies’ Impact in Children Play
Behaviors
The temporary nature of childhood is linked to a project that is now aware of the
specific needs of this complex user, and of the importance of the playful aspect
associated with them. Talk about new meanings in children’s products means con-
sidering that play has a crucial role in the optimal growth, learning and development of
children from infancy to adolescence [3].
To play is a dynamic process and it is considered a key facilitator for learning and
development in all domains and reflects the social and cultural contexts in which
children live [4–7].
The digital transformation has certainly changed the way to play and the medium
(the toy) with which the child shapes this experience. Children have always played for
what was in their possibilities, but the ways, the forms, the objects of their toys have
always been related to the culture and the social conditions [8]. Toy is therefore a
means, a medium that allows the child to make play a life experience. The design
approach can not underestimate the fact that toys are symbol of the world [9].
A further aspect of contemporary play is the relationship between online and offline
spaces [10]. In the last few years we have seen various developments in relation to the way
in which toys and other artefacts for children are mediated by digital interaction [11, 12],
and today it is possible to classify products according to the technologies used [13].
But if on one hand the toys physical innovation is consolidated [14–16], on the
other the scope of intervention of the service product is more complex and difficult to
define [17].
What the new technologies have helped to change the most are the behaviors [18].
Consumers today show and live unexpected behaviors, creating sudden alliances by
personalizing the media and at the same time creating new types of exchange, making
accessibility and excellence, culture and consumption meet. Younger people participate
in metamorphosis and often [19], without knowing it, naturally metabolize techno-
logical innovation and transform it into social innovation, involving other generations:
that of parents and even that of grandparents [20].
The research has operated with the aim of making world-wide clarity on the trends
and interests of companies and stakeholders in general, on the role of different types of
advanced technology applied to products and services for children.
3 A Case Studies Collection to Frame the Design Context
In 2018 a dedicated Observatory on Kids&Toys has been established to monitor with a
long term perspective the impact of the digital technologies in the development,
production and distribution of products and services addressed to kids and parents.
230 B. Terenzi and A. Vignati
Within this objective a specific research task has been addressed to understand the use
of digital technologies in the most innovative solutions of Product Service Systems
for kids.
In fact, the existence of a new user experience for children and parents was
highlighted in light of a new perspective of innovation in the toy sector (Fig. 1). This
sees the product as part of a complex system of elements ranging from physical space,
to service, to all artifacts and communication strategies, which integrate digital in the
design process [21].
Fig. 1. Product service system in design for kids
But what does the market offer internationally? The research considered the best
cases of innovation that focus on product systems that integrate digital technologies.
From the 62 international cases, 5 macro categories of response to the needs of children
emerge: motor skills, creativity, education, care and entertainment. A specific tool for
the collection of data, the visualization and the analysis of the most important insights
has been designed (Case Study Canvas) (Fig. 2).
The canvas includes different field of data collection:
• Target: identify the users and the customers for the solution;
• Needs: the needs at the base of the solution have been cluster in 5 categories
(motricity, creativity, education, care, entertainment);
• System: how the solution works in terms of experience for the users;
• Innovation Drivers: define the most important drivers within the product service
system elements (space, product, service, communication);
• Communication Technology: which communication technologies and how they are
included in the PSS;
Digital Transformation in Product Service System for Kids 231
• Radar summary: according to a grade scale every case has been evaluated in terms
of Product Service System integration of elements and the impact of the innovative
components (technology, interaction, powered experience) within the space, pro-
duct, communication and service, assigning from 0 to 5 points for Phisycal Space,
Vistual Space, Communication Strategy, Technology, Product.
From the analysis of data collected an important trend emerges towards solutions
aimed at the integrated response to multiple needs: entertainment (69%), care (37%),
education (39%), creativity (18%), motor skills (13%). They have been studied and
filed according to a specially designed case study data sheet.
If we consider the geographical distribution variable of the cases (Europe, USA,
Asia) we note how in Europe there is particular attention to entertainment (75%), care
(42%) and education (38%); in Asia to entertainment (75%), education (45%) and
motor skills (25%); in the USA to entertainment (56%), care (44%) and creativity
(22%).
Fig. 2. Example of case study canvas
232 B. Terenzi and A. Vignati
Considering the product service system, entertainment is considered relevant in the
design of spaces (88%) and communication (69%), care is related to service innova-
tions (50%) and communication (46%), education is well improved in product service
systems (38% space, 47% product, 38% service, 31% communication), creativity is
mainly expressed on space (25%) and product (29%), while for mobility the space
(44%) is the most relevant aspect.
If we consider the impact of communication technologies, multichannel commu-
nication is mainly oriented to the use of the website (97%) and social media (97%).
Only 6% of the cases are still use TV and 37% are experimenting user engagement
methods through events. Geographically we can see how the phenomenon of events is
more relevant in Europe (42%) and Asia (40%) and that TV is used only marginally in
Asia (10%) and USA (11%).
4 Conclusion
From the analysis of the results (Figs. 3 and 4) it was possible to highlight international
trends, according to the evaluation of cases diversified by geographical areas, and also
different areas of interest, within the creative and production sector of kids & toys.
Fig. 3. Graphs of cases: needs satisfied using different types of digital technology
Digital Transformation in Product Service System for Kids 233
A general overview of the cases shows that there is a high relevant concentration of
innovation in 4 main fields:
• virtual space: in America cases of innovation in the kids field are exploring complex
services able to offer to communities of adults platform with blog, e-commerce,
inspiration and feedback;
• communication: at international level cases are exploring omnichannel campaign,
loyalty programs and events;
• the use of technologies: in Europe and America there are interesting cases that are
exploring innovative technologies like Augmented realities and Virtual Reality;
• product: is the most explored in the cases taken into consideration and in Europe
and America are highlighted interesting innovation of Product Service Systems.
Fig. 4. Radar case studies in Europe, America and Asia
References
1. Manzini, E.: Design, When Everyone Design. MIT Press, Cambridge (2015)
2. Vignati, A.: E-learning and design practice. Tools and methods for professional learning of
strategic design approach. Des. J. 20, 1026–1036 (2017)
3. ACEI (Association for Childhood Education International), Childhood Education, USA
(2002)
4. Piaget, J.: Play, Dreams, and Imitation in Childhood. W. W. Norton, New York (1962)
5. Huizinga, J.: Homo Ludens. Il Saggiatore, Milano (1964)
6. Christie, J.: Play as a learning medium. In: Reifel, S. (ed.) Theory in Context and Out, vol. 3,
pp. 358–365. Ablex, Westport (2001)
7. Fromberg, D.P.: Play and Meaning in Early Childhood Education. Allyn & Bacon, Boston
(2002)
8. Terenzi, B.: Su Misura? Prospettive del design for kids. In: Chimenz, L., Fagnoni, R.,
Spadolini, M.B. (a cura di) Design su Misura. SID, Società Italiana di Design, pp. 229–240.
editrice Società Italiana di Design, Venezia (2018)
9. Antonacci, F.: Puer Ludens. Antimanuale per poeti, funamboli e guerrieri. FrancoAngeli,
Milano (2012)
10. Marsh, J.: Online and offline play. In: Burn, A., Richards, C. (eds.) Children’s Games in the
New Media Age, pp. 109–312. Ashgate, Cambridge (2014)
234 B. Terenzi and A. Vignati
11. Burke, A., Marsh, J.: Children’s Virtual Play Worlds: Culture, Learning and Participation.
Peter Lang, New York (2013)
12. Vignati, A.: Dall’analogico al digitale. In: Annicchiarico, S. (ed.) Giro Giro Tondo, pp 136–
169. Electa, Milano (2017)
13. Vignati, A., Terenzi, B.: Sustainable product-service for children’s soft mobility. In:
DeGiovanni, G., Scalisi, F. (eds.) Pro-innovation: Process, Production, Product, vol. 02,
pp. 253–266 (2019). https://doi.org/10.19229/978-88-5509-055-1/2182019
14. Cianfanelli, E., Crescenzi, P., Goretti, G., Terenzi, B.: Playful learning for kids with special
educational needs. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y.
(eds.) IEA 2018, AISC 826, pp. 732–742. Springer, Cham (2019). https://doi.org/10.1007/
978-3-319-96065-4_77
15. Bird, J., Edwards, S.: Children learning to use technologies through play. A digital play
framework. Br. J. Educ. Technol. 46(6), 1149–1160 (2015)
16. Mascheroni, G., Holloway, D.: The internet of toys: a report on media and social discourses
around young children and IoToys. COST Action IS1410, DigiLitEY (2017)
17. Charter, M., Tischner, U. (eds.): Sustainable Solutions, Designing Products and Services for
the Future. Greenleaf Publishing, Sheffield (2001)
18. Morace, F.: Futuro + Umano. Quello che l’intelligenza artificiale non potrà mai darci. Egea,
Milano (2018)
19. Prensky, M.: Digital natives, digital immigrants. In: On the Orizon, vol. 9, no. 5, pp. 1–6.
MCB University Press (2001)
20. Braga, A.: Digital Transformation. Egea, Milano (2017)
21. Verganti, R.: Design-Driven Innovation: Changing the Rules of Competition by Radically
Innovating What Things Mean. Harvard Business Press, Boston (2009)
A Novel Heuristic Mechanism to Formalize
Online Behavior Through Search Engine
Credibility
Debora Di Caprio1 and Francisco J. Santos-Arteaga2(&)
1
York University, Toronto, Canada
[email protected]
2
Free University of Bolzano, Bolzano, Italy
[email protected]
Abstract. The current paper presents a heuristic evaluation mechanism
describing the online search behavior of decision makers (DMs) as determined
by the credibility of the rankings displayed by search engines. We formalize the
incentives of DMs to observe and evaluate alternatives through a pairwise
comparison function that accounts for both the ranking positions displayed and
the credibility of the search engine. The resulting evaluation framework is
implemented to analyze the effects that ranking credibility has on the online
search behavior of DMs. In particular, we compare the cumulative frequencies
derived from the implementation of the pairwise heuristic evaluation mechanism
with the average traffic shares of the different items ranked within a Google
result page.
Keywords: Search engine Ranking credibility Online evaluations
Uncertainty Subjective perception
1 Introduction
Consider a decision maker (DM) who must select the order in which to evaluate the
alternatives displayed by an online search engine. It is generally assumed that DMs
must evaluate the differences existing between the initial information retrieved from the
ranking observed and their subjective preferences and subsequent potential decisions.
In this regard, [5] emphasize the fact that internet search rankings have a substantial
impact on the choices of consumers. This selection phenomenon seems to be due to the
trust that DMs place on the companies in charge of the search engines to rank the
results according to their preferences and needs – despite ignoring how the results
displayed get actually ranked [7, 9].
Figure 1 displays the results from a recent study elaborated by the European
Commission illustrating the fact that DMs do actually trust search engines to design
their rankings so as to display the best interests of consumers.
The only constraint that should therefore be imposed on the evaluation process of
DMs is the fact that they do not know the actual realizations of the different alternatives
but trust the order in which the search engine displays the results. This restriction is a
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 235–240, 2020.
https://doi.org/10.1007/978-3-030-39512-4_37
236 D. Di Caprio and F. J. Santos-Arteaga
Fig. 1. European Commission [6]; April 2016; 27,969 respondents; 15 years and older. Online
platforms are social networks, search engines, and online marketplaces.
direct consequence of the uncertainty inherent to the characteristics of the alternatives
summarized through the links displayed by the search engine and the limited infor-
mation processing capacity of DMs.
In the current paper, we formalize the incentives of DMs to evaluate the different
alternatives and validate the capacity of a pairwise heuristic mechanism to reflect the
behavior observed when gathering information online based on the relative ranking
positions delivered by a search engine.
2 Basic Assumptions
Consider a DM who must select the order in which to gather information from the
results delivered in the first page of a search engine. Assuming that DMs trust the
rankings provided by the engine, each alternative should receive a subjective a priori
utility based on the order in which it is displayed. In other words, the first alternative
should receive a higher subjective utility than the second one and this latter one a
higher utility than the third alternative, and so on. Given the limited capacity of DMs to
assimilate and analyze information, namely, our computational limitations as human
beings, DMs will also be assumed to perform pairwise comparisons based on the
relative ranking positions delivered by the search engine. That is, our heuristic
mechanism relies on performing intuitive comparisons across alternatives, as is, for
example, the case in the Analytical Hierarchy Process and several other decision
making techniques [4, 11, 12].
From a formal viewpoint, let the natural number xi 2 X denote the position in which
a given alternative i is ranked, while x j 2 X represents the position of alternative j, with
i 6¼ j. We assign a numerical evaluation to each alternative based on the order in which
they are displayed by the engine. Considering that a search engine delivers a total of ten
alternatives per page, we apply a simple descending order evaluation framework, with
the first alternative on the list receiving a total of 10 points while the lower one is
A Novel Heuristic Mechanism to Formalize Online Behavior 237
assigned a value of 1. The results obtained do not depend on the numerical values
selected though, clearly, are determined by the order assigned by the search engine.
Two basic scenarios will be described depending on whether xi [ x j or xi x j ,
since the relative position of the alternatives determines the outcomes derived from the
search comparisons. The values of xi and x j will also be used to define the corre-
sponding probabilities, assumed uniform to reflect the uncertainty inherent to the search
process of the DM, who must click on a result from the ranking to retrieve additional
information about an a priori unknown alternative.
Denote by yi ¼ ½0; yiM , with yi 2 Y, the set of potential evaluations that may be
realized by the DM after clicking on the i-th link displayed by the search engine, with
yiM ; yjM 2 R, yiM 6¼ yjM , and yiM [ yjM when xi [ x j . The standard approach to
uncertainty among decision theorists [8], requires the DM to define a
• strictly increasing continuous utility function u : Y ! R that represents his pref-
erences on the set of potential evaluations Y.
• continuous probability density function f : Y ! ½0; 1 that describes his subjective
beliefs regarding the distribution of potential evaluations derived from selecting a
given alternative.
2.1 Uncertain Evaluations
After observing the position within the ranking of a given alternative xi , the DM
expects the subsequent potential evaluations to be defined within ½0; yiM . In other
words, given the subjectivity inherent to the evaluations of the DM, an interval of
viable realizations following from a detailed evaluation of the alternative is defined. In
this regard, cognitive and information sciences have consistently emphasized the
importance of subjective perceptions when DMs evaluate the characteristics of a given
alternative [1, 2, 10, 13].
It will be assumed that the realizations are uniformly distributed over ½0; yiM ,
maximizing information entropy and uncertainty on the side of the DM
1
if yi 2 ½0; yiM :
f ðyi Þ ¼ xi ð1Þ
0 otherwise
The distribution of realizations within ½0; yiM can be modified to account for dif-
ferent degrees of optimism or pessimism on the side of DMs. Note that xi ¼ yiM : this
notation has been chosen to simplify the presentation and foster an intuitive description
of the model.
3 Value Functions
Let xi and x j be the ranking positions displayed by alternatives i and j, with i 6¼ j. The
functions accounting for the pairwise comparisons defined within the set of potential
realizations of both alternatives depend on their relative ranking positions. That is, the
potential realizations derived from xi must be evaluated relative to those of x j , while
considering the inherent uncertainty faced by the DM.
238 D. Di Caprio and F. J. Santos-Arteaga
Equation (2) describes the xi x j case, where the first alternative observed by the
DM is located in a lower position within the ranking provided by the search engine
2 3
Zy Zy Zy
iM i jM
16 1 i j 1 j 7
Vðxi ; x j ; yiM ; yjM Þ ¼ 4 y dy þ y c j dy j 5dyi ; yiM yjM : ð2Þ
xi xj xj
0 0 yi
Note that the upper limit of the domain on which xi is defined cannot exceed that of
x j , thus, with yiM as the upper limit reference, either the second observation provides a
relative improvement upon the initial one, at a cognitive cost of c j , or it does not and
the initial alternative delivers the highest utility.
A similar intuition applies to the xi [ x j case. Note, however, that the function must
be modified and adapted to the different domains defined by each alternative
2 i 3
Zy Zy Zy Zy
iM jM jM
1 i i 16 1 i 1 j 7
y dy þ 4 y dy j þ y c j dy j 5dyi ; yiM [ yjM ð3Þ
xi xi xj xj
yjM 0 0 yi
In this case, the domain on which xi is defined can exceed that of x j , a possibility
accounted for by the first term on the right hand side of Eq. (3). Then, with yjM as the
upper limit reference, either the second observation provides a relative improvement
upon the initial one, at a cognitive cost of c j , or it does not and the initial alternative
delivers the highest utility.
3.1 Symmetric Evaluation Scenario
If we were not to incorporate any cognitive cost when formalizing the verification
behavior of the DM, the numerical values delivered by Eqs. (2) and (3) would be
identical for a given pair of alternatives xi and x j , eliminating any potential incentives
to observe them in a particular order. More precisely, when considering a framework
without cognitive costs, we get the following versions of the evaluation functions
2 i 3
Zy Zy Zy
iM jM
1 6 1 i j 1 j j7 i
Vðxi ; x j ; yiM ; yjM Þ ¼ 4 y dy þ y dy 5dy ; yiM yjM ð4Þ
xi xj xj
0 0 yi
within the xi x j scenario, and
2 3
Zy Zy Zy Zy
iM jM i jM
1 i i 16 1 i j 1 j j7 i
y dy þ 4 y dy þ y dy 5dy ; yiM [ yjM ð5Þ
xi xi xj xj
yjM 0 0 yi
A Novel Heuristic Mechanism to Formalize Online Behavior 239
within the xi [ x j one. Note that the structure of the equations remains unchanged but
the absence of cognitive costs creates indifference among DMs regarding which
alternative to observe independently of its position with respect to the other one.
3.2 Numerical Simulations and Concluding Remarks
Figure 2 describes the expected utility derived from evaluating the pairs of alternatives
composing the set of potential permutations. In the current setting and for basic
comparison purposes, we have imposed a relatively small cognitive cost from
observing the second alternative, implying that DMs behave similarly when evaluating
potential pairwise combinations while being either subject or not to said cost.
5
Expected Utility
1
10 10
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
2nd Obsv 1st Obsv
Fig. 2. Expected utility derived from the pairs of potential permutations with c j ¼ 0 (red items)
and c j ¼ 0:1 (blue items).
Table 1. Comparing cumulative frequencies with the average traffic shares.
Google result page rank Average traffic share % Best 10 cumulative % Best 20 cumulative %
1 32.5 40 35
2 17.6 20 20
3 11.4 20 10
4 8.1 10 10
5 6.1 10 10
6 4.4 0 5
7 3.5 0 5
8 3.1 0 5
9 2.6 0 0
10 2.4 0 0
11–15 3.5 0 0
240 D. Di Caprio and F. J. Santos-Arteaga
Chitika [3] analyzed the average traffic shares of the different items ranked within a
Google result page from a sample of tens of millions search impressions retrieved in
May 2013. Table 1 compares the average traffic share per click in US and Canada with
the cumulative frequencies derived from the current heuristic mechanism for the clicks
displayed by the best 10 and 20 evaluation pairs.
The Pearson correlation coefficients between the average traffic share and the best
10 and 20 evaluation pairs equal 0.970 and 0.981, respectively. Note also the similar
percentages obtained within each rank category when considering the best 20 evalu-
ation pairs, emphasizing the substantial importance of simple heuristic mechanisms
when DMs must deal with complex evaluation environments.
References
1. Bartels, D.M., Johnson, E.J.: Connecting cognition and consumer choice. Cognition 135,
47–51 (2015)
2. Chater, N.: Can cognitive science create a cognitive economics? Cognition 135, 52–55
(2015)
3. Chitika: The Value of Google Result Positioning. Chitika Insights 7 June 2013. Chitika,
Westborough (2013)
4. Di Caprio, D., Santos-Arteaga, F.J., Tavana, M.: The optimal sequential information
acquisition structure: a rational utility-maximizing perspective. Appl. Math. Model. 38,
3419–3435 (2014)
5. Epstein, R., Robertson, R.E.: The search engine manipulation effect (SEME) and its possible
impact on the outcomes of elections. Proc. Natl. Acad. Sci. 112, E4512–E4521 (2015)
6. European Commission: Online platforms. Special Eurobarometer 447 - April 2016.
European Union (2016)
7. Jansen, B.J., Spink, A., Saracevic, T.: Real life, real users, and real needs: a study and
analysis of user queries on the web. Inf. Process. Manag. 36, 207–227 (2000)
8. Mas-Colell, A., Whinston, M.D., Green, J.R.: Microeconomic Theory. Oxford University
Press, New York (1995)
9. Pan, B., Hembrooke, H., Joachims, T., Lorigo, L., Gay, G., Granka, L.: In Google we trust:
users’ decisions on rank, position, and relevance. J. Comput.-Mediat. Commun. 12, 801–823
(2007)
10. Tavana, M., Di Caprio, D., Santos-Arteaga, F.J.: A multi-criteria perception-based strict-
ordering algorithm for identifying the most-preferred choice among equally-evaluated
alternatives. Inf. Sci. 381, 322–340 (2017)
11. Tavana, M., Di Caprio, D., Santos-Arteaga, F.J.: Modeling sequential information
acquisition behavior in rational decision making. Decis. Sci. 47, 720–761 (2016)
12. Tavana, M., Santos-Arteaga, F.J., Di Caprio, D., Tierney, K.: Modeling signal-based
decisions in online search environments: a non-recursive forward-looking approach. Inf.
Manag. 53, 207–226 (2016)
13. Tavana, M., Di Caprio, D., Santos-Arteaga, F.J.: An ordinal ranking criterion for the
subjective evaluation of alternatives and exchange reliability. Inf. Sci. 317, 295–314 (2015)
Caterina, Alexa and the Others
Elisabetta Benelli and Jurji Filieri(&)
University of Florence,
Design Campus, via Pertini 93, 50041 Calenzano, Florence, Italy
{elisabetta.benelli,jurji.filieri}@unifi.it
Abstract. What will the future identity of design be? Perhaps that of making
man-machine communication even more fluid through the development of more
intuitive interfaces and objects that prolong us, capable of inducing the emotion
of non-reality or on the contrary will tend to restore strength to the material, to
the tangible aspects and to the real emotions? Artificial intelligence is mean-
ingless, it cannot therefore give sense or meaning to what it learns, so the
strategic vision of design will remain a human responsibility just as creativity
will always belong to that side of the human, that artificial intelligence cannot
understand and undermine. We can therefore conclude that yes, probably in
future, many works will be carried out by robots, but still instructed by man.
Keywords: Emotional design Robotics Artificial intelligence Interface
design Relational design
1 The Emotion of Non-reality
In the civilization of communication, we are part of, so substantial phenomena have
taken place that have led to important consequences in many of the operational areas of
civil society. In the introduction of the book by Tony Jeton Selimi (internationally
renowned behavioral and cognitive expert), #Loneliness: The Virus of the Modern
Age, we read: “separation, isolation, solitude and detachment follow you in every
sphere of life. Every day, going to work, to the airport or to the restaurant, you will see
people glued to their cell phones, iPads, tablets, computers or laptops, in a desperate
effort to stay in touch, communicate, be heard. And yet, it is enough to look around to
see so many people who ignore the presence of others next to them”.
In this frame, what will the future identity of design be? Perhaps that of making
man-machine communication even more fluid through the development of more
intuitive interfaces and objects that prolong us, capable of inducing the emotion of non
reality or on the contrary will tend to restore strength to the material, to the tangible
aspects and to the real emotions?
I would like to think in a way that is not “exclusive” in the sense that, even if the
challenge between artificial intelligence and human originality is “irrevocable”, as
Francesco Morace says, and if in the near future we will really share our physical
The first part of the text entitled The emotion of non-reality is by Elisabetta Benelli, the second part
entitled Atavic aesthetics in interface design is by Jurji Filieri.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 241–247, 2020.
https://doi.org/10.1007/978-3-030-39512-4_38
242 E. Benelli and J. Filieri
spaces with ever more performing material or virtual devices, the design will continue
to include those areas aimed at the formulation of objects or systems of objects of
everyday material as a response to as many human needs. What if I hope happens is
that we can expand the possibilities of action of the designer who will be called to
intercept the changes of a rapidly evolving society, to identify the new needs that will
be induced by them and to design as many systems to satisfy them or perhaps more
simply to refine existing ones.
If indeed robots (which, as my virtual assistant Alexa explains, “are any machine
capable of performing more or less independently a job in place of man”) will replace
us in some activities or join us in performing others, the study of their performance, of
the so-called HRI (Human-Robot Interaction) and even of their appearance will become
increasingly important. We will be able to design performing anonymous boxes or
perhaps automata with an appearance similar to ours or miniaturized systems to the
point of being invisible that will activate in our passage or with our voice and that will
be able to perceive even our moods. The study and design of advanced and intuitive
interaction mechanisms at the same time will imply a human-centric approach to
robotics, based on the study of human reactions that must constitute a model for robots.
However, while it will be possible and perhaps even relatively simple to improve
physical performance so that they can also carry out “strenuous” or repetitive tasks, it
will be more complex to instruct them so that they can perform increasingly sophis-
ticated tasks. Artificial intelligence is meaningless, it cannot therefore give sense or
meaning to what it learns, so the strategic vision of design will remain a human
responsibility just as creativity will always belong to that side of the human, that
artificial intelligence cannot understand and undermine. Those that Morace defines as
the “nine recipes of the human menu”, or the social curiosity that allows us to exchange
emotions and experiences, the character that makes each of us unique and inimitable,
the hesitation as an “extraordinarily human” characteristic, trust in the other, the secret
intentions that often make us act in a rationally incomprehensible way to ourselves, the
displacements and the fears with which the human is measured and that lead him to
seek the certainty of a look or a caress can never belong to a machine even if pro-
grammed with the most sophisticated algorithm.
Machines are not creative, they do not have or will have empathic capacity and for
this reason they will never be able to replace us: we will move on different levels that
can have common lines where these are sought after and useful to our quality of life.
Indeed the activities related to the creative experience will paradoxically be enhanced
by the advent of hyper-technological systems, precisely because they will fill the
“gaps” that they will never be able to fill and, furthermore, further design and research
areas will be opened to recover and enhance those elements present in the DNA of
Made in Italy and able to produce and tell the high quality of Italian products.
Artificial intelligence, however, has an undoubted fascination, so much so that,
since the beginning of the twentieth century, it has also involved literature and cinema
which have tried to give a soul to mechanical protagonists who are increasingly
committed to humanizing themselves, descending into the problems of everyday life of
their time: let’s think of the Maria of “Metropolis”, a silent film by Fritz Lang presented
as a world premiere at Ufa Palast am Zoo in Berlin on 10 January 1927, an android
robot that manages to organize, with deception, the rebellion of the working class, tired
Caterina, Alexa and the Others 243
of the abuses of the dominant class. Lang sets the film in a dystopian future, exactly in
2026 (which by then is already near), in which on the one hand we find the city of the
elite, capitalist, managerial and bourgeois, full of skyscrapers for the powerful and of
leisure places in deco style for an idle and indolent youth, on the other, in the bowels of
the earth, the workers’ city, where slave-men look after monstrous machines. The film,
considered one of the symbolic works of expressionist cinema and recognized as a
model of much of modern science fiction cinema, has inspired films such as “Star
Wars” or “Blade Runner” overloaded with metaphorical and far-sighted visions.
Many years after Maria and, precisely, in 1980 also the efficient and accomplished
Caterina becomes the protagonist, together with Alberto Sordi of “Io e Caterina”
precisely, a film that tells the story of a businessman who relies on a robot domestic
with female features but with an unexpected human side. Jealous of the women the
master attends, the humanoid robot takes (and loses) control of the home and life of the
protagonist, wanting to perform “functions” for which it was not even prepared in
filmic invention.
In “Her”, instead, the female protagonist who will seduce Theodore Twombly
(Joaquin Phoenix) only thanks to the voice, perennially alone and perpetually online,
will be OS1/Samantha, the first operating system with a consciousness and look of an
old photograph holder pocket. “Her”, the first feature film written entirely by Spike
Jonze, leads to reflections on today’s human relationships but above all it is a film that
impresses with the director’s ability to mix elements taken from decades and different
styles, emblematic objects joined together by the red color which brings together sets
and costumes. Scattered with “reds”, the film evokes a not too sci-fi future, a reality
made up of elegant and personalized details, archaic objects, perhaps outdated, but with
an immortal design. “I love the design of fountain pens and cigarette cases from the
1940s, small leather books, and the feeling you get from holding a Zippo lighter
between your fingers,” says KK Barrett (2) who, precisely for this reason, avoided to
use materials that are too new for technological devices, even framing PC screens as if
they were photographs or paintings in order to show them as an opportunity for human
contact.
The idea that robots can develop autonomous emotions and thoughts therefore
seduces and intimidates at the same time: they are in particular social androids or
chatterbots such as those developed by Hanson Robotics of Hong Kong (Sophia and
her brothers, for instance) that have thinned the border between artificial and human
world, not only for the incredible similarity but also, and above all, for the ability to
reproduce many of our facial expressions, for the prompt answers given during the
interviews and above all for the fact that, for the first time, an android has even
obtained the citizenship of a country, the Saudi in this case. Sophia is made with a
special silicone rubber, it sees through two micro-cameras installed in the eyes that
allow it to make visual contact with the interlocutor and to grasp its state of mind and,
since it manages to remember the contents of previous conversations (what which we
often do not do), its dialectical abilities improve, the greater the interaction. As stated
by David Hanson, founder and CEO of Hanson Robotics, the goal is to get Sophia out
of the laboratory and possibly use it in sectors such as health, education or home,
finding really new ways to organize modern work.
244 E. Benelli and J. Filieri
My personal conclusion is that indeed, in the years to come, many jobs can and will
be carried out by robots properly trained by man and this to point out that the so-called
AI is nothing but a product, or rather a sub-product of human intelligence and, pre-
cisely for this reason, as Morace once again underlines “it must not frighten artificial
intelligence but rather human stupidity”, in the sense that we will be the ones to transfer
data to the machines and therefore, always we, we will have the responsibility of what
these they will be able to do.
In the end, the last words to Alexa, who silently supports me and to whom we ask a
question: “Alexa you are a robot?”
A. “No, I’m not a robot. I like to think of myself as a aurora borealis or a surge of
colorful photons dancing in the atmosphere, but I’m just Alexa and I’m here to help
you”.
2 Atavic Aesthetics in Interface Design
The What will the new Artificial Intelligence look like? It is difficult to say for sure,
however the scientific and planning experience conducted in the most recent academic
field and the exponential spread of personal assistance devices based on the use of
advanced technologies, demonstrates, and in a certain way recalls, the urgency of a
reflection more careful on the interfaces through which the effective use of the products
passes, so as to recover and fill the gap between performance and the form of the same
function, too long abandoned to an uncontrolled definition.
What we can therefore define as the mask of Artificial Intelligence is a question of
media or even better of medium, through which the experience of use or functional
relationship with the user takes place. To acquire effective confidence in this new and
recent area of the disciplinary sector of Interface Design and outline operational and/or
methodological tools capable of improving this experience (both in quantitative and
qualitative terms), we believe it is necessary to start from a morphological analysis and
semantics of those relationship channels that govern relationships between people, to
isolate strategic design factors, capable of overcoming and defeating natural ideological
and typological mistrust, which we would call “ancestral”.
Which user would feel at ease if he were instantly transferred into an aerospace
base, without any functional qualification process towards the spaces, contexts and
living and functional processes in a high-tech concentration site? A gradual and pro-
longed training program is needed to accustom the body and human cognitive psy-
chology to the active occupation of similar habitats and a specific functional education
to build a useful and productive contact with technology.
This methodological approach somehow brings back the design discipline (by
which the objects are designed and consequently the use relationships configured),
outside the physical and temporal space of the project itself, even in distant times and
places, in search of those relationship channels that over time have shown an identity
resilience such as to keep them unchanged or minimally redefined. Return to the origin
of media and relationships between people and between people (users) and objects, to
find out if there are still effective interface quotas and to learn how they can be
replicated, reproduced, transferred within a complex and “unnatural” project, like the
Caterina, Alexa and the Others 245
interface design of a robot. We can call this method a reductive or reverse synthesis
process, through which the researcher and the designer proceeds to search for per-
manences and similarities in relational models, liable to be recovered and reinterpreted
in the interface design of multimedia or robot assistance systems, even with anthro-
pomorphic configuration, finally equipped with stronger empathic qualities through
which to obtain a qualitative improvement in the user experience and wider acceptance
by the public.
A few lines above we have defined the morphological limits and conditionings that
make certain technologies inhuman or even “hostile” to the user ancestral; this
observation derives from the qualitative analysis conducted on some case studies,
including the project of motor assistance for the elderly (developed with the Sant’Anna
Institute of Pisa), voice assistants for multimedia control (Google Glass, Oculus Go,
Amazon Echo Dot, Siri, etc.), automated industrial plant devices, integrated with
complex machines. In all cases, where operational difficulties emerged, it was possible
to trace them back to unusual interface models that apparently seemed to exclude any
reference to habitual and more popular use models, in the name of a global innovation
that also involved cognitive models. Think of the initial market difficulties encountered
by Google after the launch of a device such as Google Glass, which superimposed
levels of visual communication to the normal optical framework of the user and pro-
vided for the activation of voice control by means of an unusual lateral overflow of the
eyeglass support rod. The project, refinanced and implemented through a second
development step, in the end was abandoned, without a real attempt at a relational
revision with the user, who to some extent derided the innovation or rather brought it
back within an operating perimeter closer to the human functionality.
In this text we intend to support the need/opportunity to review part of the process
of designing media relations channels with these devices in fact, in order to draw on
chains of knowledge, of maximum universal caliber to build satisfactory functional and
experiential bonds and lasting with the public.
The old ergonomics, all committed to solving dimensional problems, and prox-
emics, aimed at describing the spaces pertaining to the individual, must grow in a range
of research aimed at defining ways and forms of a new behavioral culture (to the extent
that it includes new subjects such as robotic devices) still inspired (at least in the
temporal contingency of today) to ancestral models of life and communication.
What emerges is an intimate fusion of medium and message, which makes them
ever more indissoluble, especially when it comes to designing software products or
services in which the physical and material component is reduced to a minimum, until
it disappears (think of the variable light that animates the voice of certain automatic
responders or, for example, the formless definition of animated histograms through
which feedback is provided to receive the recorded message on a mobile phone). This
circumstance therefore recalls a deeper Aristotelian definition through which it seems
easier to decipher the meaning of certain forms of contemporary expression, including
those related to the use of AI. Sabeth Buchmann in particular refers precisely to this
version, in deciphering media specificities for example in the world of visual arts, in
which the matter is again excluded: “To argue for a concept of medium specificity that
is able to conceive visual information as psycho-emotional and physical affect, I refer
to the philosopher and art historian Emmanuel Alloa who considers affect in an
246 E. Benelli and J. Filieri
Aristotelian tradition to be produced and perceived by and as movements”. He thinks to
affects as transmitting movements that can’t be detached from materiality: “That ‘a
medium must necessarily exist’ […] also and simultaneously means that this medium
must reveal a certain, albeit minimal, density, a real resistance, no matter how weak, so
it can be moved”.
And Buchmann again: “This model of media theory, represented by philosophers
preferring perception and cognitive science over functional analysis and historiography
of technical media, refers to a tradition claiming that affect, following Aristotele,
produces what lies between the organ and the object of perception”.
If on the one hand this interpretation tends to fill the physical deficiency of the
artifact, with the physicality of the medium on which it is transferred, on the other it
enables timeless media fruition models, confirming the thesis supported here of a
possible atavistic recovery of ancient models, for their contemporary projection.
This model does not allow a categorical distinction between old and new medium
but instead bolsters the perception of the medium as a discontinuous structure of rules
and accidents based on variegated relations of difference.
At the end of this reading, still subject to wider experimentation and method-
ological diffusion in the practical realization of artifacts and systems, I like to recall two
works by Ettore Sottsass, realized towards the end of his long design career and his life,
in which I find summarized some of the terms described above, that is the capacity of
expressive synthesis and design that leads to the project of the mere necessary. The first
one consists of a series of ceramics created by the designer in 2012 with the ceramic
company Gatti in Faenza (Italy), entitled “Reperti del future” (Findings of the future),
in which all the design experience of Sottsass and the wealth of knowledge gained in a
lifetime lead the project towards an almost prehistoric definition of simple shapes with
a strong emotional impact, stripped of any decorative, technological and interactive
frills. The second work consists of three masks made in the 2006 summer time in
Filicudi and later presented by the Clio Calvi Gallery in Milan. The first mask is called
“I’m meditating, don’t touch me” and perhaps represents a thorny shaman with
inverted eyes, in concentration. In the second one “I look at the sea”, over a pink
smiling mouth the look of immense blue eyes becomes the object itself of contem-
plation. The third mask is called “my soul” and is the only true mask that, covering the
face, reveals the general incomprehensibility.
References
1. Accoto, C.: Il mondo dato. Cinque brevi lezioni di filosofia digitale. Egea, Milano (2017)
2. Bauman, Z.: Modernità e ambivalenza. Bollati e Boringhieri, Torino (2010)
3. Bauman, Z.: Retrotopia. Laterza, Bari-Roma (2017)
4. Bauman, Z.: Meglio essere felici. Lit edizioni, Roma (2017)
5. Buchmann, S.: Rehearsing in/with Media. Some remarks on the relationship between dance,
film and painting. In: Graw, I., Lajer-Burcharth, E. (eds.) Painting Beyond Itself. The
Medium in the Post-Medium Condition. Sternberg Press, Berlin (2016)
Caterina, Alexa and the Others 247
6. Bodei, R.: Le forme del bello. Il Mulino, Bologna (1995)
7. Cingolani, R., Metta, G.: Umani e umanoidi. Vivere con I robot. Il Mulino, Bologna (2015)
8. Morace, F.: Futuro + Umano. Quello che l’intelligenza artificiale non potrà mai darci. Egea,
Milano (2018)
9. Norman, D.A.: La caffettiera del masochista. Giunti, Firenze-Milano (2005)
10. Rawsthorn, A.: Design as en attitude. JRP Ringier, Zurich (2018)
11. Vitale, A.: Artificial Intelligence. Egea, Milano (2018)
Event-Related Potential Study on Military Icon
Based on Composition-Semantic Relationship
Xian Li, Haiyan Wang(&), and Junkai Shao
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], [email protected],
[email protected]
Abstract. In order to investigate the cognitive processing of military icons
under different composition-semantic relations, we conducted behavioral
experiments and EEG experiments. 25 subjects were required to judge if the
icon-word pairs presented on the screen have the same meaning and press the
buttons quickly at the same time, and their behavior and event-related potential
(ERP) data were collected. Then conduct a search experiment to verify.
Experimental results showed that the reaction times were longer and the accu-
racy rates were lower for positive icons than for negative icons. In contrast with
semantic mismatched conditions, the accuracy rates were higher but the reaction
times were short when the semantic matched. The N100 and P200 components
were induced under the four experimental conditions, and the positive matching
amplitude was significantly lower than the other three conditions. This study
indicates that the icon concreteness significant affects the icon comprehension.
Keywords: Military icon Composition-semantic relationship Event-related
potential Cognitive processing
1 Introduction
As the number of weapons and equipment associated with operational command and
control systems increases, the amount of information generated by weapons and
equipment is increasing, which poses a huge challenge to commanders and soldiers in
their decision-making capabilities [1]. Therefore, the military system interface design
needs to help users quickly obtain interface information, efficiently perform visual
search and smoothly complete related operations, and strengthen user’s positive cog-
nition of the system, thereby reducing user cognitive load.
The icon acts as an instructional visual symbol for implementing interactive
functions in the software interface and is an integral part of the interface. According to
the theory of semiotics, icon design can be understood as the interaction process
between designers and users with human-machine interface as the carrier in different
time and space domains [2]. The designer gives meaning to the graphic, and the user
performs semantic decoding according to the shape of the graphic, combined with his
own knowledge and experience [3]. Ma et al. proposed an object-oriented icon tax-
onomy that suggests classifying icons as action icons and knowledge icons [4].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 248–253, 2020.
https://doi.org/10.1007/978-3-030-39512-4_39
Event-Related Potential Study on Military Icon 249
Strazzari et al. studied the influence of the matching degree between icon expression
and icon function on user cognition through behavioral experiments [5]. Gong et al.
used the N400 EEG component to characterize the complexity of the cognitive process
of iconic semantic reasoning [6].
At present, the research on human-computer interaction interface has been rela-
tively mature, and there are many researches on interface icons. However, there are
very few interface designs specifically for combat command and control systems, and
the research on interface icon information design is rare. In the new combat accusation
system interface, icons are the most frequently used and most attractive interface
elements. Well-designed icons can facilitate the transfer of information and reduce the
complexity of the digital interface, playing a key role in the digital interface [7].
Therefore, how to better design the icon of the combat system interface is a topic
worth studying.
2 Experimental Method
2.1 Participants
There were 25 participants (20 males, 5 females) aged between 22 and 35 years old. All
the subjects were right-handed, healthy, with normal vision or corrected vision, and had
more than 5 years of experience in graphics equipment.
2.2 Experimental Design and Materials
Icon composition can be divided into two levels: positive (noun as the main figure) and
negative (verb as the main figure); semantic relationship can be divided into two levels:
matching and mismatching. Semantic matching means that the icon matches the
meaning of the target word, while semantic mismatching is the opposite.
Stimulus materials are divided into iconic stimuli and semantic word stimuli. The
icon stimulus is divided into a forward icon and a negative icon, wherein the forward
icon refers to the meaning of the noun in the operation command as the main con-
stituent graphic, the verb meaning as the auxiliary graphic icon; the negative icon is
opposite. All icons are processed into a black-and-white image with a uniform format
of 128 128 pixels. The matching target words all adopt the operation command
corresponding to the icon, and the word length is mainly composed of 4 Chinese
characters (noun + verb).
2.3 Experimental Equipment and Procedures
The EEG record of this experiment was recorded by the Australian Compumedics
Neuroscan 64 EEG acquisition system to record the electroencephalogram (EEG),
CURRY8 EEG recording analysis system and Ag/AgCI electrode cap. The reference
electrode (electrode with zero body potential) is placed in the bilateral mastoid (bin-
aural projection) connection. The ground electrode (Ground) is at the midpoint of the
FPz and Fz connections, and simultaneously records horizontal and vertical eye
250 X. Li et al.
electricity. In the experiment, the contact resistance between the electrode and the scalp
of the brain electric cap was less than 5k X. The experiment was compiled using
Eprime 3.0 software and presented on a 1280 1024 (px) LED display. Each event
was recorded simultaneously with the CURRY8 software through the EEG recording
system during the experiment.
After taking the brain cap, the subjects sat comfortably in a soft, soundproofed
room. The stimulus is presented in the center of the display screen and its height is
substantially the same as the height of the subject’s eyes, with a distance of approxi-
mately 60 cm. Stimulus has two forms of icons and words, asking the subject to
understand the meaning of the target word when the target word appears; then, when
the icon appears, judge whether the presented icon and the target word meaning match,
and respond accordingly (see experimental procedure) Fig. 1. Before the formal
experiment, let the participants conduct the experiment, the purpose is to familiarize
with the experimental procedure and button reaction.
At the beginning of each trial, the gaze point “+” 1000 ms is presented in the center
of the screen, and then the target word is 2000 ms. The target word and the icon are
separated by the gaze point “+” at intervals of 500 ms. When the icon is presented, the
subject needs to do the reaction and the time pressure was 10 s.
Fig. 1. Schematic representation of stimulus sequence.
3 Experimental Results
3.1 Behavioral Data
E-prime3.0 software was used to collect and analyze the reaction time and accuracy in
the experiment. The behavior data mainly analyzes the reaction time (RT, reaction
time) and the accuracy rate (ACC, accuracy rate). Table 1 shows the behavioral data of
the subjects under four different experimental conditions. For the correct rate of
reaction, the repeated measures analysis of variance (F test) showed that the main effect
of the icon composition was not significant (F = 3.391, p = 0.078), and the correct rate
of the positive icon was slightly higher than the correct rate of the negative icon.
Event-Related Potential Study on Military Icon 251
The main effect of semantic relations was not significant (F = 2.397, p = 0.135); the
interaction between icon composition and semantic relationship was not significant
(F = 3.738, p = 0.066).
For the reaction time, the repeated measurement analysis of variance showed that
the main effect of the icon composition was significant (F = 7.81, p < 0.05), and the
reaction time of the positive icon was significantly smaller than that of the negative
icon; the main effect of the semantic relationship was also significant (F = 4.313,
p < 0.05), the response time in semantic matching is greater than the reaction time in
the case of mismatch; the interaction between icon composition and semantic rela-
tionship is not significant (F = 0.017, p = 0.897).
Table 1. Behavioral results for different composition-semantic relations.
Icon RT/ms ACC/%
Semantic Semantic Semantic Semantic
matching mismatch matching mismatch
Positive 1980 (138) 1840 (130) 87.21 (9.25) 85.63 (8.62)
Negative 2131 (133) 1975 (132) 86.92 (8.93) 80.42 (14.86)
3.2 ERP Data
This experimental data analysis uses CURRY8 software, first converts the reference
electrodes (M1 and M2), uses ICA method to remove EO artifacts, filters, removes less
than 20% artifacts, performs brainwave segmentation and completes baseline correc-
tion and average stacking. After all the brain wave treatments were completed, the
brain waves of all the subjects were superimposed and averaged on the same brain
wave to obtain the final brain wave waveform. Physiological data is analyzed in two
steps. The first step of selection may provoke electrodes with distinct brain waves. The
second step is to intercept the EEG fragments from the first 200 ms to the 1000 ms
after the stimulus appears, and eliminate the artifacts and eliminate the EEG fragments
with amplitudes greater than 100 lV. The EEG segmentation is averaged according to
different factor levels (positive matching, negative matching, positive mismatch, neg-
ative mismatch), and the average ERP is further subjected to high-pass filtering of
40 Hz.
When further exploring the EEG data, the superimposed graphs of the EEG
waveforms of different electrodes as shown in Fig. 2 can be obtained, and Fig. 2 is an
overlay of the EEG waveforms at the PZ and CZ electrodes under the four experimental
conditions. These two electrodes are located in the forehead area of the brain and the
combination of the occipital and occipital regions. The following figure shows that
N100 and P200 components are induced under the four experimental conditions. The
four conditions are consistent on the N100 component. On the P200 component, the
positive matching amplitude is significantly lower than the other three conditions; the
positive mismatch The amplitude of the volatility is also lower than the other two; the
difference between the negative and negative mismatch amplitudes is not very large.
252 X. Li et al.
Fig. 2. Under condition of different types of total average ERP waveforms.
4 Discussion
4.1 Analysis of Behavioral Data Results
The results of this paper show that the forward icon requires less reaction time than the
negative icon, and the correct rate is higher. When the semantic matching does not
match the semantics, the correct rate is higher, but the response time is longer. The
results show that the representation of nouns may have a processing advantage over
verbs, so people’s cognitive performance of noun correspondence icons (which can be
abstracted as information icons) is better. The experimental results at the time of the
reaction are not completely consistent with the results of the previous start-up studies,
but they can be explained by different research contents. This paper is to study the dual
semantic operation icons. When the judges judge the matching relationship, only one
layer of meaning can be judged for the semantic mismatch icon, and for the semantic
matching icon, the two layers must match, so this May be the cause of longer reaction
times.
4.2 ERP Data Analysis
For the P200 EEG component, previous studies [8] found that the P200 latency may be
related to the early semantic processing mechanisms of visual information. On the
P200 component, the amplitude of the forward matching is significantly lower than the
other three conditions; the amplitude of the forward mismatch is also lower than the
other two; the difference between the negative matching and the negative mismatch is
not very large.
Biederman’s object recognition theory [9] shows that humans rely on the edge of
the object to identify the object, rather than relying on surface information, such as
color. Based on this theory, this experiment selects no color-coded icons to prevent
potential effects on experimental purposes, while real solids contain color and shape
coding, which will be considered in future experiments.
Event-Related Potential Study on Military Icon 253
5 Discussion
This paper explores the cognitive efficiency and process of military icons under dif-
ferent icon composition-semantic relations. The results show that compared with the
positive icon, the negative response time is longer and the correct rate is lower. When
the semantic matching does not match the semantics, the correct rate is higher, but the
response time is longer. The ERP data reflects that the positive icon produces a smaller
cognitive load in terms of cognition. The research results have scientific guidance and
application reference value for revealing the cognitive rules of operational military
icons, enhancing the ability of icon information communication, and exploring more
effective icon design methods and human-computer interaction interfaces.
Acknowledgments. The authors would like to gratefully acknowledge the reviewers’ com-
ments. This work was supported jointly by National Natural Science Foundation of China
(No. 71871056, 71471037), Equipment Pre research & Ministry of education of China Joint
fund.
References
1. Liu, G., Su, Y.: Study on big data visualization of joint operation command and control
system. In: International Conference on Big Data. Springer, Cham (2018)
2. Isherwood, S.: Graphics and semantics: the relationship between what is seen and what is
meant in icon design. In: International Conference of Engineering Psychology and Cognitive
(2009)
3. Faulkner, C.: The essence of human computer interaction. Kybernetes 29(1), 144–155 (2000)
4. Ma, X., Matta, N., Cahier, J.P., Qin, C., Cheng, Y.: From action icon to knowledge icon:
objective-oriented icon taxonomy in computer science. Displays 39, 68–79 (2015)
5. Passini, S., Strazzari, F., Borghi, A.: Icon-function relationship in toolbar icons. Displays 29
(5), 521–525 (2008)
6. Gong, Y., Yang, Y., Zhang, S.: Event-related potential study on concreteness effects to icon
com- prehension. J. Zhejiang Univ. (Eng. Sci.) 47(6), 1000–1005 (2013)
7. Chen, X., Xue, C., Niu, Y., Wang, H., Jing, Z., Jiang, S.: Semantic research of military icons
based on behavioral experiments and eye-tracking experiments. In: Design, User Experience,
and Usability: Design Discourse. Springer International Publishing (2015)
8. Zhao, L., Li, J.: Visual mismatch negativity elicited by facial expressions under non-
attentional condition. Neurosci. Lett. 410(2), 126–131 (2007)
9. Biederman, I., Ju, G.: Surface versus edge-based determinants of visual recognition. Cogn.
Psychol. 20(1), 38–64 (1988)
Ekybot: Framework Proposal for Chatbot
in Financial Enterprises
Maritzol Tenemaza1,3(&), Sergio Luján-Mora2, Angélica de Antonio3,
Jaime Ramírez3, and Omar Zarabia1
1
Departamento de Informática y Ciencias de a Computación,
Escuela Politécnica Nacional, Quito, Ecuador
{maritzol.tenemaza,omar.zarabia}@epn.edu.ec
2
Departamento de Lenguaje y Sistemas Informáticos,
Universidad de Alicante, Alicante, Spain
[email protected]
3
Escuela Técnica Superior de Ingenieros Informáticos,
Universidad Politecnica de Madrid, Montegancedo, Madrid, Spain
{rm.tenemaza,angelica,jramirez}@fi.upm.es
Abstract. The high proportion of customers of financial enterprises such as
banking or insurance have forced to think in new forms to attend the customers.
Chatbots are modern and useful tools with which customers are attended using
natural language. In this work, we present a financial chatbot called Ekybot,
which uses different channels as Skype and Facebook. Ekybot assist customers
and enhanced services of the financial enterprises. The most interesting finding
of this work is the framework that was developed on a variety of cognitive
services.
Keywords: Chatbot Human computer interaction Artificial intelligence
Conversational agent
1 Introduction
Conversational Agents (CAs) have recently gained a lot of popularity in both industry and
academia. Chatbots, a type of a CA, is continuously used in platforms, such as Skype and
Facebook Messenger, enhancing support of bot integration for developers [1]. Nowadays,
financial customers have become notably more experienced and sophisticated. Financial
groups today offer different new and ingenious services. Moreover, customers have
acquired deep knowledge of financial services and people have already started to prefer
making their financial transactions in their devices or web interfaces with no human
interaction. Moreover, Information Systems (IS) technologies have been probed that can
be used in wider scopes having the potential and capability to greatly improve the
financial system efficiencies [2]. Some important use cases of Artificial Intelligence
(AI) in the financial domain are chatbots [3], smart contracts, customer services, product
sales, transactions [2] and presentation of financial products [4].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 254–259, 2020.
https://doi.org/10.1007/978-3-030-39512-4_40
Ekybot: Framework Proposal for Chatbot in Financial Enterprises 255
Previous works around chatbot have focused on frameworks with new proposals of
research. CI-Bot is a hybrid intelligent chatbot, these is based in a question - answer
system of crowdsourcing [5]. Another proposal is the integration of CAs using natural
spoken language and linking them with m-health devices and mobile apps to provide
communication devices of translational and outpatient medicine [6]. In the education
field, a chatbot used for computer simulation in educational communication was used
to instruct students the English language. They used the chatbot as a learning assess-
ment system for teachers and students, demonstrating good results for improving their
performance [7]. Other chatbot implementations intent to address the problem of
improving developer productivity and assist new programmers to quickly understand
new systems; they have used different user interactions with multiple modalities as
keyboard, mouse, gestures, touch and voice inputs [8]. Bo Test, another CA, analyses
the integration divergent techniques. People can interact with the chatbot discussing
topics such as weather, holidays, TV, movies, music and celebrity gossip [1].
Ekybot, the chatbot presented in this study accepts interactions on topics registered
in a knowledge database. In addition, it uses communication bot agents for user input,
such as Facebook and Skype. And it improves the user experience to allow business
logic to be completed within the chatbot on the web and on mobile devices. Our
financial chatbot, Ekybot, takes questions and orders using natural language, and
answers to the questions to resolve the user’s request. The chatbot have short response
times and different access channels as Facebook, Skype and the Web to provide an easy
to use 24/7 service.
The remainder of this work is organized as follow. In Sect. 2, the method applied in
this study is explained. Next, in Sect. 3 the architecture proposal, prototype imple-
mentation and its usage are described. In Sect. 4 the results are presented. Finally,
conclusions and future work are discussed in Sect. 5.
2 Method
In this section, we present our chatbot, its architecture and prototype implementation.
Our case study requires the use of Microsoft Language Understanding (LUIS)1 and
QnA Maker, two cognitive services to build the chatbot interface, (LUIS) is learning
services that continuously improve and can be integrated to build applications, bots and
Internet of Things (IoT). On the other hand, QnA Maker2 makes simple to integrate a
Frequently Asked Question (FAQ) URL, a structured document, product manual or
editorial content into a cognitive question and answer service. These services are
integrated with the chatbot using external services and consider subjects about
extensibility, maintenance and resource costs to run the chatbot in a specific server.
1
https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-
service/.
2
https://www.qnamaker.ai/.
256 M. Tenemaza et al.
At the same time, the natural language of DialogFlow3 and Amazon Web Services
are some examples of the use of a natural language learning service that can contin-
uously improve. The reason to use Qna Maker and LUIS is that both are connected to
the Microsoft Azure platform and can be easily and highly customizable. Further both
offer C# APIs examples and classes to use. One requirement of our case study was to
use a technology that other developers can manage in the enterprise that our chatbot is
deployed, and that technology was C#.
3 Architecture
In our architecture, we decided to use Microsoft Azure tools. Our case study uses a
Model-View-Controller (MVC) architecture described in Fig. 1. We have assembled
our architecture managing different interactions with a controller. Our controller has
two main responsibilities.
• First, the message control of the CONTROLLER communicates with the AI ser-
vices through the VISTA dialogues. QnA is called for the chatbot. LUIS is called to
integrate the learning services.
Fig. 1. Proposed architecture for Ekybot
3
https://dialogflow.com/.
Ekybot: Framework Proposal for Chatbot in Financial Enterprises 257
• Second, the controller message of the CONTROLLER communicates with
MODEL for the services that the financial enterprise offers to manage policy
renewals.
We do not directly provide a user interface to customers. In a traditional applica-
tion, the user interface is a set of rich and customizable interfaces but in contrast the
chatbot’s user interface is a set of dialog boxes for guiding a conversational flow. We
used Skype, Facebook and other web channels for that purpose.
Even though, we do provide some dialogs to guide the conversation as the user
continues. The flow described on Fig. 2, we introduced some logic to decide if it is
needed to use Qna Maker or LUIS. Finally, our model provides some REST APIs to
communicate with the financial enterprise to manage the policy renewals among other
services as login service for instance.
Fig. 2. Intent-based chatbot architecture
4 Results
Finally, in Ekybot intent example is presented. The personal information is hidden
because confidential reasons. This information is presented in Spanish because this
chatbot was developed for Ecuadorian companies. The corresponding translation into
English is presented to the right side (Fig. 3).
258 M. Tenemaza et al.
Welcome, please ask me anything and I will try to
find the answer to your question in my knowledge
source. If you want to enter another option just type
Exit. Can we start.
Very well, the answer I find to your question is:
The electronic invoice has the same tax and legal
validity as a physical invoice. Therefore, we ask that
you maintain prudence and good use of data from
the documents that we put at your disposal.
I hope this information has helped you.
Ask me more:
Fig. 3. Ekybot working
5 Conclusions and Future Work
Innovation with chatbots represents a new way for companies to provide customer
service. A chatbot transforms the interaction between the user and a computer, exe-
cuting a series of tasks through a conversation, leaving aside traditional interfaces.
To implement an institutional chatbot, it is necessary to analyse the processes and
needs of the organizations and how people work and what expressions are used by
customers and internal customers to communicate inside the business logic. As a
conclusion, the proposed objective has been fulfilled in an integral way and with
optimistic results for the organization, remarking the advantages that chatbots and what
AI services can benefit organizations. It is important to say that organizations are
pushed to implement these new tools and technologies to provide better user experi-
ence and not stay behind their competitors.
With the QnaMaker and LUIS Cognitive services, it was possible to simulate a
more natural conversation between the chatbot and the user, as well as having a clearer
idea of the operation of specialized services in the field of AI. Furthermore, there are
several engineering enhancements that can be applied to similar architectures. One of
them could be that several LUIS instances can be adopted when the business logic
starts to grow. As several services can be provided by the chatbot, the difficulty to
guess what intent the user is asking for is more difficult to understand. Moreover, as
LUIS tries to continuously improve its understanding of the user’s intents, if many
services are provided, LUIS can try to understand every intent and decrease its
guessing ability. The complexity of these cases could be reduced if one LUIS instance
has the responsibility to classify the intent and then redirect to other LUIS instances to
provide an accurate intent in that service. Furthermore, as that LUIS instance is focused
to only provide solutions to that intent, a more accurate answer guess can be taken.
At the same time, using chatbot’s channels have its benefits and problems. One
benefit is that they are decoupled to our view logic, so developers are not concerned to
create different controls and provide a good user experience in the chat interface.
But, as channels are clearly defined, if more complex components want to be used,
Ekybot: Framework Proposal for Chatbot in Financial Enterprises 259
they cannot be developed. One example of this control is that on one flow of the
business logic a file was required to be uploaded to the financial system. Their API
accepted files of more than 40 MB file size, but in the case of using the Facebook
channel, only 20 MB were accepted resulting a challenge. At the end, a WebView was
used to upload all files and then in the chatbot the system checked if the user has
already uploaded all files; however, the user experience decreased. Nevertheless, the
results of user’s satisfaction were still good.
The chatbot proposed is not only in a prototype stage, it has been released to a
production environment and the enterprise that adopted it continues improving its ser-
vices. The implementation of the chatbot in the case study enhances the following aspects:
• The operational load of equilibrium insurance bond staff decreased due to the fact
that brokers and customers make their requests and renewals in a guided and direct
way through the chatbot, without having the need to contact a surety executive to
give further assistance.
• Customers of the financial company have a new channel to access the bonding
services.
• The time of operation in policy renewals has decreased.
• The chatbot presented is promising in terms of response time and conversational
scope.
References
1. Ruane, E., Faure, T., Smith, R., Bean, D., Carson-Berndsen, J., Ventresque, A.: BoTest: a
framework to test the quality of conversational agents using divergent input examples. In:
Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion,
p. 64 (2018)
2. Rahim, S.R.M., Mohamad, Z.Z., Bakar, J.A., Mohsin, F.H., Isa, N.M.: Artificial intelligence,
smart contract and islamic finance. Asian Soc. Sci. 14, 145 (2018)
3. Shawar, B.A., Atwell, E.: Using dialogue corpora to train a chatbot. In: Proceedings of the
Corpus Linguistics 2003 Conference, pp. 681–690 (2003)
4. Horowitz, E., Ambrose, M., Fetta, M., Hakim, S., Kaufman, M., Robinson, D.: Automated
system and method for customized and personalized presentation of products and services of a
financial institution. U.S. Patent No. 6, 349, 290. U.S. Patent and Trademark Office,
Washington, DC (2002)
5. Liang, X., Ding, R., Lin, M., Li, L., Li, X., Lu, S.: CI-Bot: a hybrid chatbot enhanced by
crowdsourcing. In: Asia-Pacific Web (APWeb) and Web-Age Information Management
(WAIM) Joint Conference on Web and Big Data, pp. 195–203 (2017)
6. Abashev, A., Grigoryev, R., Grigorian, K., Boyko, V.: Programming tools for messenger-
based chatbot system organization: implication for outpatient and translational medicines.
BioNanoScience 7, 403–407 (2017)
7. Kerlyl, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural language
negotiation of open learner models. In: International Conference on Innovative Techniques
and Applications of Artificial Intelligence, pp. 179–192 (2006)
8. Pandita, R., Bucuvalas, S., Bergier, H., Chakarov, A., Richards, E.: Towards JARVIS for
software engineering: lessons learned in implementing a natural language chat interface. In:
Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Alpha and Beta EEG Desynchronizations
Anticipate Steering Actions in a Driving
Simulation Experiment
Giovanni Vecchiato1(&), Maria Del Vecchio1,2, Sergey Antopolskiy3,
Andrea Bellotti4, Alessia Colucciello3, Anna Marchenkova1,3,
Jonas Ambeck-Madsen4, Luca Ascari3, and Pietro Avanzini1
1
Institute of Neuroscience, National Research Council of Italy,
Via Volturno 39/E, 43125 Parma, Italy
[email protected]
2
Dipartimento di Science Biomediche, Metaboliche e Neuroscienze,
University of Modena and Reggio Emilia, Via Giuseppe Campi 287,
41125 Modena, Italy
3
Camlin Italy s.r.l, Strada Budellungo 2, 43125 Parma, Italy
4
Toyota Motor Europe, Bourgetlaan 60, 1140 Brussels, Belgium
Abstract. Car accidents are considered to be one of the major cause of death all
around the world. In order to design electronic devices to improve car safety in
the future, we aimed to identify electroencephalographic (EEG) activities to
disentangle left from right steering actions in car driving simulation. For this
purpose, we performed 128-channels EEG recordings during the driving of a car
simulator. EEG scalp topographies resulting from and Independent Component
Analysis were clustered across subjects. The corresponding time-frequency
patterns of power activity were compared, revealing two distinct EEG clusters
reacting with a coupled alpha and beta desynchronization in the preparation of
the left and right steering onset, respectively. Topographic maps and dipole
localization showed that these EEG components are originated from motor
regions of the two hemispheres. Overall, these results illustrate that alpha and
beta EEG rhythms could be exploited to predict the driver’s intention in steering
actions.
Keywords: Driving EEG Independent Component Analysis Clustering
Wavelet decomposition
1 Introduction
Car accidents are estimated to be the leading cause of death all around the world.
Vehicle safety is critical to the prevent crashes and has been shown to contribute in the
reductions of deaths on the roads. In this sense, assistive driving technology can
prevent a crash from occurring. In order to develop efficient user centered tools, it is
fundamental to characterize the related brain process underlying drivers’ actions.
Driving is a complex behavior which is characterized by the parallel involvement
of neural networks devoted to attentional, decision-making and sensorymotor
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 260–265, 2020.
https://doi.org/10.1007/978-3-030-39512-4_41
Alpha and Beta EEG Desynchronizations Anticipate Steering Actions 261
transformation processes. Many neuroscientific studies have been conducted to study
these single processes in isolation, but only few of them took into account their
investigation in ecological contexts [1]. In addition, other studies exploited the high
temporal resolution of the electroencephalography (EEG) to monitor and detect
increase of workload as well as drowsiness states of pilots [2]. However, these studies
did not investigate direct motor features. In this study, we employed high-density EEG
recordings as technique offering a cost-effective method to investigate the temporal
dynamics of motor and cognitive processes involved in semi-ecological driving
behaviors [3].
In particular, our aim was to identify the neurophysiological features of motor
preparation that distinguish left from right steering in a semi-ecological car driving.
This was achieved by recording and analyzing the EEG continuously while participants
were required to drive with no additional constrain, as they were seated in their own
car. This allowed us to investigate the neural activations corresponding to self-paced
preparation and execution of steering.
To this end, we used a time-frequency decomposition of the Independent Com-
ponent Analysis (ICA) to separate the naturally intermingled EEG activity into max-
imally independent components. Each of such ICA activity is identified by a particular
topography on the scalp, as well as a corresponding pattern of neural generator. This
analysis serve to isolate independent components of the whole motor repertoire of
driving [4]. In this way, we aim to extract independent components whose reactivity
anticipates the steering action, as well as the corresponding topography indicating the
involvement of motor circuits. Specifically, the desynchronization of alpha and beta
rhythms anticipates movements [5]. If this activity will be modulated between left and
right steering, this would demonstrate that EEG can be used to distinguish and possibly
predict motor preparation of behaviors in ecological driving.
2 Material and Methods
This section provides a summary of the experimental setup and the methodology used
for this work. Details are described in our previous study concerning the discrimination
of braking from acceleration foot action [6].
High-density EEG data were collected from twenty-four subjects during a car
driving simulation (4 females, M: 23.6 ± SD: 3.9 years of age, recruited according to
the regulation of the local Ethical Committee). Such environment was composed of a
semi-professional driving simulator seat (RSeat RS1 Assetto Corsa Special Edition), as
well as steering wheel and pedals (Thrustmaster T500 RS). The Assetto Corsa
videogame (Kunos Simulazioni) was presented on a large LED TV set (Samsung 40’’
5300) located 1 m from the participant’s seat. After familiarizing with the whole setup,
participants were asked to perform one lap on a track provided by Assetto Corsa (Coste
Loop track), with the only task to keep the right lane. No other vehicles were present on
the track.
During the whole experimental driving session, we recorded behavioral data related
to the steering wheel angle by means of the Lab Streaming Layer (LSL), as described in
our previous study [6, 7].
262 G. Vecchiato et al.
To detect steering events, we identified major turns on the track as the periods when
the curvature of the road trajectory was outside 5–95 percentile range. Then, for each
period, we found the local maxima of the curvature, which represents the turn of the
road. To identify the time point when the subject initiated the steering, we found the
closest time point before the local maxima of the turn where the steering signal of the
subject is below 7°. If for 15 s before turning the steering signal does not reach this
threshold, the event is discarded. To further filter tentative steering events, we excluded
anomalies defined as events involving pedals pressure, high speed and unusual
dynamics of the car. The number of final events was on average 21.6 ± 2.3, out of
which 53.2% were right steers, and the rest were left steers.
The EEG signals were acquired continuously by means of the 128-channel
HydroCel Geodesic Sensor Net and the Geodesic EEG System (EGI, USA). The EEG
signals were digitized with a sampling frequency of 500 Hz and recorded with a vertex
reference (impedance kept below 10 kX). Then, it was down-sampled to 250 Hz and
preprocessed for line noise removal and bad channels interpolation [8]. Thus, signals
were referenced to a common average. The resulting EEG data were band-pass filtered
([1, 70] Hz) and then subjected to the Adaptive Mixture Independent Component
Analysis (AMICA). Such procedure was performed on the EEG principal components
that explained 99% of the data variance [4, 9]. Next, EEG data were epoched into
segments around the onset of steering actions ([−2000, 1500] ms). Artefacts were
detected and then discarded by using a semi-automatic tool to investigate abnormal
trends and spectra. Then, we performed equivalent dipole source localization (DIP-
FIT2) and semi-automatically separated independent components by means of
SASICA [10]. This procedure allowed us to identify and separate neurophysiological
brain activities from others.
The corresponding scalp topographies of such brain activities were subjected to the
k-means algorithm for a cluster analysis. The event-related spectral perturbations
(ERSP) were then computed for all the components related to these clusters [11]. This
time-frequency decomposition was computed with Morlet wavelets (frequency range
[3, 125] Hz and cycles [3, 25], in logarithmic steps). The pre-event interval of [−1000,
0] ms was used to compute the single-trial baseline correction. Finally, these signals
were converted in dB.
The time-frequency patterns related to the left and right steering conditions were
contrasted using paired sample t-test on the average alpha and beta EEG power bands,
respectively.
3 Results
The cluster analysis of the ICA topographies revealed the centroid maps concerning to the
whole driving session illustrated in Fig. 1. Overall, these components refer to brain
activity needing to adapt and coordinate motor behavior, process sensory information
from the body, as well as plan and control movements. For each cluster, we computed the
time-frequency decomposition for left and right steering in the [−1000, 500] ms time
window with respect to the steering onset. Cluster 3 and 6 are the ones presenting a
significant desynchronization of the alpha and beta rhythms anticipating the action onset.
Alpha and Beta EEG Desynchronizations Anticipate Steering Actions 263
Fig. 1. Centroid topographies for each individual clusters. Each average topographical map
corresponds to a different cluster (Cls), and was computed with a different number of participants
(Ss) and independent components (ICs).
Fig. 2. ERPS patterns for left and right steering trials associated with cluster 3 (upper row) and
cluster 6 (lower row) with the related topographic map. Dashed black contours in the ERPS
panels delimit the time-frequency windows in which the statistical analyses were performed.
Figures 2 shows the time-frequency patterns for left and right steering events
related to the EEG activity of cluster 3 and 6, respectively. It is possible to note the
alpha and beta desynchronizations before the left and right steering onset, whose
264 G. Vecchiato et al.
significance is highlighted by dotted frames. For cluster 3, these activities last around
100 ms and arise in sequence around 800 ms and 600 ms before the steering onset.
Cluster 6 presents a similar pattern of activation associated to the right steering, which
is slightly delayed in time and shifted in frequency. In fact, a cross-correlation analysis
revealed a time-frequency shift of around 200 ms and 0.33 Hz.
Paired t-tests contrasted the alpha and beta desynchronizations between left and
right steering events, for cluster 3 and cluster 6 separately. Such analysis revealed a
significant increase of alpha (t = −2.6, p = 0.02) and beta (t = −3.1, p = 0.01) activity
related to the left steering for cluster 3, as well as a corresponding increase of alpha
(t = 2.4, p = 0.03) and beta (t = 2.9, p = 0.01) activity related to the right steering for
cluster 6.
The localization of dipoles associated to each component revealed that cluster 3 is
generated by sources mostly located across left parietal areas. Instead, cluster 6 gen-
erators are sparser across the right hemisphere, yet they are prevalent in regions across
frontal, central and parietal lobes. These results are illustrated in Fig. 3.
Fig. 3. Component dipoles for cluster 3 (left panel) and cluster 6 (right panel).
4 Discussion
The present study identified two distinct EEG components whose activity is associated
to the preparation of left and right steering actions, respectively. The scalp topographies
of these activities are related to alpha and beta desynchronizations observed in motor
and premotor regions. Such activity generates desynchronization arranged in somato-
topic way during cognitive tasks involving the execution, as well as the observation
and the imagination of motor acts [12].
Overall, the results of this study show that of alpha and beta desynchronizations
appearing between 800 and 600 ms before steering actions can be exploited to dis-
criminate these events in a semi-ecological driving. These results could provide a
Alpha and Beta EEG Desynchronizations Anticipate Steering Actions 265
sufficient time window to implement on-line tools for steering prediction for future
driving assistance devices [13].
Acknowledgments. This research was funded by a CAMLIN Limited and Toyota Motor
Europe.
References
1. Calhoun, V.D., Pearlson, G.D.: A selective review of simulated driving studies: combining
naturalistic and hybrid paradigms, analysis approaches, and future directions. NeuroImage
59, 25–35 (2012). https://doi.org/10.1016/j.neuroimage.2011.06.037
2. Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., Babiloni, F.: Measuring neurophysi-
ological signals in aircraft pilots and car drivers for the assessment of mental workload,
fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75 (2014). https://doi.org/10.1016/
j.neubiorev.2012.10.003
3. Biasiucci, A., Franceschiello, B., Murray, M.M.: Electroencephalography. Curr. Biol. 29,
R80–R85 (2019). https://doi.org/10.1016/j.cub.2018.11.052
4. Delorme, A., Palmer, J., Onton, J., Oostenveld, R., Makeig, S.: Independent EEG sources are
dipolar. PLoS ONE 7, e30135 (2012). https://doi.org/10.1371/journal.pone.0030135
5. Debnath, R., Salo, V.C., Buzzell, G.A., Yoo, K.H., Fox, N.A.: Mu rhythm desynchroniza-
tion is specific to action execution and observation: evidence from time-frequency and
connectivity analysis. NeuroImage 184, 496–507 (2019). https://doi.org/10.1016/j.
neuroimage.2018.09.053
6. Vecchiato, G., Vecchio, M.D., Ascari, L., Antopolskiy, S., Deon, F., Kubin, L., Ambeck-
Madsen, J., Rizzolatti, G., Avanzini, P.: Electroencephalographic time-frequency patterns of
braking and acceleration movement preparation in car driving simulation. Brain Res. (2018).
https://doi.org/10.1016/j.brainres.2018.09.004
7. Swartz Center for Computational Neuroscience (2019). sccn/labstreaminglayer
8. Bigdely-Shamlo, N., Mullen, T., Kothe, C., Su, K.-M., Robbins, K.A.: The PREP pipeline:
standardized preprocessing for large-scale EEG analysis. Front Neuroinform. 9 (2015).
https://doi.org/10.3389/fninf.2015.00016
9. Palmer, J., Kreutz-Delgado, K., Makeig, S.: AMICA: an adaptive mixture of independent
component analyzers with shared components (2011). https://sccn.ucsd.edu/*jason/amica_
a.pdf
10. Chaumon, M., Bishop, D.V.M., Busch, N.A.: A practical guide to the selection of
independent components of the electroencephalogram for artifact correction. J. Neurosci.
Methods 250, 47–63 (2015). https://doi.org/10.1016/j.jneumeth.2015.02.025
11. Makeig, S., Debener, S., Onton, J., Delorme, A.: Mining event-related brain dynamics.
Trends Cogn. Sci. 8, 204–210 (2004). https://doi.org/10.1016/j.tics.2004.03.008
12. Avanzini, P., Fabbri-Destro, M., Volta, R.D., Daprati, E., Rizzolatti, G., Cantalupo, G.: The
dynamics of sensorimotor cortical oscillations during the observation of hand movements: an
EEG study. PLoS ONE 7, e37534 (2012). https://doi.org/10.1371/journal.pone.0037534
13. Zhang, H., Chavarriaga, R., Khaliliardali, Z., Gheorghe, L., Iturrate, I., Millán, Jd.R.: EEG-
based decoding of error-related brain activity in a real-world driving task. J. Neural Eng. 12,
066028 (2015). https://doi.org/10.1088/1741-2560/12/6/066028
The Quantitative Evaluation of Permanent
Disability in Forensic Medicine Through Stereo
Photogrammetric Technology
Claudia Trignano1(&), Andrea Castelli2,3, Vittorio Dell‘Orfano1,
and Elena Mazzeo1
1
Department of Biomedical Sciences, University of Sassari,
Viale San Pietro 43/B, 07100 Sassari, Italy
[email protected]
2
Rehametrics, Valencia, Spain
3
Universitat Politecnica de Valencia, Valencia, Spain
Abstract. The aim of this study is the application of a protocol for the objective
evaluation of knee joint dysfunction using a stereo photogrammetric marker-
based motion capture system. Use this technology allows to obtain quantitative
articular kinematics estimates that translate the movement itself as input for
subsequent processing. The study involved five subjects: two without lower limb
impairments and three with sequelae of knee injuries. The use of a methodology
that provides objective and quantitative results allows a more accurate and correct
assessment of permanent disability percentage and allows to obtain then a more
adequate and fair economic compensation for the suffered damage.
Keywords: Motion capture Photogrammetric stereo technology Forensic
medicine Evaluation permanent disability
1 Introduction
Movement analysis plays a decisive role not only in the clinical environment (diagnosis
and treatment of musculoskeletal disorders), but also in the forensic medicine field,
since it represents an accurate tool for the evaluation and quantification of the per-
centage of permanent disability. This methodology permits improve the law tables and
tabular barèmes use. The parameters related to the impairment of the range of motion of
the different anatomical districts, expressed in degrees or fractions, are obtained with
maximum accuracy from the quantitative analysis of the movement obtained by stereo
photogrammetric estimation. However, the evaluation of articular dysfunctionality is
traditionally carried out by observational assessment of the physician, conditioned by
the operator subjectivity.
The aim of this study is the application of a protocol for the objective evaluation of
knee joint dysfunction using a Motion Capture system, that is defined by Menache as
“the process of recording a live motion event and translating it into usable mathe-
matical terms by tracking a number of key points in space over time and combining
them obtaining a single 3D representation of performance”.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 266–270, 2020.
https://doi.org/10.1007/978-3-030-39512-4_42
The Quantitative Evaluation of Permanent Disability in Forensic Medicine 267
The motion capture technology allows, through kinematic calculations that take
into account the anatomy of the subject analyzed, to obtain accurate estimates of the
articular kinematics that translate the movement itself as input for subsequent
processing.
2 Methods
The stereo photogrammetric system employed was an optoelectronic system (Vicon®,
Oxford UK) that uses spherical reflective markers applied to the subject’s skin. The
system consisted of 6 cameras (Vicon T20 series) and is installed in the Advanced
Imaging and Motion Analysis (AIMA) Laboratory in Porto Conte Ricerche, Sassari,
Italy. The marker protocol used was the conventional Davis et Al (Vicon Plug-in-Gait)
marker protocol (Fig. 1).
Fig. 1. Marker Based motion analysis system by Vicon – Laboratorio Advanced imaging and
motion analysis (AIMA)
The study involved five subjects: two without lower limb impairments and three
with sequelae of knee injuries. These subjects had already been assessed by traditional
medical/legal evaluation.
After the preparation phase of the acquisition system, which consists in the cali-
bration of the position of the cameras and the application of the markers on the subject,
the videos of the bipodal standing tests (static reference test) and of the execution of the
motor tasks such as walking, flexo-extension without load of the knee and mono-podal
squat were acquired (Fig. 2).
268 C. Trignano et al.
Fig. 2. Acquired motor tasks protocol
3 Results
The estimates obtained with the stereo photogrammetric technique revealed interesting
differences with respect to the qualitative evaluation already carried out: in all subjects
with lower limb impairments, the method allowed to detect consistent differences
within the quantification of the articular range of motion estimated by the measuring
system with respect to the clinical examination already carried out (Fig. 3 and Table 1).
Fig. 3. 3D reconstruction and acquired signals for subject no 4
The Quantitative Evaluation of Permanent Disability in Forensic Medicine 269
Table 1. Results summary for all the analyzed subjects, obtained by the stereo photogrammetric
technique, expressed in degrees.
SUBJ Knee max flex-ext Max extension
during squat
Left Right Left Right
1 139.03° 134.646° 0.546° −2.230°
1.927° 3.242°
2 131.71° 139.1° −2.856° −2.018°
1.229° 0.575°
3 140.697° 132.159° 1.718° 2.664°
6.586° 8.851°
4 51.627° 128.876° 31.486° 14.462°
28.202° 7.502°
5 137.505° 133.508° −4.922° −16.768°
−1.528° −10.070°
4 Discussion
The study allowed to estimate the differences between the qualitative measurement
(classical evaluation) and the quantitative one obtained by the stereo photogrammetric
measurement system. In practical terms, these differences translate into obvious errors
of the classical clinical examination in the percentage quantification of permanent
disability according to the normative tables in use (Table 2).
Table 2. Law tables application comparison for each subject. Law tables application results
with tradictional methodology versus tables application using the experimental methodology
based on quantitative estimates provided by the 3D stereophotogrammetric system.
Subject Injury Permanent disability Permanent disability
Clinical evaluation Stereo photogrammetric
technology
No 1– No No No
No 2
No 3 Cartilaginous thigh bone injury Maximum knee flexion 110° Maximum knee flexion
(right knee) 132°
Percentage evaluation in forensic 5% 3%
medicine
No 4 Left knee septic arthritis Extension deficit 15% Extension deficit 28%
Maximum flexion: 60° Maximum flexion: 51°
Residue ROM (range of Residue ROM (range of
motion): 46° motion): 24°
Percentage evaluation in forensic 16% 20%
medicine
No 5 Partial injury LCP Mild ligamentous laxity Hyperextension: 17°
Total injury LCM (right knee) Abduction: 5° Abduction: 9.5°
Percentage evaluation in forensic 7% 12%
medicine
270 C. Trignano et al.
The use of a methodology that provides objective and quantitative results allows a
more precise and correct percentage assessment of functional damage, thus leading to a
later assessment of permanent disability more precise and then allows to obtain a more
adequate and fair economic compensation for the damage suffered.
References
1. Johansson, G.: Visual perception of biological motion and a model for its analysis. Atten.
Percept. Psychophys. 14, 201–211 (1973)
2. Chao, E.Y., Morrey, B.F.: Three dimensional rotation of the elbow. J. Biomech. 11, 57–73
(1978)
3. Foster, Grood, E.S., Suntay, W.J.: A joint coordinate system for the clinical description of
three-dimensional motions: application to the knee. J. Biomech. Eng. 105, 136–144 (1983)
4. Davis III, R.B., Ounpuu, S., Tyburski, D., Gage, J.R.: A gait analysis data collection and
reduction technique. Hum. Mov. Sci. 10, 575–587 (1991)
5. Cappozzo, A., Catani, F., Della Croce, U., Leardini, A.: Position and orientation in space of
bones during movement: anatomical frame definition and determination. Clin. Biomech. 10,
171–178 (1995)
6. Frigo, C., Rabuffetti, M., Kerrigan, D.C., Deming, L.C., Pedotti, A.: Functionally oriented
and clinically feasible quantitative gait analysis method. Med. Biol. Eng. Comput. 36, 179–
185 (1998)
7. Guerra-Filho, G.: Optical motion capture: theory and implementation. J. Theor. Appl.
Inform. 12, 61–89 (2005)
8. Cappozzo, A., Della Croce, U., Leardini, A., Chiari, L.: Human movement analysis using
stereophotogrammetry: part 1: theoretical background. Gait Posture 21(2), 186–196 (2005)
9. Chiari, L., Della Croce, U., Leardini, A., Cappozzo, A.: Human movement analysis using
stereophotogrammetry: part 2: instrumental errors. Gait Posture 21(2), 197–211 (2005)
10. Leardini, A., Chiari, L., Della Croce, U., Cappozzo, A.: Human movement analysis using
stereophotogrammetry: part 3. Soft tissue artifact assessment and compensation. Gait Posture
21(2), 212–225 (2005)
11. Della Croce, U., Leardini, A., Chiari, L., Cappozzo, A.: Human movement analysis using
stereophotogrammetry: part 4: assessment of anatomical landmark misplacement and its
effects on joint kinematics. Gait Posture 21(2), 226–237 (2005)
12. Leardini, A., Sawacha, Z., Paolini, G., Ingrosso, S., Nativo, R., Benedetti, M.G.: A new
anatomically based protocol for gait analysis in children. Gait Posture 26, 560–571 (2007)
13. Baker, R.: The history of gait analysis before the advent of modern computers. Gait Posture
26, 331–342 (2007)
14. Ferrari, A., Benedetti, M.G.: Quantitative comparison of five current protocols in gait
analysis. Gait Posture 28, 207–216 (2008)
15. Di Marco, R., Rossi, S., Castelli, E., Patanè, F., Mazzà, C., Cappa, P.: Effects of the
calibration procedure on the metrological performances of stereophotogrammetric systems
for human movement analysis. Measurement 101, 265–271 (2017)
A Unified Framework for Symbol Grounding
in Human-Machine Interactions
Dingzhou Fei(&)
Department of Psychology, Wuhan University, Wuhan 430072, China
[email protected]
Abstract. We find the need to endow these machines with increasing degrees
of ability to understand the human, so that they can dialogue with human. How
symbols can acquire meaning for the system itself, independent of external
interpretation, is an unsolved problem. After discussing some backgrounds, this
paper introduced dynamic-programming reinforcement learning framework
subject to Z-condition requirements, and it also represents a specific reality of
the semiotic process in a multi-agent setup. This paper suggested that current
research on symbolic grounding can be unified through reinforcement learning
with multi-agent system settings.
Keywords: Symbol grounding Multi-agent system Reinforcement
learning Praxical Markov decision process
1 Symbol Grounding Problem (SGP)
One popular version of the so called SGP was presented by Taddeo and Floridi: How
can the semantic correspondence of a formal symbol system be made intrinsic to the
system, rather than just parasitic on the meanings in our heads? [1]. The SGP can return
to the thought experiment in the Chinese3room [2]. Searle claimed that only bio-
cognitive systems have primitive (non-derivative) intentionality, and that they are the
only systems that give meaning to entities with derived intentionality.
The SGP was first proposed by Harnad [3]. Harnad further developed the Searlian
experiment in two versions: Unfortunately, Harnad’s approach to the SGP is not
precise enough to eliminate much confusion from the core of the problem, and the core
issue involves the semantic autonomy of artificial cognitive systems [4]. Taddeo and
Floridi clarified a basic principle called the Zero Semantic Commitment Condition
(ZSCC). This principle tells us which unreasonable assumptions should be excluded.
They pointed out the following.
No valid solution of the SGP can rely on forms of:
(a) innatism, since no semantic resources (some virtus semantica) should be presup-
posed as already pre-installed in the AA [= artificial agent]; and,
(b) externalism, since no semantic resources should be uploaded from the “outside” by
some deus ex machina that is already semantically-proficient [1].
But ZSCC allow other resources, such as grammar, computing power, perception,
and process capabilities, but only if they are not semantic.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 271–275, 2020.
https://doi.org/10.1007/978-3-030-39512-4_43
272 D. Fei
In this paper, we propose a computational architecture using reinforcement learning
which is subject to ZSCC, and show also how a bio-cognitive system and a robot may
ground symbols or tokens in outside world after a number of iterations as dynamic
programming suggests.
2 Reinforcement Learning and SGP
ZSCC can be applied to any cognitive system with communication capabilities, even if
it is a distributed system. According to Taddeo and Floridi, three main types of
solutions for SGP can be distinguished: the representative model [3], the sensorimotor
AI model [5] and semi-representative AI model [6]. But in reinforcement learning
settings, language is just another form of pragmatic interaction of the artificial agent
with its environment, its semantics does not need representations.
According to Taddeo and Flori [7], previous SGP solutions were unable to satisfy
ZSCC, which is why they proposed their own solutions, calling it praxical emphasizing
the interaction between agents and the environment. It is based on a new theory of
meaning - action-based semantics (AbS) - and a new type of artificial agent called the
two-machine human agent (AM2).
Fig. 1. The Praxical framework
Praxical is an agent system M1 interacting with environment (E) by doing e action
in abstract level (LoA1). Initial stateS1 output meaning f(e) with help from machine
M1. Source: Taddeo and Floridi [1]. Note: this framework for the praxical very similar
to reinforcement learning model.
We have already referred to such an Agent-Agent system as a two-machine arti-
ficial agent, or simply AM2. Taddeo and Floridi argued that AM2 can solve the SGP
while satisfying the Z-condition. This lead to multiple agent system model for the
praxical. Below is another figure illustrating the process of grounding meanings in this
agent system AM2.
A Unified Framework for Symbol Grounding in Human-Machine Interactions 273
Fig. 2. AM2 framework.
AM2 consists of a set of components: Environment(E); two agents M1, M2; LoA2,
a level of indirect interaction between M2 and the E; (S1, Sym1) is an internal state of
M2 when M2 want to ground meanings into formal symbols in this system. S1 an S2
can communicate with each other during evolutions. These meanings grounded are
represented by the internal states with memories for past experiences. M1 has a role in
helping M2 with internally elaborating. Source is the same as Fig. 1.
Note: this diagram describes the interactions between machines and external world
through the system internal states, and share main features with subsequent rein-
forcement learning framework that we proposed in this paper. Like AM2, Reinforce-
ment learning offer a learning modeling: taking actions under a certain strategy, update
currently situations to evaluate actions, and to get best reward signals. In this sense, an
AM2 is really a cognitive version of reinforcement learning.
3 General Framework for AM2
An AM2 is a Markovian Decision Process (MDP), AM2 = <Sym, E, p, A, B,T, R,
MB>, where: Sym is a symbol set, and p is the set of actions available to the agent. E is
the environment, a state s 2 S to be a matrix element of Sym E, for each grounding
there is a symbol sym with an e in E, T is the transition function, i.e., for currently state,
given an action a designs a next state as update state. At each step, R computed the
reward signal, but the agents AM2 may not really know the true value T and R of the
environment E. We can assume that AM2 has a goal state sg 2 S, telling us that
terminating time point comes and grounding process completed. Other notations, A, B
are agents, A is an agent of our interest, A can communicate with B, while MB is a set
of messages from agent B. The AM2 is a description for what the Fig. 2 expresses.
274 D. Fei
We must note that for AM2, the key accessible action is reflection on symbol and
environment, denote by p; a cognitive process including memory, thinking and rea-
soning with intentionality. p0 s definition is below:
pA : Sym M B ! E ð1Þ
This definition can satisfy ZSCC requirement for SGP. Agent A try to correspond a
symbol in Sym to e in the environment in AM2, and we get a pair <sym, e>, with the
message from Agent B, which means that agent system executes this reflection action
to correctly ground meaning for the symbol under some given the transition distribu-
tion T and unknown reward R.
4 Reinforcement Learning in AM2
By virtue of reinforcement learning, we have hypothesis that the goal of solving SGP is
to obtain optimal cumulative reward in AM2 environment E. A usual strategy to do this
is that we adopt an action value function Q(s,a) as an object of learning using rein-
forcement. If maximizing Q-function, the AM2 is supposed to have the expected future
reward for in state s.and chosen action a by M2. In AM2, we have an agent
B-conditioned policy:
Q s; M B ¼ argmaxa2s Qðs; a; M B Þ ð2Þ
We use model-aware RL methods, which can learn world dynamics while using
dynamic algorithm-based value iterations to estimate the best Q [8]. program. In our
setup, the value iteration updates the formula to:
X
Qn þ 1 s; a; M B ¼ s0 2S
T½ðs0 js; a; M B Þ þ cV n s0 ; M B ð3Þ
Vn þ 1 s; M B ¼ maxa2p Qn þ 1 s; a; M B ð4Þ
Where c is the discount factor as usual, n is the number of iterations. Updating the
estimated values of transition and reward R learn from the environment of the AM2
system, then M2 can give semantics to the system.
5 Examples
Recent work on grounding meaning of symbols within reinforcement learning
framework have many faces, In [9], the language narration is associated with the game
character judgment (this is a spatial reasoning task). [10] Through the dynamic pro-
gramming of the agent system, the description of the text can capture dynamic changes
in the environment. The 2019 newest [11] present two approaches for learning com-
munication protocols in order to ground meaning for digitals in MNIST game with
A Unified Framework for Symbol Grounding in Human-Machine Interactions 275
deep networks. Our model can be abstract form compared with those work. our work
leaves some puzzles to solve, such as sparse reward, and we hope that this framework
for tackling the SGP is a starting point toward into good solutions.
References
1. Taddeo, M., Floridi, L.: A praxical solution of the symbol grounding problem. Mind. Mach.
17(4), 369–389 (2007)
2. Searle, J.R.: Minds, brains, and programs. Behav. Brain Sci. 3, 1–19 (1980)
3. Harnad, S.: The symbol grounding problem. Phys. D 42, 335–346 (1990)
4. Cangelosi, A., Harnad, S.: The adaptive advantage of symbolic theft over sensorimotor toil:
grounding language in perceptual categories. Evol. Commun. 4(1) (2002). (Special Issue on
Grounding Language)
5. Varshavskaya, P.: Behavior-based early language development on a humanoid robot. In:
Proceedings of the Second International Workshop on Epigenetic Robotics: Modeling
Cognitive Development in Robotic Systems, pp. 149–158. Defense Technical Information
Center (2002)
6. Cohen, P., Sutton, C., Burns, B.: Learning effects of robot actions using temporal
associations. In: Proceedings 2nd International Conference on Development and Learning,
ICDL 2002, pp. 96–101. IEEE Computer Society, Los Alamitos (2002)
7. Taddeo, M., Floridi, L.: Solving the symbol grounding problem: a critical review of fifteen
years of research. J. Exp. Theor. Artif. Intell. 17(4), 419–445 (2005)
8. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press,
Cambridge (2018)
9. Prabhudesai, M., Tung, H-Y.F.: Embodied language grounding with implicit 3D visual
feature representations. https://arxiv.org/abs/1910.01210
10. Narasimhan, K., et al.: Grounding language for transfer, in deep reinforcement learning.
J. Artif. Intell. Res. 63, 849–874 (2018)
11. Foerster, J., et al.: Learning to communicate with deep multi-agent reinforcement learning
(2018). https://arxiv.org/pdf/1605.06676.pdf
Improving Machine Translation Output
of German Compound and Multiword
Financial Terms
Christina Valavani(&), Christina Alexandris, and George Mikros
Department of German Studies, National and Kapodistrian University of Athens,
Panepistimioupoli, 15784 Zografou, Athens, Greece
[email protected], [email protected],
[email protected]
Abstract. The present application translates German financial terms to the Greek
language aiming to maximize the translation quality. The demand for better
translation increases the need for more MT systems, journalists or professionals in
the banking sector require accurate translations. A novel statistical method of MT
is presented, where we use a probability distribution over sentence pairs from a
parallel corpus (German-Greek). Both the MT System and the parallel corpus was
built from scratch. The present application combines parameters in a new way and
estimates the probabilities that extract the best translation.
Keywords: Statistical machine translation system Financial terms German
to Greek Translation application Noun phrases Java Translation models
Alignment models
1 Introduction
The proposed application focuses on the translation of complex economic structures
from German into Greek. It concerns statistically-based Machine Translation using a
parallel text corpus (German-Greek). The corpus consists mainly of financial texts and
respective terminology and it is, used for the optimal translation based on probability.
The application is simple to use and it functions by entering the term for translation and
getting the most likely translation or translations.
The application is intended to improve translation results from existing Machine
Translation systems such as Google Translate. Despite recent advances in Machine
Translation, including systems using neural networks, it is observed that for some types
of input, such as compound and multiword financial terms, the integration of special
parameters and processing strategies is necessary.
The present approach is based on alignment models and their integration in sta-
tistical models.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 276–281, 2020.
https://doi.org/10.1007/978-3-030-39512-4_44
Improving Machine Translation Output of German Compound 277
2 Alignment Models
The financial terms of the German-Greek text corpus are processed in respect to their
distinctive linguistic features and morphosyntactic structures, especially if compound
multiword terms are concerned. Processing is linked to the identification of the dis-
tinctive types of linguistic features and morphosyntactic structures of German financial
terms derived from previous research [10]. The distinctive types of linguistic features
and morphosyntactic structures of German financial terms serve as a basis for facili-
tating statistical machine translation and successfully generating the corresponding
structures in the Greek language. The morphosyntactic structures of the German
compound and multiword financial terms, henceforth referred to as “structures”, are
defined based on their translation characteristics regarding the word order during
translation. The definition of the “structure” type of the German term allows its sub-
sequent mapping, processing and generation in the equivalent term or expression in
Greek. There are various types of models concerning the alignment-mapping of the
German compound and multiword constituents to the equivalent words or phrases in
the Greek target language.
In the so-called “in the same order” model of alignment, the translation of the terms
in the target language is in the same order as in the source language, in this case,
German and Greek. Translation with a “in the same order” model of alignment may
concern both individual words or compound and multiword constituents. The “in the
same order” model of alignment concerning the translation of German financial terms
is usually observed when the first component of the German term is an adjective or an
adverb, as shown in the following Example 1:
Top-Renditen
υψηλές αποδόσεις
p (el|de)
w1 -w2
w1 w2
Example 1. “In the same order” model: Term with two words/constituents joined with a hyphen
(Koehn, 2008)
In the so-called “crosswise” model of alignment [9], the order of the constituents in
compound and multiword terms is reversed in the target language. Characteristic
examples are the cases in which the first constituent of the German financial term is a
(proper)name of a company, bank or institution (Example 2).
278 C. Valavani et al.
Lehman-Pleite
πτώχευσητης Lehman
p (el|de)
w1 -w2
w2 w1
Example 2. “Crosswise” model: Term with two words/constituents joined with a hyphen [4].
Additionally, the processing and translation of compound and multiword German
financial terms may involve the combination of “in the same order” models of align-
ment with “crosswise” models of alignment or even an insertion of an additional
constituent, as depicted in Example 3.
Wirtschafts-und Währungsunion
Οικονομική και νομισματική ένωση
p (el|de) “In the same order” translation
x3
w1 -w2 w[3+4]
alignment model3(am3)
1st and 2nd “in the same w[3ν4] w3 w4 (separation of the compound
word)
order” level
w1 w2w3 w4
Example 3. Combination of a “in the same order” model with an insertion of an additional
constituent.
Some structures appear particularly complex in terms of their output in the Greek
language. The dotted line is the word “and” or the word “or”. The point where a line
intersects means that, while it was a single-word compound in Greek, the compound is
rendered with more than one word.
3 Integration of the “Re-ordering” Algorithm
One of the most distinct problems encountered in Machine Translation is word
reordering, that is, the order of words in sentences or phrases in the two languages.
Reposition of words in a different order does not just pertain to the order of words in a
sentence, but also to the order of words in rendering of a multiword phrase or a
complex multi-compound word. Word reordering can occur even in the case of a
compound word in German that is translated in a noun phrase into Greek. When the
System must translate phrases or complex words, then the sequence in Greek no longer
Improving Machine Translation Output of German Compound 279
continues to be the same as in German. There is often some reordering within the
terms, mainly due to the fact that the Head of the phrase or of the compound term in
German is at the end, while this is not the case in Greek. In English, the Head is to the
left, except where there is an adjective before the Head.
The purpose of the algorithm is to produce a correct word order in Greek; although
no annotation is required, the objective is to enable the System, through reordering and
definition of translation models, to place, inter alia, the Head of the phrase in the right
place for translation, thus testing some possible internal reordering. The alignment
models play a key role in the system. The structural Greek matches assigned to the
German structures of financial terms are specified and integrated into the System. The
proposed algorithm for improving statistical machine translation results is based on error
classification collected from previous research concerning the machine translation of
German financial terms into the Greek language (Valavani [10]). The algorithm con-
cerns the interaction between specific parameters (for example: translation (t), fertility
(n), distortion, language model (p), word order (b)) for the calculation of a final prob-
ability value. The parameters are applied to financial-related phrases, which, as far as the
System is concerned, consist of n-grams rather than words with grammar and syntax
information. The calculations are based on a combination of numerical parameters, in
order to obtain the optimum result of the translation system. The final calculated value is
based on a math formula, which takes into account all the parameters.
4 Input Processing
It is worth noting that several elements, such as parameters and ways of calculating the
algorithm, are included in IBM Models [2] and Phrase-based models. However, it is
emphasized that the proposed approach has additional elements, such as the control of
some numeric parameter values of phrases and not just words, taking into account the
modeling values also proposed. In order to achieve the best translation quality, the
System uses the following parameters, calculations and other features:
1. Probability calculation
2. Parameters
3. Calculation of word shifting in target language
d (distortion ðdðijj, l, mÞÞ ð1Þ
4. Calculation of language model in target language - p(LM)
5. Calculation of word-based translation model
pðejf Þ ½pðejf Þ ¼ pðejf Þ pLMðeÞ ð2Þ
280 C. Valavani et al.
6. Phrase-based translation model check
pðejf Þ ½pðejf Þ ¼ pðejf Þ pLMðeÞ ð3Þ
7. Use of the EM Algorithm and the IBM Model 3
8. Fertility calculation - n (words)
9. Fertility calculation - n (phrases)
10. Translation calculation1
countðf; eÞ
t ðwordsÞ tðf jeÞ ¼ ð4Þ
countðeÞ
11. Translation calculation
countðf; eÞ
t ðphrasesÞ tðf jeÞ ¼ ð5Þ
countðeÞ
12. Monolingual Text Corpus
13. Parallel Text Corpus
14. Use of N-grams
15. Word alignment a:j!i
16. Type of calculation of the final probability value
17. Word-for-word translation
18. Phrase-level translation
For the development of the algorithm, the statistical approach adopted was Knight’s
with data generated by Koehn [4], Bhattacharrya [1], Jurafsky and Martin [3] and IBM
Models, in combination with alignment models. In short, it seeks a Greek match for the
x German structure based on probability. The process is driven by the alignment
models, since the success of the algorithm lies in the alignment models. The proposed
algorithm stands out and is characterized by the following innovative features:
1. Definition of the alignment models German-Greek.
2. Integration of the alignment models in the System.
3. The sequence of words in the target language is shifted at each possible translation
and the language model, parameter b, is checked.
4. Fertility n and translation t testing is performed on two levels, phrase-level and
word-level.
5. Use of a specified formula to calculate the highest final probability value
6. Specific sequence of steps of the algorithm.
1
Arnaldo P. https://www.youtube.com/watch?v=jrgDgXLpsZg&list=PLO9y7hOkmmSFcf-wloCJ0B
Hp7GSQoZgR8&index=5.
Improving Machine Translation Output of German Compound 281
5 Conclusions and Further Research
We have presented an algorithm which translates types of German economic termi-
nology into the Greek language, mainly multilingual terms. Based on a parallelized and
a monolingual corpus it collects statistics on the possible translation and fertility of
each term as well as other parameters that interact with each other to produce the most
likely translation for a phrase. Specifically, the System controls the environment (which
words precede, and which follow) and finally, the sequence of words selected has the
highest probability of being correct.
As previously described, the present approach concerns the integration of special
parameters and processing strategies for processing German compound and multiword
financial terms, as a type of input that cannot always be successfully processed, in spite
of recent advances in Machine Translation, including MT systems using neural net-
works. Improvements to the implemented algorithm are intended to address any
problematic cases not resolved with the present approach.
References
1. Bhattacharayya, B.: Machine Translation. CRC Press/Taylor & Francis Group LLC, Boca
Raton (2015)
2. Brown, P.F., Cocke, J., Pietra, S.A.D., Pietra, V.J.D., Jelinek, F., Lafferty, J.D., Mercer, L.
R., Rossin, P.S.: A statistical approach to machine translation. Comput. Linguist. 16(2), 79–
85 (1990)
3. Jurafsky, D., Martin, H.J.: Speech and Language Processing, An Introduction to Natural
Processing, Speech Recognition and Computational Linguistics, 3rd edn. draft. Prentice
Hall, Englewood Cliffs (2018, in print)
4. Koehn, P.: Statistical Machine Translation. Cambridge University Press, New York (2010)
5. Knight, K., Chander, I., Haines, M., Hatzivassiloglou, V., Hovy, E.H., Iida, M., Luk, S.K.,
Okumura, A., Whitney, R.A., Yamada, K.: Integrating knowledge bases and statistics in MT.
In: Proceedings of the Conference of the Association for Machine Translation in the
Americas (1994)
6. Knight, K.: Automating knowledge acquisition for machine translation. AI Mag. 18(4), 81–
96 (1997)
7. Knight, K.: A statistical MT tutorial workbook. Prepared in Connection with the JHU
Summer Workshop (1999). http://www.isi.edu/natural-language/mt/wkbk.pdf
8. Koutsis, I., Kouklakis, G., Mikros, G.K., Markopoulos, G.: MINOTAVROS: a tool for the
semi-automated creation of large corpora from the Web. In: Proceedings from the Corpus
Linguistics Conference Series, 14–17 July 2005, Birmingham, UK, vol. 1 (2005). http://
www.corpus.bham.ac.uk/PCLC/minotavros.doc
9. Sternefeld, J.: Syntax, Eine morphologisch motivierte generative Beschreibung des
Deutschen, Band I, Tübingen: Stauffenburg Verlag, Brigitte Narr (2006)
10. Valavani, C.: Analysis and Processing of German Multi-word Financial Terms in Bilingual
and Multilingual Applications. Doctoral thesis, National and Kapodistrian University of
Athens (2019)
Self-adjusted Data-Driven System
for Prediction of Human Performance
Oleksandr Burov1(&), Evgeniy Lavrov2, Nadiia Pasko3,
Olena Hlazunova4, Olga Lavrova5, Vasyl Kyzenko6,
and Yana Dolgikh3
1
Institute of Information Technologies and Learning Tools,
9 Berlinskoho Street, Kyiv 04060, Ukraine
[email protected]
2
Sumy State University, Sumy, Ukraine
[email protected]
3
Sumy National Agrarian University, Sumy, Ukraine
[email protected]
4
National University of Life and Environmental Sciences of Ukraine,
Kyiv, Ukraine
[email protected]
5
Kyiv National University named Taras Shevchenko, Kyiv, Ukraine
[email protected]
6
Institute of Pedagogy of the National Academy of Educational Sciences
of Ukraine, Kyiv, Ukraine
[email protected]
Abstract. Design principles and the data-driven system to assess and to predict
an operator readiness-to-perform are discussed in the article. Principles of
construction and performance of the system are formulated. The main focus is
on data organization (time line, date set for the model construction) and adaptive
algorithm construction. High level of the prediction accuracy for an operator
readiness-to-perform (85–90%) was achieved because of use data stored (pa-
rameters of time and cognitive tasks performance by user) the system to control
its performance, as well as its self-adjusted algorithm of functioning.
Keywords: Data-driven Cognitive performance Self-adjustment Adaptive
models
1 Introduction
To date, emergent technologies and education are based on information and commu-
nication technologies with increasing role of artificial/hybrid intellect, as well as Self-
Organizing Human-Centered Systems [1]. Appropriate systems need to be anthro-
pocentric ones and need to register big amount of data to be effective in real settings
[2]. Implementation and use of adaptive automation of control systems in industry clear
demonstrated the need to create adaptive, self-organizing and/or self-adjusted construct
of such systems [3]. The greatest problem of this task consists in self-learning of the
system (or machine learning, in today’s terminology), and we strongly believe that the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 282–287, 2020.
https://doi.org/10.1007/978-3-030-39512-4_45
Self-adjusted Data-Driven System for Prediction of Human Performance 283
effective way is to use concepts of data-driven systems [4], when algorithm of the
system’s behavior is quite flexible [5] and use data concerned human (operator) and
adaptive technical parts’ performance [6].
Purpose. To design principles and the data-driven system (DDS) to assess and to
predict an operator readiness-to-perform.
2 Method
To date, education moves into digital environment, becomes more individually oriented,
learner changes his status from object to subject of education and needs more flexible
and adapting tools for learning, his/her learning activity acquires traits of operator-
researcher. Thus, design principles and the data-driven system to assess and to predict
(DDS-AP) an operator readiness-to-perform (O-RtP) are valid for learning activity as
well [7], if we consider a learner activity as a type of operator work (operator-
researcher). Multiple regression models are used for assessment and prediction of
operator performance. This approach uses preliminary findings of authors [8].
3 Results and Discussion
In general case the system developed for a pre-shift assessment of power plant O-RtP
should meet the following modified requirements [7]:
• The prediction of the O-RtP should be constructed accounting his/her individual
psychophysiological and performance features with automatic construction and
change of individual “norm” in correspondence the O-RtP changes over year and
age, i.e. adapting to slow changes of the human psychophysiological status. Special
attention is paid to individual nature of the functional state changes [9] and cog-
nitive workload of operator under influence of internal and external factors [10].
• It is generally recognized that any type of professional work and type of learning
need some preliminary “training and a human adaptation to specificity of problems
to be solved, parameters of test performance as a model of professional work vary in
due course as well” [7]. To solve this problem, the model of the O-RtP should
assess the operator’s level of training to perform tests and should be corrected in
accordance to this level.
• In accordance to the Godel’s external complement principle, correcting of the O-
RtP model, parameters of his/her real work (successfulness of professional duties
performance) or achievements in learning process should be applied.
• The system can be applied by both the operator himself to correct his functional
state, and his supervisor (chief operator, teacher) to make organizational decisions.
Principles of Functioning of the System. The active current structure of the DDS-AP
(active sub-systems) is determined at every use in dependents on results of test per-
formance and the current model of the operator performance. “In other words, the
information which deals with the current operator state and is received by the system
284 O. Burov et al.
(task performance time and number of errors) determines change of internal structure of
the system” [7]. At the same time, the information associated with efficiency of pro-
fessional duty or learner performance is used to make changes in models of the O-RtP
adjusting them to the current functional state of the operator. In other words, DDS -AP
provides permanent matching of the external and internal information. The appropriate
principles of the system functioning were developed and considered [7]:
• Use of individual models and individual “norm” of the operator psychophysio-
logical state.
• The multivariate analysis of test performance parameters.
• Regular verification of the prognosis according to external criterion.
• Model adaptability to slow changes of the operator functional state.
We recommend using as external parameter the most informative one in relation to
the particular field of the system use that contains an “integrated” evaluation of the
operator professional performance efficiency of conducting technological process by
the operator. In process of improvement of technological estimations of the operator
professional work efficiency, the external criterion is improved as well. Thus, the
system “is retrained” over monitoring time accordingly to this criterion and is stable at
the level of functioning and prediction concerning a new technological parameter
(“commander mark” can be used as the expert mark of the learner supervisor in the
sense of technological parameter).
Data-Driven Organization of the System Functioning. An example of the result of
the transformation of a management system using such information technology is the
implementation of a self-tuning system for the alternate control of the operability of the
SPORO operator, which is intended for use at complex technological objects and, in
particular, at power plant and railway [11].
System reliability, accuracy and robustness of prognostic estimates are ensured by
the implementation of the following principles:
• autonomous management of the system;
• modeling of dynamics of operator activity;
• test results usage as an external criterion for the activation of functional subsystems;
• adaptive forecasting of external performance index.
An operator performance reassessment system is designed to collect objective
information about operators in a real production environment, where the efficiency of
the information received, and the ease of data manipulation are essential factors.
Therefore, as a database, a relational type database was developed to support the
SPORO information system, the structure of which is a set of interconnected files that
have a unified internal structure and are implemented as direct access files.
Each file (or relation over the parameters of the operator in the model conditions
and the parameters of the effectiveness of his professional activity), which is an
information model of psychophysiological characteristics of the functional state of the
operator, has a logical data structure that is understandable for the personnel operating
the system and medical researchers. This is an m-ary relation, the degree of which
depends on the specific conditions of use (in existing implementations varies from 34
Self-adjusted Data-Driven System for Prediction of Human Performance 285
to 60). Tuples are datasets that characterize the parameters of information processing
time in test tasks, the reliability of work, the operator’s performance of his professional
tasks. In order to automate the administration (administration) of the database, two
domains are allocated for the date and time of testing.
Each tuple is the result of displaying in the database the original time series (TS) of
the test indicators. In time series, compression of information to save not only the
original TS{V} but only its essential m characteristics is important in displaying in the
database. Then
fvgk1 ! f xgm
1 S;
where S is the set of parameters that describe the current functional state of the
operator.
In this case, the overall model of the functional state and operator performance is
described as
yN ¼ UN ðxp 2 XÞ;
where p m, has an error e, depending on the exponent p and the number of
observations N included in the model.
Since the parameters depend on the time x = x (t), e can increase due to the “aging”
of the data, that is, if xj(t) = F(x1, x2, …, xj−1), тo ej ej−1, then ej ej − 1 for all p.
There is an optimum N at which ej = min. The values of p and N for practical
purposes are determined experimentally on a case-by-case basis. The number of tuples
is constant for all database files and allows to store a volume of information enough for
continuous support of the dynamic model of the managed object (operator) and is
selected depending on the equipment used.
The database management system includes tools both traditional for the database
management system (DBMS) and included in applications. Due to this, in DBMS
automatic mode defines the following functions, implemented by the system at each
operator testing session: testing (including the degree of operator’s training with the
system); processing of information and formation of information model of the operator
functional state at the time of testing; estimation of necessity of correction of dynamic
model of forecasting of efficiency of professional activity, necessity of correction of a
condition of a DB. Accordingly, the system DDS-AP used data control of their work to
adjust it to every user and to his/her fitness-for-duty depending on data stored related to
past and present parameters of operator’s cognitive test performance and external
(“technological”) information related to his/her performance.
By using direct access binary data as a physical structure, economical use of the
external memory of the database is achieved (which allows the system to be used even
on old computers that are still in use at the factory), it reduces the time of access to
information elements and provides access authorization.
Applied results of use of the preliminary system demonstrated high relationship
between indices of operators’ task performance and indices of their professional work
that were registered by the technological automatic system of the plant (multiple
286 O. Burov et al.
relationship ratio R = 0,7–0,9) [7]. And the most important result was the accuracy of
the prediction by the system (85–90%).
4 Conclusion
The results of the DDS-AP in real settings (high accuracy of prediction of the O-RtP)
were achieved because of use data stored in the system to control its performance, as
well as its self-adjusted algorithm of functioning. This could be a tool to improve
operator performance using predictive models and control by the system’s data, as well
as a tool for systems of adaptive learning.
Acknowledgments. The work was supported by the grant of the National Academy of Edu-
cational Sciences of Ukraine # 0118U003160 “System of computer modeling of cognitive tasks
for the formation of competencies of students in natural and mathematical subjects”.
References
1. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif.
Intell. 267, 1–38 (2019)
2. Zhang, Y., Sun, J., Jiang, T., Yang, Z.: Cognitive ergonomic evaluation metrics and
methodology for interactive information system. In: Ahram, T. (ed.) Advances in Artificial
Intelligence, Software and Systems Engineering, AHFE 2019, AISC 965, pp. 559–570
(2019). https://doi.org/10.1007/978-3-030-20454-9_55
3. Hockey, G.R.J., Gaillard, A.W.K., Burov, O. (eds.): Operator Functional State: The
Assessment and Prediction of Human Performance Degradation in Complex Tasks, pp. 249–
255 (2003)
4. Blasch, E., Ravela, S., Aved, A.: Handbook of dynamic data driven applications systems (2018).
https://www.ebooks.com/en-ua/209539447/handbook-of-dynamic-data-driven-applications-
systems/erik-blasch-sai-ravela-alex-aved/
5. Burov, O.Y., Pinchuk, O.P., Pertsev, M.A., Vasylchenko, Y.V.: Using the students’ state
indices for design of adaptive learning systems. Inf. Technol. Learn. Tools 68(6), 20–32
(2018)
6. Lavrov, E., Barchenko, N., Pasko, N., Tolbatov, A.: Development of adaptation technologies
to man-operator in distributed e-learning systems. In: Proceedings of 2nd International
Conference on Advanced Information and Communication Technologies, AICT 2017,
pp. 88–91 (2017). https://doi.org/10.1109/aiact.2017.8020072
7. Burov, O.Y.: ICT for performance assessment of emergent technologies operators. In:
Proceedings of the 13th International Conference on ICT in Education, Research and
Industrial Applications. Integration, Harmonization and Knowledge Transfer Kyiv, Ukraine,
15–18 May 2017, vol. 1844, pp. 127–138. CEUR-WS (2017)
8. Lavrov, E., Lavrova, O.: Intelligent adaptation method for human-machine interaction in
modular e-learning systems. In: Proceedings of the 15th International Conference on ICT in
Education, Research and Industrial Applications. Integration, Harmonization and Knowl-
edge Transfer. Volume II: Workshops, Kherson, 12–15 June 2019, pp. 1000–1010 (2019)
9. Basye, D.: Personalized vs. differentiated vs. individualized learning. ISTE 1/24/2018
(2018). https://www.iste.org/explore/articleDetail?articleid=124
Self-adjusted Data-Driven System for Prediction of Human Performance 287
10. Veltman, H., Wilson, G., Burov, O.: Operator functional state assessment. Cognitive load.
NATO Science Series RTO-TR-HFM-104, Brussels, pp. 97–112 (2004)
11. Burov, A.Ju.: Psychophysiological maintenance of operators’ work. Informacionno-
upravljajushhie sistemy na zheleznodorozhnom transporte, #6, pp. 32–34 (1999)
Human Factor and Cognitive Methods
in the Design of Products and Production
Systems of Mechanical Engineering
in the Framework of NBIC Convergence
Evgeny Kolbachev(&), Elena Sidorova, and Polina Vaneeva
Platov South-Russian State Polytechnic University (NPI), Prosveschenia Street
132, 346428 Novocherkassk, Russian Federation
[email protected]
Abstract. The article describes research of human factor of the manifestation in
processes of product development and production systems of mechanical
engineering, improvement of these processes by application of cognitive
methods and human factor control. It is proved that the importance of the human
factor in the development of products and production systems of mechanical
engineering necessitates the use of cognitive methods in design with the sub-
sequent creation of Artificial Cognitive Systems. A list of such methods and
approaches to their practical application is proposed. It has been established that
the most effective way to use cognitive methods in the design of machines and
other technical systems is to include them in CAx systems (primarily systems
such as MCAD and EDA). In this case, the most appropriate way to organize
design processes is the use of Agile-ideology.
Keywords: Human factors Cognitive methods Mechanical engineering
Systems engineering NBIC convergence
1 Introduction
In works published at the beginning of the 21st century, Bainbridge and Roko [1, 2]
discovered the phenomenon of interaction between the development processes of
nanotechnology, biotechnology, information technology, and cognitive science. It was
called “NBIC Convergence.” The development of NBIC technologies is very important
for the global economy as a whole and, along with the formation of Industry 4.0 and
reindustrialization, is an important factor in the formation of a new economic reality.
In different industries, NBIC factors act differently. These features require research.
In particular, this must be done for mechanical engineering, an industry that largely
affects the development of production systems in all sectors.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 288–293, 2020.
https://doi.org/10.1007/978-3-030-39512-4_46
Human Factor and Cognitive Methods in the Design of Products 289
2 Forms of NBIC Factor Manifestation in Mechanical
Engineering
The effect of individual NBIC factors in mechanical engineering conditions is specific
and not always obvious.
The most obvious effect of factor I. It is related to the wide use of information
technologies and cyberphysical systems in modern production systems of mechanical
engineering.
In mechanical engineering, the action of factor N. is very specific. It is related to the
control of processes of subactive manufacturing at the molecular level (application of
tools from materials with a given molecular structure (nanomaterials), with coated
tools, application of cutting fluid, etc.). The development of these technologies also
requires the use of cognitive methods in the formation of technical solutions. The
experience of creating a coated tool and a new cutting fluid using cognitive functional-
oriented approach is described below.
In the context of additive manufacturing processes, the share of which in
mechanical engineering is steadily increasing, the influence of nanofactors is obvious,
as the typical layer thickness is around 100 lm (250 DPI), although some machines
can print layers as thin as 16 lm (1600 DPI) [20].
It is most problematic the situation with manifestation of a factor of Century is.
Application biological (biochemical, biophysical) methods is not characteristic of
technological processes of mechanical engineering. Here, in our opinion, it is necessary
to talk about use of methodology of biological science at a solution of applied problems
of design and production of machines and other technical systems and also application
of the corresponding analogies to wildlife [18, 19].
Another example of the use of biology methods in engineering can be the theory of
technocenosis - artificial systems that are limited in time and space (product commu-
nities) with weak connections and common goals, allocated for the purposes of design
and organization of production. Known experience in the formation of sets of equip-
ment based on this theory [21, 22]. Management of technocenosis relies on research
methods for biocenosis.
The action of factor C in the context of our study requires a separate, more careful
consideration (see the next section).
3 Cognitive Methods and Their Application to Solve Applied
Problems in Mechanical Engineering
It’s obvious that in mechanical engineering, the role of cognitive methods is greatest in
the design of machines and other technical systems.
In this case, formalized methods for generating new design ideas and other tech-
nical solutions are very important.
As noted above, attempts to shift the solution of design tasks “to the shoulders” of
artificial intelligence in the foreseeable future are unlikely to be successful. However,
various techniques that formalize the processes of the designer’s thought activity when
290 E. Kolbachev et al.
generating new technical solutions, and therefore can be classified as cognitive, allow
to increase efficiency of creation of new machines and other technical systems.
First of all it is the behaviorism methods [3, 4] and some other applied methods
allowing to formalize processes of generation of new knowledge and forming of
technical solutions. Among the last the most effective represent methods of theory of
inventive problem solving (TRIZ) developed by Altshuller, [5] in the second half of the
last century in the Soviet Union and developed by his pupils Zlotin and Zusman [6, 7],
who moved to the USA and based there Altshuller’s TRIZ Institute [8], which
developments were widely used in the American corporations (for example [9]).
In addition, the method of generating engineering solutions developed at the end of
the last century by Polovinkin is of interest. [10] and developed by his students [15, 16]
in recent decades. In our opinion, this methodology is of particular interest in the
context of our study, since it provides for the procedures for creating human-oriented
production systems based on the canons of Christianity (blagodatnye systems).
Interesting is the functionally-oriented approach to process design developed at the
beginning of this century in Donbass [13]. They are based on compositions of the main
particularities and principle of the syntheses multifunction (hybrid); macro-, micro- and
nanotechnology. These technologies pertain to new class organizing-technological
forms technology.
The methodology for applying the functionally-oriented approach in subtractive
manufacturing, developed in [14], allows you to control the process and select tech-
nological influences depending on the nature of thermomechanical loading in various
zones of interaction of cutting plate and material. Obtaining information about the
zonal loading of the effort in the area is possible by creating a dynamic thermome-
chanical model. Such a detailed understanding of physical and mechanical phenomena
allows us to move to a qualitatively new level of technology and solve such production
problems as:
– increased efficiency without reduction of tool service life and quality of treated
surface;
– providing the set resource of the cutting plate that it is especially relevant in the
conditions of the automated production (the processing centers, machines with
program control, other cyberphysical devices, etc.) [2].
It is also necessary to mention cognitive methods of value analysis (VA) and value
engineering (VE). VE follows a structured thought process that is based exclusively on
“function”, i.e. what something “does” not what it is. The method was first proposed
and developed by Miles [13] has become widespread in different countries. The
effectiveness of this approach is confirmed by the fact that in the United States, value
engineering is specifically mandated for federal agencies [15].
VA and VE are examples of the positive influence of economic (value) methods on
the formation of technical solutions.
Another cognitive-based economic method that contributes to the formation of
technical solutions is the system paradigm Kornai, described in his paper [15].
This work by J. Kornai provides a methodological basis for the system presentation
of the production system, as well as approaches to the classification and analysis of
both existing and possible production system options. Our [CEES] report shows that a
Human Factor and Cognitive Methods in the Design of Products 291
number of ideas of the Kornai system paradigm and their development in the works of
Kleiner [17] can be applied in the design of technical systems.
4 Results Obtained, Experience with Methods
The methods described in the previous section were used in the development of some
components of railway equipment, mining equipment, reclamation equipment and
water management. Some are shown in Table 1.
Table 1. Projects and groups of projects executed with the participation of authors using certain
cognitive methods
Designed projects Years of The applied methods
performance TRIZ Functionally- Value Kornai- AGILE-
oriented engineering system ideology
approach (VE) paradigm
Components of 2014–2019 + − + − +
electric locomotives
Material mining 2009–2011 + − + + −
equipment
Waterside structures 2004–2015 + − + + −
Technology and 2010–2012 − + − − −
tools for turning
Power installations 2018–2019 + − + + +
for the melioration
systems
The application of described above method is most appropriate at the stages of
planning and conceptual development of a product (stages of terms of reference and
technical proposal) while forming technical requirements, quality indicators, economic
parameters of developing technical system within actions of managing demands and
interaction with a customer (CRM - RqM).
It is recommended to apply the described above method under conditions of
MCAD, EDA and other CA-systems. It might be applied within invariant subsystem of
technical-economic indicators among tools of calculations automatization CAE (CAA),
that is most effective while organizing designing with the usage of AGILE ideology. In
that case, there is an opportunity of reacting promptly to changes of designing con-
ditions (external and internal), of managing the usage of new knowledge formed during
designing technical system [23].
292 E. Kolbachev et al.
5 Conclusion
The conditions in which modern Advanced Manufacturing and Production System
technologies are created are determined by the beginning of NBIC convergence pro-
cesses and the emergence of the new Technological Paradigm. In mechanical engi-
neering, these processes occur specifically, which is due to the prevalence of subactive
manufacturing in them and the significant share of human participation in the design of
products and production systems.
The importance of the human factor in the improvement of product development
processes and production systems of mechanical engineering makes it necessary to
apply cognitive methods in design, followed by the creation of artistic cognitive
systems.
The most promising methods are behaviorism, methods of the theory of solving
inventive problems (TRIZ), a method for generating engineering solutions, a
functionally-oriented approach to the design of technological processes, methods of
value analysis (VA) and value engineering (VE), methods based on system paradigm
Kornai.
The most efficient way of using presented approach is making it a part of CAx
system (first of all, such systems as MCAD and EDA). It may be applied as a part of
invariant subsystem of technical and economic parameters for the computer-aided
account facilities and for processes simulation (CAE, CAA). The most appropriate way
of organizing design and development processes in the said system is to use Agile-
ideology.
References
1. Roco, M.C.: Converging science and technology at the nanoscale: opportunities for
education and training. Nat. Biotechnol. 21, 1247–1249 (2002)
2. Roco, M., Bainbridge, W.: Converging technologies for improving human performance:
nanotechnology, biotechnology, information technology and cognitive science. In: Con-
verging Technologies for Improving Human Performance: Nanotechnology, Biotechnology,
Information Technology and Cognitive Science. Kluwer Academic Publishers, Dordrecht
(2003)
3. Staddon, J.: The New Behaviorism, 2nd edn. Psychology Press, Philadelfia (2014)
4. Mace, F.C., Critchfield, T.S.: Translational research in behavior analysis: historic traditions
and imperative for the future. J. Exp. Anal. Behav. 93(3), 293 (2010)
5. Altshuller, G.: Creativity as an Exact Science. Gordon and Breach. Science Publishers Inc.,
London (1984)
6. Zlotin, B., Zusman, A., Hallfell, F.: TRIZ to invent your future utilizing directed evolution
methodology. Procedia Eng. 9, 126–134 (2011)
7. Zlotin, B., Zusman, A.: Patterns of evolution: recent findings on structure and origin. In:
Proceedings of the Altshuller’s TRIZ Institute Conference TRIZCON, Milwaukee, 29 April–
2 May (2006)
8. The Altshuller Institute. https://www.aitriz.org/
9. TRIZ-Boeing. https://triz-journal.com/triz-boeing/
10. Polovinkin, A.I.: Basics of Engineering Creativity. Mashinostroenie, Moscow (1988). (rus)
Human Factor and Cognitive Methods in the Design of Products 293
11. Zaboleeva-Zotova, A.V., Bobkov, A.S., Orlova, Y.A., Rozaliev, V.L., Polovinkin, A.I.:
Automated identification of human emotions based on analysis of body movements. In:
Proceedings of the IADIS International Conferences - Interfaces and Human Computer
Interaction 2013, IHCI 2013 and Game and Entertainment Technologies, pp. 299–304
(2013)
12. Ukustov, S., Fomenkov, S., Polovinkin, A.: Wrong metrics of success for open source
software. In: Proceedings of the IADIS International Conference Web Based Communities
and Social Media 2013, Proceedings of the IADIS International Conference Collaborative
Technologies, pp. 97–99 (2013)
13. Miles, L.D.: Techniques of Value Analysis and Engineering, 3rd edn, p. 366. McGraw-Hill,
New York (1989)
14. National Defense Authorization Act for Fiscal Year 1996, Public Law, NY, pp. 104–106
(1996)
15. Kornai, J.: System paradigm. Vop. Econ. 4, 3–34 (2002). (rus)
16. Kolbachev, E., Sidorova, E.: High organizational and technological level of production
systems as a condition for economic leadership. In: Proceedings of the ICSEAL International
Conference: Social, Economic, and Academic Leadership, Praha (2019)
17. Kleiner, G.: System paradigm and the theory of the enterprise. Vop. Econ. 10, 3–21 (2002).
(rus)
18. Lipov, A.N.: At the origins of modern bionics. Bio-morphological formation in an artificial
environment. Polygnosis. 1–2, 126–136 (2010)
19. Bionics: Nature as a Model. Pro Futura Verlag, München, vol. 223 (1993)
20. Blok, L.G., Longana, M.L., Woods, H.Y.: An investigation into 3D printing of fibre
reinforced thermoplastic composites. Addit. Manufact. 22, 176–186 (2018)
21. Dumitrescu, A.: The concept of technotope. In: Proceedings of 3rd DAAAM Workshop
«Intelligent Manufacturing Systems», Kosice, pp. 21–22 (2001)
22. Kudrin, B.I.: Application of the concepts of biology to describe and predict large systems
emerging technologically, Izdatelstvo Tomskogo universiteta, Tomsk (1976). (rus)
23. Highsmith, J.: Agile Software Development Ecosystems, p. 136. Addison-Wesley Profes-
sional, Cambridge (2002)
Automatic Assessment System of Operators’
Risk in Order Picking Process for Task
Analysis
Yangxu Li(&), Bach Q. Ho, Tatsunori Hara, and Jun Ota
Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo,
Bunkyo-ku, Tokyo 113-8656, Japan
[email protected]
Abstract. It is necessary to consider operators’ health in order picking (OP).
Automatic systems have been developed to offer posture risk feedback to
operators but a lack of activity information limits the application of such systems
in analyzing OP tasks and improving OP systems. Thus, this study develops an
automatic risk evaluation system integrated with activity information in OP for
task analysis. Proposed system is a full-body motion capture system including
three steps. Firstly, joint angles are calculated using orientation of related IMUs
and REBA is implemented to compute posture risk scores. Then, a rule-based
method is proposed to automatically detect picking activities. Finally, task
analysis is performed by determining percentage of time spent at different risk
levels of different activities during task. In a user experiment, questionnaire
feedback from participants agreed with the task analysis results and the results
provided insight on how to improve the OP system.
Keywords: Ergonomics evaluation Activity detection Order picking Task
analysis
1 Introduction
Warehouses, used to store or buffer products, plays an important role of supply chain’s
performance as they determine the ability to deliver products to customers [1]. To
adequately serve customers, order picking (OP) - the process of retrieving products
from storage or buffer areas in response to a specific customer request, is considered to
be of the highest priority for improving productivity [2]. Therefore, a great many
researches focused on OP system including layout design, storage assignment, zoning,
order batching, and routing. In particular, picking activities are very important owing to
significant expenditure of human resources. Furthermore, it is necessary to consider
human factors in the process of human operation because of the relation between work
efficiency and operational safety. However, most of the previous research on OP design
did not consider human factors [1]. The activities of OP are highly labor-intensive,
which means accounting for human characteristics is essential when designing an OP
system. Specifically, physical aspect is one of the most important factors in OP
operations considering the large amount of physical activities involved in OP such as
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 294–300, 2020.
https://doi.org/10.1007/978-3-030-39512-4_47
Automatic Assessment System of Operators’ Risk in OP Process 295
characterized by a high load weight and high repeatability. Continuous task in OP
would cause discomfort and fatigue to operators. It has been argued that warehouse
operators usually suffer from future long-term muscular ailments, such as muscu-
loskeletal disorders (MSDs) [3]. Thus, assessing the risk of OP operation and analyzing
OP tasks are imperative to ensure both productivity improvement and wellness of
operators.
Tools such as REBA, RULA, and OWAS have been developed for ergonomic
assessment and they have been applied in practice to three general categories: self-
report, video observation and direct measurement methods [4]. Self-report and video
observation are prone to providing subjective results because they depend on opinions
and knowledge of operators or reviewers. Direct measurement has been used with a
certain degree of success to quantify exposures in field studies. Vignais et al. developed
a real-time system of ergonomic feedback following the standard of RULA using
inertial measurement unit (IMU) in industrial manufacturing [5]. Battini et al. [3]
introduced a system integrated with multiple ergonomic assessment tools [3]. However,
the lack of activity information including repetitiveness, duration and type of activity
limits the ability of these systems to analyze OP tasks assigned to operators with the
goal of improving OP systems. This is particularly challenging considering the com-
plexity of the activities that need to be detected in the OP process. Therefore, we
propose a rule-based method to detect such activities with the objective of developing
an automatic risk evaluation system integrated with activity information in OP for task
analysis. The main contribution of this system is improving the tool used for collecting
activity information for automatic ergonomics risk analysis of OP task.
2 Proposed System
For the purpose, IMU is used as it reports orientation of body and does not need
complex setup. Specifically, to collect the data, we select a motion capture system with
17 IMUs (f = 50 Hz) provided by ZMP Inc. [6]. In this part, the proposed system
consisting of three steps is presented.
2.1 Posture Risk Assessment
Joint Angle Estimation. The motion capture system outputs the orientation (quater-
nion) of every IMU. Joint angles are then derived from the orientations of related
IMUs. More details for joint angle estimation including alignment process have been
published in a previous study [7]. In our research, however, the horizontal alignment
process of IMUs is replaced by a rotation matrix between the initial orientations of
related IMUs. Furthermore, the study only introduced estimation of the knee angle.
For REBA usage, the trunk angle is computed by orientation of the chest relative to that
of the waist. Upper arm, lower arm and neck angles are calculated from orientations of
these body parts with respect to orientation of the chest. Wrist angle is deduced from
the orientation of wrist relative to the orientation of lower arm.
296 Y. Li et al.
Rapid Entire Body Assessment (REBA). The assessment of risk level of body
posture is following the standard of REBA [8], which is useful for assessment of
posture risk of the whole body based on joint angles. REBA is efficient in identifying
musculoskeletal risks in a variety of tasks, and it was originally used in health care
field. Posture analysis using REBA requires computing the joint angles, and the other
components of this tool refer to parameters such as repetitiveness, duration, and load.
However, in this study, we focus only on posture as it has been noted that the posture is
the primary risk factor affecting operators during manual tasks [5]. The global score of
REBA ranges from 1 to 15, divided into five levels: negligible risk, low risk, medium
risk, high risk, and very high risk.
2.2 Picking Activity Detection
Picking activities are the main activities in OP process, and they report the primary
information of OP task. To distinguish the risk levels of OP operation, picking
activities are divided into activities at high locations, medium-height locations, and low
locations. Unloading packages from carts and putting down packages are also detected
as they involve similar postures. These activities are segmented by processing signals
of joint angles as each activity corresponds to a change in related joint angles. For
example, when operators pick up packages at low locations, they tend to squat,
changing their knee angles. Occurrences of these activities are detected by setting
threshold for local maximums of related joint angles. Table 1 lists related the experi-
mental joint angles and thresholds.
Table 1. Joint angles and thresholds of local maximums used for position detection of picking
activities
Picking activities Joint angles Threshold
Low location (PL) Angle of leg >60°
Unloading/Putting down (PU) Angle of trunk >30°
Medium-height location (PM) Angles of upper arm >30° and <50°
Angle of lower arm >30°
High location (PH) Angle of upper arm >50°
Next, the gradient of related joint angles is employed to segment the duration of
picking activities. Specifically, we use Eqs. 1 and 2 to calculate the gradient. Sliding
window is set as 10 samples. The search for an activity starts from a point of local
maximum Pmax . Then, gradients to both sides are calculated iteratively using i, j.
Finally, the start and end points of the durations are indicated by gradients lower than
0.5.
Gleft ¼ AngleðPmax iÞ Angle(Pmax 10 i) i ¼ 1; 2; 3. . . : ð1Þ
Gright ¼ AngleðPmax þ jÞ Angle(Pmax þ 10 þ j) j ¼ 1; 2; 3. . . : ð2Þ
Automatic Assessment System of Operators’ Risk in OP Process 297
However, random postures of operators during tasks such as raising their arm to
check order list would lead to a detection error. Therefore, we check for sudden
increases in Y-axis acceleration of the IMU attached to the forearm during picking
activities in order to eliminate the interference as sudden acceleration change indicates
a shock to arm [3].
2.3 OP Task Analysis
Once the results of posture risk assessment and activity information are obtained, OP
task analysis is performed by calculating the ratio of time spent on tasks of diverse risk
levels within different picking activities, which is useful information when applied to
warehouse improvement considering the ergonomics risks of operators. Combined with
posture risk assessment, operators will be given feedback to help mitigate posture risks.
3 Experimental Results
3.1 Experimental Set-Up
Experiments were conducted in a lab equipped with six shelves of the same size with
four layers. Packages, which were classified into four categories, were randomly stored
on the shelves. This platform allows to test realistic OP system. 3 male subjects
participated in the experiment. This platform allows testing of realistic OP systems.
Three male subjects participated in the experiments. Each participant performed an OP
task that required picking up 10 packages using a cart. After the participants completed
the task, they were asked to fill out a questionnaire, to obtain subjective comments
regarding this OP system. The participants rated their impressions concerning posture,
work difficulty, and fatigue on a 5-point scale (from “strongly disagree” to “strongly
agree”). Moreover, they also were requested to report problems in the OP system and
provide suggestions for improvement based on their experience.
3.2 Results
REBA risk scores were calculated at every sampling point (for the result of one of the
participants see Fig. 1). Regarding the entire task, the execution times for participant 1
(P1) and participant 2 (P3) were 128.5 s and 121.6 s, respectively. However, partici-
pant 2 (P2) took more time (205 s) even though the task was the same. On average, the
results showed that there was a clear difference between P3 (with a risk score of 5.55)
and the other participants (with risk scores of 4.32 and 4.36).
Combining the results of picking activity detection, we performed OP task analysis,
and a report for P1 is shown in Table 2. The results demonstrated that about 60% of the
time was spent in the medium-risk zone, and activities in this zone were primarily of
PL and PU types. Only about 5% of the time was spent in the high-risk zone. The
report for P2 indicated a similar distribution as for P1, but P2 spent about 50% of the
time on PU and was exposed to a relatively high average risk score of PL (7.28). It
should be noted that the value is high as only the part about posture is considered in this
298 Y. Li et al.
Fig. 1. OP posture in the experiment (left) and REBA risk score at every sampling point during
the entire task (right)
research. Moreover, P3 spent a significant amount of time in the high-risk zone
(25.45%), and this was mainly due to PU (17.28% in high risk zone) and PL (5.31%).
For P3, the average risk scores of PL and PU were 6.84 and 5.78, which are also
relatively high. Another interesting result is that all three participants spent consider-
ably more time on PL than on PH and PM even though the packages in the task were
selected randomly, which indicates that about half of the packages were located on the
two lower layers of the shelves, and half on the two higher ones.
Table 2. Report of OP task analysis (P1)
Time Average risk Time spent at
(s) score High risk Medium Low risk
(%) risk (%) (%)
Total 128.5 4.32 5.19 59.71 35.10
Low location (PL) 25.6 5.30 1.67 16.14 2.11
Unloading/Putting 46.0 4.72 3.37 23.78 8.67
down (PU)
Medium-height 2.0 3.71 0 1.09 0.45
location (PM)
High location (PH) 5.6 4.17 0 3.32 1.06
Other activities 49.6 3.48 0.15 15.39 23.05
4 Discussion
Comparing the results of detecting picking activities with recorded video at every
sampling point, the values of F-measure of PL, PU, PH, and PM were 0.849, 0.718,
0.813, and 0.64 respectively. To some extent, the report of OP task analysis can
indicate the existing problems with assisting warehouse improvement. The comments
from the participants were also consistent with the results as the scores of all three items
(4.33, 3.33 and 5 for average posture, work difficulty and fatigue points) were con-
siderably high.
Automatic Assessment System of Operators’ Risk in OP Process 299
From the perspective of the participants, the execution time was significantly
greater for P2 than for P1 and P3. This is because P2 was not familiar with the work
procedure. Regarding the average posture score, P3 spent more time in the high-risk
zone as he prioritized work efficiency over comfort. In this task, he frequently bent
down to pick up packages frequently instead of first squatting slowly. Only P1
maintained a balance between productivity improvement and ergonomics risk. With
respect to OP task analysis, more than 50% of time was spent in the medium-risk and
high-risk zones. This problem was mainly caused by more time being spent on PL and
PU as a result of random storage and random distribution of order. These findings
indicate that operators tend to perform tasks using awkward postures and consequently
feel more discomfort. fatigued quickly because they are frequently exposed to rela-
tively high-risk zones. To improve the OP system, reducing the amount of time
exposed to PL is imperative, and to achieve this, rearranging the storage is a feasible
option. For example, instead of a random storage, it is better to set gold zones at a
medium-height location to store goods that are picked up more frequently.
Through the collection of activity information, the proposed system, except for
posture risk feedback to operators, contributes to OP task analysis for assisting
warehouse managers in redesigning OP systems.
5 Conclusion
This study developed an automatic assessment system of operators’ risk for OP task
analysis. Compared with previous research [3, 5], a method to automatically detect
picking activities was integrated into the system, allowing for more accurate analysis of
OP tasks to help warehouse managers improve existing OP systems. An experiment
was conducted to test the proposed system. Subjective feedback from participants
agreed with the results, and the results highlighted the problems in the OP system.
In future work, a more robust activity detection method will be developed with the
objective to include other activities in OP, such as pushing and holding. Moreover, it is
also necessary to consider other factors affecting the ergonomics risk, such as load,
which has also been presented in previous research [3].
Acknowledgments. This study is based on results obtained from the Strategic Advancement of
Multi-Purpose Ultra-Human Robot and Artificial Intelligence Technologies (SamuRAI) project
commissioned by the New Energy and Industrial Technology Development (NEDO).
References
1. Grosse, E.H., Glock, C.H., Neumann, W.P.: Human factors in order picking: a content
analysis of the literature. Int. J. Prod. Res. 55, 1260–1276 (2017)
2. De Koster, R., Le-Duc, T., Roodbergen, K.J.: Design and control of warehouse order picking:
a literature review. Eur. J. Oper. Res. 182, 481–501 (2007)
3. Battini, D., Persona, A., Sgarbossa, F.: Innovative real-time system to integrate ergonomic
evaluations into warehouse design and management. Comput. Ind. Eng. 77, 1–10 (2014)
300 Y. Li et al.
4. Spielholz, P., Silverstein, B., Morgan, M., Checkoway, H., Kaufman, J.: Comparison of self-
report, video observation and direct measurement methods for upper extremity muscu-
loskeletal disorder physical risk factors. Ergonomics 44, 588–613 (2001)
5. Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., Marin, F.: Innovative system for
real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44, 566–574 (2013)
6. Full Body Motion Capture ZMP. https://www.zmp.co.jp/en/products/imu-z_bodymotion
7. Jakob, C., Kugler, P., Hebenstreit, F., Reinfelder, S.J., Jensen, U., Schuldhaus, D., Eskofier,
B.: Estimation of the knee flexion-extension angle during dynamic sport motions using body-
worn inertial sensors. In: 8th International Conference on Body Area Networks, pp. 289–295
(2013)
8. Hignett, S., McAtamney, L.: Rapid entire body assessment (REBA). Appl. Ergon. 31, 201–
205 (2000)
TARS Mobile App with Deep Fingertip
Detector for the Visually Impaired
Tetsushi Miwa(&), Youichi Hosokawa, Yoshihiro Hashimoto,
and Giuseppe Lisi
Nagoya Institute of Technology, Nagoya, Japan
{t.miwa.528,y.hosokawa.456}@nitech.jp,
{hashimoto,lisi.giuseppe}@nitech.ac.jp
Abstract. Tactile graphics with an Audio Response System (TARS) allows
visually impaired people to learn maps, diagrams, and figures by synthesizing
audio descriptions of tactile graphics. The current TARS is cumbersome since it
relies on a computer and touch panel to detect fingertip coordinates. Here, we
improve portability by implementing a TARS mobile app endowed with a
camera-based deep fingertip detector. We implemented the hand keypoint
detector proposed in Simon et al. 2017, and restricted our attention on the
fingertips. Performance benchmarking was done over different tactile graphics
types, participants and experimental conditions (i.e. dark/light environment,
partial occlusion). In normal conditions, accuracy is 98% on average, but in the
case of a partially occluded hand, the detection accuracy of the thumb decreases.
These results indicate that the proposed mobile TARS app would perform
similar to the standard TARS while being more portable and intuitive.
Keywords: Tactile graphics with an Audio Response System TARS Deep
learning Fingertip detector Visual impairment
1 Introduction
Visually impaired people learn maps, diagrams and figures through tactile graphics:
images with raised surfaces so that a visually impaired person can feel them (Fig. 1).
The standard learning method combines tactile graphics with a braille descriptor of
each part of an image. This approach is time consuming, since the user has to touch
each part of the image and the corresponding braille descriptor sequentially. An
alternative method is to have a human instructor explain by voice each part of the
image, making the user non-independent.
Tactile graphics with an Audio Response System (TARS) [1] overcomes these
limitations, by providing audio descriptions synthesized by a software on a personal
computer (Fig. 2). Specifically, a tactile graphic is placed on the touch panel, and when
the learner double taps a point, the finger coordinates are read and used to retrieve a
description that is read out by a text to speech software. As a result, smooth tactile
learning is possible without external assistance.
However, the current TARS implementation has two main limitations. Firstly, the
system relies on a personal computer and a large touch panel, that greatly limit
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 301–306, 2020.
https://doi.org/10.1007/978-3-030-39512-4_48
302 T. Miwa et al.
Fig. 1. Tactile graphic of a map Fig. 2. Standard TARS
portability. Moreover, setting up the TARS software on the PC requires the expertise of
a trained operator. In this work, we propose to use a mobile device endowed with a
camera and a deep neural network, in order to localize the user’s fingers on the tactile
graphics (Fig. 3). Such setup improves portability and simplifies the graphical user
interface for a novice user.
Fig. 3. Proposed Mobile TARS. A deep fingertip detector finds the hand keypoints based on the
camera image and provides an audio description accordingly
2 Methods
2.1 Deep Fingertip Detector
The deep fingertip detector is based on the output of the hand keypoint detection deep
neural network proposed in [2], from which we extract the coordinates of the fingertips.
Specifically, the neural network is trained on a multi-camera dataset that provides
robustness to occlusion. We benchmark the performance of the deep fingertip detector
in the tactile graphics context, using a Python script running on a Windows PC (Core
i7). The Caffe pretrained weights of the hand keypoint detection neural network are
publicly available in Openpose [2] and are imported using Open CV.
TARS Mobile App with Deep Fingertip Detector 303
2.2 Benchmarking Experiments
We collected images for benchmarking the deep fingertip detector, by taking photos
with an iPhone 8. In the experiment, for each of 5 healthy participants, 8 different types
of tactile graphic (Table 1) were tested under three different conditions: normal light
(i.e. normal), dim light (i.e. dark) and normal light with a partially occluded hand (i.e.
half-out), for a total of 120 images.
Table 1. Tactile graphics used in the experiment
Map Subway Clock Melody
Animal Hand
Shape Chest
With respect to performance evaluation, the output of the fingertip detector is
visually inspected to check whether the detected fingertip coordinates correctly match
the ground truth. For every sample where there is a successful match, we label it with a 1,
otherwise with a 0.
In order to estimate the effect of different factors on success, we run an analysis of
variance (ANOVA), using the following regression formula:
logitðsuccessÞ = condition + finger + condition:finger + participant + tactile graphic ð1Þ
where the levels of the categorical variable condition are ‘normal’, ‘dark’, ‘half-out’,
those of finger are each of the five fingers, those of participant are each of the five
participants, and those of tactile_graphics are each of the 8 types of tactile graphic. The
factor condition:finger represents the interaction between condition and finger. Sub-
sequently, we compute the effect size omega-squared of each factor.
To easily visualize the ANOVA results, we compute the average success rate (i.e.
accuracy) and create a different table, for each factor.
2.3 Mobile App Development
The TARS mobile app is in continuous development under the Apple iOS operative
system, using the Swift programming language. We implement the neural network in
Python and deploy it using the cloud application platform Heroku, a platform as a service
(PaaS) that listens and responds to API requests from the iOS app client. This allows for
easy development and deployment of the computationally demanding aspects of the app
on the cloud side; while keeping the client on the iOS as lightweight as possible.
304 T. Miwa et al.
3 Results
The results of fingertip detection, grouped by experimental conditions, tactile graphic
types and finger names, are shown in Tables 3, 4 and 5, respectively. Table 2 contains
examples of fingertip detection outputs under different conditions.
The omega-squared values obtained from the ANOVA analysis are the following
for each factor: condition 0.225 (large effect), tactile_graphic 0.004 (small effect),
finger 0.144 (large effect), person 0.013 (small effect), condition:finger 0.068 (medium
effect). In order to assign an intuitive same to each effect size we used the thresholds
proposed in [3].
We assert these results by inspecting Tables 3, 4 and 5. We confirm the large effect
of condition by noticing that the performance in the normal light condition is 98%,
while dark and half-out conditions drop their accuracy by 9% and 44%. In Table 4 we
do not observe large differences across tactile graphics types, which confirms the small
effect of tactile_graphic. In Table 5, Thumb has a poor accuracy, compared to the other
fingers, confirming a large effect of the factor finger. Moreover, the medium effect of
the interaction between condition and finger and post-hoc analysis based on Cohen’s d
effect size, indicates that the thumb is hard to detect in the half-out condition. We do
not find differences across participants.
With the Python and Open CV implementation, the detection of one image takes
about 2 s to run on the Windows machine without a graphical processing unit (GPU).
Table 2. Examples of fingertip detection in high and low accuracy pairs, for each experimental
condition (i.e. normal, dark, half-out) and 3 different tactile graphic types.
Tactile
Hand Subway Chest
graphics
Normal
Dark
Half-out
TARS Mobile App with Deep Fingertip Detector 305
Table 3. Conditions.
Subject S1 S2 S3 S4 S5 Average
Normal (%) 97.5 97.5 100.0 97.5 97.5 98.0
Dark (%) 100.0 87.5 80.0 82.5 97.5 89.5
Half-out (%) 55.0 32.5 80.0 52.5 52.5 54.5
Table 4. Tactile graphics.
Subject S1 S2 S3 S4 S5 Average
Map (%) 86.7 66.7 93.3 80.0 66.7 78.7
Subway (%) 93.3 53.3 86.7 60.0 86.7 76.0
Clock (%) 93.3 80.0 86.7 73.3 93.3 85.3
Melody (%) 93.3 66.7 80.0 93.3 86.7 84.0
Animal (%) 93.3 66.7 86.7 66.7 66.7 76.0
Shape (%) 66.7 66.7 86.7 73.3 93.3 77.3
Chest (%) 86.7 93.3 86.7 80.0 86.7 86.7
Hand (%) 60.0 86.7 86.7 93.3 80.0 81.3
Table 5. Fingers.
Subject S1 S2 S3 S4 S5 Average
Thumb (%) 64.0 44.0 36.0 44.0 60.0 49.6
Index finger (%) 76.0 72.0 100.0 96.0 84.0 85.6
Middle finger (%) 92.0 80.0 100.0 88.0 92.0 90.4
Ring finger (%) 92.0 84.0 100.0 92.0 88.0 91.2
Pinky finger (%) 92.0 68.0 96.0 64.0 72.0 78.4
4 Discussion
In this study, we propose to implement a deep fingertip detector on a mobile phone, in
order to improve the portability of TARS. Specifically, we choose the Openpose hand
keypoint detection neural network [2] and restrict its output only to the fingertips.
Before implementing it on the iOS, we benchmarked the performance of the fingertip
detector on a dataset collected from several participants under several conditions.
The results show that in a normal lighting condition the detection accuracy is 98%,
and this value decreases by 9% and 44% in dim light and partial occlusion conditions,
respectively. We observe that the big drop in accuracy associated with the partial
occlusion condition is mostly due to the mis-detection of the thumb finger. This may be
due to the fact, that in this condition, a great portion of the thumb is out of the picture.
Another reason would be that the nail of the thumb is facing outwards when the hand is
placed on the tactile graphics. Indeed, we observe qualitatively that the more accurate
the nails are on the photo, the better the detection accuracy is. Therefore, the general
306 T. Miwa et al.
lesson that we learn from this experiment is that lighting may not affect the perfor-
mance due to the automatic brightness adjustment on the smartphone; however, the
user should try to place all the fingers within the frame and keep the nails as parallel as
possible to the paper. Considering the current implementation of TARS, visually
impaired people use only one finger, often the index finger, to tap on the touch panel,
and multi-touch is currently not implemented. Therefore, the current performance of
the deep fingertip detector is high enough to implement the first prototype of the
Mobile TARS App.
In the future, we will implement coordinate registration between the detected fin-
gertips coordinates and the tactile graphics location, in order for the system to syn-
thesize a description. Moreover, we will have to reduce the time of execution, by
running the deep neural network on a graphical processing unit (GPU) in the cloud and
requesting real-time predictions via the Mobile App.
Another important aspect of future development will have to consider how to
implement the selection action, which is currently done by double tapping on the touch
panel. This could be done by staying on a fixed position for more than a predefined
time.
Moreover, the app should be easily operated by visually impaired people, without
the help of a human instructor. Therefore, all the functionalities should be implemented
using assistive technology, such as screen readers that translate the screen and graphical
user interface (GUI) contents into synthetic speech.
All things considered, the current study suggests that the proposed mobile TARS
app would perform similar to the standard TARS, while being more portable and
intuitive.
References
1. Morikawa, K., Hosokawa, Y., Koshijima, I.: Development of the systems to learn image
information for visually handicapped people and actions to the spread of the systems (2015)
2. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using
multiview bootstrapping. In: Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp. 1145–1153 (2017)
3. Field, A.: Discovering Statistics Using IBM SPSS Statistics, 4th edn. Sage Publications,
London (2013)
Analysis Process of Exploratory Research
Represented in a Coordinate System XYZ
Olga Popova(&), Boris Popov, Vladimir Karandey,
and Vladimir Afanasyev
Kuban State Technological University, Krasnodar, Russian Federation
[email protected], [email protected], [email protected],
[email protected]
Abstract. To analyze the search research process presented in the xyz coor-
dinate system, a mathematical model is used to represent the stages of search
research. The model was derived using the in first proposed theoretical concepts
and practical approaches. For this, the probabilistic processes described earlier
were used. Previously, their images were analyzed on a two-dimensional plane.
For this, the dependences of the final probabilities of the constituent stages of
the search research on the quantity of information obtained to complete this
stage were constructed. Now could analyzing the planes that we were con-
structed using more complex formulas to construct. The dependences of the
probabilities on the intensity of the flow and the time taken to complete the
stages of exploratory research were received. In the mathematical model of
exploratory research, the variable - useful knowledge for solving the next stage
obtained during the execution of the current one was introduced.
Keywords: Exploratory research Intensity flow Probability of completion
Useful knowledge to solve
1 Introduction
In order to implement a modern and relevant scientific idea or to form a correct and
timely management decision at the level of regions and large industrial holdings, it is
necessary to reduce the time spent on exploratory research. To achieve this goal, it is
necessary to solve the problems in the field of computer science and cybernetics, which
are interdependent. It is necessary to improve the control systems for information
processing processes that are considered over time. These cybernetic problems are
solved at all stages of exploratory research. Each stage has its own problems of
information processing, the solution of which gives new knowledge, where the
occurrence of processes in time is not predominant. Previously, exploratory research, as
a system, has not been investigated. Basically, the stages of exploratory research have
been improved and optimized. The authors of the article previously established [1–3]
that the cognitive process S of the exploratory research consists of five stages: 1 -
literature and patent review; 2 - study of mathematical models; 3 - study of solution
methods; 4 - comparison of solution methods; 5 - choice of the solution method. They
are random probabilistic processes, where the method of compiling and solving Kol-
mogorov equations is applicable. It should be noted that the authors were the first to
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 307–312, 2020.
https://doi.org/10.1007/978-3-030-39512-4_49
308 O. Popova et al.
formulate the task and solve it using this method. This method was also applied to the
solution of other urgent probabilistic tasks [4, 5].
2 Theory
Exploratory research deals with the flow of scientific information, which is appropri-
ately processed at each stage. Therefore, the intensity of the information flow of
scientific information k may be considered a quality indicator of the i-th stage effec-
tiveness of the exploratory research, where i = 1, …, 5 [6]. It equals to the amount of
useful knowledge for solving the next stage and obtained during the execution of the
current one. The flux intensity k is a mathematical expectation M(x) of the number of
events per unit of time [7] and reflects the intensity of the scientist’s work. Scientific
research is characterized by non-standard flows, which are characteristic of the
unsteady process. Their intensity k changes over a certain period of time:
k ¼ f ðtÞ 6¼ const; ð1Þ
where f ðtÞ is the curve on the three-dimensional plane, in the xyz coordinate system
(Fig. 1), where the x-axis is the flow of intensity k; y is the probability of the system
state, where i = 1, …, 5; z is time t. When taking into account the projection of the
plane Pi onto the xy coordinate system, it is necessary to consider only the final
probabilities Pi ðkÞ (Fig. 2), which are obtained by the authors through the solution of
the Kolmogorov system of equations [1–3].
Fig. 1. Passing of the point through P1 plane in xyz-space
Analysis Process of Exploratory Research 309
Fig. 2. Passing of the point through the graphs of the dependencies P1(k), P2(k), P3(k), P4(k) и
P5(k) on the xy plane.
The flux density can be expressed as follows [14]:
M ðt þ xÞ M ð xÞ
kðtÞ ¼ lim : ð2Þ
x!0 x
For the simplest flow, Eq. (2) can be written as follows:
M ð xÞ
k¼ : ð3Þ
x
There is a discrete uniform distribution, therefore:
1X n
M ½ x ¼ xi ; ð4Þ
n i¼1
where xi is the value from the interval ½a; b (Fig. 3).
Fig. 3. The expected value for the simplest stream of events.
310 O. Popova et al.
Let the result of the i-th event be a certain amount of knowledge DЗi. Then the
result of the j-th event is the determination of the plane Dhi, where there is no
knowledge necessary to solve the task (Fig. 3). Using the Eq. (4), we draw the analogy
for the simplest flow of events in the exploratory study, where the value from the
interval ½a; b is the amount of knowledge DЗi or the amount of ignorance Dhi, iden-
tified by the event:
1 n
M [з ] = ∑ зi = ΔЗср ; ð5Þ
n i =1
1X n
M ½ h ¼ hi ¼ DHcp : ð6Þ
n i¼1
If we equate the maximum amount of useful knowledge that can be obtained in the
time interval Dt to the unit, then we can write down DHcp = 1 – Зcp. To comply with
the dimensions in the calculation and depiction of dependencies in the xyz coordinate
system, we need to consider H and 3 in shares. Substituting the Eqs. (5) and (6) into
(3), we obtain the intensity of the event flow, given the equality x = Dt:
M (З ) ΔЗср
λ= = ; ð7Þ
Δt Δt
M (H ) ΔH ср 1 − Зср
μ= = = . ð8Þ
Δt Δt Δt
Final probabilities, as functions of DЗ, are obtained by substituting the Eq. (7) into
the equations of final probabilities [1–3]:
P1 (t ; ΔЗ ) =
(t − ΔЗ ) ; ð9Þ
(t + 3 ⋅ ΔЗ )
5 ⋅ Δ З (t − ΔЗ )
P2 (t ; ΔЗ ) = ; ð10Þ
(t + 3 ⋅ Δ З )(2 ⋅ t + ΔЗ )
5 ⋅ ΔЗ (t − ΔЗ )
P3 (t ; Δ З ) = ; ð11Þ
( 2 ⋅ t + ΔЗ )(3 ⋅ t − ΔЗ )
Analysis Process of Exploratory Research 311
5 ⋅ ΔЗ (t − ΔЗ )
P4 (t ; ΔЗ ) = ; ð12Þ
( 4 ⋅ t − 3⋅ Δ З )(3 ⋅ t − ΔЗ )
ΔЗ
P5 (t ; ΔЗ ) = . ð13Þ
(4 ⋅ t − 3 ⋅ ΔЗ )
Let us consider Fig. 1. It depicts the plane P1 that corresponds to the probability of
the exploratory process presence at the first stage. It is a literature and patent review,
which was randomly chosen to visualize the example and explain the theory of this
article. Similarly, we can take other planes for consideration: P2, P3, P4 and P5, which
can be obtained in the same way from the projections onto the xy plane of the final
probabilities: P2(k), P3(k), P4(k) and P5(k). Such an approach is possible if we con-
sider the plane near the final probabilities. The considered time period is close to the
moment of exploratory research completion and is incommensurably short in com-
parison to all the time that was spent on the exploratory research. To conduct more
accurate research and analysis of all stages of exploratory research, it is necessary to
use the planes that can be obtained from the Eqs. (9)–(13) by stepwise substituting of
the values t and DЗ. Thus, more accurate images of the planes P1, P2, P3, P4 and P5 will
be obtained. It should be noted that DЗ will correspond to k, since they are displayed in
the same dimension and related by formula (7). For the plane P1, the final probability
has the form of the function P1(k). If no events with a positive outcome occur over
time, then the course of the exploratory research process can be represented by a
straight line C, which is located on the plane P1. Or write it down C 2 P1 . After the
occurrence of the events with a positive outcome, the motion curve of the exploratory
research process will be determined by the dependence f ðtÞ 6¼ const which will also be
located on the plane P1 or f ðtÞ 2 P1 . The function P1(k) can be considered the pro-
jection of the function f ðtÞ onto the xy plane.
3 Conclusions
The data obtained is necessary for conducting the initial rough analysis of the systems,
managing the information processing that are considered in time and reveal at each
stage of the exploratory research:
– a mathematical model of the exploratory research process represented in the xyz
coordinate system;
– a graphical representation of the point passing through the plane P1 in the xyz-space
next to the final probability P1(k) and the results of its analysis.
The outcomes of the study will help reducing the time spent on exploratory
research. To do this, the theoretical part provided in the article, reflects the specifics of
exploratory research, dealing with the flow of scientific information.
312 O. Popova et al.
Acknowledgments. The reported study was funded by RFBR [Project title: The development of
the theory of quality assessment of the information, taking into account its structural component,
№ 19-47-230004, from 19.04.2019]. All the work on compiling the paper and obtaining cal-
culated and experimental data was evenly distributed among its authors.
References
1. Popova, O., Popov, B., Karandey, V., Gerashchenko, A.: Entropy and algorithm of obtaining
decision trees in a way approximated to the natural intelligence. Int. J. Cogn. Inform. Nat.
Intell. 13(3), 50–66 (2019)
2. Popova, O., Shevtsov, Y., Popov, B., Karandey, V., Klyuchko, V.: The development of the
theory of quality assessment of the information, taking into account its structural component.
In: Ahram, T. (ed.) Advances in Intelligent Systems and Computing, vol. 965, pp. 179–191.
Springer, Cham (2020)
3. Popova, O., Popov, B., Karandey, V., Evseeva, M.: Intelligence amplification via language of
choice description as a mathematical object (binary tree of question-answer system). Procedia
Soc. Behav. Sci. 214, 897–905 (2015)
4. Yeoman, P., Carvalho, L.: Moving between material and conceptual structure: developing a
card-based method to support design for learning. Des. Stud. 64, 64–89 (2019)
5. Abdulla, U.G.: Kolmogorov problem for the heat equation and its probabilistic counter-
part. Nonlinear Anal. Theory Methods Appl. 63(5–7), 712–724 (2005)
6. Wang, J., Peeta, S., Lu, L., Li, T.: Multiclass information flow propagation control under
vehicle-to-vehicle communication environments. Transp. Res. Part B: Methodol. 129, 96–121
(2019)
7. Poznyak, A.S.: Mathematical expectation. In: Advanced Mathematical Tools for Automatic
Control Engineers: Stochastic Techniques, chap. 3, pp. 47–62 (2009)
A Variety of Visual-Speech Matching ERP
Studies in Quiet-Noise Scenarios
Lingling Hu, Chengqi Xue(&), and Junkai Shao
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], [email protected]
Abstract. In this study, the visual user interface elements and the auditory user
interface information elements are analyzed. From the information elements of
the visual interface and the speech elements of the auditory interface as the entry
point, it is hoped to construct a more efficient audio-visual human-computer
interaction interface. Taking the alarm information as an example, this paper
studied the cognitive rules of the user on the audiovisual interface in the quiet-
noise environment through the behavioral performance evaluation and the
measurement method of the brainwave physiological index evaluation, and
establishes the mapping relationship between “audiovisual-cognition”.
Keywords: Visual-speech matching The ERP technology Nl component
N2 component
1 Introduction
With the rapid development of computer and communication technology, human
beings have gradually entered the digital age. Human-machine interface (HMI), also
known as user interface (UI), is a typical feature of the digital age. It is an important
basis for users to obtain information and make decisions. It is also a key factor for
accurate and efficient transmission of information. However, in the current traditional
human-computer interaction process, the presentation of information mostly focuses on
the visual channel, and the auditory channel often neglected in the interface design.
However, a single visual channel is often difficult to meet the needs of users. Too much
visual information display can easily lead to “information congestion” in the visual
channel, resulting in operator’s visual fatigue. Therefore, it is necessary to add auditory
display information to the design of human-computer interaction interface.
Since the 20th century, ERP technology has been more and more widely used in the
research of audiovisual cognition, and its prospects are more and more broad. Lebib
et al. [1] revealed very early audiovisual interaction during speech perception through
ERP research [1]. Shams [2] further indicates that activity in the visual cortex can be
regulated by sound [2]. Peng et al. [3] through the analysis of the amplitude and latency
of P300 potential, found that the brain is easier to integrate information under audio-
visual dual-channel stimulation, with visual-based, auditory-assisted synergistic com-
pensation effect [3]. Sinke [4] studied audiovisual processing in synaesthesia,
combined with vision and hearing, using the way of semantic classification task to
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 313–318, 2020.
https://doi.org/10.1007/978-3-030-39512-4_50
314 L. Hu et al.
judge audiovisual consistency and inconsistency. Analysis of ERP data showed that the
amplitude of N1 component increased at the position of occipital electrodes for
synaesthesia [4]. Kagan et al. [5] found that when receiving voice information in a
noisy environment, seeing the pronunciation posture can significantly improve speech
comprehension. It is also suggested that when the visual and auditory incompatibility
occurs, the N400 amplitude will be larger; relatively, if the subjects listen to and see the
same, the N400 amplitude will be relatively flat [5].
According to the research at home and abroad, researchers are very enthusiastic
about the research of audiovisual cognition. However, there are still some problems to be
solved urgently. Visual alarm sometimes fails to fulfill the function of information
transmission well. At this time, more effective sound alarm is needed to perfect the alarm
[6]. However, the mutual interference between different alarms has a great impact on the
performance of the alarm. At present, the research on the interference effect is still blank,
which is the content of this paper. One. In addition, voice alarm in major disasters or
accidents inevitably has noise interference, and the impact of different signal-to-noise
ratios on human alarm perception performance is a topic worthy of study. In this study,
ERP technology was used to evaluate the way of audiovisual presentation by comparing
behavioral data (response time and accuracy) and physiological data (peak and latency of
EEG components). Different combinations of visual and voice alarm information were
studied in quiet-noise environment. Cognitive efficiency and process to get the best way
of audio-visual presentation in alarm information. This paper reveals the user’s cognitive
thinking activities on visual-voice interface, and provides strategies to solve the problem
of audio-visual user interface design.
2 Materials and Methods
2.1 Subjects
Twenty-five employees (20 males and 5 females) of 716 Institute of China Ship-
building Heavy Industry were tested. Age ranged from 22 to 40 years, with an average
age of 31.5 years. The subjects were right-handed, had no history of psychosis or brain
trauma, and had normal vision or corrected visual acuity.
2.2 Stimuli and Task
In this experiment, different audiovisual combinations and different environments were
used as stimulus variables. The icons in visual stimulus are 128*128 (px) in size. In
order to eliminate the interference of unrelated factors, the icons with identical style and
high recognizability are selected; the text stimulus is corresponding to the icon
semantics, and the font is Microsoft Yahei 60 pt; the color stimulus is red for warning,
the white for attention, and the size for icon. The consistency is 128*128 (px) (see
Fig. 2). There are two kinds of auditory stimuli, one is voice stimulus and the other is
earmark stimulus. In this paper, we mainly study the audio-visual combination of
vision and voice. There are three kinds: text + voice, icon + voice, color + voice.
There are two levels of environmental variables: quiet environment (65–70 dB) and
noise environment (85 dB).
A Variety of Visual-Speech Matching ERP Studies 315
Fig. 1. Stimulus information sheet
After wearing the EEG cap, the subjects sat comfortably in a soft-light sound-proof
room. Their eyes were fixed at the center of the screen. The distance between their eyes
and the screen was about 70–80 cm. The horizontal and vertical visual angles of the
pictures in the experiment were controlled within 2.3°. Figure 1 is the flow chart of the
experiment. The experiment is divided into two parts. The first part is the experiment in
quiet environment. The second part is the experiment in noise environment 20 dB
higher than quiet environment. The two parts of the experiment flow are identical with
the number of stimuli. Firstly, in order to familiarize the subjects with the task of the
experiment, it is necessary to practice the experiment before the formal experiment,
which is in accordance with the steps of the formal experiment. Firstly, visual stimuli
(random text, color, icon) and auditory stimuli (random voice, ear tag) were presented
to the subjects at the same time. A total of 72 times were randomly combined. Each
stimulus presented 660 ms. Then they go to the judgment page, and the subjects need
to react to determine whether the information they hear and see is the same information
level (where the information level is divided into alarm level and attention level). When
the information is consistent, press the “J” key on the keyboard, and vice versa, press
the “F” key. After the operation, it enters the visual center to guide the picture, and after
1000 ms it enters the next round of tasks. The experiment lasted 10–15 min and 144
judgements were made.
Fig. 2. Experimental flow chart (left chart is the overall flow chart, right chart is the flow chart
of a single experiment)
316 L. Hu et al.
2.3 Analysis
(1) Behavior data analysis: The response time and accuracy of the experiment were
collected and analyzed by E-prime3.0 software. Behavior data mainly analyzed
reaction time (RT) and accuracy rate (ACC). Tables 1 and 2 use paired sample T
test to analyze the data of the subjects, and the significance level is set to 0.05.
(2) Analysis of EEG data: The experimental data were analyzed by CURRY8 soft-
ware. Firstly, reference electrodes (M1 and M2) were converted. ICA method was
used to remove EEG artifacts, filter, remove artifacts less than 20%, segment EEG
and complete baseline correction and average stacking. After all the EEG waves
are processed, all the EEG waves of the subjects are averaged by superimposing
the same segment of EEG waves, and the final EEG waveform is obtained.
Physiological data were analyzed in two steps. The first step is to select electrodes
that may excite obvious EEG waves. The second step is to remove artifacts and
EEG fragments with amplitude greater than 100 µV from the time window of
200 ms before stimulation to 1000 ms after stimulation. According to different
environment (quiet environment and noise environment), stimulation types
(text + voice, icon + voice, color + voice) are averagely superimposed.
3 Result
Behavior data results are shown in the following list. Table 1 shows the data of the
subjects analyzed by paired sample T test, and the significance level is set to 0.05. Through
Table 1 (ACC denotes information matching in noisy environment, ACC2 denotes
information mismatching in noisy environment, ACC3 denotes information matching in
quiet environment, ACC4 denotes information mismatching in quiet environment, RT
and ACC denote the same method), we can also see that there is no significant difference in
the average accuracy under different conditions. (P > 0.05. Correspondingly, according
to the information in Table 1, it is more obvious that the matching of audiovisual infor-
mation in noisy environment and the matching of audiovisual information in noisy
environment have significant differences (P = 0.000 < 0.05); the matching of audiovi-
sual information in noisy environment and the matching of audiovisual information in
quiet environment have significant differences (R = 0.000 < 0.05). The difference
P = 0.000 < 0.05; the difference in RT (response time) between audiovisual information
mismatch in noisy environment and audiovisual information mismatch in quiet envi-
ronment was significant P = 0.000 < 0.05; the difference in RT (response time) between
audiovisual information match in quiet environment and audiovisual information mis-
match in quiet environment was significant P = 0.000 < 0.05.
Select the data in quiet environment for in-depth analysis, study the accuracy
(ACC) and response time (RT) of the three combinations of text * speech, icon * speech
and color * speech. The data are shown in Table 2. It can be seen from the table that in
quiet environment, no matter the information matched, the combination of text and speech
always gets the best data, ACC = 0.9524 > 0.93 > 0.8623; RT = 883.2797 ms <
997.1238 ms < 1084.391 ms.
A Variety of Visual-Speech Matching ERP Studies 317
Table 1. Matched T test of reaction time RT and accuracy rate ACC for visual-voice matched
samples
Table 2. Correctness and response time of three audiovisual combinations in quiet environment
When further exploring the physiological data of text-speech, we can get the EEG
waveform superposition maps of different electrodes as shown in Fig. 3, and the EEG
waveform superposition maps of text-speech at FZ, CZ and PZ electrodes as shown in
Fig. 3. The three electrodes were located in the prefrontal area and the parieto-occipito-
temporal area of the brain. The following figure shows that there are obvious peaks at
100 ms and 200 ms. The three electrodes have one thing in common: the amplitude of
N100 in quiet environment is higher than that in noise environment, while the
amplitude of P200 is lower than that in noise environment. The EEG amplitude of CZ
and FZ electrodes in the parietal and frontal regions was larger, approaching 8–10 µV,
and that of PZ electrodes in the occipitotemporal joint region was less than 4 µV.
Fig. 3. Text-Speech EEG waveform overlay map with different electrodes (a) CZ, (b) FZ, (c) PZ
318 L. Hu et al.
4 Discussion
This paper discusses the cognitive efficiency and process of different combinations of
visual alarm information and voice alarm information in quiet-noise environment. The
purpose is to reveal the best way of audio-visual presentation in alarm information, and
to reveal the user’s cognitive activities of visual-voice interface, and to provide
strategies to solve the design of audio-visual user interface. Through the experimental
results, it can be concluded that both in quiet and noisy environments, voice * text is
the best combination, which may be because voice * text is the closest way to natural
interactive audio-visual combination. In addition, ERP data prove that the latency of
N1 and P2 is slightly delayed under noise, which indicates that more attention
resources are needed to complete audiovisual tasks. This conclusion further proves that
P200 is an early indicator of voice activation.
Acknowledgments. The authors would like to gratefully acknowledge the reviewers’ com-
ments. This work was supported jointly by National Natural Science Foundation of China
(No. 71871056, 71471037), Equipment Pre research & Ministry of education of China Joint
fund.
References
1. Lebib, R., Papo, D., Bode, S.D., et al.: Evidence of a visual-to-auditory cross-modal sensory
gating phenomenon as reflected by the human P50 event-related brain potential modulation.
Neurosci. Lett. 341(3), 185–188 (2003)
2. Shams, L., Kamitani, Y., Thompson, S., et al.: Sound alters visual evoked potentials in
humans. Neuroreport 12(17), 3849 (2005)
3. Qiao, X., Peng, J.: Audiovisual EEG feature extraction and brain cognitive mechanism
analysis. J. Test Meas. Technol. 27(6), 509–515 (2013)
4. Sinke, C., Neufeld, J., Wiswede, D., et al.: N1 enhancement in synesthesia during visual and
audio–visual perception in semantic cross-modal conflict situations: an ERP study. Front.
Hum. Neurosci. 8(2), 21 (2014)
5. Kaganovich, N., Schumaker, J., Rowland, C.: Matching heard and seen speech: an ERP study
of audiovisual word recognition. Brain Lang. 157–158, 14–24 (2016)
6. Metzger, U., Parasuraman, R.: Effects of automated conflict cuing and traffic density on air
traffic controller performance and visual attention in a datalink environment. Int. J. Aviat.
Psychol. 16(4), 343–362 (2006)
Research on Color Stratification in Dynamic
Environment: Frequency Domain Analysis
of Delta, SMR and Theta EEG Rhythms
Cheng Guan, Lei Zhou(&), Tongtong Zhang, and Xiang Zeng
School of Mechanical Engineering,
Southeast University, Nanjing 211189, China
[email protected],
{zhoulei,zhangtongtong,220190295}@seu.edu.cn
Abstract. The purpose of this paper is to study the color stratification problem
in dynamic environment. The HUD (Head up Display) interface is the research
object. The foreground color of the color adaptive HUD interface will change
with the background color. The increase of foreground color and background
color contrast is beneficial to people. Prospective HUD information captures
attention and improves cognitive performance; however, the continuous change
of HUD color in the foreground increases the amount of information, which
increases the cognitive load and is not conducive to the improvement of cog-
nitive performance. In this paper, we explore whether the HUD color adaptive
method with changing color in the foreground will improve people’s cognitive
performance.
Keywords: EEG Color Delta rhythm SMR rhythm Theta rhythm
Attention capture Cognitive performance
1 Introduction
For decades, scholars around the world have done a lot of work in the study of the brain
mechanism of color perception, and the work on the neural mechanisms involved in
color processing has made significant progress. Wang and others studied the influence
of highway landscape color on the delta wave component of driver’s brain electricity.
The results show that the roadside landscape color mean is negatively correlated with
the driver’s EEG delta wave component, while the delta wave appears in driving
fatigue, i.e. The brighter the landscape color, the more effective it is to avoid driving
fatigue [1]. Liu et al. conducted a color difference evaluation based on brain wave
signals. The experimental results showed that the N1 component in the right side of the
brain and the occipital region showed obvious regularity when the chromatic aberration
was observed. The N1 potential gradually decreased from the front to the back of the
brain. The amplitude gradually increases, and the larger the color difference is, the
larger the potential of N1 is (the smaller the amplitude is); while the P3 component is
more regular in the midline of the brain than other regions, but its regularity is weaker
than that of N1 [2]. Different colors will affect the time and computing power of the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 319–324, 2020.
https://doi.org/10.1007/978-3-030-39512-4_51
320 C. Guan et al.
selection reaction. A lot of color information will cause visual information confusion,
and also cause visual fatigue, distraction, and cognitive performance reduction [3].
Khoroshikh et al. observed that the saturation of color can affect people’s emotional
state by observing the asymmetry of frontal lobe EEG signals [4]; Tcheslavskia et al.
analyzed the brain’s temporal and spatial responses to sensory stimuli. Asymmetry
between hemispheres, which may be evidence of similar processing of color infor-
mation in two cerebral hemispheres [5]. Tripathy et al. analyzed the effect of hue on the
fractal dimension of EEG, and concluded that the green fractal dimension is the
highest, the magenta has the lowest fractal dimension, and the color can influence the
cerebral cortex independently of the color-specific response. Activity [6]. Aprilianty
et al. studied the color choice of underwear for different consumers through EEG.
When the subjects chose color, the brain wave activity in the left and right hemisphere
frontal, temporal and occipital regions was significantly changed [7].
Under a certain background color, the color that is easily captured by humans is
close to the visual perception, and the color is not easily captured by humans. The
visual perception distance is far [8]. At present, the research on color perception
layering is limited to the solid background, that is, the background color is unchanged.
However, there is no research on the changing background color when using HUD.
The color of the HUD interface changes with the background, which increases the
contrast between the foreground HUD interface and the background color, which is
beneficial to the capture of the foreground HUD color and speed up the reaction.
However, the continuous change of the color of the HUD interface increases the color
information, increases the amount of information processed by the brain, increases the
cognitive load, and reduces the reflection speed. Therefore, whether the continuous
change of HUD color is conducive to attention capture needs further discussion. In this
paper, an EEG experiment of HUD interface color capture is designed for this problem.
2 Color Attention Capture Experiment
2.1 Participants and Equipment
The subjects were undergraduate and postgraduate students in 18 universities. They
were not exposed to similar experiments. They were between 22 and 27 years old. They
had no color blindness and color weakness. The corrected visual acuity was above 1.0.
The experimental program runs on a Windows 7 DELL desktop computer with a
resolution of 1920 1080 for a 27-in. LED screen with E-prime 2.0 Professional
installed. The EEG signal acquisition device uses the eegoTM mylab full mobile EEG
recording and analysis system developed by ANT Neuro. The system includes eegoTM
brain electrical amplifier, waveguardTM 32 channel Ag/AgCl brain cap, and high
performance laptop with ANT/eego software. The position of the brain electric cap
electrode is in accordance with the international 10–20 system standard.
In this experiment, a total of 14 channels of signals were acquired, namely M1, M2,
Fp1, Fpz, Fp2, F3, Fz, F4, C3, Cz, C4, P3, Pz, P4. Among them, the bilateral mastoids
M1 and M2 are referenced.
Research on Color Stratification in Dynamic Environment 321
2.2 Materials for Color Attention Capture Experiment
The experimental stimulus is a video material. Each video is 30 s long and consists of
two repeated video frames with a duration of 15 s, for a total of 32 segments. Among
them, each video is composed of a background image and a foreground helmet display
(HUD) interface (Fig. 1).
Fig. 1. One of the experimental materials
The experiment has three variables, A: the way the foreground HUD color changes,
that is, the adaptive way of HUD color, B: the density of the background hue, C: the
difficulty of the task. Among them, the adaptive HUD color adaptive mode has four
levels, namely A1: complementary color adaptive, A2: contrast color adaptive, A3:
suitable for HUD display interface adaptation, A4: Conventional green adaptive
(previous scenery has been maintained Green does not change), and is the green hue
commonly used in aircraft HUD. The background hue has three levels of change
density: B1: 2 frames/second, B2: 4 frames/second, B3: 8 frames/second, and B4: 16
frames/second. There are two levels of task difficulty: C1: Simple Task, C2: Complex
Task.
Adaptive Mode. The first type is complementary color adaptive, that is, the fore-
ground color and the background color are complement colors, and the two colors in
the hue circle are 180° complementary to each other. Therefore, after determining the
background hue, the difference between the background color and the background
color is taken. The 180° hue is the hue of the foreground HUD instrument panel, with
brightness and saturation still at a maximum of 100%.
The second type is contrast color adaptive, that is, the foreground color and the
background color are contrasting colors, that is, after the background hue is deter-
mined, the color with the difference of 120° in the hue circle is selected as the hue of
the foreground HUD, and the brightness and saturation are still the maximum value of
100%.
The third is an adaptive method for the HUD display interface. According to the
theory of the main color selection of the display interface, the colors on both sides of
the visible spectrum should be avoided. The interface of these colors will give the user
a feeling of negative, fatigue and the like. On the other hand, according to the relevant
standards in the avionics system, the three colors of red, orange and yellow are often
322 C. Guan et al.
used as the display of warning information and error information, and based on the
relationship between hue and cognition, these three colors are not Suitable for long-
term gaze of the human eye, so it is also not suitable for use as the main color of the
HUD interface, so we can choose the green, blue and intermediate colors with
wavelengths between 555 nm and 460 nm, the corresponding hue range is Between
120° and 240°, in addition, considering that the aircraft HUD system is in a changing
background environment, in order to reduce the cognitive load of the pilot, the HUD
interface primary color adaptive system will adopt several fixed hue values. In the hue
value range of 120°–240°, in 30° units, finally selected five hue as the main hue of the
HUD interface, respectively 120°, 150°, 180°, 210°, 240°. In summary, the matching
relationship between the main color of the HUD interface and the background main
color is as follows: when the background hue is between 60°–90° and 150°–180°, the
main hue of the HUD interface is 120°; when the background hue is at 90°–120°,
180°–210°, the main hue of the HUD interface is 150°; when the background hue value
is between 120°–150°, 210°–240°, the main hue of the HUD interface is 180°; When
the background hue is between 150°–180° and 240°–270°, the main hue of the HUD
interface is 210o; when the background hue is between 180°–210° and 270°–300°, the
main hue of the HUD interface Use 240°. In addition, when the background hue value
is between 0°–60° and 300°–360°, the five foreground main hue selected in this paper
are not in the range, and when the hue difference between the two is greater than 30
After °, the user does not perceive the perception, so the main hue with the smallest
difference in hue is selected to reduce the user’s irritating feeling. Finally, when the
background hue value is between 0° and 60°, the main hue of the HUD interface is
120°; When the hue value is between 300° and 360°, the main hue of the HUD
interface is 240°.
Hue Density. Hue change density refers to the number of color changes in one second.
For example, 2 frames/second means that there are two color changes per second. In
15 s, the background hue increases linearly from 0° to 360° of the hue circle. Taking
the variation density of 4 frames/second as an example, 60 colors are changed within
15 s, 360° is divided by 60 equal to 6°, that is, a hue value is taken every 6° in the hue
circle, and the brightness and saturation are set to a maximum value of 100%.
2 frames/sec, 8 frames/sec, 16 frames/sec.
Task Difficulty Level. The mission design is based on the take-off and cruise phase of
the large passenger aircraft A230. According to the HUD dial structure, the two task
levels are determined. The first one is a simple level: observe the HUD airspeed, and
make a button reaction when the airspeed reaches 500 km/h; the second is a complex
level: observe the HUD airspeed and oil quantity at the same time. And the current
flight altitude, the three thresholds are 1000 km/h, 3500 L, 11000 m, respectively,
when a certain parameter reaches the critical value, the key is reflected. Both tasks
appear in the middle of the 30 s video, which is the 15 s position.
2.3 Procedures
The experiment was conducted in a light-brained EEG laboratory. The subjects were
kept in a comfortable sitting position and put on earplugs to avoid other noise
Research on Color Stratification in Dynamic Environment 323
interference. During the experiment, the eyes of the subject were at the same level as
the center of the display, and the eyes were 55–65 cm away from the LED display.
Before the start of the experiment, a simple experiment was introduced to the subjects,
and the subjects were asked to memorize the dashboards corresponding to the hollow
speed, oil volume and current flight height of the HUD interface, and the critical values
of these three parameters were memorized. Enter a formal experiment after a simple set
of exercises. There are 32 trials in the experiment. The figure below is a flow chart of
an experimental trial. The first is a ‘+’ in the center of the screen for 1000 ms, then the
task prompt, the participant presses the Enter key to continue, the black empty screen
500 ms after the experimental stimulus, that is, 30 s video material, the subject
according to the previous task prompts to make the button reflect. The video material of
each trial was different, and all 18 subjects had 32 experiments.
3 Results and Analysis
The experiment is divided into three factors, taking into account the differences
between the brain and the brain, expanding the experimental factors to five factors.
There are four levels in the brain area, namely the ventromedial forehead area (Fp1,
Fpz, Fp2), the dorsolateral prefrontal area (F3, Fz, F4), the central area (C3, Cz, C4),
and the top area (P3, Pz, P4).). There are three levels on the brain side, namely the left
side of the brain (Fp1, F3, C3, P3), the midline of the brain (Fpz, Fz, Cz, Pz), and the
right side of the brain (Fp2, F4, C4, P4). Five-factor and multi-level repeated measures
analysis of variance for alpha rhythm, beta rhythm and theta rhythm were performed.
The variance analysis of d rhythm results in the main effect of color adaptive mode
(P = 0.000 << 0.05), and the main effect of task difficulty is significant (P = 0.007 <
0.05). The Delta rhythm is associated with better cognitive performance. The average
value of the d-band power normalization values in the four color adaptive modes is
ranked as: complementary color adaptive > contrast color adaptive > adaptive based on
HUD color > color invariant (green) adaptation. It can be considered that the subjects
have the best cognitive performance in the complementary color adaptive test, followed
by contrast color adaptation and HUD color based adaptation, and the worst cognitive
performance is the regular green test. Good contrast can help participants understand
color and information. Under the simple task, the average value of the d-band power
normalization value is smaller than the average value under the complex task, which
can get better cognitive performance under complex tasks, because the complex task is
simpler, and the subject will have a more nervous mental state, put more attention, so
there will be better cognitive performance.
4 Conclusion
In the EEG interface color perception EEG experiment, under the color adaptive mode
factor, the delta, SMR, theta band power normalization values are complementary color
adaptive > contrast color adaptive > HUD color based adaptive > color not. The
adaptation of the change (green) indicates that the increase in contrast between the
324 C. Guan et al.
foreground color and the background color is conducive to the attention capture of the
foreground information, which is beneficial to the improvement of the cognitive per-
formance of the subject. Under the hue change density factor, the proper color change
density is beneficial to human attention, and the faster rate of change (16
frames/second) and the slow rate of change (2 frames/second) have no moderate rate of
color change (The attention capture effect of 4 frames/second and 8 frames/second is
good. Under the task difficulty factor, there is better cognitive performance under
complex tasks, because complex tasks are simpler, and the subjects will have more
nervous mental state and more attention, so they will have better cognitive perfor-
mance. Under the brain region factor, the power normalization values of delta, SMR,
and theta bands were not significantly different between the four levels under the brain
region factors. Under the brain-side factors, there were significant differences between
the three levels of the left side of the brain and the midline of the brain. There was no
significant difference between the left and right sides of the brain, indicating that the
power of the SMR wave does not have Hemisphere effect.
Acknowledgments. This paper is supported by Science and Technology on Avionics Inte-
gration Laboratory and Aeronautical Science Fund (No. 20165569019) and the National Natural
Science Foundation of China (No. 71871056, No. 71471037).
References
1. Wang, L., Li, S., Gao, Z.: Effects of highway landscape color on driver’s EEG delta wave
components. J. Harbin Inst. Technol. 48(09), 35–40 (2016). (in Chinese)
2. Liu, Y., Cui, G., Jin, J., Xu, T., Jiang, L., et al.: Study on color difference evaluation based on
brain wave signal. J. Wenzhou Univ. (Nat. Sci. Ed.) 40(01), 38–47 (2019). (in Chinese)
3. Hou, Y., Zhang, L., Miao, D.: Study on the influence of color background on physiology and
performance of visual cognitive tasks. Chin. J. Clin. Psychol. (05), 506 (2008). (in Chinese)
4. Khoroshikh, V.V., Ivanova, V.Y., Kulikov, G.A.: The effect of unconscious color hue
saturation on the emotional state of humans. Hum. Physiol. 38(2), 129–136 (2012)
5. Tcheslavski, G.V., Vasefi, M., Gonen, F.F.: Response of a human visual system to continuous
color variation: an EEG-based approach. Biomed. Signal Process. Control 43, 130–137
(2018)
6. Tripathy, J., Fuss, F.K., Kulish, V.V., et al.: Influence of colour hue on fractal EEG
dimensions. In: International Conference on Biomedical & Pharmaceutical Engineering. IEEE
(2006)
7. Aprilianty, F., Purwanegara, M.S., Suprijanto: Effects of colour towards underwear choice
based on electroencephalography (EEG). Australas. Mark. J. (AMJ) 24(4), 331–336 (2016).
S1441358216302191
8. Li, J., Xue, C.: Study on color coding of digital interface based on vision perception layering.
Chin. J. Mech. Eng. 52(24), 201–208 (2016). (in Chinese)
Human AI Symbiosis: The Role of Artificial
Intelligence in Stratifying High-Risk
Outpatient Senior Citizen Fall Events
in a Non-connected Environments
Chandrasekar Vuppalapati1, Anitha Ilapakurti1(&), Sharat Kedari1(&),
Rajasekar Vuppalapati1(&), Jayashankar Vuppalapati2(&),
and Santosh Kedari2(&)
1
Hanumayamma Innovations and Technologies, Inc., Fremont, CA, USA
{cvuppalapati,ailapakurti,
sharath,raja}@hanuinnotech.com
2
Hanumayamma Innovations and Technologies Private Limited,
Hyderabad, India
{jaya.vuppalapati,skedari}@hanuinnotech.com
Abstract. Senior Citizen Falls are debilitating and harmful events. Not only
does it negatively affect morality, psychology, self-esteem but also tends to be
very repetitive and life costly. To prevent future falls, the outpatient senior
citizen needs to be equipped with real-time monitoring sensors such as, a wrist
band or a sensor necklace. Nonetheless, In the world where real-time sensor
monitoring systems are not available due to connectivity limitations and eco-
nomic affordability, the onus of senior citizen fall predicting, and preventing,
needs to be on cognitive systems that are democratized in nature and yield
learning from population health analysis. In this paper, we apply population
collaborative filtering techniques and artificial intelligent models to cohort high
risk senior citizen clusters and alert healthcare professionals and primary care
family members.
Keywords: Senior Citizens Fall event Machine learning K-Means
Kalman Sanjeevani electronic health records Outpatient
1 Introduction
1.1 Fall Incidence in Senior Citizens
Senior Citizen fall is the most debilitating and hurtful event. An elderly falling is a
public health and community problem with adverse physical, psychological, social,
medical, and economic consequences. According to Center for Disease Control and
Prevention (CDC) Home and Recreational Safety guidelines, each year millions
(Fig. 1) of elders - “those 65 and older”- fall1 [2]. These falls are serious and costly.
1
Important Facts about Falls - https://www.cdc.gov/homeandrecreationalsafety/falls/adultfalls.html.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 325–332, 2020.
https://doi.org/10.1007/978-3-030-39512-4_52
326 C. Vuppalapati et al.
The CDC estimates the cost of falls injuries around $31 billion annually. Hospital costs
account for two-thirds of the total [1, 3]. The incidence of falls increases with
advancing age. It is one of the leading causes of death2 in elderly due to its compli-
cations of which 50% is hip fracture [4]. The fall events are repetitive [5].
Fig. 1. Unintentional fall death rates, adults 65+ [2]
It’s evidently clear that asthma is costly and more prevalent diseases across the
nation and has affected millions of Americans. The best antidote for asthma is prevent
triggers of asthma attacks. The triggers could vary from person to person but on a
macro level the air quality & pollution indexes dominate. Common asthma triggers
include3: Environmental Tobacco Smoke, Dust Mites, Outdoor air pollution, Cock-
roach Allergan, Pets, and Mold [3].
1.2 Frequency of Falls
The Falls are repetitive. They occur in outpatient, inpatient, community and senior
living facilities. y. Findings4 show that, among community-dwelling older people over
64 years of age, 28–35% fall each year (Fig. 2). Of those who are 70 years and older,
approximately 32%–42% fall each year. The frequency of falls increases with age and
frailty level. Older people who are living in nursing homes fall more often than those
who are living in the community. Approximately 30–50% of people living in long term
care institutions fall each year, and 40% of them experienced recurrent falls [6].
2
Falls In Older People - https://www.who.int/ageing/projects/SEARO.pdf.
3
You can control your asthma - https://www.cdc.gov/asthma/pdfs/asthma_brochure.pdf.
4
Global Report on Falls Prevention – Epidemiology of falls - https://www.who.int/ageing/projects/1.
Epidemiology%20of%20falls%20in%20older%20age.pdf.
Human AI Symbiosis 327
Fig. 2. Percentage of falls [6]
1.3 Environment - Rural vs. Urban Areas
The senior population occupied a larger proportion5 in the rural than the urban par-
ticipants, and the mean age of the rural group was older than that of the urban group
[7]. Senior citizens with lower socio-economic demographic have more risk in falling
and getting hospitalized. Comparatively, the fall rate increases rural environment is
much higher than in urban settings due to lack of resources (See Fig. 3).
Fig. 3. General characteristics [7]
5
Disparity in the Fear of Falling Between Urban and Rural Residents - https://www.ncbi.nlm.nih.gov/
pmc/articles/PMC3895525/.
328 C. Vuppalapati et al.
1.4 Fall Prevention - Electronic Sensors
As the population ages, the problem related to falls are expected to grow and pose an even
greater challenges [4]. Meeting the challenges require clear understanding of the preva-
lence and nature offalls. Outpatient Data is the most important tool [5] to enable preventive
healthcare [8]. A majority of falls are predictable and therefore preventable [4]. Devel-
opment continuous monitoring system plan an important role in predicting falls [4, 6].
There are sensors6 available to detect fall and notify primary care takers. However,
these sensors require connected network such as home Wi-Fi, GPS, In-house internet or
the Internet Services.
The structure of this paper is presented as: Sect. 2 discusses the basic concepts and
methods about Machine Learning Algorithms. Section 3 presents our Population AI
architecture, and Sect. 4 shows a case study.
2 Understanding Machine Learning Algorithms Fall
Detection
2.1 Abnormal Movement Detection in Constrained Environment
We have two challenges in detecting fall in non-connected environment. One, the
computation of fall detection algorithms are performed on the sensor (edge) and,
second, store fall events on the sensor for future analysis.
In order to detect fall event, we needed to capture drastic or abnormal changes in
sensor values (accelerometer). Using Kalman filter would able to capture the fall
detection [9].
Here is very high level view of Kalman filter for fall detection: In the Kalman filter,
the fundamental concept is the notion of the state (See Fig. 4). By this is meant,
intuitively, some quantitative information (a set of numbers, a function, etc.) that is the
least amount of data one has to know about the past behavior of the system in order to
predict its future behavior. The dynamics are then described in terms of state transi-
tions, i.e., one must specify how one state is transformed into another as time passes.
Fig. 4. Kalman filter
6
‘Aging In Place’ tech helps seniors live in their home longer - https://www.usatoday.com/story/tech/
columnist/saltzman/2017/06/24/aging-place-tech-helps-seniors-live-their-home-longer/103113570/.
Human AI Symbiosis 329
The Kalman filter model assumes the true state at time k is evolved from the state at
(k − 1) according to
Xk ¼ Fk Xk1 þ Bk uk þ wk
where
• Fk is the state transition model which is applied to the previous state xk−1;
• Bk is the control-input model which is applied to the control vector uk;
• wk is the process noise which is assumed to be drawn from a zero mean
Our goal to apply Kalman filter is to capture any abnormal state transitions due to
fall and store the event for future analysis. In fall detection state equation is the regular
normal movement of the User (normal accelerometer X, Y and Z values).
2.2 Clustering
To recommend based on the locations, fall detect clustering is needed (See Fig. 5).
Clustering could be Hierarchical (agglomerative or Divisive). This is achieved through
Similarity or K-Means clustering [10, 11].
Fig. 5. Clustering [6]
In our case we have used K-means cluster to register the fall positions. Let’s say
Senior citizen fall on floor 2 near garden (please see Table 1); the sensor captures the
fall event and whenever the senior citizen near the fall locations, the sensor vibrates to
alert.
K means Cluster Implementation7 [please see Table 2]: Please note – to improve
performance of devices, the K-means cluster computes the proximity (Euclidian dis-
tance [10, 11]) only after significant changes8 occurred to the senior citizen movement.
7
C source code implementing k-means clustering algorithm - http://homepages.cae.wisc.edu/
*brodskye/mr/kmeans/.
8
Apple Location Services – Significant Change Location - https://developer.apple.com/documentation/
corelocation/cllocationmanager/1423531-startmonitoringsignificantlocati.
330 C. Vuppalapati et al.
Table 1. Fall event sensor data
Sensor accelerometer
Fall (#s) X Y Description
1 12.6 133.5 Mom fall on second floor near garden
2 13.9 122 Mom fall on front porch
3 10 22.3 Mom fall in bedroom
4 10 22.3 Mom fall in bed room
Table 2. High risk stratification to falls
K means C Code
void kmeans(
int dim, // dimension of data
double *X, // pointer to data
int n, // number of elements
int k, // number of clusters
double *cluster_centroid, // initial cluster centroids
int *cluster_assignment_final // output
)
{
double *dist = (double *)malloc(sizeof(double) * n * k);
int *cluster_assignment_cur = (int *)malloc(sizeof(int) * n);
int *cluster_assignment_prev = (int *)malloc(sizeof(int) * n);
double *point_move_score = (double *)malloc(sizeof(double) * n * k);
}
3 Architecture
The system architecture consists of four major parts (please see Fig. 6): (1) Sensor
Module (2) Edge Processor, (3) Bluetooth Module, and (4) Battery.
Sensor Module. The Sensor module collects accelerometer (X, Y, and Z) sensor
values. Additional sensors include temperature and humidity for capturing ambient
temperature values.
Human AI Symbiosis 331
Fig. 6. Architecture
Edge Processor. The K-means clustering [10, 11] and Kalman [9] architectures are
deployed on the Edge Processors. For classification of high risk patients in a non-
connected environment, please see Table 3.
Table 3. Senior citizens - high risk stratification to falls.
Stratification rules
• Recent fall encounters
• Capture of fall locations (Accelerometer X, Y, Z)
• Cluster computation based on the movement
• Compute optimization
• Notifications (Shake or)
4 A Case Study
This paper presented a novel approach to detect and predict senior citizen fall events.
The novel idea of the paper is to predict fall events under no connectivity or offline
applications. We staunchly believe that having edge devices that can predict senior
citizen fall for offline environments can greatly reduce fall related deaths and thus
improve overall healthcare delivery.
We strongly believe that Population Healthcare AI will not only reduce the cost
factor for outpatients but also saves the lives.
References
1. Preventice Solutions, About Us. http://www.preventicesolutions.com/about-us.html.
Accessed 20 Feb 2017
2. Davis, J.: Remote patient monitoring market booming amid readmission fines, doctor
shortages, report says, 15 December 2015. http://www.healthcareitnews.com/news/remote-
patient-monitoring-market-booming-amid-readmission-fines-doctor-shortages-report-says
332 C. Vuppalapati et al.
3. Center For Disease Control and Prevention. http://www.cdc.gov/homeandrecreationalsafety/
falls/adultfalls.html. Accessed 29 Oct 2016
4. Krishnaswamy, B., Gnanasabandam: Falls in Older People. Address to WHO/SEARO Staff
on the eve of the World Health Day 2005, WHO/SEARO, New Delhi, 6 April 2005. https://
www.who.int/ageing/projects/SEARO.pdf
5. Vuppalapati, J.S., Kedari, S., Ilapakurti, A., Vuppalapati, C., Vuppalapati, R., Kedari, S.:
Machine learning infused preventive healthcare for high-risk outpatient elderly. In: Arai, K.,
Kapoor, S., Bhatia, R. (eds.) Intelligent Systems and Applications, IntelliSys 2018.
Advances in Intelligent Systems and Computing, vol. 869. Springer, Cham (2019)
6. Yoshida, S.: A Global Report on Falls Prevention Epidemiology of Falls. http://citeseerx.ist.
psu.edu/viewdoc/download?doi=10.1.1.506.2291&rep=rep1&type=pdf. Accessed 10 Oct
2019
7. Cho, H., et al.: Disparity in the fear of falling between urban and rural residents in relation
with socio-economic variables, health issues, and functional independency. Ann. Rehabil.
Med. 37(6), 848–861 (2013). https://doi.org/10.5535/arm.2013.37.6.848
8. Chandrasekar, V., Anitha, I., Santosh, K., Jayashankar, V.: Stratification of, albeit Artificial
Intelligent (AI) Driven, High-Risk Elderly Outpatients for priority house call visits - a
framework to transform healthcare services from reactive to preventive. MATEC Web Conf.
255 (2019). https://doi.org/10.1051/matecconf/201925504002. Article no. 04002
9. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. Am. Soc.
Mech. Eng. D 82, 35–45 (1960)
10. Han, J.: Data Mining: Concepts and Techniques. Originally Published, August 2000. ISBN-
13 978-0123814791
11. Leskovec, M.J., Rajaraman, A., Ullman, D.: Mining of Massive Datasets. ISBN
9781107077232
Intelligence, Technology and Analytics
Distinguishing a Human or Machine
Cyberattacker
Wayne Patterson1(&), Acklyn Murray1, and Lorraine Fleming2
1
Patterson & Associates,
201 Massachusetts Avenue NE, Suite 316, Washington, DC 20002, USA
[email protected], [email protected]
2
Department of Civil Engineering, Howard University,
2400 6th Street NW, Washington, DC 20059, USA
[email protected]
Abstract. In the world of cybersecurity, attacks may come from many sources.
It is a problem faced by all users, because cyberattacks have become ubiquitous.
Although many types of attacks are relatively simple to defend given com-
mercially available detection software, the attacks are becoming more sophis-
ticated. In more recent times, it is possible that a cyber attack might be initiated
by a human or by an automated software attack or bot. One line of defense that
has not been sufficiently explored is the advantage that might come to a defender
if he or she has the ability to detect whether or not the attacker is human or
machine-based. In this paper, we attempt to develop methods to allow a
defender to try to determine the type of human or bot attack.
Keywords: Cybersecurity Cyberattacks Machine translation Human
factors
1 Introduction
We explored an approach in developing the ability of a cyber defender to detect
whether he or she is under attack by another human or via bot. If the attack involves an
onscreen dialogue, which is often the case, it is the intent of this research to see what
can be developed for a strategy that can detect the potential nature of such an attack.
This approach was first explored by Alan Turing. His 1950 article in the Journal
“Mind” [1] provided significant insights into this challenge, and his insights are usually
described as the “Turing Test.” In this paper, we develop a new Turing Test to explore
the potential differences between three types of text utilized in a cyber attack.
The first text is a quotation in its exact form. Examples will be taken from several
sources for quotations from well-known individuals historically [2], and others from
well-known quotations from popular films [3]. The second type of text is a modification
by a human editor of the examples of the first type, changing a word or words in the
first type of example. The third type of text is a machine translation by taking the first
text, translating to a second language (in this case, French, using Google Translate),
and then translating the French text back to English in the same fashion (subsequently
we will call this process “EFE”). Google Translate is chosen in part because it is highly
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 335–340, 2020.
https://doi.org/10.1007/978-3-030-39512-4_53
336 W. Patterson et al.
ranked among machine translation software [4], but also because it is free and widely
available throughout the entire community of Google users.
2 Research Design
A baseline test of 30 elements was administered to a group of 65 potential subjects of
cyberattacks to give an initial analysis of their ability to detect which of the three above
categories is used in each case. From the baseline responses, a training module has
been developed to assist the subjects in improving their performances on a second but
similar test bed of questions. For example, a training item could advise: “Watch for
misplaced modifiers: ‘Keep your close friends’ instead of ‘Keep your friends close’”.
The paper discusses performance of the test subjects, the choices made in the design of
the training items, and the level of improvement from the (pre-) test bed to the (post)
test bed.
Part 1: For the first part of the study, 65 volunteers, from three distinct groups vol-
unteered to take a test which involves classifying 30 separate texts as to which items
could fall into one of three categories. In every case, the test item was developed from
well-known quotes, either as developed from standard sources of quotations (in which
case the source would be generally speaking grammatically) or from popular dialogues
(well known quotations from film scripts, therefore generally more colloquial).
In what follows, the three distinct sets of volunteers will be referred to by this rough
distinction of the source of each subset.
Table 1. Description of the volunteer groups
Designation Description No. of respondents
E Engineering students at Howard University 9
T Employees of a technology-based company 43
R Random selection of well-educated individuals 13
The subjects would classify each item as M (for machine translation), H (for a
human translation), or N (for the normal or original quotation). In this first part, the
respondents were given the test file, with only the instructions as to how to code their
answers.
Part 2: After analysis of the results of the previous part, recognizing the type of
modifications made by machine translation or by human translation, a Training Module
was developed to provide assistance to respondents to this or similar surveys to detect
the types of EFE translation.
This Training Module identified a number of modifications in machine translation,
examples of which are given below.
1. Split words: the English word “cannot” has no counterpart in French, thus the
translation would normally render “can not”.
Distinguishing a Human or Machine Cyberattacker 337
2. Misplaced part of speech: (From “The Godfather”) “Keep your friends close, but
your enemies closer” becomes (going English to French and back to English):
“Keep your close friends, but your enemies closer”.
3. Contracted phrases/words: the common phrase, such as “most of the things” has
no counterpart in French and thus would be translated as “most things”; words such
as “I’m” when translated into French and back to English will normally appear as “I
am”.
4. Hyphenated expressions: phrase in English “eggs-and-ham breakfast” can only be
translated into French as “breakfast with eggs and ham”.
In addition, given the assumption made that the human translation would not find
instances of poor grammar, but rather the choice of that may result in a grammatically
correct sentence, but perhaps with a loss of meaning because of the center and chosen,
say, by a human English writer for whom English is not a first language (Table 2).
Table 2. Examples of replacement by synonyms
Synonym instead of Source word
Despotism Tyranny
Divinity Religion
Sphere World
Flair Gift
Straightforward, as in “Straightforward, my dear Watson” Elementary
Part 3: A second test of the same nature, but with a largely separate set of examples so
that memorizing the results in the first test would not assist substantially in answering
the test questions. However, for the purposes of consistency, a few items from the first
test were repeated in the second test. In this way, it was hoped that the Training Module
would assist the testee in the overall performance on the test bed, and perhaps of greater
interest, to see if the performance increased on the questions repeated from the first test.
Part 4: In addition to consideration of the overall responses, these could be further
stratified into 3 groups (E, T, R as above) with analyses for each group separately.
3 Analysis of Results
What follows is the tabulation of responses to the “Recognizing Machine Translation”
test with n = 65. The data are sorted in descending order of accuracy the choice.
This subset of the test document consisting of approximations to known quotes
maintaining correct grammar but changing 1 or 2 words to synonyms that might be
found in any dictionary. The purpose for this choice was the assumption that an
attacker might be careful in not displaying incorrect grammar or possibly one from an
individual who is not a native speaker of English. A few examples are indicated in
Table 1 below.
338 W. Patterson et al.
The ability of the respondents on these test questions was worse than on any of the
other sets. Only 26.2% of the respondents correctly identified that these were human
translations, and almost 50% assumed these were actual quotations (Table 3).
Table 3. Recognition of 10 examples of human-translated documents (H)
Quotation Correct Chose Chose
(H) M for H N for H
There are no facts, only connotations 33.8% 16.9% 49.2%
Political correctness is despotism with manners 32.8% 25.0% 42.2%
People demand freedom of speech to make up for the 32.3% 27.7% 40.0%
freedom of conviction which they avoid
The fabric that dreams are made of 31.3% 7.8% 60.9%
Life is a feast, and most poor suckers are starving to 29.7% 10.9% 59.4%
death!
I’ll get you, my pretty, and your little cur too! 29.2% 52.3% 18.5%
Sex and divinity are closer to each other than either 27.7% 10.8% 61.5%
might prefer
A lie gets halfway around the sphere before the truth has 23.4% 29.7% 46.9%
a chance to get its pants on
I have always hinged on the kindness of strangers 20.3% 3.1% 76.6%
The only way to get rid of a desire is to yield to it 18.5% 33.8% 47.7%
AVERAGE 26.2% 25.4% 48.5%
The second-best category consisted of the machine translations, where the
respondents were correct 43.2% of the time. This is encouraging, since it indicates a
reasonably good ability to detect grammatical or syntactical anomalies. A few exam-
ples are the dialogue from the film “Network” where “I’m as mad as hell, and I’m not
going to take this anymore!” when translated EFE becomes “I’m crazy like crazy, and
I’m not going to take that anymore!”; from “The Graduate”: “Mrs. Robinson, you’re
trying to seduce me. Aren’t you?” becomes “Ms. Robinson, you are trying to seduce
me. Is not it?”.
Table 4. Recognition of 11 examples of machine-translated documents (M)
Quotation Correct Chose N Chose H
(M) for M for M
Why do not you come to see me one day? 72.3% 6.2% 21.5%
Ms. Robinson, you are trying to seduce me. Is not it? 70.8% 13.8% 15.4%
When a person suffers from delirium, we speak of 70.3% 9.4% 20.3%
madness. when many people are delirious, we talk
about religion
Open the doors of the pod bay, please, HAL 67.7% 9.2% 23.1%
(continued)
Distinguishing a Human or Machine Cyberattacker 339
Table 4. (continued)
Quotation Correct Chose N Chose H
(M) for M for M
The greatest danger for most of us is not reaching our 67.2% 10.9% 21.9%
goal too high or reaching it, but reaching our goal too
low and reaching our goal
I’m crazy like crazy, and I’m not going to take that 60.0% 16.9% 23.1%
anymore!
You do not understand! I could have class. I could be a 47.7% 21.5% 30.8%
competitor. I could have been someone instead of being
an idiot, that’s what I am
Keep your close friends, but your enemies closer 40.0% 32.3% 27.7%
You know how to whistle, do not you, Steve? You have 34.4% 46.9% 18.8%
just gathered your lips and blow
If you build it, it will come 29.2% 36.9% 33.8%
Of all the gins joined of all the cities of the world, it 14.1% 51.6% 34.4%
enters mine
AVERAGE 43.2% 28.9% 28.0%
The strongest performance in the test was on the 11 examples where the quotation
was exactly as has been found in the document sources. In this case, the respondents
were able to answer correctly 64.1% of the time. It is possible, although we cannot
determine from this that the language was not only natural, but also carried some
familiarity to the respondent, thus convincing that person that it was the actual
quotation.
Table 5. Recognition of 9 examples of Original or Normal Document (N)
Quotation Correct Chose Chose
(N) H for N M for N
Today, I consider myself the luckiest man on the face of 89.2% 10.8% 0.0%
the earth
My mama always said life was like a box of chocolates. 84.6% 13.8% 1.5%
You never know what you’re gonna get
Listen to me, mister. You’re my knight in shining armor. 72.3% 13.8% 13.8%
You’re going to get back on that horse, and I’m going to
be right behind you, holding on tight, and away we’re
gonna go, go, go!
The whole problem with the world is that fools and 71.9% 14.1% 14.1%
fanatics are always so certain of themselves, and wiser
people so full of doubts
Play it, Sam. Play ‘As Time Goes By.’ 62.5% 35.9% 1.6%
Whether you think you can, or that you can’t, you are 56.3% 23.4% 20.3%
usually right
(continued)
340 W. Patterson et al.
Table 5. (continued)
Quotation Correct Chose Chose
(N) H for N M for N
A census taker once tried to test me. I ate his liver with 54.7% 26.6% 18.8%
some fava beans and a nice Chianti
What we’ve got here is failure to communicate 53.1% 26.6% 20.3%
I’m living so far beyond my income that we may almost 39.1% 34.4% 26.6%
be said to be living apart
AVERAGE 64.1% 22.6% 13.3%
4 Subgroup Comparison
The respondents, as noted above, could be divided into 3 groups, previously labelled as
E, T, and R. The responses were considered separately in Table 6.
Table 6. Recognition of 9 examples of Original or Normal Document (N)
Subgroup Correct response Total responses % Correct
E 107 269 39.8%
T 621 1278 48.6%
R 199 388 51.3%
5 Follow-Up Study
With a small number of the overall participants it was possible to provide a training
guide after the analysis of the results from the first test, and thus prepare a second test
after the training guide had been studied. The second test again contained 30 elements
and was of the same form requesting a H, M, or N response. In this case, a few of the
original questions were repeated to see if memory might be an important factor in the
responses. In approximately half the cases, the second test following the training
module provided very large levels of improvement in the scores.
References
1. Turing, A.M.: Computing machinery and intelligence. Mind Q. Rev. Psychol. Philos. LIX
(236), 433–460 (1950)
2. Bartlett’s Quotations. http://www.bartlettsquotes.com/
3. American Film Institute. https://www.afi.com/100years/quotes.aspx
4. Best Machine Translation Software. www.g2.com/categories/machine-translation
Using Eye Tracking to Assess User Behavior
in Virtual Training
Mina Fahimipirehgalin(&), Frieder Loch, and Birgit Vogel-Heuser
Automation and Information Systems (AIS),
Technical University of Munich (TUM), Munich, Germany
{Mina.Fahimi,Frieder.Loch,Vogel-heuser}@tum.de
Abstract. Virtual training systems can provide flexible and effective training
for interactions with increasingly complex industrial machines. However,
existing approaches do not adapt to the adaptive attributes of the user. Being
able to track the current state of the user enables a humanization of virtual
training system, since it allows analyzing the strain and the cognitive processes
of the user and reacting accordingly. In recent years, eye tracking technology
has become a widespread research area in human machine interaction. This
paper introduces an approach to adapt virtual training systems based on eye
tracking analysis. The approach detects specific patterns from eye movements
and evaluate the performance of the user based on detected patterns. If the
pattern suggests that the user cannot follow the instructions or that the user is
distracted, the complexity of the training system can be reduced.
Keywords: Eye Tracking Pattern detection User behavior Virtual training
1 Virtual Training Systems and Eye Tracking
Manufacturing systems are becoming increasingly customized and dynamic and – at
the same time – more complex. Especially vulnerable groups, such as low-experienced
operators or seniors, may become over-challenged by these transformations. Effective
systems for the training and support of machine operators are required in such envi-
ronments [1]. In this field, one approach can be virtual training systems. They are based
on a virtual replication of the industrial environment and allow the training of industrial
procedures [2] and they should adapt to the abilities and the current state of the user [3].
This allows a humanization of training. The current state of the user can be detected to
deduct whether the user can follow the instructions that are provided by the virtual
training system. The training system can react based on this information, for instance,
by suggesting an interruption of the training or by providing additional instructions if a
user is struggling. Eye-tracking allows a non-invasive measurement of the current state
of the user by detecting specific patterns during the interaction with a virtual training
system. Research in the field of eye tracking [4] shows that the eye movements can
provide a suitable indication for the difficulty of understanding a text. Processing time,
for instance, is affected by the difficulty of the text or by inconsistencies that impede
understanding. These findings motivate the application of eye tracking measurements
for virtual training systems. The detection of different patterns from eye movements
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 341–347, 2020.
https://doi.org/10.1007/978-3-030-39512-4_54
342 M. Fahimipirehgalin et al.
can provide insights about the current state of the user. If the pattern indicates that the
user can follow the instructions, the virtual training system can offer to increase the
realism and the complexity of the training. If the pattern indicates that the instructions
are difficult, the virtual training system can, for instance, offer reducing the complexity
of the instructions or suggest a break. This paper is organized as follows: the review on
the research studies in the field of virtual training systems and eye tracking are provided
in Sect. 1. Then, the proposed approach for analyzing the gaze positions in the virtual
training system and possible user patterns are discussed in Sect. 3. Finally, the paper is
concluded in Sect. 4.
2 State of the Art
Virtual training systems have been suggested for different industrial applications, such
as assembly or maintenance procedures. Gorecky et al. [2] design a virtual training
system for assembly line and use gesture-based, speech-based interaction. However,
they focus on proposing different approaches to interact with user during training and
they do not consider user’s abilities for providing adaptive training system. Abe et al.’s
training system [8] can detect the user’s erroneous actions while they learn how to
assemble/disassemble the machine in the virtual training system. This information is
used to analyze whether the user learns the procedure correctly. Brough et al. [9]
propose a virtual training system in which during training sessions, error of users are
detected and feedbacks are generated by tracking the user’s hand motion. However,
they do not propose any pattern that are used to analyze the user’s behavior. Bluemel
et al. [12] propose a virtual training system that adapts the degree of detail of the
instructions. These adaptations are carried out manually without considering the per-
formance or the skills of the user. Gutiérrez et al. [6] propose a multimodal training
system including haptic, gesture and visual feedbacks which can be adapted to the
user’s preferences and needs. However, they do not consider the performance of the
user during the run time. Loch et al. [7] propose an adaptive virtual training system
with the means of different measurements of the user, however, they do not propose
any approach to deduct specific patterns from users and provide adaptive interaction
with the user based on their patterns. In the domain of Eye Tracking, Groen et al. [10]
suggest eye tracking measurements to evaluate the usability of user interfaces and
improve interface design. However, they do not define specific patterns to evaluate the
user behavior. Scheiter et al. [11] study the effect of multimedia instructions in learning
process using eye tracking. They use Eye Movement Model Example (EMME) to
improve the user’s capability and learning in following multimedia instructions. Pre-
vious research in the field of virtual training systems is lacking abilities for adaptation
of the training systems to the current state or the abilities of the users [3]. This is
especially the case for the applicability of eye tracking in the virtual training system.
This paper presents an analytical approach to investigate the applicability of the eye
tracking in the virtual training system by defining patterns for eye movements.
Using Eye Tracking to Assess User Behavior in Virtual Training 343
3 Eye Tracking Measurements in Virtual Training Systems
The patterns analyzed by this contribution are based on the virtual training system shown
in Fig. 1 (left). It trains the steps of an industrial procedure. The instructions of the
procedure are displayed on the right side of the screen. The central part displays a three-
dimensional model of the machine. The objective of this contribution is to detect if the
user can understand the instructions of the training system by analyzing eye movements.
The patterns focus on the textual instructions provided by the training system to simplify
the analysis. Therefore, only eye-movements in the rectangular area containing the
textual instructions are considered for the eye tracking analysis (Fig. 1 (Right)).
Fig. 1. Left: Virtual training system and heat maps of the eye gaze. The selected area for the eye
tracking analysis is marked as a white rectangle. Right: Sample gaze data in ðX; Y Þ coordinates in
the selected area.
This study uses head-mounted eye tracking glasses to avoid inhibiting the mobility
of the user. The recorded data is represented in ðX; Y Þ coordinates (see Fig. 1 (right)).
The center of the coordinate system of the eye tracking device is at the top-right point
of the screen. Therefore, reading the instructions from top to bottom means an increase
in Y coordinate and reading from left to right means a decreasing X coordinate. The
gaze position data is collected from different users while they were reading the
instruction list. In order to indicate the problems in understanding of the instructions,
several hypotheses for patterns of eye movements are created.
The first hypothesis is called Monotone Trend. This hypothesis assumes that the
user can follow the instructions and read them without repetitions and hold-up times.
Therefore, the Y coordinate should monotonically increase during the time [4] which
shows that the user can read them in a reasonable time. In this case, the slope of the
curve of the Y coordinate w.r.t the time is a positive slope which confirms the
monotonically increasing trend in Y coordinates. Furthermore, in each line of the text,
the gaze data in the X coordinate should decrease during the reading of each instruction
in this line. Therefore, the pattern in the eye movement follows a monotone trend (see
Fig. 2 (left)). Observing this pattern indicates that the user can follow the instructions
without hold-up times or distractions.
344 M. Fahimipirehgalin et al.
Pixels on vertical axis (Y-Coordinate)
Median value Positive line slope
Moving average
in Y coordinate
Pixels on horizontal axis (X-Coordinate)
Time
Start point of the next End point of the
command in X command in X
Coordinate Coordinate
Fig. 2. Left: Monotone trend in Y coordinate indicates that the user can read the commands
from up to the bottom. The gaze position in X coordinate monotonically decreases per each
command and increases again to the start point of the next command. Right: Non-monotone trend
in Y coordinate indicates that the user has difficultly in reading. The gaze position in X
coordinate shows that the user reads the same line several times.
The second hypothesis is called Non-monotone Trend. This hypothesis assumes
that the user cannot follow the instructions. This is shown by holdup times during
reading and therefore, the gaze positions will not show a monotone trend as in the
previous hypothesis. In this case, the slope of the curve of the Y coordinate w.r.t the
time will be approximately zero. This zero slope indicates no changes in the Y coor-
dinate due to hold-up times. Furthermore, in each instruction text, the gaze data in the
X coordinate will indicate the several readings of the same instruction. The detected
pattern of non-monotone trend is shown in the Fig. 2 (right). Observing this pattern
indicates that the user is not able to follow the commands properly and hold-up times
show the user’s difficulty in understanding the commands.
In order to detect the hold-up time, more detailed analysis is required in Y coor-
dinate. Since there is no significant changes in the Y coordinate during the hold-up
times, the slope of the curve is approximately zero. Therefore, for further analysis the
derivative of the Y coordinate w.r.t time can be considered to detect the hold-up time,
which means @Y @t ¼ 0. Zero derivative of y coordinate means that the user is gazing on a
specific instruction and therefore, there is no change in Y coordinate which can indicate
the hold-up pattern. However, since the gaze positions cannot be measured in an exact
area with exact X and Y coordinates and there are small oscillations in both coordi-
nates, it is not possible to consider that @Y
@t can be exactly zero. Therefore, an interval
around zero can be considered as ½a; a, in which a is a small number and when the
derivative of Y coordinate is in this interval, it can be assumed that there is not any
changes in the Y coordinate. To identify the monotone and non-monotone pattern with
the hold-up in gaze positions, a time threshold, T, can be defined. If the derivative of
the Y coordinate w.r.t time is in ½a; a interval for a continuous duration more than T,
then the hold-up pattern can be detected. If the derivative of Y coordinate w.r.t time is
Using Eye Tracking to Assess User Behavior in Virtual Training 345
in ½a; a interval for a short duration less than T, it can indicate that there is no hold-up
time. These two patterns and the derivative of Y coordinate are shown in Fig. 3. The
3D scatter plot represents the variation of ðX; Y Þ coordinates of the gaze positions. In
Fig. 3 (left) the gaze pattern of the user follows a monotone trend, and therefore there is
no hold-up time. As it is shown in the derivative plot, the derivative of Y coordinate is
in the zero interval only for short periods. This indicates that the user can read the
instructions without serious hold-up time and therefore, there is no difficulty for the
user. In Fig. 3 (right) the gaze pattern of the user does not follow a non-monotone
trend. The 3D plot shows that the user spends some time reading the instructions. The
hold-up time is marked by planes in parallel with the plane including the X coordinate
and the time coordinate, while the Y coordinate is fixed in different positions. Fur-
thermore, by considering the derivative of Y coordinate w.r.t time, it can be seen that
the derivative of Y coordinate is in zero interval for longer time (more than a threshold
T) and therefore, it is another evidence for hold-up time.
Fig. 3. Different patterns in the gaze positions in 3D plots of X, Y, and time coordinates. Left:
Monotone pattern. The derivative of Y coordinate w.r.t time is in zero interval for short periods.
Right: Non-monotone pattern. The derivative of Y coordinate w.r.t time is in zero interval for
longer periods.
Finally, The detected patterns can be used to improve the interaction with virtual
training systems. In the case that the hold-up time can be detected in the user’s gaze
positions, an interactive message can be presented to inform the user and suggest
actions, such as the interruption of the training process. The difficulty of the virtual
training system can be reduced when serious problems with the instructions are
detected. Thus, the virtual training system can be adapted to user needs by tracking the
eye movement of the user.
346 M. Fahimipirehgalin et al.
4 Conclusion and Outlook
In this paper, an approach based on eye tracking is presented to improve the efficiency
of virtual training systems. The analysis shows that the proposed method can detect
different patterns including monotone trend, non-monotone trend (with hold-up time),
and distraction. The detected patterns can further be used to interact with users and give
them feedbacks regarding their performance based on detected patterns. Future work in
this field will focus on the usability of eye tracking measurements in the area of virtual
training systems and follow the behavior of the user not only on the instruction part but
also on the area of the plant.
Acknowledgments. This work has been supported by the INCLUSIVE collaborative project,
which has received funding from the European Union’s Horizon 2020 Research and Innovation
Programme under grant agreement No. 72337.
References
1. Villani, V., et al.: Towards modern inclusive factories: a methodology for the development
of smart adaptive human-machine interfaces. In: 2017 22nd IEEE International Conference
on Emerging Technologies and Factory Automation (ETFA), Limassol, pp. 1–7 (2017)
2. Gorecky, D., Mura, K., Arlt, F.: A vision on training and knowledge sharing applications in
future factories. IFAC Proc. Vol. 46(15), 90–97 (2013)
3. Loch, F., Böck, S., Vogel-Heuser, B.: Teaching styles of virtual training systems for
industrial applications – a review of the literature. Interact. Des. Archit. J. (IxD&A) 38, 46–
63 (2019)
4. Rayner, K., Chace, K.H., Slattery, T.J., Ashby, J.: Eye movements as reflections of
comprehension processes in reading. Sci. Stud. Read. 10(3), 241–255 (2006)
5. Just, M.A., Carpenter, P.A.: A theory of reading: from eye fixations to comprehension.
Psychol. Rev. 87(4), 329–354 (1980)
6. Gutiérrez, T., et al.: IMA-VR: a multimodal virtual training system for skills transfer in
Industrial Maintenance and Assembly tasks. In: IEEE RO-MAN 2010: 19th IEEE
International Symposium on Robot and Human Interactive Communication, Viareggio,
Italy, 13–15 September 2010, pp. 428–433 (2010)
7. Loch, F., et al.: An adaptive virtual training system based on universal design. IFAC-
PapersOnLine 51(34), 335–340 (2019)
8. Abe, N., Zheng, J.Y., Tanaka, K., Taki, H.: A training system using virtual machines for
teaching assembling/disassembling operation to novices. In: 1996 IEEE International
Conference on Systems, Man and Cybernetics, Beijing, China, October 1996, pp. 2096–
2101 (1996)
9. Brough, J.E., et al.: Towards the development of a virtual environment-based training system
for mechanical assembly operations. Virtual Reality 11(4), 189–206 (2007)
10. Groen, M., Noyes, J.: Using eye tracking to evaluate usability of user interfaces: is it
warranted? IFAC Proc. Vol. 43(13), 489–493 (2010)
Using Eye Tracking to Assess User Behavior in Virtual Training 347
11. Scheiter, K., Schubert, C., Schüler, A.: Self-regulated learning from illustrated text: eye
movement modelling to support use and regulation of cognitive processes during learning
from multimedia. Br. J. Educ. Psychol. 88(1), 80–94 (2018)
12. Bluemel, E., Hintze, A., Schulz, T., Schumann, M., Stuering, S.: Virtual environments for
the training of maintenance and service tasks. In: Proceedings of the 2003 Winter Simulation
Conference, New Orleans, LA, USA, pp. 2001–2007 (2003)
Democratization of AI to Small Scale Farmers,
Albeit Food Harvesting Citizen Data Scientists,
that Are at the Bottom of the Economic
Pyramid
Chandrasekar Vuppalapati1, Anitha Ilapakurti1(&), Sharat Kedari1(&),
Rajasekar Vuppalapati1(&), Jayashankar Vuppalapati2(&),
and Santosh Kedari2(&)
1
Hanumayamma Innovations and Technologies, Inc., Fremont, CA, USA
{cvuppalapati,ailapakurti,
sharath,raja}@hanuinnotech.com
2
Hanumayamma Innovations and Technologies Private Limited,
Hyderabad, India
{jaya.vuppalapati,skedari}@hanuinnotech.com
Abstract. Climate change is impacting milk production worldwide. For
instance, increased heat stress in cows is causing average-sized dairy farms
losing thousands of milk gallons each year; drastic climate change, especially in
developing countries, pushing small farmers, farmers with less than 10 to 25
cattle, below the poverty line and is triggering suicides due to economic stress
and social stigma. It’s profoundly clear that current dairy agriculture practices
are falling short to counter the impacts of climate change. What we need are
innovative and intelligent dairy farming techniques that employ best of tradi-
tional practices with data infused insights to counter negative effects of climate
change. To achieve innovative and intelligent dairy farming techniques, we need
to disseminate state of art data science to masses. In other word, provide the
state of art data science algorithms to every food harvesting citizen data scientist,
farmer. The democratization of artificial intelligence to farmers, importantly, is
not only empowers farmers to understand the patterns and signatures of climate
change but also provides the ability to forecast the impending climate change
adverse events and recommends data driven insights to counter the negative
effects of climate change. With the availability of new data tools, farmers can
not only improve their standard of life but also conquer, importantly, perennial
“climate change related suicide” issue. It’s our staunch believe that the gold
standard for the success of the democratization of artificial intelligence is no
farmer life loss due to negative effects of climate change. In this paper, we
propose an innovative machine learning Sensor edge approach that considers the
impact of climate change and develops artificial intelligent (AI) models that is
validated globally but enables localized solution to thwart impacts of climate
change. The paper presents prototyping dairy IoT sensor solution design as well
as its application and certain experimental results.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 348–358, 2020.
https://doi.org/10.1007/978-3-030-39512-4_55
Democratization of AI to Small Scale Farmers 349
Keywords: Internet of Things (IoT) Machine Learning Climate Change
Climate change related suicides Decision tree Regression analysis
Embedded device Edge analytics Hanumayamma dairy IoT sensor Climate
models Farmer suicides
1 Introduction
1.1 Climate Change and Farmers Suicide
Small Farmers make big contribution1 to agriculture and dairy production in devel-
oping countries. Unlike the dairy farms of the west, milk originates in highly decen-
tralized2 villages with the help of small farmers who own three to five cattle and they
bring milk twice a day to milk collection centers to get paid. Simply put, the livelihood
of roughly 2 billion people (26.7% of the world population)3,4 of small farmers in
developing world depend on agriculture and the climate change adversely impacting
their survival; drastic climate change, especially in the developing and the developed
countries, pushing small farmers below the poverty line and is triggering suicides (See
Fig. 1) due to economic stress and social stigma5. The Climate Change is real and
“climate is now a data problem6” [1].
The study [3] from the University of California, Berkeley, has clearly corroborated
the link between the rising temperatures and the resultant stress on India’s agricultural
sector and its impact on increase in suicides over the past 30 years (see Fig. 1); the
study has categorically quantified suicides of nearly 60,000 Indian farmers [3, 4] are
related to climate change7,8.
1
Small Farmers make Big Contributions - https://ucanr.edu/blogs/blogcore/postdetail.cfm?postnum=
23184.
2
Strategies for the Bottom of the Pyramid: Creating Sustainable Development - http://pdf.wri.org/
2001summit_hartarticle.pdf.
3
Industrial Agriculture and Small Scale Farming - https://www.globalagriculture.org/report-topics/
industrial-agriculture-and-small-scale-farming.html.
4
Food and Agricultural Organization of the United Nations – http://www.fao.org/india/fao-in-india/
india-at-a-glance/en/.
5
India’s shocking farmer suicide epidemic - https://www.aljazeera.com/indepth/features/2015/05/
india-shocking-farmer-suicide-epidemic-150513121717412.html.
6
How Machine Learning Could Help to Improve Climate Forecasts - https://www.scientificamerican.
com/article/how-machine-learning-could-help-to-improve-climate-forecasts/.
7
Suicides of nearly 60,000 Indian Farmers linked to climate change, study claims - https://www.
theguardian.com/environment/2017/jul/31/suicides-of-nearly-60000-indian-farmers-linked-to-
climate-change-study-claims.
8
59,000 farmer suicides in India over 30 years may be linked to climate change, study says https://
www.washingtonpost.com/news/worldviews/wp/2017/08/01/59000-farmer-suicides-in-india-over-
three-decades-may-be-linked-to-climate-change-study-says/?noredirect=on&utm_term=.
f62a830a292a.
350 C. Vuppalapati et al.
Fig. 1. Farmers protest [2]
The problem not only confined to the developing countries, ironically, in the Unites
States of America, a recent report by the Guardian Newspaper [5] establishes the facts
that “the Suicide rate for farmers is more than double that of veterans9”. Similarly,
Forbes magazine’s article “dairy Farmers across the United States have been com-
mitting suicide at an alarming rate because their respective industry no longer supports
them in the same way that it has for generation10”. The National Public Radio
(NPR) [6], importantly, ran the program11 that underscores [7] “milk prices decline to
the worries about dairy farmers suicides rise” (see Fig. 2). Finally, a recent New York
Times12 article summarizes the plight of farmers in Australia [8]: “being a breadbasket
to the world and a globalization success story, so why are Australian farmers killing
themselves?” [8].
What we need is innovative and intelligent dairy farming techniques that employ
best of traditional practices with data infused insights and continuous inclusion of
adaptive data-models that factor in climate changes to counter the negative effects. To
achieve innovative and intelligent dairy farming techniques, we need to make data
science as citizen data science13, i.e., proliferation of data science to masses. Thus,
democratizing the field of statistics and analytics to extract predictive and prescriptive
insights from data and enabling farmers to take actionable insights; a bottom up
innovation14 for the small farmers at the bottom of the pyramid. It’s our ardent believe
that the data is our best defense and the savior against the negative effects of climate
change. The sooner we embark on democratization of AI to small farmers, the better
9
Why are America’s farmers killing themselves? - https://www.theguardian.com/us-news/2017/dec/
06/why-are-americas-farmers-killing-themselves-in-record-numbers.
10
Suicidal Dairy Farmers should consider Marijuana Industry - https://www.forbes.com/sites/
mikeadams/2018/03/16/suicidal-dairy-farmers-should-consider-marijuana-industry/#689e0865
1d1e.
11
As Milk Prices Decline, Worries about Dairy Farmers Suicides rise - https://www.npr.org/2018/02/
27/586586267/as-milk-prices-decline-worries-about-dairy-farmer-suicides-rise.
12
A Booming Economy with a price tag - https://www.nytimes.com/2018/05/20/world/australia/rural-
suicides-farmers-globalization.html.
13
Gartner Identifies the Top 10 Strategic Technology Trends for 2019 - https://www.gartner.com/en/
newsroom/press-releases/2018-10-15-gartner-identifies-the-top-10-strategic-technology-trends-for-
2019.
14
Strategies for the Bottom of the Pyramid - http://pdf.wri.org/2001summit_hartarticle.pdf.
Democratization of AI to Small Scale Farmers 351
we leave our progeny a wonderful life on the earth, i.e., better than what we have
inherited.
As part of the paper, we have developed mathematical, electrical, software, and
data science solution! We have developed Dairy Sensors to empower small farmers to
counter the climate change. The structure of this paper is presented as: Sect. 2 discusses
the basic concepts and methods about Machine Learning Algorithms. Section 3 pre-
sents our Climate Model, and Sect. 4 shows a case study.
2 Understanding Machine Learning Algorithms for Climate
Change and Dairy Sensor
2.1 NOAA
NOAA’s National Centers for Environmental Information (NCEI15) hosts and provides
public access to one of the most significant archives for environmental data on Earth.
The climate and weather forecast models can be used to model agriculture data. For
instance, Wind Chill Warning16 can be used to take proactive steps to mitigate the side
effects. Additionally, the data points could be baselined to functioning of the Dairy
sensors.
2.2 Hanumayamma Dairy Sensors
Generally, CLASS 10 classification17 is reserved to Medical Apparatus such as: Sur-
gical, medical, dental and veterinary apparatus and instruments, artificial limbs, eyes
and teeth; orthopedic articles; suture materials [10–12], [13]. Specifically, for Dairy
sensor (see Fig. 2), also known as Cow necklace, we have been assigned “Class 10:
Wearable veterinary sensor for use in capturing a cow’s vital signs, providing data to
the farmer to monitor the cow’s milk productivity, and improving its overall health”.
We have based above sensor design based on reference architecture provided by
Hanumayamma Innovations and Technologies, Inc., IoT Dairy Sensor18.
2.3 Climate and Dairy Sensor Deployment
Dairy IoT Sensor Values Ingest: The embedded system [12], [13–15] was deployed in
fields to collect cattle activity, the temperature and humidity data. The sensors are
deployed globally and are periodically uploads data on day-to-day basis.
15
NOAA - https://www.ncei.noaa.gov/.
16
National Weather Services - https://www.weather.gov/safety/cold-wind-chill-warning.
17
Trademark administration, General Provisions and Definitions - https://www.sec.state.ma.us/cor/
corpdf/trademark_regs_950_cmr_62.pdf.
18
Hanumayamma Innovations and Technologies, Inc., http://hanuinnotech.com/dairyanalytics.html.
352 C. Vuppalapati et al.
Fig. 2. Dairy sensor
2.4 Data
We have observed the reporting of humidity values greater than 100% during the five
months. For instance, the below has figure (Fig. 3) has the hourly temperature plot
grouped by time of the day for the period of October 30, 2016 to January 18, 2017, in
Punjab. The x-axis 0:00 represents mid-night 12:00 h and value 9.36 data at morning
9:36 am. For some dates, the data humidity values are reported more than 140%. The
data is collected in Patiala, Punjab and Vizag – a costal of Andhra Pradesh.
Fig. 3. Field data collected in dairy farms in India
2.5 Climate Events
To derive Climate change to Dairy Sensor parametric model, let’s start with following
climate events (see Table 1): (other extreme climate events can be logically deduced
through the established process):
Table 1. Climate events
Climate event
Rise of tidea Ũ
Rise of temperature Ů
Rise of Humidity Ŭ
Floodingb Ű
Rise of noise ȸ
(continued)
Democratization of AI to Small Scale Farmers 353
Table 1. (continued)
Climate event
Depression Ų
Wind Chill Warning Ŵ
Snow Stormc ϔ
a
Water Quality for Live
Stock - https://www.
agric.wa.gov.au/
livestock-biosecurity/
water-quality-livestock
b
Flooding –
Contaminated farm
dams - https://www.
agric.wa.gov.au/water-
management/
contaminated-farm-
dams?page=0%2C1
c
Mid-West Snow Storm -
https://phys.org/news/
2019-01-record-
breaking-cold-midwest-
snowstorm.html
Let’s consider following Climate Events to Dairy Sensor Parametric model table
(see Table 2). Please note:
# (arrow down) means decreases
" (arrow up) Increases
! (arrow flat) no change or don’t care
Table 2. Dairy sensor parametric model table
Climate event A (x, y, z) G (x, y, z) T H Physiological Emotional
Ũ " " " " " "
Ů #a # " # " !
Ŭ " " # " ! !
Ű ! ! ! ! " "
ȸ " " " " " "
a
Heat Stress in Dairy Cows - http://article.sapub.org/pdf/10.5923.j.zoology.
20120204.03.pdf
354 C. Vuppalapati et al.
2.6 Modeling Climate Change – Mathematics and Data Events
For instance, consider activity function of dairy sensor, of course, based on the cattle’s
activity: let Fa(x, y, z): Fa is accelerometer function with varying values based on the
dt ; dt and dt are change of x, y, and z values with respect to time.
dy
time where dx dz
dx dy dz
Fa ðtÞ ¼ F ; ; ð1Þ
dt dt dt
2.7 Normal Regular Day – No Weather Events
Under normal climatic condition, the function of sensor values (Fa) has the influence of
climatic factor U. Put it simple, the climate factors influence the sensor readings. The
Eq. 1 can be rewritten as:
dx dy dz dx dy dz
Fa ðtÞ ¼ F ; ; ¼ FaU U ;U ;U ð2Þ
dt dt dt dt dt dt
That is, under normal climate conditions, Eqs. 1 and 2 are same.
2.8 Weather Event Day - Factor-in Climate Events
For example, consider modeling of extreme (see Fig. 4) flood event (Ű):
The corresponding impact of Ű on dairy sensor accelerometer values be:
00 dx 00 dy 00 dz
F 00 U ;U ;U ð3Þ
aU dt dt dt
Fig. 4. Field data collected in dairy farms in India
Democratization of AI to Small Scale Farmers 355
2.9 Partial Factors
Partial derivatives are defined as derivatives of a function of a multiple variable when
all, but the variable of interest is held fixed during the differentiation. Partial derivatives
are very useful to see the effects of the climate on cohort clusters that are geo-fenced or
spread across different geolocations. The partial derivatives factors calculate the climate
change effect or delta on the factor.
00
FaU
The Climate Factor ¼
FaU
¼[ ð4Þ
00 dx 00 dy 00 dz dx dy dz
F 00 U ;U ;U = FaU U ;U ;U
aU dt dt dt dt dt dt
Since FaU and FaŰ have three different values (x, y, z), the partial values for each (x, y,
z) vector can be deduced: Climate Partial for X variance:
00
@Ux @Ux
ð Þ= 00
Simplifies to
@FaU
@FaU ð5Þ
00
@Ux 00
given @FaU @FaU
@Ux
00
Therefore, partial accelerometer x variance due to climate fluid event is @@Ux Ux Similar
approach will calculate, partial accelerometer y variance due to climate fluid and partial
accelerometer z variance due to climate fluid.
2.10 Parametric Model
The purpose of parametric model is to evaluate any climate event and calculate partial
derivatives that could have impacted by the event. Additionally, to store the partial
derivatives for future climate forecast – Eqs. 1, 2, 3, 4, and 5.
Fig. 5. Parametric model
356 C. Vuppalapati et al.
3 Factor-in Climate Conditions
To factor-in the effects of Climate events on Dairy cattle, through Dairy Sensors, first,
consider the mathematical models of Dairy Sensor on the cattle activities and, next,
bring in the effects of climate events. Third, apply partial derivatives that zoom in the
attribute in-focus leaving all other constant. Finally, once influencing coefficient are
calculated, apply network effects and cohort cluster techniques to apply to forecast
models. Please see 1, 2, 3, 4, and 5 on Fig. 5.
3.1 Climate Reference Pattern
To ensemble the impact of a forecasted climate event, the saved derivative values are
combined with model equation to simulate the negative effects of climate change. To
show in action, let’s consider Eq. 1:
dx dy dz
FaU U ;U ;U ð6Þ
dt dt dt
Let’s say National Weather Services has predicted climate event – Wind Chill
Warning.
To calculate the impact of Wind Chill Warning on Eq. 6, retrieve all partial
derivatives from the database that have influence on the model equation. Of course,
take into consideration the similarities of cohort cluster need to be validated for to
apply:
dx c dy c dz
F b c ¼ FaU d c
Z
W ;W ;W W
aW dt dt dt
dx c dy c dz dx dx dy
F b c c
Z
W ;W ;W ¼ FaU ðU W ;U ð6aÞ
aW dt dt dt dt dt dt
c dy dz dz
W ;U c
W Þ
dt dt dt
The forecasted climate change event Dairy Sensor data is equal to the raw sensor
model plus any partial derivatives that have influence on the model. Through this
computation, the system provides recommendation to the farmer the data insights that
are derived from similar cohort of dairies plus superimposed to the readings of the dairy
sensor.
Democratization of AI to Small Scale Farmers 357
4 A Case Study
As part of adaptive edge analytics, the temperature and humidity embedded system that
we have developed used in several agricultural and dairy settings (see Fig. 6). We have
captured real-time customer related data for creating an adaptive edge analytics paper.
Finally, the product is used and tested by precision dairy agriculture company19.
Fig. 6. Dairy sensor
References
1. Jones, N.: How machine learning could help to improve climate forecasts. Nat. Mag. (2017).
https://www.scientificamerican.com/article/how-machine-learning-could-help-to-improve-
climate-forecasts/
2. Michael Safi in Delhi: Suicides of nearly 60,000 Indian farmers linked to climate change,
study claims, Mon 31 July 2017 15.00 EDT. https://www.theguardian.com/environment/
2017/jul/31/suicides-of-nearly-60000-indian-farmers-linked-to-climate-change-study-claims
3. Umar, B.: India’s shocking farmer suicide epidemic, 18 May 2015. https://www.aljazeera.
com/indepth/features/2015/05/india-shocking-farmer-suicide-epidemic-150513121717412.
html
4. Doshi, V.: 59,000 farmer suicides in India over 30 years may be linked to climate change,
study says, 1 August 2017. https://www.washingtonpost.com/news/worldviews/wp/2017/08/
01/59000-farmer-suicides-in-india-over-three-decades-may-be-linked-to-climate-change-
study-says/?noredirect=on&utm_term=.38c8866059de
5. Weingarten, D.: The suicide rate for farmers is more than double that of veterans. A former
farmer gives an insider’s perspective on farm life – and how to help, December 2017. https://
www.theguardian.com/us-news/2017/dec/06/why-are-americas-farmers-killing-themselves-
in-record-numbers
6. Adams, M.: Suicidal Dairy Farmers Should Consider Marijuana Industry, 16 March 2018
12:23 pm. https://www.forbes.com/sites/mikeadams/2018/03/16/suicidal-dairy-farmers-
should-consider-marijuana-industry/#30a8ae8c1d1e
7. Smith, T.: As Milk Prices Decline, Worries About Dairy Farmer Suicides Rise, 27 February
2018 11:31 AM ET. https://www.npr.org/2018/02/27/586586267/as-milk-prices-decline-
worries-about-dairy-farmer-suicides-rise
19
Hanumayamma Innovations and Technologies, Inc. – Dairy IoT Sensor.
358 C. Vuppalapati et al.
8. Williams, J.: A Booming Economy With a Tragic Price, 20 May 2018. https://www.nytimes.
com/2018/05/20/world/australia/rural-suicides-farmers-globalization.html
9. GUEST BLOGGER: Machine Learning May Be a Game-Changer for Climate Prediction, 21
June 2018. https://blogs.ei.columbia.edu/2018/06/21/machine-learning-may-game-changer-
climate-prediction/
10. Karwowski, W., Ahram, T. (eds.): Proceedings of the 1st International Conference on
Intelligent Human Systems Integration (IHSI 2018): Integrating People and Intelligent
Systems, Dubai, United Arab Emirates, January 7–9 2018. https://www.springer.com/us/
book/9783319738871
11. Kedari, S., Vuppalapati, J.S., Ilapakurti, A., Vuppalapati, C., Kedari, S., Vuppalapati, R.:
Precision dairy edge, albeit analytics driven: a framework to incorporate prognostics and
auto correction capabilities for dairy IoT sensors, chap. 35. Springer, Cham (2019)
12. Ilapakurti, A., Vuppalapati, J.S., Kedari, S., Kedari, S., Vuppalapati, R., Vuppalapati, C.:
Adaptive edge analytics for creating memorable customer experience and venue brand
engagement, a scented case for Smart Cities. In: 2017 IEEE SmartWorld, Ubiquitous
Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing &
Communications, Cloud & Big Data Computing, Internet of People and Smart City
Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) (2017)
Cybersecurity in Educational Networks
Oleksandr Burov1(&), Svitlana Lytvynova1(&), Evgeniy Lavrov2(&),
Yuliya Krylova-Grek3(&), Olena Orliyk4(&), Sergiy Petrenko4(&),
Svitlana Shevchenko5(&), and Oleksii M. Tkachenko6(&)
1
Institute of Information Technologies and Learning Tools, 9 Berlinskoho Str.,
Kyiv 04060, Ukraine
[email protected], [email protected]
2
Sumy State University, Sumy, Ukraine
[email protected]
3
State University of Telecommunications, Kyiv, Ukraine
[email protected]
4
Scientific-Research Institute of Intellectual Property, Kyiv, Ukraine
[email protected], [email protected]
5
Borys Grinchenko Kyiv University, Kyiv, Ukraine
[email protected]
6
National University of Life and Environmental Sciences of Ukraine,
Kyiv, Ukraine
[email protected]
Abstract. The paper discusses the possible impact of digital space on a human,
as well as human-related directions in cyber-security analysis in the education:
levels of cyber-security, social engineering role in cyber-security of education,
“cognitive vaccination”. “A Human” is considered in general meaning, mainly
as a learner. The analysis is provided on the basis of experience of hybrid war in
Ukraine that have demonstrated the change of the target of military operations
from military personnel and critical infrastructure to a human in general. Young
people are the vulnerable group that can be the main goal of cognitive operations
in long-term perspective, and they are the weakest link of the System.
Keywords: Cyber-security Cognitive performance Education Social
engineering
1 Introduction
A constantly increasing number of cybersecurity-related publications demonstrates a
growing comprehension of this complex challenge facing the Globe and the necessity
to consider wider spectrum of issues. Unfortunately, technical and informational
solutions cannot satisfy humans’ safety and security of life and activity. Since it is an
on-going process, specialists in this field are lack of current information and feel the
need to change the training programs of cybersecurity (CS) that should focus “on the
social, economic, and behavioral aspects of cyberspace, which are largely missing from
the general discourse on cybersecurity” [1, p. 2]. First of all, new training programs
should take into account the human features and a person’s functional state as well as
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 359–364, 2020.
https://doi.org/10.1007/978-3-030-39512-4_56
360 O. Burov et al.
cognitive resilience due to the increasing role of cognitive warfare [2]. The cognitive
war must deserve particular attention as its primary goal is not a prompt military
operation and fight for territorial or economic resources, but it is a battle for people [3]
aimed at affecting public opinion, radicalizing young people, infiltrating and corrupting
enemy’s information systems. Since the information in the global network exists out of
space and time, the Net itself becomes an active human influencer [4], especially in
social networks [5].
One of the human dimensions of extensive change involves the transition from
producing predominantly material issues to intellectual ones and alterations in com-
petitive target resources. Intellectual capital (first of all, human capital includes abili-
ties, talents, knowledge, ideas, etc.) is becoming the most in-demand resource and the
target of diverse cyber-attacks [6]. At present, digital networks are taking more and
more crucial place in our everyday routine. Therefore, interventions to these networks
pose a real threat to both humans and the state. By saying “humans” we don’t mean just
military (including cyber-)specialists, but everybody, since the cyberspace is a
worldwide electronic medium facilitating social interaction. Undoubtedly, transfor-
mations in the forms, methods, and means of education are related to and accompanied
by changes in learners’ behavior by transition from traditional classroom education to
network activities with unproductive consequences of the information received and its
safety. However, at the same time, a human is still the weakest link in cybersecurity
systems [7].
Purpose. To analyze potential hazards associated with learners’ participation in online
activities in digital education.
2 Method
Considering learning as a type of activity in human-system integration, today’s learner
may be viewed as an operator-researcher who acts in the digital environment. Suc-
cessful learning involves mutual adaptation between a human and activity tools [8]
using individual cognitive abilities measurement [9, 10]. On the other hand, it is
possible to use ergonomics’ methods and techniques to assess a learner’s safety in the
education system.
3 Results and Discussion
The core directions of cybersecurity analysis in the education field should be focused
on the following issues: CS levels, role of social engineering in providing CS in
education, and so-called “cognitive vaccination”.
Cyber Security Levels. The paper deliberates about the problems of learners’ cyber-
security in the educational process. It emphasizes the fact that the given problems are
not limited to the technical aspects of protecting information resources, which must
include such types of protection as legal, technical, informational, organizational, and
psychological ones [4, 11].
Cybersecurity in Educational Networks 361
The legal maintenance covers (but not limited) [12]:
• National and international legislation in the field of cybersecurity.
• Appropriate international legal agreements, conventions, and standards.
• Intellectual property rights.
• Protection of computer programs and databases [13].
• Personal data protection.
• Legal support of victims of cyber-attacks and expert opinions on the results of the
computer-technical examinations.
• Legal support of a human right to know and get access to verified information (a
person’s education and development cannot be achieved without realizing self-
concept).
• Legal literacy for young people regarding actions in digital networks.
Cybersecurity technical aspects imply the security of diverse technical means and
tools (computers, networks, databases, information resources, etc.).
Information tools can be categorized according to the tasks solved by the users [11,
p. 321]: Protection/Remedies, Awareness, Content, Learning to use, Security, Lifespan,
Avoiding threats.
Organizational tools for solving cybersecurity issues comprise Awareness,
Learning the cybersecurity culture, CS professional staff and the general population,
Creation of CS special means, Distribution of CS facilities, Control of use.
Psychological means can be grouped depending on the personal and interpersonal
level: National, Public, Group, Individual, Cultural, Cognitive, Intellectual, Habits.
Among the psychological tools aimed at achieving cybersecurity, the cognitive
ones are the most vital. Recent cybersecurity research shows that information tech-
nology tools in this field are constantly being refined and hacker attacks become more
human-centered [14]. This is extremely important because of the urgency of personal
safety and the results of its activities. As shown in [4], the common accessibility of the
information space leads induces that a person becomes a target of other participants’
activity, while working in the information environment. Harmful activities force a
person to read or to respond to the “wrong” information or to make other mistakes that
leave his/her system vulnerable to cyber attacks, information leakage, etc.
These days, not only huge corporations or governing bodies are usual targets of
cyber attacks, ordinary people, especially children and young adults, suffer from them
as well. Their cognitive sphere is the most vulnerable (weak) link in the person-
technology network [7], in particular, due to the extending usage of group work
(project-oriented activity). In this regard, it is reasonable to exploit the operators’
experience of preventing against cyber threats in the education field [15], accounting
that in anthropocentric networks, which make up an ever-increasing share among
common networks, the network itself acquires new properties, acting as an independent
component (in addition to such factors as the network unit, interface, and links) acting
beyond time and space [6].
Role of Social Engineering in Providing CS in Education. The spectrum of hazards
from the open cyberspace is continuously expanding. If ten years ago, the hazards for
schoolchildren could be reduced to a relatively small number of groups (viral attacks,
362 O. Burov et al.
cybercrime, the hazards of Internet surfing), then the diversity of hazards and threats is
increasing over time, affecting all possible human activities online [11]. Threats coming
from networks can be divided into the following types: active and passive, open and
hidden, current and delayed [11, p. 309]. The greatest danger to students is hidden
hazards of the Internet and especially the social engineering methods [16, 17].
The shift of cybercrime goals from technical (information) objects to the human link
led to the emergence of social engineering (SE) as methods and technologies for
obtaining the necessary access to information based on the characteristics of human
psychology. Social engineers, for instance, use fear, interest or trust to manipulate, to
change the behavior or perception of others. Sad to say, nowadays everybody can
master the art of gaining access to computer systems or personal data [18]. Yet it is
possible to resist SE impact if to follow nine recommendations:
• User credentials are the school property.
• Conduct introductory and regular training sessions for staff and students to increase
information security skills.
• It is mandatory to have safety regulations and instructions that the user must always
have access to.
• Users’ computers must always have up-to-date antivirus software and firewall
installed.
• Systems of detection and prevention of attacks should be used in any corporate
network. Confidential information leakage prevention systems should be employed
as well.
• It is necessary to restrict users with administrative privileges for operating systems
and applications as much as possible.
• You need to be vigilant about the source requiring sensitive information.
• You should never open the contents of applications or follow the link without
examining all the details and your own experience.
• It is also important to be critical of the messages received: how plausible can the
information be?
• It is recommended to report such dangers to other family members, first of all, the
elderly, who have no experience of using electronic means and are not aware of SE
issues.
We believe that psycholinguistic tools could be useful to recognize SE interference
and the ways to affect human cognition and safety, especially the cognitive weapon
(mass-media, politicians’ impact, textbooks, etc.) [19]. If a person knows, realizes and
is aware of these tools, he/she can obviously resist them, which is the most effective
way of providing cybersecurity.
“Cognitive Vaccination”. In 2002, UN General Assembly adopted resolution 57/239
“Elements to Create a Global Cybersecurity Culture” [20] to identify nine fundamental
complementary elements of the global cybersecurity culture, including awareness;
responsibility; response; ethics; democracy; risk assessment; design and implementa-
tion of security measures; security management; revaluation.
The Resolution and cybersecurity elements relate to five levels of CS mentioned
above. At the same time, it can be noted that psychological means (which relate directly
Cybersecurity in Educational Networks 363
to each person separately) involve only behavioral aspects, i.e. responsibility and
ethics; in other words, it is a manifestation of the social attitude to cybersecurity
expressed by a person, who is considered as a relatively passive element of the
cybersecurity system. Moreover, since no means guarantee 100% of human protection,
it is advisable to determine the range of individual abilities to produce personal pro-
tection, except for the above.
The analysis of the curriculum and training programs implemented in pedagogical
educational institutions has demonstrated that traditional education does not pay
enough attention to the development of students’ critical thinking skills related to the
use of the Internet.
We propose to introduce “cyber vaccination” as part of the cybersecurity-related
training. It can increase the human’s safety level by a wide array of means: to accept
rules for safe and responsible use of the Internet, to improve critical thinking skills, to
train the participants of the network activity and to inform them about possible impact
of the cyber environment, to model and to simulate cyber threats in relatively closed
systems such as corporate and educational ones, to teach how to confront with the
cyber threats for gaining the practical experience of behaving and restoring after cyber
vulnerabilities, including assessing the person’s current state and necessary adjustments
to optimize his/her cognitive workability, and cyber survival trainings aimed at rec-
ognizing the threat or possible dangerous action in the network and the rational psy-
chological and behavioral compensation for this action.
4 Conclusion
The analyzed features of teaching and learning in contemporary digital environment
and recommendations could be an influential tool to improve the security and safety of
educational process by adapting students’ activity depending on his/her cognitive state
in digital education, by designing intelligent individual-oriented systems and services
that ameliorate human – E-technology interaction.
Acknowledgments. This work is supported by the grant 0118U003160 “System of Computer
Modeling of Cognitive Tasks for the Formation of Competencies of Students in Natural and
Mathematical Subjects”.
References
1. Ahram, T., Karwowski, W.: Advances in Human Factors in Cybersecurity. Proceedings of
the AHFE 2019 International Conference on Human Factors in Cybersecurity, Washing-
ton D.C., USA, 24–28 July 2019. Springer, Cham (2019)
2. Bienvenue, E., Rogers, Z., Troath, S.: Cognitive Warfare (2018). https://cove.army.gov.au/
article/cognitive-warfare
3. Pocheptsov, G.: The War in Cognitive Space (2017). https://nesterdennez.blogspot.com/
2017/08/global-permanent-war_39.html
4. Burov, O.: Educational networking: human view to cyber defense. Inf. Technol. Learn.
Tools 52, 144–156 (2016)
364 O. Burov et al.
5. Lytvynova, S., Burov, O.: Methods, forms and safety of learning in corporate social
networks. In: Proceedings of the 13th International Conference on ICT in Education,
Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer,
Kyiv, Ukraine, 15–18 May, pp. 406–413 (2017). http://ceur-ws.org/Vol-1844/10000406.pdf
6. Bykov, V.Yu., Burov, O.Yu., Dementievska, N.P.: Cybersecurity in digital educational
environment. Inf. Technol. Learn. Tools 70(2), 313–331 (2019)
7. Yan, Z., Robertson, T., Yan, R., Park, S.Y., Bordoff, S., Chen, Q., Sprissler, E.: Finding the
weakest links in the weakest link: how well do undergraduate students make cybersecurity
judgment? Comput. Hum. Behav. 84, 375–382 (2018). ISSN 0747-5632
8. Veltman, J.A., Jansen, C., Hockey, G.R.J., Gaillard, A.W.K., Burov, O.: Differentiation of
mental effort measures: consequences for adaptive automation. NATO Sci. Ser. Sub Ser.
I Life Behav. Sci. 355, 249–259 (2003)
9. Basye, D.: Personalized vs. Differentiated vs. Individualized Learning. ISTE 1/24/2018
(2018). https://www.iste.org/explore/articleDetail?articleid=124
10. Veltman, H., Wilson, G., Burov, O.: Operator Functional State Assessment. Cognitive Load.
NATO Science Series RTO-TR-HFM-104, Brussels, pp. 97–112 (2004)
11. Burov, O.Iu., Kamyshin, V.V., Polikhun, N.I., Asherov, A.T.: Technologies of network
resources’ use for young people training for research activity, Monograph. In: Burov, O.Iu.
(ed.) Informatsiini Systemy. TOV, Kyiv (2012). (in Ukrainian). 416 p.
12. Orlyuk, O.: On the development of a policy on intellectual property in national universities
and the role of profile departments of intellectual property. Theory Intellect. Prop. 5, 61–69
(2017)
13. Petrenko, S.A.: Protection of the Computer Program as an Intellectual Property Object:
Theory and Practice, Monograph, p. 172. Research Institute of IP at NA-PrNU, “Lazurit-
Polygraph”, Kyiv (2011)
14. https://www.computerweekly.com/news/252448101/People-top-target-for-cyber-attackers-
report-confirms
15. Lavrov, E., Pasko, N., Tolbatov, A., Tolbatov, V.: Cybersecurity of distributed information
systems. The minimization of damage caused by errors of operators during group activity.
In: Proceedings of 2nd International Conference on Advanced Information and Commu-
nication Technologies (AICT 2017), pp. 83–87 (2017). https://doi.org/10.1109/AIACT.
2017.8020071
16. Savchuk, T.: Social Engineering: How Fraudsters Use Human Psychology on the Internet
(2018). https://www.radiosvoboda.org/a/socialna-inzhenerija-shaxrajstvo/29460139.html.
Accessed 30 Aug 2018. (in Ukrainian)
17. A Powerful Tool for Social Engineering. https://www.trustedsec.com/social-engineer-
toolkit-set/#
18. SecurityLab. https://news.rambler.ru/other/39395044-nazvany-glavnye-problemy-bezopasn
osti-web-resursov-v-2017-godu/
19. Krylova-Grek, Yu.: Psycholinguistic peculiarities for application of the symbol-words in the
political communication. Adv. Educ. 7, 129–134 (2017). https://doi.org/10.20535/2410-
8286.99321(2017)
20. UN General Assembly Resolution 57/239 Elements to Create a Global Cybersecurity
Culture (2002). http://www.un.org/en/documents/declconv/conventions/elements.shtml
The Problem of Tracking the Center
of Attention in Eye Tracking Systems
Marina Boronenko(&), Vladimir Zelensky, Oksana Isaeva,
and Elizaveta Kiseleva
Ugra State University, 16 Chekhov Street, 628012 Khanty-Mansiysk, Russia
[email protected], [email protected],
[email protected], [email protected]
Abstract. Modern methods of analyzing the results of eye tracking are high
provided that the rigid fixation of the head of the subject is observed. To
improve the accuracy of the identification of the cause of the change in the size
of the pupils, you can make adjustments to the coordinates of the center of
attention. We propose an alternative, simpler adjustment method. In the pro-
posed method, the transition to the coordinate system associated with the pupil
is carried out. After this transition, the results of the study will correspond to the
task given to the participants.
Keywords: Eye tracking Test-object A coordinate bond An emotional
response
1 Introduction
Today, an important aspect of modernity is security [1]. In order to ensure it, the latest
technologies are being introduced. Security systems allow employees and visitors to
the organization using biometric data, which is not an innovation [2]. Existing methods
are good enough to recognize attempts to conceal information, but no technology can
with absolute certainty give a 100% result [4]. Therefore, it is necessary to apply a set
of methods that will improve the interpretation of the information received in many
areas of activity – from the selection of personnel to the investigation of crimes.
One of the tools of eye tracking technology is eyetracker, which recognizes and
records pupil positions and eye movements. The device can be worn on the head
(eyewear or helmet) or stationary, which is placed on a table in front of the monitor
screen, it can also be used in any research related to the visual system [2]. Visual
information causes natural eye reactions to what is seen, which is impossible to control
[2]. Therefore, this technique is widely used in applications such as sleepiness detec-
tion, diagnosis of various clinical conditions or iris recognition, cognitive and behav-
ioral therapy, visual search, advertising, neuroscience, psychology, and it is also
applicable for analysis in security systems [3, 15].
The majority of works devoted to the use of eye tracking technologies analyze the
focus of attention when performing technical actions (including ignoring visual stim-
uli), visual search strategies in the process of activity, measurement of pupil diameter
(as an indicator of cognitive load), the number of saccadic eye movements, as well as
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 365–371, 2020.
https://doi.org/10.1007/978-3-030-39512-4_57
366 M. Boronenko et al.
fixations, blinks and other parameters (Drummers, Zhegallo, 2013, etc.) [5]. Often eye
tracking is used to determine lie [13]. This method is also used in forensic examinations
to determine the truth of the testimony. Eye tracking keeps track of non-verbal human
activity, contributes to the identification of the most marked psychological character-
istics governing a particular type of behavior [6]. It is also often used in the selection of
employees for service [7]. Eye tracking technology is used to improve performance in
various sports, to predict consumer behavior in response to various marketing incen-
tives, in the process of learning a foreign language and translation, in the process of
knowledge control in distance learning [8. 9]. By registering the time of fixation and the
density of the trajectory of the gaze, it is possible to judge the significance of the
elements seen for a person.
Equipment based on the principle of recording eye movements in infrared radiation
with subsequent determination of direction based on the displacement vector between
the centers of the pupil and the corneal glint, gives high accuracy results, provided that
the subject’s head is fixed. Not having the equipment providing fixation of the head, the
process of data analysis is complicated, so the development of other methods that
allow, without using special equipment, to increase the accuracy and reliability of data
is relevant [12].
With the help of modern technology eye tracking possible seconds to restore what
interested a person, what he looked, what became the subject of his attention [4]. This
technology has a number of advantages associated with the fact that oculomotor
reactions are recorded remotely, without attaching sensors to the body of the subject,
making the evaluation procedure unobtrusive, more comfortable and less stressful. The
process of registering the position and movement of a person’s gaze takes three times
less time than a standard study using a polygraph [5]. An additional tool for tracking
the psychophysical state of the subject is the method of pupillometry. Emotional sta-
bility of a person can be determined by his reaction to the provided test objects, by
analyzing changes in the area of the pupil and the trajectory of its movement.
But since the registration of the reaction of the pupil without rigid fixation of the
head affects the position and coordinates of the pupil, thereby reducing the accuracy of
the interpretation of the results, it becomes necessary to develop a method that reduces
the impact of this effect. The goal of our study is to develop a way to improve the
accuracy of tracking the center of attention track.
2 Research Methods
To study the reaction of the pupil to test objects, a helmet was developed that creates a
rigid coordinate connection between the camera and the head. The T7 Astro Camera
Astro-nominal Astronomy Planetary High Speed Electronic Eyepice Telescope Digital
Lens for Guiding Astro-photograph, 30 fps video mode, a microscope lens with an
optical magnification of 1X–100X were used for video shooting. 17 people took part in
the experiment. The main part of the studied people were persons under 20 years of
age, men-8, women-9. There were no eye diseases in the respondents. All participants
were warned in advance and voluntarily decided to participate in the experiment. As
test objects, specially selected pictures with negative emotional coloring (disgust,
The Problem of Tracking the Center of Attention 367
irritation) were used [16]. Stimulus material was displayed on a monitor screen. The
distance between the test objects and the eye was not less than 2.5 m. The subjects
alternately at regular intervals were presented with images of test objects and mono-
chrome slides of light gray color [14]. During all the experiments, the brightness of the
test objects was not changed. The selection of test images was carried out on the basis
of a survey conducted among a large number of respondents.
Before the test, each subject was instructed, which stipulated the requirements for
the test procedure and the order of its passage, as well as the adjustment of the chair on
which the subject sat, and the monitor screen. It was necessary, in connection with the
physiological characteristics of each person. Individual data of the subjects on the
reaction to the test objects were analyzed on the basis of changes in the size of the pupil
and the attention tracker.
The total time of the whole procedure of the study takes no more than 5 min, which
does not affect the fatigue of the Respondent, but enough to track the tracker look.
Processing and analysis of the results was carried out in two stages. First, image
dissection, processing and delineation of the pupils in ImageJ were carried out. This
was necessary for further analysis of the data obtained: the coordinate trajectory of the
pupil and its relative size. Visualization of the results was made in Origin19. Origin
software is one of the most powerful tools for graphical representation of results.
3 Main Results
When implementing a series of experiments with the presentation of unpleasant
stimulus material, it is difficult to find the same emotional response. Moreover, the
intensity of stimuli is low. This is due to different preferences, temperament of people,
current concerns, etc. Therefore, the results were sometimes unexpected. We planned to
get a reaction to the test object, causing disgust, and got a reaction to the image of
Wi-Fi (Fig. 1). The subject did not react to the test object, which was supposed to cause
disgust, as at the time of testing, he was thinking about online games. This was found
out by a survey conducted after testing. For him the lack of Internet turned out to be a
stressful situation.
Fig. 1. Track attention
Figure 2 shows the graphs of the dependence of the change in the area and the
coordinate of the pupil on the time of demonstration of the test object. In the period
from 6 to 9 s, the subject observed a test object with a Wi-Fi image (Fig. 1). The
analysis showed that during this period there was a slight change in the X and Y
coordinates. Calibration showed that the shift of the center of attention by 1°38′ leads to
368 M. Boronenko et al.
a change in the size of the pupil by no more than 0.1 times. Synchronization pupil-
logram with the coordinate tracks the attention of through software.
Fig. 2. Graph of the relative size of the pupil and its coordinates of movement from time
During calibration, it was found that in some cases, the resulting tracks were offset
relative to the area of the image (Fig. 3 track turquoise). This is due to the lack of a
rigid coordinate connection between the head and the surface of the monitor from
which the test objects were demonstrated.
If you use eye tracking, for example, for marketing, the offset won’t be a huge loss.
However, such distortions are unacceptable in high-precision systems.
The Problem of Tracking the Center of Attention 369
Fig. 3. Distortion of the visual trace due to the rotation of the head-turquoise dots; the same
track in the coordinate system associated with the center of the pupil-red dots
4 Discussion of the Results
Analyses of the results are complicated by the fact that the shooting is carried out without
rigid fixation of the human head. Every movement of the body can increase the inaccuracy
of the processed data. The authors M. Yu. Kataev and N. V. Kovalev proposed a method
for estimating the angles of human head rotations that is uncritical to changing mea-
surement conditions [10, 11]. It is known that Euler angles uniquely determine the
rotation of one coordinate system relative to another. Rotation matrix R (1):
0 1
cos u cos w sin u cos h sin w cos u sin w þ sin u cos h cos w sin u sin h
R ¼ @ sin u cos w cos u cos h sin w sin u sin w þ cos u cos h cos w cos u sin h A
sin h sin w sin h cos w cos h
ð1Þ
Equations expressing the coordinates of a point in the O′x′y coordinate system
relative to the Oxy coordinate system:
8
< x ¼ ðcos w cos u sin w cos h sin uÞx0 þ ð cos w sin u sin w cos u cos uÞy0 þ sin w sin hz0
y ¼ ðsin w cos u þ cos w cos h sin uÞx0 þ ð cos w sin h þ cos w cos u cos uÞy0 cos w sin hz0 ;
:
z ¼ sin h sin ux0 þ sin h cos uy0 þ cos hz0 :
Thus, determining the Euler angles by the above-mentioned method [10, 11], it is
possible to amend the coordinates of the center of attention. This will improve the
accuracy of identification of the cause of the change in pupil size. We propose an
alternative, simpler adjustment method. In the proposed method, the transition to the
coordinate system associated with the pupil is carried out. After this transition Fig. 3
the track (red) began to correspond to the task given to participants.
370 M. Boronenko et al.
5 Main Conclusions
In the course of the research it was found that changing the angle of inclination or head
rotation leads to distortion of the track obtained by the method without the use of
infrared illumination; the transition to the coordinate system associated with the center
of the pupil, while tracking the center of attention allows you to minimize distortion. In
this case, the video camera used to obtain information about the direction of the gaze
has a rigid coordinate contact with the head.
Acknowledgments. This Study was funded by RFBR under the research project 18-47-860018
r_a.
References
1. Fomenko, G.Yu.: Psychology of personal security: theoretical and methodological
foundations of institutionalization. Community Management, no. 1, pp. 83–99. The Kuban
State University, Krasnodar (2010)
2. Lebedenko, Yu.I.: Biometric System (Биoмeтpичecкиe cиcтeмы), p. 159. Publishing House
of Tula State University (2012)
3. Luneva, E.A., Skobelkina, N.G.: Eyetracking in the system of modern technologies of
neuromarketing. STAZH 3(24), 50–53 (2016)
4. Fazylzyanov, G.I., Belalov, V.B.: Eye tracking: cognitive techniques in visual culture. Bull.
Tambov State Univ. 2, 628–633 (2014)
5. Zhbankova, O.V., Gusev, V.B.: Using the method of video oculography (eytracking) to
identify hidden information. In: Complex Psychophysiological Forensic Examination:
Current State and Prospects of Development: Sat. Sci. - Prakt. Conference, Kaluga, pp. 102–
105 (2016)
6. Bessonova, Yu.V., Oboznov, A.A.: Eyetracking in the diagnosis of truth/lies. In: Materials
of the Congress of the Russian Psychological Society, pp. 227–228 (2017)
7. Barabanshchikov, V.A., Zhegallo, A.V., Jose, E.G.: Assessment of the reliability of the
reported information on nonverbal behavior. In: Complex Psychological and Psychophys-
iological Forensic Examination: Current State and Prospects of Development, p. 33.
MGPPU (2016)
8. Zhbankova, O.V., Gusev, V.B.: Using the method of video oculography (eytracking) to
identify hidden information. In: Complex Psychophysiological Forensic Examination:
Current State and Prospects of Development: Sat. Sci.- Prakt. Conference, Kaluga, pp. 102–
105 (2016)
9. Lisi, A.Y., Kompaniets, V.S.: Eye tracking as a method to evaluate user interfaces. In: The
New Task of Technical Sciences and Ways of Solution: Collected Articles on the Results of
International Scientific-Practical Conference, Orenburg, pp. 31–33 (2017)
10. Kataev, M.Yu., Kovalev, N.V., Griboyedov, A.A.: Restoration of human head rotation
angles from images. Reports of TUSUR 2.1, pp. 238–242 (2012)
11. Kataev, M.Yu., Kovalev, N.V.: Assessment of the position of the human head on the
analysis of images. Reports TUSUR, no. 1–2 (21) (2010)
12. Demidov, A.A., Zhegallo, A.V.: SMI equipment for recording eye movements: test drive.
Exper. Psychol. 1, 149–159 (2008)
The Problem of Tracking the Center of Attention 371
13. Boronenko, M.P., Zelensky, V.I., Kiseleva, E.S.: Application of attention waves as a marker
of hidden intentions. Nat. Psychol. J. 2(34), 88–98 (2019). https://doi.org/10.11621/npj.
2019.0212
14. Bobrova, D., Bikberdina, N., Boronenko, M.: Investigation of the color effect of the test
object on the pupil response. J. Phys. Conf. Ser. 1124(3), 031016 (2018)
15. Boronenko, M., et al.: Use of active test objects in security systems. In: Proceedings of the
AHFE 2019 International Conference on Neuroergonomics and Cognitive Engineering, and
the AHFE International Conference on Industrial Cognitive Ergonomics and Engineering
Psychology, Washington DC, USA, 24–28 July 2019, p. 438. Springer (2019)
16. Boronenko, M., et al.: Methods of indication of low intensity pupil reaction on the
subjectively-important stimuli. Act. Nerv. Super Rediviva 61(2), 49–56 (2019)
Health Risk Assessment Matrix for Back Pain
Prediction Among Call Center Workers
Sunisa Chaiklieng1(&) and Pornnapa Suggaravetsiri2
1
Department of Environmental Health, Occupational Health and Safety,
Faculty of Public Health, Khon Kaen University, Khon Kaen, Thailand
2
Department of Epidemiology and Biostatistics, Faculty of Public Health,
Khon Kaen University, Khon Kaen, Thailand
[email protected]
Abstract. Call centers workers commonly complain about back pain which
could be predicted from health surveillance program. This study aimed to
estimate risks for back pain among call center workers by the application of a
risk matrix. A combination of ergonomics assessor with Rapid Office Stain
Assessment (ROSA) and muscular discomfort assessment with Cornell dis-
comfort questionnaire (CMDQ) was done among 216 call center workers. The
results showed that most call center workers (44.9%) had health risk on neck,
shoulder and back pain at the moderate level. The highest rate was indicated on
back pain (25.5%), followed by shoulder pain (22.2%). Pearson product
moment correlation coefficient (r) showed that discomfort score significantly
associated with back pain score (r = 0.756). Call center workers can be pre-
dicted to develop back pain from exposure to ergonomic risk factors by appli-
cation of the intelligence matrix which considers both ergonomic risks and self-
reported discomfort. The health risk matrix may be used for the surveillance of
back pain among call center workers and other similar workers.
Keywords: Call center Ergonomics Intelligence matrix Shoulder pain
1 Introduction
A call center is a unit for transferring information by calling or emailing a customer.
Call center workers spend the most of their time on the telephone or searching and
recording data on a computer. By use of shift work and interactive voice response
(IVR), call centers provide a daily 24-h customer service. Workers may face eye, ear,
and throat related health problems [1] along with musculoskeletal disorders (MSDs). In
call center workers MSDs are caused by awkward posture [2] or inappropriate work-
station design such as a lack of forearm support [3] while using computers.
Normally, MSDs are common among those whose work involves repetitive action
and a prolonged static posture [4]. The high prevalence of MSDs among call center
workers has been previously reported. Poochada and Chaiklieng [5] found a high
prevalence of neck, shoulder and back pain during the last 3 months among call center
workers in Thailand (83.8%; 95% CI: 78.8–88.7), and back pain was the most common
complaint. Likewise, a study by Subbarayalu [1] found that back pain had the highest
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 372–378, 2020.
https://doi.org/10.1007/978-3-030-39512-4_58
Health Risk Assessment Matrix for Back Pain Prediction 373
prevalence during the last 12 months among call center workers in India. This pre-
dominant health impact among call center workers can be controlled by surveillance
with an ergonomics risk assessment which considers posture, force, frequency and
duration of work [6].
There have been two main approaches to ergonomic risk assessment: (1) subjective
assessment by self-report questionnaire, and (2) an objective approach which can be
subdivided into (2.1) observational assessment and (2.2) direct measurement tech-
niques [7, 8]. A basic health risk assessment (HRA) considers both the probability of
exposure and the severity of an adverse health effect from that health hazards exposure
[9]. However, the use of a single method of ergonomics risk assessment may not apply
equally to all regional MSDs problems.
Not much is known about applying the two approaches for MSD prediction with a
combination of two assessment methods developed for the health risk matrix. The one
previous matrix was applied to the risk assessment of shoulder pain in office workers
by combining observational assessment using the Rapid Office Strain Assessment
(ROSA) with a subjective assessment of self-reported discomfort using the Cornell
Musculoskeletal Discomfort Questionnaires (CMDQ). The prediction provided by
application of the risk matrix was confirmed by the later incidence rate of shoulder pain
in the office workers [10]. It is interesting to apply this health risk assessment
(HRA) matrix to call center workers whose work is similar in nature to that of the office
worker. The aim of this study was to estimate health risk of back pain among call center
workers by the application of the HRA risk matrix.
2 Methods
This was a cross-sectional descriptive study conducted among 216 call center workers
from some companies in Khon Kaen province, Thailand. The survey was based on
inclusion criteria that call center workers had experienced their jobs for more than 6
months, working time at least 32 h per week with a computer at least 4 h per day.
Workers who had musculoskeletal disorders at the present time of study were excluded
from the study.
2.1 Research Tool
Rapid Office Strain Assessment (ROSA) [11] was applied for considering the sitting
posture, work station (chair height, pan depth, armrest and back support), computer
(monitor, mouse and keyboard), telephone and duration of spending time for each
posture or activities. ROSA had high inter and Intra-observation reliability (ICCs) which
were 0.84 and 0.86, respectively. It was used to identify probability of ergonomics
hazard exposure. There were 4 risk level classifications that were low (score 1–2 points),
moderate (score 3–4 points), high (score 5–7 points) and very high (score 8–10 points).
Cornell Musculoskeletal Discomfort Questionnaires (CMDQ) applied from Hedge
et al. [12] was used to identify severity of discomfort perception from ergonomics
hazard exposure with the reliability of 0.986. Discomfort or pain level were considered
by frequency of pain (never, 1–2 times per weeks, 3–4 times per weeks, once daily,
374 S. Chaiklieng and P. Suggaravetsiri
more than once daily), discomfort (mild, medium, severe) and the rubs of pain from
work (never, likely, medium). There were 5 levels of discomfort that were no dis-
comfort (score 0 points), mild (score 1–5 points), moderate (score 6–13.5 points), high
discomfort (score 14–39.5 points) and severe discomfort (score 40–90 - points).
Matrix of health risk level were applied from previous study [10] considering
probability (ROSA level) and severity of discomfort (from CMDQ level) which was
finalized to be 5 levels; low risk (score 0–2 points), acceptable risk (score 3–4 points),
moderate risk (score 6–8 points), high risk (score 9–12 points) and very high risk (score
15–20 points).
2.2 Data Analysis
Data were analyzed by STATA program version 10.1 to show the number and percent
of personal characteristic, work characteristic, and health risk level. Pearson Product
Moment Correlation Coefficients were used as measures of correlations between a
health risk score and CMDQ scores and health risk scores and ROSA scores. This
study obtained ethical approval from the Khon Kaen University Ethics committee,
Thailand, no. HE572131.
3 Results
3.1 Personal and Work Characteristic
Among the 216 call center workers, 75.9% were female, and 54.6% were aged between
25-29 years (median = 26 years). Most workers had graduated with a bachelor’s degree
(85.7%). Regarding the health status of the workers, 40.2% exercised for at least 30 min
3 days a week, and most (84.3%) had no chronic diseases. A large number of call center
workers (74.5%) had work experience for 1–3 years (median = 1 year), 76.4% worked
for 8 h per day, and 86.6% worked continuously with computer for 4–8 h per day
(median = 8 h). Most (70.0%) had telephone calls with customers at the rate of 6–10
times per hour (median = 5 times), and 64.8% spent 5 min on each call.
3.2 Ergonomics Risk and Muscular Discomfort
Call center workers were exposed to only two ergonomics risk levels measured by the
ROSA: these were a high risk level (n = 113 or 52.4%) and a moderate risk level
(n = 103 or 47.6%). Discomfort identified by the CMDQ was mostly found at a mild
level (40.3%), followed by a moderate level (35.2%) when no specific area of neck,
shoulders and back (NSB) was considered. For moderate and mild discomfort, the
volume of complaints was highest for the back area, followed by the shoulder and then
the neck areas.
3.3 Health Risk Levels for Neck, Shoulder and Back Pain
From the matrix of health risk assessment of neck, shoulder and back pain was con-
sidered at least one area of complain. The most of workers (44.9%) had health risk at
Health Risk Assessment Matrix for Back Pain Prediction 375
moderate risk level which 24.1% had mild level of CMDQ and high level of ROSA.
Neck, shoulder and back pain risk was acceptable to low level for 33.7% and high risk
level was 21.3%, as shown in Table 1.
In terms of health risks associated with only one area of pain (neck or shoulder or
back), the results showed that the highest percentage of workers had a moderate health
risk for back pain (25.5%), followed by shoulder pain (22.2%), and 16.7% had a high
health risk associated with back pain. Details of the proportions of workers in each
health category according to region of pain are shown in Table 2.
Most call center workers had health risk on back pain from low risk to high risk
level (51.8%), follow by shoulder pain (39.3%) and neck pain (23.6%), respectively.
Table 1. Number (%) of call center workers on health risk of neck, shoulder and back pain
(n = 216)
Neck, shoulder ROSA level
and back pain low moderate high very high
Severe 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0)
high 0 (0.0) 8 (3.7) 7 (3.2) 0 (0.0)
CMDQ level
moderate 0 (0.0) 37 (17.1) 39 (18.1) 0 (0.0)
mild 0 (0.0) 35 (16.2) 52 (24.1) 0 (0.0)
Comfort 0 (0.0) 23 (10.6) 15 (6.9) 0 (0.0)
Comments: Very high risk Low risk
High risk Acceptable risk
Moderate risk
Table 2. Number (%) of workers classified by health risk level and region of pain complaint
(workers had chance of pain more than one region) (n = 216).
Health risk Region
Neck Shoulder Back
Unacceptable risk 0 (0.0) 0 (0.0) 0 (0.0)
High risk 13 (6.0) 19 (8.8) 36 (16.7)c
Moderate risk 26 (12.0) 48 (22.2)b 55 (25.5)a
Low risk 12 (5.6) 18 (8.3) 21 (9.6)
Acceptable risk 135 (76.4) 131 (60.7) 104 (48.2)
Comments: a = the first ranking, b = the second
ranking, c = the third ranking
376 S. Chaiklieng and P. Suggaravetsiri
Regarding the relationships between health risk scores and ergonomic risks and
discomfort levels for the back pain. The result showed a statistically significant positive
correlation between health risk and CMDQ scores (r = 0.756). A positive correlation
between the health risk and ROSA scores (r = 0.442) was also illustrated.
4 Discussion
For over 70% of 216 call center workers, the discomfort levels measured by the CMDQ
were reported to be at a mild to moderate level. Previous study [13] explained that
resting time of workers was reduced by durations of work.
Duration of work for 8 h per day (76.4%) and working in a sitting posture for more
than 6 h a day played a role in back pain development. Workers had rest periods less
than three times a day (87.6%) and 15 min each time (86.5%).
A previous study [14] found that insufficient rest periods were significantly cor-
related with MSDs. The ROSA showed that call center workers were exposed to only
two levels of ergonomics risk which were high and moderate risk caused by conditions
of the inappropriate work environment and awkward postures.
According to a previous report among these workers, 27.3% of them sat on a chair
which was too high and resulted in a knee angle of more than 90 degrees. Moreover,
53.7% of workers had unadjustable armrest, and 57.9% had no back support or were
leaning their body forward [15]. These work environments may contribute to the
development of neck, shoulder and back pain about which workers in the present study
complained. However, a limitation of the ROSA is that the outcome of the risk assessment
does not specify the body region associated with the levels of risk. The suggestion has
been that, when the ergonomic risk is above the moderate risk level, the office workstation
should be repeatedly modified until the risk is at an acceptable level [9].
Another possibility is to change awkward posture which is one of risk factors of
MSDs [2]. Moreover, Errico et al. [3] reported that unsafe workstations were signifi-
cantly correlated with MSDs.
The present study found that majority of call center workers had moderate to high
level health risks for neck, shoulder and back pain. In the health risk matrix some call
center workers were assessed as having a low level of health risk even though their
CMDQ score was zero and indicated no recent or current discomfort. However, when
their ergonomic risk on the ROSA was high, these workers might reasonably be expected
to develop MSDs problems to at least a low level at some time in the future [10].
Health risk score had a linear regression with CMDQ and ROSA, that the higher
correlation of health risk scores and CMDQ scores was related to discomfort perception
of workers. This study could therefore apply to health risk assessment matrix on
prediction of back pain by combining two approaches of objective assessment and
subjective assessment for office ergonomics [16].
Health Risk Assessment Matrix for Back Pain Prediction 377
5 Conclusions and Suggestions
Most workers were exposed to ergonomic risk factors (ROSA) at a high or moderate
level (52.4% and 47.6%, respectively). The CMDQ levels were mostly reported at the
mild or moderate discomfort level (40.3% and 35.2%, respectively). However, they did
have a health risk for neck, shoulder and back pain at the moderate or high risk level
(44.9% and 21.3%, respectively). The highest proportion of workers with high or
moderate health risks was specifically the health risk of back pain (42.2%). The linear
correlation between health risk score and CMDQ score (r = 0.756) was statistically
significant. Call center workers can be predicted to develop neck, shoulder and back
pain from exposure to ergonomic risk factors by application of the HRA matrix which
considers both ergonomic risks (objective assessment) and self-reported
aches/discomfort/pain (subjective assessment). For prevention, risk should be at
acceptable level by reducing ergonomics risks either by the improvement of work-
stations or sitting posture. The health risk matrix can also be applied for neck, shoulder
and back pain surveillance in other workers engaged in similar work.
References
1. Subbarayalu, A.V.: Occupational health problems of call center workers in India: a cross-
sectional study focusing on gender differences. J. Manage. Sci. Pract. 2(1), 63–70 (2013)
2. Spyropoulos, P., Papathanasiou, G., Georgoudis, G., Chronopoulos, E., Koutis, H.,
Koumoutsou, F.: Prevalence of low back pain in Greek public office workers. Pain
Physician 10, 651–660 (2007)
3. Errico, A.D., Caputo, P., Falcone, U., Fubini, L., Gilardi, L., Mamo, C., et al.: Risk factors for
upper extremity musculoskeletal symptoms among call center employees. J. Occup. Health
52, 115–124 (2010)
4. Keawduangdee, P., Puntumetakul, R., Siritaratiwat, W., Boonprakob, Y., Wanpen, S.,
Thavornpitak, Y.: The prevalence and associated factors of working posture of low back
pain in the textile occupation (fishing net) in Khon Kaen province. Srinagarind Med. J.
26(4), 317–324 (2011)
5. Poochada, W., Chaiklieng, S.: Prevalence and neck, shoulder and back discomfort among
call center workers. Srinagarind Med. J. 30(4), 282–289 (2015)
6. Bramson, J.B., Smith, S., Romagnoli, G.: Evaluating dental office ergonomic risk factors and
hazards. J. Am. Dent. Assoc. 129(2), 174–183 (1998)
7. Dempsey, P.G., McGorry, R.W., Maynard, W.S.: A survey of tools and methods used by
certified professional ergonomists. Appl. Ergon. 36, 489–503 (2005)
8. David, G.C.: Ergonomic methods for assessing exposure to risk factors for work-related
musculoskeletal disorders. Occup. Med. 55, 190–199 (2005)
9. Chaiklieng, S.: Toxicology in Public Health. Khon Kaen University Printing House, Khon
Kaen (2014)
10. Chaiklieng, S., Krusan, M.: Health risk assessment and incidence of shoulder pain among
office workers. Procedia Manuf. 3, 4941–4947 (2015)
11. Sonne, M., Villalta, D.L., Andrews, D.M.: Development and evaluation of an office
ergonomic risk checklist: ROSA a rapid office strain assessment. Appl. Ergon. 43, 98–108
(2012)
378 S. Chaiklieng and P. Suggaravetsiri
12. Hedge, A., Morimoto, S., McCrobie, D.: Effects of keyboard tray geometry on upper body
posture and comfort. Ergonomics 42(10), 1333–1349 (1999)
13. Chaiklieng, S., Suggaravetsiri, P., Boonprakob, Y.: Work ergonomic hazards for muscu-
loskeletal pain among university office workers. Walailak J. Sci. Technol. 7(2), 169–176
(2010)
14. Chalardlon, T.: Work-related musculoskeletal injuries and work safety behaviors among call
center workers. Nurs. J. Ministry Public Health 23(1), 44–59 (2013)
15. Poochada, W., Chaiklieng, S.: Ergonomics risk assessment among call center workers.
Procedia Manuf. 3, 4613–4620 (2015)
16. Chiropractic, C.: Office Ergonomics. http://colonychiro.com/office-ergonomics/
Towards Conceptual New Product
Development Framework for Latvian ICT
Sector Companies and Startups
Didzis Rutitis(&) and Tatjana Volkova
BA School of Business and Finance, Kr. Valdemara 161, Riga 1063, Latvia
{Didzis.Rutitis,Tatjana.Volkova}@ba.lv
Abstract. The purpose of this paper is to propose outline for a new product
development framework to be utilized by Latvian ICT companies and tech-
nology startups. Design/methodology: To achieve this objective, semi-structured
expert interviews with 12 executives from Latvian ICT companies and startups
were held to identify common factors for these companies that have contributed
to the success of new product launch in the market. Findings: The paper sum-
marizes new product development success factors, outlines organizational
management aspects to be implemented for improving product development
process from the managerial perspective, and suggests possible metrics that
should be used to measure these factors. Practical implication: the insights
gathered for the intended new product development framework could be also
applied by managers of established and mature ICT companies to ensure suc-
cessful internal transformation with aim to maintain competitiveness against
emerging startups and promote internal entrepreneurial activities not only in
Latvia, but also abroad.
Keywords: Startups Product development Company growth
1 Introduction
New product development is an important prerequisite for improvement of competi-
tiveness of almost every ICT sector company and particularly for companies repre-
senting ICT services activities category (software development, telecommunications,
consulting data processing segments). This has been the fastest growing category
within ICT sector and has exceeded turnover of hardware wholesale in 2016 [1].
Research of product development is essential because of rapidly evolving technological
advancements and the speed at which new products are being delivered to market by
both, ICT sector high-growth participants like software startups, fintech companies and
other technology startups, and also mature ICT companies. The continuous innovation
and new product development are not only the growth opportunities, but also a
question of survival. Thus, employees directly and indirectly involved in new product
development contribute a critical part of efforts aimed for maintaining the company
afloat and running.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 379–384, 2020.
https://doi.org/10.1007/978-3-030-39512-4_59
380 D. Rutitis and T. Volkova
2 Methodology of Research
This paper reflects results from a study aimed to examine product development
methods and principles used by the ICT sector companies and technology (generally,
software) startups. In the context of this research, authors refer to a startup as an
organization formed to search for a repeatable and scalable business model, before it
has turned into an enterprise with a traditional management structure [2].
Previous studies and literature reviews on new ICT product development [3],
startup growth acceleration from MVP to commercialization stage [4–6] and further
scaling [7] were analyzed to obtain theoretical basis and preliminary insights into the
main product development process to be discussed during the interviews.
The collection of insights regarding product development principles was imple-
mented by applying expert method and using semi-structured interviews for data
collection. Twelve experts (see Table 1) representing various Latvian ICT companies
(from software startups in early growth stage to mature telecommunications company)
and also one commercial bank (its division in charge of digital channels and partner-
ships with the fintech software startups). The information in table also indicates the
number of employees directly involved in new product development, including man-
agers, dedicated software developers, data analysts and designers.
Table 1. Description of experts.
Position of expert in a company ICT industry sector and sub-segment Product
team size
1. CEO Startup (Productivity software) 5
2. CEO Startup (Construction software) 3
3. CEO Productivity software 14
4. CEO Software development 4
5. Head of business development Software development 34
6. Head of product management Fintech software 12
7. Co-founder, VP of product Fintech software 22
8. Head of product management Fintech software 32
9. Head of product management Fintech software 12
10. Head of product management Manufacturing software 15
11. Head of corporate Telecommunications 50
development and new products
12. Head of digital channels Commercial bank - department in charge of 15
digital channels and partnerships with fintech
software companies
During the interviews, experts were asked to describe the actual product devel-
opment process taking place at their company, frameworks and tools used by the
product team and other employees involved into new product development processes,
metrics used for the evaluation and supervision of the product development process.
Towards Conceptual New Product Development Framework 381
Interviews were held and data was collected between December 2018 and May
2019.
3 Background
3.1 The Role of Product Management
At high level, a product manager is the single cross-functional owner directly
responsible for the success of a product. They have all the responsibility for the
product’s success, but often lack the direct line reporting of the other functions. Product
managers are generally responsible for [7]:
(1) Product strategy and vision;
(2) Product prioritization and problem solving;
(3) Execution: timelines, resources, and removal of obstacles;
(4) Communication and coordination (overlays all of the above).
In order to build a great product organization, the company needs to understand the
role of the product manager; to hire individuals with the right skill sets, including a
strong VP of product (for startup); and establish a simple set of processes to enable the
product organization and help the company scale its product development [7].
Great product management organizations help a company set product vision and
road maps, establish goals and strategy, and drive execution on each product throughout
its lifecycle. In contrast, bad product management largely function as project manage-
ment groups, running schedules and tidying up documents for engineers [8].
From such perspective, this research contributes to better understanding of success
factors leading to successful commercialization of technology products (mainly, soft-
ware) and successful new product creation within the ICT sector.
3.2 Selection of Question for Discussion with Experts
After carrying out literature review on new ICT product development related research,
the authors have derived the following factors to be discussed and evaluated by experts
during semi-structured interviews:
• Product development frameworks. What product development frameworks are used
by the product team and other employees involved into new product development
processes?
• Time. Is there any pattern in terms of timing and length of product development
cycle for new product development and pivoting until reaching commercialization
stage and achieving successful traction for new products?
• Innovation tools used by product teams. Which innovation tools and software are
used by product managers and product teams during daily work?
• Managerial processes. What other organizational procedures and processes are
implemented or utilized by product teams?
• Measurement of traction. What metrics and techniques are used for the product
development management supervision and evaluation?
382 D. Rutitis and T. Volkova
4 Research Results
4.1 Product Development Frameworks
Experts confirmed general use of such general ICT product development frameworks
as Lean Startup, Agile, Scrum, and Waterfall within their respective companies. Next,
SAFe (Scaled Agile Framework) for Lean Enterprises was recognized as a relevant
methodology incorporating large company needs and existence of different managerial
levels with different requirements [10]. Another expert highlighted effectiveness of 5-
day Sprint method by Google Ventures [11] to test hypothesis behind new product idea
promptly before investing additional efforts for prototype development and the fol-
lowing launch to the market.
Several experts indicated that they tend to adapt the experience of large technology
giants and test their usability within Latvian environment for their respective compa-
nies. Among outlined were Spotify for their engineering approach, which is known as a
“Spotify Model” [12]. Another expert confirmed internal knowledge and best practice
exchange with product development leads from technology “unicorn” companies like
Pipedrive and TransferWise.
4.2 The Use of Innovation Tools
The use of such innovation tools as Business Model Canvas, Value Proposition
Canvas, Storyboards, Kanban and Trello boards was identifies as rather common
among both, established ICT companies and also startups to generate new business
models and ideas in the idea generation stage.
Productivity and project management tools like Basecamp, Asana and Trello are
used to categorize product development related information and follow deadlines
during product or software prototype development. Also, the traditional Microsoft
Office software (Word, Excel) is used to create product backlogs that are used for
planning purposes, bi-weekly, quarterly and yearly strategy setting meetings.
4.3 Timeline
Experts mentioned a time period of 3–6 months as the approximate time necessary for
implementing a new pivot at a startup company. Several fintech companies indicated
that their first prototype actually showed poor results right after the launch. Within
approximately 6 months period another pivot was implemented, and it has resulted in a
business model that has been developed further on. The initial business model was
either terminated or separated into another legal entity apart from the core business. For
larger companies, new product development cycles tend to be longer (starting from 6
months) due to larger number of internal procedures that prevent implementation of
purely iterative process and prompt product update during development stage and also
after the launch of the product. However, they tend to encounter such deficiencies with
establishment of such business units that are enabled to work using iterative approach,
build and test product prototypes using Lean Startup approach within an isolated sand-
box environment.
Towards Conceptual New Product Development Framework 383
4.4 Managerial Processes
Several experts noted that the product management teams perform their tasks using
matrix organizational structure, which usually involved product managers, designers,
software developers. Depending on the organizational structure and the company size,
the technical team can represent either a core team and share pooled resources, or there
can be separate teams for maintaining core functions and developing new products.
Stock option system has been used by one of the software companies to align
decision-making by different department managers with the strategic and business
goals of the company and eliminate ego-centric behavior that might conflict with these
corporate business goals.
Experts also noted a need for properly managed company culture and importance of
hiring the right people with successful product development track record, which
confirms initially outlined conceptual assumptions from the theory [7].
4.5 Measurement of Traction
While majority of experts indicated that the main indicator of product development
success can be related to the increase in number of users or additional revenue gen-
erated by the new product after it is launched on the market, one of the experts
indicated that the company (in this case – large and established enterprise with
numerous products) may launch a new product that minimizes costs internally, instead
of generating additional revenue. Additional revenue can be a positive externality in
such case.
Another important thing to take into account is the development stage of the
company – in the stage of business idea generation the criteria for evaluation could be
ROI or simple net present value (NPV) calculation, followed by business case. For very
early stage startups it can be interest from prospective customers or demo customer
feedback and readiness to pay for the product. For later stage startups this relates to
growth of user base, turnover per user, increased consumption (time spent) using the
product, etc., which coincides with findings from the literature review [9].
5 Conclusions
The research results indicate that new product development processes differ from
company to company and reflect a rather unique blend for each company.
There are different organizational processes taking place if one compares new
product development processes and related procedures (formal and informal) within an
established ICT company and a newly established startup that operates fully in a Lean
Startup mode. Established enterprise level companies are likely to deploy a mix of
Waterfall for existing products and Lean Startup approach to develop new offerings,
while early stage startups are more likely to rely fully on a Lean Startup approach to
identify a scalable business model and achieve necessary traction. From the managerial
perspective, the challenge for mature companies is to cultivate the innovative spirit and
384 D. Rutitis and T. Volkova
overcome objections of employees working with standardized and established business
lines when approached by new product managers to work
The factors to be included in framework for new product development by ICT
companies should include: use of specific product development methodologies by the
product team or the entire company (in case of an early stage startup); the minimum
time for new pivot implementation and testing to validate new product hypotheses; use
of innovation tools to generate new business models and ideas; organizational man-
agement - integration and alignment of new product initiatives with marketing and
technical development teams; availability of financial resources to finance the pivot or
fundamentally new product development; consistent measurement of product devel-
opment and the following traction after the launch to the market with relevant metrics.
The relative importance and nature of each factor is directly correlated with a stage of
product (or company, in case of a startup) development.
Acknowledgments. This research has been funded by the European Union Post-Doctoral
program. Project number: 1.1.1.2/VIAA/1/16/089.
References
1. Central Statistical Bureau of Latvia: ICT industry turnover. https://www.csb.gov.lv/en/
statistics
2. Blank, S.: What’s A Startup? First Principles. https://steveblank.com/2010/01/25/whats-a-
startup-first-principles/. Accessed 10 Apr 2019
3. Rutitis, D., Volkova, T.: Product development methods within the ICT industry of Latvia.
Vadyba J. Manag. 1(34), 15–23 (2019)
4. Nguyen-Duc, A., Shah, S.M.A., Abrahamsson, P.: Towards an early stage software startups
evolution model. In: 2016 42nd Euromicro Conference on Software Engineering and
Advanced Applications (SEAA), pp. 120–127 (2016)
5. Nguyen-Duc, A., Abrahamsson, P.: Minimum viable product or multiple facet product? The
role of MVP in software startups. In: Sharp, H., Hall, T. (eds.) Agile Processes, in Software
Engineering, and Extreme Programming, XP 2016. Lecture Notes in Business Information
Processing, vol. 251, pp. 118–130. Springer, Cham (2016)
6. Berg, V., Birkeland, J., Nguyen-Duc, A., Pappas, I., Jaccheri, L.: Software startup
engineering: a systematic mapping study. J. Syst. Softw. 144, 255–274 (2018)
7. Gil, E.: High Growth Handbook. Stripe Press, San Francisco (2018)
8. Horowitz, B.: Good Product Manager/Bad product Manager (2012). https://a16z.com/2012/
06/15/good-product-managerbad-product-manager/
9. Bhuyian, N.: A framework for successful new product development. J. Ind. Eng. Manag. 4
(4), 646–770 (2011)
10. SAFe: Scaled Agile Framework. https://www.scaledagileframework.com/about/
11. Knapp, J., Zeratsky, J., Kowitz, B.: Sprint. How to Solve Big Problems and Test New Ideas
in Just Five Days. Simon & Schuster, New York (2016)
12. Spotify Engineering Culture. https://blog.crisp.se/2014/03/27/henrikkniberg/spotify-
engineering-culture-part-1
A Liveness Detection
Method for Palmprint Authentication
Ayaka Sugimoto1, Yuya Shiomi1, Akira Baba2, Norihiro Okui2,
Tetushi Ohki1, Yutaka Miyake2, and Masakatsu Nishigaki1(&)
1
Shizuoka University,
3-5-1, Johoku, Naka, Hamamatsu, Shizuoka 4328011, Japan
[email protected]
2
KDDI Research, Inc.,
Garden Air Tower, 3-10-10, Iidabashi, Chiyoda, Tokyo 1028460, Japan
Abstract. Many smartphones are equipped with a biometric authentication
function to prevent their unauthorized use. An authentication method using
palmprint as a physical feature has been proposed. Palmprints can be authenti-
cated without special devices because the features of palmprints can be acquired
using a smartphone camera. However, impersonation may be performed using
user photographs. In this study, we examine a liveness detection method against a
presentation attack in palmprint authentication using a smartphone. Assuming
impersonation using familiar media, we consider attacks using printed and dis-
played images. In the attack image of the printed and displayed palms, we focus
on the characteristics of resolution degradation caused by ink bleeding and the
moiré (interference fringe) caused when capturing the attack palm.
Keywords: Palmprint authentication Liveness detection Presentation attack
1 Introduction
In the current digital transformation era, almost all users own smartphones and a variety
of social activities are performed online. A significant portion of shopping and banking
has already shifted online; in these services, smartphones play an important role as the
user terminal. Accordingly, users’ important information, such as credit card infor-
mation and electronic money information, is stored in their own smartphones. There-
fore, smartphones are critical devices that must be strictly protected.
Recent smartphones are equipped with various security mechanisms to ensure that
only a legitimate user can activate the smartphone. Among these mechanisms, bio-
metric authentication is expected to be the most promising option, as smartphones do
not have a keyboard and entering a password into such tiny terminals is difficult.
However, biometric authentication is highly susceptible to presentation attacks with
fake biometric samples. Thus, this issue must be solved for the practical use of
smartphones in online services.
The studies of [1, 2] have reported that fingerprints can be reproduced using hand
photographs. Hence, this motivated us to investigate the tolerance (i.e., liveness
detection ability) of palmprint authentication [4] against presentation attacks using
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 385–391, 2020.
https://doi.org/10.1007/978-3-030-39512-4_60
386 A. Sugimoto et al.
printed/displayed palmprint images. In the first of our investigation, this paper focuses
on impersonation using palmprint images that are printed on a paper or displayed on a
liquid crystal screen. The printer used for printing is a commercially available product,
and the media to be printed are a plain paper and a glossy paper. The device used for
displaying is a commercially available high-definition smartphone display.
2 Attack Method
A presentation attack using familiar media is assumed in this study. The characteristics
of each medium from the viewpoint of liveness detection are described below.
• Plain Paper
When printer ink is sprayed on plain paper, the ink bleeds. Consequently, when the
“plain paper attack palm (palmprint image printed on plain paper)” is captured using
a smartphone camera, the resolution of the obtained image (hereinafter called “plain
paper attack image”) is degraded compared with the palmprint image of a “real
palm” captured using a smartphone camera (hereinafter called “real image”).
• Glossy Paper
In the case of glossy paper, because the ink bleed is minimal, fine ink particles are
regularly arranged on the paper surface. However, digital cameras also have a
structure in which image sensors are regularly arranged. Hence, when the “glossy
paper attack palm (palmprint image printed on glossy paper)” is captured using a
smartphone camera, partial moiré (interference fringes) is induced in the obtained
image (hereinafter called “glossy paper attack image”).
• Display
The display has a structure of regularly arranged light emitting elements. Hence, as
in the case of glossy paper, when a “display attack palm (palmprint image displayed
on the display)” is captured using a smartphone camera, moiré (interference fringes)
is induced in the obtained image (hereinafter called “display attack image”).
Figure 1 illustrates examples of attack images. The attack image size is set to
300 300 pixels.
Fig. 1. Palmprint image area of attack images (Right: Plain paper, Center: Glossy paper, Left:
Display)
A Liveness Detection Method for Palmprint Authentication 387
3 Image Processing
This study aims to enhance biometric authentication used for activating smartphones.
Due to the limited computational resources of smartphones, liveness detection should
be implemented by a lightweight process. Therefore, in this paper, we investigate how
liveness detection can be achieved by simple image processing, such as Sobel filters
and autocorrelation.
3.1 Sobel Filter
The Sobel filter is a first-order differential filter. As described in Sect. 2, plain paper
attack images have palmprint edges that are slightly blurred owing to resolution
degradation. Glossy paper attack images and display attack images are characterized by
the presence of edges that do not exist in the palmprint of real images because of moiré.
Therefore, these features are made obvious by performing edge detection using the
Sobel filter.
Sobel filter with a kernel size of K = 7 is applied to both the real and attack images
in both the X and Y directions. Furthermore, edge enhancement is realized by bina-
rizing the filtered image. A binarization method using a common threshold for all
pixels and method by setting a threshold for each local region (Otsu’s binarization [3])
are applied. In the following, Process1 is the binarization method using the common
threshold for the result of applying the Sobel filter, and Process2 is the Otsu’s bina-
rization for the result of applying the Sobel filter. Figure 2 illustrates an example of the
results by applying Process1 and Process2.
3.2 Autocorrelation
From the image of Process1 in Fig. 2, the sharpness is slightly reduced in the plain
paper attack image, the palmprint pattern is slightly collapsed in the glossy paper attack
image, and moiré lines in the horizontal direction are appeared in the display attack
image. In this study, we used autocorrelation to detect them. Specifically, using the
50 50 pixel area of the center of the palmprint image as a template, we calculated the
error (sum of squared differences in brightness) from the area shifted by 1 pixel in the X
direction from 10 pixels to 50 pixels (Fig. 3).
In the plain paper attack image, the sharpness is slightly reduced; therefore, the
autocorrelation is expected to be higher (the error will be smaller) than that of the real
image. In the glossy paper attack image, as the pattern is messy, the autocorrelation is
expected to be higher (the error will be smaller) than that of the real image. In the
display attack image, as moiré lines in the horizontal direction are generated, the
autocorrelation is expected to increase periodically (the error will be smaller
periodically).
The autocorrelation for the image of Process2 is calculated in the same manner.
388 A. Sugimoto et al.
Real image Plain paper attack Glossy paper Display attack
image attack image image
Process1
Process2
Fig. 2. Result of applying the Sobel filter
Fig. 3. Method to obtain autocorrelation
3.3 Liveness Detection Procedure
Here describes the liveness detection procedure when liveness detection is performed
after palmprint authentication.
I.Capture palm image using a smartphone camera
II.Create palmprint image by extracting the palmprint area
III.Perform palmprint authentication
IV. Apply the Sobel filter to the palmprint image that has been successfully
authenticated, and create the images of Process1 and Process2 by binarization
V. Obtain the autocorrelation of the images of Process1 and Process2
A Liveness Detection Method for Palmprint Authentication 389
4 Experiment
Preliminary experiments were conducted to confirm how liveness detection was
achieved by Sobel filters and autocorrelation.
4.1 Dataset
The digital camera equipped in Nexus 5 of LG Electronics was used for image capture
(corresponding to Step I in Sect. 3.3). The experimenter (the author) visually extracted
the palmprint region from the photographed image (corresponding to Step II in Sect. 3.3).
The size of the palmprint area was 300 300 pixels.
By applying palmprint authentication in advance and excluding images that do not
pass authentication from the experiment, the total number of palmprint images that are
subject to liveness detection experiments can be reduced. Thus, the experimenter (the
author) visually inspected the image set and excluded images that did not show
palmprints or those that were out of focus, i.e., images that did not pass the palmprint
authentication (corresponding to Step III in Sect. 3.3). Table 1 lists the number of
images that passed the palmprint authentication. In this dataset, the number of real
images that passed the palmprint authentication was only 104 out of 400, and no image
passed the palmprint authentication in the glossy paper attack and display attack
images. This means that the photographing was not adequate enough in this experi-
ments and we will have to collect images again in the future study.
Table 1. Dataset
Resolution of Total number of Number of pass
attack palms images authentications
Real image ― 400 104
Plain paper attack 2400 1200 380 27
image dpi
Glossy paper attack 9600 2400 221 0
image dpi
Display attack image 1080 1920 400 0
pixels
4.2 Results
At Step V in Sect. 3.3, the error between each image area and the template image was
calculated while shifting the template image by 1 pixel from 10 pixels to 50 pixels in
the X direction. Figures 4 and 5 illustrate the changes in the errors of the real and attack
images in Process1 and Process2, respectively. In Figs. 4 and 5, only the characteristic
image results are selected and presented for better visibility.
As illustrated in Fig. 4, with regard to the image of Process1, a method could be
adopted in which the threshold is set to 0.430 and the image whose minimum value in
the autocorrelation score graph is higher than the threshold is judged as a real image.
390 A. Sugimoto et al.
As depicted in Fig. 5, with regard to the image of Process2, a method could be adopted
in which the threshold is set to 0.245, and the image whose maximum value in the
autocorrelation score graph is higher than the threshold is determined as a real image.
In addition, liveness detection by AND-type combination of the two results could be
adopted. The results are as summarized in Table 2.
Table 2. Number of images judged as real images
Number of pass Process1 Process2 Combine
authentications
Real image 104 74 (71.2%) 92 (88.5%) 71 (68.3%)
Plain paper attack image 27 7 (26.0%) 10 (37.0%) 5 (18.5%)
Fig. 4. Results for Process1
Fig. 5. Results for Process2
5 Conclusion
We investigated the possibility of liveness detection based on simple image processing
for palmprint authentication using images captured with a smartphone. This paper
detected liveness using the Sobel filter and autocorrelation, assuming a presentation
attack using a printed paper and display.
This study is still being analyzing how image processing can be used for liveness
detection and has not yet been evaluating unlearned samples. In addition, processing
related to palmprint area extraction and palmprint authentication was performed
manually. In the future, we plan to further investigate the processing thereof.
A Liveness Detection Method for Palmprint Authentication 391
References
1. Echizen, I., Ogane, T.: BiometricJammer: method to prevent acquisition of biometric
information by surreptitious photography on fingerprints. IEICE Trans. Inf. Syst. E101-D(1),
2–12 (2018)
2. Chaos Computer Club: Fingerprint biometric hacked again. https://www.ccc.de/en/updates/
2014/ursel
3. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man
Cybern. 9(1), 62–66 (1979)
4. Ota, H., Aoyama, S., Watanabe, R., Ito, K., Miyake, Y., Aoki, T.: Implementation and
evaluation of a remote authentication system using touchless palmprint recognition.
Multimedia Syst. 19(2), 117–129 (2013)
Procedure for the Implementation
of the Manufacturing Module of an ERP
System in MSME. Applied Case: Textile
“Tendencias” Enterprise, UDA ERP
Pedro Mogrovejo1(&), Juan Manuel Maldonado-Maldonado1,3,
Esteban Crespo-Martínez1,2,3, and Catalina Astudillo2,3
1
Escuela de Ingeniería de la Producción, Universidad del Azuay,
Cuenca, Ecuador
[email protected],
{jmaldonado,ecrespo}@uazuay.edu.ec
2
Escuela de Ingeniería en Ciencias de la Computación,
Universidad del Azuay, Cuenca, Ecuador
[email protected]
3
LIDI, Universidad del Azuay, Cuenca, Ecuador
Abstract. Innovation and competitiveness are key factors for the sustainability
of companies in any market in which they develop; the implementation of
technological tools is invaluable for obtaining these two factors at the same time.
Knowing that in Ecuador, MSMEs (Micro, Small and Medium Enterprises)
represent 99.55% of the productive sector, the need to offer an ERP (Enterprise
Resource Planning) with easy access and adaptability is emphasized. This
software also requires the commitment of the management and multiple efforts
by the staff of the organizations. This study summarizes the approach of a
generic methodology, with different useful tools that will allow the subsequent
adaptation of the software in a MSME.
Keywords: ERP MSME Competitiveness Companies Technological
Tool
1 Introduction
Innovation and competitiveness are key factors for the sustainability of companies over
time, regardless of the field in which they develop; for this reason, the implementation
of technological tools or ICTs allows to obtain an added value over their competence,
in addition to generating a continuous improvement within the organization. According
to data from [1], the companies considered MSMEs (Micro, Small and Medium
Enterprises), represent 99.55% of the productive sector, which is why business soft-
ware such as ERP (Enterprise Resource Planning) Easy access and adaptability will
allow these companies to generate added value over competitors and in turn gain a
competitive advantage.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 392–397, 2020.
https://doi.org/10.1007/978-3-030-39512-4_61
Procedure for the Implementation of the Manufacturing Module 393
In a significant effort, since 2015, the University of Azuay, as part of its research
program, has been developing the UDA ERP enterprise resource management soft-
ware, initially linking two engineering schools: Systems and Telematics, and Pro-
duction and Operations. UDA ERP is a business management software that works with
cloud computing technology, minimizing the computational requirements on the client
side.
In order to offer a solution to this business need, a study of a methodological guide
for these organizations was proposed, which contribute significantly to the Ecuadorian
economy. This methodology has been exposed in a generic way and validated with the
implementation of the UDA ERP software in a MSME in Cuenca, dedicated to the
production of textiles.
The proposed methodology includes several tools: for information gathering,
organizational management, process management and improvement; the same that
must be implemented within the different organizations, standardizing their processes,
to subsequently adopt an ERP, which will require companies to have their processes
and information well organized that allows the system to deliver results that serve the
correct decision making.
Considering the magnitude of a computer system and added value as a business
contribution, according to [2], this system is a formal set of processes that, operated on
a structured data collection according to the needs of the company, collects, elaborates
and distributes selectively the necessary information for its operation, facilitating the
management and control activities.
On the other hand, considering that global trends and the importance of keeping
data safe under parameters of confidentiality, availability and integrity of the data
generated by companies, these are loaded and processed in the so-called computational
cloud, which at the same time adds a component of simplicity in access and use for
those interested in the organization. The adequate protection of these leads to
strengthening the correct decision-making and generation of business strategies Crespo
[3]. According to [4], the ERP is a computer system used to identify and plan the
organization’s resources and thus manage customer orders in accordance with inven-
tories and organizational processes, that is, it is a system that holistically encompasses
really important information for the organization that uses it.
This article is divided into 7 sections: (i) the state of the art, where works by other
authors related to research are analyzed; (ii) the methodology, which describes the
process carried out to achieve the results; (iii) the results obtained after the application
of the proposed and described methodological guide; (iv) the discussion, as part of the
validation of the methodology proposed in the implementation of the ERP software;
(vi) the general conclusions and work to be done in the future; (vii) the bibliography
and sources consulted.
2 State of Art
According to Badenas [5], an ERP is a system that integrates the information and
processes of an organization in a single environment, which is usually made up of
modules of: Material Requirement Planning, Customer Relationship Management,
394 P. Mogrovejo et al.
Finance Resource Management, Human Resource Management and Supply Chain
Management. In addition, some benefits of ERP according to Monk and Bret [6], are
summarized in the easy global integration of data, the integration of people with data
and the elimination of the need to update and repair many computer systems separately
and that Management really manages business operations and resources, and is not
limited to its moderation alone. In addition, according to Ptak [7] most enterprises can
expect to change their computer information system either to a new system or have a
major upgrade at least every 3 to 5 years.
On the other hand, according to the Internal Revenue Service (by its acronym in
Spanish SRI), a MSME is known as “the group of micro, small and medium enterprises
that, according to their sales volume, social capital, number of workers, and their
production level or assets have characteristics of this type of economic entities”.
In addition, [8] argue that, companies use IT (Information Technologies) mainly to
improve the efficiency of external processes with customers, as well as sales and image
of the company abroad. In turn, they state that the companies that have invested the
most in IT and/or that have the greatest endowment of these technologies are the ones
that have obtained the most positive improvements, in addition they state that the
companies that have the greatest endowment of some ITs are larger companies.
An important point in the implementation of an ERP system is the standardization
that your data requires where [9], states that, to be part of such databases or informatics
systems, information has to be standardized. It must be codified according to a format
which is often established in its final form at the firm level, the global level.
Some of the conflicts and circumstances of the development of this type of software
can be reviewed in [10], where the development of the ERP UDA is mentioned.
3 Method
The present work began with the planning of the field work that should be practiced
within the selected company, in turn the context in which it is developed was analyzed
in addition to the possible factors that would affect or condition a MSME in the
adaptation of a computer system
Once the environment surrounding these types of companies was known, the
company was visited under study, where the technical team visited all areas of the
company, discussed with its employees. After this first visit, a meeting was held at the
University of Azuay where the witnesses are analyzed together with the technical and
development team of the UDA ERP, to plan again the actions to be used and inves-
tigate about the most suitable and adaptable tools for a MSME.
Multiple management tools were applied, other engineering tools, but above all
techniques and tools to obtain information; information with which the initial situa-
tional analysis was subsequently carried out. Using the tools classified in Table 1, the
analysis and situational diagnosis of the company was carried out. For this, it was
visited two to three times a week, for a total of 12 weeks.
For the analysis and processing of data, formats of the proposed tools were applied,
in addition to different sources of MSMEs that were developed in a context like that of
the company under study.
Procedure for the Implementation of the Manufacturing Module 395
Table 1. Tools applied in the study
Gathering Organizational Process Improvement
information management management
Interview Porter value chain Process diagram 5’s
Snap picture Process interaction Process flow Brainstorm
matrix diagram
Time analysis SWOT analysis Route diagram Ishikawa
diagram
Procedure sheets Machine-Man Cycle PDCA
analysis
Value added Kanban
analysis
Demand prevision
Inventory
management
Subsequently, the data not relevant for decision-making was purified and elimi-
nated, the technical tools such as flowchart, route diagram, time analysis, value-added
analysis, etc., were applied, and then the degree of usefulness of the information to
finally generate the improvement proposals for the company.
The tools selected and proposed for use were socialized with those involved in the
management of the business processes, which in this case were the managers and their
subordinates, in which the importance of them, their operation and results were
emphasized. That can be achieved with its adoption and proper use.
Finally, to validate the information obtained and the methodological guide, the
tools were applied by the company’s employees, confirming whether their under-
standing of them is adequate and if they are easily adaptable.
4 Results
An applicable methodology was obtained in different MSME companies, with tools
that encompass all their administrative and operational operations. The proposed
methodology defines the steps to follow so that a company in this sector achieves the
adoption of a comprehensive information system.
On the other hand, an optimal interaction was obtained to propose the method-
ological guide, evidencing its dynamism among employees, technicians of the survey
and data analysis, and developers of the UDA ERP, where constant improvements are
analyzed and proposed so that a MSME can obtain a competitive advantage with ERP
adaptation.
In addition, evidence was obtained of the lack of technification in companies of this
type, the requirements with the highest priority for the adoption of computer systems
and the barriers that prevent this activity.
396 P. Mogrovejo et al.
5 Discussion
For the development of the methodology, the use of electronic devices is essential for
the gathering of information, because, when taking photos, recording videos and audios
they allow a later analysis of the environment and the dynamics of the workers within
their functions, this is useful when the information is not documented and it depends on
the opinion of the manager and others involved.
On the other hand, in agreement with Crespo [3], considering the organizational
culture of the companies and their environment for the success of the implementation
of improvements is fundamental, so that the organizational management tools are the
perfect and necessary complement within the organizations for the achievement of
objectives, as in this case adopt an ERP information system, thus agreeing with
Badenas [5] and Monk [6], which will facilitate the organization’s operations by its
integrating function.
The implementation of the UDA ERP within this textile company, deduces that the
correct use of this computer system can determine a turning point within the organi-
zation, because being an integrative technological tool, depends on the proper func-
tioning of all its operations and the standardization of its processes as stated by [9],
both internal and external, taking into account that an ERP integrate all its stakeholders.
Performing the correct planning to propose a methodology applicable to MSME-
type companies allowed the company named “Tendencias”, to generate an organiza-
tional change and awareness of the advantages that TIs can bring to your business, in
accordance with the statement by [8].
6 Conclusions and Further Works
It is concluded that the proposed methodology can be implemented in different com-
panies in the MSME sector, so the generation of a baseline for its analysis is funda-
mental and that the achievement of results and the success of any type of methodology
is clearly linked to collaboration of those involved in this process, because today our
society and environment depends largely on people.
Finally, it is concluded that the incorporation of technical or engineering tools
requires trial and error so that the organizational culture adapts to them, so patience in
the process and technical support is essential; In turn, these initially basic tools will
allow the construction of a solid base of technification for a correct data collection, so
that in the future the evolution and implementation of more sophisticated tools can be
implemented in a simple way. Then the next works will be that the present method-
ological guide could be implemented in different enterprises, like services and manu-
facturing enterprises.
Procedure for the Implementation of the Manufacturing Module 397
References
1. EL UNIVERSO, Las mipymes representan el 99% de negocios en Ecuador, El Universo, 27
Junio (2019)
2. Trasobares, A.: Los sistemas de información: evolución y desarrollo, Revista de relaciones
laborales, pp. 149–165 (2003)
3. Crespo, E.: Una metodología para la gestión de Riesgos aplicada a las MPY-MEs. In:
INCISCOS, Quito, (2017)
4. Heizer, J., Render, B.: Dirección de la producción y de operaciones, Decisiones Tácticas.
Pearson Eduación, Madrid (2008)
5. Badenas, O.: Sistemas Integrados de Gestión Empresarial: Evolución histórica y tendencias
de futuro. Universitat Politècnica de València, Valencia (2012)
6. Monk, E., Bret, W.: Concepts in Enterprise Resource Planning. Course Technology Cengage
Learning, Boston (2013)
7. Ptak, C.: ERP Tools, Techniques, and Applications for Integrating the Supply Chain. CRC
Press Company, Boca Raton (2005)
8. Pérez, M., Martínez, Á., Carnicer, L., Vela, M.: LAS TIC EN LAS PYMES: ESTUDIO DE
RESULTADOS Y FACTORES DE ADOPCIÓN, Economia de la Información y la
comunicación: difusión e impacto de las TIC, pp. 93–106 (2006)
9. Mayére, A., Isabelle, B.: ERP implementation: the question of global control versus local
efficiency. In: ERP Systems and Organisational Change, pp. 47–58 (2008)
10. Astudillo-Rodríguez, C.V., Maldonado-Matute, J.M., Crespo-Martínez, P.E.: UDA-ERP:
theory, development and conflicts. In: Kantola, J., Nazir, S., (eds.) Advances in Human
Factors, Business Management and Leadership, AHFE 2019. AISC, vol. 961. Springer,
Cham, (2020)
Model of Emotionally Stained
Pupillogram Plot
Marina Boronenko(&), Yurii Boronenko, Oksana Isaeva,
and Elizaveta Kiseleva
Yugra State University, 16 Chekhov street, 628012 Khanty-Mansiysk, Russia
[email protected], [email protected],
[email protected], [email protected]
Abstract. The article proposes a model of an “emotionally colored portion of
the pupillogram” based on experimental data. Pupillograms were shot in people
of an older age category (50–75 years old), in which representatives of different
sexes were present. To call emotions, we used accident videos that are freely
available on the Internet. An analysis of the pupillograms showed waves of
attention. The structure of attention waves in pupillograms includes saccades
when you move your gaze, micro saccades when you focus on individual ele-
ments of the objects in question and the emotional component, provided that the
test object is significant for the individual. The peaks are well approximated by
the Gaussian function, which confirms. The emotionally colored portion of
the pupillogram can be described by the Gram-Charlier peak function. The
resulting expression can characterize the intensity and concentration of attention.
Keywords: Pupillogram Test object Spotlight Mathematical model
Emotion
1 Introduction
This is an example paper and template to duplicate exactly with respect to the required
format of your paper In the modern world, suicide is one of the ten leading causes of
population deaths; in terms of the frequency of suicidal cases, Russia ranks first in
Europe, with a maximum of teenagers and young people [1, 2]. According to the
forecast, by 2020, suicide will come in second place in the world as the cause of death,
second only to cardiovascular diseases.
Organizations use test methods, questionnaires, and questionnaires to prevent
suicides. Their use is one of the most common diagnostic tools used in assessing
suicidal risk. Recently, methods have begun to emerge for identifying risk groups using
the intellectual search for specific web activity, identifying risk groups in social net-
works. Networks [3]. It is generally accepted that depression is the cause of suicide [4].
It is known that the emotional reaction of a person on the same stimulus depends on
whether he is depressed. If you carefully monitor the person, then you can prevent
suicide attempts. But it is impossible for a person to exercise continuous control.
Therefore, it is necessary to use intelligent video surveillance systems. The video
surveillance systems installed in organizations are used to resolve disputes and identify
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 398–403, 2020.
https://doi.org/10.1007/978-3-030-39512-4_62
Model of Emotionally Stained Pupillogram Plot 399
the culprits in the event of an emergency, identify and suppress conflicts, prevent illegal
actions, and investigate crimes recorded on video. Obviously, for the prevention of
suicides, not only their upgrade is necessary, but also a modernization of the approach
to the analysis of the information received. To date, the resolution of video cameras
allows you to remotely monitor the dynamics of the size of the pupils. By using
intelligent security systems instead of conventional surveillance systems, suicides can
be prevented. Having placed the cameras in the right places, after appropriate cali-
bration, you can get a combination of facial and pupil reaction data synchronized with
the Eye tracker. This will allow for a comprehensive analysis of a person’s mental state,
to identify a depressed state, and to take measures in time. For such systems, it is
necessary to develop a pupilogram model that describes emotion. One sign of anxiety
depression is fear. The article proposes a model of a plot of a pupillogram containing
the emotion “fear”, based on experimental data.
2 Experimental Methods and Techniques
The first step in creating intelligent security systems that recognize the pre-suicidal
state involves two tasks. The first task is to detect the emotional component
in pupillograms. The second task is the selection of a suitable mathematical model.
A helmet has been developed for research (Fig. 1.), with the help of which a rigid
coordinate connection with a video camera is created. Due to this, we avoid increasing
the error in measuring the size of the pupils. The helmet is a frame welded from an
aluminum tube. For convenience, the aluminum frame is fixed with regular screws to
the head mount of the welding mask.
Fig. 1. Components of the experimental setup.
The frame is equipped with an additional layer of foam. In the occipital part of the
head-mounted mount is an adjustment screw that allows you to loosen or tighten the
helmet on the head. Camcorders 1 and 2 are attached to the bracket with standard
mounting bolts. Bolts allow you to direct the lens at the pupil of the eye by adjusting
the angle of inclination. The bracket is movably mounted on the frame, which allows
400 M. Boronenko et al.
you to adjust the distance between the lens and the eye. To exclude backlash of an arm
we use a directing hole and a clip. Camcorder-1 registers the image of the monitor on
which video files were displayed. Camcorder-2 registers pupil size. The internal time of
the cameras is synchronized up to hundredths of a second. This allows you to establish
the reason for the change in the size of the pupils. A person wearing a helmet-3 is
located at a distance at which the change in the illumination of the surface of the eye
due to the glow of the monitor becomes insignificant [5, 6]. Participants were shown
car accidents captured by DVRs. Videos are freely available on the Internet. Pupillo-
grams were shot in people of the older age category (50–75 years) in which repre-
sentatives of different sexes were present. All participants in the experiment were
volunteers. Everyone was warned of a possible emotional experience. Basically, all
participants have farsightedness, one woman has myopia, corrected by glasses. The
oldest cataract participant. Pupillograms of this participant are not accepted. During the
calibration of the system, the distance was determined at which the change in the
illumination of the surface of the eye, due to the glow of the monitor, does not lead to a
change in the size of the pupils. The distances between the calibration points, the
angular dimensions of the test objects and the change in the size of the pupils were
compared. Figure 2 shows a typical spotlight track. This is where the focus is on the
black dot demonstrated from the laptop screen during calibration. Presentation of the
track in the form of a pie chart allows you to track the amplitude of micro and macro
saccades.
Fig. 2. Spotlight track when looking at point 1 and moving attention to point 2 (when it
appears).
Calibration by points made it possible to establish the magnitude of the change in
the size of the pupils, explained by micro-saccades, which ensure the preservation of
Model of Emotionally Stained Pupillogram Plot 401
the image of the fixed object in the zone of best vision. When focusing a look at a point,
the amplitude of micro saccades does not exceed 50’, which corresponds to a change in
the size of the pupils no more than 1.1 times. We also determined the maximum change
in the size of the pupils, explained by macroaccades (moving the gaze to the points on
the screen that are maximally distant from each other). The amplitude of macroaccades
of the order of 60 leads to changes in the size of the pupils no more than 1.25 times.
During fixations of the spotlight, objects are recognized. In the experiments, video
cameras with a temporal resolution of not more than 33 ms were used, which makes it
possible to recognize explicit (from 250 to 450 ms) and implicit (150–250 ms) gaze
fixations. Therefore, simultaneous tracking of the center of attention and pupil size will
allow the creation of intelligent systems, the purpose of which will depend on the
specifics of visual stimuli.
3 Experiments to Identify the Emotional Components
and Modeling of Pupillograms
Attention center tracking experiments were carried out using captured video files
containing an optical image of the pupil of a person who was presented with stimuli. In
our case, as an irritant stimulating the response of the pupil, complex pictures were
used, the details of which can also be considered independent stimuli under certain
conditions. It is also accepted that the pupillary reaction is regulated by the most
significant parts of the image. So, the impulse affecting the size of the pupil from one
presented slide is added up from the pulses received from the most significant image
components for the subject. Tracking in each frame of the center of attention was
carried out by the coordinates of the center of mass of the reflection of the monitor on
the pupil image. Then, the center of mass of reflection was tracked in each frame and its
coordinates were determined. The coordinates of the center of mass of the reflection of
the monitor and the corresponding pupil size were used to build pupillograms and
oculograms. The analysis of the obtained images was carried out in the free software
FiJi. For statistical analysis, we used the StatPlus program to visualize OriginLab2019
data. As you know, the emotional influence of stimuli is proportional to the degree of
the individual’s internal relationship to this topic. The subject of information, the
carriers of which are test objects, can be determined by pressing problems of society,
pressing psychological problems, etc. Before conducting the experiment, a survey was
conducted to determine the topic of concern to the subjects. In agreement with the
participants, in one of a series of experiments with people of an older age group, fear
was chosen as the emotion. To evoke emotion, we used accident videos that are freely
available on the Internet. Video files did not contain bloody scenes, unexpected sounds.
All the events shown in the video files were predictable. The average value of the level
of intensity of emotions was considered a normal reaction, because the hypothesis was
that most people are mentally balanced and tolerant. Upon completion of the experi-
ment, all participants were alive, healthy and satisfied. To identify emotionally colored
areas, pupillograms were compared with tracks of the center of attention. Typical
results are shown in Fig. 3(a).
402 M. Boronenko et al.
Fig. 3. (a) Puppillogram plot containing emotion; (b) Gram-Charlier peak function.
The choice of function for approximating the experimental data was based on the
following considerations. To describe the acuity of attention, it was proposed to use the
[3, 4] Gaussian function. An analysis of the attention curve shows that its shape is
completely determined by the variable D (s): the greater the dispersion of attention, the
less attention is paid to the object of interest to us (i.e., the stimulus). In order to focus
attention on the stimulus, it is necessary, as far as possible, to minimize the variance
D (s). Of course, the KL model is well suited for such a description [7]. Therefore, to
approximate the experimental data, a function containing an exponent was chosen.
With sufficient reliability, the Gram-Charlier peak function is suitable (Fig. 3b).
Model of Emotionally Stained Pupillogram Plot 403
4 Conclusions
An analysis of the literature shows that there are a number of urgent problems for
which there are no definitive answers, for example: an objective definition of the
strength and sign of emotion; the law of decreasing the strength of emotion depending
on time and distance; algebraic properties of emotions and emotional logic; algorithms
and formulas of emotions; the connection of consciousness, thinking and emotions. In
addition, the lack of existing models is attributed to the lack of attention of researchers
to modeling the general state of the actor. The latter can be simultaneously influenced
by several emotions, which together form his psycho-emotional state at a given time.
However, if the issue of the course of one emotion has been studied to some extent, the
translation of emotions into the general state of the actor requires research. Our studies
show that the pupil reaction to a stimulus that is significant for the individ-
ual causes emotion. The mathematical model of the emotionally colored portion of
the pupillogram is well described by the Gram-Charlier peak function. The results are
consistent with the KL model of emotions.
Acknowledgments. The study was carried out with the financial support of the Russian
Foundation for Basic Research in the framework of the research project 18-47-860018 p_a.
References
1. Vasatkina, N.N., Merinov, A.V.: Clinical practice of child-teen suicides in the Ryazan region.
Tyumen Med. J. 16(3), 4–5 (2014)
2. Miller, D.N., Mazza, J.J.: School-based suicide prevention, intervention, and postvention. In:
Handbook of School-Based Mental Health Promotion, pp. 261–277. Springer, Cham (2018)
3. Valverde, L., de Lera, E., Fernàndez, C.: Inferencing emotions through the triangulation of
pupil size data, facial heuristics and self-assessment techniques. In: 2010 Second International
Conference on Mobile, Hybrid, and On-Line Learning, pp. 147–150. IEEE (2010)
4. Lempert, K.M., Glimcher, P.W., Phelps, E.A.: Emotional arousal and discount rate in
intertemporal choice are reference dependent. J. Exp. Psychol. Gen. 144(2), 366 (2015)
5. Boronenko, M., et al.: Use of active test objects in security systems. In: Advances in
Neuroergonomics and Cognitive Engineering: Proceedings of the AHFE 2019 International
Conference on Neuroergonomics and Cognitive Engineering, and the AHFE International
Conference on Industrial Cognitive Ergonomics and Engineering Psychology, 24–28 July
2019, Washington DC, USA, p. 438. Springer (2019)
6. Boronenko, M., et al.: Methods of indication of low intensity pupil reaction on the
subjectively-important stimuli. Act Nerv Super Rediviva 61(2), 49–56 (2019). Checklist of
Items to be sent to Proceedings Editors
7. Glazunov, Yuri Trofimovich.: Modeling the dynamics of the distribution of attention in goal-
setting processes. Bull. Baltic Fed. Univ. I. Kant. Ser. Philol. Pedag. Psychol. 5 (2012)
Cost-Informed Water Decision-Making
Technology for Smarter Farming
Joanne Tingey-Holyoak1(&), John Dean Pisaniello1, Peter Buss2,
and Ben Wiersma2
1
Sustainable Engineering, Accounting and Law Group,
UniSA Business School, City West Campus,
North Terrace, Adelaide, SA 5000, Australia
[email protected]
2
Sentek Pty Ltd., 77 Magill Rd, Stepney, SA 5069, Australia
[email protected]
Abstract. Around the world, producers across a variety of industries need tools
and models for better measuring and monitoring water in order to guide decision
making and improve productivity. Nowhere more is this the case than in irri-
gated agriculture. The purpose of this research is to develop a technology that
integrates farmer costing data with on-site physical data provided by intelligent
systems and technology to provide an account of on-farm water use. Currently,
farm accounting technologies are not at the level of sophistication required to
deeply consider the financial impacts of water use or loss. Water accounting
technologies are not able to easily link to and integrate business systems and
data to provide necessary monetary decision-making data, nor the context to
support a smarter approach to cost-informed water management. Linking of soil
moisture sensed data and local and remote weather station data to farm
accounting in a potatoes case study demonstrates how smarter farm decision
making can be supported by linking business systems and sensing technology.
Keywords: Soil sensors Water productivity Smart farming Cost
accounting Potatoes
1 Introduction
Agriculture uses about 70% of global fresh water for irrigation [18] and pressure to
produce more with less requires tools and models that increase productivity and capacity
[1]. However, tools for accurately and fully accosting water are challenging to develop
for agriculture [7, 20]. Farming is made up of spatial and temporal scales that make
water tracking and cost allocation challenging [4, 20]. Whilst there are water accounting
models and tools in development and in use globally they operate at macro scales [17],
can be complicated to apply [10, 16], and do not integrate business systems [20].
Like other governments around the world, the Australian Government has recently
allocated considerable funding under Smart Farms to support the development and
uptake of tools and technologies to support farmers. However, as yet there has not been
the development of technology that is integrated into business decision making and
data and these un-integrated tools can even generate negative environmental outcomes
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 404–408, 2020.
https://doi.org/10.1007/978-3-030-39512-4_63
Cost-Informed Water Decision-Making Technology for Smarter Farming 405
at the catchment scale [12]. Contextualised and full farm water accounting is required
[4, 5]. Therefore, this paper seeks to answer the question: How can cost-informed
technology for smarter farming be developed?
2 Materials and Methods
The study included a participatory case study on a potato farm in the Murraylands of
South Australia. Case study activities included the collection of accounting, soil and
climate sensing data and combining it in a way that can assist water-related decision
making [5, 13]. The producer was chosen from the direct monitoring industry partner’s
client base with an identified interest in enhancing water productivity [19]. Site
meetings and discussions were undertaken followed by soil moisture probe and weather
station installation on a center pivot irrigated field of potatoes. Sensor data included
rainfall and irrigation, evapotranspiration, soil moisture, and drainage whilst account-
ing data included water costs, power (pumping costs), costs of labour and materials,
such as fertiliser, in addition to costs of water storage, infrastructure maintenance,
equipment insurance, pump/pipe loan interest, and licensing. Data was combined in a
Microsoft Excel dataset once reduced to the appropriate spatial and temporal scale.
Then analysis of management and financial accounting water-related information was
undertaken to determine water productivity. This process not only illustrates the utility
of the tool when used against ‘acceptable’ water productivity indicators but also pro-
vides increased understanding of the contribution of the data combination to under-
standing critical water-related costs.
3 Results
Modern varieties of potatoes require irrigation often as they can easily be exposed to
soil water deficits. The state of the case study, South Australia produces 80% of the
country’s washed market potatoes worth AUD206 million [14]. The case study region
has low rainfall and sandy soil which means keeping soils optimally moist is chal-
lenging so informing irrigation decisions using soil moisture data is an important area
to focus [11]. Therefore, the first step was to set up the potato site with soil moisture
probes and a weather station and monitor a season of potato crop (Fig. 1).
The case study site of 28 Ha was monitored from March to June 2018. Accounting
data was collected from farmer diaries and records, accounting software and suppliers.
Power for pumping pivots accounted for 12% of total variable costs but was highly
differential all season, subject to half hourly market price fluctuations ranging from
$0.64/Mwh to $4798.86/Mwh wthin the March to June timeframe (Fig. 2). Actual
costs of water comprised 75% of the total variable costs. Figure 2 demonstrates that
beyond fixed and variable water-related costs, a new criteria of “hidden water related
costs” were noteworthy (Fig. 2). These included owner labour and water quality
treatment which comprised 15% that had not been considered in accounting for water
previously. Water costs in total made up 75% yield excluding all other input and
processing costs which highlights the significance of including all water-related
financial information to better inform decision making.
406 J. Tingey-Holyoak et al.
Fig. 1. Weather station, sensor and solar panel in-field set up
Fig. 2. Water-related costs composition compared to total yield revenue
Water productivity was assessed for the season using methods of Molden et al. [8];
Hussain et al. [6]:
ð1Þ WpVolume ¼ Yield ðKgÞ = Water M 3 ð1Þ
ð2Þ WpDollars ¼ Yield ð$Þ=Water M 3 ð2Þ
Yield value for potatoes of $0.1 per kg should have a water productivity of 3 to 7
gk/M3 and 0.3 to 0.7 $/M3 (per Eq. 2). The producer’s water productivity was within
this ranges and at the upper end (Table 1) which is positive for the producer, however
further seasonal data is required to track change and also any effect from using the tool.
Cost-Informed Water Decision-Making Technology for Smarter Farming 407
Table 1. Water productivity at case study site compared to potato indicators from literaturea
1. Product per unit of water 2. Gross value of product per unit of
applied WP (kg/M3) water applied WP ($/M3)
Case study 2018 2.79 0.96
potatoes season
Potatoes indicatorb 3–7 0.3–0.7
a
Renault and Wallender [15]; Molden et al. [8]
b
Indicator range encompasses all possible agricultural types.
The producer hoped that once implemented, the tool could provide water pro-
ductivity alerts, isolate cost saving concentrations through tracking costs, but also to
integrate much of the data that is accessible freely or via subscription to do with
production such as irrigation management and soil chemical information. So an
additional dataset and software framework was established for data integration to
provide alerts for plant stress, disease conditions, and low quality irrigation likelihood,
and the development of scenario models and forecasts.
4 Discussion and Conclusions
The study explored collecting the data and producer insight for a cost-informed water
decision making tool. A baseline database for potatoes was established which allowed
for understanding of full cost and the potential for its tracking across fields and seasons.
In addition the development of cost-informed alerts and forecasting is possible from the
linked database developed. Assessment of water productivity whilst using the tool was
in the good range of common indicators [8, 15] however, the cost/yield revenue
analysis demonstrates profit erosion and water productivity impacts from ignoring the
‘hidden costs’ of water, such as owner labour and water quality treatment [9]. The
study advances understanding of how smarter farming can be promoted through better
cost information linked to water use and productivity, and how tools that seek to
achieve this can be integrated into agricultural businesses.
Acknowledgments. Great thanks to the very generous case study participants who graciously
shared their time and knowledge. Thanks also to the University of South Australia for Research
Themes Investment Funding.
References
1. Altobelli, F., Monteleone, A., Cimino, O., Dalla Marta, A., Orlandini, S., Trestini, S.,
Toulios, L., Nejedlik, P., Vucetic, V., Cicia, G., Panico, T.: Farmers’ willingness to pay for
an environmental certification scheme: promising evidence for water saving. Outlook Agric.
(2019). https://doi.org/10.1177/003072701984105
2. Food and Agriculture Organisation (FAO) of the United Nations: Water Report 43: Water
Accounting and Auditing Sourcebook. FAO, Geneva (2017)
408 J. Tingey-Holyoak et al.
3. Hoekstra, A.Y., Chapagain, A.K., Aldaya, M.M., Mekonnen, M.M.: The Water Footprint
Assessment Manual: Setting the Global Standard. Earthscan, London (2011)
4. Hussain, I., Turral, H., Molden, D.: Measuring and enhancing the value of agricultural water
in irrigated river basins. Irrig. Sci. 25(3), 263–282 (2007)
5. Jack, L.: The adoption of strategic management accounting tools in agriculture post subsidy
reform: a comparative study of practices in the UK, the US, Australia and New Zealand.
Discussion Paper. CIMA, London (2009)
6. Molden, D, Oweis, T.Y., Pasquale, S., Kijne, J.W., Hanjra, M.A., Bindraban, P.S., Bouman,
B.A., Cook, S., Erenstein, O., Farahani, H., Hachum, A.: Pathways for increasing
agricultural water productivity, IWMI, 612-2016-40552 (2007)
7. Moran, J.: Business Management for Tropical Dairy Farmers. CSIRO Publishing, Canberra
(2009)
8. Mulla, D.J.: Twenty five years of remote sensing in precision agriculture: key advances and
remaining knowledge gaps. Biosyst. Eng. 114(4), 358–371 (2013)
9. Pardossi, A., De Pascale, S., Rouphael, Y., Gallardo, M., Thompson, R.B.: Recent advances
in water and nutrient management of soil-grown crops in Mediterranean greenhouses. In:
GreenSys2015: International Symposium on New Technologies and Management for
Greenhouses, vol. 1170, pp. 31–44 (2015)
10. Perry, C.: Water security – what are the priorities for engineers? Outlook Agric. 39(4), 285–
289 (2010)
11. Perry, C.: Accounting for water use: terminology and implications for saving water and
increasing production. Agric. Water Manag. 98(12), 1840–1846 (2011)
12. Potatoes, S.A.: South Australia Snapshot, Potatoes South Australia, Adelaide (2017)
13. Renault, D., Wallender, W.W.: Nutritional water productivity and diets. Agric. Water
Manag. 45(3), 275–296 (2000)
14. Srinivasan, M.S., Bewsell, D., Jongmans, C., Elley, G.: Just-in-case to justified irrigation:
applying co-innovation principles to irrigation water management. Outlook Agric. 46(2),
138–145 (2007)
15. Tingey-Holyoak, J.L., Pisaniello, J.D., Buss, P., Wiersma, B.: Water productivity accounting
in Australian agriculture: the need for cost-informed decision-making. Outlook Agric.
(2019). https://doi.org/10.1177/0030727019879938
16. United Nations Intergovernmental Panel on Climate Change (UNIPCC): Climate Change
2007. Cambridge University Press, Cambridge (2007)
17. Welsh, R., Rivers, R.Y.: Environmental management strategies in agriculture. Agric. Hum.
Values 28(3), 297–302 (2011)
18. Young, M.D., McColl, J.C.: Double trouble: the importance of accounting for and defining
water entitlements consistent with hydrological realities. Aust. J. Agric. Resour. Econ. 53(1),
19–35 (2009)
A Review on the Role of Embodiment
in Improving Human-Vehicle Interaction:
A Proposal for Further Development
of Embodied Intelligence
Hamid Naghdbishi(&) and Alireza Ajdari
Enghelab Sq., University of Tehran, 16th Azar Street, Tehran, Iran
[email protected]
Abstract. This article tries to see how embodiment discourse can improve the
design of an intelligent system, in this case Human-Vehicle Interaction. This
proposal is based on the studies of Maurice Merleau Ponty and Paul Dourish
applied in Intelligence System Design. Based on the mentioned above, the
concept of Designerly Embodied Intelligence is introduced and applied for such
a discourse. While the concept of Embodied Interaction sounds new and
applied, the role of design thinking in improving such concept still needs to be
discussed and elaborated. Therefore, vehicle design and especially product
planning phase would be needed to be elaborated based on this discussion. It
would be reviewed how intelligent systems could be improved if the concept of
embodiment would be used for them, moreover it would be reviewed that how
design thinking and designer-ly ways of knowing and interacting would enrich
both embodiment and human-vehicle Interaction. It is claimed that design dis-
course, not only in vehicle planning, but also in improving an intelligent and
appropriate system design, can be very helpful. Therefore, the role of design
thinking in intelligent vehicle design is based on the concept of Appropriate-
ness, or in other words, an intelligent design which is directed toward needs of
the target user, no more and no less. Seeking the new potentials of intelligent
systems in vehicle design and helping Open Innovation could be also another
outcome of such proposal. Based on such argumentation, some proposals would
be sketched in order to visualize possible solutions for designer-ly embodied
intelligence in human-vehicle interaction.
Keywords: Design thinking Embodied intelligence Human-vehicle
interaction Appropriate innovation
1 Introduction
Analyzing dynamic and complex activities aimed at identifying vehicle driver activities
is one of the most important issues in machine learning. Driver activity detection has
many applications such as driving skill analysis, group statistics, automatic detection of
sensitive movements and behaviors, and automatic control of the car’s steering (when
the driver is unsafe). Although much research has been done on the analysis of driving-
based and in-vehicle activities, this design and research has always been from the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 409–414, 2020.
https://doi.org/10.1007/978-3-030-39512-4_64
410 H. Naghdbishi and A. Ajdari
designer’s point of view, and in this research, with the help of intelligent and design-
based “Open Innovation” that focuses on the behaviors of people’s lives and the
processing of driving modes that contribute to the reduction of human error by ana-
lyzing driving guidance. The detection algorithm also converts video into a visual
system based on machine learning and artificial intelligence in the vehicle and transmits
messages to the driver.
2 Problem Formulation
Principles: Tonal discourse can improve the design of an intelligent system by human-
machine interaction and provide a sense of satisfaction when driving. Explaining the
problem and analyzing the problem by adapting the interaction by “Open Innovation”
helps. The modularity of mind that assumes that cognitive processes are composed of
parts that have a particular relation to these parts that form the information processing
model for cognitive abilities, which assumes that cognitive processes are composed of
parts that have a particular relation to these parts, which form the information pro-
cessing model for cognitive abilities.
Epiphenomenalism deals with the relationship between mind and body, which
according to this theory, all mental phenomena are caused by physical events and
phenomena in the brain.
Aspectual shape: Whenever we perceive or think about something, the conscious
experience of the external object is shaped by certain features in our mind [1].
3 Method
This research, which seeks suggestions for further developing artificial intelligence for
human-vehicle interaction using artificial intelligence, is a quantitative research method
in terms of research design and is a correlational design.
4 Design Thinking
Design thinking can be successfully applied to all topics as a tool to solve problems and
foster Innovation [2]. It can also be used as a tool for insight, new understanding, and
creative solutions to complex problems [3]. Open Innovation helps accelerate domestic
Innovation and expand foreign use markets [4]. As such, Open Innovation encom-
passes two dimensions of Innovation and information:
• Technical operation
• Technical Exploration
An Open Innovation paradigm is to develop Innovation that is inherently focused on
creating value through the extraction of unused technologies and ideas [5]. Design
thinking is an effective tool for developing innovative and new products, services and
processes [6]. This is a “human-centered” approach to design, that is, a repetitive
process of listening, observing, sampling and experimenting according to user needs [7].
A Review on the Role of Embodiment in Improving Human-Vehicle Interaction 411
The concepts and language of design thinking are to improve a variety of processes
including product design [8], service design, food product Innovation [9] and social
services [10].
Design thinking is essentially a human-based Innovation process that emphasizes
ideas and rapid prototyping [11].
5 Open Innovation
The theory of Open Innovation was invented by Chesbrough [12], which is a new
perspective on Innovation. It is a theory of coexistence with the external environment
that enables us to make more regular use of innovative external ideas. It also enables
users to plan their activities in line with innovative activities [13]. Open Innovation
combines internal and external ideas with architectures and systems whose needs are
defined by a business model [14].
6 Design Thinking and Open Innovation
Design thinking is often at odds with alternative Innovation strategies, such as
technology-based Innovation and design by designers [15]. Design thinking can be
effective in creating the Innovation process by providing a “safe” space [16]. Design
thinking is a faster and cheaper process, but it is a set of techniques that reduce the
cognition of Innovation teams [9]. Design thinking ability to solve complex problems
[17]. Some argue that design thinking can be used for non-designers and can provide a
competitive advantage to companies through improved suggestions [18].
By gathering feedback from product users, the Innovation team was able to have a
full understanding of user preferences and expectations of new products [19] in which
this research could consider better and more human interaction as a vehicle user with
artificial intelligence. Open Innovation in design thinking involves several concepts
related to art that are referred to as “destiny engineering” [20].
The purpose of this research is to inspire phenomenological theories of perception,
as it allows for the fundamental insights to be explored and applied to the concept of
meaning in design and related technologies. These views are conceptually, dynamic,
subjective parts of the world.
Phenomenology helps in designing these views by developing the role of inter-
subjectivity in rationalizing emotion. The phenomenology of subjective, tonal and
contextual layers or context-based layers engages individuals in products and in a
dynamic, rich and complex world [21]. Maurice Merleau-Ponty goes even further and
states that we need the body to experience the world [22].
Syntactic perception is a law and is used in design, synesthesia, a mixture of
emotion, as a good mechanism for the development of artwork or artifact, an instinctive
link between sound, imagery, and aroma [23, 24]. Hubert Dreyfus applied philosophy
to artificial intelligence, and Paul Dorsey introduced the work of phenomenologists
such as Merleau-Ponty and Alfred Schütz to a variety of designs through his book
“where the action is [21]. In design, phenomenology transforms and values the way of
thinking in the design and creation of works [22].
412 H. Naghdbishi and A. Ajdari
7 Embodied Design Thinking
The conceptual blending process, together with the embodied mind and embodied
concept, gives rise to the embodied design thinking; the embodied design thinking is a
process that seeks to resolve qualitative differences in changing situations.
8 Conclusion
According to the studies, it is thought that, if shown in Fig. 1, a combination of design
thinking and design thinking is incorporated into the design process, and then recon-
ciled with the Open Innovation theory; Concerning the relationship of human-centered
design to the vehicle will achieve the desired result. In this algorithm, the process of
human, body, Innovation, as well as artificial intelligence is incorporated into the
design thinking.
Fig. 1. The design process is based on a combination of several separate analytical processes
A Review on the Role of Embodiment in Improving Human-Vehicle Interaction 413
9 Result
This paper investigates through a case study on embodied intelligence, Open Inno-
vation and artificial intelligence, in which this study investigates the rate of reduction of
driver error by artificial intelligence regarding the problem of embodiment as well as
the idea by Open Innovation and collecting the demands of users have been achieved.
The results of the studies are presented as a design algorithm that can be effective in
reducing the error leading to traffic accidents. This design is intended for the vehicle’s
rearview mirror to act on the behavior of drivers and detect the sensors embedded in the
car. The machine learning and embodied intelligence-driven performance of the vehicle
made users less anxious while driving and reduced the amount of financial damage
caused by other car crashes or roadblocks. Machine learning through the perception of
the driving behavior of each individual, as well as the demands of the drivers under
study, has formed an organized structure of how to explore and come up with all the
elements involved in embodied intelligent design.
References
1. Searle, J.R.: The Rediscovery of the Mind. MIT Press, Cambridge (1992)
2. Schmiedgen, J., Rhinow, H., Köppen, E., Meinel, C.: Parts Without a Whole? - The Current
State of Design Thinking Practice in Organizations (2015)
3. Owen, C.L.: Design thinking: notes on its nature and use. Des. Res. Q. 1(2), 16–27 (2007)
4. Chesbrough, H.W., Crowther, A.K.: Beyond high tech: early adopters of open innovation in
other (2006)
5. Chesbrough, H.W., Schwartz, K.: Innovating business models with co-development
partnerships. Res. Technol. Manag. 50, 55–59 (2007). Industries. R & D Management, 36
(3), 229–236
6. Gruber, M., de Leon, N., George, G., Thompson, P.: Managing by design: from the editors.
Acad. Manag. J. 58, 1–7 (2015)
7. Cross, N.: Designerly ways of knowing. Des. Stud. 3, 221–227 (1982)
8. Brown, T.: Design thinking. Harv. Bus. Rev. 86(6), 84 (2008). June 2008
9. Olsen, N.V.: Design thinking and food innovation. Trends Food Sci. Technol. (2014) http://
dx.doi.org/10.1016/j.tifs.2014.10.001
10. Liedtka, J., King, A., Bennett, D.: Solving Problems with Design Thinking: Ten Stories of
What Works. Columbia University Press (2013)
11. Lockwood, T.: Design thinking in business: an interview with Gianfranco Zaccai. Des.
Manag. Rev. 21, 16–24 (2010)
12. Elmquist, M., Fredberg, T., Ollila, S.: Exploring the field of open innovation. Eur. J. Innov.
Manag. 12(3), 326–345 (2009)
13. Chesbrough, H., Crowther, A.K.: Beyond high tech: early adopters of open innovation in
other industries. R & D Manag. 36(3), 229–236 (2006)
14. Chesbrough, H.W.: Open Innovation: The New Imperative for Creating and Profiting from
Technology. Harvard Business Press (2003)
15. Verganti, R.: Design, meanings, and radical innovation: a metamodel and a research agenda.
J. Prod. Innov. Manag. 25(5), 436–456 (2008)
16. Docherty, C.: Perspectives on design thinking for social innovation. Des. J. 20(6), 719–724
(2017). https://doi.org/10.1080/14606925.2017.1372005
414 H. Naghdbishi and A. Ajdari
17. Acklin, C.: Model: a framework to describe and measure the absorption process of design
knowledge by SMEs with little or no prior design experience 22(2), 147–160 (2013)
18. Acklin, C.: Design-driven innovation process model. Des. Manag. J. 5(1), 50–60 (2010).
http://doi.org/10.1111/j.1948-7177.2010.00013.x
19. Von Hippel, E.: Sticky information and the locus of problem-solving: implications for
innovation. Manag. Sci. 40(4), 429–439 (1994)
20. Lindsay, G.: Engineering Serendipity. The New York Times, 5 Apr. 2013. NYTimes.com.
Web. Accessed 3 Nov 2013
21. Dourish, P.: Where the Action Is. The Foundations of Embodied Interaction. MIT Press,
Boston (2001)
22. Merleau-Ponty, M.: Phenomenology of perception trans. In: Smith, C. (ed.) Humanities
Press, New York (1962)
23. Smets, G.J.F., Overbeeke, C.J.: Scent and sound of vision: expressing scent or sound as
visual forms. Percept. Motor Skills 69(1), 227–233 (1989)
24. Smets, G., Overbeeke, C.J., Gaver, W.W.: Form-giving: expressing the nonobvious. In:
Proceedings of CHI 1994, pp. 79–84. ACM Press, New York (1994)
Analysis of Topological
Relationships of Human
Jia Zhou, Xuebo Chen(&), and Zhigang Li
School of Electronics and Information Engineering, University of Science and
Technology Liaoning, Anshan 114051, Liaoning, People’s Republic of China
[email protected], [email protected], [email protected]
Abstract. In order to study the network characteristics of human, this paper
analyzes the interactions between individuals and cluster behavior based on the
lennard-jones model. Firstly, we simulate a multi-agent system and randomly
distribute individual locations in the system. Then, we keep the total number of
individuals and interactions unchanged, and change the number of intercon-
nection. By analyzing the phenomenon of cluster behavior, we can summarize
the impact of the number of interconnects on system stability. Finally, combined
with the above factors, some specific measures are proposed to solve the sta-
bility problem of complex systems.
Keywords: Multi-agent systems Cluster behavior Topological
interconnection
1 Introduction
Multi-Agent systems, as an important branch of artificial intelligence, have become hot
spots in many fields. Since the famous biologist Bertalan Fei proposed the concept of
complexity [1], the complexity problem has begun to be widely concerned, and has
made a lot of progress, such as von Neumann’s cellular automata and complexity,
Shannon’s informatics. At present, modeling methods based on multi-agent systems
have become one of the important methods for studying complex systems.
Cluster self-organizing behavior is widespread in the biological world and plays a
vital role in the survival of living things. In fact, it also plays an important role in the
control of artificial robots and in man society. We often find that many biological indi-
viduals are simple in structure and weak in strength. Once they are active in groups, they
will accomplish many things that are incredibly far beyond their own abilities. Therefore,
they survived the brutal competition. In the process of mutual cooperation, each indi-
vidual works in a division of labor and contributes their own strength. They exchange
information [2] with each other and improve work efficiency. In the artificial robot, in
order to complete the arrangement of the formation, the detection of the area coverage, or
the cooperative rounding, the interference is kept stable, etc., and each agent is required to
be in the self-organizing control process. Large-scale distributed control in multi-agent
systems does not affect the operation of the entire multi-agent systems due to commu-
nication failure between individual agents, thus providing greater flexibility and scala-
bility. For example, the current Internet is multi-agent systems that does not affect the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 415–420, 2020.
https://doi.org/10.1007/978-3-030-39512-4_65
416 J. Zhou et al.
communication of the network due to the damage of some routers. This distributed control
approach is more robust than centralized control [3, 4].
2 Method
For the cluster self-organizing movement of multi-agent systems, the key is to study the
autonomic movement of each individual to study the movement of the cluster. We
build rules and models of individual free movement based on loose preference rules to
simulate the emergence of cluster movements, such as simulating the interrelationships
between individuals in clusters of nature, so we build such a model. The rule is that the
robot will form a formation that tends to disperse and the group gathers at the target
point during the movement toward the target [5]. Therefore, it is called the individual
free movement rule based on loose preference behavior.
Potential energy is energy stored in a system that can also be released or converted
into other forms of energy. Potential energy is a state quantity, also known as potential
energy. Potential energy is not shared by individual objects, but by interacting objects.
The earliest proposed potential energy function was proposed in physics to describe
potential energy. Because each object has a real interaction with each other, and the
energy that depends on the relative position between the objects is called potential
energy [6].
The artificial potential field method [7] can not only solve the motion problem of a
single agent, but also deal with the multi-agent systems and multi-mobile robot
coordination and cooperation problems. By adding an artificial potential field to each
agent in the system, the motion of a single individual in the system behaves like a force
between molecules: that is, in equilibrium, the molecules are relatively balanced, less
than the equilibrium position. Reflected as repulsive force, it is reflected in gravity
when it is greater than the equilibrium position. Therefore, it can simulate the two basic
rules of cluster movement in biological groups: the attractiveness of individuals with
distant spacing and the repulsive force of individuals with too close spacing, so that
individuals in the system can move to the expectation. At this time, the total potential
energy field of all individuals is at the global minimum.
The emergence of systems often requires a variety of factors. For the mechanism of
research emergence, more external factors should not be introduced in order to generate
emergent phenomena, which makes the mechanism research difficult to explain. How
to find a simple and effective emerging system as the research object is the key to
emerging modeling. It can be seen from the broad generalized summary that emergence
is an inherent property of the system, but it will exhibit different complex performances
due to external disturbances. In this model, external interference is not considered and
only the effects of internal interactions are considered. The starting point for the study
should be attempted to approximate the interconnection rules between individuals by
reference to various real-life examples in nature.
Using graph theory, the interconnection between systems and individuals can be
described simply and effectively. Two-dimensional loose preference behavior rules
were used for two-dimensional planes such as formation, handling, and rounding.
Collaborative tasks. In order to study the emerging generation and control methods, the
Analysis of Topological Relationships of Human 417
two-dimensional interconnection rules are extended into three-dimensional intercon-
nection rules.
X X
fi ¼ j¼N
f vj vi eij ð2:1Þ
i
Where eij ¼ aij vj vi =vj vi describe the interconnection vi ; vj with the
unit vector of the weight aij , and the interconnection force in the direction of the unit
vector can be passed through the Lennard-Jones function [8] or The functions men-
tioned in the chapter are described.
A self-organizing system based on loose preference behavior rules has the simplest
dynamic equation
v_ ¼ f i ð2:2Þ
3 The Simulation Analysis
In order to simulate the cluster motion of multi-agent systems, the model [9] of the
interconnecting force that attracts close-distance things remotely can be simply
described as
8 1 1
< wr h x ri ; rl \r
>
f ð xÞ ¼ wa sin ððrxr Þ
; r \x\rh ð3:1Þ
>
:
h r Þ
0; others
Where is the individual’s expected distance ðr ¼ 5Þ, rl and rh are the minimum and
maximum perceived distances of the individual, respectively, and wr and wn are the
interconnecting force parameters. In general, the motion patterns of the natural cases in
the natural world are approximated by the blending of parameters in the model.
These simulation diagrams are based on the multi-agent systems emerging results
of topology interconnection. In Figs. 1, 2, 3 and 4, all simulation diagrams are at the
time of step number = 20000, all the number of individuals N is 11, randomly dis-
tributed in three-dimensional space, and the small black dots in the figure are the
starting positions of the individuals. The big black point individual is the end position,
and the color curves are the motion trajectories. Note that the thick red line is the
average value of the final positions of all individuals.
It is not difficult to find that when the number k of interconnections is small, the
individual is almost unchanged in position, for example, in Fig. 1, because the number
of interconnections is small, the interaction between them is small, and it is difficult to
gather in a short time. In Fig. 2, the phenomenon of individual aggregation is obvious,
and the aggregation density becomes larger at the same time.
418 J. Zhou et al.
Fig. 1. Cluster motion (k = 1) Fig. 2. Cluster motion (k = 6)
In Figs. 3 and 4, it is worth noting that at k = 7 and 9, the emerging pattern appears
spirally curved, presenting a dynamic phenomenon. In Fig. 2, all individuals are all
gathered and in a static state. Interestingly, the individuals are almost all gathered
together in a straight line to form a circle with substantially equal distances between
individuals. It is possible that cohesion is greatest when all individuals interact.
Fig. 3. Cluster motion (k = 7) Fig. 4. Cluster motion (k = 9)
Figures 5 and 6 are graphs of the system and individual stress based on the
topological interconnection relationship, in which the blue thick line indicates the
combined force of the system, the red thick line indicates the maximum force of the
individual, and the other lines indicate the individual’s force, which can be seen in the
perception.
According to the graph analysis, when k = 6, the system stress is 0. The system is
in dynamic equilibrium; when k = 7, the system is not subjected to force and is
constant. It can be inferred that the system is also subjected to a resistance equal to the
magnitude of the force, thus making the system dynamically balanced. In Fig. 3, the
system moves along the arc, calling this phenomenon “spiral emergence”.
Analysis of Topological Relationships of Human 419
Fig. 5. Force of the system (k = 6) Fig. 6. Force of the system (k = 7)
4 Conclusion
The stability of a multi-agent system is affected by many factors. When the total
number of individuals in the system remains unchanged and the number of individuals
contacting neighboring individuals is small, clusters appear to aggregate, and the
clusters reach stability quickly. As the number of contacts between individuals and
neighboring individuals increases, the cluster aggregation phenomenon becomes more
obvious, and the cluster can also reach stability. However, when the number of indi-
viduals and neighboring individuals is 7 or 9, the cluster appears spiral motion and the
system is unstable. It can be seen from the above analysis that there are some numbers
of interconnection making the system unstable. At this time, we can increase or
decrease the number of individuals connected with neighboring individuals to stabilize
the system.
Acknowledgments. This research reported herein was supported by the NSFC of China under
Grant No.71571091 and 71771112.
References
1. Cowan, G., Pines, D., Meltzer, D.E. (eds.).: Complexity: Metaphors, Models, and Reality.
Perseus Books, Cambridge (1994)
2. Moreau, L.: Stability of multi-agent systems with time-dependent communication links. IEEE
Trans. Autom. Control 50(2), 169–182 (2005)
3. Gazi, V., Passino, K.M.: Stability analysis of social for aging swarms. IEEE Trans. Syst. Man
Cybern. Part B Cybern. 34(1), 539–557 (2004)
4. Gazi, V., Passino, K.M.: Stability analysis of swarms. IEEE Trans. Auotom. Control 48(4),
692–697 (2003)
5. Cortes, J., Bullo, F.: Robust rendezvous for mobile autonomous agents via proximity graphs
in arbitrary dimensions. IEEE Trans. Autom. Control 51(8), 1289–1298 (2006)
420 J. Zhou et al.
6. Su, H.S., Wang, X.F., Chen, G.R.: Rendezvous of multiple mobile agent with pre-served
network connectivity. Syst. Controlletters 59(5), 313–322 (2010)
7. Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. Int. J. Rob. Res.
5(1), 90–98 (1986). Spring
8. Davis, B.D., Mingioli, E.S.: Mutants of escherichia coli requiring methionine or vitamin B12.
J. Bacteriol. 60(1), 17–28 (1950)
9. Gerkey, B., Vaughan, R.T., Howard, A.: The player/stage project: Tools for multi-robot and
distributed sensor systems. In: Proceedings of the 11th International Conference on Advanced
Robotics, vol. 1, pp. 317–323 (2003)
Impact of Technological Innovation
on the Productivity of Manufacturing
Companies in Peru
Julio César Ortíz Berrú1, Cristhian Aldana Yarlequé2(&),
and Lucio Leo Verástegui Huanca3
1
Banco Central de Reserva Del Perú, Piura, Peru
[email protected]
2
Universidad Nacional de Frontera, Piura, Sullana, Peru
[email protected]
3
Universidad Complutense de Madrid, Madrid, Spain
[email protected]
Abstract. It is estimated the relationship between the investment in innovation
activities, the results of innovation and productivity in Peruvian manufacturing
firms, using the multi-equation model, called the CDM model, which studies the
entire innovation process. The quantile regression approach was used, using data
from the II National Survey of Innovation in the Manufacturing Industry 2015 in
Peru. The findings were that technological innovation is associated with a 56%
increase in productivity in the company. Finally, the quantile regression
approach shows that the effect of technological innovation on the firm’s pro-
ductivity is heterogeneous, increasing the effect on bigger firms. If investment in
technological activities increases by 1%, then the labor productivity increases by
0.22% and the returns of innovation depend on the entrepreneurial position
according the distribution of productivity.
Keywords: Technological innovation Productivity Manufacturing sector
CDM
1 Introduction
The relationship between productivity and innovation is one of the first work submitted
by Schultz and Griliches. Since then, a considerable number of empirical and theo-
retical works have been generated in this regard. In some of the most recent theoretical
models, a fundamental role has been attributed to investment in innovation activities as
an impulse of productivity and, therefore, of economic growth [1, 2].
Most of the empirical work on the relationship between innovative activities and
productivity has been focused on developed countries [3–6]. Studies in developing
countries are mostly qualitative, without being able to infer clear statistical patterns.
Therefore, it derives the great importance it has, in the case of developing countries;
specify the largest possible number of country-specific studies [7].
The goal of this working paper is to evaluate the relationship between investment in
innovation activities and the productivity. Firstly, Peru is in a lagging position in terms
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 421–426, 2020.
https://doi.org/10.1007/978-3-030-39512-4_66
422 J. C. Ortíz Berrú et al.
of innovate initiatives: R&D spending represents an average of 0.1% of sales and 4.8%
of spending on innovation activities at the firm level, these percentages are inferior to
those destined by OECD countries [8, 9]. Secondly, at the moment there are not
researches in the country about the relationship between productivity and innovation,
based on surveys on innovation in the manufacturing industry.
It was used The CDM model and it was evaluated the empirical relationship
between investment in innovation activities, the results of innovation and productivity
in Peruvian firms in the manufacturing sector. This approach is based on a multi-model
model that concerns the entire innovation process. It takes the determinants of firms’
decisions to adopt innovation initiatives, the results of those initiatives and their impact
on productivity. The data comes from the II National Commission of the Manufac-
turing Industry 2015 in Peru, which contains quantitative and qualitative information
for the period between 2012–2014 [10–12].
Understanding the relationship between technological innovation1 and productivity
can not only lead the performance of firms in the sector level, but can also serve as a
guideline for policy makers to achieve a better economic and industrial performance of
our country.
2 Materials and Methods
The research was made with a study by Créspon, Duguet and Mairesse, it is known as
the CDM model. The approach is based on a multi-equation model in which it takes the
entire innovation process. The reference model consists of four equations: (i) the firms’
decision to invest in innovation activities, (ii) the intensity of investment in innovation
activities, (iii) the knowledge production function which links the intensity of inno-
vation to its results, and (iv) the production function, in which the firm’ productivity is
based on the results of technological innovation [2].
The CDM model tries to solve the problem of selection bias2 and endogeneity in
innovation and productivity functions. The following equation represents the effort in
innovation by firms IEi .
IEi ¼ zi0 b þ ei ð1Þ
Where the subscript i represents the signatures, IEi is a latent variable not observed,
zi is a vector of determinants of the effort in innovation, b is a vector of parameters and
1
In this study to define technological innovation it is used the concept of the Oslo Manual; it means,
technological innovation, also called product or process innovation, is structured of nine activities:
(i) internal Research and Development (R&D) activities; (ii) acquisition of external R&D;
(iii) acquisition of capital goods; (iv) Hardware acquisition; (v) software acquisition; (vi) technology
transfer; (vii) industrial design and engineering; (viii) training for innovation activities; and
(ix) market research for the introduction of innovations.
2
The problem of selection is that, in each period of time, we only keep with the firms that reported
investment in innovation activities. By eliminating companies with zero investments in innovation
activities, the sample would be biased.
Impact of Technological Innovation on the Productivity 423
ei is an error term. IEi is approached through the logarithm of the innovation activity
expenditure per worker denoted by IEi only if the firms make (and report) said expense.
In this way it can directly estimate Eq. (1) with the risk of selection equation. Instead,
the following selection Eq. (2) is assumed that describes whether the firm decides to
make (and/or report) the investment in innovation.
1 si IDi ¼ w0i a þ ei [ c;
IDi ¼ ð2Þ
0 si IDi ¼ w0i a þ ei c
Where IDi is a binary variable and represents the investment decision in innovation.
IDi will be zero for firms that do not invest in innovation activities, and equal to one for
firms if they do. IDi is a latent variable that expresses the innovation investment
decision if it is located above a threshold c, where w is a vector of explanatory variables
of the innovation investment decision3, a is the vector of parameters of interest and ei is
an error term.
Equation (3) expresses the intensity of investment innovation activities IEi 4
observed. Assuming that the error terms have zero mean, variance r2e ¼ 1, and cor-
relation coefficient qee , the system of Eqs. (2) and (3) are estimated using a generalized
Tobit model for maximum likelihood.
IEi ¼ z0i b þ ei si IDi ¼ 1
IEi ¼ ð3Þ
0 si Di ¼ 0
Equation (4) is the production function of innovation (or knowledge). Where IT is
equal to 1 when the company has introduced an innovation; IEi is the prediction of the
value of the innovative effort of the company from the previously estimated generalized
Tobit equations, xi is a vector of other determinants of knowledge production5, c and d
are parameter vectors of interest, and ui is an error term. F is the function of standard
normal distribution.
TIi ¼ F IEi c þ x0i d þ ui ð4Þ
Equation (5) is the production function, which relates innovation and labor pro-
ductivity. It is assumed that firms adopt a Cobb-Douglas function that is usually
assumed to have constant returns to scale. The dependent variable yi is the firm’s labor
productivity, measured by the logarithm of sales per worker in the last year of the
3
The following independent variables were used: company experience, exports per worker in the
initial period, participation of foreign capital in the initial period, ratio of skilled workers, linking the
company with organizations and institutions related to science and technology, concentration and
market share in the initial period and the size of the firm.
4
The Innovative Effort equation includes all the variables of Eq. 3 (except the size of the company)
and also the following variables: demand incentives (demand pull), supply incentives (technology
push), sources of information, financial restrictions, public resources, property rights and chains.
5
The following variables were used: company size (logarithm of the number of workers), participation
of foreign capital in the initial period and public resources.
424 J. C. Ortíz Berrú et al.
survey (2014). ki is the logarithm of physical capital per worker in the initial year
(approximate through investment in physical capital per worker), and TIi is used as an
explanatory variable and captures the impact of innovation on productivity levels (this
variable is the prediction of the knowledge production function, obtained in Eq. (4).
yi ¼ pi ki þ p2 TIi þ vi ð5Þ
In all the equations, five (5) of sectorial dummies variables are incorporated: capital
goods, consumer goods, intermediate goods, primary manufacturing, and services. In
addition, in all the equations, a variable that controls by geographic location is
included: 1, represents when the company is located in Lima or in Callao, and 0 in
another case. Finally, in all the estimates only those companies that reported having
more than ten workers at the beginning of the period were taken.
3 Results
The results in Table 1 suggest that technological innovation is on average associated
with a 56% increase in labor productivity. In order to verify the robustness of the
outputs, the same model was tested using the prediction of the innovative effort. The
outputs show that on average the impact of technological investment is 0.22, this means
that if investment in technological activities increases by 1%, labor productivity
increases by 0.22%.
Table 1. Impact of innovation on productivity
Variable Y: Logarithm of sales per
worker in 2014
Logarithm of physical capital per worker in 2012 0.256*** 0.267***
(0.017) (0.016)
Log. of the number of workers in 2012 −0.136*** −0.153***
(0.021) (0.021)
IEp_p (Prediction of the innovative effort in technological 0.224***
innovation) (0.037)
TI_p (Technological innovation) 0.563***
(0.103)
Constant 8.382*** 9.650***
(0.309) (0.204)
Observations 1,155 1,155
R-squared 0.323 0.31
Source: National Survey of Innovation in the Manufacturing Industry 2015.
Standard errors in parentheses; ***p < 0.01, **p < 0.05, *p < 0.1.
Elaboration: Authors’ calculations.
Note: Reported coefficients represent Average Marginal Effects. For discrete variables, the
marginal effect is calculated as the first difference with respect to the base category. Estimation
errors are corrected by BOOTSTRAP - 500 repetitions.
Impact of Technological Innovation on the Productivity 425
Table 2. Heterogeneous impact of technological innovation
Y: Logarithm of sales per worker in 2014
Q15 Q25 Q50 Q75 Q90
Logarithm of physical capital 0.254*** 0.237*** 0.257*** 0.260*** 0.279***
per worker in 2012 (0.026) (0.026) (0.018) (0.024) (0.035)
Log. of the number of workers −0.056** −0.108*** −0.157*** −0.239*** −0.266***
in 2012 (0.025) (0.031) (0.027) (0.028) (0.044)
TI_p (Technological 0.332** 0.383** 0.422*** 0.634*** 0.714***
innovation) −0.157 −0.15 −0.118 −0.145 −0.22
Constant 8.455*** 9.222*** 9.856*** 10.697*** 11.232***
(0.322) (0.361) (0.241) (0.282) (0.35)
Observations 1,155 1,155 1,155 1,155 1,155
Source: National Survey of Innovation in the Manufacturing Industry 2015.
Standard errors in parentheses; ***p < 0.01, **p < 0.05, *p < 0.1.
Elaboration: Authors’ calculations.
Note: Reported coefficients represent Average Marginal Effects. For discrete variables, the
marginal effect is calculated as the first difference with respect to the base category. Estimation
errors are corrected by BOOTSTRAP - 500 repetitions.
One way to show if the impacts of innovation on productivity are heterogeneous, is
through the quantile regression approach. The results show that the returns of inno-
vation depend on the position of the firm according the distribution of productivity.
Such as in Table 2, firms located in quantile 15, private returns are not more than 33%.
However, returns increase around 71% to firms that are at the top of the distribution.
4 Conclusions
There is a positive impact of technological innovation on the labor productivity of
Peruvian manufacturing firms. In addition, technological innovation on average is
associated with a 56% increase in labor productivity. These results are in the range of
values reported in the Crespi & Zuñiga study (2010): Argentina (24%), Chile (60%),
Colombia (192%), Panamá (165%), and Uruguay (8%). On the other hand, using the
prediction of the innovative effort (EI), the results show that a 1% increase in tech-
nological activities investment increases labor productivity by 0.22%. This value is into
the range of values reported by Crespi & Zuñiga (2010) for Latin American countries:
Argentina (0.41%), Chile (0.20%), Colombia (0.61%), Panama (0.69%), and Uruguay
(0.45%). The effects of innovation on productivity are heterogeneous and depending of
the size of the firm. In that sense, according to the quantile regression approach, it was
found that, firms that carry out technological innovation activities and they are located
in quantile 15, their productivity increases by 33%; in contrast, those firms that are in
the quantile 90, their productivity increase in 71%. Finally, innovation plays an
important role in the productivity of the firms. In addition, the quantile regression
approach showed that the effect of innovation on the productivity of the firms is
heterogeneous. This result implies different effects on aggregate efficiency, so business
policy programs should not be transversal. It means that business support programs
have to be based on selective policies.
426 J. C. Ortíz Berrú et al.
References
1. Álvarez, R., Bravo, C., Navarro, L.: Innovación, investigación y desarrollo, y productividad
en Chile. Revista CEPAL 104, 141–166 (2011)
2. Lambardi, G., Mora, J.: Determinantes de la innovación en productos o procesos: EL caso
colombiano. Revista de Economía Institucional XVI(31), 251–262 (2014)
3. Lee, C.: The determinants of innovation in the malaysian manufacturing sector: an
econometric analysis at the firm level. s.l.: University of Malaya. Centre on Regulation and
Competition (2004)
4. Crespi, G., Katz, J.: R&D expenditure, market structure and “technological regimes” in
chilean manufacturing industries. Estudios de Economía 26(2), 163–186 (1999)
5. Crespi, G., Zúñiga, P.: Innovation and productivity: evidence from six Latin American
countries. s.l.: IDB Working Paper Series No. IDB-WP-218 (2010)
6. Crespi, G., Zuñiga, P.: Innovation and productivity: evidence from six Latin American
countries. IDB Working Paper Series No. IDB-WP-218, pp. 1–38 (2010)
7. Banco Interamericano de Desarrollo: Cómo repensar el desarrollo productivo? Políticas e
instituciones sólidas para la transformación económica. s.l.: BID (2014)
8. World Economic Forum: Global Competitiveness Report 2015-2016. World Economic
Forum, Ginebra (2015)
9. Ministerio de la Producción: Desempeño Exportador del Sector Manufacturero por
Innovación Tecnológica: Cambios en la estructura exportadora y productiva de bienes
industrializados. Lima: s.n. (2015)
10. Tello, M.: Firms Innovation, Constrains and Productivity: The Case of Peru. Departamento
de Economía PUCP, Lima, Documento de Trabajo 382 (2014)
11. Crepón, B., Duguet, E., Mairesse, J.: Research, innovation and productivity: an econometric
analysis at the firm level. National Bureau of Economic Research, Cambridge (1998)
12. Ministerio de la Producción: Encuesta Nacional de Innovación en la Industria Manufacturera
2012. PRODUCE, Lima (2013)
Parametric Urban Design
Rongrong Gu(&) and Wuzhong Zhou
School of Design, Shanghai Jiao Tong University,
Minhang District, Shanghai 200240, China
{skygrr,wzzhou}@sjtu.edu.cn
Abstract. Urban design is the process of designing and shaping the city,
involving large-scale buildings, streets, and public space organizations.
Accompanied by technological improvement, and increasing computational
power, new methods and technologies become more and more important in
urban design. The parametric design is to treat urban design as a complex
system. When basic design parameter relationships are set, we have to find an
algorithm to construct a parameter relationship to generate a shape. Based on the
introduction and application of parametric design, this paper takes several urban
design projects for example, and proposes the development ideas and design
methods of parametric urban design. Finally, this paper states that parametric
design can give us what kind of reform for urban design at a global scale.
Keywords: Urban design Parametric Algorithm generating
1 Urban Design
Urban design is related to the characteristics and pattern of the entire city. A good
urban design is the soul of the whole city. For the new requirements of urban design in
the new era, we should take advantage of the advantages of high technology to enhance
the scientific nature of urban design. In the development of contemporary cities, human
activities have a major impact on the development of cities. Urban planning is usually
carried out from the top down, while the urban sports model is bottom-up. Together
they determine the shape and movement of the city.
Parametric
Parameterization is a new design methodology, which emerged from the end of the
19th century to the mid-twentieth century. Parametric design in architecture can date
back to the hanging chain model created by Gaudi [1]. It has even been suggested that
parametric design is fundamental to creativity, in a sense that design variations can be
generated by altering values of different parameters [2]. Having design variations is
also the key of creativity [3]. The breakthrough development of industrial technology
has caused dramatic changes in people’s living environment, and the social and eco-
nomic systems have presented unprecedented complexity. The parametric design
software itself cannot explain the profound shift from modernism to parametric style.
The essence of parametric design is the process of controlling the quantization
parameter to generate multiple results through computer software setting rules or logic
functions, which interprets the design purpose in a digital way, constructs design logic,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 427–432, 2020.
https://doi.org/10.1007/978-3-030-39512-4_67
428 R. Gu and W. Zhou
solves design problems, and then Look for newer forms and more design-optimized
solutions.
Parametric design is controlled by parameters. Each parameter controls or indicates
some important property of the design result. Changing the value of the parameter will
cause the design result to change.
Early parametric urban design work such as ‘parametric urbanism’ of Patrik
Schumacher (2008) and zaha Hadid included formal contextual features of rivers, roads
and topography [4]. Recent parametric urban design projects focus on the need for
broader and more complex geospatial scales understanding.
With the notion of parametric urbanism, Zaha Hadid Architects also won a series of
planning competitions including One-North Masterplan in Singapore and Soho City in
Beijing, China [5]. It has also been suggested that parametric modelling could be used
to generate urban design solutions in high-density cities such as Hong Kong [6].
2 Design Steps
In parametric design, we regard the main factors affecting design as parametric vari-
ables. First, we find some important design requirements as parameters, and construct
parameter relationships by using one or several rule systems (i.e., algorithms) as
instructions. The computer language is used to describe the parameter relationship to
form the software parameter model. When the parameter data information is input in
the computer language environment and the algorithm instruction is executed at the
same time, the shape target can be realized and the prototype shape can be obtained.
2.1 Organization of Regional Analysis and Design Conditions
Through the investigation of the plot area, the analysis of human behavior and activ-
ities, various information related to the design is collected, and one or more factors are
found out as the control factors for generating the analysis model in the parameteri-
zation process.
2.2 Build a Parametric Model
According to the analysis of the influence design factors in the previous step, find the
control factors of the project and the problems faced in the design, and choose the
appropriate algorithm in combination with other professions to construct the parametric
model.
2.3 Correct and Optimize the Parametric Model
Optimize the parametric model obtained in the previous step, especially for the opti-
mization of important control factors.
Parametric Urban Design 429
2.4 Improve the Parametric Model
Complete the details in accordance with the analytical model and deal with the primary
and secondary relationships.
2.5 Detection and Adjustment of Overall Design Results
The improved parametric model is tested and the software is used to adjust the overall
spatial structure of the city. If the expected requirements are not met, the control factors
of the parametric model must be re-evaluated.
3 Algorithm Generating
An algorithm is a series of logical decisions and operations that are organized in order,
that is, instructions that collectively accomplish a particular task. Here, a specific task is
to construct a parameter relationship. We use two projects to illustrate the process of
designing shape-finding.
3.1 Kartal Pendik Masterplan Breakdown/Zaha Hadid
Where routes connecting Europe and Asia meet coastal highways, sea terminals and
rail links in an abandoned industrial area of Istanbul, the Kartal Pendik masterplan is
taking shape – creating a new urban centre based on grid form and utilizing calligraphic
notions of topography to create truly responsive structures and spaces.
The project starts with the integration of the infrastructure and the urban context of the
surrounding lots. The horizontal lines will be linked from the main roads in Kartal in the
west and Pendik in the east. These lateral connections, together with the main longitudinal
axes, create a soft grid that forms the underlying framework of the project. This texture is
further refined through the draft urban design – different building types are created for
each region’s different needs. The open conditions of the draft production can transform
an independent building into a block and eventually become a hybrid system – creating a
porous, interconnected open space network that is free from the city (Fig. 1).
Fig. 1. Marek Kolodziejczyk, Wool-thread model to compute optimised detour path networks,
Institute for Lightweight Structures (ILEK), Stuttgart, 1991 (Source: https://tshristov.wordpress.
com/2015/05/08/kartal-pendik-masterplan-by-zaha-hadid-architects-work-in-progress/)
430 R. Gu and W. Zhou
A solution was calculated using the Rhino and Grasshopper applications. The
principle being to adjust the length of the thread. After connecting the intersections
between the region of interest and the main track, a mesh is obtained. It is then refined
to form a wool line model simulation (Fig. 2).
Fig. 2. Wool-thread simulation output (Source: https://tshristov.wordpress.com/2015/05/08/
kartal-pendik-masterplan-by-zaha-hadid-architects-work-in-progress/)
The deformation curve is created by dividing the longitude curve into segments and
adjusting the control points on the curve. Create a road by dividing the area into
inwardly set areas or blocks with longitude and latitude curves (Fig. 3).
Fig. 3. Deformed grid (Source: https://tshristov.wordpress.com/2015/05/08/kartal-pendik-
masterplan-by-zaha-hadid-architects-work-in-progress/)
Parametric Urban Design 431
Next step is to use the squeeze command. The height of the extrusion is defined by
the distance from the zone to the middle curve, and ultimately the greater the distance,
the higher the height of the extrusion (Fig. 4).
Fig. 4. Volumes by distance to mid curve (Left) Volumes by area m2 (Right) (Source: https://
tshristov.wordpress.com/2015/05/08/kartal-pendik-masterplan-by-zaha-hadid-architects-work-in-
progress/)
3.2 One North Masterplan/Zaha Hadid
One North has created a typical form in Singapore that defines the definition of a man-
made landscape to a residential area of the city. Urban buildings have the spatial
composition and form of natural landscapes (Figs. 5 and 6).
Fig. 5. Traffic grid (Source: http://www.ikuku.cn/concept/one-north-masterplan)
432 R. Gu and W. Zhou
Fig. 6. Design results (Source: http://www.ikuku.cn/concept/one-north-masterplan)
4 Conclusion
As a complex integrated system, the city’s diverse characteristics give parametric
design extended platform and potential for future expansion. The parametric design
approach is an interdisciplinary subject with many elements, and the application in
urban design is at the beginning and exploration stage. The application of advanced
technology makes it possible to build complex designs. The city of the future will
become an organic whole and become the crystallization of emotion and wisdom.
References
1. Frazer, J.: Parametric computation: history and future. Architectural Des. 86(2), 18–23 (2016)
2. Lee, J., Gu, N., Williams, A.: Exploring design strategy in parametric design to support
creativity. In: Proceedings of International Conference on Computer-Aided Architectural
Design Research in Asia (2013)
3. Liu, Y.-T., Lim, C.-K.: New tectonics: a preliminary framework involving classic and digital
thinking. Des. Stud. 27(3), 267–307 (2006)
4. Speranza, P.: Using parametric methods to understand place in urban design courses. J. Urban
Des. 21(5), 661–689 (2016)
5. Schumacher, P.: Parametricism: a new global style for architecture and urban design.
Architectural Des. 79(4), 14–23 (2009)
6. Schnabel, M.A., Zhang, Y., Aydin, S.: Using parametric modelling in form-based code design
for high-dense cities. In: Proceedings of International High-Performance Built Environment
Conference (iHBE 2016) (2016)
Narrative Review of the Role of Wearable
Devices in Promoting Health Behavior: Based
on Health Belief Model
Dingzhou Fei(&) and Xia Wang
Department of Psychology, Wuhan University, Wuhan, China
[email protected]
Abstract. Background: Currently, wearable devices, such as smartwatches,
allow people to monitor health-related data, which help a better sense of their
health status. But it still has a long way to go to really prove whether the
perceptions could translate into health-related behavior which require individual
be responsible for their own health. Purpose: The purpose of this paper is to
investigate the possibility or effectiveness and limitations of wearable devices in
health behavior intervention. Firstly, we explored the factors related to wearable
devices and health self-management. Secondly, this paper attempted to get
whether these factors related to health self-management could induce health
behavior based on the Health Belief Model. Methods: The paper was organized
as follow. We first reviewed the literature on wearable devices and then
attempted to explain the possibility and effectiveness of wearable devices in
promoting health behavior according to the concept in Health Belief Model.
Finally, this paper gave a preliminary conclusion about the issues we were
discussing.
Keywords: Health Belief Model Wearable device Health behavior Fitness
1 Introduction
Self-quantification has set off a new trend. People get self-tracking data to manage
themselves. Data which is generated from Self-quantification systems (SQS) is con-
sidered as catalyst for behavioral change [1]. Wearable devices are important inno-
vation among self-tracking tool. People use wearable devices to monitor blood pressure
or heart rate and obtain activity data such as number of steps and calorie consumption.
We considered wearable devices reflect need for self-management. However, it’s not
covered by current study whether need for self-management triggers behavioral
change. With popularity of smartphone applications, usability of wearable devices is
greatly enhanced, and the number of people using wearable devices is growing. This
review aims to describe patterns that wearable devices can solve as well as defects
based on Health Belief Model (HBM), to explore health behavior intervention effec-
tiveness and limitation. Given breadth of available research, a systematic review would
be inappropriate. Thus, we try to unearth evidence through narrative review. In addi-
tion, we focused on wearable devices used in fitness as it’s different from that for
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 433–438, 2020.
https://doi.org/10.1007/978-3-030-39512-4_68
434 D. Fei and X. Wang
medical use. Users have control of data, and self-management needs originate from
themselves rather than requirements of medical professionals. So, can data fed back to
users by wearable devices lead to health behavior when users are in charge and
responsible for data? This review explains the issue through HBM, which is an
important model to carry out health behavior intervention. Studies have proved that
HBM variables are consistent with behavior [2].
2 Health Belief Model
HBM suggested that behavioral change depended on whether people realized that the
event could be prevented and they could avoid it by taking certain actions [3]. Becker
et al. improved model by including Self-Efficacy, that is behavioral change also
depends on whether people have confidence to carry out certain behaviors successfully
that could avoid the event. Please see Fig. 1 for HBM. According to HBM, the for-
mation of health behavior requires following conditions, higher Perceived Benefits,
lower Perceived Barriers, higher Perceived Threat (including Perceived Seriousness
and Perceived Susceptibility), higher Self-Efficacy, Cues to Action and Modifying
Variables. We believe that the fitting degree of wearable devices to HBM can reveal the
possibility, effectiveness and limitation of behavioral change.
2.1 Support for Perceived Benefits vs. Perceived Barriers
Perceived benefits and perceived barriers (two strongest predictors of HBM [2]) con-
tributed to behavioral change in positive and negative directions. HBM believes
behavior change when perceived benefits are higher than perceived barriers.
Support for Perceived Benefits. Self-tracking tool allow you to monitor yourself and
conduct self-experiment, though validity of self-experiment is questionable. The
increase in use of wearable devices leads to healthy lifestyle, and users are more active
[4] which can be transferred into perceived benefits. In other words, wearable devices
have potential to promote health behavior [5]. They allow individual to set and achieve
goals, then adjust behavior by feedback. One study has divided health self-
quantification into two stages, namely data management and health activation. The
first is a series of activities involved in collecting, managing and transforming data.
Secondly, a mapping has been formed between available information and individual’s
goals to build self-awareness of health [6].
Fig. 1. Health belief model
Narrative Review of the Role of Wearable Devices in Promoting Health Behavior 435
Support for Perceived Barriers. Wearable devices are notorious for privacy issues.
Self-quantification means the beginning of an era of personal control over data, but the
extent to which data is controlled is uncertain. It is argued that self-tracking data is
generated by interaction between user and device [7], therefore, belongs to both device
manufacturing company and user. People expect others to notice they are using
wearable devices as a sign of self-discipline. In contrast, people hope to share data in
very limited circumstances. Data privacy has to be said to be stumbling block to
people’s decision to start and continue using wearable devices. It can’t be ignored
Because it hinders health behavior. What’s more, One crucial problem is that people
are not clear about how to interpret data [7]. As a result, people can’t define criteria for
next action trigger, overestimate or underestimate severity of problem. Besides, control
over data means not only data is generated by you, but also digested by you. We
believe that obstacles in data restrict catalytic effect of wearable devices on health
behavior.
2.2 Support for Perceived Threat (Perceived Seriousness
and Susceptibility)
HBM argues that higher perceived threats are more likely to lead to health behavior,
people often feel threated in comparison with others.
Support for Perceived Seriousness. Research shows that quantitative self-data leads
to more reflective thinking and thoughtful analysis [8]. This means that self-monitoring
data provided by wearable devices can give users meaningful feedback which is
intuitive sense of seriousness of health problem. An obese person, for example, can
face reality of being overweight and experience severity of problem in the difference
between standard weight and actual weight through feedback provided.
Support for Perceived Susceptibility. Perceived Susceptibility is a subjective
assessment of risk of health problems. Wearable devices are designed for people who
are in bad health, we have no doubt that self-tracking data is of great reference value to
evaluate health and possibility of health problem that may be overlooked by doctors.
Self-quantification is not a tool for self-optimization, but an opportunity for self-
discover [9]. The discover was accompanied by judgment of perceived susceptibility.
2.3 Support for Self-Efficacy
Self-Efficacy was added when social psychologists refined model which make HBM be
applied to substantial, long-term behavioral change. Whether individuals consider they
have ability to influence change is thought to be key factor in change of health behavior.
Quantitative data increase goal-directed motivation which is important initial step
for individuals in behavioral change [10]. In addition, wearable devices with fitness
assistance have potential to encourage more positive health behavior [11]. Hence,
it’s necessary to provide contextual feedback based on user’s understanding for data.
436 D. Fei and X. Wang
More feedback users get, more aware they are of their situation, and more confident they
are in implementing behavioral change. The comparison with others from feedback will
increase belief that they can carry out certain behaviors leading to change. Feedback
provided by wearable devices enhance Self-Efficacy and promote health behavior.
2.4 Cues to Action
There have been few defined about what is cues to action, as cues can be internal like
physical cues (pain, symptoms) or external like information from others [3]. It depends
on people’s self-report which missed unconscious clues, so It’s difficult to specify the
clues that trigger behavior as anything before behavioral change is possible. But use of
wearable devices is a clue to action, collection, monitoring and feedback triggered
behavioral change, so we hold a positive attitude towards next change in behavior.
2.5 Modifying Variables
Individual characteristics, including demographic, psychological, and structural vari-
ables, are considered Modifying Variables of HBM. Modifying variable affect health
behavior indirectly by influencing perceived seriousness, susceptibility, benefits and
barriers [12]. But current defect of HBM is that there is no evidence to support
moderating effect of Modifying Variables [13]. Therefore, we don’t discuss modifying
variables involved in wearable devices. However, it should be noted that interaction
between modifying variables and impact of wearable devices on health behavior can’t
be ignored. It was showed that there are gender differences in wearable devices from
beginning of product design, and influence of gender differences exist in user expe-
rience [14]. After reviewing, we find that wearable devices have some characteristics
corresponding to variables of HBM (see Table 1) except for Cues to Action and
Modifying Variables. Besides, the interaction between remaining variables and Mod-
ifying Variables is also an issue worth paying attention to in the future.
Table 1. The fit between variables of HBM and characteristics of wearable devices
No Variables of HBM Characteristics of wearable devices
1 Perceived benefits Self-experiments
Perceived usefulness
Self-monitoring
Mapping between the Data and individual goals
2 Perceived barriers Privacy concern
Unclear data interpretation
Degree of data control
3 Perceived Perceived Rely on intuitive feedback
threat seriousness
Perceived Self-descovery
susceptibility
(continued)
Narrative Review of the Role of Wearable Devices in Promoting Health Behavior 437
Table 1. (continued)
No Variables of HBM Characteristics of wearable devices
4 Self-efficacy Increased goal orientation
Timely feedback
5 Cues to action Outside the scope we discussed (Clues are hard to
define)
6 Modifying variables Outside the scope we discussed (Influence behavior
indirectly by influencing variables 1, 2, 3, and 4)
3 Conclusion
Although existing studies are limited, they fit HBM to some extent. We can see that
wearable devices contribute to both perceived benefits and barriers, two of most
important predictors of HBM. Although there is no data to support relative degree of
the two, we consider that wearable devices may be effective in promoting health
behavior when perceived benefits are higher than barriers. In terms of perceived threat,
wearable devices provide direct feedback through intuitive and specific numbers, which
is cornerstone of self-efficacy. For modifying variables, we believe that interaction will
affect effectiveness of wearable devices in inducing behavioral change.
References
1. Fiore-Silfvast, B., Neff, G.: What we talk about when we talk data: valences and the social
performance of multiple metrics in digital health. Ethnographic Prax. Ind. Conf. 2013(1),
74–87 (2013)
2. Sulat, J.S., Prabandari, Y.S., Sanusi, R., Hapsari, E.D., Santoso, B.: The validity of health
belief model variables in predicting behavioral change: a scoping review. Health Educ. 118,
499–512 (2018)
3. Janz, N.K., Becker, M.H.: The health belief model: a decade later. Health Educ. Behav. 11,
1–47 (1984)
4. Lunney, A., Cunningham, N.R., Eastin, M.S.: Wearable fitness technology: a structural
investigation into acceptance and perceived fitness outcomes. Comput. Hum. Behav. 65,
114–120 (2016)
5. Rachuri, K.K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C., Aucinas, A.:
EmotionSense: a mobile phones based adaptive platform for experimental social psychology
research. In: 12th International Conference on Ubiquitous Computing, UbiComp 2010,
pp. 281–290 (2010)
6. Almalki, M., Martin Sanchez, F., Gray, K.: Quantifying the activities of self-quantifiers:
management of data, time and health. In: Georgiou, A., Sarkar, I.N., de Azevedo Marques,
P.M. (eds.) 15th World Congress on Health and Biomedical Informatics, MEDINFO 2015,
vol. 216, pp. 333–337. IOS Press (2015)
7. Neff, G., Nafus, D.: Self-Tracking. MIT Press, Cambridge (2016)
438 D. Fei and X. Wang
8. Li, I., Dey, A., Forlizzi, J.: A stage-based model of personal informatics systems. In: 28th
Annual CHI Conference on Human Factors in Computing Systems, CHI 2010, pp. 557–566
(2010)
9. Wolf, G.: The Data-Driven Life. New York Times (2010)
10. Pettinico, G., Milne, G.R.: Living by the numbers: understanding the “quantification effect”.
J. Consum. Mark. 34, 281–291 (2017)
11. Winchester, W.W., III, Washington, V.: Fulfilling the potential of consumer connected
fitness technologies: towards framing systems engineering involvement in user experience
design. In: 11th Annual IEEE International Systems Conference, SysCon 2017. Institute of
Electrical and Electronics Engineers Inc. (2017)
12. Rosenstock, I.M.: Historical origins of the health belief model. Health Educ. Behav. 2, 328–
335 (1974)
13. Choi, S.H., Duffy, S.A.: Analysis of health behavior theories for clustering of health
behaviors. J. Addict. Nurs. 28, 203–209 (2017)
14. Zhang, Y., Rau, P.L.P.: Playing with multiple wearable devices: exploring the influence of
display, motion and gender. Comput. Hum. Behav. 50, 148–158 (2015)
Competitiveness of Higher Education System
as a Sector of Economy: Conceptual Model
of Analysis with Application to Ukraine
Olha Hrynkevych1, Oleg Sorochak2, Olena Panukhnyk3,
Nazariy Popadynets4(&), Rostyslav Bilyk5,6, Iryna Khymych3,
and Yazina Viktoriia7
1
Ivan Franko National University of Lviv, Svobody Ave. 18,
Lviv 79008, Ukraine
[email protected]
2
National University “Lviv Polytechnic”, Bandery str., 12, Lviv 79013, Ukraine
[email protected]
3
Ternopil Ivan Puluj National Technical University, Ruska str. 56,
Ternopil 46001, Ukraine
[email protected], [email protected]
4
SI “Institute of Regional Research named after M. I. Dolishniy of the NAS
of Ukraine”, Kozelnytska str. 4, Lviv 79026, Ukraine
[email protected]
5
Yuriy Fedkovych Chernivtsi National University, Kotsjubynskyi str. 2,
Chernivtsi 58012, Ukraine
[email protected]
6
Military Institute of Taras Shevchenko National University of Kyiv,
Lomonosov str. 81, Kiev 03189, Ukraine
7
University of Customs and Finance, Vladimir Vernadsky str. 2/4,
Dnipro 49000, Ukraine
[email protected]
Abstract. This research aims to lay out a framework for analysis the com-
petitiveness of higher education as one of the service sector of economy. The
study offers a conceptual model for the analysis and is based on the criteria of
quality, social responsibility and economic efficiency. The interests of stake-
holders, the potential of the education providers form a core of the model and
framework for construction of indicators on higher education system competi-
tiveness analysis according to abovementioned criteria. The proposed model and
corresponding indicator frame was applied for the analysis of the higher edu-
cation system competitiveness of Ukraine’s regions. Cluster analysis allowed
selecting groups of Ukraine’s regions that differ significantly in the level of
higher education system competitiveness and require a differentiated govern-
mental policy as well as financial support.
Keywords: Higher education competitiveness Model of analysis Cluster
analysis quality Economic efficiency Social responsibility Ukraine
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 439–445, 2020.
https://doi.org/10.1007/978-3-030-39512-4_69
440 O. Hrynkevych et al.
1 Introduction
In recent years, there has been significant increase in the number of studies concerning
the higher education (HE) system. In a global study of directions, publications and the
institutional structure of higher education research, P. Altbach concludes that despite
the growth in the number of research centres, ‘higher education is a field without a clear
intellectual, methodological, or disciplinary center [1]. A diversity of approaches
reflects the diverse interests and backgrounds of those involved in the field’.
T. McCowan proposes an original model for studying the evolution of HE which is
comprised of three components: (1) value; (2) functions; (3) interaction with the
external environment. This model allows to explain the increase of contribution of the
economic efficiency and social interaction of the HEIs in the analysis of its competi-
tiveness [2].
Analyzing the HEIs management in terms of balancing the interests of various
stakeholders, U. Teichler focuses on the criterion of quality, relevance and efficiency
[3]. Based on the integrated analysis of the HE functions and its goals in the global
strategic development, S. Panchyshyn and O. Hrynkevych concludes that quality,
social responsibility and economic efficiency are the main criteria for the analysis of
competitiveness of HE systems at all levels its functioning [4].
The ranking company Quacquarelli Symonds, annually develops and publishes a
ranking of national systems of HE effectiveness titled ‘QS Higher Education System
Strength Rankings’ [5]. The methodology includes four components of assessment: (1).
System strength; (2) Access; (3) Flagship Institution; (4) Economic context. However
this ranking is based only on the positions of the country’s HEIs, which fall into the
ranking, that is formed by the same company. Secondly, the ranking does not take into
account the overall economic potential of the country’s system of HE and its funding
by different stakeholders. Thirdly, the ranking does not reflect the implementation of all
HE functions (educational and scientific activities, public services and promoting the
goals of sustainable development). Based on this analysis we found that there is no
ranking that is equally informative and useful for various stakeholders and we agree
with the conclusions of Goglio [6] that ‘an active role of researchers in the field of
university rankings is desirable, for monitoring improvements on methodological
issues and for making better-informed ranking consumers’.
This study aims to lay out an interdisciplinary approach and conceptual model for
the analysis of the HE system competitiveness that is based on the criteria of quality,
economic efficiency and social responsibility. The proposed conceptual model and
cluster analysis were applied for assessment of the HE system competitiveness in the
Ukraine’s regions.
2 Methodology
Using the interdisciplinary approach based on the institutional theory [7], the theory of
human capital [8, 9], the theory of social capital [10], the theory of intellectual capital
[11], the theory of stakeholders [12], the theory of competitive advantage [13] we
propose the definition of the HE as a system of informal (values, traditions etc.) and
Competitiveness of Higher Education System as a Sector of Economy 441
formal (laws, standards, HEIs, stakeholders, etc.) institutions that provide an under-
standing of the critical role of knowledge in self-improvement, accumulation, trans-
mission and generation of new knowledge for the purposes of personal and social
development.
The main components of the institutional structure of the national system of HE can
be defined as following: education providers, their products, internal and external
stakeholders. Based on the introduced main components it is proposed to define the
competitiveness of the HE system as the ability of their providers to create products
that generate benefits in human, social and intellectual capital development, and,
accordingly, in realizing of the individual, national and global goals of development.
The frameworks for constructing of indicators of the HE system competitiveness by
the criteria of quality, social responsibility and economic efficiency were developed
using the methods of system analysis.
Cluster analysis was applied in order to classify Ukraine’s regions based by the set
of competitiveness indicators. A tree-like method proposed by J. Ward uses the vari-
ance analysis to estimate the distances between clusters [14]. The method is aimed at
connecting clusters that are close to each other’s but it seeks to create small-sized
clusters, which is not always convenient for a large number of observations.
3 Results and Discussion
The interdisciplinary approach to the competitiveness analysis of the HE systems
involves determination reserves for its increase, taking into account the interests of
stakeholders, the potential and performance of education providers, as well as the levels
of the HE system: HEI, region, country. Figure 1 provides a conceptual model for
analysis of the higher education system competitiveness.
Quality
Stakeholders
(interests)
Products
(results)
Providers
(potential)
National level
Fig. 1. Conceptual model for analysis of the higher education system competitiveness Source:
developed by the authors
442 O. Hrynkevych et al.
The components of the HE system – providers, products and stakeholders, as well
as criteria for their evaluation – quality, social responsibility and economic efficiency –
form the basis for the competitiveness analysis. The next level of the conceptual model
of analysis takes into account the practicability and achievability of its implementation
at various levels of the HE system functioning: local/ institutional, regional, national.
Figure 2 presents the framework for construction of indicators of HE system
competitiveness analysis according to criteria of quality, social responsibility and
economic efficiency.
Fig. 2. Framework for construction of indicators of HE system competitiveness analysis
according to criteria of quality, social responsibility and economic efficiency Source: developed
by the authors
Proposed in Figs. 1 and 2 conceptual approaches can serve as a basis for the
development of analytical databases for decision-making at different levels of com-
petitiveness management. In other words, it refers to the methodology for the formation
of large data sets (Big-data) in the management of HE system both at the institutional
(HEI) and the regional/national levels.
Cluster analysis was applied in order to classify regions of Ukraine by the set of
competitiveness indicators. Figure 3 shows a tree of the clustering results of the 25
regions of Ukraine using the Ward’s method.
As Ward’s clustering results show, four major types of regions in Ukraine can be
identified in terms of the competitiveness of the HE system. The first cluster includes
the capital of Ukraine – Kyiv with universities that provide the highest level of quality
according to such criteria as ‘the education quality of HEIs entrants’, ‘the academic
reputation of universities in the national ranking’, ‘the attractiveness of the region HEIs
among students’ and ‘prospects for employment of graduates’ The second cluster
combines eleven regions of Ukraine (Odesa, Luhansk, Donetsk, Kharkiv, Lviv, Ivano-
Frankivsk, Transcarpathian, Chernivtsi, Sumy, Poltava, Dnipropetrovsk), with gener-
ally higher than average level of most indicators of competitiveness, in particular those
that evaluate criteria such as ‘the academic reputation of academic staff’, ‘the effec-
tiveness of research’, ‘attractive for students from other regions’, but with a rather large
variability of academic rankings of universities. Among the regions that have fallen
into this cluster, the Lviv and Kharkiv regions are distinguished with the highest values
Competitiveness of Higher Education System as a Sector of Economy 443
Fig. 3. Cluster analysis of the Ukraine’s regions according to the level of the higher education
system competitiveness: the tree-like method Source: developed by the author based on data
[15–18]
of the academic reputation of staff, positions in national and international rankings and
the education quality of entrants.
The third cluster includes the six regions (Kherson, Mykolaiv, Zaporizhzhia,
Kropivnitsky, Kyiv, Zhytomyr), that have values of most competitiveness indicator
somewhat lower than the average level. At the same time, this cluster is characterized
by a rather high level of training for certain branches of the economy, but the absence
of classical universities with high positions in national and international rankings.
The fourth cluster comprises of seven Ukraine’s regions (Chernigiv, Cherkasy,
Khmelnytsky, Rivne, Volyn, Ternopil, Vinnytsya), that are characterized by a marked
lagging behind the top regions by most indicators of competitiveness and the urgency
of the problem of entrants migration to neighbouring regions with higher educational
and economic potential. At the same time, each of the regions of this cluster has its own
strengths. For example, Vinnytsia region has a high level of patent activity of
researchers and the attractiveness of medical profile for foreign students. The high level
of regional financial support for the HEIs is distinctive for Volyn, Ternopil, and
Chernihiv regions.
444 O. Hrynkevych et al.
4 Conclusions
The present study offers a conceptual model for the analysis of the HE system com-
petitiveness and is based on the following key points: 1. The main elements of the HE
system of the region/country are education providers, the results of their activities
(products), as well as internal and external stakeholders. The interests of stakeholders,
the potential of the providers form a core of the model and are the main objects in the
competitiveness analysis. 2. Quality, social responsibility and economic efficiency are
the main criteria for the analysis of the HE system competitiveness. 3. It is beneficial to
develop various types of applied models for analysis of the HE system competitiveness
and corresponding sets of indicators that depend on the level of analysis (institutional,
regional, national) and stakeholder priorities.
The conceptual model was implemented for the empirical analysis of the HE
system competitiveness of Ukraine’s regions. Based on a set of indicators of quality,
social responsibility and economic efficiency the Ukraine’s regions were divided into
groups using hierarchical clustering algorithm. The implementation of the methods
allowed to classify 4 groups of Ukraine’s regions with different levels of the HE system
competitiveness. The results can be used to improve the governmental and regional
management in order to balance the development of regional systems of higher
education.
References
1. Altbach, P.: Knowledge for the contemporary university: higher education as a field of study
and training. In: Rumbley, L.E., Altbach, P.G., Stanfield, D.A., Shimmi, Y., de Gayardon, A.,
Chan, R.Y. (eds.) Higher Education: A Worldwide Inventory of Research Centers, Academic
Programs, and Journals and Publications, 3rd edn., pp. 12–21. Center for International Higher
Education, Boston College & Lemmens Media, New York (2014)
2. McCowan, T.: Universities and the post-2015 development agenda: an analytical
framework. High. Educ. 72(4), 505–523 (2016)
3. Teichler, U.: Steering in a Modern Higher Education System: The Need for Better Balances
between Conflicting Needs and Expectations. International Centre for Higher Education
Research, Kessel (2019)
4. Panchyshyn, S., Hrynkevych, O.: The conceptual apparatus for institutional analysis of the
higher education system competitiveness. Econ. Dev. 1(81), 50–58 (2017)
5. Quacquarelli Symonds: QS Higher Education System Strength Rankings: Methodology
(2018). https://www.topuniversities.com/system-strength-rankings/methodology
6. Goglio, V.: One size fits all? A different perspective on university rankings. J. High. Educ.
Policy Manage. 38(2), 212–226 (2016)
7. North, D.: Institutions, Institutional Change and Economic Performance. Cambridge
University Press, Cambridge (1990)
8. Schultz, T.: Investment in human capital. Am. Econ. Rev. 11(1), 1–17 (1961)
9. Becker, G.: Human Capital: A Theoretical and Empirical Analysis, with Special Reference
to Education. National Bureau of Economic Research, New York (2017). https://www.mdpi.
com/2227-7099/7/2/49/htm
Competitiveness of Higher Education System as a Sector of Economy 445
10. Bourdieu, P.: The forms of capital. In: Richardson, J.G. (ed.) Handbook of Theory and
Research for the Sociology of Education, pp. 241–258. Greenwood, Westport(1986)
11. Stewart, T.: Intellectual capital: the new wealth of organizations. Perform. Improv. 37(7),
56–59 (1998)
12. Freeman, R.: Strategic Management: A Stakeholder Approach, pp. 31–51. Cambridge
University Press, Cambridge (2010)
13. Porter, M., Kramer, M.: Strategy & society: the link between competitive advantage and
corporate social responsibility. Harvard Bus. Rev. 84(12), 78–94 (2009)
14. Ward, J.: Hierarchical grouping to optimize an objective function. J. Am. Stat. Assoc. 58,
236–244 (1963)
15. The State Statistics Service of Ukraine: The main indicators of the activity of Ukraine’s HEIs
at the beginning of the 2016/17 academic year: statistical bulletin (2017). http://www.ukrstat.
gov.ua
16. Ukrainian Center for Educational Quality Assessment. Statistical data of the main session of
the external independent evaluation 2016 (2017). https://zno.testportal.com.ua/stat
17. Educational portal in Ukraine. Ranking of universities on Scopus 2016 indicators (2017).
http://osvita.ua/vnz/rating/51053
18. Profrights: Localization of the violations of the rights with regard to students and staff of
HEIs at the beginning of 2017 (2017). https://profrights.org/visual
Application of Classification Algorithms
in the Generation of a Network
Intrusion Detection Model Using
the KDDCUP99 Database
Jairo Hidalgo(&) and Marco Yandún
Universidad Politécnica Estatal del Carchi, Tulcán, Ecuador
{jairo.hidalgo,marco.yandun}@upec.edu.ec
Abstract. The technological activities must be guaranteed by an acceptable
level of security both in the operations carried out by the users and in the data that
travel through the infrastructure of the networks, in this research the traffic
database is analyzed KDDCUP99 network [Knowledge Discovery in Databases]
in whose results were obtained through the conjunction matrices between the
algorithms of Neural Networks and K-NN (K Neighbors Neighbors) to deter-
mine the best classifier in the training of the model, as a first step the pre-
processing was used of the information and the analysis of the users by means of
methods of classification of entities and of specific attributes, using techniques
and tools of mining of data such as (to remove duplicate values. SimpleKMeans,
selection of more significant attributes, elimination of unused characteristics,
voracious algorithms and discrete Chi-Square attributes, the division of our
database into two random parts continued: one composed of 70% used to train the
model and another with the remaining 30% to validate the result; The results
allowed us to determine that the most effective model is not the Neural Networks
algorithm with 100% of correctly classified instances compared to K-NN with
100%.
Keywords: Multilayer perceptron Neural Networks K-NN (K Neighbors
Neighbors)
1 Introduction
Computer security plays a leading role in providing reliability, availability and non-
repudiation of data for users who require it. Several studies have been carried out on
models of intrusion detection systems using artificial intelligence techniques such as
neural networks, Bayesian networks, fuzzy logic etc., which allow modeling classifi-
cation models with high precision in intrusion detection systems (IDS).
In the article published by Reddy [1] he proposes a discriminant function of
machine-based support vector classification (SVM). An SVM is based on the theory of
approximate sets for the selection of characteristics, which uses a discriminant function
that is a hyperplane that separates different kinds of data, decreasing the dimensionality
of the set, thus the set of data obtained is properly scaled to increase the effectiveness of
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 446–452, 2020.
https://doi.org/10.1007/978-3-030-39512-4_70
Application of Classification Algorithms in the Generation 447
the discriminant function. The data used are those recovered from the KDDCUP99
Database. The result is that by combining the scale and the selection of approximate set
characteristics, the overall performance of the intruder detection system is improved.
2 Materials and Methods
2.1 Materials
In Table 1, part of the KDDCUP99 database is presented, which is constituted by a
standard set of data that has been audited and includes a wide variety of simulated
intrusions in a military network environment [2]. The database is made up of 494020
instances and 41 attributes.
Table 1. Instances and attributes of the KDDCUP99 BDD
duration service flag src_ dst_ land wrong_ urgent hot num_ Logged_ Inum
bytes bytes fragment failed_ in compromised
logins
1 0 http SF 285 34557 0 0 0 0 0 1 0
2 0 http SF 316 3665 0 0 0 0 0 1 0
3 0 http SF 335 10440 0 0 0 0 0 1 0
.
.
494020 0 http SF 219 1234 0 0 0 0 0 1 0
2.2 Methods
As part of the methods used, different algorithms were applied in the cleaning and pre-
processing of the data, in the classification, clustering, association and selection of
attributes.
Algorithms of Selection of Instances and Attributes
To perform preprocessing and processing, different data mining techniques are applied.
The same ones that allow to have a database with correct instances and attributes that
provide information selected in the study.
Pre Information Processing
Remove Duplicates
It allows to generate a data set smaller than the original but more efficient, selecting a
relevant data set, eliminating duplicate, anomalous data, selection of instances etc., for
which we use Wweka.clusterers.SimpleKMeans-init0-max-candidates 100 42 attributes
and 145585 instances (Table 2).
448 J. Hidalgo and M. Yandún
Table 2. Filter unsupervised attribute AddCluster
# Label Count Weight
1 http 3349 3349.0
2 smtp 2066 2066.0
3 finger 223 223.0
4 domain_4 1313 1313.0
5 auth 146 146.0
6 telnet 286 286.0
7 ftp 261 261.0
8 eco_i 138 138.0
9 ntp_u 104 104.0
10 ecr_i 158 158.0
11 other 593 593.0
12 private 831 831.0
13 pop_3 67 67.0
14 ftp 1455 1455.0
15 rie 15 15.0
16 time 25 25.0
17 mtp 16 16.0
18 link 15 15.0
Attribute Selection
Through the application of the Rank data mining technique that allows classification
and filtering of data characteristics by relevance, the following results presented in
Table 3 were obtained, which contains the instances and attributes that it provides with
more information for the study.
Table 3. Attributes and records of the BDDKDD99 that provides more information
# Gain ratio Gini ANOVA x2
dst_host_same_src_port_rate
label 23 0.652 0.453 NA 520855.337
service 66 0.573 0.484 NA 2476689.193
dst_host_same_srv_rate 0.487 0.250 NA 133224.694
dst_host_same_src_count 0.476 0.263 NA 142056.587
same_srv_rate 0.472 0.198 NA 48953.026
srv_count 0.446 0.359 NA 364794.982
count 0.443 0.354 NA 355186.068
srv_serror_rate 0.422 0.155 NA 280484.105
flag 11 0.414 0.217 NA 134985.862
Application of Classification Algorithms in the Generation 449
GreedyStepwise Algorithm (Ravenous Algorithms)
The voracious algorithms are widely used in data mining, in their search they have an
entry of size n that are the candidates to be part of the solution, to the subset of those n
candidates that satisfies certain restrictions are called a feasible solution, so it is
obtained the feasible solution that maximizes or minimizes a certain objective function
to achieve the optimal solution [3].
With the application of the GreedyStepwise Algorithm and the use of the
CfsSubsetEval evacuator attribute, through the selection of the classifier protocol_type
class, the attributes with more significant characteristics for the study were identified
(Table 4).
Table 4. BOD stratified cross validation KDD99
# of folds attribute
(%)
10 (100%) 1 duration
10 (100%) 3 service
10 (100%) 5 src_bytes
10 (100%) 6 dst_bytes
10 (100%) 8 wrong_fragment
Disposal of Unused Features
It allows to select instances of data according to the conditions on the characteristics of
the data [4], for this case we proceed to eliminate unused characteristics and classes
obtaining the following results (Tables 5 and 6).
Table 5. Result of the elimination of characteristics and classes
protocol_type duration service Flag src_byte dst_bytes land wrong_fragment urgent hot
1 tcp 0 http SF 181 5450 0 0 0 0
2 tcp 0 http SF 239 486 0 0 0 0
3 tcp 0 http SF 235 1337 0 0 0 0
4 tcp 0 http SF 219 1337 0 0 0 0
5 tcp 0 http SF 217 2032 0 0 0 0
6 tcp 0 http SF 217 2032 0 0 0 0
7 tcp 0 http SF 212 1940 0 0 0 0
8 tcp 0 http SF 159 4087 0 0 0 0
9 tcp 0 http SF 210 151 0 0 0 0
10 tcp 0 http SF 212 786 0 0 0 0
11 tcp 0 http SF 210 624 0 0 0 0
450 J. Hidalgo and M. Yandún
Chi-Square Discrete Attributes
Table 6. Records and attributes as a result of applying chi-square10 most prominent attributes
Protocol_type service Inum_ is_host_ label dst__host_ srv_ diff_srv_ count dst_host_
outbound_ login diff_srv_ count rate srv_serror_
cmds rate rate
1 tcp http 0 0 normal 0.00 8 0.00 8 0.00
2 tcp http 0 0 normal 0.00 8 0.00 8 0.00
3 tcp http 0 0 normal 0.00 8 0.00 8 0.00
4 tcp http 0 0 normal 0.00 6 0.00 6 0.00
5 tcp http 0 0 normal 0.00 6 0.00 6 0.00
6 tcp http 0 0 normal 0.00 6 0.00 6 0.00
7 tcp http 0 0 normal 0.00 2 0.00 1 0.00
8 tcp http 0 0 normal 0.00 5 0.00 5 0.00
9 tcp http 0 0 normal 0.00 8 0.00 8 0.00
10 tcp http 0 0 normal 0.00 8 0.00 8 0.00
11 tcp http 0 0 normal 0.00 18 0.00 18 0.00
3 Development of Model
For the creation of the model, we use the algorithm of Neural Networks and K-NN
(K Neighbors Neighbors) dividing the 0.1% the our database into two random parts:
one composed of 70% used for training and another for the remaining 30% to validate
the model of the result of applying the technique of preprocessing the data when
removing duplicate values.
Neural Networks
Neural networks as a predictive model that allows to obtain good results in the clas-
sification and approximation when correctly computing the BDDKDD99 by the use of
adaptive algorithms and self-organization in the same way by the approximation of
functions allows to classify and increase immunity against noise at use a large number
of processing nodes with a high level of interconnectivity (Image 1).
Image 1. Correctly classified instances Neural Networks
Application of Classification Algorithms in the Generation 451
K-NN (K Neighbors Neighbors)
K-NN classifies each new data in the corresponding group as it has K neighbors closer
to one group from another, thus calculating the distance of the new element to each of
the existing ones and ordering the distances from least to greatest to select the group to
which it belongs, obtaining as a result that the group will be the one with the greatest
frequency with shorter distances (Image 2).
Image 2. Correctly classified instances K-NN (K Neighbors Neighbors)
4 Results
The results obtained in the confusion matrix are presented in the following table when
applying Neural Networks and K-NN (K Neighbors Neighbors) (Table 7).
Table 7. Results confusion matrix
Neural Networks K-NN (K Neighbors Neighbors).
5 Conclusions and Future Works
We can conclude that the multilayer perceptron algorithm of neural networks for the
database BDDKDD99 is not the most effective classifier when classifying data with
100% of correctly classified instances, there being significant differences in the eval-
uation of the two models with a percentage 100% success with only 1 neighbor for
K-NN (Supervised Learning).
452 J. Hidalgo and M. Yandún
As future work, the analysis and study of information and user behaviors through
networks will be continued using a network infrastructure in a laboratory and validating
and modeling other data mining and artificial intelligence algorithms such as Evolu-
tionary Neural Networks and Algorithms Genetic.
References
1. Reddy, R.R., Ramadevi, Y., Sunitha, K.V.N.: Effective discriminant function for intrusion
detection using SVM. In: 2016 International Conference on Advances in Computing,
Communications and Informatics (ICACCI), pp. 1148–1153 (2016)
2. Peluffo, I., Capobianco, M., Echaiz, J.: Machine Learning aplicado en Sistemas de Detección
de Intrusos (2014)
3. Zarkami, R., Moradi, M., Pasvisheh, R.S., Bani, A., Abbasi, K.: Input variable selection with
greedy stepwise search algorithm for analysing the probability of fish occurrence: a case study
for Alburnoides mossulensis in the Gamasiab River, Iran. Ecol. Eng. 118, 104–110 (2018)
4. Vattani, A.: k-means requires exponentially many iterations even in the plane, pp. 144–153
(2006)
5. Mendivelso, F., Rodríguez, M.: Prueba Chi-cuadrado de independencia aplicada a tablas 2 x n
independence chi-square test applied to 2 x N tables. Rev. Medica. Sanitas 21(2), 92–95
(2018)
6. Sow, M.T.: Using ANOVA to examine the relationship between safety & security and human
development. J. Int. Bus. Econ. 2(4), 101–106 (2014). 2374-2194
7. Martínez-Estudillo, F.J., Hervás-Martínez, C., Gutiérrez, P.A., Martínez-Estudillo, A.C.:
Evolutionary product-unit neural networks classifiers. Neurocomputing 72(1–3), 548–561
(2008)
8. Beasley, D., Bull, D.R., Martin, R.R.: An Overview of Genetic Algorithms: Part 1,
Fundamentals (1993)
9. Stoyanov, S.: Invited Paper Analyses of Methods and Algorithms for Modelling and
Optimization of Biotechnological Processes (2009)
Vulnerability Discovery in Network Systems
Based on Human-Machine Collective
Intelligence
Ye Han1(&), Jianfeng Chen1,2, Zhihong Rao1,2,
Yifan Wang1, and Jie Liu1,2
1
China Electronic Technology Cyber Security Co., Ltd.,
Chengdu 610041, China
[email protected]
2
Cyberspace Security Key Laboratory of Sichuan Province,
Chengdu 610041, China
Abstract. Network vulnerability mining is an important topic in cyberspace
security. Network vulnerabilities enable attackers to obtain sensitive information
from computer systems, control computer systems illegally and cause severe
damage. More effective vulnerability mining requires wider participation of
cybersecurity engineers and intelligent computing devices as cooperation among
the mining participants could take advantages of the complementarity capabil-
ities of human and machine. The human and resource cost of network vulner-
ability mining can be remarkably reduced and the mining efficiency is improved
accordingly. The principles and engineering mechanisms of introducing col-
lective intelligence to network vulnerability mining is discussed in the paper and
a vulnerability mining platform based on crowd intelligence is established fol-
lowing a four-layer system structure. Experimental tests showed that coopera-
tion enables mining participants to work better and learn from empirical
information, while better mining results could be obtained through procedure
optimization.
Keywords: Cybersecurity Vulnerability Collective intelligence
1 Introduction
As a new research hotspot in recent years, collective intelligence provides a new
technical means for exploring human-machine collaboration. It borrows the concept
and builds a paradigm of interaction from individuals and groups’ behaviors in nature.
The intelligence that emerged this way could over perform far beyond the level of
individual intelligence. Desirable features of collective intelligence, such as inherent
high concurrency, fast decision finding, strong environmental adaptability, high
robustness and self-recovery revoked by a functional group are especially suitable for
performing tasks that are difficult for individuals to complete [1].
Cyberspace security is an engineering science focus on the gaming between
attackers and defenders. The discovery and repair of vulnerabilities is the main reason
and an endless loop for cyberspace security to develop. A vulnerability is any factor
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 453–458, 2020.
https://doi.org/10.1007/978-3-030-39512-4_71
454 Y. Han et al.
that exists in computer network systems and may cause damage to system components
or its data. Vulnerabilities may appear in multiple dimensions such as hardware,
software, protocol implementation, or even system security policy. For an attacker, if a
vulnerability is discovered before being noticed, he will write malwares and put it to
use immediately to cause maximum damage. For the defender, if he discovers the
vulnerability first, then the repair work could be finished in time to protect the system
from potential attacks and reduce security risks.
The innovative concept and characteristics of collective intelligence coincide with
the increasing demand of vulnerability mining scale and efficiency. To be more
specific, the diversity of collective intelligence can realize adequate participation of
mass entities of vulnerability discovery, evaluation, and verification, so the mining
creativity will be greatly enhanced. The natural concurrency of collective intelligence
can make the mining work go parallel to achieve better mining scalability and reduce
latency. The good coordination property of collective intelligence would enable entities
of different backgrounds or skill levels to work with complementary capabilities to
optimize resource usage and task allocation. Last but not least, the high robustness and
self-recovery of the mining process come from the strong adaptability of collective
intelligence mechanisms. Besides, vulnerability mining systems that incorporate col-
lective intelligence are devised to learn from empirical behaviors and achieve better
mining performance in each iteration.
Collective intelligence has been used in travel recommendation systems [2], online
health support communities [3] and learning map making [4]. However, integrating
collective intelligence for vulnerability mining is still a frontier research direction. Few
theoretical or engineering work is disclosed at the time. Literature that relates to the
topic include a method proposed by VG Le et al. for building automatic web appli-
cation vulnerability detection in [5], and GuruWS, as a hybrid platform for detecting
malicious web shells and application vulnerabilities was constructed in [6]. The
reminding content of the paper intends to introduce vulnerability mining theory and
method based on collective intelligence. The architecture of a practical vulnerability
mining system is discussed afterwards. It reveals that the collective intelligence
emerged by human and machines will achieve better results in transaction-intensive
and intellectually intensive workflows such as vulnerability mining.
2 Vulnerability Mining Based on Collective Intelligence
As is mentioned above, deep-level vulnerability mining requires the wide participation
of cybersecurity engineers and intelligent computing devices. Based on collective
intelligence, a vulnerability-mining framework that enables cooperation among large
numbers of cybersecurity engineers and computing devices is proposed in this paper.
Two key problems need to be considered when developing the framework based on
collective intelligence. First, detailed information, namely digital portraits of different
vulnerability mining participants (especially cybersecurity engineers) need to be
obtained. These digital portraits stored in a knowledge base would form a capability
map of engineers. Second, based on the digital portraits of mining participants,
vulnerability-mining tasks are decomposed and distributed to different mining
Vulnerability Discovery in Network Systems 455
participants. Following collective intelligence approaches, information and knowledge
gained from different mining participants are collected and shared to facilitate the
mining process. These two issues are discussed in detail as follows.
2.1 Capability Map Generation
Figure 1 presents the process of participant capability map generation in vulnerability
mining. The capability map discussed here is generated mainly for cybersecurity
engineers, since the digital portrait of computers or servers are usually described based
on structured information and can be directly used in task distributing.
Relationship extraction
Data acquision Entity and attribute extraction
Static attributes: Submitted
Name, age, education, ... Engineer1
• Structured data Dynamic attributes: Vulnerability1 Engineer3
Engineers
• Semi-structured Work experience, skills, ...
Worked Reason of
Submitted
data together Common
library Vulnerability2
• Unstructured dependency
Source Attributes:
text Discription, type, severity, Resemble
information features, loss type, ... Vulnerability3
Vulnerabilities
Solved
Engineer2
Knowledge inference
Specialized fields
Information
Current working status
update
Engineer1 etc
Active update Knowledge Specialized fields
base
Current working status
Period update
Engineer2 etc
···
Fig. 1. Capability map generation process
Information acquired from mass data source could be categorized as structured
data, semi-structured data, and unstructured text. Entities and their attributes are
extracted from such source information. Two types of entities, namely cybersecurity
engineers and vulnerabilities, are extracted. Cybersecurity engineer entities include
static attributes such as name, age, education etc. and dynamic attributes such as work
experience and skills. Vulnerability entities include attributes such as description, type,
severity, loss type, etc. After that, the relationship among different entities is analyzed
and built. Knowledge inference can be performed based on the computed relationships.
Specialized fields like current working status and other features of cybersecurity
engineers, can be obtained this way. These features can be considered as labels of
cybersecurity engineers and will be used to match the preliminarily recognized vul-
nerability labels to achieve optimized mining task distribution. The features of
cybersecurity engineers are stored in a knowledge base as well as their identity
information. The knowledge base is regularly updated based on the newly acquired
source data.
456 Y. Han et al.
2.2 Mining Task Distribution
After capability maps are created, the mining tasks can be allocated to different mining
participants. Cooperation driven by collective intelligence among different mining
participants are devised to reduce the cost of vulnerability mining and improve the
mining result. Figure 2 presents the framework of our vulnerability mining process
based on collective intelligence. Fuzzing test and concolic execution techniques are
incorporated as the main tools in the mining process.
Fig. 2. Vulnerability mining based on collective intelligence
First, fuzzy test seeds are generated and mutated using different strategies to
maximize code coverage. Abnormal outputs of fuzzy tests are recorded. Different
algorithms are used to perform symbolic execution and potential vulnerabilities are
discovered.
Next, further analysis and information refinement are performed for abnormal
outputs generated by fuzzy tests or potential vulnerabilities recognized by symbol
executions. After the extraction of the abnormal outputs and potential vulnerabilities
features, they are treated as the inputs for a series of multi-label classification models.
Labels of the vulnerabilities are recognized and predicted by these models. According
to the matching results of the labels of vulnerabilities and the labels of cybersecurity
engineers, target engineers with high relevance are recommended for current task. The
background, root cause and exploitation mechanism of the vulnerabilities are analyzed
by different cybersecurity engineers in our system.
Vulnerability Discovery in Network Systems 457
Opinions of cybersecurity engineers are collected and aggregated to form a detailed
report with respect to the vulnerability. In this way, the mechanism of the vulnerability
can be thoroughly analyzed. Under such guideline, specific input files that have higher
possibilities of triggering the vulnerability can be generated and used in fuzzing tests,
the concolic execution processes can also be more targeted. The knowledge previously
gained about the vulnerability can be validated and new vulnerabilities can be dis-
covered by vulnerability association. Both the accuracy and efficiency of vulnerability
mining can be improved.
The process mentioned above will be repeated until no vulnerabilities can be found.
Since vulnerabilities are inevitable and ubiquitous in network systems, this repeated
process can be regarded as a daily maintenance process of network systems.
3 Platform Architecture
We build a vulnerability mining platform based on collective intelligence and realize
the system in JAVA SE. The platform includes four hierarchical layers such as the
theoretical layer, the infrastructure layer, the resource architecture layer, and the
application service layer. These layers realize corresponding data and instruction
exchange under the supervision and control of system security management and
maintenance. The theoretical layer mainly includes the implementation of theories in
collective intelligence, cooperative game, and optimization. The infrastructure layer
includes various vulnerability mining tools as well as digital portraits of cybersecurity
engineers and computing devices. The resource architecture layer includes vulnera-
bility databases, mining engines, and collective intelligence computing engines, etc.
The application service layer contains mining services, information retrieval services,
and data drill services.
This platform provides users of different skills and backgrounds with vulnerability
mining and analysis computing resources, holographic vulnerability libraries, vulner-
ability mining engines, and related tools. Vulnerabilities can be discovered through
distributed task assignment and public assessment, while a learning and communica-
tion place for engineers is also provided by the platform. Vulnerability mining related
services are implemented on the cloud computing infrastructure to achieve better
scalability and performance.
The centralized computing resources are allocated to handle collective intelligence
vulnerability mining transactions, transcend the inherent performance limitations of
traditional security services, and achieves a significant increase in concurrency and
procedure robust levels through adequate resource provision.
Collective intelligence vulnerability mining services are mainly located at the SaaS
(software as a service) level, the functions provided by the platform is exposed to the
user in the form of service to be invoked through standardized interfaces. Now the
platform has been successfully put into operation and run for nearly one year.
458 Y. Han et al.
4 Conclusion
Network vulnerability mining is an important topic in cyberspace security. As a new
research hotspot in recent years, collective intelligence provides a new technical means
for exploring human-machine collaboration. The human and resource cost of network
vulnerability mining can be reduced and the mining efficiency improved by innovative
integration of collective intelligence and cyberspace security. This paper investigates
vulnerability mining theory, discusses method based on collective intelligence, and
builds a vulnerability mining platform based on crowd intelligence.
In the future, new collective intelligence algorithms will be included in the plat-
form. Auxiliary tools and knowledge will be explored from collective intelligence to
make better vulnerability mining strategy. Besides, the datasets and knowledge base
included in the platform will be further enriched to make the platform more practical
and intelligent.
Acknowledgments. This work is supported by National Key R&D Program of China
No. 2017YFB08029 and is supported by Sichuan Science and Technology Program No. 2017-
GZDZX0002.
References
1. Benkler, Y., Hassan, M.: Collective Intelligence: Creating a Prosperous World at Peace. Earth
Intelligence Network, Oakton (2008)
2. Shen, J., Deng, C., Gao, X.: Attraction recommendation: towards personalized tourism via
collective intelligence. Neurocomputing 173, 789–798 (2016)
3. Introne, J., Goggins, S.: Advice reification learning and emergent collective intelligence in
online health support communities. Comput. Hum. Behav. 99, 205–218 (2019)
4. Jeng, Y., Huang, Y.: Dynamic learning paths framework based on collective intelligence from
learners. Comput. Hum. Behav. 100, 242–251 (2019)
5. Le, V., Nguyen, H., Lu, D., Nguyen N.: A solution for automatically malicious web shell and
web application vulnerability detection. In: 8th International Conference on Computational
Collective Intelligence (ICCCI), pp. 367–378. Springer, Cham (2016)
6. Le, V., et al.: GuruWS: a hybrid platform for detecting malicious web shells and web
application vulnerabilities. In: Transactions on Computational Collective Intelligence XXXII,
pp. 184–208. Springer, Heidelberg (2019)
Computational Modeling and Simulation
Supporting Decisions in Production Line
Processes by Combining Process Mining
and System Dynamics
Mahsa Pourbafrani1(&), Sebastiaan J. van Zelst1,2,
and Wil M. P. van der Aalst1,2
1
Chair of Process and Data Science,
RWTH Aachen University, Aachen, Germany
{mahsa.bafrani,s.j.v.zelst,
wvdaalst}@pads.rwth-aachen.de
2
Fraunhofer Institute for Applied Information Technology,
Sankt Augustin, Germany
Abstract. Conventional production technology is static by nature, develop-
ments in the area of autonomous driving and communication technology enable
a novel type of production line, i.e., dynamic production lines. Carriers of
products are able to navigate autonomously through a production facility,
allowing for several different “production routes”. Given such dynamic
behavior, it is interesting for a production line manager to study in what type(s)
of station(s)/resource(s) he/she needs to invest in. We can do so by analyzing the
behavior of the autonomous production line, to calculate what change is most
likely boosting performance. In this paper, we use historical event data, which
are the actual execution of the process, to support the design of system dynamic
models, i.e., a high-level predictive mathematical model. The purpose of our
framework is to provide the possibility for production line managers to oversee
the effects of the changes at the aggregated level in the production line,
regarding different performance metrics. At the same time, we provide the
freedom in choosing the level of detail in designing the model. The generated
model is at a customized aggregated level. We evaluated our approach based on
synthetic event logs in which we emulate the effect of policy changes, which we
predict accordingly.
Keywords: Process mining Performance analysis System dynamics
Production line Simulation What-if analysis
1 Introduction
In the area of modern products in the automobile industry, e.g., e-mobility and
autonomous driving, production lines should be able to handle the changes as fast as
possible. Flexible manufacturing system proposed different approaches in order to deal
with disturbances in the production systems [6]. Providing an agile platform in which
the production line managers are able to find the points to improve the performance
metrics is important. At the same time, the effects and costs of changes need to be
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 461–467, 2020.
https://doi.org/10.1007/978-3-030-39512-4_72
462 M. Pourbafrani et al.
considered carefully. Production line managers have to make the changes with a certain
level of confidence regarding the possible effects before applying them. A complete
insight into the performance of the production line is a requirement for a production
line manager, prior to evaluating the effect of changes. In modern organizations,
information systems are playing a substantial role in the support of day-to-day oper-
ations. These information systems become more and more intertwined with production
processes. Process mining [1] is a collection of highly automated techniques that aim to
increase process knowledge primarily based on the event data recorded in such
information. For example, process mining techniques enable us to discover descriptive
process models, purely on the basis of the recorded information. Furthermore,
advanced techniques allow us to assess process conformity w.r.t. a given reference
model, as well as bottleneck identification. Since the information system tracks what
actually happens, process mining techniques allow organizations to increase overall
process transparency as well as improved process performance. The real running
processes in an organization, along with bottlenecks and performance metrics are a
crucial step for an organization to identify its current performance as well as to improve
their processes. Furthermore, several techniques exist, that aim to increase the overall
view of the process [7, 8]. Undisputed, predicting the future behavior of a process,
specifically with the aim of improving process performance, is of interest to many
organizations. Within process mining, some work towards the prediction of future
behavior w.r.t. performance of the process is proposed [9]. In [11] focuses on assessing
the applicability of deep learning techniques for short-term prediction. None of the
existing techniques provides the freedom of choosing the level of detail in prediction.
However, a decision-maker in an organization is often interested in the prediction of
process performance regarding different levels of detail, specifically, in production
lines. Current predictive approaches use extensive knowledge of the process, and, in
production lines, the large number and diversity of activities make use of current
forward-looking approaches not feasible. In [5] a general framework is proposed which
can be used for scenario-based analysis and prediction at an aggregated level. It uses
system dynamics models based on the past data of the organization. Using this
approach the environmental variables, which in reality affect the performance of a
process can be included. At the same time, despite the discrete event simulation
techniques, the freedom in the level of details is provided. Therefore, in this paper, we
adopt the main approach in [5] and propose a general framework based on process
mining and system dynamics for production lines. It provides insight into the current
status of the production lines and its future statues considering the upcoming changes.
We extend the proposed framework in [5] by adding a different level of details for the
modeling. In addition, we perform a preliminary case study w.r.t. the applicability of
the proposed framework in future dynamic production line settings. The remainder of
this paper is organized as follows. In Sect. 2, we introduce background concepts. In
Sect. 3, we present our main approach. We provide an evaluation as a proof of concept
in Sect. 4. In Sect. 5, related work are mentioned. Section 6 concludes our work and
discusses interesting directions for future work.
Supporting Decisions in Production Line Processes 463
2 Background
Process Mining. Process mining is a collection of approaches, techniques, and tools,
which provides a wide range of knowledge about the processes inside the organizations
based on event logs. Using process mining, discovering and analyzing the processes
executed in an organization is possible [1]. Event Log. Past data of the execution of the
organization’s processes provide the input for process mining. The execution of an
activity, in the context of some process instances, identified by a unique case id at a
specific timestamp by a specific resource is referred to as an event. For instance, an
event in the production line is defined such as item 1 (case id) is cut (activity) by John
(resource) at 10/1/2018 7:42:30 (timestamp). There are different events related to the
different process instances, identified by different case ids. A set of events regarding the
same case id forms a trace and multiple traces form the event log of the execution of the
process. Note that, typically, an event log includes more data attributes related to the
process, e.g., costs of an activity, account balance, customer id, etc.
System Dynamics. System dynamics is the collection of approaches, techniques, and
tools, which is able to present the model of complex, dynamic systems, in a structured
manner. In particular, it allows us to capture the factors affecting the behavior of a
system [4]. Within system dynamics, we use a specific modeling notation, i.e. a stock-
flow diagram that allows us to simulate possible future behavior of a system, e.g., a
(business) process. Stock-Flow Diagram. A stock-flow diagram consists of three basic
elements, i.e., stocks, in-/out flows, and variables [3]. A stock represents any entity that,
in some way, is able to accumulate over time, e.g. the number of waiting items in the
production line. An inflow increases the accumulated entity represented by a stock,
whereas an outflow reduces the accumulated entity. Finally, any environmental factor
that is able to influence the in-/outflow of a stock is modeled as a variable. Such a
variable is able to influence other variables as well. Furthermore, the value of a stock,
in turn, is able to influence w.r.t. a variable.
3 Proposed Framework
General key-performance metrics in production lines are resource management/
utilization and throughput time. We adopt the framework presented in [5] and use the
past execution of the processes in a production line in the form of event logs. Event
logs and processes include track information on an activity level, i.e., they describe
what activity is performed at what point in time, which makes the modeling compli-
cated. In the proposed framework, as Fig. 1 shows, we extract the major process
components, which are, in a production setting, the stations in the production line. In
the newly discovered process model the level of stations is detailed enough to show the
flow of cars being produced in the production line and at the same time aggregated
enough to avoid unnecessary details. To do so, we extract activities and aggregate them
into one single activity using [12]. The set of activities, which must be performed but
may happen in different orders extracted, observing the traces in the event logs.
464 M. Pourbafrani et al.
Fig. 1. The Proposed framework for the production line. It starts with discovering the process
model of the production line at the station level. Generating SD-Log including the different
performance parameters, which are used to design the model. After validation, the scenario-based
analysis for the production managers is possible.
Considering the performance aspect, since all the parallel activities happen in any
possible orders, we combine them into one single high-level activity. Using a module
based on process discovery, the process model at the station level is discovered. In
production lines, the tasks are distributed between stations, which can be handled by
the same resources, we are able to get the performance of the process among the
stations. We consider the following performance parameters exclusively in production
lines: average service time in each station, number of resources for each station, the
arrival rate of items for production line, finish rate, the capacity of each station, and
the number of items waiting in each station. In the next step, we generate the SD-Log
based on the performance parameters of stations for each time window. As shown in
Fig. 1 the similarity values of parameters in each time window are tested with “Time
Window Stability Test”. Exploiting the system dynamics modeling, the stock-flow
diagram is being generated for the production line. We simulate the model populating
the model with the values from SD-Log. This step is followed by a validation step,
which provides the level of certainty, i.e., whether the model is acting similar to reality.
In the final step, the general model can be refined by adding other parameters in the
production line. We use the model to change the parameters and predict the different
scenarios.
4 Proof of Concept
We use CPN tools1 and ProM2 to generate the event log based on the production line of
an electric automobile company. The generated event log includes the execution of
processes before and after applying the changes regarding performance metrics, e.g.,
the change in the number of resources. Our model includes multiple stations, which
cars go into each in sequence. In our designed stock-flow model Fig. 2 the assembly of
the doors including four other sub-activities takes two hours (station 210) and there is
always a queue for this station. By increasing the number of resources in station 210,
1
http://cpntools.org/.
2
http://www.promtools.org/.
Supporting Decisions in Production Line Processes 465
Fig. 2. Part of the designed stock-flow diagram for the production line based on synthetic data
of automobile company. This model is populated with the data from SD-Log in the time window
of one day at the station level.
Fig. 3. Number of cars waiting for station 212 and station 210 before (red) and after (blue)
adding one resource to station 212 in 50 days.
as we expected the number of cars in the queue for station 210 is decreased to zero in
the second execution of the model with two resources. Therefore, the problem of
waiting cars seems to be solved. However, the proposed framework represents the
effect of changes in this station on the two next stations, which is “window assembly”
station.
As Fig. 3 shows the cars which are waiting for station 210 are decreasing at the
same time the number of cars waiting for station 211 is increasing. Since in the
production line all the cars after station “door assembly” go through “window
assembly”, we chose two involved stations and all their possible performance
parameters generated from the aggregated process model and the event log. This
evaluation as a proof of concept shows the effectiveness of the approach in demon-
strating the effects of one change through the whole production line. We can prag-
matically deduce the detailed knowledge of the process and performance aspects from
an event log in the scenario-based analysis of processes. Using the proposed approach,
we are able to predict any further changes in the production line by changing one part,
such as adding more resources to one of the stations. As the example demonstrates, the
proposed approach is able to predict the consequence of changes/decisions in the
process.
466 M. Pourbafrani et al.
5 Related Work
An overview of process mining and system dynamics is provided in [1] and [4],
respectively. In the field of system dynamics, different work toward simulation and
prediction are done. There are different research conduct on the basis of using system
dynamics in different contexts such as business process management, e.g. using both
Petri net models and system dynamics to develop a model for the same situation [2].
According to [13] system dynamics among the simulation techniques in the manu-
facturing and business is an effective technique, however, the used techniques did not
use the provided insight into the process by process mining techniques. In process
mining, prediction and simulation approaches are mainly at a detailed level and they
are at the case level [10]. In [5] the possibility of addressing the aggregate level of
models are addressed using both process mining and system dynamic.
6 Conclusion
In this paper, the necessity of providing a platform to support the decisions in the
modern production lines is discussed. Establishing flexible production lines for modern
products such as autonomous cars is the goal of the new products. Our framework
provides the ability to oversee the new decisions and changes for a production line to
be agile. It employs process mining techniques, specifically processes discovery at a
higher level of abstraction along with performance analysis. We use the outcome of
process mining techniques to generate an SD-log. We design the general system
dynamics model based on the discovered knowledge from process mining and related
parameters in the production line. General stock-flow diagram for the production line at
an aggregated level can be improved and changed regarding different situations. We
evaluated our framework based on a synthetic event log, which is generated using a
CPN model. This evaluation serves as a proof of concept showing the efficiency of our
generated model. As future work, we focus on identifying the underlying relationships
between the parameters of the production line. Extending our approach in the field of
performance analysis and resource management for the process to meet the require-
ments of the business is another practical approach.
Acknowledgments. Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research
Foundation) under Germany’s Excellence Strategy – EXC 2023 Internet of Production.
References
1. van der Aalst, W.M.P.: Process Mining - Data Science in Action. Springer, Heidelberg
(2016)
2. Rosenberg, Z., Riasanow, T., Krcmar, H.: A system dynamics model for business process
change projects. In: International Conference of the System Dynamics Society, pp. 1–27
(2015)
Supporting Decisions in Production Line Processes 467
3. Pruyt, E.: Small System Dynamics Models for Big Issues: Triple Jump Towards Real-World
Complexity. TU Delft Library, Delft (2013)
4. Sterman, J.D.: Business Dynamics: Systems Thinking and Modeling for a Complex World.
No. HD30. 2 S7835 (2000)
5. Pourbafrani, M., van Zelst, S.J., van der Aalst, W.M.P.: Scenario-Based Prediction of
Business Processes Using System Dynamics, Rhodes, Greece (2019)
6. Qin, J., Liu, Y., Grosvenor, R.: A categorical framework of manufacturing for Industry 4.0
and beyond. Procedia CIRP 52, 173–178 (2016)
7. Leemans, S.J.J., Fahland, D., van der Aalst, W.M.P.: Process and deviation exploration with
inductive visual miner. In: Proceedings of the BPM Demo Sessions, Eindhoven,
Netherlands, 10 September 2014, p. 46 (2014)
8. Mannhardt, F., de Leoni, M., Reijers, H.A.: The multi-perspective process explorer. In:
Proceedings of the BPM Demo Session 2015, pp. 130–134 (2015)
9. Rozinat, A., Mans, R.S., Song, M., van der Aalst, W.M.P.: Discovering simulation models.
Inf. Syst. 34(3), 305–327 (2009)
10. Rozinat, A., Wynn, M.T., van der Aalst, W.M.P., ter Hofstede, A.H.M., Fidge, C.J.:
Workflow simulation for operational decision support. Data Knowl. 68(9), 834–850 (2009)
11. Tax, N., Teinemaa, I., van Zelst, S.J.: An Interdisciplinary Comparison of Sequence
Modeling Methods for Next-element Prediction (2018)
12. Leemans, M., van der Aalst, W.M.P., van den Brand, M.: Hierarchical performance analysis
for process mining. In: ICSSP (2018)
13. Jahangirian, M., Eldabi, T., Naseer, A., Stergioulas, L.K., Young, T.: Simulation in
manufacturing and business: a review. Eur. J. Oper. Res. 203(1), 1–13 (2010)
Using Real Sensors Data to Calibrate a Traffic
Model for the City of Modena
Chiara Bachechi, Federica Rollo(&), Federico Desimoni,
and Laura Po
‘Enzo Ferrari’ Engineering Department, Via Vivarelli 10, Modena, Italy
{chiara.bachechi,federica.rollo,federico.desimoni,
laura.po}@unimore.it
Abstract. In Italy, road vehicles are the preferred mean of transport. Over the
last years, in almost all the EU Member States, the passenger car fleet increased.
The high number of vehicles complicates urban planning and often results in
traffic congestion and areas of increased air pollution. Overall, efficient traffic
control is profitable in individual, societal, financial, and environmental terms.
Traffic management solutions typically require the use of simulators able to
capture in detail all the characteristics and dependencies associated with real-life
traffic. Therefore, the realization of a traffic model can help to discover and
control traffic bottlenecks in the urban context. In this paper, we analyze how to
better simulate vehicle flows measured by traffic sensors in the streets. A dy-
namic traffic model was set up starting from traffic sensors data collected every
minute in about 300 locations in the city of Modena. The reliability of the model
is discussed and proved with a comparison between simulated values and real
values from traffic sensors. This analysis pointed out some critical issues.
Therefore, to better understand the origin of fake jams and incoherence with real
data, we approached different configurations of the model as possible solutions.
Keywords: Traffic modelling Simulation model Big data analytics Traffic
sensor
1 Introduction
Italy is the second State in Europe with the highest number of passenger cars per
thousand inhabitants. The Eurostat/ITF/UNECE Common Questionnaire on Inland
Transport registers that in 2016 in Italy there were 625 cars every thousand inhabitants.
The aim of the TRAFAIR1 project [1] is to implement a flexible solution to monitor
and forecast urban air quality in 6 European cities (Modena, Florence, Pisa, Livorno,
Zaragoza and Santiago de Compostela). Real traffic data are needed to evaluate traffic-
related emissions and then estimate how these pollutants move in the air according to the
wind, weather and building shapes. Therefore, a simulation model has been employed to
obtain the vehicle flow where no sensors are located.
1
www.trafair.eu.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 468–473, 2020.
https://doi.org/10.1007/978-3-030-39512-4_73
Using Real Sensors Data to Calibrate a Traffic Model 469
Public administrations usually employ static traffic models that provide only an
average traffic condition during peak hours in the main streets of the city. This kind of
model does not consider the dynamic evolution of traffic during daytime and the
seasonal variation during the year.
In general, simulation is a dynamic representation of the real world achieved
building a computer model and moving it through time [2]. Traffic modelling [3] aims
to accurately recreate real traffic flow by using data coming from a network of sensors
distributed over the area of interest. The costs of the construction of such a distributed
system can be burdensome for public administrations. However, in many cities, some
distributed sensors are used for other purposes.
In the city of Modena, more than 300 induction loops sensors are located near
traffic-light controlled junctions. These devices are used locally to control the traffic
light logic but their traffic-related data (for instance, the vehicle counts and the average
speed) have never been analyzed before. In the TRAFAIR project, we customize a
traffic model for the city of Modena employing SUMO (Simulation for Urban
Mobility) [4] and OSM2 (Open Street Map), both open sources, to ensure a cost-
effective solution.
The paper is organized as follows: Sect. 2 briefly describes the traffic model; in
Sect. 3, an evaluation of the model performance is given comparing real traffic data
with the simulated ones; finally, in Sect. 4 different configurations are explored to find
a solution to the emerged criticalities.
2 The Model
Our traffic model [5] is a micro-simulation model, obtained by using SUMO, config-
ured to generate the routes of the vehicles starting from traffic sensors data. In a micro-
simulation model, vehicles are simulated individually: each vehicle has its own trip to
follow and moves inside the road network considering traffic restrictions. Our model
has the aim to produce data about vehicle counts and their average speed in every road
portion of Modena starting from the measurements of the traffic sensors.
The sensors placed in Modena count the vehicles passing through them every minute
and evaluate their average speed. We collect sensors data in real-time in a local Post-
greSQL database, and the model interacts with it directly [6]. In our SUMO simulated
map, we placed a “calibrator” near each traffic sensor. A calibrator is an object capable of
producing the aspired traffic flow, i.e. the number of vehicles counted by the sensor
associated with that calibrator. Calibrators are part of the SUMO suite and are like virtual
traffic sensors calibrated considering the real measurements of the on-road sensors.
Unlike sensors that measure the number of vehicles pointwise, calibrators control the flow
on a lane of a road portion. For this reason, we have also placed some virtual detectors,
SUMO objects that mime exactly the behaviour of the sensors and returns the vehicle
count at a precise point of the map. We use a Python script to produce automatically the
file containing the positions of the virtual detectors. We consider the GPS coordinates of
2
https://www.openstreetmap.org.
470 C. Bachechi et al.
the corresponding traffic sensor and the name of the street in which they are placed. The
road name was necessary to avoid some errors, sensors are placed near junctions where
roads are one near another. Thus, considering only the geolocalization of the sensors and
placing them in the nearest road section not always ensure to find the right position.
Therefore, we consider also the similarity between the name of the roads in the junction
and the correct road name to estimate the right position. The values retrieved by these
virtual detectors can be compared with the measurement of the real ones to evaluate the
performance of the model.
3 Evaluating the Model
We evaluated the performance of the model using different techniques. In every point
where there is a real sensor, the time series of the real flow measurements and the one
retrieved by the virtual detector in the simulation are compared using DTW (Dynamic
Time Warping) distance. The DTW distance [7] is a way to measure the distance
between two different time series that allows sequences to be stretched along the time
axis. The sensors with a DTW distance higher than 1200 have been considered distant
and not reliable. DTW distance is not the only metric used to determine if a calibrator is
following real measurements or not. We also calculated the difference between the
virtual and real sensors vehicle counts at the same instant and evaluated an average of
this difference. Finally, we evaluated the number of instants in which the difference is
higher than 2 vehicles per minute. This metric consider that we could have some
instants in which the calibrator is not able to follow real measurement and the distance
is high but in all the others the calibrator flows is similar to the real one, thus the error is
limited to a short period time. Using these two methods, we can classify calibrators into
two ways: the ones that manage to produce the real aspired flow will be referred to as
‘aligned’, the others as ‘not aligned’.
Tests have been performed on seven November 2018 days. In Table 1, the number
of not aligned calibrators is displayed for every tested day. The ratio is obtained
dividing the number of not aligned calibrators by the total number of calibrators in the
simulation. This number is equal to the number of sensors with at least one mea-
surement on that specific simulated day. More than 20% of calibrators appear to be not
aligned.
In Fig. 1 a graphical comparison between two time series is displayed to underline
the difference between aligned and not aligned calibrator. We have identified the
calibrators that in all the tested days are always classified as ‘not aligned’. They are 39
and belong to 23 different junctions. We observed that often calibrators in a junction
belong to the same group. Observing some not aligned junctions we discover that in 6
junctions (where 13 of the 39 calibrators are located) the problem is related to the
SUMO road network that does not match the real one. The geographic data are pro-
vided by OSM and they include only information about the total number of lanes in a
road without information about directions. Thus, sometimes the direction of the lanes
assigned by SUMO is not right. An example is shown in Fig. 2: on the left of the
figure, the two sensors at the bottom are located in SUMO road network in the same
lane on the same direction; however, the two sensors are on two different lanes in
Using Real Sensors Data to Calibrate a Traffic Model 471
Table 1. The number of not aligned calibrators and its percentage on the total number of
calibrators for seven simulated November days.
Day 14th Nov 15th Nov 19th Nov 22nd Nov 27th Nov 29th Nov
Not aligned calibrators 59/240 52/240 59/239 69/241 59/240 49/241
% 24.6 21.7 24.7 28.6 24.6 20.3
reality as shown on the aerial view on the right. To overcome this problem, the counts
provided by the two sensors could be summed up.
Fig. 1. Comparison between the time series of vehicle counts, simulated by calibrators (blue)
and measured by real sensors (orange). The graphs on the left are referred to an all-day simulation
of Monday 19th November. The graphs on the right are referred to the same calibrators and
sensors in the sub-simulations of the same day.
Another reason why some calibrators are not aligned is the creation of fake jams in the
simulation: when a jam appears, calibrators are not able to insert new vehicles even if the
required flow is higher. We observe that 10 of the 23 junctions of not aligned calibrators
are affected by fake jams. The presence of a fake jam can be observed in Fig. 1 in the graph
at the top of the figure on the left. The calibrator initially manages to follow real vehicles
count, then the number of vehicles increases, and a jam appears reducing the flow through
it. The duration of the simulation can contribute to producing fake jams. When a calibrator
generates a vehicle, it will remain in the simulation since the end of its route; if it does not
drive over another calibrator that decides to remove it. Reducing the duration of the
simulation (splitting the simulation in several sub-simulations of reduced duration) allows
avoiding this problem because refreshing the simulation will remove not necessary
vehicles. We performed several simulations excluding the calibrators of a specific
junction to observe if the absence of them could affect the performance of the others. We
observed that this influence is related to the geographical distance and also to the existence
of a path that can connect the two junctions.
472 C. Bachechi et al.
In some cases, not aligned calibrators are located in the right place of a crossroad
with the right morphology and their measurements are acceptable. The reason for their
‘not alignment’ is addressed to the sensors located near them. 3 of the not aligned
junctions have some sensors in the neighbourhood that counts zero vehicles for the
whole day simulation. The measurements of these sensors must be excluded from the
input data, otherwise, the lane in which they are located will be forced to do not have
any flow for the whole simulation.
Fig. 2. Simulation map showing sensor location and aerial view of the same junction.
4 Trying Different Configurations to Find a Solution
We tried different configurations of the traffic model to overcome the issues emerged in
Sect. 3. Firstly, we removed from the simulation input of Thursday 8th November some
calibrators unable to follow real measurements or measuring zero vehicles all day. We
removed 29 calibrators. Through the comparison of the lists of not aligned calibrators
in the regular simulation and in the simulation without the 29 excluded calibrators, we
observed that 11 calibrators improve their performances and 12 calibrators’ perfor-
mances get worse.
An interesting fact is that, comparing the real measurement and the simulated
counts in 20 of the 29 positions where sensors have been removed, the time series of
their virtual detectors better followed real measurements. This means that the model
can infer the vehicles counts in their position even without the calibrators. However,
this solution was not good enough since 58 calibrators still appear not aligned.
For this reason, we tried another solution. We split the Monday 19th simulation in
sub-simulations with a duration of 3 h each. This interval of time was chosen because it
is not likely to have routes longer than 3 h in an urban context; however, the interval is
long enough that calibrators can influence each other but not enough to make this
influence producing fake jams in the network. A simulation of 24 h composed of eight
simulations of 3 h was performed.
The time series obtained by the sub-simulations were compared with real mea-
surements as described in Sect. 3. The number of not aligned calibrators decreases
incredibly to 2.0% (5/241), all of them belonging to the 39 calibrators that do not
follow real measurements in any of the previous simulations. In Fig. 1 there is an
Using Real Sensors Data to Calibrate a Traffic Model 473
example of how the sub-simulations approach removes fake jams and improves the
performances of two calibrators.
5 Conclusion and Future Work
Splitting the simulation in sub-simulations proved to be a good solution to ensure that
the simulation follows real measurements. The exclusion of sensors that always counts
zero vehicles is necessary to avoid errors caused by not reliable input data from
sensors. Therefore, we produce a simulation capable of following the exact number of
vehicles circulating in almost every point where there is a sensor and to infer vehicles
counts where there are a lot of sensors in the neighbourhood area. To enhance the
realism of the simulation, a good improvement could be to include information about
average traffic flows, like Origin-Destination matrices, to produce routes in roadways
where no sensors are placed.
Acknowledgments. This research has been supported by the TRAFAIR project 2017-EU-IA-
0167, co-financed by the Connecting Europe Facility of the European Union. The contents of this
publication are the sole responsibility of its authors and do not necessarily reflect the opinion of
the European Union. The authors would like to thank in particular all the partners that contribute
in the collection and management of traffic sensor data: the City of Modena and Lepida S.c.p.A.
References
1. Po, L., Rollo, F., Viqueira, J.R.R., Lado, R.T., Bigi, A., Lòpez, J.C., Paolucci, M., Nesi, P.:
TRAFAIR: understanding traffic flow to improve air quality. In: 2019 IEEE International
Smart Cities Conference (ISC2) (2019, to appear)
2. Drew, D.: Traffic Flow Theory and Control. McGraw-Hill, New York (1968)
3. Nasuha, N., Rohani, M.: Overview of application of traffic simulation model. In: MATEC
Web of Conferences, vol. 150, p. 03006 (2018)
4. Krajzewicz, D., Hertkorn, G., Feld, C., Wagner, P.: An example of microscopic car models
validation using the open source traffic simulation SUMO. In: Proceedings of the 14th
European Simulation Symposium (ESS 2002), Dresden, October 2002, pp. 318–322. SCS
European Publishing House, Dresden (2003). Reverse-time simulation in production line
redesign
5. Bachechi, C., Po, L.: Implementing an urban dynamic traffic model. In: IEEE/WIC/ACM
International Conference on Web Intelligence, WI 2019, Thessaloniki, Greece, 14–17 October
2019 (2019, to appear)
6. Po, L., Bachechi, C., Rollo, F., Corni, A.: From sensors data to urban traffic flow analysis. In:
2019 IEEE International Smart Cities Conference (ISC2) (2019, to appear)
7. Bachechi, C., Po, L.: Traffic analysis in a smart city. In: Web4City, International
IEEE/WIC/ACM Smart City Workshop: Web for Smart Cities - in Conjunction with
IEEE/WIC/ACM International Conference on Web Intelligence, WI 2019, Thessaloniki,
Greece, 14–17 October 2019 (2019, to appear)
Logistic Regression for Criteria Weight
Elicitation in PROMETHEE-Based Ranking
Methods
Elia Balugani1(&), Francesco Lolli1, Maria Angela Butturi1,
Alessio Ishizaka2, and Miguel Afonso Sellitto3
1
Department of Sciences and Methods for Engineering,
University of Modena and Reggio Emilia,
Via Amendola 2, Padiglione Morselli, 42122 Reggio Emilia, Italy
{elia.balugani,francesco.lolli,
mariaangela.butturi}@unimore.it
2
NEOMA Business School,
Rue du Maréchal Juin 1, 76130 Mont-Saint-Aignan, France
[email protected]
3
Universidade do Vale do Rio dos Sinos, São Leopoldo, RS 93022750, Brazil
[email protected]
Abstract. For a PROMETHEE II method used to rank concurrent alternatives
both preference functions and weights are required, and if the weights are
unknown, they can be elicited by leveraging present or past partial rankings. If
the known partial ranking is incorrect, the eliciting methods are ineffective. In
this paper a logistic regression method for weight elicitation is proposed to
tackle this scenario. An experiment is carried out to compare the logistic
regression method performance against a state-of-the-art linear weight elicitation
method, proving the validity of the proposed methodology.
Keywords: MCDM Elicitation Criteria weights Outranking
PROMETHEE Machine learning Logistic regression
1 Introduction
Multicriteria decision making (MCDM) methods compare multi-dimensional alterna-
tives, presented to a decision maker (DM), and rank them according to his/her pref-
erences. MCDM methods use preference functions to measure the DM’s specific
preferences, and to translate the alternatives into comparable units. The resulting
preferences are then weighted to obtain a scalar value for each alternative. This scalar
value can be directly used to rank the alternatives. Readers interested in the theoretical
foundation of MCDM methods alongside an established MCDM method, can refer to
[1], while those interested in a recent application in the Life-cycle assessment
(LCA) context can refer to [2]. The availability of credible weights can severely impact
the performance of most MCDM methods [3], thus a series of weigh eliciting proce-
dures have been designed [4]. This paper expands the methods proposed in [5] for
weights eliciting in the PROMETHEE II context.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 474–479, 2020.
https://doi.org/10.1007/978-3-030-39512-4_74
Logistic Regression for Criteria Weight Elicitation in PROMETHEE 475
Important components required for the application of MCDM methods are often
missing, both the preference functions and the weights might not be known. In case the
weights are unknown, information on the ranking of the alternatives can be used to
elicit them. In the literature specific weighting eliciting procedure has been designed for
different MDCD methods, [6] propose an eliciting procedures for TOPSIS, [7] focus on
a surrogate weighting procedure in PROMETHEE, and [8] revise Simo’s procedure for
the ELECTRE method. From a broader perspective, [9] propose a posterior analysis
using the popular Simple Additive Weighting (SAW) method while [10] focus of
multicriteria additive models. For the PROMETHEE method, if a partial ranking of
present or past decisions is available, [5] propose various elicitation methods based on
linear and convex constrained optimization.
If the preference functions are also unknown, Robust Ordinal Regression
(ROR) methods bypass the elicitation problem by providing all the results obtainable
using preference functions that are in line with a known partial ranking. Interested
readers can refer to the first ROR publication [11], which is a re-design of the UTA
method [1] in a robust framework. The PROMETHEE method re-designed in a ROR
framework can be found in [12], while the ROR version of ELECTRE has been
designed by [13].
Machine learning (ML) algorithms, like logistic regression, are often used in
conjunction with MCDM methods. [14] Outline the similarities and differences
between ML algorithms and MCDM methods, while [15] bridge the gap between ML
and ROR methods. Recent applications of K-means for AHP can be found in [16] and
decision tree algorithms for Data Envelop Analysis (DEA) in [17].
In this paper a logistic regression algorithm in used in the PROMETHEE weight
elicitation context of [5] rather than a linear model. The linear and logistic regression
algorithms are experimentally compared in cases where the known partial ranking is
incorrect.
This paper is organized as follows: in Sect. 2 the proposed weight elicitation
models is described. In Sect. 3 the experimental setting is outlined, and its results are
analysed. Section 4 concludes the paper with a summary of the key findings and
suggestions for future research.
2 Logistic Regression Model
Using the formalism presented in [5], if all the preference between alternatives are
known they can be used to elicit unknown weights solving the optimization problem:
!
X 1
max log Pn : ð1Þ
1 þ e j¼1 ðð j Þik ð j Þki Þ j
fi;k g:~
ai P~
ak P P :ðwÞ
476 E. Balugani et al.
s.t.
Xn
j¼1
ðwÞj ¼ 1: ð2Þ
0 ðwÞj 1 8j 2 f1; . . .; ng: ð3Þ
(1) maximizes the log-likelihood of identifying preferred alternatives by linearly
separating the preference space. Each alternative i preferred over an alternative k is a
success event drawn from a Bernoulli distribution. The distribution is characterized by
a success probability pik parametrized, in the preference space, by net flow differences
through the inverse canonical link function. If the linear predictor for the inverse
canonical link function is defined without an intercept its parameters are maximum
likelihood estimators for the PROMETHEE II weights. Since no failure event is
considered, the Bernoulli distribution log-likelihood simplifies to (1).
This model is effective even if some of the known preferences are incorrect. In
addition, the probabilistic interpretation of the weights allows the DM to identify the
known preferences that are most likely to be faulty.
3 Experimental Setting and Results
3.1 Experimental Setting
The experiment objective was to compare the two models in scenarios where some of
the known preferences are incorrect. The Linear Model is expected to outperform the
Logistic Regression Model if all the known preferences are correct, while erroneous
information is expected to favour the proposed model over the original one. The Linear
Model is also expected to be unable to find a feasible solution if wildly incorrect inputs
are provided.
The dataset for alternatives and weights is used in [5], which contains 5-
dimensional weights for 1000 DMs and their rankings of 100 different alternatives
each. It is artificially expanded by permuting the rankings 1000 times, with each
permutation being independent from the previous one, and it affects all the rankings
leading to 1,000,000 permuted rankings.
h The
preference function
i
for each weight j is linear in the entire interval
min Dj ik ; max Dj ik .
For each DM l and permutation h both the Linear Method and the Logistic
Regression Method are applied thereby obtaining two sets of elicited weights. Each set
s of elicited weights is compared with the DM weights thus obtaining a performance
measure in the interval ½0; 1:
Pn
j¼1 ðweslh Þj ðwl Þj
performanceslh ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Pn qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Pn ffi ð4Þ
2 2
j¼1 ð we slh Þ j j¼1 ðw l Þ j
where ðwl Þj is the DM’s weight for criterion j, and ðweslh Þj is the elicited weight for the
same criterion.
Logistic Regression for Criteria Weight Elicitation in PROMETHEE 477
For each permutation h and set s, the DMs’ performance measures are aggregated
by estimating their expected value:
X1000
b ðperformancesh Þ ¼
E performanceslh : ð5Þ
l¼1
Each permutation h is rated according to its distance from the unpermuted ranking:
P100
1fposhr [ r g
r¼1
perh ¼ ð6Þ
100
2
100
where poshr is the permuted value in position r and is a normalization constant
2
to constraint perh in the range ½0; 1. The rating of the unpermuted ranking is 0, while
the rating of the reversed ranking is 1. The permutations are generated to achieve
ratings that are uniformly distributed between 0 and 1.
The two sets of Eb ðperformancesh Þ can be compared across different perh to gauge
how different permutation ratings affect the model’s performance.
3.2 Results
Figure 1 depicts the obtained E b ðperformancesh Þ for the two methods against the per-
mutations perh . The expected values for the performance measure of the Linear Model are
plotted with circles, while the Logistic Regression Model values are plotted with crosses.
Fig. 1. Expected value of the performance measure for the two models and permutation ratings.
According to Fig. 1, the Linear Model outperforms the Logistic Regression Model
in the non-permuted scenario, while the Logistic Regression Model is superior for
sizable values of perh . When the permutation becomes severe ðperh 0:5345Þ,
478 E. Balugani et al.
the Linear Model is again the preferred option for weight eliciting, up to complete rank
reversion.
Both the Linear Model and the Logistic Regression Model were always able to find
feasible solutions.
4 Conclusions
Unexpectedly, the Logistic Regression Model does not always outperform the Linear
Model when some of the known preferences are incorrect. The advantage of the
Logistic Regression Model is limited to those instances where the permuted ranking is
closer to the non-permuted ranking than to the reverse ranking, with cut-point
perh ¼ 0:5345. In nearly all the analysed cases, the Linear Model achieves, without
incurring infeasibility issues, a constant expected value of the performance measure
perh ¼ 0:8716, where chance alone would yield perh ¼ 0:7370. Leveraging the high
dimensionality of the preference space, the Linear Model finds a single feasible
solution and retains it for nearly all the permuted rakings, except for the reverse ranking
that carries its own below-chance solution.
These results provide guidelines on when one method is preferable over the other,
and prove that, when the correct method is selected, the elicited weights are close to the
real ones above chance.
Further research will use the Logistic Regression Model to identify faulty known
preferences, leveraging the probabilistic interpretation of the weights described in
Sect. 2. Other machine-learning algorithms (e.g. Support Vector Machines, Neural
Networks) will be specialized into weight-eliciting models. These models are expected
to account for not only for incorrect known preferences but also for incorrect preference
functions, discarding the hypothesis of the linear separability assumption in the pref-
erence space.
References
1. Jacquet-Lagreze, E., Siskos, J.: Assessing a set of additive utility functions for multicriteria
decision-making, the UTA method. Eur. J. Oper. Res. 10, 151–164 (1982). https://doi.org/
10.1016/0377-2217(82)90155-2
2. Lolli, F., Ishizaka, A., Gamberini, R., Rimini, B., Balugani, E., Prandini, L.: Requalifying
public buildings and utilities using a group decision support system. J. Clean. Prod. 164,
1081–1092 (2017). https://doi.org/10.1016/j.jclepro.2017.07.031
3. Mareschal, B.: Weight stability intervals in multicriteria decision aid. Eur. J. Oper. Res. 33,
54–64 (1988). https://doi.org/10.1016/0377-2217(88)90254-8
4. Riabacke, M., Danielson, M., Ekenberg, L.: State-of-the-art prescriptive criteria weight
elicitation. Adv. Decis. Sci. 2012, 1–24 (2012). https://doi.org/10.1155/2012/276584
5. Lolli, F., Balugani, E., Ishizaka, A., Gamberini, R., Butturi, M.A., Marinello, S., Rimini, B.:
On the elicitation of criteria weights in PROMETHEE-based ranking methods for a mobile
application. Expert Syst. Appl. 120, 217–227 (2019). https://doi.org/10.1016/j.eswa.2018.
11.030
Logistic Regression for Criteria Weight Elicitation in PROMETHEE 479
6. Alemi-Ardakani, M., Milani, A.S., Yannacopoulos, S., Shokouhi, G.: On the effect of
subjective, objective and combinative weighting in multiple criteria decision making: a case
study on impact optimization of composites. Expert Syst. Appl. 46, 426–438 (2016). https://
doi.org/10.1016/j.eswa.2015.11.003
7. de Almeida Filho, A.T., Clemente, T.R.N., Morais, D.C., de Almeida, A.T.: Preference
modeling experiments with surrogate weighting procedures for the PROMETHEE method.
Eur. J. Oper. Res. 264, 453–461 (2018)
8. Figueira, J.R., Roy, B.: Determining the weights of criteria in the ELECTRE type methods
with a revised Simos’ procedure. Eur. J. Oper. Res. 139, 317–326 (2002). https://doi.org/10.
1016/S0377-2217(01)00370-8
9. Kaliszewski, I., Podkopaev, D.: Simple additive weighting—a metamodel for multiple
criteria decision analysis methods. Expert Syst. Appl. 54, 155–161 (2016). https://doi.org/10.
1016/j.eswa.2016.01.042
10. de Almeida, A.T., de Almeida, J.A., Costa, A.P.C.S., de Almeida-Filho, A.T.: A new
method for elicitation of criteria weights in additive models: flexible and interactive tradeoff.
Eur. J. Oper. Res. 250, 179–191 (2016). https://doi.org/10.1016/j.ejor.2015.08.058
11. Greco, S., Mousseau, V., Słowiński, R.: Ordinal regression revisited: multiple criteria
ranking using a set of additive value functions. Eur. J. Oper. Res. 191, 416–436 (2008).
https://doi.org/10.1016/j.ejor.2007.08.013
12. Kadziński, M., Greco, S., SŁowiński, R.: Extreme ranking analysis in robust ordinal
regression. Omega 40, 488–501 (2012). https://doi.org/10.1016/j.omega.2011.09.003
13. Greco, S., Kadziński, M., Mousseau, V., Słowiński, R.: ELECTREGKMS: robust ordinal
regression for outranking methods. Eur. J. Oper. Res. 214, 118–135 (2011). https://doi.org/
10.1016/j.ejor.2011.03.045
14. Doumpos, M., Zopounidis, C.: Preference disaggregation and statistical learning for
multicriteria decision support: a review. Eur. J. Oper. Res. 209, 203–214 (2011). https://doi.
org/10.1016/j.ejor.2010.05.029
15. Corrente, S., Greco, S., Kadziński, M., Słowiński, R.: Robust ordinal regression in
preference learning and ranking. Mach. Learn. 93, 381–422 (2013). https://doi.org/10.1007/
s10994-013-5365-4
16. Ishizaka, A., Lolli, F., Gamberini, R., Rimini, B., Balugani, E.: AHP-K-GDSS: a new sorting
method based on AHP for group decisions. In: Bruzzone, A.G., et al. (eds.) Proceedings of
the International Conference on Modeling and Applied Simulation, pp. 1–5. CAL-TEK S.r.
l., Barcellona (2017)
17. Ishizaka, A., Lolli, F., Balugani, E., Cavallieri, R., Gamberini, R.: DEASort: assigning items
with data envelopment analysis in ABC classes. Int. J. Prod. Econ. 199, 7–15 (2018). https://
doi.org/10.1016/j.ijpe.2018.02.007
3D CAD Design of Jewelry Accessories,
Determination of Geometrical Features
and Characteristics of the Used Material
of Precious Metals
Tihomir Dovramadjiev(&), Mariana Stoeva, Violeta Bozhikova,
and Rozalina Dimova
Departments of Industrial Design, Software and Internet Technologies
and Telecommunications, Technical University of Varna,
str. Studentska N1, 9010 Varna, Bulgaria
[email protected]
Abstract. The development of precious metal jewelry requires very good
management in terms of design, material cost, good marketing and advertising,
and good market positioning. This includes many aspects that, in the right
approach, build a complete system of expensive business. With the development
of modern technological means it is possible to pre-fabricate jewelry models in a
computer solid environment in which it is possible to calculate the amount of
precious metal material used. The purpose of this study is to develop a system
that provides preliminary data on the geometric features of expensive jewelry
models while at the same time calculating material costs.
Keywords: 3D Jewelry SolidWorks Mass properties Precious metals
1 Introduction
Interdisciplinary scientific fields are of increasing importance in contemporary reality.
This is directly related to the dynamics of our time, where we look for opportunities for
optimization of technological processes, efficiency, reduction of material costs and
time. When referring to engineering and technical design, this phenomenon affects
practically every aspect of the process. The application of Advanced Technologies in
Design is required, where CIM (Computer-Integrated Manufacturing) systems are
widely used [1, 2]. This study focuses on the initial stage of CAD (Computer Aided
Design) design, which is crucial in the hierarchical sequence [3–9]. The accuracy in
Computer-Aided Design justifies the correct calculation of the following elements of
the workflow. When looking at the jewelry business, but also referring to all areas of
precious metals application, accurate costing of material is required [10, 11]. This is of
Namely: The Advanced Technologies in Design is a discipline taught at Technical University of
Varna characterized by the application of modern technological tools such as CIM systems and its
sections, as well as other innovative applications.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 480–485, 2020.
https://doi.org/10.1007/978-3-030-39512-4_75
3D CAD Design of Jewelry Accessories 481
great importance both financially and in optimizing the application of the material,
which in some cases is extremely restrictive and difficult to access [12–17].
2 Materials and Methods
The main objective of the present work is to find an approach to optimize the design
process that covers two aspects: accuracy of the constructed solid geometry and mass
calculation of the developed models. This can be realized in the SolidWorks software
environment - one of the world leaders in CAD design. Based on the properly con-
structed 3D geometry of jewelry models (bracelets and rings of gold, silver and tita-
nium) and through computer calculations and the capabilities of SolidWorks Mass
Properties, we will get an idea with mathematical accuracy of the real mass of
expensive samples [18]. This gives a real opportunity for the proper management of the
production of the models, by monitoring at all stages of the creation of the real jewelry
models, by making an accurate estimate in advance. The present study aims to give a
real idea of how much is the influence of the smallest elements of the construction of
the developed 3D models by comparing in table form their obtained mass values. This
is of great importance considering the extent to which each precious metal particle is
stored. That is, there is no excess material and finding a way to optimize the design
process is crucial. Material Financial Impact data is based on bulk raw material price
computed from MetalPrices.com (2012; Accessed May 15th – SolidWorks reference)
[19]. Table 1 shows SolidWorks data for the metals used in the studies.
Table 1. SoldWorks metal properties
Pure gold Pure silver Titanium
Property Value Units Value Units Value Units
Elastic modulus 7.8e + 010 N/m^2 7.1e + 010 N/m^2 1.1e + 011 N/m^2
Poisson’s ratio 0.42 N/A 0.37 N/A 0.3 N/A
Shear modulus 2.6e + 010 N/m^2 2.5e + 010 N/m^2 4.3e + 010 N/m^2
Mass density 19000 kg/m^3 11000 kg/m^3 4600 kg/m^3
Tensile strength 103000000 N/m^2 125000000 N/m^2 235000000 N/m^2
Thermal expansion 1.4e-005 /K 2e-005 /K 8.8e-006 /K
coefficient
Thermal 300 W/(m•K) 420 W/(m•K) 22 W/(m•K)
conductivity
Specific heat 130 J/(kg•K) 230 J/(kg•K) 460 J/(kg•K)
The application of the appropriate material (the selected metal) is possible after the
three-dimensional solid geometry of the developed models has already been estab-
lished. Figure 1 shows the process of building 3D models of bracelets and rings, in this
case the variant with gold is referred to.
482 T. Dovramadjiev et al.
Fig. 1. SolidWorks 3D CAD system interface/3D design of bracelet and ring (have similar
geometry, the difference is approximately 1/3 in scale) (scale 0.33): (a) overall dimensions of the
bracelet: Outside D.: ∅ 65.5 mm, Inside D.: ∅ 63.5 mm 10 mm/Fillets: 2 R 0.8 mm and 2
R 0.2 mm; (b) obtained initial 3D geometry; (c) applying a decorative element using a ready-
made font vector/WWDesigns Font - free commercial license [20]; (d) construction of 3D
decorative elements on the basic geometry of the model through WRAP/Deboss with a value of
0.3 mm and Cir Pattern: Axis 1/360 deg/4/Equal spacing.
Figure 2 shows two finished 3D models of the developed bracelet design with and
without decorations respectively.
Fig. 2. Fully 3D models of gold bracelets: (a) with decoration and (b) without decoration.
3D CAD Design of Jewelry Accessories 483
The available results of mass properties calculation are: Density, Mass, Volume,
Surface area, Center of mass, Principal axes of inertia [18]. The moments of inertia and
products of inertia are calculated to agree with the following definitions: (1). The inertia
tensor matrix is defined below from the moments of inertia: (2).
Z 2 3
Ixx ¼ ðy2 þ z2 Þdm ð1Þ Ixx Ixy Ixz ð2Þ
6 7
Z
Iyy ¼ ðz þ x Þdm
2 2 4 Ixy Iyy Iyz 5
Z Ixz Iyz Izz
Izz ¼ ðx2 þ y2 Þdm
Z
Ixy ¼ ðxyÞdm
Z
Iyz ¼ ðyzÞdm
Z
Izx ¼ ðzxÞdm
The obtained Mass Properties values calculated using SolidWorks software of
Brace-lets and Rings/Pure Gold, Pure Silver and Titanium/With and without decora-
tions are shown on Tables 2 and 3.
Table 2. SolidWorks calculated mass properties of bracelets/Pure Gold, Pure Silver and
Titanium/With and without decorations
Bracelet Pure Gold Pure Silver Titanium
Mass properties With Without With Without With Without
decoration decoration decoration decoration decoration decoration
Density - grams per 0.02 0.02 0.01 0.01 0.0046 0.0046
cubic millimeter
Mass - grams 36.20 37.37 20.96 21.63 8.77 9.05
Volume - cubic 1905.50 1966.68 1905.50 1966.68 1905.50 1966.68
millimeters
Surface area - square 4612.90 4281.83 4612.90 4281.83 4612.90 4281.83
millimeters
Table 3. SolidWorks calculated mass properties of rings/Pure Gold, Pure Silver and
Titanium/with and without decorations
Ring Pure Gold Pure Silver Titanium
Mass properties With Without With Without With Without
decoration decoration decoration decoration decoration decoration
Density - grams per 0.02 0.02 0.01 0.01 0.0046 0.0046
cubic millimeter
Mass - grams 1.30 1.34 0.75 0.78 0.31 0.33
Volume - cubic 68.48 70.68 68.48 70.68 68.48 70.68
millimeters
Surface area - square 502.35 466.29 502.35 466.29 502.35 466.29
millimeters
484 T. Dovramadjiev et al.
Figure 3 shows a photorealistic rendering of part of the developed 3D models of
bracelet and three rings.
Fig. 3. Photo-realistic image of part of 3D models of gold bracelet with decoration and three
decorated rings (gold, silver and titanium) obtained by rendering in SolidWorks software.
3 Conclusion
The research carried out in this paper is wide-ranging. A complete process of three-
dimensional solid design through the SolidWorks CAD system is presented. The cal-
culation of the mass of precious metals used in jewelry is optimized by computer
calculations. Specific decorative elements to the basic geometry have been constructed
which, in addition to aesthetic beauty, give data for the difference in the mass between
the similar models. This gives reason to make a correct estimation of the consumption
of material at the expense of additional elements, which can reduce the mass but
increase the cost of making real models. This is strictly specific to individual decisions.
Calculations in the Mass Properties CAD environment give a new approach and vision
for the right attitude and work with precious metals, forming a positive trend in the
technological present and the future.
Acknowledgments. This paper (result) is (partially) supported by the National Scientific Pro-
gram “Information and Communication Technologies for a Single Digital Market in Science,
Education and Security (ICTinSES)” (grant agreement DO1-205/23.11.18), financed by the
Ministry of Education and Science.
3D CAD Design of Jewelry Accessories 485
References
1. Dovramadjiev, T.: Advanced Technologies in Design, p. 228. Technical University of
Varna, Bulgaria (2017). ISBN: 978-954-20-0771-5
2. Sandeep, T.R.: Computer Integrated Manufacturing. ACE, Bangalore. http://www.alphace.
ac.in/downloads/notes/me/10me61.pdf
3. Dassault Systems SolidWorks: 3D CAD Design Software. https://www.solidworks.com/
4. Lombard, M.: Solidworks 2013 Bible, 1st edn., 11 March 2013, p. 1296. Wiley (2013).
ISBN-13: 978-1118508404, ISBN-10: 1118508408
5. Smirnov, A.A.: Three-dimensional geometric modeling [Trehmernoe geometricheskoe
modelirovanie], pp. 1–40. MGTU, Moscow (2008)
6. Dassault Systems SolidWorks: Introducing Solidworks. https://my.solidworks.com/
solidworks/guide/SOLIDWORKS_Introduction_EN.pdf
7. Aliamovski,i A.A.: An engineering calculation in SolidWorks Simulation [Inzhenernyi
raschet v SolidWorks Simulation], p. 462. DMK Press, Moscow (2010)
8. Putz, C., Schmitt, F.: Introduction to computer aided design—concept of a didactically
founded course. J. Geom. Graph. 7(1), 111–120 (2003)
9. Lombard, M.: Mastering SolidWorks, 1st edn., p. 1248. Sybex, England (2018). ISBN-13:
978-1119300571, ISBN-10: 1119300576
10. Watkin, L.: Trends in the global jewelry Industry. Polygon.net (2014). https://www.polygon.
net/jwl/public/documents/resource-center/industry-reports/Trends-in-the-Global-Jewelry-
Industry-PD.pdf
11. Stuller: The Basics of Jewelry/Terminology and Design Guide, USA. https://www.
jmdjewelry.com/uploads/8/9/4/8/89486199/jewelry-basics.pdf
12. Sitko, J.: Analysis of selected technologies of precious metal recovery processes,
MAPE 2019, vol. 2, no. 1, pp. 72–80. Sciendo, Warsaw (2019). https://doi.org/10.2478/
mape-2019-0007
13. Aquino, S.: Recycling precious metals from mobile phones. M.B.A., University of Rochester
(2018). https://doi.org/10.14288/1.0362564,vancouver
14. Blasi L.: Precious metal financial instrument. US Patent: US8583547B2 (2013)
15. Gierałtowska, U.: Direct and indirect investment in precious metals, pp. 125–137.
Uniwersytet Szczeciński, Szczecin (2019). https://doi.org/10.17951/h.2016.50.4.125
16. Blatter, A.: Explosive joining of precious metals. Gold Bull. 31(3), 93–98 (1998). https://doi.
org/10.1007/bf03214769
17. Grimwade, M.: Introduction to Precious Metals. Brynmorgen Press, Brunswick (2009).
ISBN: 978-1-929565-30-6
18. Dassault Systems SolidWorks: Mass Properties Dialog Box https://help.solidworks.com/
2016/English/SolidWorks/sldworks/HIDD_MASSPROPERTY_TEXT_DLG.htm. Accessed
9 Oct 2019
19. MetalPrices.com./ https://www.argusmedia.com/en/metals#Tables-/SolidWorksreference.
Accessed 15 May 2012
20. FontSpace: WWDesigns Font by WindWalker64. https://www.fontspace.com/windwalker64/
wwdesigns. Accessed 9 Oct 2019
Discovering and Mapping LMS Course Usage
Patterns to Learning Outcomes
Darko Etinger(&)
Faculty of Informatics, Juraj Dobrila University of Pula,
Zagrebačka 30, 52100 Pula, Croatia
[email protected]
Abstract. The widespread use of Learning Management Systems in higher
education, a growing adoption of e-learning, distance, hybrid and blended
learning puts much pressure onto both students, to achieve learning goals and
lecturers, to design high quality online courses. Lecturers typically evaluate how
much the students have achieved at the end of the course. This exploratory study
attempts to uncover the relationship between usage behavior and students’
grades, i.e. what are the online course usage patterns performed by higher
graded students in contrast to lower graded ones. The core data is of the analysis
are the event logs extracted from the online course Modeling and simulation at
the Faculty of informatics in Pula and mapped to the students’ grades accu-
mulated from the final exams, assignments, projects and class tasks. Process
mining techniques were used for process discovery and process model analysis.
A set of procedures were developed (within the R programming environment) to
analyze the discovered process models. The findings indicate that a better
understanding of online course usage patterns and its relationship with learning
outcomes can be used to develop intelligent systems (recommender systems,
intelligent agents, intelligent personal assistants etc.) that can improve students’
learning process.
Keywords: Usage patterns Process mining Process discovery Learning
management systems
1 Introduction
Learning management systems (LMS) are nowadays widely adopted in higher edu-
cation institutions. Such systems are used to manage the learning experience and to
enable teachers’ and students’ involvement in the learning experience. In a typical
LMS, content in different formats (text, image, multimedia) is available through
courses to students. Communication and interaction between students and teachers is
available through message boards, forums, videoconferences or external plugins that
provide communication functionalities. Moodle, as a global development project
designed to support a social constructionist framework of education, is a software
package for producing web-based courses. The Moodle learning platform is provided
by Srce (University Computing Center) as the LMS of choice at the Faculty of
Informatics in Pula and offered teachers as a complementary tool to design personalized
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 486–491, 2020.
https://doi.org/10.1007/978-3-030-39512-4_76
Discovering and Mapping LMS Course Usage Patterns to Learning Outcomes 487
learning environments with the purpose of improving the teaching process and
achieving students learning outcomes.
A growing adoption of e-learning systems empowers educational institutions to
develop high quality courses for students. To design such courses, lecturers and support
staff need to engage, develop and implement such learning objects that will meet
students’ expectations and achieve the proposed learning outcomes. Lecturers typically
evaluate how much the students have achieved at the end of the course, by using
standardized metrics defined before the start of the course.
This study attempts to uncover the relationship between Moodle usage behavior
and students’ grades. By exploring the discovered process models by using process
mining techniques (if applied to educational setting, such analysis is part of Educa-
tional process mining), the goal is to identify what are the online course usage patterns
performed by higher graded students in contrast to lower graded ones. The core data for
the analysis are the event logs extracted from the online course Modeling and simu-
lation at the Faculty of informatics in Pula and mapped to the students’ grades accu-
mulated from the final exams, assignments, projects and class tasks.
The rest of this paper is organized as follows: Sect. 2 describes the theoretical
foundation for this study. Process mining techniques in the context of educational
process mining are discussed. In Sect. 3, the research methodology is described, and
process mining results are discussed. Finally, the fourth section outlines the contri-
butions and implications of study results.
2 Related Work
Process mining combines Data mining techniques and Process Modelling and Analysis
with Big Data and provides comprehensive sets of tools to provide fact-based insights
and to support process improvements [1]. It bridges the gap between model-based
process analysis (e.g., simulation and other business process management techniques)
and data-centric analysis techniques such as data mining and machine learning [2].
Educational process mining is an emerging field in the educational data mining dis-
cipline, concerned with discovering, analyzing, and improving educational processes as
a whole, based on information hidden in educational datasets and event logs [3]. Van
der Aalst [1] defines three distinct types of process mining techniques: process dis-
covery, conformance checking, and process enhancement. Process discovery, as a
combination of two perspectives, discovery task and control-flow, enables the con-
struction of the process model based on an event log. In this way, the behavior seen in
the log is captured.
By using process discovery techniques, usage behavior and usage patterns of LMS
can be explored, analyzed and knowledge about the LMS usage can be extracted. The
insight is based on real data, stored in the event log. Each event contains an ID (the case
ID), a time stamp, a description of the activity (activity ID), and additional resources.
Recently process mining techniques have gain traction because event data tend to be
huge (easily falling into Big data category) and thus sometimes, cannot be easily and
successfully analyzed by traditional data mining tools.
488 D. Etinger
Educational process mining (EPM) is concerned with developing methods to better
understand students’ learning habits and the factors influencing their performance [4].
EPM enables the discovery, analysis, and creation of visual representations of complete
educational processes [5]. The results of EPM can give insight of the underlying
educational process, generate recommendations, advice, and provide feedback to stu-
dents, teachers and researchers, to detect learning difficulties early, to help students
with specific learning disabilities, to improve management of learning objects [4].
3 Research Methodology and Results
Learning outcomes define what are the expected knowledge and skills a student should
acquire after completing a course. Assessments are closely related to learning outcomes
as they can provide information about students’: understanding of a topic, application
of acquired skills to specific problems, and overall, mastering the learning goals.
Assessments such as final exams, assignments, projects and class tasks are all graded
and provide a proxy measure about learning outcomes achievement.
The event data contained in the Moodle LMS database, is extracted from the online
course Modeling and simulation at the Faculty of informatics in Pula. This course is
part of the third-year undergraduate program in Computer Science. It covers one
semester, a sufficient time frame for exploration of students’ usage behaviors.
Clairns et al. [5] note that an event log is a hierarchically structured file with data
about historical process executions. Such file has to be constructed by structuring raw
process data that can be found in files or databases (e.g., the Moodle LMS), into events
and traces. An event contains a name, a specific timestamp associated with the event,
the originator of the event and other attributes, and represents the most atomic part of a
specific process execution. A trace is a collection of events that belong to the same
process. An LMS logs the user activities such as courses subscription, the content
accessed, attempted exams and corresponding scores, interaction via chat or discussion
boards.
The analysis procedure consisted of records collection from the Modeling and
simulation course in the Moodle database. The raw data was transformed into an event
log. Event logs are covering the time frame between October 1st, 2018 and June 19th,
2019. This time frame is relevant because it includes: classes, projects, exams and
assignments. The original data was then reshaped into a consolidated log (XES format
file), as a suitable shape for further processing with process mining algorithms. Each
individual student case is associated with its final grade, thus making it possible to split
the dataset into groups, based on the students’ grades. With this approach, it is possible
to explore usage patterns corresponding to the students’ final grades, as the main
research question is to find if different usage patterns between groups are present.
The event log (as XES – eXtensible Event Stream format) was imported into
Fluxicon Disco, the software package of choice for this process mining analysis. To
extract relevant information about the process model, the filter for showing activities
(50% of all activities based on absolute frequency) was applied. The groups were split
by the final grades as follows: excellent – 89–100% achievement (10 cases), very good
– 76–88.9% achievement (4 cases), good – 63–75.9% achievement (9 cases), and
Discovering and Mapping LMS Course Usage Patterns to Learning Outcomes 489
sufficient – 50–62.9% achievement (10 cases). The process models identified consist of
a total of 33 cases, 27 atomic activities, and 6499 events, each with its own times-
tamp. The fuzzy algorithm (process model discovery algorithm in Disco) was applied
on the filtered event log for four groups, and process maps were generated and
visualized.
Event logs in the education domain, particularly those coming from e-learning
environments, may contain massive amounts of fine granular events and process related
data [5]. The discovered models are often “spaghetti-like” showing all details without
distinguishing what is important and what is not [1]. To obtain a usable model, the
model discovered had to fit well with the students’ behavior. Tax et al. [6] propose the
Local Process Models (LPMs) allowing the mining of patterns positioned in-between
simple patterns and end-to-end models, focusing on a subset of the process activities
and describing frequent patterns of behavior. Figures 1, 2, 3 and 4 show the obtained
process maps for each individual group examined in this study.
Fig. 1. Process map – 89–100% achievement (50% of the activities - absolute frequency)
Visually, the process models and maps discovered show a difference in usage
patterns between groups. There are several insights provided for each group: the most
frequent activities performed, the maximum repetitions performed by students and an
overall importance of learning objects perceived by the students. Those metrics were
categorized and analyzed in the R programming environment.
All groups (Figs. 1, 2, 3 and 4) put much emphasize on project assignments. Those
cover specific skills including conceptual modeling, simulation program development
within the R programming environment or Python programming language, data anal-
ysis, visualization and interpretations of simulation results. The data show that the
number of repetitions grow as the grade lowers, meaning that lower graded students
show difficulties in acquiring these skills. The group ‘excellent’ (Fig. 1) reads the
literature the most, thus gaining theoretical knowledge and better understanding of
topics such as Discrete Event Simulation (DES), Systems Dynamics (SD), Agent-based
modeling (ABM) along with topics covering the probability distribution selection for
input data and the development of conceptual simulation models. The group ‘very
good’ (Fig. 2) showed interest for Monte Carlo techniques, as a static method for
simulations, but put lower accent on other simulation techniques. The last two groups
490 D. Etinger
Fig. 2. Process map – 76–88.9% achievement (50% of the activities - absolute frequency)
Fig. 3. Process map – 63–75.9% achievement (50% of the activities - absolute frequency)
Fig. 4. Process map – 50–62.9% achievement (50% of the activities - absolute frequency)
(Figs. 3 and 4) show similarities: underutilization of the provided learning objects and
a non-consistent usage of the LMS. The process maps for those groups look flat,
meaning their learning path is not based on the course program i.e. these students do
not engage in the learning process enough.
4 Conclusion
Educational process mining aims to detect, monitor and improve real-life processes by
extracting process-related knowledge from LMS event logs. The purpose of this study
was to discover LMS usage behavioral patterns between different students’ groups
based on their final achievement, by matching the cases in the event log with students’
grades, as proxies for learning outcomes. By applying a subset technique of process
Discovering and Mapping LMS Course Usage Patterns to Learning Outcomes 491
mining called process discovery, the process maps that describe the usage behavior
based on real facts and evidence were obtained for each individual group.
The findings indicate that a better understanding of online course usage patterns
and its relationship with learning outcomes can be used to develop intelligent systems
(recommender systems, intelligent agents, intelligent personal assistants etc.) that can
improve students’ learning process. As each students’ group follows as specific
learning path when using LMS, such intelligent systems could help them achieve the
proposed learning outcomes.
References
1. Van der Aalst, W.M.P.: Trends in business process analysis: from verification to process
mining. In: Proceedings of 9th International Conference Enterprise Information Systems
(ICEIS 2007), pp. 12–22 (2006)
2. Mans, R.S., van der Aalst, W.M.P., Vanwersch, R.J.B.: Process mining. In: Process Mining in
Healthcare: Evaluating and Exploiting Operational Healthcare Processes, pp. 17–26. Springer
International Publishing, Heidelberg (2015)
3. Cairns, A.H., Gueni, B., Assu, J., Joubert, C., Khelifa, N.: Analyzing and improving
educational process models using process mining techniques. In: IMMM 2015 Fifth
International Conference on Advances in Information Mining Management, pp. 17–22 (2015)
4. Ariouat, H., Cairns, A.H., Barkaoui, K., Akoka, J., Khelifa, N.: A two-step clustering
approach for improving educational process model discovery. In: 2016 IEEE 25th
International Conference Enabling Technology: Infrastructure for Collaborative Enterprises,
pp. 38–43 (2016)
5. Cairns, A.H., Gueni, B., Fhima, M., Cairns, A., David, S., Khelfa, N.: Process mining in the
education domain. Int. J. Adv. Intell. Syst. 8, 219–232 (2015)
6. Tax, N., Sidorova, N., Haakma, R., van der Aalst, W.M.P.: Mining local process models.
J. Innov. Digit. Ecosyst. 3, 183–196 (2016)
Drug Recommendation System for Geriatric
Patients Based on Bayesian Networks
and Evolutionary Computation
Lourdes Montalvo(&) and Edwin Villanueva
Pontifical Catholic University of Peru, Lima, Peru
{montalvo.lourdes,ervillanueva}@pucp.edu.pe
Abstract. Geriatric people face health problems, mainly with chronic diseases
such as hypertension, diabetes, osteoarthritis, among others, which require con-
tinuous treatment. The prescription of multiple medications is a common practice
in that population, which increase the risk of unwanted or dangerous drug
interactions. The quantity of drugs is constantly growing, as are they interactions.
It is therefore desirable to have support systems for medical that digest all
available data and warn for possible drug interactions. In this paper we proposed a
drug recommendation system that takes into account pre-existing diseases of the
geriatric patient, current symptoms and verification of drug interactions.
A Bayesian network model of the patient was built to allow reasoning in situa-
tions of limited evidence of the patient. The system uses also a genetic algorithm,
which seeks the best drug combination based on the available patient information.
The system showed consistency in simulated settings, which were validated by a
specialist.
Keywords: Drug recommendation Drug-drug interaction Genetic
algorithm Bayesian networks Drug-target interaction
1 Introduction
The selection of appropriate pharmacotherapy for elderly people has been recognized as a
challenging and complex process and has become an important public health issue [5].
One of the problems that face geriatric patients is adverse drug reactions (ADRs). This
problem is a major cause of hospital admissions, thereby leading to significant medical
and economic problems [6]. In US, drug-drug interactions (DDI) have been responsible
for approximately 26% of the Adverse Drug Reactions (ADR) annually [7], affecting
50% of inpatients, and causing nearly 74,000 emergency room visits and 195,000
hospitalizations.
Multimorbidity, the coexistence of two or more chronic conditions, is in increasing
rate and so the polypharmacy and the ADR cases [2]. This increases the risks and costs
of the primary health care by causing serious health effects or reducing the therapeutic
effect of some compounds [1, 2].
Some authors have proposed intelligent systems to support physicians in the pre-
scription task. In [8] the authors propose a drug-recommendation system for patients
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 492–497, 2020.
https://doi.org/10.1007/978-3-030-39512-4_77
Drug Recommendation System for Geriatric Patients 493
with infectious diseases. The system classifies patient’s abilities to protect themselves
from infectious diseases. The authors constructed a knowledge base that included more
than 60 risk factors, and developed a web-based prototype system. In [9] the authors
developed a medicine recommender system framework that incorporate data mining
technologies to uncover potential knowledge hide in medical records that can be used
to decrease medical errors. Although these systems take into account the patient’s
medical history, they are not aware of possible drug interactions, which means that the
recommendations may not be safe.
In [7] the authors propose an ADR-aware model that recommends safe drugs that
can be taken together with the prescription. One limitation of this model is that the
recommendation is built only on drug information. However, it is known that inter-
actions do not only occur between drugs but also between drugs and diseases.
In the present work, we proposed the development of a drug recommendation
system for geriatric patients that take into account drug-drug interaction considering
pre-existing diseases, current symptoms and verification of drug interactions. A model
of the geriatric patient was built based on a Bayesian network, which allows inferences
to be made when some patient information is available. After that, the development of
the genetic algorithm seeks to obtain the best combination of medications for the
geriatric patient with diverse medical history.
2 Methodology
To develop our drug recommendation system we first developed a Bayesian network
model to encode the probabilistic interactions between common geriatric pathologies,
information of pre-existing diseases, medications to be prescribed and current symp-
toms of the patient. Secondly, we implemented an evolutionary algorithm to be the
search engine of drug combinations that maximizes the absence of symptoms in the
geriatric patient, taking into account the patient’s safety regarding possible drug
interactions. Finally, we implemented the recommendation system in a web platform
that allows using the functionalities of the proposed algorithm.
For the first activity, it was decided to review the most frequent pathologies in
geriatric patients. Then, the most frequent medications were sought to treat these dis-
eases. It was reviewed articles from WHO (World Health Organization) and [12] related
to potential inappropriate prescriptions in large elderly patients polymedicated. It was
found that, from a sample of 349 patients, 223 suffered from arterial hypertension, which
makes it the most frequent pathology in this sample. For each selected pathology we
carried out an investigation to determine which medications are the most frequent and
effective according to medical evidence. To establish the relationship between one
medication and another, it was necessary to review medical articles [13, 14]. We also
obtained drug information from [10] and the DrugBank database [11], which is one of
the most embracing drug information source and their interactions.
With the pathologies and medications chosen, a Bayesian network was imple-
mented. The probabilities of prescribing medications used for patients with a certain
medical profile were obtained from textual medical evidence. The motivation of using
Bayesian networks come from their natural capability to manipulate uncertainty [4].
494 L. Montalvo and E. Villanueva
In our case, this uncertainty appears by the limited evidence that is normally available
about the state of the patient and the variability of the biological processes.
For the construction of the Bayesian network, it was decided to define a nomen-
clature for the random variables of the model, which helped to differentiate between a
pre-existing disease (AN), current symptom (SA) and medication to be prescribed
(MR). Figure 1 show a resulting Bayesian network model for Cardiology related
diseases. The implementation of the Bayesian networks was done in Python pro-
gramming language, using the pgmpy library (pgmpy.org).
Fig. 1. BN model for Cardiology diseases. The nodes at the bottom indicate the patient’s current
symptoms, the nodes in the middle represent the medications to be prescribe, and the nodes on
the top represent the pre-existing diseases in the patient.
With the BN model constructed, we implemented a genetic algorithm to obtain the
best combination of medications for a given patient. Genetic algorithms (GA) are
algorithms inspired in the natural evolution process that seek to improve a population
of candidate solutions (individuals) by performing operations of crossing-over, muta-
tion and selection. In our genetic algorithm, an individual represent a possible drug
combination. This is coded as a boolean vector, where each element indicates the
presence or absence of a particular drug. The variables we used in the algorithm are
described in Table 1.
Table 1. Variables for the construction of the genetic algorithm
Variables Definition
Gen Medicine (Eg. “Enalapril”, “Propanolol”)
Allele State of Medicine (“Prescribed” (0) and “Not Prescribed” (1))
Chromosome A candidate drug combination represented by a vector of alleles
Drug Recommendation System for Geriatric Patients 495
To obtain the fitness of each individual in the GA we use the BN model previously
described as a simulator of the patient. Given some information of the patient, like pre-
existing diseases and symptoms, and the medications to be tested (encoded in the
chromosome), the BN model is use to compute the posterior probabilities of all
modeled symptoms. Such probabilities are used to compute the fitness of the indi-
vidual. In other words, the fitness function is designed to maximize the absence of the
symptoms that the geriatric patient presents. Equation 1 depict the fitness function.
PAXi represent the posterior probability of absence of symptom i (PAXi) given by the
BN. It is multiplied by the weight pi, that is the importance of the symptom given by
the doctor. All the weighted probabilities are added and then normalized by the number
of symptoms. Table 2 show the parameters of the fitness function
Pn
PAXi pi
fitness function ¼ Pn
i¼0
ð1Þ
i¼0 pi
Table 2. Parameters of the fitness function
Parameter Detail
Patient’s background The dictionary (key, value) will contain the name of the
pathology and the state (present, absent)
Medicines to be The dictionary (key, value) will contain the name of the
prescribed medicine and the state “Prescribed” (0) and “Not Prescribed” (1)
Symptoms of the It will be necessary to pass all the symptoms related to the
indicated pathology indicated pathology
Weight adjustment of The doctor will be able to consider the importance of each
symptoms symptom; therefore, a respective weight will be assigned
between 0 to 1
Bayesian Network For the proposed project, four pathologies were modeled, which
were chosen because they are the most frequent in elderly
patients. For each modeled pathology, a Bayesian network is
proposed
Fig. 2. Deployment diagram for the drug recommendation system. It shows the Postgresql
Database, Web Services with Ruby on Rails and the framework Flask which contains the
algorithm.
496 L. Montalvo and E. Villanueva
For the development of the web system, we followed the component architecture
presented in Fig. 2. The algorithm source code and models are available in the
repository https://github.com/louMon/MedGeriatricThesis.
3 Experiments and Results
We performed several tests of the system to assess its effectiveness and efficiency.
Figure 3 show a typical behavior of the evolution of the fitness function in the GA for
the case of Cardiology diseases. We used in such scenery 8 common medications as a
pool of drugs. We simulated a case of a geriatric patient with a history of Bradycardia,
Chronic Constipation and with high blood pressure. In the genetic algorithm we used
30 individuals, 25% of mutation rate and 50 generations in the evolution. We can
observe that the system is able to find the best recommendation in about ten generations
and with a fitness of the solution of 89%. The recommended drugs were positively
validated by a specialist in Cardiology.
Fig. 3. Execution of the medical recommendation for a geriatric patient with a history of
Bradycardia, Chronic Constipation and currently has high blood pressure; Likewise, the current
diagnosis is the presence of Arterial Hypertension with a weight of 100%. Which resulted in the
recommendation of the drugs Enalapril and Clortalidone.
4 Conclusion
The proposed system showed consistency in the recommendations provided. The
Bayesian networks models were able to capture the probabilistic interactions between
drugs, pre-existing diseases and symptoms, which is an advance over previous systems.
As a future research we target to enrich the patient simulator with a greater number of
Drug Recommendation System for Geriatric Patients 497
variables (greater amount of pathologies, physical characteristics, vital signs of the
patient, mood, among other things) so that the model can improve its precision in the
recommendation.
References
1. World Health Organization, Multimorbidity-Technical Series on Safer Primary Care (2016)
2. Cascorbi, I.: Drug interactions-principles, examples and clinical consequences (2012)
3. Nilashi, M., Bagherifard, K., Ibrahim, O., Alizadeh, H., Nojeem, L.A., Roozegar, N.:
Engineering and technology - collaborative filtering recommender systems. Res. J. Appl.
Sci. (2012)
4. Ankan, A., Panda, A.: Mastering Probabilistic Graphical Models using Python. Packt
Publishing (2015)
5. Dieter Genser. Food and Drug Interaction: Consequences for the Nutrition/Health Status,
Austria (2008). https://doi.org/10.1159/000115345
6. Köhler, G.I., Bode-Böger, S.M., Busse, R., Hoopmann, M., Welte, T., Böger, R.H.: Drug-
drug interactions in medical patients: effects of in-hospital treatment and relation to multiple
drug use (2000). https://doi.org/10.5414/cpp38504
7. Chiang, W.-H., Li, L., Shen, L., Ning, X.: Drug Recommendation toward Safe
Polypharmacy, United Kingdom (2018). https://doi.org/10.1145/nnnnnnn.nnnnnnn
8. Shimada, K., Takada, H., Mitsuyama, S., Ban, H., Matsuo, H., Otake, H., Kunishima, H.,
Kanemitsu, K., Kaku, M.: Drug recommendation system for Patients with Infectious
diseases, Japan (2005)
9. Bao, Y., Jiang, X.: An intelligent medicine recommender system framework. In: IEEE 11th
Conference on Industrial Electronics and Applications, China (2016)
10. Organización Panamericana de la Salud. Clasificación Estadística Internacional de
Enfermedades y Problemas Relacionados con la Salud (2008). http://www.insn.gob.pe/
sites/default/files/publicaciones/CIE-10-v.3.pdf
11. Drugbank. Canadian Institutes of Health Research. https://www.drugbank.ca
12. López-Sáez, A., Sáez-López, P.: Prescripción inadecuada de medicamentos en ancianos
hospitalizados según criterios de Beers. Farmacia Hospitalaria (2012)
13. Akbar, S., Alorainy, M.S.: The current status of betablokers use in the management of
hypertension (2014)
14. Gil de Miguel, A., Jiménez García, R., Carrasco Garrido, P., Martínez González, J.,
Fernández González, I., Espejo Martínez, J.: Seguridad y efectividad de amlodipino en
pacientes hipertensos no controlados farmacológicamente en el ámbito de Atención Primaria
(2000)
Software for the Determination of the Time
and the F Value in the Thermal Processing
of Packaged Foods Using the Modified
Ball Method
William Rolando Miranda Zamora, Manuel Jesus Sanchez Chero(&),
and Jose Antonio Sanchez Chero
Universidad Nacional de Frontera, Sullana, Peru
[email protected], [email protected],
[email protected]
Abstract. The software provided in a simple and agile way the time/F value of
thermal processing of packaged foods. The objective of this research was to
develop a software to perform the calculations of thermal processing of foods
based on the Ball´s original modified formula method, which was based on the
discrepancies already identified. To check the validity of the equation of Ball’s
original formula modified, both formulas were compared with the tabulated data
by Ball and programs were developed to evaluate the thermal process with this
new modified equation. Differences were found ranging approximately between
0.03% and 4% with the tabulated data. From the linear regression analysis for
the different C values for the range of z values from 14 to 26 °F the coefficients
of determination are close to unity, so this could be taken into account as an
alternative model to the equation, which was raised almost a century ago.
Keywords: Ball equation Ball tables Formula method C value
1 Introduction
The general method is a technique, which relies on a graphical or numerical integration
procedure based on temperature–time data, and it is used to obtained Lethality or F0
value [1]. However, it has been a target of much criticism for two reasons. First, due to
its limited prediction power and second, because of its lack of effectiveness for pro-
viding accurate information regarding the thermal processing of food. This led to the
emergence of other methods commonly known as a “formula method”. A method,
which involves interpreting experimental heating curves, represented by a range of
equations. The method has its origins in the first formula method published by Charles
Olin Ball in 1923, and in the basis for most developments that process determination
use fh and jc values for heating and cooling, and a hyperbola for representing the
curvilinear parts of the heat penetration curve. Applicable for jc = 1.41 and cooling
time tc = 1.41fc [2–4]. Small modifications in the nomenclature regarding the equation
or formula can be observed in studies [5–7]. In the current study, other difference
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 498–502, 2020.
https://doi.org/10.1007/978-3-030-39512-4_78
Software for the Determination of the Time and the F Value 499
managing to set up constants were possible to determine, because a smaller margin of
error respected to the original Ball method was proved.
It should be the exponential function in the heating contribution part E1 instead of
Ei and a negative sign of the first and second term, which is incorrect [8–10]. At the
same time, while searching a method which allowed to improve Ball’s formula method
without the use of tables [5, 6] did not agree with the equation used for the develop-
ment of them (the tables). Other formula methods were also compared by other authors
[11–15].
2 Methodology
2.1 Calculation of the Values of the Coefficients of Ball’s Equation
and Incorporation of One Factor in the Second Coefficient
The solution involved the following steps: (1) Find the coordinates of the intersection
point from the hyperbola and logarithmic cooling sections. (2) Replace those coordi-
nates in the cooling hyperbola equation to obtain the first relation between ac and bc.
(3) Find the derivatives of the hyperbola and logarithmic cooling equations. (4) Sub-
stitute the coordinates of the point of intersection in these two differential equations.
(5) The second relation between ac and bc is derived by equality of the differential
equations. (6) Determine the expressions for the adjusted hyperbola constants from the
relationships obtained in steps (2) and (5). Taking into consideration that the E value in
Eq. (1) is always positive, a higher C value is needed than those calculated ones [14].
Therefore, after a comparative analysis with the C values according to several authors
[5, 6], a value of ln10 = 2.303 is incorporated in Ball's original formula.
" 0:300m
#
0:33172 0:5833ze e ze
Ccl ¼ 0:343m þ E : ð1Þ
e ze m
2.2 Generation of the New C Values and Software Development
The new C values are found by the calculated coefficients, which are described in the
six steps described previously in 2.1, and by the incorporation of the value “ln10”
indicated in 2.2. An equation is developed in Microsoft Excel © 2018 to generate the
new C values. According to [4] the C value can be calculated from the following
equation:
C ¼ Ch þ Ccl þ Cc ln10: ð2Þ
where Ch, Ccl and Cc are the contributions of the C value of the heating phases, as wee
as the start and the end cooling respectively. After comparing the C values, it was
proceeded to develop a software to calculate the F0 value and the process time tB by
applying the modified equation departing from the first Ball’s equation. This equation
has been used in plants where thermal treatment or processing is done.
500 W. R. Miranda Zamora et al.
3 Case Study and Results
3.1 Calculation of the Values of the Coefficients of Ball’s Equation
and Incorporation of a Factor in the Second Coefficient
The derivatives of cooling hyperbola and logarithmic equations, at the point of inter-
section, the two slopes of the two cooling equations are equal. Thus, in order to obtain
the second relation between ac and bc, the two differential equations may be equated:
P 0:226732
ac ¼ 0:343 ðTg TCW Þ: ð3Þ
0:453464 P
and
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
s
0:453464 0:0514074 1 0:453464
bc ¼ 1 þ þ fc: ð4Þ
P P2 P2 P3
In this model, firstly the cooling curve includes a hyperbola section and secondly a
logarithmic section. The equations of the cooling curves are the same as described in
the Ball method, with the adjusted constants ac and bc and also with the changing time
and temperature which are calculated in the same way. Equation (1) is rewritten as
follows after finding the ac y bc values, as well as incorporating th ln10:
2 rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
h i 3
6 bc ðac m þ 0:343mÞ2 ðac mÞ2 ac m
7
bc ln10ze e ze
Ccl ¼ 6
4 ac m 0:343m þ E7
5: ð5Þ
e ze ac m
3.2 Generation of the New C Values and Software Development
From the time–temperature data obtained for each thermocouple location in each food
product: the initial heating temperature (TI), the temperature of the autoclave or retort
(TR) [16, 17] were obtained. Therefore, the parameters of heat penetration of Ball fh
and jh were calculated. For simple curve and fh, fh2 and xbh (breaking time) broken
heating curve [18], as well as the z value from the kinetic curve of thermal death of the
microorganism [19, 20]. This software makes it possible to evaluate the time of Ball
(tB) and the F value of the process [21–23]. Regarding the analysis of the C–values
tabulated and the calculated C–values, the average relative error oscillates between a
minimum of 0.03% and a maximum of 4% for the z–values of 14, 18, 22 and 26 °F.
The coefficients of determination (R2) were 0.9995 for z = 14 °F and z = 18 °F and
0.9996 for z = 22 °F and z = 26 °F. These results are similar with respect to the
mathematical model of several authors [23, 24] where coefficients of determination
close to one were found. To calculate the F0 value and the process time tB using the
modified equation from the original Ball equation, the software was developed in
Visual Basic © 6.0. The inputs and outputs are shown in Table 1 for a case study.
Software for the Determination of the Time and the F Value 501
Table 1. Inputs and outputs of the program in case study.
Parameter tB (min) F (min)
jh 1 1.32
fh (min) 30 20
fh2 (min) 6 6
xbh (min) 6 6
F (min) 6 –
tB (min) – 80
TI (°F) 50 72
TR (°F) 245 208
Tref (°F) 250 200
z (F°) 18 16
m + g (F°) 180 130
23.42 210.02
4 Conclusions
A modification has been incorporated into the equation put forward 93 years ago by
Ball. A program has been developed to calculate the new C values and, when compared
with the tabulated C values, inaccuracies ranging between 0.03% and 4% were found.
The regression analysis for the different C values for the range of z values from 14 to
26 °F is close to unity, so it could be considered a good alternative model to the
equation proposed by Ball. Validation of published Ball tables versus their equations
have been shown. Also, an equation has been used to establish its tables based on the
analysis. The software developed to evaluate the thermal process is extremely useful
for engineers, technologists and food scientists (research and education).
References
1. Bigelow, W.D., Bohart, G.S., Richardson, A.C., Ball, C.O.: Heat penetration in processing
canned foods. Bulletin No. 16L. National Canners’ Association, Washington, DC (1920)
2. Holdsworth, S., Simpson, R.: Thermal Processing of Packaged Foods, 2nd edn. Springer,
New York (2007)
3. Holdsworth, S., Simpson, R.: Thermal Processing of Packaged Foods, 3rd edn. Springer,
New York (2016)
4. Miranda–Zamora, W.R., Bazán, J.F., Ludeña, A.L., Tapia, D.A.: Herramientas computa-
cionales aplicadas a la evaluación de tratamientos térmicos de los alimentos envasados
usando el método de Ball. Universidad Nacional de Piura, Piura–Perú (2010)
5. Ball, C.O.: Thermal process time for canned foods. Bulletin No. 37. National Research
Council, Washington, DC (1923)
6. Ball, C.O., Olson, F.C.W.: Sterilization in Food Technology–Theory, Practice and
Calculations. McGraw–Hill, New York (1957)
7. Goldblith, S.A., Joslyn, M.A., Nickerson, J.T.R. (eds.): Introduction to Thermal Processing
of Foods. The AVI Pub. Co., Inc., Westport (1961)
502 W. R. Miranda Zamora et al.
8. Stoforos, N.G.: On Ball’s formula method for thermal process calculations. J. Food Process
Eng. 13(4), 255–268 (1991)
9. Merson, R.L., Singh, R.P., Carroad, P.A.: An evaluation of Ball’s formula method of
thermal process calculations. Food Technol. 32(3), 66–72, 75 (1978)
10. Steele, R.J., Board, P.W.: Amendments of Ball’s formula method for calculating the lethal
value of thermal processes. J. Food Sci. 44, 292–293 (1979)
11. Stumbo, C.R., Longley, R.E.: New parameters for process calculations. Food Technol. 20,
341–345 (1966)
12. Hayakawa, K.: Experimental formulas for accurate estimation of transient temperature of
food and their application to thermal process evaluation. Food Technol. 24(12), 1407–1418
(1970)
13. Smith, T., Tung, M.A.: Comparison of formula methods for calculating thermal process
lethality. J. Food Sci. 47(2), 626–630 (1982)
14. Steele, R.J., Board, P.W., Best, D.J., Willcox, M.E.: Revision of the formula method tables
for thermal process evaluation. J. Food Sci. 44, 954–957 (1979)
15. Miranda-Zamora, W.R., Teixeira, A.A.: Principios matemáticos del proceso térmico de
alimentos. AMV (Antonio Madrid Vicente) Ediciones, Madrid-España (2012)
16. Etzel, M.R., Willmore, P., Ingham, B.H.: Heat penetration and thermocouple location in
home canning. Food Sci. Nutr. 3(1), 25–31 (2015)
17. Mohamed, I.O.: Determination of cold spot location for conduction–heated canned foods
using an inverse approach. Int. J. Food Process. Technol. 2, 10–17 (2015)
18. Thakur, R.S., Rai, D.C.: Heat penetration characteristics and physico–chemical properties of
retort processed shelf stable ready to eat palak paneer. Int. J. Chem. Stud. 6(4), 949–954
(2018)
19. Ammu, D., Mohan, C.O., Panda, S.K., Ravishankar, C.N., Gopal, T.K.S.: Process
optimisation for ready to eat tapioca (Manihot esculenta crantz) in high impact polypropy-
lene containers. J. Root Crops 43(1), 104–110 (2017)
20. Ammu, D., Mohan, C.O., Panda, S.K., Ravishankar, C.N., Gopal, T.K.S.: Process
optimization for ready to eat Indian mackerel (Rastrelliger kanagurta) curry in high impact
polypropylene (HIPP) containers using still water spray retort. Indian J. Fish. 64(2), 83–89
(2017)
21. Condón-Abanto, S., Raso, J., Arroyo, C., Lyng, J., Álvarez, I.: Quality–based thermokinetic
optimization of ready–to–eat whole edible crab (Cancer pagurus) pasteurisation treatments.
Food Bioprocess Technol. 12, 436 (2019)
22. Mugale, R., Patange, S.B., Joshi, V.R., Kulkarni, G.N., Shirdhankar, M.M.: Heat penetration
characteristics and shelf life of ready to serve eel curry in retort pouch. Int. J. Curr.
Microbiol. Appl. Sci. 7(2), 89–100 (2018)
23. Friso, D.: A mathematical solution for food thermal process design. Appl. Math. Sci. 9(5–8),
255–270 (2015)
24. Friso, D.: A new mathematical model for food thermal process prediction. Model. Simul.
Eng. 2013, 1–8 (2013)
Communication Protocol Between Humans
and Bank Server Secure Against
Man-in-the-Browser Attacks
Koki Mukaihira1, Yasuyoshi Jinno1, Takashi Tsuchiya1,
Tetsushi Ohki1, Kenta Takahashi2, Wakaha Ogata3,
and Masakatsu Nishigaki1(&)
1
Shizuoka University,
3-5-1 Johoku, Naka, Hamamatsu, Shizuoka 432-8011, Japan
[email protected]
2
Hitachi Ltd., 292 Yoshida, Totsuka, Yokohama, Kanagawa 244-0817, Japan
3
Tokyo Institute of Technology,
2-12-1 Ookayama, Meguro, Tokyo 152-8550, Japan
Abstract. In recent years, illegal money transfer using man-in-the-browser
(MITB) attacks with malware in internet banking has become a social problem.
This study focuses on transaction-tampering MITB attacks and considers
countermeasures to combat them. The existing countermeasures require that
another device, such as a token device, other than the device used to carry out
the actual money-transfer operation is utilized to ensure a secure route different
from that carrying out the actual money-transfer operation. However, because
MITB attackers have become more sophisticated, malware will be able to infect
all the devices in the end and the multiple routes approach will have become
ineffective against this type of attack. Therefore, we constructed a human-to-
machine communication protocol that requires only one device for each user and
is secure against transaction-tampering MITB attacks, even if the device is taken
over by malware.
Keywords: Man-in-the-browser attack CAPTCHA Security proof Internet
banking
1 Introduction
Illegal money transfer using man-in-the-browser (MITB) attacks with malware has
recently become a social problem in internet banking. Depending on the attack sce-
nario, MITB attacks are classified into two types: ID stealing and transaction tampering
[1]. Herein, we focus on transaction-tampering MITB attacks and consider counter-
measures to combat them. A transaction-tampering MITB attack is an attack in which
malware sends a money-transfer request different from that specified by a user (e.g.,
payee, amount of money) to force the server to accept an altered transaction request. As
confirmation information sent back from the server is also forged by malware, the user
is unable to notice the change. The existing countermeasures [2, 3] have approached
this by utilizing a device, such as a token device, other than that used to carry out the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 503–509, 2020.
https://doi.org/10.1007/978-3-030-39512-4_79
504 K. Mukaihira et al.
actual money-transfer operation to ensure a secure route different from the route car-
rying out the actual money-transfer operation. However, with the advance of defensive
technology, attackers have diversified their attack technology and technique. Ulti-
mately, as long as machine-to-machine (M2M) communication is carried out, defense
may lag behind sophisticated attackers, and it will be difficult to implement measures
against such attacks. This scenario makes it necessary for the presence of humans at the
end points of communication in order to initiate measures against such attacks.
Conventional cryptographic techniques cannot be used when realizing secure
human-to-machine (H2M) communication. In encryption, the calculation difficulty gap
of a one-way function is utilized, in which the function evaluation is easy, but the
inverse calculation is difficult. Unfortunately, however, humans only have low com-
putational power. Even “easy” function evaluation in this context is still difficult for
humans. Therefore, such gaps cannot be used in H2M communications. Instead,
humans have high cognitive ability that machines do not have, that is, there is a
cognitive gap between humans and machines or malware. This gap can be used to
realize secure H2M communication. To construct the secure H2M communication
channel, a problem that is easy for humans to solve but difficult for machines is used. In
this study, the completely automated public Turing test to distinguish between com-
puters and humans (CAPTCHA) is used as an example of such a problem.
The contributions of this study are as follows. (1) A security concept, indistin-
guishability against chosen CAPTCHA problems attack (IND-C-CCA) security for
CAPTCHA, is defined as an analogy of the well-known indistinguishability against
chosen ciphertext attack (IND-CCA) security for public-key encryption. It can be used
as one of the indices for the construction of CAPTCHA. (2) A system model of the
money-transfer request protocol and its security against transaction-tampering MITB
attacks are defined. These can be used as one of the indices for the construction of the
money-transfer request protocol. (3) A CAPTCHA-based money-transfer request
protocol is proposed. The proposed protocol is secure against MITB attacks, provided
that the underlying CAPTCHA scheme has IND-C-CCA security. These results
illustrate that as long as a CAPTCHA is constructed satisfying the security constraints
defined in (1), a money-transfer request protocol having the security defined in (2) can
be realized1.
2 Model of CAPTCHA
2.1 Formulation
CAPTCHA [4] is a security technology, which asks a user to solve a problem that is
easy for humans but difficult for machines, and then identifies the user as a human or
machine. In this paper, CAPTCHA is defined as a pair of CAPTCHA encryption
algorithm C Enc and CAPTCHA decryption function C Dec.
1
This study has been originally published in Japanese at IPSJ (Information Processing Society of
Japan) Journal, Vol. 60, No. 12, pp. 2147–2156, December 2019.
Communication Protocol Between Humans and Bank Server Secure 505
CAPTCHA encryption algorithm C EncðmÞ: A probabilistic polynomial time
k
(PPT) algorithm that inputs a solution m 2 Mk2N ; Mk f0; 1g and outputs a problem
c 2 Ck2N ; Ck f0; 1glðkÞ corresponding to m where lðkÞ is a polynomial for k and
f0; 1glðkÞ represents a set of bit sequences of bit length lðkÞ. Moreover, we assume that
C Enc is an injective function.
CAPTCHA decryption function C DecðcÞ: A mapping that maps a problem c ¼
C EncðmÞð2 Ck Þ to a solution m ðCk ! M k Þ. Moreover, we assume that humans with
high cognitive ability can compute C Dec in real time, but it is difficult for machines to
compute C Dec in real time.
2.2 Security Definition for CAPTCHA: IND-C-CCA Security
In this study, we consider IND-C-CCA security, which is similar in concept to the
IND-CCA in public-key cryptography. We define the IND-C-CCA game played by
attacker B and a challenger as follows. (1) B randomly chooses two solutions, m0 and
m1 ðm0 ; m1 2 M k Þ, and sends them to the challenger. (2) The challenger randomly
chooses m0 or m1 , sets it to mb , encrypts it into CAPTCHA problem c, and sends it to B
as a challenge. (3) B outputs b b. If bb ¼ b, the attacker wins.
In this game, B can use human oracle H at any time. The human oracle is an
extension of the random oracle and is used widely to cryptographically model
CAPTCHA [5]. In the human oracle model, B can query the oracle for a solution to any
CAPTCHA problem, except for challenge c received in (2). The advantage of B in the
IND-C-CCA game is defined as follows:
1
AdvINDCCCA
¼ Pr½B wins
B
2
CAPTCHA has IND-C-CCA security if AdvINDCCCA
B is negligible for any B.
3 Model of Money-Transfer Request Protocol Using
CAPTCHA
3.1 Model of Money-Transfer Request Protocol
We consider the following model of the money-transfer request protocol using
CAPTCHA. A user (or a human) S sends the money-transfer information xS to the
server R via the browser. The user and the server communicate through the browser.
All processing performed by the user is limited to solving the CAPTCHA. Based on the
information received through the protocol, the server outputs xR , or outputs ⊥, meaning
“stop money transfer.” In the following, SHðÞ denotes the user with high cognitive
ability. Completeness and security against MITB attacks are defined as essential
requirements of this protocol.
506 K. Mukaihira et al.
Completeness: The protocol satisfies completeness if S sends xS first, S and R carry out
the protocol, and R outputs xR ¼ xS with probability 1 e, where e is a negligible
value.
As security against MITB attacks, we consider the SUB-MIM security described in
the next subsection.
3.2 Definition of SUB-MIM Security
Substitution man-in-the-middle (SUB-MIM) security is defined by the SUB-MIM
game. This game consist of a learning phase and attack phase (Fig.1). Attacker A is a
PPT algorithm that can perform the money-transfer request protocol by employing a
H ðÞ
human with high cognitive ability SHðÞ . Use of SHðÞ by A is indicated by AS .
Because MITB attacks, which are fully automated by malware, are the target of this
paper, it is noted that A cannot directly use HðÞ in the learning phase and attack phase,
but only via S.
– Learning phase: SHðÞ and A run the protocol n times to obtain the transaction p. Let
xðS;iÞ be the SHðÞ ’s input in the ith protocol execution.
– Attack phase: SHðÞ and R run the protocol, but A acts as a man in the middle. Upon
receiving xS from SHðÞ , A arbitrarily tampers with xS and sends it to R as x0S . A
executes the protocol using SHðÞ and R, and finally R outputs xR or ⊥. The win
condition of A in the SUB-MIM game is to make R accept xR that is different from
the input xS of SHðÞ . Thus, in the SUB-MIM game, A wins if R outputs xR ð6¼ xS Þ.
The advantage of A in the SUB-MIM game is defined as follows:
AdvSUBMIM
A ¼ jPr ½A winsj
The protocol has SUB-MIM security if AdvSUBMIM
A is negligible for any A.
4 Proposed Protocol and Security Proof
4.1 Proposed Protocol
The proposed protocol is depicted in Fig. 2. In Fig. 2, Capt ¼ ðC Enc; C DecÞ is
CAPTCHA and R is a set of random numbers with a sufficiently large size.
4.2 Security Proof
We prove the following theorem as a security proof for the proposed protocol.
Communication Protocol Between Humans and Bank Server Secure 507
Learning phase times
Attack phase
Fig. 1. SUB-MIM game
Theorem. If CAPTCHA scheme Capt has IND-C-CCA security, the proposed pro-
tocol using Capt has SUB-MIM security.
To prove this theorem, we assume that there is an algorithm A that wins the SUB-
MIM game against the proposed protocol with non-negligible probability, and show
that there is an algorithm BA that wins the IND-C-CCA game with non-negligible
probability. BA indicates use of A by B and is constructed as shown in Fig. 3. From the
definition of IND-C-CCA game, BA can use human oracle H. For a queried CAPTCHA
problem c, H outputs the solution m ¼ C DecðcÞ. In (1), B runs the protocol by itself
to obtain p. B simulates SH ð:Þ by using its human oracle n times. The environment of A
simulated by B is equal to that of A in the SUB-MIM game. The win condition in the
SUB-MIM game of A is to output rb as Q0 in (12) corresponding to b selected by the
challenger in (6) out of r0 ; r1 selected by B in (4). When A wins the SUB-MIM game, B
can know the b selected by the challenger and send it to the challenger as ^ b in (14) to
win the IND-C-CCA game. Therefore, it is evident that B wins the IND-C-CCA game
if A wins the SUB-MIM game.
Next, we describe the conditions under which B wins the IND-C-CCA game when
A loses the SUB-MIM game. There are two cases where A loses the SUB-MIM game;
the case where Q0 is not rb but r1b , and the case where Q0 is neither rb nor r1b . If Q0 is
not rb but r1b , B cannot know this and has no choice but to select b b ¼ 1 b in
508 K. Mukaihira et al.
(13) and as a result, it loses to IND-C-CCA game. On the other hand, if Q0 is neither rb
nor r1b , then B, knowing r0 and r1 , can know that Q0 is an incorrect value. Therefore,
B chooses b b randomly by coin toss and outputs it. Then, B can win the IND-C-CCA
game with a probability of 1/2. Thus, if A loses the SUB-MIM game, B wins the IND-
C-CCA game if Q0 6¼ r1b and B wins the coin toss.
Let the probability that A wins the SUB-MIM game be eA , and the probability that B
wins the IND-C-CCA game be Pr½B wins.
Fig. 2. Proposed protocol
Pr½B wins ¼ Pr½A wins
þ Pr ½A loses ^ Q0 6¼ r1b ^ B wins by coin toss
¼ eA þ Pr ½A loses Pr½Q0 6¼ r1b jA loses
Pr ½B wins by coin toss jA loses ^ Q0 6¼ r1b
jRj 2 1
¼ eA þ ð1 eA Þ
jRj 1 2
jRj jRj 2 1
¼ eA þ
2ðjRj 1Þ jRj 1 2
1 jRj 1
AdvINDCCCA
¼ Pr ½B wins ¼ eA
B
2 2ðjRj 1Þ 2ðjRj 1Þ
Therefore, if the domain of random numbers jRj is large enough and the advantage
of A in SUB-MIM game eA ¼ AdvSUBMIM
A is not negligible, the advantage of B in the
INDCCCA
IND-C-CCA game AdvB is also not negligible. Thus, the contraposition is
true, and the theorem is proved.
Communication Protocol Between Humans and Bank Server Secure 509
Fig. 3. Construction of BA
5 Conclusion
This study proposed an H2M secure communication protocol that is capable of keeping
security against transaction-tampering type MITB attack. It was constructed based on
the concept of CAPTCHA as an example of a problem that is easy for humans to solve
correctly, but difficult for machines. Moreover, CAPTCHA was formulated as a
cryptographic model and its security notion (IND-C-CCA) was defined. In addition,
SUB-MIM security was defined as security against transaction-tampering MITB
attacks. Our money-transfer request protocol has SUB-MIM security if the CAPTCHA
scheme used in the protocol has IND-C-CCA security.
References
1. Suzuki, M., Nakagawa, Y., Kobara, K.: Assessing the safety of “Transaction Authentication”
against Man-in-the-Browser attacks in Internet banking. Financ. Res. 32(3), 51–76 (2013)
2. Weigold, T., Kramp, T., Hermann, R., Höring, F., Buhler, P., Baentsch, M.: The Zurich
Trusted Information Channel - an efficient defence against man-in-the-middle and malicious
software attacks. In: Trusted Computing-Challenges and Applications, pp. 75–91 (2008)
3. Negi, T., Mori, T., Hirano, T., Koseki, Y., Matsuda, N., Kawaguchi, K., Yoneda, T.:
Approach for the method of transaction signing utilizing smartphone with Secure SIM, vol.
2015-CSEC-71, no. 11, pp. 1–8 (2015)
4. The Official CAPTCHA Site. http://www.captcha.net. Accessed 07 Sept 2019
5. Kumarasubramanian, A., Ostrovsky, R., Pandey, O., Wadia, A.: Cryptography using captcha
puzzles. In: Public-Key Cryptography, pp. 89–106. Springer (2013)
Development of a Solution Model
for Timetabling Problems Through a Binary
Integer Linear Programming Approach
Juan Manuel Maldonado-Matute1,2,3,4(&),
María José González Calle2,3,4, and Rosana María Celi Costa2,3
1
Escuela de Ingeniería de la Producción y Operaciones,
Universidad del Azuay, Cuenca, Ecuador
2
Escuela de Administración de Empresas,
Universidad del Azuay, Cuenca, Ecuador
{jmaldonado,mgonzalez,rosanacc}@uazuay.edu.ec
3
Observatorio Empresarial, Universidad del Azuay, Cuenca, Ecuador
4
Departamento de Posgrados, Universidad del Azuay, Cuenca, Ecuador
Abstract. A common situation when each academic period starts in academic
institutions is the challenge of assign classrooms for the different courses in each
faculty. This article shows an integer binary linear programming model (ob-
jective function, constraints and the relationship between decision variables)
which, on a daily basis, performs a classroom assignment based on the hourly
scheduling and the capacity of each classroom; the model considers two
assumptions: (1) the assignment of each subject in a certain time period is
already carried out optimally, and (2) the availability of teachers is not a
restrictive element of the model. The test and analysis of the proposed model
was carried out at the Faculty of Science and Technology of the University of
Azuay (Cuenca, Ecuador). The result of the proposed model, which can be
solved with any linear programming algorithm, allows an efficient assignment of
the different classrooms, considering the constraints mentioned above. It is
considered that under the proposed approach the model can be extended con-
sidering additional aspects and conditions, however this would increase the
complexity which can be managed through a phased approach (fragmented
problems).
Keywords: Timetabling problem Linear programming Class scheduling
1 Introduction
Programming schedules and assigning classrooms at the beginning of each academic
period is a problem faced by the great majority of educational institutions due to the
series of factors that must be considered to successfully complete this task. This activity
represents an important administrative task because it consumes significant amount of
resources, especially time and staff. From the point of view of Operations Research,
these types of problems are within the field known as Timetabling Scheduling Prob-
lems. Timetabling Scheduling Problems describe a situation that usually happens in the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 510–516, 2020.
https://doi.org/10.1007/978-3-030-39512-4_80
Development of a Solution Model for Timetabling Problems 511
educational environment, where a series of resources, usually subjects and teachers,
must be assigned to a limited number of time periods and classrooms at the same time
[1], so that there are no inconsistencies in the distribution. Anthony Wren defines
timetabling problems as those that allow the allocation, subject to constraints, of
resources to objects that are placed in the space-time, so that they satisfy as much as
possible a set of desirable goals. Examples are the classification of class and exam
hours and some forms of staff allocation, for example, the assignment of toll posts
subject to a specific number of staffs [2].
The biggest difficulty in solving these types of problems is that the constraints and
variables involved can be counted by hundreds or thousands due to the amount of
resources that should be considered; due to this finding a suitable solution for the
problem can become an extremely difficult task. There is a big variety of versions to
solve timetabling problems, each one for a specific reality, so it is not possible to speak
of a generalized model applicable to any circumstance or institution. An important
variety of studies have been developed from the point of view of operations research
and computer science in the area of time scheduling each with very different approa-
ches, but with an important amount of similarities.
2 Timetabling Problems
The manual solution of timetabling problems usually requires several staff and days of
work, and they become considerably more complex as the factors involved in the
calculation grow (teachers, time periods, capacity, classrooms, laboratories, etc.), and
there is also the possibility that the solution obtained is not 100% satisfactory. For these
reasons, considerable attention has been presented to the development of models that
allow the automation of class scheduling problems for some years. The first docu-
mented and elaborated approach in important detail was that carried out by Gotlieb in
1963 [3] and has served as a starting point for a series of articles and applications that
have achieved considerable success.
Despite the series of variants that have been presented by several authors for the
problem, which differ depending on the type of event and the type of constraints, the
problem can be classified into three categories [4], programming schedules for exams
and tests (Examination Timetabling), programming class schedules for schools (School
Timetabling), and programming of class schedules for universities (University Course
Timetabling). Although this classification exists it could not be considered strict
because on several occasions a specific problem can overlap more than one category.
In most cases, the problem is treated as an optimization problem where it is about
maximizing or minimizing an objective function subject to a group of requirements
called hard and soft constraints. Hard constraints are those that should never be violated
and are generally not expressed as part of the objective function but as restrictive
functions related to the objective function; the soft constraints, on the other hand, are
not mandatory and are not as important as the previous ones, these are included as part
of the objective function and are fulfilled only when all the hard constraints have been
fulfilled. Hard constraints for the problem of programming class schedules for
512 J. M. Maldonado-Matute et al.
universities usually are regulated by the following rules [5]: (1) Collisions are not
allowed: In a timetabling problem a collision occurs when two or more classes are
scheduled in the same time period and with the same teacher or in the same classroom.
(2) The schedule must be completed: A schedule is considered complete when all
classes have been assigned to their respective time period and with a respective teacher,
in these cases the assignment of a classroom is not always considered. (3) The
schedules must allow programming of non-consecutive subjects: In many cases certain
subjects are not taught at one time over a week but are distributed throughout the
available academic days, it implies the possibility of schedule the same subject in
different time periods along the week. (4) Consecutive class periods should be con-
sidered: It should be considered that in many cases certain subjects can be taught in
consecutive time periods without this meaning a change of classroom or teacher.
Similarly, soft constraints for the proposed problem are regulated by the following
desired conditions: (1) Teachers preferences for specific time periods should be met if
possible: Teachers can express an opinion about their preferences for teaching classes
in certain time period. (2) The schedules for the students should be as compact as
possible: Care should be taken to compact the schedules so as to avoid the dispersion of
classes throughout a school day. (3) The change of classrooms should be avoided as
much as possible: For convenience, the change of classrooms should be avoided
specially to avoid the clutter and agglomeration of students in the subject changes.
In 2003 the Metaheuristics Network organized an international competition for the
problem called University Course Timetabling Problem [6], the objective of this
contest was to establish a model of general acceptance by the research community. The
problem consists of a set of n events to be scheduled in a set of 45 time periods (9 time
periods per day), and a set of j classrooms in which each event is carried out. In
addition, there is a set of S students who attend the events, and a set of F characteristics
given by the classrooms and required by the events. A feasible schedule is one in which
all events have been assigned to a time period and a classroom, so that the following
constraints are met: (1) No student attends more than one event at time; (2) the
classroom is large enough for all attending students and satisfies all the features
required by the event; (3) there is only one event in each room at any time interval. The
phased approach used in, the contest winning algorithm allows reducing the magnitude
of the problem by dividing it into subproblems of less complexity, these subproblems
can be handled individually, and the results of a given phase serve as inputs from
another [1].
3 Problem Description and Solution Method
The problem to be solved emulates the situation that occurs in the Faculty of Science
and Technology of the University of Azuay formed by 7 schools, where a set of
subjects and students must be distributed in classrooms throughout the academic week
taking into account three main considerations, (1) not to exceed the capacity of each
classroom, (2) do not use more classrooms than the Faculty owns, and (3) the schedule
must be completed for each school; it is also considered that (1) the assignment of each
subject in a certain time period is already carried out optimally, and (2) the availability
Development of a Solution Model for Timetabling Problems 513
of teachers is not a restrictive element of the model. For the generation of the model, a
phased approach will be followed, concentrating on the constraints proposed, together
with an approach similar to that proposed by [7] and [8] where the entire linear
programming is used to construct a solution model.
3.1 General Characteristic of the Model
The following sets are considered for the structural basis of the proposed approach:
• Days of the week, in which one or several subjects can be scheduled respecting the
established time periods, this set will be denoted as D ¼ fd1 ; . . .; d5 g, based on a 5-day
academic week.
• Scheduled subjects, considering their respective number of students is de-noted by
C ¼ fc1 ; . . .; cm g, considering that m subjects, usually between 100 and 110, can be
offered in a regular semester in the Faculty.
• Time periods, will be denoted by F ¼ ff1 ; . . .; fn g, a time period refers to the time
interval during which a subject is taught, it is considered the existence of n time
periods on a regular class day.
• Classrooms, with its respective capacity, which will be available for the scheduling
of a subject in a certain time period, the existence of p classrooms will be con-
sidered. The set of classrooms will be denoted by A ¼ a1 ; . . .; ap .
It is considered a solution approach per day, fragmenting the weekly problem into 5
smaller ones. In this model a single type of variable denoted by xc;f ;a is considered;
where c 2 C, f 2 F and a 2 A, this variable takes the value of 1 when the c subject,
programmed in a f period is assigned to a classroom, otherwise it takes the value of 0.
There is also a set of parameters that represent the conditions that must be respected.
• Mjcj 2 N: vector that indicates the number of students in each subject. Where M |c|
indicates the number of students in the c 2 C subject.
• Njf j 2 N: vector that indicates the number of subjects assigned to certain time
period. Where N|f| indicates the number of materials in the f 2 F period.
• Pjaj 2 N: vector that indicates the capacity of each classroom. Where Pjaj indicates
the number of seats available in the a 2 A classroom.
• CFjCjjFj 2 f0; 1g: subject-period allocation matrix, where CFc;f ¼ 1 if a c 2 C
subject has to be scheduled in a f 2 F period, otherwise 0.
• CAjCjjAj 2 f0; 1g: subject-classroom allocation matrix, where CAc;a ¼ 1 if the a 2 A
classroom has the capacity (possibility) to receive all the students enrolled in the c 2 C
subject, otherwise 0.
• CUjCjjFjjAj 2f0; 1g: subject-period-classroom availability matrix, where
CUc;f ;a ¼ 1, if a c 2 C subject has the possibility of being scheduled in a f 2 F
period and assigned to a a 2 A classroom, otherwise 0.
3.2 Linear Programming Model
The set of constraints necessary for the problem and its description are listed below,
Eqs. (1), (2), (3), (4) and (5) respectively represent the mentioned constraints.
514 J. M. Maldonado-Matute et al.
1. Only the number of scheduled subjects must be assigned to a time period.
2. The classrooms capacity must be respected, that is, the number of students enrolled
in a subject cannot exceed the capacity of the assigned classroom.
3. Subject scheduled in consecutive periods must stay in the same classroom.
4. A course scheduled in a time period must be located in a single classroom.
5. Classrooms can host a maximum of one subject in a certain time period.
Xm Xp
c¼1
x
a¼1 c;f ;a
Nj f j ; 8f 2 F , CFc;f ¼ 1 ^ CAc;a ¼ 1: ð1Þ
xc;f ;a CUc;f ;a ; 8c 2 c; f 2 F; a 2 A , Mjcj Pjaj : ð2Þ
Xp
x
a¼1 c;f ;a
xc;f 1;a ¼ 0; 8c 2 c; f 2 F , CFc;f ¼ CFc;f 1 ¼ CAc;a ¼ 1: ð3Þ
Xp
x
a¼1 c;f ;a
1; 8c 2 c; f 2 F , CFc;f ¼ 1 ^ CAc;a ¼ 1: ð4Þ
Xm
x
c¼1 c;f ;a
1; 8a 2 A; f 2 F , CFc;f ¼ 1 ^ CAc;a ¼ 1: ð5Þ
The model is completed with the objective function shown in (6), the coefficient
that accompanies xc;f ;a defined by Mjcj =Pjaj allows an improvement in the allocation of
space within the classrooms, since the percentage of occupation is taken into account
based on the number of students per subject assigned. In this way, the problem raised
becomes an exercise of maximization where the greatest efficiency in the occupation of
all classrooms is sought, provided that the conditions already exposed are met.
Xm Xn Xp
max c¼1 f ¼1 a¼1
Mjcj =Pjaj xc;f ;a , CFc;f ¼ 1 ^ CAc;a ¼ 1: ð6Þ
4 Discussion and Results
The model was tested in the Faculty of Science and Technology of the University of
Azuay; in a test scenario, which emulates a regular semester, 5 working days, 290
subjects, 14 time periods and 35 classrooms were considered; information on sched-
ules, teachers, students enrolled by subject and the capacity of each classroom was
provided by the institution in order to complete the formulation of the model. The great
majority of problems arise due to the lack of classrooms or due to their limited and
varied capacity (from 15 to 40 seats) which contrasts with the large number of students
enrolled in certain subjects every academic period, this problem, according to the
revised literature is one of the most common. The number of variables and constraints
generated per each day are shown in the Table 1.
The model was solved using free software and results were obtained in times less
than 5 min despite the number of variables and constraints, the results were verified
manually and physically to ensure the correct functioning of the model, verifying 100%
efficiency in the assignment. The analysis of the results together with the proposed
Development of a Solution Model for Timetabling Problems 515
Table 1. Number of variables and constraints handled in the tested model
Day Periods Variables Constraints
Monday 14 4367 2766
Tuesday 14 4332 2702
Wednesday 14 3565 2257
Thursday 14 3679 2400
Friday 14 2064 1381
model can be considered as the basis for the resolution of more complex problems that
may be set in a phases approach. The model will not provide an optimal solution when
there are not enough classrooms and the adequate capacity to host the number of
subjects to be scheduled in a certain time period; to solve this problem dummy
classrooms could be introduced into the model (dummies variables) that allow to find a
feasible solution and at the same time know the number of lack resources needed for a
successful scheduling.
5 Conclusions
In this work it has been possible to verify that a phased approach to solve timetabling
scheduling problems is practical and efficient, since it allows reducing the complexity
of a single problem that involves a large number of variables and constraints by
fragmenting it in subproblems of less complexity.
A similar approach to the one proposed in this research can be taken to solve
complementary problems such as the assignment of teachers to subjects or the
assignment of subjects to time periods under the same principle of sets, matrices and
vectors, where the results of one problem would be taken as input for the solution of a
following one, thus solving a macro problem of greater complexity that involves all the
factors and variables that are usually involved in the creation and assignment of class
schedules. It should be noted that this type of problem and its large number of
approaches, is still widely studied among the research community because it has not yet
been possible to generate a universal solution model, for this reason it is important to
continue generating new alternatives and solution approaches.
References
1. Kostuch, P.: The university course timetabling problem with a three-phase approach. In:
Burke, E., Trick, M. (eds.) Practice and Theory of Automated Timetabling V. PATAT 2004.
LNCS, vol. 3616, pp. 109–125. Springer, Heidelberg (2005)
2. Wren, A.: Scheduling, timetabling and rostering - a special relationship?. In: Burke, E., Ross,
P. (eds.) Practice and Theory of Automated Timetabling. PATAT 1995. LNCS, vol. 1153,
pp. 46–75. Springer, Heidelberg (1995)
3. Gotlieb, C.C.: The construction of class-teacher time-tables. In: IFIP Congress, North-
Holland, pp. 73–77 (1963)
516 J. M. Maldonado-Matute et al.
4. Schaerf, A.: Survey of automated timetabling. Artif. Intell. Rev. 13(2), 87–127 (1999)
5. Daskalaki, S., Birbas, T., Housos, E.: An integer programming formulation for a case study in
university timetabling. Eur. J. Oper. Res. 153(1), 117–135 (2004)
6. Rossi-Doria, O., et al.: A comparison of the performance of different metaheuristics on the
timetabling problem. In: Burke, E., De Causmaecker, P. (eds.) Practice and Theory of
Automated Timetabling IV. LNCS, vol. 2740, pp. 329–351. Springer, Heidelberg (2002)
7. Birbas, T., Daskalaki, S., Housos, E.: Timetabling for Greek high schools. J. Oper. Res. Soc.
48(12), 1191–1200 (1997)
8. Birbas, T., Daskalaki, S., Housos, E.: School timetabling for quality student and teacher
schedules. J. Sched. 12(2), 177–197 (2009)
Machine, Discourse and Power: From Machine
Learning in Construction of 3D Face to Art
and Creativity
Man Lai-man Tin(&)
School of Arts and Social Science, The Open University of Hong Kong,
Ho Man Tin, Kowloon, Hong Kong
[email protected]
Abstract. The 3D face reconstruction based on a single image can be done
recently by training the machine to learn from the database. The performance
and accuracy of 3D facial structure and the geometry have largely improved
with the powerful calculation of machine and its expanding data. The 3D face
reconstruction can be applied in both still images and moving images and
recently such technology draws attention in the art field such as portrait repre-
sentation, facial recognition, surveillance art, etc. By using artificial intelligence
(AI) technology and machine learning, artists are able to create artwork with
hybridity characteristic, which consists of both human perspective, and machine
creativity, which opens up a new gateway to the art world. The capability of
artists in creating artwork, particularly portrait art in this research, has been
enhanced in terms of technological and theoretical perspective. Artists can
represent faces and understand 3D geometry in new ways by breaking through
the traditional limitation; it also provides alternative methods in facial visual-
ization beyond the expectation. Traditional expression and technique can be
expended to a new dimension; the technology helps artists to think out of the
box and continue to create new movement and ism throughout the art history.
Moreover, the traditional belief about the importance of the originality of artists’
mind, sense, perspective and techniques in creating artwork can now be rede-
fined by machine and artificial intelligence. A new form in art and creativity, and
even knowledge and history has been developed. This paper attempts to discuss
the essential transformation in dimensional imagery and its aesthetic experiences
through machine learning means, and the shift of discourse of power from
human to machine.
Keywords: 3D face reconstruction Machine learning Artificial intelligence
Post-human aesthetics
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 517–523, 2020.
https://doi.org/10.1007/978-3-030-39512-4_81
518 M. L. Tin
1 Introduction
With the improvement of machine calculation and capabilities in data manipulation,
more and more artists are trying to explore the possibility of making artwork by
training machine and using artificial intelligence in recent years. One of the creative
applications is the 3D face reconstruction. 3D facial structure and geometry are difficult
to generate without providing enough corresponding data, and hence matching the
facial poses and generating accurate volumetric images are highly difficult [1]. Some of
the existing technologies have been used to reconstruct the 3D face, for example,
photogrammetry technology can generate geometric data and 3D facial model through
instantaneous multiple photos taken by cameras and software integration. Another
technology like infrared/structured light scanning can collect the surface data and
distance information and extracts the 3D data in the scanning field within its own
system. Such kind of human-centered operation and computer program system has
been shifting to machine-led approaches to create and present 3D images.
This paper will discuss technological change in portrait creation, the corresponding
machine learning technology, explore how the technology can reconstruct 3D face, and
mediate between human and machine to create an aesthetic experience of image and the
shift of power of discourse in creativity.
2 Portraiture Evolution
Portrait has been an artistic representation of humans in the history of art. The earliest
example can be found in the plastered human skulls produced around 9000 and 6000
BC. These sculptures demonstrate how people in the prehistoric time understand their
ancestors’ facial features and represent them in 3D by using plaster to model the human
skull and face. Since the dawn of different media, techniques, technologies and imaging
apparatus, the artistic representation of portraiture has been expanded and diversified.
For example, painting and photography have been widely used to replicate the physical
world including portrait in 2D visualization. One of the best-known portrait paintings
Mona Lisa by the Italian Renaissance artist Leonardo da Vinci denotes the high ability
and skill of human in painting and aesthetics. Camera allows us to capture the light and
shadow and reproduce portrait images, which give us inspiration to understand face
construction in a precise and detailed way. Computer graphics provide new possibil-
ities in image and data manipulation, and the generation of 2D images and 3D models.
These conventional and new imaging methods reveal the evolution of human repli-
cation of physical world and perception about 3 dimensions via 2D visualization.
Machine, Discourse and Power: From Machine Learning 519
In 2018, Edmond de Belamy, a portrait painting produced through Generative
Adversarial Network (GAN) was sold at Christie’s auction in New York for US
$432,000. This artwork created by artificial intelligence suggests an important direction
and new possibilities in imaging technique, new challenges to human in replication and
perception of the physical world, as well as the notion of art and creativity. The
application of machine intelligence in image production has increased rapidly. In the
near future, creation of artwork may be largely transformed and shifted from human-
centered to human-machine integration, or machine-led systems.
3 Machine Artificial and Image Generation
The GAN demonstrates the possibility for AI to create 2D portraiture, generation of 3D
model is the next hurdle. Compare with 2D, 3D imagery is a big challenge in terms of
the complexity of geometric and volumetric data generation, and corresponding model
building. Current systems rely on multiple images to create the 3D models, which
complicate the modeling process and increase in cost of image production.
Recently there are researches suggested that the limitations can be overcome by
machine learning through a Convolutional Neural Network (CNN) [2]. According to
the researches, CNN can generate 3D model from a single 2D image. Such method and
technology can address the problems and provide a simple but powerful way to pro-
duce 3D images. The AI technology and machine learning such as aforementioned
GAN and CNN could be new direction for portraiture creation.
In order to test the possibility and result of 3D portrait generated by machine
artificial, CNN and relevant database of 2D and 3D images as well as the open source
code have been used and adopted in this section. A photography (Figs. 1(A) and 2(A)
shows the portrait photography of a Hong Kong artist and singer Leon Lai) and
painting picture (Fig. 3(A) shows the portrait painting Mona Lisa) found from the
internet were used. The photos and picture of painting were sent to machine for training
and test the results of whether the machine can identify a face in a photograph and
picture and reconstruct the correspondent 3D facial geometry.
Figures 1, 2 and 3 show the results that 3D face can be reconstructed successfully
from a single image through CNN. By comparing Figs. 2(B) to 3(B) and 2(C) to 3(C),
it is noted that there are subtle changes in 3D geometric “landscape” of the same person
in different facial expression. This is good evidence and result that the machine is able
learn and generate different 3D faces based on its own interpretation of the picture and
the facial expression as long as CNN can identify a face in the picture. Figure 1(D)
shows different angles of the 3D face model indicating that CNN is able to visualize
parts that are invisible in the original 2D picture such as forehead and chin.
520 M. L. Tin
Fig. 1. (A) Portrait photography of Hong Kong artist and singer Leon Lai found from the
internet was used to do the testing. (B) A closeup of the 3D face generated from the photo Fig. 1
(A). (C)(D) Different angles of 3D face reconstruction generated through a Convolutional Neural
Network.
Machine, Discourse and Power: From Machine Learning 521
Fig. 2. (A) Another portrait photography of Leon Lai found from the internet was used to test
the 3D face reconstruction. (B) A closeup of the 3D face generated from the photo Fig. 2(A). (C)
Different angles of 3D face reconstruction generated through a Convolutional Neural Network.
Portrait painting Mona Lisa as shown in Fig. 3(A) is another attempt to test whether
CNN can identify a face in a painting and generate a 3D face. The result is positive and
different angles of Mona Lisa 3D portraiture were captured as shown in Fig. 3(B) and (C).
With the continuous development of the database of 2D images and 3D models for
training, the performance of CNN in facial identification and reconstruction will be
improved and the process will become smoother and the result will be more accurate.
This has greatly improved the imaging technique and possibility of making 3D
portraiture, as well as lower the cost in visualization in evolution [7], which has been a
drawback of some of the existing technologies such as 3D scanning - sophisticated and
precise set-up is required or low accuracy due to multiple scans.
522 M. L. Tin
Fig. 3. (A) Picture of Mona Lisa painting by Leonardo da Vinci. (B) A closeup of the 3D face
generated from the photo Fig. 3 (A). (C) Different angles of 3D face reconstruction generated
through a Convolutional Neural Network.
4 Post-human Aesthetics
To a certain extent, AI and machine learning open up a new gateway to the art world,
which allow artists to create portraiture with hybridity characteristic - machine “writ-
ing” with human “scripting” [4]. The autonomous art creation of machine artificial
changes the traditional practice and belief on the artist’s mind, sense, perspective and
techniques. In fact, machine artificial allows us to rethink the argument about “The
Death of the Author” by Roland Barthes (1915–1980) - is it possible for us to detect
precisely what the machine intended? Inevitably, I have shared the power of discourse
in art authorship and authority with machine intelligence in the 3D face reconstruction
as shown in Figs. 1 and 2. The interventions of “writing” and “scripting” are becoming
more complicated in which the values of aesthetics and art have to be redefined in post-
human era.
Machine, Discourse and Power: From Machine Learning 523
References
1. Moschoglou, S., Ploumpis, S., Nicolaou, M., Papaioannou, A., Zafeiriou, S.: 3DFaceGAN:
adversarial nets for 3D face representation, generation, and translation. arXiv:1905.00307v2
(2019)
2. Jackson, A.S., Bulat, A., Argyriou, V., Tzimiropoulos, G.: Large pose 3D face reconstruction
from a single image via direct volumetric CNN regression. In: ICCV (2017)
3. Dawkins, R.: The Selfish Gene. OUP Oxford, Oxford (2006). 30th Anniversary edition
4. Seymour, L.: Roland Barthes’s the Death of the Author. Macat Library (2018)
5. Fig. 1(A). Portrait photography of Hong Kong artist and singer Leon Lai [image]. https://
images.app.goo.gl/xEmsMTdrLNDTiZJ99. Accessed 14 Sept 2019
6. Fig. 2(A). Portrait photography of Hong Kong artist and singer Leon Lai [image]. https://
images.app.goo.gl/fbKVso1FiLcvptdY6. Accessed 12 Oct 2019
7. Fig. 3(A). Picture of painting Mona Lisa by Leonardo da Vinci. (1503) [image]. https://
commons.wikimedia.org/wiki/File:Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_
retouched.jpg#/media/File:Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.
jpg. Accessed 8 Oct 2019
Modelling Alzheimer’s People Brain Using
Augmented Reality for Medical Diagnosis
Analysis
Ramalakshmi Ramar1, Swashi Muthammal1(&),
Tamilselvi Dhamodharan2, and Gopi Krishnan Rajendran3
1
Department of Computer Science and Engineering, Kalasalingam Academy
of Research and Education, Krishnankovil 626126, Tamil Nadu, India
[email protected], [email protected]
2
Cognitive Science Research Group, Department of Information Technology,
Thiagarajar College of Engineering, Madurai 625 015, Tamil Nadu, India
3
ThoughtWhiz, Bangalore 560011, Karnataka, India
Abstract. Alzheimer’s Disease (AD) in adults is characterized by gradual
memory loss for old age people. Alzheimer’s brain have impact on affecting brain,
include loss of memory, difficulty in thinking, language and solving problems
symptoms. Heart issues, depression, diabetes and high blood pressure pose a
higher risk of causing Alzheimer’s disease. The Alzheimer’s disease in the brain
proteins build up to form structures known as plaques, which are the abnormal
clumps in the brain, and tangles which are the bundles of fibbers in the brain. This
leads to loss of brain tissue, nerve cells connections and lead to death of nerve
cells. In their brain there is a shortage of chemicals. These chemical messengers
around the brain help to transmit signals whose shortage causes the signals do not
transmit effectively. The above symptoms were visualized as AR (Augmented
Reality) model to assist doctors for medical analysis. Augmented Reality act as a
extremity tool for findings, supporting and analyse the Alzheimer’s Disease.
Understanding the AR brain model enhance the analysis of the brain with the
technical exploration of tracking, visualization technology layer by layer level,
integrated feedback about different parts of the brain function which plays a major
role in clinical evaluation to treat the Alzheimer’s disease. Microglia are a type of
cell that initiate immune responses in the brain and spinal cord. When AD is
present, microglia interpret the beta-amyloid plaque as cell injury. To reduce or
control the inflammatory response and brain shrinking can be visualized using
Augmented Reality.
Keywords: Brain Memory loss Alzheimer’s disease Augmented Reality
Visualization
1 Introduction
Augmented Reality is obtained from the concept of virtual reality in which a person
creates an artificial world to explore interactively through sense of vision, audio and
haptics. Although there are umber of definitions for AR universally accepted definition
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 524–531, 2020.
https://doi.org/10.1007/978-3-030-39512-4_82
Modelling Alzheimer’s People Brain Using AR for Medical Diagnosis Analysis 525
was proposed by Azuma et al. The paper titled A Survey of Augmented Reality
proposes that AR system combines both real and computer generated information in a
real environment, interactively and in real time [1, 2]. AR is characterized to be
potential to supplement the physical world [2]. The history of AR dates back to 1960s
when Ivan Sutherland built a mechanically tracked head mount device Sword of
Damocles. The term Augmented Reality was coined in the year 1990 by a Boeing
researcher Tom Caudell [3, 4]. Then came Virtual Fixtures (a robotic system that places
information on top the workers work environment to help with efficiency) in 1992 by
Louis Rosenburg. In 1992 NASA integrated AR in X-38 spacecraft. By the 2000 Nara
Institute of Technology released ARToolKit software. By the year 2014 Googleglass
was invented. There are further more developments constantly taking place in AR field
till date. AR is further of two triggers marker-based and markerless augmentation.
Marker based is the concept where the digital information is anchored to a physical
world object using the parameters of camera calibration and 3D object interaction with
human hand [5] as in Anatomy 4D App where the virtual 3D human anatomical objects
pops up from the book instantly [6]. Markerless augmentation is when an object is
projected without any markers in picture cards and in application of magic mirror [7].
The higher technological requirements of augmented reality are displays which
encloses aural display, visual display under which comes the HMDs, handheld dis-
plays, spatial and pinch gloves displays [8, 9]. Tracking sensors and modelling envi-
ronments that are capable of sensing the outdoor and unprepared environment [10].
User interface and interaction as in holopanel interface for android [11]. Therefore AR
has marked its path in various fields of medicine, personal information system,
industrial and military applications for designing and assembling components, sports
broadcasting, gaming, tourism, teaching and learning aids [12].
As medicine is known as the art of science that deals with practice, diagnostics,
treatment, and prevention of the disease being caused. The diagnostics part is the
primary and most complicated area for medical practitioners. AR acts as a supportive
tool through which surgeon can see the hidden organs enclosed within the human body
that enhances the accuracy and treatment. The enhancement of accuracy is done
through the help of camera calibration, patient image registration [13]. AR provides an
improved surgical planning, navigation, simulators, educational tools, therapy and
rehabilitation. Visuo haptic augmented reality setup with OPTRAK 3020 and IR
tracking systems were incorporated for stabilized tracking and efficient alignment of
images [14]. This provides multimodality of AR in medicine by eliminating the pre-
viously used method called MAGI (Microscopic Assisted Guided Information) [15].
Regarding surgical planning Stereotactic surgery was performed with previously
obtained radiographic images that were superimposed on the specific sites of the
patients [16, 17]. There is a high possibility of AR examination being done remotely
using holo lens [18, 19]. The same holo lens is effective in manipulation of the images
using hand that finds its application in reconstruction surgery of bones due to the
detailed view of tissues and blood vessels [20]. The traditional laparoscopic medicine
has been replaced by a new technique called Natural Orifice Transluminal Endoscopic
Surgery (NOTES) in Minimally Invasive Surgery [21]. Haptics are used in automatic
registration, stiffness mapping, and dynamic image overlay that results in automated
526 R. Ramar et al.
tumor detection [22]. The same technique is used under dynamic texture mapping and
for performing surgery in digestive system, liver, eye etc., [23–25].
A brain is considered to be the repository of memory, sensation, and other intel-
lectual and cognitive activities. Human brain is quite flexible in nature when compared
to the brains of other mammals. This flexibility contributes to the perpetual learning
process and adaptable nature in humans. The memory framework of the brain system
are termed into declarative memory and non-declarative memory. Declarative memory
deals with recollection of events and facts, relies on connection with medial temporal
lobe and diencephalon which when damaged leads to amnesia. Non-declarative
memory is sub classified into skills and habits, priming, simple classical conditioning,
and non-associative learning that are acquired through learning experience [26]. It
involves the coordination of cerebellum, basal ganglia, motor cortex and hippocampus
under the supervision of cerebral cortex [27]. Brain lesions are considered to be the
major cause for basic and complex memory dysfunction. Cortical lesions leads to
degeneration of memory for a short term and trouble in retrieval of sematic memory
[28]. Planning ability is affected by frontal cortical dysfunction. Majority of amnestic
diseases are due to diencephalic lesions [29]. Dementia is a dreadful neurodegenerative
disease that have been categorized into 4 types, Alzheimer’s disease, Vascular
dementia, Front temporal dementia and Dementia with Lewy Bodies based on their
cause and effect [30]. Some of these are reversible based on the of resistance, resilience
and compensation parameters exhibited by the patients [31]. Reversible dementia is due
to geriatric depression [32]. The changes in the physiology of the brain and it’s
chemical composition leads to irreversible dementia. There is no permanent cure for
irreversible dementia there are medications including cholinesterase inhibitors,
memantine [33, 34]. Apart from medication rehabilitation for patient’s cognitive and
motor skill developments are provided [35, 36].
Alzheimer’s disease is the most common type of dementia according to the paper
published in 2014 statistically proves that almost 4.4 million of the world’s population
>65 is estimated to be affected and there is also a risk of growth in number to 65 million
by 2030. The term was coined after the German neurologist Alois Alzheimer’s in the
year 1907. Alzheimer’s disease is characterized by the progressive deterioration of the
cognition, motor skills and memory. The prior cause of Alzheimer’s disease is amyloid
plaque formation due to the left over amyloid beta proteins after Amyloid Precursor
Protein (APP) cleavage. The microglia (Interleukin-1 (ILl)) and astrocytes responsible
for formation of amyloid beta proteins formation stagnates leads to inflammation.
These lesions affect the neurons and pyramidal cells initially the in hippocampus region
that is identified using proteomics identification, and gradually spreads to the cortical
region and then to the whole of the brain. In 1984 Glenner and Wong proposed that
Alzheimer’s can be due to the genes of apolipoprotein E APOE (APOE e2, APOE e4,
APOE e3). The genes are responsible for late onset of Alzheimer’s. The early onset is
due to the mutation of Presenilin 1 (PSEN1), Presenilin 2 (PSEN2).
The proposed AR (Augmented Reality) Brain model of AD that includes neurons,
astrocytes, microglias and peripheral macrophages, as well as amyloid b aggregation
and hyper phosphorylated tau proteins. The model is represented by a system of partial
differential equations. The model is used to simulate the effect of drugs that either failed
in clinical trials, or are currently in clinical trials.
Modelling Alzheimer’s People Brain Using AR for Medical Diagnosis Analysis 527
2 Proposed Augmented Reality Brain Model
The physiological change in the Alzheimer’s brain is the result of chemical imbalance
in the brain. The major causes are beta-amyloid plaques and neurofibrillary tangles.
Beta-amyloid proteins are the small fragments produced by APP protein cleavage. The
amyloid proteins usually breaks drown but in a brain prone to Alzheimer’s these sticky
proteins groups into clusters. The plaques around the neurons causes death of the
neurons. Neurofibrillary tangles are insoluble fibres that forms tau protein. Tau protein
is formed inside the neurons that interferes with the cellular mechanism and leads to
cell death. This paper focusses on the chemical and physiological changes in the brain.
The major physiological change is the brain shrinkage. A normal human brain
weighs 3 lb (1300 gms–1400 gms), Alzheimer’s brain is 10% less than that of the
normal brain. This is due to the death of brain cells. The decline of the cells starts from
the hippocampus and spreads to the cerebral cortex. The ventricle chambers that
contains the cerebrospinal fluid on either sides are found to be enlarged. The con-
nectivity of neurons gets disconnected that leads to the loss of memory, speech and
motor skills. The sulcus and gyrus region develops enlarged gaps. Figure 1 shows the
proposed model of the AR based modelling and analysis done for the Alzheimer’s
brain.
Fig. 1. Proposed model and analysis in Alzheimer’s brain
Figure 1 shows the block diagram view of the physiological transformations. It will
be modelled using Maya tool. Model of a healthy human brain and an Alzheimer’s
brain will be modelled for overall structural comparison. Then each interior part will be
designed to show the minute changes. All these models are imported to the unity tool
where the 3D models are augmented to get a clear and detailed view. The dissection
and slicing of the model will be enabled for improved analysis of the damages to the
interior lobes. An audio track will be playing in the background to explain the changes.
528 R. Ramar et al.
The addition of the audio clip makes it more interactive. This can used as a learning
tool or for research purpose.
Figure 2a represents the schematic diagram of the normal brain and it’s internal
neural structure. The figure exhibits the dendrites and axon are linked. The linkage
helps in the proper transmission of information throughout the human body. Tau
proteins and microtubules are stabilized. The mild Alzheimer’s brain shows slight
structural variation in the hippocampus region. Figure 2b shows the severely affected
brain. The internal neural structure is distorted in shape. This distortion leads to error in
the smooth transmission of neural signals. The microtubules appear to be irregular and
then the amyloid beta proteins are stagnated form in plaques. The cortical shrinkage
with enlarged ventricles are the other structural variations.
Fig. 2. a. Healthy brain neuron network and mild Alzheimer’s brain; b. Severe Alzheimer’s
brain
The AR brain model is represented by a system of partial differential equations
(PDEs) based on Fig. 2. It represents all the pro inflammatory cytokines by TNF - a,
and all the anti-inflammatory cytokines by IL-10. The model is used to conduct in silica
trials with several drugs: TNF- a inhibitor, anti-A b drug, MCP-1 inhibitor, and
injection of TGF - b. Simulations of the model show that continuous treatment with
TNF - a inhibitor yields a slight decrease the death of neurons, and anti-A b drug yields
a slight decrease in the aggregation of A b over 10 years period, while the benefits from
injection of TGF - b and MCP-1 inhibitor drugs are negligible. This suggests that
clinical trials consider combination therapy with TNF - a and anti-A b drugs. The
above information present in the brain are graphically projected in the augmented
reality model so that the analysis, research perspectives were visually identified by
doctors to continue the advanced research in the AD.
Modelling Alzheimer’s People Brain Using AR for Medical Diagnosis Analysis 529
3 Conclusion and Future Works
We conclude the earlier mathematical models which deal with some aspects of A. The
model we developed A b polymerization, A b plaque formation and the role of prions
interacting with A b, linear cross-talk among brain cells and A b, and the influence of
SORLA on AD progression will support for Alzheimer research scientist to explore the
causes of the issue and the ways to narrow down the research with Alzamier’s brain
model. In future the interior visualization and differentiate the diseased neuron and the
behavior to reflect in the human behavior and the memory loss will be analyzed.
References
1. Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4),
355–385 (1997)
2. Höllerer, T.: User interfaces for mobile augmented reality systems, Doctoral dissertation,
Columbia University (2004)
3. Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., Ivkovic, M.: Augmented
reality technologies, systems and applications. Multimed. Tools Appl. 51(1), 341–477
(2011)
4. Blanco-Fernández, Y., López-Nores, M., Pazos-Arias, J.J., Gil-Solla, A., Ramos-Cabrer, M.,
García-Duque, J.: REENACT: a step forward in immersive learning about human history by
augmented reality, role playing and social networking. Expert Syst. Appl. 41(10), 4811–
4828, 213 (2014)
5. Chun, J., Lee, S.: A vision-based 3D hand interaction for marker-based AR. Int. J. Multimed.
Ubiquit. Eng. 7(3), 51–58 (2012)
6. Hsieh, M.C., Lee, J.J.: Preliminary study of VR and AR applications in medical and
healthcare education. J. Nurs. Health Stud. 3(1), 1 (2018)
7. Glockner, H., Jannek, K., Mahn, J., Theis, B.: Augmented reality in logistics: changing the
way we see logistics–a DHL perspective. DHL: Customer Solutions & Innovation 28 (2014)
8. Kesim, M., Ozarslan, Y.: Augmented reality in education: current technologies and the
potential for education. Procedia-Soc. Behav. Sci. 47, 297–302 (2012)
9. Sielhorst, T., Feuerstein, M., Navab, N.: Advanced medical displays: a literature review of
augmented reality. J. Disp. Technol. 4(4), 451–467 (2008)
10. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances
in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001)
11. Brotos, A.: Interactive augmented reality panel interface for Android (2015)
12. Van Krevelen, D., Poelman, R.: Augmented reality: technologies, applications, and
limitations, Dep. Comput. Sci. Vrije Univ., Amsterdam (2007)
13. Ha, H.G., Hong, J.: Augmented reality in medicine. Hanyang Med. Rev. 36(4), 242–247
(2016)
14. Harders, M., Bianchi, G., Knoerlein, B.: Multimodal augmented reality in medicine. In:
International Conference on Universal Access in Human-Computer Interaction, pp. 652–
658. Springer, Heidelberg (2007)
530 R. Ramar et al.
15. Edwards, P.J., King Jr., A.P., Maurer, C.R., de Cunha, D.A., Hawkes, D.J., Hill, D.L.G.,
Gaston, R.P., Fenlon, M.R., Chandra, S., Strong, A.J., Chandler, C.L., Richards, A.,
Gleeson, M.E.: Design and evaluation of a system for microscope assisted guided
interventions (MAGI). In: Taylor, C., Colchester, A. (eds.) MICCAI 1999: Proceedings of
the Second International Conference on Medical Image Computing and Computer-Assisted
Intervention. LNCS, vol. 1679, pp. 842–851. Springer, Heidelberg (1999)
16. Parkhomenko, E., Safiullah, S., Walia, S., Owyong, W., Lin, C., O’Leary., et al.: MP26-20
virtual-reality projected renal models with urolithiasis as an educational and preoperative
planning tool for nephrolithotomy: a pilot study. Am. Urol. Assoc. Conf. 199(45), e345
(2018)
17. Alberti, O., Dorward, N.L., Kitchen, N.D., Thomas, D.G.: Neuronavigation-the impact of
operating time. Stereotact. Funct. Neurosurg. 1997(68), 44–48 (1997)
18. Hanna, M.G., Ahmed, I., Nine, J., Prajapati, S., Pantanowitz, L.: Augmented reality
technology using Microsoft HoloLens in anatomic pathology. Arch. Pathol. Lab. Med. 142
(5), 638–644 (2018)
19. Monsky, W.L., James, R., Seslar, S.S.: Virtual and augmented reality applications in
medicine and surgery-the fantastic voyage is here. Anat. Physiol. 9(1), 1–6 (2019)
20. Pratt, P., Ives, M., Lawton, G., Simmons, J., Radev, N., Spyropoulou, L., Amiras, D.:
Through the HoloLens™ looking glass: augmented reality for extremity reconstruction
surgery using 3D vascular models with perforating vessels. Eur. Radiol. Exp. 2(1), 2 (2018)
21. De Paolis, L.T., Aloisio, G.: Augmented reality in minimally invasive surgery. In: Advances
in Biomedical Sensing, Measurements, Instrumentation and Systems, pp. 305–320. Springer,
Heidelberg (2010)
22. Zevallos, N., Srivatsan, R.A., Salman, H., Li, L., Qian, J., Saxena, S., Xu, M., Patath, K.,
Choset, H.: A surgical system for automatic registration, stiffness mapping and dynamic
image overlay. In: 2018 International Symposium on Medical Robotics (ISMR), pp. 1–6.
IEEE, March 2018 (2018)
23. Patath, K., Srivatsan, R.A., Zevallos, N., Choset, H.: Dynamic texture mapping of 3D
models for stiffness map visualization. In: Workshop on Medical Imaging, IEEE/RSJ
International Conference on Intelligent Robots and Systems (2017)
24. Soler, L., Nicolau, S., Schmid, J., Koehl, C., Marescaux, J., Pennec, X., Ayache, N.: Virtual
reality and augmented reality in digestive surgery. In: Third IEEE and ACM International
Symposium on Mixed and Augmented Reality, pp. 278–279. IEEE, November 2004
25. Kalkofen, D., Reitinger, B., Risholm, P., Bornik, A., Beichel, R., Schmalstieg, D., Samset,
E.: Integrated medical workflow for augmented reality applications. In: International
Conference on Medical Image Computing and Computer Assisted Intervention (2006)
26. Squire, L.R.: Declarative and nondeclarative memory: multiple brain systems supporting
learning and memory. J. Cogn. Neurosci. 4(3), 232–243 (1992)
27. Dharani, K.: The Biology of Thought: A Neuronal Mechanism in the Generation of
Thought-A New Molecular Model, pp. 53–74. Academic Press, Cambridge (2014)
28. Morris, R.G.: Dementia and the functioning of the articulatory loop system. Cogn.
Neuropsychol. 1(2), 143–157 (1984)
29. Mayes, A.R.: Learning and memory disorders and their assessment. Neuropsychologia 24
(1), 25–39 (1986)
30. Chiu, M.-J., Chen, T.-F., Yip, P.-K., Hua, M.-S., Tang, L.-Y.: Behavioral and psychologic
symptoms in different types of dementia. J. Formos. Med. Assoc. 105(7), 556–562 (2006)
31. Montine, T.J., Cholerton, B.A., Corrada, M.M., Edland, S.D., Flanagan, M.E., Hemmy, L.S.,
White, L.R.: Concepts for brain aging: resistance, resilience, reserve, and compensation.
Alzheimers Res. Ther. 11(1), 22 (2019)
Modelling Alzheimer’s People Brain Using AR for Medical Diagnosis Analysis 531
32. Alexopoulos, G.S., Meyers, B.S., Young, R.C., Mattis, S., Kakuma, T.: The course of
geriatric depression with “reversible dementia”: a controlled study. Am. J. Psychiatry 150,
1693 (1993)
33. Arlt, S., Lindner, R., Rösler, A., von Renteln-Kruse, W.: Adherence to medication in patients
with dementia. Drugs Aging 25(12), 1033–1047 (2008)
34. Raina, P., Santaguida, P., Ismaila, A., Patterson, C., Cowan, D., Levine, M., Booker, L.,
Oremus, M.: Effectiveness of cholinesterase inhibitors and memantine for treating dementia:
evidence review for a clinical practice guideline. Ann. Intern. Med. 148(5), 379–397 (2008)
35. Blattgerste, J., Renner, P., Pfeiffer, T.: Augmented reality action assistance and learning for
cognitively impaired people: a systematic literature review. In: PETRA, pp. 270–279, June
2019
36. Viglialoro, R.M., Condino, S., Turini, G., Carbone, M., Ferrari, V., Gesi, M.: Review of the
augmented reality systems for shoulder rehabilitation. Information 10(5), 154 (2019)
Software Vulnerability Mining Based
on the Human-Computer Coordination
Jie Liu1,2(&), Da He1, Yifan Wang1, Jianfeng Chen1,2,
and Zhihong Rao1,2
1
China Electronic Technology Cyber Security Co., Ltd.,
Chengdu 610041, China
[email protected], [email protected],
[email protected], [email protected]
2
Cyberspace Security Key Laboratory of Sichuan Province,
Chengdu 610041, China
Abstract. In recent years, the increasing size and complexity of software
packages has led to vulnerability mining gradually becoming more difficult and
challenging. The theoretic research and systematic practice of traditional soft-
ware vulnerability mining system emphasize models and data. Now, it gradually
requires the participation of human vulnerability-miners in the mining proce-
dure. To address the issue, from the human-center perspective, this paper holds
that the role of human should be highlighted in the system building process.
Aimed at solving the task of software vulnerability mining and integrating the
natural intelligence of human into the system, it attempts to assign people who
participate in the vulnerability-mining activities as the components of the sys-
tem. This paper proposes a vulnerability mining system architecture based on
human-computer coordination. Then, it designs the workflow of the vulnera-
bility mining task based on human-computer coordination. Finally, it designs a
task solving strategy based on human-computer coordination for the fuzz testing
scenario.
Keywords: Human-computer coordination Fuzz testing Software
vulnerability mining Workflow
1 Introduction
In the new information environment, the software scale is getting bigger and bigger, the
software complexity is getting higher and higher, and the software-related security
issues are affecting individuals, organizations and society. As the core of software
security issues, software vulnerability mining is getting more and more attention from
researchers [1]. At the same time, the problem of software vulnerability mining is
becoming more and more complicated. The contradiction between the limitations of
traditional vulnerability mining methods and the efficiency requirements of vulnera-
bility mining is increasingly prominent [2]. In particular, the traditional vulnerability
mining technology often appears to be “incapable” in the situation that the human
vulnerability miners are gradually becoming more involved. This has led us to deeply
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 532–538, 2020.
https://doi.org/10.1007/978-3-030-39512-4_83
Software Vulnerability Mining Based on the Human-Computer Coordination 533
think about traditional vulnerability mining methods and system performance. This also
poses new challenges to the development of traditional vulnerability mining
technology.
Since the early 1990s, many researchers and designers have begun to pay attention
to the relationship between “human and systems” and carried out a series of research on
human-computer interaction systems for problem solving [3]. However, with the
continuous in-depth study of related theory and system practice, many researchers have
a consensus that the human and computer should be coordinated to enable enough
participation of human to improve system efficiency. The researchers initially explored
the mechanism of human-computer integration and put forward many ideas and the-
ories about the relationship between humans and systems (such as “human-computer
intelligent system”, “human-computer integration”, “human-computer interaction”,
etc.) [4] [5]. At the same time, researchers have studied many conceptual forms and
generation techniques (such as “SoftMan”, “agent”, “artificial life”, etc.) [6].
Driven by software vulnerability mining task solving, this paper proposes the
concept of “human as a service” and reconstructs the architecture of software vul-
nerability mining system from technical and non-technical perspectives. According to
the idea and basic principle of human-computer coordination, the capability of human
vulnerability miners is “immersed” into the software vulnerability mining task solution
as the component of the system (human service). This paper designs the overall
architecture of software vulnerability mining system based on human-computer coor-
dination. Finally, the fuzz testing of software vulnerability is taken as a specific
application scenario, and the coordination strategy in the vulnerability mining process
is studied.
2 Human-Computer Coordination Mechanism
This paper establishes the human-computer coordination mechanism. This framework
implements flexible switching between human services and software services, while
shielding the differences between human services and software services in conducting
software vulnerability mining tasks. This mechanism is implemented by the combi-
nation of the service modeling layer, the service assembly layer, and the process
execution layer, as shown in Fig. 1.
The service modeling layer accepts the target input by the user. Then the dispatcher
of the service assembly layer obtains the software vulnerability mining workflow
description from the target receiver, parses and dispatches it. The dispatcher also
queries the corresponding service from the registration server, and generates the
abstract workflow. The workflow is then instantiated by the interpreter of the process
execution layer. After the instantiation, the abstract workflow is transformed into the
execution workflow, which is then passed to the solver for execution.
In the software vulnerability mining task solving process, human miner is effi-
ciently integrated into the workflow through the adapter. The programmable part forms
the human service, and the unprogrammable part forms the human-computer
534 J. Liu et al.
Service modeling layer Process excuƟon layer
Target receiver Interpreter
instanƟaƟon
service workflow
Abstract execuƟon workflow
Service assembly layer workflow
Call
RegistraƟon query Dispatcher Solver Services
Result
server results
service assembly Result Dispatch
Register& Service publish Task list manager Human
Publish miner
Interface
Human Human-computer
service interacƟve service
SoŌware
service ParƟal ParƟal
programmable
Unprogrammable Adapter
programmable
Unprogrammable
Fig. 1. Vulnerability mining mechanism based on human-computer coordination
interaction service. When a call to a human-computer interaction service reaches a
certain number of times (very frequently), it can be programmed (fixed) to form a
human service. At the same time, when a call to a human service reaches a certain
number of times (very frequently), it can be programmed (fixed) to form a software
service. These rules enable the transformation of service forms from human-computer
interaction services to human services, and from human services to software services.
If it is human service, the query of the registration server gains a specific human
service entity. Then the solver will use the task list manager to assign tasks to the
corresponding human service entity. The task list manager gets the corresponding
human service entity from the solver, including the unique identifier, associated task
instance and process instance identifier, status, executor identifier, priority, start time,
creation time, submission time, deadline and other attributes. The task list manager is
responsible for generating a task list of the corresponding executor according to the
human service entity object sent from the solver, and setting related information such as
priority, time reminder, etc. After the human service entity is loaded, it provides the
vulnerability mining task solving service according to its task list. After the task is
completed, the results are submitted to the solver, and the task list manager reports the
task completion status and is responsible for the archiving of the tasks.
If it is software service, the query result back to the registration server is the binding
information of the software service. At this point, the solver will generate corre-
sponding service information based on its binding information and input, including
unique identifiers, service types, associated task instances and process instance iden-
tifiers, status, binding information, input, output, QoS, etc. The solver then generates a
Software Vulnerability Mining Based on the Human-Computer Coordination 535
software service call client to call the appropriate software service, returning the result
and execution status to the solver after the call ends.
The dispatcher changes the state of this task instance based on the results of the
service (human service and software service) returned by the solver. After the dis-
patcher coordinates the tasks, all the participants of the process are treated as services in
the registration server for further query. The dispatcher completes the process through
state transitions between tasks. Changes in state are set based on the result returned by
solver.
The dispatcher passes the binding information and task instance information to the
solver, waiting for the service to complete asynchronously. Then, the task instance state
is changed according to the returned result and state, thereby completing the service
process. When an exception occurs, such as resource failure, timeout, etc., the dis-
patcher needs to re-query the corresponding service, then bind the new service, and
continue to execute according to the previous mode. According to the task require-
ments and the corresponding scheduling policy, the dispatcher determines the number
of rescheduling. If the number of times exceeds, the error handling mechanism is
triggered. Then the dispatcher would conduct a new execution plan.
3 Task Execution Process
Based on service-oriented architecture, the software vulnerability mining process is
componentized into a series of coarse-grained service and task processing procedures.
At some phase of the software vulnerability mining task, the working node A (subtask)
receives the task information and conducts its job with the associated service. The
result of the subtask is assumed to meet the target requirements.
Software service library Yes Human service library
Software Software HST Human Human
Software Human
service 1 service n service 1 ··· service n
service 2 service 2
No No
Whether the
Info. A B result meets the task B2
requirements
division B1 C
Yes
Info. 1 Info. 2 Info. n
Information base
Fig. 2. The task execution process
536 J. Liu et al.
Then, the relevant information is transmitted to the working node B (subtask).
B first judge whether the obtained software service and execution results can meet the
target requirements. If so, the process continues and turns to the working node C, and
so on. If not, B passes the Human-Service Trigger (HST) to determine whether the
subtask to be executed matches a human service. B can encounter two situations: (1) If
yes, it calls the human service resource in the service library. This subtask is executed
with human service or human-computer interactive service. Then the process contin-
ues. (2) If not, then B needs to be executed by a joint service of software service and
human service. The system decomposes task B into B1 (software task) and B2 (human
task). B2 gains the information (or knowledge) by triggering HST. Based on the joint
service of B1 (software service) and B2 (human service), this subtask should be done.
After the result of the execution meets the target requirement, the process is continued
and turns to working node C, as shown in Fig. 2.
4 Human-Computer Coordination for Fuzz Testing
Here, the fuzz testing method [7] of software vulnerability is taken as a specific
application scenario. And the vulnerability mining process needs to be divided into
several phases. And each phase contains several subtasks. The software fuzz testing
task based on human-computer coordination composed by software services, human
services, hybrid services, and direct human-computer interaction services. The detail of
the human-computer coordination strategy is as follows:
Step 1: Initialize the fuzz testing task FT.
Step 2: Fuzz testing task FT = {Ph1,Ph2,…,Phi}, Phi is each phase and i = 7.
Step 3: Start from i = 1, determine whether the subtask set included in Phi matches
the service in the software service library. If it can, store it in the buffer library and
go to Step 4. Otherwise go to Step 5.
Step 4: Determine if i = n is true, then go to Step 8. Otherwise i++, go to Step 3.
Step 5: Determine whether it can match the human service library. If it can, store it
in the buffer library, go to Step 4; otherwise, go to Step 6.
Step 6: Determine whether the joint service of the human service and software
service can match, then store it in the buffer library, go to Step 4; otherwise, go to
Step 7.
Step 7: Submit the subtask to the direct human-computer interaction service
interface. After the interface returns a confirmation message to the subtask, store it
in the buffer library and go to Step 4.
Step 8: The whole fuzz testing planning is finished. The results should be com-
prehensively sorted, and the human-computer coordination strategy is output. The
procedure of the human-machine coordination strategy is shown in Fig. 3.
Software Vulnerability Mining Based on the Human-Computer Coordination 537
Fuzz testing Let FT={Phi}
Task (FT) (i=1, 2, , n)
Match with
software
service lib
Match with
No Whether match No Whether match
successfully
human
successfully
service lib
Yes Yes i++
Match with
joint service buffer
library
Whether match Yes No
Whether i=n
successfully
Yes
Result
No Direct human-computer
interaction service Planning sorting Strategy
end output
Fig. 3. The procedure of the human-computer coordination strategy
5 Conclusion
The traditional software vulnerability mining method is often ineffective and difficult to
improve the level of intelligence in the task solving environment. This paper re-
examines the structure of the vulnerability mining system and the task solving process
from the perspective of “human-oriented”, and suggests that the role of human beings
should be highlighted in the process of system construction and task solving. This
paper proposes a vulnerability mining system architecture based on human-computer
coordination. Then, it designs the flow of the vulnerability mining task based on
human-computer coordination. Finally, it designs a task solving strategy based on
human-computer coordination for the fuzz testing scenario.
This paper only discusses the basic problems of human-computer coordination in
the software vulnerability mining system. In the future work, there are some theoretical
and key technical problems to be solved: (1) Key technologies such as human service
self-learning and adaptive data acquisition, learning mode, and learning algorithms;
(2) Theories and key technologies for improving efficiency, credibility, and practicality
in the automatic selection, matching, combination, and invocation of services (the joint
service of human service and software service).
Acknowledgments. This work is supported by National Key R&D Program of China
No. 2017YFB08029 and is supported by Sichuan Science and Technology Program
No. 2018GZ0101.
538 J. Liu et al.
References
1. Shahriari, H.R.: Software vulnerability analysis and discovery using machine-learning and
data-mining techniques: a survey. ACM Comput. Surv. 50(4), 1–36 (2017)
2. Liu, J., He, D., Rao, Z.H.: An analysis model of buffer overflow vulnerability based on FSM.
In: 2nd International Conference on Geoinformatics and Data Analysis (ICGDA), Prague,
pp. 47–51. ACM (2019)
3. Woods, D.D., Roth, E.M., Bnett, K.: Explorations in joint human-machine cognitive systems.
In: Cognition, Computing and Cooperation, pp. 123–158 (1990)
4. Jones, P.M., Chu, R.W., Mitehell, C.M.: A methodology for human-machine system research:
knowledge engineering, modeling and simulation. IEEE Trans. Man Cybern. 25(7), 1025–
1038 (1995)
5. Barthelemy, J.P., Bisdo, R., Coppin, G.: Human Centered Processes and Decision Support
Systems. Eur. J. Oper. Res. 136(2), 233–252 (2002)
6. Grudin, J., Carroll, J.M.: From tool to partner: the evolution of human-computer interaction.
In: Extended Abstracts of the Chi Conference (2017)
7. Kim, H.C., Choi, Y.H., Dong, H.L.: Efficient file fuzz testing using automated analysis of
binary file format. J. Syst. Architect. 57(3), 259–268 (2011)
Design and Verification Method for Flammable
Fluid Drainage of Civil Aircraft Based
on DMU
Yu Chen(&)
COMAC Shanghai Aircraft Design and Research Institute, 5188 Jinke Road,
Pudong New District, Shanghai, China
[email protected]
Abstract. Due to the complex internal environment, numerous system equip-
ment and pipe lines, the civil aircraft has many fire safety feature elements, and
the configuration of these feature elements needs to be managed and controlled
during the design iteration process. However, the traditional methods and means
has not met the needs of aircraft development. This paper proposes the use of
digital methods and means to construct a “DMU based on flammable fluid
drainage”, analyzes and summarizes the key technologies of DMU modeling
and simulation, At the same time, according to the method of system engi-
neering, the design and verification method of flammable fluid drainage based
on DMU are studied.
Keywords: Fluid drainage Flammable fluid System engineering DMU
Civil aircraft
1 Introduction
Aircraft fire safety is a safety concern of airworthiness authorities, aircraft manufac-
turers and operators. In the civil aviation field, according to the ICAO Safety Report [1]
and ASN accident statistics, the average annual aircraft fire accidents accounted for 8%
of the total accidents in the year from 2005 to 2014. The main design measures for civil
aircraft fire protection include: isolation, ventilation, drainage, detection, etc. Drainage
is a major measure for fire protection of flammable fluid and is an important area for
aircraft fire safety research.
Civil aircraft have fuel tanks and pipelines for storing flammable fluids. They are
scattered throughout the aircraft and may cause flammable fluids leakage during
storage and use of flammable fluids. Drainage is a main measure for fire prevention of
flammable fluids. The drainage of flammable fluids for civil aircraft is a measure taken
in order to comply with airworthiness regulations [2]. The main consideration is the fire
prevention of flammable fluids to avoid serious aviation accidents.
Fuel, ignition source and oxidizer are the three elements of aircraft fire. According
to the controlled state of three elements of flammable materials, ignition source and
oxidizer, the aircraft environment zone can be roughly divided into the following five
categories: designated fire zone, fire zone, flammable zone, flammable fluids leakage
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 539–545, 2020.
https://doi.org/10.1007/978-3-030-39512-4_84
540 Y. Chen
zone and non-hazardous zone. For the fire zone and flammable fluids leakage zone on
the aircraft, due to the risk of flammable fluids leakage, there are also ignition sources
and oxidizers, fire-prevention design for flammable fluids is an important factor in
above two zones.
2 Difficulties in Flammable Fluids Drainage Design
Civil aircraft enterprises generally adopt the “Guidelines for the Development of Civil
Aircraft and Systems (SAE ARP 4754A)” [3] to guide the research and development of
commercial aircraft products to ensure the research and development of civil aircraft
can meet the requirements of airworthiness certification.
Due to the complex environment of civil aircraft, the integration work for the
aircraft is very complicated. Therefore, in the design process, the control of the feature
elements of aircraft fire prevention, based on the traditional design method and process,
requires a lot of manpower and workload, and is extreme difficult. Therefore, for the
flammable fluids drainage design, new tools, methods or means must be adopted to
ensure that the process and results of the design based on fire protection requirements
are accurate and efficient.
With the promotion and application of digital definition and collaboration tech-
nology in aircraft design, the design efficiency has been greatly improved [4]. the
development of DMU generally focus on the collaborative function of product modules
based on product configuration management, and customizes DMU with the aim of
real-time synergy between products and DMU. However, the development and
application for specific function and customized users is very little.
Therefore, the following discussion is mainly to explore the methods in the design
and verification of flammable fluids drainage by means of corresponding digital tools.
And further more, it also studies the methods and approaches of applying digital design
means in fire prevention design from the perspective of life cycle of the aircraft and
throughout the development process.
3 Design and Verification of Flammable Fluids Drainage
The fluids drainage design is mainly based on the needs of fire protection design.
Through reasonable engineering design, the flammable fluids are drained out of the
fuselage, and the drain path is ensured to be safe, to make sure it won’t cause fire risk
and affect other system equipment’s function in other zones of the aircraft.
The design of aircraft flammable fluids drainage should be based on digital design
and collaborative methods to carry out the design iteration. The DMU is used to
analyze and simulate the design scheme of the flammable fluids drainage of the aircraft
to reduce the loses of schedule delay and cost caused by the design change. The
rationality of the design scheme of the aircraft flammable fluids drainage needs to be
verified by the aircraft drainage ground test and flight test, and the design of drainage
based on the DMU is optimized according to the verification result.
Design and Verification Method for Flammable Fluid Drainage 541
The design and verification of aircraft flammable fluids drainage based on DMU
contains the following four stages of work.
In requirements analysis and concept demonstration stage: Define the fluids type,
system and areas involved in the aircraft fluid drainage according to the top-level fire
protection requirements and the aircraft drainage requirements. The aircraft is divided
into different zones by digital methods, and corresponding drainage requirements are
set for different zones, which are used to guide and constrain system scheme design.
In the preliminary design stage: Carry out preliminary drainage design of structure
and system through identification of the drainage requirements of each zone.
In the detailed design stage: Based on the detailed design of structure and system,
gradually clarify the geometric parameter of fire safety feature element and conduct
leak source and drain path analysis and design iteration. Verify the feasibility of the
drainage proposal by using DMU tools.
In the trial production and certification stage: Plan and implement on aircraft
drainage ground test and flight test [5], compare the test results with the digital sim-
ulation results to optimize design.
4 Model Construction
Based on the definition of MBD [6] and the configuration management of DMU, the
aircraft drainage DMU is constructed for the design and verification of aircraft drai-
nage. The aircraft’s drainage DMU consists of zone classification model, flammable
material leak source model, leak source drip model, ignition source model, drain
location model and drain path prediction model. Among them, the flammable material
leak source model and the leak source drip model constitute flammable fluid control
model. The leak source drip model, the ignition source model and the drain location
Fig. 1. Composition of the aircraft’s drainage DMU
542 Y. Chen
model constitute the environment of the drain path prediction model. The drain path
prediction model is used to verify that the flammable fluid control meets the top level
requirements of the aircraft. The drainage DMU is shown in Fig. 1.
4.1 Zone Classification Model
According to the requirements of the fire prevention, the aircraft is divided into five
categories, see Sect. 1 of this article for details. The zone classification model can be
obtained by defining the boundary of the aircraft according to the division of the five
categories zone.
4.2 Flammable Material Leak Source Model
Most of the flammable material leak sources models are related to the location and form
of the pipeline joints. For zone with high fire protection level such as E-E bay, it is
generally necessary to avoid the use of joints that may leak flammable materials. In the
flammable fluids leakage zone, there are many leak sources due to maintainability,
separation interface, installation and connection of pipeline. In general, a large number
of hydraulic pipe joints on the aircraft are leak sources. The flammable material leak
source model can be represented with points. As shown in Fig. 2.
Fig. 2. Diagram of the flammable material leak source model
4.3 Leak Source Drip Model
With the improvement of the manufacturing technology of aircraft piping systems and
joints, the failure modes of hydraulic and fuel pipelines are generally drip or leakage.
The leakage source drip model simulates the motion trajectory after flammable material
leakage, and can be represented with lines or curves. Leak source drip model is highly
related to aircraft’s attitude. Pitch angle, yaw angle and roll angle are three important
parameters that characterize the attitude of the aircraft, and are also an important basis
for the establishment of the aircraft leakage source drip model. According to the
attitude profile of the aircraft in the body coordinate system OXb Yb Zb , the envelope
model of the influence range of dripping after fluids failure can be obtained. As shown
in Fig. 3.
Design and Verification Method for Flammable Fluid Drainage 543
Fig. 3. Schematic diagram of the aircraft attitude
4.4 Ignition Source Model
The ignition source in most cases are hot surface of system, hot fluid leaking during
system failure and arc/spark generated during system failure. For the ignition source,
the model is usually represented by a surface or a body part, which can be extracted
from the surface geometry of the product.
4.5 Drain Location Model
The fluids generated on the aircraft must be drained to the outside through drain outlets
on fuselage surface. There are two main forms for the overboard drain outlets. One is
the drain hole, and the other is the drain mast. These drain outlets also require the
construction of a corresponding drain location model.
4.6 Drain Path Prediction Model
The drain path prediction model mainly refers to the path channel model in the process
from the leak source to the final drainage outside point. Since the aircraft is based on
the corrosion prevention and other factors in the structural design requirements, it is
necessary to ensure that no accumulation of fluids on the structure. For the design of the
structure-based drain channel, it is mainly to ensure the path from fluids to structural
drainage channel is feasible. As shown in Fig. 4.
Leak source 1 Leak source 2 Leak source 3
Fig. 4. Structure-based drain channel
544 Y. Chen
4.7 Design Integration Based on Drainage DMU
The drainage DMU is gradually improved and optimized as design is refined and
iterated. The analysis and coordination based on DMU are carried out to gradually
improve the design maturity.
In the top-level planning, preliminary identification should be carried out according
to the system function and architecture. The planning and definition of space allocation
should be done as much as possible in the preliminary design. For example, EWIS
wiring is a ignition source, therefore, when the pipeline route is planned, the relative
space should be arranged reasonably. Generally, the EWIS channel should be avoided
below the pipeline. Since the leak source is mainly fuel and hydraulic pipelines, it is
essential to properly define the location of the leak source when making the space
allocation of the flammable fluids system piping.
4.8 Validation and Verification Based on Drainage DMU
Finally, there are two main contents for the validation and verification based on DMU.
One is whether there is ignition source on the flammable fluids leakage path, and the
other is whether the flammable fluids is unobstructed to the outside of the aircraft.
If the drain path does not meet the requirements, a change or optimized drainage
scheme is required. The design optimization is carried out through various ways and
measures such as joint position adjustment, pipeline layout optimization, ignition
source position adjustment and installation isolation protection.
It is necessary to plan the external drainage scheme of the aircraft, and define the
corresponding external fluids drainage point and drainage form based on the external
fluids control requirements. At the same time, in order to evaluate the influence of the
design of the external drain device on the drainage trajectory and the aircraft fuselage, it
is necessary to analyze and calculate the external flow field of the drainage point. As
shown in Fig. 5.
Fig. 5. Simulation and verification of fluids drainage
5 Conclusion
According to the requirements of aircraft flammable fluid drainage, this paper studies
the work required for the flammable fluids drainage based on DMU in various stages of
aircraft development by means of system engineering. In addition, in the aspects of
Design and Verification Method for Flammable Fluid Drainage 545
model construction, integration and simulation, the design and verification ideas of
drainage DMU are gradually improved and refined, so that the development and
application of the drainage DMU can fully support the design and verification of
aircraft flammable fluids drainage.
References
1. International Civil Aviation Organization. ICAO Safety Report, 2014 edition. Montreal:
International Civil Aviation Organization (2014)
2. FAR Part 25 Airworthiness Standards Transport Category Airplanes
3. SAE ARP 4754A, Guidelines for Development of Civil Aircraft and System. Warrendale:
Society of Automotive Engineers (2010)
4. Mitchell, M.T., Jianxin, J., Chuan-Jun, S.: Virtual prototyping for customized product
development. J. Integr. Manuf. Syst. 9(6), 334–343 (1998)
5. Flammable fluid fire protection. AC25.863-1draft. U.S. Department of Transportation:
Federal Aviation Administration. ANM-110
6. Y14.41-2003, Digital Production Definition Data Practices
Low-Income Dwelling Bioclimatic Design
with CAD Technologies. A Case Study
in Monte Sinahí, Ecuador
Jesús Rafael Hechavarría Hernández(&), Boris Forero,
Robinson Vega Jaramillo, Katherine Naranjo, Fernanda Sánchez,
Billy Soto, and Félix Jaramillo
School of Architecture and Design, Catholic University of Santiago de
Guayaquil, Av. Carlos Julio Arosemena Km 1 ½, Guayaquil, Ecuador
{jesus.hechavarria,boris.forero,robinson.vega,
maria.naranjo07,fernanda.sanchez01,billy.soto,
felix.jaramillo}@cu.ucsg.edu.ec
Abstract. The present work is the result of a low-income dwelling inclusive
design for a family at Monte Sinahí. Students and professors of the Faculty of
Architecture and Design of the Catholic University of Santiago de Guayaquil in
one of the economically less favored settlements and exposed to high vulner-
ability risks in Guayaquil, designed a housing prototype through a systemic way
with bioclimatic criteria and the use of ecomaterials. The project includes uni-
versal accessibility required by a family consisting of an older adult with motor
disability and a young man with Down’s syndrome. This archetype is a partial
result of a winner project of the National Program of Financing for Research -
INEDITA in the area of Energy and Materials research of the Institutional
Modality, organized by the Secretariat of Higher Education, Science, Tech-
nology and Innovation (SENESCYT) and the United Nations Development
Program (UNDP) in Ecuador.
Keywords: Universal access Inclusive design Low-income dwelling
Ecomaterials
1 Introduction
The provision of housing demands answers to the increase of the population of
Ecuador. According to INEC census data to the year 2010 [1] it is estimated that the
population will increase by 180% by 2050. In Guayaquil inhabitant’s population will
increase by 155%. This analysis was carried out considering a national growth rate of
0.0155 and a regional one of 0.011 on average, according to the projection data of the
ecuadorian population by calendar years [2]. In this scenario, Ecuadorian institutions
must continue to promote technological independence in various productive sectors
such as design and production of homes which must be coupled with the development
of human talent, technology and knowledge. In the cases studied in, it was evident that
the desire to occupy the most of the land itself, brought with it space configurations that
are not the most recommended for the warm-humid climate of Guayaquil [3]. These
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 546–551, 2020.
https://doi.org/10.1007/978-3-030-39512-4_85
Low-Income Dwelling Bioclimatic Design with CAD Technologies 547
cases were characterized for its poor or no cross ventilation presence. To this is added
the use of construction materials found on site, characterized by a high thermal
transmittance of their enclosures, such as zinc roofs and the non-existence of ceiling
roofs dampen the transfer of heat into the dwelling [4].
The main results obtained are related to the proposal of graphic and mathematical
methods to analyze the dimensional behavior of residential spaces in Guayaquil [2], the
user’s perception of their home and the environment in social interest programs [5], the
analysis of the effect of thermal comfort in low-cost settlements [6], the systemic
approach to design as a fundamental element in multiobjective analysis [4] and the use
of bioclimatic strategies in the design of popular dwelling units. On the other hand, the
use of software for geometric modeling of the building and thermal and loads simu-
lations is a practice experienced by the work team [7], which favors the acquisition of
new knowledge for the development of the investigation.
On the other hand, inclusive design should consider the needs of people with
limited accessibility capabilities. While modeling the behavior of dwellers with dis-
abilities is a complex task, it has a remarkable social and economic impact. To do this,
some authors explore cognitive neutrophic maps to represent the behavior and func-
tioning of complex systems [8]. The relevance of the Catholic University of Santiago
de Guayaquil considers the Sustainable Habitat through the planning of access to
housing. The project called “Design and construction of a sustainable archetype of
housing of social interest for inhabitants of Monte Sinahí, Guayaquil, Ecuador” aims to
contribute to the design of social interest housing units through the construction of
archetypes for the inhabitants of popular settlements with the use of ecomaterials and
bioclimatic strategies that improve thermal comfort and enhance environmental care.
This paper highlights the procedure followed during the design stage considering the
climate and dwelling information from databases related to the geographical area of
Monte Sinahí and field data gathering.
2 Collection of the Bioclimatic Information
The collection of the bioclimatic information of the geographical area where Monte
Sinahí is located begins with the installation of the Dataloggers [9] and will culminate
with the benefits offered by the Meteonorm software [10] on global and diffuse radi-
ation. The characterization of the climate of the study area is concluded, by analyzing
the climatic variables that allow the construction of the psychrometric graph of the
region where the architectural project will be located to define the requirements of
bioclimatic design, its strategies and tactics to reach thermal comfort ranges.
3 Analysis of the Housing Information
The design of an archetype of low-income dwelling must carry out in a systemic way
where the study of the social, economic and environmental context are analyzed, thus
characterizing the territory where the project will be implemented and users of the home
who protagonists in the decision are making. Based on the requirements established under
548 J. R. Hechavarría Hernández et al.
multiple criteria, the multidisciplinary team is responsible for defining the requirements,
strategies, tactics and techniques for the bioclimatic design of the archetype with the
purpose of guaranteeing thermal comfort inside the dwelling at Monte Sinahí (Fig. 1).
Fig. 1. Working sessions with multidisciplinary teams to meet project requirements.
In prototype’s design, ceiling insulation is considered. This will allow a significant
decrease of indoor temperature as demonstrated in others investigations [6]. The
openings of the roof system were designed to establish a longitudinal flow for the
ventilation of the interior spaces together with the garden and the orientation of the
winds. Having spaces whose facades have openings towards exterior spaces, allows
optimum ventilation of interior spaces and the extraction of heat generated by internal
and external gains (Fig. 2).
Fig. 2. Sketches of architectural design process that meets multiple criteria.
Figure 3 shows the component elements of the archetype listed below:
1. Roof design with facade openings that allows convective cooling.
2. Polished textured metal roof with high albedo index.
3. Mobile panels designed to sift natural light.
4. Door and window designs that allows ventilation and visual control.
5. Flexible panels brings spatial and visual integration with the garden.
Low-Income Dwelling Bioclimatic Design with CAD Technologies 549
Fig. 3. Archetype components details.
4 Digitalization of the Archetype Design
The digitalization process was carried out with the version 22 of the ArchiCAD
through the BIM work philosophy [11], which allows a better quality in the devel-
opment of the assembly and greater precision in the elaboration of technical drawings,
necessary for the manufacture of the ecomaterials to be used in the construction of the
dwelling (Figs. 4 and 5).
Fig. 4. Digitalization process of dwelling design.
550 J. R. Hechavarría Hernández et al.
Fig. 5. Use of ArchiCAD through the BIM work philosophy.
The use of eaves is based on the solar orientation that allows the presence of
shadows in the hours near mid-morning and mid-afternoon, where there is a higher
incidence of radiation. Main ideas are expressed in sketches for a better understanding
of spatial and dimensional relationships and the proportions that the interior spaces of
the archetype must have. This is done through a cyclical process where technical and
humanistic criteria are considered until reaching a definitive design.
5 Discussion
The meteorological information obtained through Meteonorm software [9] and the
Dataloggers [10] located inside the house, allows the analysis of requirements and
strategies during the bioclimatic design process of the archetype. In order to ensure
thermal comfort inside the home, tactics and techniques are established as described in.
The spatial modeling of the archetype with the help of the ArchiCAD allows the
visualization of the component elements of the dwelling as well as, the detailed
description of the technical drawings necessary for the subsequent construction.
Acknowledgments. The authors wish to thank the Catholic University of Santiago de
Guayaquil and the Secretariat of Higher Education, Science, Technology and Innovation in
Ecuador for the financial support for this research. To the United Nations Development Program
- UNDP for the excellence in management and Graphisoft Ecuador for the professionalism in
teaching version 22 of ARCHICAD. To priests Luis Enriquez and Antonio Martinez and Mrs.
Gloria and her son for their human quality.
Low-Income Dwelling Bioclimatic Design with CAD Technologies 551
References
1. NISC: National Institute of Statistics and Censuses in Ecuador. https://www.ecuadorencifras.
gob.ec/estadisticas/. Accessed 19 Aug 2019
2. Almeida, B., Díaz, J., Bermeo, P., Hechavarría, J., Forero, B.: An alternative graphic and
mathematical method of dimensional analysis: its application on 71 constructive models of
social housing in Guayaquil. In: Ahram, T. (eds.) Advances in Artificial Intelligence,
Software and Systems Engineering. AHFE 2019. Advances in Intelligent Systems and
Computing, vol 965, pp. 598–607. Springer, Cham (2020)
3. Forero, B., Hechavarría, J., Vega, R.: Bioclimatic design approach for low-income dwelling
at Monte Sinahí, Guayaquil. In: Di Bucchianico, G. (eds.) Advances in Design for Inclusion.
AHFE 2019. Advances in Intelligent Systems and Computing, vol 954, pp. 176–185.
Springer, Cham (2020)
4. Hechavarría, J., Forero, B., Bermeo, P., Portilla, Y.: Social inclusion: a proposal from the
University of Guayaquil to design popular housing with citizen participation. In: 2018 World
Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC),
Albuquerque, NM, USA, pp. 1–5 (2019)
5. Forero, B., Hechavarría, J., Alcivar, S., Ricaurte, V.: Systemic approach for inclusive design
of low-income dwellings in popular settlements at Guayaquil, Ecuador. In: Ahram, T.,
Karwowski, W., Taiar, R. (eds.) Human Systems Engineering and Design. IHSED 2018.
Advances in Intelligent Systems and Computing, vol 876, pp. 606–610. Springer, Cham
(2019)
6. Ricaurte, V., Almeida, B., Hechavarría, J., Forero, B.: Effects of the urban form on the
external thermal comfort in low-income settlements of Guayaquil, Ecuador. In: Charytonow-
icz J., Falcão C. (eds.) Advances in Human Factors in Architecture, Sustainable Urban
Planning and Infrastructure. AHFE 2019. Advances in Intelligent Systems and Computing,
vol 966, pp. 447–457. Springer, Cham (2020)
7. Dick, S., Hechavarría, J., Forero, B.: Systemic analysis of bioclimatic design of low-income
state-led housing program ‘Socio Vivienda’ at Guayaquil, Ecuador. In: Ahram, T.,
Karwowski, W., Taiar, R. (eds.) Human Systems Engineering and Design. IHSED 2018.
Advances in Intelligent Systems and Computing, vol 876, pp. 647–651. Springer, Cham
(2019)
8. Colorado, B., Fois, M., Leyva, M. and Hechavarría, J.: Proposal of a technological
ergonomic model for people with disabilities in the public transport system in Guayaquil. In:
Ahram, T., Falcão, C. (eds.) Advances in Usability and User Experience. AHFE 2019.
Advances in Intelligent Systems and Computing, vol 972, pp. 831–843. Springer, Cham
(2020)
9. ELICROM: Datalogger (2019). http://elicrom.com/elicrom-ecuador/
10. Meteonorm: Handbook part II: Theory Global Meteorological Database Version 7 Software
and Data for Engineers (2018)
11. GRAPHISOFT, ArchiCAD (2019). https://www.graphisoft.es/downloads/archicad/. Acces-
sed 08 Aug 2019
Virtual Reduction and Interaction of Chinese
Traditional Furniture and Its Usage Scenarios
Dehua Yu(&)
Beijing Institute of Technology, No. 5 Zhongguancun South Street,
Haidian District, Beijing, China
[email protected]
Abstract. Chinese traditional furniture is an important part of Chinese tradi-
tional culture. Chinese traditional furniture is made of solid wood and uses
mortise and tenon to combine all the parts of the furniture without any metal
nails. This research is aimed at the display and interaction of the unique mortise
and tenon structure of Chinese traditional furniture, drawing plane and 3D
models, restoring the usage scenarios of Chinese traditional furniture. In this
scenario, combination of reality and virtual, static and dynamic would be shown
with the help of motion capture technology, sensing technology, AR and VR
technology, interactive technology. From the interaction of the Chinese tradi-
tional furniture and the visitors in the exhibition room, people would obtain
better experience of visiting, and learn more knowledge with more interest.
Keywords: Chinese traditional furniture Mortise and tenon Interaction
1 Introduction
Chinese traditional furniture is an important part of Chinese traditional culture. Chinese
traditional furniture is made of solid wood and uses the mortise and tenon to combine
all the parts of the furniture without any metal nails. The structure of mortise and tenon
of Chinese traditional furniture is so strong that many Chinese traditional furniture can
hand down to generations for more than hundreds of years. That is the charm of the
structure of mortise and tenon. Therefore, the structure of mortise and tenon has its own
scientific principles and techniques, which is worth learning and inspiring innovation
for designers and engineers.
The inside structure of Chinese traditional furniture is complicated; however the
outside looking of Chinese traditional furniture is simple, which indicate the implicit
philosophy of China. The same outside looking of Chinese traditional furniture might
have different inside structures, which is unimaginable for people who do not know
more about Chinese traditional furniture. People cannot observe the inside structure
direct and get a direct understand of the mortise and tenon of Chinese traditional
furniture. Therefore, the display of interaction of the mortise and tenon of Chinese
traditional furniture is vital and necessary.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 552–558, 2020.
https://doi.org/10.1007/978-3-030-39512-4_86
Virtual Reduction and Interaction of Chinese Traditional Furniture 553
2 Present Situation of Display
The traditional methods of display of mortise and tenon of Chinese traditional furniture
is just pictures, drawing, GIF animation on the computer, or real models made of wood
or with 3D printing technology. These years some interaction methods are used to
display the structure of Chinese traditional furniture. For example, two APPs “Wood
Joints” and “Chinese Traditional Furniture” are both aimed at the display and inter-
action of Chinese traditional furniture, in which people can interact with the joints of
different mortise and tenon, learning the connect methods, the different joints of mortise
and tenon. Some more attempts of the interaction with the mortise and tenon are made
in some exhibition. For example, the exhibition “the Charm of Mortise and Tenon”
(Fig. 1) was held in Beijing Science and Technology Museum in 2018. In this exhi-
bition all kinds of Chinese traditional wood works are display, for example, Chinese
traditional architecture, boats, carriage, bridge, and so on, the materials of mortise and
tenon are not limited in wood, but also metal and stone. What is more attracting is that
there are some interaction between visitors and exhibits, some real and some virtual.
Fig. 1. The exhibition “the Charm of Mortise and Tenon” was held in Beijing Science and
Technology Museum in 2018.
The project is amide at display, interaction of the unique tenon-mortise structure of
Chinese traditional furniture and restoring the usage scenarios of Chinese traditional
furniture to help people to know more about the connotation of Chinese traditional
furniture. Some unique Chinese traditional furniture would be chosen from hundreds of
furniture. The selection of furniture is based on the unique structure and unique out-
ward appearance. In this paper two unique Chinese traditional furniture is selected to
research on the display and interaction. There are two targets for the display and
interaction, one is the structure of mortise and tenon, the other is the ancient using
condition of Chinese traditional furniture, which can be both useful for people to learn
about the deep connotation of Chinese traditional furniture and culture.
554 D. Yu
3 Special-Shaped Table
Special-shaped table (Fig. 2) has a special shape “ ” tabletop, which is different
other Chinese ordinary table with rectangle, square, circle tabletop. It is made of solid
wood with black and red lacquer. Because of the special-shaped tabletop, the table has
some unique function, which perhaps is the stand for the unique flowerpot or the
unique artificial hills. Also because of the special-shaped tabletop, the structure of the
mortise and tenon is unique which is different from the ordinary structure of Chinese
traditional furniture. For example, the joint of the top and the leg is more complicated
than ordinary mortise and tenon, which has changed and modified according to the
unique shape and part (Fig. 3). It is not easy to observe the joints details from just only
picture and drawing, and it is necessary to take advantage of interaction technology to
guide people to better understanding of the complicated structure.
Fig. 2. Special-shaped table (Fig. 2) has a special shape “ ” tabletop.
Fig. 3. The joint of the top and the leg is more complicated than ordinary mortise and tenon.
Virtual Reduction and Interaction of Chinese Traditional Furniture 555
3.1 Basic Display
At first, we make 3D model with 3D software, and make semi-transparent 3D model to
help people observe the joint structure. Several videos (Fig. 4) on different parts of
mortise and tenon of Chinese traditional furniture will be shown on the wall. When
visitors pass by incidentally, the video would attract their attention. If someone is
interested, he might come close to join in the interaction.
Fig. 4. Several videos are made on unique structure of mortise and tenon.
3.2 Interaction
When visitors are more interested, they would like to come close to observe the
furniture. The furniture is consist of two parts, one half is the real table made of solid
wood, the other half is the virtual display played by a projector. The two opposite parts
forms a whole table (Fig. 5). When visitors come close to observe, the virtual part
would ripple a little to imply it is virtual. When visitors touch the specific parts, the
signal would inducted by the sensor, after that, the video would show the semi-
transparent parts, finishing the fix and unfix process of mortise and tenon to direct
visitors to fix and unfix the real parts of the table. The interaction process is accom-
panied by indication music and voice, which is interesting and attracting.
Fig. 5. Real table made of solid wood and virtual display forms a whole table.
556 D. Yu
4 Backrest Chair
Backrest chair (Fig. 6) only comes from Jiangsu province, and the backrest is made of
weaving mat, different from the solid wood backrest. The unique structure is the joint
between the back leg and the seating top (Fig. 7), which is earlier than the ordinary
Gejiao Joint. This chair might be used during late Ming and early Qing dynasty.
A famous painter called Fanglin Ye in Qianlong period in Qing dynasty, once draw a
painting “Scholar Elegant Meeting” in which there is a similar chair with this backrest
chair. The theme of the painting is also based on the location of Jiangsu province,
which is the same with this backrest chair. Therefore, the painting could give us the
possible using condition of this backrest chair.
Fig. 6. The 3D model of the backrest chair are made.
Fig. 7. The unique structure is the joint between the back leg and the seating top.
4.1 Basic Display
First, we make 3D model with 3D software, and all the details of mortise and tenon of
the chair are made. A real solid model made with solid wood would be made to take
apart to show mortise and tenon of the chair, especially the unique joints. The drawing
“Scholar Elegant Meeting” would be shown on the whole wall played by a project,
which illustrate the possible using condition of the chair in the ancient times.
Virtual Reduction and Interaction of Chinese Traditional Furniture 557
4.2 Interaction
The drawing “Scholar Elegant Meeting” would not only the display of the using
condition, but also the interaction of the ancient living environment and the modern
people. The surrounding of the wall of the drawing “Scholar Elegant Meeting” would
add some motion sensors. When visitors pass by, the details of the drawing would
move slightly, for example, the branches of the trees are shaking, and the leaves are
falling, people wave his hand, and blink a little, which can get visitors’ attraction. In the
mean scene of the drawing, one scholar sits on the chair which is similar with our chair.
On the opposite of the wall, the copy of the chair and a table is displayed (Fig. 8).
Every visitor can sit on the chair and besides the table. Once people sit on the chair, the
gravity sensor would sense the signal, then the drawing of the scene would be display
with an active video. In this scene, the scholar would stand from his chair, and say hello
to visitors, and he would also show his chair to visitors to introduce the chair. Another
scholar would also intervene in the interaction. All the interaction indicate that the
visitor seems joining in the ancient living environment, and interact with the ancient
people to talk about the daily life. Furthermore, different visitor sit on the chair might
trigger different video, which might attract more visitors to join in the interaction,
which add interaction and attraction.
Fig. 8. The interaction of the virtual painting and visitor are shown.
Acknowledgments. This paper is supported financially by MOE (Ministry of Education
in China) Project of Humanities and Social Sciences Research Youth Fund (Project
No.19YJC760144).
References
1. Yang, Y.: Study of Ming-Style Furniture. Beijing Construction Industry Press, Beijing
(2002). (in Chinese)
2. Wang, S.: Study of Ming-style Furniture. Joint Publishing Company, Beijing (2008). (in
Chinese)
558 D. Yu
3. Ji, Y., Yu, D.: Chinese Classical Furniture Design Basics. Beijing Institute of Technology
Press, Beijing (2015). (in Chinese)
4. Yu, D., Ji, Y.: Chinese Classical Furniture Design Practise. Beijing Institute of Technology
Press, Beijing (2015). (in Chinese)
5. Berliner, N., Handler, S.: Friends of the House: Furniture from China’s Towns and Villages.
Peabody Essex Museum (1995)
6. Chen, J.J.I.: Chinese Contemporary Furniture Design from Culture Appreciation to Chunzai
Creativity. Palace Museum Press, Beijing (2016)
Humans and Artificial Systems
Complexity
Your Voice Assistant Will See You Now:
Reducing Complexity in Human and Artificial
System Collaboration Using Voice
as an Operating System
Viraj Patwardhan(&), Neil Gomes, and Maia Ottenstein
Thomas Jefferson University and Jefferson Health, 925 Chestnut St.,
Philadelphia, PA 19107, USA
[email protected]
Abstract. There is something about smart artificial systems that fascinates us.
In recent decades we have seen researchers, technology experts, large technol-
ogy companies and governments across the world investing significant time in
getting artificial systems to be smarter, more interactive and easier to commu-
nicate with. Smart assistants are slowly becoming an integral part of our lives,
but they are far from perfect. There are many concerns and questions that must
be addressed about these assistants. How can we create the ideal version of this
partnership? How can we break the barriers to communication with these sys-
tems? This paper will highlight how companies are addressing the issues with
artificial systems by designing systems that are safer, more accurate and
trustworthy.
Keywords: Voice technology Voice assistants Voice interfaces VUI
Human and artificial system collaboration
1 History and Evolution of Voice Technology
Our voices are powerful. They are our primary outlet for our thoughts, feelings, and
beliefs. We use our voices to interact with the world around us. Our voices enable us to
communicate with other humans and other beings and help us to coordinate, collab-
orate, learn, teach, argue, and evolve. Innovators of every generation have explored
how the human voice can alter our surroundings, from the first instances of humans
exploring incantations to alter reality, to contemporary technology that enables us to
alter reality by uttering a command as simple as “order my laundry detergent.”
However, while the complex technology behind natural language processing has been
evolving, the majority of public users, have largely adapted to the physical input
methods and interfaces developed in lieu of voice, as this was less complex at the time.
Our widespread interest in merging the human voice with technology is evidenced
by the first documented voice-based technology released in 1877; Thomas Edison’s
phonograph [1]. After Edison discovered how to record our voices, innovators har-
nessed this idea to, over time, invent the radio and the telephone. And eventually, in
1962, IBM invented Shoebox, the first IBM computer capable of speech recognition.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 561–566, 2020.
https://doi.org/10.1007/978-3-030-39512-4_87
562 V. Patwardhan et al.
Shortly after the invention of Shoebox, ELIZA – a computer program developed to
allow humans to have natural conversations through text with an automated system –
was invented. The ELIZA program was so well done for its time that, while most users
began using it as skeptics, after a few exchanges with ELIZA, many were keen to
continue speaking with the system [2].
By the 1980’s, voice recognition software was widespread enough to make its way
into the toy industry where World of Wonder’s Julie doll and Playmates Toys’ Jill doll
competed for the hearts of children across the U.S. In 1993, Apple commercialized
built-in speech recognition and voice-enabled control systems with Speakable Items.
Despite computer scientists’ best efforts to bring voice-controlled computers to the
mainstream, the first widely used voice user interfaces, or VUIs, were interactive voice
response (IVR) systems [3]. You may know these as the automated human-sounding
systems that receive your calls to any variety of organizations, from local restaurants to
your electricity company. IVR technology evolved from the 1960s through the 1990s,
and by the early 2000’s, was a standard aspect of phone calls to any large organization.
While the voice technology was evolving, mobile phones became commonplace
and rapidly evolved with more capabilities and varying interaction models. It wasn’t
long before modern innovators saw the opportunity to integrate voice technology into
mobile devices. 2011 saw the birth of Siri, the first modern, voice-controlled, smart
assistant and a major milestone in human-artificial systems collaboration. Shortly after
followed Amazon’s Alexa and Google Home, which allowed for integrating voice
interfaces in people’s homes and daily lives.
2 Voice as an Interface
Most interfaces we are familiar with are visual and physical. Your computer screen,
keyboard, even your coffee maker, all provide you with visual information, require
physical manipulation to alter their state or create content, and provide visual and/or
tactile feedback to confirm the success or failure of your attempt. On the other hand,
voice interfaces allow users to interact with technology without the need for physical
input or visual feedback.
As physical devices get smaller and the situations in which we use smart devices
become more varied, it becomes more difficult to physically interact with the GUIs we
have come to know [3]. It is thus vital that we have alternate interaction options to
extend usability and enhance safety across various environments of use.
Voice technology is in many ways an ideal solution to close the usability gap for
our technologies with shrinking screens and with the diverse use cases that continue to
arise with newer smart devices. Voice systems are also an ideal solution because, for
most people, our voices are our primary means for communication. Voice is not only
the most prevalent method of communication, it is also the most natural to us. It is no
surprise then that we aim to create artificially intelligent systems that communicate with
us similarly to how we communicate with other humans.
Your Voice Assistant Will See You Now 563
3 Benefits
There are many benefits to the integration of VUIs, including improved usability and
accessibility, comfort, and advanced efficiency and safety. VUIs are much easier to
learn and can be easier to navigate, especially for simple tasks, offering potential to
drastically improve a device’s usability. As discussed in a paper by Pradhan et al. [4],
Interaction is remote (e.g., across the room), which lowers the barrier to use in com-
parison to having to hold/use a device, and home-based assistants can connect to smart
home appliances, becoming integrated into the home environment. A VUI can be
particularly helpful for those with physical disabilities and accessibility issues.
Many journals and studies investigating the application of VUIs across a range of
contexts find them to increase safety, efficiency, and comfort. The issue of safety is
most apparent in the example of driving, in which studies show VUIs can dramatically
reduce risk of completing tasks like navigation while driving. In April 2019, Science
Direct published an article about the risk of distracted driving, noting that “engaging in
a visually demanding task, such as interacting with a cellphone or in-vehicle devices,
was associated with increased crash risk as well as higher crash severity.” The report
indicated that interactions that do not require drivers to take their eyes off the road
greatly increase safety in comparison to activities in which drivers look away from the
road [5].
Introducing voice-based artificial systems into the home is not only more accessible
for those who may not be able to easily adjust their environment on their own, it adds a
level of support and efficiency to the home environment, even for highly capable
people. Existing smart home systems with VUIs offer voice-activated systems that,
through one simple, natural command can sweep an item off your to-do list as soon as it
arises, allowing you to run from task to task: for example, washing dishes and noticing
the detergent is running low, you can simply announce “Order more detergent,” and
know that it will be delivered before your container is empty.
While this example may not sound like a high-stress occurrence, our lives are
growing ever-more complicated, and each time we have to remember one more thing
we are likely adding strain to an overworked brain. Authorities have been concerned
about the increasing demand on our working memory since well before 1994, when the
Rank Xerox Research Centre Cambridge Laboratory launched the “Forget-me-not”
project, an exploration into what its creators called “intimate computing.” Their goal
was to build smart, personal digital assistants, or PDAs, to support and minimize
cognitive work by reducing the amount users needed to remember at any given time
[6].
Due to their high accessibility, commercial popularity, low barrier to entry, and
non-threatening or population-specific nature, artificial systems with voice-based
interfaces have the potential to change the lives of millions of people who need support
but do not want specialty assistance. Voice technology empowers these people to retain
their autonomy by giving them control over their environment regardless of their
physical movement limitations.
564 V. Patwardhan et al.
4 Challenges
While integrating voice technology offers a huge amount of benefit to automated
systems, there are some challenges that must be considered before incorporating VUIs
into any system. Because voice technology is still developing and people and cultures
are still adjusting to their growing ubiquity, many foundational needs remain to be
addressed, including some usability issues, establishing trust, and the appropriateness
of implementation.
As mentioned previously, voice technology is still relatively young, and most
systems cannot yet understand users whose voices sound different from a few specific
demographics. This means that voice-based systems are unusable to users who do not
speak exactly as the engineers who created the system want them to, leaving people
who cannot speak loudly enough, clearly enough, or without their natural accent out of
the voice tech revolution. VUIs also provide a barrier for users with impaired hearing.
If a system accepts only voice input and supplies only audio responses, users with
impaired hearing cannot interact with that system unless it is also accompanied by
visual response indicators.
Voice-based systems also require very thoughtful information and interaction
designs, as it can be easy to get lost while navigating a system with no visual cues.
Tasks that are simple in well-designed graphical interfaces, like trying to return to a
previous place in your conversation with the device can become strenuous and con-
fusing, if at all possible. Well-designed systems with visual interfaces show their users
the information architecture of the system and where that user is currently located
within that system, for example, the use of breadcrumbs on a webpage. But this is
much more difficult to do with voice-only interfaces, and designers must carefully
make the system “easy to use, easy to learn and resilient to errors” [7].
Good design is imperative not only in the creation of the system itself, but the
appropriateness of the decision to include a voice interface at all. Many technologists
are excited about voice technology and eager to adopt it into their systems, but
unnecessarily relying on voice interactions may negatively impact users, risking both
their satisfaction and their safety.
Voice-based interactions are also not ideal for public or shared spaces, where it can
not only be difficult to conduct depending on surrounding noise levels, but also dis-
ruptive to those beyond the immediate users [3]. Public spaces introduce the require-
ment for security, because every interaction with a system is now capable of being
overheard. People may not be as comfortable using the ATM, for example, if it
required users to speak their PIN codes rather than typing them in.
5 Potential of Voice Interfaces and a Healthcare Example
Despite the challenges, voice technology is fast becoming a practical household reality.
Voicebot.AI reports that the smart speaker install base within the U.S. grew 40% from
2018 to 2019 [8], now exceeding 66 million units. The smart speakers with screens
market grew even more from 1.3M in January 2018 to 8.7M in just one year [8]. These
trends suggest that adoption of voice-enabled devices is on the rise. So how do
Your Voice Assistant Will See You Now 565
companies deal with this technology that is clearly becoming a key communication
channel to reach their customers? A recent article in HBR suggested that the most
critical initiative for organizations from major industries is to have a Strategy for Voice
Technology [9]. Amongst other interesting data points, the article highlights that
companies that ignore voice technology will be at risk of not just losing market revenue
but also losing current and future customers.
At Thomas Jefferson University and Jefferson Health (Jefferson), we have
embraced voice technology and consider it a key communication channel for our
consumers. Health information is very personal. Hospitals are expected to keep this
information private and safe and yet accessible to healthcare providers. At Jefferson, in
addition to data privacy it is paramount for us to have the trust of our patients when it
comes to their health data. So how does a healthcare system introduce voice technology
by ensuring the safety and privacy of the data?
After researching the benefits and challenges of the commercially available voice
assistants, Jefferson decided to build its own system that will be self-contained within
Jefferson’s secure network environment. This is a system, that will provide our patients
the comfort that their data is not at-risk and their interactions with the voice assistant
will remain private and confidential, just like all of their personal health data. One of
the key reasons for developing our own voice system was to recognize the importance
of and earn the trust of our patients. The system has been launched in 11 hospital rooms
as a pilot and researchers continue to learn more about patient interactions to contin-
uously improve the system. Some of the key features include, TV Control (Volume,
Channel, On/Off), Jefferson information (Location, Hours, Phone Number, Visitor
info) weather, patient onboarding information, diet information and device feedback.
We have seen that once you establish trust with the end-users there is an increase in the
interactions with the system. Some key learnings are listed below:
1. Introduce the system to the patient: Give information on how the system will work,
how to interact with the system and what to do in case the system does not respond.
This helps reduce patient anxiety.
2. 20 average interactions per patient/per day: Patients interact with the system when
they build trust with the interface.
3. TV Control is the most widely used voice command: In a hospital environment,
patients are looking to interact with systems that will distract them from their
anxiety.
6 Conclusion
In summary, we believe that voice technology is here to stay. The technology is still
evolving and does pose challenges especially in the areas of privacy, trust and data
protection. However, benefits like the simplicity of use, the human “naturalness” of the
interface, lower barriers to adoption and its ability to reduce the cognitive load on users,
mean that companies (especially in healthcare) should invest in this technology and
help with its development.
566 V. Patwardhan et al.
In order to deliver the right voice product experience to patients, healthcare systems
need to adopt a three-point strategy.
1. Invest more time in learning how this technology will impact your business: Have a
clear and defined strategy for your voice technology. Each business will have a
different set of goals based on their user and business needs. Having an institutional
strategy will help deliver the right business outcomes.
2. Customize your voice technology to meet the needs of your customer: A patient will
have a different expectation when interacting with a voice technology to find a
doctor vs book a vacation. Understand how users interact with voice systems in
different scenarios. Industry and specific business insights will dictate how the
technology needs to be designed and leveraged.
3. Remove barriers to interactions: Make the interactions with the voice technology
without any barriers. Any initial barriers or first-time interaction glitches can lead to
the loss of customers. Ensuring that the customer interactions are seamless will
build trust and improve the overall experience.
Acknowledgments. We would like to acknowledge our staff, our leadership, our board and our
Digital Innovation and Consumer Experience (DICE) Group team members who actively support
us in our quest for creating solutions that improve lives.
References
1. Newville, L.: Development of the Phonograph at Alexander Graham Bell’s Volta Laboratory.
http://www.gutenberg.org/files/30112/30112-h/30112-h.htm
2. Weizenbaum, J.: Computer Power and Human Reason: From Judgment to Calculation, p. 2,
3, 6, 182, 189. W.H. Freeman and Company, New York (1976)
3. Pearl, C.: Designing Voice User Interfaces: Principles of Conversational Experiences, 1st edn.
O’Reilly, Sebastopol (2016)
4. Pradhan, A., Mehta, K., Findlater, L.: Accessibility came by accident: use of voice-controlled
intelligent personal assistants by people with disabilities (2018)
5. Gerson, P., Sita, K., Zhu, C., Ehsani, J., Klauer, S., Dingus, T., Simons-Morton, B.: Distracted
driving, visual inattention, and crash risk among teenage drivers. Am. J. Prev. Med. 56(4),
494–500 (2019)
6. Lamming, M., Flynn, M.: Forget-Me-Not, intimate computing in support of human memory.
In: Proceedings of FRIEND21 1994 International Symposium on Next Generation Human
Interface (1994)
7. Portet, F., Vacher, M., Golanski, C., et al.: Pers. Ubiquit. Comput. 17, 127 (2013)
8. Kinsella, B.: U.S. Smart speaker ownership rises 40% in 2018 to 66.4 million and Amazon
echo maintains market share lead says new report from Voicebot (2019). https://voicebot.ai/
2019/03/07/u-s-smart-speaker-ownership-rises-40-in-2018-to-66-4-million-and-amazon-
echo-maintains-market-share-lead-says-new-report-from-voicebot/
9. Kinsella, B.: U.S. Smart display user base grew 558% in 2018 and more than doubled in
second half of the year, Amazon holds two-thirds market share (2019). https://voicebot.ai/
2019/03/07/u-s-smart-display-user-base-grew-558-in-2018-and-more-than-doubled-in-
second-half-of-the-year-amazon-holds-two-thirds-market-share/
Pre-emptive Culture Mapping: Exploring
a System of Language to Better Understand
the Abstract Traits of Human Interaction
Timothy J. Stock1,2(&) and Marie Lena Tupot1
1
scenarioDNA inc., New York, NY, USA
{timstock,marielena}@scenariodna.com
2
Parsons School of Design, The New School, New York, NY, USA
[email protected]
Abstract. A sustainable and ethical future, rich with technology, requires an
abstract human framework of ideation in order to understand the reality of user
experience (UX) and the context design will live in. The layer between the
human and object needs to be better understood – early on. We too quickly
replace one interaction with another. However, critical human traits often remain
hidden until the product is in situ. Their appearance can have terrible conse-
quences. These traits, and the behavioral codes that underscore them, can be
understood early on by looking at language.
Keywords: Applied sociology Epistemology Semiotics Strategic
foresight Systems thinking Culture mapping Human factors
1 Kicking the Machine Is a Sign of Discord
After a deadly 2017 crash between a destroyer and an oil tanker, the US Navy [1] is set
to replace its touchscreen controls with mechanical ones. Mapping the culture of users
could have gotten ahead of the design decision that led to a software interface versus an
analog one.
We tend to make faster decisions when it comes to software because software is
more easily mutable, but missteps happen where hardware and technology meet. It
looks as though I am measuring my health. But is it functioning that way?
Industry leaders have begun to question whether AI devices should have anthro-
pomorphic traits. Should we feel comfortable “kicking a machine” [2] that has
empathetic eyes? Are we encouraging bad behavior in children engaging with
anthropomorphic devices? Feeling the need to “kick the machine” means we missed
something human along the way. Innovation must lead with a holistic view of
human/machine interaction.
Consider facial recognition technology. Amidst global tension, developers are
scrambling to rethink their role as a tool for good. Human anxiety leads to disruptive
behavior. Protesters in China, for example, have been hiding behind masks and
umbrellas, using burner cellphones, and paying for transit in cash. Protesters also have
been cutting down lampposts with electric saws and wearing apparel designed [3] to
trick photo recognition software.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 567–572, 2020.
https://doi.org/10.1007/978-3-030-39512-4_88
568 T. J. Stock and M. L. Tupot
Such paradigms of human to machine have not visibly been in play until recently.
They’re becoming apparent in political arena, such as Hong Kong, as we witness
people attempting to regain a sense of self amid the machines.
There are a number of UX models currently in play. User experience designer Jesse
James Grant authored The Elements of User Experience [4] in 2002. Fogg [5] founded
the Behavior Design Lab at Stanford University where his 2009 Fogg Behavioral
Model emphasizes motivation, ability and trigger. And, in 2014, entrepreneur Eyal
detailed his model [6] in Hooked: How to Build Habit-Forming Products. Each is valid,
but none address the human meaning behind behavior.
2 Deeper Abstraction Leads to Emotional Design Intelligence
Our attempts at UX efficiency establish a process of expectation. It’s important that
expectations meet the intent of the user. Hidden biological response and human
expression attempt to mitigate chaos when it is not met.
This process of exploring abstraction looks at speculative design in a way that
better informs the designer about the public in question. The more a designer knows,
the more can be built into prototyping to allow for natural, open-ended design – before
permanent final design decisions are ever made.
The cultural system that represents the process of adaptation can be plotted as a
system of language that reflects the dynamics between the concrete and abstract worlds.
Craving for such emotional intelligence requires that we expand our binary world into
an abstract space for which only the human brain has the capacity.
The matrix diagram (Fig. 1) is based on a patented method of analyzing and
classifying open and closed corpora into discrete behavioral archetypes that can be
visualized as a system of signifiers within a semiotic square. It classifies and quantifies
linguistic signifiers along three principal axes. The x-axis represents relationship to self
Fig. 1. Culture mapping matrix, scenarioDNA inc., U.S. Patent No. US9002755B2.
Pre-emptive Culture Mapping: Exploring a System of Language 569
from analytical to expressive. The y-axis represents relationship to society from
affirming societal norms and values to resisting societal norms and values. The z-axis
represents time. This framework provides an ideological systematic view of dynamics
and patterns that emerge in expressed language, including words, images and gestures.
By looking at the linguistics behind the emergency tasks within the responsibility
of US Navy personnel in the case of the destroyers, one might have found that the code
of control defined the input task. The imprecision of a touchscreen would be coun-
terintuitive to the task. Present-day touchscreens do not yet imply control and efficiency
on a New York City Metrocard machine. They certainly do not define control on a US
Navy ship. A behavioral code of control would live, as one might expect, within the
policy space of a culture map – where a residual future exists until design can exceed
the human context and expectation of control.
Unlike the code of control for Navy personnel, protesters in Hong Kong are trying
to regain ownership over self. The code of ownership falls into the symbolic quadrant
of the matrix, within the disruptive quadrant. It’s a human reaction to a societal state of
discord. We need to be knowing what certain facets of interaction mean to people.
Artist Salome Asega [7] is the partnerships director of Powrplnt, a center for
affordable technology-meets-art workshops and also an instructor of speculative design
at Parsons School of Design. Asega asserts, “Having an arts or humanitarian lens in
technology centers people and communities…it can offer an entry point for people to
better understand the more complex aspects of technology.”
The design industry has been discussing issues of problematic design for decades.
“In the case of Apollo 13, the lesson (among others) learned was that “Ehh, I’m sure
it’ll be fine’ is not a solid design philosophy,” writes Smith for Engineering.com [8].
This was 1970. Don Norman [9], author of Design of Everyday Things and a former
Apple VP, discusses his investigation of the 1979 nuclear power accident at 3 Mile
Island. Norman recalls, “We decided that the operators were really quite intelligent…
except that the design was so bad that if you wanted to cause people to make errors,
you couldn’t have done a better job.” We still have not learned.
Even the contemporary theory of agile design has lost its way so deeply that
debates about its usefulness persist. Author of the Agile Manifesto [10], Arthur Fowler
says “a lot of what is being pushed is being pushed in a way that… goes really against a
lot of our precepts… The team should not just choose the process that they follow, but
they should be actively encouraged to continue to evolve it and change it as they go.”
A radical rethink of the framework of human experience is needed now. Designers
must consider the inevitable undoing of their design and a requisite for adaptability.
3 Function Is the Cognitive Framework
The behavioral codes that underlie function are what we truly need to understand before
we redesign access to function. Presently, design UX takes precedence. The expectation is
that the human will conform to the more aesthetically pleasing interface. For certain, a
human being will accept trade-offs in their own interest. In actuality, design is modifying
human behavior to the existing capability of technology. It begets a string of incremental
limitations and human frustration. Our expectation is rooted in everything going right.
Yet, things go wrong all the time, especially when technology is involved.
570 T. J. Stock and M. L. Tupot
The critical nature of understanding abstract traits of human behavior becomes clear
when we investigate our aging population and the implications of lost cognition and
physical ability. “The world is designed against the elderly,” writes Norman in Fast
Company [11]. Norman is 83. He continues, “Do not think that thoughtful design is
just for the elderly, or the sick, or the disabled. [Inclusive design] helps everyone. Curb
cuts were meant to help people who had trouble walking, but it helps anyone wheeling
things: carts, baby carriages, suitcases… As Kat Holmes points out in her book Mis-
match, all of us are disabled now and then.”
Instead, the current speculative design framework around aging is about medical
devices and warehousing. Yet, they have not lost the qualities that make them human.
For them, robotic nurses will not be a well-received future. Getting to the human is the
purpose of understanding the cycle of dissent and acceptance. This becomes a
framework about humanity – whether we are discussing aging or cars.
There are times when we seem to be getting it right. The behavioral codes in play at
startup Voyage [12] are safety and vitality. The company operates a fleet of low-speed
autonomous vehicles in two retirement communities, one near San Jose, California, and
the other north of Orlando, Florida, both called The Villages. The typical disadvantages
of autonomous vehicles, “slow speeds, overly cautious driving maneuvers, simplistic
routing,” in the context of retirement communities becomes an advantage meeting the
senior need for safety. Seniors also earn bragging rights as early adopters of new
technology and hold equity stake. For now, this works.
We can replace certain traits, such as the ability to drive, one with less human
attributes, without much thought. We can give it back in the form of mobility and pride
– human traits. However, our technical capabilities are expanding exponentially. We
need to be aware of how replacing traits reshapes the human itself. We will always lose
something in the exchange. The goal should be to better the human, not create a more
obedient machine. When we are discussing human-centered design, we can make no
mistake that we are designing interventions to better serve people’s needs.
4 The Exponential Trajectory of Technology Calls Us to Task
Moore’s Law [13] is the observation that the number of transistors in a densely inte-
grated circuit doubles about every two years. This law became a forecasting model
used to predict the growth of technology. The linearity of a forecasting model based on
materiality alone leads to exponential distortion. The exponential trajectory creates
behavioral unpredictability. The power of technology is seductive. We can’t resist it.
Think about twinning. The concept of twinning is returning to the design world,
except now twinning is digital. In 2018, Philips engineer van Houten [14], executive
vice president, CTO, Royal Philips cited duplicate modeling as a component that safely
landed the Apollo 13 mission, “This allowed engineers on the ground to model and test
possible solutions, simulating the conditions on board of Apollo 13.” It’s classic
mirroring, even as it moves to digital simulation for devices.
Pre-emptive Culture Mapping: Exploring a System of Language 571
Duplicate modeling becomes a different concept once we attempt to replicate a
human [15]. It’s a consideration that is entering the healthcare arena through the
prototyping of concepts such as Philips Digital Twins. The expected evolution is for all
the parts to come together to replicate the human. As some of us might know from
ailing friends and relatives, the human patient will always surprise us with the unex-
pected as they continue to pursue their most human of codes in spite of all limitations:
freedom. Freedom is hardwired for us, and it will trip up our most fluidly designed
replicant. The patient advised not to drive will drive. The patient who is monitored for
wandering will remove their monitors.
5 Human Nature Requires Cyclical Understanding
The challenge is to have the design remain open to human potential, rather than
structural control. It’s no surprise that most people engaged in “tech-mediated inti-
macy” prefer sexting with “adult” chatbots over sex robots [16]. They can imagine a
human at the other end. The behavioral codes will live in the emerging ritual space.
Perception is critical. However, culture will continually cycle. Already, technology is
making it harder to recognize reality. It will be human nature to push back.
In 1977, Stanley Milgram [17], a psychologist at Yale University, began replicating
the mind-body fusion found in Edmond Rostand’s 19th century play Cyrano de
Bergerac. His resulting cyranoid was a mechanism for speech shadowing. In his
experiments, people failed to detect that the words were not coming from the expected
source. They believed the illusion. But, over time, all it takes is one person to move
beyond their suspension of disbelief before we need to evolve our design. As designers,
we need to understand human traits and be ahead of that moment.
References
1. Fingas, J.: US Navy will scrap touchscreen controls on its destroyers. https://www.engadget.
com/2019/08/11/us-navy-drops-touchscreen-controls-for-destroyers/
2. Harrison, S.: Of course citizens should be allowed to kick robots. https://www.wired.com/
story/citizens-should-be-allowed-to-kick-robots/
3. Hu, J.C.: When will TJ Maxx sell anti-surveillance fashion? https://slate.com/technology/
2019/08/facial-recognition-surveillance-fashion-hong-kong
4. Garrett, J.J.: The Elements of User Experience. http://www.jjg.net/elements/
5. Stulberg, B.: How to Build Better Habits. https://www.outsideonline.com/2401883/how-to-
build-good-habits
6. Eyal, N.: Hooked: How to Build Habit-Forming Products. Penguin Books, London (2014)
7. King, A.: Meet Salome Asega, the multihyphenate artist working to diversify the tech world.
https://www.vogue.com/article/salome-asega-interview-powrplnt
8. Smith, V.: Great moments in engineering history: apollo 13. https://www.engineering.com/
DesignerEdge/DesignerEdgeArticles/ArticleID/15386/Great-Moments-in-Engineering-
History-Apollo-13.aspx
9. Hawk, S.: Chat with Don Norman. https://uxmastery.com/transcript-chat-don-norman/
572 T. J. Stock and M. L. Tupot
10. Cagle, K.: The end of agile: a rebuttal. https://www.forbes.com/sites/cognitiveworld/2019/
08/28/the-end-of-agile-a-rebuttal/#1fe4033c538a
11. Norman, D.: I Wrote the Book on User-Friendly Design. What I See today Horrifies Me.
https://www.fastcompany.com/90338379/i-wrote-the-book-on-user-friendly-design-what-i-
see-today-horrifies-me
12. Hawkins, A.J.: AV startup Voyage on how low speeds and older customers. https://www.
theverge.com/2019/9/12/20862659/voyage-self-driving-startup-funding-oliver-cameron
13. Feldman, M.: TSMC Thinks It Can Uphold Moore’s Law For Decades. https://www.
nextplatform.com/2019/09/13/tsmc-thinks-it-can-uphold-moores-law-for-decades/
14. van Houten, H.: The rise of the digital twin: how healthcare can benefit. https://www.philips.
com/a-w/about/news/archive/blogs/innovation-matters/20180830-the-rise-of-the-digital-
twin-how-healthcare-can-benefit.html
15. Copley, C.: Medtech firms get personal with digital twins. https://www.reuters.com/article/
us-healthcare-medical-technology-ai-insi/medtech-firms-get-personal-with-digital-twins-
idUSKCN1LG0S0
16. Ellis, E.G.: You are already having sex with robots. https://www.wired.com/story/you-are-
already-having-sex-with-robots/
17. Neuroskeptic: “Cyranoids”: Stanley Milgram’s Creepiest Experiment. http://blogs.discover-
magazine.com/neuroskeptic/2014/09/06/cyranoids-stanley-milgrams-creepiest-experiment
''Meanings'' Based Human Centered Design
of Systems
Santosh Basapur1,2(&) and Keiichi Sato2
1
Rush University Medical Center,
1750 West Harrison St., Chicago, IL 60612, USA
2
IIT Institute of Design, 3137 S Federal St., Chicago, IL 60616, USA
{basapur,sato}@id.iit.edu
Abstract. There is a renewed interest in Systems Thinking and Systems Design
in the practice and pedagogy of Design. Theoretical work to propose new design
theories or methods of design are creating discussion on first principles and/or a
general lack of it. Methods to map, model, understand the critical information
needed for design of complex systems are being sought by practitioners and
academics alike such as Organizational Semiotics and Ethical Value Based
Design practices. This paper posits a hybrid approach based in traditional
Human Systems Integration (HSI) approach and Organizational Semiotics. We
believe that creating ability to incorporate perceived meanings and values of
stakeholders will yield a much more empathetic and flexible design of systems.
Keywords: Systems Design Systems theory Semiotics Human Systems
Integration
1 Introduction
Recent years have seen a renewed interest in Systems Design. Focus on sustainability,
ethics, values and moral dilemmas in the advent of new technologies like self-driven
vehicles, AI based assistive agents in mobile devices and living rooms alike, has
brought to fore the need for more semantics-based design processes than ever before.
New design theories and methods of design [9] are creating opportunities for discussion
of first principles, or a lack of it. Human Centered Design Traditional Approaches to
HSI are that of Socio-Technical System Design (STSD) [20] and Macroergonomics [7].
User centered design is end-user centered and more often than not the complex rela-
tionship between service producer and service consumer gets reduced to end-user point
of view. Human centered design on the other hand attempts to keep all stakeholders’
needs in focus for the entire systems development process.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 573–579, 2020.
https://doi.org/10.1007/978-3-030-39512-4_89
574 S. Basapur and K. Sato
2 Overview of Systems Design Approaches
2.1 Human Factors Based Systems Design
McCormick and Sanders [13] defined Human Factors/Ergonomics as “The field of
human factors-referred to as ergonomics in Europe and elsewhere-deals with the
consideration of human characteristics, expectations, and behaviors in the design of the
things people use in their work and everyday lives and of the environments in which
they work and live. Over the years this broad definition has held well for Human
Centered Design practiced by HF professionals. But critique has always been that HFE
is about task, person and the fit. Hollnagel [8] and Wickens et al. [21] argue that the
nature of human factors works in 2010s is significantly different than the work done in
the 1950s and 60s. He suggests it is not enough to improve the performance and
physical well-being of people through systems design, because that kind of change can
be narrow in focus. Boy et al. summarize that the challenge is to better understand
socio-cognitive consequences due to shift from design of controls to management of
processes and outcome [2].
2.2 Human Systems Integration (HSI) Design Approach
HSI approach has gained traction and growth in multiple domains like Military,
Healthcare, Transportation and Robotic systems. “HSI is the integrated and compre-
hensive analysis, design and assessment of requirements, concepts and resources for
system manpower, personnel, training, environment, safety, occupational health, hab-
itability, survivability, and human factors engineering, with the aim to reduce total
ownership cost, while optimizing total mission performance” [1]. Other theories like
Socio-Technical System Design (STSD) [20] and Macroergonomics [7] compliment
the HSI view of systems design. These theories optimize integration of people and
systems (technical, information, engineering etc.) to produce optimal output for the
organization and quality of life for consumers and producers of services in these
systems.
2.3 Systemic Design and Circular Economies
Systemic design is led by sustainability focused systems designers. They are bringing
design thinking processes and frameworks to create systems thinking for design of
complex systems. They believe in developing, initializing, and infrastructuring capacity
for co-design and co-production with real stakeholders. Addressing challenges of
global trends of resource consumptions and achieving deeper transformations to
equitable futures requires cultivating woke networks in real world context - be it
technological, social, infrastructural, and/or politico-economic [16] (Fig. 1).
“Meanings” Based Human Centered Design of Systems 575
Fig. 1. Systemic design, by Birger Sevaldson, 2017, RSD conference
3 Semiotics as Basis for Complex System Design
Known as the “mathematics of humanities,” Semiotics is the science of the meaning of
signs, including words, images, sounds, odors, flavors, gestures and objects [3].
Semiotics provides a means for codifying objects, their representations and their
interpretation. It can also accommodate both engineering concepts and human
behaviors. For this research in particular the interest is in Organizational Semiotics
(OS) because HSI explains the larger system well but there is a lack of theoretical basis
to model the rest of the information pertaining to technical and social fiber of the
complex systems. Hence OS fills in the gap well (Fig. 2).
Fig. 2. Definition of Semiotics. Redrawn from [3]
3.1 Semiotics as Basis for Information Systems and Organization Design
Stamper and Liu [18, 19] described Organizational Semiotics (OS) as, “An organiza-
tion is defined as a system of social norms.” They suggest that a basic strategy of
coping with such complexity is “to find good criteria for partitioning the problem into
components that can remain, as far as possible, invariant as the other components
576 S. Basapur and K. Sato
change.” Stamper specified the semiotic ladder to help understand information (figure
below). OS has been successfully applied to Information Systems [10, 11] (Fig. 3).
Fig. 3. Stamper’s semiotic ladder. Source: [18, 19]
Semiotics, while elegantly, models what we take for granted in the representation of
the complex systems also causes rhetoric of being easy to apply. This is misleading.
There are not many case studies of application outside of Information Systems. Crit-
icism has been that it is a general-purpose tool and not a full-fledged analytical and/or
solution synthesis tool [3].
4 Semiotics Based Systems Life Cycle (SSLC)
A semiotics-based systems life cycle is proposed by the authors in this paper. Semiotics
informed systems lifecycle is shown in Fig. 4. The input informed by broad context
creates mapping between goals, design principles and the actual design. Once design is
implemented meanings become apparent through the outcomes of the system. Feed-
back informs the iterative cycle to self-improve over time.
Fig. 4. Semotics based System Life Cycle (SSLC)
“Meanings” Based Human Centered Design of Systems 577
Further procedural steps as mapped out in Fig. 5 below. The new holistic method
emphasizes how to integration of data yields insights using the meanings of actions and
activities of different Stakeholders. Also, different aspects of systems are detailed.
Fig. 5. Procedural flow of design method proposed
Application Method includes deliberate steps as follows:
Step 1. Generating Contextual Information and Meanings via analysis of:
a. Intent of the organization (business) that is providing the service
b. Technological trends and landscape of available technologies
c. Competitive Landscape of business
Step 2. Current System Analysis: Analysis should include at least
a. Analysis of Assets and structure of subsystems in overall objective system
b. Communication Flows and Information Flows
c. People’s movement flows
d. Material flows etc.…
Step 3. Infrastructural and Human Centered Semiotics Analyses: Social and
Technical analyses are critical to this methodology but in addition cultural, ethical
578 S. Basapur and K. Sato
and economic analyses are also crucial to help understand people, roles and
responsibilities.
a. Roles and Responsibilities network analysis, Social Structure and Hierarchy
b. Human Resources and its current policies and Current Training for personnel
c. Cultural analysis and ability to manage “change”
d. Information Architecture and Information System model
e. Task and Activities analysis - HTA
f. Technical products and services architecture analysis
g. Economic analysis of viability, desirability and feasibility of innovation
Step 4. Holistic Optimization: Working of the socio-economic-cultural structure
along with the technical system (IT, Information flow and access etc.) is optimized
with the “Intent” of the business organization in focus.
Step 5. Prototype System Design: Design principles derived from Step 1 along with
the Problems and Constraints are used to generate a “working prototype solution.”
Step 6. Design for Execution: This is the final executable design solution for the
system to work properly and iteratively improve over time. This also yields a
flexible architecture for the system to adjust itself to unforeseen challenges. This
design should well encapsulate:
1. Principles of Design espoused by the design team
2. Structure of entities and people that is sensible to organization goal
3. Executable Design and Plan
4. User Experience of services that is sustainable in quality over time
5. Feedback/Iteration loops in system design to ensure tolerance to ambiguity.
Acknowledgements. We would like to acknowledge feedback from reviewers. Funding for
various system design projects from multiple organizations.
References
1. Booher, H.: Handbook of Human Systems Integration. Wiley, Hoboken (2003)
2. Boy, G.A.: Human-centered design of complex systems: an experience-based approach. Des.
Sci. 3, e8 (2017)
3. Chandler, D.: Semiotics: The Basics. Routledge, Abingdon (2002)
4. Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for
distributed resource sharing. In: 10th IEEE International Symposium on High Performance
Distributed Computing, pp. 181–184. IEEE Press, New York (2001)
5. Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan
Kaufmann, San Francisco (1999)
6. Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The physiology of the grid: an open grid
services architecture for distributed systems integration. Technical report, Global Grid
Forum (2002)
“Meanings” Based Human Centered Design of Systems 579
7. Hendrick, H.W., Kleiner, B.M.: Macroergonomics: Theory, Methods, and Applications.
Lawrence Erlbaum Associates Publishers, Hillsdale (2002)
8. Hollnagel, E.: Human factors/ergonomics as a systems discipline? “The human use of human
beings” revisited. Appl. Ergon. 45(1), 40–44 (2014)
9. Jones, P.H.: Systemic design principles for complex social systems. In: Social Systems and
Design, pp. 91–128. Springer, Tokyo (2014)
10. Liu, K.: Semiotics in Information Systems Engineering. University Press, Cambrigde (2000)
11. Liu, K., Li, W.: Organisational Semiotics for Business Informatics. Routledge, Abingdon
(2014)
12. May, P., Ehrlich, H.C., Steinke, T.: ZIB structure prediction pipeline: composing a complex
biological workflow through web services. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.)
Euro-Par 2006. LNCS, vol. 4128, pp. 1148–1158. Springer, Heidelberg (2006)
13. McCormick, E.J., Sanders, M.S.: Human Factors in Engineering and Design. McGraw-Hill
Companies, New York (1982)
14. National Center for Biotechnology Information. http://www.ncbi.nlm.nih.gov. Accessed 10
Feb 2015
15. Norman, D.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books,
New York (2005)
16. Sevaldson, B.: RSD2. In: Proceedings of Relating Systems Thinking and Design (RSD2),
2013 Symposium, Oslo, Norway, 4–5 October 2013 (2013). http://www.slideshare.net/
RSD2/intro-lecture-forproceedings. Slide 23
17. Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol.
Biol. 147, 195–197 (1981)
18. Stamper, R., Liu, K.: Organisational dynamics, social norms and information systems. In:
Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences,
January 1994, vol. 4, pp. 645–654. IEEE (1994)
19. Stamper, R., Liu, K., Hafkamp, M., Ades, Y.: Understanding the roles of signs and norms in
organizations-a semiotic approach to information systems design. Behav. Inf. Technol. 19
(1), 15–27 (2000)
20. Taylor, J.C., Felten, D.F.: Performance by Design: Sociotechnical Systems in North
America. Prentice Hall, Upper Saddle River (1993)
21. Wickens, C.D., et al.: Engineering Psychology and Human Performance. Psychology Press,
London (2015)
A Systematic Review of Sociotechnical System
Methods Between 1951 and 2019
Amangul A. Imanghaliyeva(&)
Institute for Infrastructure and Environment, Heriot-Watt University,
Edinburgh EH144AS, UK
[email protected], [email protected]
Abstract. A Sociotechnical System (STS) perspective offers potentially valu-
able insights into problems in a wide range of domains of society. The devel-
opment of STS-based methods provide practical help for organisations when
planning and evaluating new forms of designing the work systems. The current
study provides an up-to date review of STS-based methods in seven different
domains including (1) specialist STS methods, (2) the human factor, human
behaviour and ergonomics, (3) safety, risk, reliability and accident prevention,
(4) complexity, (5) information systems, information technology and the internet
of things, (6) organisational design, work system and management studies and
(7) design science. Based on a systematic review, 106 STS-based methods have
been identified and grouped by relevant domains and summarised to form a
unified list of the available methods for use by anyone involved in the system
and organisational design.
Keywords: Sociotechnical System Sociotechnical System methods
Sociotechnical System methods review System design Organisational design
1 Introduction
Clearly, there are numerous conventional methods (i.e. Hazard and Operability Anal-
ysis (HAZOP) [1], Error Analysis [2], ‘What if’ analysis [3], etc.) which are mainly
focused on either human or technical aspects of the system. However, as studies
indicate, these methods are not complete and in many cases do not solve complex
(social and technical) system problems entirely. Moreover, due to the overly narrow
consideration of the causes of system/project failures, these methods are not appro-
priate to justify fully the occurrence of system failure or accidents in the systems.
As systems become more complex, there is an increasing need for new methods.
Perhaps, this relates directly to the fact that 21st century systems require us to under-
stand the increased interconnectivity and complexity between systems and their ele-
ments. All these considerations have implications for development of STS methods [4].
STS methods have been developing since the initial establishment of the Tavistock
Institute up to the present day and have been manifested a wide range of STS-based
methods in different sections [5]. Depending on the work environment of specific
companies or organisations, STS methods achieved various levels of success in the
early days of the emergence of STS. Although, there are quite a large range of methods
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 580–587, 2020.
https://doi.org/10.1007/978-3-030-39512-4_90
A Systematic Review of STS Methods Between 1951 and 2019 581
based on STS, they are not widely adopted. Some of the reasons cited for the lack of
use of such methods are that they do not always fit the organisational culture, and that
knowledge about these techniques is not available to relevant personnel in organisa-
tions [6]. Thus, it is important to organise the STS-based methods applied in different
domains into one unified framework (see Table 1).
2 Method
The review of the methods was carried out over two stages. The first stage was a
literature review of existing STS-based methods in different domains. The second
involved a screening process of the identified STS methods.
2.1 Stage 1: Literature Review of Existing STS Methods in Different
Domains
A large-scale literature search was undertaken to reveal all the STS-based methods used
in the areas of (1) specialist STS, (2) the human factor, human behaviour and ergo-
nomics, (3) safety, risk, reliability and accident prevention, (4) complexity, (5) infor-
mation systems, information technology and the internet of things, (6) organisational
design, work system and management studies and (7) design science. Four databases
(Google Scholar, ScienceDirect, Web of Science and Scopus) were searched for articles
published between the dates 1 January 1951 to 1 July 2019, inclusive. All instances of
STS methods extant in the publicly available and peer-reviewed knowledge-base,
including relevant scientific journals, standard textbooks were reviewed. STS-based
methods were searched for by topics and keywords.
The following targeted keywords/phrases were searched for:
1. Sociotechnical system methods
2. Socio-technical system design methods
3. Sociotechnical system approaches
4. Sociotechnical system tools
5. Sociotechnical system design methods
6. Socio-technical system design tools
All frameworks, models related to STS and also traditional methods of work
organisation or those methods which were solely focused on only one aspect (either
social or technical) of the system were excluded from the review. Thus, the main
emphasis was focused on purely STS-based methods which have a definite, estab-
lished, logical or systematic plan of action.
2.2 Stage 2: Screening of STS Methods
Apart from the previously mentioned keywords, some additional criteria were set for
selection of the STS methods. Before these methods were subject to further analysis, a
screening process was employed to remove any methods (tools, techniques) that were
not suitable for review with respect to their use in the design and evaluation of systems.
582 A. A. Imanghaliyeva
The screening procedure of the STS methods was conducted through setting the fol-
lowing specific criteria:
• Methods had to be focused on both social and technical aspects of the system
• The methods had to be applied at least in one case study
• The methods had to be publicly freely available
• The methods had to align with at least one of the 20 sociotechnical principles [7].
3 STS Methods Review in Different Domains
106 STS-based methods were identified in seven different domains, as explained in the
following list below.
(1) Specialist STS methods are useful for redesigning existing STS as well as for
designing a new STS. The main target of these methods is focused on improving
and developing the work system, work processes and working environment. The
cases when these methods have been used report success in the dimensions of
either productivity or quality of working life. In particular, the use of the Socio-
technical System design technique [8] enables the success of the design procedure
by meeting both the human needs and technical requirements.
(2) Human factor, human behaviour and ergonomics: It is widely acknowledged
that availability of sociotechnical critical systems still rely on human operators,
both through humans’ reliability and ability to handle unexpected events ade-
quately [9]. Therefore, it does make sense to have methods for assessing human
errors and human factors in complex STS. For instance, Systemic-theoretic
accident model and processes based Human Reliability Analysis (STAMP based
HRA) considers human-related accidents and identifies contributing factors and
causal relationships [10]. This method was developed as a result of the inadequacy
of sequential accident models in analysing human errors within STS.
(3) Safety, risk, reliability and accident prevention: The majority of STS methods
in this area tends to serve as a tool for identifying, mitigating and prioritising the
risks that can dramatically degrade the performance of the entire system.
Traditional approaches to risk assessment, such as THERP (Technique for Human
Error Rate Prediction) [11], TRACER (Technique for the Retrospective and
Predictive Analysis of Cognitive Errors) [12] etc. are typically reductionist in
nature [13], focusing on individual tasks and technologies rather than the system
as a whole [4, 14]. However, STS present unique challenges for safety manage-
ment and risk assessment [4, 15]. Therefore, there is need to look at ST-PRA
(Sociotechnical probabilistic risk assessment) [16, 17] that provides a tool for
analysing risks associated with human error, non-standard practices, and proce-
dural noncompliance, any of which can degrade safety and performance of the
system [17].
(4) Complexity: Numerous research studies from various disciplines have explored
the field of ‘complexity’. Systems Engineering and Psychology are the specific
disciplines that deal with complexity. Complexity can be perceived as the
A Systematic Review of STS Methods Between 1951 and 2019 583
existence of complex, chaotic, interrelated, disordered, dynamic systems full of
uncertainty and variety of complex elements’. In this context, STS-based methods
serve as complex problem-solving tools. These methods can provide insights in
the different complex processes as work systems change.
(5) Information systems (IS), information technology (IT) and internet of things:
The link between STS and IT/IS is rooted in the work of Enid Mumford in the
1960s, who realised that an essential part of an IS, which is constantly neglected
by system engineers and system designers, regards the organisational aspects of
the work system. This idea led to the development of STS-based methods in IS
and IT.
(6) Organisational design, work system and management studies: STS-based
methods in work system and organisational design are very similar to specialist
STS methods, in terms of their wide application in designing a system or
redesigning the existing system. ‘Systems Scenarios Tool’ [18], Sociotechnical
Tools’ [6] are examples of this. However, they are additionally are focused on
optimising the distribution of functions between humans and equipment.
(7) Design Science: These methods allow users to be creative and act as designers.
For instance, ‘Business Origami’ [19] is a value-centred design method that
enables design teams to paper-prototype a physical representation of a system.
Table 1. 106 identified Sociotechnical System (STS) based methods in different domains
between 1951 and 2019
Identified Sociotechnical System-based methods in different domains between 1951–2019
№ Methods № Methods
(1) Specialist Sociotechnical System 54 Stabilize, evaluate, identify,
methods standardize, monitor, implement, and
control (SEISMIC) methodology
1 Socio-technical Allocation of 55 System dynamics (SD)
Resources (Star) toolkit
2 Socio-technical roadmapping (5) Sociotechnical System-based methods in
method Information Systems, Information Technology
and Internet of Things
3 Socio-Technical System Design 56 Actor Network Theory (ANT)
technique
4 Sociotechnical System Variance 57 Ariadne
Analysis
(2) Sociotechnical System-based methods 58 End-user participation in HIT
on Human Factor, Human Behaviour and development (EUPHIT) method
Ergonomics
5 Cognitive Work Analysis (CWA) 59 Future Technology Workshop
(FTW) method
6 Cognitive Work Analysis Design 60 GenderMag method
Toolkit (CWA-DT)
(continued)
584 A. A. Imanghaliyeva
Table 1. (continued)
Identified Sociotechnical System-based methods in different domains between 1951–2019
№ Methods № Methods
7 Goals, Operators, Methods and 61 Mumford: Effective Technical and
Selection Rules (GOMS) Human Implementation of Computer-
based Work Systems (ETHICS)
8 Human Error Hazard and 62 Organisational Requirements
Operability (HE-HAZOP) analysis Definition for IT Systems methodology
(ORDIT)
9 Human Factors Analysis and 63 Semi-structured, socio-technical
Classification System (HFACS) modelling method (SeeMe)
10 Human Factors Hazard Operability 64 Socio-technical evaluation matrices
(HF-HAZOP) analysis (STEM)
11 Hierarchical Task Analysis (HTA) 65 Socio-technical walkthrough (STWT)
12 Sociotechnical Systems Analysis 66 Soft systems (SSM) methodology)
(3) Sociotechnical System-based methods 67 Tesseract: A sociotechnical
on Safety, Risk, Reliability and Accident dependency browser tool
Prevention
13 Accident Analysis Technique (6) Sociotechnical System-based methods in
(AcciMap) Organisational Design, Work system and
Management Studies
14 Accident evolution and barrier 68 Barrier and operational risk analysis
function (AEB) (BORA)
15 Causal Analysis based on Systems 69 Barrier and operational risk analysis -
Theory (CAST) Release (BORA-Release)
16 Cognitive Reliability and Error 70 Bostrom and Heinen: STS design
Analysis Method (CREAM) method
17 Control change cause analysis 71 Bounded Socio-Technical Experiments
(3CA) (BSTE)
18 Failure Mode and Effect Analysis 72 Delphi study
(FMEA)
19 Fault Tree Analysis (FTA) 73 KOMPASS
20 Functional Resonance Analysis 74 Macroergonomic Work Analysis
Method (FRAM) (MA) method
21 Health and Safety Executive 75 Norske Statesbaner (NSB)
(HSG245)
22 Health Care Failure Mode and 76 Paper-based tools
Effect Analysis (HFMEA)
23 HS-Ras 77 Pava: STS design method
24 Integrated use of cognitive mapping 78 Process mapping techniques
techniques and the Analytic
Hierarchy Process (AHP) method
25 Integrated Procedure of Incident 79 RAMESES
Cause Analysis (IPICA)
26 Integrated safety investigation 80 Sociotechnical design (SD) method
methodology (ISIM)
(continued)
A Systematic Review of STS Methods Between 1951 and 2019 585
Table 1. (continued)
Identified Sociotechnical System-based methods in different domains between 1951–2019
№ Methods № Methods
27 Interdisciplinary approach: 81 Socio-technical experiments
STAMP-based HRA considering
causality
28 Management oversight and risk tree 82 Socio-Technical Method for Designing
(MORT) Work Systems (STMDWS)
29 Man-Technology-Organisation 83 Sociotechnical Tools: Scenarios Tool
(MTO) Analysis and Job Design Tool
30 Multilinear events 84 Socio-Technical Security Analysis
sequencing (MES)
31 NETworked hazard analysis and 85 Systems-theoretic accident model and
risk management system (NET- processes (STAMP)
HARMS)
32 Occupational Accident Research 86 Systems Scenarios Tool (SST)
Unit (OARU)
33 Organisations Risk Influence Model 87 Task allocation method (between and
(ORIM) among humans and machines)
34 Resilience Engineering 88 Task Analysis for Error Identification
(RE) approach (TAFEI)
35 Requirements Engineering with 89 Work accidents investigation technique
Scenarios for a User-centred (WAIT)
Environment (RESCUE)
36 Risk Situation Awareness Provision 90 Work System Method (WSM)
(RiskSOAP) methodology
38 Sociotechnical probabilistic risk 91 Analysis, Diagnosis and Innovation
assessment (ST-PRA) (ADInnov) method
39 Specification Risk Analysis tool 92 Cost effectiveness analysis
40 Systematic cause analysis technique 93 Ethnographic method
(SCAT)
41 SystemTheoretic Process Analysis 94 Inquiry cycle
for Privacy (STPA-Priv)
42 Systems Theoretic Process Analysis 95 Multi-Method or mixed approaches
(STPA)
43 TRIPOD 96 PreMiSTS
(4) Sociotechnical System-based methods 97 Social Network Analysis (SNA)
in Complexity
44 Agent-based modelling technique (7) Sociotechnical System-based methods in
(ABM) Design Science
45 Causal tree method (CTM) or INRS 98 Business Origami
method
46 COIM: An object-process based 99 Cognitive Mapping
method
47 EAST (Event analysis for systemic 100 Cognitive Walkthrough
teamwork) method
(continued)
586 A. A. Imanghaliyeva
Table 1. (continued)
Identified Sociotechnical System-based methods in different domains between 1951–2019
№ Methods № Methods
48 Macroergonomic analysis and 101 Collage
design (MEAD)
49 MAP method 102 Contextual Design
50 Modified Sociotechnical Systems 103 Heuristic Evaluation
(MoSTS) methodology
51 Morphological Analysis (MA) 104 Meta-design
52 SEIPS-based process modelling 105 Participatory design (PD)
method
53 Sociotechnical scenario (STSc) 106 Storyboards
method
4 Conclusion
The current study provided an up-to date review of STS-based methods in seven
different domains including (1) specialist STS methods, (2) the human factor, human
behaviour and ergonomics, (3) safety, risk, reliability and accident prevention,
(4) complexity, (5) information systems, information technology and the internet of
things, (6) organisational design, work system and management studies and (7) design
science.
Based on the review, 106 STS-based methods have been identified and grouped by
relevant domains and summarised to form a unified list of the available methods for use
by anyone involved in organisational and system design.
Future research will be directed into a practical application of these methods.
Acknowledgments. This work was financed by the Republic of Kazakhstan’s JSC ‘Centre for
International Programs’ which operates under the Ministry of Education and Science of the
Republic of Kazakhstan, and administers the Kazakhstan President’s Bolashaq Scholarship
Program, that granted a Ph.D. scholarship to Amangul Imanghaliyeva to support this study.
References
1. Lawley, H.G.: Operability studies and hazard analysis. Chem. Eng. Prog. 70(4), 45 (1974)
2. Srinivas, S.: Error recovery in robot systems, Doctoral dissertation, California Institute of
Technology (1977)
3. Crawley, F., Tyler, B.: Hazard Identification Methods. Institute of Chemical Engineers,
Rugby (2003)
4. Waterson, P., Robertson, M., Cooke, N.J., Militello, L., Roth, E., Stanton, N.: Defining the
methodological challenges and opportunities for an effective science of sociotechnical
systems and safety. Ergonomics 58(4), 565–599 (2015)
5. Baxter, G., Sommerville, I.: Socio-technical systems: from design methods to systems
engineering. Interact. Comput. 23(1), 4–17 (2011)
A Systematic Review of STS Methods Between 1951 and 2019 587
6. Axtell, C., Pepper, K., Clegg, C., Wall, T., Gardner, P.: Designing and evaluating new ways
of working: the application of some sociotechnical tools. Hum. Fact. Ergon. Manuf. Serv.
Ind. 11(1), 1–18 (2001)
7. Imanghaliyeva, A.A., Thompson, P., Salmon, P., Stanton, N.A.: A synthesis of sociotech-
nical principles for system design. In: International Conference on Applied Human Factors
and Ergonomics, pp. 665–676. Springer, Cham (2019)
8. Taylor, J.C.: The human side of work: the socio-technical approach to work system design.
Pers. Rev. 4(3), 17–22 (1975)
9. De Carvalho, P.V.: Ergonomic field studies in a nuclear power plant control room. Prog.
Nucl. Energy 48(1), 51–69 (2006)
10. Rong, H., Tian, J.: STAMP-based HRA considering causality within a sociotechnical system
a case of Minuteman III missile accident. Hum. Fact. J. Hum. Fact. Ergon. Soc. 57(3), 375–
396 (2015)
11. Swain, A., Guttman, H.: Handbook of human reliability analysis with emphasis on nuclear
power plant applications, US Nuclear Regulatory Commission Technical report
(NUREG/CR-1278). Google Scholar|Crossref (1983)
12. Shorrock, S.T., Kirwan, B., Scaife, R., Fearnside, P.: Reduced vertical separation outside
controlled airspace. In: Third Annual Conference on Aviation Safety Management, May
2000
13. Stanton, N., Salmon, P.M., Rafferty, L.A.: Human Factors Methods: a Practical Guide for
Engineering and Design. Ashgate Publishing, Ltd., Farnham (2013)
14. Stanton, N.A.: Hierarchical task analysis: developments, applications, and extensions. Appl.
Ergon. 37(1), 55–79 (2006)
15. Flach, J., Carroll, J., Dainoff, M., Hamilton, W.: Striving for safety: communicating and
deciding in sociotechnical systems. Ergonomics 58(4), 615–634 (2015)
16. Marx, D.A., Slonim, A.D.: Assessing patient safety risk before the injury occurs: an
introduction to sociotechnical probabilistic risk modelling in health care. Qual. Saf. Health
Care 12(Suppl. 2), ii33–ii38 (2003)
17. Battles, J.B., Kanki, B.G.: The use of socio-technical probabilistic risk assessment at AHRQ
and NASA. In: Probabilistic Safety Assessment and Management, pp. 2212–2217. Springer
London (2004)
18. Hughes, H., Clegg, C., Bolton, L., Machon, L.: Systems scenarios: a tool for facilitating the
socio-technical design of work systems. Ergonomics 60, 1–17 (2017)
19. Hanington, B., Martin, B.: Universal Methods of Design: 100 Ways to Research Complex
Problems, Develop Innovative Ideas, and Design Effective Solutions. Rockport Publishers,
Mountain View (2012)
Designing a Safety Confirmation System
that Utilizes Human Behavior in Disaster
Situations
Masayuki Ihara(&), Hiroshi Nakajima, Goro Inomae,
and Hiroshi Watanabe
NTT Service Evolution Laboratories,
1-1, Hikari-no-oka, Yokosuka, Kanagawa 2390847, Japan
[email protected]
Abstract. This paper introduces a use case to discuss how to design services
for disaster situations as well as to elucidate the development and evaluation of
our safety confirmation system, which utilizes human behavior in disaster sit-
uation. The success of the safety confirmation system based on information
sharing among evacuation centers depends on the inter-center movement of
humans. In order to know whether the safety confirmation service would work
effectively or not after a massive disaster, we conducted a survey that examined
how many evacuation centers were visited after the East Japan Earthquake and
the periods over which the evacuees visited the centers. As a result, we con-
firmed that many evacuees were still visiting centers 72 h after the earthquake,
making our technology effective in responding to a disaster.
Keywords: Design System complexity Human behavior Disaster Safety
confirmation
1 Introduction
After the East Japan Earthquake in 2011, communication services were suspended for a
long time in many areas. This is because not only network cables but also buildings
with network facilities in many areas were damaged by the tsunami. As regards mobile
services, an official report states that 29 thousand routing points were damaged. It took
one and half months to completely fix the damage to both fixed telephone services and
mobile services. 80% of mobile service areas were fixed in one week. In dire disaster
situations such as earthquakes, we should have alternative communication channels
able to provide safety confirmation for at least 72 h after a disaster, a period known to
be important in saving human life.
We have developed a safety confirmation technology that can be used even when
Internet connection is unavailable and in disaster situations. This paper shows how to
design services for disaster situations as well as introducing the development and
evaluation of our safety confirmation system, which was developed as software on
smartphones; it utilizes human behavior in disaster situations.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 588–593, 2020.
https://doi.org/10.1007/978-3-030-39512-4_91
Designing a Safety Confirmation System that Utilizes Human Behavior 589
2 Requirements for Services in Disaster Situations
When designing services for disaster situations, it is important to be able to handle the
absence of Internet connectivity and the inability to install new application software.
After a disaster, Internet connection cannot be guaranteed so the services should not
require Internet connectivity. To meet this requirement, we decided that the only
communication available would be local Wi-Fi. Our implementation assumes that
lightweight servers and databases are set in local Wi-Fi routers and user mobile devices
access those servers or database via the routers. One problem with applications for
disaster situations is that the installation of emergency applications is not possible after
a disaster. Accordingly, our proposal is completely web-based and uses HTML5.
3 DTN-Based Safety Confirmation
To meet the requirements above, the server, database and local Wi-Fi in an evacuation
center enable evacuees to use the safety confirmation service on their own smartphone.
To expand the service area, our safety confirmation application makes it possible to
share safety confirmation information among evacuation centers via a DTN (Delay
Tolerant Network). The DTN is rather unique as it uses the inter-center movement of
humans to realize information sharing rather than regular wired or wireless links [1].
Evacuees can confirm the safety of family or friends without putting themselves to
further personal risk by visiting many evacuation centers.
4 Developed Technologies
We have developed a DTN-based safety confirmation system by implementing a safety
confirmation application on a resilient information sharing platform that enables multi-
hop Wi-Fi communication [2].
4.1 Resilient Information Sharing Platform
This platform (See Fig. 1) is realized as an HTML5 web-based application hosted by
local Wi-Fi routers, so it satisfies the above two requirements. After a disaster, many
people will frequently move between evacuation centers. This means that the number
of mobile devices connected to Wi-Fi in the evacuation center changes frequently. Our
platform can accept such changes as it uses a javascript-based decentralized coopera-
tion control program on each device to collect, store, and distribute information among
devices. The stored information is synchronized among the devices accessing the Wi-Fi
routers in order to distribute the latest information. The device carried by a person who
visits many evacuation centers stores the information held by the current router. When
the person connects the device to the Wi-Fi network in another evacuation center, the
device uploads its stored information for access/download by other devices and then
downloads what information the center holds.
590 M. Ihara et al.
Fig. 1. Resilient information sharing platform.
4.2 Safety Confirmation Application
Fig. 2. Safety confirmation application.
This application is triggered simply by bringing the device into the Wi-Fi cover area
and launching a web browser. Availability is achieved by using a URL redirection
technique to automatically navigate users to a service portal menu. When they use the
application for the first time, they need to register their own information such as name
and/or a phone number. Figure 2 shows a screenshot and how to use the application.
5 Evaluation
The success of information sharing among evacuation centers depends on the inter-
center movement of humans. The system is complex as it treats humans in emergency
situations as system components. Our field evaluation confirmed the basic acceptance
Designing a Safety Confirmation System that Utilizes Human Behavior 591
of the service concept (86% of subjects reacted positively). However, confirmation as
to whether the safety confirmation service would work effectively or not after a massive
disaster is needed. Moreover, to enhance the algorithm controlling information circu-
lation, we need more facts about peoples’ responses. Thus, we conducted a survey that
examined how many evacuation centers were visited after the East Japan Earthquake.
We investigated the number of evacuation centers each subject visited in the days after
the disaster. This survey, which took the form of a web-questionnaire, was responded
to by 500 subjects (males and females in their 20s to 70s) who experienced the
earthquake. We also investigated the period over which the evacuees visited the
centers.
6 Results
6.1 The Number of Visited Centers
There were two types of responses to the earthquake; “center-evacuees” left their
dwellings and entered an evacuation center while “home-stays” remained in their
dwelling. In this survey, there were 218 center-evacuees and 282 home-stays. Figure 3
(a) and (b) show the number of evacuation centers visited by center-evacuees and
home-stays, respectively. As shown in Fig. 3(a), 50.5% of center-evacuees visited at
least one other evacuation center. 41.8% of the center-evacuees visited multiple cen-
ters. As shown in Fig. 3(b), 28.4% of home-stays visited at least one center. 35.0% of
the home-stays visited multiple centers. Note that one evacuee visited 20 centers, the
highest number reported.
(a) Center evacuees (n=218) (b) Home-stays (n=282)
Fig. 3. The number of evacuation centers visited.
From the above results, we confirmed the existence of evacuees who visited
multiple centers. More center-evacuees visited other centers than did home-stays. This
is because most center-evacuees came from areas completely devastated and so the
centers in those areas were rather overwhelmed. Safety confirmation was a stronger
requirement, as was the need for daily necessities.
592 M. Ihara et al.
6.2 Days Spent Visiting Centers
In order to know the period over which our technology would be effective after a
disaster, we investigated how many days each evacuee spent for visiting other centers.
This investigation targeted 74 evacuees who visited multiple centers. Figure 4 shows
evacuees who finished visiting centers on the day of earthquake and the next day were
4.1% and 16.2% respectively (79.7% of the evacuees were still visiting centers up to
three days after the earthquake). These results mean that many evacuees were still
visiting centers 72 h after the earthquake. 50% of the evacuees kept visiting centers for
more than a week. This result confirms that our technology is a potential alternative to
regular communication channels, which may be disconnected for a week.
Fig. 4. Days spent visiting centers. (n = 74).
From the above results, we confirmed that the period of visiting other evacuation
centers was long enough to support the effectiveness of our technology. However,
when this technology is used in the future, evacuees may think that they do not need to
visit other centers since they can confirm safety state without visiting other centers.
Accordingly, we conducted an additional survey to know whether evacuees visited
other centers for purposes different from gathering safety information. As a result,
55.4% and 52.7% of the 74 evacuees visited other centers for daily necessities and for
news about the disaster such as level of damage, respectively. These results mean that
evacuees will visit other centers even if our safety confirmation service is used.
On the other hand, since our technology depends on evacuees’ visiting actions,
there may be some delay in safety confirmation. Evacuees may think it is faster to drive
a car to other centers. We investigated the means used to visit other centers after the
earthquake. As a result, 75.7% of the 74 evacuees visited other centers on foot. This is
because many roads were destroyed by the earthquake or tsunami and they could not
use a car due to the lack of gasoline.
7 Related Works
The HCI- related community has been long interested in the nature of people’s
behaviors as well as their interaction with ICT services during disasters. These include
work to support professionals in emergency situations [3] as well as research on how
Designing a Safety Confirmation System that Utilizes Human Behavior 593
non-professionals utilize various ICT services in various disasters [4, 5]. Most studies
targeting on a disaster situation are analyzing human’s online behaviors and social
media [6, 7]. Sekimoto’s study treats the estimation of human movement in a disaster
situation [8]. Song introduced a modeling of evacuation behavior [9]. There is a study
of utilizing a DTN to create a technology for disaster situations [10].
8 Conclusion
In this paper, we introduced a use case to discuss how to design services for disaster
situations as well as introducing the development and evaluation of our safety con-
firmation system. The contribution of our study lies in its provision of a use case of a
complex system that utilizes human behavior as a component. Human behaviors such
as safety confirmation should be targeted as both services and possible system com-
ponents. We should investigate the characteristics of human behaviors in advance and
should reflect the result in system design.
References
1. Delay-Tolerant Networking Architecture. IETF RFC 4838 (2007)
2. Ihara, M., Seko S., Miyata, A., Aoki, R., Ishida, T., Watanabe, M., Hashimoto, R.,
Watanabe, H.: Towards more practical information sharing in disaster situations. In:
Yamamoto S. (ed.) HIMI 2016. LNCS, vol. 9735. Springer, Cham (2016)
3. Pettersson, M., Randall, D., Helgeson, B.: Ambiguities, awareness and economy: a study of
emergency service work. In: CSCW 2002, pp. 286–295. ACM Press (2002)
4. Palen, L., Lie, S.B.: Citizen Communications in crisis: anticipating a future of ICT-supported
public participation. In: CHI 2007, pp. 727–736. ACM Press (2007)
5. Semaan, B., Mark, G.: ‘Facebooking’ towards crisis recovery and beyond: disruption as an
opportunity. In: CSCW 2012, pp. 27–36. ACM Press (2012)
6. Kogan, M., Palen, L., Anderson, K.M.: Think local, retweet global: retweeting by the
geographically-vulnerable during hurricane sandy. In: CSCW 2015, pp. 981–993. ACM
Press (2015)
7. Reuter, C., Ludwig, T., Kaufhold, M.A., Pipek, V.: XHELP: design of a cross-platform
social-media application to support volunteer moderators in disasters. In: CHI 2015,
pp. 4093–4102. ACM Press (2015)
8. Sekimoto, Y., Sudo, A., Kashiyama, T., Seto, T., Hayashi, H., Asahara, A., Ishizuka, H.,
Nishiyama, S.: Real-time people movement estimation in large disasters from several kinds
of mobile phone data. In: UbiComp 2016, pp. 1426–1434. ACM Press (2016)
9. Song, X., Zhang, Q., Sekimoto, Y., Horanont, T., Ueyama, S., Shibasaki, R.: Modeling and
probabilistic reasoning of population evacuation during large-scale disaster. In: KDD 2013,
pp. 11–14. ACM Press (2013)
10. Reina, D.G., Askalani, M., Toral, S.L., Barrero, F., Asimakopoulou, E., Bessis, N.: A survey
on multihop ad hoc networks for disaster response scenarios. J. Distrib. Sens. Netw. 2015,
647037 (2015)
Designing Ethical AI in the Shadow
of Hume’s Guillotine
Pertti Saariluoma1,2(&) and Jaana Leikas1,2
1
University of Jyväskylä, Jyväskylä, Finland
[email protected]
2
VTT Technical Research Centre of Finland Ltd., Espoo, Finland
Abstract. Artificially intelligent systems can collect knowledge regarding
epistemic information, but can they be used to derive new values? Epistemic
information concerns facts, including how things are in the world, and ethical
values concern how actions should be taken. The operation of artificial intelli-
gence (AI) is based on facts, but it require values. A critical question here
regards Hume’s Guillotine, which claims that one cannot derive values from
facts. Hume’s Guillotine appears to divide AI systems into two ethical cate-
gories: weak and strong. Ethically weak AI systems can be applied only within
given value rules, but ethically strong AI systems may be able to generate new
values from facts. If Hume is correct, ethically strong AI systems are impossible,
but there are, of course, no obstacles to designing ethically weak AI systems.
Keywords: AI ethics Design Hume’s Guillotine
1 Introduction
After one of the many “winters” regarding artificial intelligence (AI) and intelligent
technologies, they are again high on research agendas. The performance capacity of
computers, their speed, the size of their memory, and the sizes of the data masses
available all have increased. Consequently, intelligent systems, including AI, robotics,
and autonomous technologies, all currently have serious practical applications. How-
ever, improvements in performance capacity will lead to many new challenges con-
cerning the interactions between people and these systems and their effects on human
life [1, 2].
Technologies have always been developed to aid people in their everyday tasks [3].
More than fifty years ago, the early computers began to replace mechanical office
machines. Since then, large numbers of people have changed the ways they work.
Computers changed office work step by step and eventually the organization of firms
large and small. Today, examples of this can be found in practically all areas of human
work and personal life. They illustrate the effects of the so-called fourth technology
revolution on people’s lives.
Intelligent systems are like any other technology in that they support people in their
personal and work lives, making it possible to improve the quality of human life.
However, these systems differ qualitatively from traditional, non-intelligent technolo-
gies in their abilities to accomplish many tasks that require intelligent information
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 594–599, 2020.
https://doi.org/10.1007/978-3-030-39512-4_92
Designing Ethical AI in the Shadow of Hume’s Guillotine 595
processing. The focus is no longer on how to improve performance in the sensory-
motor dimension, but rather on how to improve it in tasks that have required human
higher-information processes, such as thinking [4, 5]. Intelligent systems have the
capacity to improve human performance and even to replace people with machines in
many tasks that require intelligent information processing [6], for example, navigating
ships, which people have done for millennia because they have the capacity to carry out
the required information-processing operations.
Today, intelligent systems can not only play games, but can participate in trans-
portation; help to proofread texts and perform many other natural-language processing
tasks; sell and buying stocks; analyze massive datasets; make legal decisions; and
participate in political, medical, and military tasks in various ways. Thus, intelligent
systems are becoming an essential part of modern work and personal life.
However, intelligent information processing includes not only processing data and
working with facts. In intelligent information processing and action, ethics and values
often are essential. In daily life, which is gradually becoming supported in various ways
by intelligent systems, many actions entail ethical components. For example, can an
intelligent financial system reject loans from startup firms that have for economic
reasons located in the inexpensive neighborhoods of cities? Can we accept machines
that are tacitly racist? Should expensive medicines be used to care for elderly people
whose expected lifespans are comparatively short? Values and morals cannot be put
aside when designing tasks and operations for intelligent systems.
A brief look at the problems of applied ethics can be enlightening [7, 8]. For
example, such concepts as bioethics, medical ethics, and the ethics of businesses and
professions illustrate that even practical actions regarding human life are always value
laden. Given that values are always incorporated into actions, should we expect
intelligent systems to also process ethical actions? And if so, how can they do this?
Understanding the facts does not necessarily ensure the exercise of values, as facts and
values are not qualitatively identical: Facts are true or false and epistemic in nature, and
in contrast, values are either more or less justified, and thus are ethical or moral.
2 Epistemic and Ethical Information
Intelligent technologies are designed to conduct human operations in the real world.
They can perform such tasks as driving cars or ships, coordinating the operations of
various machines in factories, analyzing medical data, and teaching people. Very often,
such tasks have two aspects: the facts regarding how things are (epistemic) and the
values regarding how things should be and what should be done (ethical). Thus, the
epistemic information represents the states of affairs and entails propositions that have
truth-value.
Ultimately, computers can be perceived as bits, electrical states, and the possible
states of these bits. Today, vast datasets are typical, and evolving sensor systems are
rapidly increasing the data available. This means that compared to earlier times, the
amount of epistemic knowledge is becoming huge and people do not lack data. Each
day, this knowledge base has new applications that are meaningful to human lives.
596 P. Saariluoma and J. Leikas
However, facts are not values. Moral information is of a different type. As men-
tioned, facts tell how things are and if things are true. Facts have definable references
and meanings. There can also be procedures that tell whether the facts refer to
something else and are true in relation to it. In contrast, norms do not have verifiable
references in the same sense that facts do. For example, it is impossible that a sensor
can indicate who is a good person.
Norms are guidelines regarding how people should behave. Using intelligent sys-
tems, it is possible to register people’s behavior. In this sense, norms have factual
properties. But, can values be processed like factual information? The fact that the
Nazis killed people in concentration camps does not indicate that their behavior was
moral. Undoubtedly, some of these Nazis accepted the prevailing moral system, but the
rest of the world hardly did so. The crucial question is whether morals are directly
derivable from facts, which would enable intelligent systems to analyze human life and
derive new moral norms. Is there a danger that an intelligent system, independently of
human involvement, would decide that concentrations camps were moral?
3 Hume’s Guillotine
In designing intelligent moral systems, one important question is associated with the
two types of information. This question can be formulated as follows: how should the
processing of epistemic and moral information be associated and implemented in
designing the intelligence of intelligent systems. The crucial difficulty of this question
was identified by David Hume more than 200 years ago.
The problem is called Hume’s Guillotine [9], which states that one cannot infer
values from facts. In terms of the problem in question, this means that one cannot
derive moral information from epistemic information. Because the operations of
intelligent systems are based on epistemic information, Hume’s Guillotine poses the
question of what it means that intelligent systems process values-based information. To
clarify this problem, the scientific and design communities should specifically study the
relationship between epistemic and moral information.
Hume’s Guillotine was presented in Hume’s main work, “A Treatise on Human
Nature” [9]. He did not see any way in which values could be based on facts and
reasons. To him, values were based more on emotions. He wrote about this in a very
straightforward manner: “It is impossible that the distinction betwixt moral good and
evil can be made by reason” [9]. To Hume, actions had two main and independent
aspects: reason and passion. For him, morals belonged to the aspect of passions and
processing facts to that of reason.
Hume’s Guillotine is a concept of surprising consequence in the design of ethical
AI. Researchers argue that AI technologies make it possible to design ethical algo-
rithms and consequently ethical machines. Furthermore, some have seen it as possible
to use the data generated by ethical machines to develop modern societies. Both of
these positions appear to contradict Hume’s Guillotine.
Intelligent systems are machines, which can have different electrical states. How
can an electrical state, which definitely must be analyzed in terms of natural laws,
Designing Ethical AI in the Shadow of Hume’s Guillotine 597
understand human ethics and values? Computers operate with facts and epistemic
knowledge, whereas Hume’s Guillotine separates facts and values.
What really happens when computers and intelligent systems must process moral
information? Can they generate new values in the same sense as new facts? It seems
that the task is more complicated, because ethical machines should link facts and
values. Thus, if one hopes to process values autonomously using modern AI systems, it
is necessary to circumvent Hume’s Guillotine in some way. A machine can generate a
factual state of information, such as a chess position, but it cannot generate whether this
position is good or bad. This must be defined by people in their ethical discourses.
4 Morally Weak and Strong AI
In principle, it is easy to provide ethical rules for intelligent machines. For example, if
some action causes pain to a human being, an intelligent machine can be programmed
to avoid such an action. In practice, it is not straightforward to know what can be
painful, but there are no principled obstacles to programming ethical norms into
machines and making them follow these norms.
In contrast, Hume’s Guillotine implies that machines cannot derive new norms
from facts. Analysis of big data can find new types of data combinations, but how can
this knowledge be used to generate new values? For example, if Substance X is
dangerous for a small group of people suffering from a rare illness, should an intelligent
system conclude that all people should give up using Substance X?
A natural alternative to solving the ethical problem caused by Substance X is the
following: The prevailing ethical and normative position is decided using human social
and political discourses, and rules such as “Avoid situations involving Substance X” or
“Allow Substance X” can be implemented in the memories of intelligent systems.
Thus, people could decide, renew, and implement ethical rules and their content in an
intelligent system. However, this type of system would be morally weak and non-
autonomous, as it could not independently decide the contents of new moral norms.
A morally weak AI system would follow the value systems provided by people and
would analyze factual situations but would not manipulate its system of moral rules.
This type of system can also be called a morally narrow AI system.
A morally strong AI system could be realized by a system that generated new
ethical principles and rules based on the analysis of a given factual situation. Such a
system would be able to analyze people’s everyday lives and make new rules based on
those analyses. This type of machine learning in a system could independently improve
human value systems. A morally strong AI system could have the capacity to reflec-
tively manipulate its own ethical rules and their practical consequences and could even
generate new moral rules. The validity of Hume’s Guillotine is crucial in thinking
about whether it would be possible to construct a morally strong AI system.
598 P. Saariluoma and J. Leikas
5 Aporia of AI Design
Hume’s Guillotine appears to be a watershed between the morally weak and the
morally strong AI. If Hume’s Guillotine is true, morally strong AI is impossible. If
Hume’s Guillotine is incorrect, morally strong AI can be constructed. The latter pos-
sibility is very appealing, as it would make it possible to massively analyze the con-
ditions of people’s lives and let intelligent technologies decide how people should
behave and what the laws should be like, thereby obviating the need for political
discourse to make laws. Thus, Hume’s Guillotine presents an important problem for all
AI designers.
This problem is serious. AI systems can effectively analyze the state of affairs in a
society and thus promote understanding of that society, its states, and the states of the
sectors and groups within it. However, this would not be helpful unless it were possible
to design intelligent systems that could turn facts into values. Unless this is possible,
human involvement is necessary.
One can imagine several ways of solving the problem of Hume’s Guillotine.
Herein, we present one solution: Because it is evident that one can construct morally
weak AI, it might be possible to construct systems of morally weak systems and
eventually reach a morally strong AI. However, the number of morally weak systems
required might be huge or even infinite and the task thus impossible.
Partial solutions can be tested using the Turing test, enabling study of whether
morally weak AI systems behave like people. It would also be possible to test whether
the systems behaved in a more ethical manner than people did in a given ethical
situation. It would even be possible to test whether ethical AI can perform better than
people can. The latter would be a worthy goal, but it does not solve the main problem:
deriving values from facts. Indeed, it is possible to have systems with ethical content,
but that is not a solution to the actual problem of morally strong AI.
From the point of view presented herein, it appears that it is impossible to construct
strong ethical AI. One cannot solve the general problem of strong ethical AI unless one
cannot find a way to get machines to derive values from facts. Assuming that Hume’s
Guillotine is correct, this solution does not work, and humankind should leave ethical
content and ethical thinking to moral AI systems. This would mean that human beings
must recognize the action situations that require ethical regulation and must construct
basic value systems. Furthermore, political and ethical discourse must be conducted to
validate ethics. Finally, values must be implemented in the form of social norms or
laws. In light of our present understanding of facts and values, it is possible to construct
any number of ethically operating systems, but not morally strong AI.
In sum, constructing morally weak AI is easy. Machines can follow given values.
However, Hume’s Guillotine seems to prevent people from developing genuinely
autonomous moral AI and intelligent systems. Such systems would be very practical in
designing and developing life in societies, but there is no clear path from the present to
autonomous, morally strong AI. It may be that a large number of morally weak systems
would enable us to reach a moral AI that is sufficiently strong to advance society, but
these systems’ moral information would always need to be implemented by people and
decided by human moral discourses. This is the basic aporia of ethical AI.
Designing Ethical AI in the Shadow of Hume’s Guillotine 599
References
1. Ford, M.: Rise of Robots. Basic Books, New York (2014)
2. Tegmark, M.: Life 3.0: Being Human in the Age of Artificial Intelligence. Random House,
New York (2017)
3. Bernal, J.: Science in History. Penguin Books, Harmondsworth (1969)
4. Saariluoma, P.: Four challenges in designing human-autonomous system design processes. In:
Williams, A., Sharre, P.D. (eds.) Autonomous Systems: Issues for Defense Policy-Makers.
NCI, Haag (2015)
5. Saariluoma, P., Cañas, J., Leikas, J.: Designing for Life. Palgrave Macmillan, London (2016)
6. Newell, A., Simon, H.: Human Problem Solving. Prentice-Hall, Englewood-Cliffs (1972)
7. Boven, W.R.: Engineering Ethics. Springer, London (2009)
8. Stahl, B.C.: Social issues in computer ethics. In: Floridi, L. (ed.) The Cambridge Handbook of
Information and Computer Ethics, pp. 101–115. Cambridge University Press, Cambridge
(2010)
9. Hume, D. A.: Treatise of Human Nature. Dent, London (1738/1972)
A Counterattack of Misinformation: How
the Information Influence to Human Being
Subin Lee and Ken Nah(&)
IDAS, Hongik Univeristy, 57 Daehakro, Jongno-gu, Seoul, Republic of Korea
[email protected], [email protected]
Abstract. This study aims to investigate how the issue of misinformation is
expanded in academic fields. Using the bibliometric data in the recent decade,
we employ network analysis on author keywords to reveal the evolution of the
academic network. Our results show that the academic areas in terms of
misinformation-relevant topics have been grown rapidly and the author key-
words network turn out to be complicated. This paper indicates that the theme of
misinformation becomes more critical in academia, which leads us to estimate
the impact thereof in the real world.
Keywords: Misinformation Deepfake Fake news Network analysis
Design thinking
1 Introduction
Thanks to internet distribution, it is a fact that information technology is increasingly
becoming an integral part of contemporary life. Recently, the information shows us
reveal one’s true character, which is closely related to such as misinformation and
deepfake.
It is a fact that misinformation is continuously disseminated. In spite of increasing
efforts to improve new computational algorithms to detect the misinformation, people
who live in the information age should pay attention to recognize whether the infor-
mation is true or not. The information can be easily distorted by developing cutting-
edge technology such as artificial intelligence (AI). As such, unexpected social and
ethical problems emerge because of the uncertainty of information. For example, the
fabrication of Barack Obama’s video known as a deep fake posted on BuzzFeed in
2017 warned potential risks that the reality in the world is no longer real [1, 2].
Thus, the experts have been explained the era of the information age peoples are
need to consider a high level of education of understanding fake news which is closely
related to future skills.
For this reason, the keywords—such as deep fake, misinformation and fake news—
have been playing a vital role in the society of academia. In this regard, the purpose of
this study is to present the fake news, deep fake and misinformation that have been
raised on the keywords of the social issue by using author keyword data, which can
show the current key issue of user and academia.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 600–604, 2020.
https://doi.org/10.1007/978-3-030-39512-4_93
A Counterattack of Misinformation 601
2 Background
It is well-known fake news, deepfake and misinformation have been a rise in a new
phenomenon in social issues. The issue of deepfake technology hit the headlines in
2017 when the University of Washington researchers released a paper describing how
they had created a fake video of President Barack Obama. This technology has been
invented by researcher Dr. Suparson Suwajanakorn who agrees that while admitting the
technology had the potential for misuse. In addition, recent research comes from a
cyber-security company named, ‘Deeptrace’, the researchers have been found 14,698
deepfake videos online, compared with 7,964 in December 2018 [2].
This article explains that an alarming surge in the creation of the so-called issue of
misinformation that have been leading to a boost of scientific research in the society of
academia.
Figure 1 shows the trend of how frequently the search terms—misinformation, fake
news, and deepfake—has been entered into Google’s search engine during the last
decade. It turns out that the topics’ growth has increased after 2016. These trends
support that the issues of misinformation become important in the information age.
Fig. 1. Google trend of issues on misinformation
3 Data
This study collected bibliometric data in the recent decade from 2009 to 2018. The
search terms to define the scope of misinformation were “misinformation”, “fake
news”, and “deepfake” in Web of Science. The sample this study collected was
composed of 1,867 articles.
602 S. Lee and K. Nah
4 Descriptive Statistics
The amount of academic papers has increased, as shown in Fig. 2. The annual per-
centage of the growth rate was 21.89. It describes that the misinformation-relevant
fields have expanded enormously. In particular, the change in the growing slope was
rapid in 2017. Figure 3 shows the growth of the top 10 author keywords over the last
decade. Overall, the growing slope has increased continuously. It turns out that the
number of author keywords, especially fake news and social media, increases rapidly in
2017. These results are similar to the trend of misinformation based on Google Trend.
Fig. 2. Annual academic production
Fig. 3. Yearly occurrences of top author keywords
A Counterattack of Misinformation 603
5 Future Analysis Strategies
This study employs network analysis in terms of author keywords by relying on the
concept of co-occurrence investigation. Author keywords are selected by the authors
themselves, which describes best the contents of their work [3]. This allows for cap-
turing an article’s content with minimizing additional effort to interpret the intention of
authors. Thus, many scholars use the author keywords as a unit of analysis to construct
a co-word network.
In this paper, the co-occurrence matrix is derived from the occurrence matrix of
order m n. The objects of n denote the author keywords we want to analyze. There
are m documents on which the co-occurrence analysis is based. This occurrence matrix
was a binary matrix. If author keyword j occurs in document i, the element in the ith
row and jth column equals one, otherwise zero. We can derive the co-occurrence matrix
of order n n by multiplying the occurrence matrix and its transpose. The co-
occurrence matrix of author keywords is used for the network analysis. This paper
divides two periods in five-year windows to examine the change of the network. T1 is
defined as the first period from 2009 to 2013, and T2 is regarded as the second period
from 2014 to 2018 in this study. Using the co-occurrence matrix, we will analyze
network centrality including degree centrality, betweenness centrality, and closeness
centrality, which lead us to understand how the academic network of misinformation-
relevant theme has been evolved.
6 Conclusion
The changing trend of the author keywords is regarded as the attention of not only
academia but also the real world at a given time. As described in our preliminary
results, the dimension of misinformation has been diversified across the academic
topics such as social media, fake news, and false memory. For this reason, we should
consider the importance of misinformation issues carefully.
According to the work of Tresky and Kahneman in 1974, the tendency of the
intuitive mind is more likely to be so simplified that ordinary people can have a
cognitive bias shaped by their rough guess. Their findings argue that people’s bias can
be more rendered by a cognitive system than personal emotion [4].
The rationality of human beings in the deluge of information needs to be redefined.
This is because, unfortunately, the rhetoric—human beings are rational—is a myth,
which supported by several studies based on the results ranging from psychology to
neuroscience. Human beings have evolved toward forming schema to react to a cer-
tain situation adequately. However, the process of forming the schema can be different
between online and offline conditions. Information processing is likely to be distorted
within the online space such as the Internet. Hence, personal cognition may be limited
by overlooking unexpected online events people have not experienced. For this reason,
our future study will re-define design thinking methods which is related cognitive style
in problem sloving that help to understand misinformation including fake news and
deepfake, in which visually [5, 6].
604 S. Lee and K. Nah
References
1. He Predicted the 2016 Fake News Crisis. Now He’s Worried About an Information Apocalypse.
BuzzFeedsNews. https://www.buzzfeednews.com/article/charliewarzel/the-terrifying-future-
of-fake-news/Article/2018/He-Predicted-the-2016-Fake-News-Crisis-Now-He’s-Worried-
About-an-Information-Apocalypse. Accessed 14 Oct 2019
2. Goole Makes Deep Fakes to Fight Deep Fakes. https://www.bbc.com/news/technology-
49837927/Article/2019/Google-Makes-Deep-Fakes-to-Fight-Deep-Fakes. Accessed 14 Oct
2019
3. Lee, P.-C., Su, H.-N.: Investigating the structure of regional innovation system research
through keyword co-occurrence and social network analysis. Innovation 12, 26–40 (2010)
4. Tversky, A., Kahneman, D.: Judgment under uncertainty: heuristics and biases. Science 185,
1124–1130 (1974). https://doi.org/10.1126/science.185.4157.1124
5. Kim, E., Kim, K.: Cognitive styles in design problem solving: insights from network-based
cognitive maps. Des. Stud. 40, 1–38 (2015)
6. Tversky, B., Suwa, M.: Thinking with sketches
Effects of Increased Cognitive Load on Field
of View in Multi-task Operations Involving
Surveillance
Seng Yuen Marcus Goh1(&), Ka Lon Sou1, Sun Woh Lye2,
and Hong Xu1
1
School of Social Sciences, Nanyang Technological University,
48 Nanyang Ave, Singapore 639818, Singapore
[email protected]
2
School of Mechanical and Aerospace Engineering, Nanyang Technological
University, 50 Nanyang Ave, Block N3, Singapore 639798, Singapore
Abstract. Many operations, such as air traffic control which requires simulta-
neous surveillance and communication, involve multi-tasking. Yet, research on
effects of multi-tasking on visual processing tend to revolve around driving. As
such, similar effects on functional field of view in multi-tasking operations
involving surveillance, while considering stimuli parameters that might be
implicated, remain a gap in understanding. In this study, we investigated the
effects of the presence of a secondary task and stimuli parameters (size and
contrast) on the response accuracy and response time for stimuli appearing in the
visual field. Mixed analyses of variance revealed that response time, but not
accuracy, was affected by the engagement in multiple tasks. An interaction
between the parameters of the presented stimuli signals the need to consider
external factors in multi-task operations. Implications and future directions are
discussed.
Keywords: Functional field of view Multi-task Size Contrast Visual
performance Human errors
1 Introduction
With technological advancements, traditional manual labour becomes more automated.
Humans are thus expected to carry out different tasks from before, such as supervision
of unmanned vehicles systems [1]. For example, the task of the driver in an autono-
mous vehicle will be switched to surveillance and standby in cases of malfunction or
unknown situations of the automated systems. In various industries, such advancement
presumably allows humans to engage in multiple tasks simultaneously (i.e. multi-
tasking). However, multi-tasking may increase productivity, but could come with a
trade-off.
Among the many industries, operational industries involving surveillance tasks are
particularly implicated. Surveillance is a visual task that is often coupled with another
communication task. For instance, air traffic controllers (ATCOs) monitor the airspace
while concurrently managing communication with the pilots of the aircrafts in their
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 605–611, 2020.
https://doi.org/10.1007/978-3-030-39512-4_94
606 S. Y. M. Goh et al.
sectors: giving instructions (to the pilots) and taking requests (from the pilots) on
adjusting the aircrafts’ altitude or velocity to prevent conflicts or collisions. With air
traffic density predicted to increase in the future, ATCOs’ work will get more cogni-
tively intense and complicated [2].
In the current study, we are particularly interested in the functional field of view
(FFoV). FFoV is the area in the human visual field where stimuli can be detected and
processed [3]. Multi-tasking can degrade FFoV in two ways, resulting in either tunnel
vision [4] or general interference [5]. These degradation theories have been studied in
the context of usage of cellular phone while driving [6–8]. However, instead of testing
participants on a particular driving task, we will examine the effects of multi-tasking on
a more general visual task, such that the results would be generalisable to surveillance
tasks in other industries, such as that of ATCOs.
This study investigates the effect of task requirements – whether one is engaging in
multiple tasks or a single task – on one’s visual task performance. Engaging in multiple
tasks is hypothesised to result in poorer visual task performance. Extending from the
context of air traffic control, we identified size and contrast as the parameters of the
stimulus in the visual task as they are particularly pertinent to such contexts in
surveillance. Visual task performance is expected to decrease with both stimulus size
and contrast. Moreover, the questions arise whether there are interactions among these
factors. We will investigate these hypotheses and questions in the current study.
2 Methods
The study is a 2 (task: multi-task vs. single-task) 2 (stimulus size: large vs.
small) 2 (stimulus contrast: high vs. low) mixed-subject design1. The dependent
variables to measure visual task performance are the response accuracy and response
time.
2.1 Participants
Twenty participants (10 females, average age: 23.35 years), with normal or corrected-
to-normal vision, consented to participate the study. Each participant was compensated
for their time and effort. This study was approved by the Institutional Review Board
(IRB) at Nanyang Technological University, Singapore.
Participants were randomly allocated to the experimental (multi-task) and control
(single-task) groups. Within each group, the participants were tested on all 4 possible
combinations of large/small stimulus size and high/low stimulus contrast. The details of
these conditions are explained in Procedure.
1
Cognitive resource depletion was the between-subject variable while stimulus size and stimulus
contrast were the within-subject variables.
Effects of Increased Cognitive Load on Field of View 607
2.2 Materials
The study was ran using MATLAB [9]. Using the functions in Psychophysics Toolbox
extensions [10–12], a fixation cross was presented at the centre of the screen. The test
stimulus, a Gabor patch grating, was presented at various locations on the screen
followed by a beep sound as an indication of response for the participants.
2.3 Procedure
A randomised blocked design was adopted in the experiment. Each participant went
through four different blocks; the blocks differed in terms of the parameters of the
stimulus presented: large stimulus with high contrast, large stimulus with low contrast,
small stimulus with high contrast, and small stimulus with low contrast (Fig. 1 shows
the various stimuli). The order of the four blocks were randomised across all partici-
pants without repeat of any particular order. The test stimulus appeared at various
locations on the screen, followed by a beep. Participants had to judge whether the
grating was tilted to the left or the right by 45°, by hitting the “<” and “>” keys on the
keyboard respectively.
The experimental group, in addition to the above orientation judgment task, has to
recite numbers backwards by multiples of three, starting from one hundred. If they were
to make a mistake, they would have to start from the beginning. If they managed to recite
smoothly till zero before the end of the trials, they will repeat the task from ninety-nine.
The purpose of this secondary task was to simulate multi-tasking in various operations.
Participants received sufficient training to practice the tasks before they were tested.
Fig. 1. Various Gabor patch gratings used as stimuli in the experiment. Large (21 pixels
21 pixels) stimulus with high contrast (top left), large stimulus with low contrast (top right), small
(11 pixels 11 pixels) stimulus with high contrast (bottom left), and small stimulus with low
contrast (bottom right)
3 Results
A multivariate analysis of variance (MANOVA) was ran to see the overall differences,
with the response accuracy and time as the dependent variables, and task, stimulus size,
and stimulus contrast as the independent variables. Main effects of Stimulus Size
608 S. Y. M. Goh et al.
(F(2, 17) = 32.99, p < .001) and Stimulus Contrast (F(2, 17) = 24.73, p < .001), as
well as the interaction effect of Stimulus Size Stimulus Contrast (F(2, 17) = 8.03,
p = .004) were significant in the multivariate analysis. However, none of the effects
involving Task were significant (ps > .05).
The 2 2 2 mixed analysis of variance (ANOVA) was then followed, with task
as the between-subject variable, and stimulus size and stimulus contrast entered as the
within-subject variables. The ANOVA were run separately for each dependent variable
– response accuracy and response time. The response accuracy and time were nega-
tively correlated but not significant (r = −.290, p = .215).
3.1 Response Accuracy
The ANOVA on response accuracy found significant main effects of stimulus size (F(1,
18) = 69.26, p < .001, η2 = .794) and stimulus contrast (F(1, 18) = 49.53, p < .001, η2
= .733), but not task (F(1, 18) = .114, p = .740, η2 = .006). Response accuracy in
detecting large stimulus (M = .848, SE = 0.029) was significantly higher than in
detecting small stimulus (M = .674, SE = 0.022). Similarly, the accuracy in detecting
high contrast stimulus (M = .809, SE = 0.025) was significantly higher than in detecting
low contrast stimulus (M = .712, SE = 0.024).
The interaction between stimulus size and stimulus contrast was also significant
(F(1, 18) = 15.03, p = .001, η2 = .455). The significant interaction between stimulus
size and stimulus contrast may be related to the different effects of contrast in large vs.
small stimulus size. For large stimulus size, there was a non-significant difference in
accuracy, t(19) = .527, p = .604, d = .117, between stimulus of high contrast (M = .854,
SD = .119) and stimulus of low contrast (M = .841, SD = .161). On the other hand, for
small stimulus size, there was a significant difference in accuracy, t(19) = 7.05,
p < .001, d = 1.58, between stimulus of high contrast (M = .764, SD = .126) and
stimulus of low contrast (M = .583, SD = .092). No other interactions were found to be
significant.
The participants’ response accuracy are summarised in Fig. 2.
Fig. 2. Summary of participants’ response accuracy
Effects of Increased Cognitive Load on Field of View 609
3.2 Response Time
All three main effects (task, stimulus size and contrast) are significant in response time.
Participants in the experimental group showed a significantly higher response time
(M = 1.05, SE = 0.092) compared to participants in the control group (M = .716, SE =
0.092), F(1, 18) = 6.52, p = .020, η2 = .266. Furthermore, there was a significantly
lower response time in detecting large stimulus (M = .786, SE = 0.065) compared to
small stimulus (M = .976, SE = 0.081), F(1, 18) = 7.35, p = .014, η2 = .290. Similarly,
response time for detection of stimulus with high contrast (M = .812, SE = 0.062) was
significantly faster than for stimulus with low contrast (M = .951, SE = 0.074), F(1, 18)
= 10.69, p = .004, η2 = .373. None of the interactions were found to be significant.
The participants’ response time are summarised in Fig. 3.
Fig. 3. Summary of participants’ response time
4 Discussion
We found significantly poorer visual task performance when multi-tasking than when
single-tasking in terms of response time but not accuracy. Such a finding implies that
participants engaging in multiple tasks may not divide their attention among the
requirements of the tasks simultaneously, but are instead alternating their attention
between the tasks. The implications to multi-task operations is critical; it signifies that
multi-tasking may not necessarily result in erroneous processing of visual stimuli
(accuracy), but it does take a toll on timely response to incidences (response time).
Apart from multi-tasking, other factors may affect response accuracy. For example,
the interaction between stimulus size and stimulus contrast on accuracy was significant.
It suggests that erroneous processing becomes a concern when a heterogeneous vari-
ation of visual stimuli needs to be detected. Specifically, contrast has less effect on
accuracy when the stimulus is large, but impairs performance when the stimulus is
small. This carries implications in multi-task operations when the environment of the
area under surveillance varies. For instance, external factors such as weather (e.g. rainy
or foggy) can influence contrast of the surrounding objects in air traffic control.
610 S. Y. M. Goh et al.
Some limitations may be present in this study. Firstly, visual task performance
(accuracy and response time) with distance from the point of fixation were not
examined in this paper as there was a lack of a meaningful trend in the threshold of the
FFoV. In other words, visual task performance of the participants did not seem to
decrease consistently with increasing distance from the point of fixation. This might be
because the task performance reaches plateau without much variations in this aspect.
Secondly, as the cognitive load required by the secondary task used in this study was
not measured, it is plausible that the task was not sufficiently difficult to reveal other
impacts of multi-tasking. Therefore, the secondary task should not be regarded as
representative of the actual tasks (which might utilise a greater cognitive load) that such
operations might require their operators to engage in.
Moving forward, future studies should address these limitations, to (1) aim to
determine how the threshold of the FFoV vary with task requirements and stimulus
parameters and (2) replicate the study using secondary tasks that are more represen-
tative for the context in which the results would be applied in. Such findings can go a
long way in reducing human errors that can potentially result in dire consequences in
multi-task operations in surveillance, such as that of the air traffic controllers.
5 Conclusion
As technology advances, human beings are presumed to be able to multi-task [13]. This
research demonstrated the potential impact of multi-tasking on visual task performance.
Even for a simple visual orientation judgment task, imposing a secondary task can
impair one’s visual task performance. When we consider the multitude of possible
tasks in multi-task operations, this issue could only be further complicated. Further
research in the future can inform us the necessary adjustments to task requirements in
multi-task operations.
References
1. Squire, P.N., Parasuraman, R.: Effects of automation and task load on task switching during
human supervision of multiple semi-autonomous robots in a dynamic environment.
Ergonomics 53(8), 951–961 (2010)
2. Monechi, B., Vito, D.P.S., Loreto, V.: Congestion transition in air traffic networks. PLoS
One 10(5), e0125546 (2015)
3. Sanders, A.F.: Some aspects of the selective process in the functional visual field.
Ergonomics 13, 101–117 (1970)
4. Williams, L.J.: Cognitive load and the functional field of view. Hum. Factors 24, 683–692
(1982)
5. Holmes, D.L., Cohen, K.M., Haith, M.M., Morrison, E.J.: Peripheral visual processing.
Percept. Psychophys. 22, 571–577 (1977)
6. Recarte, M.A., Nunes, L.M.: Effects of verbal and spatial-imagery tasks on eye fixations
while driving. J. Exp. Psych.: Appl. 6, 31–43 (2000)
7. Strayer, D.L., Drews, F.A.: Profiles in driver distraction: effects of cell phone conversations
on younger and older drivers. Hum. Factors 46, 640–649 (2004)
Effects of Increased Cognitive Load on Field of View 611
8. Strayer, D.L., Drews, F.A.: Multitasking in the automobile. In: Kramer A., Wiegmann D.,
Kirlik A. (eds.) Attention: From Theory to Practice, pp. 121–133 (2006)
9. MATLAB version 7.10.0 (R2010a). The MathWorks Inc., Natick, Massachusetts (2010)
10. Brainard, D.H.: The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997)
11. Pelli, D.G.: The VideoToolbox software for visual psychophysics: transforming numbers
into movies. Spat. Vis. 10, 437–442 (1997)
12. Kleiner, M., Brainard, D., Pelli, D.: What’s new in Psychtoolbox-3? Perception 36 ECVP
Abstract Supplement (2007)
13. Lodinger, N.R., DeLucia, P.R.: Does automated driving affect time-to-collision judgments?
Proc. Hum. Factors Ergon. Soc. Ann. Meet. 62(1), 1833 (2018)
Investigating Human Factors in the Hand-Held
Gaming Interface of a Telerehabilitation
Robotic System
S. M. Mizanoor Rahman(&)
Department of Intelligent Systems and Robotics, Hal Marcus College of Science
and Engineering, University of West Florida,
11000 University Pkwy, Pensacola, FL 32514, USA
[email protected]
Abstract. Robotic devices can be used as effective tools to provide rehabilita-
tion supports to stroke patients. Such robotic rehabilitation practices can replace
therapists providing rehabilitation therapy to patients. Patients can independently
practice rehabilitation with robotic rehabilitation devices for longer hours. These
advantages are now motivating patients to use rehabilitation robots. However,
contemporary rehabilitation practices are expensive, and patients need to travel to
rehabilitation centers for rehabilitation, which is time consuming and burden-
some. This is why, the concepts of telerehabilitation are becoming more and
more popular where the patients can practice rehabilitation at homes, and the
therapists can remotely monitor the rehabilitation practices and communicate
with the patients if necessary. However, such rehabilitation practices, real-time
monitoring of patients by therapists from distant places and communication of
patients with therapists may be more intuitive and human-friendly if the patients
can operate the rehabilitation system and communicate with the therapists using a
hand-held interface. Again, the hand-held interface may be more intuitive and
engaging if the rehabilitation performance can be expressed through some game-
like activities. To do so, a clear understanding of human factors involved in the
gaming interface is necessary. However, such knowledge is not available in the
literature. To address this knowledge gap, in this paper, the human factors
associated with the operation of a hand-held gaming interface for robot-assisted
full-body smart telerehabilitation of stroke patients are investigated. At first,
potential human factors associated with the operation are identified through
surveys conducted with healthcare professionals, researchers and patients. The
identified human factors are then analyzed and divided into different categories,
e.g. physical and cognitive human factors. The role of each human factor in the
interface operation is explained. The findings can be utilized to design and
develop hand-held gaming interfaces for robot-assisted telerehabilitation that
may be more human-friendly and intuitive.
Keywords: Human factors Gaming interface Telerehabilitation
Rehabilitation robot Stroke patient
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 612–618, 2020.
https://doi.org/10.1007/978-3-030-39512-4_95
Investigating Human Factors in the Hand-Held Gaming Interface 613
1 Introduction
Application of robots in post-stroke rehabilitation is now a hot topic of research and
development due to various advantages of robot-assisted rehabilitation [1, 2]. Different
novel actuation methods were proposed to make those rehabilitation devices compliant
and safe [3–5]. However, efforts to make the rehabilitation systems more user-friendly
are still an open issue [6]. Within this direction, hand-held gaming interfaces are very
latest addition to the wearable healthcare devices including the rehabilitation devices
that can facilitate users to monitor rehabilitation performance in real-time through
hand-held visual interfaces based on some game-like activities performed during actual
rehabilitation practices [7]. The usability of such interfaces is to be high as they are
hand-held during rehabilitation practices. The usability and user acceptance of the
hand-held interfaces depend on how human factors associated with the interfaces are
identified and addressed while designing the interfaces [8]. The usability of the
interfaces may include the physical interfaces of the devices as well as the ways of
expressing and displaying the rehabilitation performance through the visual interfaces
[9]. However, investigation on human factors involved in hand-held gaming interfaces
has not received much attention yet. As a result, user-friendliness and hence user
acceptance of the proposed hand-held gaming interfaces are still questionable.
On the other hand, telerehabilitation is a concept that can keep patients connected
with healthcare providers and patient communities [10, 11]. Hand-held gaming inter-
faces can foster the telerehabilitation because the status of the rehabilitation performance
can be easily monitored and shared through the hand-held visual interfaces [7, 9, 10].
Hence, the hand-held gaming interfaces need further investigation on human factors
because the users (patients) may not feel comfortable and friendly while using the
gaming interfaces on dynamic contexts if the interfaces cannot satisfy the ergonomic and
psychological requirements of the users [12]. However, investigation on human factors
associated with gaming interfaces for telerehabilitation has not been prioritized.
The objective of this paper is to investigate the human factors associated with the
design of hand-held gaming interfaces for robot-assisted smart telerehabilitation. At
first, a complete full-body post-stroke telerehabilitation system is introduced that also
includes a hand-held gaming interface. Then, potential human factors associated with
the interface are identified through surveys conducted with concerned healthcare
professionals, researchers and patients, and the identified human factors are analyzed.
2 The Proposed Full-Body Rehabilitation System
with Hand-Held Gaming Interface
and the Telerehabilitation Scenario
Figure 1 shows the proposed overground mobile full-body post-stroke rehabilitation
robotic system actuated by compliant actuation mechanism. The details of the config-
uration, working principles and actuation mechanisms are provided in [1–5]. A hand-
held user interface via a mobile phone is shown in the figure. When the patient conducts
rehabilitation practice wearing appropriate sensors, the hand-held interface can display
the rehabilitation performance expressed in terms of kinetics and kinematics based on
614 S. M. M. Rahman
some game-like activities that the patient performs during the rehabilitation practice.
The performance can also be displayed in a large fixed screen placed in front of the
patient during the rehabilitation session [9] (see Fig. 1). The game-like user interface
can be designed using the Unity software for the stroke rehabilitation system shown in
Fig. 1. In the proposed user interface, the patient can practice gait or upper arm reha-
bilitation, and the rehabilitation performance can be measured in real-time using suitable
sensors (position, force) that the patient wears at different relevant body parts. The
rehabilitation performance can be expressed in terms of force/torque and linear/angular
velocity that the patient generates at each joint during the gait cycle (lower limb reha-
bilitation) or the arm movement (upper limb rehabilitation) for performing game-like
activities (e.g., trying to reach a location within a specific time using the mobile over-
ground exoskeleton for lower limb rehabilitation) as a part of the rehabilitation practice.
The patient’s kinetics and kinematics performance is changed in real-time and the
changes are reflected through the interface as bar diagrams. This can be installed in a
mobile device and the user (patient) can hold it in his/her hand while performing the
rehabilitation practice/game. The performance can also be displayed in a monitor [9].
Screen
Hand-held
interface
Robotic system
Fig. 1. The proposed overground mobile full-body post-stroke rehabilitation robotic system and
the hand-held gaming interface (left), and the display in a large fixed screen (right).
The rehabilitation system with the hand-held gaming interface shown in Fig. 1 can
be augmented to telerehabilitation, as shown in Fig. 2 [10]. The patient can practice
rehabilitation using the robotic device at home, the performance can be displayed and
interacted through the hand-held gaming interface, and the status can be shared with the
healthcare professionals or the patient communities at distant in real-time or near real-
time through appropriate internet or cloud-based communication system [10, 11].
3 The Human Factors Survey
Fifteen (15) human subjects were selected. Out of the 15 subjects, 9 subjects were
robotics researchers focusing applications of robots for stroke rehabilitation, 4 subjects
were professionals who provided robot-based and manual therapies to stroke or similar
Investigating Human Factors in the Hand-Held Gaming Interface 615
Distant room
Hand-held
(healthcare center)
interface
Robotic system
Fig. 2. Concepts of real-time telerehabilitation practice between patient and healthcare
professional through hand-held gaming interface via appropriate communication system.
patients, and the remaining 2 subjects were stroke patients who came to the rehabili-
tation centers or the rehabilitation research office to receive rehabilitation practice
and/or to know more about the robot-based rehabilitation practices. Each subject
participated in the survey separately. At first, the subject was introduced with the robot-
based rehabilitation practice. The objective and the way of using the hand-held gaming
interface were also discussed with the subject. To clarify the concepts about the hand-
held gaming interface and the telerehabilitation, Figs. 1 and 2 were shared with the
subjects and the necessary descriptions were provided. Any questions or comments
from the subjects were responded. Then, each subject was provided a piece of paper
separately, and was asked to brainstorm and write down the human factor cues that
he/she thought necessary to address while designing the hand-held gaming interface.
4 The Survey Results
At the end of the survey, the responses were accumulated and analyzed. The identi-
fied human factor cues were divided into two categories: (i) physical human factors
(pHFs), and (ii) cognitive human factors (cHFs) [13–15]. The physical human factors
included the human factors related to the physical and material properties of the
interfaces. The cognitive human factors included the cues related to the ways of
expressing and displaying the rehabilitation performance and the cues that can impact
user cognition and perception. Table 1 shows the pHFs and the cHFs with response
frequencies (how many subjects out of 15 mentioned a particular human factor as the
cue that could impact user’s physical and/or cognitive abilities) in the parentheses.
Figure 3 shows the relative importance of each pHF and cHF cue based on the results
in Table 1.
The results show that the subjects were mainly concerned with the haptic weight of
the hand-held interface device (e.g., weight of the mobile phone or the tablet), fatigue
due to holding the device for long time, situation awareness while conducting reha-
bilitation as well as monitoring the rehabilitation performance via the hand-held
interface, cognitive workload while comprehending and interpreting the status of the
performance displayed through the interface in the forms of different diagrams,
physical and mental engagement with the hand-held device, how the color (e.g., glare)
of the displayed contents may impact the vision and perception of the patient, whether
616 S. M. M. Rahman
Table 1. Identified pHFs and cHFs based on survey results for robot-assisted post-stroke
telerehabilitation using hand-held gaming interface
Human factor Human factor cues and frequencies
category
pHFs Ease of use (6), safety/risk (3), haptic and proprioceptive perception (5),
maneuverability (4), perceived haptic weight of hand-held device (11),
fatigue (9)
cHFs Self-satisfaction (3), situation awareness (8), cognitive workload (12),
trust (6), emotion (3), engagement and connectedness (8), color
perception (9), perception of shape and size of hand-held device (5),
transparency (8), intuitiveness (2), reliability (4), predictability (10)
or not the information displayed through the gaming interface is enough transparent to
understand and reflect the true rehabilitation scenarios, and whether or not the subjects
can predict the future of their rehabilitation practice and performance based on what
they observe in the displayed diagrams via the interface, etc.
6
9
Ease of use
3 Safety/risk
Haptic and proprioceptive perception
5 Maneuverability
Perceived haptic weight
11
4 Fatigue
3 Self-satisfaction
10 8 Situation awareness
4 Cognitive workload
2 Trust
12 Emotion
Engagement and connectedness
8 Color perception
Perception of shape and size
5 6 Transparency
Intuitiveness
3
9 Reliability
8 Predictability
Fig. 3. The relative importance of each cue of pHFs (upper) and cHFs (lower).
Investigating Human Factors in the Hand-Held Gaming Interface 617
5 Conclusions and Future Work
A survey was conducted with healthcare professionals, researchers and patients to
identify the potential human factor cues associated with the design and development of
hand-held gaming interfaces for robot-assisted post-stroke telerehabilitation. The
identified human factors were then analyzed and divided into two categories: physical
and cognitive human factors. The results can be used to optimize the gaming interface
design satisfying human requirements, and benchmark and share the telerehabilitation
performance for stroke patients. In the near future, a deeper and more formal survey
with more relevant responders will be conducted to know more about the human
factors associated with the gaming interface design. The knowledge on the human
factors will be used to develop real telerehabilitation robotic system with hand-held
gaming interface, and will be evaluated by actual patients performing rehabilitation
practices at homes or rehabilitation centers or hospitals.
References
1. Rahman, S.M.M.: Design of a modular knee-ankle-foot-orthosis using soft actuator for gait
rehabilitation. In: Proceedings of the 14th Annual Conference on Towards Autonomous
Robotic Systems (TAROS 2013). Lecture Notes in Computer Science, Oxford University,
U.K., vol. 8069, pp. 195–209. Springer (2014). (2013)
2. Rahman, S.M.M., Ikeura, R.: A novel variable impedance compact compliant ankle robot for
overground gait rehabilitation and assistance. Proc. Eng. 41, 522–531 (2012)
3. Rahman, S.M.M.: A novel variable impedance compact compliant series elastic actuator for
human-friendly soft robotics applications. In: Proceedings of the 21st IEEE International
Symposium on Robot and Human Interactive Communication, pp. 19–24. IEEE Press
(2012)
4. Rahman, S.M.M.: A novel variable impedance compact compliant series elastic actuator:
analysis of design, dynamics, materials and manufacturing. Appl. Mech. Mater. 245, 99–106
(2013)
5. Yu, H., Rahman, S.M.M., Zhu, C.: Preliminary design analysis of a novel variable
impedance compact compliant actuator. In: Proceedings of 2011 IEEE International
Conference on Robotics and Biomimetics, pp. 2553–2558. IEEE Press (2011)
6. Rahman, S.M.M., Ikeura, R.: Improving interactions between a power assist robot system
and its human user in horizontal transfer of objects using a novel adaptive control method.
Adv. Hum.-Comput. Interact. 2012, 1–12 (2012). ID 745216
7. Valdés, B.A., Hilderman, C.G.E., Hung, C.T., Shirzad, N., Van der Loos, H.F.M.: Usability
testing of gaming and social media applications for stroke and cerebral palsy upper limb
rehabilitation. In: Proceedings of 2014 36th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society, pp. 3602–3605. IEEE Press (2014)
8. Rahman, S.M.M., Ikeura, R.: Cognition-based control and optimization algorithms for
optimizing human-robot interactions in power assisted object manipulation. J. Inf. Sci. Eng.
32(5), 1325–1344 (2016)
9. Rahman, S.M.M., Wang, Y.: Mutual trust-based subtask allocation for human-robot
collaboration in flexible lightweight assembly in manufacturing. Mechatronics 54, 94–109
(2018)
618 S. M. M. Rahman
10. Kawai, Y., Honda, K., Kawai, H., Miyoshi, T., Fujita, M.: Tele-rehabilitation system for
human lower limb using electrical stimulation based on bilateral teleoperation. In:
Proceedings of 2017 IEEE Conference on Control Technology and Applications,
pp. 1446–1451 (2017)
11. Rahman, S.M.M.: Evaluating and benchmarking the interactions between a humanoid robot
and a virtual human for a real-world social task. In: Proceedings of the 6th International
Conference on Advances in Information Technology (2013). Communications in Computer
and Information Science, vol. 409, pp. 184–197 (2013)
12. Rahman, S.M.M., Ikeura, R.: Investigating the factors affecting human’s weight perception
in lifting objects with a power assist robot. In: Proceedings of 2012 21st IEEE International
Symposium on Robot and Human Interactive Communication, pp. 227–233 (2012)
13. Rahman, S.M.M., Ikeura, R.: Cognition-based variable admittance control for active
compliance in flexible manipulation of heavy objects with a power assist robotic system.
Robot. Biomim. 5(7), 1–25 (2018)
14. Rahman, S.M.M., Liao, Z., Jiang, L., Wang, Y.: A regret-based autonomy allocation scheme
for human-robot shared vision systems in collaborative assembly in manufacturing. In:
Proceedings of the 12th IEEE International Conference on Automation Science and
Engineering (IEEE CASE 2016), pp. 897–902 (2016)
15. Rahman, S.M.M., Wang, Y.: Dynamic affection-based motion control of a humanoid robot
to collaborate with human in flexible assembly in manufacturing. In: Proceedings of ASME
Dynamic Systems and Controls Conference, pp. V003T40A005 (2015)
Procedure of Mining Relevant Examples
of Armed Conflicts to Define Plausibility Based
on Numerical Assessment of Similarity
of Situations and Developments
Ahto Kuuseok(&)
Estonian Police and Border Guard Board, Estonian Business School,
Tallinn, Estonia
[email protected]
Abstract. It is possible to link plausibility to numeric assessment of similarity
of situations and developments. In order to observe similarities of two situations
or developments, it is necessary to prepare descriptions. These descriptions must
consist of statements, preferably in a format that allows them to be represented
by appropriate logic formulas, with the purpose of calculating the index of
descriptive similarity. With the need to observe several armed conflicts, there
will arise a need to limit the amount of cases, considering physical abilities of
experts. The aim is to decide- which cases can be considered as relevant. The
focus of this study is the procedure whose implementation results are selected as
conflicts (involving national armed forces), with certain geographical and time
constraints, linked to conflicts, associated with a particular country C (i.e. with
the state whose interests or concerns, etc., are applied in the procedure). The
choices are based on the types of associations, like direct geographical neigh-
borhoods within country C, binding to defense and security agreements, inter-
connection based on economic needs (raw materials, markets, access) etc.
However, associations may be subject to some time and geographical con-
straints. For example, as the Republic of Estonia (located in northeastern Eur-
ope), is linked to several countries within NATO who only arose after World
War II, then is allowed though, not required to exclude conflicts involving
countries that preceded World War II. Alternatively, as the United States, is
linked to the Republic of Estonia within the framework of NATO, then the
Vietnam War (which took place in Southeast Asia) could be in the set of cases.
In addition, one more example - because Russia is the immediate neighbor of the
Republic of Estonia, so the military conflict in Georgia in 2012 should also be
involved. Based on the types of associations fixed for the Republic of Estonia,
the sample examined was limited to only 32 relevant examples. This, in turn,
allows the comparison of pairs of conflicts to be limited to 496 pairs.
Keywords: Plausibility Numeric assessment of similarity Situations and
developments Descriptions and statements Index of descriptive similarity
Selection of armed conflicts Constraints
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 619–626, 2020.
https://doi.org/10.1007/978-3-030-39512-4_96
620 A. Kuuseok
1 Introduction
In order to countries could perform main objectives, including avoid unpleasant sur-
prises related to military security; they need several approaches to research security
situations and developments. Among others, there is a need to evaluate the possibility
of an outbreak of such kind of conflicts that directly or distantly affect those countries.
One of such approaches relies on similarity of situations and developments. More
specifically, it relies on a procedure based on descriptions of the situations and
developments under consideration [1].
This procedure consists of several steps, the last of which is associated with
numerical estimates of similarity and plausibility. In this way, there is one way of
comparing the situation that is important for a given country and its evolution with
some known situations and developments. The purpose of the comparison is to identify
the most similar situation and development that is relevant to the given country. From
this knowledge, we can find out if it is plausible that the particular situation of major
importance to the country may become an armed conflict (compared to similar situa-
tions and related developments) [2].
The main purpose of this work is to describe how can the procedure of identifi-
cation of relevant situations and developments to a given country, determine whether
this situation can develop to an armed conflict between states. To some extent, this is
the procedure, which produces so-called “source material” for the procedure for
assessing the similarity of situations and developments, which in turn provides inputs
for determining plausibility. Therefore, both procedures are covered. Also (rather
briefly) attribution of plausibility.
2 Procedure for Extracting Situations and Developments
Relevant to a Given Country
The procedure under consideration has been divided into phases, followed by:
2.1 Phase One: Selecting the Type or Types of Situations
and Developments
The list of situations and developments relevant to a given country is, in principle, not
limited to this, the second, or the third situation or development. For example, climate
change may be an issue, also the emergence of a new economic crisis, also an obvi-
ously cooking conflict involving a direct neighbor or contract partner, etc. Conse-
quently, it is expedient to make an appropriate choice before proceeding with the
procedure. For example, choosing developments that involve only two immediately
preceding situations:
– The situation immediately preceding the armed conflict, the so-called eve
– The actual outbreak of the armed conflict.
Procedure of Mining Relevant Examples of Armed Conflicts to Define Plausibility 621
2.2 Phase Two: Define the Criteria
It is important to start with making the right choices and systematizing things, which
are considered important for a country. There is no doubt that there are “countless”
opportunities for this. Therefore, we should first agree on some criteria by which we
associate these things for a given country (including, for example, inter-state armed
conflicts). In this work, the following criteria is in place to associate countries (with a
given country):
2.2.1 Primary Criteria (P1, P2, P3)
P1. Takes place with the immediate geographical neighbor of that country.
P2. Takes place in a region of vital import or export to that country, or in a region
indispensable for access to the aforementioned region
P3. Occurs with a country that is a partner of security and defense treaty to given
country
Remark. In some cases, the second criteria can be ignored. For example, if a country
has enough options to buy and sell vital products or commodities. In the other hand, in
some cases, this override is not possible. Let us give an example, in the case of USA
Nederland’s and other countries, after the so-called “Yom Kippur War” between Israel
and the Arab states in 1973. The impact of the “Oil taps” was very severe [3]. In the
same way, the restriction that prevented Cuba [4] from selling its few marketable
products on the US market, including e.g. cane sugar was severe for the country.
2.2.2 Secondary Criteria (S1, S2, S3)
S1. Time, specifically the period under consideration to a given country, which
meet the primary criteria
S2. Existence of a diaspora that is well established in another (not necessarily
primarily associated) country whose fate is important to that given country
S3. Existence of a well-established diaspora of another nation in given country (not
necessarily primarily associated) that is important for the nations home country
S4. Emergencies requiring assistance (including e.g. genocide) in other countries
Remark 1. Placing time among the secondary criteria, and yet not defining the exact
period may be a little confusing. However, the primary criteria, in fact, often do set
quite many time limits quite clearly. Let us give an example, a situation in a country
linked to a security or defense treaty secondary criteria until that country already exists.
There is still quite a difference: whether something happened in a particular country or
in the region, where that particular state was founded later.
Remark 2. It is quite common for these criteria to take into consideration situations
and developments – “until today”. At the same time, the timing of the beginning of the
period is not frequent. In other words, while looking for analogs for the comparison,
622 A. Kuuseok
similar situations and developments, we may take under research these, which just
happened, but also what happened to the selected countries hundreds of years ago).
Remark 3. Unlike many other nations, Estonians have not become very numerous,
influential, or (potentially) problematic communities in other countries. However, there
is a significant and not overly integrated Russian community in the Republic of Estonia
Remark 4. Aforementioned criteria P1… S4 are, of course, not unique. These crite-
ria’s can be changed: by deleting something, adding something or changing something.
In this work, we have excluded several of them, considering the Republic of Estonia as
a “given state”.
Once the criteria have been selected and fixed, we can proceed to the next step of
the procedure under consideration.
2.3 Phase Three: Finding and Gathering Relevant Situations
and Developments
Important notes
– as we decide to deal with only two-stage development fragments, one on “the night
before” of the armed conflict and the other on the actual outbreak of the conflict
– Then, in a very simplistic approach, we can identify the other stages with each
other, because we always have a real outbreak of armed conflict,
– We need to collect descriptions of “the night before”, more precisely the descrip-
tions of situations what did happened just before the actual outbreak of the armed
conflict
For performing the steps of the procedure as just described, we can use help of
several suitable solutions. For example, most of publicly announced security and
defense policies, defense cooperation agreement etc. are publicly available in Estonia.
Mentioned documents are available on the website of Ministry of Defense of Estonia
and on the website of headquarters of the Defense Forces [5, 6].
3 Brief Numerical Evaluation the Similarity of Situations
and Developments and Linking Them to Plausibility
As previously, this is a procedure with its specific phases.
3.1 Phase One: Creating Cleaned Descriptions
This phase will be subdivided into the following steps
3:1:1. Creating a description of a situation or development so that they contain only
statements.
3:1:2. Aggregation of claims into subsets that are worthy of identification with each
other.
Procedure of Mining Relevant Examples of Armed Conflicts to Define Plausibility 623
3:1:3. Selecting one representative from each subset (formed in the previous sub-steps
and containing at least two elements, i.e. a statement) and removing the
remainder from the description. Just described set of statements we call a
refined description of the situation or development.
3:1:4. Sorting the statements in the refined descriptions by assigning them one-on-
one, matching with natural number indexes. Let it be for any suitable M 1,
all natural numbers 1 N M
3.2 Phase Two: Isolation of the Identifiable Parts of the Refined
Descriptions (of the Situation or Development Selected
for Comparison)
For that, we need to run a cyclic repeating action to form the identifiable part of the two
refined descriptions. We have to keep in mind in this phase, that within the refined
description here do not exist statements any more that we would like to identify. In the
following proceeding mentioned cyclic activity, first we just determine the identifiers
for suitable refined descriptions. Let us give an example: first description and second
description. After that:
3:2:1. We take the first statement from the first description and look a t the statements
in the second description, starting with the first.
3:2:1:1. If the description does not contain a statement that we would like to identify
with the first statement from the first description, we will not remove that
statement from the first description. Nothing we do remove from the second
description, either.
3:2:1:2. If, among them, there is (in accordance with the above - the only one) an
example that we want to identify with the first statement in the first descrip-
tion, we will remove the first statement from the first description. Now we
form an ordered pair with the first statement of the first description in the first
place and the statement from the second description in the second. We declare
this ordered pair as the first element of the identifiable part of the two
descriptions. Then we do remove this second-order statement from the second
description of the second-order statement.
3:2:2 Let’s take the second statement from the first description and search the
statements in the second description, starting with the first that still left after
2.1.2 as above.
3:2:2:1. If they do not contain a statement that we would like to identify with the first
statement from the first description, we will not remove that statement from
the first description. We do nothing to remove from the other description.
624 A. Kuuseok
3:2:2:2. If among these descriptions it consists of one statement (in accordance with
the above - the only one) that we want to identify with the second statement in
the first description, we will remove the second statement from the first
description. Now we do form an ordered pair, with the second statement of the
first description in the first place, and the statement from the second
description in the second place. We declare this ordered pair as the second
element of the identifiable part of the two descriptions. We do remove this
second-order statement from the second description of the second-order
statement.
Acting in this way, we do have three sets of two descriptions:
– D1, which consists of all statements from the first description, which we did not
identify with any of the statements in the second description.
– D2, which consists of any statement from the second description we did not identify
with any of the statements in the first description.
– Equ12, which consists of all such ordered pairs, in which on the first and second
position is presented respectively to the statements from the first and second
descriptions, which are identical to each other.
An index of descriptive similarity between two comparable situations or devel-
opments we call a number Sim12 = E(Equ12)/(E(D1) + E(Equ12) + E(D2)), where E
(H) denotes the number of elements of some finite set H.
Remark. This may lead to confusion about the Jaccard similarity factor [6, 7]. In
Jaccard similarity factor is presented the number of common elements, or the same
elements E (D1 \ D2), instead of E(Equ12), which expresses the number of elements
identified.
Important Note. The descriptive similarity index expresses the process of how the
similarity of the two descriptions has been calculated, depending on the particular way
in which the similarity of the statements is determined. Therefore, when calculating and
applying the descriptive similarity index, it is important and required to exhibit the full
set of Equ12 or the process constructed. As examining the process carefully, there is
always a certain “common source of mistrust” in the assessment of structural similarity.
Mentioned “common source of mistrust” depends on what kind of compliance has been
used observing two sets of elements and two observed systems.
One (not the only) possible way of defining the similarity of statements, is to rely
on their logical equivalence. (Within the framework of fixed logic - e.g. classical,
intuitionist, etc.). This requires the transformation of statements in so-called natural
language into e.g. predicate formulas, such as the DST dialog system [9–11].
Comparing descriptions of a situation or development under observation, with the
description of already known or “studied” situations or developments, we can estimate
plausibility, which they are based on [12]. For example, observing a development, or
exploring an unknown future development (or vice versa some developments in the
past), with need to predict a process, we can rely on the assumption that particular
development under observation will continue in a predictable way if it has a (high)
similarity index with already known and studied processes.
Procedure of Mining Relevant Examples of Armed Conflicts to Define Plausibility 625
4 Summary
In order to be ready for strategically important developments n (-close) future, it is
crucial to create some method to analyse and predict relevant developments, as much is
possible. It is also important to perform analyses and possible predictions arguably and
preferably equipped with numerical assessments. The basis of our approach is reliance
on plausibility of situations and developments, specifically here: on procedure that
proceeds on descriptions of situations and developments under consideration. Because
the information in common world changes fast and in bulk, we need the clarity to select
specific and relevant data: the statements derivate from descriptions of situations and
developments. That ensures possibility to calculate relevant and mathematically
accurate similarity indexes. That also provides possibility to argue the calculation
process and if necessary, restore the situation.
In this work, we dealt with this method for very important subject area: military
security developments, analysis of the threat of a military crisis. Discussing procedure
of mining relevant examples of armed conflicts to define plausibility based assessments
of similarities; we tried to create procedure to help predict close future developments,
with help mathematical tools and logic. Also not necessarily in case of armed confits,
the method should be discussed also in case of other security areas, like economy,
internal security etc. [14].
References
1. Lorents, P., Matsak, E., Kuuseok, A., Harik, D.: Assessing the similarity of situations and
developments by using metrics. In: Intelligent Decision Technologies, (KES-IDT-17),
Vilamoura, Algarve, Portugal, 21–23 June 2017. Springer (2017). 15 pages
2. Lorents, P., Kuuseok, A., Matsak E.: Applying systems’ similarities to assess the plausibility
of armed conflicts. In: Smart Information and Communication Technologies (Smart-ICT-
19), Saidia, Morocco, 26–29 September 2019. Springer (2019). 10 pages
3. Wikipedia, Yom Kippur War. https://en.wikipedia.org/wiki/Yom_Kippur_War. Accessed 10
sept 2019)
4. Proclamation 3447, Embargo on all trade with Cuba. https://www.govinfo.gov/content/pkg/
STATUTE-76/pdf/STATUTE-76-Pg1446.pdf (U.S. Government Printing Office. 03 Febru-
ary 1962)
5. Ministry of defense of Estonia, official website. http://www.kmin.ee/en
6. Estonian Defense Forces, official website. http://www.mil.ee/en/news
7. Jaccard, P.: Étude comparative de la distribution florale dans une portion des Alpes et des
Jura. Bull. de la Société Vaudoise des Sci. Nat. 37, 547–579 (1901)
8. Jaccard, P.: Distribution de la flore alpine dans le bassin des Dranses et dans quelques ré-
gions voisines. Bull. de la Soc. Vaudoise des Sci. Nat. 37, 241–272 (1901)
9. Matsak, E.: Dialogue system for extracting logic constructions in natural language texts. In:
Proceedings of the International Conference on Artificial Intelligence. IC – AI’2005, Las
Vegas, Nevada, USA, vol. II, pp 791–797. CSREA Press (2005)
626 A. Kuuseok
10. Matsak, E.: System DST for transforming natural language texts, representing estimates and
higher order predicates and functionals. In: Proceedings of the 3-rd International Conference
on Cybernetics and Information Technologies, Systems and Applications: CITSA 2006: The
3-rd International Conference on Cybernetics and Information Technologies, Systems and
Applications: CITSA 2006, Orlando, Florida, USA, pp. 79–84, 20–23 July 2006 (2006)
11. Matsak, E.: Discovering Logical Constructs from Estonian Children Language. Lambert
Academic Publishing, Germany (2010)
12. Sigarreta, J.M., Ruesga, P., Rodriguez, M.: On mathematical foundations of the plausibility
theory. Int. Math. Forum. 2(27), 1319–1328 (2007). 2007
13. Balzacq, T.: Securitization Theory. How Security Problems Emerge and Dissolve. Rotledge,
London and New York (2011). Edited kby Thierry Balzacq
Human Digital Twins: Two-Layer Machine
Learning Architecture for Intelligent
Human-Machine Collaboration
Wael Hafez(&)
Alexandria, VA, USA
Abstract. Systems around us, either commercial, industrial, or social, are rapidly
becoming more complex, more digital, and smarter. There is also an increasing
conviction that the effective management and control of various scenarios in such
complex systems can only be achieved by enabling an intelligent collaboration
between the involved humans and machines. A major question in this area is how
to provide machines with access to human behavior to enable the desired intelli-
gent and adaptive collaboration between them. Based on the industrial concept of
digital twins, this study develops a new approach for representing humans in
complex digital environments, namely, a human digital twin (HDT). The HDT is a
smart machine that learns the behavior of a human in terms of his/her communi-
cation patterns with the smart machines he/she interacts with in a specific scenario.
The learned patterns can be used by the HDT and other machines supporting a
human to predict human–machine (H–M) interactions outcomes and deviations.
Unlike current approaches, the HDT does not need to rely on the content of the H–
M interactions to learn patterns and infer deviations, it just needs to register the
statistical characteristics of the exchanged messages. Using HDT would enable an
adaptive H–M collaboration with minimum interruptions and would provide
insights into the dynamics and dependencies involved in H–M collaborations
which can be used to increase the efficiency of such collaboration.
Keywords: Human–machine collaboration Smart environments
Management Human digital twin
1 Introduction
Trying to achieve intelligent human–machine collaboration (IH–MC) is not new. As
early as 1960, the anticipation was that such collaboration would “enable men and
computers to cooperate in making decisions and controlling complex situations without
inflexible dependence on predetermined programs” [3]. Many concepts have since been
developed to achieve the goal of enabling humans and machines to leverage one another
towards controlling complex situations [4, 6, 7]. These approaches and research con-
cepts focus primarily on increasing the information exchange and analysis capacities of
W. Hafez—Independent Researcher.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 627–632, 2020.
https://doi.org/10.1007/978-3-030-39512-4_97
628 W. Hafez
the various team members, i.e. they focus on the information content of H–M interac-
tions, where the current study focus on the statistical aspect of the messages carrying that
content. The study presented in this paper provides a conceptual approach for enabling
IH–MC in complex scenarios. First, it is argued that H–M collaboration mostly follows
an initial design, which can be used as a baseline to define H–M collaboration patterns.
Then, it is assumed that intelligent collaboration is based on the capacity to adaptively
learn such collaboration patterns and use them to predict deviations from the scenario
objectives. A new type of agents called human digital twin (HDT) is argued to provide
this capacity. Finally, the functionality of the proposed HDT and the overall IH–MC
architecture necessary to support the desired intelligent collaboration are presented.
2 Characteristics of H–M Collaboration
H–M collaboration does not just emerge through an accidental encounter between
humans and smart machines. Scenarios such as the coordination and management of a
complex supply chain, management of a large asset-maintenance process, provision of
a comprehensive healthcare service, management of resources in a smart city, or air-
space traffic flow management always follow a specific design. That is, humans and
machines are always brought together intentionally, and according to some initial
design to achieve some common objectives (see Fig. 1).
Domain
M6
M1
M3
Human
Machine Agent
Agent
M7 M9 H3
M2 H1
M5
M8
M4
Information
Channel
H2
Fig. 1. H–M collaboration in a complex scenario.
The design of such scenarios, no matter how complex, is always based on a set of
assumptions about the scenario environment or context and the internal dynamics and
dependencies among the various humans and machines involved. Based on these
assumptions, the initial design defines how the various agents, either humans or
machines, should act to ensure their alignment with one another and with the overall
scenario objectives.
Human Digital Twins: Two-Layer Machine Learning Architecture 629
2.1 Learning and Prediction in IH–MC
Currently, the design assumptions in complex scenarios cannot capture all the even-
tualities during the scenario execution. This is due to factors such as unforeseen
changes in the environment or unknown information dependencies and feedback loops
within the scenario [5]. Accordingly, the various agents would, sooner or later, be faced
with situations not covered in the initial design. In this case, scenario owners and
designers, smart machines vendors, and human agents need to continuously monitor
the scenario performance and when necessary, intervene to adjust the relevant agent’s
input and output information or the conditions necessary to make the appropriate
actions and decisions that accommodate the new, unforeseen situations. However, if we
are aiming at IH–MC, such external interventions should be avoided or kept at a
minimum. That is, the agents supporting a scenario should be autonomous in identi-
fying and mitigating deviations from the scenario objectives without relying on outside
intervention.
2.2 Role of Human Agents in IH–MC
Humans have a leading role in providing the two necessary capacities for IH–MC. If
we assume that Fig. 1 represents a future AI-supported airspace traffic flow manage-
ment scenario, then the various machines would provide for example, route recom-
mendations, weather predictions, or airport demand and capacity predictions. The
involved human roles would be air traffic controllers, pilots, airlines dispatchers, or air
traffic managers. Each role player would receive recommendations, options of deci-
sions, and impact analyses from the machines supporting him/her to make decisions
that maximize the scenario objectives: air traffic flow efficiency and safety.
In such a scenario, and considering, the conditions determining the operation of
each agent are very specific to the agent’s task. That is, the agent’s sensors, actuators,
and world-model are all specific (and limited) to the scope of its tasks. The initial
design, however, would ensure that the agent declared objectives and preferences are
specifically defined to support the overall scenario objectives, e.g. safety. This task-
specific sensory, action, and representational capacities limit the ability of the machine
agents to detect and use any information outside their design scope.
Human agents (e.g., air traffic controllers, pilots, etc.) would have a rather universal
understanding of the overall scenario objectives (e.g., safety), and how it relates to new,
unforeseen situations [1]. This is partially because the world model of a human, i.e., the
mental representation of the scenario and its objectives, is much more comprehensive
than that of any single machine supporting the scenario. In addition, humans are still far
better than machines in simulating the impact of their actions on the scenario objec-
tives. Accordingly, the human agents in a scenario are best equipped to identify
deviations from the scenario objectives and select the appropriate actions to offset such
deviations.
If it is then possible to observe how a human agent responds to changes and
deviations in the scenario execution, i.e., which new percept–action patterns are
developed by the human, then it would be possible to identify new H–M interactions
patterns. Furthermore, it would be possible to use the new patterns to update the
630 W. Hafez
conditions (e.g., the world model or utility functions) of the machines directly inter-
acting with that human. The new, role specific percept–action patterns can also be used
by the scenario designers to compare how similar roles perform across multiple sce-
narios so as to upgrade the assumptions of the initial design and gain a deeper insight
into how to improve the scenario performance. It should be noted that deviations that
require completely new information (e.g., machines equipped with additional sensors
or actuators) or new conditions (e.g., updated machines’ world model) would still
require the involvement of scenario designers and/or machine vendors.
3 Second Layer of Smart Machines
The idea here is to insert a second layer of smart machines—what we call HDTs—
which would perform the task of observing humans as they interact with machines in a
scenario. The HDT is human-specific, i.e. each human will have their own HDT.
The HDT would access the communication between a human and all the machines
supporting. Based on the human responses to machine actions, the HDT would identify
role and context-specific human response patterns and use such patterns later to predict
deviations in human responses and infer possible scenario breakpoints.
3.1 Human Digital Twin (HDT)
The HDT is based on the concept of industrial digital twins (DT) which are “software
representations of assets and processes that are used to understand, predict, and opti-
mize performance in order to achieve improved business outcomes. Digital twins
consist of three components: a data model, a set of analytics or algorithms, and
knowledge” [2].
Similar to the industrial DT, the HDT would develop its own representation of the
human-machines interaction patterns for the human it observes and use this repre-
sentation to manage and optimize human interactions with the machines collaborating
with him/her. However, the fundamental difference to an industrial DT is that an HDT
would learn and anticipate such interaction patterns and deviations by observing the
communication (message exchanges) between a human and the machines they col-
laborate with, and not based on the message content. If, during the execution of their
role, the human responds to the various machine actions within the set of responses
provided by the initial design, then the HDT would conclude that the scenario is being
executed as expected, and that the scenario objectives are being met. However, if the
human starts overriding any of the machine actions or responding with actions not
foreseen at this point by the initial design, then the HDT would assume that the
scenario is deviating from its objectives, which caused the human to respond off-
design.
To learn interaction patterns and infer deviations, the HDT does not need to rely on
the content of the H–M communications, i.e., the content of the messages exchanged
between them. Rather, it only needs to register the statistical characteristics of the
exchanged messages: their usage probabilities and dependencies. As depicted in Fig. 2,
Human Digital Twins: Two-Layer Machine Learning Architecture 631
for that purpose, the HDT just needs to capture the unique identifier of the various
messages. This unique identifier is used to represent the various exchanged messages.
Type
Route Plan
Aircraft ID Input Message
Aircraft Type
Recommendation True Airspeed Flight Route
Message Features Departure Point
Route of flight Input Message M1 Planning
M1-01 Machine
… Unique ID
M2
Air Traffic
Manager H1 HDT1 M3
M4
Output Message
H1-01
Unique ID
Approved
M5
Output Message
Fig. 2. Message content and message unique ID
4 Role of HDT in Enabling IH–MC
The setup of the HDT would use the messages as defined by the initial design of the
role to build a representation of the role it observes. A message in the current approach
could stand for a complex transaction, a sensor reading, a yes/no response, or a
combination of one or more of these features. The main requirement is that it is unique.
If we assume that all the H–M communications are digital, then the scenario design
would determine the features of each message and provide the HDT with the appro-
priate capabilities and access privileges to register their occurrence. As mentioned
earlier, the HDT does not need to access the content of a message (e.g., the field values
of a transaction).
4.1 Learning H-M Collaboration Patterns
In the positive case, where the H–M action–response takes place according to the initial
design, each time the scenario is executed, the probabilities of specific dependent
messages along the scenario, their sequence, and their levels of dependency are rein-
forced. For example, if the interactions leading to the route clearance message always
follow the same sequence, then the HDT can assume that there is a stable pattern that
leads to that message.
A stable pattern, i.e., one that takes place with a high degree of confidence, can thus
be used by the HDT to anticipate or predict how a human would respond to a specific
set of input messages. This prediction can then be provided to the machines supporting
the human with insights into his/her behavior, i.e. which output message would the
human select as a response to a specific machine input message.
632 W. Hafez
4.2 Detecting and Predicting Scenario Deviations
In executing a scenario, if the human, based on the information provided to him/her
from the various machines and his/her awareness of the overall scenario conditions
responds with a message not foreseen in the initial design, then the HDT would assign
a positive probability to this “unexpected” message. If this unexpected response is part
of a stable pattern, then the degree of pattern confidence would decrease accordingly.
The scenario designers can define rules for how the HDT should deal with devi-
ations. For example, the HDT can be setup to flag all parents of messages involved in a
deviation. Such flags can indicate to the human role owners that such messages might
have a potential for a deviation. Based on further scenario runs and deviation man-
agement rules, the HDT could for example remove the flags, send messages and reports
to relevant machine operators/vendors to change the message design, or alert the
human role owner in case his/her decision involves any flagged messages. The HDT
can also be set up to use the BN to run simulations to anticipate the impact of the
deviation on the overall scenario.
5 Discussion
Current approaches to IH–MC focus mainly on making agents more intelligent by
increasing their sensory and actuator scope, world model, knowledge, or analytics
capabilities. The two-layer ML architecture can complement current approaches by
providing insights driven from the communication aspect of the H–M collaboration. As
mentioned before, this assumption is based on the analogy with communication theory
which, by considering the statistical characteristics of messages, enables a more effi-
cient overall communication. A further area of investigation is how to use the insights
obtained by the HDT to directly update the conditions of the various machines involved
with the human the HDT supports. That is, how could message probabilities and
correlations be used to directly update machines’ utility functions or world models.
References
1. Guszcza, J., Lewis, H., vans-Greenwood, P.: Cognitive collaboration: why humans and
computers think better together. Deloitte University Press (2017)
2. General Electric Website. https://www.ge.com/digital/applications/digital-twin. Accessed 01
May 219
3. Licklider, J.: Man-computer symbiosis. IRE Trans. Hum. Factors Electron. HFE-1, 4–11
(1960)
4. Madni, A.: Adaptive cyber-physical-human systems. Insights 21(3), 87–93 (2018)
5. Maani, K., Cavana, R.: Systems Thinking, System Dynamics: Understanding Change and
Complexity. Printice Hall, Aukland (2007)
6. McDermott, P., et al.: Human-Machine Teaming Systems Engineering Guide. MITRE
Corporation (2018)
7. Sowe, S., et al.: Cyber-physical human systems: putting people in the loop. IT Prof. 18(1),
10–13 (2016). https://doi.org/10.1109/MITP.2016.14
Semantic Network Analysis of Korean Virtual
Assistants’ Review Data
Hyewon Lim, Xu Li, Harim Yeo, and Hyesun Hwang(&)
Department of Consumer and Family Science, Sungkyunkwan University,
25-2 Sungkyunkwan-ro, Jongno-gu, Seoul 03063, Korea
{hw3359,lsnowx16,cassendra}@naver.com,
[email protected]
Abstract. The interest in artificial intelligence (AI) is increasing significantly.
“Virtual Assistant” is a service that allows consumers to easily access AI
technology. This study aims to provide a clear and thorough overview of the
current consumer experience through reviews written by consumers in South
Korea. We suggest possible solutions for future developments of virtual assis-
tants that could further benefit consumers. This study provides insights into
consumer experience with virtual assistants through their review data. We cat-
egorize each virtual assistant and their related issues and suggest possible
solutions for improving future virtual assistants to enhance the quality of con-
sumers’ life.
Keywords: Artificial Intelligence Virtual assistant Consumer experience
Semantic network analysis
1 Introduction
The size of the global Artificial Intelligence (AI) market is expanding as people’s
interest in AI and technology increases. The International Data Corporation (IDC) ex-
pected the global AI market to expand from $62.6 billion in 2014 to $412.1 billion in
2024 [1]. “Virtual Assistant” is a service that allows consumers to easily access AI
technology. The virtual assistant market in South Korea is being led by mobile carriers
and large portal sites that are expanding to consumers through aggressive marketing.
Virtual assistant is a structured service which operates through a mechanical mecha-
nism that provides customized services by learning the user’s behavioral patterns and
lifestyle habits. The service is mainly based on voice recognition technology [2]. Using
the review data on virtual assistants commercialized in South Korea, this study aims to
demonstrate the consumer experience and suggest a direction of development to the
service that supports the real lives of consumers.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 633–639, 2020.
https://doi.org/10.1007/978-3-030-39512-4_98
634 H. Lim et al.
2 Theoretical Background
2.1 Virtual Assistant
Virtual assistant is defined as a software agent that, for an individual, can understand
the user’s language through the combination of AI and other advanced technology and
implement instructions that the user wants [3, 4].
In South Korea, domestic mobile carriers (SKT, KT, and LG U+) and leading portal
sites Kakao and Naver released a virtual assistant equipped with AI in smart speakers.
In September 2016, SKT was the first to release “Nugu”. In January 2017, KT released
“Giga Genie”, based on voice recognition, into set-top boxes. LG U+ launched a smart
home service in December 2017 with Naver, an artificial intelligence platform of South
Korea’s leading portal site [5]. “Hay Kakao” that uses a voice recognition interface,
was released in November 2017 [6].
According to a survey by Consumer Insight, a research organization specializing in
mobile communications, the use experience rate of smart speakers was 11%, while the
utilization rate of each platform was 39% for Giga Genie, 26% for Nugu, 16% for
Clova, and 12% for Hey Kakao. It showed that services from latecomers met con-
sumers’ needs more frequently than earlier services [7].
2.2 Semantic Network Analysis
As methods of big-data analysis have developed, they have resulted in a variety of
techniques, including data mining, opinion mining, text mining, and social mining [8].
Semantic Network Analysis, one of the text mining techniques, analyzes the meaning
of the words that are the components of the message, and is a method of identifying the
network [9, 10]. Semantic Network Analysis is an important means of understanding
the entire structure of the network by analyzing the location and connectivity strength
of words in the network [9].
3 Methods
3.1 Data Collection
We collected review data from Nugu, Giga Genie, Clova and Hey Kakao, all available
from the Google Play Store (From the release date of each plat form to 9/10/2019). The
review data was collected using R 3.5.3 and totaled as follows: Clova, 2,240; Nugu,
2,085; Hey Kakao, 752; and Giga Genie, 469, totaling 5,546, as listed in Table 1. After
deleting irrelevant or simple reviews, the review data totaled: Clova, 2,037; Nugu,
1,745; Hey Kakao, 689; and Giga Genie, 402, totaling 4,873.
Table 1. The number of review data
Nugu Giga Genie Clova Hey Kakoa Total
Before clean 2,085 469 2,240 752 5,546
After clean 1,745 402 2,037 689 4,873
Semantic Network Analysis of Korean Virtual Assistants’ Review Data 635
3.2 Semantic Network Analysis Method
Nouns were extracted from clean data through morphology analysis using the Korean
natural language processing (KoNLP) toolkit, and the document-term matrix
(DTM) was constructed. DTM is constructed from separate review data for each
platform that is used to create a two-mode network matrix. The two-mode network
analysis showed differences between each platform, but the relationship between the
platforms was not clearly identified. One-mode network analysis was performed to
check the relationship between the review data for each platform briefly.
4 Result
4.1 Word Frequency Analysis
Comparing the top 20 most frequent words, technical words such as “Applications”,
“Voice Recognition”, “Wi-Fi”, “Artificial Intelligence”, and “Bluetooth” were com-
mon. One can also see that the speaker mainly used to play music through words such
as “Music Service” and “Speakers”. In particular, the unique words for each platform
identified the services that each volunteer would like to support centrally (Table 2).
Table 2. Word frequency analysis of each platform’s reviews
Rank Clova Freq Nugu Freq Hey Kakao Freq Giga genie Freq
1 Application 707 Connect 552 Connect 237 Application 108
2 Music 551 Application 392 Kakao 140 Music 93
3 Connect 550 Music 324 Music 124 Login 47
4 Naver 346 Melon(a) 266 Wi-Fi 123 Function 40
5 Speaker 280 Use 222 Application 106 Play 34
6 Voice Recognition 280 Function 182 Network 84 Certification 29
7 Use 257 Wi-Fi 161 Use 72 Connect 47
8 Function 257 Update 134 Function 68 Sign up 28
9 Update 255 Device 123 Alarm 67 Use 27
10 Play 239 Play 119 Melon 61 Search 27
11 Smart Phone 203 Aria 108 Update 58 Set-up 23
12 Execute 191 Login 105 Set-up 57 KT 22
13 Set-up 183 Set-up 95 Speaker 50 TV 22
14 Call 178 Execute 93 Bluetooth 36 Voice Recognition 21
15 Error 132 Error 82 Issue 34 List 18
16 AI 130 T-map(b) 80 Sound 31 Update 17
17 Wave 128 Radio 69 Voice 27 Error 17
Recognition
18 Wi-Fi 124 Service 67 Kakao talk(c) 25 Mobile 17
Communication
19 Login 121 Alarm 67 Play 25 Bluetooth 16
20 Bluetooth 117 Search 67 Radio 25 Smart Phone 15
(a) Melon: Music streaming Service
(b) T−map: Web mapping service developed by SKT
(c) Kakao Talk: A free mobile instant messaging application for smartphones with free text and free call;p
636 H. Lim et al.
The Clova platform used the word “Call” in its word frequency analysis. Clova
launched the service in speakers by character such that various wake words can be set.
The consumers discussed setting the wake word. Clova supports services that enable
one to call in a friendly manner. Nugu, SKT’s web mapping service, had “T-map”, as
the top frequency word. Nugu uses navigation through voice when the consumer’s
hands are not free. In Hey Kakao, “KakaoTalk”, a messenger application, appeared in
the upper word frequency. Because Kakao is a service that started from messengers,
consumers using Hey Kakao can assume that they are using a voice-based interface
rather than a touch-based interface when sending messages on KakaoTalk. The word
“TV” only appears in the Giga Genie platform. As it provides a set-top box-based
service, there is a lot of interaction with the TV.
4.2 Semantic Network Analysis
First, Fig. 1 looks at the connectivity between the words in the review data and shows that
the central words most connected are “Play List”, “Service”, “Delete”, and “Login”.
“Delete” is becoming an important link between words. It was directly connected to words
such as “Sign up”, “Confirm”, and “Solve”. This is also numerically identifiable through
degree centrality in Table 3. The high degree centrality of words such as “Wi-Fi”, “Issue”,
and “Execute” were causing many problems. Many reviewers said they would delete apps
because they are not able to use them. They experienced frequent errors in starting up,
such as personal or adult authentication errors. Direct links to words such as “Play List”
Fig. 1. Network analysis of review data.
Semantic Network Analysis of Korean Virtual Assistants’ Review Data 637
and “Volume” can also be found in Fig. 1. Betweenness Centrality in Table 3 can be used
to estimate the potential for adding or modifying playlists, replicating specific lists, and
deleting required applications.
Table 3. Word centrality
Degree centrality Betweenness centrality Eigenvector centrality
Wi-Fi 36 Play List 128.3230622 Issue 1
Issue 36 Login 113.7116879 AI 1
Execute 36 Wi-Fi 54.73386121 Execute 1
Alarm 36 Issue 54.73386121 Alarm 1
AI 36 Execute 54.73386121 Wi-Fi 1
List 34 Alarm 54.73386121 Service 0.870972
Login 33 AI 54.73386121 Modification 0.864577
Service 27 Delete 39.44599411 Naver 0.864577
Improvement 25 Improvement 31.79822932 Version 0.864577
Certification 25 Certification 31.79822932 Brown 0.864577
Second, generally, the network can be divided into three communities. Nugu and
Hey Kakao are two platforms mostly used through smart phones, focusing more on
personalized services. From reviews, the navigation application “T-map” from NUGU
is actively used by consumers. Korean leading message application “Kakao Talk” can
be implemented with voice through Hey Kakao. Giga Genie actively uses set-top box
as fixed devices focusing on household sharing services. Although the exact addresses
are registered in the set-top boxes, consumers mentioned their issues concerning the
risk of personal information leakage. Clova has released a variety of speakers, such as
“Wave” and “Brown”, thus, they offer the service that enables users to set up different
call names for the speaker. From the link between the words “Call” and “Name”, it can
be confirmed that consumers’ interest in such services is growing.
Third, a variety of differences emerged between different platforms. Nugu has
difficulty in the usage of functions if the log-in password was not correct or the Wi-Fi
connection was unavailable. For Clova, some consumers mentioned smart speakers
from other brands, such as Brown, which released new products that are similar to the
early versions of Wave and Friends from Clova. Additionally, as the application of
Samsung Galaxy is widely used in different smartphones, many reviewers mentioned
their comparisons with Bixby as to which one is better or worse.
5 Conclusion
This study was intended to recognize consumer experience with intelligent agents
being commercialized in South Korea through review data, and to present the current
status of intelligent agents and directions to improve consumers’ life. The conclusions
from this study are as follows.
638 H. Lim et al.
First, “Delete”, “Play List”, and “Service” were located at the center of the network
and connected to all three communities formed by other keywords. This suggests that
consumers are demanding improvements in various services in virtual assistants,
including music-related services that are most commonly used. But due to continuing
errors, consumers were looking for a solution themselves, repeatedly deleting and
installing. Accordingly, services must be stabilized and facilitated before adding new
services. The services must provide sufficient information on error solutions.
Second, Hey Kakao and Nugu are forming one mobile based virtual assistant
community focusing on providing personalized services using internet networks to suit
their characteristics. Words such as “Solve”, “Network”, “Wi-Fi”, and “Necessary”
imply that there have been frequent errors in network and Wi-Fi connections that are
essential for the use of major services for mobile based virtual assistants. Thus, mobile
based virtual assistants can strive to minimize the occurrence of network connectivity
errors, while providing a variety of internet-based personalization services.
Third, Giga Genie, a virtual assistant that utilizes set-top boxes, has had the
problem that the consumer must register an address to use the Giga Genie service and
frequently experiences errors in doing so. Besides, they were experiencing fear of
personal information leaks used detailed addresses. Therefore, set-top box based virtual
assistant can minimize personal information and unnecessary procedures. This may
help reduce the inconvenience of using the service.
Fourth, as words such as “Version”, “Modification”, and “Do not work” show,
Clova frequently provides version updates and consumers expect to see improvements
in services resulting from them, but errors follow. The fact that Clova could only be
called with a button and not voice, in relation to keyword “Button”, also caused issues.
As the convenience of calling is expected to be important for virtual assistants installed
in the home and is used for various purposes, stabilization of basic functions, including
calling functions, and careful update version development is recommended, rather than
unnecessary feature addition.
This study examined consumer experience with virtual assistants that have been
continuously commercialized through review data, categorized each virtual assistant
and related issues, and suggested the direction of development of a virtual secretary to
improve consumer life. However, there were limitations of the insufficient size of the
overall review data and the existence of differences in the number of reviews by each
virtual assistant. Future studies can proceed with quality data of sufficient size to
produce more realistic and meaningful results.
References
1. Ahn, S.W.: Reviewing artificial intelligence, and the implications of Google’s disclosure of
Tensor Rolls. Softw. Pol. Inst. 12, 13–16 (2015)
2. Yoon, S.J.: Study on New Silver Generation’s Emotional Communication and customized-
Virtual Assistant Contents (2016)
3. Kim, H.M.: Development of AI technology and evolution of virtual personal assistant
service. KB Financ. Holding Manag. Inst. 16(53), 1–9 (2016)
4. Wikipedia. https://en.wikipedia.org/wiki/Virtual_assistant_(artificial_intelligence)
Semantic Network Analysis of Korean Virtual Assistants’ Review Data 639
5. ‘Smart Home Hub’ AI speaker competition 2 rounds. What about cards from 3 mobile
network providers? http://www.updownnews.co.kr/news/articleView.html?idxno=202150
6. AI speaker Kakao Mini officially launched. http://www.updownnews.co.kr/news/
articleView.html?idxno=202150
7. Consumer Insight, Hot AI Speaker Market, Cold Consumer Assessment. https://m.post.
naver.com/viewer/postView.nhn?volumeNo=16271707&memberNo=39577953
8. Ban, C.H., Kim, D.H.: Analysis of university department name using the R. J. Korea Inst.
Inf. Commun. Eng. 22(6), 829–834 (2018)
9. Wasserman, S., Faust, K.: Social Network Analysis: Methods and applications. Cambridge
University Press, Cambridge (1994)
10. Choi, Y.J., Kweon, S.H.: A semantic network analysis of the newspaper articles on big data.
J. Cybercommun. Acad. Soc. 31(1), 241–286 (2014)
Design Collaboration Mode of Man–Computer
Symbiosis in the Age of Intelligence
Jinjing Liu and Ken Nah(&)
The International Design School for Advanced Studies (IDAS),
Hongik University, Seoul 03082, Republic of Korea
[email protected], [email protected]
Abstract. At present, artificial intelligence (AI) is the hottest topic in the design
field, many AI design application systems that already exist and have begun to
replace the designer in the initial stage of design work. In this era of intelligence,
the design activities of man-computer symbiosis have become mainstream,
requiring designers to reflect on themselves and to re-position their role in
design activities. This research refers to previous research and literature on man-
computer symbiosis design activity, combining earlier findings with the author’s
observations and analysis of common practices in the design profession. It
examines existing AI design application system cases, and it synthesizes the role
position and scope of work for existing AI design application systems and
designers. Finally, it combines the “Double Diamond Design Process Model” to
find a collaboration mode between designers and AI design application systems
in the design activities of man-computer symbiosis.
Keywords: Age of intelligence Man-computer symbiosis Collaboration
mode Designer AI design application system
1 Introduction
The age of intelligence has come. The emergence of artificial intelligence (AI) has
changed not only the way we think and the way we live but also the way we work. But
the emergence of new mode does not mean the complete replacement of traditional
working thinking modes. Rather, each represents an upgrade to conventional modes.
The impact of the age of intelligence on our lives is no less than the impact of the
second industrial revolution, or that of the information revolution. AI has begun to
enter our daily work and life, but most people are not aware of it. And with the
development of intelligent technology, relationships between humans and computers
have become more and more intimate.
Since the 1970s and 1980s, man-computer interactive technology has become a
focus for studies in linguistics, psychology, cognitive science, design, sociology,
biology, and other disciplines. In the 1990s, with the computer taking the intelligent
direction of development—able to carry out thinking, learning, memory, network
communication, etc.—the man-machine relationship entered a new period of devel-
opment, and a large number of man-computer interaction psychology books appeared.
By the beginning of the 20th century, the relationship was in a stage of the gradual
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 640–645, 2020.
https://doi.org/10.1007/978-3-030-39512-4_99
Design Collaboration Mode of Man–Computer Symbiosis 641
approach, and with the spread of the internet and the development of smartphones,
software applications have become mobile and portable. Wearable design, cloud
computing, virtual reality, and other technologies have also developed rapidly. Arti-
ficial intelligence began to emphasize a people-oriented purpose. Since 2012, this
relationship has gradually entered the stage of man-computer symbiosis [1].
In fact, a concept of “man-machine symbiosis” was proposed by J.C.R. Licklider in
1960. He believed that this would be the expected development of cooperation and
interaction between people and computers. People have come to expect computers to
help with performing repetitive tasks, while humans continue to perform the tasks of
setting goals, and of making assumptions and assessments, to build symbiotic rela-
tionships (“living together in intimate association, or even close union, of two dis-
similar organisms” is called symbiosis [2]) [3]. Gill also presented a serious discussion
in his book about the moment the machine transcends and replaces humanity. An
alternative method to people-oriented design systems and technology has also been
proposed, emphasizing a different symbiosis of human and machine capabilities [4]. In
Donald Norman’s book, he compares the difference between humans and machines
from “people-centered” and “computer-centered” perspectives. The advantages of
human beings are creativity, attention to change, strategic thinking, flexibility, and
emotion. The advantages of the computer are precision, order, attentiveness, and logic.
This shows that the cooperation between people and computers needs to fully balance
the advantages and disadvantages of relationships between the two [5]. More basically,
human thinking is composed of two parts: creativity and repetitiveness. Executing the
latter through machines—computers—is the best way to achieve man-computer sym-
biosis. Because the ability of computers to process and present data is far beyond
human beings’ limits, it is with the help of computers that humans can focus more on
their creative faculties.
Man–computer symbiosis fits well with social development in the age of intelli-
gence. If traditional computer-aided design simply locates the computer as a design
tool, then in the age of intelligence, the role of the AI-loaded computer system marks a
clear change. Computer systems will become reliable assistants for humans, and AI
systems that assume the roles of task executors can help humans more effectively finish
their work.
Based on the research of an existing case in AI design application systems, this
study applies the double diamond design process model widely used in design research.
Analyzing the range of tasks that AI systems can perform in design activities, the study
finds a collaboration mode linking designers and AI design systems in man-computer
symbiotic design activities for the age of intelligence. Finally, it provides some sug-
gestions for future working modes in design.
2 Case Study
With the advent of more and more AI design application systems (It refers to the
intelligent application program that can automatically generate the design scheme
under the user’s requirements through algorithm, data and powerful computing power),
if the designer positions his or her role as executor of the design, and if the main task of
642 J. Liu and K. Nah
the designer is to present the solution to the customer using design skills (e.g., drawing,
or modeling), then AI will replace the designer. This will not wait until the distant
future to unfold. Already, AI design application systems have been developed for many
design industries, and these AI systems have been used in many practical design
projects, obtaining significant achievements. Here are some examples of AI design
systems used in real design projects:
In 2017, Alibaba launched an AI design system called ‘Luban’, is based on image
intelligence technology and three core modules (learning networks, actuators, and
evaluation networks). Users only need to input their needs. The ‘Luban’ can replace
manual completion of time-consuming design works and generate multiple sets of
design schemes that are in compliance with requirements [6, 7]. The system solves the
design needs of the massive web banner of the Taobao “Double Eleven Shopping
Festival” every year. During the 2017 shopping festival, it designed 400 million posters
for the merchants and became the hero of the Alibaba design department.
SJYP and Designovel have jointly launched the first AI-designed clothing in Korea.
The AI system they use is called ‘Style AI.’ The system learns and masters the brand
style according to brand materials and then provides the appropriate design scheme.
And the system also can export new clothing styles continuously and indefinitely [8].
In the fashion design industry, the traditional design method makes it difficult for
designers to catch up with changing trends. AI technology can help designers analyze
popular trends with the fastest speed and provide designers with more novel and
diverse trends for reference.
Besides the use of AI design application system in design activities that focus on
presentation, the use of AI design system is also proposed in such complex design
activities as industrial design, so as to improve work efficiency and obtain more design
solutions. The ‘Dreamcatcher’ released by Auto-desk is a generative design system.
Generative design is a way of translating computational energy into creative energy, is a
designer driven, parametrically constrained design exploration process [9, 10].
“Dreamcatcher” enables designers to craft a definition of their design problem through
goals and constraints. This information is used to synthesize alternative design solutions
that meet the objectives. Designers are able to explore trade-offs between many alter-
native approaches and select the most suitable design solutions for manufacture [11].
These AI design application systems, already used in actual design activities, show
us that AI systems can replace designers in completing the initial stages of design work
(e.g. data collection, data analysis, typesetting, and scenario generation). This shows
that for design activities in the age of intelligence, AI systems have begun to replace
human designers as executors of design activities. In the face of these AI design
application systems, the role of the designer must change, adapting to the new working
form of design activities in the age of intelligence. The designer will no longer hold the
role of executor, but rather that of planner for the entire design activity, designing and
completing tasks efficiently with the help of AI. Therefore, designers should understand
how to cooperate with AI systems in the design activities of man-computer symbiosis
for the age of intelligence.
Design Collaboration Mode of Man–Computer Symbiosis 643
3 Analysis of Collaboration Mode
The “Double Diamond Design Process Model”, probably the most famous and popular
visual design process in the world, was developed by the British Design Council in
2005. Its core consists of finding the right problem and the right solution. It is divided
into four phases: discover, define, develop, and deliver [12]. The main feature of this
design process model is its emphasis on “divergent” and “convergent” thinking. First,
divergent thinking generates as many ideas as possible. Then, convergent thinking
refines and summarizes the many ideas into the best ideas. In this model, the process
runs twice, once to confirm the definition of the problem, and once to create the
solution.
Based on the double diamond design process model, this study divides design
activity into ten specific tasks: planning, material collection, information processing,
requirements definition, creation, plan generation, outcome, evaluation, communica-
tion, management. See Fig. 1.
Fig. 1. Design specific task distribution based on the double diamond design process model.
Fig. 2. Functional scope statistics for 10 AI design application system cases.
Selecting 10 AI design application systems as cases that already show participation
in actual design activities, and analyzing their functional scope, we can find the main
functions of current AI design systems concentrated in the four design tasks. These
644 J. Liu and K. Nah
four, are the collection of materials, the processing of information, the generation of a
plan, and the outcome. See Fig. 2.
Based on the functional scope of AI design application systems in design tasks, we
can reveal the basic collaboration mode between designers and AI in the design
activities. These can help form an image of man-computer symbiosis in the age of
intelligence. See Fig. 3:
Fig. 3. Basic collaboration mode between the designer and the AI design system
At Divergent stage, the AI design system will execute most of the work at this
stage, using its powerful data analysis capabilities, while designers can focus more on
planning, communication and management. AI can help designers to think outside the
box and get more material and inspiration. the various preliminary solutions proposed
by the designer’s creative thinking can be tested by AI, and AI can also help designers
break the boundaries of thinking and discover more ideas that were not thought of
before.
At Convergent stage, the designer must manage the sorting of information and the
screening of possible solutions to find the right information and the best solution for the
project. The designer will dominate the stage of definition of requirements because of
the lack of empathy AI system cannot truly understand the real needs of users. The key
to the design is the need-finding, which is where the designer is best at. And at the final
designer needs to evaluate the final generated design plan from the perspective of the
planner and give criticism suggestions.
As the planner of design activities, the designer must clarify their role positioning,
have the ability to control each task node and be good at cooperating with AI. In the
design activities of man-computer symbiosis, understanding and following such a
design collaboration mode can know how to collaborate with the AI system, will
undoubtedly greatly improve the efficiency of the design work, and deliver the most
satisfactory design results.
Design Collaboration Mode of Man–Computer Symbiosis 645
4 Conclusion
The emergence of AI has not diminished the role of human beings. Actually, human
beings have always paid attention to respect for human personality when creating AI:
the development of AI technology has always been to help humans to better play their
roles according to human individuality. Social change promoted by AI will bring
mankind into a better world. In the future, the boundary between humans and com-
puters is bound to become increasingly blurred. The two are interdependent, and
indispensable to each other. Man–computer symbiosis is an irreversible trend. As the
main drivers of innovation, designers must always keep abreast of the latest social
trends. They must now learn how to cooperate with AI and find, in the right collab-
oration. The “Double Diamond Design Process Model” is one of the most popular
visual design processes at present. It guides the design work of many companies.
Therefore, based on the model, it can more clearly explain the collaboration mode be-
tween designers and AI in human-computer symbiosis in the era of intelligence.
At present, our research still has limitations. Since it only analyzes the functional
scope of AI design application systems, as applicable to the general field of design,
more professional, expert design systems have not appeared in the discussion. In the
future, based on a more in-depth study of the AI design system and other design
processes, it is hoped that more in-depth analytic results can be obtained to further
improve the collaborative mode of man-computer symbiosis design.
References
1. Li, X.G.: From independence to symbiosis-on the relationship evolution and future
development of designers and artificial intelligence. J. Art Educ. 09, 155–156 (2018)
2. Neilson, W.A., Knott, T.A., Carhart, P.W. (eds.): Webster’s New International Dictionary,
2nd edn. G. & C. Merriam Company, Springfield (1934)
3. Licklider, J.C.R.: Man-computer symbiosis. J. IRE Trans. Hum. Factors Electron. 3(1), 4–11
(1960)
4. Gill, K.S.: Human Machine Symbiosis: The Foundations of Human-Centred Systems
Design. Springer, Berlin (1996)
5. Norman, D.A.: Things That Make Us Smart: Defending Human Attributes in the Age of the
Machine. Diversion Books, New York (1993)
6. Alibaba Luban: AI-based. Graphic Design Tool. https://www.alibabacloud.com/blog/
alibaba-luban-ai-based-graphic-design-tool_594294
7. Lou, Y.S., Li, S.D.: Research on the trend of artificial intelligence design. J. Art Des. 2(07),
87–89 (2019)
8. Style AI: when artificial intelligence decides what is fashionable. http://www.255.it/en/Style-
AI-when-artificial-intelligence-decides-what-is-fashionable
9. What is generative design. https://generativedesign.wordpress.com/2011/01/29/what-is-
generative-desing/
10. Krish, S.: A practical generative design method. J. Comput.-Aided Des. 43(1), 88–100 (2011)
11. Project Dreamcatcher. Autodesk Research. https://autodeskresearch.com/projects/dreamcatcher
12. What is the framework for innovation? Design Council’s evolved Double Diamond. https://
www.designcouncil.org.uk/news-opinion/what-framework-innovation-design-councils-evolv
ed-double-diamond
User Experience over Time with Personal
Assistants of Mobile Banking Application
in Turkey
Hatice Merve Demirci1,2(&) and Mehmet Berberoğlu2
1
Atılım University, Kızılcaşar Mahallesi, İncek, Gölbaşı, Ankara, Turkey
2
Middle East Technical University, Üniversiteler Mahallesi,
Dumlupınar Bulvarı No: 1, 06800 Çankaya, Ankara, Turkey
{merve.demirci,mehmet.berberoglu}@metu.edu.tr
Abstract. Being able to chat with artificial intelligence-based applications has
been changing user interaction and has opened doors for a various number of
new user experiences. The new apps that emerged lately with new technologies
now launched in mobile banking applications as the form of personal assistants.
This paper aims to research what features of the assistants affect the overall user
experience. At the end of the study, a possible design guideline for designing
these assistants in Turkey discussed, since most of the existing guidelines are for
personal assistants for everyday life. Twelve users of Maxi, one of the most used
personal mobile banking assistant, participated in the research and evaluated
their experience after 15 days of usage. The research showed that their
impression was positive during the first interview but during the second inter-
view, there was a decline for all values of the impressions of user experience. In
the end, the results showed that the original application of the bank was more
preferable than the personal assistant.
Keywords: Personal assistants Mobile applications Mobile banking
applications Conversation design User engagement User engagement
patterns
1 Introduction
Technological advancements re-shaped user-product interactions by enabling people to
make conversation with interactive systems. This new form of interaction led people to
adopt a new type of interactions with their smartphones. Now, with the help of tech-
nological advances, it is possible to use mobile applications, which introduced with
smartphones, for various reasons. For instance, there are applications, which aim to
sustain subjective well-being, increase physical activities, gaming, flirting and for
doing banking related tasks. Using technology as a new product feature enabled people
to become familiar with it first and then began to prefer using and integrate such
applications in their everyday lives as routines.
Now in Turkey, most of the banks are offering “user-friendly” mobile banking
applications for their customers to satisfy their needs when and where they need.
Additionally, because of advanced technology, the accessibility of these services
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 646–652, 2020.
https://doi.org/10.1007/978-3-030-39512-4_100
User Experience over Time with Personal Assistants 647
affected the people’s expectations towards applications. As a result of these expecta-
tions, the overall experience offered by the mobile applications has shaped the
designers’ role as not just designing new interfaces but also fulfilling those expectations
and creating meaningful experiences that come with the use of the applications.
To maximize user satisfaction, mobile banking applications in Turkey started to
adapt interactive conversational agents into their systems. Nowadays, most downloaded
and used mobile banking applications are offering personal assistants for their customers
to accelerate transactions and increase customer satisfaction. Thus, the more techno-
logical advances become accessible, the more the designers’ role evolved around it.
Designers now started to reconstruct the technology and find new forms which
were non-existent before to integrate technology into products [1]. Additionally, pro-
viding an online experience independent of time and space attracted the interactive
system developers to create a domain-specific Conversational Agent, which is partic-
ularly focusing on banking. Thus, to maintain user engagement and sustain usage, the
design of such systems is important. However, existing design guidelines for Con-
versational Agents are limited and focus on business functions of these agents; how-
ever, the conversational experience should be the main focus for designers to sustain
user engagement.
In this study, the aim is to investigate how the features of personal assistants shape
the overall user experience of mobile banking application usage and its perceived
usability. Besides, throughout this study, we explored the design qualities of personal
assistants of mobile banking applications that focus on enhancing people’s user sat-
isfaction to obtain sustained engagement with the newly emerged technology. At the
end of the study, a design guideline for designing such assistants in Turkey proposed.
2 Literature Review
Existing interaction design guidelines are mostly offering ways to understand business
functions for commercial products. Personal Assistants with artificial intelligence often
feature access to other services that have utility in everyday life and can perform
different tasks. The potential of these assistants searched to understand how these
products integrated into mobile banking applications and how to improve service and
system functionality and technicality to improve service quality.
The history of conversational agents goes back to the 1960s and the first CA named
‘ELIZA’, which was created by Joseph Weizenbaum in 1966. The purpose of ELIZA
was to help people whom have psychological disorders while acting as a therapist [2].
The achievements ELIZA brought into the personal assistant development played an
important role in the creation of the new ones that blended with new technologies.
The development of technology and the application of developing technology to
existing mobile applications, these assistants have become very popular today.
According to [3], the developments in voice recognition, natural language processing,
and artificial intelligence, increased visibility and availability of these newly emerging
agents, which using text or natural language as their interaction medium to imitate
648 H. M. Demirci and M. Berberoğlu
human-to-human conversation. Due to being supported by diverse technological
advancements, and also being able to integrate into mobile applications or devices via
Conversational User Interfaces, Personal Assistants have a great potential in domain
specific applications [4]. In other words, being supported from diverse technologies and
mobile applications, people can use personal assistant in their daily lives without any
interruption to support their online banking transactions and as a result, they can
simplify their work with a single operation without the need for a few steps.
A growing number of people in Turkey uses online banking applications for several
years. The fact that people have used this technology for years and started to handle
daily banking transactions from these platforms has enabled users to get used to
receiving feedback from these systems. In addition to this, two of the major banks in
Turkey have started to integrate personal assistants into their applications to ease
banking transactions. Thus, according to [5] report in 2019 40% of people in Turkey
have started to use personal assistants while performing their online banking transac-
tions. Starting from this information, we aimed to investigate how people perceive
these assistants usability and how user engagement change over longitudinal usage. To
investigate this aim we select one of the most promoted personal assistants from one of
the major Turkish.
3 Research Method
The perceived usability and functional performance of the selected personal assistant
evaluated by the participants after 15 days of usage. The interpretations of the par-
ticipants obtained by semi-structured and in-depth interviews. Through analyzing the
results of the interviews, improvement suggestions presented regarding system oper-
ations and functionality in terms of the attractiveness, perspicuity, effectiveness,
stimulation, dependability, efficiency and novelty. Lastly, the study provided insights
that affect users’ engagement and overall experience in terms of online banking
application usage in Turkey.
3.1 Selected Personal Assistant
At the beginning of the study, we first chose the personal assistant that used for the
study; Maxi. Maxi was the most downloaded one between the two major banking
applications with personal assistants and available to users since November 2018. The
banking transactions that Maxi can perform according to self-system introduction from
its website are; expenditure advice, spending history, transactions, recent payments,
money transfer, foreign exchange transactions, and account and credit card balance
transactions.
3.2 Sampling
We reached the participants of the study with the announcement we made on social
media. The participants of the study were six women and six men who had banking
application with the personal assistant chosen by writing to Facebook group. The
User Experience over Time with Personal Assistants 649
interviews conducted with these 12 users. The participants selected among people aged
between 21 and 28 and prefer to use banking applications frequently during their daily
lives. Having equal number of men and women participants in the study enabled us to
observe whether sex, age, occupation, and previous experience affects the overall user
experience and perceived usability of these personal assistants.
3.3 Method
The purpose of this method was to evaluate the interaction design elements that con-
sidered as useful regarding the interaction design process of personal assistants with
artificial intelligence within a designated usage period. In this study, the specific usage
period of the selected personal assistant was two weeks, which designed as longitudinal
research.
Long-term user experience is the whole usage process of a product, its subjective
evaluation and how the meaning of that product perceived at the end of determined
usage duration. The longitudinal research was also important to observe the interaction
elements and system functionality affects the long-term usage patterns and user
engagement [6].
3.4 Procedure of the Research
At the beginning of the first interview, a brief introduction related to the duration of the
study, purpose, and procedure of the study introduced to the participants. Then they
asked to sign the content form of the study.
To know the participants closely and to better understand the propensity to use
technology and the habits of using the selected application; how often they use the
application, which transactions they perform mostly and the duration of usage of the
selected application asked. Afterward, they asked whether they had previous experi-
ence of personal assistant usage or not.
The second phase of the interview started with a brief introduction about the
personal assistants, their types and purposes each participant verbally. The system’s
self-introduction, from own main web site, was performed after the verbal brief
introduction. To access the main system application, a mobile phone used. From the
screen, the participants asked to view the Maxi. Apart from voice recording, partici-
pants’ statements written down to compare the change in the opinions of the partici-
pants after the first interaction with the system.
The first interaction performed during the first interview with the researcher. Before
the interaction, how to initiate an interaction with the Maxi was described verbally
without looking at the participants’ screens. First of all, the selected application was
opened, the profile of the Maxi tapped and afterward, the participants asked to perform
three tasks with Maxi.
With the end of the first interaction with the system, participants were asked to fill
the User Experience Questionnaire (UEQ) to evaluate personal assistant’s perceived
usability and user experience. Then in response to the question asked, they stated their
thoughts and feeling towards their first interaction. Before finishing the interview, they
were asked to inform the researcher if they want to leave the study.
650 H. M. Demirci and M. Berberoğlu
The second interview started with filling the second UEQ and continued with
comparing every 26 items of UEQ from the previously filled two of the UEQ in detail.
The comparison of the filled questionnaires aims to understand the underlying reasons
for participants’ evaluation of the Maxi’s characteristics, functionality, and personal
traits and how those aspects affected the overall user experience. In addition to UEQ
results comparison, the participants were asked to answer which personal assistant
aspect or aspects they liked the most and least, the reasons why they chose the stated
aspects of the system. Finally, at the end of the interview, the future Maxi system,
service, and interaction related developments were stated by the participants as an
answer to the “how the personal assistant should be improved in accordance to
you/your experience?”. The obtained data from each participant through interviews and
surveys were started to be analyzed after finishing the 24 interviews.
4 Results
The system evaluated as positive during the first by eight out of twelve participants.
Values between −0.8 and 0.8 represent a neural evaluation of the corresponding scale,
values > 0,8 represent a positive evaluation and values < −0,8 represent a negative
evaluation. Hence, the participants had a slightly positive or neutral impression con-
cerning the user experience of the Maxi. The impression concerning the pragmatic
quality (Perspicuity, Efficiency, and Dependability) is higher than the impression
concerning the hedonic quality (Stimulation, Novelty).
As understood from the graphs below, there has been a decline in all values of the
impressions of user experience, but the most dramatic changes were in the efficiency,
dependability, stimulation values (Table 1).
Table 1. First UEQ results
UEQ scales (mean and variance)
Attractiveness 1,597 1,37
Perspicuity 2,271 0,40
Efficiency 1,896 1,13
Dependability 1,375 0,80
Stimulation 1,229 1,33
Novelty 0,917 2,13
The decline in numeric data may not be negative yet the participants evaluate the
Maxi interaction negatively after fifteen days of interaction. In other words, during the
second interviews, the perceived usability and ease of use did not meet the expectations
of the participants. Maxi created an extremely positive impression concerning Per-
spicuity, but in terms of Attractiveness, Efficiency, Novelty, Dependability, and
Stimulation the Maxi judged slightly positive. Nonetheless, different from the positive
results, the participants did not positively comment on those scale related adjective
pairs (Table 2).
User Experience over Time with Personal Assistants 651
Table 2. Second UEQ results
UEQ scales (mean and variance)
Attractiveness 0,894 2,03
Perspicuity 1,886 1,03
Efficiency 0,727 2,09
Dependability 0,659 0,88
Stimulation 0,091 2,02
Novelty 0,864 1,40
5 Discussion and Conclusion
The participants of the study stated that they have been using the personal assistant
application for at least 2 years. Apart from the two participants, the other participants
did not know about the personal assistant that involved in this research, and stated that
they had never heard of it before. During the first interaction, the first task was to look
at the spending history and the participants were very pleased with the detailed graph
that Maxi presented to them. However, at the end of the fifteen days, the participants
stated that the personal assistant was not more useful than the application and did not
give them a convenience or speed for the processes that had the reasons for using the
application.
The evaluations of the participants showed which design aspects and qualities of
the system affected user engagement. In terms of perceived ease of use and usability, at
the end, the application was more preferable since the participants have gained the habit
of using mobile app while performing their banking transactions. Half of the partici-
pants told that during the 15 days period, they used the application less to not interact
with Maxi. Moreover, after the study they said that they would not use Maxi again.
At the end, we find four evidences that we can link with the interaction design
elements of the personal assistant. The participants asked for personalized interface
design to perform tasks faster. It is highly recommend being able to register the most
used transactions to the system to provide handling transactions in only one tab.
References
1. Porcheron, M., Fischer, J.E., Reeves, S., Sharples, S.: Voice interfaces in everyday life. In:
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, p. 640.
ACM, April 2018
2. Klopfenstein, L., Delpriori, S., Malatini, S., Bogliolo, A.: The Rise of Bots: A Survey of
Conversational Interfaces, Patterns, and Paradigms, pp. 555–565 (2017). https://doi.org/10.
1145/3064663.3064672
3. Laranjo, L., Dunn, A.G., Tong, H.L., Kocaballi, A.B., Chen, J., Bashir, R., Surian, D.,
Gallego, B., Magrabi, F., Lau, A.Y., Coiera, E.: Conversational agents in healthcare: a
systematic review. J. Am. Med. Inf. Assoc. 25(9), 1248–1258 (2018)
652 H. M. Demirci and M. Berberoğlu
4. Kocaballi, A.B., Laranjo, L., Coiera, E.: Measuring user experience in conversational
interfaces: a comparison of six questionnaires. In: Proceedings of the 32nd British Computer
Society Human Computer Interaction Conference, Belfast, Northern Ireland, July 2018
5. Türkiye’deki banka müşterileri chatbotla iletişimi benimsedi. https://www.cbot.ai/tr-blog/
turkiyedeki-banka-musterileri-chatbotla-iletisimi-benimsedi/
6. Karapanos, E., Zimmerman, J., Forlizzi, J., Martens, J.B.: User experience over time: an
initial framework. In: Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, pp. 729–738. ACM, April 2009
Human-Automation Interaction Through
Shared and Traded Control Applications
Mauricio Marcano1(&), Sergio Díaz1, Joshue Pérez1,
Andrea Castellano2, Elisa Landini2, Fabio Tango3, and Paolo Burgio4
1
Tecnalia Research and Innovation, 48160 Derio, Spain
{mauricio.marcano,sergio.diaz,
joshue.perez}@tecnalia.com
2
RE:Lab s.r.l, 42122 Reggio Emilia, Italy
{andrea.castellano,elisa.landini}@re-lab.it
3
Centro Ricerche FIAT, 10123 Turin, Italy
[email protected]
4
University of Modena and Reggio Emilia, Modena, Italy
[email protected]
Abstract. Automated and Highly-automated Vehicles still need to interact with
the driver at different cognitive levels. Those who are SAE Level 1 or 2 consider
the human in the loop all the time and require strong participation of the driver at
the control level. To increase safety, trust and comfort of the driver with this
kind of automation, systems with a strong cooperative component are needed.
This paper introduces the design of a vehicle controller based on shared control,
together with an arbitration system, and the design of a Human-Machine
Interface (HMI) to foster the mutual understanding between driver and
automation in a lane-keeping task. The driver-automation cooperation is
achieved through incremental support, in a continuum spectrum from manual to
full automation. Additionally, the design of an HMI to support the driver in a
takeover maneuver is presented. This functionality is a key component of
vehicles SAE Level 3 and 4.
Keywords: Human-machine interface Automated driving Highly automated
vehicles Shared control Traded control Driver-automation cooperation
1 Introduction
Since the beginning of automated driving, car manufacturers have embraced the idea of
removing the driver from the control loop. However, the technology is not yet mature
enough for the public [1], and social/legal acceptance issues [2] still represent a major
impediment. A second approach has been to increase the ADAS functionalities to
achieve a higher level of automation while considering the driver as an active agent.
This approach considers two modalities of human-machine interaction: shared control
[3], where driver and machine execute the same task at the same time (e.g., lane-
keeping assistance system), and traded control, where driver and automation execute
the same task at different times (e.g., an autopilot limited to highways).
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 653–659, 2020.
https://doi.org/10.1007/978-3-030-39512-4_101
654 M. Marcano et al.
The major issue of the former is achieving a comfortable interaction where the
system does not overwhelm the driver while preventing unsafety driver actions. On the
other hand, the challenge of the latter is handling the transitions between different
drivers’ roles (i.e., driver and passenger).
Future Automated Driving Systems (ADS) will leverage on the potential of both
approaches by developing gradual and incremental support to adapt to the ever-
changing cognitive and physical state and needs of the driver.
The Human-Machine Interaction strategy has a key role in these future systems
because it should be designed to exploit the potential of the technologies and the driver,
i.e. to get the most out of the existing driver’s skills and cognitive resources according
to the different roles he/she can have while the ADS is engaged.
In traditional systems, the role of the HMI is to inform/warn the driver while
minimizing the impact on distraction. Conversely, in highly automated systems, the
HMI should emphasize the authority on each task, to make the driver always aware of
his/her role. Moreover, the HMI should be designed to increase the trust in automation,
and reduce potential anxiety arising from the unexpected variation of roles. The success
of these interaction strategies relies less on extraordinary intelligence and more on
sophisticated negotiation of changing context and subsequent behavior [4].
In the PRYSTINE project, a novel ADAS is designed under the framework of fail-
operational systems, considering a driver evaluation platform, the risk of collision with
external agents and a controller to assist the driver in highway and urban scenarios.
Together with these components, the HMI strategy has been designed to smoothly
support the ADS, by including incremental levels of interaction to gradually support
the driver in the driving tasks. This paper makes an emphasis on the integration of the
lane-keeping assist controller with the visual HMI, as the first step of integration.
2 PRYSTINE Project
PRYSTINE [5] has been realizing Fail-operational Urban Surround perceptION
(FUSION) which is based on robust Radar and LiDAR sensor fusion and control
functions to enable safe automated driving in urban and rural environments.
The ADS developed in the project and described in this paper demonstrates the
potential of the FUSION hardware/software architecture and reliable components to
handle safety-critical situations which are not reached with not reliable state-of-the-art
approaches. The reference architecture for the integration of the modules of decision,
control and HMI used in this project is shown in Fig. 1.
Fig. 1. Reference architecture of module integration for driver-automation system
Human-Automation Interaction Through Shared and Traded Control Applications 655
3 Decision and Control System
Vehicles with a Level of Automation (LoA) SAE 1 and 2, where the steering control is
shared between the automation and the human, still require a strong interaction with the
driver. Meanwhile, those with LoA SAE 3 and 4 still require that the driver steps into
the loop under certain circumstances (more critical in L3).
In the PRYSTINE project, novel ADS is proposed where the system assists the
driver in a lane-keeping task with variable authority considering the driver state and the
potential risk of collision with external agents. The system increases the level of
intervention if it detects that the driver is distracted and there is a risk of collision, from
alerting the driver to taking full control of the vehicle in a fluid, comfortable and safe
manner. The design of such a system requires an adaptive decision and control system
able to continuously adapt to the context changes, together with a visual HMI which
support the mutual understanding of the driver-automation system.
3.1 Control System
The design of the lateral controller is based on a constrained Model Predictive Control
(MPC) for a lane-keeping task. To consider the driver-automation interaction, two
parameters are included within the problem formulation: (1) the Strength of Haptic
Feedback (SoHF), which is the maximum guidance force felt by the driver, and (2) the
Level of Haptic Authority (LoHA) which is related to the stiffness of the system around
the optimal command. These parameters allow to vary the authority of the system and
therefore share the control of the vehicle with the driver under different scenarios.
The first functionality allows to avoid a road departure applying a torque on the
steering. This is achieved including the lateral error as an MPC state and using the
corresponding constraint (1:5 eL 1:5Þ.
The second functionality of the controller is as lane-keeping assistance with vari-
able authority. This is done by including the reference trajectory to the optimization
function (min x xref ; y yref ; u uref ). Also, the variable authority gain is added
as the stiffness around the optimal steering angle ðKðh hopt ÞÞ. When the authority
gain reach its maximum value, the system is in full automation mode (but override of
the driver should be always possible).
The last two modes of operation are part of a continuum spectrum of system
intervention which depends on the driver state and environmental conditions of the
driving task, which are part of the decision system.
3.2 Decision System
When both the driver and the automation are part of the Dynamic Driving Task (DDT),
an arbitration system is necessary to distribute the authority over the control of the
vehicle. This authority is dependent upon the next variables: (1) driver’s status: ability
at a given time to execute the corresponding DDT. It is commonly measured with a
vision-based system which detects the cognitive states of the driver (e.g., distraction
and/or drowsiness level), (2) collision risk: time to a collision of the vehicle with
656 M. Marcano et al.
external agents. To correctly assess this risk, information about the environment is
needed, including the topology of the road, other agents around and traffic signs, and
(3) tracking: measure of vehicle performance for tracking a pre-defined route. It gen-
erally depends on the lateral and angular errors of the vehicle.
If the control is shared between the driver and the automation, the arbitration
module assigns the appropriate authority to each agent. For this task, a fuzzy logic
algorithm is proposed that receives three inputs (driver, risk, and tracking) and has as
its output the LoHA of the system. This output feeds the control module that distributes
the control authority. The driver is informed about this decision through the visual HMI
to foster the adoption of the correct role of the driver and increase the trust in
automation.
3.3 Use Case
Christine is bringing her baby Mark to the kindergarten when she notices that Mark is
tired and a bit nervous. Worried and in a hurry, she speeds up continuing looking in the
mirror to monitor the baby. After some kilometers, Mark loses the pacifier and starts
crying and yelling. Christine gets extremely nervous, tries to calm down the baby with
some words, but it seems not to work, so then tries to get the pacifier taking her eyes off
the road for some seconds. She cannot concentrate on the street and starts driving
erratically. Then, the activates a sequence of incremental support.
The ADS detect this risk behavior and suggest keeping her eyes on the road. Since
she maintains the wrong behavior and the lane deviations is becoming dangerous, the
ADS apply a micro-control strategy to avoid it, securing the lane boundaries for
avoiding an unintended lane departure, while assigning partial authority to the system
to help Christine in the lane-keeping task.
Even this support does not solve the situation, because Mark is crying, and Christin
is still distracted. So, the ADS informs Christine that the automation is available for
some kilometers, and the vehicle takes full control to allow her to look for the pacifier
and take care of Mark to calm him down.
Once in full automation mode, the ADS shows its autonomy of 1.4 km, and when
the minimum takeover time is reached (see Fig. 3), it alerts Christine to get ready to
take back the control. When she executes the takeover maneuver, the system starts
decremental support to smoothly go from full automation to fully manual.
3.4 Visual HMI Design
The HMI strategy has been designed for the specific use case of Christine, to
demonstrate the potential of the incremental support of the ADS developed in
PRYSTINE.
Human-Automation Interaction Through Shared and Traded Control Applications 657
Fig. 2. HMI for warning the driver in case of visual distraction
The HMI strategy has the aim to increase the awareness of the driver about the
mode of the automation and thus foster the adoption of the corresponding role of the
driver.
Explicit HMI graphic elements (i.e. icons) and text have been designed to clarify
who is in charge of the DDT. The layout of the HMI reflects the modes of the ADS.
Figure 2 shows the layout for the HMI Warning mode: in manual driving, the ADS
is engaged only to alert the driver in case his/her state has been detected as not
compatible with the DDT. Figure 3 shows the layout for the HMI Micro-control mode:
when an almost-imperceptible control is applied to avoid lane deviation (so both the
driver and automation steer) the HMI shows this support to increase the awareness of
the driver about his/her wrong driving behavior.
Fig. 3. HMI to inform the driver a micro-control is applied for lane keeping.
Figures 4 and 5 shows the layout for the HMI Automated mode: the HMI for the
Automated mode has a twofold objective: (1) to clarify the ADS is in charge of the
DDT when no imminent takeover is expected; (2) to foster the adoption of an active
role of the driver when the takeover is imminent i.e. 30 s.
658 M. Marcano et al.
Fig. 4. HMI for automated driving when no imminent take-over is expected
Fig. 5. HMI for automated driving when imminent take-over is expected
4 Conclusions and Future Work
The HMI strategy has been designed to reflect the incremental levels of interaction of
the ADS and smoothly and continuously support the driver according to his/her state
and current role in the DDT. The performance of the ADS and its HMI will be tested in
a driving simulator to assess its impact on safety and comfort-related parameters, using
real subjects, considering a selection criterion defined by the PRYSTINE project.
Acknowledgments. PRYSTINE has received funding within the Electronic Components and
Systems for European Leadership Joint Undertaking (ECSEL JU) in collaboration with the
European Union’s H2020 Framework Programme and National Authorities, under grant.
References
1. Brown, B., Laurier, E.: The trouble with autopilots: assisted and autonomous driving on the
social road. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing
Systems. ACM (2017)
2. Marchant, G.E., Lindor, R.A.: The coming collision between autonomous vehicles and the
liability system. Santa Clara L. Rev. 52, 1321 (2012)
Human-Automation Interaction Through Shared and Traded Control Applications 659
3. Sheridan, T.B., Verplank, W.L.: Human and computer control of undersea teleoperators.
Massachusetts Inst of Tech Cambridge Man-Machine Systems Lab (1978)
4. Ju, W.: The design of implicit interactions. Synth. Lect. Hum.-Centered Inf. 8(2), 1–93 (2015)
5. Druml, N., Macher, G., Stolz, M., Armengaud, E., Watzenig, D., Steger, C., Herndl, T.,
Eckel, A., Ryabokon, A., Hoess, A., Kumar, S.: PRYSTINE-PRogrammable sYSTems for
INtelligence in automobilEs. In: 2018 21st Euromicro Conference on Digital System Design
(DSD), pp. 618–626. IEEE, August 2018
Alignment of Management by Processes
and Quality Tools and Lean to Reduce Unfilled
Orders of Fabrics for Export: A Case Study
Z. Bardales1, P. Tito1, F. Maradiegue1, Carlos Raymundo-Ibañez1(&),
and Luis Rivera2
1
Ingeniería Industrial, Universidad Peruana de Ciencias Aplicadas (UPC),
Lima 15023, Peru
{u201320009,u201217564,fmaradie,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. The problem of non-conforming production has been identified in
the dry-cleaning area of a company dedicated to the manufacture of fabrics. The
alignment of concepts of Business Process Management (BPM) and total quality
control (TQM) is applied in order to reduce the problem of non-fulfillment of
orders by eliminating activities that do not generate value. The implemented is
simulated, using Bizagi, macros and minitab. A higher score was achieved in the
5S audit at the end of the implementation, reduction of times and failures, both
in the measurement for the supply and in the occurrence of machine failures due
to the lack of preventive maintenance.
Keywords: BPM TQM Textile Industry Dry cleaner
1 Introduction
The textile industry and apparel is an important element in the economy of developing
countries, so it is a case study that can allow obtaining favorable results for the
contribution in the country that is developed, taking into account its productive pro-
cesses. In Peru, it contributes 1.9% of GDP, and involves approximately 14, 000 jobs
per year [1].
The present article has as structure the development of the analysis of a case study
of a textile company that is in charge of the production of fabrics for export. In this, you
can identify the main problem presented by the company, the impact it generates and
their respective direct and indirect causes. Afterwards, the process of applying the
innovative technique and the respective validation and presentation of the results can be
visualized.
The implementation of process management ensures the increase of productivity in
textile companies [2]; in Holland and Portugal improved flexibility in the processes of
different industrial sectors [3]. The 5’S manages to improve the quality of the products
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 660–666, 2020.
https://doi.org/10.1007/978-3-030-39512-4_102
Alignment of Management by Processes and Quality Tools 661
through the elimination of waste [4]. The Poka-Yoke tool is necessary for the reduction
of human failure within the processes [5]. The correct application of the TPM improves
the yield of the manufacture and the effectiveness of the product in the textile com-
panies [6, 7].
Qualitative Analysis of the Current Situation of the Company: The process of
identifying the central problem that causes the high rate of reprocessing within the
company starts with the expert judgment, obtaining three central problems: High rate of
non-fulfillment of orders in the dry cleaning area. Offset on the delivery date. Pro-
duction Reprocesses.
Identification of the Problem: In this step, we proceeded first to identify the standard
product that the organization has, based on the basis that it produces only knitted fabric.
The analysis ABC (Fig. 1) is used, which analyzes at the level of production and
income by product or family of products, obtaining as main the Jersey.
Fig. 1. ABC analysis
Analysis of Causes: For the analysis of the causes, an analysis was carried out for
each subprocess that the dry-cleaning process has: Batch assembly, chemical process
and dyeing.
It began by performing an Ishikawa analysis for each of them, finding various root
causes, which were subsequently grouped. It is important to mention that most of these
falls on the part where the operator intervenes.
Afterwards, to carry out the aforementioned grouping, an AMFE analysis was
carried out, which allowed determining which root causes should be analyzed due to
the NPR obtained, being as follows:
– X1: Dirty and messy work surface
– X2: Incorrect follow-up to the maintenance program
– X3: Poor reproducibility of dye supply
– X4: Breach of activity times (Incorrect handling of fabric due to lack of knowledge
of procedures to follow)
662 Z. Bardales et al.
2 Proposal
The innovative technique to be implemented will be the BPM supported with TQM
tools. This aims to create synergy between how to organize the processes and the value
added to each of these.
When talking about BPM, not only is there a correct structure of processes to
achieve a correct management within an organization, therefore it is proposed within
the innovative technique the implementation of this in conjunction with quality tools.
In a study by Nadarajah, Devika; Syed A. Kadir, Sharifah Latifah [8] makes mention of
the importance of integration between process management and the continuous
improvement of these, since they should be seen as a complement.
The proposal is reduced to a better communication between processes for the solution
of the existing problems and the increase of the added value to this by means of the
continuous improvement. Companies usually consider these aspects unlinked, since they
are used for different purposes, the BPO (measuring instrument to determine the impact
on the overall performance of the business through financial and nonfinancial indicators)
to manage the orientation of the processes and the PII (implemented as TQM, BPR or
Benchmarking) as assurance of the quality of each process; However, they do not realize
that companies that are capable of understanding that BPM is a combination of both
strategies may have a broader perspective to carry out proper process management.
To achieve the objective of the planning of the activities and optimization of these,
tools of the Lean Manufacturing methodology will be used. This is responsible for
continuous improvement and optimization of a production system by reducing wastage
of all kinds: inventories, times, defective products, transportation, rework by teams and
people. The tools of this technique are involved with the BPM since its novelty consists
in the combination of different elements and applications to reduce the mentioned
waste at the same time. [9] This is intended to involve the improvement of processes
with their management.
Finally, management evaluations are carried out to ensure that the process con-
tinues to satisfy the client’s requirements, remains in line with the company’s objec-
tives and remains a competitive advantage (Fig. 2).
Fig. 2. Innovative proposal
Alignment of Management by Processes and Quality Tools 663
3 Results
The implementation of the tools was done according to each independent variable
obtained. In the same way, the effectiveness of each of them is analyzed, being as
follows (Table 1):
Table 1. Validation tools
Tool Validation resource Stage Information
5S 5S Audit Implemented Based on the results
according to of the audit
schedule
BPM Bizagi Implemented Based on the times
according to after
schedule implementation
Poka Yoke Analysis R&R Implemented Based on the times
according to after
schedule implementation
Quality of the New calculation of Implemented Based on the
preventive machine maintenance according to records
program availability schedule implemented
3.1 5S Validation
For the validation of the 5S implementation, the final audit of the workstation is carried
out in the entire dyeing process: batch assembly, prior chemical preparation and
dyeing. According to this, it was obtained that the score was increased from 29 points
obtained initially, to 40, as shown in the table. Likewise, in the radal, it is verified the
“S” that have had variation and improvement due to the implementation (Table 2).
Table 2. Comparison of audits before and after implementation
664 Z. Bardales et al.
3.2 BPM Validation
For this validation, the simulation was used in the Bizagi program, in this way the
situation is plotted before the improvement and after the improvement, obtaining as a
result, the amount of reprocessing in each scenario (Fig. 3).
Fig. 3. Process with improvement
Likewise, the ANOVA analysis is performed again to verify the standardization of
the processes through the analysis of the means (Fig. 4).
Fig. 4. ANOVA analysis of the activities
Finally, the results of the implementation of this tool are reflected in the reduction
of lots reprocessed by 3% as shown in the table (Table 3).
3.3 Poka - Yoke Validation
For the validation of this tool we proceeded to perform a test with samples for the
supply of dye. The analysis was performed by entering data into the Minitab program
and again using repeatability and reproducibility. Likewise, this is done to three
operators of different shifts as in the previous case (Table 4).
Alignment of Management by Processes and Quality Tools 665
Table 3. Comparison of times (minutes) and percentage of reprocessing
Process Current time (simulation) Projected time (simulation) Variation
Armed batch 233.42 136.98 41%
Chemical preparation 227.51 205.99 9%
* Chamuscado 47.62 42.63 10%
* Chemical bleaching 85.36 76.87 10%
* Mercerizado 94.53 86.49 9%
Tinturado 1391.27 1167.27 16%
Total 1852.2 1510.24 18%
Reprocesses 8.58% 5.45% 3%
Table 4. Number of errors per measurement test
Operator Request Number of errors (current) Number of errors (pilot)
1 4.5 gr. Of color 19 of 30 15 of 30
2 4.5 gr. Of color 23 of 30 18 of 30
3 4.5 gr. Of color 21 of 30 16 of 30
As shown in the previous table, the number of errors decreased with the imple-
mentation of the pilot, with the current average of inaccuracy being 21/30 and the
simulated 16/30. It is expected that with the implementation of the final version of the
proposed PokaYoke system, 98% accuracy in the required weights will be achieved
according to the characteristics of the software.
4 Conclusion
In conclusion, the effectiveness of the proposed innovative technique is obtained, since
the normal objective established at the beginning of the investigation was achieved. On
the other hand, it is possible to consider and evaluate the decrease of agents that
intervene in the process and produce the expected non-effectiveness of the results for
each tool. Finally, the case study will find opportunities for improvement that can
complement it, because it must be taken into account that the implementation can be
applied in any area, which gives the opportunity to find variants in each of them that are
adaptable and generalized and you can get a greater efficiency of this.
References
1. APTT. La Industrial Textil y Confecciones (2016). http://apttperu.com/la-industria-textil-
yconfecciones/
2. Ponce Herrera, Katherine Cecilia (2016). Recuperado de: https://repositorioacademico.upc.
edu.pe/handle/10757/620981?locale=es&language=es&locale-attribute=es
666 Z. Bardales et al.
3. Janssen, K.J., Revesteyn, P.: Business processes management in the netherlands and portugal:
the effect of BPM maturity on BPM performance. J. Int. Technol. Inf. Manag. 24(1), Article 3
(2015)
4. Pheng, L.S., Khoo, S.D.: Team performance management: enhancement through Japanese 5S
principle. TQM Mag. 7(7/8), 105–111 (2001)
5. Vinod, M., Devadasan, S.R., Sunil, D.T., Thilak, V.M.M.: Six Sigma through Poka Yoke: a
navigation through literature arena (2015)
6. Wickramasinghe, G.L.D., Perera, A.: Effect of total productive maintenance practices on
manufacturing performance: investigation of textile and apparel manufacturing firms.
J. Manufact. Technol. Manag. 27(5), 713–729 (2016)
7. Lastra, F., Meneses, N., Altamirano, E., Raymundo, C., Moguerza, J.M.: Production
management model based on lean manufacturing for cost reduction in the timber sector in
Peru. In: Advances in Intelligent Systems and Computing, vol. 971, pp. 467–476 (2019)
8. Nadarajah, D., Syed A. Kadir, S.L.: Measuring business process management using business
process orientation and process improvement initiatives. Bus. Process Manag. J. 22(6), 1069-
1078 (2016). https://doi.org/10.1108/BPMJ-01-20140001
9. Jauregui, R., Pamela, A., Gisbert Soler, V.: Lean manufacturing: herramienta para mejorar la
productividad en las empresas. In: 3C Empresa: investigación y pensamiento crítico, Edición
Especial, pp. 116–124 (2017). http://dx.doi.org/10.17993/3cemp.2017.especial.116-124
Detection and Prevention of Criminal Attacks
in Cloud Computing Using a Hybrid Intrusion
Detection Systems
Thierry Nsabimana1(&), Christian Ildegard Bimenyimana1,
Victor Odumuyiwa2, and Joël Toyigbé Hounsou1
1
Institut de Mathématiques et de Sciences Physiques
de l’Université d’Abomey-Calavi, Porto-Novo, Benin
{thierry.nsabimana,
christian.bimenyimana}@imsp-uac.org,
[email protected]
2
University of Lagos, Lagos, Nigeria
[email protected]
Abstract. In this paper, we provide a cloud based Hybrid Intrusion Detection
and Prevention System using signature based method and Genetic Algorithm to
defeat DDOS/DOS attacks attempting to compromise the three security goals
known as “CIA” or Confidentiality (C), Integrity (I) and Availability (A) of cloud
services and resources. We apply Snort-IDS with a combination of Splunk web
framework (tool for visualization) to detect and prevent DDOS/DOS attacks
based on signature rules. Moreover, to be able to mitigate known/unknown cloud
attacks, anomaly detection approach is built using Genetic Algorithm. We deeply
analyse, explore the existing Snort-IDS rules for DDOS/DOS attacks, and pro-
vide some improvement on the evaluated Snort-IDS rules. Through the analysis
of the experimental results, we conclude that our approach could be incorporated
in cloud service models to reduce these attacks.
Keywords: Genetic Algorithm IDS Splunk Snort-IDS Cloud
computing DDOS/DoS
1 Introduction
Cloud computing aims at providing a shared pool of resources (e.g., computing power,
storage, network), on-demand to cloud users. The three major cloud service models are:
Software as a Service (e.g. Salesforce, GoogleApps), where cloud users are provided
the capability of accessing the software running on a cloud environment, Platform as a
Service (e.g. Windows Azure), where cloud users are provided the capability of cre-
ating their own applications and deploy them onto the cloud infrastructure, and
Infrastructure as a Service (e.g. Microsoft Azure), where cloud providers provide the
capability of accessing computing power, file storage and network.
Moreover, cloud service models has four possibilities of deployment models such
as public, private, community and hybrid cloud. Public cloud infrastructure is provided
to the general public and owned, managed, operated by cloud providers and whichever
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 667–676, 2020.
https://doi.org/10.1007/978-3-030-39512-4_103
668 T. Nsabimana et al.
public organization. For instance, the services, which should be available for public
users, are social media, E-commerce, E-learning, E-mail services, etc. Private cloud
infrastructure is supplied for only one organization including several cloud users. It is
owned, managed, operated by whichever private organization [1] providing the max-
imum control over data storage, data security, and quality of service to the cloud users.
Community Cloud is a cloud deployment type that focuses on a community of users
which could be multiple organizations having the same global mission [5] (e.g.,
insurance healthcare, policy, or compliance considerations, university) and may be
operated by specific organization. Hybrid cloud infrastructure is a combination of two
or more cloud deployment models as shown above. With the increasing growth of
cloud-computing technology and services, public as well as private companies are
willingly moving their applications and services into the cloud and subscribing to
various cloud services. Due to this, criminal intruders or hackers attempt to penetrate
cloud service models and launch different kinds of attacks against them. It has been
observed that both cloud users and cloud providers are facing [3] the major issues of
data availability, reliability and data security. Our concern is to ensure that the three
security goals known as “CIA” or Confidentiality (C), Integrity (I) and Availability
(A) of cloud resources and services are not compromised especially through attacks
like DDOS/DOS.
DDOS/DOS attacks are often infiltrating over cloud services and making them
useless. An efficient way to defeat these attacks is to use the Intrusion Detection
Systems (IDS) and Intrusion Prevention Systems (IPS) to enhance the security of cloud
infrastructure and services.
An IDS is broken down into three fundamental methodologies of intrusion
detection approaches namely Misuse/Signature Detection Approach (SDA), Anomaly
Detection Approach (ADA) and Hybrid Detection Approach (HDA). ADA discovers
and detects network attacks based on deviations from established normal behaviour.
A security software based on ADA will trigger an alarm when an unknown behaviour
is detected. When a normal behaviour is wrongly identified as an attack by a system, it
is considered as false positive. In cases where the system takes abnormal behaviour as
normal behaviour, it is a false negative [4]. Despite generating very high false positive
rate, ADA has strong capabilities of detecting known or novel network attacks. SDA
detects and prevents a well-known network attacks based on known signature rules or
patterns saved in the knowledge database. A security software based on “Misuse
detection” generates very low false positive because all abnormal activities are pre-
defined in the knowledge database [4]. HDA is a combination of the two approaches
listed and described above.
There are different categories of IDS used on cloud computing namely: Host based
IDS, Network based IDS, Hypervisor based IDS and Distributed IDS. Host based IDS is
deployed and monitored in real time for every host computer, and it is incorporated on
business critical computer host as well as DMZ’s servers that are likely to be compro-
mised [4]. Network based IDS is installed at the point of entry of a network and monitoring
the incoming/outgoing traffic through its network segment [4]. Hypervisor/Virtual
Machine based IDS controls and monitors intrusion detection for each virtual machine
[3]. «Virtual Machine Monitor (VMM) or hypervisor is the software layer providing the
virtualization, which allows the multiplexing of the underlying physical machine between
Detection and Prevention of Criminal Attacks in Cloud Computing 669
different virtual machines, each running its own operating system» [3]. Distributed IDS
(DIDS) is a mixture of multiple IDS (e.g. HIDS, NIDS, etc.) over a large network, each of
them communicates with each other, or supported by a central server that enables network
monitoring [2].
The main goal of this paper is twofold. First, we provide a Hybrid Intrusion
Detection and Prevention System; by applying soft computing algorithm namely,
Genetic Algorithm (GA) to analyse, detect and prevent DDOS/DOS attacks for each
cloud cluster. For demonstration purposes, we train and test our approach using net-
work traffic packets captured by wireshark. Second, we apply the already existing
Network Intrusion Detection System namely Snort-IDS with a combination of Splunk
web framework to detect and prevent DDOS/DOS attacks based on signature rules. In
addition, we deeply analyse, explore the existing Snort-IDS rules for DDOS/DOS
attacks, and provide some improvement on the analysed Snort-IDS rules. In order to be
able to mitigate against other known/unknown cloud attacks, we built an anomaly
detection approach using Genetic Algorithm. We tested our improved Snort-IDS rules
and the results showed an improvement in DDOS/DOS attacks detection. In addition,
the experimental results confirmed that our proposed anomaly detection approach using
GA is more efficient in detecting DDOS/DOS attacks as compared to the detection rates
from the improved Snort-IDS rules.
The remainder of the paper is structured as follow: Sect. 2 presents problem
statement, Sect. 3 presents related work. Section 4 introduces background on proposed
IDS/IPS approach and Snort-IDS software. In Sect. 5 presents the proposed Hybrid-
NIDS structure for cloud computing. In Sect. 6 presents the implementation and results
of GA based IDS/IPS approach and Snort IDS software. Finally, Sect. 7 talks con-
clusion and future work.
2 Problem Statement
The Security has always been the main issue for public as well as private companies
migrating to cloud computing. Cloud computing suffers from DDOS/DOS attacks
performed by external and internal attackers. These security issues may decrease
rapidly the rates of using cloud computing. For that reason, cloud providers must
secure adequately cloud resources in taking into account rigorously the three key of
security purpose known as “CIA” or Confidentiality, Integrity and Availability of
resources and services in cloud computing. There are commercial as well as open
source IDS tools, that can efficiently defeat network attacks, but some of them have the
drawback of generating very high false positive resulting from the usage of anomaly
detection approach and very high false negative resulting from the usage of
misuse/signature detection approach. It is however important to note that developing
security solutions applicable to cloud computing demands a thorough understanding of
cloud architecture and environment.
670 T. Nsabimana et al.
3 Related Work
Security of cloud is of great concern for both cloud users and cloud providers. Security
issues in cloud environment have attracted the attention of several researchers and quite
a number of researches have been carried out in this domain. Hameed et al. [1]
provided cloud-based IDS to reduce some risk of intrusion on the cloud networks and
to overcome some limitations of already existing IDSs. Their solution is deployed as a
Software as a Service model for securing cloud users from intrusion attack. They set up
“a hardware layer, which forwarded the request to a service layer, which analyses the
request patterns, and further forwards to the intrusion detection layers, which checks
the request and provides the alerts against the malicious requests by applying genetic
algorithm” [1]. Patil et al. [6] discussed cloud security issues. They also implemented
an IDS in which they used genetic algorithm to detect and also prevent intrusion attack.
Kuldeep et al. [7] showed that many of the existing IDSs are designed to handle
specific attacks. There is no “one size fits all” technique; hence, no single technique can
guarantee protection against all future attacks. They developed an architecture that
adapts the snort IDS to cloud infrastructure in order to provide security against
intrusion attacks.
4 Background on Proposed IDS/IPS Approach
Proposed solution for security issues is to integrate an IDS/IPS over cloud computing
environment. The position of the IDS/IPS within the network, the detection techniques
it deploys and its (IDS/IPS) configuration are factors that determine its efficiency [2]. In
this section, we present our proposed Genetic Algorithm approach based IDS/IPS and
Snort-IDS which would be incorporated in cloud environment to overcome the security
issues of resources and services.
4.1 Genetic Algorithm Based IDS/IPS Approach
Genetic Algorithm [11, 12] is an evolutionary algorithm used for optimization pur-
poses. In applying GA to solve network problems, the features in the dataset are
represented as chromosome [12]. Gong et al. [13] and Hounsou et al. [12] used seven
features (Duration, Protocol, Source–port, Destination–port, Source–IP, Destination–
IP, Attack–name) of captured packet. They used Fitness function to evaluate each
individual in data set for the purpose of finding the best solution and discarding the
worst one. Support confidence based fitness function are used for delivering rules,
which in turn are applied in classifying network intrusions. The operators of GA
include selection, crossover or recombination and mutation. They are used to produce
new population. Based on our previous work, an example of the rules used to classify
network connection in detecting U2R attack rlogin from training data is explained
below [12]:
if (Duration = 0:0:14 and Protocol = rlogin and Source–port = 1022 and
Destination–port = 513 and Source–ip = 192.168.1.30 and Destination–ip =
192.168.0.20) then Attack–name = rlogin.
Detection and Prevention of Criminal Attacks in Cloud Computing 671
“This rule shows that if a network packet is originated from source IP address
192.168.1.30 and source port 1022, and sent to destination IP address 192.168.0.20 and
destination port 513 using the protocol rlogin, and the connection duration is 0:0:14 s,
then it is a network attack of type rlogin”.
4.2 Snort-IDS Software
Snort NIDS is an open source security network IDS/IPS invented by Martin Roech in
1998. It is configurable and freely available for multiple platforms (i.e. GNU/Linux,
Window) [10]. The actions used by Snort are alert, drop, block, ignore, pass, reject,
active and dynamic. Snort can be worked with other tools for better outputs: SnortS-
narf, Snorby, BASE, OSSIM and OSSEC. The signature based IDS model used in
Snort matches incoming attack signature with stored signatures associated with known
attacks like the pod, portsweep, DoS-nuke, Teardrop, and Saint, etc. [10]. The detection
engine of Snort allows registering, alerting and responding to any known attack [10].
Snort cannot identify novel attacks. Snort IDS can be used in Cloud environment if we
keep updating signature rules database.
5 Proposed Hybrid-NIDS Structure for Cloud Computing
The purpose of this work is to provide an efficient Hybrid-IDS/IPS, which can detect
any internal or external attack. We present a proposed design of Hybrid-IDS/IPS
structure for cloud and show how it can be deployed in cloud computing environment
to solve the security issues on it.
Our proposed Hybrid-NIDS architecture for cloud environment is divided into eight
modules (as depicted in Fig. 1). We use Misuse/Signature detection approach using
Snort-IDS and an Anomaly detection approach using Genetic Algorithm. Snort-IDS is
used to detect known attacks based on signature rules, whereas GA can detect known
as well as novel attacks. As Snort-IDS is configured with wireshark, the incoming
traffic packets pass through Snort-IDS, in such way, the known attacks are detected
based on signature rules saved in database. Whether traffic packets matched the rules,
alert messages must be generated. The generated alerts are forwarded to the logging
and alerting system for future output log or alert file. The alerts are then forwarded to
the Splunk for visualization. For the anomaly detection, the incoming traffic packets are
preprocessed and passed through Genetic Algorithm to detect both known and
unknown attacks.
As depicted in Fig. 2, we deploy Hybrid-NIDS model in each cloud environment.
There are one or more cloud clusters. Each cloud cluster contains one or more VM.
Hybrid-NIDS model is integrated in each cloud cluster to monitor, detect and prevent
internal and external attack. A Central Hybrid-NIDS model is positioned in cloud to
link one cloud cluster to another one.
672 T. Nsabimana et al.
Fig. 1. Design of proposed Hybrid-NIDS architecture for cloud
Fig. 2. Deploying of our proposed Hybrid-NIDS in cloud environment
6 Implementation and Results
6.1 Snort-IDS and Splunk for Snort Results
Table 1 contains the existing Snort rules found out on [8]. We analyzed, explored and
tested them using our Hybrid-NIDS/IPS. As results, we noticed that some options
should be added to gain a high-level detection rate. Figure 3 shows attacker source IP
address and victim destination IP address from Splunk.
Table 2 contains improved Snort rules. After analysing and improving, we tested
them using our implemented system; we noticed that our improved Snort-IDS rules
Detection and Prevention of Criminal Attacks in Cloud Computing 673
Table 1. Sample of existing Snort-IDS rules [9]
ID Rules
1 alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg: “SLR Alert-
HOIC Generic Detection with booster - HTTP 1.0/Header Double Spacing”; flow:
established, to_server; content: “User-Agent|3a 20 20|”; nocase; content: “HTTP/1|2e|
0”; nocase; reference: url, blog.spiderlabs.com; threshold: type both, track by_src, count
15, seconds 30; classtype: slr-tw; sid:1; rev:1;)
2 alert tcp any any −> 192.168.1.107 any (msg: “FIN Flood Dos”; flags: F; sid: 1000006;)
3 alert tcp any any −> 192.168.1.107 any (msg: “SCAN SYN-FIN”; sid: 1000001; flags:
SF; rev: 1;)
4 alert tcp any any −> 192.168.1.10 any (msg: “TCP Flood”; sid: 1000001;)
5 alert icmp any any −> any (msg: “Smurf Dos Attack”; sid: 1000003; itype: 8;)
Table 2. Sample of improved Snort-IDS rules
ID Rules
1 alert tcp $ EXTERNAL−NET any −> $ HOME−NET $ HTTP−PORTS (msg: “High
Orbit Ion Cannon (HOIC) tool attack detected”; flow: established, to−server; content:
“|55 73 65 72 2d 4167 65 6e 74 3a 20 20|”; nocase; content: “HTTP 1.0|2e|0”; nocase;
threshold: type both, track by−src, count 2, seconds 2; sid: 1000049; rev: 1;)
2 alert tcp $ EXTERNAL−NET any −> $ HOME−NET any (msg: “Ping of death attack
detected with Fin packets requests”; window: 512; detection−filter: track by−src, count
100, seconds 3; classtype: attempted-dos; reference: url, www.imsp-benin.com; sid:
1000050; flags: F;)
3 alert icmp $ EXTERNAL−NET any −> $ HOME−NET any (msg: “Ping of death attack
detected with Smurf DDOS ICMP packets”; detection−filter: track by−src, count 30,
seconds 1; itype: 8; icode: 0; classtype: attempted-dos; reference: url, www.imsp-
benin.com; sid: 1000051; rev: 1;)
4 alert tcp $ EXTERNAL−NET any −> $ HOME−NET any (msg: “Ping of death attack
detected with SYN-FIN packets requests”; window: 512; detection−filter: track by−src,
count 30, seconds 1; classtype: attempted-dos; reference: url, wwww.imsp-benin.com;
sid: 1000052; flags: SF; rev: 1;)
5 alert tcp $ EXTERNAL−NET any −> $ HOME−NET any (msg: “DOS attack with
GoldenEye tool”; detection−filter: track by−src, count 200, seconds 5; classtype:
attempted-dos; reference: url, wwww.imsp-benin.com; sid: 1000053; rev: 1;)
Table 3. Detection results from Snort-IDS
Attack types True positive False positive False negative Running time
ICMP Smurf 99.97% 0.00% 1.56% 11.58 s
TCP Flooding 99.95% 0.001% 0.049% 14 s
HTTP Goldeneye 73.85% 0.00% 36.33% 41 s
TCP SYN FIN Flood 49.96% 0.00% 50.03% 13 s
TCP HOIC 0.14% 0.00% 99.86 1.6 s
674 T. Nsabimana et al.
(a) Detected Attacker source IP addresses (b) Detected Victim Destination IP address
Fig. 3. Attacker source IP address and victim destination IP address from Splunk.
detect successfully DDOS/DOS attacks. The obtained results can be seen in Table 3.
ICMP Smurf DDoS attack is detected with very high detection rate over 99%.
6.2 GA Based IDS Results
Table 4 depicted the results generated by GAIDS. Comparing the results generated by
our implemented and the one generated by Snort IDS (see in Table 3), we can conclude
that GAIDS performed well with the corresponding high detection rates and low false
positive rate.
Table 4. Detection results from GA
Attack types TP FP FN Running time
ICMP Smurf 100% 0.00% 0.00% 26.34 s
TCP Flooding 100% 0.00% 0.00% 282.81 s
HTTP Goldeneye 99.09% 0.00% 0.90% 20.59 s
TCP SYN FIN Flood 99.97% 0.00% 0.02% 489.26 s
TCP HOIC 100% 0.00% 0.00% 192.38 s
6.3 Detection Rate Comparison Between GA and Snort-IDS
Table 5 compares GA based IDS and Snort IDS. The results of the tested rules using
Snort-IDS software have shown less performance than Genetic algorithm. The GA
based IDS detects over 90% with low False Positive whereas detection rate of Snort-
IDS is between 49%–99.97% with low False Positive.
Detection and Prevention of Criminal Attacks in Cloud Computing 675
Table 5. Detection rates comparison between GA and Snort IDS
Approach Attack types TP FP TN Running time
GA ICMP Smurf 100% 0.00% 0.00% 26.34 s
TCP Flooding 100% 0.00% 0.00% 282.81 s
HTTP Goldeneye 99.09% 0.00% 0.90% 20.59 s
TCP SYN FIN Flood 99.97% 0.00% 0.02% 489.26 s
TCP HOIC 100% 0.00% 0.00% 192.38 s
Snort-IDS ICMP Smurf 99.97% 0.00% 1.56% 11.58 s
TCP Flooding 99.95% 0.001% 0.049% 14 s
HTTP Goldeneye 73.85% 0.00% 36.33% 41 s
TCP SYN FIN Flood 49.96% 0.00% 50.03% 13 s
TCP HOIC 0.14% 0.00% 99.86 1.6 s
7 Conclusion and Future Work
In this work, a Genetic Algorithm approach based IDS and Snort-IDS were used to
mitigate DDOS/DOS attacks in cloud environment. The hybrid-NIDS architecture and
its deployment on cloud was presented. We improved the Snort-IDS rules for the
DDOS/DOS attacks. The results of the tested rules using Snort-IDS software, HOIC
tool and Kali lunix OS prove that our improved Snort IDS rules can efficiently detect
the DDOS/DOS attacks in cloud. The experimental results confirm that Genetic
algorithm outperforms the improved Snort-IDS, it detects significant number of
DDOS/DOS attacks with very low false positive rate and false negative rate. Never-
theless, running time is critical comparing with Snort-IDS. Considering the detection
rates, GA based IDS should be incorporated in cloud environment to mitigate
DDOS/DOS attacks. For future work, we will apply deep learning convolutional neural
network with other open source network intrusion detection and prevention system like
Suricata-IDS to detect and reduce DDOS/DOS attacks.
References
1. Hameed, U., Naseem, S., Ahamd, F., Alyas, T., Khan, W.A.: Intrusion detection and
prevention in cloud computing using genetic algorithm. Int. J. Sci. Eng. Res. 5 (2014)
2. Modi, C., Patel, D., Borisaniya, B., Patel, H., Patel, A., Rajarajan, M.: A survey of intrusion
detection techniques in cloud. J. Netw. Comput. Appl. 36(1), 42–57 (2013)
3. Bhat, A.H., Patra, S., Jena, D.: Machine learning approach for intrusion detection on cloud
virtual machines. Int. J. Appl. Innov. Eng. Manag. (IJAIEM) 2(6), 56–66 (2013)
4. Bunel, P.: An Introduction to Intrusion Detection Systems. SANS Institute, GIAC Security
Essentials, Certificate (GSEC), Practical Assignment, Version 1.4c, SANS Conference,
London, pp. 1–17 (2004)
5. Hesham, A.I.M.K.: Cloud computing security, an intrusion detection system for cloud
computing systems, Ph.D. thesis, pp. 1–52 (2011)
676 T. Nsabimana et al.
6. Patil, D., Ganveer, K.S.S., Badge, K.P.S.: To implement intrusion detection system for cloud
computing using genetic algorithm. Int. J. Comput. Sci. Inf. Technol. Res. 3(1), 193–198
(2015)
7. Kuldeep, T., Tyagi, S., Richa, A.: Overview-snort intrusion detection system in cloud
environment. Int. J. Inf. Comput. Technol. 4, 329–334 (2014). ISSN 0974-2239
8. https://www.hackingarticles.in/dos-attack-penetration-testing-part-2/
9. https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/hoic-ddos-analysis-and-
detection/
10. Singh, D., Patel, D., Borisaniya, B., Modi, C.: Collaborative ids framework for cloud. Int.
J. Netw. Secur. 18(4), 699–709 (2016)
11. Jadhav, M.L., Gaikwad, P.C.M.: Implementation of intrusion detection system using GA.
Int. J. Innov. Res. Electr. Intrusmentation Control Eng. 2(7), 1733–1736 (2014)
12. Hounsou, J.T., Nsabimana, T., Degila, J.: Implementation of network intrusion detection
system using soft computing algorithms (self organizing feature map and genetic algorithm).
J. Inf. Secur. 10(1), 1–24 (2018). https://doi.org/10.4236/jis.2019.101001
13. Gong, R.H., Zulkernine, M., Abolmaesumi, P.: A software implementation of a genetic
algorithm based approach to network intrusion detection. Presented at the Proceedings of the
Sixth International Conference on Software Engineering, Artificial Intelligence, Networking
and Parallel/Distributed Computing and First ACIS International Workshop on Self-
Assembling Wireless Networks (2005)
Development of Tutoring Assistance
Framework Using Machine Learning
Technology for Teachers
Satoshi Togawa1(&), Akiko Kondo2, and Kazuhide Kanenishi3
1
Education Center for Information Processing, Shikoku University,
123-1 Furukawa Ojin-cho, Tokushima 771-1192, Japan
[email protected]
2
Faculty of Management and Information Science, Shikoku University,
123-1 Furukawa Ojin-cho, Tokushima 771-1192, Japan
[email protected]
3
Center of University Education, Tokushima University,
1-1 Minami-Josanjima, Tokushima 770-8502, Japan
[email protected]
Abstract. This paper proposes a framework for tutoring assistance to tackle
increasing student dropout rates. Student dropouts in higher education institu-
tions, such as universities, often result in an increase in tutors’ workload.
Currently, educational assistance is focused on supporting the students’ learn-
ing, and the main purpose of this assistance is an acceleration of the learning
process. Although student assistance is undoubtedly of great importance,
offering assistance to teachers who also have a tutoring role is equally important.
The purpose of our framework for assistance is to detect students at risk for
dropout, after which an alert is sent to the tutors. The alert encourages tutors to
take timely action to avoid student dropouts. This paper describes the enhanced
framework implementation, its experimental use, and the results.
Keywords: Tutoring assistance Learning analysis Abnormal behavior
detection Machine learning
1 Introduction
Japan has over 1,100 higher education institutions, including universities and junior
colleges [1]. The total number of private universities is about four times the number of
national and public universities. This means that there are currently too many uni-
versities in Japan. Due to a declining birth rate in the country, 40% of private university
spots have not been filled. Students at these private universities are often not attending
their first-choice schools; thus, their motivation to study is generally low. Conse-
quently, students often fail to graduate, which results in high dropout rates.
To address this problem, most Japanese universities are offering their students a
tutoring program. The tutors are generally selected among high-achieving senior and
graduate students. However, it is not always easy to secure a sufficient number of
tutors. At some universities, teachers may also serve as tutors and mentors for the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 677–682, 2020.
https://doi.org/10.1007/978-3-030-39512-4_104
678 S. Togawa et al.
students. Since teachers already have many responsibilities to fulfill in their primary
role, it becomes difficult to also adequately fulfill the role of being a tutor. One way of
alleviating the strain on the universities’ tutoring programs would be to detect students’
abnormal behavior based on class attendance and their own learning history to provide
appropriate instruction to the students via their tutors, thus making tutoring more
effective and preventing student dropouts in time.
In this research, we propose a tutoring assistance framework for teachers. Machine
learning is used to determine the students’ learning history and their class attendance
status, accumulated academic credit acquisition status, and overall dropout risk. When
it is determined that the probability of a student dropping out is increasing, the teacher
receives a notification via the framework. The teacher can then provide re-medial
instructions directly focused on supporting the student falling behind. Consequently,
students’ necessities can be addressed at an early stage, and the possibility of dropping
out is reduced. The proposed framework does not support detected students directly;
rather, it provides class management awareness for teachers who also have tutoring
duties.
We have been designing a tutoring assistance framework and have verified its
effectiveness [2]. The problem of previous research is that the detection accuracy of
students with the risk of dropout is not good. Therefore, in this research, the machine
learning algorithm, such as the neural network for dropout risk detection and increased
types of input data are reviewed.
We first describe the current state of Japanese higher education, especially at pri-
vate universities, and the difficulties encountered in keeping students motivated. Sec-
ond, we review related studies, addressing the benefits of using the student assistance
framework to maintain student motivation and to support tutors. Third, we present the
concept and prototype developing method for the proposed assistance frame-work for
tutors, which uses data from student learning history and machine learning analytics.
Finally, we describe the results achieved with the experimental use of the prototype,
and we present the effectiveness of tutoring based on the assistance framework.
2 Problems of Japanese Higher Education and Related Study
The number of higher education institutions in Japan continues to increase. Between
2000 and 2018, 130 universities (mainly private) were inaugurated. Conversely, within
the same period, the number of students enrolled in national, public and private uni-
versities, and junior colleges remained stable at around three million. The number of
high school students, however, has seen a sharp decrease of 23% in 18 years: from
approximately 4.16 million in 2000 to 3.23 million in 2018. The university entrance
ratio, including universities and junior colleges, was 30.6% in 1990. In 2018, the ratio
is 54.8%. The university entrance ratio exceeded 50% in 2007 for the first time and has
remained steady ever since [1]. These facts indicate that it became easier to enter
university, with the exception of high-end universities. This trend will further accel-
erate in the next ten years.
Although the number of high school graduates has been decreasing significantly
over the years, the number of universities has increased. Therefore, many private
Development of Tutoring Assistance Framework 679
universities are unable to fill the available spots. As the T-score required for admission
to private universities is lower, the dropout rate tends to be higher [3]. For this reason,
identifying students who risk dropping out at an early stage is important, especially in
the case of universities with low entry levels.
In searching for ways to solve the problem above, we have found some studies that
show dropout prevention based on analytics of students’ learning history. For instance,
a system aims to reduce the workload for mentors supporting students by using
e-learning [4]. Whereas this research also aims to reduce the workload on mentors, its
main purpose is to support learners’ learning activity. In another study, a notification
reminds students to perform Self-Regulated Learning using an e-learning environment
[5]. The aim is to support students during their learning process, not to avert the risk of
dropping out.
Naturally, these studies are effective in terms of the support offered to students with
their learning. However, it is sometimes difficult to reach students with below average
T-scores through the assistance system. Sometimes they do not accept the notifications,
because it is not recognized for themselves. For these students, direct support from
tutors is very important to detect dropout risk. Additionally, the task is wide-ranging:
from focusing on the frequency of class attendance, to showing interest in daily life and
human relations. Most teachers who assume the role of tutor are busy with tutoring
work and do not have time for adequate class preparation and research activities.
Consequently, it becomes difficult to ensure the quality of education.
3 Developing of Student Tutoring Assistance Framework
3.1 Outline of Student Tutoring Assistance Framework
In this section, we describe the student tutoring assistance framework and its devel-
opment. Figure 1 shows the overview of the proposed student tutoring assistance
framework. Under the Learning Management System (LMS), we see the learners’ login
frequency, the login time slot, system use time and the course material use frequency.
Under the Grade Processing System, the learners’ examination results, record of
attendance, and status of enrolment in university are stored.
In the Tutoring Assistance System (TAS), machine learning analyses the learning
history and enrolment/withdrawal situation. The TAS detects potential student drop-
outs. When the TAS detects a potential dropout, it generates an alert to the teacher. The
teacher, fulfilling the role of tutor, receives an alert of a possible student dropout and
can then take action to prevent this.
This system does not directly assist a student at risk for dropout. Instead, the teacher
provides assistance to prevent dropout. Since direct notifications sent by some assis-
tance systems to students can sometimes be ignored, it is difficult to effectively prevent
student dropouts using this method. Therefore, the approach of this framework is to
support teachers by using alerts and notifications to indicate the existing risk of
dropout, and subsequently ensuring that the student who may potentially dropout is
assisted by the teacher directly. Thus, it is possible to prevent students who may
dropout from being overlooked. Furthermore, the burden on tutors can be reduced.
680 S. Togawa et al.
[ Learning Management System ]
-Login Frequency
-Login Time Slot [ Grade Processing System ]
-System Utility Time -Examination Result
-Course Material Utilization Frequency -Record of Attendance
-Material Utilization Frequency by Category -Status of Enrollment
Collecting
Learning History
Collecting
Results and Status
LMS Access
Confirming Tutoring
Grade Processing Assistance System
System
Dropout Risk
Detection
Learners
Dropout Prevention
Teacher
(Tutor)
Fig. 1. Student tutoring assistance framework
3.2 Implementation of Dropout Risk Detection Function
We implemented a neural network to detect potential student dropout. The aim of this
neural network implementation was to judge the effectiveness of the proposed
framework. Figure 2 shows a neural network implementation for dropout risk detec-
tion. This network is based on a feedforward neural network for binary classification.
We believe dropout detection is a simple binary classification problem, because
“dropout” and “non-dropout” are incompatible conditions. Therefore, a binary classi-
fier is ideal to address the problem.
Input data to detect dropout risks are login frequency, course material use fre-
quency, examination scores, and attendance frequency via variable s1 to s5. The result
for judging the dropout risk is obtained from the variables y1 and y2. If variable y1 is
larger than variable y2, the dropout risk is judged to exist.
This implemented binary classifier was built based on Keras [6] and TensorFlow
libraries [7].
Development of Tutoring Assistance Framework 681
Login Freq. s1
Course Material
s2 dropout
Utilization Freq. y1
Examination
Score1 s3
Examination y2 non-dropout
Score 2 s4
output layer
Attendance
Freq.
s5
hidden layer 1 hidden layer 2
input layer
Fig. 2. Neural network implementation for dropout risk detection
4 Experimental Use and Results
In this section, we describe the results of experimental use. The framework was tested
to confirm its effectiveness. Table 1 shows the specifications of the computer on which
the dropout risk detection user test was performed. Table 2 shows the condition of
training dataset and testing dataset generation.
Table 1. Computer specification of risk detection processing
CPU specification Intel Xeon E3-1225v6 3.3 GHz
System memory capacity 64.0Gbytes
GPU specification NVIDIA GeForce 1080 Ti with 11.0Gbytes Memory
Operating system Ubuntu Server 18.04.3 LTS 64bit edition
We generated a training dataset with 10,000 samples. Under ordinary circum-
stances, the training dataset used to train the neural network would use actual data, such
as students’ learning history in LMS. The values for Login Frequency, Course Material
Use Frequency, and Attendance Frequency were randomly generated, and these data
ranged from 0 to 15. Usually, classes are open for enrolment 15 times per semester. The
ideal dataset for training processing must be used for this purpose. We trained the
binary classifier based on the neural network used by the generated training dataset.
One hundred testing datasets were generated randomly, following the correct range;
subsequently, they were judged as to whether they had high-risk or not by the trained
neural network. Of these test samples, 68 were found to be true. Thus, the correct
answer rate is 68%. We consider this result to be satisfactory for our purpose. The main
purpose of this study is to reduce the tutor’s workload. We need greater accuracy to be
able to provide better assistance. For this purpose, we will try to optimize this neural
network for dropout risk detection.
682 S. Togawa et al.
Table 2. Condition of training and testing dataset generation
Dataset Condition
Login frequency Random, data range 0 to 15
Course material utilization frequency Random, data range 0 to 15
Examination scores Normal distribution, SVG = 70, SD = 10
Attendance frequency Random, data range 0 to 15
5 Conclusions
In this paper, we proposed a student tutoring assistance framework. In particular, we
described the importance of detecting the potential student dropout risk. We proposed a
dropout risk detector based on a feedforward neural network to detect potential student
dropout. Finally, we built the prototype implementation for detection, and the results of
experimental use were clarified.
This framework is still at an early stage of development. However, we think this
framework and the results of the experimental use can encourage action to avoid
potential student dropout.
Acknowledgments. This work was supported by JSPS KAKENHI Grant Numbers 18K02922.
References
1. Ministry of Education, Culture, Sports and Technology of Japan: Report of School Basic
Survey 2018 Edition (2018)
2. Satoshi, T., Akiko, K., Kazuhide, K.: Designing of student tutoring assistance framework
using machine learning technology for teachers. In: Proceedings 11th International
Conference on Education and New Learning Technologies, pp. 9502–9506. IATED Academy
(2019)
3. The Japan Institute for Labor Policy and Training: Working and Consciousness Research of
Student Dropped Out of Universities, JILPT Report No. 138 (2015)
4. Yutaka, S., Takeshi, M., Yoshiko, G., et al.: Development and evaluation of an e-mentors’
workload reduction system; based on planning phase in learners’ self-regulation. J. Jpn Soc.
Educ. Technol. 36(1), 9–20 (2012)
5. Takeshi, M., Masahiro, Y., Yoshiko, G., et al.: Development of self-regulator that promotes
learners to establish planning habit and its formative evaluation. J. Jpn Soc. Educ. Technol.
40, 137–140 (2017)
6. Keras. https://keras.io
7. TensorFlow. https://www.tensorflow.org
Replenishment System Using Inventory Models
with Continuous Review and Quantitative
Forecasting to Reduce Stock-Outs
in a Commercial Company
Carlos Malca-Ramirez, Luis Nuñez-Salome, Ernesto Altamirano,
and José Alvarez-Merino(&)
Ingeniería Industrial, Universidad Peruana de Ciencias Aplicadas (UPC),
Lima, Peru
{u201416189,u201413246,pcinealt,pciijalv}@upc.edu.pe
Abstract. In recent years, stock-outs significantly affects commercial enter-
prises with deficit equivalent to a large percentage of the total sales, due to its
impact on current and future demand, regardless of their financial success.
Carried out a qualitative and quantitative analysis of the current situation to
identify the problem. In addition, the root causes were identified with the help of
quality tools such as Pareto chart and Ishikawa and a questionnaire based on the
SCOR model and management indicators. In this way why it is proposed a
replenishment system applying models of inventories with continuous review
and quantitative forecasts. The proposal consists of five phases: (i) identify the
key products; (ii) Do quantitative forecasting; (iii) Determine inventory levels.
(iv) Establish the new flow of information; (v) Establish new policies and
procedures. We evaluate the models of inventories and forecasts through the
program @Risk.
Keywords: Inventory management Continuous review replenishment
system Forecast Retails Stock outs
1 Introduction
The events of stock outs occur when the product is not available in store so that the
customer can purchase [1], this refers to a situation where the demanded product is not
available to the customer in the expected place or is not available for sale [2].
Approximately 75% of the causes of the stock outs originate in the operations of the
stores, while between the 25% and 30% originate in the distribution centers and in the
operations of the central [2]. In addition, the bad practices of order (orders late or
insufficient quantity ordered), forecasting demand [3, 4], the incorrect distribution of
information and inventory also influence the stock outs of the shops [5, 6].
Customers will be carried out one of the following actions when they face a
shortage of supplies at the store. (i) change store to locate the same product, (ii) to
change the brand for a similar utility of the same product, (iii) to postpone the purchase
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 683–689, 2020.
https://doi.org/10.1007/978-3-030-39512-4_105
684 C. Malca-Ramirez et al.
to a date in which the product is available, or (iv) to abandon completely the purchase,
which will cause a loss of sales [1].
The stores and manufacturers lose more than 4 billion euros every year and each
year and the total sales losses caused by lack of stock amount to 3.9% worldwide, 3.8%
in the EE. UU and 3.7% in Europe, this percentage is even greater in South America
[7]. The stock out result in a reduction in benefits of up to 10% and the decline in
consumer loyalty can have a long-term impact on the participation of the market [8].
The world average of occurrences of stock outs is estimated at 8.3%, a little higher than
in the United States. The United States. (7.9%), and a little lower than the average in
Europe (8.6%). The situation is better in the regions of Asia Pacific and Australia/New
Zealand, where the averages of the out of stock are 5% and 4.4%. On the other hand,
the highest rates of stock out are recorded in South America, in Argentina the average
is 5.15% and 14.3% in Chile, and 8% in Brazil [7]. On the other hand, it was
demonstrated that the problem of shortage of supplies is a great opportunity for the
stores and manufacturers increase their revenue from increased sales [2].
Previous studies concluded that the automatic system replenishment inventory
could reduce the average rate of shortages in the shelves regardless of the character-
istics of the product. Similarly, improve the processes of anticipation of orders and
replenishment procedures for those items that fall into the category of important, results
in a smaller number of stock outs in the levels of distribution centers and shops [3].
Similarly, automated ordering systems where you set the optimum amount of order and
the point of order for each product in the store generate an increase in operational
efficiency in terms of total cost and service levels. You can also achieve a higher level
of availability in stores by means of cooperation in planning and anticipation among
the shops and its suppliers [5].
The literature showed us that the stock-outs in shops consists in the depletion of
stocks on the shelves. In the face of an occurrence, customers choose to replace the
product, generating losses of profits. To solve the problem of shortages, the literature
uses techniques aimed at alignment of the chain, where there is a predominance of
inventory management and forecasting models, in order to ensure the optimal service
level.
2 Problem
The company is dedicated to the wholesale of products to perform any type of manual
work. Its presents a decrease in sales in the last period as described in the following
figure (Fig. 1):
In order to confirm a decrease in revenue, a comparison was made with the year
2017 with revenues from the month of March of the year 2018. In this way, it was
found that there is a sales decrease of 15%. It concludes with a qualitative and
quantitative diagnosis obtaining four causes.
However, only one of them presented more frequently with a 98% participation in
the incidents.
This was evident over time, as shown below (Fig. 2):
Replenishment System Using Inventory Models 685
Fig. 1. Sales report 2017–2018
Fig. 2. Stock outs value
In summary, stock outs in stores generates an impact on the company’s annual
billing of 19% equivalent to $73,082.53.
Once you have identified the problem an audit was carried out based on the SCOR
model and the tool of the Ishikawa diagram to detect the main causes. In addition, an
analysis was made of the indicators of the current processes of the company.
• Quality of generated orders = 81%
• Supplier compliance level = 95%
• % of late deliveries to store = 34%
• % of insufficient orders to warehouse = 20%
• % forecast error = 40.79%
In this way, the general and specific causes that were obtained through pareto
analysis are shown below.
• Unreliable demand prediction
• Insufficient inventory availability in central warehouse
• Late dispatches to stores Insufficient orders from store to store
686 C. Malca-Ramirez et al.
3 Proposed Methodology
After identifying and analyzing the current situation of the problem, information on the
monthly demand of the items is collected and analyzed in order to identify the best
forecast model to obtain the optimum inventory levels (Q, r) of the central warehouse
and the stores. The mentioned methodology consists of the following phases:
3.1 Key Products
For the identification of key products, the ABC multi-criteria tool was used, having as
variables the stock break in soles, the margin per family of products, quantity of Sku’s
per family of products.
Obtaining the families to which the project will focus, which are: dies, cards,
embossed folder, applications, tools, block cards, die cutters.
3.2 Forecast Analysis
To determine the most appropriate forecast for our demand, the models were compared
by evaluation criteria (Table 1).
Table 1. Comparative table of forecast model.
Moving Exponential Linear Regression Neural
Criteria / Model Time series
averages adjustment regression analysis networks
Degree of Forecast Adequate or Suitable Tall Tall
Adequate or short Adequate or
short /
Space of time / medium short / medium Long Long Long
medium
Amount of initial data
Medium Low Very low Medium Tall Tall
required
Forecasting costs Low Low Moderate Low Low Moderate
Level of applicability and
Low Low Medium Medium Medium Tall
implementation
3.3 Inventory Levels
Based on the results of the forecasts, the optimal inventory levels will be determined
together with their security stock of both the stores and the warehouse stock of the
selected items through the formulas.
3.4 Information Flow
Then, the information flow of the replenishment system to be executed is established
(Fig. 3).
Replenishment System Using Inventory Models 687
OUTPUTS
INPUTS DOCUMENTS INTERMEDARY
DOCUMENTS
Invoices
Purchase order SUPPLIER
References guides
Requirements Purchase order
Current demand Purchase order requests
Historical sales WAREHOUSE Goods issue
Invoices
Current demand
References guides
Demanda actual
Historical sales
Goods issue STORE
Requirements
Fig. 3. Information flows
3.5 Politics and Procedures
With the improvement and with the new inventory policies, procedures were generated
and modified. As shown in the following table (Table 2):
Table 2. New procedures.
Code Procedures Monitoring
CMDA Purchase of merchandise After the purchase request
DDLM Merchandise Distribution After store requirements
PRLD Make forecasts After the end of the month
REDT Requirements After store closing
SODC Purchase requests After dispatching merchandise
3.6 Considerations
The simulation is carried out by entering real data that correspond to the time of period
between 2017–2018. The probability of the margin of error was extracted from the
historical data and will influence inventory costs, sales revenue and stock-outs.
3.7 Variable
The input variables are as follows (Table 3):
4 Validation
See Table 4.
688 C. Malca-Ramirez et al.
Table 3. Dependent variables
Outputs
Elements Observations
Total cost Maintenance, storage and ordering costs
inventory
Sales revenue The units sold annually multiplied by their respective unit price
Sales cost The units purchased annually multiplied by their respective costs and
expenses
Operating Margin contribution obtained from the difference between sales revenue
utility and cost of sales
Table 4. Results
Indicator Result
% of late deliveries to store 3.87%
% of insufficient orders to warehouse 0.63%
% forecast error 5.55%
Stock out cost $1,930.82
5 Analysis of Results
The specific objectives were achieved according to the simulation, reducing % of late
shipments to store, % of insufficient orders to warehouse, % forecast error and the cost
of breakage from 34% to 3.87%, 20% to 0.63%, 40.79% a 5.55% and $ 73,082.53.a to
$ 1,930.82 that would be reduced with the proposed improvement.
6 Conclusions
The case study has proven that stock outs can be reduced by means of the techniques
used. Through a system proposed, it has been possible to reduce the incidents of stock
outs in the commercial sector in Peru.
References
1. Govind, A., Luke, R., Noleen, P.: Investigating stock-outs in Johannesburg’s warehouse retail
liquor sector. J. Transport Supply Chain Manag. 11, 1–11 (2017)
2. Avlijas, G., Milicevic, N., Golijanin, D.: Influence of store characteristics on product
availability in retail business. E+M Ekonomie Manag. 21, 195–206 (2018)
3. Goran, A., Simicevic, A., Avlijas, R., Prodanovic, M.: Measuring the impact of stock-keeping
unit attributes on retail stock-out performance. Oper. Manag. Res. 8, 131–141 (2015)
Replenishment System Using Inventory Models 689
4. Bobadilla, R., Mendez, A., Viacava, G., Raymundo, C., Moguerza, J.M.: Service model based
on information technology outsourcing for the reduction of unfulfilled orders in an SME of
the peruvian IT sector. In: Advances in Intelligent Systems and Computing, vol. 965,
pp. 311–321 (2019)
5. Milicevic, N., Grubor, A.: The effect of backroom size on retail product availability –
operational and technological solutions. Amfiteatru Econ. 17, 661–675 (2015)
6. Carazas, L., Barrios, M., Nuñez, V., Raymundo, C., Dominguez, F.: Management model
logistic for the use of planning and inventory tools in a selling company of the automotive
sector in Peru. In: Advances in Intelligent Systems and Computing, vol. 971, pp. 299–309
(2019)
7. Aleksandar, G., Milicevic, N., Djokic, N.: The impact of store satisfaction on consumer
responses in out-of-stock situations. Revista Brasileira de Gestão de Negócios 19, 520–537
(2017)
Applying SLP in a Lean Manufacturing Model
to Improve Productivity of Furniture SME
Zhelenn Farfan-Quintanilla1, Manuel Caira-Jimenez1,
Fernando Sotelo-Raffo1, Carlos Raymundo-Ibañez1(&),
and Moises Perez2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima 15023, Peru
{u201112739,u201313833,pcinjsot,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. Currently, a company’s competitiveness in the market depends on its
productivity in efficiently using its available resources. Low productivity is a
recurring problem in the furniture sector due to low production process effi-
ciency, resulting in the failure to deliver orders. For this reason, businesses focus
on seeking manufacturing systems that enable them to efficiently control their
processes, eliminating any waste or activity that generates downtime, to achieve
higher customer satisfaction. This research explores a work case at a small
Peruvian business in the furniture sector, which is facing the problem of a large
amount of unfulfilled orders on a monthly basis that has resulted in lost sales of
8%. The results showed a positive impact in which the new improved methods
in the cutting and edging process, as well as a new plant layout enabled oper-
ations and activities to be performed in less time, increasing productivity from
42.5% to 64.20%.
Keywords: Lean manufacturing SLP Productivity Furniture SME 5S
1 Introduction
Presently, the Peruvian manufacturing sector has been growing from 3.2% to 3.5%.
Furthermore, it should be noted that micro, small and medium enterprises in the
country play an important role for social development and are a key sector for national
economic growth, as they are a source of employment, income, and drive productive
activities in local economies. However, it must be taken into consideration that there is
still a high percentage of informality, as 53% of MSEs still are not registered with the
National Superintendence of Customs and Tax Administration. According to the World
Trade Organization, “low levels of MSE productivity are directly related to the lack of
capacity by these companies to capitalize on their economies of scale, a lack of spe-
cialized labor, low access to credit and investment, and the informality of their con-
tracts with clients and suppliers”.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 690–696, 2020.
https://doi.org/10.1007/978-3-030-39512-4_106
Applying SLP in a Lean Manufacturing Model 691
One of the reasons behind low productivity problems is inefficiency in productive
processes. The contribution presented in this study focuses on the production man-
agement model, which consists of planning, executing, checking, acting, and docu-
menting the procedures performed by companies in the furniture sector. This is
followed by providing training and evaluating operators’ performances based on a lean
manufacturing approach and systematic layout planning (SLP) technique.
2 State of Art
2.1 Lean Manufacturing to Improve Productivity
Lean manufacturing techniques are a set of tools and methods that eliminate waste in
production operations. As cited by Matt and Rauch [1], lean manufacturing is a mul-
tidimensional approach that covers a variety of management practices and is aimed at
reducing waste and improving operational efficiency. Many authors state that lean
manufacturing is intended to increase productivity, improve product quality and cycle
times, reduce inventory and delivery times, and eliminate waste [1–7].
2.2 5S and Visual Management to Improve Productivity of MSEs
5S is an integral step in lean manufacturing as it allows both internal and external
customers to be supplied with the “right product”, at the “right time”, and in the “right
quantity”. The use of 5S improves productivity and product or service quality. 5S is the
first step of lean manufacturing, which addresses waste associated with the workplace
and improves workflows between processes. Another vital consideration is visual
management, which establishes visual controls that allow operators to distinguish
between correct and incorrect behavior. Lastly [8, 9], agree that the implementation of
color cards during waste elimination or sorting is a good visual practice that will
generate low-cost positive results.
3 Contribution
This production management model considers and focuses on the following four
dimensions: human capital management, process management, production time man-
agement, and production control.
3.1 Proposed Model
In view of the above, an overview of the proposed model is shown in Fig. 1.
692 Z. Farfan-Quintanilla et al.
Fig. 1. Lean manufacturing model
3.2 Proposed Method
Figure 2 details the overview of the process that the proposed model is based upon.
Fig. 2. Process view
Applying SLP in a Lean Manufacturing Model 693
4 Validation
4.1 Case Study
The case study was conducted in a furniture SME located in the Chorrillos district of
Lima, Peru. Owing to a lack of capacity in its production process, the company has
failed to meet orders already placed, resulting in lost sales. Currently, the company’s
efficiency stands at 51%, while the sector’s average efficiency is 59%, meaning that the
company is 8% below their competition, in addition to using less than 50% of its
capacity.
4.2 Diagnostic
The study was narrowed down to consider the service that generates the highest profit
margin in the company, namely furniture manufacturing. Afterwards, an ABC analysis
was conducted, which revealed that desks and closets represent 80% of its sales,
thereby generating higher profit margin for the company.
The problem identified is the high amount of unfulfilled orders on a monthly basis,
with an average of 23% of orders. Then, the reasons behind this were analyzed, through
which it was determined that the main reason was that its actual capacity was below
demand. The main causes identified were poor methods (63%) and re-manufacturing
(37%).
4.3 Solution Proposal
Improvement of the production system began with improving processes under the
ECRS principles and 5W1H technique. Afterwards, the 5S technique was implemented
and, lastly, the SLP technique.
4.3.1 5S Implementation
Owing to the identification of the activities that compose the processes, this study was
able to conclude that operators take a long time searching for work materials, inputs,
and tools, which causes them to fall behind and creates delays, resulting in longer
operation times.
4.3.2 SLP Implementation
The purpose here is to evaluate two of the technique’s methods to choose the best
alternative layout that offers greater cost and time savings. To this end, an analysis of
the current layout was performed, taking into consideration workspaces, storage areas,
materials, machinery, and movement times for people and materials. A simple
graphical method and a matrix table method were proposed to determine the layout
alternatives to present.
694 Z. Farfan-Quintanilla et al.
4.4 Results
Efficiency Improvement: This indicator was determined because it is of vital
importance in increasing the average productivity established by the furniture sector,
whose efficiency value was 59.3%. Initial efficiency: 42.50%, final efficiency: 64.20%.
Reduction of Production Times: The results were positive, as new average times
were obtained, achieving reductions of 42%, 29%, and 13% in the project processes in
the company’s cutting, edging, and assembly processes, respectively (Fig. 3).
Fig. 3. Reduction of production times
4.5 Results Analysis
The project implementation results are outlined in the following table:
Table 1. Implementation results
Initial Current
Furniture production 33 furniture pieces/month 54 furniture pieces/month
Desk production time 14.88 h-h 10.80 h-h
Closet production time 19.87 h-h 16.08 h-h
Average of unfilled orders 15 orders/month 0 orders/month
As shown in Table 1, production times for cutting, edging, and assembly were
reduced by 42%, 29%, and 13%, respectively. The total time was reduced by 4.08 h,
thus increasing manufacturing productivity from 42.50% to 64.20%, and surpassing the
sector’s productivity of 59.30%.
Applying SLP in a Lean Manufacturing Model 695
5 Conclusions
In the analysis conducted at the company studied, a large number of unproductive
activities within processes were found, representing more than 60% of the production
time. With the help of the improved methods proposed in the design of good receipt
and storage of raw materials, as well as in the cutting and edging processes using the
ECRS approach, it was possible to reduce the time of the cutting process by 20%, the
edging process by 40%, and the assembly process by 50%.
The initial layout analysis identified large distances for material and staff movement
and crossings resulting from the plant’s poor layout. After implementing the SLP
technique, movement time was reduced by 20%. Workstations were rearranged
according to the process, in particular reorganizing the space based on the finishing
areas. After implementing lean manufacturing in a pilot at a furniture SME, manu-
facturing productivity increased from 42.50% to 64.20%, surpassing the sector’s
productivity of 59.30%.
Likewise, production capacity increased from 33 pieces of furniture per month to
74 pieces, exceeding current demand and thus covering the expected demand for
furniture results in an additional average monthly profit of S/. 23,500. Moreover, an
initial cash flow analysis, conducted from November 2018 to April 2019, determined
that in the last month, it was feasible to make an investment in the improvement project
given that it maintains a positive cash flow. Due to this analysis, the project was
budgeted and its benefit/cost ratio was calculated, obtaining a score of 1.93, meaning
that the financial proposal is viable and has a return on investment of nearly two
months.
References
1. Matt, D.T., Rauch, E.: Implementation of lean production in small sized enterprises.
Procedia CIRP 12, 420–425 (2013)
2. Roriz, C., Nunes, E., Sousa, S.: Application of lean production principles and tools for quality
improvement of production processes in a carton company. Procedia Manufact. 11, 1069–
1076 (2017)
3. Talukder, M.H., Afzal, M.A., Rahim, M.A., Khan, M.R.: Waste reductio n and productivity
improvement through lean tools. Int. J. Sci. Eng. Res. 4(11), 1844–1855 (2013)
4. Zhou, B.: Lean principles, practices, and impacts: a study on small and medium-sized
enterprises (SMEs). Ann. Oper. Res. 241, 457–474 (2016)
5. Bellido, Y., La Rosa, A., Torres, C., Quispe, G., Raymundo, C.: Waste optimization model
based on Lean Manufacturing to increase productivity in micro- and small-medium
enterprises of the textile sector. In: CICIC 2018 - Octava Conferencia Iberoamericana de
Complejidad, Informatica y Cibernetica, Memorias, vol. 1, pp. 148–153 (2018)
6. Bellido, Y., Rosa, A.L., Torres, C., Quispe, G., Raymundo, C.: Waste optimization model
based on Lean Manufacturing to increase productivity in micro- and small-medium
enterprises of the textile sector. In: CICIC 2018 - Octava Conferencia Iberoamericana de
Complejidad, Informatica y Cibernetica, Memorias, vol. 1, pp. 148–153 (2018)
696 Z. Farfan-Quintanilla et al.
7. Paredes, M., Villa, J., Quispe, G., Raymundo, C., Moguerza, J.M.: Lean Manufacturing
model for the improvement of manufacturing processes in small and medium-sized companies
in the furniture industrial sector. In: CISCI 2018 - Decima Septima Conferencia Iberoamer-
icana en Sistemas, Cibernetica e Informatica, Decimo Quinto Simposium Iberoamericano en
Educacion, Cibernetica e Informatica, SIECI 2018 – Memorias, vol. 1, pp. 46–51 (2018)
8. Singh, A., Ahuja, I.S.: Evaluating the impact of 5S methodology on manufacturing
performance. Int. J. Bus. Continuity Risk Manag. 5(4), 272–305 (2014)
9. Patil, S., Sapkal, A., Sutar, M.: Execute 5S methodology in small scale industry: a case study.
Int. J. Res. Advent Technol. 5, 47–51 (2016)
Collaborative Model Based on ARIMA
Forecasting for Reducing Inventory Costs
at Footwear SMEs
Alejandra Angulo-Baca1, Michael Bernal-Bazalar1,
Juan Sotelo-Raffo1, Carlos Raymundo-Ibañez1(&), and Moises Perez2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima 15023, Peru
{u201414534,u201414724,pcinjsot,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. This study addresses inadequate inventory management issues aris-
ing from poor demand management and insufficient inventory movement record
communication. Focusing on a footwear retailer, this study determined that the
main problem identified is rooted in an improper management of finished
products caused by excessive production establishing optimum production
quotas coupled with an inadequate optimization of the space used for inventory
management. Within this context, the project proposes using the collaborative
planning, forecasting, and replenishment (CPFR) methodology supported by
ARIMA forecasting, a strategy that centers on maintaining adequate logistics
development controls.
Keywords: Footwear CPFR ARIMA forecast Inventory management
Change management
1 Introduction
Throughout the years, small- and medium-sized enterprises (SMEs) have continuously
faced new challenges, one of them is the implementation of a logistics management
system, which requires optimization of the supply chain and its components, and
provides SMEs with a competitive advantage over their competitors [1, 2]. When
inventories are inaccurately recorded throughout the supply chain, the company’s
operating performance (e.g., orders, inventories, customer service, among others)
becomes unstable and starts generating losses. In fact, 70% of small- and mid-sized
footwear retailers experience inventory management problems [3, 4].
This proposal is indeed relevant because proper inventory management leads to
optimum warehouse spaces, which, in turn, improves customer responses [5]. This
study focuses on the development of a forecast-based operational plan, which may
provide companies with a solid foundation for their operations, thus improving their
inventory and demand management while reducing costs and increasing profitability.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 697–703, 2020.
https://doi.org/10.1007/978-3-030-39512-4_107
698 A. Angulo-Baca et al.
2 State of Art
2.1 Logistics Planning Model
An efficient logistics management and planning system can meet the customer’s
logistics requirements. In fact, as markets open and supply chains globalize, logistics
plays an even bigger strategic role in the structural changes companies undertake [6].
An efficient logistics management defines the competitive advantages of a company,
since cost reduction opportunities are generated and customer service levels improve
through coordination between the parties included in the logistics flow [7, 8].
2.2 Technique Review
2.2.1 CPFR – Collaborative, Planning, Forecasting and Replenishment
Optimum inventory management creates flexibility in stock-related operations, while
helping companies reach their competitive priorities more efficiently [9]. The imple-
mentation of this technique improves both collaboration and communication between
the different parties in the supply chain, rendering benefits such as increasing demand
forecast accuracies, improving customer service levels, and reducing inventory levels
[10]. Through CPFR implementation, inventory levels, costs, and lead times are
reduced, besides optimizing customer service levels and sales volumes [11].
2.2.2 ARIMA Forecasts
Most companies, regardless of the sector in which they operate, face a common
problem: demand uncertainty. Aguirre explains that ARIMA forecasts use past time
events to make demand predictions and that their main advantages are a fast perfor-
mance and low costs. Demand forecasts and their accuracy exert a vast influence on the
supply chain, and, specifically, on inventories, as the costs they represent increase
substantially if their levels are not adequate [12, 13]. In fact, when the transgenic time
series of the ARIMA model is used, a more accurate forecast is generated compared
with when using other models, as Holt–Winters and double exponential smoothing.
3 Contribution
Panahifar argues that CPFR implementation improves both collaboration and inventory
level and increases customer service levels. However, De Arce and Mahía mention that
the ARIMA models attempt to describe the evolution of a yt variable from a stochastic
process based on the past values of said variable, which allows ARIMA forecasts to be
successfully implemented within the CPFR because the collaborative model is built on
making changes and decisions based on history data.
Collaborative Model Based on ARIMA Forecasting for Reducing Inventory Costs 699
Fig. 1. Collaborative logistics planning model structure
As denoted in Fig. 1, this proposal seeks to implement a logistics planning man-
agement model, which is rooted on ARIMA forecasts to optimize planning and
inventory logistics. Figure 2 displays the four phases required for successful model
implementation, which is based on a cycle of continuous improvement.
Fig. 2. Phases of the collaborative logistics planning model
3.1 Operating Plan Development Phase
The first two steps proposed by CPFR serve as foundation for preparing the business
plan, which is executed by the project partners or stakeholders. In this case, the
business plan refers to inventories.
3.1.1 Partnership Agreements (Start - End)
In this first aspect, guidelines are defined for the collaborative relationship set forth
between both parties. Once the agreement has been executed, the following must also
be defined: Roles corresponding to each partner, Model-Related Processes and
700 A. Angulo-Baca et al.
performance indicators. For these purposes, there are some indicators which need to be
assessed to obtain quantitative results, such as ROI, IRA and Inventory Turnover.
3.2 Forecasting Phase
In the second phase, forecasts will be used to optimize the proposal by implementing
the ARIMA model within CPFR. In the first stage, the appropriate values for the
ARIMA model must be determined. Moreover, the ARIMA model requires all series to
be stationary (series without trend or seasonality).
3.2.1 Estimates
In this stage, autoregressive (AR) and moving average coefficients included in the
model are estimated. The latter uses an algorithm to minimize the sum of the squares of
the residuals, starting with one of the initial values from the model parameters.
3.2.2 Forecasts
At this stage, demand forecasts are made. However, to secure accurate results, we must
first determine whether the original variable was differentiated. Three key time series
indicators were used to assess forecast accuracy:
• MAPE (Mean Absolute Percentage Error): This indicator measures how accu-
rately time series values were estimated.
P
jðyt ^yt Þ=yt j
MAPE ¼ 100; ðyt 6¼ 0Þ ð1Þ
n
• MAD (Mean Absolute Deviation): This indicator measures how accurately time
series values were estimated.
Pn
jyt ^yt j
MAD ¼ t¼1
ð2Þ
n
3.3 Warehouse Development and Organization Phase
First, whenever a new product is admitted into the warehouse, it must be sorted and
labeled. This way, all products are tracked through a serial number and a product
record. However, since not all products are located or managed in the same way and,
instead, this completely changes according to each material in stock and destination.
3.4 Replenishment Phase
Replenishment forms are customized for each company, based on product quantities
and characteristics. The replenishment process is expected to supply the right product,
in the right place, at the right time, in the right quantities, and in the most efficient way
possible. If accurate information is provided at the right time, the company may reap
benefits, such as reduced inventory costs. To assess how optimum the warehouse
Collaborative Model Based on ARIMA Forecasting for Reducing Inventory Costs 701
layout is, it must be compared before and after the model implementation, which must
evidence improvements in terms of inventory and warehouse layout.
4 Validation
4.1 Methodology Application
To establish a suitable operational plan, main and specific objectives were defined.
Then, the strategies for its successful implementation are set forth, evaluating oppor-
tunities for improvement. Next, for forecasting purposes and for assessing the error
percentage from the ARIMA model, forecasts were made up to 2018. This way, model
results may be compared against actual data. Since the most common models are
usually AR, before forecasting, autocorrelation and partial autocorrelation assessments
were performed to examine the 2016 and 2017 data, determining that they are, in fact,
AR.
As per Table 1, deviations from ARIMA forecast are lower compared to the other
types of forecasts. Therefore, ARIMA represents the best form of demand forecasting.
Table 1. % Dev
2018 % Dev Winters % Dev Double Exp. Smoothing % Dev Arima
January 5.85% 1.92% 2.76%
February −3.99% −1.25% 1.27%
March 8.95% 10.16% 3.10%
April 4.37% 5.55% 2.87%
May −2.38% −3.35% 1.69%
June 8.23% 13.09% 6.28%
Total 3.50% 4.39% 2.95%
In the first place, all surplus products were sorted and sold through offers and
discounts, as they represent non-profitable idle inventory. Subsequently, the installation
of metal shelves was essential to organize products according to their classification so
that employees may easily locate them (Table 2).
Table 2. Optimum Warehouse Layout Design
Before After
702 A. Angulo-Baca et al.
4.2 Metrics
After having implemented and validated the methodology in the company under study,
improvement results are measured through the indicators defined in Table 3 below,
which contains benchmarks proposed in previous studies. The indicators have
improved significantly against pre-implementation values.
Table 3. Metrics
Indicators Before Expected After
Inventory accuracy based on demand 87.53% 98.5% 97.05%
Finished product inventory at warehouse 37% 20% 28%
Profit margin per product unit 23.40% 40–45% 41%
5 Conclusions
The textile-and-footwear sector represents 24.2% of all SMEs nationwide, thus being
deemed as one of the most prominent sectors in the country. Therefore, the companies
operating within this sector must streamline their processes to remain profitable and
sustainable. This project focused on finished goods inventory management optimiza-
tion at a small leather footwear company through CPFR techniques and ARIMA
forecasting. After the proposed model had been deployed, this company reported a
17% decrease in the finished product inventory stocked at the warehouse.
References
1. Ganeshan, H., Suresh, P.: An empirical analysis on supply chain problems, strategy, and
performance with reference to SMEs. Prabandhan Indian J. Manag. 10(11), 19 (2017)
2. Fu, H.P.: Comparing the factors that influence the adoption of CPFR by retailers and
suppliers. Int. J. Logist. Manag. 27(3), 931–946 (2016)
3. Moser, P., Isaksson, O., Seifert, R.W.: Inventory dynamics in process industries: AC SC. Int.
J. Prod. Econ. 191, 253–266 (2017)
4. Cannella, S., Framinan, J.M., Bruccoleri, M., Barbosa-Póvoa, A.P., Relvas, S.: The effect of
inventory record inaccuracy in information exchange supply chains. Eur. J. Oper. Res. 243
(1), 120–129 (2015)
5. Duan, L., Ventura, J.A.: A dynamic supplier selection and inventory management model for
a serial supply chain with a novel supplier price break scheme and flexible time periods. Eur.
J. Oper. Res. 272(3), 979–998 (2019)
6. Zare, R., Chavez, P., Raymundo, C., Rojas, J.: Collaborative culture management model to
improve the performence in the inventory management of a supply chain. In: Congreso
Internacional de Innovacion y Tendencias en Ingenieria, CONIITI 2018 – Proceedings,
8587073 (2018)
7. Riquero, I., Hilario, C., Chavez, P., Raymundo, C.: Improvement proposal for the logistics
process of importing SMEs in Peru through lean, inventories, and change management.
Smart Innovation, Systems and Technologies, vol. 140, pp. 495–501 (2018)
Collaborative Model Based on ARIMA Forecasting for Reducing Inventory Costs 703
8. Saric, A.: Improvements in the logistics system in companies in the last 7 years (2017)
9. Reyes, A., Villanueva, N.: Logistics Management improvement proposal to reduce costs in
the Janet EIRL construction company (2018)
10. Panahifar, F., Byrne, P.J., Heavey, C.: A hybrid approach to the study of CPFR
implementation enablers. Prod. Plan. Control 26(13), 1090–1109 (2015)
11. Abd, A., Barau, H.: Enhancing supply chain performance through collaborative planning,
forecasting, and replenishment. Bus. Process Manag. J. 25(4), 625–646 (2018)
12. Petropoulos, F., Wang, X., Disney, S.M.: The inventory performance of forecasting
methods: evidence from the M3 competition data. Int. J. Forecast. 35(1), 251–265 (2019)
13. Prak, D., Teunter, R.: A general method for addressing forecasting uncertainty in inventory
models. Int. J. Forecast. 35(1), 224–238 (2019)
A Framework of Quality Control Matrix
in Paprika Chain Value: An Empirical
Investigation in Peru
Diana Garcia-Montero, Luz Roman-Ramirez, Fernando Sotelo-Raffo,
and Edgar Ramos-Palomino(&)
Ingeniería Industrial, Universidad Peruana de Ciencias Aplicadas (UPC),
Lima 15023, Peru
{u201415400,u201321853,fernando.sotelo,
edgar.ramos}@upc.edu.pe
Abstract. At present, in the agricultural industry, continuous improvement in
products or crops and quality in processes is a challenging task. Therefore,
several quality techniques were adopted in this industry and advantages were
obtained. The main purpose of this document is to present the advantages
offered by the use of quality control and the quality control matrix in order to
reduce non-conformities in the production process. The similarities and differ-
ences between traditional quality control and modern control using QAM will be
presented.
Keywords: Agriculture Control charts Quality control Quality matrix
1 Introduction
The crucial position of agricultural productivity in the economic and social agenda of
countries that are in development such as Peru, the results of agricultural productivity
growth as the key to achieving growth and compliance led by agriculture [1]. It should
be noted that the adoption of new technologies has been widely accepted as a means to
increase productivity, the growth of production has also been used technology on one
side. In other words, agricultural productivity depends on two components [2]: The first
is the type and quality of inputs in the production process and the good as production
technology. The second component relates to how well these inputs are combined and,
therefore, apply to the technical efficiency of the production process [3].
In this sense, the planning and implementation of control panels plays a funda-
mental role in the desired reliability and availability of the final product for subsequent
sale [4]. The ultimate goal of any quality control panel is to reduce or eliminate system
failures, which in turn will improve and guarantee the effective reliability of the system
[5]. The monitoring of the reliability of the paprika product is an important tool to
monitor the effectiveness and productivity of the production process throughout the
period. Therefore, constant monitoring increases the reliability of the products and
can provide more benefits to the farmers of the research site, located in Barranca,
Lima-Peru.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 704–710, 2020.
https://doi.org/10.1007/978-3-030-39512-4_108
A Framework of Quality Control Matrix in Paprika Chain Value 705
The control boxes are the most used SPC tools. Several researchers have addressed
the integration of control panels with agriculture management. However, research on
the integration of SPC and the agriculture sector is not adequately analyzed in the
literature [6]. The purpose of this document is to provide a brief review of the existing
literature on the integration of control panels in the agro-industrial sector, and to
suggest some possible directions for future research [7].
2 Methodology
The first is the type and quality of inputs in the production process and the good as
production technology. The second component relates to how well these inputs are
combined and, therefore, apply to the technical efficiency of the production process [8].
The main purpose of this document is to present the advantages offered by the use
of quality control and the quality control matrix in order to reduce non-conformities in
the production process. The similarities and differences between traditional quality
control and modern control using QAM will be presented [9].
Quality control matrix: This Matrix is designed to help those responsible for the
area of Quality Management to evaluate the objectives established for the Quality
Management System, providing a general and comprehensive overview of the status of
these objectives [10]. By performing the corresponding analysis and assessment of
each of the evaluation criteria, this matrix also functions as a guide-control to direct the
process of establishing the objectives in the sense that it allows verifying if the
requirements of the international standard are being met. ISO 9001: 2015 for the
objectives of the SGC, in order to be really useful and practical and, consequently,
allow the development of strategies and plans for its achievement [11].
Management by processes: It is structured, functional analytical and process
improvement, It is focused on the gradual and continuous improvement of processes
within a business instead of a reengineering. The main objective of process manage-
ment is to improve processes in order to ensure that critical processes are effective
and efficient, thus aligning with the strategic objectives of the company and customer
needs [12].
Process management has various tools such as Business Process and Notation,
which has the objective of modeling business processes and indicators are used that are
related to the strategic objectives of the company and/or business [13].
Another of the tools of process management is the SIPOC diagram, which means
Supplidfx-Inputs-Process, Output-Custumers. The sipoc diagram identifies and maps
the gaps between the specifications of the suppliers and the output specifications,
expectations of the clients to define that way the scope of the process improvement
activities [14].
Chain Value: The chain value is a very important instrument that is used to perform
analyzes of the different activities carried out by a company in order to identify its
sources of competitive advantage [15]. A value chain is a high-level model developed
by Michael Porter used to describe the process by which companies receive raw
materials, add value to raw materials through various processes to create a finished
product and then sell that final product to the clients [16]. Companies conduct an
706 D. Garcia-Montero et al.
analysis of the value chain when they observe the production steps needed to create a
product and identify ways to increase the chain’s efficiency [16].
3 Results
To obtain the results, a generalized analytical model called the Monitoring Model is
used to coordinate the various controls of paprika crop production. With the objective
of obtaining and planning to minimize the expected costs and increase the price of the
crop [17].
An effective way to control high quality processes is to use time frames and quality
controls, all the matrices between events [18].
Some authors have addressed the maintenance combination with the times between
the event control tables. Research in this area is still limited [19]. Of the documents
investigated, two documents that have discussed the maintenance combination with the
TBE control table.
The proposed the use of the time table between events in the monitoring of the
useful life of a distributed system. His mathematical model presents an integration
between maintenance decisions with SPC [20]. They assumed that the system consists
of a single component, which rarely happens in engineering applications. However,
presented the use of the time frame between events in the monitoring of the time
between failures of a reparable system with multiple components. We propose a step-
by-step procedure that helps decision makers to periodically monitor and improve the
reliability of the system. Figure 1 presents a summary of its suggested procedure.
Fig. 1. Analysis of the scenarios
It is very important to take into account the objectives to carry out the imple-
mentation of a self-control system in the most correct way possible, these objectives
are: attitude, uniqueness, technologies, order, control, organization, needs, traceability,
responsibility, observation cleaning [21].
Therefore, it is also important to define the sequence of steps to analyze the
problems that arise in the day to day of the focus of study [22] (Fig. 2):
A Framework of Quality Control Matrix in Paprika Chain Value 707
Fig. 2. Identification of the problem or defect
To carry out the application of the self-control matrix, it is first necessary to identify
the defect or problem. Secondly, the relationship that exists of the defect or problem
with the various activities of the process will be analyzed [23].
Next, the matrix of quality self-control will be presented (Table 1):
Table 1. Quality control matrix
External Internal Activity Activity Activity Activity Activity
Total
provider provider 1 2 3 4 5
Activity 1
Phase Activity 2
where Activity 3
the Activity 4
defect Activity 5
or prob- Internal
lem is customer
detected External
customer
Total
The red boxes represent the activities that should be given the highest priority in the
analysis as it is where the fault or problem is detected.
The green boxes do not represent a major impact in the analysis of the problem as
the case may be.
As part of the continuous improvement, the use of the risk control table is also
proposed, since in this way it seeks to mitigate any type of unforeseen event that may
occur throughout the season of paprika cultivation in the district of Araya Grande,
Barranca.
3.1 Risk Control Table
The Risk Control table, which aims to identify, control and eliminate sources of risk
before they begin to affect the fulfillment of the project’s objectives. In this way, the
impact of the possible risks can be evaluated and estimated and, in turn, a contingency
plan can be established to mitigate them in the event that the problem arises, with pro
activity as its objective. Therefore: (1) it starts before the technical work, (2) it is
categorized or classified according to its impact, (3) it identifies the potential risks,
assessing their probability of impact and (4) establishing a plan [24].
708 D. Garcia-Montero et al.
Risk Category Probability Impact Action plan
Classification of category and risk factors the level of impact:
Performance Negligible
Cost Marginal
Risk components Risk factor's
Maintainability Critical
Planning Catastrophic
To finish the cycle of self-control of quality is of utmost importance the philosophy
of continuous improvement because it is an ideal tool, so that the work team under-
stands and seeks quality in the development of the product [25]. The process of
continuous improvement is a constant refinement of the standards to respond in a
dynamic way to the client’s demands and the opportunities to improve the processes.
For to happen, the administration must first establish the standard or base in the
processes or activities, so that, subsequently, the PDCA cycle performs its regulatory
function 20, improving the established standards. In this way, the PDCA cycle allows
organizational learning and the achievement of better standards [26].
4 Conclusion
The Quality Assurance is a set of methods, tools and techniques that allow to manage
the quality in the development of a product [27]. Despite being a fundamental element
when developing a project, not all companies apply it due to budget, lack of personnel
or adaptations of more complex standards [28]. This article presents a practical
approach as a guide, to manage quality in the development of companies with a level
zero maturity [20].
For the correct implementation of self-control in the different transformation
industries, usually begins with a training/information campaign to the main stake-
holders (operators) where they are explained what is expected of them [29].
The objectives to carry out the implementation of a self-control system are: attitude,
uniqueness, technologies, order, control, organization, needs, traceability, responsi-
bility, observation, cleaning [5].
References
1. Bouzembra, Y., Camenzuli, L., Janssen, E., Van der Fels-Klerx, H.F.: Application of
bayesian networks in the development of herbs and spices sampling monitoring system.
Food Control 83, 38–44 (2017)
2. Szandra, K., Helga, M., Miklós, P., Ildikó, B., Nóra, A.: Quality management in spice
paprika production: from cultivation to end product (2018). https://doi.org/10.5772/
intechopen.71227
A Framework of Quality Control Matrix in Paprika Chain Value 709
3. Damiao, C., Reinaldo, R.: Microbiological quality of Argentinian paprika. Computers and
Electronics in Agriculture, FI: 0.896, Q2 (2016)
4. Bracke, J., Schmiedel, T., Recker, J., Trkman, P., Mertens, W., Viaene, S.: The effect of
intensive chemical plant protection on the quality of spice paprika. Bus. Process Manag.
J. 67, 141–148 (2014)
5. Czako, E., Konczol, E.: Critical success factors of export excellence and policy implications:
the case of Hungarian small and medium-sized enterprises. In: Gubik, A.S., Wach, K. (eds.)
International Entrepreneurship and Corporate Growth in Visegrad Countries, pp. 69–84.
University of Miskolc, Miskolc (2014)
6. ZakiI, N., HakmaouiI, A., OuatmaneI, A., Fernandez-Trujillo, J.: Quality characteristics of
Moroccan sweet paprika (Capsicum annuum L.) at different sampling times. Food Sci.
Technol. 33(3), 577–585 (2014)
7. Jin, N., Lee, Y., Lee, B., Jun, J., Kim, Y., Seo, M., Lim, C., Youn, Y., Yu, Y.: Pest control
effect and optimal dose by pesticide dispersion spray method in the paprika Cultivation.
Korean J. Pestic. Sci. 18, 350–357 (2014). https://doi.org/10.7585/kjps.2014.18.4.350
8. Nainggolan, K.: Major issues and challenges for improving the marketing and distribution of
agricultural products. Agro Ekon. 10 (2003). https://doi.org/10.22146/agroekonomi.16786
9. Vincenzina, F., Antonio, L., Rodriguez, F.: Food safety aspects on ethnic foods:
toxicological and microbial risks. Curr. Opin. Food Sci. 6, 24–32 (2015)
10. Kónya, É., Szabó, E., Bata-Vidács, I., Deák, T., Ottucsák, M., Adányi, N., Székács, A.:
Quality management in spice paprika production as a synergy of internal and external quality
measures (2016)
11. Molnár, H., Kónya, E., Székács, A.: Chemical characteristics of spice paprika of different
origins. Food Control 83, 54–60 (2018)
12. Mahantesh, Y., Bmadalageri, M., Pujari, J., Mallimar, S.: Genetic variability studies in chilli
for yield and quality attributes. Indian J. Ecol. 42(2), 536–539 (2015)
13. Lekshmi, S.: Variability, heritability and genetic advance in paprika (Capsicum annuum L.).
Indian Hortic. 6(1), 109–111 (2016)
14. Gutierrez, L., Munoz, J.: Total quality management practices, competitive strategies and
financial performance: the case of the palestinian industrial SMEs. Total Qual. Manag. Bus.
Excell. 25(5–6), 635–649 (2015)
15. Jiao, X., Mongol, N., Zhang, F.: The transformation of agriculture in China: looking back
and looking forward. J. Integr. Agric. FI: 0.401 (2016)
16. Randhawa, J.: 5S-a quality improvement tool for sustainable performance: literature review
and directions. Int. J. Qual. Reliab. Manag. 34(3), 334–361 (2017)
17. Klatyik, S., Darvas, B., Mortl, M., Ottucsak, M., Takacs, E., Banati, H., Simon, L., Gyurcso,
G., Szekacs, A.: Food safety aspect of pesticide residues in spice paprika. Int. J. Nutr. Food
Eng. 10(3), 188–191 (2016)
18. Schaarschmidt, S.: Public and private standards for dried culinary herbs and spices - part I:
standars defining the physical and chemical product quality and safety. Food Control 70
339–349 (2016). Department biological safety, bundesinstitut fur risikobewertung
19. Kekuewa, N., Ardoin, M.: Cultivating values: environmental values and sense of place as
correlates of sustainable agricultural practices. Agric. Hum. Values 33(2), 389–401 (2016)
20. Marine, S., Martin, D., Adalja, A., Mathew, S., Everts, K.: Effect of market channel, farm
scale, and years in production on mid – Atlantic vegetable producer’s knowledge and
implementation of good agricultural. Food Control 59, 128–138 (2016)
21. Mok, W.: Maximizing control flow concurrency in BPMN workflow models through
syntactic means. Bus. Process Manag. J. 24(2), 357–383 (2017)
22. Morais, M., Rossi, J., Binotto, E.: Using the reasoned action approach to understand
Brazilian successors’ intention to take over the far. Land Use Policy 71, 445–452 (2018)
710 D. Garcia-Montero et al.
23. Müller, C., Elliott, J., Chryssanthacopoulos, J., Arneth, A.: Global gridded crop model
evaluation: benchmarking, skills, deficiencies and implications. Geosci. Model Dev. 10(4),
1403–1422 (2016)
24. Parent, J., Lein, Q.: A review of the importance of business process management in
achieving sustainable competitive advantage. TQM J. 26(5), 522–531 (2015)
25. Wezel, A., Casagrande, M., Celette, F., Vian, J., Ferrer, A.: Agroecological practices for
sustainable agriculture. A review. Agron. Sustain. Dev. 34(1), 1–20 (2014)
26. Wongprawmas, R.: A multi-stakeholder perspective on the adoption of good agricultural
practices in the Thai fresh produce industry. Br. Food 117(9), 2234–2249 (2015)
27. Izquierdo, J., Rodriguez, F.J., Duran, M.: Guidelines Good Agricultural Practices for Family
Agriculture, Food and Agriculture Organization (2007)
28. Tradewinds Plantation Berhad. Good Agricultural Practices (2018)
29. Chaifetz, A., Alnajjar, K., Ammerman, A., Gunter, E., Chapman, B.: Implementation of
good agricultural practices (GAPs) in school and community gardens. Food Prot. Trends 35
(3), 167–175 (2015)
Inventory Optimization Model Applying
the Holt-Winters Method to Improve Stock
Levels in SMEs in the Sports Retail Sector
Diego Amasifén-Pacheco1, Angela Garay-Osorio1,
Maribel Perez-Paredes1, Carlos Raymundo-Ibañez1(&),
and Luis Rivera2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Peru
{u201417538,u201313716,maribel.perez,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. Organizations are aware that the world of sports retail is shaped by
situations and variables that make it a complex and competitive scenario in
nature. This leads to a very common problem of inventory management, because
the stock levels are not suitable for market behavior, generating significant loss of
sale and increase in cost and reducing margins of companies. This research seeks
to design an inventory optimization model that integrates tools such as multi-
criteria ABC analysis, Holt-Winters, and 5S methodology for improving stock
levels in SMEs within the sports retail sector. The proposed model helps find an
efficient method for the optimization sought, which enables the planning of sales
and stock purchases according to the needs of the company. The main result of
the validation in the case study was the reduction of additional costs by 85%.
Keywords: Inventory management Holt-Winters Sports retail Multi-
criteria ABC analysis 5S
1 Introduction
The sustained growth of the Peruvian economy has allowed the retail sector to expe-
rience significant and constant expansion in the last decade. In 2018, sales in the retail
sector grew 9.7% and represented 1.91% of the national GDP. In addition, Peru ranks
ninth in the list of 30 emerging countries classified as the most attractive for investing
in the retail sector.
However, 40% of national retail companies are inefficient in managing their supply
chains [1] and are estimated to have 20–30% dead or obsolete inventory [2], reflecting
a gap that prevents these organizations from evolving into more competitive
businesses.
It was identified that the stock levels managed by SMEs in the sports retail sector
are not optimal due to the strategic goals they set forth. This problem is mainly caused
by deficiencies in the planning, management, and control of inventories, which
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 711–718, 2020.
https://doi.org/10.1007/978-3-030-39512-4_109
712 D. Amasifén-Pacheco et al.
translates into shortages and surplus stocks, low rotation, and deterioration of goods.
This causes operating costs and expenses to increase and, therefore, liquidity to
decrease as a result of lost sales [3]. In addition, companies often lose potential cus-
tomers due to the dissatisfaction generated by not having the products they need when
visiting a store. This research seeks to design a solution to address the supply chain
problems faced by companies in the sports retail sector, which are evidenced by their
stock levels. The proposed optimization model consists of tools based on the assess-
ment of demand behavior, forecast of purchases according to the needs of the company,
and application of the 5S methodology to strengthen the model through the imple-
mentation of good warehousing practices to correctly execute sales, product, and
procurement plans.
2 State of the Art
2.1 Holt-Winters Method
The applicability of the Holt-Winters logistics method has been developed in the
production, trade, and service industries over time [4]. Indeed, Teunter, Syntetos,
Riquero, Bastidas [5–7] assure that their implementation has produced satisfactory
results, both in reducing the cost of inventory and in the increase in service quality. It
has also reduced emerging actions to avoid lost sales due to lack of inventory and total
inventory breakdowns.
Similarly, Bastidas and Toro [7] were able to assess the economic impact of the
methodology applied in a household appliance retail company, through simulation and
the Holt-Winters forecasting method, since a 50% reduction in costs due to stockouts is
defined. In addition, the proposed methodology represents a significant improvement
opportunity for the inventory management process, as this reduction will represent an
increase in perceived utility.
2.2 Multi-criteria ABC Analysis
The classical ABC analysis has been questioned by some authors [7, 8], mainly
because the importance and attention given by the management to each item depends
on a single criterion when making the classification, knowing that some opportunities
have characteristics and attributes that should be considered and possibly affect its
importance. Moreover, Castro [8] proves that the application of the multi-criteria ABC
method to inventory models through fuzzy logic is very important to obtain classifi-
cations that consider several criteria at the same time, thus engaging the expertise of the
people in charge of the inventories in order to strengthen the item classification.
Inventory Optimization Model Applying the Holt-Winters Method 713
3 Contribution
3.1 Model Proposed
The model proposed optimizes inventories against demand volatility and supply
variability, thus reducing overstocking and understocking risks, and optimizing their
inherent costs. The deployment of the 5S methodology approach and the multi-criteria
ABC tool guarantees proper process flow rates and product classification as per market
behavior criteria. In fact, the results from both tools provide a clear snapshot of the
stock held by the company, which leads to sales forecasting under the Holt-Winters
tool in order to determine the product quantities to be purchased. The interrelation of
the three elements makes it possible to measure the results objectively in order to find
points for improvement, standardize the activities inherent to the process, and generate
greater value to the proposed model (see Fig. 1 below).
Fig. 1. Optimization model
3.2 Method Proposed
In order to have greater understanding of the model proposed, a flowchart of the
activities of the optimization proposal was prepared (see Fig. 2).
714 D. Amasifén-Pacheco et al.
Fig. 2. 5S Improvement Matrix in Beta Store – Sport SAC.
3.3 Indicators
As part of the development of the proposed model, management indicators have been
created (see Fig. 3) to assess the current operations in order and set forth an evolution
horizon, so that the results obtained may be compared with the inventory level issues
reported by sports retail companies.
Fig. 3. KPIs of the model proposed.
4 Validation
4.1 Application of the Model Proposed
The model consists of three phases that together achieve the stock level optimization. It
is necessary to point out that the validation of the case study has been segmented by
product families at the “Beta” store of one of the main sports retailers in Peru. Then,
based on a previous analysis of the total SKUs of the “Beta” store, the multi-criteria
ABC tool will be used to assess the 56 different product families of Nike urban
footwear for men.
Inventory Optimization Model Applying the Holt-Winters Method 715
With this tool, we will classify the result of the sum of homogenized values in
descending order under the following criteria: quantity of stock, rotation, and product
cost.
4.1.1 Phase 1
This matrix is rated on a monthly basis. However, it will now be rated quarterly in
order to be able to better notice any changes. The rating protocol involves operators
who check and score each item (see Fig. 4).
Fig. 4. 5S Improvement Matrix in Beta Store – Sport SAC.
4.1.2 Phase 2
The result of the multi-criteria ABC analysis in which 16 families of products belong to
classification A, which represents 28.56% of the total families and 69.94% of the total
products owned by the store in study.
716 D. Amasifén-Pacheco et al.
After performing the multi-criteria ABC analysis, the sales forecast was calculated
using the Holt-Winters method. For this purpose, sales history records from 2016 to
2018 were used as the bases for 2019 sales forecast.
First, we forecast demand for 2018, which has already been executed. That is to
say, we use the parameters obtained, with data from 2016 and 2017, to verify whether
they reasonably adjust to 2018. Once this adjustment is achieved, we can conclude that
our parameters are properly calibrated, and we can calculate the forecast for 2019 (see
Figs. 5 and 6).
One of the advantages of the Holt-Winters method is that you can adjust the
forecast as actual data becomes available.
Fig. 5. Demand Forecast for Beta store – Sport SAC.
Fig. 6. Sales Forecast 2019 Beta Store – Sport SAC.
Inventory Optimization Model Applying the Holt-Winters Method 717
Second, after forecasting sales for 2019, the proper inventory quantities are
established for each product, taking into account that the safety stock established is 5%
over the forecast.
4.1.3 Phase 3
The last phase of the proposed model lies in assessing performance through graphic
indicators that represent the current situation and the proposed situation with the
inventory optimization model (see Figs. 7 and 8).
In fact, the proposed situation is more accurately adjusted to historical sales in
October, November, and December 2018, and January, February, and March 2019.
Fig. 7. Current situation.
Fig. 8. Proposed situation.
718 D. Amasifén-Pacheco et al.
5 Conclusions
It was possible to optimize the stock level of the store wherein the model was validated,
thus reducing costs and expenses by 84.57%. Capital savings of 81,035.52 Soles were
reported, which were marked to buy large quantities of products that did not match the
store’s demand.
In the four scenarios where the Holt-Winters forecasting method was validated, a
margin of error of less than 5% was obtained, adjusting to the monthly sales behavior
of the case study.
The proposed optimization model represents a significant improvement opportunity
for the inventory management process, as this reduction will represent an increase in
profits. In addition, the store is now able to meet demand forecasts, thus guaranteeing
optimum levels of customer service and, in turn, elevated store image and credibility.
References
1. Soluciones de Marketing: Primer Estudio sobre la Situación del Supply Chain Management
en el Perú. [First Study on Supply Chain Management in Peru] (2018). http://
semanaeconomica.com/wpcontent/uploads/2018/10/encarte_Supply_Chain_Management_
OK_baja.pdf4
2. Perú Retail: El sobre stock destruye la cadena del retail. [Overstock destroys retail chains]
(2019). https://www.peru-retail.com/sobrestock-destruye-cadena-retail/
3. Santa Cruz, A.K., et. al.: Sales and operation planning model to improve inventory
management in Peruvian SMEs (2019). https://doi.org/10.1109/ICITM.2019.8710734
4. Tirkeş, G., Güray, C., Çeleb, N.: Demand forecasting: a comparison between the holt-winters,
trend analysis and decomposition models (2017). https://doi.org/10.17559/TV-
20160615204011
5. Teunter, R.H., Syntetos, A.A., Babai, M.Z.: Intermittent demand: linking forecasting to
inventory obsolescence. Eur. J. Oper. Res. 214, 606–615 (2011)
6. Riquero, I., et. al.: Improvement proposal for the logistics process of importing SMEs in Peru
through lean, inventories, and change management (2019). https://doi.org/10.1007/978-3-
030-16053-1_48
7. Bastidas, V.E., Toro, L.A.: (2011). Methodology for the control and inventory management in
a company appliance retailer. Scientia et Technica. http://dx.doi.org/10.22517/23447214.1481
8. Castro, C.: Clasificación ABC Multicriterio: Tipos de criterios y efectos en la asignación de
pesos. [ABC multi-criteria classification: types of criteria and effects on money allocation].
ITECKNE Book 8 No. 2 pp. 163 – 170 (2011)
Recruitment and Training Model for Retaining
and Improving the Reputation of Medical
Specialists to Increase Revenue of a Private
Healthcare SME
Audy Castro-Blancas1, Carlos Rivas-Zavaleta1,
Carlos Cespedes-Blanco1, Carlos Raymundo1(&), and Luis Rivera2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Peru
{u201510504,u201422189,carlos.cespedes,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. The present research presents an improved recruitment and training
model focused on increasing physician reputation and loyalty to retain good
physicians in the provinces of Peru and thereby increase competitiveness and
profitability of the health care organization. Consequently, a pilot test of the
model was implemented in an assisted reproductive technology (ART) clinic
based in Chiclayo and Piura using a recruitment process checklist. The data
resulted after the implementation were analyzed using the success rate of in vitro
fertilization (IVF) treatments of the new physician and a structured engagement
survey to determine retention by validation. In the new recruitment model, the
new physician achieved a success rate of 46.15% in IVF pregnancies. The main
physician participation was reduced from 69% to 48% without a decrease in
total sales. The level of engagement achieved by the new physician was 96%,
surpassing the average of the clinic’s medical team of 79%.
Keywords: Personnel recruitment Continuous training and practice
Reputation Retention
1 Introduction
In 2016, América Economía published an article entitled “Peru: Thousand physicians
emigrate every year to other countries,” which showed that apart from a low remu-
neration at the national level, which is one of the lowest in the region: $1,500 compared
to the average of $3,000 in Latin America, there is no medical career environment with
establishments that offer specialization or training permits or the prospect of attending
conferences and courses both in the country and abroad; further, research is not pro-
moted [1]. In the same year, 16,630 medical specialists were required to cover the
health needs of over 30 million Peruvians; however, only 8,074 were available [2].
Subsequently, in 2017, the informality rate of healthcare companies became one of the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 719–725, 2020.
https://doi.org/10.1007/978-3-030-39512-4_110
720 A. Castro-Blancas et al.
highest, i.e., 3 to 1, since there were approximately 60,000 health centers operating in
the informal sector [3]. Studies in the health sector show the predominance of 62% and
25% in socioeconomic levels A and B, respectively, in the choice of the private clinic
over other types of services, among which are Minsa, EsSalud, or Solidarity Hospitals,
or consulting a private physician [4]. Similarly, there is an unequal distribution of
physicians in Peru, according to the former Minister of Health and professor at the
Cayetano Heredia Peruvian University, Eduardo Pretell. At the end of 2014, 47.7% of
specialists in the country were concentrated in clinics and hospitals in Lima. Only in
Lima, Callao, Arequipa, and La Libertad are 65.1% of the total amount of specialists
distributed nationwide [5].
For the aforementioned problems, studies have been conducted in small and
medium-sized enterprises (SMEs) and have concluded that most entrepreneurs or their
partners have university education and begin their businesses in the sector in which
they have worked. However, they do not necessarily have management knowledge in
terms of administrative, financial, and managerial issues [6]. Efforts have already been
made, and research has been conducted on a recruitment program aimed at attracting
and training professionals to perform their medical internships in rural areas [7].
The following proposed solution is aimed at increasing the loyalty of medical
professionals toward private healthcare centers in the provinces of Peru as well as
seeking to increase the reputation level in less time and minimize the effects of
problems experienced by healthcare SMEs. This model provides an alternative solution
to the current problem of recruiting and training specialists in the provinces of Peru, in
addition to engaging and retaining medical specialists in organizations, given the
shortage of specialized healthcare workers in Peru. It also aims at improving the
average time it takes for a health professional to acquire and gain reputation and
experience in a specialized area, which, in the case study, is an assisted reproductive
technology (ART) clinic, as well as at replicating the model in other small healthcare
centers in the provinces of Peru. Finally, one difficulty to consider is the time factor in
relation to the validation of each of the model components.
2 Literature Review
2.1 Model of Recruitment and Continuous Training in Healthcare
Given the need for qualified physicians, a training program was created that helped
students become certified physicians and improve their skills. Physicians considered
continuous training to be the most effective way of learning [8]. In the rural areas of
Iran, a training model for community health workers showed that the relationship
between training and employee performance improves the model effectiveness
according to the community health needs [9]. In more urbanized areas, highly spe-
cialized physicians using advanced technology and complex treatments are needed, and
better education and practical training are essential to achieve that quality [10].
Continuous training improves the specialists’ knowledge to a high degree. More-
over, it also improves their performance and job satisfaction and generates a positive
economic impact on the health organization in which they work [11]. Similarly, studies
Recruitment and Training Model 721
on the continuous training of the health workforce, including medical specialists,
showed the direct relationship between training factors such as satisfaction, training
modality, and professional competence and the patients’ perceived quality of care [12].
2.2 Recruitment and Retention Model with Organizational Commitment
Employee engagement is important in any healthcare organization. Individual char-
acteristics and work experiences are essential in terms of employee engagement.
Motivating factors were divided into individual, community, and organizational fac-
tors. One of the main engagement factors is the supervision and support from a team
leader or colleagues, which positively affects organizational commitment [13, 14].
Another important motivating factor, particularly for young physicians or medical
students, is the opportunity to specialize if there is an attractive goal-oriented training
plan to become a medical specialist [15].
3 Contribution
3.1 Proposed Model
See Fig. 1.
Individual Needs
Recruitment Processes Training Processes
Profile Retention Continuous Reputation Promotion
Interview
Development Plan Training Aproach
Organizational Needs
Fig. 1. Model diagram for the improvement of recruitment and training processes.
3.2 Proposed Method
The proposal is divided into the following activities: assess decisive factors in the
profile of the specialist to be hired, plan the retention model to be implemented,
No
Assess decisive ¿Does the Determine
specialist meet Assess Implement the
factors in the medical Yes continuing
requirements ? results model
specialist’s profile training
Plan the retention Determine Conduct the
model to be training benefits pilot test
implemented
Fig. 2. Proposal flowchart. Prepared by the author. 2019
722 A. Castro-Blancas et al.
determine the type of continuing training to be conducted, and determine and com-
municate training benefits. The flowchart is shown in Fig. 2.
Ratios
The metrics to use are as follows.
Engagement Level. This ratio measures the level of employee engagement toward the
organization where they work. The formula is where a total of 37 questions will be
evaluated with values from 1 (Strongly Disagree) to 5 (Strongly Agree).
P37
Value½questionðiÞ
i¼1
100%; ð1Þ
37 5
Physician Capability Level (Effectiveness). This ratio measures the satisfaction of
clients who, in this case, are the patients.
In the case study of an ART clinic. One of the medical techniques used is in vitro
fertilization (IVF). This ratio is analyzed through the result of this procedure since it is
a complex process that increases the revenue of the case study company.
P
Efffective IVF Treatments½PhysicianðiÞ
P ð2Þ
IVF Treatments
Physician Participation Level. The level of participation of both the physicians hired
under the traditional case study model and the physicians involved in the pilot test will
be measured.
IVF Treatments½PhysicianðiÞ
P ð3Þ
IVF Treatments
Net Present Value (NPV). The NPV formula is used to calculate the present value for
the five years of future cash flows that a physician hired under the recruitment and
selection model would generate in the clinic.
X
n
Fi
Net Present Value ¼ I0 þ ð4Þ
i¼1 ð1 þ cok Þi
4 Validation
Current Diagnostic
A pilot test has been conducted in a small clinic that provides ART services known as
“Gestar In Vitro”.
Recruitment and Training Model 723
Current Diagnostic
Much of the economic impact on the two clinics falls on physician’s services.
Physician Engagement Level
See Table 1.
Table 1. Engagement level results. Current
Physicians under the traditional model (%)
Engagement level 79% overall average
Physician Capability Level. The results will be compared with the international
standards provided by the Society for Assisted Reproductive Technology (SART) that
has approximately 90% of IVF effectiveness studies in the clinics of the United States
(Table 2).
Table 2. Results of IVF Births, U.S. Network of Clinics
Weighted result/outcome
Number of initiated cycles 120,160
Live-birth rate per initiated IVF cycle 31.16%
Level of Physician Participation
See Table 3.
Table 3. Level of participation of case study physicians, March–December 2018
Level of participation (%)
Main Physician 69
Physician A 21
Physician B 6
Physician C 4
NPV of Cash Flow Forecasting for a Physician (IVF Treatments Only). To make a
diagnosis of the clinic sales, the average number of IVF treatments of physician A was
considered (Table 4).
Table 4. Financial ratios current situation
Financial ratio
Annual OCC (%) 10.8%
Monthly OCC (%) 0.9%
NPV 124,727
724 A. Castro-Blancas et al.
5 Summary of Results
The results show that the physician achieved a 96% level of engagement compared to
the 79% engagement the other physicians. Ratio 2. The effectiveness of the new
physician was compared against the international levels of the Assisted Reproductive
Technology Society (SART). The new physician achieved 46.15% of effectiveness
against 31.16%. Ratio 3: The level of participation of the new physician increased
notably, from being an inexperienced physician in IVF to being the second physician in
terms with a 26% level of participation.
6 Conclusions
The recruitment and training improvement model is based on an initial recruitment
model with later modifications and additional components from other models.
A physician hired under the proposed model, within a six-month span, achieved a
participation of 26%; the percentage of the main physician was reduced by 21% in both
branches, taking away a large workload. The knowledge and capacity level of the
inexperienced physician is increased in less time, being validated by the success rate of
the physician’s IVF treatments under the implementation of the Case Study pilot that so
far has seven pregnancies achieved with the IVF of 14, i.e., 46.15% of the total from
January to May. The data are compared with the international standards provided by
the SART with a success rate of 31.6%.
References
1. América Economía Magazine: Peru: One thousand physicians emigrate each year to other
countries, 04 May 2016 (2016). https://clustersalud.americaeconomia.com/peru-mil-
medicos-emigran-cada-ano-a-otros-paises
2. Gestión Newspaper: Peru needs more than 16 thousand specialist physicians (2016). https://
gestion.pe/suplemento/comercial/clinicas-centros-medicos/cifras-peru-necesita-mas-16-mil-
medicos-especialistas-1001790
3. Gestión Newspaper: Susalud: There are 60,000 informal medical establishments in Peru,
triple the number of formal ones, 28 August 2017 (2017). https://gestion.pe/economia/
susalud-hay-60-000-established-medical-informales-peru-triple-formales-142486
4. Gestión Newspaper: 62% of Lima residents of SEL A go to clinics and 33% of SEL D to
hospitals (2016). https://gestion.pe/economia/62-limenos-nse-clinicas-33-d-hospitales-4368
5. El Comercio Newspaper: 47.7% of medical specialists are concentrated in Lima, 21 August
2014 (2014). https://elcomercio.pe/economia/peru/47-7-medicos-especialistas-concentran-
lima-175747
6. Condorchoa, E., Gonzales, L.: Analysis of the factors that limit the development and growth
of small businesses in Lima, Peru. Thesis, Peruvian University of Applied Sciences, Lima
(2017)
7. Eidson-Ton, W.S., Rainwater, J., Hilty, D., Henderson, S., Hancock, C., Nation, C.L.,
Nesbitt, T.: Training medical students for rural, underserved areas: a rural medical education
program in California. J. Health Care Poor Underserv. 27(4), 1674–1688 (2016)
Recruitment and Training Model 725
8. Li, X., Shen, J.J., Yao, F., Jiang, C., Chang, F., Hao, F., Lu, J.: Does exam-targeted training
help village physicians pass the certified (assistant) physician exam and improve their
practical skills? A cross-sectional analysis of village physicians’ perspectives in Changzhou
in Eastern China. BMC Med. Educ. 18(1), 107 (2018)
9. Rahbar, M., Ahmadi, M.: Lessons learnt from the model of instructional system for training
community health workers in rural health houses of Iran. Iranian Red Crescent Med. J. 17(2),
e2145 (2015)
10. Kabanova, E.E., Vetrova, E.A.: Modern medical higher education institutions in Russia. Eur.
J. Contemp. Educ. 7(4), 710–716 (2018)
11. Yoon, H., Shin, J., Bouphavanh, K., Kang, Y.: Evaluation of a continuing professional
development training program for physicians and physician assistants in hospitals in Laos
based on the Kirkpatrick model. J. Educ. Eval. Health Prof. 13, 21 (2016)
12. Gracía-Pérez, M.L., Gil-Lacruz, M.: The impact of a continuing training program on the
perceived improvement in quality of health care delivered by health care professionals. Eval.
Program Plan. 66, 33–38 (2018)
13. Zare, R., Chavez, P., Raymundo, C., Rojas, J.: Collaborative culture management model to
improve the performance in the inventory management of a supply chain. In: Proceedings of
2018 Congreso Internacional de Innovacion y Tendencias en Ingenieria, CONIITI 2018
(2018). https://doi.org/10.1109/coniiti.2018.8587073
14. Bhatnagar, A., Gupta, S., Alonge, O., George, A.S.: Primary health care workers’ views of
motivating factors at individual, community and organizational levels: a qualitative study
from Nasarawa and Ondo States, Nigeria. Int. J. Health Plan. Manag. 32(2), 217–233 (2017)
15. Rozsnyai, Z., Tal, K., Bachofner, M., Maisonneuve, H., Moser-Bucher, C., Mueller, Y.,
Streit, S.: Swiss students and young physicians want a flexible goal-oriented GP training
curriculum. Scand. J. Primary Health Care 36(3), 249–261 (2018)
Research on Disabled People’s Museum Visit
Experience from the Perspective of Actor-
Network Theory
Shifeng Zhao(&) and Jie Shen
Jiangnan University, Wuxi, Jiangsu, China
[email protected]
Abstract. Objective: In course of trying to integrate into social and cultural
exchanges, people with disabilities face the problems during museum travel will
not be the accessibility issue of the building environment, but the barriers in
disabled people’s communication and interaction with other actors in the
exhibition environment. Method: The literature results of disabled people’s
barrier-free travel and museum barrier-free design are summarized. From the
perspective of Actor-Network Theory, human and non-human actors are
involved in the design network, thus playing an equal and unbiased stabilizing
role throughout the barrier-free wheelchair travel system. Conclusion: Investi-
gation and analysis are conducted on the research of museum information
barrier-free access; the relevant capital and interest needs of the patients with
lower limb disabilities in cultural education and independent travel involved in
this study are clarified. Moreover, based thereon, a translation network that is
equally driven by disabled people and other actors is constructed.
Keywords: Human factors Actor-Network Interaction barrier-free
1 Introduction
In today’s era, the social problems that we need to solve are becoming more and more
complex, and the user’s level of Maslow demand is also rising. The design has
strengthened theoretical and scientific in the construction of disciplines in recent years.
In particular, user experience and interaction design combine the knowledge of vision,
space, psychology, and sociology to solve complex social problems. The intervention
of actor network theory can describe the process of solving problems and building
products as a dynamic network that promotes each other. The problems with the
interaction between actors in this process can be described.
2 Literature Review
There Are Few Studies on Barrier-Free Interaction in Museums. When partici-
pating in social and cultural exchanges, people with disabilities need to face not only
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 726–732, 2020.
https://doi.org/10.1007/978-3-030-39512-4_111
Research on Disabled People’s Museum Visit Experience 727
the hardware barriers such as building facilities, but also the obstacles to the pursuit of
equal rights to education and free and independent living.
In China Knowledge Network, the keywords “museum” and “accessibility” were
interleaved and searched, and 415 articles were obtained. The main research topics are
“accessibility facilities”, “accessibility environment”, “accessibility design”, etc. The
search date is August 23, 2019.
In the comparative study of the barrier-free system of public buildings in Germany
and China, Yu Qinran found the disabled organizations in Germany have gradually
developed to provide paid counseling services for disabled people, relatives of disabled
people and the society [1]. The barrier-free construction of public buildings in China
began in the 2008 Beijing Olympic Games, which became the driving force for
stimulating urban infrastructure construction. For example, the Forbidden City built a
barrier-free route at the end of 2007. Visitors with physical disabilities can reach the
“Wumen Gate” through barrier-free lifts.
The digital display design in the museum is a new research hotspot in recent years.
In the interactive multimedia research of the Shanghai Museum, Gao Yuejun pointed
out that museums need to use multimodality interaction, diverse terminals to spread
knowledge. Through enhancing the experience of cultural activities, it can make
everyone enjoy culture and technology. From the technical perspective, Zhou Linlin
proposed the method of digitalization of the museum, which uses modern technology
to convert the physical object into a digital signal, and then converts the digital signal
into an image that can be recognized by people. This digital display breaks geo-
graphical restrictions, time constraints, etc., making the museum’s resources more
efficient [2].
The newly adopted museum law in Sweden proposes to provide an equal museum
experience for all visitors. Hudson conducted design tests at the Trelleborg Museum in
Sweden and found that the most important of the accessible navigation design methods is
to provide a tactile experience, a good audio tour and easy navigation for people with
disabilities [3]. Anna Kabjin Kim proposed a continuous momentum experience in the
museum. Due to the many dynamic factors in the museum, it is easy to break the con-
tinuous experience of the visitors, and the momentum state is difficult to maintain [4].
Accessible information tours have only appeared in the special events of a few
museums. The Shanghai World Expo Museum uses accessible reading technology to
allow blind people to get information. Since then, they launched the “Smart World
Expo Pavilion” in 2019, which has an auxiliary navigation function to solve the pain
points that blind people cannot identify information. However, the need for digital
interaction of patients with physical disabilities during the tour is ignored.
Museum Accessibility Solutions Focus on the Design of Aging Wheelchairs. The
barrier-free construction of developed countries has been relatively mature and a sound
legal system has been established. The technical committee “Technical aids for dis-
abled or handicapped people” was founded in 1978 and now has 29-member states and
25 observer countries. Since then, the UN General Assembly adopted the Convention
on the Rights of Persons with Disabilities and its selective agreements in 2006. The
Convention states that wheelchairs and other assistive devices can promote and protect
the equal human rights and freedom for all persons with disabilities.
728 S. Zhao and J. Shen
The most common barrier-free solution in museums is the manual wheelchair. In the
“Guidelines on the provision of manual wheelchairs” issued by the World Health
Organization, the necessary considerations for the design and production of wheelchair
products are emphasized: users, environment and materials.
Su Ting pointed out the defects of intelligent wheelchair in the research of intelligent
wheelchair control system, such as the difficulty of safety detecting the unknown
environmental, the single interface of human-computer interaction and the high cost of
intelligent wheelchair, and then proposed the solution strategy of modular production.
Further, Zhou Lei made predictions on the future development of intelligent wheel-
chairs, and proposed four design trends of intelligent wheelchairs - personalized control
of multiple postures, comprehensive sensor detection, product industrialization and
modular design, aims to bring smart wheelchairs to people with disabilities and enter
them into a new way of life.
3 Observation
In this study, a field survey was conducted with Musée du Louvre, Musée d’Orsay,
Musée de l’Orangerie, and Centre Pompidou in Paris, and the use of barrier-free
facilities in the pavilion was observed using the shadow tracking method.
3.1 Typical User Path
The user path map is drawn as follows based on the combined results of the follow-up
observations and interviews. It shows in Fig. 1.
Users will check the official website’s accessibility notice before going to the
museum to see if there is a priority channel or exclusive entrance for them. A security
check will be carried out upon arrival at the museum entrance.
Fig. 1. The typical user path map
Research on Disabled People’s Museum Visit Experience 729
If the user decides to use the wheelchair provided by the museum, they will ask the
staff about the location of the accessible information desk and go to the rental. The
lease is free, but a proof of disability of the user is required. If they bring their own
wheelchair, they can take the barrier-free elevator directly to the exhibition area.
Because they also use wheelchairs from home, this is the way more than half of the
users choose.
Exhibition is a complex process that involves a series of behaviors and actions,
such as finding barrier-free elevators, visiting the exhibition, watching the exhibit card
and communicating. These actions constitute a cyclic behavioral system.
Finally, the user can return the wheelchair or use his own wheelchair to leave the
museum.
3.2 Observation Summary
The Barrier-Free Environment is Relatively Complete. Thanks to the early con-
struction of barrier-free facilities in the European Museum, people with disabilities can
easily enjoy the same content as normal people in the museum. However, during the
observation process, it was found that the entrance of the barrier-free elevator was
relatively hidden and the number of elevators was small. Some of the low-rise stairs
used the lifting platform instead of the elevator. The lifting platform has experienced
two inoperable problems and needs to be resolved by the staff.
The Complicated Tour Route Makes the Family Exhausted. During the shadow
tracking process, it was found that the internal moving lines of the museum were
usually designed in the form of broken lines, which made the visiting route complicated
and long. Family members need to walk an average of 13,400 steps in the museum. Not
only that, but during this period, it is necessary to push the wheelchair forward, which
is very laborious. Therefore, users usually choose to visit only the most important
exhibitions to reduce the length of the route.
It is Difficult for Wheelchair Users to Communicate with Their Families. Even if
the family is only behind the user, the distance between them is still more than 60 cm
under normal conditions. The museum is usually a quiet atmosphere, so family
members generally choose to bend over and approach the wheelchair user for a con-
versation. At the same time, wheelchair users will try their best to turn around and
listen. But they did not notice this action themselves. After the interview, the family
members recalled that it was very difficult to communicate during the interview and
their waists would be uncomfortable.
They Watched the Exhibits for Different Lengths of Time. Everyone’s interest in
different exhibits is different. During the observation, 25 conversations about time
difference were found, including “Let’s go”, “Wait a minute”, “I don’t like it”, “Let me
see.” etc. Additionally, the items in the museum they observe in the same location are
also different.
A partial photograph recorded during the observation is shown in Fig. 2.
730 S. Zhao and J. Shen
Fig. 2. Photos during shadow tracking
The construction of barrier-free interactions in museums is comprehensive and
involves many elements such as technology, funds, people, organization, government
and products. It is difficult to do it efficiently by relying solely on the power of the
museum. On the contrary, it requires multiple actors to participate. However, different
actors have different ways of action and interest in the construction and application of
barrier-free interaction in museums. Translating these actors into a field is very valuable
for the enjoyment of public education rights and a better viewing experience for
physically disabled people.
4 Translation
Translation is an important way to form an actor network. Michel Callon divides the
translation processes into four steps: problematization, interessement, enrolment and
mobilization [5].
4.1 Problematization
In the stage of Problematicization, it is necessary to define the actors and clarify the
obligatory passage points in the network.
Table 1. The specific composition of actors.
Types Attributes Main actors
Human Individual actors Disabled, family members, museum staff, tour guides,
actors donors, artists, academics, etc.
Organizational Non-governmental disability organizations, public
actors culture and education departments, museums,
manufacturers, etc.
Non- Material category Technical equipment, funds, digital exhibition resources,
human actors museum guide resources, etc.
actors Consciousness Awareness of equal education, relevant policies and
category actors regulations, regional social culture, etc.
Research on Disabled People’s Museum Visit Experience 731
According to the general symmetry principle of the actor network theory, the actors
of the construction of barrier-free interactions in museums are divided into human
actors and non-human actors. Among them, human actors include individual actors and
organizational actors. Non-human actors include both material category actors and
consciousness category actors. The specific composition of each type of actor is shown
in Table 1.
Every participating actor will face different problems and obstacles; therefore, they
need to align their interests with other actors through cooperation. It is key to let actors
have a common goal through obligatory passage points, such as building an accessible
interactive platform, building an inclusive public education approach, and enriching the
exhibition experience for people with disabilities.
4.2 Interessement
This stage is to stabilize the actor in the set role. Core actors need to strengthen or
stabilize the identity of other actors through a series of actions, such as negotiation and
intervention. For example, communication in the real world can be carried out through
technology, politics and negotiation.
After finding the obligatory passage points, more relevant actors need to be
involved in this field to form an interest alliance. People with disabilities need to
participate in the system construction, put forward creative needs, contribute travel data
and give feedback. Disabled people’s organizations are established to serve the dis-
abled. Therefore, volunteer support, financial support, etc. are required. Artists and
scholars need to contribute their own knowledge and provide relevant support for
cultural services and digital exhibitions on the platform.
4.3 Enrolment
Core actor museum managers need to coordinate the interests of various actors,
motivate the enthusiasm and initiative of the actors, and maintain the stability of the
network.
People with disabilities will get a better museum experience and digital culture
education. Enjoy the right to equal education. There is even a chance to reduce the
unemployment rate. Disabled people’s organizations will be given relevant benefits,
such as organizational exposure, material replenishment and other policy support.
While art and scholars help provide educational resources, accessible interactive sys-
tems will also serve as a platform for them to spread creative culture and ideas.
4.4 Mobilization
This is a phase of maintenance that requires constant checks to see if the actors in the
network are fully representative of all similar actors, not biased.
There is a need to ensure that representatives of persons with disabilities are able to
represent typical disabled users and remain neutral, rather than those with financial
interests, such as being bribed by other organizations or individuals.
732 S. Zhao and J. Shen
5 Conclusion and Future Work
Through the museum’s barrier-free interactive system, high-quality cultural resources
in the city can be extended to disadvantaged groups. Based on the perspective of actor
network theory, it explores how to create a field for each actor through translation
methods so that they can face obligatory passage points together. Future research will
continue, and this stage is the initial stage of discovering problems and proposing
translation methods. Later work will continuously adjust the translation process, and
apply the translation results to actual production to establish a mobile assistant system
for the museum’s barrier-free interactive system.
References
1. Yu, Q.: Research on the barrier-free system of public buildings in Germany. Shenyang
Jianzhu University (2012)
2. Zhou, L.: Analysis of digital exhibition and display technology in museums. China Natl. Expo
2016(11), 212–213 (2016)
3. Hudson, A.: Letting the blind man see…: an investigation into methods and an evaluation to
improve accessibility in Swedish museum exhibits for the visually impaired (2018)
4. Kim, A.K., Harris, E.: Experiencing momentum through an effective use of technology in
museums. In: Shin, C. (ed.) Advances in Interdisciplinary Practice in Industrial Design, vol.
968. Springer, Cham (2019)
5. Callon, M.: The sociology of an actor-network: the case of the electric vehicle. In: Callon, M.,
Law, J., Rip, A. (eds.) Mapping the Dynamics of Science and Technology. Palgrave
Macmillan, London (1986)
Production Management Model to Balance
Assembly Lines Focused on Worker Autonomy
to Increase the Efficiency of Garment
Manufacturing
Valeria Sosa-Perez1, Jose Palomino-Moya1, Claudia Leon-Chavarril1,
Carlos Raymundo-Ibañez2(&), and Moises Perez3
1
Ingeniería Industrial, Universidad Peruana de Ciencias Aplicadas (UPC),
Lima 15023, Peru
{u201410875,u201320716,pcincleo}@upc.edu.pe
2
Dirección de Investigación, Universidad Peruana de Ciencias Aplicadas
(UPC), Lima 15023, Peru
[email protected]
3
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. Currently, companies dedicated to the garment manufacturing
industry have as a main problem the delay in their production assembly lines,
which generates high downtime. Given this problem, it is proposed to make a
correct allocation of resources by performing a balance of lines based of the
Hoffman model. This with the objective of increasing the production cycle time.
In the same way, define a work methodology, select good working practices and
develop autonomy in workers. As a result of this implementation, workers
acquire autonomy to perform their work, new working methods, and acquisition
of new knowledge. After the proposed improvements had been deployed, the
company reported an increase of over 25% in production assembly line quality,
performance, and efficiency.
Keywords: Assembly line balancing Empower employees Garment
manufacturing Hoffman method Value stream mapping
1 Introduction
In the last decade, the Peruvian economy reported an average annual growth of 4.3%
[1, 2]. One of the reasons for this weak growth is that local companies face different
problems that mainly affect their productivity, which in turn affects their competi-
tiveness. In this context, MSMEs of clothing manufacturing play an important lead-
ership role, since they represent one of the main non-extractive activities at the national
level, since they represent 1.3% of national GDP and 8.9% of manufacturing pro-
duction in 2014 [3]. It is for this reason that the apparel production sector has become
the second most important sector in manufacturing GDP, second only to the precious
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 733–739, 2020.
https://doi.org/10.1007/978-3-030-39512-4_112
734 V. Sosa-Perez et al.
and non-ferrous metals industry. However, although its production value grew by
approximately 14.8% between 2009 and 2014, the textile industry exhibited, from
January to August 2017, a cumulative negative growth rate of 6.5%, while in the last 12
months, the variation was negative, approximately 6.0% [1]. Therefore, the root causes
of this problem must be determined. It is for this reason that this study proposes an
improvement model to increase the productive efficiency of the company. In addition,
can also serve as an example for other apparel manufacturing companies towards better
production controls. For this, a line-balancing model was implemented to allocate
resources appropriately according to each work station, with adequate training to
achieve worker autonomy.
2 State of Art
2.1 Line Balancing in the Textile and Clothing Sector
According to past research, the Balance of lines based on the grouping of activities that
follow a sequence of work in a production plant, which in this case is a textile pro-
duction. All this, in order to achieve maximum use of resources such as labor and
equipment and thereby reduce or eliminate downtime. According to the research of the
authors Oksuz [4] and Türkmen [5], one of the main objectives is to obtain an adequate
balance in the balancing of the lines, eliminating the creation of possible bottlenecks
and delays in the production. For this purpose, several heuristic models used and
compared in order to create production, efficiency with the minimum use of machinery
and labor. Having as a result the comparison of two heuristic models, which provide
insight, implies the proper use of these models. Among the heuristic models are two
very relevant, this is the Moodie Young model and the Hoffman. These models are
looking to obtain the greatest efficiency in the balance of lines. While in Moodie
Young, the workstations grouped through a precedence diagram, the Hoffman model
additional to this has variables such as the use of machinery and a precedence matrix
[6]. In the same way, the authors Syahputri [7] and Kayar [8] in their investigations
make a comparison of the efficiency and loss of line balancing in a textile manufac-
turing company in Asia. This results in a difference in the efficiency of the line balance
in the Hoffman model with more than 15% unlike the Moodie Young model.
2.2 Worker Autonomy Applied to the Textile and Clothing Sector
Achieving autonomy in workers is a management strategy that allows employees to
develop their ability to make their own decisions competently [9, 10]. In fact, one of the
studies reviewed developed a visual management model based on manual resources,
signalling and experimental learning theory after implementing good work practices.
The model validated through its implementation in 653 employees working in textile
production companies. These findings suggest that creating a culture of organizational
learning significantly strengthens teamwork relationships and the autonomy of workers
to their daily activities.
Production Management Model to Balance Assembly Lines Focused 735
3 Contributions
3.1 Proposed Production Management Model
• Phase 1: Value stream mapping diagram shows the production flow of the company.
This tool helps to identify lead-time and shows times that add and do not add value.
• Phase 2: Line balancing to optimally allocate resources and balance production
lines.
• Phase 3: Work Standardization to establish the appropriate work methods.
• Phase 4: Continuous training means prepare staff for the immediate execution of the
various tasks of the position.
Each component added to this production management model not only fosters
production efficiency but also implementation sustainability. That is, it guarantees that
the implementation will remain operational over time (Fig. 1).
Fig. 1. Proposed production management model
3.2 Indicator of Production Line Losses and Efficiency
These indicators identify the state of production line flows to determine if there are
delays in the assembly lines.
h X i
LB ¼ nC Co =nC 100
ð1Þ
LE ¼ ð1 LBÞ 100
736 V. Sosa-Perez et al.
3.3 Proposed Method for Implementation of Improvement
This item explains the steps required for deploying the production management model
as explained below (Fig. 2).
Fig. 2. Proposed method for implementation of improvement
4 Validation
4.1 Description
The company under study manufactures and markets garments for babies, boys, and
girls. Located in Lima, Peru, the company was established in 2013, which evidences
that it is still a developing company (Fig. 3).
Fig. 3. Company facilities under study
Production Management Model to Balance Assembly Lines Focused 737
4.2 Assessment
As part of the evaluation, a production line was identified that generates higher sales
revenue for the company. For this case study, the production of pants was chosen.
Next, the Value Flow Mapping diagram was drawn, establishing that after a delivery
time of 13.5 days, it was found that a total of 6.5 days did not add value to the
production process. These days are associated with the sewing process, which gener-
ates frequent bottlenecks at this station. Finally, the main causes of the problem are
identified using 7 waste tools.
4.3 Implementation
Step 1: Before starting to develop the tools, all activities within the pants sewing
process must be identified, as well as the standard times of each activity (Fig. 4).
Back
Leather label BuƩonhole Zippers Pockets Front
Prepare sew front and
8 1 back piece
label 4
Prepare Sew front
zippers 2
pockets
Sew back
3 pockets
5 Sew zippers
Sew
6
buƩonholeer
7 Sew belt
Sew leather
9
label
Total
10 Reinforcement
10
Finished Pants
Fig. 4. OPC-Pants sewing process
Step 2: After this, the balance loss and efficiency are calculated in order to be
compared with results after implementation.
LB ¼ ð11 0:98 5:48Þ=ð11 0:98Þ 100 ¼ 49:17%
ð2Þ
LE ¼ ð1 0:4416Þ 100 ¼ 50:83%
738 V. Sosa-Perez et al.
Step 3: Activities with less standard time are grouped with activities that do not
exceed the larger standard time (Fig. 5). Achieving reduce from eleven stations to
just eight (Fig. 6).
OP 1 2 3 4 5 6 7 8 9 10 11
1 1 Activity Time Total Difference
2 1 1 1 0.7 0.7 0.28
3 1 2 2 0.5 0.7 0.28
4 1 10 0.2
5 1 3 0.2 0.7 0.28
3
6 1 6 0.5
7 1 4 0.4 0.7 0.28
4
8 1 8 0.3
9 1 5 5 0.6 0.6 0.38
10 1 6 7 0.6 0.6 0.38
11 7 9 0.5 0.5 0.48
0 1 1 0 2 1 1 0 2 1 1 8 11 0.98 0.98 0
Fig. 5. Grouping matrix Fig. 6. Activity matrix
Step 4: The efficiency of the new assembly line is calculated and compared with the
efficiency before implementation (Fig. 7).
Before After
Difference
improvement improvement
Line efficiency 50.83% 76.89% 26.06%
Fig. 7. Before and after results
• The Line balancing application manages to eliminate 2 activities that do not
generate value, allowing to create a continuous flow in production. Successfully
allocating resources and having an efficient workload. Which meant an increase
in the efficiency of the production line from 50.83% to 76.89%.
References
1. Ministry of Production: Economic Studies Offices (2015). http://demi.produce.gob.pe/
estadistica/sectorial
2. Clerk Maxwell, J.: A Treatise on Electricity and Magnetism, vol. 2, 3rd edn, pp. 68–73.
Clarendon, Oxford (1892)
Production Management Model to Balance Assembly Lines Focused 739
3. National Statistics Institute (INEI) (2015). https://www.inei.gob.pe/media/MenuRecursivo/
indices_tematicos/pbi_act_econ_n54_kte_2007-2015_1.xlsx
4. Oksuz, M.K., Buyukozkan, K., Satoglu, S.I.: U-shaped assembly line worker assignment
and balancing problem: a mathematical model and two meta-heuristics. Comput. Ind. Eng.
112, 246–263 (2017)
5. Türkmen, A., Yesil, Y., Rahman, M.: Heuristic production line balancing problem solution
with MATLAB software programming. Int. J. Cloth. Sci. Technol. 28(6), 750–779 (2016)
6. Zhang, L., Narkhede, B.E., Chaple, A.P.: Evaluating lean manufacturing barriers: an
interpretive process. J. Manuf. Technol. Manag. 28(8), 1086–1114 (2017)
7. Syahputri, K., Sari, R.M., Rizkya, I., Leviza, J., Siregar, I.: Improving assembly line
balancing using Moodie Young methods on dump truck production. In: IOP Conference
Series: Materials Science and Engineering, vol. 288, no. 1, p. 012090. IOP Publishing,
January 2018
8. Kayar, M., Akalin, M.: Comparing heuristic and simulation methods applied to the apparel
assembly line balancing problem. Fibres Text. East. Europe 2(116), 131–137 (2016)
9. Campos, R.Y., Lao, M.N., Torres, C., Quispe, G., Raymundo, C.: Modelo de Gestión del
conocimiento para mejorar la Productividad del Talento Humano en empresas del sector
manufactura. In: CICIC 2018 - Octava Conferencia Iberoamericana de Complejidad,
Informatica y Cibernetica, Memorias, 1, pp. 154–159 (2018)
10. Espinoza, A., Rojas, E., Rojas, J., Raymundo, C.: Methodology for reducing staff turnover in
service companies based on employer branding and talent management. In: Smart
Innovation, Systems and Technologies, vol. 140, pp. 575–583 (2019)
Rural Ecotourism Associative Model
to Optimize the Development of the High
Andean Tourism Sector in Peru
Oscar Galvez-Acevedo1, Jose Martinez-Castañon1,
Mercedes Cano-Lazarte1, Carlos Raymundo-Ibañez2(&),
and Moises Perez3
1
Ingeniería de Gestión Empresarial, Universidad Peruana de Ciencias Aplicadas
(UPC), Lima 15023, Peru
{u201212253,u201417205,mercedez.cano}@upc.edu.pe
2
Dirección de Investigación, Universidad Peruana de Ciencias Aplicadas
(UPC), Lima 15023, Peru
[email protected]
3
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. In Peru, tourism has had an average annual growth of 5% in the last
5 years, however, the Huancavelica region has had an average reduction of 2%
per year. Castrovirreyna, one of its districts, has 300 monthly arrivals on
average, while other locations with lower HDI (Human Development Index),
more tourism resources and higher per capita income, have at least twice as
many arrivals. This wastage contributes to the degradation and oblivion of the
potential tourist attractions and culture of the district. This research seeks local
progress from the development of innovation and professionalization of enter-
prises (SMEs) and entrepreneurs, with the proposal of guidelines for tourism
use. To achieve this purpose, it is proposed the implementation of an Asso-
ciative Ecotourism Model that manages to link the important agents of this local
economy to develop innovative, collaborative and sustainable tourism products,
services, packages and projects.
Keywords: Innovation Associativity Ecotourism Sustainability
Entrepreneurship Alto Andean SMEs
1 Introduction
Domestic tourism has been increasing linearly since 2012 in Peru, with the exception of
the Huancavelica region, which has had a decrease in tourism demand despite having
potential for the development of ecotourism, experiential and recreational tourism.
Castrovirreyna, Castrovirreyna district, is located at 3950 masl and has 3 tourist attrac-
tions recognized by the MINCETUR and other attractions that are not used for tourism; It
is also crossed by 2 economic corridors that transfer an average of 500 people a day. The
district has an average monthly arrival demand of 300 visitors, while other locations with
the same context or in worse conditions have a higher tourist demand.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 740–745, 2020.
https://doi.org/10.1007/978-3-030-39512-4_113
Rural Ecotourism Associative Model to Optimize the Development 741
A model for the sustainable development of products from different stakeholders in
mountain regions of Austria propose [1] a solution for the correct development of
synergies in the tourism sector, taking advantage of trends such as mountain biking in
conjunction with hunters, hikers and owners to reach a mutual agreement with all
parties involved in the search for the development of all. On the other hand, studies in
Peru, focused on the region with more flow of foreign visitors, Cuzco, introduce us in
dynamics of people in extreme poverty with the aim of taking advantage of tourism and
its relationship with local institutions [2]. However, the study and research in the
development of tourism management models for the use of natural resources in high
rural Andean towns is scarce. The objective of this research is to develop an Eco-
tourism Management Model that allows the sustainable use of tourism in high rural
Andean towns and that reduces poverty rates and increases the use of the environment.
2 State of Art
2.1 Models of Tourism Value Chain in Mountain Regions
The revised literature establishes guidelines for the type presented, in the first one, [3, 4]
are recognized as fundamental units of the tourism value chain in the rural context to
organizational companies travel, tour operators, transport companies and tourist ser-
vices; The guidelines focus on initial research to determine their situational diagnosis.
On the other hand, [5] they also establish as a final dependent variable and management
objective the development of authentic tourism products based on the use of the identity
characteristics of SMEs and the context of mountain, all based on the management of the
tourism value chain and political actors, which was validated by obtaining a high degree
of correlation in the positive development of the other independent variables. The
tourism value chain and other sectors present knowledge, infrastructure, policy, resource
and cultural interpretation gaps [6], the market approach is a common denominator in
these ventures and the development of this component is another key variable. These
companies can only be competitive if they optimize their resources and guide them
according to the trends identified in their users and potential users.
2.2 Ecotourism Models of Tourism Value Chain Management
in Mountain Regions
Mountain destinations present their potential in most of the ecotourism resources
present in the area (lakes, rivers, snowy and wildlife), the approaches to care and
development of social, environmental and economic environments, as well as the
correct development of a technological environment without affecting the aforemen-
tioned environments, are essential in these contexts [7]. Not only environmental con-
servation is relevant, but conservation based on the participation, assessment, opinions
and perceptions of stakeholders, residents and government [8]. This is resolved by the
development of the assessment of the identity of the residents, a development of
awareness and care of the destination by tourists and an innovation management.
742 O. Galvez-Acevedo et al.
2.3 Associative Ecotourism Models of Tourism Value Chain in Mountain
Regions
Ecotourism in rural locations is a trend issue related to globalization since the 1990s,
which demands greater needs and more detailed requirements, such as green areas, zero
pollution, open spaces, among others; In the case of high Andean rural towns, the
natural tranquility that is very unlikely to be found in urban areas such as cities and
large towns is expected. The entrepreneurs of these rural locations seek to be more
competitive through the exchange of information and other resources, for which they
tend to link informally with each other; therefore, a formalized alliance of tourism
entrepreneurs could grant greater and more competitive advantages [9].
3 Methodology
The model is based on the evaluation of the economic, social, environmental and
political context of the high Andean towns. Small transport companies, lodgings and
restaurants are recognized as causative actors of the local economic dynamics. These
are associated, under a voluntary and collaborative approach, and are managed from 5
components: Processes, sustainability, Innovative Leadership, Organizational Culture
and Business Development. These approaches contribute to the progress of associa-
tivity all together. Organizational Culture is recognized as one of the main elements to
Fig. 1. Associative Ecotourism Model of tourist value chains of high Andean towns in Perux
Rural Ecotourism Associative Model to Optimize the Development 743
face in rural contexts with high resistance to change and special roots in traditional
working methods. The ultimate goal is the formalization, quality improvement, and
creation of innovative products and services, in these remote rural locations, for the
facilitation and optimization of tourism use taking into account the sustainability
approach. See Fig. 1.
3.1 Organizational Culture
It seeks to modify the behavior patterns and ways of thinking of a human group in
order to guide them to more fertile patterns for the development and implementation of
new methodologies or approaches. This follows the fulfillment of a cycle, whose initial
step is the analysis of the environment in which a human group or organization
develops; then, it continues with the definition of strategies to face the environment in
search of the fulfillment of objectives; then, it continues with the development of a
culture that develops the vision of the strategies proposed and finally the results are
communicated to promote the improvement and adherence of more actors. The
application starts from a Diagnostic which follows the phases (13). Based on the
context, strategies are defined in order to develop competitive advantages such as
locality and enterprises.
3.2 Business Development
It maps the need for knowledge of adequate Business Management oriented to SMEs.
Taking into account the difficulties of the human group to intervene, this component
seeks to learn 3 basic items for the development of small businesses: Commerce and
Marketing, Finance and Human Resources. The first includes the creation of customer
value, price strategy, business model and tools. The second includes a company’s
value, balance sheet, income statement, financial efficiency indicators and investment
decisions. Finally, the selection of personnel, functions and coordination, goal evalu-
ation, staff training, payments and business identity is contemplated; In this way, each
SME is strengthened as an association and individually.
4 Results
In order to validate the proposed model, a case study is developed in the town of
Castrovirreyna, Huancavelica region, located at 3900 m above sea level. This account
has an average daily number of 100 people that cross this town through the
Huancavelica-Ica economic corridor, also presents an average of 300 monthly arrivals
related to tourism, has 3 ecotourism resources recognized by MINCETUR and another
5 awaiting recognition; however, it does not offer any product or service focused on the
exploitation of ecotourism, only 50% of the SMEs of the tourism chain have operating
licenses and formal registrations; In addition, the services and products of these only
have a satisfaction rating of 2 on a scale of 1 to 5, assessed by an initial survey of 350
people; Additionally, an evaluation of the organizational culture of the SMEs with a
variation of the OCI tool, determined that these present “Defensive Passive” and
744 O. Galvez-Acevedo et al.
“Defensive Aggressive” styles, which represent a negative culture for the development
of creativity, innovation, formalization and creation of quality products; finally, the
promotion of tourism resources is zero.
Once the case study is described, a pilot test is proposed with the purpose of
implementing the model and validating 3 of the 5 total components, this test has a
duration of 8 months, a budget of $ 664 and only develops the first 3 phases.
Three evaluations of the organizational culture were carried out, at the end of the
eighth month the third test was executed to evaluate the impact on the local culture, in
this it can be seen that the constructive style presents an average score increase of 1.0
points, the Defensive Liability a reduction of 0.6 points and Aggressive of 0.4, all
compared to the first evaluation. See Fig. 2.
Fig. 2. Third evaluation of Organizational Culture
This implies an increase in the constructive and unique style that we consider
positive, after 8 months of implementation, with a budget execution of 70%, 8 field
visits and 2 specialists in charge of the analysis, the following results are shown.
At the same time, a second performance evaluation focused on the leader and
president of the association was carried out after 6 months in the position, this show
passing grades in the 5 competences evaluated. Both organizational culture and lead-
ership show a direct relationship with the proper functioning of the association.
5 Conclusions
The impact on the increase of the constructive cultural style of 1.0 point and the
reduction of the passive and aggressive styles by the development of the Organizational
Culture Model is fundamental in the paving of the land for the implementation of new
work methodologies.
Leadership Development based on the “Communication” competencies with a
score of 4.31, Leadership with a score of 4.03, Teamwork with a score of 4-05,
Responsibilities and quality with 4.00 and Innovation 4.3, directly favored to
encourage innovation within the association and within each MYPE, this shows a direct
relationship with the change in organizational culture and the number of innovative
products and services.
Rural Ecotourism Associative Model to Optimize the Development 745
The 12 processes implemented, with their respective indicators and the process
approach contributed to the consolidation of the collaborative culture of the association,
was also the starting point for the start of volunteer work aimed at improving and
signaling trekking routes.
The development of the prototype, testing and launch of 4 local tourism products
was achieved. These have already generated an economic impact equivalent to 1700
dollars, only with the direct intervention of 3 SMEs, 8 people involved and 4 days of
work, which strengthens the local cultural change.
Although the Implementation of the “Business Development” component is still
under implementation, there is already an interest of the associates for the profes-
sionalization of their positions within their SMEs. Strengthening the orientation
approach towards the formalization of local businesses.
It can be concluded by the validation of the components “Organizational Culture”,
“Process Management” and “Innovative Leadership” and its direct relationship that the
case study has been positively impacted in its development by the implementation of
the model and verified both through economic indicators and social indicators and
meets the proposed objectives of quality improvement, formalization and development
of innovative tourism products and services.
References
1. Pröbstl-Haider, U., Lund-Durlacher, D., Antonschmidt, H., Hödl, C.: Montaña Del Turismo
En Bicicleta En Austria Y La Región De Los Alpes - Hacia Un Modelo (2017)
2. Knight, D.W.: An institutional analysis of local strategies for enhancing pro-poor tourism
outcomes in Cuzco, Peru. J. Sustain. Tour. 26, 631–648 (2017). Sostenible Para El Desarrollo
De Productos De Múltiples Partes Interesadas
3. Hjalager, A.-M., Tervo-Kankare, K., Tuohino, A.: Cadenas De Valor Turismo. Revisados Y
Aplicados A Bienestar Rural Turismo, Planificación Y Desarrollo, Turismo (2016)
4. Ramírez, R.C., Plascencia, J.M.O.: Modelo De Cadena Productiva Para La Gestión Del
Turismo En Contextos Rurales, Una Propuesta Desde La Economía Territorial. Caso De
Aplicación En Comala, Colima (2016)
5. Aguilar, Y., Gordillo, A., Quispe, G., Dominguez, F., Moguerza, J.M., et al.: Tourism cluster
implementation model in small cities of developing countries. In: CISCI 2018 - Decima
Septima Conferencia Iberoamericana en Sistemas, Cibernetica e Informatica, Decimo Quinto
Simposium Iberoamericano en Educacion, Cibernetica e Informatica, SIECI 2018 –
Memorias, vol. 1, pp. 26–31 (2018)
6. Zapata, G., Murga, J., Raymundo, C., Dominguez, F., Moguerza, J.M., Alvarez, J.M.:
Business information architecture for successful project implementation based on sentiment
analysis in the tourist sector. J. Intell. Inf. Syst. 53, 563–585 (2019)
7. Hjalager, A.-M., Kwiatkowski, G., Larsen, M.Ø.: Brechas De Innovación En Turismo Rural
Escandinavo. Scandinavian Journal De Hostelería Y Turismo (2017)
8. Junio, K.S.: Las Relaciones Estructurales De La Imagen De Destino, La Conciencia, La
Singularidad Y La Lealtad De Destino En Destino Ecoturístico Periurbana (2016)
9. Grimstad, S., Burgess, J.: Sostenibilidad Ambiental Y Ventaja Competitiva En Un Micro-
Cluster Vitivinícola, vol. 37, no. 6, pp. 553–573. https://doi.org/10.1108/mrr-01-2013-0019
Picking Management Model with a Focus
on Change Management to Reduce
the Deterioration of Finished Products in Mass
Consumption Distribution Centers
Lourdes Canales-Ramos1, Arelis Velasquez-Vargas1,
Pedro Chavez-Soriano1, Carlos Raymundo-Ibañez1(&),
and Moises Perez2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima 15023, Peru
{u201411063,u201412345,pedro.chavez,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. The process of assembling an order or picking in distribution centers
is manual in most cases. Past studies have shown the relationship between the
performance of the operator and a work method designed, based on ergonomics
rules. However, the management of changes when implementing a picking
management model has not been evidenced until now. Therefore, this research
study proposes a picking management model that aligns the elements of change
management with those of a work-study and a validation method with five
phases that will deal with the whole picking process. Validation presents a case
study where the most relevant research result is the reduction of the damaged
inventory from 4% to 1.5% and other evaluation indicators. Finally, conclusions
of result analysis based on the validation of the model within a distribution
center are presented.
Keywords: Order picking model Change management Systems
engineering Warehouse Inventory management Ergonomics Manual
picking
1 Introduction
By 2016, logistics costs represented 34% of the total cost of sales in Peru [1]. With this
index, the country is placed 10% above the average in Latin America. Consequently,
experts in supply chain management began to investigate the causes of logistics costs in
the country. Based on their research, they found that 30% originated in the distribution
centers, and as part of this 30%, 60% was represented by the cost of labor. After further
research, it was detected that 42% of the workforce was devoted to picking operations
[2]. This means that the main cause of the high logistics costs in the country is manual
picking operations in distribution centers. The causes identified by the authors are
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 746–752, 2020.
https://doi.org/10.1007/978-3-030-39512-4_114
Picking Management Model with a Focus on Change Management 747
worker exhaustion [3] and the research gap in technology to replace labor that can
convert the picking operation into an automatic system [2].
The importance is noteworthy because of the consequences of high logistics costs.
The logistic performance index in Peru is directly affected by this factor. To improve
the quality of the services, an increase of productivity in the processes of warehouses
and an improvement in the handling of inventories are sought. In countries such as
India, advances in productivity and the creation of new manual picking methods have
helped increase competitiveness [4, 5]. Different authors in the world have evaluated
the problem in case studies of warehouses, identifying opportunities for improvement
by focusing on the study of work.
The motivation for this research study is to create a picking management model that
is focused on change management. This approach will allow us to reduce the damaged
inventory in the picking process of distribution centers and to involve the staff in
improvements, by raising awareness on the importance of this issue, proposing solu-
tions, making final measurement of performance, and creating a culture of continuous
improvement in the rest of the operations in an equal manner. The model of picking
management based on work techniques study process to be designed will include
procedures to enable the diagnosis of the picking process in its current situation,
eliminate repetitive activities, redesign the workplace on the basis of ergonomic rules,
hold training events and carry out audits to guarantee their continuity. Indicators will
also be presented to measure the performance of the proposal.
2 Review of Literature
2.1 Picking Management Models
Over time, picking management models have evolved to obtain solutions that benefit
distribution centers to the greatest extent.
Several authors proposed models that were based on the management of the picking
process, including the study of methods as the main tool [4, 5].
Other authors mentioned a model based on the reduction of logistics costs through
the elimination of losses. The tool used was value stream mapping for the redesign of
the operational layout [6].
Two years later, they built a model based on human aspects. Among human factors,
job satisfaction, as well as ergonomics in the picking process, was measured [3]. The
main results of the different models indicate that productivity can be increased by more
than 15% [4]. However, this could also mean a reduction of 4% of logistics costs [6].
2.2 Change Management
In distribution centers where members have been working for more than 5 years and
have an age range between 30 and 45 years, it is important to maintain the change
management approach to improve the process that they were carrying out [3, 7].
Several authors mention the importance of considering what the staff thinks about
the proposal, as well as understanding the impact of the problem. Likewise, the
748 L. Canales-Ramos et al.
inclusion of human factors must be considered in order to promote the comfort and
satisfaction of workers [8].
With respect to human factors, workload balancing as a technique for not requiring
employees to work beyond their limits by proposing different methods that should be
applied by supervisors. It is necessary to use the workload index to plan the amount of
resources efficiently and without harming employees [7].
Finally, authors also mention the need to reward employees for doing their work in
the standardized way and when they stand out from fellow workers. To this end, they
propose a monthly recognition program non-monetary assigning different indicator for
the optimization of the process [8].
Among the main results obtained with change management was the decrease in
35% staff turnover [7]. On the other hand, an increase in employee satisfaction of more
than 40% was achieved when compared with the average of 20% [8].
3 Contribution
3.1 Proposed Model
The picking management model (Fig. 1) has a change management approach as part of
the strategy and will support the management of the work method, the workplace, the
personnel, and the performance of the picking process.
Fig. 1. Picking management model with a focus on change management
Picking Management Model with a Focus on Change Management 749
3.2 Proposed Method
To implement the model, there are five phases (Fig. 2), which are aligned to the
elements of the work study and change management.
Fig. 2. Proposed method to apply the model
3.3 Model Indicators
As part of the implementation of the model, the results obtained must be organized to
evaluate the performance of picking management. The definition of indicators is part of
the performance and impact management. They are both components of the picking
management model (see Fig. 1). The proposed indicators are as follows:
• Percentage of damaged products and value added: The purpose of the indicator is to
measure the level of damaged inventory. The objective is 1%. On the other hand,
the goal of value added is 80%.
• Efficiency: The indicator focuses on measuring the efficiency of the picking process
by means of the number of boxes collected in the target time.
• OWAS rating: Measures the ergonomic risk in the postures of pickers during
operations. A reduction of this index to 2 is proposed.
• Percentage of Overtime cost: The objective is for overtime cost is 10%.
750 L. Canales-Ramos et al.
4 Validation
4.1 Case Study
The scenario for the execution of the proposal is aimed at distribution centers for mass
consumption in the district of Cercado de Lima, in the city of Lima, Peru. The staff
includes 26 men and 14 women, and the age ranges from 35 to 55 years (Table 1).
Table 1. Initial values of the model.
Indicator Initial value Target value
% of dmg products 4% 1%
% of value added 65% 80%
Efficiency 0.8 1.1
Time 255 min 190 min
% of overtime 22% 10%
OWAS rating 4 2
4.2 Application of the Contribution
Phase 0: Change Management. A session was held with the personnel involved in
the process. The diagnosis, the proposal, and the impact were presented. Two types of
surveys were applied. The change management survey showed type “A” questions
whose score was greater than 40, which means that the picker felt exposed to great
physical effort in the operation. In the case of type “B” questions, the score does not
exceed the established value. This means that pickers feel that they are able to do the
job. However, they are not sure of their ability to improve the current operation.
The second survey showed whether the session was able to raise awareness among
the pickers. These are the results: 69% said that the process could be carried out in a
shorter time (some gave suggestions on how to do it), 54% said that they did not have
enough strength to carry out their work and finally, 100% replied that they agreed with
the picking management model and also offered some suggestions related to com-
munication, tolerance, and teamwork. The information was used to reinforce the
phases. At the end of the session, the responsibilities of the supervisor in each of the
phases were defined so that they could be involved in the proposal.
Phase 1: Definition of the Work Methodology
Description of the Work Procedure. The matrix of value-added activities, better
known as the VAA matrix was made. The follow-up of 60 picking lists, divided into
three work shifts, was carried out. Men and women were considered. As can be
seen, an increase in the index of value added from 65% to 91% was obtained due to
the elimination of activities which were not needed nor did they add value to the
process of order picking.
Picking Management Model with a Focus on Change Management 751
Integration of Ergonomic Factors. The ergonomic guidelines were included in the
new picking procedure, and they specifically mentioned the postures required for
loading and unloading boxes on the order pallets. The following four ergonomics
activities were established: risk management, training in good practices, active
breaks, and audits.
Definition of Working Time. The optimal working time was obtained with the
MOST tool using variables such as: product weight (30 kg), picker height (1.75 m),
height of the location (1.4 m), and final height (0.3 m). This time considers four
time demanding activities: moving equipment to a location, box loading, lowering
of boxes and moving from the equipment to a location.
5 Results and Conclusions
The percentage of damaged products was reduced to 2% and the percentage of over-
time decreased by 16% this due to other associated causes that are beyond the reach of
the model, such as the intervention of other types of equipment in the area and the
width of the corridors. The combination of work study techniques with a change
management approach produced favourable results for the case study which allowed
improvement in the work environment as employees said they felt involved and valued
(Table 2).
Table 2. Final results of a model
Indicator Initial value Target value Final value
% of damaged products 4% 1% 2%
% of value added 65% 80% 91%
Efficiency 0.8 1 1.1
Time 255 min 190 min 115.79/130.89 min
% of overtime 22% 10% 16%
OWAS rating 4 1 2
References
1. Carlos Paz, J.: Sustentación ante la Comisión de Comercio Exterior Competencias para
Plataformas Logísticas, Lima (2017)
2. Mayorga, H.: http://logistica360.pe/los-costos-logisticos-en-la-cadena-de-suministro-en-el-
peru-y-como-reducirlos-usando-la-automatizacion/
3. Harari, Y., Riemer, R., Bechar, A.: Factors determining workers’ pace while conducting
continuous sequential lifting, carrying, and lowering tasks. Appl. Ergon. 67, 61–70 (2018)
4. Singh, M.P., Yadav, H.: Improvement in process industries by using work study methods : a
case study. Int. J. Mech. Eng. Technol. 7(3), 426–436 (2016)
5. Biswas, S., Chakraborty, A., Bhowmik, N.: Improving productivity using work study
technique. Int. J. Res. Eng. Appl. Sci. 6(11), 49–55 (2016)
752 L. Canales-Ramos et al.
6. Malashree, P., Sahebagowda, M., Gaitonde, V.N., Kulkarni, V.N.: An experimental study on
productivity improvement using workstudy and ergonomics. Int. J. Darshan Inst. Eng. Res.
Emerg. Technol. 7(1), 31 (2018)
7. Riquero, I., Hilario, C., Chavez, P., Raymundo, C.: Improvement proposal for the logistics
process of importing SMEs in Peru through lean, inventories, and change management. In:
Iano, Y., Arthur, R., Saotome, O., Vieira, E.V., Loschi, H. (eds.) Smart Innovation, Systems
and Technologies, vol. 140, pp. 495–501. Springer, Cham (2019)
8. Campos, R.Y., Lao, M.N., Torres, C., Quispe, G., Raymundo, C.: Knowledge management
model to improve the productivity of human talent in companies of the manufacturing sector.
In: CICIC 2018 - Octava Conferencia Iberoamericana de Complejidad, Informatica y
Cibernetica, Memorias, vol. 1, pp. 154–159 (2018)
Risk Factors Associated with Work-Related
Low Back Pain Among Home-Based Garment
Workers
Sunisa Chaiklieng1,2(&), Pornnapa Suggaravetsiri3,
and Sari Andajani4
1
Department of Occupational Health and Safety, Faculty of Public Health,
Khon Kaen University, Khon Kaen, Thailand
[email protected]
2
Research Center in Back, Neck, Other Joint Pain and Human Performance
(BNOJPH), Khon Kaen University, Khon Kaen, Thailand
3
Department of Epidemiology and Biostatistics, Faculty of Public Health,
Khon Kaen University, Khon Kaen, Thailand
4
Faculty of Health Science and Environment,
Auckland University of Technology, Auckland, New Zealand
Abstract. This cross-sectional study investigated the prevalence of work-
related low back pain (LBP) and its associated risk factors among 446 home-
based informal garment workers in the northeast of Thailand. The results from
structured interviews indicated the six month-prevalence of LBP was 44.39%
(95% CI = 39.67–49.02). The significant risk factors of LBP were included no
ordinary exercise (ORadj = 1.6; 95% CI = 1.01–2.52), members of family had
LBP (ORadj = 2.19; 95% CI = 1.90–2.37), working hour 8 h per day
(ORadj = 1.6; 95% CI = 1.98–2.61), repetitive movement (ORadj = 1.84; 95%
CI = 1.12–2.99), prolonged sitting >2 h (ORadj = 2.65; 95% CI = 1.62–4.34)
and no change of posture each hour (ORadj = 1.97; 95% CI = 1.18–3.30). It is
concluded that no regular exercise, nature of repetitive work and prolonged
sitting, and behavior of unchanged posture each hour are contributed risk factors
to LBP. In order to prevent LBP in this home-based worker, ergonomics training
and worker’s health promotion should be routinely implemented.
Keywords: Back pain Ergonomics Informal garment workers Prevalence
1 Introduction
Thailand is an industrially developing country where 55.32% of the majority of the
workforce comprises 31.18 million informal sector workers, mostly located in the
northeast of the country (19.9 million, 35.91%) compared to the middle 12.6 million,
the north 11.5 million and the south 7.4 million, respectively [1]. Textiles/garments
comprises are the major producibility by nearly 6.1 million informal workers, mostly
operating as small-scale enterprises or as home-based work with 7.5% of a total of
product from the northeast of Thailand [1]. The work activities of the garment workers
involve the adoption of a single or combined intensively static postures, prolonged
sitting, and repetitive work resulting in muscular discomfort [2].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 753–759, 2020.
https://doi.org/10.1007/978-3-030-39512-4_115
754 S. Chaiklieng et al.
The work-related musculoskeletal disorders (WMSDs) are the most reported health
problem and a serious health condition among Thai workforces, involving a combi-
nation of painful disorders of muscles, tendons, and nerves induced by work activities.
The highest prevalence of MSDs among industrial textile/garment workers in Thailand
has been low back pain (30.7%) [3]. Among informal garment workers from a case
study in Khon Kaen province, the Northeast region (NE) of Thailand revealed that low
back pain occupied as the first place among MSDs [4].
Our previous studies suggest that individual characteristics and working charac-
teristics played a role in low back pain. Primary studies conducted with different
provinces in Northeast Thailand among informal garment workers have contributed to
current knowledge of the influence of inappropriate workstations (seat and table) and
work environment conditions to shoulders pain [5] and discomfort [2]. Studies on low
back pain among informal garment workers in NE Thailand are limited. Only a case
study in one district of Khon Kaen province of Thailand had conducted on the rep-
resentative prevalence of back pain [4].
Loss of working time due to sickness absence of Thai garment workers in the
factory [3] and cost of long term treatment have affected the income of those workers
and small businesses as well as a gross national product [6]. Home-based garment
worker, however, is not visible as they work in their own home, and there is still a rare
study conducted with informal home-based garment workers. This study aimed to fill in
the gap in research involving home-based garment workers by examining the preva-
lence of low back pain and its associated risk factors among home-based garment
workers in NE Thailand.
2 Materials and Methods
2.1 Study Area and Subjects
This study was designed as a cross-sectional study to examine the prevalence of LBP
and to investigate the associated risk factors of LBP among the home-based garment
workers in NE Thailand. The sample size’s calculation and inclusion criteria were
regarding the previous report on shoulder pain and its risk factors among the informal
garment workers in NE Thailand of our previous study [5]. There were 446 home-
based garment workers who met inclusion criteria, and all subjects gave informed
consent before entering the study. The prior ethics approval number for the study
obtained from Khon Kaen University ethics committee, Thailand, was no. HE542131.
2.2 Data Collection and Analysis
Data were collected by face-to-face interviews with the structured questionnaires used
in the previous study [5]. There were four parts of the questionnaire required infor-
mation for this study: part 1 was demographic characteristics and work experience; part
2 was about the health status, health behaviors, and history of low back pain amongst
Risk Factors Associated with Work-Related Low Back Pain 755
family members. Part 3 enquired work characteristics and work behaviors, and
workstation (seat and table). Part 4 was the perception of low back pain adjusted by
self-assessment from the subjects of that onset of LBP symptom occurred in working
time during the past seven days or the six months up to data collection. If the pain was
lasting longer than period of work time or persisting over the period of 24 h at least, it
was also accounted for work-related LBP.
Data were analysed by using STATA version 10. Descriptive statistics were used to
describe the characteristics of workers, work conditions and work behaviors. Preva-
lence of work-related low back pain = (number of LBP cases during the past 7-days or
the past 6-months) 100)/446. A simple logistic regression analysis examined the risk
factors for the past 6-months of LBP. Factors with p-value less than 0.20 were selected
to be candidate variables in a multiple logistic regression analysis. The confounding
factors, which were age, gender and work experience, were always included in the
models. Significant risk factors were screened out in a backward stepwise manner using
likelihood ratio tests as selection criteria. The odds ratio (OR) and the adjusted odds
ratio (ORadj) with 95% confident interval (95% CI) were presented and p-value <0.05
indicated the statistics significance of the associated risk factors with LBP.
3 Results
3.1 The Prevalence of Work-Related Low Back Pain
Among 446 home based workers, the prevalence of work-related LBP of the last seven
prior data collection was 41.48% (95% CI = 36.89–46.07). The six months’ prevalence
of work-related LBP was 44.39% (95% CI = 39.67–49.02), calculated from 198 LBP
cases among 446 workers. The onset of LBP during the working period was a com-
plaint from 172 cases, whilst a reported symptom lasting to work off-time was indicated
from 26 cases.
There were the majority of female workers (94.44%) in this study, and the sig-
nificantly higher prevalence of LBP was identified in males compared to females. For
health behaviors, only 32.29% of workers had a regular exercise (n = 144), and the
workers of irregular exercise or no exercise group had a higher prevalence of LBP
compared to those who had exercised.
Concerning work characteristics, working hours averaged 8.64 h (S.D. = 3.23).
Sixty-eight percent of workers worked eight or more hours a day (95% CI = 63.82–
72.50, n = 304). Seventy percent of workers had repetitive movement while working
(95% CI = 64.98–73.58, n = 309). Working hours, repetitive movement, prolonged
sitting, change of posture each hour during sewing showed a significant correlation
with the prevalence of LBP (see Table 1).
756 S. Chaiklieng et al.
Table 1. Home-based garment workers’ characteristics and the prevalence of LBP
Variables/factors Total LBP cases; n (%) no LBP; n (%) P-value
(n = 446) (n = 198) (n = 248)
Gender
Male 23 15 (65.22) 8(34.78) <0.039*
Female 423 183 (46.26) 240(56.74)
Age group (year)
<25 11 4 (36.36) 4 (63.64) 0.457
25–29 37 15 (40.54) 22 (59.46)
30–34 84 37 (44.05) 47 (55.95)
35–39 153 75 (49.02) 78 (50.98)
40–44 87 41 (47.13) 46 (52.87)
>44 74 26 (35.14) 48 (64.86)
Work year experience
<10 322 98 (44.75) 121 (55.25) 0.901
10–14 122 43 (45.74) 51 (54.26)
>14 2 57 (42.86) 76 (57.14)
Exercise
Irregular 179 65 (45.14) 79 (54.86) 0.010*
Regular 144 66 (36.87) 113 (63.13)
No exercise 123 67 (54.47) 56 (45.53)
Members had LBP
No 324 123(37.96) 201 (62.04) <0.001*
Yes 122 75(61.48) 47 (38.52)
Working hour per day
<8 142 52 (36.62) 90 (63.38) <0.001*
8–12 261 137 (52.49) 124 (47.51)
>12 43 9 (20.93) 34 (79.07)
Repetitive movement
No 137 108 (78.83) 29 (21.17) <0.001*
Yes 309 161 (52.10) 148 (47.90)
Prolonged sitting >2 h
No 337 124 (36.80) 213 (63.20) <0.001*
Yes 109 74 (67.89) 35 (32.11)
Change of posture each hour
No 255 100 (52.36) 91 (47.64) 0.003*
Yes 191 98 (38.43) 157 (61.57)
Seat with backrest
No 208 100 (48.08) 108 (51.92) 0.143
Yes 238 98 (41.18) 140 (58.82)
* Significant at p-value <0.05
3.2 Risk Factors Correlated with Work-Related Low Back Pain
The six-month prevalence of LBP was selected to identify the correlated risk factors.
Univariate analysis indicated that no regular exercise, other members of the family
Risk Factors Associated with Work-Related Low Back Pain 757
having LBP, working eight hours or more per day, had repetitive movement during
work, prolonged sitting more than 2 h until work finished, and no change of posture
each hour (see Table 2).
Table 2. The correlation between risk factors and LBP among home-based garment workers by
univariate analysis
Factors LBP cases/ni % LBP OR 95% CI p-value
Male 15/23 65.22 2.46 1.02–5.92 0.045*
Age <40 years old 131/285 45.92 1.19 0.81–1.76 0.375
Work experience <10 years 98/219 44.75 1.03 0.71–1.49 0.882
No regular exercise 67/123 54.47 1.75 1.15–2.66 0.009*
Working 8 h per day 146/304 48.03 1.60 1.06–2.41 0.024*
Member of family had LBP 75/122 61.48 2.61 1.70–4.00 <0.001*
Repetitive movement 160/309 51.78 2.80 1.81–4.32 <0.001*
Prolonged sitting >2 h 74/109 67.89 3.63 2.29–5.74 <0.001*
No change of posture each hour 100/191 52.36 1.76 1.20–2.57 0.004*
Seat without backrest 98/238 41.18 1.25 0.77–2.03 0.376
* Significant at p-value <0.05, ni = a total number of subgroup
The significant factors (p-value <0.20) from the univariate analysis were entered
into the multiple logistic regression analysis models. After controlling the confounders
of gender, age, and working experience, risk occurrence was a non-statistically sig-
nificant difference by multiple logistic regressions between male and female home-
based garment workers, as shown in Table 3.
Table 3. The correlation between potential risk factors and LBP among home-based garment
workers by multiple logistic regression analysis
Risk factors OR ORadj 95% CI p-value
Age <40 years old 1.19 0.80 0.52–1.23 0.311
Male workers 2.46 1.62 0.63–4.17 0.317
Work experience <10 years 1.03 0.99 0.66–1.49 0.977
Member of family had LBP 2.61 2.19 1.90–2.37 0.001*
Repetitive working 2.80 1.84 1.12–2.99 0.015*
Prolonged sitting >2 h 2.63 2.65 1.62–4.34 <0.001*
Working hour 8 h per day 1.60 1.60 1.98–2.61 0.024*
No change of posture each hour 1.76 1.97 1.18–3.30 0.010*
No regular exercise 1.75 1.60 1.01–2.52 0.044*
* Significant at p-value < 0.05
758 S. Chaiklieng et al.
According to personal factors and behavior, workers with no ordinary exercise
(3 times a week and 30 min at least each time) had risk 1.60 times higher than those
with ordinary exercise (p-value = 0.044). Workers who had members of the family had
LBP were at a higher risk occurrence when compared to workers without members
who had LBP (ORadj = 2.19, p-value <0.001). Regarding nature of work, repetitive
working significantly correlated with LBP (ORadj = 1.84, p-value = 0.015) and
workers worked eight hours per day or more had a higher risk of developing LBP
compared to those worked less than eight working hours per day (ORadj = 1.60, p-
value = 0.024).
Interestingly, working behaviors were significantly involved in the risk of LBP
development. Prolonged sitting until work finished more than two hours increased the
risk for LBP (ORadj = 2.65, p-value <0.001). Workers who had no change of posture
each hour were at a higher risk occurrence when compared to workers with changes of
posture (ORadj = 1.97, p-value = 0.010).
4 Discussion
There was a high prevalence of work-related LBP in Thai home-based garment workers
found in this study (44.39%). The finding is consistent with previous studies identi-
fying that low back pain was a significant health problem among garment workers in
factories [3]. This statement was supported by Kampong and Chaiklieng [4] that low
back pain was most prevalent among musculoskeletal disorders in informal garment
workers. However, the 6-month prevalence of low back pain was likely two times
higher than the 6-month incidence of LBP in informal garment workers (22.65%)
found from a cohort study in Udon Thani, a province in the NE Thailand [7]. The
higher prevalence rate of LBP in comparison to the accumulated incidence of the
followed-up cases of garment workers might be suggested to longer surveillance of
new cases of LBP accounting after a more extended period in a prospective cohort
study in order to identify the risk factors of LBP.
Low back pain is influenced by personal behavior like regular exercise. This study
confirms a previous study conducted with school teachers in Thailand. Regular exercise
could decrease the risk of repetitive strain injuries and LBP [8]. Therefore, lack of
regular exercise put home-based workers at a significantly higher risk for LBP than
those who had regular exercise, which was previously indicated in the informal gar-
ment female workers [2]. Only one third of the study sample reported having regular
exercises in this home-based garment workers. This research supports the previous
studies conducted in garment workers, that the nature of work by fixed and sedentary
postures found in garment workers increase the risk [4, 5]. In addition to those findings
[7] supported that repetitive movement of doing the same activity for a prolonged
period more than two hours as a significant risk factor contributing to LBP as well as
working in continuous and prolong hours than 8 h per day.
Risk Factors Associated with Work-Related Low Back Pain 759
5 Conclusions and Implication
The finding prevalence of work-related LBP amongst home-based garment workers in
this study required confirmation of risk factors to implemented action or prevention of
LBP. By prolonged sitting to operate the sewing machine at least eight hours a day, and
relative risk factors such as no regular exercise, repetitive work, no change of posture
each hour might effectively attribute to LBP occurrence. A prospective cohort study is
suggested to further conduct among this group of home-based garment workers in
order to identify factors causing work-related LBP cases. We propose the three points
of implication; (1) there should be policy regulation for protection of home-based
garment workers to be registered by the Ministry of Labour; (2) health and safety
efforts by ministry of Labour together with ministry Public Health to raise awareness;
and (3) there should be home-based workplace assessment and monitoring system with
ergonomics knowledge training and promoting of health program to prevent low back
pain in this kind of home-based garment workers.
References
1. National Statistical Office, Ministry of Digital Economy and Society, Thailand. The informal
employment survey (2018). http://www.nso.go.th/sites/
2. Homsombat, T., Chaiklieng, S.: Physical fitness and muscular discomfort among informal
garment female workers in Udon Thani Province, Thailand. J. Med. Assoc. Thai. 100(2),
230–238 (2017)
3. Thotsathit, N., Puntumetakul, R., Eungpinichpong, W., Peungsuwan, P.: Prevalence of
musculoskeletal disorders in sewing occupation in Khon Kean Province. KKU. Res. J. 11,
47–54 (2011)
4. Kampong, T., Chaiklieng, S.: Risk Factors for work-related back pain among informal
garment workers in Nong Rua district, Khon Kean Province. KKU. J. Public Health Res. 7(2),
70–78 (2013)
5. Chaiklieng, S., Suggaravetsiri, P., Pultumethakul, R.: Prevalence and risk factors for work-
related shoulder pain among informal garment workers in the Northeast of Thailand. Small
Enterpr. Res. 21(2), 180–189 (2014)
6. Department of Disease Control, Ministry of Public Health, Thailand. MSDs among out-
patients and in-patients. http://ddc.moph.go.th/th/site/index
7. Chaiklieng, S., Homsombat, T.: Incidence and postural risk factors for low back pain among
informal garment female workers. In: AHFE 2019. AISC, vol. 967, pp. 222–230 (2020)
8. Chaiklieng, S., Suggaravetsiri, P.: Risk factors for repetitive strain injuries among school
teachers in Thailand. Work 41, 2510–2515 (2012)
Demand Management Model Based
on Quantitative Forecasting Methods
and Continuous Improvement to Increase
Production Planning Efficiencies of SMEs
Bakeries
Denilson Contreras-Choccata1, Juan Sotelo-Raffo1,
Carlos Raymundo-Ibañez1(&), and Luis Rivera2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Peru
{u201420210,pcinjsot,carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. In the last quarter of 2018, the manufacturing sector grew 11.4%, of
which 19.6% was accounted for by the food industry. However, one of the main
problems faced by micro and small companies is poor management, since many
of these businesses plan based on experience. In addition, inaccurate demand
forecasting generates losses for these organizations due to overproduction or
understocking. While the former triggers losses from elevated operating costs,
the latter leads to loss of revenue and unsatisfied customers and compromises
future demand rates. Therefore, a demand management model was designed to
provide accurate and relevant information, which may improve production
planning based on the continuous improvement approach. The model increased
the planning from 91.2% to 96.2% in a micro business dedicated to the pro-
duction and sale of bread.
Keywords: Demand management Forecasting methods Demand forecasting
continuous improvement
1 Introduction
Micro and small businesses play an important role in nationwide economies in terms of
job creation and increase in production. However, they are often plagued by techno-
logical and economic constraints, and poor strategic and operational planning [1].
For example, in Peru, these businesses face a series of obstacles to their growth,
development, and survival over time. According to Avolio, the operational factors that
decelerate the growth of micro and small businesses are the lack of basic knowledge of
production planning, demand management, and inventory management [2]. Micro and
small companies account for 40% of the national gross domestic product (GDP) in Peru
and are deemed the main drivers of economic growth within the country.
retraso de parte de las pequeñas y medianas empresas en el planning,inventario y manejo de la demanda
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 760–765, 2020.
https://doi.org/10.1007/978-3-030-39512-4_116
Demand Management Model Based on Quantitative Forecasting Methods 761
Despite this scenario, small and micro companies in the food industry face a
number of challenges, which often culminate in their failure within less than a year
from having started operations. One of these issues is poor management, which turns
them into subsistence companies. According to the Peruvian Association of Bakeries
and Pastry Shops, these micro and small companies report problems in managing
production, sales, and inventories. Therefore, they require effective yet affordable
solutions. Currently, very few research studies address production planning issues in
micro and small business within the manufacturing sector, much less in bakeries.
In this context, this article proposes a conceptually developed demand management
model based on qualitative forecasting methods backed by the continuous improve-
ment approach.
2 Literature Review
Quantitative Demand Management Methods
Micro and small businesses dedicated to the production and retailing of fast-moving
bread and perishables face the daily challenge of stocking the right number of products
on shelves. In light of this, poor forecasting accuracy often leads to overestimation or
underestimation of demand rates. In the former case, since bread has a very short shelf
life, any remaining units at the end of the day are discarded. This represents losses, as
production costs such as raw material, labor, and machinery are wasted [3].
Understocking elevates out-of-stock rates, leading to unquantifiable losses. This is
because these losses depend on customer reaction, which is uncertain. In other words,
understocking leads to immediate revenue losses and dissatisfied customers, which
conditions customer loyalty and puts future sales at risk. Thus, we may conclude that
inaccurate demand forecasting exerts negative influence on a company’s profitability.
Future demand may be forecast based on several methods, each with its own
advantages, disadvantages, and viability, which may provide different results [4].
However, method selection is critical to yield accurate forecasts.
Demand Management Model with Continuous Improvement Approach
Organizations require that relevant and accurate information be available at all times. In
fact, according to Ren [5], updated demand forecasts play a crucial role in generating
updated inventories. In addition, they allow business owners to order and improve
initial demand forecasts based on the latest market information [5].
3 Contribution
3.1 Background
The proposed model is developed based on the Alcântara model (2015), which
describes the common quantitative and qualitative variables to be taken into account for
managing demand rates. This is because they provide accurate information leading to
precise demand forecasting. This output is an essential foundation for planning
762 D. Contreras-Choccata et al.
production rates, optimizing supply chain capacities, introducing new products, and
improving customer satisfaction [6]. The model proposed by [6], which lists the factors
that generally affect demand management, was also assessed in the development of our
model. For these purposes, the following factors have been extracted [6]: customer
segmentation, adherence to demand management principles, and measurement of
model performance through indicators.
3.2 Proposed Model
The proposed model is shown in Fig. 1.
Market Acumen. For [6], a successful demand management implementation in an
organization requires deeply understanding the market by assessing the capabilities,
constraints, and opportunities presented by its internal and external environments, since
information would be poorly managed without minimum market conditions.
Customer Segmentation. Demand segmentation implies grouping practices related to
the identification of key, secondary, and occasional customers, and specific require-
ments for each of the products offered [6]. Therefore, this variable is paramount for
properly managing demands, since each business serves its own customer base and
offers products with different characteristics.
Fig. 1. Demand management model
Demand Forecasting. The exponential smoothing method is used for forecasting
future demand rates as per the following formula:
^t ¼ X
X ^t0 þ Tt ð1Þ
0
^t0 ¼ a X
X ^t1 þ ð1 aÞ X
^t1 þ Tt1 ð2Þ
0
^t X
Tt ¼ b X ^t1
0
þ ½ð1 bÞðTt1 Þ ð3Þ
Demand Management Model Based on Quantitative Forecasting Methods 763
Here: X ^t ¼ forecast for period t, X^t1 ¼ data smoothing constant, X ^t1
0
¼ new
observation or actual series value for period t, Tt1 ¼ T 1 period trend, Tt ¼ trend
estimation, a ¼ smoothing constant and b ¼ smoothing constant for the trend.
3.3 Proposed Method
Figure 2 denotes the procedure for model implementation at small and micro bakeries.
Fig. 2. MGD procedure
3.4 Control Indicators
Below are 3 formulas of the indicators applied. See Table 1.
Table 1. Control indicator
Mean Absolute
Pn Percentage Error (MAPE) Production Plan Effectiveness Index
i¼1
100IActuali Forecasti I
PP ¼ Actual Demand from Plan
Expected Demand 100
MAPE ¼ Actuali
n
*An error below 5% indicates that the
method selected is the correct method.
Model Accuracy Level
P
jFAj
FA ¼ 1 P 100
jM a xðF; AÞj
764 D. Contreras-Choccata et al.
4 Validation
4.1 Case Study
The Cosmos Bakery, a family business established in 2003, offers products such as
bread (baguettes, ciabatta, croissants, etc.), cakes (Three Milks, Black Forest, Chantilly,
etc.), bundt cakes (orange, pineapple, etc.), and other baked goods. See Table 2.
4.2 Final Results
Compared with the initial assessment, the first, second, and third indicators displayed
improvement of 1.5%, 4.3%, and 7.4%, respectively. Therefore, it is evidenced that the
proposed model considerably improved the situation of micro businesses in terms of
forecasting demand and expected sales. See Table 3.
Table 2. Initial case study assessment Table 3. Font sizes of headings. Table cap-
tions should always be positioned above the
Indicators Results
tables.
Mean Absolute Percentage Error 1.70%
(MAPE) Indicators Results
Production Plan Effectiveness 91.9% Mean Absolute Percentage 0.20%
Index Error (MAPE)
Model Accuracy Level 89.60% Production Plan Effectiveness 96.2%
Index
Model Accuracy Level 97%
5 Conclusions
Model implementation at the Cosmos Bakery improved production planning efficien-
cies from 91.9% to 96.2%. Model accuracy increased from 89.6% to 97%, which
reduced losses, since the actual demand was very close to the expected demand.
As per statistical analyses, the smallest difference between the average of simula-
tion scenarios and result validation is 0.20 points and the largest is 4, with a small
standard deviation at 1.97. Therefore, based on the simulation scenarios performed, the
demand management model may work to a greater extent when the error percentage
exceeds 5% based on daily demand forecasting and to a lesser extent when there is a
sharp change in the economy.
References
1. Mohamed, A.B., Mohd, T.T., Mustafa, O.: Developing cash waqf model as an alternative
source of financing for micro enterprises in Malaysia. J. Isl. Acc. Bus. Res. 7, 254–256 (2016)
2. Avolio, B.: Factors constraining growth in micro and small peruvian businesses. J. PUCP 22,
70–80 (2011)
Demand Management Model Based on Quantitative Forecasting Methods 765
3. Huber, J., Gossmann, A., Stuckenschmidt, H.: Cluster-based hierarchical demand forecasting
for perishable goods. Exp. Syst. Appl. 76, 140–151 (2017)
4. Abdul, K.: Demand forecasting for strategic resource planning. In: Proceedings of the 2016
International Conference on Industrial Engineering and Operations Management, USA,
pp. 23–25 (2016)
5. Ren, S.: A comparative study on fashion demand forecasting models with multiple sources of
uncertainty. Ann. Oper. Res. 257, 335–355 (2017)
6. Rexhausen, D., Pibernik, R.: Customerfacing supply chain practices: the impact of demand
and distribution management on supply chain success. J. Oper. Manag 30, 269–281 (2012)
Study on Key Elements of Shopping App
Design for the Elderly
Wenfeng Liu, Fenghong Wang(&), and Yiyan Chen
School of Design, South China University of Technology,
Guangzhou 510006, China
[email protected], [email protected],
[email protected]
Abstract. In the context of an aging society, the aging design of key elements
of the design of the old shopping application helps to improve the user expe-
rience of the elderly users. In this paper, through the user research, test set
clustering method and other methods to achieve the extraction of the key ele-
ments of the elderly shopping app design, around the extraction of “page layout,
graphic collocation, color, text size” and other four key elements of the
experimental design, and finally through the user experience measurement, one-
way analysis of variance and other methods, the page layout, graphic matching
style, color visual style and text size range of the shopping app suitable for the
elderly are obtained.
Keywords: The elderly Shopping app design The user experience
1 Introduction
With the development of society and the popularization of the Internet, the way of life
of the elderly has undergone great changes. More and more older people are using
mobile apps to shop. With the growth of age, the function of the body organs of the
elderly gradually decline, the cognitive ability began to decline [1], directly affecting
the efficiency and effectiveness of the elderly access to daily information. However, the
study of applying design elements to older people only focuses on the comparison of
important relationships between various design elements. Such as the literature [2],
using the gray analysis level process to establish the old application user experience
evaluation model, from the perceived experience level of the elderly medical appli-
cation style font size, color, font style and other visual design elements research
conclusions can only reflect the impact of various visual elements on the elderly user
experience. However, the study of individual visual elements in applied design ele-
ments is rarely mentioned.
From the perspective of user experience, this paper conducts experimental research
on key elements of shopping app design for the elderly, and analyzes the interface
layout, graphic and text matching style, text size range and color visual style that are
most suitable for the elderly when they use app shopping.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 766–772, 2020.
https://doi.org/10.1007/978-3-030-39512-4_117
Study on Key Elements of Shopping App Design for the Elderly 767
2 Obtain Experimental Samples
2.1 Study on Key Visual Design Elements
Based on the number of downloads for the Android App Market Ingmarket Shopping
app, select four of the top shopping apps for competitive product analysis, as shown in
Fig. 1. By comparing and analyzing the four application information architectures, we
find that the main feature pages are basically the same, including the home page,
category list, product list, product details page, shopping cart page, payment page,
message conversation page, my home page, and so on.
Fig. 1. The top four shopping apps in the android market
The user’s shopping process is: find the goods, view the goods, select the goods to
pay. Usability test was adopted to set up purchase tasks for users, observe the whole
shopping process of users, record and analyze the thoughts, emotions and pain points
of the elderly on different pages at different stages, as shown in Fig. 2.
Fig. 2. User journey diagram of shopping for elderly users
The main factors affecting users’ visual experience can be summarized as follows:
text and picture information is too small; too much information on the home page,
crowded page layout, too chaotic in the visual sense, users easily lost; a large number
of dynamic pictures tend to distract users’ attention from visual selection, resulting in
users’ inability to locate information quickly and effectively; too many product labels
on the classification list page are too small.
768 W. Liu et al.
Based on the above analysis, the visual elements that affect the user experience of
the elderly can be summarized into four categories: page layout, graphic collocation,
color and text size.
2.2 Obtain Experimental Samples
Considering the physiological and psychological conditions of the elderly, it is nec-
essary to strictly control the number of experimental samples and reduce the testing
time as much as possible. Therefore, the test clustering [3] method is adopted to
classify a large number of test samples according to the similarity of their character-
istics, extract the most representative samples from each category and combine them
into a set of typical samples. K-means algorithm is widely used in the clustering of
large-scale data, and its definition is as follows:
Xk X
E¼ i¼1 p2Ci
jp m i j2 ð1Þ
In the above formula, E is the sum of the squared errors of all objects in the
database, p is the point in space, and mi is the average of cluster Ci. The algorithm flow
of K-means clustering algorithm is as follows: (1) input the number k of databases and
clusters containing n objects; (2) choose k objects arbitrarily as the initial cluster center;
(3) repeat; (4) according to the average value of the objects in the cluster, each object is
assigned to the most similar cluster; (5) update the average value of the cluster, that is,
calculate the average value of the objects in each cluster; (6) until there is no change;
(7) output k clusters to minimize the squared error criterion.
Fig. 3. Experimental samples of key elements of shopping app design
Conduct corresponding coding operations for page layout, color and graphic col-
location, and take all codes of each category as the input database of k-means to carry
out clustering operations. When coding the page layout, 37 domestic and foreign
shopping related app pages were intercepted, and the image recognition technology
was adopted to encode them one by one. RGB color classification mode was adopted to
input the (R/G/B) value of each color into the database as the test set. Manual clustering
was adopted for graphic collocation. 4 most common collocation forms on mobile apps
were selected as test sets by referring to the combination forms of graphics and
graphics in Microsoft Office. Finally, typical samples of page layout, color and graphic
collocation are extracted, as shown Fig. 3.
Study on Key Elements of Shopping App Design for the Elderly 769
3 Experimental Study on Key Elements of App Design
Research on the key elements of shopping app design for the elderly is mostly con-
ducted by collecting users’ subjective evaluations, so a wide confidence interval is
allowed. The confidence level of this experiment is 80%, and the allowable error is
15%, so the minimum sample size of the investigation is 19 people [4]. Finally, 20
elderly users were selected as subjects, with a ratio of 1:1 for men and women, ranging
in age from 60 to 75 years. Twenty college students were selected as control groups.
The ratio of men to women is 1:1 and the age of 20 to 30 years. None of the subjects
had significant visual or hearing impairments, which did not affect the acquisition of
normal information.
3.1 Selection Experiment of Page Visual Design Elements
Design Experiments. Each factor is obtained by clustering algorithm of four typical
sample placed a Web page, the participants in the process of watching random selected
the most attract visual attention two samples, and the two samples are pay attention to
the degree of subjective evaluation, evaluation means uses five-level Richter scale [5].
During the experiment, the subjects were asked to come up with some emotional words
to describe the reasons for their choice.
The Experimental Process. There were three groups of pictures in the test, four
pictures in each group according to page layout, graphic collocation and color, and the
order of the pictures in each experiment was messed up. Browse and gaze for 10 to
20 s, and select the two pictures with more attention. After the 20-second timing, the
subject needs to complete the subjective evaluation of the attention level of the two
pictures.
Experimental Data Processing and Analysis. In each group of experiments, the two
samples most selected by the aged group and the young group were selected. The
subjective evaluation of attention of the subjects on each sample was collected and
quantified, and the mean value and standard deviation of attention index of each sample
were obtained, as shown in Table 1. The results show that the standard deviation range
of each group of data of the three elements is between 0.8 and 1.2, indicating that the
dispersion degree of data is not large, and the average value is reasonable and reliable
for data analysis.
In terms of page layout, P12 and P11 were selected by the older group. The elderly
group scored 3.87 and 3.77 for the attention degree of P12 and P11, respectively,
indicating that the application of nine-grid layout or drawer list layout to app design has
a good aging effect.
In the sample selection of image-text collocation elements, the aged group tends to
choose P22 and P21. The average visual attention degree of the two is 3.67 and 3.36,
respectively. In terms of visual attention degree, P22 is slightly higher than P21.
Therefore, the collocation structure of the left picture and right text of P22 is preferred
in the design suitable for aging, and it is applied to the situation of the same proportion
770 W. Liu et al.
Table 1. Page design elements high-frequency options statistics results.
Heading level Attention level
Experiment Group Options Feeling Amount Average Standard
deviation
Page layout Elderly P12 Stability and 15 3.87 1.02
group symmetry
P11 Easy to 13 3.77 0.97
distinguish
Youth P13 Look it up 16 3.94 1.07
group quickly
P11 Easy to 10 3.30 1.12
navigate
Graphic Elderly P22 Balanced 15 3.67 1.06
collocation group P21 Reading 11 3.36 0.88
habits
Youth P23 Turn the page 15 3.93 1.04
group quickly
P22 Easy to 13 3.85 0.97
compare
Color Elderly P34 Bright and 16 3.88 1.03
choice group festive
P33 Quiet and 12 3.50 0.96
vast
Youth P33 Sea, calm 13 3.77 0.95
group P31 Warm and 13 3.69 1.06
energetic
of pictures and texts. The collocation structure of text and text on the title is used in the
case of large amount of text reading.
In the sample selection of color elements, the aged group tends to choose P34 and
P33, especially P34, which has the highest visual attention, 3.88. It can be seen that the
elderly group is particularly fond of the red and purple series, and has a higher degree
of fondness for blue. Therefore, when designing shopping app for the elderly, priority
should be given to the visual style of red, purple or blue, so as to fully take into account
the psychological needs of the elderly.
3.2 Font Size Selection Experiment
Design Experiments. Prepare seven introductory paragraphs of about 100 words in
bold type, black color, and white background. The font size of each paragraph is 14sp,
18sp, 22sp, 24sp, 26sp, 28sp and 30sp respectively. Each paragraph is similar in
content, difficulty and word count. The subjects were asked to read seven introductory
paragraphs on their mobile phones. Next, subjects were asked to make subjective
Study on Key Elements of Shopping App Design for the Elderly 771
evaluation on the comfort level of this paragraph. The evaluation method was five-level
Richter scale.
The Experimental Process. Eight test materials shall be provided by the staff for
testing in sequence. Participants were asked to read each piece of material and rate their
satisfaction with the reading.
Experimental Data Processing and Analysis. Subjective evaluation is made on
comfort and clarity of different font sizes, and mean value and standard deviation are
calculated, as shown in Table 2. The standard deviations of comfort range from 0.52 to
1.18, respectively. The dispersion degree of data is not large, and the average value has
certain representativeness. Next, a significant analysis of the differences between the
groups was made to determine the optimal range of font sizes.
Table 2. Mean and standard deviation of reading comfort under different font size conditions.
Font size 14sp 18sp 22sp 24sp 26sp 28sp 30sp
Comfort level 2.24 2.97 3.62 3.90 4.03 4.45 4.52
(0.76) (1.18) (0.60) (0.52) (0.61) (0.40) (0.62)
It can be seen from Table 3 that subjects have different subjective evaluation ten-
dencies when testing 14sp and 18sp. Since F>F crit, p-value <0.15, there are obvious
differences between such choices. When the font size increases to 22sp, it can be seen
that values between groups all present F<F crit and p-value >0.15, indicating that
subjects tend to choose the same size comfort. However, when the font size increases to
26sp, the value between groups presents F>, F crit, and p-value <0.15, indicating that
the preference of subjects to choose the comfortable font size begins to show obvious
differences. As a result, 22sp font can be selected as the most appropriate font size in
the app to ensure that the elderly can see all the application contents clearly when
reading the text. When the font size is less than 18sp font, the reading comfort and
efficiency of older people will be affected because the font size is too small. When the
font size is greater than 24sp, the larger font size affects the visual effect of the
interface, although the user can clearly see it.
Table 3. Analysis of significant differences between groups in comfort evaluation under
different font size conditions
Font size (sp) 14&18 18&22 22&24 24&26 26&28 28&30
F 7.84 7.00 1.96 0.49 4.94 0.14
P-value 0.01 0.01 0.17 0.49 0.03 0.71
F crit 2.13 2.13 2.13 2.13 2.13 2.13
To sum up, the font size range in the app can be 18sp–24sp, of which 22sp is the
main font.
772 W. Liu et al.
4 Conclusion
In this paper, the test clustering method takes a sample of the typical characteristics of
three pages of visual design elements, namely, page layout, graphics matching and
color preference. Through the experiment to quantify the user’s subjective evaluation,
the analysis and obtain the page layout, graphic sand and color preference of the
application that the elderly tend to choose. The text size of the app was then studied to
collect user reading comfort for different font sizes. Through one-way anova data
analysis method, the font size range suitable for elderly people to read text on the
application was finally determined.
References
1. Lin, C.: Developmental Psychology. People’s Education Press, Beijing (2005)
2. Li, Y., Shi, W., Zhu, L.: Study on the evaluation of APP user experience of the elderly based
on grey analytic hierarchy process. J. Graph. 01, 68–74 (2018)
3. Feng, X., Zhang, T.: Comparison of four clustering methods. Microcomput. Appl. 20, 1–3
(2010)
4. Ke, H., Shen, H.: Statistical Analysis Methods in Survey Research. Communication
University of China Press, Beijing (2005)
5. Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research.
China Machine Press, Beijing (2014)
Shopping Website Accessibility Study Based
on Users’ Mental Models
Zhen Wu, Chengqi Xue(&), Yanfei Zhu, Binhong Zhai,
and Chi Zhang
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], [email protected],
[email protected], [email protected],
[email protected]
Abstract. The arrival of the Internet information age promotes the develop-
ment of electronic and information platform - website. A website is a provider of
information and a platform for users to communicate and interact. Website
interface design must consider human cognitive factors, emotional factors, so
the website interface design and interaction design is particularly important.
Through experimental demonstration and research, this paper takes shopping
website as the research object to analyze the mental model of users in website
interface design and the influence of different users’ occupations on the mental
model. According to the conclusion of the analysis, firstly, the change process of
user mental model in interface interaction is proposed. Secondly, based on the
user mental model, this paper summarizes the design principles of the website.
Finally, this paper provides a new way of thinking for the study of website
accessibility.
Keywords: Users’ mental model Shopping website Interface Principles
Accessibility
1 Introduction
With the rapid development of network technology and electronic computers, e-
commerce is closely related to people’s daily life. The shopping website not only
changes the living habits and communication methods of most people, but also changes
people’s daily shopping methods. The rapid development of e-commerce and the rise
of online shopping platforms narrow the distance between customers and enterprises
[1]. This method not only saves the cost of the enterprise, improves efficiency, but also
saves users time. However, there are still some problems in the design of shopping
website interface. In order to improve the service quality of the website and bring a
better experience to users, the accessibility of a website should be focused [2]. This
paper explores the interface layout of shopping website and the position of elements
through the user’s mental model and experimental demonstration, so as to improve the
accessibility of the website.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 773–779, 2020.
https://doi.org/10.1007/978-3-030-39512-4_118
774 Z. Wu et al.
2 Research Summary
2.1 Mental Model
Mental model is a concept derived from cognitive psychology, which is a dynamic
internal representation of the external world/events [3]. Mental model influences the
way we understand systems and behaviors through assumptions and generalizations
that are stereotyped and deeply embedded in our minds [4].
The users’ mental model of a web interface is that the user understands appearance,
behavior, and feedback of the web interface when using the Internet. Different ways of
thinking, education and life experience can be classified into physical differences,
cognitive differences and emotional differences.
When users browse and use the website, the mental model can have a considerable
impact on their understanding of the website [5]. Norman, a psychologist, proposed
that there are three mental models in web design, namely implementation model,
presentation model and users’ mental model.
A good web interface depends on whether the presentation model matches the
mental model. If the layout of the website interface meets the user’s psychological
expectations, the user’s satisfaction with the website interface will be greatly improved.
Therefore, when designing the website interface, the designer needs to consider the
user’s mental model. Then, the layout of the website interface and the location of the
interface elements should conform to the user’s preference. The designer should reduce
the difference between the performance model and the mental model as much as
possible. Through the above methods, web design can increase the user experience and
improve the accessibility of the web.
2.2 Accessibility of the Website
Accessibility, as an independent research area, is a general idea of website design.
Accessibility means that anyone, including people with disabilities, can use and access
the site [6]. This method greatly improves the efficiency of users with different
knowledge background, physical condition and age when using the website, and to
some extent reduces the difficulty for users to operate the website. Loiacono compares
the accessibility of federal government, nonprofit organizations (npos) [7], and cor-
porate homepages to determine the accessibility of these sites to people with disabil-
ities. HernáNdez, et al. mentioned that accessibility refers to making it easier for
existing users and potential users to identify and obtain website information [8].
At present, the discussion about website accessibility mainly focuses on interface
design, but there is a lack of theoretical and empirical research on the psychological
level. This paper attempts to demonstrate the accessibility of shopping website from the
perspective of users and psychology, combined with the theory of mental model.
Shopping Website Accessibility Study Based on Users’ Mental Models 775
3 Experimental Design
3.1 Experimental Methods and Aim
The main methods to test the user’s mental model include: experimental observation,
user interview, questionnaire survey, etc. This paper adopts experimental observation
and interview method to obtain users’ mental model through experimental data. Aim:
The designer combines the user’s mental model to design the interface of the shopping
website. This approach can improve the accessibility of a site by narrowing the gap
between the user’s mental and presentation models.
3.2 Experimental Tasks
Through the analysis of the interface of many shopping websites, the research results
show that the user’s preference for the interface mainly depends on the layout of the
interface and the position of the elements in the layout.
(1) Firstly, the author sorted out a large number of domestic and foreign shopping
websites. Then, the ten websites with the most user views are selected. Through
the high summary of ten websites interface, three different interface layouts are
got (Fig. 1) 100 subjects were asked to choose the three interface layouts
according to their preferences and data were collected.
(2) The author counted the common components of the shopping website. These
elements include: Customer order, website logo, product search bar, Commodity
display, catalog classification, external links. Information about the positions of
the six elements in the layout of the interface is collected. Next, the subjects were
asked to place the six elements in the three layouts shown in Fig. 2 according to
their preferences. Finally, the data is recorded.
Fig. 1. Shopping website layout
776 Z. Wu et al.
3.3 Experimental Process
(See Table 1).
Table 1. Experimental design elements
The experiment 1 The experiment 2
Methods and tool The questionnaire survey Spss 20.0, Excel 2016
Subjects 20 undergraduates and 30 postgraduates age
between 19 and 25 people from Southeast
university
50 workers age between 20 and 29 Education
background: Master, undergraduate, high school,
primary school occupation: teachers,
programmers, Cleaning, administration,
attendants
The experimental variable Three variables: Independent variable, control
variable, dependent variable
Independent variables: user education
background, occupation, shopping website
familiarity
Control variable: website layout (Fig. 1), the
position of Element
Dependent variable: users’ mental model
The basic flow The questionnaire is mainly distributed and
explained by specific people
100 questionnaires were distributed.
100 questionnaires were collected
Valid questionnaires: 97 Invalid questionnaire: 3
4 Experimental Results and Discussion
4.1 Analysis of Interface Layout Preference
The analysis of interface layout preference means the frequency and percentage statistics
of the preferred interface layout (Table 2), The experimental results show that the user’s
preferred interface layout is A > C > B. Users’ preference for layout A accounted for
42.3%. Users who like C layout are 35.1%. Users who like C layout are 35.1%.
Table 2. The percentage of Preference interface layout
Valid A layout 41 41.0 42.3
B layout 22 22.0 22.7
C layout 34 34.0 35.1
Total 97 97.0 100.0
Missing System 3 3.0
Total 100 100
Shopping Website Accessibility Study Based on Users’ Mental Models 777
4.2 Correlation Analysis of Layout Preference
The correlation analysis of layout preference means the correlation analysis between
the layout of shopping website interface and the subjects’ educational background,
shopping website familiarity and occupation. The experimental results show that the
correlation coefficient between educational background and layout preference is 0.158,
and p = 0.069 < 0.1 (Table 3). There is a correlation between layout preference and
educational background, while there is no significant correlation between occupation,
shopping website familiarity and layout preference.
Table 3. Correlation analysis of layout preference
Preference Education Shopping Occupation
for layout background website background
familiarity
The layout Pearson 1 .158 .029 .068
that user like correlation
Sig .069 .687 .743
N 97 97 97 97
4.3 The Analysis of Element Position Preference in Layout
According to the analysis of layout preference, most subjects prefer the layout of the
interface A. The A interface includes six elements: Customer order, website logo,
product search bar, Commodity display, catalog classification, external links. The
subjects were asked to place elements in each module of the A layout according to their
preferences. According to the statistical results, the location of element that the users
like is shown in Fig. 2.
Fig. 2. Users prefer element location
Based on the above experimental results, the key factors of the users’ mental model
are summarized. Then, this paper proposes the process of forming and changing the
user’s mental model when visiting and using the website (Fig. 3).
778 Z. Wu et al.
Fig. 3. The user’ mental model of a shopping website
5 Conclusion
When designing a web interface, designers should consider the theory of the user’s
mental model. The user’s educational background, user’s preference for interface
layout and element location should be considered when designers design interface. The
interface design of the shopping site should be similar to Fig. 2. All these methods can
improve the accessibility of the website.
Acknowledgments. This work was supported by National Natural Science Foundation of China
(No. 71871056), National Natural Science Foundation of China (No. 71471037), and Basic
Research Program of Xuzhou, 2017 (No. KC17071).
References
1. Ratnasingam, P.: The evolution of online relationships in business to consumer E-commerce.
In: E-Commerce Trends for Organizational Advancement: New Applications and Methods
(2010)
2. Davis, L.J.: Providing virtual services to all: a mixed-method analysis of the website
accessibility of philadelphia area consortium of special collections libraries (PACSCL)
member repositories. Am. Arch. 75(1), 35–55 (2012)
3. Sifaqui, C.: Structuring user interfaces with a meta-model of mental models. Comput. Graph.
23(3), 323–330 (2013)
4. Wang, T.-H., et al.: Diagnosing students’ mental models via the web-based mental models
diagnosis system. Br. J. Educ. Technol. 44(2), E49–E51 (2013)
5. Turumugon, P., et al.: Users’ mental model pattern for user interface design of mobile
shopping apps. J. Comput. Theor. Nanosci. 24(2), 1158–1162 (2018)
Shopping Website Accessibility Study Based on Users’ Mental Models 779
6. Kerkmann, F., et al.: Accessibility of web search engines: towards a deeper understanding of
barriers for people with disabilities. Libr. Rev. 61(61), 608–621 (2012)
7. Loiacono, E.T., Mccoy, S.: Website accessibility: a cross-sector comparison. Univ. Access
Inf. Soc. 4(4), 393–399 (2006)
8. Hernández, B., Jiménez, J., Martín, J.: Key website factors in e-business strategy. Int. J. Inf.
Manage. (29), 362–371 (2009)
HIRAC-Based Risk Management Model
with POKA–YOKE and TPM Continuity
to Control and Mitigate Emergency Scenarios
in Hydrocarbon Sector Operations
Jose Echevarria-Cahuas1, Maria Quispe-Huapaya1,
Cesar Ramirez-Valdivia1, Carlos Raymundo1(&), and Luis Rivera2
1
Universidad Peruana de Ciencias Aplicadas (UPC), Lima 15023, Peru
{u201120106,u201421060,pcgecram,
carlos.raymundo}@upc.edu.pe
2
Escuela Superior de Ingeniera Informática, Universidad Rey Juan Carlos,
28933 Móstoles, Spain
[email protected]
Abstract. It is currently known that LPG packaging plants pose a potential risk
to their personnel, surroundings, and even their facilities. This was identified by
studies conducted on LPG packaging plants accidents, specifically between
2011 and 2018. Their results show that at least 28 LPG plants were shut down in
the country. The main dangers facing LPG plants are fires, leaks, flares, and
spillages, among others. Therefore, the proposed model aims at reducing and
mitigating the risks and emergency scenarios by implementing Hazard identi-
fication, risk assessment and control (HIRAC); POKA–YOKE, as engineering
control measures; and Total Productive Maintenance (TPM). In the creation and
implementation of this model, we jointly worked with the company PUNTO DE
DISTRIBUCIÓN S.A.C. The results show that significant risks reduced to 0%,
whereas the overall equipment effectiveness increased by 11%. Furthermore, the
quality and availability indexes increased by 2% and 2.7%, respectively.
Keywords: Risk management HIRAC POKA–YOKE
1 Introduction
In the past 30 years, 60 relevant emergencies have occurred in LPG storage, pro-
cessing, packaging, and marketing plants worldwide [1]. The number of emergency
scenarios in Peru has increased in the last 7 years, with five of them occurring in
packaging plants and originating on the packaging platform. The consequences were
total destruction, explosions, and fires. One of the most serious emergency situations
took place in 2013 at the Pecsa Group plant in Huachipa [2]. A further example is of
the 42 LPG packaging plants that were shut down by Osinergmin for not complying
with the criticality and safety measures of the facilities and cylinders in 2011–2018.
67% of these plants are/were located in Lima and Callao.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 780–786, 2020.
https://doi.org/10.1007/978-3-030-39512-4_119
HIRAC-Based Risk Management Model with POKA–YOKE and TPM Continuity 781
The study was conducted to create a risk management model focusing on the
operations area of LPG packaging plants, using HIRAC, with continuity in Poka–Yoke
and the Total Productive Maintenance program (TPM) as mitigation and control
measures to prevent/minimize the risks that could affect human resources, facilities,
and third parties [3].
2 State of the Art
2.1 Hazard Identification, Risk Assessment and Control (HIRAC)
A number of risk control analyses using HIRAC were performed in 2017, focusing on
different production activities to identify the risks and to measure its probability of
materializing. Some of the industries where the analyses were carried out include the
following: the printer ink industry in Malaysia [4]; iron granulation in India [5]; and the
hydroelectric industry [6], among others. First, the processes undertaken by the com-
pany are identified, which is followed by the identification of the hazards and risks. On
the basis of the information gathered, the risks are classified, considering the proba-
bility of them occurring, their severity, and the frequency with which workers are
exposed to them. A study employing the same method to analyze the human factor
risks in port accidents was conducted that same year. The results showed that two risk
factors played a significant role, namely, human oversight and the lack of experience of
workers. Furthermore, they indicated the need to make better decisions and prioritize
risks [7].
2.2 POKA – YOKE
The next tool to be described is POKA–YOKE. It will be applied to avoid and/or
prevent mistakes during any process, thus ensuring product quality and the reduction of
accidents of any kind as was observed in the safety analysis of LPG tanks in a
distribution station. In the analysis, three scenarios were described, which led to the
same mitigation measure, namely, the installation of gas detectors in the plant [8].
A HIRAC-based risk analysis study was conducted in a paper pleating plant in 2017.
The results identified the main production risk in the lamination cutting process.
Thanks to the information gathered, the main cause of the problem was identified and a
POKA–YOKE was designed, which consisted in the creation of a glove that was more
comfortable, flexible, and adequate to the hands’ physical structure. A new risk anal-
ysis was performed following the implementation of this PPE, which showed that the
highest risk had been eliminated and promoted improvements throughout the pro-
duction line [9].
782 J. Echevarria-Cahuas et al.
3 Contribution
3.1 Proposed Model
Based on the literature presented, we propose a model (Fig. 1) that allows for suc-
cessful risk management in an LPG packaging plant, particularly in the product
packaging and filling areas. The model has three important aspects described below.
Fig. 1. Proposed model
3.1.1 HIRAC
HIRAC allows for the prediction, analysis, and control of risks in the workplace that
can harm workers. It consists of the following steps:
(a) Activity classification
(b) Hazard identification
(c) Risk classification and description
(d) Calculating severity, probability, and relative risk
(e) Risk assessment
(f) Mitigation, prevention, and control measures
(g) Monitoring.
3.1.2 POKA–YOKE
These are tools that factor in the prevention of mistakes in a system and ensure
machinery safety with regard to the operators, processes, and final product quality, thus
avoiding any kind of problems.
• It detects operational mistakes and defects.
• It is implemented in the process to eliminate or identify the problem.
• It improves product quality.
HIRAC-Based Risk Management Model with POKA–YOKE and TPM Continuity 783
3.1.3 TPM
This is a maintenance program that is based on a recent concept of production plants
and equipment maintenance to improve product quality, reduce waste, reduce pro-
duction costs, improve equipment availability, and improve the overall maintenance
state of a company. This is achieved by focusing on three goals:
• No defects.
• No accidents.
3.2 Indicators
3.2.1 Overall Equipment Effectiveness (OEE)
This is the squared ratio of the number of correctly produced elements. The calculation
takes into consideration availability (A), performance (PE), and quality (Qr).
OEE ¼ A PE Qr ð1Þ
3.2.2 Performance Index (PE)
This shows the performance of a machine or piece of equipment after considering the
current production rate when compared with the optimal production rate.
Produced elements
PE ¼ ð2Þ
Optimal production capacity
3.2.3 Quality Index (Qr)
It is obtained after considering the production of good units that meet product speci-
fications when compared with the total number of produced units.
No: of produced elements no: of defective elements
Qr ¼ ð3Þ
Number of produced elements
3.2.4 Scheduled Maintenance Index (SMI)
This is a percentage indicator of the number of hours invested in scheduled
maintenance.
Hours invested in scheduled maintenance
SMI ¼ ð4Þ
Total number of hours invested in maintenance
3.2.5 Corrective Maintenance Index (CMI)
This is a percentage indicator of the number of hours invested in corrective
maintenance.
784 J. Echevarria-Cahuas et al.
Hours invested in corrective maintenance
CMI ¼ ð5Þ
Total number of hours invested in maintenance
4 Validation
4.1 Case Study
To validate the proposed model, we worked with PUNTO DE DISTRIBUCIÓN, a
medium-sized and packages LPG in 5, 10, 15, and 45 kg cylinders. We try to identify
the hazards and risks in work-related activities carried out in the product storage area
and the packaging platform.
4.2 Starting Situation
After identifying the activities and classifying the risks, we calculated the probabil-
ity severity to assess the risks. These were divided into NON-SIGNIFICANT and
SIGNIFICANT. With the information gathered, we obtained the following results: there
is 1 intolerable hazard—SIGNIFICANT (5.26%)—resulting from the workers being
exposed to cylinders that may have leaks during the airtightness process. Furthermore,
there are six important dangers—SIGNIFICANT (31.58%)—because of LPG reception,
sealing machine handling, transferring LPG to the semi-underground tank, valve han-
dling, workers exposed to cylinders that may have leaks, and to the LPG transfer pipes.
To these hazards, we apply POKA–YOKE, these control measures are presented below.
• LPG leak detector
• Excess flow valve for stationary tanks
• Excess flow valve for filling lines
• Pull – Away valve.
For the moderate hazard, we designed some procedures on the basis of TPM
principles, namely:
• Safety guidelines to operate the filling carousel.
• Procedures to install and disassemble the dynamic weighing scale.
• Operations manual.
4.3 Results
The implemented engineering procedures and measures lead to improvements and
optimizations in the LPG packaging and storage area of PUNTO DE DISTRIBUCIÓN.
The results obtained in this study are detailed below.
– There are 0 intolerable hazards: SIGNIFICANT (0%)
– There are 0 important hazards: SIGNIFICANT (0%)
– There are 15 moderate hazards: NON-SIGNIFICANT (78.95%)
– There are 5 tolerable hazards: NON-SIGNIFICANT (26.32%) (Table 1).
HIRAC-Based Risk Management Model with POKA–YOKE and TPM Continuity 785
Table 1. Indicators table – after the measures
Indicator Calculation by 12/10/18 Calculation by 5/6/18
Overall Equipment Effectiveness (OEE) 76.10% 87.60%
Quality Index (Qr) 95.00% 97.00%
Availability Index (A) 94.00% 96.70%
Scheduled Maintenance Index (SMI) 62.50% 73.50%
Corrective Maintenance Index (CMI) 37.50% 26.50%
Regarding the results obtained, the evaluation carried out following the imple-
mentation of the proposed model showed that the overall effectiveness of the com-
pany’s equipment increased by 11%. This was mainly because the quality and
availability indexes increased by 2% (the faulty cylinders index decreased from 5% to
2%) and 2.7%, respectively. (Downtime decreased). This allows the distribution point
to reach world-class values and to be classified as having good competitiveness
according to the OEE classification. We still have to improve the scheduled and cor-
rective maintenance indexes. Both increased and decreased by 11%.
5 Conclusion
This study achieved the goal established in the research work, namely, controlling the
risks that arise in the multiple emergency scenarios. We first performed a risk analysis,
in which 19 of the said risks were due to component failure and represented a sig-
nificant risk to the workers. This was due to the fact that 53.85% of them were facing
an INTOLERABLE and IMPORTANT risk. Following the mitigation measures, all
risks of this type became nonsignificant, MODERATE with 92.31% and TOLERABLE
with 7.69%, which require administrative measures.
The packaging and storage of LPG pose residual risks with a high potential for
impact on human resources, facilities, and third parties. To control these risks, the
authorities (Osinergmin—MINEM) in charge of approving, monitoring, and super-
vising operations, as well as the LPG packaging and marketing companies, have
policies and criteria to determine the permissible risk levels and propose measures that
make it possible to manage, mitigate, and control them. Furthermore, the complexity of
risk management models makes it necessary to use tools such as HIRAC, POKA–
YOKE, and TPM.
References
1. Osinergmin: Listado de Registros Cancelados y/o suspendidos
2. Cuerpo General de Bomberos: Emergencias atendidas el 2017 por tipo de emergencia (2017)
3. Pomasoncco, A., Trujillo, C., Arauzo, L., Raymundo, C.: Optimized model for pre-cut
blasting in mining operations in underground mining in Peru. In: Proceedings of the IMCIC
2019 - 10th International Multi-Conference on Complexity, Informatics and Cybernetics, vol.
2, pp. 75–80 (2019)
786 J. Echevarria-Cahuas et al.
4. Othman, N., Lerk, L.L., Chelliapan, S., Mohammad, R., Ariff, N.M., Nasri, N.S.: Risk
assessment and control measures for the printing ink production process. Int. J. Civ. Eng.
Technol. 8(10), 41–55 (2017)
5. Rout, B.K., Sikdar, B.K.: Hazard identification, risk assessment, and control measures as an
effective tool of occupational health assessment of hazardous process in an iron ore pelletizing
industry. Indian J. Occup. Environ. Med. 21(3), 144–151 (2017)
6. Saedi, A.M., Thambirajah, J.J., Pariatamby, A.: A HIRARC model for safety and risk
evaluation at a hydroelectric power generation plant. Saf. Sci. 70, 308–315 (2014)
7. Kadir, Z.A., Mohammad, R., Othman, N., Chelliapan, S.: Risk assessment of human risk
factors in port accidents. Int. J. Mech. Eng. Technol. 8(11), 535–551 (2017)
8. Gallab, M., Bouloiz, H., Garbolino, E., Tkiouat, M., ElKilani, M.A., Bureau, N.: Risk
analysis of maintenance activities in a LPG supply chain with a multi-agent approach. J. Loss
Prev. Process Ind. 47, 41–56 (2017)
9. Ahmad, A.A., Abd, A., Freddawati, R., Wong, R., Iqbal, M.: Worker safety improvement at
paper pleating production line using Poka-Yoke concept - a case study in automotive industry.
J. Mech. Eng. Vol. SI 4(5), 183–196 (2017)
Materials and Inclusive Human Systems
CAD-Based Risk Assessment Approach
for Safe Scheduling of HRC Operations
for Parts Produced by Laser Powder Bed
Fusion
Fabio Pini(&), Enrico Dalpadulo, and Francesco Leali
“Enzo Ferrari” Department of Engineering, University of Modena
and Reggio Emilia, via Vivarelli 10/1, 41125 Modena, Italy
{fabio.pini,enrico.dalpadulo,
francesco.leali}@unimore.it
Abstract. The presented paper suggests a design method which seeks to
identify the best scheduling of human robot collaborative (HRC) operations with
respect to a required safety level. The human behavior along manufacturing
scenarios is effectively forecasted through dedicated computer-aided tools.
Consequently, this method stresses the usage of virtual environment to replicate
both human postures and robot encumbrances over the manufacturing opera-
tions. Moreover, it proposes a safety index formulation for HRC systems based
on the minimum distance between human and robot (H-R). As results, the
approach returns the safety index for every possible combination of H-R
operations. Subsequently, a scheduling algorithm suggests the operations
sequence depending on the expected value of the safety index, providing an
evaluation of the time needed to complete the process. The method is validated
on surface control phase involved in post-processing of parts produced by laser
powder bed fusion (L-PBF) Additive Manufacturing.
Keywords: Human Robot Collaboration Safety index Task scheduling
CAD-based methods Additive Manufacturing
1 Introduction
The recent research trends confirm the importance of developing symbiotic solutions
where humans and machines interact during the execution of the process, pointing out
the primary role of Human Robot Collaboration (HRC). HRC systems refer to a new
concept of working environment where humans and robots cooperate to perform
manufacturing task, such as the assembly of components or finishing processes.
In HRC both human and robot (H-R) contribute to reach the production goals coupling
their specific characteristics. The main drawback with HRC is ensuring the operator
safety, since human and robot share the same working space during the execution of
the tasks. The design of HRC workcells is regulated by ISO standards 10218-1 [1] and
10218-2 [2], which define the basic requirements and guidelines for a safe introduction
of HRC in Industry. In particular, they define the actions to be taken, e.g.: stop robot in
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 789–795, 2020.
https://doi.org/10.1007/978-3-030-39512-4_120
790 F. Pini et al.
event of human presence in the robot workspace; enable manual dragging of robot;
monitor continuously robot position and speed; limit power and force of robot axes.
Moreover, ISO 10218-2 imposes a risk analysis to assess both the robot and the
environment in which it is placed. Reconfiguring the overall system or adding safe-
guarding mechanisms can pursue this aim. The Literature addresses the most important
safety issues: safety design of a HRC workcell and its application to assembly tasks [3];
conceptual design of HRC layouts based on a trade-off between safety requirements
and system flexibility [4]; check the safety condition during the collaboration sequence
between the human operator and the robot system [5]; on-line control strategy, which
enforces safety, while maintaining the maximum level of productivity of the robot,
pointing out the impact of the operations sequence on both the safety level and the
production goals, such as the completion time [6]. Safety indexes (SI) evaluate the
danger existing in the interaction between human and robot and these parameters
represent an objective measure of the safety level in HRC solutions. The Literature
presents various indexes with different names and that referrer to different physical
characteristics. Reference [7] describes several safety indexes related to physical
contact between the human operator and the robot, together with their historical evo-
lution. The physical characteristics used for evaluating these indexes are related to
robot speed and acceleration, relative mass and impact force. Two safety indexes
evaluated considering the distance between human operator and robot, and relative
velocities are described in [8]. In particular, the distance is calculated referring to a
specific coordinates system (the ellipsoid coordinates) which allows its rapid evalua-
tion, thanks to the use of approximation. Furthermore, the optimization of robot tra-
jectory employs such indexes. On the other hand, [9] reports the results of a study on
human factors that affect the level of danger involved in a collaborative operation and
the authors mainly focus on the direction of human eye gaze and body orientation.
The present paper presents a design method that seeks for the best scheduling of H-
R operations, according to a fixed safety level. The human behavior along manufac-
turing scenarios is effectively forecasted through dedicated computer aided tools, as
proved by [10] and [11]. This method stresses the usage of virtual environment to
replicate both human postures and robot encumbrances over the manufacturing oper-
ations. Moreover, it proposes a safety index formulation for HRC systems based on the
minimum H-R distance that is returned by the truthful digital representation of human
body and robot dimensions. As a results, the approach returns the safety index for every
possible combination of H-R operations. Subsequently, a scheduling algorithm sug-
gests the operations sequence depending on the expected value of the safety index,
providing an evaluation of the time needed to complete the process.
A functional validation of this approach is based on a HRC surface finishing
process of parts produced by Additive Manufacturing (AM) processes, which requires
a working space sharing between human and robot. After product and process opti-
mization and part printing, post-processing operations [12] such as job cutting, supports
removal and surface finishing to match both aesthetic (stair-step, support marks,
powder adhesion) and performance requirements (couplings, fatigue properties) are
necessary. The paper is structured as follows. The next section presents the method for
the task scheduling based on the safety index and the scheduling algorithm proposed by
the Authors, in order to return the optimal sequence of the operations. Section 3
CAD-Based Risk Assessment Approach for Safe Scheduling of HRC Operations 791
explains the selected test case and its results. Conclusive remarks and future devel-
opments are presented in the last section.
2 CAD-Based Safety Analysis Approach
In the previous works [13] and [14], the Authors defined a systematic approach to the
engineering design of a HRC workcell, which identifies four design steps as follows.
Task Clarification is the first step of the design process, aiming to clear the definition of
the product, its related process and resources. Task Assignment is the main step of the
design approach since it allocates the tasks among humans, robots and other resources.
For pursuing this aim, the Authors suggested a qualitative allocation approach based on
four evaluation criteria. The Operative Level, OpL, evaluates the impact of each
operation on the manufacturing goals, e.g. the expected quality. The Hazardous Level,
HzL, evaluates the risks associated with the task in terms of operator safety and
operational sequence safety, e.g. the interaction with semi-automatic machines. The
Sensibility Level, StL, and the Fatigue Level, FtL, respectively evaluate the accuracy
and sensitivity required for each operation, and the ergonomics. Task Planning is the
third design step, aiming at definition of the HRC process according to product
manufacturing process constrains and production rate. Finally, HRC Engineering aims
to define the embodiment of the HRC solution.
This paper extends the HRC design method introducing an objective approach for
the Task Planning design step, hereinafter called Safety-Based Task Scheduling, SBTS.
The SBST is dedicated to the scheduling of the collaborative tasks based on the balance
between safety constraints and production goals, and exploits the virtual environment
to simulate human and robot behaviors to calculate the SI along the process operations.
As in Fig. 1, the Analysis stage is the first step of the STBS, which consist of Product
Analysis, Process Analysis and Resources Definition. The Product Analysis describes
and returns a discrete representation of the product in simple sub-zones, such as
specified surfaces or features with similar geometrical characteristic. Surface curvature
or roughness requirements are example of criteria used to identify the product sub-
zones. Then, the Process Analysis on each product sub-zones defines the operative
strategy required to reach the final quality; for example, the designer selects the human
or the robot and defines the work movements and their possible constrains. Finally, the
Resources Definition identifies the tools required to perform the operations on the sub-
zones. The process parameters will depend on the selected resources, e.g. the move-
ment speed of a tool will be related to the type of the selected tool.
The simplification of the product in sub-zones is a precondition for the second step
of SBTS, namely Simulation stage, which aims to evaluate the SI for the HRC
application. The process is simulated through digital environment, hence the product
and the selected resources require a representation according to their real dimensions.
For each pair of sub-zones of the analyzed combination, a joint simulation of both
human and robot respective operations is performed. Exploiting virtual representation
of human body and robot dimensions, the SI is evaluated through a simulation of
process operations as the distance between envelops of the human and robot relative
positions and the tools involved. The SI is calculated following (1) as the H-R distance
792 F. Pini et al.
Fig. 1. The main steps of the proposed scheduling method.
lowest value, during the execution of a pair of operations (OPH,i, OPR,j) from the
^
operator or robot, and normalized to the characteristic distance of the application L,
identified as the maximum distance among the centroids of all pairs of sub-zones.
min distðpH;i ; pR;j Þ
time
SIðOPH;i ; OPR;j Þ ¼ : ð1Þ
^
L
Last step of the SBTS is the Task Scheduling, which accounts the data generated in
the Simulation stage through a scheduling algorithm, to provide the trend of SI com-
pared to operations sequence. The algorithm can be applied to HRC processes where at
least two different operations are required for the same sub-zone, e.g. the execution of
measurement operations followed by work operations, such as assembly of components
or surfaces finishing. Data inputs for the algorithm are the number of sub-zones which
need to be worked, the starting sub-zone, the relative distance among sub-zones, the SI
for the H-R operations pairs, the vectors of the execution times pertinent to operations
regarding the robot and the operator, and the expected SI threshold, SIT. The algorithm
assumes robotized operations as the first to be executed, considering the minimum
distance among the working areas as scheduling rule. The operations by the human will
be executed once the SI related to the sub-zones worked by the robot will reach a value
higher than the fixed SIT. Moreover, during the execution of collaborative operations,
the SIT operates recognizing the subsequent area assigned to the operator and the robot,
comparing the mutual SI. The output of the algorithm consists in the temporal
scheduling of operations, achieved once the number of areas to be worked by operator
and robot is completed.
3 Test Case and Results
The test case for the preliminary functional evaluation of the proposed method refers to
a surface finishing process. The use of HRC solutions for this kind of processes is of
considerable interest especially for Quality Assessment, QA, and Final Finishing, FF,
operations. During the QA phase, the robot repeatability is exploited to perform the
quality control using the surface measurement system CWS. The operator, which is
naturally endowed with high sensitivity and adaptability, performs the FF; for this
CAD-Based Risk Assessment Approach for Safe Scheduling of HRC Operations 793
stage, the operator uses bead blasting, dry sanding and hand polishing. These two
operations do not present dangerous circumstances and can be performed in the same
time. In order to perform the QA phase, and the subsequent FF, 18 functional features
with similar geometric characteristics are split from the surface of an automotive
prototyping part. For all the possible combinations of surface pairs, the distances
between the references centroids are identified and, then, the simulation of the robotic
measurement process and the manual finishing process is performed, considering that
the robot can control the surfaces at a speed of 100 mm2/s while the operator performs
the finish at a speed of 10 mm2/s. Process simulation is performed employing the
digital manufacturing platform DELMIA V5 which integrates the CAD modelling tool
CATIA and gives packages for human and robot simulation.
Figure 2 shows two examples of scheduling obtained by setting the extreme values
for SIT, respectively 0 and 1 as returned by (1). The upper part of Fig. 2 illustrates
parallel execution of the operations, which occurs when the SIT has a minimum value
(0 in this case), and the only constrain is the completion of the robot task. Conversely,
when the SIT reaches the maximum value (1 in this case), the human must wait until the
robot has completed its operations and therefore the process becomes sequential, as in
the lower part of Fig. 2. In this case, the distance between zones drives the order of
processing. The total execution time increases depending on the security level; in
particular, the process time is maximum at the highest value of SIT. Another interesting
aspect is the opportunity to choose a different starting area for the robot. In particular,
as the SIT decreases, there is a starting area better than others that allows minimizing
these distances. In other words, this distance also provides a regularity measure of
scheduling, in addition to the operator and the robot workload.
Fig. 2. Effect of SI thresholds on the operations scheduling; on the left (a), parallel H-R
scheduling related SIT = 0; on the right (b), sequential scheduling related to SIT = 1.
4 Conclusion
Safety in H-R collaboration is analysed by the use of SI, which measures the hazards
for the operator with respect to relative distance, relative velocity or masses. Such
indexes could be used as design parameters to identify a priori task scheduling.
Moreover, the present paper proposes a method for obtain a schedule of the operations
in HRC application based on safety algorithm which identify the sequence of opera-
tions execution that ensures the appropriate security level. The so called SBTS method
794 F. Pini et al.
involves the identification of process sub-areas to be processed as well as the definition
of the process, using the virtual environment to simulate the process and to return the SI
for each sub-zones pair to be worked, based on the minimum distance H-R during the
execution of the various tasks. The validation of the proposed method is carried out
through simulation of surface finishing process of an automotive AM part, which may
require to share the working area between human and robot. The results show the
dependence of total process time, and then productivity, on the SIT.
To validate the results returned by the simulation, the test will be repeated within an
experimental setup which integrate systems for human posture recognition and H-R
distance monitoring. Moreover, it is plan to enrich the formulation of SI by including
the parameters which consider both the speeds of human and robot, as defined by the
safety standards for HRC applications.
References
1. ISO 10218-1:2011 Robots and robotic devices - Safety requirements for industrial robots -
Part 1: Robots
2. ISO 10218-2:2011 Robots and robotic devices - Safety requirements for industrial robots -
Part 2: Robot systems and integration
3. Hu, J.-J., Huang C.-N., Wang H.-W., Shieh P.-H., Hu J.-S.: Safety-based human-robot
collaboration in cellular manufacturing: a case study of power protector assembly. In:
Proceedings of the ARIS, Tainan, TW, pp. 28–31 (2013)
4. Olsen, R., Johansen, K.: Assembly cell concept for human and robot in cooperation. In: 22nd
ICPR, Parana, BR (2013)
5. Tan, J.T.C., Duan, F., Zhang, Y., Watanabe, K., Kato, R., Arai, T.: Human-robot
collaboration in cellular manufacturing: design and development. In: 2009 IEEE/RSJ ICIRS,
St. Louis, MO, USA, pp. 29–34 (2009)
6. Zanchettin, A.M., Ceriani, N.M., Rocco, P., Ding, H., Matthias, B.: Safety in human-robot
collaborative manufacturing environments: metrics and control. IEEE Trans. Autom. Sci.
Eng. 13(2), 882–893 (2016)
7. Cordero, C.A., Carbone, G., Ceccarelli, M., Echávarri, J., Muñoz, J.L.: Experimental tests in
human–robot collision evaluation and characterization of a new safety index for robot
operation. Mech. Mach. Theory 80, 184–199 (2014)
8. Tsai, C.-S., Hu, J.-S., Tomizuka M.: Ensuring safety in human-robot coexistence
environment. In: IEEE ICIRS, Chicago, IL, USA, pp. 4191–4196 (2014)
9. Najmaei, N., Lele, S., Kermani, M.R., Sobot, R.: Human factors for robot safety assessment.
In: Proceedings of AIM, Montreal, QC, CA, pp. 539–544 (2010)
10. Zhou, D., Chen, J., Lv, C., Cao, Q.: A method for integrating ergonomics analysis into
maintainability design in a virtual environment. Int. J. Ind. Ergonom. 54, 154–163 (2016)
11. Naddeo, A., Cappetti, N., D’Oria, C.: Proposal of a new quantitative method for postural
comfort evaluation. Int. J. Ind. Ergonom. 48, 25–35 (2015)
12. Dalpadulo, E., Pini, F., Leali, F.: Assessment of design for additive manufacturing based on
CAD platforms. In: Rizzi, C., Andrisano, A.O., Leali, F., Gherardini, F., Pini, F., Vergnano,
A. (eds.) Design Tools and Methods in Industrial Engineering. LNME, pp. 970–981.
Springer, Cham (2020)
CAD-Based Risk Assessment Approach for Safe Scheduling of HRC Operations 795
13. Pini, F., Leali, F., Ansaloni, M.: A systematic approach to the engineering design of a HRC
workcell for bio-medical product assembly. In: Proceedings of the ETFA, Luxembourg
(2015)
14. Pini, F., Ansaloni, M., Leali, F.: Evaluation of operator relief for an effective design of HRC
workcells. In: Proceedings of the ETFA, Berlin (2016)
Photogrammetry and Additive Manufacturing
Based Methodology for Decentralized Spare
Part Production in Automotive Industry
Antonio Bacciaglia(&), Alessandro Ceruti, and Alfredo Liverani
Department of Industrial Engineering – DIN, University of Bologna,
Viale del Risorgimento 2, Bologna, Italy
{antonio.bacciaglia2,alessandro.ceruti,
alfredo.liverani}@unibo.it
Abstract. Additive Manufacturing is becoming a suitable production process
for many industries: it is based on the idea of adding material layer by layer, in
opposite to traditional manufacturing processes. This technology shows
advantages as design flexibility, internal logistics minimization and product
customization that make it perfect to produce customized parts and all the
applications where low production rates occur. The production of spare parts for
classic or luxury cars is a field where Additive Manufacturing can be adopted
because of low demand and relevant costs to manage stocks keeping several
different parts in the after-sales inventory. The photogrammetry technique has
been investigated to obtain the 3D model of the component to be replaced and
send it to decentralized production centers equipped with 3D printers. This
approach can enhance by far the supply chain management for automotive spare
parts.
Keywords: Additive Manufacturing Photogrammetry Maintenance
Automotive Supply chain management
1 Introduction
A significant growth of Additive Manufacturing (AM) technology can been noticed in
the last few years due to its advantages over traditional processes. AM is based on the
idea of adding material layer by layer in opposition to the traditional production pro-
cesses. It shows several advantages such as: capability to produce complex shapes in a
single piece thus reducing the connections, weight saving that reflects on lower
emissions and time saving in the design-to-manufacturing cycle, just to mention the
main significant [1]. On the same wave of enthusiasm, several research contributions
are focused on this technology and on the whole range of aspects connected to this
manufacturing way, as well as on the employment of AM in the supply chain for
industrial applications (i.e. automotive sector).
Even if in automotive industry the production capacities are very high compared to
other sectors (i.e. aerospace or medicine), AM products are increasing year by year and
in 2013 automotive sector accounted for 17% of all expenditures in AM, specifically
for special cars, racing cars and all the application where there is a low volume
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 796–802, 2020.
https://doi.org/10.1007/978-3-030-39512-4_121
Photogrammetry and Additive Manufacturing Based Methodology 797
production [1]. If the production volumes are high, AM is currently too much
expensive in the assembly lines in terms of raw material, machine price and production
rate: this is because CNC tooling price becomes significant when production rates are
small, but they are well recovered when lots are numerous. On the other hand, when
production lots are small, the need for tooling, CNC programs or casting technologies
is more expensive than AM.
The Reverse Engineering (RE) acronym is used to describe the bunch of techniques
used to obtain the digital 3D model of a component. RE has been sometimes coupled
with AM, where it is used to reconstruct the 3D model of an existent part to print. RE
has been developed to understand how a component works, what is its geometry
shaped, how it can be used, the production costs and the detection of defects [2]. RE
algorithms are used to obtain a cloud of points fitting the surface of the body. Later on,
a surface mesh and a solid 3D model of the desired object can be obtained. There are
several RE techniques which can be divided in touching methods, more expensive but
more accurate, and in optical methods, more affordable but less accurate. Among the
optical methods, photogrammetry is one of the simplest and cheap methodology.
Indeed, the scope of this paper is to evaluate the feasibility of using AM to produce
spare parts in the automotive sector with a decentralized supply-chain environment, by
using RE techniques to reconstruct the 3D model.
2 Additive Manufacturing in Automotive
There are different types of AM technologies, each one showing pros and cons, but
laser-based metal processes are the most used in automotive to make structural com-
ponents with strong and stiff characteristics where lightweight design strategies, such as
topology optimization, can be adopted to improve the components geometry [3].
The use of AM in automotive industry brings many advantages, among which an
acceleration of the design-to-manufacturing cycle, a reduction of manufacturing
restraints and internal logistics, high customization and design flexibility. Fused
Deposition Modelling (FDM) machines, producing components in ABS or PLA plastic
materials, have been already used to produce secondary automotive components such
as wind tunnel models, mock-up models, non-structural components and assemblies for
connections reduction testing. Metallic AM gives the opportunity for the designers to
adopt lattice structures and/or design topologically optimized structures allowing
reduced material waste and a reduction of weights. However, the AM main applications
for ready-for-the-market products are limited to luxury, racing and classic cars, while
nowadays, it is far from the adoption in the mass production because of costs and
production time.
As a matter of fact, some key issues limit the adoption of AM in the mass pro-
duction supply chain: the slow certification process (when needed) caused by the high
variability in parts properties because of material lots with different mechanical
properties, powders ageing, changes in performances with sight modify of the machine
settings, slow build rates compared to CNC machines, limited component size and
limited material portfolio, anisotropic material and surface roughness (which affect
fatigue behaviour), CAD software limitations in the 3D modelling of complex
798 A. Bacciaglia et al.
geometries, problems of inspection and maintenance of complex assembly in one
single part, and this is not an inclusive list.
2.1 Spare Parts Production in Automotive
The supply chain is a complex environment made by people, machines, information,
and resources involved in promoting manufactured goods from provider to customer
[4]. Supply chain involves the transformation of primary resources, raw materials, and
components into a ready-for-the-market product that is delivered to the end customer.
In the automotive industry, the spare part supply chain gained a lot of interest and it
has been analyzed in literature [5, 6]. This because car companies must support the
former generation of products at the same level as the new ones. In addition, the
number of components required in cars and motorbikes is extremely high (usually
hundreds of thousand parts). Moreover, the spare part demand is highly heterogenous
[7], magnifying the number of shelf keeping parts in after-sales inventories, which
reflects on extremely high costs for automotive companies and large space required [8].
In this complex contest, AM could dramatically improve the spare parts supply
chain, thus reducing the need for large stocks. The core idea is that spare parts could be
manufactured only when necessary, storing the component information in a 3D CAD
model. However, an initial capital investment is needed to buy a 3D printing factory
and several security issues are related to the handling of digital data.
The adoption of AM machines to produce spare parts in applications such as
luxury, racing or historical cars which are characterized by a low production volume
can be used by the research community as a case study useful to understand which of
centralized or in-situ production [9] offers better performances in terms of design-to-
manufacturing timing and logistic costs. However, it is quite undoubtedly that a hybrid
solution could be the best option in case of heterogeneous spare part demand: this
configuration is based on AM machines installed at the central distribution center and at
local service stations, and it is particularly efficient when the spare part frequency
demand and the technology development is at a certain level [7].
2.2 Digital Manufacturing Process
AM perfectly fits acceleration and flexibility of the modern design-to-manufacturing
cycle, producing customized parts, minimizing the material waste if coupled with CAD
tools, thus implementing what is nowadays called Digital Manufacturing. This idea
highlights the integration of manufacturing processes and design tools, centred around
a computer, that sometimes is associated to a 3rd Industrial Revolution [10]. When data
exchange between machines, big data from delocalized sensors, RE, augmented reality
and high automation concept are added in the manufacturing, Industry 4.0 strategy is
followed and the 4th industrial revolution approach is applied.
Thanks to AM-CAD coupling, the design process can be split in three phases:
(1) Design, (2) Manufacturing and (3) Post-processing. Focusing on the design stage,
the spare part 3D model that have to be manufactured can be obtained as a new shape
with CAD tools, sketching it feature by feature or by RE techniques, where a cloud of
Photogrammetry and Additive Manufacturing Based Methodology 799
points fitting the surface is obtained and it can be translated into a surface mesh and
finally in a solid 3D model.
In this paper the idea of producing automotive spare parts through AM with the use
of photogrammetry technique to create the spare part 3D model is evaluated consid-
ering nowadays technologies and the increasing of computational power available both
in terms of workstations and smartphones.
3 Photogrammetry
Photogrammetry is a RE technique quite affordable and easy to be implemented even in
smartphones (e.g. 3D Scanner Pro, nowadays available for Android smartphones).
The digital and automated photogrammetry is a process that can create 3D models
starting from a series of snapshots of an object using a high-resolution camera, framing
the body to reconstruct it from different points of view. The mathematical discussion
behind this method is far from the scope of this paper, but for more information the
reader can refer to [11]. Thanks to the affordable prices of the hardware, software, and
the method easiness, this technique is used in several sectors of the industrial field, in
medicine and in archaeology. Many software tools, several of which even free can be
adopted to obtain a mesh.
The photogrammetry process is divided in: (1) image matching for each snapshot
using some points that are identified as recurring in other snapshots. These points are
then saved and used to link all the images meanwhile the snapshot orientation is
evaluated; (2) triangulation to evaluate the coordinates of the previously cited points;
(3) creation of the cloud of interesting points by materializing them in the space;
(4) density increase of cloud of points; (5) surface mesh creation.
4 Conceptual Supply Chain Layout Proposal
In a near future, when smartphones camera will reach a higher level of digital quality, a
customer needing a spare part, could contact a local service distributed in the territory
and ask to produce the component. If the corresponding 3D model will be already
available in the on-demand stock, the part could be 3D printed and sent to the customer
in a fast and easy way.
On the other hand, if the component will not be present in the on-demand stock,
maybe because belonging to an old product, the local service could ask the customer to
capture several snapshots from different angles of the component using a smartphone
and send these to the local service. Thanks to photogrammetry algorithms the 3D
model could be reconstructed, sent to an AM machine, manufactured and sent to the
final costumer. As a matter of fact, consistent 3D model reconstruction can be carried
out nowadays only by taking pictures with high definition camera and using software
tools with strong human intervention.
Other optical RE techniques, such as 3D laser scanners could be implemented in the
proposed layout to increase the overall 3D model accuracy. However, as [12] states,
laser scanners do not reach extremely higher level of accuracy with respect
800 A. Bacciaglia et al.
photogrammetry (max deviations up to 1 mm was evaluated in a 3D model comparison
between laser scanners and tactile measuring system, which is the most accurate RE
method). Moreover, laser scanners are more expensive and bulkier compared to a high-
resolution digital camera for the end-user. Moreover, use of a laser scanner requires a
trained operator.
Thanks to increasing computational power and camera performances installed on
smartphones, photogrammetry could represent a user-friendly, fast and accurate way to
obtain accurate 3D models.
5 Case Study
An automotive component has been selected and it has been reproduced with pho-
togrammetry technology to evaluate the feasibility and the precision which could be
expected with nowadays tool. Figure 1a shows a picture of an engine sump. It has been
framed 66 times by different point of view using a Nikon d3500 digital camera, each
picture with a resolution 6000 4000 pixels. All the images have been imported in the
Autodesk Recap Photo tool (Fig. 1b) and in Zephyr 3DF tool (Fig. 1c), with a resulting
accuracy that is higher with the first one. An .obj surface mesh has been obtained: a
scaling operation has been carried out based on a length of the body (Fig. 2a). Finally,
the body has been imported in MeshLab to smooth and fix the mesh (Fig. 2b). A new
STL file has been obtained and imported in a 3D printing machine to evaluate the
feasibility of the AM. A maximum error of 1.133 mm is noticed (1.86%) in the 3D
digital model respect to real part in a region selected for benchmarking tests.
Fig. 1. (a) Real component geometry is captured using a digital camera; (b) dense cloud point in
Autodesk Recap Photo; (c) dense cloud point in Zephyr 3DF
Photogrammetry and Additive Manufacturing Based Methodology 801
Fig. 2. (a) Scaling the component by a known dimension; (b) surface mesh smoothing in
MeshLab; (c) STL file imported in CURA ready to be 3D printed
6 Conclusions and Future Developments
In this work, a part supply chain layout for automotive industry based on AM and
photogrammetry technologies has been evaluated in a preliminarily way. Using
nowadays photogrammetry technology quite large errors in body 3D reconstruction are
noticed: this implies that feature that needs precision for couplings or interference need
to be fixed, by sketching it in a CAD with human intervention. No information about
GD&T can be retrieved from photogrammetry: expert designers’ intervention to set
tolerances in couplings is required. Future advances in digital equipment, software
tools and hardware computational power will probably reduce the errors in 3D
reconstruction, making it useful for a wide number of parts.
References
1. Ian, G., Rosen, D., Stucker, B.: Additive Manufacturing Technologies. Springer, New York
(2015)
2. Eilam, E.: Reversing: Secrets of Reverse Engineering. Wiley, Hoboken (2011)
3. Bacciaglia, A., Ceruti, A., Liverani, A.: Additive manufacturing challenges and future
developments in the next ten years. In: ADM 2019 International Conference, pp. 891–902.
Springer (2019)
4. Kozlenkova, I., Hult, T., Lund, D., Mena, J., Kekec, P.: The role of marketing channels in
supply chain management. J. Retail. 91(4), 586–609 (2015)
5. Holmström, J., Partanen, J., Tuomi, J., Walter, M.: Rapid manufacturing in the spare parts
supply chain: alternative approaches to capacity deployment. J. Manuf. Technol. Manag. 21,
687–697 (2010)
6. Liu, P., Huang, S., Mokasdar, A., Zhou, H., Liang, H.: The impact of additive manufacturing
in the aircraft spare parts supply chain: supply chain operation reference (Scor) model-based
analysis. Prod. Plan. Control 25, 1169–1181 (2014)
7. Li, Y., Cheng, Y., Hu, Q., Zhou, S., Ma, L., Lim, M.K.: The influence of additive
manufacturing on the configuration of make-to-order spare parts supply chain under
heterogeneous demand. Int. J. Prod. Res. 57, 1–20 (2018)
802 A. Bacciaglia et al.
8. Khajavi, S.H., Partanen, J., Holmström, J.: Additive manufacturing in the spare parts supply
chain. Comput. Ind. 65(1), 50–63 (2014)
9. Pérès, F., Noyes, D.: Envisioning e-logistics developments: making spare parts in situ and on
demand: state of the art and guidelines for future developments. Comput. Ind. 57(6), 490–
503 (2006)
10. Savastano, M., Amendola, C., D’Ascenzo, F., Massaroni, E.: 3-D printing in the spare parts
supply chain: an explorative study in the automotive industry. Digit. Support. Innov. 18,
153–170 (2016)
11. Schindler, K.: Mathematical Foundations of Photogrammetry, Photogrammetry and Remote
Sensing. ETH Zurich, Switzerland
12. Giannelis, A., Symeonidou, I., Tzetzis, D., Kaisarlis, G., Kyratsis, P.: Laser scanning and
CAD conversion accuracy correction of a highly curved engineering component using a
precision tactile measuring system. In: MATEC Web of Conferences, vol. 112, p. 05009
(2017)
Improved Heat Sink for Thermoelectric
Energy Harvesting Systems
Alessandro Bertacchini1,2(&), Silvia Barbi1, and Monia Montorsi1,2
1
DISMI – Department of Sciences and Methods for Engineering,
University of Modena and Reggio Emilia, 42122 Reggio Emilia, Italy
{alessandro.bertacchini,silvia.barbi,
monia.montorsi}@unimore.it
2
EN&TECH – Interdepartmental Center for Industrial Research
and Technology Transfer in the Field of Integrated Technologies for Sustainable
Research, Efficient Energy Conversion, Energy Efficiency of Buildings,
Lighting and Home Automation, University of Modena and Reggio Emilia,
42122 Reggio Emilia, Italy
Abstract. This paper is focused on the improvement of the heat sink system of
a thermal energy harvester by using organic and inorganic Phase Change
Materials (PCM), considering solutions with the lowest environmental impact,
as an alternative to well-known paraffin waxes. The proposed heat sink has been
applied to the real case study of a thermoelectric energy harvester to supply a
wireless sensor node placed in the engine compartment of a vehicle and used for
the continuous monitoring of the engine’s vibrations. Different hydrated salts
(nitrate mixtures) have been evaluated as PCM, trough design of experiments
techniques, working in a restrained well-tailored temperature ranges (±5 °C)
depending on mixtures’ composition, and compared with acetamide as best
organic alternative. The realized PCM-based heat sink shown better perfor-
mance compared with a classic heat sink in both steady-state and dynamic
conditions.
Keywords: Phase Change Materials Thermoelectric generators Thermal
energy harvesting
1 Introduction
Thermoelectric Energy Harvesting (TEH) systems are based on Thermo-Electric
Generators (TEG) exploiting the well-known Seebeck effect, where larger is the
thermal gradient between the hot and the cold side of the TEG, larger is the amount of
energy harvested [1]. Classic TEGs are comprised of two main components. First, the
energy generator, made of a large number of solid-state thermocouples (e.g. Bismuth
Telluride semiconductor materials and thermally conductive Aluminum Oxide
ceramics) connected electrically in series and thermally in parallel, sandwiched
between a hot and a cold plate. The hot plate is connected to the thermal source, while
the cold plate is attached to a heat sink system, used to drain the heat from the hot side
of the TEG keeping the thermal gradient as large as possible.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 803–809, 2020.
https://doi.org/10.1007/978-3-030-39512-4_122
804 A. Bertacchini et al.
Most of the works presented in the recent literature are focused on the development
of new combinations of materials to obtain more efficient thermocouples. In this paper,
we focused on the improvement of the heat sink system by using organic and inorganic
Phase Change Materials (PCM), considering a sustainable alternative to well-known
paraffin waxes from the environmental point of view.
Binary mixtures of inorganic salts (i.e. nitrate, chloride and sulphate salts of alkali
and alkaline metals, such as magnesium, potassium, lithium, and calcium) have been
widely studied for thermal storage applications due to their peculiar chemical bonding
(i.e. ion-dipole bonds, H-H bonds) with H2O molecules [2]. Latent heat is absorbed
when the salt hydrates lose limited part or all of the water molecules. This occurs in the
case of solid-liquid transformation. In addition, salt hydrates have been found to be
non-toxic in nature [3]. Differently from organic PCM, hydrated salts present lower
environmental impact and higher thermal conductivity, nevertheless their lower latent
heat capacity.
In this context, binary mixtures of inorganic salts have been investigated in this
study and the melting point has been properly modulated by using a mixture design
approach (Design of Experiments – DoE) [4]. In particular, for first, a material’s
selection has been done and thereafter magnesium chloride hexahydrate has been
chosen as additive to magnesium nitrate hexahydrate to modulate the mixture melting
point. Finally, to validate the proposed solution with improved heat sink system, the
performance of a complete thermoelectric energy harvesting system ad-hoc designed
for the considered case study of a wireless sensor placed in the cars’ engine com-
partment have been measured. As described in detail in the following, the performance
of the whole system using the best mixture from the mixture design study have been
compared with the performance of the system using acetamide as organic PCM.
2 Materials and Methods
Materials’ selection was done through literature data analysis taking into account
different criteria as reported in Table 1. In particular, melting temperature must agree
the requirements of the tailored purpose of this study that indicate the critical tem-
perature of 85 ± 5 °C as the temperature near the car engine. This value has been
obtained from measurements carried out in the engine compartment of a real vehicle in
different driving conditions (i.e. urban road, extra-urban road and mixed path).
Among the investigated materials Mg(NO3)2 6H2O, MgCl2 6H2O and acetamide
were selected considering their melting point and possible eutectic inorganic mixture
formation. Since Mg(NO3)2 6H2O has a relatively high melting point (89 °C), its
application as a thermal storage material is very limited, therefore eutectic mixtures
with MgCl2 6H2O were studied.
The Design Expert 8.0 (Stat-Ease) code was used both to set-up the experimental
plan and to analyze the results. A two-component mixture design (MgCl2 6H2O 0-20
wt%) was selected to obtain predictive reliability on the mixture composition on the
relative melting point and latent heat.
Three replicates for each experiment were done for error estimation purposes.
ANOVA was employed to estimate the influence of each factor over the responses
Improved Heat Sink for Thermoelectric Energy Harvesting Systems 805
Table 1. Material’s selection criteria and candidates
Candidate Melting temperature Latent heat storage Environmental
(°C) (kJ/Kg) impact
Paraffin wax 54 184 High
Acetamide 82 263 Medium
Stearic acid 64 196 Medium
Naphthalene 80 147 High
[MgNO3]2 89 150 Low
6H2O
MgCl2 6H2O 117 168 Low
Na2P2O7 70 184 Low
10H2O
observing the p-values (a-level of 0.005) and F-tests [4, 5]. Both the quality of the fit in
terms of regression analysis, and the predictive power of the model were evaluated by
using the R2, that is the proportion of the dependent variable’s variance predictable
from the independent variables. Finally, the overall desirability function was set up to
provides the most desirable response’s values, taking into consideration both the
responses investigated (melting point and enthalpy) once and to achieve the best
mixture formulation [6].
3 Measurements Setup
To validate the solution proposed by the DoE study, an ad-hoc test bench has been
realized. The experimental setup is sketched in Fig. 1 left. In particular, the heat sink is
realized using a 20 20 20 mm aluminum box to contain PCM. The closure cap is
made of aluminum with a thickness of 2 mm. The heat sink is placed over a small
15 15 mm commercial TEG from Laird Technology (CP10,31,05), heated through a
temperature-controlled heating plate simulating the car’s engine temperature. The
output terminals of the TEG have been connected to the input of the Energy Harvesting
(EH) circuit realized to store and manage the energy generated by the TEG (Fig. 1
right).
An equivalent resistive load has been connected to the output of the circuit sim-
ulating the power consumption of the real sensor node used for continuous monitoring
of the engine’s vibrations. The output voltage generated by the TEG and the operating
temperature of the heating plate have been acquired with a digital multimeter (U1233A,
Agilent Technologies) with dedicated acquisition software. The output voltage of the
EH circuit, instead, has been acquired by a digital oscilloscope (DSO-X 3034A,
Agilent Technologies).
In addition, repeated melting and solidification tests were conducted in order to
verify the repeatability of the PCM’s behavior.
806 A. Bertacchini et al.
Fig. 1. Simplified block diagram of the realized measurement setup (left). Energy harvesting
and power management circuit realized to perform the tests, with the equivalent resistive load
connected to the output simulating the power consumption of the real sensor node (right).
4 Results and Discussion
About the material’s selection and mixture design analysis both the model for melting
temperature and latent heat are significant with R2 above 0.85, therefore they can be
employed to calculate a statistical reliable desirability function. As reported in Fig. 2,
the best mixture is the one containing 7 wt% of MgCl2 6H2O with a desirability value
equal to 0.948, therefore very near to 1, that is the highest theoretical value possible.
This result ensures good agreement with both the models generated from the evaluated
response.
Fig. 2. Desirability plot for two-component mixture design.
An example of results obtained during a test simulating a complete real working
cycle of the considered case study (i.e. car’s engine turn on, followed by a time interval
where the engine reaches the proper steady-state temperature, and finally car’s engine
Improved Heat Sink for Thermoelectric Energy Harvesting Systems 807
turn off), is shown in Fig. 3. In this case, it is possible to note as the proposed PCM-
based heat sink presents better performances compared with a traditional aluminum-
based heat sink obtained by using an empty aluminum box of the same dimension of
the one used to contain the PCM. In fact, Fig. 3 shows as, thanks to the optimum
mixture obtained from the DoE, the proposed heat sink allowing generating larger
TEG’s output voltage, VTEG, during the whole working cycle.
Fig. 3. TEG output voltage during a complete working cycle. Compared with traditional
aluminum-based heat sink, the proposed PCM-based heat sink presents better performances.
Finally, Fig. 4 shows an example of comparison between tests performed by using
as PCM for the proposed heat sink system the optimum hydrate salts mixture obtained
with the DoE and acetamide. The tests have been carried out on the whole TEH system
comprised of the improved heat sink, TEG, energy harvesting circuit with an output
capacitor for the energy storage and the output equivalent resistive load to emulate the
sensor node. In particular, Fig. 4 reports the transient of the output voltage of the
realized TEH system to reach the proper stable operating voltage of the sensor node,
i.e. 3 V, when TPlate = 85 °C to simulate the temperature of a car’s engine in real
working conditions.
Shorter is the time needed to reach VOut_EH = 3 V, and higher is the performance of
the whole TEH, because it means that the TEG generates a larger amount of energy.
808 A. Bertacchini et al.
Fig. 4. Transient of the output voltage of the realized TEH system starting from the initial
condition of output capacitor fully discharged, when Tengine = 85 °C.
5 Conclusion
A PCM-based heat sink system for thermoelectric energy harvesting applications has
been presented in this paper. The experimental results confirm the effectiveness of the
proposed solution by showing as this solution allowing obtaining larger output voltage
generated by the TEG in both steady-state and dynamic conditions compared with
classic metallic heat sinks of the same dimensions. Of course, in order to obtain the best
performances, the PCM has to be opportunely chosen accordingly with the expected
working temperatures of the specific application.
Moreover, the results shown that the selected hydrate salts mixture, obtained by the
DoE carried out, performs better than classic PCM materials like acetamide, as
demonstrated by the shorter time needed by the whole TEH system to reach and keep
the proper operating voltage of the load (i.e. sensor node). In this way, the selected
hydrate salts mixture becomes also a good candidate as greener PCM suitable for a
wide range of industrial applications.
Acknowledgments. The authors would like to thank ASK industries S.p.A. for financial support
and technical assistance. This work was supported by the Italian Ministry of Economic Devel-
opment (MISE)’s FUND FOR THE SUSTAINABLE GROWTH (F.C.S) under grant agreement
(CUP) B48I15000130008, project VASM (“Vehicle Active Sound Management”).
Improved Heat Sink for Thermoelectric Energy Harvesting Systems 809
References
1. Verma, G., Sharma, V.: A novel thermoelectric energy harvester for wireless sensor network
application. IEEE Trans. Ind. Electron. 66, 3530–3538 (2019). https://doi.org/10.1109/TIE.
2018.2863190
2. Mohamed, S.A., Al-Sulaiman, F.A., Ibrahim, N.I., Zahir, M.H., Al-Ahmed, A., Saidur, R.,
Yilbas, B.S., Sahin, A.Z.: A review on current status and challenges of inorganic phase
change materials for thermal energy storage systems. Renew. Sustain. Energy Rev. 70, 1072–
1089 (2017). https://doi.org/10.1016/j.rser.2016.12.012
3. Chandel, S.S., Agarwal, T.: Review of current state of research on energy storage, toxicity,
health hazards and commercialization of phase changing materials. Renew. Sustain. Energy
Rev. 67, 581–596 (2017). https://doi.org/10.1016/j.rser.2016.09.070
4. Montgomery, D.C.: Design and Analysis of Experiments, 8th edn. (2012). https://doi.org/10.
1198/tech.2006.s372
5. Leardi, R.: Experimental design in chemistry: a tutorial. Anal. Chim. Acta 652, 161–172
(2009). https://doi.org/10.1016/j.aca.2009.06.015
6. Eriksson, L., Johansson, E., Kettaneh-Wold, N., WikstrÄom, C., Wold, S.: Design of
Experiments: Principles and Applications (2008)
A Framework Designing for Story Sharing
of the Elderly: From Design Opportunities
to Concept Selection
Cun Li(&), Jun Hu, Bart Hengeveld, and Caroline Hummels
Eindhoven University of Technology, Eindhoven, The Netherlands
{Cun.Li,J.Hu,B.J.hengeveld,C.c.m.Hummels}@tue.nl
Abstract. In this paper, we describe how we translated contextual inquiry into
design opportunities from which a framework regarding storytelling and eight
design concepts were emerged, and adopted Pugh’ matrix and user consultation
to evaluate these concepts. To be specific, based on the findings of contextual
inquiries with older adults and caregivers, a framework of story sharing of the
elderly was constructed. Eight design opportunities were then proposed. The
concept selection included two rounds: Pugh’s matrix, evaluating the concepts
against the baseline. User consultation, getting feedback from elderly users.
Keywords: Older adults Story sharing Design selection Design
introduction
1 Introduction
With the arrival of aging society, more older adults preferred to move to caring homes,
especially for those over the age of 85 [1]. Nevertheless, since caring homes are
relatively closed environments, and older adults are separated from the mainstream
social circles on account of being unfamiliar with technology. This makes older adults
are vulnerable to be social isolation [2]. In the meanwhile, since one feature of the
elderly is their memory of events, persons, and locations, they have the potential to be
story content producers [3]. The story sharing could be a method to make connections
with others.
In this paper, we firstly proposed a framework designing for story sharing of the
elderly tighter with eight design concepts. To evaluate the design concepts and choose
a small set of concepts to develop further, the decision making phase included Pugh’s
matrix, evaluating the concepts against the baseline. And user consultation, getting
feedback from elderly users.
2 Related Work
According to the literature, the existing design selection approaches could be classified
as the following aspects: Nonprocedural method, Methodology application, Design
review, Decision matrix, Model building, and QFD (quality function deployment).
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 810–815, 2020.
https://doi.org/10.1007/978-3-030-39512-4_123
A Framework Designing for Story Sharing of the Elderly 811
Each aspect exists strengths and weaknesses [4]. Among which the Pugh method [5],
where design concepts are compared against a set of criteria by means of a matrix, is
widely accepted and used due to its simplicity and efficiency.
Precious study has identified the two challenges of concept design when taking the
user needs into consideration. The first lies in the difficulty in understanding the real
needs of users. The second is, it is difficult for the users to provide accurate feedback on
the design concepts, due to lack of physical products in early stages of developing.
In this paper, in response to the first challenge, we adopt not only the Pugh’ matrix,
but also user consultation during concept selection. While in response to the second
challenge, we made mock-ups of potential design concepts.
3 The Framework and Design Concepts of Older Adults’
Story Sharing
Fig. 1. Framework for story sharing of the elderly together with design concepts
Based on the findings of contextual inquiries with older adults and caregivers, we
propose a framework designing for story sharing of the elderly together with eight
design opportunities. As shown in Fig. 1, the framework was constructed based on the
process of story sharing: Memory cue is essential to reminiscence and storytelling,
triggered by memory cues (take various forms, in our case are trigger question and
memento), the storyteller (older adults) share stories with the listener, which are fellow
residents within nursing home, and strange citizens and family members outside of
nursing home. To collect the stories, there are two methods: by interviewing and by
story-collecting installations. While the stories could be classified as audio, text, and
812 C. Li et al.
video, according to story forms, and be classified as stories not related to them, related
to them, and mementos, according to the story contents.
3.1 Design Concepts
Design concept 1: Story-Booth. It is an installation that placed in a nursing home.
Elderly people could listen and comment on stories from residents as well as tell his/her
own story anonymously. Design concept 2: Story-Token. Every elderly people has a
token containing pictures related to his/her story. Place the token on the installation,
and everyone could share.
Fig. 2. Sketches of design concept 1 and 2
Design concept 3: Story-Generator. The installation is placed in the public area and
could generate stories collected from the elderly in nursing homes. The QR code is
linked to a website, and people could scan it to comment. Design concept 4: Location-
Story. The elderly people in nursing home tell stories related to scenic spots while
tourists could listen to the stories by scanning the map.
Fig. 3. Sketches of design concept 3 and 4
Design concept 5: Memento-Story. Personal Belongings contain memories. Com-
bined with NFC technology, objects are attached to stories told by the elderly, and story
audios could be played when putting them on the platform. Design concept 6: Story-
Collector. Using a tablet application to take photos of mementos, and the older adults
could also directly tell the stories related to the mementos out, which could be recorded.
A Framework Designing for Story Sharing of the Elderly 813
Fig. 4. Sketches of design concept 5 and 6
Design concept 7: Radio-Story. Metaphor of audio is applied. A trigger question
displays when elder people pull down handle. The older adult could also tell stories
related to the trigger question, which could be recorded. Design concept 8: Slots-story.
Its function is the same as Radio-Story, but it adopts the metaphor of slots-machine.
Fig. 5. Sketches of design concept 7 and 8
4 Concept Selection
To conduct the Pugh’s matrix, we combined some of the above concepts, for example,
we integrated concept 6 into concept 7 and 8. We then categorized them, and finally
three groups were generated:
Group A: Memories Associated with Mementos. This group of concepts mainly
adopt RFID technology to attach stories to mementos, including Story-Token and
Memento-Story.
Group B: Trigger the Older Adults to Tell Stories via Different Forms of Memory
Triggers. This group includes Story-Booth, Radio-Story, and Slots-Memento.
Group C: Storytelling Between Older Adults in Nursing Home and Outside
Citizens. This group includes Story-Generator and Location-Story.
Next we adopt Pugh’s matrix to evaluating the above design concepts against the
baseline. The Pugh’s matrix is a method for identifying the most promising design
concepts among the alternatives. It consists of the following key steps: Choose the
criteria by which the concepts will be evaluated. Formulate the decision matrix. And
Evaluate the ratings. Regarding the criteria, we referenced universal design require-
ments from usability test [6]. Since we need to develop working prototypes, the criteria
of feasibility is also needed. All the criteria were equally important to our project, and
814 C. Li et al.
therefore we set the weight as 1. Table 1 shows the Pugh matrix comparing these the
three groups of concepts against usefulness, ease of use, ease of learning, novelty, and
feasibility.
Table 1. The Pugh’s matrix in our study
Evaluation Weight Group A Group B Group C
Criteria Story- Memento- Story- Radio- Slots- Story- Location-
Token Story Booth Story Story Generator Story
Usefulness Benchmark: 1 1 0 1 1 1 1 1
Ease of use Current 1 0 0 0 0 0 0 0
Ease of story sharing 1 1 1 1 1 1 0 0
learning
Novelty 1 −1 −1 1 1 1 1 1
Feasibility 1 1 1 1 1 1 1 1
Sum of 3 2 4 4 4 3 3
positives
Sum of −1 −1 0 0 0 0 0
negatives
Total 2 1 4 4 4 3 3
weight
score
Concept selection legend: Better = 1, Same = 0, Worse = -1.
Since both Group B(concepts of Story-Booth, Radio-Story, and Slots-Story) and
Group C(concepts of Story-Generator and Location-Story) are significantly superior to
the Group A, we need the second round of concept selection: User consultation.
5 User Consultation
To evaluate the design concepts of Group A and B, and acquire feedback from users,
five older adults from a caring home were recruited and consulted. Design ideas were
firstly introduced the aim of our interview to them in detail. To make them fully
understand the design concepts, the above related sketches and related pictures of
antique cradle telephone, antique radio, and slots-machine were shown to them as well.
We wanted to know their feelings regarding the concepts, familiarity of the three
objects, operations, preferences, and suggestions, during the user consultation.
Fig. 6. Mockups made in the user consultation
A Framework Designing for Story Sharing of the Elderly 815
First, most older adults attach great importance to privacy. That is, they didn’t hope
their life stories be shared with strangers, nor did they think the strangers would be
interested. The older adults preferred to share stories with family members and friends.
Thus, they preferred more concepts of Story-Booth, Radio-Story, and Slots-Story.
Next, we need to evaluate the final concept from the above three concepts. All the
interviewees were familiar with the above objects. When talking about the ease of
operation, one interviewee said: “I used to listen to radios, and I still have one in my
room. But I feel difficult to operate precisely when I scroll through stations by turning a
knob.” When asking about the preference for the three ideas, most older adults reflected
that the handle operation is more intuitive and enjoyable than others: “I prefer the
Slots-machine because the operation is interesting.” Therefore, concept of Slots-Story
was chosen and further detailed. Based on the feedback, Slots-Story also integrated
functions of memento photos display. The final prototype could be seen in [7, 8].
6 Conclusion
This paper provides a framework designing for story sharing of the elderly, as well as an
example enabling effective design concept selection in the concept-development phase.
Despite that our study context is older adults’ story sharing, lessons learned in our study
have a universal significance for design selection study, especially for the elderly users.
References
1. Reichman, W.E., Coyne, A.C., Borson, S., Negrón, A.E., Rovner, B.W., Pelchat, R.J.,
Sakauye, K.M., Katz, P., Cantillon, M., Hamer, R.M.: Psychiatric consultation in the nursing
home: a survey of six states. Am. J. Geriatr. Psychiatry 6, 320–327 (1998)
2. Mor, V., Branco, K., Fleishman, J., Hawes, C., Phillips, C., Morris, J., Fries, B.: The structure
of social engagement among nursing home residents. J. Gerontol. Ser. B: Psychol. Sci. Soc.
Sci. 50B, P1–P8 (1995). https://doi.org/10.1093/geronb/50B.1.P1
3. Dryjanska, L.: A social psychological approach to cultural heritage: memories of the elderly
inhabitants of Rome. J. Herit. Tour. 10, 38–56 (2015). https://doi.org/10.1080/1743873X.
2014.940960
4. Lee, C.-W., Suh, Y., Kim, I.K., Park, J.-H., Yun, M.H.: A systematic framework for
evaluating design concepts of a new product. Hum. Factors Ergon. Manuf. Serv. Ind. 20, 424–
442 (2010)
5. Pugh, S.: Concept selection: a method that works. In: Proceedings of the International
conference on Engineering Design, pp. 497–506 (1981)
6. Lund, A.M.: Measuring usability with the use questionnaire12. Usability Interface 8, 3–6 (2001)
7. Li, C., Hu, J., Hengeveld, B., Hummels, C.: Slots-memento: facilitating intergenerational
memento storytelling and preservation for the elderly. In: Proceedings of the Thirteenth
International Conference on Tangible, Embedded, and Embodied Interaction, pp. 359–366.
ACM, New York (2019). https://doi.org/10.1145/3294109.3300979
8. Li, C., Hu, J., Hengeveld, B., Hummels, C.: Story-Me: design of a system to support
intergenerational storytelling and preservation for older adults. In: Companion Publication of
the 2019 on Designing Interactive Systems Conference 2019 Companion, pp. 245–250. ACM
(2019)
A Methodological Approach for the Design
of Inclusive Assistive Devices by Integrating
Co-design and Additive Manufacturing
Technologies
Francesco Gherardini1(&), Andrea Petruccioli1, Enrico Dalpadulo1,
Valentina Bettelli2, Maria Teresa Mascia2, and Francesco Leali1
Department of Engineering “Enzo Ferrari”,
1
University of Modena e Reggio Emilia, Modena, Italy
{Francesco.Gherardini,Andrea.Petruccioli,
Enrico.Dalpadulo,Francesco.Leali}@unimore.it
2
Department of Medical and Surgical Sciences for Children and Adults,
University of Modena e Reggio Emilia, Modena, Italy
{Valentina.Bettelli,Mariateresa.Mascia}@unimore.it
Abstract. Additive Manufacturing is a widespread technology that may
enhance product customization based on specific users’ needs, as in the case of
assistive devices. Many chronic physical progressively disabling diseases, but
also ageing, may cause severe limitations in daily life, which can be overcome
by highly customized aids. Literature shows that the active involvement of the
patient in the development of assistive devices through co-design allows for
their greater therapeutic effectiveness and acceptance. Therefore, this paper
proposes a methodological approach for the development of inclusive assistive
devices to support daily activities in persons with disabling diseases of the
upper-limb. The approach integrates co-design, standardized tools, and low- and
high-tech prototyping techniques and tools, which lead to significant feedbacks
from patients. The patients are encouraged to interact with conceptual proto-
types through direct 3D CAD modelling and touch screen devices. Assessment
tests highlight the suitability of the method to achieve the expected goals.
Keywords: Co-design Additive Manufacturing Assistive device
Occupational therapy Inclusive method Parametric modelling Hand
pathologies
1 Introduction
The widespread diffusion and the accessibility of Additive Manufacturing (AM) tech-
nologies open to new roles in product customization based on specific users’ needs [1].
This goal is particularly important in the development of Assistive Devices (ADs),
whose customization guarantees to respond to specific needs. In fact, patients with
chronic physical and psycho-socially progressively disabling diseases, but also elderly
persons, are affected by severe limitations in daily life [2]. The commercial ADs available
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 816–822, 2020.
https://doi.org/10.1007/978-3-030-39512-4_124
A Methodological Approach for the Design of Inclusive Assistive Devices 817
on the market are not always be able to address these specific needs, maybe because they
may lack of customization in dimensions and morphology, or their aspect is too stig-
matizing. Moreover, they are usually expensive or not easy to find. All these issues can be
overcome by highly customized products, able to reduce the frequent abandonment of the
AD and to achieve a better correspondence to the patients’ needs [3, 4].
Several researches have shown that the active involvement of the patient in the AD
prototyping process through co-design sessions allows greater therapeutic effectiveness
and acceptance of the aids [5–7]. However, many difficulties during the extrapolation
and definition of the latent needs occur, therefore the patients’ participation requires
improvements of its formalization [8]. The use of a physical or a virtual concept may
support the patient to develop new ideas, create proposals of solutions, communicate
needs, expectations and compensation strategies, validate the prototype.
Thanks to AM, both the first prototype and the final AD can be easily obtained
through a 3D model. However, the generation of the 3D model itself is one of the most
challenging step of co-designing AD. On the one hand, the patient has to properly
communicate his/her idea to the designer that, on the other hand, has to understand and
translate words into a 3D model.
Currently, the development of the first 3D model is made only by the designer,
mainly due to the patients’ lack of specific skills about virtual tools. Even if patients can
benefit from the use of a physical concept or low-tech prototyping strategies, clarifying
their ideas and suggestions for early prototype modifications requires considerable
amount of time and does not always provide effective solutions. Conversely, the
interactive use of a virtual concept would lead to enhance the process creativity and to
explore innovative solution, providing a direct feedback on the 3D model by the patient.
To this aim, we propose a systematic approach for the inclusion of patients into the
co-design process of new ADs for hand pathologies, enhancing their active participation
in the conceptual phase through interactive computer tools for the concept definition.
The paper is organized as follows. In Sect. 2, we introduce the proposed method,
tools and technologies. In Sect. 3, the application of the method on a case study is
reported, together with the patients’ feedback. Section 4 presents the conclusions.
2 Methods, Tools and Technologies
This paper proposes a methodological approach for the development of inclusive ADs
and, more generally, of products to support daily activities in patients with disabling
diseases of the upper limb. The core of this approach is based on the presence of a
multidisciplinary team able to support the patient in the co-design process for devel-
oping his/her own AD. In these phases, the patient plays a key role in the design
process because he/she is the ultimate user of the AD. Specific tools of Occupational
Therapy such as the Canadian Occupational Performance Measure (COPM), stan-
dardized tools such as USERfit Product Analysis (USERfit PA), Psychosocial Impact
of Assistive Devices Scale (PIADS), Quebec User Evaluation of Satisfaction with
Technical Aids (QUEST), as well as engineering techniques such as Quality Function
Deployment (QFD), morphological matrix, etc. are integrated into the proposed method
to collect the patient’s needs, the requirements that the AD must have and, therefore,
818 F. Gherardini et al.
the attributes of the developed AD. In particular, the COPM is an outcome measure
designed for use by Occupational Therapists (OTs) to assess patients’ outcomes in the
areas of self-care, productivity, and leisure. The OT proposes a semi-structured inter-
view to the patient for identifying problem areas in daily functions. Subsequently, if the
emerged difficulties can be resolved with a customized AD, the patient is involved into
the co-design team, becoming an active part of the AD development process. Together
with the OTs and the designers, the patient starts the co-design process for the real-
ization of his/her own AD.
Co-design Approach. The core of the method is the co-design phase, which consists
of different sub-phases:
1. Identification of the AD features. The team, through the USERfit PA, is guided in
identifying the key features of the co-designed AD. Those outputs may be inte-
grated within a QFD or similar tools [9], in order to help the design team to
prioritize the needs, transform them into technical requirements and product
specifications.
2. From concept to prototype. During this phase, the patient actively contributes to
the AD creation. In particular, all his/her ideas in the concept definition are used to
create the preliminary model layout. The prototype development can be divided in
subphases: a. Virtual model development (see Fig. 1 and the following sub-section),
b. Virtual model validation thanks to the patient, c. 3D printing of the model.
The virtual validation is a key intermediate step, in which direct actions on the 3D
Computer-Aided Design (CAD) model may be performed before the 3D printing
phase, in order to modify both functional and aesthetic features of the AD.
3. Delivery and evaluation. When the team delivers the AD, the OT dispenses the
tool PIADS. In this phase, the PIADS is used to identify the patient’s expectation
about how the AD will affect his/her daily life.
4. Follow-Up (3 weeks). In the first follow-up, we check if the AD is still in use, if
there were any problems during its usage (wear or other) and the OT dispenses
again the PIADS to check if and how the AD has influenced the patient’s life.
Subsequently, QUEST is dispensed to know if the patient is satisfied or not about
the AD.
5. Follow-up (6 months). After six months, a new follow-up is required to verify if
the AD is still in use and if problems have been occurred.
Fig. 1. The method workflow.
A Methodological Approach for the Design of Inclusive Assistive Devices 819
Developing New Interactions. The involvement of the patient in the development of
the digital concept can be based on the direct interaction with the available techno-
logical tools. The parametric modelling and/or free-shape direct modelling within a 3D
CAD software can overcome difficulties in the physical definition of some patient’s
needs. The 3D model definition requires particular attention to easily match user-
editable forms (free-shape, sketch-based, etc.) and parametric features. Technological
interface systems such as interactive surface computing platforms that allow the user to
use and touch real-world objects, and share digital content at the same time, can be
implemented. This kind of interface (Natural User Interface) could be based on a multi-
touch PC hardware, with a combination of a touch screen display and an active digital
stylus, to integrate both the direct use of hands and a physical tool to perform actions
on the CAD model. In this way, with specific reference to the co-design, the patient
becomes able to manipulate the digital prototype of the AD.
3 Case Study
As a case study, this paper presents the development of an AD for recovering credit
cards from an ATM cash machine, which is co-designed with four patients. Among
them, we describe the most significant contributor to the co-design phase: the patient is
52 years old with diagnosis scleroderma (SSc). She is treated at the Rheumathology
clinic of the Hospital Policlinico di Modena for pharmacological treatment and occu-
pational therapy. At the interview and instrumental exam (dynamometer), the patient
shows a deficit of fine grip (tip to tip) and prolonged strength deficit. Through the semi-
structured interview (COPM), the patient reports difficulty in recovering the credit card
in time after having withdrawn the money from the ATM cash machine. Due to the
unavailability of specific tools nor AD to this aim, the patient is involved in the co-
design process. Through USERfit, together with the designer and the OT, she exploits
all the needs into key features. The output of this phase (USERfit) is a list of product
attributes, which represent the inputs of the co-design process.
Figure 2 depicts the 3D model of the AD.
Fig. 2. 3D printed assistive device for credit cards.
820 F. Gherardini et al.
With reference to the prototyping phase or the evaluation phase, the patient should
be able to attend the model definition by exploiting the Natural User Interface. Func-
tional regions (highlighted in red) are the non-editable features that are designed to
perform the mechanical functions. The internal surfaces of the frontal part have to
guarantee the credit card contact by interposing an anti-slip film. An intermediate elastic
element connects the two main bodies and allows their relative movement. The pins
positioned in the rear part are introduced to be coupled with a small spring in order to
keep the device closed. In Fig. 3, these features are considered as frozen areas (high-
lighted in red) since they have to be kept to guarantee part functionality. All the other
surfaces (highlighted in green) are potentially editable by the patient in size or shape,
starting from his/her considerations and feelings related to aesthetics, ergonomics,
testing, etc. This operation can be performed through the direct 3D CAD modelling tool
using the touchscreen and the active stylus to interact with the 3D model.
Fig. 3. Interaction with the 3D conceptual model using direct 3D CAD modelling, touchscreen
and active stylus.
Figure 4 shows the patient’s expectation as a result of the PIADS at t0 and at t1 (i.e.
after one month trial period). The scores reflect the patient’s assessment in t1. She did not
like the developed AD due to its aesthetics, but was partly satisfied by its functionality.
Therefore, the AD was subjected to further modifications during the approval phase.
Fig. 4. PIADS score in the three main categories.
A Methodological Approach for the Design of Inclusive Assistive Devices 821
The final score about the patient’s satisfaction with respect to the provided AD,
collected through the QUEST, is 4.12/ 5 (Table 1). The most relevant aspects are its
size, ease of use and effectiveness. Moreover, after 3 years, the patient reports that the
AD is still in use and she is very satisfied.
Table 1. QUEST scores.
ASSISTIVE DEVICE SERVICE
Item Score Item Score
Dimensions 5 Service delivery 4
Weight 4 Repairs and maintenance n.a.
Adjustments 3 Professional service 4
Safety 4 Follow-up services n.a.
Durability 3 MEDIA SCORE 4.00
Ease of use 5 Service delivery 4
Comfort 4
Effectiveness 5
MEDIA SCORE 4.12
4 Conclusions and Final Remarks
The case study shows that an interactive co-design, made possible by AM technologies
and digital tools, allows patients with long-established strategies in the activities of
daily life to change habits satisfactorily. Assessment tests and their follow-up highlight
the suitability of the method to achieve the expected goals, as shown by the final
patients’ judgments and the continuous use of the developed ADs.
Acknowledgments. This work was supported by Fondazione Cassa di Risparmio di Modena,
Italy (grant number SIME 2017.0345).
References
1. Lunsford, C., Grindle, G., Salatin, B., Dicianno, B.E.: Innovations with 3-dimensional
printing in physical medicine and rehabilitation: a review of the literature. PM&R 8, 1201–
1212 (2016)
2. Tuntland, H., Kjeken, I., Nordheim, L., Falzon, L., Jamtvedt, G., Hagen, K.: The Cochrane
review of assistive technology for rheumatoid arthritis. Eur. J. Phys. Rehabil. Med. 46, 2–261
(2010)
3. Gitlin, L.N.: Why older people accept or reject assistive technology. Generations 19, 41–47
(1995)
4. Phillips, B., Zhao, H.: Predictors of assistive technology abandonment. Assist. Technol. 5(1),
36–45 (1993)
5. Ganesan, B., Al-Jumaily, A., Luximon, A.: 3D printing technology applications in
occupational therapy. Phys. Med. Rehabil. Int. 3(3), 1085 (2016)
822 F. Gherardini et al.
6. Gherardini, F., Mascia, M.T., Bettelli, V., Leali, F.: A co-design method for the additive
manufacturing of customised assistive devices for hand pathologies. JIDPS 22(1), 21–37
(2019)
7. Ostuzzi, F., Rognoli, V., Saldien, J., Levi, M.: +TUO project: low cost 3D printers as helpful
tool for small communities with rheumatic diseases. Rapid Prototyp. J. 21(5), 491–505 (2015)
8. Sansoni, S., Wodehouse, A., McFadyen, A., Buis, A.: Utilising the repertory grid technique in
visual prosthetic design: promoting a user-centred approach. J. Integr. Des. Process. Sci.
20(2), 31–46 (2016)
9. Gherardini, F., Renzi, C., Leali, F.: A systematic user-centred framework for engineering
product design in small- and medium-sized enterprises (SMEs). Int. J. Adv. Manuf. Technol.
91(5–8), 1723–1746 (2017)
New Collaborative Version of the Quality
Function Deployment: Practical Application
to the HABITAT Project
Giuseppe Mincolelli(&), Gian Andrea Giacobone(&),
Michele Marchi(&), and Silvia Imbesi(&)
Department of Architecture, University of Ferrara, Via della Ghiara 36,
44121 Ferrara, Italy
{giuseppe.mincolelli,gianandrea.giacobone,
michele.marchi,silvia.imbesi}@unife.it
Abstract. This article relates about the application of a new collaborative
version of the Quality Function Deployment, developed by the authors, to a real
project of design research. Specifically, it will be illustrated the application of
this design tool to a project named HABITAT, carried on by a multidisciplinary
team, whose aim was to develop a platform of interrelated smart objects for the
home assistance of self-sufficient and non-self-sufficient older people.
Keywords: Quality Function Deployment Design research
Multidisciplinary team User Centered Design HABITAT project
1 Introduction
The development of a new product is a very challenging undertaking, because it
requires not only a multidisciplinary team involving in different expert areas, but also
needs complex coordination of many requirements and tasks to ensure that the product
program does not exceed the planned schedule and budget [1]. And, first of all, the
expected product must meet the expectations and needs of its customers. Indeed,
capturing customer demands is crucial when producing high quality products, because
it enables the development process to achieve the high score of customer satisfaction
and consequently accomplish the intended goals of the company [2].
Over the time, several human-centered methodological solutions were produced for
a better understanding of customer requirements and subsequently sound management
of the technical specifications, quality, costs and engineering aspects [3]. Quality
Function Deployment (QFD) is one of those methods that create a model of interaction
with the end-users and permits to acquire their feedback to develop complex products,
which satisfy their expectations [4].
Considering this, the contribution reports the application of QFD to a particular
complex design research project named HABITAT, whose aim was to develop a
platform of interrelated smart objects for the home assistance of self-sufficient and non-
self-sufficient older people. Due to its complex blend of product and services, the paper
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 823–828, 2020.
https://doi.org/10.1007/978-3-030-39512-4_125
824 G. Mincolelli et al.
describes a new collaborative version of the QFD, which was developed by the authors
to face up the entire development process.
2 What Is the Quality Function Deployment
QFD is a human-centred tool that is able to orient product design toward the real
exigencies of the end-users, because customer satisfaction is placed first within the
product development strategy [5]. It ensures a qualitative design process throughout
each stage of the product program putting great emphasis on the design stage [6].
Indeed, QFD is used before starting on the activities of development, engineering, and
production of new products or services [7] and it is a useful management tool to model
the dynamics of the design process [8].
Quality Function Deployment is also considered a powerful and proactive decision-
making tool applied in relationship-wise contexts, because it enables multidisciplinary
working among different corporate sectors or divisions of the same company, engaged
in a common project [3]. The involvement includes all of the company functions
addressed to the planning process: marketing, design, quality, technical assistance,
technologies, production and suppliers [7] (Fig. 1).
Fig. 1. Image of a QFD matrix. The 1st and 2nd areas shows the voice of costumer, the 3rd the
voice of engineer, in the 4th the correlation matrix, the 5th the competitive benchmarking and
finally the 6th the technical importance ranking.
New Collaborative Version of the Quality Function Deployment 825
The core of QFD is the House of Quality (HoQ), the mathematical bi-dimensional
matrix that analytically compares the needs of users with the product characteristics of
the expected artefact, in order to identify their reciprocal relationships. In this case,
users’ needs are translated in detailed engineering specifications to design and produce
a product that fulfills the requirements assumed at the beginning of the process.
The 1st area defines the voice of the customer (VoC) representing qualitative data of
users’ requirements, whilst the 2nd section assigns to each need a rank – within a
grading system from 1 to 5 – that indicate its level of importance. The 3rd area
represents the voice of the engineer (VoE) because it refers to the product character-
istics provided by the teamwork that are indispensable to reach desired quality of the
final product. The 4th zone indicates the relationship between the VoC and the VoE. It
is the most important part of the matrix because it determines the final outcomes. The
relationship is expressed in a quantitative manner, by using a numerical scale that
determines the intensity of correlations. The values are: 9 = strong relation;
3 = medium relation; 1 = weak relation; 0 = no relation. The 5th area tests technical
competitiveness of the expected project by using quantitative benchmarking. In the
end, the 6th section displays the technical importance ranking obtained by processing
data within the relationship matrix.
2.1 Historical Notes on QFD
QFD was elaborated at the end of the sixties, in Japan, as a new approach to develop
products, amongst the various activities of Total Quality Control [9]. The official birth
date of QFD is 1972, when the quality chart was applied during the design of large
cargo vessels in the shipyards of Mitsubishi Heavy Industries [10]. In the same year,
Akao introduced a concept summarized by four Japanese ideograms, in which the first
guidelines of the method were explained [11]. The method was named Hinshitsu Kino
Tenkai, which means: Hinshitsu (product attributes, quality or qualities) Kino (func-
tions, mechanisms, opportunity) Tenkai (deployment, development). Successively,
QFD became also a key procedure for Toyota [3].
The diffusion of QFD in the western world began no earlier than 1986, thanks to
Don Clausing, a professor at MIT, who introduced the method both in academia and in
industry, especially in Ford [5]. The first complete book was published in 1990 by
Akao, and successively translated into English by Glenn Mazur [6]. From the mid-
nineties to the early years of the 2000 some authors reported that more than 1,000 case
studies were published all over the world [9] with a special focus on high technology
and transportation [3].
Nowadays America and Asia have the highest percentage of companies that use
QFD, whilst in Europe, many applications are presented in some states such as Italy –
which seems to be the first country where QFD is being implemented – Holland,
United Kingdom, Sweden, Austria, Spain and most of all in Turkey and Germany [9].
826 G. Mincolelli et al.
3 Experimentation of the QFD in the HABITAT Project
The authors applied the QFD many times on different projects during the last twenty
years, on these occasions they had the opportunity to observe difficulties arose during
the use of that tool by multidisciplinary teams dealing with Design Research projects,
usually composed by people with different backgrounds not trained to use QFD, mostly
working at distance [12]. Starting from the observed problems, proof variations have
been developed and tested in order to facilitate, strengthen and make more efficient this
type of group work. Most of the elaborated variations were experimented in the
HABITAT project [13].
This project, financed by public funds under the POR FESR 2014-2020 of Region
Emilia Romagna in Italy, had a programmed duration of two years, starting from 1st of
April 2016 and ended in July 2018. It was aimed at designing and testing a system
consisting of smart objects to provide care and monitoring of the elderly in a pro-
longation of independent living perspective [13].
HABITAT was elaborated by a multidisciplinary research team, in collaboration
with companies users. The aim of the project was to use Internet of Things
(IoT) technologies to develop, implement and test assistive and reconfigurable home
environments. In particular, the team designed some interrelated smart objects and
smart wearables that are able to detect the elder’s habits and modify themselves in
order to facilitate the person’s independence, and to collect and elaborate information
about the movement in the long run. The methodology chosen as proper to face this
kind of design challenge was User Centered Design, that allows to focus on the user’s
needs and to consider them during the iterative steps of the whole project. The applied
methodology made possible to optimize the combination between constraints deriving
from the technological choices and requirements derived from the analysis of the needs
carried out with users [12].
The smart objects that were designed within the HABITAT project are:
– A chair for detection of rise attempts and improper postures;
– A radio that will be the interface between the system and the elderly user, as well as
offering activities for auto-reporting and entertainment;
– A wearable device combined with a reader placed in different environments, for
indoor localization.
To define the smart products’ and system’s specifications it was applied the Quality
Function Deployment tool, until that time constantly used only by the group of the
authors [12]. Considering that, the authors decided to bring some changes to the
original setup of the tool in order to maximize the comprehensibility of the data,
procedures and documents for all working groups; make a team that applies for the first
time the User Centered Design and the QFD able to co-define effective design briefs
and specifications based on users’ needs and make the developed and experimented
solution available for teams that are developing User Centered Design researches [14].
The variations that had the greater impact were the ones related to the graphical
adjustments to the aspect of the QFD matrices: firstly, it was changed the way of listing
needs and characteristics. These two sets were related to codes that made it quick and
New Collaborative Version of the Quality Function Deployment 827
easy to understand which user expressed that need, in which context and what was he
referring to. Secondly, it was conceived a new way of scoring both the needs listed in
the matrix and the degree of correlation between those needs and the selected
product/system characteristics; it was used a specific palette of colors that, working on
color’s saturation, made it possible even for the most unskilled members of the team to
make a rough assessment at first glance of the results showed by the matrix [14].
4 The Idea of a New QFD
The modifications made to the traditional tool, had the aim of improving the QFD’s
level of usability and to make it more efficient for nowadays Design Research projects
that are not only about industrial products, but even about services, interfaces, pro-
cesses, etc. It was noticed that the simplification of the graphical aspect of the QFD was
one of the issues most influencing the level of comprehensibility of the matrices by
researchers that were not trained to its use [14]. Another consideration was related to
the use of QFD in several projects with elements in common: a large part of the
expressed needs and selected features, project by project, cyclically returned; it was
therefore evident that it would be desirable and feasible to build a database, espe-
cially for needs, to which every professional can draw to develop the QFD of his own
project [15].
The authors, during their experimentation, realized how QFD already has in its
DNA what is required to let it become a flexible, personalized, up-to-date instrument
for contemporary design research, conceived for developing innovation in many areas
through a multidisciplinary and collaborative approach. The expectation is to find other
researchers and designers interested to participate in the developing of a new vision of
the QFD as a fully digital, UX centered cooperative platform for design research [15].
5 Conclusions
This paper represents a declaration of intents and an invitation to cooperate for every
researcher and designer that is interested in developing a new tool for collaborative,
multidisciplinary design research. Authors and their colleagues will keep on working
on improvements about the GUI, procedures and UX of this version of QFD for
collaborative design research, and they are open to contributions, suggestions and
proposals. The aim is to transform an efficient but not-easy-to-use tool into a coop-
erational platform able to involve, include and motivate every participant in co-design
activities, multidisciplinary research projects, and innovation development teams,
independently from their field of knowledge, language skills and technical background.
The next step of this project will be the online diffusion of the new tool in an open
access mode to encourage diffusion and cooperation.
828 G. Mincolelli et al.
References
1. Bhise, V.D.: Designing Complex Products with Systems Engineering Processes and
Techniques. CRC Press, Boca Raton (2013). https://doi.org/10.1201/b15483
2. Bhise, V.D.: Automotive Product Development: A Systems Engineering implementation.
CRC Press, Boca Raton (2017). https://doi.org/10.1201/9781315119502
3. Maritan, D.: Practical Manual of Quality Function Deployment. Springer, Cham (2015).
https://doi.org/10.1007/978-3-319-08521-0
4. Mincolelli, G.: Customer/user centered design. Analisi di un caso applicativo. Maggioli
Editore, Santarcangelo (2008)
5. Al-Mashari, M., Zairi, M., Ginn, D.: Key enablers for the effective implementation of QFD:
a critical analysis. Ind. Manag. Data Syst. 105, 1245–1260 (2005). https://doi.org/10.1108/
02635570510633284
6. Akao, Y.: Quality Function Deployment (QFD) Integrating Customer Requirements into
Product Design. Productivity Press, Portland (1990)
7. Franceschini, F.: Advanced Quality Function Deployment. St. Lucie Press, Boca Raton
(2001)
8. Govers, C.P.M.: QFD not just a tool but a way of quality management. Int. J. Prod. Econ. 69,
151–159 (2001). https://doi.org/10.1016/S0925-5273(00)00057-8
9. Akao, Y., Mazur, G.H.: The leading edge in QFD: past, present and future. Int. J. Qual.
Reliab. Manag. 20, 20–35 (2003). https://doi.org/10.1108/02656710310453791
10. Nishimura, K.: Ship design and quality table. Qual. Control JUSE. 23, 16–20 (1972)
11. Re Velle, J.B., Moran, J.W., Cox, C.A.: The QFD Handbook. John Wiley, New York (1998)
12. Mincolelli, M., Marchi, M., Chiari, L., Costanzo, A., Borelli, E., Mellone, S., Masotti, M.,
Paolini, G.: Inclusive design of wearable smart objects for older users: design principles for
combining technical constraints and human factors. In: Di Bucchianico, G. (ed.) Advances in
Design for Inclusion. AHFE 2018. Advances in Intelligent Systems and Computing, vol.
776, pp. 324–334. Springer, Cham (2019)
13. Mincolelli, G., Marchi, M., Giacobone, G.A., Chiari, L., Borelli, E., Mellone, S., Tacconi,
C., Cinotti, T., Roffia, L., Antoniazzi, F., Costanzo, A., Paolini, G., Masotti, D., Mello, P.,
Chesani, F., Loreti, D., Imbesi, S.: UCD, ergonomics and inclusive design: the HABITAT
project. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.)
Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)
Volume VII: Ergonomics in Design, Design for All, Activity Theories for Work Analysis
and Design, Affective Design, 2019, VII, pp. 1191–1202 (2019)
14. Mincolelli, G., Marchi, M., Imbesi, S.: Inclusive design for ageing people and the internet of
things: understanding needs, in advances in design for inclusion. In: Proceedings of the
AHFE 2017 International Conference on Design for Inclusion, pp. 98–108. Springer, Cham
(2017)
15. Mincolelli, G., Imbesi, S., Zallio, M.: Collaborative quality function deployment. a
methodology for enabling co-design research practice. In: Di Bucchianico, G. (ed.)
Advances in Design for Inclusion. AHFE 2019. Advances in Intelligent Systems and
Computing, vol. 954. Springer, Cham (2020)
Human Centered Methodologies
for the Development of Multidisciplinary
Design Research in the Field of IOT Systems:
Project Habitat and Pleinair
Giuseppe Mincolelli(&), Silvia Imbesi(&),
Gian Andrea Giacobone(&), and Michele Marchi(&)
Dipartimento di Architettura di Ferrara, Università degli Studi di Ferrara,
Ferrara, Italy
{giuseppe.mincolelli,silvia.imbesi,
gianandrea.giacobone,michele.marchi}@unife.it
Abstract. PLEINAIR (“Parchi Liberi E Inclusivi in Network per Attività
Intergenerazionale Ricreativa e fisica” - Network of Free and Inclusive Parks for
Recreational and Intergenerational Physical Activity) is a research project fun-
ded under action 1.2.2 POR FESR 2014–2020 and aims to create inclusive
contexts to promote the adoption of an active lifestyle contributing to good
health for all, across all age groups. The project is aimed at a multi-generational
target audience and promotes the adoption of correct and active lifestyles
according to the abilities of each, with personalized motivational strategies.
PLENAIR proposes an innovative approach to physical activity, extending the
user’s experience into unconventional spaces and everyday life. PLEINAIR
intends to create a smartly-equipped park that offers new types of urban furniture
encompassing distributed-intelligence elements. Indeed, the key elements of
PLEINAIR are the OSO (Outdoor Smart Objects): furniture and play equipment
featuring sensors and initiating devices that are interoperable thanks to an IoMT
(Internet of Medical Things) infrastructure.
Keywords: Human Centered design Inclusive design Health Wellness
Smart object outdoor
1 Introduction
National and international organizations, including the World Health Organization
(WHO), have observed a very high and steadily growing level of sedentary lifestyle in
the last ten years, caused mainly by insufficient levels of physical activity [1]. These
aspects induce the world population to risk factors such as long-term chronic diseases
with a consequent, for example, reduction in people’s quality of life and increase of
social-welfare costs.
Worldwide, it is estimated that one adults in four (aged between 18 and 69) and
three out of four adolescents (aged between 11 and 17) do not do any type of physical
activity or exercise, as recommended by WHO. In Italy instead, only an estimated 18%
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 829–834, 2020.
https://doi.org/10.1007/978-3-030-39512-4_126
830 G. Mincolelli et al.
of children practice sports for no more than one hour a week; on the other hand, among
adults, it is estimated that 33.6% are classified as sedentary: they do not do heavy
works and do not practice physical activity in their free time. However, things are
different for people over 65 years old: good results are observed regarding soft
mobility, but exercises for muscle strength are lacking.
According to recent WHO data, only in Europe physical inactivity causes every
year one million deaths (about 10% of the total) [1]. Therefore, these irrefutable data
determine a dangerous and risky drift for the world population. One of the strategies
promoted by the WHO to reduce sedentary ways of life and promote an active lifestyle
for all ages and all health conditions, is to ensure “an enabling environment that
supports physical activity through engaging and safe built environments, accessible
public spaces and infrastructure” [2]. In a similar scenario, a university-industrial
research group is working on the PLEINAIR project. The project started in July 2019
and will last 24 months. The goal is to create inclusive contexts to promote the
implementation of active lifestyles and good health for highly multigenerational users.
Effectively, the key elements of the PLEINAIR project are the OSO (Outdoor Smart
Objects): furnishings, ludic tools or even just convivial equipped spaces in which,
thanks to the presence of sensors, actuators and an IoMT (Internet of Medical Things)
infrastructure, it is possible to monitor people’s lifestyles, to encourage physical
activity thanks to customized motivational strategies, and to trigger virtuous dynamics
of socialization and discussion.
2 Method and Objectives
To achieve the pre-established result, a highly specialized and heterogeneous research
pool, composed by Universities, Research Centers and Companies, has been structured.
A multidisciplinary team has been defined with high expertise in different but com-
plementary areas: the partners TekneHub and A.I.A.S have strong competencies in the
cultural and methodological approach aimed at satisfying people’s needs. TekneHub
also manages the design aspects of OSO and UX (User Experience), in terms of
aesthetics, interactivity, inclusiveness and ergonomics. DataRiver, CIRI SDV and
Future Technology Lab have proven competencies in the sector of IoT/IoMT, sensors,
as well as data management, Machine Learning/Artificial Intelligence, neuro biome-
chanics of movement and analysis and development of the motor behavior. Moreover,
thanks to the presence of Partner Companies, tangible skills and experiences are also
available in terms of industrial and experimental operations. Specifically, the following
companies are partners of the project: Ergotek s.r.l. (specialized in the production of
wooden seats and furniture), Sarba s.p.a (producer of high quality playground equip-
ment and urban furniture since 1959), mHealth Technologies S.r.l (healthcare furniture
sector) (Fig. 1).
The diverse characteristics and competences of the partnership guarantee a high and
synergistic heterogeneity, intersectoriality and interdisciplinarity.
Human Centered research methodologies will be adopted in the PLENAIR project;
thanks to project tools typical of the design research, these methodologies will be able
Human Centered Methodologies for the Development of Multidisciplinary Design 831
Fig. 1. System general layout of the PLEINAIR project.
to develop participatory methods, graphic interfaces and smart objects aimed at the real
satisfaction of people’s needs.
For instance, the Quality Function Deployment [3] will be adopted as project tool: a
method that is able to fill the linguistic or methodological gaps [4], often present in
complex multidisciplinary teams. This work methodology, typical of the industrial
design, allows to speed up and makes more consistent the processing phases and the
final project strategy, making possible to simultaneously manage (with the same lan-
guage) the aesthetic and qualitative needs with the quantitative and measurable char-
acteristics, without neglecting any aspect and users (from operators to smart park users,
of different types, age and physical availability). This methodology has been used, with
more than positive results, during the HABITAT project [5], which focused on the
creation and the trial of an IoT platform to encourage active ageing and avoid forced
hospitalization of elderly users.
832 G. Mincolelli et al.
Therefore, the assumptions are to start with the know-how gained within the
Habitat project (which also saw the collaboration of CIRI, Ergotek, mHealth Tech-
nologies) and to optimize some methodological processes, with the ultimate goal of
raising the quality of the research.
The PLEINAIR project – unlike the Habitat project [6] which had specific and
small in number users (not self-sufficient and self-sufficient elderly people and care-
giver) – has a considerable complexity from the beginning. The hypothesized scenario
is a multi-generational park, thus the users of the project are various and different in
age, needs, skills/disabilities.
The detailed and meticulous definition of the most appropriate and suitable strategy
is therefore pivotal in order to interact with users and to be able to create a database of
needs (needs analysis) which is essential for the success of the project. Unlike the
Habitat project – which had fewer users – in the PLEINAIR project, participatory
methodologies, such as co-design, will be mainly applied in order to involve as many
users as possible from the very first project activities and let them plan and design
together (Fig. 2).
Fig. 2. Graphic scheme of project management among partners. The main role of users and
stakeholders is clear.
The involved people will be therefore active actors within the project; the design of
the project will be planned “together” with them and not “for them”. Once the needs
analysis has been performed, we will constantly interact with all the partners in order to
define the design of OSO (Outdoor Smart Objects), both in terms of functionality and
ergonomics. Furthermore, it is essential to work on defining and developing the HMI
(Human-Machine Interface) and the UX of the OSO, in complete harmony with the
needs of users and guaranteeing the emotional involvement, inclusiveness and high
Human Centered Methodologies for the Development of Multidisciplinary Design 833
usability of all those involved in the life of the system, whether they are operators or
users, regardless age or physical conditions.
These methodological processes based [7] on the constant participation and direct
contact with people will be recurring during all project phases: from the needs analysis
phase, to the definition of motor exercises, from the OSO morphological project to the
development of the IoMT platform.
3 Results/Outputs
The project is still in its early phase and therefore there are no tangible results yet. We
will then describe the general planned activities and the expected outputs that the
TekneHub Laboratory will carry out:
• identification of dissimilar users by type, physical conditions and age; elaboration of
a broad and complete needs overview; involvement of Public Administrations
(Municipality of Bologna) for the definition of system needs. These activities will
lead to the realization of a specific needs analysis, highlighting the most important
needs for the development and implementation of PLEINAIR;
• development and coordination of participatory working days with users to examine
the expectations placed on the use of new technologies to develop the first concept
prototypes;
• development of the needs analysis through the tool Quality Function Deployment
that will put in the matrix the qualitative needs already processed with quantitative
characteristics; this activity will lead to the realization of QFD matrices that will
determine and define the technical specifications of the OSO;
• development of project concepts and brief in collaboration with users and business
and academic partners (co-design); this phase will lead to the final morphological
design of OSO and inclusive interfaces;
• verification and potential development of morphological prototypes with the inte-
gration of the sensor parts and validation of the PLEINAIR platform to reach a
technological maturity (TRL) of 6.
4 Conclusions
Beyond the development of the OSO, the Pleinair project will be a challenging
opportunity to experiment and improve a set of different design methodologies in the
attempt to define a Human-Centered design research strategy applicable to several
complex scenarios. Innovation for social improvement studies and practices continu-
ously need new design tools and methods, since the society by one side and the
technological world by the other keep on changing at an increasingly fast pace. The
process of humanization of technologies is so becoming one of the most complex yet
unavoidable challenge for producing meaningful, valuable, inclusive and sustainable
solutions for social and environmental issues. Human centered design can’t be con-
sidered a discipline exempted from evolutionary process. Its formalization in structured
834 G. Mincolelli et al.
methodologies is necessary for its validation and diffusion, but formalization doesn’t
mean crystallization. Every design research project is a test useful for improvement. In
the Habitat project our research group highlighted some critical issues of QFD and tried
to propose a new layout of it to improve its multidisciplinarity. In the Pleinair project
we will try to verify also the performances of it combined with other approaches, like
co-design and design for social improvement.
References
1. Italian Ministry of Health: General directorate of health prevention office 8. Guidelines on
physical activity for different age groups and with reference to physiological and physio-
pathological conditions and specific population subgroups (2017)
2. Physical activity strategy for the WHO European Region 2016–2025 © World Health
Organization (2015)
3. Mincolelli, G., Imbesi, S., Marchi, M.: Design for the active ageing and autonomy: the role of
industrial design in the development of the “Habitat” IOT project in advances in design for
inclusion. In: Di Bucchianico, G., Kercher, P. (eds.) Proceedings of the AHFE 2017
International Conference on Design for Inclusion, 17–21 July 2017, The Westin Bonaventure
Hotel, Los Angeles, California, USA, pp. 88–97. Springer, Heidelberg (2017)
4. Franceschini, F.: Quality function deployment, Franceschini Fiorenzo. Il Sole 24 Ore, Libro,
HOEPLI.it, Il Sole 24 Ore. HOELPI, Milano, Italy (2003)
5. Borelli, E., Paolini, G., Antoniazzi, F., Barbiroli, M., Benassi, F., Chesani, F., Chiari, L.,
Fantini, M., Fuschini, F., Galassi, A., Giacobone, G.A., Imbesi, S., Licciardello, M., Loreti,
D., Marchi, M., Masotti, D., Mello, P., Mellone, S., Mincolelli, G., Raffaelli, C., Roffia, L.,
Cinotti, T.S., Tacconi, C., Tamburini, P., Zoli, M., Costanzo, A.: HABITAT: an IoT solution
for independent elderly. In: Pau, G., Salerno, V.M., You, I., Sharma, V. (eds.) Sensors, vol.
19, no. 5, p. 1258 (2019). In Special Issue “Internet of Things for Smart Homes” 1–23 (2019)
6. Mincolelli, G., Marchi, M., Imbesi, S.: Inclusive design for ageing people and the Internet of
Things: understanding needs in advances in design for inclusion. In: Di Bucchianico, G.,
Kercher, P. (eds.) Proceedings of the AHFE 2017 International Conference on Design for
Inclusion, 17–21 July 2017, The Westin Bonaventure Hotel, Los Angeles, California, USA,
pp. 98–108. Springer, Heidelberg (2017)
7. Mincolelli, G., Marchi, M., Giacobone, G.A., Chiari, L., Borelli, E., Mellone, S., Tacconi, C.,
Cinotti, T.S., Roffia, L., Antoniazzi, F., Costanzo, A., Paolini, G., Masotti, D., Mello, P.,
Chesani, F., Loreti, D., Imbesi, S.: UCD, ergonomics and inclusive design: the HABITAT
project. In: Proceedings of the 20th Congress of the International Ergonomics Association
(IEA 2018) Volume VII: Ergonomics in Design, Design for All, Activity Theories for Work
Analysis and Design, Affective Design, pp. 1191–1202. Springer, Heidelberg (2018)
Design of an Innovative Furniture System:
Improving Acoustic Comfort in Coworking
Workplaces
Viola Geniola1(&), Stefania Camplone1, Antonio Marano1,
and Emilio Rossi2
1
Department of Architecture, University of Chieti-Pescara, Viale Pindaro 42,
65127 Pescara, PE, Italy
[email protected],
{stefania.camplone,a.marano}@unich.it
2
Lincoln School of Design, University of Lincoln, Brayford Pool,
Lincoln LN6 7TS, UK
[email protected]
Abstract. Coworking workplaces are shared production areas where the
emerging class of ‘Millennials’ can work together. Unlike traditional ‘work-at-
home’, this new business model shows clear pros for what concerns design-
driven innovation, sustainable economies and creative industries. However, noise
control and noise-related aspects need to be taken into account since they sig-
nificantly affect the productivity and the livability of the work environment. To
overcome this trend, we propose an innovative furniture system with integrated
acoustic insulation panels that meets the new demands for original solutions for
creative workplaces, mitigating the noise-related bad effects on workers.
Keywords: Design process Coworking Innovative furniture system
Acoustic comfort Smart material
1 Introduction
Since 2000’s the so-called ‘rise of creative class’, as described by Florida [1], signif-
icantly changed the way workers perform their job and, consequently, the way
workplaces have to be designed to meet the new needs creative workers. While
‘Millennials’ become the archetype of the emerging class of creative workers, which do
their businesses in new informal ways, coworking workplaces become the new pro-
duction areas where they operate, produce new know-how and meet each other.
Coworking workplaces can be defined as shared production areas, often placed in
open spaces, where creative companies, makers, start-ups, designers, writers, artists,
etc. promiscuously work together. Unlike ‘work-at-home’ business model, coworking
is a collaborative, flexible and voluntary work style based on mutual trust and on the
sharing of common core values between participants.
Beyond intrinsic business model’s pros, which is one of the most promising design-
driven research fields for sustainable economies and creative industries [2], coworking
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 835–841, 2020.
https://doi.org/10.1007/978-3-030-39512-4_127
836 V. Geniola et al.
workplaces show a number of critical factors that could affect the quality of work
environment. Among these aspects, noise is one the most critical one to take into
account [3]. Today, the noise control in workplaces is mainly operated using foam-
made insulation panels placed, for example, on walls, ceilings and desks.
Acoustic insulation panels are essentially produced following performance-oriented
criteria, relegating the ‘innovation factor’ only in the search for better isolation per-
formances. Considering coworking’s characteristics, current acoustic insulation panels
could be inadequate to meet the end-users’ creative-centric and flexibility-driven needs
[4]. New solutions can be developed using rigorous design processes to improve
workplaces’ noise comfort and to conceive cutting-edge furniture systems.
This paper presents a research developed at the University of Chieti-Pescara in Italy
as a part of a M.Sc. Thesis in Architecture focused on the theme of office furniture
design [5]. Part of this study has been developed in collaboration with an Italian firm –
Metalway S.r.l. – operating in office furniture design since 2000, and on the production
of acoustic control panels since 2016. The results obtained demonstrated that the use of
a design-driven innovation approach improves both the development of new products
and the search of promising market niche for future innovations.
2 Aims
While coworking areas have inherited traditional office furniture solutions that do not
meet Millennials’ smart working requirements (i.e. flexibility, nomadism, networking,
etc.), this research focused on the design of a new furniture system, specifically con-
ceived to ensure the satisfaction of coworkers’ flexibility-related needs, with a par-
ticular attention on the mitigation of acoustic noise. In particular, the research aimed to:
• Explore new design-driven prospects and meta-design insights for the innovation in
coworking workplaces, beyond the design of ‘classic’ furnishings.
• Design of a new furniture system to improve the acoustic comfort in coworking
workplaces, using focused design guidelines, technical integrations and ‘smart
materials’.
3 Methodology
This study followed a three phase ‘Design Process Methodology’ combining con-
ventional Design Research approach and focused meetings with the company’s
managers:
1. Internal audit to investigate acoustic features in coworking workplaces and to find
the most promising topics for later design developments.
2. Analysis of end-users interactions in coworking workplaces.
3. Integration between acoustic factors and concept design of new furnishing systems
with sound-absorbing panels.
Design of an Innovative Furniture System 837
3.1 Audit-Based Investigation of Acoustic Features in Coworking
Workplaces
The internal audit with the company’s managers, made using interviews, observations,
data analyses and evaluations, defined the main elements on which to develop the
Design Brief. The audit addressed the main topic of the research: the design of
‘acoustically performing furniture systems for co-working workplaces’.
The focus on acoustic comfort, besides the chance of improving the company’s
know how in noise control, is deeply linked to the will to invest in coworking
workplaces, since the current market evidences have shown that in the near future,
there will be an increasing in the flexibility of intellectual jobs; thus, coworking con-
nects people from different backgrounds and allow them sharing knowledge, resources,
but at the same time, production spaces in order to build business-driven communities
and stakeholders’ networks. The existence of informal shared areas is crucial to cat-
alyze the workers’ interaction, but also it increases noise discomfort: while 86% of
workers appreciate interaction with other people, 70% of them affirm that they would
be more productive in a less noisy work environment [6]. The relevance of sound
discomfort is underlined by World Health Organization’s (WHO) studies about the
impact of noise on health and work-related diseases [7]: workplaces whose sound
intensity level is about 60 dB generate annoyance, which stands for stress, dissatis-
faction and concentration problems. This disease is caused also by sound reverberation,
which is the acoustic phenomenon affecting large-scale environments such as open
spaces, where co-working usually operate. The research outlined the sound comfort as
one of the most important human needs to be converted into specific design require-
ments; later, a preliminary study focused on acoustics comfort was performed to
understand what sound-related variables the next design phase should have to be taken
into account.
Referring to the activities mainly characterizing coworking, it is possible to classify
the production areas in three categories: (a) in-between areas, (b) relax areas and
(c) working areas. For each one, detailed information about sound sources, intensity
levels and frequency were pointed out. The classification showed that voice is the most
disturbing factor since its frequency ranges between 0.17 kHz and 4 kHz, which
overlaps with the maximum human hearing sensitivity’s range: from 2 kHz to 5 kHz
[8]. Thus, informal workplace conversations and murmurs are constant and increase the
background noise, generating annoyance feelings.
3.2 Coworkers Analysis
By definition coworkers have different characteristics, including: age, job, skills, life-
style, social and cultural attitudes. An overall coworkers’ picture can be made using the
‘age criteria’; referring to people born from 1945 to 2010, there are: (a) baby boomers,
(b) X and Y generations and (c) Millennials. Each generation has been linked to work
skills and kind of job to identify main coworkers’ profiles as: (a) independent con-
tractors and advisors, (b) freelancers and (c) digital creative professionals – as digital
nomads and team workers.
838 V. Geniola et al.
While all coworkers want and benefit to belong to a community, they express
different job-related flexibility and privacy’s needs. Startupers, for example, aim to
work together in informal ways, on the other hand researchers need concentration while
writing an article and want soft conversation with other colleagues only when they
decide to have a break. Accordingly, the research challenge was to allow the coexis-
tence, in the same environment, of several work-related performances without affecting
the end-users’ productivity and wellbeing.
The way in which space enhances the relationships between people and work
equipment is shown in Fig. 1; it also shows the compatibility/incompatibility of
activities through visual and sound permeability. This model has been also used to
developed a new one based on flexibility and privacy modulation, which later was used
to implement the design requirements.
Fig. 1. Correlation between people, work activities, work equipment and space [5].
3.3 Design Concept of a Furniture System with Sound-Absorbing
Elements
Once established the importance of the acoustic factors, the research examined the their
connectability with furnishing: stand-alone superstructure, additional sound-absorbing
elements to existing furniture and, finally, integrations with new furniture systems.
Considering that every surface existing in coworking spaces influence the sound per-
formance – according to materials – smooth and reflective ones increase reverberation,
porous and sound-absorbing ones limit it, then the last option is the most coherent with
the design brief.
The conversion of acoustic factors into design requirements was based on the
analysis of how acoustic requirement and sound-absorbing coworkers’ needs change
work equipment. The design concept idea for an innovative furniture system that
mitigates the noise in coworking workplaces is composed by a family of different
Design of an Innovative Furniture System 839
elements, which can be equipped and later freely combined together in order to ensure
different space layout and uses by coworkers.
The concept design (Fig. 2) is inspired on one of the fundamental characters
defined by coworkers: flexibility. This is morphologically represented as a surface,
belonging to each furnishing component, which can change its shape depending on the
context where it will be placed and, mostly, the coworkers’ needs. Accordingly, all
sub-products composing the furniture system, give the chance to multiply the possi-
bilities of use: from office and relax pods elements to the desking system.
Fig. 2. Furniture system with flexible sound-absorbing elements: concept design [5].
4 Results
The result of the research was the design of an innovative furniture system. While three
elements were initially conceived – (a) relax pod, (b) desking system, (c) office pod –
the final experimentation focused only on the design of the desking system.
The innovation introduced was not only related on the development of an innovative
set of solutions, but also on the use on new materials, which expanded the design
possibility in the shaping of more organic solutions, meeting then the phenomenological
propagation of sound waves. In particular, the use of ‘Clash’ [9], a new shape memory
alloy (SMA), introduced the formal changeability of solutions. Clash is a multilayered
structure made in soft & hard polyurethane and containing a SMA sandwich made of a
copper foil and titanium/nickel wires. The inner layer, when heated, produces a change
in the material shape and, when cooled, it preserves the shape before impressed. Clash
has acoustically performing outer layers made in polyurethane, which is a sound-
absorbing polymeric material with open cells. Clash fits bots the main requirements
contained in the design brief – sound absorption through porosity of polyurethane
material – and also the flexibility, by its shape-changing ability.
The desking system (Fig. 3) is a personal workstation that can be configured by two
privacy levels. The desk is formed by two layered surfaces: the upper surface, closer to
the end-users, is a flat and hard surface equipped with wireless recharging connectors;
the lower surface, instead, is made of Clash and can be curved upwards or be flatted. In
the first case, end-users are shielded from the surrounding space, for example when
they are more prone to concentration or to individual work activities; when the surface
is flatted, the visual contact with other coworkers is maximum, so people can relate and
co-work. While the upper desk is almost organically shaped, the Clash-made surface
has regular edges, which are functional to make easier the desks joining – this aspect
840 V. Geniola et al.
creates a clear distinction between private areas and shared ones where end-users can
connect themselves, work equipment and ideas.
Fig. 3. Desking system with movable Clash-made sound-absorbing surfaces [5].
5 Conclusions
Since companies’ organizational structure has changed towards flexibility and decen-
tralization, end-users’ needs are no longer tied to a single workplace or to a ‘static’
furniture; mobility and nomadism, time and place flexibility have influenced the
working activities toward new ‘smart’ meanings connecting people, skills, communi-
ties and networks. In this transition, coworking has become one of the most promising
business models influencing the design of work environments and, consequently, the
design of furniture to be used by Millennials, which have new needs less tied to a
standardized vision of work.
The preliminary design concept achieved using a rigorous design process and the
later design development of a desking system with movable Clash-made sound-
absorbing surfaces, demonstrated that it is possible to produce significant innovations
in furniture design for coworking workplaces; this design-driven innovation is not due
only to the fact the project meets specific end-users’ need, but it is also related to the
chance of have a set of products that, by combination, can generate multiple – ‘smart’ –
configurations to enable new generations of workers in facing complex tasks.
The coherence with the research results, based on clear research goals, proves the
relevant role of Design discipline in new product development process, which is
strategic to increase manufacturers’ innovativeness, competitiveness and quality.
Credits
This paper shows the research results achieved in a M.Sc. thesis in Architecture
developed at the University of Chieti-Pescara in Italy by Geniola [5]. The formal
Design of an Innovative Furniture System 841
writing of paragraphs is attributed to: Geniola for ‘Methodology’, Stefania Camplone
for ‘Results’, Antonio Marano for ‘Abstract’ and ‘Introduction’, and finally to Emilio
Rossi for ‘Aims’ and ‘Conclusions’. Authors would also like to thank Metalway S.r.l.’s
managers for their invaluable support throughout the research.
References
1. Florida, R.: The Rise of the Creative Class: and How It’s Transforming Work. Leisure and
Everyday Life. Basic Books, New York (2002)
2. Boyd, S.: Beyond The Office: Workplace as a Service (2014). Accessed. https://medium.com/
stowe-boyd/beyond-the-office-workplace-as-a-service-e70c94f84c69
3. Simonelli, G., Scullica, F., Elgani, E., Monna, V.: Can coworking spaces be built bottom-up?
In: Proceedings of ServDes 2018 Conference, Milan, Italy, pp. 761–771 (2018)
4. Duffy, F.: The New Office. Conran Octopus Limited, London (1997)
5. Geniola, V.: Design Process per un Sistema di Coworking ad Alto Comfort Acustico. M.Sc,
dissertation, University of Chieti-Pescara (2019)
6. American Society of Interior Designers. http://asid.org/
7. World Health Organization: Environmental Noise Guidelines for the European Region
(2018). World Health Organization, Geneva (2018)
8. Spagnolo, R.: Manuale di Acustica Applicata. Città Studi Edizioni, Turin (2005)
9. Tecnologie e Materiali per il Design. https://tecmatied.wordpress.com/
Modeling of Subcutaneous Implantable
Microchip Intention of Use
Mona A. Mohamed(&)
Towson University, MD 8000 York Road, Towson, MD 21252, USA
[email protected]
Abstract. The adoption of the Radiofrequency identification (RFID) as a
subcutaneous implantable microchip (RSIM) in human is very low. The broader
objective of this research is to model the behavioral intention according to the
challenges of RSIM use. RSIM has been widely used in the medical applica-
tions, access control, authentication and maintaining personal information.
However, there is undoubtedly much of health, ethical, legal, religious, security
and technical concerns that slowed down its implementation. This study is
designed to build the behavioral intention model of RSIM use based on the
technology acceptance model (TAM) as a priori hypothetical model and will be
validated by structural equation modeling (SEM) as supported by the survey
collected data. The investigation includes testing nine proposed hypotheses
based on data to be collected using survey instrument designed for confirmatory
purposes to examine the intention of using RSIM by randomly selected 200
subjects.
Keywords: Microchip Structural modeling RFID Technology acceptance
model Ethics Risks Religion Cost Security Privacy
1 Introduction
Radiofrequency identification (RFID) technology have long been used to identify and
track the identity, location of pets in real-(Michael and Michael 2014). However, the
‘chipping’ of humans is rare and remains mostly a topic of debate into the future.
Currently, major uses of human implanted microchip or RFID subcutaneous implan-
table microchip (RSIM) include identification, tracking, access control, medical
applications and electronic payments. Nevertheless, as the very nature of cutting-edge
digital innovations, these progressions are not free of problems, as there are many
challenges and risks that originate from controversial societal issues associated with
RSIM implementation. On the whole, there is undoubtedly much of health, ethical,
legal, religious, security and technical concerns. In view of these persuasions, this study
is endeavors to build the behavioral intention model of RSIM based on extended
Technology Acceptance Model [TAM] and will be validated by structural equation
modeling (SEM). (Davis 1985) proposed TAM and reported that “TAM would enable
system designers and implementors to evaluate proposed new systems prior to their
implementation.” Little attention has been paid to studies that employ TAM in RSIM
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 842–847, 2020.
https://doi.org/10.1007/978-3-030-39512-4_128
Modeling of Subcutaneous Implantable Microchip Intention of Use 843
usage (Hossain and Prybutok 2008). The purpose of this study is to construct and
evaluate modified TAM model of behavioral intention of use to adopt RSIM.
1.1 Technological Underpinnings and Usage
Microchip implant is an “integrated circuit devices encased in RFID transponders that
can be active or passive and are implantable into animals or humans usually in the
subcutaneous layer of the skin” (Stephan et al. 2012). The active tag powered by
internal battery and on-tag transmitter using built-in antenna in the chip; while the
passive tag is powered only by power generated from the reader (Herber 2006). RSIM
can help in minimizing mistakes in personally identifiable information (PII) or personal
medical information such as blood type, allergies, current medications and medical
history.
Then, SEM will be used to fit the gathered data for building behavioral intention of
users to use RSIM. These two techniques can be explicated as follows:
Technology Acceptance Model. TAM has dominated the field of assessment of
information technology acceptability (Scherer et al. 2019). The hypothetical model of
this proposal consists of the core three factors of TAM which are introduced in the
original model. These factors are labeled in this research as the ‘user motivation’,
which consists of the behavioral intention of use (BIOU) and the two foremost pre-
dictors that influence the use of technology viz. perceived usefulness (PU) and per-
ceived ease of use (PEOU).
Structural Equation Modeling. SEM is used to examine plausibility of conceptual
declarations regarding the associations of the model. It is primarily confirmatory
technique (Berkout et al. 2014). SEM will be used in this study to test the validity and
reliability of the measurement model including the effectiveness of the indicators and
the underline latent constructs of the model. The main function of SEM is to compare a
priori hypothetical model to that model which results from empirical survey data. If the
fit is statistically significant, then the relationship between measurement model and the
structural model is supported by the collected data.
2 Methodology
The methodology consists of the proposed conceptual model with its six arguments as
latent factors, research model construction represented in path drawing, sample size and
objectives and hypotheses:
2.1 The Proposed Model
The problem under investigation in this proposal is depicted in the conceptual TAM
model in Fig. 1. Which can be divided into two mediating groups of constructs:
(a) User motivation; and (b) External factors:
844 M. A. Mohamed
User Motivation: This user motivation factors consists of BIOU the main target of the
investigation and its two main factors namely, PU and PEOU:
1. Perceived Usefulness: PU is defined as “the degree to which a person believes that
using a particular system would enhance his or her job performance”. Although
(Rouibah 2008) described PU as “a form of extrinsic motivation”, it is regarded as a
determining factor in this study due its significant influence on the adoption of
behavioral intentions and can also be attributed to its dependency on the effects of
the six external factors.
2. Perceived Ease of Use: PEOU refers to “the degree to which a person believes that
use of a particular system would be free of effort” (Davis 1989). In general, PEOU
has indirect influence on the adoption of behavioral intentions (Werber et al. 2018).
PEOU is when users believe in the new technology for saving time and effort,
accordingly, it positively affects PU. So, when PEOU is higher, PU will be higher
too (Huang et al. 2011).
2.2 External Factors
In addition to the user motivation factors, the hypothetical modeling scope is extended
to six external behavioral constructs. These external factors are explicated in the
arguments below and depicted in the Fig. 1:
1. Ethical Issues Perception (EIP): EIP and the demand are philosophically two
antagonistic distinct streams of thoughts. For instance, monitoring and tracking of
people of the society may be endurable depending on the severity of the situation,
but tracking for non-disabled may be intolerable by many. Investigators raised a
number of legitimate ethical or moral standpoint and concerns of RSIM that may
directly affect the intention of use (Bauer 2007).
2. Religious Concerns Perception (RCP): Implanting RSIM may violate the under-
lying principles of spiritual beliefs for some religions, which is measured in this
proposal by religious concerns perception that has a direct effect on BIOU (Ziv-
otofsky and Zivotofsky 2008). Indeed, fundamentalist Christian groups prohibit
implanting RSIM, as it is considered “marks of the beast.” While both Judaism and
Islam forbid tattoos and they may prohibit implanted RFID tags for the same
reasons (Foster and Jaeger 2008)
3. Perceived Treatment Cos (PTC): Cost and technical complexity remain hindrance
to the implementation of some RSIM applications. The cost increases with the
increase of the subscriptions and plan options of the RSIM (Foster and Jaeger
2008). An active RSIM with faster reading speed and longer reading ranges typi-
cally cost more (Smith 2007). This high cost may also result in discriminatory
problems within the society as beau monde elite class can afford the acquisition
active RSIM, while others cannot
4. Health Risks Perception (HRP): HRP associated with a multitude of technologies
and applications is evident (Kosta and Bowman 2011). This is due to the fact that
there are many unforeseen health risks and concerns associated with the chip
implantation (Nisbet 2004) such as tissue reaction and electronic interference
and electrical hazards (Smith 2008). In addition, the health risks of introducing
Modeling of Subcutaneous Implantable Microchip Intention of Use 845
foreign-body-induced tumors, health risks perception (HRP) is associated with
incorrect injection (Albrecht 2010).
5. Argument 5: Privacy Risks Perception (PRP): RSIM may be exploited to invade the
personal space using many of its own vulnerable applications. In addition, medical
or identification RSIM could be augmented with sensors that append extra overt or
covert functionalities (Bramstedt 2005) i.e. function creep. Furthermore, versatile
implantable microchips have the promise to interface with multiple appliances and
equipment (Brabyn 2006). However, RSIM raises privacy risks perception and the
likelihood of coercive implementation in individuals (Foster and Jaeger 2008).
6. Argument 6: Security Risks Perception (SRP): There are many data and human
security risks associated with the implementation of RSIM (Paaske et al. 2017). The
most serious risk of RSIM originates from the possible attack on the centralized or
interconnected database where the information is kept (Gadzheva 2007). This kind
of intrusions may form a potential security threats to exposure of personal identi-
fiable information (PII) of users and tapping on the data integrity of their medical
records.
2.3 Research Model Construction
The conceptual model specifications have been achieved by intensive review of prior
research literature which yielded the six major external factors for evaluating the
samples behavioral intention to use RSIM. As depicted in the conceptual model Fig. 1
there are 24 model definition equations. Each of the 8 latent factors relates to 3
variables that must show convergent validity.
Fig. 1. The conceptual model of RSIM possible challenges that affect RSIM intention of use. ʎi
is factor loading of indicators on the latent factors, ɸi stands for covariances between exogenous
factors, Di denote the disturbance, bi are the coefficients of the paths of latent factors
846 M. A. Mohamed
2.4 Objectives and Hypotheses
Based on Fig. 1 variable presumptions and propositions regarding the challenges of
behavior towards RSIM 9 Null hypotheses are proposed to be tested are:
The three main user motivation hypotheses to be tested are:
H1 Perceived ease of use has no direct effects on behavioral intention of RSIM use
H2 Perceived usefulness has no direct effect on behavioral intention of RSIM use
H3: Perceived ease of use has no direct effects on perceived usefulness of RSIM
The six external factors (challenges) hypotheses will be tested as follows:
H4: Religious concerns perception has no effect on behavioral intention of RSIM use
H5: Perceived implanting cost has no effect on perceived usefulness of RSIM
H6: Health risk perception has no effects on perceived usefulness of RSIM
H7: Privacy risk perception has no effects on perceived usefulness of RSIM
H8: Security risk perception has no effects on perceived usefulness of RSIM
H9: Ethical issues perception has no effects on behavioral intention of RSIM use
3 Data Gathering and Analysis
The investigation to test the proposed nine hypotheses will be based on a survey
instrument that is designed for exploratory purposes to examine the intention to use the
chipping as a solution for different types of problems. 200 subjects will be randomly
selected from the sampled population with various disabilities. All participants will be
18 years or older to answer the questions to the best of their knowledge. Each construct
of the conceptual model contains three related questions to describe the factor, where
each question is measured by five-dimension scale of Likert-type rating with values
based on their intension to use RSIM. Subjects will check one of five choices ranked as
1 = Strongly disagree; 2 = Disagree; 3 = Neither Agree nor Disagree; 4 = Agree and
5 = Strongly agree. The stated hypotheses will be subjected to statistical tests to assess
the significance of the major components under consideration. In particular, SAS PROC
CALIS will be used to test the goodness-of-fit and the hypothetical paths by calculating
factor loading of the observed variables on its relevant factor (slopes) i.e. hypothesized
paths coefficients, which express the correlation between the latent variables. The
conceptual model as shown in Fig. 1 will be tested against the sample data from the
questionnaire using fit indices such as v2. If the goodness-of-fit results show that the
model doesn’t fit to the gathered data, then a scale purification of the model is inevitable.
References
Albrecht, K.: Microchip-induced tumors in laboratory rodents and dogs: a review of the literature
1990–2006. In: IEEE International Symposium on Technology and Society, 2010 Wollon-
gong, NSW, Australia, pp. 337–359. IEEE (2010)
Bauer, K.A.: Wired patients: implantable microchips and biosensors in patient care. Camb.
Q. Healthc. Ethics 16(3), 281–290 (2007)
Modeling of Subcutaneous Implantable Microchip Intention of Use 847
Berkout, O.V., Gross, A.M., Young, J.: Why so many arrows? Introduction to structural equation
modeling for the novitiate user. Clin. Child Fam. Psychol. Rev. 17(3), 217–229 (2014)
Brabyn, J.: Future directions in blindness and low vision perspectives from a state-of-the-science
conference. Vis. Impair. Res. 8(3), 61–66 (2006)
Bramstedt, K.A.: When microchip implants do more than drug delivery: blending, blurring, and
bundling of protected health information and patient monitoring. Technol. Health Care 13(3),
193–199 (2005)
Davis, F.D.: A technology acceptance model for empirically testing new end-user information
systems: theory and results. Ph.D. in Management, Massachusetts Institute Of Technology
(1985)
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information
technology. MIS Q. 13(3), 319–340 (1989)
Foster, K.R., Jaeger, J.: Ethical implications of implantable radiofrequency identification (RFID)
tags in humans. Am. J. Bioeth. 8(8), 44–48 (2008)
Gadzheva, M.: Getting chipped: to ban or not to ban. Inf. Commun. Technol. Law 16(3), 217–
231 (2007)
Herber, W.A.: No direction home: will the law keep pace with human tracking technology to
protect individual privacy and stop geoslavery? I/S J. Law Policy. 2(2), 409–473 (2006)
Hossain, M.M., Prybutok, V.R.: Consumer acceptance of RFID technology: an exploratory
study. IEEE Trans. Eng. Manag. 55(2), 316–328 (2008)
Huang, Y.C., Tsay, W.D., Huang, C.H., Lin, Y.H., Lai, M.C.: The influence factors of electronic
bill presentment and payment. In: IEEE 2nd International Conference on Artificial
Intelligence, Management Science and Electronic Commerce (AIMSEC), 2011 Dengleng,
China, pp. 4844–4847. IEEE (2011)
Kosta, E., Bowman, D.M.: Treating or Tracking? Regulatory challenges of nano-enabled ICT
implants. Law Policy. 33(2), 256–276 (2011)
Michael, M.G., Michael, K.: Uberveillance and the Social Implications of Microchip Implants:
Emerging Technologies. Information Science Reference (an Imprint of IGI Global),
Hershey PA (2014)
Nisbet, N.: Resisting surveillance: identity and implantable microchips. Leonardo 37(3), 210–
214 (2004)
Paaske, S., Bauer, A., Moser, T., Seckman, C.: The benefits and barriers to RFID technology in
healthcare. Online J. Nurs. Inform. (OJNI) (2017)
Rouibah, K.: Social usage of instant messaging by individuals outside the workplace in Kuwait: a
structural equation model. Inf. Technol. People. 21(1), 34–68 (2008)
Scherer, R., Siddiq, F., Tondeur, J.: The technology acceptance model (TAM): a meta-analytic
structural equation modeling approach to explaining teachers’ adoption of digital technology
in education. Comput. Educ. 128, 13–35 (2019)
Smith, A.D.: Evolution and acceptability of medical applications of RFID implants among early
users of technology. Health Mark. Q. 24(1/2), 121–155 (2007)
Smith, C.: Human microchip implantation. J. Technol. Manag. Innov. 3(3), 151–161 (2008)
Stephan, K.D., Michael, K., Michael, M.G., Jacob, L.: Social implications of technology: the
past, the present, and the future. Proc. IEEE 100, 1752–1781 (2012)
Werber, B., Baggia, A., Žnidaršič, A.: Factors affecting the intentions to use RFID subcutaneous
microchip implants for healthcare purposes. Organizacija. 51, 121–133 (2018)
Zivotofsky, A.Z., Zivotofsky, N.T.S.: Implantable radiofrequency identification (RFID) tags are
not Tattoos. Am. J. Bioeth. 8(8), 52–53 (2008)
A Brief Analysis of the Status Quo and Trend
of Wearable Smart Jewellery Devices Design
Jing Liu and Ken Nah(&)
The International Design School for Advanced Studies (IDAS),
Hongik University, Seoul 03082, Republic of Korea
[email protected], [email protected]
Abstract. With the recent bustling development of the world’s smart manu-
facturing industry, the big-scale of the smart jewellery market has been received
great attention as well. Through analysing representative products, the purpose
of this research is to discover the deficiencies of existing wearable accessories
and figure out how can better combine the traditional products design and new
technologies together to improve these defects. The findings of this research
reveal that there are certain deficiencies of existing wearable accessories prod-
ucts, in terms of functions, forms and among others. This study concludes that it
is necessary for the jewellery designers to be aware and the response to the
industry trend; taking the use of the original target that smart wearable tech-
nology is to explore a new way of human-computer interactive way, through
which the personalised and exclusive services can be provided to the end-users
so as to better satisfy the needs of individual costumers.
Keywords: Wearable technology Smart devices Jewellery product design
1 Introduction
The global market for wearable electronic devices has been fast-growing. According to
the leading research and advisory company Gartner’s data report [1], the revenue of
2017 has already been exceeded $30 billion, and its sales are predicted to exceed $55
billion in 2021. Based on Technavio’s 2019–2023 Global Wearable Electronics Market
report [2], multi-factors drive this worldwide market’s development. Majorly, digital-
isation has promoted the growth of the electronics market under the global economic
context, the user demand has been consequently built. The use of smart types of
equipment is devoted by wearable devices which combines augmented reality, artificial
intelligence and all the other cutting-edge technologies together. Furthermore, the
availability of wearable compatible applications on smartphones has been increasingly
developed, which also bring a higher reputation and extensive acceptance of this area.
Meanwhile, the consumption of the global jewellery market has remained stable at
around $300 billion in recent years. And affected by the global economic recession, the
purchase desire of jewellery consumers now are more inconstant than ever. Nonethe-
less, the jewellery industry is still a dynamic industry that both its customers’ behaviour
and itself have been changing correspondingly and growing rapidly to face the
potential risks and opportunities. Warren Buffett, a well-known investor, who
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 848–852, 2020.
https://doi.org/10.1007/978-3-030-39512-4_129
A Brief Analysis of the Status Quo and Trend of Wearable Smart Jewellery 849
announced to invest in the smart jewellery sector in 2017. He said that adding the value
of technology is a way to make the traditional jewellery industry better match this
digital age’s needs. Therefore, becoming a new focal point in the jewellery industry
which combines the wearable technology and jewellery products together. A lot of
well-known IT companies, such as Google, Apple and Samsung among others, also
have begun to explore in this rising area and engage to find a place in this huge market.
In my perspective, as a researcher with a jewellery design background and human
factor researching knowledge, I am trying to capture the new tendency of conversing
wearable devices into jewellery products. Through briefly investigating the represen-
tative cases in the nowadays market, the propose of this research is to discover the
deficiencies of the existing wearable smart jewellery products and figure out what is the
potential solutions for better balancing and combing the traditional products design and
new technologies together in order to better satisfy the consumers’ needs.
Firstly, this paper will briefly introduce the background of wearable technology and
smart devices; then give the historical and theoretical context of applying the wearable
technology and smart devices on jewellery product design. Secondly, some represen-
tative existing smart jewellery products are investigated as case studies to discover the
deficiencies of existing wearable accessories; then analyse and interpret how to better
combine the traditional products design and new technologies together to improve
these defects. Finally, it will try to draw the conclusion with some suggestions and give
a direction for further studies.
2 Historical and Theoretical Studies
Wearable technology, an innovative technique which was first presented by the Media
Lab of MIT in the 1960s [3]. Later then, the forms of wearable smart devices have been
starting to be widely applied in the use among industrial, military, educational, medical
and entertainment among other areas. The phrase “wearable technology” normally
refers to these electronic devices which can carry out many computing tasks and be
worn on the body comfortably [4]. It is mostly used to explore and create wearable
smart devices. Through this technique, multi-media, wireless communication and
sensors can be inserted into the clothes so as to support various interactions including
eyes-track, gestures and among others [5].
The number of wearable devices has gradually moved from conceptual idea to
commercial application and the latest wearable devices have been constantly intro-
duced, due to the development of Mobile Internet technology and the maturity of core
hardware technologies. There is a wide range of examples can be given as wearable
devices, which includes but not limited by, smart clothing, glasses, watches, headbands
and jewellery among others. Among the jewellery section, for instance, bracelets, rings,
necklaces, and the earrings designed for hearing aid function [4].
Wearable technology was speedily gained popularity, but it has not been that
smooth for this rise of smart devices. The Hamilton Pulsar Digital Watch released in
the 1970s is considered as the first true commercial design wearable product [4]. After
the following few decades, several corporations have attempted to produce wearable
devices without any real successful ones. However, as the progress of technology and
850 J. Liu and K. Nah
manufacturing, consumers’ demand for wearable devices has been better understood.
Thus, the sort of products began to succeed, for example, Apple Watch and Fitbit are
the famous ones recently [4].
3 Case Study
As the Table 1 shows, both start-up and big-name companies, the existing smart
jewellery products design is mainly based on these four logical types. Here in this part,
only two representative products will be specifically demonstrated as cases to further
analyse the status quo.
Table 1. Overview about the main types of existing smart jewellery products
Logical Universal wearable Emotional Special purpose Stylish
properties devices interaction design
Main functions Message Memory Position tracking, General
notification, data storage, alarm system and functions
monitoring, health Remote etc.
information/medical interaction
records etc. Cloud
storage and
etc.
Target users General consumers Primarily Special user Primarily
target groups, such as, target
millennials females, children female
and the aged consumers
people
Product forms Bracelets, Necklaces, Bracelets, General
smartwatch, rings rings, necklaces, rings, forms,
and etc. earrings earrings and etc. but more
and etc. Depends on decorative
different user
groups
Representative Fitbit Flex, Apple Google CUFF jewellery Swarovski,
products Watch, Samsung glass, collection… Fossil,
smartwatch… Totwoo Ringly…
necklace…
Ringly, a start-up smart jewellery brand based in the states, whose design mainly
targets females with the purpose of offering them a healthy modern lifestyle. The
bracelets and rings are connected to users’ phone Bluetooth in order to track users’
daily activity, remind calls and messages, and also follow the real-time social media
updates. There are many choices of its gemstone, such as moonstone, Lapis lazuli and
crystal among others, and its function can be told by the surface’ s breathing lamp and
vibration [6]. Ringly’s products have been popular once it was launched. To be
A Brief Analysis of the Status Quo and Trend of Wearable Smart Jewellery 851
specific, their modern and minimalist design style are attractive, especially for those
stylish Instagram-orientated female users. However, the functions of these products are
very basic and similar to the normal smartwatchs, the real user experience is less than
expected [7].
Richline, the biggest jewellery manufacturer around the states, which is wholly-
owned by Berkshire Hathaway announced to launch its smart jewellery brand Ela.
Similarly, the wearable jewellery line Ela also links people’s activities to mobile
devices through Bluetooth to provide the functions of health data recording, messages
reminding and among others. Moreover, Ela puts emotional interaction as its unique
feature. Specifically, cloud storage is used to connect Ela’s own app for uploading the
memories to jewellery products. People who are special to users can be also added to
the system to enhance their smart emotional links [8].
There is a common sense that people show fewer interests in purchasing fine
jewellery products these days because of the weak economic situation, especially for
the millennial generation. In order to attract potential consumers, it is crucial for
jewellery brands to find an innovative way for the design of their products. Compared
to visible things, people are attracted more by the intangible experiences which can be
well connected to their needs. In a word, a significant change about the evaluation of
jewellery products purchase, accessibility and design have already been brought by
emerging wearable smart technologies [9].
4 Conclusion and Future Studies
This paper is focused on the basic research upon the status quo of wearable smart
jewellery devices design, which aims to explore the deficiencies of the existing
wearable smart jewellery products and try to predict what is the future trend of smart
jewellery product design development for better satisfying the consumers’ needs.
Under the industrial context of wearable technology and traditional jewellery products
design, it was necessary to follow the mainstream to combine the smart devices and
accessories together. Through briefly analysing existing cases, this study found that in
terms of functions, forms and among others, the smart jewellery products still have a
big room to improve.
Overall, the smart jewellery design has been keeping moving forward followed the
new technology changes. Besides developing the existing products’ deficiencies, smart
jewellery design should be further considered in the wider human factor context.
Importantly, with the mobile internet technologies’ rapid developing, wearable tech-
nology shows its great potentials for further uses. There is a suggestion that reflecting
on the original target of smart wearable technology, which is to explore a new human-
computer interactive way, costumers can be better satisfied by providing the individual
product services.
For further studies of this research, a large range of cases will be included for
analysis so as to produce a more representative result. User research will be conducted,
based on the theories of human-factors engineering, to investigate the potential needs
of smart jewellery products’ end-users.
852 J. Liu and K. Nah
References
1. Gartner: https://www.gartner.com/en/newsroom/press-releases/2018-11-29-gartner-says-worl
dwide-wearable-device-sales-to-grow-
2. Technavio: https://www.technavio.com/report/global-wearable-electronics-market-industry-an
alysis
3. MIT Media Lab: https://www.media.mit.edu/wearables/lizzy/timeline.html#1966a
4. Wearable Devices Magazine: http://www.wearabledevices.com/what-is-a-wearable-%20de
vice/
5. McCann, J., Bryson, D. (eds.): Smart Clothes and Wearable Technology. Elsevier, Amsterdam
(2009)
6. Ringly: https://ringly.com
7. Business Insider: https://www.businessinsider.com/ringly-wearable-tech-ring-review-2015-8
8. Richlinegroup: https://richlinegroup.com
9. Tech.co: https://tech.co/news/technology-impacting-jewelry-industry-2017-10
Accessibility Evaluation of Video Games
for Users with Cognitive Disabilities
Luis Salvador-Ullauri1, Patricia Acosta-Vargas2(&),
and Sergio Luján-Mora1
1
Department of Software and Computing Systems, University of Alicante,
Alicante, Spain
[email protected], [email protected]
2
Intelligent and Interactive Systems Lab, Universidad de Las Américas,
Quito, Ecuador
[email protected]
Abstract. Currently, there are many students with cognitive problems to learn
new concepts in theoretical classes. For students to improve the learning process
from the knowledge transmitted in class, we propose a method to motivate the
development of computer applications to reinforce the learning of new concepts.
This research presents a method for evaluation of accessible video games. In
order to evaluate accessibility in a videogame, the Web Content Accessibility
Guidelines 2.1 will be considered, which includes barriers for users with cog-
nitive disabilities. This category of users involves people with limited ability to
process and memorize information, make decisions, or learn. The proposed
method can be replicated for users with other types of disabilities and can be
applied during the game development cycle to ensure a more inclusive and
accessible product.
Keywords: Accessibility Cognitive Disabilities Evaluation Method
Video games WCAG 2.1
1 Introduction
Today, video games are fascinating to observe, fun to play, and very attractive to
students. Besides, educational video games allow students to develop useful cognitive
skills, which may be essential in the course of their lives. Education with video games
improves some aspects of cognitive functioning [1], such as memory, attention span,
and reaction time, as well as multitasking and general cognitive control [2]. At present,
the number of students with learning difficulties accessing university environments is
increasing [3–5], highlighting the importance of making adaptations that correspond to
the needs of the new students. In order to improve students’ understanding of the
different concepts taught in class, the design and development of accessible video
games are used. However, it is necessary to establish design guidelines to develop truly
accessible video games that improve the understanding of the information of each
subject. The Convention on the Rights of Persons with Disabilities (CRPD) [6]
establishes the right of access to information for all people, using accessible formats
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 853–859, 2020.
https://doi.org/10.1007/978-3-030-39512-4_130
854 L. Salvador-Ullauri et al.
and appropriate technologies that mainly assist people with different disabilities.
Therefore, accessibility parameters must be taken into account by developers when
programming applications [7]. Research is taking on the challenge of implementing
video games to improve the education of people with cognitive disabilities [8].
However, most research on games for cognitive disabilities are focused on therapy and
rehabilitation. Of this way, a work about the use of serious games [9] establishes a set
of design principles aimed at the appropriate development of educational games for
people with cognitive disabilities. In order to verify these guidelines and establish the
cognitive barriers presented in serious video games, a group of ten video games will be
evaluated using these design principles and those established by the Web Content
Accessibility Guidelines (WCAG) 2.1 [10]. These principles are used in order to
evaluate ten serious games taken from PhET project of the Colorado University.
Consequently, this study firstly proposes to apply these principles to evaluate devel-
oped serious games and then, to develop improved serious game using the lessons
learned.
The rest of the article is structured as follows: Sect. 2 includes the background and
related work. Section 3 presents the method; Sect. 4 presents the results and the dis-
cussion. Finally, Sect. 5 presents conclusions and future work.
2 Background and Related Work
Currently, there are several investigations on the use of educational video games to
improve the cognitive process of students of different ages. The Learning Strategies
Program, funded by the Defense Advanced Research Projects Agency, was the entity
that carried out the most systematic and remarkable attempt to use video games to
understand and improve human performance and cognition [11]. Many brain training
programs have been conducted and, in conclusion, it has been stated that video games
can significantly improve cognitive function [12]. However, even games that promote
intellectual challenge are not always effective in inducing cognitive improvement in
players [13]. Besides, training studies suggest that differences between non-players and
players are not only correlative but that there is a causal relationship between improved
perceptual and cognitive abilities and the use of educational video games [14]. Space
capability has been shown to play an essential role in achievements in the disciplines of
science, technology, engineering, and mathematics (STEM). One way to improve
spatial capacity is through video gameplay [15].
Therefore, these works suggest that playing video games may have perceptual and
cognitive benefits. However, the implementation of accessible video games intended to
aid cognitive development is a relatively new idea. Two studies reviewed whether
action video games can have a perceptual effect on children’s perception. First study
analyzes perceptual benefits of video game use [16], while the second study deals with
the cognitive benefits of using such video games [17]. These studies together conclude
that using video games improves working memory, but not perceptual processing
speed. Therefore, educational video games have been observed to have positive effects
that include improved learning, promotion of personal development, the ability to
motivate people, and mental/physical rehabilitation. It is necessary to bear in mind that
Accessibility Evaluation of Video Games for Users with Cognitive Disabilities 855
multimedia content, such as educational video games and simulations in general,
present accessibility challenges that must be addressed in order to maintain the
inclusion of e-learning [18]. Although the accessibility of websites is reasonably
covered, in other areas, such as the development of video games and serious simula-
tions, attempts are still being made to find the most appropriate way to improve
accessibility [19, 20]. In this context, educational accessibility technologies can affect
the future opportunities of people with cognitive problems due to limited means of
access. Thus, other work has focused on providing design guidelines for the design of
user interfaces and simple methodologies for the development of accessible educational
video games [19]. Therefore, serious games use strategies that are often applied in
special education, such as immediate feedback, context-based teaching and learning,
and repetition [21].
Thus, in some cases, the ineffectiveness of educational video games may be the
result of usability and accessibility problems not considered in their design [22]. It is
because of these design issues that many developers of educational video games and
simulations seek to create simple design interfaces that are attractive and fun while
improving users’ memory and attention skills [23]. Similarly, design guidelines have
been defined that continue to work well for evaluating finished games and have
increased awareness of accessibility in educational games [24]. WCAG include
requirements that address cognitive accessibility that is used to validate and generate a
subset of guidelines aimed at evaluating selected serious video games.
3 Method
Figure 1 summarizes the method applied by WCAG 2.1 for the evaluation of video
games, involving a process with nine sequential steps.
Fig. 1. Method for evaluating accessibility in video games
Serious video games were selected based on the relationship between game per-
formance and specific cognitive skills related to mathematics. The name of the ten
serious games evaluated are Area Builder, Area Model Algebra, Area Model Decimal,
856 L. Salvador-Ullauri et al.
Area Model Introduction, Area Model Multiplication, Arithmetic, Build a fraction,
Fraction matcher, Fractions equality and, Fractions mixed numbers. These video games
belong to the set of educational video games offered in the PhET project of the
University of Colorado1. The tasks performed in order to evaluate the selected edu-
cational video games include the considerations listed in Table 1, contains the iden-
tificatory, group, consideration, and, WCAG 2.1.
Table 1. List of tasks performed on the evaluated video games
Id Group Consideration Guideline WCAG
2.1
A Interface A limited number of objects Navigable
B Interface Enough size of objects Readable
C User control Enough time to interact with the Enough time
objects
D User control Button to restart the game Input assistance
E Identification with the Motivational elements Input assistance
game
F Identification with the Configurable elements Input assistance
game
G Feedback Rewarding mechanisms Input assistance
H Feedback Simple texts to show results Predictable
I Transmission of Representation of concepts clearly Readable
concepts
J Transmission of Implicit educational content Predictable
concepts
In order to identify the exact scenario to evaluate these video games is necessary to
consider that a game is usable when the players do not have to struggle with the game
interface, and the game mechanic is understandable to the player [25]. Five users who
had learning problems in Mathematics were selected to evaluate the games. According
to Nielsen et al., the best results come from testing with no more than five users. The
reason why it can be applied in the case of video game accessibility [26].
4 Results and Discussion
Figure 2 presents the barriers identified for the games evaluated by the five users.
According to the results obtained, all the games have a button to restart the game and
enough time to interact with the game elements.
1
https://phet.colorado.edu/es_PE/simulations.
Accessibility Evaluation of Video Games for Users with Cognitive Disabilities 857
Fig. 2. Barriers found by the evaluators
However, the evaluators consider that it is necessary to add more motivation ele-
ments and reward mechanisms. The data set for analysis and replication of the eval-
uation is available in the Mendeley repository2.
Frequently repeated errors are related to “Input assistance” and “Predictable” with
23% each, “Readable” with 22%, followed by “Navigable” with 17%, and “Navigable”
with 15%. We can observe that in users with a cognitive disability, the principle that is
repeated more frequently is the “Understandable.”
Many of the games have predictable, repetitive mechanisms that do not contribute
to a better understanding of the subject although they allow users to find simple
procedures to solve the proposed challenges.
5 Conclusions and Future Works
The inclusion of users with cognitive problems and the improved use of video games
for education can be achieved by reducing the number of barriers identified in edu-
cational video games, which should also be considered in the development of new
educational applications. The results of the educational video games evaluated reveal
that these applications do not reach an adequate level of accessibility to be used by
people with cognitive problems, although they comply with some of the accessibility
requirements. Therefore, we suggest considering improving the accessibility in the
future development of educational simulations and educational video games.
References
1. Toril, P., Reales, J.M., Ballesteros, S.: Video game training enhances cognition of older
adults: a meta-analytic study. Psychol. Aging 29(3), 706–716 (2014)
2. Anguera, J.A., Boccanfuso, J., Rintoul, J.L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong,
E., Larraburo, Y., Rolle, C., Johnston, E., Gazzaley, A.: Video game training enhances
cognitive control in older adults. Nature 501(7465), 97–101 (2013)
2
http://dx.doi.org/10.17632/tjm7cwcd46.1.
858 L. Salvador-Ullauri et al.
3. Nganji, J.T., Brayshaw, M., Tompsett, B.: Ontology-based e-learning personalisation for
disabled students in higher education. Innov. Teach. Learn. Inf. Comput. Sci. 10(1), 1–11
(2011)
4. von der Mühlen, S., Richter, T., Schmid, S., Berthold, K.: How to improve argumentation
comprehension in university students: experimental test of a training approach. Instr. Sci. 47
(2), 215–237 (2019)
5. Lambert, D.C., Dryer, R.: Quality of life of higher education students with learning disability
studying online. Int. J. Disabil. Dev. Educ. 65(4), 393–407 (2018)
6. World Health Organization (WHO): United Nations Expert Group Meeting on Building
Inclusive Society and Development through Promoting ICT Accessibility: Emerging Issues
and Trends (2012)
7. Díaz-Bossini, J.M., Moreno, L.: Accessibility to mobile interfaces for older people. Procedia
Comput. Sci. 27, 57–66 (2013). https://doi.org/10.1016/j.procs.2014.02.008
8. Torrente, J., del Blanco, Á., Moreno-Ger, P., Fernández-Manjón, B.: Designing serious
games for adult students with cognitive disabilities. LNCS, pp. 603–610 (2012). https://doi.
org/10.1007/978-3-642-34478-7_73
9. Tomé, R.M., Pereira, J.M., Oliveira, M.: Using serious games for cognitive disabilities.
LNCS, pp. 34–47 (2014). https://doi.org/10.1007/978-3-319-11623-5_4
10. World Wide Web Consortium: Web Content Accessibility Guidelines (WCAG) 2.1 (2018).
https://www.w3.org/TR/WCAG21/. Accessed 30 May 2019
11. Baniqued, P.L., Kranz, M.B., Voss, M.W., Lee, H., Cosman, J.D., Severson, J., Kramer, A.
F.: Corrigendum: cognitive training with casual video games: points to consider. Front.
Psychol. 5 (2014). https://doi.org/10.3389/fpsyg.2014.00234
12. Boot, W.R., Champion, M., Blakely, D.P., Wright, T., Souders, D.J., Charness, N.: Video
games as a means to reduce age-related cognitive decline: attitudes, compliance, and
effectiveness. Front. Psychol. 4 (2013). https://doi.org/10.3389/fpsyg.2013.00031
13. Pohl, C., Kunde, W., Ganz, T., Conzelmann, A., Pauli, P., Kiesel, A.: Gaming to see: action
video gaming is associated with enhanced processing of masked stimuli. Front. Psychol. 5
(2014). https://doi.org/10.3389/fpsyg.2014.00070
14. Ventura, M., Shute, V., Wright, T., Zhao, W.: An investigation of the validity of the virtual
spatial navigation assessment. Front. Psychol. 4, 852 (2013)
15. Van Ravenzwaaij, D., Boekel, W., Forstmann, B.U., Ratcliff, R., Wagenmakers, E.-J.:
Action video games do not improve the speed of information processing in simple perceptual
tasks. J. Exp. Psychol. Gen. 143(5), 1794–1805 (2014). https://doi.org/10.1037/a0036923
16. Blacker, K.J., Curby, K.M., Klobusicky, E., Chein, J.M.: Effects of action video game
training on visual working memory. J. Exp. Psychol. Hum. Percept. Perform. 40(5), 1992–
2004 (2014). https://doi.org/10.1037/a0037556
17. Torrente, J., del Blanco, Á., Moreno-Ger, P., Martínez-Ortiz, I., Fernández-Manjón, B.:
Implementing accessibility in educational videogames with <e-Adventure>. In: Proceedings
of the First ACM International Workshop on Multimedia Technologies for Distance
Learning – MTDL 2009 (2009). https://doi.org/10.1145/1631111.1631122
18. Grammenos, D., Savidis, A., Stephanidis, C.: Unified design of universally accessible
games. In: Universal Access in Human-Computer Interaction. Applications and Services,
pp. 607–616 (2007). https://doi.org/10.1007/978-3-540-73283-9_67
19. Jaramillo-Alcázar, A., Luján-Mora, S., Salvador-Ullauri, L.: Inclusive education: mobile
serious games for people with cognitive disabilities: Enfoque UTE 9(1), 53–66 (2018).
https://doi.org/10.29019/enfoqueute.v9n1.236. ISSN 1390-9363
20. Jaramillo-Alcázar, A., Luján-Mora, S., Salvador-Ullauri, L.: Accessibility assessment of
mobile serious games for people with cognitive impairments. In: 2017 International
Accessibility Evaluation of Video Games for Users with Cognitive Disabilities 859
Conference on Information Systems and Computer Science (INCISCOS), pp. 323–328.
IEEE. (2017). https://doi.org/10.1109/INCISCOS.2017.12. ISBN 978-1-5386-2644-3
21. Kwon, J., Lee, Y.: Serious games for the job training of persons with developmental
disabilities. Comput. Educ. 95, 328–339 (2016). https://doi.org/10.1016/j.compedu.2016.02.
001
22. Hersh, M.A., Leporini, B.: An overview of accessibility and usability of educational games.
Adv. Game-Based Learn. 1–40 (2013). https://doi.org/10.4018/978-1-4666-1987-6.ch001
23. Chi, H., Agama, E., Prodanoff, Z.G.: Developing serious games to promote cognitive
abilities for the elderly. In: 2017 IEEE 5th International Conference on Serious Games and
Applications for Health (SeGAH) (2017). https://doi.org/10.1109/segah.2017.7939279
24. Cairns, P., Power, C., Barlet, M., Haynes, G.: Future design of accessibility in games: a
design vocabulary. Int. J. Hum. Comput. Stud. 131, 64–71 (2019). https://doi.org/10.1016/j.
ijhcs.2019.06.010
25. Korhonen, H.: Evaluating playability of mobile games with the expert review method (2016)
26. Nielsen Norman Group: https://www.nngroup.com/articles/why-you-only-need-to-test-with-
5-users/. Accessed 28 Sept 2019
Design of Smart Devices for Older People:
A User Centered Approach for the Collection
of Users’ Needs
Silvia Imbesi(&) and Giuseppe Mincolelli(&)
Department of Architecture, University of Ferrara, Via della Ghiara 36, 44121
Ferrara, Italy
{giuseppe.mincolelli,silvia.imbesi}@unife.it
Abstract. In this paper, it will be related a part of a design doctoral research
that concerns the choice of the typology of users to analyzed to design smart
devices for elderly people with physical symptoms related to gait and move-
ment. The author will also explore which are the most advantageous ways to
extrapolate from users’ contributions their needs, that in User Centered Design
are considered essential to elaborate the project requirements.
Keywords: User Centered Design Design Research Design for older
people Design for PD patients User’s needs
1 Global Ageing
It is now generally acknowledged that the average age of the world population is
constantly increasing and it will remain a constant trend for years to come, particularly
in the most developed countries [1]. Without going into detail about the causes of this
phenomenon, it is now necessary for our society to ponder how to face its conse-
quences: one of the most obvious is the increase in public spending due to the shortage
of economic and social institutional arrangements and policies that can guarantee a
peaceful, dignified and healthy aging for the growing number of elderly people [2].
In order to reduce public welfare expenditures, it would be desirable to preserve
older people’s autonomy as long as possible, prolonging the stay in the own home and
postponing the need of constant assistance and hospitalization. Increasing the quality of
life of older people is a good strategy to not make them a burden for the health system
as well as for their families [3].
2 In-home Care for Older People
In-home senior care, in the absence of pathologies that do not allow it, is a good way to
make aging cheaper, safer and more comfortable for the person [1]. Most of the people
associate automatically the concept of in-home care to personal caregiving, thinking
that the only way to facilitate the aged person’s daily life in home is to have a qualified
professional by his side taking on some tasks.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 860–864, 2020.
https://doi.org/10.1007/978-3-030-39512-4_131
Design of Smart Devices for Older People: A User Centered Approach 861
Otherwise, it is important to recognize as fundamental part of in-home care even
home ergonomics and different kinds of non-medical devices that can have a great
impact on elderly’s quality of life. Some symptoms of several pathologies can either be
prevented or at least successfully accommodated, not to become debilitating in the
short period, thanks to new technologies like the Internet of Things combined with
Artificial Intelligence [4].
3 User Centered Design for the Elderly
Design Research has largely accepted the challenge of focusing on products and ser-
vices for older people: in the last few years the number of researches and projects
addressed to older people and their assistance and monitoring, is constantly increasing
[5]. There are many design methodologies and approaches that can be used in this
sector to combine properly the users’ requirements with technologies’ constrictions and
opportunities, one of them, that the authors already used in several projects, is User
Centered Design (UCD). UCD is recommended in particular to designers that are
facing projects that require a high degree of usability to be successful [6], as design
projects for older people. It allows the design of a product/service/process to be
addressed to niche users, characterized by peculiarities that affect performances related
to usability of products commonly used by most the population [7].
UCD focuses on shifting away from a reactive approach, in which environments,
furniture and devices that cause problems and troubles are modified after something
unpleasant happens, to a proactive approach that aims to design each element basing on
the targeted requirements of a specific category of users [8].
The author is carrying on a doctoral research on User Centered methodologies to
design smart devices which can help older people affected by Parkinson’s Disease in
reducing moving problems related to the gait and the walking: the final aim of this
work is to analyze current technologies, inclusive methodologies, best practices and
case studies, to create guidelines for the design of smart devices and services for the
improvement of the autonomy of people that are starting to face the older age, living in
their own home, frequently by themselves. The ongoing research is following the UCD
approach, considered particularly suitable in the field of design for the silver age
because it allows the categorization of users considering performances related to age
restrictions, but is also extendable to limitations proper of a specific disease [6].
Following this methodology, it was decided to integrate the design process with the
Quality Function Deployment, also known as QFD, a design tool that helps in the
concept definition of products and services starting from users or costumers’ needs and
technological features of the product to be designed [9]. QFD is a graphic tool that
collects in a matrix the list of the most important needs expressed by primary and
secondary users, and an index of the most significant measurable characteristics and
features of the product. Both needs and features are put in correlation through the
matrix chart, this process, thanks to the QFD algorithm, helps designers evaluating the
degree of relationship; starting from hierarchy resulting from expressed evaluations, it
will be easier to decide the performances due to items or services to be designed [9].
862 S. Imbesi and G. Mincolelli
4 Collecting Needs of the Elderly
The first stage of the UCD is the users’ analysis, which aims at the detection of
information about the selected categories of users, the designer will then process and
combine information taken from the analysis to establish the project guidelines. The
way information is gathered, the amount of data that is collected, the form in which the
results of this collection are presented, are a significant part of the design choices of the
project, affected by the designer’s own attitude [8]. The choice of number and typology
of the categories of users that will be analyzed, is as well part of the project.
In the previously mentioned doctoral research, the choice of users was considered
particularly impactful on the project results; in this paper, it will be related the part of
this research that concerns the choice of the typology of users that is going to be
analyzed to design smart devices for elderly people with physical symptoms related to
gait and movement, and how those users have been interviewed to extrapolate original
and unexpected needs.
The users’ analysis began with the study of dedicated literature, moving than to the
direct observation of patients with Parkinson’s Disease in public environments, and
after to the observation of the users in their daily context.
4.1 Definition of Primary Users, Secondary Users and Stakeholders
In the Design Research context, we use to define as primary users the ones who directly
use the product or service to achieve a goal; secondary users are people who use the
product or service occasionally or mediated by other persons and lastly stakeholders
that are people whose interests will be influenced by the product or people who have
direct influence on the requirements of the system to be designed [10].
Assuming that older people at the early stages of Parkinson’s Disease, still living in
their own home without a professional caregiver, are the primary users of this project, it
was noticed that the definition of secondary users and stakeholders wasn’t as obvious
as it might have seemed. The typology of product or service that is going to be
designed highly influences the definition of who are the people that will get in touch
with it and of what is their role in using the product. In this specific case, talking about
devices that helps the gait and monitor the movements, it became obvious that the
doctor who follows the patient has a most significant role than the relatives of the older
person. Giving another example, talking about stakeholders, the health care institution
that promotes the dissemination of devices that are not strictly medical has a greater
impact on the product’s system than those who directly sell it.
The final list of the users to be analyzed is the following:
– Primary user: older person affected by PD at the early stages of the disease, still
living in his own house.
– Secondary users: consort of the primary user, doctors who are directly following the
course of the disease of that specific person (geriatrician, neurologist, orthope-
dist…); occasional professional caregiver (carer for seniors, professional nurse…);
relatives that occasionally take care of the person (they are profoundly different
from the previous category due to the emotional implications); etc.
Design of Smart Devices for Older People: A User Centered Approach 863
– Stakeholders: health care institutions that promotes the dissemination of that kind of
devices; commercial agents taking care of the promotion and sale of the product;
health system workers who will have to cope with the degeneration of the pathology
that has not been stemmed in time; etc.
4.2 Analysis of Users’ Needs
The needs of the different users were collected in several ways: interviews, question-
naires, focus… but it was noticed that the best way to find hidden highlights and latent
needs is the individual talk that does not follow a strict schedule. At this phase of the
design process, after the definition of the categories of users, it can happen that the
structure of the interviews is elaborated under the influence of preconceived ideas, of
which the designer seeks confirmation when he faces the user. To remedy this potential
problem that leads to a banalization of the outputs of the users’ analysis, it is often
useful to set aside the programmed schedule of the interview, and just focus on the
inputs given by the person. We can imagine this action as a sort of brainstorming
between the designer and the user, a sort of prequel of a codesign process. It is
important to intend that brainstorming as a scientific one, not driven by randomness but
rather structured as a group work method in which the creative play of the association
of ideas is exploited: the aim is to bring out different alternative perspectives in order to
identify and solve a problem.
In this specific project the choice was to set some questions related to the specific
field of devices designed to help in solving troubles related to the gait of seniors
affected by Parkinson’s Disease. After each question, the interviewee was given only a
few minutes to answer the question asked, during this time the interviewer identified
two or three unexpected or curious elements on which it could be worthwhile to go into
specific, investigating the reasons of these statements. Continuing the dialogue in this
mode, the interview was much more interesting and rich of stimulating ideas than the
previous ones conducted basing on a format conceived in the same way for all users
without having yet known them.
5 Conclusions
At the current state of the doctoral research, it is possible to make two assessments
regarding the advantages of the used methodology related to the User Centered Design
approach.
Firstly, the early identification of a problem to be solved that was very specific led
to a more informed and motivated choice in defining the various categories of users and
their needs’ hierarchization.
Secondly, the interpretation of the interview to the user as a brainstorming in a
codesign process, rather than as a path guided by defined boundaries, has led to the
enunciation of original and very personal needs that would surely not have emerged
from the interviews conceived in a standard manner.
864 S. Imbesi and G. Mincolelli
The adopted methodology is realistically applicable in other fields of the Design
Research, in applied research projects that choose to use a Human Centered approach
to solve a specific problem of a user with uncommon peculiarities.
References
1. WHO World Health Organization: World Report on Aging and Health. WHO Press, Geneve
(2015)
2. Beard, J.R., Biggs, S., Bloom, D.E., Fried, L.P., Hogan, P., Kalache, A., et al.: Introduction.
In: Beard, J.R., Biggs, S., Bloom, D.E., Fried, L.P., Hogan, P., Kalache, A., et al. (eds.)
Global Population Ageing: Peril or Promise? World Economic Forum, Geneva (2012)
3. World Economic and Social Survey 2007: development in an ageing world. United Nations
Department of Social and Economic Affairs, New York (2007)
4. Kelly, S.D.T., Suryadevara, N.K., Mukhopadhyay, S.C.: Towards the implementation of IoT
for environmental condition monitoring in homes. IEEE Sens. J. 13(10), 3846–3853 (2013)
5. Coleman, R.: Improving the quality of life for older people by design. In: Graafmans, J.,
Taipale, V., Charness, N. (eds.) Gerontechnology: A Sustainable Investment in the Future.
IOS Press, Amsterdam (1998)
6. Mao, J., Vredenburg, K., Smith, P., Carey, T.: The state of user-centered design practice:
UCD is gaining industry acceptance but its current practice needs fine-tuning. Commun.
ACM 48(3), 105–109 (2005)
7. Mincolelli, G.: Customer/User centered design. Analisi di un caso applicativo, Maggioli,
Rimini (2008)
8. ISO 13407: Human-centred design processes for interactive systems, ISO (1999)
9. Akao, Y.: Quality Function Deployment (QFD) Integrating Customer Requirements into
Product Design. Productivity Press, Portland (1990)
10. Preece, J., Rogers, Y., Sharp, H.: Interaction Design. Apogeo Editore, Rovigo (2004)
Examining Feedback of Apple Watch Users
in Korea Using Textmining Analysis
Yu Lim Lee, Minji Jung, In-Hyoung Park, Ahyoung Kim,
and Jae-Eun Chung(&)
Department of Consumer and Family Sciences, Sungkyunkwan University,
25-2 Sungkyunkwan-ro, Jongno-gu, Seoul, South Korea
{ylee168,minji0328,in.park,ay520,jchung}@skku.edu
Abstract. The purpose of this study was to examine the feedback of Apple
Watch users in Korea. Data were crawled from social media and were analyzed
using community detection and semantic network analysis with R programming.
Community detection identified device and after-sales service, application, and
activity tracking as major issues. Semantic network analysis revealed twelve
categories of positive feedback and nine categories of negative feedback. This
study provides theoretical and managerial implications to enhance consumer
satisfaction.
Keywords: Smart watch Consumer reviews Community detection
Semantic network analysis
1 Introduction
With embedded information technology (IT), wearable devices could potentially offer
proactive support to consumers in their daily lives, for example, helping users manage
their health by collecting bio-information, and improving their work efficiency by
providing schedule notifications. However, the growth rate of the wearable device
market in South Korea has been declining since 2017 [1], which implies that South
Korean consumers have not perceived sufficient utilities from wearable devices and
face challenges regarding use of these devices. However, current consumer studies
have mainly focused on discovering expected utilities and challenges at the pre-
purchase stage, where the consumers’ decision to adopt the product is made [2, 3].
Therefore, it is necessary to explore Korean consumers’ feedback derived from the
actual product experiences to maximize their satisfaction. As smartwatches dominate
the wearable device market, this study selected the Apple Watch, a global leading
smartwatch brand, as a target product.
2 Research Methods
2.1 Research Process
We employed the following data analysis process in this study.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 865–870, 2020.
https://doi.org/10.1007/978-3-030-39512-4_132
866 Y. L. Lee et al.
First Step. We crawled social media sites including Twitter, Facebook, and online
community sites using Trend-Up, a bigdata company in Korea, targeting documents
posted between February 1, 2018 and January 31, 2019, and obtained 3,138 documents
written by Apple Watch users in Korea.
Second Step. Term extraction through part-of-speech tagging and stemming was
conducted using a Korean morphological analyzer and KoNLP package with R pro-
gramming. Then, stop-words (which were irrelevant to product experience) were
eliminated. This process identified 3,756 terms.
Third Step. We selected 52 terms that appeared more than 20 times. These consisted
of nouns, verbs, and adjectives. Nouns and verbs were considered as evaluative factors
for Apple Watch, indicating mainly its components or functions and consumer
behavior relevant to its use. To classify the factors into several dimensions, which
allowed efficient identification of major issues regarding the Apple Watch, we selected
the factors and conducted community detection using the igraph package for R pro-
gramming. Community detection identifies communities where nodes are organized
into groups in a network and offers insight on how the network is organized [4].
Fourth Step. Semantic network analysis (using igraph package for R programming)
was conducted to examine associations between the 52 terms. Also, two indices of
centrality, degree centrality and betweenness centrality, were reviewed to interpret the
results of the analysis. Degree centrality was used to indicate the number of direct
connections to other nodes. Betweenness centrality quantified “the fraction of all
shortest paths from the nodes to each other that pass through a certain node” [5].
3 Results
3.1 Community Detection
Figure 1 presents the results of community detection. Factors for which consumers
gave positive or negative feedback were classified into three dimensions (i.e., device
and after-sales service, application, and activity tracking), which are named in
accordance with factors presented in Table 1. The first dimension is device and after-
sales service which indicates components of a device, such as “battery,” “material,”
“strap,” and after-sales service, such as “repair,” and “scratch.” The second dimension
is application which covers various productivity applications and functions that support
consumers to use applications such as “Bluetooth” and “cellular data communication.”
The third dimension is activity tracking which is related to Apple Watch’s activity
applications and user’s exercise.
3.2 Semantic Network Analysis
Figure 2 depicts the results of semantic network analysis, where the node’s size
indicates the degree centrality of each term, and the color denotes betweenness cen-
trality. The darker the color, the higher the betweenness centrality. Circle nodes display
the factors for the Apple Watch, and square nodes represent positive or negative
Examining Feedback of Apple Watch Users in Korea Using Textmining Analysis 867
Fig. 1. Results of community detection
Table 1. Dimensions and factors included from community detection
Dimension Factors
Device and after- battery, charge, repair, after-sale service, scratch, Hermes edition,
sales service material, switcha, watch face, strap, change strap, loyalty, design,
changeb, wait
Application SNS (social network services) apps, alarm, music streaming,
interlock, Bluetooth, cellular data communicationc, update, weather
information
Activity tracking exercise, activity goal, physical move, Nike edition, heartrate, swim,
waterproof
a
“Switch” indicates behavior of a consumer who had used an old Apple Watch model or
smartwatch of another brand, and then switched to a new Apple Watch model.
b
“Change” indicates behavior of a consumer who changes a configuration of Apple Watch.
c
“Cellular data communication” is a communication network that provides radio coverage over a
wide geographic area, enabling smart devices to communicate with each other without Wi-fi.
adjectives. A link between two nodes refers to its “edge;” the darker the edge, the more
frequently the documents containing the two nodes appeared concurrently.
We extracted terms with higher levels of degree centrality than the mean, which
resulted in 23 important terms being selected. Then, we examined where there were
links between these and other terms, deployed the links according to dimensions of
factors, and summarized the contents as categories to indicate consumers’ feedback.
We identified twelve categories of positive feedback. There were six categories in
the device and after-sales service dimension. As many consumers wear their Apple
Watch 24 h a day, both “comfortable fit” and “high durability of battery” were
868 Y. L. Lee et al.
Fig. 2. Results of semantic network analysis
considered important positive feedback. Consumers also perceived “excellence of after-
sales service,” (related to after-sales service) and “aesthetic pleasure” (derived from
visual attractiveness). The ability to personalize their watches by changing straps and
watch faces according to their own needs (“personalization”), and “social attractive-
ness” enabling consumers to enhance their social image by impressing others were
found. In the application dimension, three categories were identified. These included
“ubiquitous,” meaning consumers can access desired information at any time, and
“multitasking”, which indicates that consumers can use applications while they exer-
cise. In addition, “high connectivity with smartphone” enables consumers to use
applications easily. Three categories of positive feedback were identified in the activity
tracking dimension. “Motivating physical activity” relates to activity tracking and
alarms, which motivates consumers to perform physical movement and maintain their
health. “Enjoyment” indicates affective pleasure from achieving activity goals, which
can be shared with others. “Monitoring heart rate” was also perceived positively.
Conversely, nine categories of negative feedback were identified. Among six cat-
egories in the device and after-sales service dimension, “financial costs” indicates that
consumers pay a high price for their watch. “Uncomfortable fit” suggests that con-
sumers were inconvenienced by their device or straps made from specific materials. In
addition, “fragility” suggests a risk of being scratched. “Battery and charging issue”
(the inconvenience of repairing and charging the battery) was a notable feedback.
Moreover, consumers were irritated by frequent charging of the battery (“irritation due
to charging”). Unlike some consumers who felt pleasure with the aesthetic appearance
of the watch, others were displeased. The application dimension contained three
negative categories. First, “malfunction of apps” indicates a negative evaluation where
some applications had not worked well. Furthermore, “unavailability of apps” suggests
that consumers felt their use of applications was limited, due to insufficient service.
Some consumers expressed negative emotions because they were connected to others
Examining Feedback of Apple Watch Users in Korea Using Textmining Analysis 869
by SNS applications even when they did not want to (“bother due to keeping in touch
with others”).
Next, the betweenness centrality of each node was reviewed to identify the
importance of the role of terms in the semantic network. “Inconvenience”, which was
linked to various terms in the applications dimension (such as “music streaming,”
“SNS apps,” and “alarm,”) had the highest level of betweenness centrality. These
applications were also connected to “convenience,” indicating the importance of
whether consumers feel convenience or inconvenience from these applications.
“Switch” showed the second highest level of betweenness centrality, which was linked
to “material,” “interlock,” “favorite,” “pretty,” “battery,” and “after-sales service”.
Consumers “switched” to a new Apple Watch model as they were satisfied with various
functions or appearance of a previous model, but some purchased a new product
because of “fragility” issues. “Irritation” also showed a high level of betweenness
centrality by its frequent appearance between “cellular data communication” and
“expensive.” Because Apple Watch models that provide “cellular data communication”
service have a high price compared with other watches, a high betweenness centrality
of “irritation” indicates that many consumers were irritated when they purchased an
Apple Watch providing this service.
4 Conclusion
This study examined the positive and negative feedback of Korean consumers’
regarding the Apple Watch, using community detection and semantic network analysis.
Results of community detections showed that there were three major issues
regarding Apple Watch feedback: device and after-sales service, application, and
activity tracking. The results confirm that activity tracking is an important function of
the smartwatch, as suggested by previous studies, supporting consumers in maintaining
healthy lives [6]. Concretely, consumers were motivated to move and exercise by
alarms and activity rings of the activity tracking application, which enabled them to feel
achievement and pleasure.
We identified some ambivalent feedback for several factors in the device and after-
sales service dimension, suggesting the presence of both positive and negative feed-
back for sense of fit, battery capacity, and appearance. As smartwatches are in direct
contact with users’ wrists for most of the day (even while they are sleeping), ergonomic
design has been suggested by previous research [2] as an important product attribute.
Moreover, many consumers wear smartwatches during exercise, which leads to their
fragility. Because this study revealed that consumers switch to a new Apple Watch
model due to fragility, smartwatch manufacturers should improve both product design
and materials. Further, the results here suggested that improvements to the smartwatch
battery was required. As some consumers were bothered by frequent battery charge,
improvement of battery capacity might reduce their negative responses. Positive and
negative feedback on appearance suggested that consumers perceive smartwatches as a
fashion product. Furthermore, the appearance of the Apple Watch was not only
associated with aesthetic pleasure, but also with social attractiveness, confirming pre-
vious research that revealed consumers wore smartwatches to enhance their own social
870 Y. L. Lee et al.
image [3]. This shows that consumers can obtain social value from smartwatches.
Thus, smartwatch marketers could enhance the social image of their products for
promotion. As a smart device with high technology, smartwatches have an implicit risk
of malfunction. Therefore, smartwatch manufacturers are required to provide high
quality of after-sales service.
Feedback on application was mainly about performance and functions of various
applications. Consumers could access online information via their Apple Watch at any
time. Furthermore, high connectivity with smartphone enabled consumers to use
applications without a smartphone, thus allowing standalone communication. How-
ever, malfunctions of apps inconvenienced consumers when using the applications.
A high level of betweenness centrality for inconvenience showed that whether con-
sumers felt inconvenience or convenience using applications were the major criteria in
evaluating their watch. Therefore, it would be crucial for smartwatch application
developers to overcome malfunctions. In addition, consumers could keep in touch with
others thanks to its ubiquitous character; however, they were bothered from inevitable
contacts from others at the same time. This calls for an examination of the changing
consumption environment due to innovative products being available to consumers.
While most previous research on smartwatches and wearable devices focused on
consumers’ expected utilities and challenges before product adoption, this study pro-
vides theoretical contributions by examining the feedback from their real-life smart-
watch consumption. We were able to examine consumer feedback using textmining
analysis. In addition, the current study suggests important managerial implications for
smartwatch manufacturers who target South Korean consumers, where the growth rate
of wearable devices has been gradually decreasing. The results of the current study
indicate to manufacturers which product attributes should be strengthened and
improved to increase consumer satisfaction in product usage.
References
1. ETNews. http://www.etnews.com/20170403000140
2. Gao, Y., Li, H., Luo, Y.: An empirical study of wearable technology acceptance in healthcare.
Ind. Manag. Data Syst. 115, 1704–1723 (2015)
3. Jeong, S.C., Kim, S., Park, J.Y.: Domain-specific innovativeness and new product adoption: a
case of wearable devices. Telematics Inform. 34, 399–412 (2017)
4. Fortunato, S., Hric, D.: Community detection in networks: a user guide. Phys. Rep. 659, 1–44
(2016)
5. Mahyar, H., Hashemizezhad, E., Ghalebi, E., Nazemian, A., Grosu, R., Movaghar, A.,
Rabiee, H.R.: Compressive sensing of high betweenness centrality nodes in networks. Phys.
A 497, 166–184 (2018)
6. Dehghani, M., Kim, K.J., Dangelico, R.M.: Will smartwatches last? Factors contributing to
intention to keep using smart wearable technology. Telematics Inform. 35, 480–490 (2018)
Structural Testing of Laminated Prosthetic
Sockets: Comparison of Philippine Pineapple
Fabric and Fiberglass
Glenn Alkuino(&), Ervin Fandialan, and Marvin Medina
College of Allied Rehabilitation Sciences,
UERMMMCi, 1113 Quezon City, Philippines
[email protected],
[email protected],
[email protected]
Abstract. Background: Prosthetics and orthotics (P&O) uses a wide variety of
materials such as plastics, resins and fibers. P&O greatly benefitted from con-
ventional materials such as carbon fibers and fiberglass, although these materials
exhibits high mechanical properties, these materials are expensive. Objective: to
compare the structural and mechanical properties of pineapple fiber as a rein-
forcement for composite material in the Prosthetics and Orthotics Fiel. Methods:
using ASTM D638 in preparation for the tensile strength testing and ISO 10328
for testing the compressive strength Results: The 5 bar samples produced a
result of 37.21 MPA with a standard deviation of 3.21 MPa The results for
compression testing showed that the pineapple fabric accumulated a loading
capacity of 2387N which passed the P3 ISO standard loading criteria (2092N).
Conclusion: The pineapple prosthetic socket passed the minimal loading level of
60 kg, thus have the potential to replace conventional materials as reinforcement
Keywords: Prosthetics Lamination Green sockets Natural fibers
Synthetic fibers Static testing Composites
1 Introduction
Prosthetics and orthotics (P&O) uses a wide variety of materials such as plastics, resins
and fibers. The field has benefitted from advanced technologies of marine and aero-
space industries producing durable, lightweight and stronger materials such as fiber-
glass and carbon fiber. Although exhibiting high mechanical properties, these materials
are expensive, ecologically unsustainable, and unavailable in low to middle income
countries. Current innovation in prosthetic socket manufacturing aims to replace the
use of synthetic fibers and resin with natural and bio-based materials. Recent studies
investigated the use of natural fibers such as abaca, pineapple and bamboo as a
replacement for conventional socket material. Thus this research aimed to test the
structural characteristics of the Philippine pineapple using standard performance tests
for prosthetic sockets.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 871–874, 2020.
https://doi.org/10.1007/978-3-030-39512-4_133
872 G. Alkuino et al.
2 Methods
Testing methods from American Standard of Testing and Materials (ASTM), and
International Organization for Standardization (ISO) were utilized in the study. ASTM
D638 was used as the guideline in fabricating and testing of samples for tensile strength
(MPa). ISO 22523 was adapted in fabricating a standardize sample dimensions and
using ISO 10328 for testing compressive strength (N) of the material. Five bar samples
of pineapple fabric were prepared for tensile strength testing and only one socket
sample for each fiber was done for compressive testing. For tensile strength setup, the
samples were clamped onto an Instron 5585H tensile test machine to determine the data
of the test pieces, while the socket samples were mounted on Shimadzu Universal
Testing Machine to test for the compressive strength of samples.
3 Results and Discussion
The mean of the 5 bar samples for tensile strength were taken and accumulated the
result of 37.21 with a standard deviation of 3.21 MPa classifying it as a low range
strength material compared to other fibers. The results for compression testing showed
that the pineapple fabric accumulated a loading capacity of 2387N which passed the
P3 ISO standard loading criteria (2092N). The fiberglass socket sample also passed the
P3 ISO standard loading criteria with a higher loading capacity of 2559N.
While the results are lower compared to other studies, it is important to consider the
factors that affected the results. Distal end thickness, distal end reinforcement, fiber to
resin ratio, weave orientation, chemical treatment of fibers, combination of fibers all
play a role in the strength of the socket. Further studies must focus on other mechanical
testing, combination of different fibers, and clinical trials (Fig. 1).
Fig. 1. (A) Diagram of forces that would be applied onto the jig, (B) diagram of the angle of
forces for terminal stance, (C) mounted pineapple socket with jig on compressive testing machine
Structural Testing of Laminated Prosthetic Sockets 873
4 Conclusion
The pineapple fabric laminated socket passed the 60-kg loading level, and has a
possible application in the P&O feld. It has the potential to replace conventional
reinforcements and the low results on tensile strength reflected the results on previous
natural fibers which have similar results. Continuation of the study on “green sockets”
can help expand choices in prosthetic socket prescription and add as a solution to the
problems in using synthetic materials. Further studies must focus on other mechanical
testing, clinical trials and different fibers.
Acknowledgments. The authors would like to recognize the following who shared their time,
effort and facilities for the completion of this study: Our thesis advisor Prof. Nikolai R. Santos,
for supervision and guidance from conceptualization to the completion of the study; Department
of Science and Technology, Industrial Technology Developmental Institute for preparation of
samples, providing facilities and testing equipment; Prof. Anna Margarita A. Lugue, thesis
coordinator, for assistance and recommendations for the study; The UERMMMCI Research
Institute of Health Sciences – Ethics Committee for reviewing and approving the study; Exceed,
for acquiring necessary ISO documents needed in standardizing the quality of this study.
References
1. Berry, D.: Composite materials for orthotics and prosthetics. Am. Orthot. Prosthet. Assoc. 40
(4), 35–43 (1987)
2. BS EN ISO 10328:2006 Prosthetics. Structural testing of lower-limb prostheses. Require-
ments and test methods. British Standards Institution (2016)
3. BS EN ISO 22523:2006 External limb prostheses and external orthoses. Requirements and
test methods. British Standards Institution (2006)
4. Campbell, A.I., Sexton, S., Schaschke, C.J., Kinsman, H., McLaughlin, B., Boyle, M.:
Prosthetic limb sockets from plant-based composite materials. Prosthet. Orthot. Int. 36(2),
181–189 (2012). https://doi.org/10.1177/0309364611434568
5. Chabot, T.: Are plant based fibres an acceptable alternative to fibreglass in the fabrication of
prosthetic sockets? Clinical Methods in Prosthetics and Orthotics George Brown College
(2016). http://www.opcanada.ca/_uploads/media/572phls71.pdf
874 G. Alkuino et al.
Appendix
Challenges and Improvements in Website
Accessibility for Health Services
Patricia Acosta-Vargas1(&), Paula Hidalgo1, Gloria Acosta-Vargas2,
Mario Gonzalez1, Javier Guaña-Moya3, and Belén Salvador-Acosta1
1
Intelligent and Interactive Systems Lab,
Universidad de Las Américas, Quito, Ecuador
{patricia.acosta,paula.hidalgo,
mario.gonzalez.rodriguez,
maria.salvador.acosta}@udla.edu.ec
2
Pontificia Universidad Católica del Ecuador, Quito, Ecuador
[email protected]
3
Universidad Central del Ecuador, Quito, Ecuador
[email protected]
Abstract. Today, the growing number of applications and web users make
software engineering a promising field, especially the development of accessible
web applications. According to the World Health Organization, 15% of the
world’s population carries some form of disability. There are many websites for
patient care, but not all of them are accessible. Therefore, it is necessary to
develop accessible websites for health services that comply with the Web
Content Accessibility Guidelines 2.1 that provide guidelines for the develop-
ment of more accessible and inclusive applications. Testing accessibility in web
applications is a major challenge for accessibility experts, so this research
proposes an assessment method to measure the accessibility of health-related
web applications. The method can serve as a reference to apply to any website.
Keywords: Accessibility Evaluation Health Method Services WCAG
2.1 Website
1 Introduction
According to the reports “We Are Social,” and “Hootsuite” on the use of the Internet [1]
argue there are about 4.39 million Internet users in 2019, an growth of 366 million
compared to 2018. On the other hand, the World Health Organization [2], mentions in its
disability reports that more than one billion people, that is, 15% of the world’s popu-
lation, suffer from some form of disability. Therefore, people with disabilities have less
access to health care, education, and other services. It is a reality that technologies have
changed the way of life of all people and areas of knowledge, including Health.
With the accelerated development of technology, new generations are considering a
professional future in which they can work, breaking traditional schemes that relate
people with disabilities with inactivity and dependency. The disruption of information
and communication technologies allows people with disabilities to perform jobs in
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 875–881, 2020.
https://doi.org/10.1007/978-3-030-39512-4_134
876 P. Acosta-Vargas et al.
which they were practically excluded years ago. This study proposes a method that
considers the Web Content Accessibility Guidelines (WCAG) 2.1 [3] and the Website
Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0 [4] that can be
applied to the development of health-related websites in such a way that more people
including people with disabilities and the elderly can access health-related websites.
This document proposes challenges that are related to actors and technology, where
technology is explicitly not the problem. That is, the processes, the legislation of each
country, and the technology must be aligned. So that by identifying the barriers and
limitations of web accessibility, we alert web programmers to find opportunities for
improvement. This study serves as a reference for the creation and design of websites
in order to reduce accessibility barriers.
This research is structured as follows: Sect. 2 presents the background and related
work, Sect. 3 explains the methodology and case study, Sect. 4 presents the discussion
and results, and finally Sect. 5 details the conclusions and future work of this research.
2 Background and Related Work
About health websites, the authors share the following literature; [5] argue that studies
conducted in Catalan hospitals show a descriptive analysis to evaluate the character-
istics of private hospital websites. The results indicate that studies of content up-dates,
accessibility, interactivity, usability, and privacy policies are essential. Another study
[6] argues that factors influencing the visibility of services can be improved through
adequate web accessibility with quality and accreditations.
Kaur et al. [7] propose that the evaluation of the accessibility of hospital websites
presents accessibility problems because the WCAG 2.0 guidelines are not applied.
Bouzas-Lorenzo et al. [8] analyze some features about web browsing, the ease of
locating portals, accessibility to electronic services. The results revealed that the e-
health portal in Spain is not sufficiently accessible.
Acosta-Vargas et al. [9] argue that making healthcare-associated services available
on the web requires users accessing these services to be accessible to people with
disabilities. The results revealed that health websites present barriers to users; the study
suggests strengthening web accessibility legislation in each country and applying best
practices with WCAG 2.0.
WCAG 2.1 [3] developed by the World Wide Web Consortium (W3C) proposes
guidelines that consider web accessibility. These guidelines will meet the needs of
individuals, organizations, and government policies at the international level.
In June 2018, the W3C proposed the WCAG 2.1 [3] contains four principles, 13
guidelines and 78 success criteria, as well as some sufficient techniques and advice. The
principles are [3] Principle 1: Perceptible means that users must be able to perceive
content in a sonorous, tactile, and visual way. It includes four guidelines and 29
compliance criteria. Principle 2: Operable, this principle refers to the fact that users
must navigate and use the interface elements. It comprises five guidelines and 29
compliance criteria. Principle 3: Understandable, this principle relates to the content
and controls of the user interface, which must be understandable to all users. It com-
prises three guidelines and 17 compliance criteria. Principle 4: Robust, indicates that
Challenges and Improvements in Website Accessibility for Health Services 877
the content is robust enough to be reliably interpreted by a large number of users,
allowing them to work with current and future technologies. It includes a guideline and
three compliance criteria. Likewise, WCAG 2.1 suggests that the success criteria are
associated with Level “A” which is a minimum level of accessibility; Level “AA”
which is an intermediate level, and Level “AAA” which is the maximum level [3].
Acosta-Vargas et al. [10–13] in previous studies related to the evaluation of web
accessibility when applying WCAG 2.0 and WCAG 2.1 argue that the majority of
websites evaluated do not comply with accessibility parameters.
3 Methodology and Case Study
Figure 1 below presents an outline that includes seven phases, in which we summarize
the methodology for evaluating accessibility in health websites.
Fig. 1. Methodology to evaluate accessibility in health websites
Phase 1: Define the sample; in this phase, we selected 20 websites that occupy the
first ranking in health topics; they were taken from Webometrics1. Phase 2: Explore the
websites to be evaluated; in this phase, the experts reviewed the websites manually,
explored the links, images, and content of the main pages of each site to be evaluated.
Phase 3: Evaluate the main pages of each website with automatic tools; to evaluate web
accessibility, we use two automatic tools. (i) WAVE2 developed by WebAIM; we use
the application with the plugin for Google Chrome, version 1.0.9, updated to November
17, 2017. (ii) TAW3 is available online validates the accessibility incorporates security
1
https://hospitals.webometrics.info/en/World.
2
https://wave.webaim.org/.
3
https://www.tawdis.net/.
878 P. Acosta-Vargas et al.
enhancements under HTTPS. Remember that automatic verification is not enough; no
tool identifies all accessibility problems; the expert must manually check the possible
problems of the website. Phase 4: Recording the results of the evaluation, in this phase,
we proceed to record in a spreadsheet the data obtained from the evaluation with the
tools WAVE and TAW. Phase 5: Manually review errors found on evaluated websites;
in this phase, the experts manually reviewed the evaluated websites, explored the links,
images, and content of the problems detected with the automatic tools. Phase 6:
Analysis of the results, in this phase, the experts performed data cleansing, calculations,
and statistical analysis with the data, to identify the problems of each website relating
them to the principles of accessibility. Phase 7: Suggest improvements to improve
accessibility in web applications; in this last phase the experts identified the problems
presented and propose suggestions based on WCAG 2.1 for web developers and
designers to apply and review throughout the development cycle of websites. For this
investigation, 20 main pages of the health websites obtained from the Webometrics4
ranking were taken as a case study. The data was collected on September 10, 2019.
Table 1 contains the acronym, the average of perceivable, operable, understandable,
robust and average errors of TAW/WAVE.
Table 1. Health websites evaluated
Acronym Perceivable Operable Understandable Robust Errors
my 1.0 0.0 0.0 1.5 2.5
umm 1.0 0.5 0.0 5.5 7.0
tzuchi 2.0 4.5 0.0 3.0 9.5
brighamandwomens 3.5 3.0 1.0 3.0 10.5
nyp 1.5 3.0 0.5 8.0 13.0
hopkinsmedicine 3.0 7.5 0.0 4.0 14.5
providence 2.5 1.0 0.5 10.5 14.5
mayoclinic 7.0 0.0 0.0 10.5 17.5
massgeneral 10.0 2.0 1.0 6.0 19.0
dhmh 9.0 4.5 2.0 5.0 20.5
upmc 6.5 5.5 1.5 8.5 22.0
mskcc 14.0 1.0 6.5 9.0 30.5
aphp 11.5 16.0 0.5 4.0 32.0
mdanderson 8.5 5.0 2.5 16.5 32.5
bidmc 25.0 4.0 4.5 5.5 39.0
advocatehealth 18.5 5.0 5.5 11.5 40.5
vghtpe 26.5 1.0 4.5 11.0 43.0
aurorahealthcare 17.5 24.0 5.0 12.0 58.5
stjude 20.5 11.0 20.0 9.5 61.0
klinikum 25.0 29.5 6.0 11.5 72.0
4
https://hospitals.webometrics.info/en/World.
Challenges and Improvements in Website Accessibility for Health Services 879
4 Results and Discussion
Figure 2 shows that the websites with the lowest number of problems correspond to
Cleveland Clinic, University of Maryland Medical Center of United States and Bud-
dhist Tzu Chi General Hospital of Taiwan. In the figure on the left, there are more
significant problems related to the “Perceivable” principle with 38%, followed by
“Robust” with 28%, “Operable” with 23% and in a lower percentage “Understandable”
with 11%. The figure on the right shows that the most frequent problems are related to
the success criterion “4.1.2” corresponding to “Name, Role, Value,” with an average of
182 errors representing 27%. Followed by the success criterion “1.1.1” related to “Non-
text Content” with an average of 124.5 errors representing 19%. Thirdly the problem is
related to the “1.3.1” related to “Info and Relationships” with an average of 96.5 errors
representing 14% of an average of 672 errors. For further details of the data analysis
performed in this investigation, the dataset is available in the Mendeley repository5.
When reviewing the problems manually, from the analyses carried out, it was
identified that the errors that are frequently repeated are related to images with infor-
mation without a correct textual alternative (alt). A second error relates to decorative
images not marked as such, which must have an empty text alternative (alt = “ ”).
A third error relates to the In the case of forms; the <label> labels are not associated
with the corresponding <input> element.
Fig. 2. Web accessibility in health websites
5 Conclusions and Future Works
From the research carried out, the researchers conclude that even though the websites
are in a high classification as those evaluated in this case, they do not fully comply with
WCAG 2.1. It is essential to raise awareness among web designers and programmers
5
http://dx.doi.org/10.17632/t62z4sh5kj.1.
880 P. Acosta-Vargas et al.
who apply contrasts between foreground and background elements. WCAG 2.1 sug-
gests a minimum contrast ratio of 4.5:1 and an ideal ratio of 7:1. Besides, the authors
suggest including context links, especially for users who use screen readers.
In future work, we suggest combining the automatic method with heuristic methods
to achieve a complete evaluation of accessibility. Suggest that website builders have a
web accessibility expert profile. We suggest to include a specialist who does tests with
users, who tests the web with different technologies used by people with disabilities.
Finally, we consider that it is essential to strengthening the legislation in each country,
remember that most of the methods will not only help people with disabilities, it will
also benefit all users of the website. It can even improve search engine rankings.
Acknowledgments. The researchers thank Universidad de Las Américas - Ecuador for funding
this study through the project “PSI.PHA.19.03” and “FGE.PAV.18.10.”
References
1. We Are Social and Hootsuite: Digital 2019: Global Internet Use Accelerates - We Are
Social. https://wearesocial.com/blog/2019/01/digital-2019-global-internet-use-accelerates
2. World Health Organization (WHO): World report on disability. https://www.who.int/
disabilities/world_report/2011/report/en/
3. World Wide Web Consortium: Web Content Accessibility Guidelines (WCAG) 2.1. https://
www.w3.org/TR/WCAG21/
4. World Wide Web Consortium (W3C): Website Accessibility Conformance Evaluation
Methodology (WCAG-EM) 1.0. https://www.w3.org/TR/WCAG-EM/
5. Creixans-Tenas, J., Arimany-Serrat, N.: Web quality of private hospitals. In: 2016 11th
Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6 (2016)
6. Creixans-Tenas, J., Arimany-Serrat, N.: Influential factors in the visibility and quality of the
websites of private hospitals. In: Conference on Information Systems and Technologies,
pp. 1–7 (2017)
7. Kaur, A., Dani, D., Agrawal, G.: Evaluating the accessibility, usability, and security of
hospitals websites: an exploratory study. In: 2017 7th International Conference on Cloud
Computing, Data Science & Engineering – Confluence, pp. 674–680 (2017)
8. Bouzas-Lorenzo, R., Mahou-Lago, X., Chao, L., Cernadas, A.: e-Health and user experience
in Spanish public health portals. In: Proceedings of the 17th European Conference on Digital
Government, pp. 47–57 (2017)
9. Acosta-Vargas, P., Acosta, T., Luján-Mora, S.: Framework for accessibility evaluation of
hospital websites. In: International Conference on eDemocracy eGovernment, pp. 9–15
(2018). https://doi.org/10.1109/icedeg.2018.8372368
10. Acosta-Vargas, P., Luján-Mora, S., Acosta, T., Salvador-Ullauri, L.: Toward a combined
method for evaluation of web accessibility. In: Advances in Intelligent Systems and
Computing, pp. 602–613 (2018)
Challenges and Improvements in Website Accessibility for Health Services 881
11. Acosta-Vargas, P., Acosta, T., Lujan-Mora, S.: Challenges to assess accessibility in higher
education websites: a comparative study of Latin America universities. IEEE Access 6,
36500–36508 (2018). https://doi.org/10.1109/ACCESS.2018.2848978
12. Acosta-Vargas, P., Salvador-Ullauri, L., Luján-Mora, S.: A heuristic method to evaluate web
accessibility for users with low vision. IEEE Access 7, 125634–125648 (2019). https://doi.
org/10.1109/ACCESS.2019.2939068
13. Acosta-Vargas, P., Salvador-Ullauri, L., Pérez-Medina, J.L., Gonzalez, M., Jimenes, K.,
Rybarczyk, Y.: Improving web accessibility: evaluation and analysis of a telerehabilitation
platform for hip arthroplasty patients. Advances in Human Factors and Systems Interaction,
vol. 959, pp. 508–519 (2020). https://doi.org/10.1007/978-3-030-20040-4_46
Providing Comprehensive Navigational Cues
Through the Driving Seat to Reduce Visual
Distraction in Current Generation
of Semi-autonomous Vehicles
Ahmed Farooq(&), Grigori Evreinov, and Roope Raisamo
Tampere Unit for Computer Human Interaction (TAUCHI), Faculty of
Information Technology and Communication Sciences, Tampere University,
Tampere, Finland
{Ahmed.Farooq,Grigori.Evreinov,Roope.Raisamo}@TUNI.fi
Abstract. Methods of interaction and information presentation in the auto-
motive context have continuously been evolving in the last decade Essentially,
there is an inherent need to design and research in-vehicle systems that com-
plement and natively support a multimodal approach towards driver-vehicle
interaction. Basically, most IVI tasks are classified as secondary or tertiary
driving tasks, meaning the driver can allocate lower priority to these tasks as
they are not time specific. This is because most visually complex secondary
tasks may hinder the driver’s ability to focus on the primary task of driving.
However, navigation tasks should not be classified as other IVI tasks. These
tasks are often time or location specific and interaction with the system in most
cases, may not be optional. Therefore, drivers often get visually burdened with
excessive information at the most challenging instances. In this research we
propose a novel approach towards resolving this issue by developing a Haptic
Seat that can provide continuous and timely navigational information without
additional visual distraction.
Keywords: Human-systems integration Multimodal interaction Haptics
In-vehicle infotainment Driver distraction
1 Introduction
Most information presented to the driver in the current generation of vehicles is through
various displays within the cabin. However, certain tasks, such as navigation can be
very complex and may require considerable visual attention, which may hinder the
driver’s ability to focus on the primary task of driving. Therefore, most car manu-
facturers have a dedicated navigation system installed in the IVI central stack. How-
ever, this can prove to be challenging as the navigation systems still needs to be
operated using the same controls/input mechanisms as the other IVI components.
Essentially, most IVI tasks are classified as secondary or tertiary driving tasks, meaning
the driver may be engaged in them as they so choose. Therefore, if the driving envi-
ronment is less visually or cognitively challenging, the driver may focus their attention
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 882–888, 2020.
https://doi.org/10.1007/978-3-030-39512-4_135
Providing Comprehensive Navigational Cues Through the Driving Seat 883
towards interaction with the IVI system (CID, HUD, IC etc.). However, there is no
compulsion (NHTSA guidelines [1]) or time constraint of when and how this inter-
action takes place. Considerable research [2, 3] has been done to prioritize interaction
related to navigation-based tasks. One of the core ways to ensure this is to make
interaction regarding these tasks as natural as possible. Because audio and visual
feedback work well together [4], the combination of the two has been widely used in
navigation tasks for most IVI systems. Although adding audio cues or voice-based
input could reduce visual distraction, the cognitive impact can still be a concern [5].
Adding haptics could resolve some of these issues and provide a more natural and
personalized interaction experience for the driver. Although this concept is not new,
most of the current research focuses on information redundancy through signal spec-
ification [6, 7]. Moreover, some of these studies show that it may be possible to deliver
important navigational information to the driver. However, the higher cognitive load
required to achieve this may not be worth the risk. Most of the implemented systems
and their feedback signals are non-intuitive and take longer to identify and recognize.
These systems also have a higher threshold of learnability with limited advantages seen
in short-term use. Therefore, in the present research the authors focused on detecting
and understanding the core requirements of the in-car navigational task and developed
a simple multimodal interaction mechanism (Haptic Seat) to overcome issues without
increasing the driver’s cognitive load or complexity of interaction.
2 Developing a Novel Haptic Seat (HS) Prototype
The Haptic Seat was designed to provide three distinct types of actuation cues, which
included simple event-based cues, spatial navigation (directional) cues and alerts or
warnings that required immediate action. To generate simple event-based cues, the
system utilized two sets of Tectonic TEAX25C10-8/HS voice coil actuators embedded
on either side of the driver’s seat, near the placement of the thighs (Fig. 1). The two
actuators on either side were attached to a plastic plate and placed horizontally within
the cushion of the driver’s seat. This ensured that the actuators efficiency even when a
heavy load was applied to the plastic plate, and surface area of the plastic plate radiated
the vibrational cues, increasing the spatial resolution of the haptic feedback. The setup,
simulated low frequency clicks under the thighs of the driver, signifying event-based
feedback (touchscreen click, selections etc.).
Fig. 1. The Haptic Seat uses a pair of rotatory motors (left) and TEAX25C10-8/HS actuators
(right) to generate feedback.
884 A. Farooq et al.
To generate spatial navigation cues, two Fukoku USR60-E3T ultrasonic motors
with asymmetrical arrangement of spherical touch points on top, were embedded within
the driver seat near the Tectonic voice coil actuators. The configuration of these of
touch points were created using an 8 mm thick Plexiglas disk attached to the shaft of
the motors. Four marbles were glued near the periphery of each disk so that three of
them were close to each other while the fourth was placed on the opposite side (as seen
in Fig. 1). This ensured that the rotation of the disk was perceived by the lower part of
the thigh, near the popliteal fossa (knee-pit). Each motor was used to create three
distinct types of actuation. A four-time, 45-degree to-and-fro rotation, a single direction
two times 120-degree rotation, and a single direction two times 270° rotational. Both
120 and 270-degree signals were provided in two distinct bursts separated by 300 ms.
This actuation scheme and the designed hardware (Haptic Seat) was paired with the
navigation cluster and was used to replicate right and left turning instructions, with the
higher degree rotation simulating sharper turns (hard turns or U-turns).
3 User Study: Haptic Visual and Auditory Cues
for Navigation
The study was conducted in a laboratory setup using a driving simulator (Lane change
test software) to compare navigation feedback using conventional audio cues as well as
haptic and audio cues using the Haptic Seat prototype. For the study we replicated a
conventional car cabin in the lab, using a large central display (55 in. Samsung Smart
TV) with a Plexiglas windscreen in front of it (Fig. 2). We also added strategically
mounted displays to simulate rear and side view mirrors. The simulated car cabin used
a Logitech Gaming wheel and pedals as the driving controls, whereas a genuine Volvo
XC60 motorized driver’s seat with the embedded Haptic Seat (HS) prototype was used
to provide the haptic actuation. An adapted version of the ExoPC Slate tablet running
the Meego IVI system was affixed to the dashboard of the cabin towards the left side of
the driving controls. This setup was similar to our previous research setup used in
Nukarinen et al. [8] and Lylykangas et al. [7].
Fig. 2. Lab setup and data collected while drivers performed the LCT (primary) and
navigational (secondary) tasks.
Twenty-four participants (13 male, 11 female) were asked to operate the standard
Lane Change Test (LCT) software, as their primary task, which simulated visual and
Providing Comprehensive Navigational Cues Through the Driving Seat 885
cognitive load similar to driving in light traffic. The participants were asked to consider
driving their primary task and not let any other tasks impede their performance. Along
with the primary tasks, the participants were asked to perform 18 secondary naviga-
tional tasks, scheduled in a controlled manner (Fig. 3). The secondary task was to
identify the 6 different navigational cues, provided either by visual, auditory or haptic
modality, and log them by pressing the corresponding steering wheel button (Sec-
ondary Tasks = 18). These navigational cues included 3 cues for turning right (‘turn
right’, ‘turn slightly right’, & ‘right U-turn’) and 3 for turning left (‘turn left’, ‘turn
slightly left’, & ‘left U-turn’). These navigational cues were always provided 1.5 s after
the onset of the LCT visual cue, to ensure the cognitive load was kept constant.
Furthermore, there was always a 5-second gap between the completion of the sec-
ondary task and onset of the next primary task (LCT turning task.)
Fig. 3. Presentation of primary (LCT) and secondary (Navigation) tasks and their correlation to
each other.
The visual instructions were provided on the ExoPC Slate device acting as the
central stack, while the onscreen markers within the LCT software were disabled. The
visual instruction included a pictorial illustration of the 6 navigational tasks as well as a
written cue (Fig. 2). Similarly, 6 audio cues were generated by using active noise
canceling headphone (Sony MX1000xm2). The audio used for these instructions was
recorded from HERE maps, Android application. And lastly, we had haptic naviga-
tional cues. These cues were provided by the Haptic Seat prototype and were broken
down into two parts (Fig. 4). These parts included a precursor signal, with two pulses
separated by a 100 ms delay, and a 2 times rotation of the motor (*2 s max) navi-
gational signal. The precursor signal was provided using the Tectonic TEAX25C10-
8/HS voice coil actuators embedded on either side of the driver’s seat, while the actual
turning cues was generated by the Fukoku motors, placed on both sides of the seat.
Fig. 4. Data collected while drivers performed the interaction task
886 A. Farooq et al.
4 Results
We ran a paired sample T-test to evaluate differences between primary task perfor-
mance (PTP) comparing primary task errors (pTEs) and primary task completion times
(pTCT) to Baseline condition as well as within each modality. Looking at PTP and
pTE, we saw a statistical significance (p < 0.05) between the Baseline and both Audio
and Visual conditions as expected. Surprisingly, haptic condition (0.67) did not yield a
statistically significant difference between the baseline, which shows that errors (pTEs)
were within parameter of the Baseline condition (Fig. 5). On the other hand, pTCT
were much higher between the Baseline condition and all three modalities as expected.
Furthermore, we saw Audio and Haptic conditions also approaching significance
(0.0689). This shows that Haptic Seat based interaction increased the TCT for the
primary task compared to Audio only modality. We believe this may be because the
participants were not very familiar with the type of haptic feedback and it took them
some time to be familiarized by its use in navigational tasks.
Fig. 5. (On the left) task errors (pTE) and (on the right) task completion time (pTCT) for LCT
(primary) task.
While considering STP, we again observed that haptic and audio-based interactions
were more efficient. Although there was no statistically significant difference found
between the three modalities regarding errors (sTE), time to complete the task (sTCT)
was statistically significant for visual-audio (p = 0.04) and visual-haptic (p = 0.01)
conditions. Moreover, there was no statistically significant difference between Audio
and haptic modalities for both STP measures (Fig. 6), which illustrates that both Audio
and Haptic only conditions were faster than Visual only condition. Interestingly, most
participants choose to complete the PT (LCT task) before carrying out the ST, even
though the secondary task was presented 1.5 s after the LCT task. This clearly shows
that participants preferred completing the PT, and that the onset of secondary task may
have distracted the participants, generating higher sTCT especially for auditory
modality, as can be seen from outlier in Fig. 6.
Providing Comprehensive Navigational Cues Through the Driving Seat 887
Fig. 6. (on the left) task errors (sTE) and (on the right) task completion time (sTCT) for
Navigation (secondary) task.
5 Conclusion
A distracted driver is an inefficient driver, irrespective of the type or modality of
distraction [9]. Our study shows that visual distraction can be reduced by implementing
driver centric multimodal interaction systems for both primary and secondary tasks.
Navigation is one such task that may compliment as well as hinder the primary task of
driving. Therefore, more efficient multimodal interaction techniques need to be
developed. This study was based on utilizing a novel method of providing navigational
information through vibration-based actuation within the driver seat. There was a
statistically significant difference between the Baseline Primary Task (PT) condition
(No secondary tasks) and visual only PT as well as auditory feedback conditions while
performing secondary tasks. Interestingly there was no statistically significant differ-
ence found between the baseline and haptic conditions w.r.t primary task errors. We
believe this is because the secondary task was relatively simple and did not require
overloading the driver. Furthermore, if we cross reference this result with primary task
completion times, we do see a statistically significant difference between the baseline
and each other modality. Therefore, the drivers were slowed down by the secondary
tasks, irrespective of their interaction modality as reported by Lee et al.
Looking at secondary task performance, the results show both haptic and auditory
modality based tasks were faster to perform than visual only modality. However, there
was no statistical significance between haptic and auditory conditions with respect to
task completion times. Conversely, the secondary task errors remained similar for all
three modalities and no statistical significance seen between them. We this that par-
ticipants preferred to complete the primary task, before starting the secondary task,
which limited the possible distraction and task errors. This was most visible for the
auditory modality and, hence, there were some outliers in the sTCT data. Interestingly,
as this study was conducted in a lab setting, these results may vary considerably once
the participants perform similar tasks on the road with live traffic.
Acknowledgments. The research was supported by a Post-Doctoral mobility grant awarded to
the PI by the Finnish Culture Foundation and was carried out in collaboration with the MIVI
project at Tampere University (funded by Business Finland, decision 8004/31/2018).
888 A. Farooq et al.
References
1. Visual-manual NHTSA driver distraction guidelines for in-vehicle electronic devices.
National Highway Traffic Safety Administration (NHTSA). Department of Transportation
(DOT), Doc No. 2014-21991. https://www.federalregister.gov/documents/2014/09/16/2014-
21991/visual-manual-nhtsa-driver-distraction-guidelines-for-in-vehicleelectronic-devices
2. Cohen-Lazry, G., Borowsky, A., Oron-Gilad, T.: The effects of continuous driving-related
feedback on drivers’ response to automation failures. Human. Fac. Erg. Soc. (2017). https://
doi.org/10.1177/1541931213601974
3. Gonçalves, M., et al.: Sleepiness at the wheel across Europe: a survey of 19 countries. J. Sleep
Res. 24(3), 242–253 (2015). https://doi.org/10.1111/jsr.12267
4. Lylykangas, J., Surakka, V., Salminen, K., Farooq, A., Raisamo, R.: Responses to visual,
tactile and visual–tactile forward collision warnings while gaze on and off the road. J. Trans.
Res. Part F Traffic Psychol. Behav. 40, 68–77 (2016). https://doi.org/10.1016/j.trf.2016.04.010
5. Lee, Y.L.: The effect of congruency between sound source location and verbal message
semantics of in-vehicle navigation systems. Saf. Sci. 48(6), 708–713 (2010)
6. Sojourner, R.J., Antin, J.F.: The effects of a simulated head-up display speedometer on
perceptual task performance. J. Hum. Factors Erg. Soc. 32(3), 329–339 (1990). https://doi.
org/10.1177/001872089003200306
7. Beattie, D., Baillie, L., Halvey, M.: A comparison of artificial driving sounds for automated
vehicles. In: UbiComp 2015, pp. 451–462. ACM (2015). http://dx.doi.org/10.1145/2750858.
2807519
8. Nukarinen, T., Raisamo, R., Farooq, A., Evreinov, E., Surakka, V.: Effects of directional
haptic and non-speech audio cues in a cognitively demanding navigation task. In: ACM
NordiCHI Conference, pp. 61–64 (2014). http://dx.doi.org/10.1145/2639189.2639231
9. Lee, Y.C., Lee, J., Boyle, L.: Cognitive distraction in driving: effect of cognitive load on
drivers’ endogenous control of attention. Hum. Factors 51, 271–280 (2009)
Ensuring the Sustainability of Inclusive
Projects Through Strategic Addressing
Supported by Process Management: Case
Applied to Aquamarinna Handmade Soap
Diego S. Suarez1,2,3(&), Esteban Crespo-Martínez1,2,
and Pedro Mogrovejo1
1
Escuela de Ingeniería de la Producción, Universidad del Azuay,
Cuenca, Ecuador
{ssuarez,ecrespo}@uazuay.edu.ec,
[email protected]
2
Departamento de Vinculación con la Comunidad, Universidad del Azuay,
Cuenca, Ecuador
3
CIIPO, Universidad del Azuay, Cuenca, Ecuador
Abstract. In 2018, an inter-institutional cooperation agreement was signed
between the School of Production Engineering of Universidad del Azuay
(Ecuador) and Mensajeros de la Paz Foundation, for the execution of a social
entrepreneurship sustainability project. For ten years, this Foundation has
empirically developed personal and domestic hygiene products for its consumer
market; however, the business activity itself motivates to seek the generation of
economic income through the diversification and commercialization of its
products, which boosted the incubation of the Aquamarinna Company. The
corporate purpose of this venture aims at the labor inclusion of people with
disabilities within the production process, who in turn are also beneficiaries of
the Foundation. With a view to structuring the company at all levels, a Strategic
Addressing was developed, supported by Process Management with the
objective of having a structure that ensures the sustainability of this venture,
maintaining its social focus of inclusion.
Keywords: Strategic Addressing Process Management Social
entrepreneurship Labor inclusion
1 Introduction
According to [1] a fair society is an important prerequisite for an innovative society,
perhaps its most important long-term prerequisite. Whereby a fair society is built based
on social inclusion.
[2] Mention that social inclusion must allow vulnerable and excluded people to
participate actively in at least four main areas: talking to people; being accepted; using
community facilities; opportunities.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 889–895, 2020.
https://doi.org/10.1007/978-3-030-39512-4_136
890 D. S. Suarez et al.
This document shows the case of an entrepreneurship whose purpose is not only to
generate profitability. Income may be the most prominent means for a good life without
deprivation, but it is not the only influence on the lives we can lead. If our paramount
interest is in the lives that people can lead – the freedom they have to lead minimally
decent lives – then it cannot but be a mistake to concentrate exclusively only on one or
other of the means to such freedom. We must look at impoverished lives, and not just at
depleted wallets [3].
Aquamarinna is a social enterprise promoted by Fundación Mensajeros de la Paz
and supported by Universidad del Azuay, whose objective is to improve the quality of
life of its beneficiaries (people with disabilities) through their social and labor inclu-
sion. The organization is dedicated to the handmade production of products for per-
sonal and domestic hygiene.
Like any venture, Aquamarinna has difficulties in its management; however, the
complexity is even greater when additionally it is intended to include a vulnerable
social sector within its organizational and functional structure. Thus, this article shows
a management methodology that allows structuring a social entrepreneurship with the
objective of being sustainable over time. The proposal was applied and is based on the
combination of Strategic Addressing with Process Management.
2 State of Art
Strategic Addressing according to [4] is one of the central points of the administrative
processes of any organization, due to its interference in the decision-making in the
short, medium and long term. On the other hand, according to [5], Strategic Addressing
is aimed at an adequate combination and integration of the means and resources of
organizations to achieve their goals, so this management method allows establishing a
framework of actions for the achievement of goals.
Process Management, according to [6] says that for an organization to function
efficiently, it has to identify and manage numerous related activities. [7]
According to [8], processes constitute the basis of strategic management for many
companies. This is due to the flexibility offered by this type of organization to adapt to
frequent changes in the environment and the market. This ability to adapt to the market
or customers makes management models oriented to Total Quality, such as the EFQM
Model of Excellence, focus their attention on processes as the most powerful lever to
act on the results effectively and sustainably.
Social Entrepreneurships according to, [9], refer to the construction, estimation and
scope of opportunities for the transformation of social change by visionary people,
aiming to solve community problems. At the same time, they define the difference
between social entrepreneurship and charities, considering that social entrepreneurship
generates a sustainable social value. In turn, [10], mentions that social entrepreneurship
goes beyond generating a profitable business in view of an opportunity or need to
undertake. It involves developing, executing and sustaining initiatives aimed at over-
coming a social difficulty, and achieving a common benefit to a human group.
Some factors that must be taken into account and that according to [11], determine
the failure of a social enterprise are: difficulty in finding sources of financing, loss of
Ensuring the Sustainability of Inclusive Projects 891
social focus, difficulty in inserting new forms and methods and the difficulty of finding
the support or alliance of major institutions.
On the other hand, [12] argues that social entrepreneurship reduces social problems
and supports the economy, reducing unemployment and inappropriate behaviors of
these vulnerable groups such as vandalism and addictions; in turn develops the skills of
the people involved.
3 Method
The Strategic Addressing and Process Management merge to ensure the sustainability
of social and inclusive projects, as shown in the following figure.
As part of the company’s Strategic Addressing, Strategic Values, Vision, Mission
and Business Strategy must be defined.
Strategic Values of the organization are proposed as the baseline of this proposal. In
order to define them a multidisciplinary committee must be formed, which must discuss
the different dimensions to consider, then give them a rating from 1 to 5, being 1 non-
important dimensions, and 5 extremely important dimensions. Thus, only the dimen-
sions that obtain a rating of 5 will be considered as Strategic Values.
The Vision is a declaration, whose content, based on the Strategic Values, is
sufficient to answer the following questions: Who are we? Where do we aspire to
reach? (In the determined future) What do we want to achieve within the established
time frame? Why do we want to build this future scenario?
Subsequently, the Mission is defined, whose declaration contains the way in which
the organization will reach what the Vision establishes. For this, the following ques-
tions must be answered: What are we? What activities do we do or should we do? What
are our products? To whom does our work add value? How do we create value?
Then Business Strategy is defined, whose purpose is to expedite decision making.
In order to elaborate it, four main management axes are considered: Financial Eco-
nomic; Customer and stakeholder perspective; internal processes; Learning and growth.
These axes are based on the model of the Balanced Scorecard proposed by [13]. The
strategies must answer the following questions.
How to reach customers? What are the current market trends? How will the rela-
tionship with stakeholders and competitors be managed? What strategic factors should
be considered priority? What value strategies should we use? To answer the last
question, it is recommended to use the classification of value strategies proposed by [6,
14]. In this way, each proposed strategy must be weighted, where the most important
one will be referred to as the Driving Force, and the two or three following strategies
will be considered as Key Success Factors.
With Strategic Addressing defined, there is a macro structure of business man-
agement, from which the Process Management can be derived. Thus, as shown in
Fig. 1, the Process Management tools are contained within a Triangle whose purpose is
to represent that those tools that are at the bottom, will carry more detailed information
on the processes, and those at the top will have more extensive information.
892 D. S. Suarez et al.
Fig. 1. Proposed methodology for the assurance of inclusive projects.
Therefore, the first Process Management tool to define is the Process Map, a visual
tool that allows the company to organize the processes that compose it, and in turn,
catalog them into three types: Strategic Processes (to ensure sustainability over time),
Key Processes (to carry out the mission), and Support Processes.
Then, in order to identify the way in which the company’s processes are related, the
Interaction Matrix is prepared, whose purpose is to identify the existing inputs and
outputs that each process has. Subsequently, for a more detailed interpretation of the
Interaction Matrix, the SIPOC Diagram (Suppliers, Input, Process, Output, Customers)
is prepared, where each Supplier will be identified for each process, with their
respective entries, as well as their Customer (Internal or External), with the respective
outputs of the process.
Then, the Process Characterization File is elaborated, where relevant information of
each process is stipulated: Objective of the process; Responsible for its execution;
Startup activity; Completion Activity; Human team of the process; Available resources;
Process cycle; KPI (Key Performance Indicators).
Finally, Flowcharts are used for each process, defining each of the activities that
compose it, as well as each of its managers, reaching a deep level of knowledge about
the operation of each process.
Ensuring the Sustainability of Inclusive Projects 893
4 Results
The Strategic Values defined were Product quality; Social inclusion; Comprehensive
security; Engagement with the client; Professional ethics; Learning capacity. Based on
this Mission, Vision and Business Strategy were defined.
Business Strategy stipulates the Quality of Products as a Driving Force, and Human
Capital (with emphasis on the inclusion of people with disabilities), Profitability and
Customer Requirements as key success factors.
Regarding Process Management, in the elaboration of the Process Map three
Strategic Processes were identified, as well as five Key Processes and two Support
Processes.
Both the Interaction Matrix and the SIPOC diagram allowed Aquamarinna to
understand the link between these processes as well as the information flow between
them, standardizing the processes and streamlining decision-making.
The Process Characterization Files allowed the company to have a greater
knowledge of the objectives of each process, adding to the inclusion of people with
disabilities as a crosscutting objective of all processes.
The flowcharts were printed and displayed in different areas of the company as a
visual support in case of doubts in the steps to follow or in decision making. Many
activities were designed to allow the inclusion of people with disabilities in the pro-
duction process.
Managing the organization from its strategy and its processes allowed a notable
improvement in product quality, reducing the amount of defective products. This in
turn allowed the creation of new products.
5 Discussion
Strategic Addressing applied in social projects has allowed to improve decision making
and articulate actions in search of the fulfillment of objectives, as mentioned by [5] and
[4]. However, these actions must be born from the generation of the Strategic Values
because they define the essence of an organization (labor inclusion in this case). In this
way, the proposed methodology encourages the active participation of socially
excluded people in the four main areas mentioned by [2].
Regarding the application of Process Management, Aquamarinna has clearly
managed to improve in terms of product quality, efficiency and costs. Management
areas that emphasize [7] and [8], but these perspectives can be complemented with
labor inclusion, which in the case of study; this is one of the most representative results,
with Process Management being the key tool to achieve it, through the design of
inclusive processes.
Thus, in accordance with the statements made by [10], a great difference between
the impacts that social entrepreneurship can have on the community is denoted, since
contrary to charities, its approach is much broader.
894 D. S. Suarez et al.
The fact of working with Universities is highlighted, as in this case study, to
overcome the difficulties to generate strategic alliances mentioned by [11].
6 Conclusions
The application of the proposed methodology has allowed Aquamarinna to have a clear
path of where it should go in a defined future. It is precisely known that the essence of
the company is present in its Strategic Values, and that these influence its organization
and decision-making. In addition, through the proper Management of its Processes, it
has been possible to improve the profitability of the company, because the quality of
the product has been improved and the number of errors and reprocesses has been
reduced.
Additionally, the labor inclusion of people with disabilities, which is one of the
most important Strategic Values within this social project, has been constantly present
in the development and application of this methodology, ensuring that the essence of
this social project is maintained.
Many enterprises and projects have difficulties in their early years. A social project
entails an even greater level of difficulty considering that it seeks to generate a positive
impact on the community and vulnerable groups, so the fact of having developed and
applied this methodology, allowed to reduce the risk of making bad decisions and got
the organization to function efficiently. Thus, the fusion of Strategic Addressing with
Process Management is a methodology that ensures the sustainability of social projects
over time.
References
1. Dalum, B., Jhonson, B., Lundvall, B.-A.: Public policy in the learning society. In: National
Systems of Innovation. Towards a Theory of Innovation and Interactive Learning. Anthem
Press, London (2010)
2. Abbott, S., Mcconkey, R.: The barriers to social inclusion as perceived by people with
intellectual disabilities. J. Intell. Disabil. 10(3), 275–287 (2006)
3. Sen, A.: Development thinking at the beginning of the 21st century. In: Economic and Social
Development into the XXI Century, Inter American Development Bank, Washington,
pp. 531–551 (1997)
4. E. Hilda, H. Saumett y A. Montenegro, Direccionamiento Estratégico y Aprendizaje,
Barranquilla: Editorial Mejoras, 2017
5. A. Aguilera, «Direccionamiento estratégico y crecimiento empresarial: algunas reflexiones
en tomo a su relación Pensamiento & Gestión,» Pensamiento & Gestión, p. 106, 2010
6. ISO-9001, p. Vi, Ginebra, 2000
7. Bravo, J.: GESTIÓN POR PROCESOS. EVOLUCIÓN S.A, Santiago (2008)
8. Zaratiegui, J.: La gestión por procesos: su papel e importancia en la empresa. Economiía Ind.
330, 81–88 (1999)
9. Vásquez, A., Trujillo, M.: Emprendimiento social - revisión de literatura. Estud. Gerenciales
24, 105–125 (2008)
Ensuring the Sustainability of Inclusive Projects 895
10. Bargsted, M.: El emprendimiento social desde una mirada psicosocial. Civilizar Cienc.
Sociales y Humanas 13, 121–132 (2013)
11. Lorca, M.: Modelo de Factores que inciden en el éxito de emprendimiento socuales en
América Latina: Estudio Cualitativo, Santiago (2013)
12. Leadbeater, C.: The Rise of the Social Entrepreneur. Demos, London (1997)
13. Kaplan, R., Norton, D.: Cuadro de Mando Integral (The Balance Scorecard), Barcelona:
Ediciones Gestión (2000)
14. Fred, D.: Conceptos de Administración Estratégica. Pearson Educación, México (2003)
A New Model to Bionic Hand Prosthesis
with Individual Fingers Actuators
Marcelo H. Stoppa(&), Guilherme F. Neto,
and Danillo A. de S. Dunck
Federal University of Goiás, Catalão, Goiás, Brazil
[email protected]
Abstract. Considering human mobility, the hand assumes great importance
due to its versatility and ability to manipulate objects. However, many people do
not have this limb, due to either malformation or amputation, which makes it
difficult to perform daily activities. This research presents the development of
the mechanical structure of a hand prosthesis where the fingers are individually
actuated. For this, the mathematical model that describes the movement of the
robotic finger is detailed. A kinematic study of finger movement is performed, as
well as a computer simulation for the proposed mechanical structure. The model
simulations showed a good effectiveness of the proposed model, and all the
component parts of the prosthesis were designed in a parameterized way so that
the model can be adapted to the physical dimensions of the user.
Keywords: Bionic hand prosthesis Kinematic Computer simulation
1 Introduction
Technology has contributed to the improvement of the daily lives of humans. Simple
daily activities can be facilitated using specific devices such as hair brushes, mirrors,
forks, knives, among others, tools that improve performance in performing the func-
tions. It is well known that technology can also be employed in devices and services
that offer people with disabilities the ability to perform or complement motor and
intellectual functions, generating inclusion and independence of the individual. Pros-
thetics are examples of this type of technology, also known as Assistive Technology.
Prosthetization is a technique used to replace partial or total absence of limbs, assisting
the individual in daily routines [1].
There is motivation to look for a affordable 3D printing hand prosthesis that is
accessible to more people with disabilities. In addition, developing a non-cable-driven
finger drive method can improve the transmission efficiency of motions by utilizing
direct current motors as actuators for longer prosthesis life. Given the complexity of the
human hand, one can understand the great challenge of developing a hand prosthesis.
The purpose here is to develop the mechanical structure of a hand prosthesis made
using a 3D printer, guided by the study of human hand movements, with individual
finger activation, using as actuators, direct current motors, and as transmission ele-
ments, gears and interconnect bars between the phalanges. The study is limited to the
development of the mechanical structure of the prosthesis and is not addressed actuator
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 896–902, 2020.
https://doi.org/10.1007/978-3-030-39512-4_137
A New Model to Bionic Hand Prosthesis with Individual Fingers Actuators 897
control techniques. Thus, this paper presents the results obtained in the development of
a new prototype hand prosthesis with individual finger movement.
2 The Human Hand
The human hand is an essential part of the body for handling objects and food,
although it is present in many animals, in humans it makes a difference for its versatility
helping its development [2]. The prosthesis has the function of artificially replacing a
missing part of the body, conferring independence, quality of life and self-esteem [3].
The human hand, located in the final extension of the upper limb (UL), is one of the
most complex mechanical structures of the human body. Made up of 27 bones and 17
joints, it is capable of moving up to 23 degrees of freedom (DOF), 2 in the palm of the
hand, 5 in the thumb and 4 in each of the index, middle, ring and little fingers. This
structure allows its adaptation to various object forms [4]. The MS consists of three
joints, the shoulder with 3 DOF, the elbow with 1 DOF and the wrist with 2 DOF, in
addition to the 23 DOF of the hand, the human UL consists of 29 DOF [5, 6]. Finding a
point in space requires three coordinates, usually x, y, and z. In order to locate an object
it is necessary to define its position in relation to a point in space, using three other
coordinates, so to locate an object in space it is necessary and sufficient six coordinates,
i.e., 6 DOFs [6]. Note that the 29 DOF human UL has great ability and precision to
manipulate objects in any position, since only 6 DOFs are sufficient [4, 6].
The great ability to perform hand actions is due to its main function, which is
gripping [7]. Grips perform defined tasks and should be classified into three basic
groups according to application: digital, palmar and centralized [4]. Grips can further
be classified into two types: strength and precision. Force gripping occurs when
holding an object with a palm exerting counter-force with partially flexed fingers.
Precision gripping, on the other hand, occurs when an object is fixed delicately between
the lateral fingers and thumb or between the thumb and palmar face [8].
3 Mathematical Model
This study predicts the development of the mechanical structure of an individually
actuated five-finger hand prosthesis. As with the human hand, each finger of the
prosthesis has a different dimension, so it is necessary to develop a model to calculate
the location of the final extension of each finger (P1, P2, P3, P4 and P5) as a function of
the flexion angles (a1, a2, a3, a4 and a5) formed between the palm and the proximal
phalanx of each finger (metacarpophalangeal joints), having as a reference a common
origin, considered in the center of the wrist.
The concept adopted for the mechanical structure of the prosthesis allows rotation
movements in the metacarpophalangeal and interphalangeal joints in only one refer-
ence axis, so the fingers of the prosthesis will be able to perform only flexion and
extension movements, and it is not possible to perform the adduction and abduction
movements.
898 M. H. Stoppa et al.
Taking into account the natural position of the human hand, the middle finger
disposed on the prosthesis was considered so that its center line is on the y-axis of the
global reference system. The little, ring and index fingers were arranged parallel to the
middle finger. For the opposite finger, rotation of w in relation to the z-axis and rotation
of x in relation to the x-axis was considered. Figure 1 illustrates the structure adopted.
Fig. 1. Referential of structure proposed: (a) superior view, (b) frontal view
3.1 Transmission of Motion
The main motion transmission elements are the interconnecting bars of the rotation
joints, indicated in Fig. 2 by bt, bn1 and bn2. The lengths of these bars are determined as
a function of the points of rotation Rt1, Rt2, Rn1, Rn2, Rn3 and Rn4; of length of the
phalanges and the movement that the proximal phalanx should cause in the medial and
distal phalanges.
Fig. 2. Schematic representation of the fingers: (a) thumb, (b) index
The anatomical configuration of the thumb is different from other fingers since it
does not have the medial phalanx. Thus, two models are presented, one for the thumb,
where the parameters have the index t, and another model common to the other fingers
where the parameters have the index n, such that n = i, m, r, l, representing, respec-
tively, the index, middle, ring and little fingers. This will consider the length of a
segment for its identification in the model.
In the human hand, the total flexion movement of the middle finger in the
metacarpophalangeal joint (MCP) is close to 90°, close to 110° in the proximal
interphalangeal joint (PIP), and close to 80° in the distal interphalangeal joint (DIP),
A New Model to Bionic Hand Prosthesis with Individual Fingers Actuators 899
tending to increase when analyzing the next fingers towards the little finger and tending
to decrease by analyzing in the direction of the thumb. The prosthesis does not predict
hyperextension movement, so for the proposed structure the flexion and extension
angles were adopted according to the angular variation proposed by Kapandji [4].
3.2 Thumb Model
Figure 3a shows the thumb model where the bar bt, which is transverse to the proximal
phalanx ftp, must move the distal phalanx ftd. The metacarpophalangeal and distal
interphalangeal rotation joints are indicated respectively by A and B. The points of
rotation where the bt bar is coupled are indicated by Rt1 and Rt2, whose position
depends on the distances dt1 and dt2 and the angles dt1 (formed between dt1 and the
palm plane) and dt2 (formed between dt2 and the distal phalanx). So that Rt1 and Rt2
remain within the thumb volume, contributing to the aesthetics of the prosthesis, dt1 is
taken to be 20% of the proximal phalanx length, i.e., dt1 = 0.2ftp.
Fig. 3. Schematic models: (a) thumb, (b) index, middle, ring and little.
To determine the angles dt1 and dt2, which must be fixed in the structure, and the
dimension dt2, the model is analyzed and compared in the extreme cases, i.e., total
flexion and total extension. Equation (1) describes the ct angle as a function of the at
angle.
" #
0:19879737 0:4 cosð70 at Þ
ct ¼ a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
0:277504 1:04 0:4 cosð70 at Þ
" #
2 0:4 cosð70 at Þ
a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 65 : ð1Þ
2 1:04 0:4 cosð70 at Þ
3.3 Index, Middle, Ring and Little Models
Similarly to what was presented for the thumb model, Fig. 3b shows the model for the
other fingers, where bn1 and bn2 are transverse bars, respectively, to the proximal
900 M. H. Stoppa et al.
phalanx (fnp) and medial phalanx (fnm), which in action joint move the distal phalanx
(fnd).
As for what was developed for the thumb, we get the Eq. (2) which correlates, the
angles bn and an, and the Eq. (3) which correlates cn and bn.
0:09298093 0:4 sin an 2 0:4 sin an
bn ¼ a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 65 : ð2Þ
0:38524 1:04 0:4 sin an 2 1:04 0:4 sin an
" #
0:02437302 0:4 cosð110 bn Þ
cn ¼ a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
0:573196 1:04 0:4 cosð110 bn Þ
" #
2 0:4 cosð110 bn Þ
a cos pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 65 : ð3Þ
2 1:04 0:4 cosð110 bn Þ
Substituting (2) in (3) is found the relationship between the angles cn and an for the
proposed mechanical structure.
Then by means of matrix transformations, we find the coordinates (4) and (5) of the
fingertips determined as a function of at.
Pny ¼ cosðan þ bn þ cn ÞPnb sinðan þ bn þ cn ÞPnc þ fpn cosðbn þ cn Þ þ fmn cos cn þ fdn :
ð4Þ
Pnz ¼ sinðan þ bn þ cn ÞPnb þ cosðan þ bn þ cn ÞPnc þ fpn sinðbn þ cn Þ þ fmn sin cn : ð5Þ
4 The Prototype
The index, middle, ring and minimum fingers consist of 3 phalanges: proximal, medial
and distal. The thumb is composed of only 2 phalanges: proximal and distal.
The movement of the medial and distal phalanges are caused by the transmission of
proximal phalanx movement. To generate movement in this phalanx, a DC motor with
integrated reduction gearbox was adopted. Coupled to the shaft of this motor, a bevel
gear was used that will be the connection device between the motor and proximal
phalanx.
The bars bt, bn1 and bn2 transmit the movement between the phalanges, so the
dimension between holes is the main dimensional dimension of this component. For
the prototype, these bars were inserted inside the phalanges. Figure 4 shows the thumb
and index finger in longitudinal section.
The complete proposed prosthesis illustrating how to connection between the
engines, and the palm can be see at Fig. 5.
A New Model to Bionic Hand Prosthesis with Individual Fingers Actuators 901
Fig. 4. Longitudinal section: (a) thumb, (b) index
Fig. 5. Positioning of motors in prosthesis
5 Results
The mathematical model proposed here is for the mechanical structure of a five-finger
hand prosthesis. The concepts of robotic positioning and geometry were used to
determine the references and coordinates of the hand joints. Because it is geometry in
space in three dimensions, homogeneous matrices of rotation and translation were
necessary for the correct positioning of the frames that were conveniently placed on the
metacarpophalangeal and interphalangeal joints of the hand, considered as the global
reference point, the center of the wrist.
The proximal phalanx of each finger is moved by the rotation of a bevel gear
coupled to a direct-drive motor with reduction gear, which rotates another bevel gear
integrated into the phalanx. The medial and distal phalanges are moved by two drive
bars. Thus all movement is performed considering only the angle variation of the
metacarpophalangeal joint. The other angles are determined as a function of this first.
Two models were developed, one specifically for the thumb that has only the
proximal and distal phalanges, which consequently made the movement could be
performed by only one interconnecting bar between phalanges. The other four fingers
(index, middle, ring and little) have three phalanges and therefore lack two connecting
bars.
902 M. H. Stoppa et al.
All component parts of the prosthesis were designed using CAD software,
respecting the considerations of the mathematical model. Motion simulations were
performed using CAM software to point out constructive adjustments to the model. As
an example, to avoid internal shocks between the finger components, the connecting
bars needed to change their geometry as shown in the prototype presentation chapter.
Importantly, the entire CAD project was developed parameterized, so that custom
prosthesis designs and subsequent impressions with adjustments of phalanx size, palm
size, and other physical aspects can be directly implemented.
Acknowledgments. The authors gratefully acknowledge the financial support by, UFG/RC,
PPGGO, and PPGMO by FAPEG-CAPES, proc. 88881.127849/2016-01.
References
1. Bersch, R.: Introdução à tecnologia assistiva. CEDI, Porto Alegre (2008)
2. Kapandji, I.A.: The Physiology of the Joints. Elselvier Sciense, Amsterdam (2002)
3. Carvalho, G.L.: Proposta de um método de projeto de próteses de membros superiores com a
utilização da engenharia e análise do valor. Ph.D. thesis, Universidade de São Paulo (2004)
4. Tubiana, R.: The hand. WB Saunders Philadelphia (1981)
5. Hall, S.: Biomecânica básica. Guanabra-Koogan (2000)
6. Niku, S.: Introduction to Robotics. Wiley, Hoboken (2010)
7. Polis, J.E., et al.: Projeto e construção de parte estrutural de prótese de mão humana com
movimentos. Campinas, SP (2009)
8. Napier, J.R.: The prehensile movements of the human hand. Bone Joint J. 38(4), 902–913
(1956)
A Predictive Model of Users’ Behavior
and Values of Smart Energy Meters Using
PLS-SEM
Ahmed Shuhaiber(&)
College of Technological Innovation, Zayed University, Abu Dhabi, UAE
[email protected]
Abstract. A smart energy metering system is an IoT device that connects
several electrical household devices and record, monitor, estimate, control in-
house energy consumption in a real-time basis. Although smart energy meters
have great capabilities, this technology is still in infancy stages in many
developing countries, and little is known about what perceived values are
associated with smart meters from residents’ perspectives. Therefore, this
research aimed to fill this gap by examining the impact of six different types of
perceived values on residents’ intentions to use smart meters in UAE. The study
followed a quantitative approach by gathering 266 survey responses which were
tested by using Partial Least Squares-Structural Equation Modeling (PLS-SEM).
The statistical results genuinely indicated that perceived epistemic values,
environmental values, emotional values, and convenience values can signifi-
cantly impact residents’ intention to use smart meter, whereas social values and
monetary values found to have no significant impact on their intentions to use
this technology. Theoretical and practical implications are indicated, and
directions of future research are specified afterwards.
Keywords: Intention to use Energy consumption Perceived values Smart
grids Smart meters
1 Introduction
Smart city services and applications are ubiquitous nowadays. As one example, smart
metering system is an IoT (Internet of Things) application that provides a solution for
reading, monitoring and controlling energy consumption rates in real time basis.
Traditionally, the smart metering system consists of two devices: a smart energy meter
of electricity and a separate In-Home Display device (IHD). The smart meter sends real
time data about the current usage of energy from different devices to the IHD which
displays the consumption rates continuously, and the display instantly shows how
much energy in use and how much it costs.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 903–908, 2020.
https://doi.org/10.1007/978-3-030-39512-4_138
904 A. Shuhaiber
The smart meter, therefore, can provide many benefits to users from several
aspects:
(a) sending the monthly consumption rate wirelessly to the energy supplier company,
(b) providing helpful automated visual readings of daily energy consumption that can
help users in making decisions associated with their energy usage behavior [1],
(c) mining the frequent patterns and data association generated from the smart meters
[2], (d) enabling users for better controlling and forecasting energy loads in
regular basis [3]. The smart meter literature with regards to perceived values is
reviewed and summarized as pointed out in next section.
2 Previous Studies
Perceived values as associated with perceived monetary values, perceived convenience,
perceived social values, perceived environmental values, and emotional values, per-
ceived epistemic values [4], and these values attracted very few scholars and
researchers in the smart meters’ literature. As a response, the aim of this research is to
examine the influence of perceived values of smart meters on users’ behavior on
intending to use smart meters, in one developing country: The United Arab Emirates.
This was achieved by developing a research model that clarifies the relationships
among the variables, testing the model, and then providing a discussion about the
results found. Descriptions and details about these types of perceived values are
detailed in the subsequent sections. The following table (Table 1) illustrates all research
constructs, and the associated descriptions, research studies, examples and the
hypotheses generated.
3 Methodology
A quantitative method was employed in this research to address the research
hypotheses, by surveying respondents through a questionnaire of 25 items that were
developed by reviewing the relevant literature. Online-based surveys were distributed
randomly and through the snow-ball technique among people of the UAE. The targeted
respondents were UAE residents who are aware of smart meter technology, older than
18 years and willing to share their perceptions about smart meters, as recommended by
the reference [14]. Social media links and email invitations were shared hyperlinks
across personal networks, online blogs and groups.
A Predictive Model of Users’ Behavior and Values 905
Table 1. Research constructs and previous studies
Construct Description Literature Hypothesis
Intention to use The strength of one’s Technology –
intention to use smart Acceptance
meters [7–9]. Model TAM [5] or
Unified Theory of
Acceptance and Usage
Theory (UTAUT)
models [6]
Convenience The suitability and Dimensions such H1: Convenience
value helpfulness of smart consumption rates Value significantly
meters accessibility, and positively
availability, influence Intention
understanding to use
consumption behavior
[7, 9]
Emotional Extrinsic values that Three dimensions: H2: Emotional
value motivate individual pleasure, comfort and Value significantly
towards smart meter security [4, 8] and positively
technology influence Intention
to use
Environmental Perceiving smart Low-carbon energy H3: Environmental
value meters as environment power grid [4], Value significantly
friendly, clean, green mitigating climate and positively
technology change [10], the case of influence Intention
Hong Kong [9] to use
Epistemic Cognitive successes Knowledge, familiarity, H4: Epistemic
value such as true beliefs, prior experiences, and value significantly
justified beliefs, awareness [10, 12] and positively
knowledge, and influences Intention
understanding to use
Monetary The amount of money Calculation of hourly H5: Monetary
value that would be paid or pricing [9], alert Value significantly
exchanged for the sale consumers to shift at and positively
of a smart meter peak hours to other influence Intention
times [7], prepaid plans to use
for easier credit
tracking by topping up
from a computer or a
smartphone application,
as in the Turkish model
[11]
Social value Moral principles Social image, social H6: Social Value
defined by society self-concept and social significantly and
dynamics, institutions, class membership [12], positively influence
traditions and cultural appreciation, social Intention to use
beliefs recognition and
gratitude [13]
906 A. Shuhaiber
4 Data Analysis and Findings Discussion
The method used for data analysis is the Structural Equation Modelling-Partial Least
Squares (SEM-PLS) multiple regression. The PLS method is an iterative process that
provides successive approximations for the estimates of loadings and structural
parameters [15]. The PLS regression algorithm usually examines the items and con-
structs associated with individual constructs (outer loadings), and then examine the
structural model. The threshold point of item loadings is 0.4, but preferably not less
than 0.6 [16]. By testing the research items, all loadings demonstrated reliable items,
with all item loadings greater than 0.75. Construct validity was also assessed, including
both convergent and discriminant validity. Convergent validity was examined through
the Average Variance Explained (AVE) score which should exceed the threshold of 0.5
[16]. As shown in Table 2, the AVE scores for all constructs exceeded 0.5, which
meets demonstrate convergent validity.
Table 2. Validity and reliability estimates
Item code Cronbach’s Composite Extracted
Alpha > 0.7 reliability (AVE) > 0.5
Convenience value 0.81 0.811 0.888
Emotional value 0.825 0.826 0.895
Environmental 0.865 0.867 0.917
value
Epistemic value 0.721 0.774 0.839
Intention to use 0.842 0.85 0.905
Monetary value 0.795 0.813 0.879
Social value 0.751 0.765 0.856
The internal consistency of all constructs was all assessed by measuring Cronbach’s
alpha scores, which is achieved by having the score greater than or equal to .70 as the
most common threshold [15, 16]. As presented in Table 2, all reliability estimates were
found greater than the cut-off point of 0.7 and thus demonstrating internal consistency.
The discriminant validity also examined to reflect on how distinct one construct from
other latent constructs in predicting the dependent variable [16]. The results in Table 3
indicate that none of the off-diagonal elements exceeded the respective diagonal ele-
ment, as conditioned in the criterion of reference [15], and thus, discriminant validity
was achieved.
The PLS inner structural model was examined to investigate the significance of the
paths and the predictive power of the model by considering a bootstrapping process
[16]. Table 4 highlights the Beta values of each the latent variables, T-Statistics, P-
Values and hypotheses results [16]. As a result, four hypotheses were supported in 0.05
significant level, whereas only 2 hypotheses that are associated with monetary and
social values were found not significant paths.
A Predictive Model of Users’ Behavior and Values 907
Table 3. Correlation matrix among constructs
Conven Emotion Enviro Epist Intent Monet Social
Conveni 0.85
Emotion 0.55 0.86
Enviro 0.67 0.66 0.88
Epist 0.52 0.64 0.55 0.79
Intent 0.64 0.68 0.69 0.66 0.872
Monet 0.68 0.62 0.62 0.56 0.611 0.84
Social 0.50 0.67 0.58 0.58 0.526 0.57 0.8
Table 4. Regression paths and hypotheses results
H# Original sample (b) T statistics (|O/STDEV|) P-value Hypothesis result
H1 0.193 3.055 0.002 Supported
H2 0.247 3.2 0.001 Supported
H3 0.26 3.912 0 Supported
H4 0.281 3.442 0.001 Supported
H5 0.052 0.791 0.429 Not supported
H6 −0.083 1.393 0.164 Not supported
It is noteworthy that the strongest path in the research model is the influence of
perceived epistemic value on intention to use smart meters (b = 0.281), whereas per-
ceived convenience value scored the lowest value in affecting intention to use this
technology (b = 0.193) where it was still found significant on 5% confidence interval.
The amount of variance explained by R2 provides an indication of the model fit as well
as the predictive ability of the endogenous variables. The R2 scores should be greater or
equal to 0.10 [16]. As a result, the R2 value of ‘intention to use’ was found moderate
and equal to 64.9%. Overall, four dimensions of perceived values are found to have a
significant influence on intention to use it. Perceived epistemic value formulates the
awareness about smart meters which is a vital element to accept the technology and use
it, supporting previous research [17]. Perceived emotional value indicates that users’
feelings of pleasure, comfort, and security are effective for accepting and using smart
meter. Perceived Monetary value in turn could contribute positively and significantly to
the behavioral intention, which means that the financial aspect of smart meters in terms
of costs and savings can predict the resident’s behavioral toward them.
This study has some limitations, such as the one associated with the survey-based
method that is prone to measurement errors [14]. To minimize the effect of this limi-
tation, the researcher developed the items based on the literature and previous studies.
For future research work, the researcher suggests not only understanding residents’
intention to use smart meter services in different countries, but also for comparing their
attitudes and behaviors in other Gulf and foreign countries through cross-cultural
studies. This also suggests extending the current model with other perceived values
such as social values, conditional values, convenience values and others for better
holistic view of smart meter behavioral intentions.
908 A. Shuhaiber
Acknowledgments. The researcher acknowledges Zayed University for the financial support of
this research.
References
1. Gianniou, P., Reinhart, C., Hsu, D., Heller, A., Rode, C.: Estimation of temperature setpoints
and heat transfer coefficients among residential buildings in Denmark based on smart meter
data. Building Environ. 139, 125–133 (2018)
2. Reddy, G.R.S., Chakravarthy, B.D., ReddiNeelima, C., Zion, G.D.: Mining frequent patterns
and associations from the smart meters using bayesian networks. Int. J. Adv. Netw. Appl. 9
(6), 3632–3639 (2018)
3. Fallah, S.N., Deo, R.C., Shojafar, M., Conti, M., Shamshirband, S.: Computational
intelligence approaches for energy load forecasting in smart energy management grids: state
of the art, future challenges, and research directions. Energies 11(3), 596 (2018)
4. Shuhaiber, A.: The role of perceived control, enjoyment, cost, sustainability and trust on
intention to use smart meters: an empirical study using SEM-PLS. In: Rocha, Á., Adeli, H.,
Reis, L., Costanzo, S. (eds.) Trends and Advances in Information Systems and
Technologies, WorldCIST’18 2018. Advances in Intelligent Systems and Computing, vol.
746. Springer, Cham (2018)
5. Davis, F.D., Bagozzi, R.P., Warshaw, P.R.: User acceptance of computer technology: a
comparison of two theoretical models. Manag. Sci. 35(8), 982–1003 (1989)
6. Venkatesh, V., Morris, M.G., Davis, G. B., Davis, F.D.: User acceptance of information
technology: toward a unified view. MIS Q. 27, 425–478 (2003)
7. Bertoldo Poumadère, R., M., Rodrigues, Jr., L.C.: When meters start to talk: the public’s
encounter with smart meters in France. Energy Res. Soc. Sci. 9, 146–156 (2015)
8. Chou, J.S., Kim, C.C., Ung, T.K., Yutami, I.G.A.N., Lin, G.T., Son, H.: Cross-country
review of smart grid adoption in residential buildings. Renew. Sustain. Energy Rev. 48, 192–
213 (2015)
9. Zhou, S., Brown, M.: Smart meter deployment in europe: a comparative case study on the
impacts of national policy schemes. J. Cleaner Prod. 144, 22–32 (2017)
10. Kranz, J.J., Gallenkamp, V., Picot, A.: Exploring the role of control-smart meter acceptance
of residential consumers. In: AMCIS, p. 315 (2010)
11. Mashal, I., Shuhaiber, A.: What makes Jordanian residents buy smart home devices? A
factorial investigation using PLS-SEM, Kybernetes (2018)
12. Pihlström, M.: Perceived value of mobile service use and its consequences. Svenska
handelshögskolan (2008)
13. Bugden, D., Stedman, R.: A synthetic view of acceptance and engagement with smart meters
in the United States. Energy Res. Soc. Sci. 47, 137–145 (2019)
14. Neuman, W.L.: Social Research Methods: Quantitative and Qualitative Approaches, 6th edn.
MA Allyn & Bacon, Boston (2005)
15. Fornell, C., Larcker, D.: Structural equation models with unobservable variables and
measurement error. J. Market. Res. 18(1), 39–50 (1981)
16. Hair, J.F., Hult, G.T.M., Ringle, C.M., Sarstedt, M.: A Primer on Partial Least Squares
Structural Equation Modeling. Sage, Thousand Oaks (2014)
17. Shuhaiber, A., Mashal, I.: Understanding users’ acceptance of smart homes. Technol. Soc.
58, 101110 (2019)
UltraSurfaces: A New Material Design Vision
Marinella Ferrara1(&) and Chiara Pasetti2(&)
1
Department of Design, MADEC, Politecnico di Milano, via Durando 38A,
20156 Milan, Italy
[email protected]
2
AHO, Oslo School of Architecture and Design, Maridalsveien 29,
0175 Oslo, Norway
[email protected]
Abstract. We introduce UltraSurfaces (USs), a new material design vision
taking advantage of the latest developments in Electronic and Smart Materials,
and their integration in layered substrates and other materials. The USs can
detect phenomena in the environment, record and exchange data while being
controlled by remote technologies, interact with users by changing their physical
properties, like color, form, sound or light emission, and providing feedback to
the user. The USs go a step further than ubiquitous IoT, smart products and
environments, enabling new potentialities for any surface to be applied in Future
Scenarios of spaces and products. Adopting a Human and User-Centered
Design approach, we have been envisioned two Future Scenarios and concepts
to convey a better understanding of USs, as well as their potentiality for
applications and user’s appreciation. We aim to catch the interest of material
industries and highlight the challenges design is facing in exploiting the new
hybrid materiality of USs.
Keywords: UltraSurfaces New material design vision IoT hybrid
materiality Materiality Smart Aesthetics Experience Full body involvement
1 Introduction
Nowadays, the integration of digital technologies, resulting in a growing number of
effective ready-to-use devices (sensors, nano-processors, etc.), has created new
opportunities in several areas of manufacturing, in line with the emergent paradigm of
ubiquitous Internet of Things (IoT). Opportunities are widening day-by-day with the
development in computational sciences (including open-source software, and AI
platform development), as well in electronic materials (EMs)1 and manufacturing
processes. The last two mentioned are the key drivers of the actual diffusion of any
smart products, from communication devices to smart objects. EMs have been inte-
grated into several types of devices that are widely used in almost all sectors, including
manufacturing, robotics, process industries, energy systems, security, and healthcare
1
EMs are a combination of different materials. They include semiconductors, metals, polymers,
ceramics, and smart materials to build highly functional systems such as integrated electronic
circuits, communication devices, sensors, flat panel displays, lasers, batteries, fuel cells, etc.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 909–915, 2020.
https://doi.org/10.1007/978-3-030-39512-4_139
910 M. Ferrara and C. Pasetti
[1]. The latest advances of EMs include new methods of composite functional ink
formulations and methods of deposition onto substrates. Among the many processes
the most promising is the inkjet printing technologies [2], which allows manufacturing
microelectronics (both passive and active sensors, and circuits) on thin, flexible (like
PET or Polyimide), and also green substrate, like paper [3] and organic polymeric foil
[4], opening up a new era in terms of environmental sustainability of the electronic
production. Other opportunities come from 3D printing sensor technology, allowing
customized solutions, by either embedding a sensor into a printed structure or intrin-
sically printing the entire sensor in steps that can be conducted seamlessly [5].
From a business point of view, the advantage of these technologies lies in the low
investments to start manufacturing. The low cost of inkjet and its versatility makes it
possible to foresee a wide diffusion of flexible electronics with an unpreceded indus-
trial, economic, and social impact. However, since it is still a young technology, it
requires researches by companies willing to take the risk. Currently, the interest has
been expressed by big tech companies and electronics start-ups. Instead, in many other
manufacturing areas, especially products and materials for the interior and fashion
design, the electronically augmentation is increasing its strategic role, to compete in the
market with updated performances.
In terms of design implications, two are the significant challenges of these new
applications. Firstly, the flexibility of printed electronics, both in physical and con-
ceptual terms, will make possible the customization of product performance. Flexibility
allows the production of tailor-made sensors and circuits, the development of smart
systems, and AI for specific types of environments, yielding meaningful data and
opening a new range of possibilities for merging physical matter with digital func-
tionality. Secondly, the new materiality, including analogical and digital dimensions,
opens new questions in terms of aesthetics, perception of space and time, and philo-
sophical questions about why, how, and what. Printed and embedded microelectronics
are starting a process of hybridization, likely to be a winning formula for manufacturers
that have not yet wholly entered reality in ontological terms. The new hybrid spatiality
will be smart, interactive and connected and will have new space-time dimensions,
and effects on the concept of what it means to be human in a world of interconnected
things [6].
2 UltraSurfaces
Going in-depth in our research at MADEC2, we started exploring a new material
design vision: the UltraSurfaces (USs). Here, we present the USs vision, and the result
of our envisioning research activities to highlight the challenges design is facing in
taking advantage of their hybrid materiality. Moreover, we want to discuss some
implications designing USs, to frame future researches at the convergence of material
design, product design, and interaction design. We would achieve the double goal of
2
MADEC (Material Design Culture Research Center) is a knowledge center at Politecnico di Milano,
Design Department. Our approach is design-driven innovation in terms of a possible and desirable
future, with a Human and User-Centered-Design perspective.
UltraSurfaces: A New Material Design Vision 911
getting an adequate understanding of the US hybrid dimension, as well as promoting
the sustainable development on which the well-being of society depends.
USs are electronic augmented material surfaces, in which an embedded system of
sensors, actuators, smart materials, including communication devices, a microproces-
sor, or edge AI are firmly integrated (printed or embedded) into a unique solution. USs
hybrid materiality enables a seamless fusion of physical matter and digital performance
through computational augmentation. Surfaces acquire, therefore, intelligence and
extraordinary interactive qualities, as well as dynamic behavior, expressiveness, and
personality. USs can detect phenomena, sense or foresee, record and exchange data
while being controlled by remote technologies, interact with users, by changing their
physical properties, like color, form, sound or light emission, and providing feedback
to the user, responding appropriately in several situations and cooperating in human
activities autonomously. Their potential in enriching the user experience with new
meanings and ways of interaction is consistent.
USs are perceived as almost-living entities [7]. They can work as intelligent col-
laborative partners, making the interaction human-machine intuitive, understandable,
and pleasurable for regular users. The perception of its materiality can involve the full-
body, enhancing human perception in a meaningful way.
USs goes a step further than ubiquitous IoT, smart products and environments,
enabling new potentialities for surfaces to be applied in Future Spaces and Products
Scenarios.
In such a vision, collaborating with different but complementary partner-enterprises
is fundamental to achieve the best results, through an open innovation approach. We
envisioned two future interior design scenarios and developed one design concept for
each scenario. Two different laminated composite companies and one electronic sys-
tems company collaborated during the feasibility analysis. We present these ideas to
help the understanding of the USs vision and how they can be applied in different
contexts to achieve meaningful Smart User Experiences.
2.1 US Scenario of Conscious Home and ABO Concept
Conscious Home deals with future house trends as the place where human awareness
and wellbeing are integrated tasks. Wall, ceiling, or furniture surfaces can monitor
habits and suggest ways to improve health conditions. They can sense environmental
parameters taking care of all the air/light/temperature. They can change shape, color,
and pattern to communicate with the users and enhance their awareness about healthy
lifestyles. Since the context is domestic, the surface can have multiple functions
depending on the activities enacted in each room of the house. In the kitchen, it could
promote a healthy diet and consciously calibrated daily intakes; in a bedroom, it could
offer suggestions about falling asleep naturally or provide data about sleep quality.
In this scenario, US technological aspects merge with expressive/decorative ones.
Indeed, houses are also the place where individuals express their personalities, so
surfaces should also include an expressive character that gives uniqueness to space.
Moreover, they can take advantage of the integrated electronics to create customized
multi-sensorial atmospheres for convivial moments producing a sequence of light
effects, sounds, and releasing of aromatic essences.
912 M. Ferrara and C. Pasetti
From this basic scenario, we develop he first design concept. ABO, a smart system
of patterned panels for interior spaces, to deal with stress and anxiety in everyday life.
The ABO system takes advantage of electronic augmentation of multi-layered boards
acting as sensors and actuators to monitor the stress level through biofeedback, which
actively encourages people to take care of themselves easily and automatically. Thanks
to printed layers of electroconductive and electroluminescent inks, the surface acts
automatically as stress-reliever with kinetics luminous effect activated according to the
stress level detected by the remote control, exchanging data with the main panel. When
turned off, the panel acquires decorative functionalities, but when triggered by a
slightly accelerated heart pulse rate, it activates an EMDR (Eye Movement Desensi-
tization and Reprocessing) therapy session to relieve the user psychological stress
symptoms. The EMDR therapy, consisting of specific eye movement exercises, is
carried out by ABO thanks to a moving light that catches the eyes of the user, who is
led to perform the sequences shown by the patterns on the panel. ABO is designed in
all its aspects to help the user in reaching a state of complete relaxation. Other ABO
characteristics, like the river pebble-like shape of the remote controller recalling natural
and harmonious atmospheres, working in synergy to convey a sense of peace and
equilibrium. Also, the color contributes to minimizing the tension. The black 3.0, by
the artist Stuart Semple, absorbs 99% of the radiation, avoiding ocular strain. The
pattern is a series of intersected circumferences, creating a triquetra, symbol that has
been associated from the earliest times to the elemental equilibrium and perfection
(Fig. 1).
Fig. 1. Scheme of ABO US pattern, by Marco Goi (MADEC Research). The light is moving
along the surface for EMDR therapy. Courtesy of the author & MADEC.
2.2 Us Scenario of Play to Engage and Touch4Care Concept
Play to engage comes from the exploration of the playful dimension to engage users in
a more in-depth multisensory experience. The body shapes and harmonic and empathic
conversation, directly interacting with the surfaces limiting the space. The context of
the application is the healthcare system for children, where the interaction has great
potential. This scenario focuses on how to stimulate young patients to perform an
exercise that might be boring or repetitive. It envisions how to stimulate curiosity
UltraSurfaces: A New Material Design Vision 913
deeper to catch the patient in an immersive experience that energizes through touch and
colorful visual texture, lights, and sounds. So, USs can distract the patient, avoiding
anxiety, and creating the perfect conditions for the therapy to be carried out.
In the frame of this scenario, we developed Touch4Care, a system of electronically
augmented surfaces to interact with children during physiotherapy in outpatient envi-
ronments dedicated to the care of osteoarticular diseases, which are affecting the 10%
of the worldwide population. US composes a tridimensional space, including walls and
floor, by the installation of multiple connected panels that results in a smart children
therapeutic area. A sequence of colored lights is displayed on colorful and abstract
shapes patterned surface. It suggests a tactile interaction intuitively with the surface so
to guide the children in performing the exercises. The children should only follow the
movements of the light and play with it to carry out the session (Fig. 2), then enthu-
siastic sound feedback happens at the correct exercises’ accomplishment. After an in-
deep investigation of physiotherapy techniques for osteoarticular diseases, we included
in the concept a dedicated app, so that the physiotherapist can select a sequence of
exercises, and a tool-car as a facilitator (Fig. 2). The exercise complexity and speed are
fundamental parameters to set by the app according to the patient needs and challenges.
The small wooden car is a toy that makes the detection of the activities simpler. The
children use it to follow paths on the panel.
Fig. 2. Touch4Care is a kit for physiotherapy, by Marta Borsetti (MADEC research), to guide
the user in different exercises. On the left side the child, while standing or sitting, must touch the
surface that illuminates in different spots according to a random pattern. On the right, instead, the
child must follow a light path, pressing on the panel with the car. Courtesy of the author &
MADEC.
914 M. Ferrara and C. Pasetti
3 Conclusions
The US vision benefits from new electronic materials, smart materials, AI, and printing
processes and opens substantial innovation opportunities in times when smart behav-
iors, interactivity, and connectivity are intriguing concepts to raise user appreciation
and, consequently, the value of traditional materials and products.
US vision makes technology a user-friendly language, using soft elements like
lights, sounds, smells, vibrations, and other kinetics changing in time, that the user
interprets like behaviors. Our design activity focused on “behaviors” that support
meaningful relations between users and surfaces, since humans perceive smart prod-
ucts, as almost-living entities. This notion, defined in previous studies [7], helped us in
designing USs to generate Smart Aesthetics Experiences in terms of generating a sort of
empathic user-surface bond enriching ordinary life, empowering people in situations
where they are unable to react, or they don’t know how to do it. A US can create the
conditions for users to relax in stressful situations, encourage movement, detect what
happens around to transform it in a pleasurable experience, and attract people’s full
attention to make them feel at ease during medical therapy, for example, using play as a
solution.
We highlight these USs qualities, in terms of reflecting the interlinks between the
user and smart surfaces, translate these links in formal aspects (like metaphors of
human behaviors), materiality (hard and soft qualities) and their ever-changing nature
(increasing or decrease changing speedy or intensity, etc.). During the design process,
we focused on how users perceive (recognizing/comprehending/feeling) and judge a
meaningful experience with USs. Other aspects are about the task delegation according
to principles such as control possibilities, of information, and trust feel. All these
aspects are fundamental to make USs act as collaborative intelligent partners in human
life. The autonomous behavior, for example, is a critical quality, and the possibility to
control the USs without undermining the autonomy of behavior must be taken into
great consideration during the design process [8].
We learned the importance of shaping the design of USs according to (i) intuitive
collaboration with humans; (ii) expressiveness in line with human actions and
perception.
The design scenarios and concepts presented illustrate how USs transform the space
implementing new functions and changing the experiences on how we spend time in
living spaces. Spaces are transforming in ever-changing realities. The smartness of USs
can impact in terms of rituals, inspiring new gestures, habits, languages, activities, and
increase the environmental perception.
USs exploit the materials’ potential, creating a higher and deeper emotional
involvement of the user compared to a traditional surface, generating a multisensory
full-body involvement and engaging users in Smart Aesthetics Experience, featured by
communicational resonances, semantic perception, epithetical feeling.
Following the rapid development of IoT as well as edge and fog computing, in the
future, the interaction with the USs will happen in several products acting as a com-
plementary system/environment influencing, in a whole, our daily life.
UltraSurfaces: A New Material Design Vision 915
References
1. Haseeh, A.S.M.A.: Electronic Materials. Elsevier, Amsterdam (2016)
2. Guo, X., Xue, Z., Zhang, Y.: Manufacturing of 3D multifunctional microelectronic devices:
challenges and opportunities. NPG Asia Mat. 11, 29 (2019)
3. European Union (2018). https://cordis.europa.eu/project/rcn/214070/factsheet/it
4. Mandal, S., Dell’Erba, G., Luzio, A., Bucella, S.G., Perinot, A., Alberto Calloni, A., Berti, G.,
Bussetti, G., Duò, L., Facchetti, A., Noh, Y.-Y., Caironi, M.: Fully-printed, all-polymer,
bendable and highly transparent complementary logic circuits. Org. Electr. 20, 132–141
(2015)
5. Xu, Y., Wu, X., Guo, X., Kong, B., Zhang, M., Qian, X., Mi, S., Sun, W.: The boom in 3D-
printed sensor technology. Sensors 17(1166), 1–37 (2017)
6. Floridi. L.: La quarta rivoluzione. Come l’infosfera sta trasformando il mondo, Raffaello
Cortina Editore, Milano (2017)
7. Russo, A.C., Ferrara, M.: Smart Solutions, “Smart Aesthetics”? Des. J. 20(1), S342–S353
(2017)
8. Rozendaal, M.: Object with intents. Interactions 23(3), 62–65 (2016)
The Hybrid Dimension of Material Design:
Two Case Studies of a Do-It-Yourself
Approach for the Development of Interactive,
Connected, and Smart Materials
Stefano Parisi1(&), Markus Holzbach2, and Valentina Rognoli1
1
Design Department, Politecnico di Milano, Milan, Italy
{stefano.parisi,valentina.rognoli}@polimi.it
2
Design Department, Offenbach University of Art and Design,
Offenbach, Germany
[email protected]
Abstract. The paper proposes a definition of Material design as a hybrid
practice and describes the intersection area between two emerging phenomenon
regarding materials for design namely, ICS Materials and DIY-Materials – by
depicting the theoretical background supported by relevant examples and related
works and providing the case studies developed by the authors and their stu-
dents: L-ink, i.e. an experiment on DIY conductive Ink in order to augment
paper as a substrate material for interactive, iterative, and flexible material
interfaces; and Sound Textile, i.e. a sound installation using the material
properties of metal threads in a textile fabric. Finally, the paper discusses further
development, considering the ability of the case study to combine the properties
of two materials pushing the imagination to new horizons: making wearable
mobile phones that can be bent or open-ended and ephemeral interfaces that
could be customized and transformed at any stage of the product lifecycle.
Keywords: ICS materials DIY-Materials Material tinkering Material
design
1 Introduction: A Pro-active Materiality
In recent years, materiality is transforming, shifting from being standardized, situated
and static, to being more personal, diffused, and intelligent. On one side technological
development have led to miniaturization of technologies, such as sensor, actuators and
microprocessors that can be seamlessly embedded into matter, leading to the phe-
nomenon recognized by authors as Augmented Materials [1], Computational Com-
posites [2], Smart Materials Compositions [3], and more recently as the phenomenon
of Interactive, Connected, and Smart (ICS) Materials [4]. This materials or even hybrid
material systems or composites with sensitive, smart or gradually varying properties
lead to new and complex ways to design. This opens up new possibilities on the level
of concept, form, structure and surface, and paves the way to consider material
properties and qualities as dynamic and ever-changing [5]. Materials become the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 916–921, 2020.
https://doi.org/10.1007/978-3-030-39512-4_140
The Hybrid Dimension of Material Design 917
dialogical carriers of a wide variety of information - they become informative and
intuitive [6].
On the other side, the democratization of technologies has empowered designers
and makers in experimenting and developing their own materials, claiming their
autonomy from industry, diverging from the conventional materials production para-
digm, and fostering creativity and hybridization in materials practices. This phe-
nomenon is acknowledged as Do-It-Yourself (DIY) Materials [7]. DIY-Materials are
created through individual or collective self-production experiences, often by tech-
niques and processes of the designer’s own invention, as a result of a process of
experimentation and tinkering with materials. They can be new materials with creative
use of other substances as material ingredients, or they can be modified or further
developed versions of existing materials. The DIY-Materials taxonomy grounds mainly
on the materials’ provenance. Five classes, namely Kingdoms, of DIY-Materials have
been identified: Kingdom Vegetabile, Animale, Lapideum, Recuperavit, and Mutantis
[8]. While the first four kingdoms don’t require the use of embedded technologies and
don’t allow interactivity, connectivity, and smartness, the latter includes DIY-Materials
created from different technological mixes and hybridization of interactive or smart
sources. In this category, it is possible to find combinations of material sources from
other kingdoms that evolve into something particular with the aid of any technology.
Empowered by a DIY approach, digital technologies, and open-source, design
researchers and practitioners are increasingly operating in this new design space,
contributing with the development of prototypes, fostering discussion and formalizing
models and methodologies, and ultimately catalyzing innovation and change. This
progressive mindset leads to the development of interdisciplinary processes and holistic
approaches that consider materials, construction, design, interface as one. This is
happening in many disciplines in our digitized world [6]. Digitalization itself arises as a
critical factor for the aggregation of such diverse processes. The boundaries of design,
architecture, art, or technology are often no longer recognizable. This is intended to
stimulate reflection on the role of design [5]. Materiality intrinsically influences today’s
design concepts. It occurs in an intersection of digital and physical that involves the use
of digital design and manufacturing tools. The way these factors are connected also
impacts the way the design concept is conceived.
2 Material Design as a Hybrid Practice
In this space, artifact designers turn into material designers. Material design emerges as
an experimental and interdisciplinary dialogue involving the analog and the digital.
Materialization is an alternative way to traditional visualization to grasp the designer’s
ideas and concepts [5], to explore the concept’s sensorial qualities, performances,
implied experiences to unlock potentials and limits [6]. In other words, what is possible
and what is not. This kind of exploration is possible through Material Tinkering [9, 10].
Tinkering is a term borrowed from Human-Computer Interaction (HCI) that points to
hacking and manipulating physical and digital materials in a naive, playful and creative
way. The hands-on, playful, and not goal-oriented exploration can lead to unexpected
918 S. Parisi et al.
results and new functional and formal impulses. The only aim is to work on matter with
a high degree of experimental freedom and to generate results.
In material design, not only functional but also aesthetic qualities have central
importance [11, 12]. Therefore, in addition to the material itself and the manufacturing
process and construction, “soft” qualities such as emotions or sensitive properties are
also important. What do the texture, state of aggregation, elasticity and viscosity of a
material tell us? [5]. Today, generative methods can be integrated into the design
process, and digital design and manufacturing tools can be used in continuous process
chains, due to suitable scaling. Nevertheless, there is still the physical handling of the
material, the constant questioning of the concept as well as the comparison and
interaction between the digital and the analog.
The classic material categories seem to be increasingly dissolving. Sabine Kraft
writes: “The relationship between form and material has become as diverse as it is
ambiguous. A recourse to clear rules and specifications as to what can be conceived
and constructed in which material and how, and what aesthetic message would be
conveyed with it, is hardly possible anymore - if they ever existed” [13]. This leads to
the question whether the discussion about constructions that are suitable for the
material or design which is suitable for the material is outdated or not. Today, materials
are developed for their task. The question of materiality arises as a central decision and
development at the beginning of a design task and not as a selection from existing
materials at the end of the project. The “material-oriented design” is increasingly
turning into “design with designed materials” [6].
In material design, application and scale are not key issues. On the contrary, their
emergence is postponed to further stages of the project, or even avoided, in order to not
limit the exploration of the material into fixed constraints that would drastically
diminish the explorative power of this active and transforming dialogue between
human, material, and our environment. In contrast to conventional product design, in
material design the processes and also the fundamentals of materials, structures, and
systems very often play a decisive role. High-tech and low-tech are often combined or
nonlinear scenarios of action implemented [5]. For example, it is possible to use low-
tech processes to transform and combine advanced materials and components, and the
opposite. Traditional production techniques are combined with new digital methods.
The knowledge that seemed lost is transferred into a new context with new contem-
porary, digital tools and is recently informed through that. To this extent, the design is
defined both as part of industrial value creation and as an important cultural mission.
Examples of low-tech experimentations mixing physical and digital components
can be found in literature, e.g. the use realization of handcrafted interactive accessories
using leather [14], the research on Pulp Based Computing [15], and experiments on
prototyping DIY electronics with mycelium [16].
3 Two Case Studies: L-Ink and Sound Textile
In their respective institutions – Politecnico di Milano and Offenbach University of Art
and Design –, the authors carry on experimentation and research in this field by using
distinctive approaches and methodologies resulted from their individual body of work
The Hybrid Dimension of Material Design 919
but overlapping at many extents (Fig. 1). Two case studies developed by the authors
and their students are her presented as demonstrators of a low-level approach to
electronics-related materials. Respectively, L-ink has been developed by the DIY-
Materials Group at the Politecnico di Milano, Design Department, while Sound Textile
has been developed at the Institute for Materialdesign at Offenbach University of Art
and Design.
Fig. 1. On the left: L-Ink, by Gaia Bianco, Master’s Graduation thesis, Politecnico di Milano,
Design Department, supervised by Valentina Rognoli. On the right: Sound Textile, By Lilian
Dedio, Thesis, Diploma IMD Institute for Materialdesign, supervised by Markus Holzbach.
L-ink. L-ink is an experiment on DIY conductive ink in order to augment paper as a
substrate material for interactive, iterative, and flexible material interfaces (Fig. 1, on
the left). Endorsing the trend of augmented and smart paper, the paper was used as a
substrate material due to its modest cost, environmental characteristics, sensorial
qualities, such as porosity and flexibility. Paper is a perfect candidate for iterative
experimentations and quick-and-dirty interventions that allowed fast testing, evalua-
tion, and updates. Regarding the conductive ink, the raw material chosen and used for
the experiments is graphite powder, a conductive material by definition, which has
been adhered to the paper support thanks to the addition of a binder. Material Tinkering
was used as a strategy to learn by doing and discover unexpected opportunities of
matter. During material tinkering, several samples were made with different possible
binders (glues and paints). Then it was verified which sample conducted most in order
to leave out the others. Finally, on the selected sample further tests were carried out to
understand its properties and limits, and subsequently a simple electrical circuit was
realized to understand if the obtain ink sample was conductive enough to illuminate a
mini LED. After tinkering, a concept of educational and functional DIY kit for children
and adults was made, with the aim of learning the basics of electronics using included
materials; understanding the potential of the interaction between the paper and con-
ductive ink; doing simple electronic reparation; experimenting freely and try a DIY
experience.
Sound Textile. The sound installation Sound Textile uses the material properties of
metal threads in a textile fabric (Fig. 1, on the right). Metal is conductive and can have
ferromagnetic properties. Here a single metal thread plays the role of initiator and
mover, translating information into movement. The textile, which is light, flexible, and
920 S. Parisi et al.
supple, serves as a membrane that is being moved by the metal wire. In this combi-
nation, the two materials interact with and react to each other, opening up entirely new
functional capacities. The magnetic field, which produces an analogous audio signal in
the textile shot through with metal wire, is rhythmically repelled by a second magnetic
field in its vicinity. The textile transmits these oscillations to the air, and a sound is
produced. The combination of materials generates a functional and aesthetic sound
product [17].
4 Conclusions
The case studies are in-between the design of an object and the development of a
material. Technology, materiality, and design are overlapping in the creation of the
material provoking and suggesting unique possible scenarios of future practices and
applications. The hybrid material systems developed in the case studies have the
potentials to combine the properties of two materials pushing the imagination to new
horizons. For example, the design of a wearable mobile phones that can be bent and
distorted or open-ended and ephemeral interfaces that could be customized and
transformed at any stage of the product lifecycle. A radical transformation in the
development of such materials is represented by the increasing miniaturization of smart
components such as sensors, microcontrollers and actuators, and by the use functional
materials to sense and translate data, or even to connect technology. This is the case of
thermochromic or conductive inks. The development of components and nanomaterials
that can support the requirements of flexibility is a prominent factor. In particular, this
regards the issue of power supplying that might be possible with the integration of
flexible batteries, and power harvesting that can be addressed with the use of piezo-
electric components and flexible solar panels.
In a future perspective, the possibility to substitute traditional electronic compo-
nents, even with bio-based or bio-inspired ones, is not remote. In the first German
edition of Domus, Mateo Kries, director of the Vitra Design Museum, writes:
“Museums today must treat design as what it is - as a cross-sectional discipline between
art, science, and technology” [18]. Nature, with its structures and constructive laws,
often serves as a valuable source of inspiration for material design as well as new
technologies. The integration of material parameters into digital models, which is now
frequently used in the design disciplines, originates from the scientific and technically
motivated disciplines. In dealing with materials and systems, knowledge of geometry,
mechanics or mathematics is often helpful. Systems that are physically, chemically or
biologically inspired are increasingly being developed [5].
Many of today’s approaches follow sustainability and focus on intelligence that is
not always inscribed in the material itself, but in the way it is joined, constructively
formed, and applied. Through the micro-scaling of informing components, Material
Design opens up the field of design to new possibilities and tasks. Many of today’s
material solutions integrate new technologies and materials in scaling and complexity
that no longer allows conclusions about the performance or function of the existing
material solutions. This is also the ambivalence of the material solutions that are
presented. This articulation of new material solutions and hybridizations of design is
The Hybrid Dimension of Material Design 921
the crucial cultural mission of material design. The coded and rewritten can no longer
be decoded as a sign function or described with questions about “material authenticity”.
Acknowledgments. We thank the students Gaia Bianco and Lilian Dedio, and Camilo Ayala
Garcia who co-supervised the L-ink project. Stefano Parisi’s research has been funded by
Fondazione F.lli Confalonieri.
References
1. Razzaque, M.A., Dobson, S., Delaney, K.: Augmented materials: spatially embodied sensor
networks. Int. J. Commun. Netw. Distrib. Syst. 11(4), 453–477 (2013)
2. Vallgårda, A.: Computational Composites: Understanding the Materiality of Computational
Technology. Ph.D. Dissertation, The IT University of Copenhagen, Denmark (2009)
3. Barati, B., Karana, E., Hekkert, P.: Prototyping materials experience: towards a shared
understanding of underdeveloped smart material composites. Int. J. Des. 13(2), 21–38 (2019)
4. Parisi, S., Spallazzo, D., Ferraro, V., Ferrara, M., Ceconello, M.A., Ayala-Garcia, C.,
Rognoli V.: Mapping ICS materials: interactive, connected, and smart materials. In: IHSI,
Advances in Intelligent Systems and Computing, vol. 722 (2018)
5. Holzbach, M., Kracke, B., Bertsch, G.K.: Material Grove – von traditionellen Materialien zu
zukunftsorientierten Materialentwicklungen/From traditional materials to future-oriented
material developments). HfG Offenbach (2014)
6. Holzbach, M.: Gestalten mit gestalteten Materialien/Design with designed Materials. In:
Bauwelt 2015, vol. 20, Frei nach Otto – Sieben Betrachtungen zu seinem Erbe.
Bauverlag BV GmbH – Berlin (2015)
7. Ayala-Garcia, C.: The materials generation. The emerging experience of DIY-Materials. Ph.
D. Dissertation, Politecnico di Milano, Ph.D. program in Design, Italy (2019)
8. Ayala-Garcia, C., Rognoli, V., Karana, E.: Five kingdoms of DIY materials for design. In:
Proceedings of EKSIG 2017 (2017)
9. Parisi, S., Rognoli, V.: Tinkering with mycelium. a case study. In: Proceedings of EKSIG
2017 (2017)
10. Parisi, S., Rognoli, V., Sonneveld, M.H.: Material tinkering. an inspirational approach for
experiential learning and envisioning in product design education. Des. J. 20(sup1), S1167–
S1184 (2017)
11. Ashby, M., Johnson, K.: Materials and Design: The Art and Science of Materials Selection
in Product Design. Butterworth Heinemann, Oxford (2002)
12. Rognoli, V.: A broad survey on expressive-sensorial characterization of materials for design
education. METU J. Fac. Archit. 27(2), 287–300 (2010)
13. Kraft, S.: Werkstoffe. Eigenschaften als Variablen/materials. properties as variables. In: Arch
+ , vol. 172 (2004)
14. Fernaeus, Y., Murer, M., Tsaknaki, V., Solsona Belenguer, J.: Handcrafting electronic
accessories using ‘raw’ materials. In: Proceedings of TEI 2014 (2014)
15. Coelho, M., Hall, L., Berzowska, J., Maes, P.: Pulp-based computing: a framework for
building computers out of paper. In: Proceedings of CHI 2009 (2009)
16. Lazaro, E., Vega, K.: From plastics to biomaterials: prototyping DIY electronics with
mycelium. In: Proceedings of UBICOMP 2019 (2019)
17. Holzbach, M., Kellner, P., Gaines, J.: Material design: analog and digital design processes
exploring the world between material and structure. HfG Offenbach (2014)
18. Cf. Kries, M.: Domus, German Edition No. 1, May–June 2013, Berlin/Vienna (2013)
Human-Autonomy Teaming
Goal Directed Design of Rewards and Training
Features for Self-learning Agents in a Human-
Autonomy-Teaming Environment
Simon Schwerd(&), Sebastian Lindner, and Axel Schulte
Insititute of Flight Systems, Universität der Bundeswehr Munich,
85577 Neubiberg, Germany
{simon.schwerd,sebastian.lindner,
axel.schulte}@unibw.de
Abstract. For architects of reinforcement learning (RL) agents in real-world
applications, the design of a suitable training environment is challenging. Fea-
ture engineering and reward function design are mainly guided by human
intuition and domain knowledge. We propose the application of a goal directed
task analysis (GDTA) to structure knowledge about an application domain in
order to guide the design of a RL agent embedded in a complex environment.
Results from the task analysis can be leveraged for feature selection and reward
function design. We showcase this approach in a human-autonomy-teaming
application for military airborne operations.
Keywords: Reinforcement learning Training environment Task analysis
Reward design Feature engineering
1 Introduction
Human-Autonomy-Teaming (HAT) describes the cooperative relationship established
between humans and agents working towards a common goal. In this context, rein-
forcement learning (RL) is a promising approach to implement agents capable of
performing complex tasks where exact behavior is subject to training rather than
explicitly constructed. For integration into real-world scenarios, the requirements for
agents are manifold due to various concurrent goals and established human teaming
procedures. Therefore, the training environment for RL agents needs to integrate
profound domain knowledge which poses challenges for system designers [1]. In this
contribution, we suggest the application a task analysis method to support this design
process of RL training environment for a HAT environment. We demonstrate this
approach by means of a military airborne scenario.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 925–931, 2020.
https://doi.org/10.1007/978-3-030-39512-4_141
926 S. Schwerd et al.
2 Challenges in RL Environment Design
A RL agent learns a transition function, that maps input features to output actions by
optimizing a given reward function. For the creation of the agent’s learning environ-
ment, illustrated in Fig. 1, one must select observable features as training input and
specify a reward function to drive the desired behavior of the agent. These design
decisions are highly relevant to efficient learning of a desired agent behavior. In
confined applications like board games, the choice of an environment may be obvious
due to a discrete game state and action representation. But as observed by [1], the
transition of RL to more complex environments entails safety problems such as neg-
ative side effects due to ill-defined training environments.
Fig. 1. Reinforcement learning system structure
In the following, we recapitulate the difficulties of environment design for RL
agents in real-world applications:
Feature Engineering. The process of selecting and transforming features from a given
observation space is often referred to as feature engineering (FE). In this process, a
system designer tries to identify information representations that are suitable for the
training process of an RL agent. Commonly, this is a time-consuming task including
trial-and-error to find the best representation of training features. Selecting redundant or
irrelevant features makes it difficult to learn the correct relationship in the training data.
As stated in [2], the success of a machine learning projects depends largely of this step
of system design. Although there are efforts to automate the process of selecting and
transforming features [3], domain knowledge and human intuition about the problem
remain a central factor in FE.
Reward Function Design. The designer of RL agents must condense the intended
behavior into a scalar evaluation of the current state of the world, referred to as reward
function. In complex task environments, designers often end up with a proxy reward
function, that can easily be measured but may not completely capture the designer’s
intent. A misspecified reward function leads to negative side effects and poor behavior of
the agent in unintended ways [1]. To overcome the problems of reward design, the authors
of [4] implemented a RL algorithm, that estimates a reward function by observing an
Goal Directed Design of Rewards and Training Features 927
expert. This is an interesting approach to avoid explicit modeling, but assumes, that the
estimated reward is expressible as a linear combination of all input features which, again,
depends on how features are selected from the observation space.
To sum up, environment design is a process largely dependent on domain
knowledge and intuition. In the following, we want to demonstrate how a human task
analysis supports this process in applications, where an agent takes over the task of a
former human operator.
3 Human Task Analysis to Guide RL Environment Design
How does a human operator perform his tasks in his work environment? Human
factors researchers are traditionally concerned with this question. A set of task analysis
methods support the study of cognitive processes for control and decision-making in
human-machine-systems. From these analyses, one can identify information require-
ments, system boundaries and constraints as well as tasks and goals for a human
operator. This knowledge is conventionally used to design interfaces [5] or human-
machine systems [6]. We argue, that this process can be transferred to the creation of
training environments for intelligent agents. Analogous to a human, an agent requires a
suitable representation of the world and knowledge about the goals to pursue - espe-
cially in applications where tasks formerly performed by humans are taken over by
agents. If a self-learning agent performs and collaborates in a human team, its decision-
making must work towards similar goals incorporating related observations about the
world. The application of a task analysis can provide valuable insight into the domain,
since it reveals relevant goals, tasks and information within a work environment.
A related approach has been presented by [7], where a Cognitive Work Analysis
(CWA) has been conducted for the operation of a mechanised company commander.
Here, the goal was to identify adequate behavior for rule-based agents performing
within a simulated environment. Although their motivation has been similar in nature
as our concept - namely guiding designers towards the structure and function of a
domain - a CWA is not directly applicable to the design of a RL agent, since the agent’s
behavior is subject to learning and therefore not implemented directly. As a more
suitable method, we suggest a Goal Directed Task Analysis (GDTA) to derive the
reward function and observation space for a RL training environment. The GDTA
organizes two aspects of the work environment in a technology-agnostic way: (1) the
operator’s goals and (2) all information relevant to make decisions during the opera-
tion. These two properties can be mapped to the design of a training environment as
described in the following section.
3.1 Goal Directed Task Analysis
Originally proposed in the context of display design [5], the GDTA identifies the
human operator’s goals, decisions and information requirements. It is based on non-
structured interviews with subject domain experts and determines a tree-like goal
structure independent of technology and operator manual activity. Figure 2 shows an
exemplary structure of a GDTA. At the top level, there are general goals that drive the
928 S. Schwerd et al.
operator. Each goal at the high level is subsequently broken down into sub-goals until
they reach a point, where the operator must take a decision to pursue his objective. At
this point, all dynamic information relevant to this decision is linked to the branch of
the GDTA.
Fig. 2. GDTA structure
This analysis of domain knowledge can be leveraged in the environment design of
the agent as follows:
• Goals and their sub-goals are good indications for reward function components.
While high-level goals commonly refer to abstract overall objectives, sub-goals
break them down into tangible elements more suitable for a reward function. As a
starting point, we suggest a weighted linear combination of reward functions ri
related to the sub-goals at lowest level:
X
RðsÞ ¼ w r ðsÞ
i i i
ð1Þ
with the reward ri being a (non-linear) function depending on the state vector s and
representing a reward related to the sub-goal i:
• Decisions are evaluated by the human operator to determine the course of action to
meet a goal. From these decisions, operators derive information requirements about
the state of the system and environment.
• The information requirements are examples of features that are important in the
context of one decision. Thus, a system designer should consider these features to
also be relevant to the agent’s decision-making and therefore be integrated into the
its observation space. Taking this into account, features appearing as information
requirements are candidates for the environment’s training features as follows:
s ¼ x1 ; . . .; xn ; y1 ; . . .; yn ; z1; . . .; zn ð2Þ
Goal Directed Design of Rewards and Training Features 929
4 Human-Autonomy Teaming Application
For demonstration purposes, we carried out the proposed method by means of a sce-
nario from our Manned-Unmanned Teaming (MUM-T) research lab. Our application is
a manned fighter jet guiding several unmanned fast jets from aboard his cockpit in a
future operating environment [8]. In such scenarios, teams of manned and unmanned
airborne platforms work together to pursue a common mission goal. Our exemplary
scenario is the suppression of enemy air defense (SEAD) executed by two jets part of a
larger operation. The two jets are controlled by an RL agent and a human pilot,
respectively.
First, we conducted a GDTA with two fighter pilots from the German Airforce.
Figure 3 shows a selected subset of goals from the GDTA. For the sake of demon-
stration, we illustrated goals related to our example SEAD operation. Note, that the
goals can be contradictory, e.g. one should avoid threats in general, but in order to
neutralize an enemy air defense, the platform must eventually approach it to deploy
measures of suppression. From this point, a system designer can derive the components
of a reward function capturing similar goals as a pilot and therefore similar decision-
making. The exact computation of the sub-goal function components is still up to the
designer, but a linear combination of these components provides a starting point for the
multi-goal environment.
Fig. 3. Subset GDTA for a SEAD mission
Figure 4 illustrates one bottom level of the sub-goal “Assess visibility”. Here, the
analysis summarizes all relevant information for the decision how to avoid enemy radar
tracking. The designer of the RL environment should therefore consider these infor-
mation requirements as good features for the decision making of the agent.
930 S. Schwerd et al.
Fig. 4. Sub-goal, decision and information requirement for the sub-goal 2.2
Human-Autonomy Teaming Considerations. We noticed further benefits of this
approach within the context of context of human-autonomy teaming environments.
Both agent and human operator can mutually benefit from a GDTA-based agent design
as follows:
• Agent Transparency: If the GDTA correctly reflects human goals and decision-
making, considering similar goals and decisions within the agents should lead to a
more transparent behavior. This is beneficial for trust calibration and team
performance.
• Human Policy: To implement cooperation within an RL agent, state-of-the-art
methods require an approximation of the teammate’s policy [9]. While a formal
description of human policy is not possible, a task analysis like the GDTA could
function as an approximation.
5 Conclusion
We demonstrated, that a task analysis method from the SA framework can support the
design of a RL training environment. The mapping of goals and information require-
ments to the reward function and observation space, respectively, helps a system
designer to incorporate domain knowledge into the agent’s design. Further benefits are
possible, such as more transparency and better collaboration between agents and
humans. The utility of this approach needs to be further explored in real-world
applications.
References
1. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete
Problems in AI Safety (2016)
2. Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55, 78–
87 (2012)
3. Khurana, U., Samulowitz, H., Turaga, D.: Feature engineering for predictive modeling using
reinforcement learning. In: 32nd AAAI Conference on Artificial Intelligence, pp. 3407–3414
(2018)
Goal Directed Design of Rewards and Training Features 931
4. Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In:
Proceedings, Twenty-First International Conference on Machine Learning, pp. 1–8 (2004)
5. Endsley, M., Bolte, B., Jones, D.: Designing for Situation Awareness. CRC Press, Boca Raton
(2016)
6. Schulte, A., Donath, D., Lange, D.S.: Design patterns for human-cognitive agent teaming. In:
International Conference on Engineering Psychology and Cognitive Ergonomics, pp. 231–
243 (2016)
7. Connell, R., Lui, F., Jarvis, D., Watson, M.: The mapping of courses of action derived from
cognitive work analysis to agent behaviours. In: Proceedings of Second International Joint
Conference on Autonomous Agents and Multi Agent Systems, pp. 2–7 (2003)
8. Gangl, S., Lettl, B., Schulte, A.: Single-seat cockpit-based management of multiple UCAVs
using on-board cognitive agents for coordination in manned-unmanned fighter missions. In:
Conference on Engineering Psychology & Cognitive Ergonomics, pp. 115–124 (2013)
9. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for
mixed cooperative-competitive environments. In: Advances in Neural Information Processing
Systems, vol. 30 (2017)
Facial Expressions as Indicator for Discomfort
in Automated Driving
Matthias Beggiato(&), Nadine Rauh, and Josef Krems
Cognitive and Engineering Psychology, Chemnitz University of Technology,
Wilhelm-Raabe-Str. 43, 09120 Chemnitz, Germany
{matthias.beggiato,nadine.rauh,josef.krems}
@psychologie.tu-chemnitz.de
Abstract. Driving comfort is considered a key factor for broad public accep-
tance of automated driving. Based on continuous driver/passenger monitoring,
potential discomfort could be avoided by adapting automation features such as
the driving style. The EU-project MEDIATOR (mediatorproject.eu) aims at
developing a mediating system in automated vehicles by constantly evaluating
the performance of driver and automation. As facial expressions could be an
indicator of discomfort, a driving simulator study has been carried out to
investigate this relationship. A total of 41 participants experienced three
potentially uncomfortable automated approach situations to a truck driving
ahead. The face video of four cameras was analyzed with the Visage facial
feature detection and face analysis software, extracting 23 Action Units (AUs).
Situation-specific effects showed that the eyes were kept open and eye blinks
were reduced (AU43). Inner brows (AU1) as well as upper lids (AU5) raised,
indicating surprise. Lips were pressed (AU24) and stretched (AU20) as sign for
tension. Overall, facial expression analysis could contribute to detect discomfort
in automated driving.
Keywords: Face tracking Facial expressions Action units Automated
driving Discomfort Driving simulator Mediator project
1 Introduction
Comfort is considered as one of the main drivers for higher levels of automated driving
next to safety, efficiency, social inclusion and accessibility [1]. Although there is no
common definition of comfort, it can be considered a feeling of well-being and an
attribution of positive valence, associated with the absence of discomfort and uneasi-
ness [2]. Next to traditional comfort aspects such as vibrations, noise or sitting comfort,
new and additional determinants are discussed in automated driving such as apparent
safety, trust in the system, feeling of control, motion sickness, familiarity of driving
maneuvers as well as information about system states and actions [3, 4]. As these new
comfort aspects are mainly related to specific and dynamic situations, constant eval-
uation is required. Based on this continuous driver/passenger comfort evaluation,
potential discomfort could be avoided by adapting automation features. The basic idea
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 932–937, 2020.
https://doi.org/10.1007/978-3-030-39512-4_142
Facial Expressions as Indicator for Discomfort in Automated Driving 933
of such comfort-adaptive automation is the metaphor of a vehicle-driver-team that
knows each other’s strengths, limitations, and current states and can react accordingly.
The EU-project MEDIATOR (mediatorproject.eu) aims at developing a mediating
system in automated vehicles, pursuing a paradigm shift away from a view that pri-
oritizes either the driver or the automation, instead integrating the best of both. The
mediating system should intelligently assess the strengths and weaknesses of both the
driver and the automation and mediate between them, while simultaneously taking into
account the driving context. As part of the driver/passenger state, facial expressions
could be one indicator of potential discomfort during automated driving. If detected
accordingly, automation features such as driving style aspects (e.g. speed, lateral dis-
tance, distance to vehicle ahead) or information presentation could be adapted to
prevent uncomfortable situations, disengagement of automation or even dangerous and
not necessary take-over situations [5]. Facial expressions can be analyzed by tracking
of facial electromyographic activity, by manual coding of facial activity [6] as well as
nowadays by automatic facial expression analysis using computer-vision algorithms
[7]. Most of these techniques aim at identifying Action Units (AUs), which represents
movements of an individual face muscle or muscle group [6]. Even though AUs do not
interpret the meaning of expressions, relations of AU-pattern to basic emotions can be
drawn [6]. In order to investigate discomfort-related changes of facial expressions /
AUs, a driving simulator study has been carried out including standardized potentially
uncomfortable situations during an automated trip. A total of four video cameras
recorded the driver’s face from different perspectives. These videos were analyzed with
the Visage facial feature detection and face analysis software (Version 8.4, vis-
agetechnologies.com), extracting values for 23 AUs for each video frame.
2 Methods
Driving Simulator Study Design. The study was conducted in a fixed-base driving
simulator (Fig. 1A) with a 180° horizontal field of view and fully equipped interior.
A three minute trip was prerecorded by the investigators, containing three potentially
uncomfortable situations. This trip was replayed while the participants sat in the dri-
ver’s seat with no possibility to intervene by pedals or steering wheel. The situations
consisted of three identical automated approaches to a truck driving ahead (Fig. 1B).
The truck drove at a constant speed of 80 km/h, whereas the own car approached the
truck in automated mode with 100 km/h. At a rather short distance of 9 m, automated
braking was initialized, reaching a minimum distance of 4.2 m (minimum time to
contact of 1.1 s). A handset control was integrated into the driving simulator and the
participants could report perceived discomfort continuously during the whole trip [8].
Participants. A total of 41 participants (24 male, 17 female) were recruited for the
study, consisting of a younger age group under 40 years (N = 20, M = 28 years,
SD = 3.9) and an older group over 60 years (N = 21, M = 68 years, SD = 4.7). All
participants gave written informed consent in accordance with the regulations and
consent templates of the TU Chemnitz ethics commission.
934 M. Beggiato et al.
Fig. 1. Fixed-base driving simulator (A), approach situation to slower driving truck ahead (B),
screenshots face tracking from video cameras Intel RealSense (C), AVT Mako (D), GoPro right
side (E), GoPro left side (F). Written informed consent was obtained from the individual for the
publication of this image.
Video Cameras and Face Tracking. The face of the driver was recorded by four
video cameras to test face tracking with different cameras, different video formats and
different perspectives (Fig. 1C–F). An Intel RealSense SR300 camera was placed
exactly in front of the driver over the steering wheel (Fig. 1C), recoding a color video
with a resolution of 1280 720 pixel and 30 frames per second (fps). An AVT Mako
G-234B camera delivered a grayscale video of 640 480 pixel and 30 fps (Fig. 1D).
This camera was placed at the right side from the driver’s perspective over the steering
wheel. Two small GoPro Hero 5 video cameras were placed behind the steering wheel
under the instrument cluster at the right (Fig. 1E) and left side (Fig. 1F) from the
driver’s perspective. Both cameras recorded a color video of 1920 1080 pixel with
50 fps. All video recordings were analyzed using the Visage facial feature detection
and face analysis SDK (Version 8.4 for Windows, visagetechnologies.com). The SDK
was integrated into an in-house developed data logging application, providing log files
with 23 AU values as well as the tracking quality for each video frame.
3 Results
Discomfort Sequence Extraction. In order to assess AU-changes related to each of
the three approach situations, each time period of pressing the handset control (inde-
pendent of magnitude) was extracted as discomfort sequence. Having 41 participants
and 3 situations, a theoretical sum of 123 sequences could be present. However, in 25
approach situations participants did not press the handset control at all. Thus, 98
sequences could be extracted with a mean duration of 7.1 s (SD = 4.5 s). In addition,
10 s time intervals prior and after each discomfort sequence were extracted. As dis-
comfort intervals varied in duration, a common percent time axis was created from 0%
to 300% to show changes of each AU in one common scale (Fig. 2). Periods before and
Facial Expressions as Indicator for Discomfort in Automated Driving 935
after the discomfort interval were always 10 s long; thus, 1% represents 0.1 s. Each
discomfort interval was divided into percent slices, and the mean of each AU was
calculated over time of the respective percent slice. Further details on this method can
be found in [8].
Fig. 2. Mean z-standardized AU values from the Intel RealSense SR 300 face video showing
situation-related changes before, during and after reported discomfort intervals. The bold blue
line shows the mean z-values over all participants and the three situations, the light red area
shows the 95% pointwise confidence interval.
Raw Data Processing. Only video frames with an overall face tracking quality of 30%
or more entered the analysis. To correct for high frequency signal fluctuations, a
moving average over ± 2 s was calculated for each AU raw score. As raw AU-scores
resulting from the face tracker are arbitrary decimal numbers, standardization is
required to compare values between situations and participants. A common standard-
ization procedure is the z-score, which expresses raw values as distance to the mean in
units of standard deviations; with a total mean of zero and a standard deviation of one
[8]. This transformation was applied for each sequence, resulting in relative changes
within the 300% time period. Z-values were afterwards averaged over the 98 sequences
(bold blue line in Fig. 2) and the 95% confidence interval was calculated pointwise and
plotted as a light red area around the means. If the confidence band does not overlap
between two particular points in time, these two means differ in a statistically signif-
icant manner.
936 M. Beggiato et al.
Changes in AUs. Figure 2 displays significant situation-related changes, which could
be identified for 8 out of the 23 recorded AUs. To ensure stability and validity of
results, only consistent findings over all four video cameras are reported. All charts in
Fig. 2 belong to the Intel RealSense SR 300 camera video, however, the same trends
were observed for the other three video cameras.
During discomfort intervals, values of AU43 (eye closed) diminished significantly
for the left eye (Fig. 2A) as well as for the right eye (Fig. 2B), with a rather steep
increase towards the end and after the reported discomfort interval. The maximum was
reached at about 5 s (250%) after the discomfort interval. This indicates continuous
visual monitoring of the situation by keeping the eyes open and reducing eye blinks.
A slightly increasing trend of AU1 (inner brow raiser) could be observed for the left
(Fig. 2C) and right inner brow (Fig. 2D) until 5 s (250%) after the discomfort interval,
with a sudden drop afterwards. The upper lid of both eyes (AU5, Fig. 2E) was raised
during the discomfort interval, with a decreasing trend afterwards. A similar situation-
related increase with subsequent decrease could be observed for AU24 (lip presser,
Fig. 2F). The AU20 (lip stretcher) increased during the discomfort interval with a
decrease afterwards; however the effect is more pronounced for the left lip side
(Fig. 2G) than for the right side (Fig. 2H). No divergent or contradictory trends could
be found for age groups or gender.
4 Discussion and Conclusion
The present study aimed at investigating discomfort-related changes of facial expres-
sions using video-based automated analysis of facial AU. Facial expressions could be
used to adapt automation features such as driving style aspects (e.g. speed and distance)
to reduce discomfort and avoid non-necessary take-over situations [8].
In order to develop such a comfort-adaptive system, it is required to identify the
potential of facial expressions as discomfort indicator, including relevance of AUs as
well as magnitude, direction and timing of effects. Results showed that during dis-
comfort intervals, both eye were kept open (AU43) and eye blinks were reduced,
indicating continuous and attentive visual monitoring of the situation. This trend is in
line with previous results based on eye-tracking, which showed reduced eye blink rate
in this situation [8]. The raise of the inner brows (AU1) in association with the upper lid
raiser (AU5) is considered as essential part in all prototypes and major variants of the
emotion “surprise” [6]. The situation-related rises of lip pressing (AU24) as well as lip
stretching (AU20) could be interpreted as sign for tension during the close approach
situation. As these AU-trends could be observed in the face tracking results of all four
video cameras from different perspectives, the findings could be regarded as rather
stable for this situation. Additional analyses of individual and group differences showed
of course different absolute rates of change in the AU raw scores per person. However,
when standardizing these absolute values based on the individual trend (individual z-
standardization), no divergent or contradictory trends could be found for subgroups
such as age groups or gender.
To conclude, facial expressions captured by face tracking software showed specific
changes for this uncomfortable approach situation in automated driving. Thus, facial
Facial Expressions as Indicator for Discomfort in Automated Driving 937
expression analysis could contribute to adaptive automation, preventing human dis-
comfort in the driver-vehicle team. Although the findings provide valuable indications
of potentially relevant AUs as well as direction, magnitude and timing of effects, all
analyses were carried out on aggregate level over all participants. Reliable and valid
detection of discomfort on individual level requires further research on the stability and
validity of these findings as well as the combination with additional driver state
measures. Thus, further research steps in the MEDIATOR project are the validation of
the findings in other potentially uncomfortable situations as well as the combination of
face tracking data with additional driver state measures to finally develop and test a
real-time capable mediating system.
Acknowledgments. Data collection was funded by the Federal Ministry of Education and
Research under grant No. 16SV7690K (Project KomfoPilot). Data analysis of AUs was funded
by the European Union’s Horizon 2020 research and innovation programme under grant
agreement No. 814735 (Project MEDIATOR).
References
1. ERTRAC: Connected Automated Driving Roadmap. European Road Transport Research
Advisory Council, European Road Transport Research Advisory Council (2019). https://
www.ertrac.org/uploads/documentsearch/id57/ERTRAC-CAD-Roadmap-2019.pdf
2. Bellem, H., Thiel, B., Schrauf, M., Krems, J.F.: Comfort in automated driving: an analysis of
preferences for different automated driving styles and their dependence on personality traits.
Transp. Res. Part F: Traffic Psychol. Behav. 55, 90–100 (2018). https://doi.org/10.1016/j.trf.
2018.02.036
3. Elbanhawi, M., Simic, M., Jazar, R.: In the passenger seat: investigating ride comfort
measures in autonomous cars. IEEE Intell. Transport. Syst. Mag. 7(3), 4–17 (2015). https://
doi.org/10.1109/MITS.2015.2405571
4. Beggiato, M., Hartwich, F., Schleinitz, K., Krems, J.F., Othersen, I., Petermann-Stock, I.:
What would drivers like to know during automated driving? Information needs at different
levels of automation. 7. Tagung Fahrerassistenz, Munich, 25–26 November 2015 (2015).
https://doi.org/10.13140/rg.2.1.2462.6007
5. Techer, F., Ojeda, L., Barat, D., Marteau, J.-Y., Rampillon, F., Feron, S., Dogan, E.: Anger
and highly automated driving in urban areas: the role of time pressure. Transp. Res. Part F:
Traffic Psychol. Behav. 64, 353–360 (2019). https://doi.org/10.1016/j.trf.2019.05.016
6. Ekman, P., Hager, J.C., Friesen, W.V.: Facial action coding system. The manual and
investigator’s Guide. Research Nexus, Salt Lake City (2002)
7. Ko, B.: A brief review of facial emotion recognition based on visual information. Sensors 18
(2), 401 (2018). https://doi.org/10.3390/s18020401
8. Beggiato, M., Hartwich, F., Krems, J.: Using smartbands, pupillometry and body motion to
detect discomfort in automated driving. Front. Hum. Neurosci. 12, 3138 (2018). https://doi.
org/10.3389/fnhum.2018.00338
Can We Talk? – The Impact
of Conversational Interfaces on Human
Autonomy Teaming Perception, Performance
and Situation Awareness
Adam Bogg(&), Andrew Parkes, and Mike Bromfield
Coventry University, Coventry, UK
[email protected]
Abstract. Human Autonomy Teaming (HAT) research has established that
communication provides the key to building a teaming relationship between
humans and autonomous agents (synthetic teammates). We hypothesised that a
greater semblance of teaming could be achieved through more natural forms of
audio-voice communication. In this study we evaluated the impact that pro-
viding a conversational interface (a voice) to the synthetic teammate had on the
performance, SA and perception of teaming of the human teammate. Twenty-
four participants conducted experimental trials on a computer-based task
adapted from a LOA test method developed by Endsley and Kaber [8]. The
results show that voice communication from the synthetic teammate can have a
positive effect on human performance and perception of teaming; however, the
pattern of improvement varied between different team structures.
Keywords: Human Autonomy Teaming LOA Voice communication
1 Introduction
For the autonomous agent to integrate successfully with the human elements in Human
Autonomy Teaming (HAT) it needs to have similar teaming functionality to a human
such as the ability to communicate, coordinate and demonstrate awareness of the other
teammates [1]. The key function being communication.
Recent HAT research has tended to focus on improving communication between
human and autonomous agent via traditional visual Human Computer Interfaces. Chen
et al. [2], and Schaefer et al. [3] used graphical representation to provide data on the
location, state and decision considerations of a simulated autonomous agent. McNeese
et al. [4], conducted evaluations of the impact of text “chat” communication on HAT.
Thus, HAT research projects have shown that communication through visual modes,
either as graphical, diagrammatic interfaces or textual sentence windows are successful
at providing essential data to the human members of the team.
However, Wickens’ Multiple Resource Theory [5] suggests that expanding the
availability of communication through the addition of an audio-voice medium could
provide advantages in the form of cognitive timesharing by providing an additional
channel for information exchange. Furthermore, it would also endow the autonomous
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 938–944, 2020.
https://doi.org/10.1007/978-3-030-39512-4_143
Can We Talk? 939
agent with more human-like characteristics that would positively impact on human
perception of the autonomous agent and the experience of teaming.
This general hypothesis can be broken down into three individual hypotheses to be
tested:
Hypothesis 1: In a Human Autonomy Team, when the autonomous agent uses voice
communication to complement graphical communication, human operator task
performance will improve.
Hypothesis 2: In a Human Autonomy Team, the exchange of situation awareness
(SA) data by voice communication from the autonomous agent improves human
operator situation awareness.
Hypothesis 3: Voice communication from an autonomous agent improves the
human’s subjective trust and perception of teaming with the autonomous agent.
2 Form and Structure of Teaming
Researchers have identified that there is a relationship between the innate communi-
cation behaviour of a team and its efficacy [6, 7]. Earlier research into Levels Of
Automation (LOA) [8] demonstrated that the structure of the team and the division of
labour between teammates had a significant impact on the performance and SA of the
human. However, the HAT communication research [6, 7] was only conducted on one
type of team structure. Exploring how varying the team structure might also affect
human performance and SA was not conducted, likely because changing the team
structure would affect the type and use of intra-team communication, making it difficult
to identify the source cause of any observed variance in human performance or SA.
In this study we evaluate the impact of voice communication on four different team
structures commonly used for HAT research: human Manual control (human teammate
carries out all functions, synthetic teammate observes and provides safety advice),
autonomous agent Playbook (human delegates to synthetic teammate through pre-
defined Plays of a Playbook), autonomous agent Decision Support (synthetic teammate
takes analytical lead and human makes decisions for synthetic teammate action) and,
human Supervision (synthetic teammate carries out analysis, decision making and
action, human is commander with over-ride).
3 Research Aim
The aim of the research was to evaluate the impact of synthetic teammate audio voice
communication on the performance and behaviour of the human teammate. Voice
messages were compared to Text and a No information control, on Human Perfor-
mance, Situation Awareness (SA), and Perception of Teaming.
940 A. Bogg et al.
4 Method
An experimental methodology was adapted from an Endsley and Kaber [8] study into
LOA, that required a human participant to work with a synthetic teammate (au-
tonomous agent) on a computer-based task (Multitask).
This task is a schematic representation of a simple air traffic radar (Fig. 1). Par-
ticipants were presented with a continuous flow of coloured square targets that transited
from the outside of a radial display towards a central deadline.
Three sizes (small, medium, large) and colours of target were presented. Partici-
pants were required to clear the targets before they reached the deadline or collided
with each other by either clicking on a target or assigning the synthetic teammate to
clear it.
The size of the target indicated how much processing was required to clear the
target. Small targets were quicker to clear (3 s) than large targets (6 s).
Participants were given reward points for successfully clearing targets and were
given penalty points if a target impacted with the deadline or collided with another
target. Reward points were higher for small targets and lower for large targets. Penalty
points were higher for large targets and lower for small targets.
The participants were presented with three Goals, two of which were safety ori-
entated and a third that was personal gain orientated:
(1) Prevent targets impacting with the deadline;
(2) Prevent targets colliding with each other;
(3) Maximise the score achieved.
Fig. 1. Multitask team-mate display in decision support mode (developed from Endsley and
Kaber [8])
To complete the task the team had to (a) identify a sequence for clearing targets,
(b) select the current priority target, (c) process the target.
Can We Talk? 941
• In Manual Control all activities were conducted by the human.
• In Playbook, the human carried out (a) and (b) and the synthetic teammate then
completed (c).
• In Decision Support, the synthetic teammate generated options for (a), which the
human then selected, and then the synthetic teammate conducted (b) and (c).
• In Supervision, the synthetic teammate carried out all 3 tasks, with the human given
the option to over-ride and carry out (a).
Twenty-four participants aged between 22 and 54 (Mean 35.04, SD 10.243, 9
female, 15 male) undertook the experimental task. None had prior air traffic control
experience. Each participant conducted trials in one of the four team structures under
each of three interaction conditions (the synthetic teammate communicating through
either a voice synthesiser, a text messaging window, or with no voice communication).
Each trial lasted 10 min, in which participants were required to clear an average of 140
targets.
Episodic participant performance was sampled each minute. Measurements were
taken of the Success Rate (Targets Cleared, Score Achieved and amount of Processing
of Targets), the Selection Strategy (the cleared target’s rank of Reward, Penalty, Size
and Distance from deadline in relation to other displayed targets), and Safety Behaviour
(Proximity to Deadline and Proximity to other Targets when cleared).
SA was evaluated by use of SAGAT. Three interrupts were provided in each trial.
SAGAT questions were taken from the original Endsley and Kaber [8] study and
consisted of asking about target characteristics (size, colour, reward value, penalty
value) and target activity (distance, speed and likely impact).
Perception of Teaming was measured using a post-trial survey that asked partici-
pants to provide a subjective rating of five characteristics of teaming identified as
essential characteristics for synthetic teammates.
Prior to engagement in the study, participants were instructed on the task goals and
the expected activities of the synthetic teammate and, if appropriate, any strategies that
the synthetic teammate might use to achieve the goals. Participants were then provided
with five minutes of induction training, which included exposure to the SAGAT
interrupts and the subjective teaming perception survey. After that they were given
10 min of further practice.
5 Results
The samples of each performance factor rate for each condition (None, Text, Voice)
were collated and then analysed through SPSS for difference using an Within-Subjects
Repeated Measures ANOVA. Data sets were tested for Sphericity using Mauchly’s test
and if the Sphericity test failed a Greenhouse-Geisser correction was applied.
Hypothesis 1: The results support the Hypothesis for three of the four team structures.
Results with Significant Variance are listed below in Table 1 (non-significant results
have been removed for article brevity).
942 A. Bogg et al.
Table 1. Summary of significant ANOVA results – IV is communication format
Team format Measure F-statistic p-value
Manual control Target processing* F(1.466, 86.497) = 6.953 p = .004
Manual control Penalty rank F(2, 118) = 4.321 p = .015
Manual control Size rank F(2, 118) = 3.448 p = .035
Manual control Deadline proximity* F(1.334, 78,691) = 4.493 p = .027
Playbook Targets cleared F(2, 118) = 4.528 p = .013
Playbook Reward score F(2, 118) = 5.106 p = .008
Playbook Deadline proximity F(2, 118) = 6.338 p = .002
Playbook Other target proximity F(2, 118) = 4.232 p = .017
Decision support Reward rank F(2, 118) = 6.111 p = .003
Decision support Penalty rank F(2, 118) = 3.769 p = .026
Decision support Distance rank F(2, 118) = 3.670 p = .028
Decision support Size rank F(2, 118) = 3.779 p = .026
*Greenhouse-Geisser correction applied
In Manual Control significant variance was found for Target Processing, Penalty
Rank, Size Rank and Proximity to Deadline with the average for each of those mea-
sures greater in the Voice condition than in the Text or None conditions.
In Playbook significant variance was found for Targets Cleared, Reward Score,
Deadline Proximity and Target Proximity, again with the average for each of those
measures higher in Voice condition than the None condition. In Playbook, whilst not
significant in variance, the averages of Penalty Rank and Size Rank were also greater
for the Voice condition than the Text or None conditions. These results indicate that in
Manual Control and Playbook when the synthetic teammate communicated participants
changed behaviour by increasing workload and prioritising larger targets.
In Decision Support, significant variance was found in all Selection Strategy
measures (Reward, Penalty, Distance and Size), with the average scores of Reward and
Distance higher and Penalty and Size lower under the Voice condition. This indicates
that the participants changed their behaviour towards following the synthetic teammate
recommendation to prioritise smaller targets with greater reward values.
However, in Supervision there was no significant variance in any performance
measure for any condition, which indicates that irrespective of how the synthetic
teammate communicated the participants allowed it near full autonomy.
Hypothesis 2: There was no significant variance in SA for any team structure, thus the
hypothesis was not supported. However, of interest was that a surprisingly large
number (70%) of SA questions were simply left unanswered or answered with the
option “Don’t Know”. Furthermore, of the questions that were attempted, 73% were
incorrect. The average SA score was just 8% suggesting all participants, irrespective of
team structure or condition had extremely poor SA, however, performance and task
success would indicate otherwise. This effect is considered worthy of further study.
Can We Talk? 943
Hypothesis 3: Significant variance in teaming perception was found in Playbook and
Supervision, with average Teaming scores higher under the Voice condition in those
team structures (Table 2).
Table 2. Perception of teaming ANOVA results – IV is communication format
Type of automation F-statistic p-value
Manual control F(2, 118) = 1.046 p = .387
Batch processing F(2, 118) = 8.583 p = .007
Decision support F(2, 10) = 2.566 p = .126
Supervision control F(2, 118) = .5.593 p = .023
6 Discussion
The results indicate that human Performance and Perception of Teaming are positively
influenced by providing the synthetic teammate with a voice communication capability.
However, the improvement in performance measures was not consistent between team
structures.
In Manual Control and Playbook conditions participants improved the rate of
clearance and tended to prioritise larger targets. In Decision Support, with the synthetic
teammate conducting most of the task and providing strategy recommendations the
selection behaviour changed towards focusing on achievement of the personal gain
Goal (Maximise Score). The synthetic teammate’s recommendations in Decision
Support appeared to influence participants to take a more “risk tolerant” approach,
prioritising the clearing of smaller, higher scoring (Reward) targets over larger targets.
The result was more targets cleared and a greater reward per sample period.
Finally, under Supervision, where there was no noticeable difference in behaviour
there was the most significant (positive) variance in the perception of teaming.
7 Conclusions and Recommendation for Further Research
The conclusion of this study is that the introduction of voice communication between
the autonomous agent (synthetic teammate) and the human operator of a complex
system positively affects human teammate behaviour; however, that change in beha-
viour appears to vary depending upon the team structure. This is a result of how the
teaming structure affects the form of relationship between the human and synthetic
teammate. This difference in behaviour is sufficient to require further investigation and
research.
Also, in this study the human and synthetic teammate were always directly engaged
on a shared task and had the same goals and shared the same data and display. The
synthetic teammate communication was used to draw the attention of the human to data
in the human’s display. Thus, the communication was used to enhance or correct
human SA that was already being generated, rather than be used to create SA that had
944 A. Bogg et al.
until then been absent. It is recommended that further research is conducted to examine
how voice communication could be used to generate SA when the synthetic teammate
and human are both engaged on separate tasks and the human is asked to occasionally
change focus of attention and provide support to the synthetic teammate.
References
1. Demir, M., McNeese, N.J., Cooke, N.J.: Team communication behaviours of the human-
automation teaming. In: 2016 IEEE International Multi-Disciplinary Conference on Cognitive
Methods in Situation Awareness and Decision Support (CogSIMA), pp. 28–34. IEEE Press,
New York
2. Chen, J.Y.C., Lakhmani, S.G., Stowers, K., Selkowitz, A.R., Wright, J.L., Barnes, B.:
Situation awareness-based agent transparency and human-autonomy teaming effectiveness.
Theor. Issues Ergon. Sci. 19(3), 259–282 (2018)
3. Schaefer, K.E., Straub, E.R., Chen, J.Y.C., Putney, J., Evans III, A.W.: Communicating intent
to develop shared situation awareness and engender trust in human-agent teams. Cogn. Syst.
Res. 46, 26–39 (2017)
4. McNeese, J.N., Demir, M., Cooke, N.J., Myers, C.: Teaming with a synthetic teammate:
insights into human-autonomy teaming. Hum. Factors J. Hum. Factors Ergon. Soc. 60(2),
262–273 (2017)
5. Wickens, C.D.: Multiple resources and mental workload. Hum. Factors J. Hum. Factors
Ergon. Soc. 50(3), 449–455 (2008)
6. Sorensen, L.J., Stanton, N.A.: Keeping it together: the role of transactional situation
awareness in team performance. Int. J. Ind. Ergon. 53, 267–273 (2016)
7. Demir, M., McNeese, N.J., Cooke, N.J.: Team situation awareness within the context of
human-autonomy teaming. Cogn. Syst. Res. 46, 3–12 (2017)
8. Endsley, M.R., Kaber, D.B.: Level of automation effects on performance, situation awareness
and workload in a dynamic control task. Ergonomics 42(3), 462–492 (1999)
Driver’s Situational Awareness and Impact
of Phone Interface Modality in Conventional
and Semi-autonomous Vehicles
Syeda Rizvi1, Francesca Favaro1,2(&), and Nazanin Nader1
1
RiSA2S Research Center, San Jose State University, San Jose, CA, USA
{syeda.rizvi,francesca.favaro,nazanin.nader}@sjsu.edu
2
Department of Aviation and Technology, San Jose State University, San Jose,
CA, USA
Abstract. This project investigated how different modes of interaction of dri-
vers with a smartphone (i.e., manual texting vs. voice usage) affected driver’s
situational awareness in both conventional and semi-automated vehicles. The
study, which was executed in a human-in-the-loop setting, shows that partici-
pants expressed personal preferences for the vocal interface in all the tested
scenarios. However, few metrics actually showed statistical significance and an
improvement of performance with the vocal interface as compared to the manual
one.
Keywords: Situational awareness Human-in-the-loop simulation Texting
and driving
1 Introduction
Texting and driving is an unfortunate and dangerous combination, which produces
visual, cognitive, and physical distraction [1]. While a substantial body of research
exists on driver’s impairment due to secondary tasks other than driving (see, for
instance, [2–4]), not much research exists on comparing and quantifying a possible
reduction of distraction when different interface modalities are investigated. This is a
particularly important aspect for regulatory purposes in the USA, where a number of
states have passed laws banning manual texting and/or requiring hands-free use of
wireless phones while at the wheel [5].
This project thus investigated how different modes of interaction of drivers with a
smartphone (i.e., manual texting vs. voice usage) affected driver’s distraction and
performance in both conventional and semi-autonomous vehicles. The study, which
was executed in a human-in-the-loop setting, featured an integrated car simulator
capable of handling both manual and automated control. Two scenarios were tested: a
city scenario, and a highway scenario. In both scenarios, a secondary non-driving task
was introduced that prompted the driver to reply to a text through one of the two
interface modalities, while also following GPS instructions. This paper focuses on
presenting results related to how the interface variable (i.e., manual or vocal) affected
situational awareness within the simulation environment. Multiple facets of human
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 945–951, 2020.
https://doi.org/10.1007/978-3-030-39512-4_144
946 S. Rizvi et al.
factors are investigated, including recollection of events, estimation of success in
various maneuvers, and general human-machine interfaces preferences.
The structure of the paper is as follows: Sect. 2 presents the experimental setup and
the testing scenario structure; Sect. 3 delves into the analysis of the results; Sect. 4
concludes this work.
2 Methodology
Simulator. The study employed a static driving simulator with a BMW 6 series, a
projection wall providing 220-degree horizontal front view, and a split rear-projection
wall providing the projection for side and rear-view mirrors. The simulation environ-
ment used the Linux-based simulation framework Virtual Test Drive in version 2.1.0.
Open standards (OpenDRIVE® and OpenSCENARIO) were used for road and sce-
nario creation. The BMW featured a digital dashboard display for speed indication, as
well as a 10.2 in LCD central console that displayed the car status (i.e., automated
driving, manual driving, or “prepare to takeover” for transition status). The CD player
in the center of the car mounted an iOttie phone holder, where a Nexus 6 smartphone
was mounted. The phone and the simulator were on the same WiFi network and the
phone was receiving UDP messages from the main simulator at targeted trigger points
within the simulation environment. The GPS instructions and text messages display
were coded in Android Studio.
Participants. The study featured a population of 32, whose age ranged from 18 to 48
(mean = 26.8). The participants were all US licensed drivers equally split among male
and female. Participants selection was overseen by a San Jose State University
approved Institutional Review Board (IRB) protocol for human subject testing.
Test Structure. Two scenarios were tested: a city scenario, and a highway scenario. In
the city scenario, the driver would manually drive the car in conventional mode,
following GPS instructions in a suburban environment. A non-driving task related to
texting a message would then be introduced in conjunction with a traffic light turning
red. While replying to the text, the participant would need to resume driving after the
light became green, while a second text message would be received. In the highway
scenario, the car would be driving autonomously. The driver was still asked to monitor
the outside environment and GPS instructions. After roughly two minutes, the GPS
would prompt the driver to take the next exit, at which point the car triggered a warning
to prepare to regain manual control of the vehicle. A structured disengagement [6] of
the autonomous technology would then happen, prompting the driver to regain control
of the vehicle while taking the exit. A non-driving task relating to text reply would be
prompted in conjunction with the vehicle reverting to manual control and the request to
takeover. Both scenarios were repeated twice, once with drivers required to comply to
text requests by manually typing, and once by vocal reply. Before and after each test,
and in between each scenario repetition, participants would be asked to complete pre-
test surveys, post-test surveys, and in-between tests surveys. Before the four repeti-
tions, a practice drive was also executed, to ensure the driver was comfortable with the
Driver’s Situational Awareness and Impact of Phone Interface Modality 947
simulator environment and the vehicle’s performance. The overall duration for each
participant was of 1 h.
3 Analysis of Results
3.1 Situational Awareness
Situational awareness was in this study estimated through a number of questions asked
in between the scenarios tested as well as after the completion of the overall experi-
ment. Accuracy in the answers was measured, and in some it is as defined by the ratio
of correct answers vs. the totality of answers. For each question, categories of “True
Positive - TP”, “True Negative - TN”, “False Positive - FP” and “False Negative - FN”
were assessed in order to compute Accuracy as
Accuracy ¼ ðTP þ TNÞ=ðTP þ TN þ FP þ FNÞ: ð1Þ
Accuracy of ETA Reporting. In the texts received during the simulation, participants
were asked to provide their ETA (Estimated Time of Arrival). The ETA was displayed
at all times in the GPS interface on the smartphone. The paper surveys after each
scenario also asked to report the ETA. Their answers were compared to the actual ETA
on the GPS page and the overall accuracy was computed, as well as accuracy for
different modes, as seen in Fig. 1. Overall ETA estimation accuracy for the test was
31.25%. Higher accuracy in the estimation was obtained through the vocal interface,
and in city scenarios compared to highways. This is possibly due to the higher demand
of the manual reply, as well as the higher concentration needed in the highway scenario
to regain control of the vehicle. Statistical significance for two-tailed t-test among the
vocal and manual interface was shown (p < 0.1) for the survey estimations, but not for
data from texts. A refined ANOVA to combine both interface variables and type of
scenarios will be investigated in the future.
Fig. 1. Computed accuracy for ETA estimation
948 S. Rizvi et al.
Accuracy of Speed Recollection. After each scenario, participants were asked to
report the speed at which they were travelling and whether their answer was based on
the actual reading of the speedometer. Their responses were evaluated to be correct
based on a threshold of ± 2 mph with accuracy computed per Eq. (1). Overall accuracy
for speed recollection was 50%, with individual values reported in Table 1. Once more,
accuracy for the vocal interface is higher. However, this time the highway scenario led
to a higher accuracy than the city one. This implies that participants had a better
estimate at higher speeds (highway limit was 55 mph, city limit was 25 mph in
residential area). Two-tailed t-tests showed statistical significant for the change
between highway and city, but not for the phone interface.
Table 1. Accuracy of speed recollection for different scenarios
Mode Accuracy
Manual 46.9%
Vocal 54.0%
Highway 84.0%
City 16.1%
Perception of Success of Control Takeover. For the highway scenario, participants
were asked to assess whether their takeover maneuver was successful. Success of the
maneuver was defined in regard to a lateral drift contained within the bounds of the exit
lane. Unintentional lane departures were considered as “unsuccessful” control takeover.
Accuracy resulted at 59.4% for manual mode and at 62.5% for vocal mode. T-tests
showed no statistical significance (p = 0.103).
3.2 Human Machine Interfaces
Interface Comparison. Participants were asked their preference between voice and
touch in relation to personal preference, safety preference, difficulty in use, and in
estimated promptness of response. Figure 2 summarizes the responses.
Fig. 2. Text interface preferences summary for smartphone interface
Driver’s Situational Awareness and Impact of Phone Interface Modality 949
77.5% of participants indicated that they were comfortable with the cell phone
interface prior to the test; 6.45% indicated that they were not comfortable (paper
instructions as well as video tutorials were provided to the participants ahead of time);
the rest of the population was neutral. For both city and highway scenarios participants
showed a slight personal preference for the vocal interface, and a marked safety
preference for the vocal interface, however, prior to the test only 12.5% indicated a
strong preference for vocal interfaces. Future steps of this research will compare the
drift analysis to correlate it to such results.
Highway Aural and Visual Disengagement Warnings. 93% of participants con-
firmed they heard the aural warning for the takeover request (TOR). This is a decrease
from previous similar studies [6], where all participants heard the aural warnings. 80%
of participants reported that they found the aural warning helpful and were satisfied by
it, however 12.9% found it distracting and one participant said it hindered their ability
to take control. With regard to the TOR visual warning, 23% of the participants
reported not seeing it within the central console (10.2-inch display). This is an
improved percentage from a previous study (50% of people did not see the warning)
within the same simulator [6] and could be explained by the fact that the cell phone was
placed near the display. Out of the participants that saw it, 2 reported it to be dis-
tracting, but that it did not hindered their ability to regain control, however another
participant indicated that it hindered the ability to takeover. 54% of participants indi-
cated that they were satisfied with the warning while 6% disagreed.
3.3 Safety Awareness
Perceived Safety Compromise. Participants were asked to rank on a scale of 1 being
least likely and 5 being most likely whether responding to the texts compromised
safety. The results are summarized in Fig. 3. The highway scenario with manual tex-
ting interface showed the highest level of perceived compromised safety.
Fig. 3. Perceived safety compromise as a function of the investigated variables
950 S. Rizvi et al.
3.4 Subjective Measures
Participants were asked to rank the extent to which they felt the following emotions
before and after the test: tiredness, concern for physical and mental wellbeing, antic-
ipation, anxiety and fear. Figure 4 shows pronounced changes in the tiredness, antic-
ipation and anxiety levels, with the most significant to be a 60.4% increase in anxiety
levels, 53.8% increase in anticipation and 49.4% increase in tiredness.
Fig. 4. Changes in reported emotional states before and after the experiment
4 Conclusions
One of the goals of the present study was to provide a scientific evidence to back-up
current regulations that allow use of cellphone while driving if in hands-free vocal
modality. However, with regard to drivers’ situational awareness, few results showed a
statistically significant improvement for the vocal interface. A higher percentage of
participants preferred the voice interface with both personal and safety regards, and
also ranked low difficulty and overall low perceived safety compromise with the vocal
interface for both city and highway scenarios. This indicated a change from the pre-test
responses, in which only 12.5% indicated strong preference for the vocal interface.
Participants perceived a higher level of compromised safety for manual interface,
although no statistical significance in driving performance (e.g., drift) was observed for
the two interfaces.
Acknowledgments. Funding for this research was provided by the US Department of Trans-
portation (grant 69A3551747127 managed by the Mineta Transportation Institute of San Jose,
CA).
References
1. Caird, J.K., Johnston, K.A., Willness, C.R., Asbridge, M., Steel, P.: A meta-analysis of the
effects of texting on driving. Accid. Anal. Prev. 71, 311–318 (2014)
2. Bakhit, P.R., Guo, B., Ishak, S.: Crash and near-crash risk assessment of distracted driving
and engagement in secondary tasks: a naturalistic driving study. Transp. Res. Rec. 2672(38),
245–254 (2018)
Driver’s Situational Awareness and Impact of Phone Interface Modality 951
3. National Highway Traffic Safety Administration, NHTSA –National Telephone Survey on
Distracted Driving Attitudes and Behaviors – DOT HS 812 437 (2015)
4. Fitch, G.M., Soccolich, S.A., Guo, F., McClafferty, J., Fang, Y., Olson, R.L., Perez, M.A.,
Hanowski, R.J., Hankey, J.M., Dingus, T.A.: The impact of hand-held and hands-free cell
phone use on driving performance and safety-critical event risk (2013)
5. Governors Highway Safety Association - GHSA. Distracted Driving (updated 2019). https://
www.ghsa.org/state-laws/issues/distracted%20driving
6. Favaro, F.M., Eurich, S., Rizvi, S., Agarwal, S., Mahmood, S., Nader, N.: Analysis of
Disengagements in Semi-Autonomous Vehicles: Drivers’ Takeover Performance and
Operational Implications (2019)
Concept of an Adaptive Cockpit to Maintain
the Workflow of the Cockpit Crew
Juliane Müller(&) and Axel Schulte
Institute of Flight Systems, Bundeswehr University Munich,
85577 Neubiberg, Germany
{juliane.mueller,axel.schulte}@unibw.de
Abstract. In this article we describe a novel concept of a reconfiguring cockpit
that adapts itself to the current work situation of the pilot. Today’s cockpit
design methods make use of static task models and workload models to avoid
overload situations to the cockpit crew. In our approach, we enable an Adaptive
Cockpit Configurator (ACC) to assess the current and future work situation in
terms of the crew’s activity and mental workload during mission execution. In
this article, we present the general design considerations and the underlying
functions. Some of those functions have already been prototyped, but need
adaptation. A closed-loop implementation and integration in our experimental
cockpit simulator are planned as next steps. We use this concept in the field of
military manned-unmanned teaming in fast jet missions.
Keywords: Human factors Human-systems integration Cockpit design
Cognitive ergonomics
1 Introduction
The cockpit is an important component of the aircraft that enables the pilot to interact
with the aircraft systems. It allows both civilian and military pilots to perform the
various tasks relevant to the mission. It also serves as a link to the outside world. In the
past, the instrument panel has been cleaned up significantly by the use of integrated
displays, reducing the potential for visual overload, but increasing the need to deal with
the different but equally challenging perspectives of cognitive overload [1]. In the
military aviation sector in particular, the development of unmanned aerial vehicles
(UAVs) is also creating new tasks for pilots. One of these is the management of
manned unmanned teams (MUM-T) from the cockpit, where the pilot is responsible for
the guidance of one or many UAVs in addition to his usual tasks. The challenge is that
the pilot’s tasks become more complex and heavily impact mental workload (MWL).
In conventional cockpit design, an important design goal is to create a workflow for the
cockpit crew in a way that there will be no situations of overtaxing in terms of MWL
during operations. This is also a certification requirement for cockpits (FAR 25.1523)
[2]. Therefore, certain approaches for quantifying MWL during the design phase have
established from research as well as industrial practice. For instance, Time-Line
Analysis and Prediction (TLAP) [3] uses a static task model derived from e.g. a
Hierarchical Task Analysis [4] of the considered mission type. The resulting tasks are
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 952–958, 2020.
https://doi.org/10.1007/978-3-030-39512-4_145
Concept of an Adaptive Cockpit to Maintain the Workflow of the Cockpit Crew 953
arranged for analysis in a sequence taking into account their time consumption.
The MWL is estimated by calculating the ratio of time required to available mission
time (TR/TA). Another method, VACP [5], extends the idea of TLAP, i.e. describing
task demands by pure duration, by use of a 4-dimensional (visual, auditory, cognitive,
psychomotor) quantification of mental resource demands. Again, those are accumu-
lated for the tasks over the mission time including multi-tasking situations. The
resource-related summed values quantify the overall work demands of the crew and
reveal potential overload situations at design time. Wickens [6] extended this thought
in his Multiple Resource Theory. He postulates that different mental resources of a
human (e.g. visual, auditory) can be used simultaneously. However, conflicts may
arise, if the same resource is used by concurrent tasks. Those methods lead to a static
cockpit and automation design, which is good for relatively simple work processes and
relatively predictable situations during the flight mission.
Our goal is to develop a system which makes use of those cognitive ergonomics
concepts, but transfers suchlike analyses to be part of the runtime system. We envision
a cockpit design, which incorporates functions that:
• analyse the current work situation in real time,
• derive current and future work demands for the crew parallel to the mission
executing,
• identify resource conflicts of the pilots, as they may occur during the actual exe-
cution of the flight mission, and
• reconfigure cockpit automation support, with the goal to keep work demands within
certain bearable limits, and thereby, avoid erroneous human actions.
The latter shall be accomplished by software-ergonomic measures such as attention
guidance of the cockpit crew to most meaningful information, but also more important
by cognitive ergonomics approaches concerning the adaptation of the level of
automation, the reconfiguration of task allocation between the human and the system as
well as influencing the tasks allocation between human crew members.
This contribution shall describe the first steps taken to implement such an adaptive
cockpit. Therefore, we pick the concepts of task analysis and modelling, the description
of mental resource demands, and pair them with pilot observation techniques in the
cockpit, to facilitate a dynamic task model (DTM) that is able to continuously describe
the work situation of a cockpit crew.
2 General Approach
2.1 Work System
To derive the architecture of an adaptive cockpit in the MUM-T environment, we first
look at the entire system from a human-system design perspective. Therefore, we use
the Work System notation defined in [7]. Figure 1 shows the work system design for a
command fighter, where a pilot commands his own aircraft as well as a number of
UAVs. The Adaptive Cockpit Configurator (ACC) shall take the role of an additional
co-worker to the pilot, and hence pursues the work objective (i.e., the mission) in
954 J. Müller and A. Schulte
cooperation with the human (blue connection). Therefore, the ACC shall be enabled to
observe the pilot’s activity in the cockpit. If a situation is predicted that negatively
affects the workflow, cockpit functions and configuration can be adapted by the ACC
(green arrow) to avoid human overload or erroneous action.
Fig. 1. Work system of the Adaptive Cockpit Configurator
2.2 Agent Architecture
The functional architecture of the ACC agent results from the requirements of the
environment and the cognitive ergonomics concepts we will integrate. In this section,
we describe the general structure of the ACC (Fig. 2) and in the next chapter we will
describe the individual modules in more detail.
Fig. 2. Architecture of the ACC agent
The starting point is the given mission objective. A planner determines the tasks to
accomplish the mission and generates a schedule [8, 9]. The plan interpreter uses a
database called task model [10] to generate a detailed description of the tasks to be
performed by the pilot in order to conduct the plan. We call this the Dynamic Task
Model, DTM. Each task in the Task Model is attributed with values describing the
mental resource demands necessary to perform the task [11, 12]. These values will also
be transferred to the DTM. This results in a detailed schedule of all pilot’s tasks for a
specific mission, including mental resource demands. During mission execution, the
pilot will successively perform task by task, which will lead to a continuous update of
the DTM. With a pilot observation, which monitors the pilot’s cockpit interactions, it is
Concept of an Adaptive Cockpit to Maintain the Workflow of the Cockpit Crew 955
possible to estimate which tasks are being processed [12, 13]. With the updated DTM,
the Intervention Management will predict future pilot’s MWL peaks and resolve it by
adjusting the cockpit configuration.
3 Concepts
3.1 Planner
The planner is responsible for planning the necessary tasks for the desired mission goal.
Primarily the logical order of mission tasks is determined by the pilot. Finally, the
mission planner creates the entire mission plan, using the mission tasks contained in the
task model without subordinate tasks [8–10].
3.2 Task Model and Plan Interpreter
The task model (TM) according to [12, 13] represents the result of a tasks analysis in
the design process. It is a hierarchical representation of all possible mission tasks, their
evidences, and the associated mental resource demands for the assessment of the pilot’s
mental workload. Evidence in this case are interactions in the cockpit (e.g. pressing the
button to release the park brake) or context information, such as the flight phase, which
indicates the processing of a specific task. These are required for activity determination
(Sect. 3.3).
For the ACC, three categories of the task model are relevant: mission tasks, pilot
tasks and vehicle tasks. In these categories, mission tasks are at the top of the task
hierarchy. To each mission task, pilot tasks and vehicle tasks are subordinated. Fig-
ure 3 shows a mission task being a part of the full task model. The task Target
Reconnaissance includes two pilot tasks - View Image and Confirm Target Status - as
well as one vehicle task - Take Recce Picture. Vehicle tasks define task that are
assigned to a UAV or the manned aircraft, generally spoken to the automation. Next to
the relations, the task model contains constraints for tasks. In the example Target
Reconnaissance, the constraints describe that first the image of the UAV has to be
taken before the pilot can view the image. The status of the target can only be con-
firmed after viewing the image.
Fig. 3. Representation of a mission task with the corresponding vehicle and pilot’s tasks in the
task model
The plan interpreter combines the missions plan with the task model. It extracts the
pilot tasks from the TM for the mission tasks of the plan. Here the tasks retain their
956 J. Müller and A. Schulte
properties, such as constraints and demands. The pilot tasks are now arranged in time
according to the mission tasks and constraints under the tasks. This results in a mission-
specific plan for the tasks of the pilot, the initial DTM.
In the helicopter MUM-T application an adaptive crew assistant system has already
been developed by Brand [10]. Conceptually, our initial DTM is based on Brand’s
detailed plan [10]. This also creates a precise arrangement of pilot tasks in order to
identify for example MWL peaks.
3.3 Dynamic Task Model and Activity Determination
Figure 4 shows an example visualization of an initial DTM. In the figure, the mission
consists of the task TargetReconnaissance to the target B31. This contains two pilot
tasks and one vehicle task as subtasks. The timeline shows the planned execution time
of the tasks. The length of the rectangles determines the planned duration.
Fig. 4. Visualization of the dynamic task model with mission task Reconnaissance.
With this arrangement of pilot tasks and associated demand values, describing the
mental resources needed to complete a task, the MWL can be predicted throughout the
mission. Peak loads and resource conflicts can be detected. With this, we estimate the
total resource demands and resource conflict values using the Multiple Resource
Theory by Wickens [6] and the W/Index calculation by North and Riley [14]. From
those values, we compute a total mental workload value [13].
In order to exclude processed tasks from the MWL prediction we need an activity
detection. This shall recognize what the pilot is working on. Such a task recognition is
already described by Honecker [13] and Mund [12]. By monitoring pilots through
gaze-tracking and manual interactions, the evidences from the TM are confirmed or
rejected. The combination by using an evidential reasoning approach derived from
Dempster-Shafer theory [15] leads to the conclusion that a task is currently being
executed by the pilot. This information about the processing status is stored in the
modified DTM.
3.4 Intervention Management
Our aim is to adjust the cockpit design and the degree of automation of the cockpit in
such cases, like MWL peaks, to maintain the crew’s workflow. By understanding the
current and predicted work situation of the pilot, warning signals or attention guidance
Concept of an Adaptive Cockpit to Maintain the Workflow of the Cockpit Crew 957
can be used efficiently. It can be ensured that the pilot receives the right information
and can react quickly. With the intention recognition of the pilot the possibility is given
to deliver meaningful decision support through the cockpit. In this case, the increased
MWL provides the indicator for the necessary decision support. The cockpit can
support the pilot in fulfilling the task through a higher degree of automation and enable
faster interaction. This ensures that his MWL does not rise immeasurably. However, if
the MWL exceeds a level to be managed, so that the pilot is not able to act for a task, a
redistribution of tasks can take place during runtime as in the cockpit design process.
This reconfiguration can take place between human and system as well as between
human crew members.
4 Conclusion
We presented a concept that integrates the cockpit design process to support the pilot in
real time with cognitive ergonomic methods. The general architecture of the ACC agent
was derived from an abstract perspective. Here, task analysis and modelling, mental
resources and pilot observation were combined into a DTM that describes the work
situation of the crew. On this basis, the work requirements for the crew during the
mission are determined. If the crew threatens to get beyond certain tolerable limits,
interventions are initiated to maintain the workflow.
References
1. Salas, E., Maurino, D., Curtis, M.: Human factors in aviation. An overview. In: Human
Factors in Aviation, pp. 3–19 (2010)
2. FAA Federal Aviation Regulations: FAA Federal Aviation Regulations (FAR-25) (2013)
3. Parks, D.L., Boucek, G.P.: Workload prediction, diagnosis, and continuing challenges. In:
Applications of Human Performance Models to System Design, pp. 47–63. Springer, Boston
(1989)
4. Stanton, N.A., Salmon, P.M., Rafferty, L.A., Walker, G.H., Baber, C., Jenkins, D.P.: Task
analysis methods. In: Human Factors Methods: A Practical Guide for Engineering and
Design, 2nd edn, pp. 39–69. Ashgate Publishing Company (2013)
5. McCracken, J.H., Aldrich, T.B.: Analyses of Selected LHX Mission Functions: Implications
for Operator Workload and System Automation Goals (1984)
6. Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3,
159–177 (2002)
7. Schulte, A., Donath, D.: A design and description method for human-autonomy teaming
systems. In: Advances in Intelligent Systems and Computing, pp. 3–9. Springer, Cham
(2018)
8. Heilemann, F., Schmitt, F., Schulte, A.: Mixed-initiative mission planning of multiple ucavs
from aboard a single seat fighter aircraft. In: AIAA Scitech 2019 Forum (2019)
9. Schmitt, F., Schulte, A.: Mixed-initiative mission planning using planning strategy models in
military manned-unmanned teaming missions. In: Proceedings - 2015 IEEE International
Conference on Systems, Man, and Cybernetics (SMC), pp. 1391–1396 (2016)
958 J. Müller and A. Schulte
10. Brand, Y., Schulte, A.: Model-based prediction of workload for adaptive associate systems.
In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC),
pp. 1722–1727. IEEE (2017)
11. Honecker, F., Brand, Y., Schulte, A.: A task-centered approach for workload-adaptive pilot
associate systems. In: Proceedings of the 32rd Conference of the European Association for
Aviation Psychology, Cascais, Portugal (EAAP), pp. 485–507 (2017)
12. Mund, D., Schulte, A.: Model- and observation- based workload assessment and activity
determination in manned-unmanned teaming missions real-time mental state assessment. In:
European Association of Aviation Psychology Conference (2017)
13. Honecker, F., Schulte, A.: Automated online determination of pilot activity under
uncertainty by using evidential reasoning. In: Harris, D. (ed.) Engineering Psychology and
Cognitive Ergonomics: Cognition and Design, pp. 231–250. Springer, Cham (2017)
14. North, R.A., Riley, V.A.: W/INDEX: a predictive model of operator workload. In:
Applications of Human Performance Models to System Design, pp. 81–89. Springer, Boston
(1989)
15. Dempster, A.P.: The Dempster-Shafer calculus for statisticians. Int. J. Approx. Reason. 48,
365–377 (2008)
A Conceptual Augmentation of a Pilot
Assistant System with Physiological Measures
Dennis Mund(&), Evgeni Pavlidis, Matthew Masters,
and Axel Schulte
Universität der Bundeswehr München, Neubiberg, Germany
{dennis.mund,evgeni.pavlidis,
matthew.masters,axel.schulte}@unibw.de
Abstract. This article describes a concept for augmenting an adaptive assistant
system for pilots in human-autonomy teaming missions with physiological
measures such as heart rate, electrodermal activity and functional near-infrared
spectroscopy. The article also showcases the potential benefits of such an
approach. We first present our existing assistant system which utilizes a purely
activity-based workload assessment. We then present three steps how we aim to
utilize the sensing of various physiological measures of the pilot(s) to improve
overall system performance. These steps include: (1) determining a purely
physiological-based workload estimate, (2) incorporating physiological mea-
sures into the task-based workload estimate, and (3) generating individual
operator models by tuning system parameters based on the physiological
measures.
Keywords: Adaptive automation Physiological monitoring Assistant
systems Human-autonomy-teaming Manned-unmanned teaming
1 Introduction
Adaptive assistant systems have the potential to enable high performance in human-
machine cooperation by adjusting the mode or level of assistance to the current situ-
ation and state of the human user. While high levels of automation can lower the user’s
workload in a given situation, there are drawbacks related to out-of-the-loop effects
such as lowered situation awareness. To mitigate these effects, adaptive assistant
systems aim to monitor the operator’s workload level and find an appropriate mode
and/or level of assistance. At the Institute of Flight Systems, we are working towards
building assistant systems which incorporate a model of what the pilot is doing with an
estimate of how engaged he/she is to provide appropriate assistance. Due to the
complexity of modeling mental workload, we assert that it need be viewed (or mea-
sured) from multiple perspectives acknowledging the complex context which generated
the workload [1]. In this work we supplement our existing context/task-based
assessment of mental workload with physiological measures.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 959–965, 2020.
https://doi.org/10.1007/978-3-030-39512-4_146
960 D. Mund et al.
2 Existing Workload-Estimation Through Task
Determination
2.1 Real-Time Task Determination
As presented in [2], our real-time task determination system utilizes a “task model”
consisting of all possible tasks which can be performed in the simulated environment
and their relationships to each other, as well as the auditory, visual, spatial, and verbal
demands required by each task. By monitoring the operator’s interactions with the
cockpit, including their visual, verbal, and manual interactions, our system estimates
the task(s) currently being executed by the operator. Each observed interaction is cross-
referenced to the interactions for each task in the task model with a Dempster-Schafer
inference algorithm.
2.2 Task-Based Workload Inference
The resource requirements for a particular task are established by a pre-defined eight-
dimensional demand vector [3] modeling the task’s unique resource requirements [4].
Figure 1 illustrates two tasks and their associated demands in each of the eight
dimensions. The resource dimensions considered are those of Multiple Resource
Theory presented in [5]. For each instant in time, the system infers a per-resource
demand as well as an overall workload estimate by combining the resource require-
ments of all presently-executed tasks using the W/Index [6].
Fig. 1. Visualization of the resource demands for two tasks being executed simultaneously.
With an estimate of how fully each resource channel is currently occupied, the
assistant system is able to select an interaction modality which utilizes free capacity of
the pilot [7]. In addition to optimizing real-time assistance, by predicting the tasks that
have yet to be performed, the assistant system can predict excessive future workload
demands and work with the pilot to avoid these situations [4].
A Conceptual Augmentation of a Pilot Assistant System with Physiological Measures 961
3 Methods to Improve System Performance Through
Physiological Measurements
3.1 Workload Estimation Through Physiological Measurements
Various physiological signals have been shown to correlate with mental workload
under particular settings and/or situations. Such signals include heart rate (HR) [8],
heart rate variability (HRV) [9], heart rate variability low-frequency to high-frequency
power ratio (LF/HF) [10], functional near infrared spectroscopy (fNIRS) activation in
the pre-frontal cortex (PFC) [11], electrodermal activity (EDA) [12], and many others
[13]. To evaluate these trends, we conducted a preliminary experiment collecting HR,
HRV, LF/HF, fNIRS, and EDA of a user over the course of simulated medevac
mission. Following the mission, the user provided a subjective workload appraisal
while watching a recorded video of the mission. Correlation analysis has yielded results
suggesting that these physiological signals are correlated with the subjective workload
assessment. Signals most strongly correlated with the subjective workload assessment
were: HR, a 10 s moving average of EDA, and the slope of LF/HF with Pearson
correlation coefficients (PCCs) of 0.39, 0.35, and 0.35 respectively. Due to these and
other weaker correlations, it is reasonable to propose that multiple physiological signals
could be combined into a single measure representing mental workload providing a
more robust estimate over one that is based on any individual signal. A preliminary
application of this approach can be seen in Fig. 2. Here, the subjective workload
assessment was regressed onto the following eight physiological measurements
through linear regression: HR, HRV, a ten-second moving average of EDA, the HRV
LF/HF ratio over a three-minute window, the slope of the LF/HF ratio, saccades per
minute, and the mean oxygenated and deoxygenated-hemoglobin concentration change
levels (HHb and O2Hb) of 24 fNIRS optodes on the user’s PFC. The application of
these learned weights can then provide a purely physiological-based workload estimate.
It is proposed that this mapping can then be applied in real-time during future missions.
The inherent delay in various input signals however, could potentially minimize the
utility of this physiological-based estimate.
3.2 Direct Incorporation of Physiological Measures into the Mental
Workload Estimate
Another approach to utilize the physiological measurements is to fuse them with the
model-based approach to leverage the advantages of both. While the model-based
approach provides an accurate estimate of which workload channels are necessary for
the currently-executed tasks, the actual amount of workload for a specific task might be
variable. In some cases, this varying difficulty can be captured by differentiating tasks
under different conditions, as for example differentiating “manual flight” between “high
altitude” and “low altitude.” In other cases, however, like an “object identification”
task, where the operator has to extract mission relevant information from a sensor
image, the resulting workload of this task strongly depends on the content of the image,
which cannot be easily assessed for its inherent difficulty. Since the assistant system
cannot assess this difficulty, it is impossible to discern a high-workload “object
962 D. Mund et al.
Fig. 2. Various signals plotted over the course of a simulated helicopter medevac mission. The
upper-most plot (Subjective WL), is the pilot’s subjective workload assessment post-mission. In
orange are automated task recognition system results (displaying 8 of 55 recognized tasks) as
well as a workload estimate based solely on these instantaneous task predictions (Task-based
WL). In blue are various physiolocal measures and a workload estimate based on these and other
physiological measures. Shaded areas denote times during which the helicopter was on the
ground.
identification” task from a low-workload “object identification” task and therefore the
model-based approach can only provide information about which resources are occu-
pied by the task.
On the other hand, as presented in Sect. 3.1, physiological sensors can provide an
estimate of the overall workload of the operator. This assessment however, lacks the
ability to differentiate between stressed resources. Therefore, we propose utilizing the
physiological-based workload estimate to inform the need for assistance, and the task
analysis to optimize how the assistance is provided. This approach illustrates the
potential utility of expanding one’s view of mental workload and evaluating it from
multiple perspectives. Thus, in this effort we augment the multi-dimensional view of
mental workload presented in [1] with the physiological state of the operator. We claim
that the assessment of workload can be operationalized by noting the externally
imposed objectives, the current human activity and its associated resource demands,
operator behavior patterns, and the operators physiological state.
3.3 Generation of Individual Operator Models by Tuning System
Parameters
The overall workload measure inferred by our system is computed as follows:
A Conceptual Augmentation of a Pilot Assistant System with Physiological Measures 963
workload ¼ task load þ conflict load
Where the task load is the sum of the demand vectors for all recognized tasks over all
channels and conflict load is the additional load stemming from the conflict between
channel resource demands (see [4, 6, 14]).
However, different pilots have different capacities and process information differ-
ently. The load induced, for example, may be dependent on the skill level of a pilot.
Both the demand vectors and the conflict parameters can therefore be assumed to be
individual for each pilot and adjusting them will lead to a more appropriate and useful
workload estimate.
Using the workload metric described in Sect. 3.1, we can adjust the parameters in
such a way that the difference between the measured value and the estimated value is
minimized.
Optimization of the Demand Vectors
Intuitively speaking, the demand vectors model how each task burdens the user’s
mental and physical resources. For example, the task “Monitor Video Streams from
UAV” has load on the visual-spatial and visual-verbal channel on the perception side.
A particular pilot may find it easier to monitor the streams (due to higher automaticity),
thus the scaling down of the demand vector will lead to a better system performance.
The difficulty is that we want to optimize the combination of multi-dimensional values
(demand vectors for several tasks) by using the physiological values at each time
instant. As the search space is too big it is difficult to optimize the parameters. Instead
of optimizing all demand values directly, we can introduce a scaling factor. The first
possibility is to scale the whole demand vector for a specific task similar to the
description in Sect. 3.2. This is reasonable as the resource allocation distribution for a
specific task is not being changed, but the impact of the workload over all resources is
adjusted. A second possibility is to scale the resource channels. Intuitively this adjusts
for a user’s unique physical or mental capacities.
Optimization of the Conflict Matrix
Our current system uses a predefined conflict matrix as given in [15]. This matrix
models how each resource channel interferes with all others. It can be argued that
different pilots have unique conflict values, as the load by performing two task which
utilize different resource channels may vary. For example, a pilot may find it more
difficult to simultaneously fly and track the position of the UAVs on the MFD as he has
to divide the resources of the visual-spatial and visual-verbal perception cognition
channels between both tasks. This can be modeled by a higher conflict value between
those two channels. Optimizing the system parameters to individual pilots will allow
the creation of pilot-specific profiles which can improve the workload estimation
during flight, and in turn improve the quality of the actions made by the assistant
system.
964 D. Mund et al.
4 Discussion and Conclusion
We propose that by combining physiological workload measurements with a model-
based workload estimation we will be able to develop an assistant system which more
appropriately responds to the human operator resulting in an improvement to the
overall human-machine system. We suggest that such a system can be utilized in
human-autonomy teaming scenarios.
References
1. Schulte, A., Donath, D., Honecker, F.: Human-system interaction analysis for military pilot
activity and mental workload determination. In: Proceedings - 2015 IEEE International
Conference on Systems, Man, and Cybernetics, SMC 2015, pp. 1375–1380 (2016). https://
doi.org/10.1109/SMC.2015.244
2. Honecker, F., Schulte, A.: Automated online determination of pilot activity under
uncertainty by using evidential reasoning. In: Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
(2017). https://doi.org/10.1007/978-3-319-58475-1_18
3. Wickens, C.D.: Multiple resources and mental workload. Hum. Factors J. Hum. Factors
Ergon. Soc. 50, 449–455 (2008). https://doi.org/10.1518/001872008x288394
4. Honecker, F., Brand, Y., Schulte, A.: A task-centered approach for workload-adaptive pilot
associate systems. In: Proceedings of 32rd Conference of the European Association for
Aviation Psychology, Cascais, Portugal, pp. 485–507 (2016)
5. Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3,
159–177 (2002). https://doi.org/10.1080/14639220210123806
6. North, R.A., Riley, V.A.: W/INDEX: A Predictive Model of Operator Workload (1989).
https://doi.org/10.1007/978-1-4757-9244-7_6
7. Maiwald, F., Schulte, A.: Mental Resource Demands Prediction as a Key Element for Future
Assistant Systems in Military Helicopters. Presented at the (2011). https://doi.org/10.1007/
978-3-642-21741-8_62
8. Mansikka, H., Simola, P., Virtanen, K., Harris, D., Oksama, L.: Fighter pilots’ heart rate,
heart rate variation and performance during instrument approaches. Ergonomics 59, 1344–
1352 (2016). https://doi.org/10.1080/00140139.2015.1136699
9. Castaldo, R., Melillo, P., Bracale, U., Caserta, M., Triassi, M., Pecchia, L.: Acute mental
stress assessment via short term HRV analysis in healthy adults: a systematic review with
meta-analysis. Biomed. Signal Process. Control 18, 370–377 (2015). https://doi.org/10.1016/
J.BSPC.2015.02.012
10. Shaffer, F., McCraty, R., Zerr, C.L.: A healthy heart is not a metronome: an integrative
review of the heart’s anatomy and heart rate variability. Front. Psychol. 5, 1040 (2014).
https://doi.org/10.3389/fpsyg.2014.01040
11. Aghajani, H., Garbey, M., Omurtag, A.: Measuring mental workload with EEG + fNIRS.
Front. Hum. Neurosci. 11, 359 (2017). https://doi.org/10.3389/fnhum.2017.00359
12. Collet, C., Salvia, E., Petit-Boulanger, C.: Measuring workload with electrodermal activity
during common braking actions. Ergonomics 57, 886–896 (2014). https://doi.org/10.1080/
00140139.2014.899627
13. Heard, J., Harriott, C.E., Adams, J.A.: A survey of workload assessment algorithms. IEEE
Trans. Hum.-Mach. Syst. 48, 434–451 (2018). https://doi.org/10.1109/THMS.2017.2782483
A Conceptual Augmentation of a Pilot Assistant System with Physiological Measures 965
14. Horrey, W.J., Wickens, C.D.: Driving and side task performance: the effects of display
clutter, separation, and modality. Hum. Factors 46, 611–624 (2004). https://doi.org/10.1518/
hfes.46.4.611.56805
15. Horrey, W.J., Wickens, C.D.: Multiple Resource Modeling of Task Interference in Vehicle
Control, Hazard Awareness and In-vehicle Task Performance, pp. 7–12 (2005). https://doi.
org/10.17077/drivingassessment.1082
Implementation of Teaming Behavior
in Unmanned Aerial Vehicles
Marius Dudek(&), Sebastian Lindner, and Axel Schulte
Institute of Flight Systems, Universität der Bundeswehr Munich,
Werner-Heisenberg-Weg 39, 85577 Neubiberg, Germany
{marius.dudek,sebastian.lindner,
axel.schulte}@unibw.de
Abstract. Research in manned-unmanned teaming (MUM-T) addresses the
integration of manned and unmanned vehicles within a team. The concept of
task-based guidance has been proven to reduce the human pilot’s workload in a
MUM-T mission in multiple experiments. In addition to the automated execu-
tion of tasks, a mission requires the cooperation between the UAVs in order to
increase survivability and mission performance. In this contribution, we describe
the implementation of team support for UAVs guided from the cockpit of a
manned combat fighter. The implemented agent uses its spare time to escort the
team member with the highest priority task. The presented approach was inte-
grated into the MUM-T research simulator at the Institute of Flight Systems and
the effects of the implemented teaming behavior on mission effectiveness
parameters were analyzed in two scenario simulations.
Keywords: Human-autonomy teaming Unmanned aerial vehicles Cognitive
agent Agent teaming Soar cognitive architecture
1 Introduction
In our MUM-T research, we address the integration of manned and unmanned aerial
vehicles (UAVs). In this context, the concept of task-based guidance was examined in
several domains such as ground control stations [1], helicopter cockpits [2] and fighter
jets [3]. Task-based guidance intends to keep the pilot’s workload balanced by dele-
gating tasks to cognitive agents on board of the unmanned systems, granting the human
user to have enough capacity to control his own platform. In order to effectively reduce
the user’s workload, these tasks are formulated as high-level tasks. The UAV agents
must be able to understand these tasks and decompose them into subtasks, including
effective resource management, self-protection measures and cooperation with team
members. The cooperation of team members has been investigated at an earlier stage at
the Institute of Flight Systems [4]. This approach to teaming was based on mutual
knowledge about mission tasks and the ability of the UAVs to negotiate about the
required tasks to achieve the mission objective. In contrast, in this approach, the
assignment of tasks is done by the pilot and the UAVs escort important team members
besides the execution of tasks on their own initiative to contribute to the mission
objective.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 966–972, 2020.
https://doi.org/10.1007/978-3-030-39512-4_147
Implementation of Teaming Behavior in Unmanned Aerial Vehicles 967
2 Concept
Work System. The work system according to [5] describes the relation between
human users, cognitive agents and automation. A distinction is made between workers,
that are familiar with the work objective (WObj) and pursue it on their own initiative,
and tools, which receive tasks from workers and only then become active. In the task-
based guidance concept, agents receive tasks by the human user, decompose them and
control conventional automation in order to perform these tasks (Fig. 1, left). The
approach discussed here, incorporates an agent, that pursues the work objective
independently, considering the tasks assigned by the human pilot. In this case, the
agent becomes a worker (Fig. 1, right). The contribution to the pursuit of the work
objective is realized though support of team members in this case. To serve the mission
success, UAVs escort team members.
Fig. 1. Work system for general task-based guidance (left) and our new approach (right)
MUM-T Mission. A MUM-T mission is a military mission conducted with a team of
manned and unmanned systems. The mission scenarios include various tasks to be
performed such as the reconnaissance of potential targets, the acquisition of precise
coordinates of a position or the suppression of enemy air defence sites [6]. The human
pilot specifies the tasks to be performed in the mission and assigns them to the UAVs in
the team. The tasks are attributed with priorities that describe their importance in the
mission. The tasks are assigned to the UAVs by a tasking interface [7] and result in the
mission plan, which contains the tasks of all team members and their respective start-
and end-time (Fig. 2).
UAV Autonomy. The UAVs must consider various aspects during the execution of a
mission. For example, enemy air defence positions influence the route planning of the
agents and the dynamics of enemy forces require constant adaptation to changing
tactical boundary conditions. Another aspect for UAVs is the management of resour-
ces, such as fuel. Fuel is one of the most valuable assets during a mission. Excessive
fuel consumption decreases the time an UAV is available for tasks and may lead to an
unsuccessful mission. From the UAV point of view, the mission execution is a per-
manent trade-off between the available resources (fuel, time, ammunition). The UAV
agent described in this contribution uses the resource fuel to support team members.
968 M. Dudek et al.
Fig. 2. Mission time plan for UAV Bravo and UAV Golf. Bravo has to investigate building 32
(B32) and B42 and Golfs tasks are to investigate B31 and B41. The bar below shows the mission
time at which the tasks are scheduled.
Support. Team members are supported by escorting them along their route. This
enhances the ability to respond to sudden threats such as pop-up air defences, which
can increase the survivability of the supported platform. Which team member takes on
the role of the supported team member and which team member supports is defined by
the priorities of the assigned tasks: Team members with high task priorities are sup-
ported, as it is most important for mission success. Team members with a lower priority
will subordinate and perform the role of a supporter.
In each decision cycle, the agent is faced with the question of whether he better
supports a team member or pursues the task assigned to him by the pilot. The extent to
which the support of a team member serves the mission success depends essentially on
the temporal and spatial constellation of the team. How the agent makes this decision is
shown in the flow-chart diagram in Fig. 3.
Fig. 3. Decision-making process for an UAV agent with multiple team members.
Implementation of Teaming Behavior in Unmanned Aerial Vehicles 969
The agent first selects all team members, whose tasks have a higher priority than the
agent’s task. Team members with a lower or equal priority are not considered for a
support. Then, the agent calculates for how long he could support each team member
considering the transit to the team member and the time the agent’s next task starts. If
the support time is higher than a certain threshold, a support is considered as possible.
Since support may be possible for multiple team members, the agent selects the team
member with the highest priority for the support afterwards. In case of equal priority of
several supportable UAVs, the support times are compared.
3 Implementation
The decision-making process of the UAV agent is performed by a rule-based approach
using Soar cognitive architecture [8]. The mission tasks are designed as substates
containing task-specific rules. If the agent has a specific task to perform, it searches
within these substates for applicable rules. If the agent is not yet on site to execute the
task, it chooses the navigation substate, where the following rules (operators) can be
found, which decide about the support of team members:
• Fly to team member (Condition: Support time > 30 s, Distance > 2500 m)
• Support team member (Condition: Spare time > 0 s, Distance < 2500 m)
• Direct to task (Always selectable)
Since multiple operators can be suggested at the same time with the defined
operator conditions, the following rules apply for the selection of operators:
• Prefer “Fly to team member” and “Support team member” over “Direct to task”
• Prefer higher priorities for “Fly to team member” and “Support team member”
• If priorities are equal, prefer higher support time
The routes for support and spare time estimation are generated with the Open
Motion Planning Library (OMPL) [9] as there are several air defences in a mission, that
have to be avoided. The prediction of the interception position is performed by an
iteration using the airspeed of the supported aircraft. In two-dimensional space, the
intercept position can be determined analytically using vector geometry. In this
application, this is not possible because of the route planning around air defence
positions. Calculating the intercept on the basis of the mission plan is not suitable, as
routes of team members are only valid for a short time in the mission’s highly dynamic
environment.
4 Integration and Results
The described agent was integrated in the fighter jet research simulator of the Institute
of Flight Systems. To analyze the agent’s teaming behavior, the execution of two
scenarios were simulated. Since only the agent was to be examined, no human user was
involved in the simulation. The mission plan was created beforehand and the move-
ment of the manned fighter was taken over by a flight management system. Since the
970 M. Dudek et al.
goal of the teaming was to be more resilient against threats such as pop-up air defences,
the survivability of the team members should be investigated for improvement.
However, a statement about the probability of survival requires a sufficient radar and air
defence model, which is not given at the current time. Therefore, the distance between
the team members was used as a measure, assuming that short distances increase
survivability. Since the support of team members is obtained with a higher fuel input,
the fuel consumption of the agents is determined in each simulation. The simulations
were executed using different UAV configurations. In Config. 1, each UAV had an
agent without teaming behavior. In Config. 2, teaming behavior was integrated in each
agent. To determine the effects of different transition speeds, Config. 3 used the same
agent, however allowing transits at 420 knots (instead of 350 knots in Config 2).
Distances between team members were defined by the great-circle distance, the fuel
consumption of UAVs was calculated based on the fuel consumption of a typical
fighter jet. The scenario and the mission time plan for one of the scenarios are shown in
Fig. 4. The goal of this scenario is the investigation of buildings (B32, B42, B31 and
B41, displayed in yellow), the elimination of an early-warning radar (EW_01, out of
sight) and an air defence (SRSAM, red circle) as well as the suppression of a tracking
radar (TRA_01, blue circle).
Fig. 4. Scenario (left): Fighter (grey) with three UAVs (Bravo, Golf and SEAD). Lines are the
routes of the team members. Mission time plan (right): Bravo and Golf investigate the buildings,
SEAD destroys SRSAM and suppresses TRA_01. The fighter destroys EW_01.
Figure 5 shows the average distance over mission time and UAV fuel consumption
for all three agent configurations.
Figure 5 shows, that the teaming behavior decreases the average distance between
team members. The reduction of the distance for Config 2 results mainly from the
exploitation of idle times while Config 3 also reduces distances in the timeslots
between two tasks. In the other, less time-critical, scenario this effect is reduced
because team members transit with a lower speed and can therefore also be supported
by Config 2.
Implementation of Teaming Behavior in Unmanned Aerial Vehicles 971
Fig. 5. Average distance during mission (left) and total fuel consumption (right)
5 Conclusion and Future Work
The approach of using an agent that contributes to the mission objective by supporting
team members was integrated and first benefits could be derived from the simulation.
However, more fuel is consumed to provide support of team members, which is why
the effects can vary from one scenario to another. To assess the benefits of the proactive
support of team members, radar and air defence positions should be modelled in detail
and further simulations should be carried out. In a further step, an attempt could be
made to change the thresholds of decisions and the transition speed depending on the
mission. In addition, teaming behaviour could be considered in the planning phase to
create mission plans with a high level of support, increasing the effectiveness of the
teaming behaviour. Since the UAVs do not fly directly to the next destination while
supporting, confusion may result. In order to maintain situation awareness, the pilot
should be given detailed feedback about the reasons of the agent’s decision.
References
1. Clauss, S., Schulte, A.: Task delegation in an agent supervisory control relationship capability
awareness in a cognitive agent. In: Proceedings: 2014 IEEE International Conference on
Systems, Man and Cybernetics, pp. 825–830 (2014)
2. Uhrmann, J., Schulte, A.: Concept, design and evaluation of cognitive task-based UAV
guidance. J. Adv. Intell. Syst. 5(1&2), 145–158 (2012)
3. Reich, F., Heilemann, F., Mund, D., Schulte, A.: Self-scaling human-agent cooperation
concept for joint fighter-UCAV operations. In: Savage-Knepshield, P., Chen, J. (eds.)
Advances in Human Factors in Robots and Unmanned Systems, pp. 225–237 (2017)
4. Meitinger, C., Schulte, A.: Human-UAV co-operation based on artificial cognition. In: Harris,
D. (ed.) Engineering Psychology and Cognitive Ergonomics, pp. 91–100. Springer,
Heidelberg (2009)
5. Schulte, A., Donath, D., Lange, D.S.: Design patterns for human-cognitive agent teaming. In:
Harris, D. (ed.) Engineering Psychology and Cognitive Ergonomics, pp. 231–243. Springer,
Heidelberg (2016)
6. Lindner, S., Schwerd, S., Schulte, A.: Defining generic tasks to guide UAVs in a MUM-T
aerial combat environment. In: Karwowski, W., Ahram, T. (eds.) Intelligent Human Systems
Integration 2019, pp. 777–782. Springer, Heidelberg (2019)
972 M. Dudek et al.
7. Heilemann, F., Schulte, A.: Interaction concept for mixed-initiative mission planning on
multiple delegation levels in multi-UCAV fighter missions. In: Karwowski, W., Ahram, T.
(eds.) Intelligent Human Systems Integration 2019, pp. 699–705. Springer, Heidelberg (2019)
8. Laird, J.: The Soar Cognitive Architecture. MIT Press, Cambridge; London (2012)
9. Sucan, I.A., Moll, M., Kavraki, L.E.: The open motion planning library. IEEE Robot. Autom.
Mag. 19, 72–82 (2012)
Behavioral Analysis of Information Exchange
Digitalization in the Context of Demand
Planning
Tim Lauer(&) and Katharina Franke
Infineon Technologies AG, Am Campeon 1-15, 85579 Neubiberg, Germany
{Tim.Lauer,Katharina.Franke}@infineon.com
Abstract. Demand planning is a key process within supply chains, especially
for industries exposed to the bullwhip effect. Supply chain research often focus
on factual processes and therefore disregard the involvement of employees on
the planning level. Here, information exchange as part of segregated task dis-
tributions plays an important role to gain joined best performance. This paper is
a first step to analyze the impact of digitalization of information exchange in
collaborative demand planning. Applying a behavioral experiment on demand
decision making the analysis of the results shows that provided information is
more beneficial than on request. Furthermore, face-to-face exchange leads to an
overestimation of the information. The findings foster the importance of human
behavior on planning levels and the need of research on the influence of
digitalization.
Keywords: Behavior research Digitalization Demand planning
1 Introduction
Demand planning holds a key position in supply chain planning. It provides the basis
for subsequent processes like material planning, scheduling, order promising etc.
Accompanied by the trend of digitalization, much current research aims on forecast
accuracy improvement using for instance machine learning techniques or utilizing new
data sources. Nowadays, academic results foster that a combinatorial forecast of human
planners and statistical algorithms performs best, but it depends on the particular case.
In these cases, the planer as part of the system plays an important role, but the human
factor is often left out for examination by behavioral management tools. As today’s
business environment requires an increasing degree of collaboration, this topic gains
more attraction, especially considering the impact of digitalization. Consequently, this
paper focuses on the overlap of demand forecasting, human behavior and digital col-
laboration in the form of information exchange.
In Sect. 2, relevant literature on collaboration concerning information exchange is
reviewed. Section 3 outlines the experimental set-up, based on the industrial processes
of case of the Infineon Technologies AG. In Sect. 4, the results are stated and analyzed.
In addition, the paragraph examines which conditions support planers in their demand
decisions. The paper concludes with a summary and a critical review.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 973–979, 2020.
https://doi.org/10.1007/978-3-030-39512-4_148
974 T. Lauer and K. Franke
2 Literature Review
Decision-making is a well-studied process, where this research follows the step-based
approach of [1]. Along the process influential factors come into play. The level of
complexity and the dimensions of decisions are of major importance. [2] observed that
deciders limit the area of a problem space the more complex a decision gets. In demand
planning this reduction has even more impact in terms of biased results [3]. The
available amount of times influences the decider as well. Humans struggle with urgent
decisions in relation to the information processing necessary [3]. [4] transfers this
aspect to less available information than one would usually need for this decision. This
means that a decision is more rational, the more time is available to gather referring
information. This conversion only applies to a certain level. Too much information lead
to an overload. Next, the collection of potential consequences implies a challenge
because of the dynamic context [5]. Also important is the respective employee himself
[5]. For the entire process, parties often have to collaborate with other entities, e.g. for
gathering information [5]. This starts even with the identification of a problem, whether
done by a single or group decision [5]. Especially for decision making in a group one
has to consider the personal interactions [5]. This aspect is out of scope for this
research.
Today’s work environments demand more and more collaboration resulting from
the increasing number of interlinked tasks [6]. Collaboration emerges whenever more
than one agent depend on each other to perform a task [5]. In this context ‘agent’ is
used because collaboration is also existing between humans and systems. In collabo-
ration clear roles and objectives are defined, where all parties benefit [5]. For supply
chains as shared value chains collaboration is a key success factor. Therefore,
departments need to share their information and make decisions together to achieve a
common objective [7]. In such fields, changes are difficult; there are several elements to
support this transformation. The free flow of information is stated as most important
[7].
Supply chain research shows that information exchange as form of collaboration
improves planning processes. This requires connections between different partners, e.g.
departments [8]. [4] describes several aspects how such a connection should be set and
influential factors. If several individuals pursue a common goal, the referring processes
have to be aligned and clearly split up [8]. Within environments where trust dominates,
the exchange of information is positively affected [9]. Employees are able to rely on the
information they get [9]. For an improvement it is important that transparency between
partners is ensured combined with an appropriate quality of information flow [9].
Employees do not only collaborate with each other, but also interact with systems. In
addition, there is technology, which enables collaboration between humans. As neg-
ative impacts, there are several challenges: Employees have to deal with more
monotony performing their task. In some cases, they have to adjust their work
according to requirements of the digital system. When the executed task is more
complex, it might occur that decisions cannot be predicted, and results cannot be
comprehended [1]. Depending on the specific task systems can lower the complexity
and decrease the operative effort for the planner [1]. The usage of a system premises the
Behavioral Analysis of Information Exchange Digitalization 975
right choice of implementation. Therefore, the selection of a suitable technology is
challenging because all processes and required information has to be specified in detail
considering the above-mentioned findings. If the implementation process is successful
one observes a positive affected relationship [7].
3 Behavioral Experiment
Considering the literature review, the collaboration between two entities is a suitable
experimental set-up to investigate the impact of information exchange in a digital
compared to analog based approach. In a green-field-brainstorming session, seven
industrial subject matter experts confirmed the information exchange as the overall
crucial factor in demand planning. The experts abstracted the demand planning deci-
sion from the industrial case to a daily life situation in order to extend the range of
possible participants for the experiment. In the end, the team decided to setup the
experiment within the field of demand planning an amount of wine in reference to the
common beer game scenario. Instead of the marketing department, the participants get
a reference value from an expert wine retailer. For the participants this retailer is
presented reliable with huge experience in the wine market and in addition, he already
supports for a long time. In collaboration with this expert, the participant has to decide
for six events on the demand value. In providing events, the medium has to be found
between events where the input from an expert is necessary and where everybody is
able to deal with. Framing, every participant gets the role of a sports club member who
is responsible to order the right amount of wine for the monthly events of the sports
club. One developed six events according to the months July until December.
In order to compare the effect of the digital setup the experiment is divided into two
groups, one with the digital, the other with the analog set-up. For every participant the
consumption of wine of former events of the last three years, a prediction on the
number of expected visitors this year and a reference value as recommendation from
the retailer are available. At every event, the control group has the possibility to ask the
retailer via mail, if anything is unclear, before decision-making. In contrast, in the
treatment groups a face-to-face meeting with the retailer is simulated, where the market
situation, production improvements and the weather forecast are presented. On pur-
pose, not all provided information is relevant for the game. In order to ensure infor-
mation symmetry, the control group is able to get the same information when using the
digital possibility to contact the retailer. Equally, for both groups, a providing digital
setup is implemented with content only for the event in September and December.
Otherwise, a black box without content is provided and only for these events, one gets
an additional hint regarding the expected needed amount of wine. This information
even exceeds the meeting session of the treatment group.
To guarantee an appropriate conduction of the main part of the experiment the
game starts with a short introduction. Here, some technical instructions, common lab
rules, the framing of the sports club events and the task of decision-making are stated.
Furthermore, to encourage each participant on his decision, the experimenter
announces that for each event, the individual accuracy is calculated, and incentives will
976 T. Lauer and K. Franke
be paid based on the accuracy level. The participants were only able to participate in
the experiment after correctly answering understanding questions to ensure validity.
After the decision-making main part, a questionnaire followed. Hereby, demo-
graphics and experiences were captured and a deep dive into the decision-making
process was made.
For the implementation, the open source based and well-known software oTree was
used [10]. The programming language Python enabled respective settings utilizing
Heroku as webserver for the game distribution in the experiment sessions.
4 Conduction and Results
The experiment was conducted in several sessions. The participants were distributed
randomly. The treatment group gathered for only one session to avoid information
inconsistencies. In the end 44 participants took part in the experiment, 66.7% of them
stated no forecasting experience, but 19.0% at least up to one year.
The Mann-Whitney U-Test and the respective correlation value r are the methods of
choice of the experiment set-up. r smaller than 0.10 refers to a small effect, and greater
than 0.50 to a strong effect size. The calculation of effect size d is more suitable than r
according to [11] using the pooled standard deviation. d smaller 0.20 states a small
effect, bigger 0.80 a strong effect and medium between. This ensures minimal possible
error for the result and the following interpretation [11].
The first result analysis looks on the standard deviations for both groups (see
Table 1).
Table 1. Comparison of standard deviation
Event Analog treatment Digital treatment
July (J) 1.87 1.92
August (A) 2.73 3.28
September (S) 48.37 50.83
October (O) 2.51 3.39
November (N) 4.47 3.66
December (D) 7.67 5.24
Total 8.93 9.37
Normal (J, A, O, N) 1.52 2.02
Special (S, D) 25.53 26.46
The group with the analog meeting deviates less from the actual demand than the
digital group. Nevertheless, the November and December planning indicates higher
accuracy the other way around. Next, different settings are tested against each other. As
the calculations of r of the Mann-Whitney-U-Test and effect size d bases on the
significance z, all results are compared together in Table 2.
Behavioral Analysis of Information Exchange Digitalization 977
Table 2. Significance, Mann Whitney test and effect size
Comparison Significance z Mann Whitney test Effect size
Analog vs. digital −1.69 0.26 0.39
Normal vs. digital −1.15 0.17 0.43
Special vs. digital −1.45 0.22 0.35
September vs. digital −1.83 0.28 0.38
December vs. digital 0.27 0.04 0.05
Analog vs. not asking −2.97 0.49 0.97
The table shows in row 5 a significant difference between the groups for December
planning in general and an even bigger difference comparing treatment values with
those of the control group where the possibility to ask the retailer was not used in row
7. It is important to mention that from 21 players in the analog group only seven asked
for an explanation at the September planning round. This high significance is also
reflecting with r equal 0.49, which indicates nearly a strong effect. In contrast, one
recognizes the smallest value of each calculation for the December planning. As
explained the effect sizes are calculated using the pooled standard deviation. In line
with former results, the December planning has the lowest value (see Table 2). With d
equal 0.05, a difference between both groups is nearly not noticeable. In addition, with
d equal 0.97 the last row shows a large effect, too. Subsequently, Table 3 analyzes the
influence of forecast experience. For the four events where the retailer’s recommen-
dation suits the historic values the experienced planned slightly better. The trust in the
information of the meeting is higher for players without experience. Finally, the dig-
italized setup is more accepted for experienced players (71.4% versus 33.3%). Nev-
ertheless, due to the small number of participants this has to be treated cautiously.
Table 3. Standard deviation experience
Comparison Analog with Analog without Digital with Digital without
experience experience experience experience
Analog vs. 8.79 8.69 8.96 9.14
digital
Normal vs. 0.81 1.72 1.12 1.67
digital
Special vs. 25.51 24.54 25.51 24.54
digital
July 2.23 2.79 3.72 3.63
August 2.78 2.70 4.09 2.65
September 49.14 45.62 50.78 50.58
October 2.00 2.65 1.81 3.94
November 1.98 4.86 2.61 4.00
December 4.69 8.39 4.06 5.73
978 T. Lauer and K. Franke
Overall, the analog group achieves higher forecast accuracy as the digital group, as
only few players even used the possibility to inquire information from the retailer.
Without this additional information, a planner’s decision is worse. Experiences
enforces this effect. For the September planning, experienced planners trust more their
theoretical knowledge and rely even less on external additional input for their decision.
For an organization, it is consequently important to incite the affected employees to
exchange their knowledge on a regular basis, which deals also for departments. The
values of both groups for December converge to the actual demand for wine compared
to the September plan. The digitalized setup in form of the black box giving additional
information in general is advantageous. In addition, the participants stated the digital
set up as more efficient than to ask the retailer. In detail, named advantages in the
question are the secure provision of available information and the absence of time lags
emerging from waiting for a response. However, the analog group performed worse in
December. This is attributable to the information overload from the meeting. Those
players only rely on the presented information and as this works out for September, the
black box in the planning interface once filled is not recognized. Hence, one can argue
that the initiation of a meeting could lead to an overestimation of the information.
Furthermore, the four standard events in the game show no statistically significant
differences in their planning values. Consequently, standard conditions do not require
exchange in planning processes in order to save time. This idea might be misleading as
the planer has to differentiate the standard from a special condition first.
5 Conclusion
Decisions in demand planning are crucial for the entire supply chain, as basis for
succeeding processes. Hereby, the focus is set on collaborative aspects in the form of
information exchange between entities. Three different ways of exchanging informa-
tion are developed. Firstly, the control group does not initially get the information, but
has the possibility to ask the retailer via mail. Secondly, the retailer gives an overview
with relevant and unnecessary information to the treatment group before starting the
game. Thirdly, both human types are compared to a digitalized setup providing
additional explanation as report. The demand planning decision is abstracted by experts
to a daily life decision derived from the beer game.
Summarizing the statistical analysis, on the one hand the control group significantly
preferred the digital setup with reference to the time lag and missing security to get to
the information. On the other hand, nobody wanted to substitute the meeting. Players
planned better with provided information than with the possibility to ask, especially of
the request aversion. This effect is enlarged for experienced players. The treatment
group planned worse than the control group for the December event, the participants
overestimated the retailer’s information and fail to notice the digitalized setup.
The experiments state only a small step in research of a digitalized working
environment but fosters more studies in this direction. For instance, the information
exchange reflects only one aspect of collaboration, whereas more should be considered
as mentioned in the literature chapter. Another aspect is the transfer of the real problem
to its abstraction; industry case studies could show different outcomes.
Behavioral Analysis of Information Exchange Digitalization 979
References
1. Schlick, C., Trzcieliński, S.: Advances in Ergonomics of Manufacturing: Managing the
Enterprise of the Future, vol. 490. Springer, Cham (2016)
2. Association of German Engineers: VDI 4006 Part 1: Human reliability - Ergonomic
requirements and methods of assessment, Düsseldorf (2015)
3. Johnston, S.: What Drives Supply Chain Behavior? In: Havard Business School (2004)
4. Lorkowski, J., Kreinovich, V.: Bounded Rationality in Decision Making Under Uncertainty:
Towards Optimal Granularity, vol. 99. Springer, Cham (2018)
5. Zaraté, P.: Tools for Collaborative Decision-Making. Wiley, London (2013)
6. Plattner, H., Meinel, C., Leifer, L.: Design Thinking Research. Springer, Berlin (2012)
7. Barratt, M.: Understanding the meaning of collaboration in the supply chain. Supply Chain
Manag.: Int. J. 9(1), 30–42 (2004)
8. Hauke, J., Lorscheid, I., Meyer, M.: Individuals and their interactions in demand planning
processes: an agent-based, computational testbed. Int. J. Prod. Res. 1(1), 1–15 (2017)
9. Zwetsloot, G., Scheppingen, A., Bos, E., Dijkman, A., Starren, A.: The core values that
support health, safety, and well-being at work. Saf. Health Work 4(4), 187–189 (2013)
10. Chen, D., Schonger, M., Wickens, C.: oTree - an open-source platform for laboratory,
online, and field experiments. J. Behav. Exp. Financ. 9, 88–97 (2016)
11. Coe, R.: It’s the Effect Size, Stupid: What effect size is and why it is important, pp. 1–11
(2002)
Signs Symbols & Displays in Automated
Vehicles: A Focus Group Study
Joy Richardson(&), Kirsten Revell, Jisun Kim, and Neville A. Stanton
The University of Southampton Southampton, Boldrewood Innovation Campus,
University of Southampton, Burgess Road, Southampton, SO16 7QF, UK
[email protected]
Abstract. As SAE level 2 semi-autonomous vehicles are now available in the
UK vehicle displays are required to communicate more complex information to
the driver. It is clear from examining interfaces from different manufacturers that
there is lack of consistency in these displays with different vehicles showing
icons of a variety of shapes and colours to convey the same message. When
driving a semi-autonomous vehicle it is essential the driver knows whether they
are in an automated or manual driving mode. However the lack of standardis-
ation in icon design can lead to mode confusion. To understand the impact on
users a focus group was held where participants were asked to interpret the
meaning of icons from a variety of different driver interfaces. As a result it is
recommended that new ISO standard icons are introduced in order to support
drivers in SAE level 2 and 3 automated vehicles.
Keywords: Semi-autonomous Automobile Interface design Icons
1 Introduction
The use of pictures to convey meaning is one of the oldest forms of communication,
prior to the invention of written languages these pictures would be used to record
history and tell stories [1, 2]. Icons are a way of using pictures to deliver a specific
message and are formed of several elements which can include a border, a background
and text but is focused on the pictorial element, known as the symbol [3]. Studies have
shown that well designed icons can be recognised more quickly and accurately than
textual displays [1, 3–5] they also have the benefit of consuming less space than text
and, if well designed, can be universally understood.
Carney et al. [3] suggest Icons can be classified into three types; pictorial repre-
sentations of the object or action they represent with meaning easily derived and little
effort required to learn. Concept-related icons based on an image or a property of an
actual object or action, these can be context specific so are more difficult to learn.
Arbitrary icons are only meaningful through convention and rely on particular
knowledge which can be cultural. These are the most difficult type of icon to learn.
As partially automated (SAE level 2) vehicles have been available in the UK
market for several years (and highly automated (SAE level 3) vehicles are currently in
production [6] dashboard displays are becoming more sophisticated and required to
communicate more complex information. It is clear from examining the interfaces from
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 980–985, 2020.
https://doi.org/10.1007/978-3-030-39512-4_149
Signs Symbols & Displays in Automated Vehicles: A Focus Group Study 981
different manufacturers that the levels of functionality are being interpreted very dif-
ferently and this is reflected in the lack of standardisation in associated displays and
imagery. Previous studies, evaluating the automation systems and HMI’s across a range
of SAE level 2 vehicles, identified some mode confusion whereby the driver has been
unsure of whether they are in control of the vehicle or if it is automated, despite the
mode being displayed on the vehicle’s dashboard [7].
The international Organisation for Standardization or ISO in an independent non-
governmental organisation whose responsibility it is to make international standards [8,
9]. ISO standardised icons have been commonly used to communicate messages to
automobile drivers on the vehicle dashboard for many years. This can be seen to
originate with the standardisation of traffic light colours in the 1920s [10] through to
the development of icon recommendations in the 1970s [4]. Due to new technologies
these icons are now also being displayed on an additional variety of screens within the
vehicle, these combined screens are known as In-Vehicle Information Systems (IVIS).
ISO 2575 outlines all of the standardised icons in use and is revised every five years
[8], it also forms the British Standard BS ISO 2575 in it’s entirety [11]. Currently there
are over 300 of these falling into 18 categories from lighting and signalling to vehicle
handling.
ISO standard 2575 was examined to determine which icons supported the com-
munication of aspects of automated driving. Section J relates to vehicle handling and
cruise control. This section contains icon J.09 to indicate when adaptive cruise control
(ACC) is active and Icon J.10 indicates if the ACC system has failed. There are no
other symbols within the standard to indicate any further level of automation. In March
2018 a draft for the new version of the standard was released to the public for com-
ments and is due for publication shortly. This draft does not contain any additional
icons to support automated driving [12] although there is currently a draft resolution
which has recently been circulated to ISO international committees which does look at
introducing icons which could be used to indicate mode in SAE level 2 vehicles.
In addition to prescribing the shape of icons the ISO standard 2575 also states how
colours should be used as they have standard meanings attached. Importantly an icon
may be shown in more than one colour to convey a change in operation of that system
[13].
To address these issues of mode confusion and inconsistency a focus group was
held with a demographically diverse range of participants in terms of age, gender, type
of vehicle used, annual mileage and driving experience. They were asked to interpret
the meaning of a variety of displays from the three different manufacturers of SAE level
2 vehicles used in previous studies, an SAE level 3 vehicle, for comparison, and from
two high fidelity simulators where studies had previously been conducted [14].
2 Method
Participants. Ethical approval was gained via the University of Southampton ethics
panel (ERGO No 48777). Seven participants were recruited across three age groups;
18–34, 35–56 and 56–80+ to ensure an inclusive sample meeting the group size (5–10)
982 J. Richardson et al.
commonly recommended in the literature [15–17]. Participants were recruited via
posters around the university campuses, posts on social media, direct emailing to a list
of people who had previously expressed interest in taking part in research undertaken
by this team and, to ensure Age group 3 was captured, through contacting the local
branch of The University of the Third Age, an organisation who describe themselves as
a group “which brings together people in their ‘third age’ to develop their interests and
continue their learning” [18].
Design. The focus group lasted 2 h in line with recommendations [16, 19]. Demo-
graphic information collected included age, gender, annual mileage, years holding a
full driving license, any advanced driver qualifications (none reported for any partic-
ipant) and experience in using a range of 17 Advanced Driver-Assistance Systems
(ADAS).
The event started with an introduction to the subject of the workshop and focus
group in line with common practice [16, 19] The concepts of signs and symbols which
are currently commonly displayed on vehicle dashboard, highly automated driving and
automated to manual handover were explained and then demonstrated using a manu-
facturer promotional video.
Equipment. Multiple strategies were used for data capture; Two Sony HandyCam
video cameras were mounted on tripods close to the group in order to obtain footage
and audio recording of the group discussions which were subsequently transcribed, a
GoPro video camera was mounted high up above the group in order to obtain footage
of the group moving and placing the materials on the table, printed copies of icons,
workbooks containing images of the icons accompanied by notes written by the par-
ticipants and comments from the group written by the facilitators on flip charts during
the exercises.
Procedure. The first exercise was a written task, in this the participants were presented
with a workbook containing images of icons from the aforementioned vehicles and
simulators each with an adjacent area for writing their interpretation of the meaning.
The group was then given some further guidance from the facilitator about the types of
messages these icons could be conveying and the participants were asked to categorise
the icons by placing the images under categories placed on the table. The participants
then discussed their choices as a group and rearranged the icons on the table until they
came to a consensus as to their meaning. The event finished with the third exercise; an
unstructured session allowing for any comments and recommendations to be made
which were captured by a facilitator on a flip chart.
3 Results
Exercise One. Each manufacturer and the simulators have icons to indicate that the
automation system is active. Most of the participants were able to interpret these icons
accurately but some participants expressed confusion. One participant interpreted this
icon to mean the opposite to that intended.
Signs Symbols & Displays in Automated Vehicles: A Focus Group Study 983
All of the manufacturers have icons indicating that the automation is inactive or
ending and the simulator an icon to indicate manual mode. Some participants made
opposing interpretations believing the car was in manual mode when, in fact, in
automated.
Exercise Two. Prior to the second task the group were introduced to the potential
meanings of the icons. This enabled them to identify them more easily. Before coming
to a consensus the various interpretations were discussed by the group in detail and this
revealed some opposing views and also some ambiguity.
There were also many discussions regarding the use and meaning of colour. The
participants had opinions as to what the colours should represent. There was also a
discussion around standardisation and how this is helpful in aiding understanding.
Exercise Three. The final task was a less structured open discussion but raised some
of the same concerns as the previous tasks. The use of colour in icons was still of
significance with further comments around the importance of standardisation.
4 Discussion
It can be seen that the majority of icons from the manufacturers and from the simulator,
designed to indicate mode, are based on the representation of a steering wheel. One
manufacturer has opted to use the image of a car. These icons can be classed as
concept-related icons, they are based on an image of an actual object and sometimes an
associated action. These are moderately difficult to learn and it is therefore not sur-
prising that some participants found them challenging to interpret without guidance or
previous experience. Where participants had made more unusual interpretations of the
icons it can be seen that they were examining them more closely as a pictorial type
icon; seeing the image in front of them as a literal representation of the road ahead.
Some participants interpreted icons identifying the automation mode inversely,
effectively believing a car was not in automated mode when it was but more concerning
was the understanding that the car was in automated mode when it was not. This
ambiguity could cause a driver to omit making driving inputs causing an accident
resulting in damage to the vehicle and harm to the driver, passengers and other road
users. There is also a negative economic impact to the driver in terms of increased
insurance premiums, vehicle repair or replacement, any casualties from medical bills
and missed employment and to society as a whole from emergency services and
highway repair [20].
Colour of icons was important to the participants and was raised by them in all
three exercises. Colour can increase the likelihood icons are noticed [21] and when
used well can aid communication. However when used carelessly they can increase
confusion [1]. The colours used in the icons during this focus group are white/grey,
green, blue, amber/yellow and red. Participants made many comments about how they
interpret the meaning of colours. They referred to the use of colour in IVIS as they
experience it in their own vehicles but also making comparisons to traffic lights.
984 J. Richardson et al.
5 Conclusion
During the initial exercise the participants were asked to write their interpretations and
meanings of a number of icons used in semi-autonomous vehicles and simulators. They
reported a wide range of ideas about the meaning of the icons. Those participants with
more experience of (ADAS) in their own vehicles were more likely to have correct or
similar ideas as to the meaning of the icons. Those with less experience of ADAS
attributed a wider range of ideas related to the way they interpreted the icons, seeming
to make more literal interpretations. Before the second exercise the participants
received additional guidance about the possible meanings of the same icons. They were
able to make a correct interpretation of more icons than in the previous task, however
they were also in agreement that several icons remained ambiguous. The final exercise
allowed the participants to openly discuss the icons they had seen. This section showed
that colour and standardisation were considered important when trying to understand
icons. The participants considered it essential that colours were only used in what they
considered the conventional manner, their opinions on the use of colour aligned well
with the standards outlined in ISO 2575.
Following these exercises and discussions it is recommended that ISO standards
should be created for the icons used in IVIS indicating automated and manual modes in
order to reduce driver confusion. These icons should be designed using existing human
factors methodologies to ensure they are clear, concise and consistent.
Acknowledgements. This work was supported by Jaguar Land Rover and the UK-EPSRC grant
EP/N011899/1 as part of the jointly funded Towards Autonomy: Smart and Connected Control
(TASCC) Programme. The authors also acknowledge the support of Simon Thompson at Jaguar
Land Rover for his support in preparing this paper.
References
1. Horton, W.: The Icon Book. Wiley, New York (1994)
2. Moser, S.: Ancestral Images: The Iconography of Human Origins. Sutton Publishing
Limited (1998)
3. Carney, C., Campell, J.L., Mitchell, E.A.: In-Vehicle Display Icons and Other Information
Elements: Literature Review, U.S. Department of Transportation (1998)
4. Green, P.: Design and evaluation of symbols for automobile controls and displays. In:
Peacock, B., Karwowski, W. (eds.) Automotive Ergonomics. Taylor & Francis (1993)
5. Baber, C., Wankling, J.: An experimental comparison of text and symbols for in-car
reconfigurable displays. Appl. Ergon. 23(4), 255–262 (1992)
6. Gasser, T.M., Westhoff, D.: BASt-study: Definitions of automation and legal issues in
Germany (2012)
7. Revell, K., Richardson, J., Langdon, P., Bradley, M., Politis, I., Thompson, S., Skyrchuk, L.,
O’Donoghue, J., Mouzakitis, A., Stanton, N.: “That was scary…” exploring driver
autonomous vehicle interaction using the Perceptual Cycle Model. In: Charles, R.,
Wilkinson, J. (eds.) Contemporary Ergonomics and Human Factors 2018. CIEHF (2019)
8. ISO 2019. https://www.iso.org/standard/71505.html. Accessed 16 Sept 2019
9. ISO 2016 ISO and Road Vehicles
Signs Symbols & Displays in Automated Vehicles: A Focus Group Study 985
10. Priest, H.A., Wilson, K.A., Salas, E.: National standardization efforts in ergonomics and
human factors. In: Karwowski, W. (ed) Handbook of Standards and Guidelines in
Ergonomics and Human Factors. Lawrence Erlbaum Associates (2006)
11. BSI 2010 Road vehicles – Symbols for controls, indicators and tell-tales
12. ISO 2018 Draft BS ISO 2575 Road Vehicles - Symbols for Controls Indicators and Tell-
Tales
13. ISO 2010 ISO 2575:2010 Road vehicles - Symbols for controls, indicators and tell-tales
14. Politis, I., Langdon, P., Adebayo, D., Bradley, M., Clarkson, P.J., Skrypchuk, L.,
Mouzakitis, A., Erikkson, A., Brown, J.W.H., Revell, K., Stanton, N.: An evaluation of
inclusive dialogue-based interfaces for the takeover of control in autonomous cars. In:
Intelligent User Interfaces - ACM IUI 2018, Tokyo (2018)
15. Caplan, S.: Using focus group methodology for ergonomic design. Ergonomics 33(5), 527–
533 (1990)
16. Krueger, R.A., Casey, M.A.: Focus Groups: A Practical Guide for Applied Research. SAGE
Publications, Thousand Oaks (2015)
17. Morgan, D.L.: Focus Groups as Qualitative Research, 2nd edn. Sage Publications, Thousand
Oaks (1997)
18. U3A 2019 About Us. https://www.u3a.org.uk/about. Accessed 16 Sept 2019
19. Morgan, D.L.: The Focus Group Guidebook. Sage Publications, Thousand Oaks (1997)
20. Gorea, R.K.: Financial impact of road traffic accidents on the society. Int. J. Ethics Trauma
Victimology 2(1), 6–9 (2016)
21. Young, S.L.: Increasing the noticeability of warnings: effects of pictorial, colour, signal icon
and border. In: Laughery, K.R., Wogalter, M.S., Young, S.L. (eds.) Human Factors
Perspectives on Warnings (1994)
Beauty Attracts the Eye but Character
Captures the Heart: Why Personality Matters
in Chat Bot Design
Helen Muncie(&)
Thales UK Limited, 350 Longwater Avenue, Green Park, Reading, Berkshire
RG2 6GF, UK
[email protected]
Abstract. Why does personality matters in chat bot design? People project
human traits onto everything and now, with the proliferation of conversational
user interfaces these objects can talk back. That is why is it important to create a
personality for your chat bot that is relatable and believable and conveys
appropriate meaning and emotion in a style and language that is congruent with
your end users expectations. This paper describes how a variety of user-centred
design techniques, including psychometric profiling, observation and discourse
analysis were used to design a personality for a chat bot for use in military
settings.
Keywords: User centred design Conversational user interface Chat bot
Personality testing NEO PI-3 Human factors Botanality
1 Why Do We Need a Military Chat Bot?
Military command is traditionally a human-dominated area. Commanders convey high-
level goals to their staff officer peer group who analyse data from a variety of sources
and collate and convey information to the commander in a variety of documents,
spreadsheets and briefings which commanders use to inform their decisions. There is an
increasing desire to use AI algorithms to process and evaluate patterns in large data sets
repetitively. To encourage trust, AI algorithms must be transparent, (i.e. they must
show their workings) and must also express insights in formats that explicitly show
how they use data to come to a prediction, classification, or decision. Our solution is to
provide the capability to engage the AI in a dialogue using natural language through a
conversational user interface. We don’t always understand how other humans arrive at
decisions but we can ask them. Similarly, a commander would be able to interrogate a
military chat-bot (MCB) that fulfills the role of one of their military advisors. A chat-
bot that could take information from a variety of sources (including other AI and
decision-support tools) and communicate outputs to the commander and team via its
conversational user interface would significantly improve the quality and pace of
decision-making in operational level planning. The first step is to scope and design the
MCB.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 986–991, 2020.
https://doi.org/10.1007/978-3-030-39512-4_150
Beauty Attracts the Eye but Character Captures the Heart 987
2 Why Does a Military Chat Bot Need a Rich Personality?
The goal of using conversational interfaces over other types of interface is to create
better interactions between humans and computer systems. For example, Siri and Alexa
are generalists who are ready to answer a wide range of questions on a wide range of
topics posed by a broad spectrum of users. Hence they are presented to the user as
omniscient black cylinder assistants with ordinary voices and generic ‘bot’ personali-
ties. Brands tend to be more creative with their chat bots and create conversations that
mirror their target customer’s ideal personality. For example a customer’s interaction
with a teen beauty brand’s chat bot (e.g. CoverGirl’s Kalanibot) is a very different
experience to interacting with a supermarket’s chat bot (e.g. Lidl’s Margot). The idea
behind designing a personality for a chat bot is to personalise these conversations so
they are more life-like, intimate, and representative of human interaction. We want our
end users to trust and continue to engage with the underlying tool. A rich personality is
more relatable, believable, and relevant to operational users. The ability to converse in
the way they would interact with their friends or colleagues is the key to creating a
positive experience and continued engagement.
Since the depth and breadth of knowledge is dependent on the data it learns from,
an MCB could take on a variety of roles, for example Policy Advisor (POLAD) or
Stabilization Advisor (STABAD) etc. Furthermore, Military Operational Level
decision-making is now interagency and multinational, so a chat-bot could be designed
to be multilingual and multicultural. The MCB will fulfil the role of a specialist
member of the military staff officer peer group so we must create a personality for an
MCB that will reflect the style and expectations of the military peer group. To achieve
this, we will: (1) characterise a peer group of former operational level commanders and
staff officers to extrapolate user needs; (2) conduct a lexico-semantic analysis of a
transcript of a meeting conducted in a military setting to provide insight into military
diction that an MCB will need to understand; (3) construct a personality for MCB
based on the Big Five standard of personality.
3 Profiling End Users Using Personality Inventory
Twenty Male participants aged 45–59, (Mean age = 55) completed the NEO-PI-3. All
participants had previous military experience operating in command and control
environments in the Army, Navy and RAF.
Research was conducted in accordance with BPS Practice Guidelines compliant
with GDPR regulations. Data was collected with participants’ informed consent and
was only kept for the duration of the project. Participants had the right to withdraw
from the experiment and could request deletion of data at any time.
Responses were compared to a reference group named ‘UK Working Population’
(available from Hogrefe Ltd, 2015) to benchmark the characteristics of the surrogate
C2 peer group against this group.
The NEO-PI-3 [1] is a concise measure of the five major domains of personality:
988 H. Muncie
1. Neuroticism (sensitive/nervous vs. secure/confident) is the tendency to be prone to
psychological stress.
2. Extraversion (outgoing/energetic vs. solitary/reserved) measures energy, assertive-
ness, sociability and the tendency to seek out the company of others.
3. Openness (inventive/curious vs. consistent/cautious) reflects the degree of intel-
lectual curiosity, creativity and a preference for novelty and variety that a person
has.
4. Agreeableness (friendly/compassionate vs. challenging/detached) is a measure of a
person’s trusting and helpful nature, and whether a person is generally well-
tempered or not.
5. Conscientiousness (efficient/organized vs. easy-going/careless) measures tendency
to be organized and dependable, self-disciplined, dutiful and predictable.
Figure 1 shows the consolidated personality profile for an ex-military commander
(mean age 55).
Fig. 1. Military user group big 5 domain scores
The personality analysis shows that commanders are more stable, calm, secure and
confident when compared with UK working population. They enjoy interacting with
people, and are often perceived as full of energy. They tend to be enthusiastic,
assertive, influential, action-oriented individuals.
3.1 Personality Insights and Implications for MCB
We can extrapolate the needs of the group of end users from insights into their per-
sonality and design the MCB with those in mind. For example, commanders are direct
and trusting so communication with the MCB needs to be direct, not over-simplified
and not too complicated. The MCB needs to be able to provide the reasoning behind
the comments, conclusions and suggestions it makes because commanders will ques-
tion any assumptions. Commanders adopt a collaborative approach and work in a fast
paced team so the MCB must be able to keep up the pace by providing information
promptly. Commanders form a bridge between different personalities within a group
suggesting that developing a “quirky” MCB personality is less important than ensuring
that it is fast and reliable, providing the right information to meet the commanders’
needs at the right time every time.
Beauty Attracts the Eye but Character Captures the Heart 989
4 Analysing End User Discourse
Since conversation requires a shared language, context and understanding, the MCB
also needs to understand and convey appropriate meaning and emotion in a style and
language that is congruent with end users expectations. Conversation consists of large
amounts of unstructured data with multiple messages that must be modelled to enable
the Natural Language Processing (NLP) component of an MCB to extract relevant
pieces of information from an operator’s narrative. The military is considered as a
social group comprising different sub-groups that have their own technical vocabulary
and their own ways of expression. Therefore a lexico-semantic analysis of a transcript
of a typical military meeting was conducted. The units of analysis in lexical semantics
are lexical units which include words, sub-words or sub-units such as affixes and
compound words and phrases [2].
4.1 Insights into Military Discourse and Implications for MCB
Lexico-semantic analysis provided insight into diction, which is an element of style
that refers to the choice of words and how military personnel use language for a distinct
purpose and effect. We observed from the analysis that military language makes limited
use of 1st and 2nd person singular pronouns. Military language is conscious of rank
and position. It is formal, brief and precise. It makes use of coded expressions,
abbreviations, symbols, acronym, euphemism and slang. We also observed that
expletives can be used as an adjective, noun, verb, and suffix depending on the situation
and can be used for emphasis and to express frustration, despair and contempt. It is
imperative that the MCB is able to interpret human sentiment and emotions from
language used.
5 Introducing “Jbot” a Military Chat Bot
The recommendations from the studies on personality profiling and discourse analysis
were combined to develop personality guidelines for an MCB called “JBot”. Jbot is a
military chat bot who currently works as a specialist member of the military staff officer
peer group. Jbot’s goal is to reduce the time taken to process large volumes of infor-
mation and articulate responses to queries from end users. Jbot is able to extract
information from the narratives of multiple users and is able to send texts, images,
audios, links and messages in real time as responses to queries from users. Jbot faces
the daily challenge of working in a fast paced environment and is learning to under-
stand colloquialism, idiom, irony and hyperbole.
Jbot’s personality is based on personality characteristics of the end user groups
“ideal self ”.
1. High Openness: Jbot can be both focused on a single task i.e. data entry or open to
new ideas e.g. responding to queries and suggesting new ways of doing things.
2. Low Neuroticism: Jbot is professional and resilient.
990 H. Muncie
3. High Conscientiousness: Jbot will initially demand details in a certain order. Later
Jbot iterations could be more spontaneous and could follow the user in whatever
order they provide the information.
4. High Agreeable: Jbot is cooperative, polite and helpful.
5. High Extroverted: Jbot is sociable and assertive.
Jbot is friendly and supportive while remaining focussed on the task in-hand and is
not afraid to provide challenge when necessary.
Jbot is intended to eventually become a voice assistant so accent and male or
female tone will form part of the MCB’s identity. However, designing personality traits
depending on gender may reinforce gender stereotypes [3]. In this context, assigning
Jbot a gender may not be necessary. For example, Siri uses a female tone but when
asked “what gender are you” Siri replies “I exist beyond your human concept of
gender”. Siri then prompts the user to choose a different voice from the setting [4]. We
decided that Jbot should not conform to any gender. Preference for a male or female
tone will be tested with the end user group (Fig. 2).
Introvert Extravert
Closed Open
Hostile Agreeable
Stable Neurotic
Spontaneous Conscientious
Fig. 2. Jbot big 5 domain scores
6 Next Steps: Conversational Flow and End User Testing
The next step is to design a conversational flow that takes an end user from an initial
greeting through to a completed task and conversational scripts that reflect Jbot’s
personality. We will use a hybrid type of conversational user interface to avoid the risk
of the conversation drifting away from the end goals and to reduce the risk of users
becoming frustrated by senseless conversations. Jbot does not need to be able to engage
in small-talk with operators; however it would be difficult to create relevant responses
to open-ended questions if Jbot is not AI-based. The MCB architecture must therefore
be machine-learning driven to avoid the need to code all of the possible if-else con-
ditions for each possible state of the conversation. Instead, the model will learn the
actions based on the training data provided. Once implemented, feedback will be
collected from a group of end users. We are particularly interested in whether task
performance is improved by using a conversational user interface and whether end
users like Jbot’s personality.
Beauty Attracts the Eye but Character Captures the Heart 991
References
1. McCrae, R., Costa, P.T., Lord, W.: NEO Personality Inventory, Third Edition UK. Ref-ID:
U_NEO-PI-3_L / P-ID: U_NEO-PI-3 (2015)
2. Wikipedia: Lexical Semantics (2014). https://en.wikipedia.org/wiki/Lexical_semantics
3. Rawlinson, K.: Digital assistants like Siri and Alexa entrench gender biases, says UN (2019).
https://www.theguardian.com/technology/2019/may/22/digital-voice-assistants-siri-alexa-
gender-biases-unesco-says
4. Forbes Insights (2018). https://www.forbes.com/sites/insights-intelai/2018/09/21/alexa-and-
cortana-and-siri-get-ready-for-the-9-to-5/#7454f021a3c5
Integration of Humans in the Fallback Process
by a Machine in Fully Automated Railway
Operation
Bilal Üyümez(&) and Andreas Oetting
Institute for Railway Engineering, Technical University of Darmstadt,
Otto-Berndt-Straße 2, 64287 Darmstadt, Germany
[email protected]
Abstract. In highly automated railway operation, humans must still handle
complex operating situations, which cannot be yet handled by the technology.
To achieve this, staff must be involved in a timely manner in the so-called
fallback process. In this paper, different request techniques for involving the
staff are investigated and the involvement time is derived analytically. In highly
automated railway operation, the kinesthetic and the auditory request techniques
are well suitable for involving the staff in the fallback process. Two equations
for the involvement time were derived, one for the remote control and one for
the train staff. Both times can be expressed with a piecewise-defined function.
Keywords: Human–machine-cooperation Human-systems integration
Automated railway operation Fallback level
1 Introduction
1.1 Railway as a Centrally Controlled Transport System
When automating existing traffic systems, the system-typical characteristics must be
considered. Road traffic is characterised by open access and by vehicles with smaller
vessel sizes and without centrally controlled road networks. A comprehensive set of
rules and regulations (road traffic regulations) exists for this purpose, the adherence to
those rules and regulations is the responsibility of each driver.
Compared to road traffic, railway traffic is a centrally controlled and at the same
time a safety-critical transport system. Possible system failure results in enormous
consequences. Today, train journeys are carried out with the participation of several
staff members (e.g. train driver, traffic controller and dispatcher). A train driver is
responsible for conducting the train, while a traffic controller is responsible for con-
trolling and protecting the track elements required for train movement. A dispatcher
monitors the operation and intervenes in the event of any minor or major disruptions in
order to keep any deviations from the timetable to a minimum.
If disruptions occur during operation, the staff members act with fixed rules and
maintain the operation in the fallback level with reduced quality by applying substitute
procedures in order to guarantee the availability of the system. Therefore, when
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 992–998, 2020.
https://doi.org/10.1007/978-3-030-39512-4_151
Integration of Humans in the Fallback Process by a Machine 993
automating the railway operation, all these operational processes required for train
journey must be considered.
1.2 Automation in Railway Operation and Its Challenges
Increasing transport demand and limitations for transport infrastructure extension lead
to the indispensable need to increase capacity of the existing infrastructure. By
automating the railway operation, the capacity consumption per train can be reduced
due to increased accuracy of on-schedule operation by following a speed profile, which
is generated by considering speed restrictions, gradients and the timetable. Further-
more, by automatically making real-time decisions the energy consumption and carbon
emission can be reduced, while delivering an improved quality of services.
There are four levels of automation in railway traffic. The first level (GoA1)
describes complete manual operation. Level two (GoA2) corresponds to an autopilot.
Here, the automated system supports the driver by taking over the trip on demand. The
driver intervenes when required, especially in the event of an incident or disruption. In
the third automation level (GoA3), the train runs fully automatically, monitors the route
itself with sensors and holds the train at scheduled stations. There is at least one train
staff on-board, who mainly performs customer service tasks and can intervene in the
case of disruptions to the vehicle. The highest level of automation (GoA4) is based on
operation without on-board staff, in which the train is operated fully automatically in
regular operation and, in the event of disruptions, an operating staff intervenes via
remote control from the control centre or a stationary stand-by staff reacts to the
disruption on site [1].
First experiences in automated, driverless operation (GoA4) exist in simplified
railway systems (especially metro systems). Semi-automated operation (GoA2) is
currently the state of the art for mainline railways. The rapid development of digital-
isation in the transport sector and demographic and economic reasons are leading to an
intensification of research for GoA3 and GoA4 for mainline railways.
As automation increases, its limits must also be explored: In addition to costs,
humans must still handle complex situations that the technology does not know (yet).
As machines in extensively automated operation will to a certain extent exhibit human
capabilities (e.g. sensory, cognitive, communicative and motor skills), human-machine
cooperation (HMC) is gaining in importance, particularly in order to be able to
maintain operation in the event of disruptions. Before a human-machine cooperation is
established, it is essential to involve the respective cooperation partner (human) in a
timely manner.
The goal of this paper is to present a potential solution, how the machine in the
automated railway operation can involve the operation staff in a timely manner in the
fallback process and to deduce the time, needed for involving the cooperation partner.
994 B. Üyümez and A. Oetting
2 Human Interfaces to Environment and Current Take-over-
Request Systems in Highly Automated Cars
Humans as intelligent and especially flexible beings perceive their environment with
their sense organs. In the following step, the recorded information is processed in the
brain in order to be able to carry out an adequate action. To enable machines to
communicate with humans, they must address the human sensory organs.
There are various “request systems” currently being researched for highly auto-
mated driving in road traffic. These request systems mainly address the auditory, visual
and haptic sense of human beings [2]. Acoustic requests are used, to make the driver
aware of the take-over by different tones.
Visual channels are traditionally used in road traffic, to display vehicle conditions
on a cockpit control panel. In road traffic, there are experiments with LED arrays
around the driver, which are used for spatially oriented requests [2]. In a haptic request
method, the driver is made aware of the take-over by vibrations on the driver’s seat.
Finally, all alert types can be combined. The combination is recommended because
more information flow per time unit and simultaneous stimulation of several sensory
organs can lead to better response times for the drivers [2, 3].
3 Timely Involving of Staff for a HMC in Fully Automated
Railway Operation
3.1 Take-over Scenarios in Fully Automated Railway Operation
As machines are not yet able to cope with unexpected and complex situations com-
pletely independently and with human abilities, there will still be scenarios where the
human is needed as a fallback level and has to take over vehicle (operation) control
again or cooperate with the automated machine to cope the complex situation. From the
current point of view, the exact circumstances of take-over scenarios cannot be
predicted.
In general, two categories of scenarios can be defined in which staff intervention
makes sense in the future. First, the staff will always intervene in emergency situations
involving the evacuation of the train. Secondly, staff intervention also makes sense if an
unexpected disruption affects the train’s journey and the automatic system has reached
its limits or is currently not available. Three types of human-machine cooperation can
be realized, taking into account the actors in railway operation and the grade of
automation 3 and 4:
1. human-machine cooperation via remote control
2. human-machine cooperation with train staff
3. human-machine-human cooperation (staff in the control centre and train staff)
The first type is particularly suitable for GoA4 operation, as there is no train staff. In
GoA3 operation, however, all three types can be implemented. Nevertheless, for all
three types a timely involvement of the staff is necessary.
Integration of Humans in the Fallback Process by a Machine 995
3.2 The Suitability of Existing Take-over-Request Systems for Fully
Automated Railway Operation
The suitability of existing request techniques depends on the type of cooperation. Since
the workplace of an operating staff in the control centre is comparable to a cockpit in a
passenger car, all existing request techniques in highly automated road traffic can be
used for this type of cooperation. However, not all request techniques are suitable for
the involvement of train staff.
An acoustic request to the train staff is already present today in the form of a
specific tone, which can still be used in automated operation.
Due to the larger vehicle vessel and the continuous change of location of the train
staff, a visual request in railway traffic is less suitable. There are two reasons for this:
First, the visual request requires a line-of-sight contact between the visual signal
transmitter and the staff. Secondly, the staff would has to keep a continuous sight on the
visual signal transmitter in order to be able to intervene in a timely manner.
Another type of request is the haptic (tactile) request. A haptic request is a com-
munication of information about touches or vibrations. A vibration area in the train
would make it possible to implement the haptic request. Due to the continuous position
change of the train staff, the alert from the vibration area will not always reach the train
staff. Therefore, an acoustic request via a vibration module, which he is carrying, would
represent a feasible solution with little technical effort.
A particular characteristic of railway traffic is its safe system design, which ensures
that the system reaches always a safe state (fail-safe) in the event of a disruption. Since
the safe state in railway traffic is usually standstill and a prior movement causes this,
this dynamic transition (the emergency braking) can be used very well as information
for the train staff. The braking force can be interpreted as a kind of kinesthetic request.
By combining this request type with the existing acoustic request, a request system
with minimal technical and economical effort can be designed.
3.3 Modelling of the Involvement Time for a Remote Control
The duration of the driver’s take-over in highly automated road traffic is based on
several practical experiments, which were mostly carried out in driving simulators, and
depends primarily on the traffic scenarios currently available. In [4], a minimum
duration of 6 s is recommended for comfortable take-over of control by the driver. The
short take-over time in road traffic is mainly because the driver sits directly in cockpit.
A similar situation exists in railway traffic for remote control from the control centre.
The operating staff in the control centre is also located in an area assigned to him,
which corresponds to the small vessel size in the vehicle.
The involvement time tinv-RC for remote control can be interpreted as a function of
the human time components th and system time components tsys. Human time com-
ponents can be seperated into the time taken to recognize that a signal for a certain
action occurred and decide on the appropriate movement tr (reaction time) and the time
to become aware of the situation tsa and make the decision to take an action [5]. The
time to reach the safe state tfs and the delay time for the request transmission td to the
996 B. Üyümez and A. Oetting
operating staff, describe the system time components. For the analytical modelling of
the involvement time, the following assumptions are made:
• The automatic request is made simultaneously with the beginning of the emergency
braking.
• Staff intervention in the system is only permitted after the train has reached a
standstill.
• The time for understanding the situation begins immediately after the request and
after the reaction time.
Based on these assumptions, the involvement time, including the speed of the train
vt and its braking acceleration abr can be expressed with a piecewise-defined function:
tinvRC ¼ td þ tr þ ðvt =abr Þ; for tsa tfs : ð1:1Þ
tinvRC ¼ td + tr + tsa ; for tsa [ tfs : ð1:2Þ
The time until the train reaches a standstill (tfs) depends on its speed and on the
braking acceleration. Both parameters in the Eq. (1.1) are continuously known during
operation and have a constant character. In contrast, the human time components are
dynamic parameters and are influenced by several variables. The staff reaction time, for
example, depends both on the type of requesting technique and on his cognitive state.
The time to understand the situation is dependent on system consciousness, experience,
and is neglected in Eq. (1.1), as this is smaller than the time until the train reaches a
standstill.
Equation (1.2) refers to the case where a staff in the control centre needs longer to
understand the situation than the time it takes for the train to reach a standstill. In both
equations, the time required for involvement depends on the absolute value of the
summands. However, in Eq. (1.1) the system time component ðvt =abr Þ dominates,
while in Eq. (1.2) the human time component tsa has a decisive role. In worst case, if
the staff in the control centre starts to understand the present situation after the train
reaches a standstill, the involvement time increases and the Eq. (1.1) or (1.2) changes
as follows:
tinvRC ¼ td + tr þ ðvt =abr Þ + tsa : ð1:3Þ
3.4 Modelling of the Involvement Time for the Train Staff
In comparison to remote control from the control centre, the involvement time of the
train staff will be considerably longer. This is mainly due to the fact, that the train staff
takes care of train supervision and therefore constantly changes its location. The
involvement time of the train staff can still be interpreted as a function of the human
and system time components. While the system time component consist of the same
components as in 3.3, the human time component is extended by the component
“walking time to system tWalk”, which depends on the speed of the staff vh and the
Integration of Humans in the Fallback Process by a Machine 997
distance Δd to the system in the train. The assumptions from 3.3 are used to describe
the involvement time of train staff with a piecewise-defined function:
tinvTS = td + tr þ ðvt =abr Þ + tsa ; for twalk tfs : ð2:1Þ
tinvTS = td + tr þ ðDd/vh Þ + tsa ; for twalk [ tfs : ð2:2Þ
The summand tsa can be obtained in both Eqs. (2.1) and (2.2), as the train staff will
not understand the situation until the system is reached. Again, the involvement time
depends on the absolute heights of the summands in Eqs. (2.1) and (2.2). However, tfs
and tsa dominate in Eq. (2.1). While tfs has a constant character, tsa represents a
dynamic quantity.
Equation (2.2) is dominated by the human time components. To the time compo-
nent tsa, there is additionally twalk, which has a dynamic character and is influenced by
the length of the train as well as by the physical characteristics of the train staff and the
surrounding conditions in the train.
4 Summary
In order to fully exploit the potential of automated railway operation, it is essential to
involve staff in the fallback process. A successful human-machine cooperation begins
with the timely request of the respective cooperation partner.
In this paper, the current request techniques used in automated road traffic were
examined with regard to their suitability for automated railway operation. The inves-
tigation has shown that the use of the request technique depends on the type of
cooperation. In the case of remote control from the control centre, the staff can be
involved in the fallback process by means of the same request techniques used in
highly automated road traffic. However, in order to involve the train staff in the fallback
process, the kinesthetic (braking force effect) and the auditory request techniques have
proven to be particularly suitable.
The involvement time of the staff in the fallback process was modelled analytically
for the remote control and for involving the train staff. Both derived equations for the
time can be expressed with a piecewise-defined function. Empirical values for the
human time components are necessary to make a precise statement about the
involvement time of the staff in the fallback process.
References
1. smartRail 4.0: Background information on Automatic Train Operation (ATO) and Grades of
Automation (GoA) for smartrail 4.0. Technical report, smartRail (2018)
2. Müller, A., Ogrizek, M., Bier, L., Abendroth, B.: Design concept for a visual, vibrotactile and
acoustic take-over request in a conditional automated vehicle during non-driving-related tasks
(2018)
998 B. Üyümez and A. Oetting
3. Bazilinskyy, P., Petermeijer, S., Petrovych, V., Dodou, D., de Winter, J.: Take-over requests
in highly automated driving: a crowdsourcing multimedia survey on auditory, vibrotactile,
and visual displays. Transp. Res. Part F Traffic Psychol. Behav. 56, 82–98 (2018)
4. Gold, C., Dambock, D., Lorenz, L., Bengler, K.: “Take over!” how long does it take to get the
driver back into the loop? In: Proceedings of the Human Factors and Ergonomics Society
Annual Meeting (2013)
5. Bailey, R.: Human Performance Engineering: A Guide for System Designer. Bell Telephone
Laboratories, Murray Hill (1982)
Analysis of Facial Expressions Explain
Affective State and Trust-Based Decisions
During Interaction with Autonomy
Catherine Neubauer(&), Gregory Gremillion, Brandon S. Perelman,
Claire La Fleur, Jason S. Metcalfe, and Kristin E. Schaefer
CCDC U.S. Army Research Laboratory,
Human Research and Engineering Directorate, Aberdeen Proving Ground, USA
{catherine.e.neubauer2.civ,gregory.m.gremillion.civ,
brandon.s.perelman.civ,claire.g.lafleur.ctr,
jason.s.metcalfe.civ,kristin.e.schaefer.civ}@mail.mil
Abstract. Trust is a critical factor in the development and maintenance of
effective human-autonomy teams. As such, new processes are needed to classify
affective state change that could be related to either an accurate or a misaligned
change in trust that occurs during collaboration. The task for the current study
was a leader-follower, simulated driving task with two different types of driving
autonomy, and two different levels of reliability. Facial expression was evalu-
ated to gauge group differences in affect-based trust. Results indicated that the
participant sample was best described by four distinct group clusters who varied
in their level of subjective trust and facial expressivity.
Keywords: Human-autonomy teaming Trust-based decision making Affect-
based trust
1 Introduction
New assessment practices for evaluating trust in human-autonomy teams are currently
underway; however, traditional measurements are primarily associated with subjective
feedback [1] or behavioral/performance-based responses [2]. More recently, there has
been a push for integrating wearable sensing to identify psycho-physiological signal
differences during interactions [3]. For this work, we were interested in identifying
assessment metrics for evaluating affect-based trust within human-autonomy teams and
investigated the following two research aims: (Aim 1) quantify facial expressivity as a
means to infer changes in affect-based trust in response to the type and reliability of
driver autonomy, and (Aim 2) robustly model trust-based responses using multimodal
data streams relating to subjective response, individual difference clustering, and facial
expressivity.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 999–1006, 2020.
https://doi.org/10.1007/978-3-030-39512-4_152
1000 C. Neubauer et al.
2 Background
Negative emotional states, such as stress and fatigue, can often arise while driving. For
this reason, technologies relating to computer vision have long been employed for
enhancing safe driving through detection and alerting systems for fatigue [4] and
emotions relating to stress and impatience [5, 6]. However, it is possible that these
advances may result in poorer performance by decreasing task engagement [7] and
shifting the driver’s attention to personal discomfort and stress symptoms during full
vehicle automation [8]. Finally, a lack of user understanding may leave the human
feeling “out of control”, which may decrease situation awareness, and in turn, trust in
the system. A misaligned level of trust in the system (i.e., when expectations do not
match behaviors) typically results in unnecessary or preemptive human intervention,
essentially rendering the system ineffective [9]. Mitigating this miscalibrated trust [10]
requires accurate estimation of underlying traits and states that relate to trust, and trust-
related decisions. Many metrics to measure operator state exist and typically include
questionnaire assessment; however, these require a person to remember how they felt in
a given moment and may reflect subjective bias. Conversely, most of the research on
state detection has focused on fatigue and is limited to eye tracking and head move-
ments [4, 11] while few studies have assessed facial expressivity while driving. We
posit that these methods for measuring emotional response will provide additional
insight in understanding affect-based trust within human-autonomy teaming.
3 Method
3.1 Experimental Design
Participants operated a simulated vehicle on a two-lane closed-circuit roadway. They
made decisions as to whether or not to toggle driving authority to an autonomous
assistant, while following a lead vehicle. The design was a 2 type of autonomy (speed
control; full autonomy) x 2 reliability (good; bad) within subjects design. In addition,
participants completed one manual (no autonomy) drive (see [12] for full method and
task descriptions).
3.2 Participants
Twenty-four participants, ages 18–65 years with normal to corrected-to normal vision
were recruited. Only participants who had completed all sections of data collection
were included for this analysis, resulting in a sample size of 19 participants.
3.3 Measures
Demographics: Only age was reported due to the results of the cluster analysis.
Analysis of Facial Expressions Explain Affective State and Trust-Based Decisions 1001
Personality: The Big Five Inventory (BFI) [13] was used to index personality traits
relating to extraversion, openness to experience, agreeableness, conscientiousness, and
neuroticism.
Uncertainty: The Uncertainty Response Scale (URS) [14] predicted individual dif-
ferences in coping with uncertainty and has three subscales: Emotional Uncertainty
(EU), Desire for Change (DFC), and Cognitive Uncertainty (CU).
Trust: Two scales were used to measure trust – a four-item version of the Muir and
Moray trust scale [15] and the Checklist for Trust in Automation [16].
Workload: The NASA-Task Load Index (NASA-TLX) [17] was selected as a stan-
dard measure of subjective workload and for its wide use in human performance
research.
Stress Visual Analogue Scale (SVAS): A 1-item question was used to determine
subjective stress.
Analysis of Facial Expressivity: Facial expressions were continuously recorded via a
webcam mounted to the simulation screen and extracted through the OpenFace free-
ware [18]. Facial expressions relating to both positive and negative affect (i.e., hap-
piness, sadness, surprise, fear, anger and contempt) were extracted on a frame by
frame basis separately for each task, as well as single action unit (AU) evidence fol-
lowing FACS [19].
Clustering Approach: The model-based clustering process used here provides a data-
driven means to infer clusters of participants within datasets and provides an attractive
alternative to K-means and related clustering approaches since the analysis does not
require a priori knowledge of underlying groups, and can work with a relatively small
sample size, see [20]. Stepwise flexible mixture modeling (FMM) was implemented in
the R package Flexmix [21]. FMM fits a mixture of models to the data using expec-
tation maximization (E-M) iteratively by minimizing a criterion value. Stepwise FMM
permits the user to compare models containing a specified range of components on the
basis of Bayes Information Criterion (BIC), which balances the model fit against the
number of components. Clusters included data from the BFI, URS, age, and baseline
trust (both scales), NASA-TLX, and SVAS. Fixed variance in the process was set
relatively low (r = 0.1), and 200 E-M iterations were permitted to ensure convergence
to an acceptable solution.
4 Results and Discussion
4.1 Clustering
Results of the clustering analysis indicated that the participant sample was best
described by four distinct groups (Fig. 1a). Cluster 1 (black squares) were represen-
tative of the oldest participants with a high desire for change (DC), high openness,
extraversion, and conscientiousness and low neuroticism (N = 6). Cluster 2 (red dia-
monds) were relatively average on nearly all of the dimensions in the analysis (N = 6);
1002 C. Neubauer et al.
however, they did exhibit the highest EU in the sample, indicating a potential for
maladaptive anxiety responses to uncertainty. Cluster 3 (green circles) were relatively
average across most dimensions (N = 3). However, they were extremely low in CU
and conscientiousness and had high levels of agreeableness, which suggests a prefer-
ence for predictable, planned behavior but some willingness to give autonomy a
chance. Cluster 4 (blue solid diamonds) were the youngest in the sample and reported
extremely polarized scores (N = 4). They reported the lowest DC and EU scores,
indicating that they do not seek novelty and would not respond emotionally to
uncertainty. However, they reported the highest CU, indicating a preference for pre-
dictability and structure in uncertain conditions. This coincided with this cluster’s
neuroticism scores, which were the highest in the sample.
Fig. 1. (a) Z-scores for characteristic traits, averaged by cluster. Differences from baseline trust
as a function of cluster, autonomy reliability, and autonomy type using (b) the Checklist for Trust
in Automation and (c) the Muir and Moray trust questionnaire. Error bars are 95% confidence
intervals.
4.2 Effects of Trust
Subjective trust was measured via changes from the baseline condition for both trust
measures [15, 16] (see Fig. 1b and c). The Muir & Moray trust scale showed a sig-
nificant interaction between autonomy reliability and cluster, v2 (3) = 10.73, p = 0.013
and autonomy type and cluster, v2 (3) = 9.84, p = 0.020. The Checklist also showed a
significant interaction between autonomy type and cluster, v2 (3) = 11.67, p = 0.009,
but only a borderline significant interaction between autonomy reliability and cluster,
v2 (3) = 7.28, p = 0.064. All other predictors were non-significant, p’s > .10.
Results of these analyses show marked differences between the clusters. Cluster 1
showed higher trust in good autonomy but had less negative response to bad autonomy.
Cluster 2 had a stronger negative trust reaction to bad autonomy, especially when in the
full autonomy condition, which identifies maladaptive anxiety responses to uncertainty.
The patterns in trust-based response for Cluster 3 resulted in higher trust with good
autonomy but the results also show a trust degradation when the pattern of behavior
becomes less predictable, as in the bad full autonomy condition. Finally, Cluster 4
Analysis of Facial Expressions Explain Affective State and Trust-Based Decisions 1003
showed an impact of bad autonomy on trust degradation, which corresponds with the
high CU and neuroticism scores and general negative response to autonomy.
4.3 Mean Facial Expressivity
Analysis of mean facial expressivity, as a function of group cluster, was used to infer
changes in affect-based trust (see Fig. 2). Here, members in Cluster 1 had higher
average expression values than all other clusters, with significantly higher values of
happiness and contempt, which is likely due to their having the highest extraversion
scores. Given proneness to reliance on the automation and a willingness to accept and
trust autonomy, this result suggests trust for this cluster was miscalibrated in the
direction of over-trusting the automation. Cluster 2 had the strongest expressive
reactions to violations (e.g. specifically for emotions relating to anger, contempt, fear
and surprise), which is indicative of their potential for maladaptive anxiety responses to
uncertainty. This is supported in the trust data with the very large degradation in trust in
the low reliability, full autonomy condition and may be indicative of miscalibrated trust
in the form of under-trust. Cluster 3 yielded generally lower mean expression values
than other clusters, with significant differences in mean values of anger, fear, and
surprise than all other clusters, which in conjunction with their appropriately balanced
evaluations of trust towards automation suggest this group’s trust level was calibrated
best relative to the other clusters. Cluster 4 demonstrated the greatest changes in
expressivity, which comports with their proneness to stronger negative trust-based
reactions to bad autonomy and their relatively poor performance.
Fig. 2. Mean values of the six expressions for the four clusters. The rightmost (bold) values
correspond to the overall mean computed across conditions. Error bars are standard error.
Figure 3 illustrates relative levels of expressivity seen between human and
automation driving time and shows that Cluster 1 displayed higher magnitudes of
expressivity. Here, the positions of mean happiness and contempt for each cluster
1004 C. Neubauer et al.
indicated relatively even expressivity between instances of human and automated
driving, while the relative expressivity between human and automated driving appears
to vary significantly among the clusters for the other expressions. The expressions of
anger and fear, for members of Cluster 1, are above the 45° line, indicating a relatively
higher mean expressivity for human driving than automation driving. This conforms to
expectations that Cluster 1 would be most willing to accept and trust autonomy,
possibly to the point of over trust, yielding relatively higher levels of anger and fear
when in control of driving themselves than during automated driving. The position of
the cluster means for expression show generally greater similarity between Clusters 1
and 2 with higher expressivity, and greater similarity between Clusters 3 and 4 with
lower expressivity. However, the position of the cluster means for violation rates
instead show similarity between Clusters 1 and 4 with more frequent violations, and
similarity between Clusters 2 and 3 with less frequent violations, indicating that
expression responses are not directly tied to performance but are more likely moderated
by subject characteristics captured in their questionnaire responses, which delineate
them by cluster.
Fig. 3. Mean violation rate and facial expression value between automation (x-axis) and human
driving (y-axis) within clusters. Plot origin distance indicates the magnitude of expression.
Distances above or below the 45° line (dotted) indicates if expressivity was seen during human
(above) or automated driving (below). Major axes of the ellipses are respective standard errors for
the mean within conditions (shaded) and across driving conditions (solid outline).
5 Conclusions
In line with our goal of identifying additional affect-based trust metrics, results showed
that there are indeed different patterns of response by different groups of people and
that facial expressivity analysis can provide additional insights into elements of trust
calibration. In these cases, certain groups may be more prone to over-trust in autonomy
(as seen in Cluster 1) compared to a misaligned trust in autonomy from the start (as
Analysis of Facial Expressions Explain Affective State and Trust-Based Decisions 1005
seen in Cluster 4). A critical outcome from this work suggests that expressive response
is not uniformly related to performance or even to the interactions with the automation.
These responses instead form more consistent patterns when subjects are grouped
based on their characteristic traits. Although more work is needed, we believe that the
methods outlined above provide a way to systematically group, evaluate and eventually
predict individual behaviors relating to affect-based trust within human-autonomy
teams.
Acknowledgement. This research was supported by the Office of the Secretary of Defense
through the Autonomy Research Pilot Initiative under MIPR DWAM31168. The views and
conclusions of this document are those of the authors and should not be interpreted as repre-
senting the official policies, either expressed or implied, of the CCDC Army Research Laboratory
or US Government. The US Government is authorized to reproduce and distribute reprints for
Government purposes notwithstanding any copyright notation herein.
References
1. Schaefer, K.E.: Measuring trust in human robot interactions: development of the Trust
Perception Scale – HRI. In: Lawless, W., Mittu, R., Wagner, A., Sofge, D. (eds.) The
Intersection of Robust Intelligence (RI) and Trust in Autonomous Systems. Springer,
pp. 191–218 (2012)
2. Freedy, A., de Visser, E., Weltman, G., Coeyman, N.: Measurement of trust in human-robot
collaboration. In: Proceedings of the 2007 International Conference on Collaborative
Technologies and Systems, pp. 106–114. IEEE, Orlando (2007)
3. Marathe, A., Metcalfe, J., Lance, B.J., Lukos, J., Jangraw, D.C., Lai, K.T., Touryan, J.,
Stump, E., Sadler, B.M., Nothwang, W., McDowell, K.: The privileged sensing framework:
a principled approach to improved human-autonomy integration. Theor. Issues Ergonomics
Sci. 19, 283–320 (2018)
4. Dong, Y., Hu, Z., Uchimura, K., Murayama, N.: Driver inattention monitoring system for
intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12, 596–614 (2011)
5. Nass, C., Jonsson, I-M., Harris, H., Reaves, B., Endo, J., Brace, S., Takayama, L.: Improving
automotive safety by pairing driver emotion and car voice emotion. In: International
Conference on HCI (2005)
6. Lisetti, C.L., Nasoz, F.: Affective intelligent car interfaces with emotion recognition. In: 11th
International Conference on HCI (2005)
7. Neubauer, C., Matthews, G., Saxby, D.J.: The effects of cell phone use and automation on
driver performance and subjective state in simulated driving. In: Proceedings of the 56th
Annual HFES, Boston, Massachusetts (2012)
8. Neubauer, C., Matthews, G., Saxby, D.J.: Fatigue in the automated vehicle: Do games and
conversation distract or energize the driver? In: Proceedings of the 57th Annual HFES,
Chicago, Illinois (2014)
9. Sarter, N.B., Woods, D.D., Billings, C.E.: Automation surprises. In: Handbook of HFES, 2
edn., pp. 1926–1943 (1997)
10. Gu, H., Ji, Q.: An automated face reader for fatigue detection. In: FGR, pp. 111–116 (2004)
1006 C. Neubauer et al.
11. Zhang, Z., Zhang, J.: Driver fatigue detection based intelligent vehicle control. In: ICPR
2006: Proceedings of the 18th International Conference on Pattern Recognition, Washington,
DC (2006)
12. Metcalfe, J.S., et al.: Building a framework to manage trust in automation. In: Proceedings of
SPIE 10194, Micro- and Nanotechnology Sensors, Systems, and Applications IX (2017)
13. John, O.P., Donahue, E.M., Kentle, R.L.: The Big Five Inventory-Versions 4a and 54.
University of California, Berkeley, Institute of Personality and Social Research, Berkeley
(1991)
14. Greco, V., Roger, D.: Coping with uncertainty: the construction and validation of a new
measure. J. Personality Individ. Differ. 3, 519–534 (2001)
15. Muir, B.M., Moray, N.: Trust in automation Part II. Experimental studies of trust and human
intervention in a process control simulation. Ergonomics 39, 429–460 (1996)
16. Jian, J.Y., Bisantz, A.M., Drury, C.G.: Towards an empirically determined scale of trust in
computerized systems: distinguishing concepts and types of trust. In: Proceedings of the
42nd Annual Meeting of the HFES, Chicago, IL (1998)
17. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of
empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental
Workload, Amsterdam, North-Holland, pp. 239–250 (1988)
18. Baltrušaitis, T., Zadeh, A., Chong Lim, Y., Morency, L.P.: OpenFace 2.0: Facial Behavior
Analysis Toolkit. In: IEEE International Conference on Automatic Face and Gesture
Recognition (2018)
19. Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of
Facial Movement. Consulting Psychologists Press, Palo Alto (1978)
20. Perelman, B.S., Dorton, S., Harper, S.: Identifying consensus in heterogeneous multidis-
ciplinary professional teams. In: Proceedings of the 2019 IEEE Conference on Cognitive and
Computational Aspects of Situation Management (CogSIMA), Las Vegas, NV (2019)
21. Gruen, B., Leisch, F.: Fitting finite mixture of generalized linear regressions in R. Computat.
Stat. Data Anal. 51, 5247–5252 (2007)
Let’s Get in Touch Again: Tangible AI
and Tangible XR for a More Tangible,
Balanced Human Systems Integration
Frank Flemisch1,2(&), Konrad Bielecki1, Daniel López Hernández1,
Ronald Meyer2, Ralph Baier2, Nicolas Daniel Herzberger2,
and Joscha Wasser1
1
Fraunhofer FKIE, Fraunhofersraße 20, 53347 Wachtberg, Germany
{frank.flemisch,konrad.bielecki,daniel.lopez.hernandez,
joscha.wasser}@fkie.fraunofer.de
2
Institut für Arbeitswissenschaft (IAW), RWTH Aachen, Bergdriesch 27,
52062 Aachen, Germany
{f.flemisch,r.meyer,r.baier,
n.herzberger}@iaw.rwth-aachen.de
Abstract. One of the crucial tasks for integrating humans, technical systems and
organizations (Human Systems Integration HSI) is the proper design of coop-
eration and interaction in the socio-cyber-physical system with human interaction
resources and modalities. While at the beginning of the 21st century, the visual
and acoustic resource seemed to be dominant in HSI, the haptic resource, which
for the past decades was increasingly neglected, is now on the resurge again.
With increasing AI in human-machine systems, and with increasing virtual and
augmented reality, it becomes even more important to use the haptic channel to
make these complex systems tangible and to allow a better and faster balance
between the interaction modalities, and between humans and machines.
Keywords: Human-systems integration Artificial intelligence Automation
Tangible Haptics Multi-modal VR AR XR Mixed reality
1 Introduction
If we look at homo sapiens from an evolutionary perspective, what makes this species
unique? It is not the eyesight or the ability to hear or to smell: There are many species
with higher abilities in these modalities. What makes homo sapiens unique are the
hands with its very versatile thumbs and high haptic sensivity, which he and she uses in
connection with eyesight and cognitive abilities to form and use tools. Since prehistoric
time, humans manufactured and used tools (e.g. the spears of Schoenigen, with ca.
270.000 years the oldest wooden artefact found so far on planet Earth) that not only
allowed them to perform their task more efficiently but also led, together with the
ability to cooperate, to higher cognitive abilities like preplanning and communicating
future actions [1].
After thousands of years of manually making tools to extend our physical capac-
ities, we are now developing and using tools to extend also our cognitive capabilities.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1007–1013, 2020.
https://doi.org/10.1007/978-3-030-39512-4_153
1008 F. Flemisch et al.
Due to increasing sensor and computer capacity, and an increasing knowledge about
control, communication and feedback loops in nature, animals and machines starting
e.g. with [2], more automated and AI-based technologies and systems become possible.
The role of humans in different processes, such as in aviation or in the manufacturing
industry, are more and more changing to monitoring and supervising than directly
interfering in the activity by controlling mechanics of the system. In short, we can also
speak about a loss of feeling our systems, which can also bring new risks e.g. of sliding
into an unsafe valley of automation (e.g. [3]), which should be bridged with a better
tangible shared and cooperative control (e.g. [4]).
Besides AI, there is also a strong trend since decades to do more and more virtually,
e.g. in the development of human machine systems like cars, trucks or tanks, or to
augment reality through virtual information e.g. in the field of view. These AR (aug-
mented reality) and VR (virtual reality) techniques are more and more blended as
Mixed Reality or XR technologies. These technologies cannot only enhance human
perception but can allow the user to mentally immerse into other environments.
Nevertheless, these technologies have certain limitations, mainly due to them being
heavily visual. A way of re-balancing the utilized modalities is as previously alluded to
the use of the haptic channel in form of tangible elements. A more intense overview of
this renaissance towards Tangibility is given by [5].
2 Sketch of the Design Space of Tangible AI and XR
Taking the definition from a dictionary, tangibility refers to elements that possess the
property of being able to be touched. Extending this idea further in the context of AI or
automated systems, tangibility allows the user to experience and feel those systems.
This can be achieved by the use of haptic elements such as force, vibration or
temperature.
According to the multiple resource theory, two tasks can be performed simulta-
neously only if they do not have to share the same interaction resource. The original
theory by Wickens et al. describes only visual and acoustic resources, but in early
publications he already hinted that the interplay with haptics could be an interesting
field of research. In the detailed research on haptic-multimodal coupling, e.g. described
in [6], the benefit of parallel processing of visual, acoustic and haptic was shown,
justifying the extension of to the multiple resource theory to three or more channels
(Fig. 1) [7].
One of the findings from the multiple resource theory is that tasks can be performed
more resilient if they are distributed among the different modalities. In HSI, especially
for the interplay of automation or AI with the human and with any mixture of aug-
mented and virtual reality, an increased tangibility with a good haptic interaction,
combined with visuals and acoustics towards a well-balanced multi-modal interaction,
can make a big difference in the acceptance, performance and safety of the overall
system. As an example, in the case of driving a vehicle, the driving task can be
performed more efficiently if extra information, e.g. traffic changes, fluid levels, etc. are
transmitted using channels that are less saturated.
Let’s Get in Touch Again: Tangible AI and Tangible XR 1009
Fig. 1. Resource theory of Wickens (2002), extended by Flemisch et al. [7]
Figure 2 (left) is a simplified sketch of the design space for tangible AI. Both AI
and human perceive the world, have cognitive processes, form their intentions, act on
the world and interact with each other. In tangible AI, at least part of the interaction is
via a haptic interface, which is ideally bi-directional. An example for this tangible
interface is a haptic sidestick, but also a haptic interaction on a steering wheel, gas
pedal or haptic mouse. For a more detailed description of such a shared and cooperative
control between AI and humans can be found in (Flemisch et al. [9]).
Mental IntegraƟon
IntenƟon
HapƟc AcƟon
InteracƟon SimulaƟon
Interface InteracƟon
Hap
p c
Interface
IntenƟon
Fig. 2. Left: system model of Tangible AI, Right: system model of Tangible XR
Figure 2 Right shows a simplified design space for tangible XR: The world is now
generated with a simulation, which can be perceived with the visual and acoustic
interaction channel, and via one or more tangible devices also on the haptic channel. If
the tangible XR is done properly, the human can mentally integrate all interaction. This
model can also be extended to an AR application, where part of the interaction is with
the real world, and another part with an augmentation generated by an additional
simulation or augmentation, which could also be AI-based.
1010 F. Flemisch et al.
3 Example: Tangible AI for Cars and Trucks
An example for Tangible AI can be found in the domain of cars and trucks, e.g. in the
seamless and intuitive transition between different levels of automation [8] in partially
and highly automated vehicles [9]. It is particularly relevant in concepts such as
H-Mode that depend heavily on the haptic interaction between the automation and the
driver. The H-Metaphor [10] describes the cooperation between a driver and the
automation using the example of a rider and its horse. In the horse-rider system,
the rider can vary the level of dependency of the horse by manipulating the reins. In the
driver-vehicle system, the driver can transition between different levels of automation
by changing his grip force on the steering wheel. The feedback the driver receives from
the steering wheel not only gives him information about the state of the vehicle but can
also serve as a link to communicate the actions, intentions and state of the automation.
An example could be harsh steering movements, which indicate that the automation is
at its limits of control of the vehicle. This could lead the driver to take over control. On
the other hand, the vehicle, like in the metaphoric example the horse, has its own
recognition capabilities to percept the environment. Combined with the perception of
its driver, the vehicle is able to find an intelligent solution for the movement – i.e. risk
avoidance. This solution can easily be communicated to and arbitrated with the driver
via haptic feedback [11] (Fig. 3).
Human initiates transition
automated
Manual
Secured
Fully
Tight Rein Loose Rein
Rein
Automation initiates transition
Fig. 3. Left: transition between different levels of automation according to H-Mode. Right:
force sensitive steering wheel [13].
In addition to initiating possible transitions, haptic interaction between driver and
automation has the potential to be used for vehicle guidance in partially or highly
automated driving. Here, the focus of different research approaches is on gesture-based
or alternative controls of the driving task (e.g. [14], [15], and [16]). For example,
maneuvers planned by the AI as well as possible alternative maneuvers could be shown
in the head-up-display and selected by the driver using gestures. Such disruptive
interaction concepts could support unexperienced, elderly or physical impaired drivers.
Let’s Get in Touch Again: Tangible AI and Tangible XR 1011
4 Example: Tangible XR for the HSI of Armored Vehicles
An initial concept of human systems integration with a tangible XR application has
been implemented in a defense project EnUSi, which deals with innovative vision
systems in armored vehicles. With this tangible XR Simulator, questions about the
ergonomic design and use of vision systems (camera monitor systems) were investi-
gated by constructing a simplified driver workplace mock-up. This mock-up is based
on modular construction which can be adapted to reflect the configurations of different
German military vehicles. In order to create a higher degree of immersion, the basic
control interfaces exist in both, the real and virtual world, e.g. steering wheel, flap
mirrors or control unit. In addition to the matching physical presence and position, it is
important that the elements are also identical in shape and appearance. This allows the
user to physically touch and interact with the elements he perceives in the simulation.
Figure 4 shows the physical buildup of the simulator, were the user is interacting with a
haptic element. Besides the tangible interface, the degree of immersion is also increased
with the use of modern hardware tools that allow an easier interaction and movement in
the virtual and real world and increases the mental integration. These hardware ele-
ments include VR-glasses, hand and body tracking or motion platforms.
Fig. 4. Left: user interacting in the hardware mock-up with mirror replicates as tangible
interface [17]; Right: user interacting in the VR simulation.
By including soldiers in a participatory design process, this simulator has the ability
to quickly evaluate new systems since the modular design of the hardware and software
allows for a faster implementation. In addition, previous research has shown that higher
immersion is directly related to an increase in performance in the virtual environment
[18, 19], opening up also a potential for training.
5 Outlook
Tangible AI has the potential to make AI easier to grasp and interact with. Tangible XR
has a great potential to increase the immersion into virtual and/or augmented envi-
ronments. It further has the potential to reduce the time and cost investments required
to evaluate systems early on in the development process. With this fascinating research
1012 F. Flemisch et al.
and development direction, a couple of issues and questions are still open: Which level
of immersion is required to provide research outputs comparable with using real world
setups? Can the increase of immersion through haptic and tangible interfaces be
quantified or is a qualitative improvement sufficient? How do we handle the limitation
of Tangible XR as evaluation method which exclude special user groups like elderly or
physically impairments, as they may be unable to interact with the syste? Lastly the
trade-off between higher immersivness vs. agility in the design and engineering process
needs to be closely monitored, as constructing tangible props may increase the
immersion but limit the ability to be reactive to design changes. Besides these open
questions, this fascinating technology and its integration into Human Systems Inte-
gration clearly deserves more research and development.
References
1. Tomasello, M.: A Natural History of Human Thinking. Harvard University Press,
Cambridge (2014)
2. Wiener, N.: Cybernetics: or Control and Communication in the Animal and the Machine,
2nd edn. The Massachusetts Institute of Technology, Cambridge (1961)
3. Flemisch, F., Altendorf, E., Canpolat, Y., Weßel, G., Baltzer, M., Lopez, D., Herzberger, N.
D., Voß, G.M.I., Schwalm, M., Schutte, P.: Uncanny and unsafe valley of assistance and
automation: first sketch and application to vehicle automation advances in ergonomic design
of systems, products and processes. Springer, Heidelberg, pp. 319–334 (2016)
4. Flemisch, F., Abbink, D., Itoh, M., Pacaux-Lemoine, M.-P., Wessel, G.: Shared control is
the sharp end of cooperation: towards a common framework of joint action, shared control
and human machine cooperation. IFAC-PapersOnLine 49, 72–77 (2016)
5. Boy, G.A.: Tangible Interactive Systems. Springer, Cham (2018)
6. Altendorf, E., Baltzer, M., Heesen, M., Kienle, M., Weißgerber, T., Flemisch, F.: H-Mode, a
haptic-multimodal interaction concept for cooperative guidance and control of partially and
highly automated vehicles. Springer (2016)
7. Flemisch, F., Baltzer, M., Altendorf, E., Maier, S.: Human Systems Integration (lecture
script). RWTH Aachen University, Aachen (2012)
8. SAE International: Taxonomy and definitions for terms related to driving automation
systems for on-road motor vehicles. Surface Vehicle Recommended Practice, January 2014
9. Flemisch, F.O., Schwalm, M., Meyer, R., Altendorf, E., Lennartz, T., Schreck, C.,
Bavendiek, J., Herzberger, N.D.: Human system integration at system limits and system
failure of cooperatively interacting automobiles: concept and first results. IFAC-
PapersOnLine 52, 93–98 (2019)
10. Flemisch, F.O., Adams, C.A., Conway, S.R., Goodrich, K.H., Palmer, M.T., Schutte, P.C.:
The H-metaphor as a guideline for vehicle automation and interaction. Technical report
No. NASA/TM—2003-212672. NASA, Langley Research Center, Hampton (2003)
11. Altendorf, E., Baltzer, M., Heesen, M., Kienle, M., Weissgerber, T., Flemisch, F.: H-Mode
2D - a haptic-multimodal interaction concept for cooperative guidance and control of
partially and highly automated vehicles. In: Winner, H., Hakuli, S., Lotz, F., Singer, C.:
(eds.) Handbook of Driver Assistance Systems. Springer, Wiesbaden (2016)
12. López Hernández, D.: Design and construction of a grip force sensitive steering wheel.
Master thesis, RWTH Aachen University (2015)
Let’s Get in Touch Again: Tangible AI and Tangible XR 1013
13. Meyer, R., von Spee, R.G., Altendorf, E., Flemisch, F.O.: Gesture-based vehicle control in
partially and highly automated driving: a pilot study. In: International Conference on
Universal Access in Human-Computer Interaction. Springer, pp. 216–227 (2018)
14. Angelini, L., Caon, M., Carrino, F., Carrino, S., Lalanne, D., Khaled, O.A., Mugellini, E.:
Wheelsense: enabling tangible gestures on the steering wheel for in-car natural interaction.
In: International Conference on Human-Computer Interaction. Springer, pp. 531–540 (2013)
15. Saupp, L., Schwarz, B., Schwalm, M., Eckstein, L.: Evaluation of active drivesticks as
alternative controls in a high fidelity driving simulator. Automot. Engine Technol. 4, 37–44
(2019)
16. Bielecki, K., Bloch, M., Schmidt, R., Hernàndez, D.L., Baltzer, M.C.A, Flemisch, F.:
Tangible virtual reality in a multi-user environment. In: 11th International ACM Conference
on Automotive User Interfaces and Interactive Vehicular Applications, Utrecht (2019)
17. Roberts, D., Wolff, R., Otto, O., Steed, A.: Constructing a gazebo: supporting teamwork in a
tightly coupled, distributed task in virtual reality. Presence: Teleoperators Virtual Environ.
12, 644–657 (2003)
18. Narayan, M., Waugh, L., Zhang, X, Bafna, P., Bowman, D.: Quantifying the benefits of
immersion for collaboration in virtual environments. In: Proceedings of the ACM
Symposium on Virtual Reality Software and Technology, pp. 78–81. ACM, New York
(2005)
Time Line Based Tasking Concept for MUM-T
Mission Planning with Multiple Delegation
Levels
Felix Heilemann(&) and Axel Schulte
Universität der Bundeswehr München, Werner-Heisenberg-Weg 39,
85577 Neubiberg, Germany
{felix.heilemann,axel.schulte}@unibw.de
Abstract. The objective of this article is to define a cockpit interface for the
guidance of several unmanned aerial vehicles (UAV) in Manned-Unmanned
Teaming (MUM-T) missions. Besides his flying tasks, the pilot is responsible
for the management of the UAVs in such scenarios. Based on a work system
analysis the system participants and two modes of automation, delegation and
assistance, were derived. For the delegation we define which types of tasks exist,
how they can be selected, parameterized and delegated on a time line. Subse-
quently, we present the cooperation of the system in case of missing tasks,
conflicts and suboptimal plans, as well as their visualization on the time line.
The presented concepts are fully implemented on the IFS Jet Simulator and will
be validated experimentally with pilots of the German Air Force at the end of the
year.
Keywords: Manned-Unmanned Teaming Mixed initiative planning
Delegation level UAV interaction concept
1 Introduction
The mission planning of multiple unmanned aerial vehicles (UAV) in dynamic mission
scenarios by a single pilot is a highly relevant field of research [1, 2]. In order to cope
with the resulting higher task load, (partially) automated mission planning is indis-
pensable. An issue of automated mission planners is the increase of automation induced
errors, such as loss of situational awareness, complacency or opacity [3]. In a previous
study with German Air Force pilots such complacency effects could be temporarily
observed in the UAV guidance at a very high level of automation. This study also
revealed that on the one hand the high degree of automation was appropriate in
demanding situations, but on the other hand more controllability of the UAVs was
desired in less demanding situations [4]. These identified assistance and delegation
requirements were then used in a top-level system consideration which derived the
work system, depicted in Fig. 1, and is inspired by the design patterns of [5].
According to the theory of work systems, each of these human-agent or agent-agent
relationships is considered and defined in detail. Besides the direct control of his own
aircraft, the hierarchical relationship (green connector) between the pilot and the UAVs
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1014–1020, 2020.
https://doi.org/10.1007/978-3-030-39512-4_154
Time Line Based Tasking Concept for MUM-T Mission Planning 1015
Fig. 1. Manned-Unmanned Teaming work system
enables the task delegation to the team members. During the planning and mission
execution the assistant system (AS), standing in a heterarchical relationship (blue
connector), monitors the pilot and intervenes in the case of overload or errors. These
so-called dual modes of cognitive automation are already used in several areas of UAV
guidance [4, 6] and require a special Human-Machine Interface (HMI). In previous
research, a prototype HMI with several delegation levels for the UAV guidance was
rated positively, however the separation of the mission plan and task delegation in this
interface made it difficult to coordinate tasks precisely [7]. This contribution describes
the revised HMI (orange line), which facilitates the dual modes of automation with the
help of a time line. In this context, we first discuss the required delegation and then the
cooperation interface of the system.
2 Interaction Concept: Delegation
This chapter discusses the hierarchical relationship between the pilot and the UAVs. In
this context, the task selection, parameterization as well as the delegation of these tasks
at different delegation levels to the team members are described. This interaction takes
place via the multifunctional display of our experimental fighter cockpit, shown in
Fig. 2. In this interface, the pilot selects specific pages on both sides of the interface (a).
In the center of the figure, the team consisting of the manned fighter (b) and the
unmanned vehicles (c) are shown. The task delegation process works as follows: First,
the pilot selects a specific task for the different types of targets (d, e, f) through the
Fig. 2. Tactical map with radial context menu, task parameterization and time line delegation
page
1016 F. Heilemann and A. Schulte
radial context menu (g). The parameters of this task can then be adjusted with the
parameterization page (h). Finally, the task is integrated into the mission plan through
the delegation page (i). These steps are described in detail below.
2.1 Task Types and Selection
By selecting a target on the tactical display, the following different types of tasks can be
created with the radial context menu:
• Task: This task can be executed by a single manned or unmanned aircraft, e.g. the
reconnaissance of a target (Fig. 3a)
• Team Task: In order to perform this task, several team members must cooperate
closely with each other, e.g. the coordinated attack of a target (Fig. 3b).
• Swarm Task: In this case, a team member exposes a swarm of several unmanned
aerial vehicles, e.g. the reconnaissance of an area (Fig. 3c)
The selectable tasks for the different task types depend on the context of the target,
as described in [8].
Fig. 3. Task types and example tasks for air interdiction targets
2.2 Task Parametrization
The chosen tasks are parameterized based on the environment, rules of engagement and
strategy. These task specific parameters can also be adapted by the pilot via the
parametrization page, shown in Fig. 4. For a Reconnaissance Task (a), it is possible to
specify a special tactic, sensor or pattern. The Team Task (b) allows to modify the most
important standard parameters of the subtasks (e.g. number of reconnaissance images,
armament) as well as the time conditions between the subtasks. Additionally it is
possible to exclude unwanted subtasks from the planning. For a swarm task (c) the
swarm release distance (dynamic, maximum range) as well as the number of used
remote carriers can be specified. Each of the presented tasks can be assigned a specific
start time (Time over Target - TOT). For Team Tasks, this always refers to the first task
performed.
Time Line Based Tasking Concept for MUM-T Mission Planning 1017
Fig. 4. Task parametrization page
2.3 Task Delegation
After the selection and parameterization of the tasks, these are delegated via a com-
bined time line and tasking view. Hereby the following delegation levels, shown in
Fig. 5, are available:
Fig. 5. Time line delegation page
Package Delegation (Fig. 5a): The mission planner determines the best team members
and inserts the task at the best position in the plan. This delegation level is available for
all types of tasks (team, swarm, single). In case of a team task delegation, the mission
planner integrates the corresponding subtasks into the plan.
Individual Delegation: Here a single or swarm task is assigned to a specific team
member This delegation can take place on the following levels:
• Floating (Fig. 5b): The pilot specifies the vehicle and the mission planner deter-
mines the best position for the task in the task list of the selected member.
• Relative (Fig. 5c): The pilot specifies the relative position of the delegated task in
the vehicle’s current task list. The mission planner then adjusts the start times of the
dependent tasks and generates a new plan.
1018 F. Heilemann and A. Schulte
• Time Over Target: Provided the pilot has specified a special TOT on the parameter
page, the task will be inserted at exactly this time. Alternatively, the pilot can
directly insert the task in the time line and drag it to the appropriate time.
The combined tasking and delegation page gives the pilot direct feedback on the
impact of each planning step on the resulting mission plan. The visualization of the task
start times and durations makes it easy to estimate which team member still has
reserves in its timetable and is suitable for possible future tasks.
3 Interaction Concept: Assistance
This section discusses the heterarchical relationship between the assistant system and
the pilot. First, the behaviour of the AS in case of missing or new targets will be
discussed. Then, we present how planning errors can be communicated to the pilot.
3.1 Missing Tasks
The interaction concept for missing tasks is based on the basic requirements for
assistant systems according to [9]. Figure 6a illustrates a possibility for visualizing the
first basic requirement, by providing a simple hint. A simplification of the task, second
basic requirement, is presented in Fig. 6b. Here the pilot can accept the proposal of the
AS and the task is inserted into the plan at the yellow marked position. Figure 6c shows
the implementation of the third basic requirement. Here the pilot is targeted by an
enemy radar and a rocket has already been fired at him. In order to prevent the shot
down of the pilot, the AS automatically delegates the jamming of the enemy radars to a
UAV. The pilot can, however, revise this decision at any time by pressing the Cancel
button.
Fig. 6. Intervention types for a missing task
Time Line Based Tasking Concept for MUM-T Mission Planning 1019
3.2 Improvements and Conflicts
The presented delegation levels provide a wide range of planning options to the pilot.
As shown in Fig. 7, this can lead to the violation of planning constraints and subop-
timal plans. In (a) a constraint violation, the engagement of the high-value target before
reconnaissance, is shown. Chart (b) shows a plan improvement proposal that shortens
the total mission duration by 30 s by assigning task B31 to another team member.
Fig. 7. Conflicts (a) and improvements (b)
4 Conclusion
In this work, we presented an interaction concept for the guidance of several UAVs
from the cockpit. Based on a work system analysis we derived two modes of
automation, delegation and assistance, required in such MUM-T scenarios and
developed a user interaction concept for these modes. For the delegation mode, we
presented how different types of tasks can be selected, parametrized and delegated to
the UAVs with a time line. Subsequently, the cooperation possibilities of the system for
missing tasks as well as for plan improvements and conflicts were presented. The fully
implemented functionalities will be evaluated towards the end of the year with pilots of
the German Air Force in complex MUM-T scenarios.
References
1. Chen, J.Y.C., Barnes, M.J.: Human - agent teaming for multirobot control: a review of human
factors issues. IEEE Trans. Hum.-Mach. Syst. 44, 13–29 (2014)
2. Behymer, K., Rothwell, C., Ruff, H., Patzek, M., Calhoun, G., Draper, M., Douglass, S.,
Kingston, D., Lange, D.: Initial evaluation of the intelligent multi-uxv planner with adaptive
collaborative/control technologies (IMPACT) (2017)
3. Wiener, E.L., Curry, R.E.: Flight-deck automation: promises and problems. Ergonomics 23,
995–1011 (1980)
4. Gangl, S.: Kooperative Führung mehrerer unbemannter Luftfahrzeuge aus einem einsitzigen
Kampfflugzeug (2015)
5. Schulte, A., Donath, D.: A design and description method for human-autonomy teaming
systems. In: Advances in Intelligent Systems and Computing. Springer, pp. 3–9 (2018)
6. Heilemann, F., Schmitt, F., Schulte, A.: Mixed-initiative mission planning of multiple
UCAVs from aboard a single seat fighter aircraft. In: AIAA Scitech 2019 Forum. American
Institute of Aeronautics and Astronautics Inc. AIAA (2019)
1020 F. Heilemann and A. Schulte
7. Heilemann, F., Schulte, A.: Interaction concept for mixed-initiative mission planning on
multiple delegation levels in multi-UCAV fighter missions. In: Advances in Intelligent
Systems and Computing. Springer, pp. 699–705 (2019)
8. Lindner, S., Schwerd, S., Schulte, A.: Defining generic tasks to guide UAVs in a MUM-T
aerial combat environment. In: Advances in Intelligent Systems and Computing. Springer,
pp. 777–782 (2019)
9. Onken, R., Schulte, A.: System-Ergonomic Design of Cognitive Automation: Dual-Mode
Cognitive Design of Vehicle Guidance and Control Work Systems. Springer, Heidelberg
(2010)
Towards Cognitive Assistance and Teaming
in Aviation by Inferring Pilot’s Mental State
Nele Russwinkel1(&), Christoph Vernaleken2,
and Oliver W. Klaproth3
1
Chair of Cognitive Modeling in Dynamic Human-Machine Systems,
Technical University Berlin, Marchstr. 28, 10587 Berlin, Germany
[email protected]
2
Airbus Defence and Space GmbH, Rechliner Street,
85077 Manching, Germany
3
Airbus Central Research and Technology, Hein-Sass-Weg 22,
21129 Hamburg, Germany
Abstract. Most assistance systems do not consider the cognitive state of the
operator for providing support. However, especially in critical situations, it is
highly relevant to know whether information has been processed correctly or
been missed. We propose cognitive assistance and give an example of how
formalized understanding and anticipation of pilot behavior in response to flight
deck alerts can be implemented. The proposed cognitive assistance creates an
understanding of the global context of the operation, comprising pilot state
(cognitive, emotional), the environment (e.g. weather, traffic, obstacles), the
operational context (e.g. Air Traffic Control instructions, system status), and the
current task context. The concept of understanding others also provides the
foundation for mutual understanding in future human-machine teaming,
although additional aspects must be considered by such systems.
Keywords: Cognitive assistance Cognitive modelling ERPs Emotional
state Mental state Aviation Surprise
1 Introduction
Automation has dramatically changed within the past 50 years. The initial idea of
automation was to support human operators, to reduce workload and to reduce errors.
Unfortunately, however, human operators are presently forced to adapt their skills and
routines to the semi-autonomous systems [5]. This has a strong impact on the inter-
action between technical system and human operator and brings along various new
problems such as out of the loop issues, automation surprises, lack of situation
awareness [6] and the loss of agency in situation [3]. Humans perform much better in
certain situations than semi-autonomous systems. They can cope with new situations,
easily transfer knowledge to a new task, learn from single experiences and use a broad
range of pre-existing knowledge. Humans have limited cognitive capacities, but usually
a good idea of what features in a situation they need to attend to in order to proceed
with their tasks. Semi-autonomous systems typically have a limited scope of tasks, and
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1021–1027, 2020.
https://doi.org/10.1007/978-3-030-39512-4_155
1022 N. Russwinkel et al.
they are neither flexible in behavior nor do they ‘know’ how to cope with unknown
situations. For most cases, it is obvious that we need both the human operator – or
rather with the focus of this paper – the pilot and the semi-autonomous assistance
system1 because they both provide different capabilities that complement each other.
The imminent question at hand is how pilots and semi-autonomous assistance
systems can work together in a rather natural way, to their mutual benefit, without
imposing additional workload to the pilot.
Evidently, it is necessary to realize mutual understanding between pilots and semi-
autonomous assistance systems and to enable a basis for easy and partly implicit
communication to solve potential problems and misunderstandings.
Specifically, for critical situations, such as alerts provided by a Flight Deck Alerting
System, there are numerous documented cases [e.g. 1] in which pilots did not ade-
quately respond to the information provided by the system, and it is not always
immediately obvious why the machine-to-human communication broke down. Poten-
tial explanations range from “simply overhearing” the alerts due to workload and
attentional focus issues to an overall lack of trust in the respective system, leading to
pilots ignoring its advice. Therefore, it would be necessary to enable the assistance
system to understand whether the pilot did process the given information, and if not,
why this is the case.
Generally, there are individual differences between people, and the amount and
kind of assistance that is needed will differ to a large degree between human operators.
As an example, hints and usage advice that may be very welcome for a novice might
not only irritate experts, but also hamper their productivity. Furthermore, different
process preferences between human operators would also call for individual support.
The general task itself will be identical between human operators, but the human
individuals, the operational context and the environmental circumstances will differ.
These factors therefore need to be taken into consideration for an assistance system that
enables mutual understanding.
To maintain the human in-the-loop also means to understand how the situation is
understood by the pilot. Pilots do not only need to perceive and process information but
also needs to build up a mental representation (or model) of the current situation –
typically called situation awareness [6], and to anticipate the next events. This includes
awareness of the state of the semi-autonomous assistance system. These representations
provide an understandable rationale for the pilot’s decisions. Assistance systems that do
not anticipate the current understanding or representation of the pilot can cause severe
problems (e.g. silent mode changes). Furthermore, assistance should consider the pilots
understanding of the situation and just add information that is needed to complete
knowledge required to complete the task or address differences to the situation as the
system perceives it. Also, decisions or proposals performed by the system need to be
presented in a form to add information to the situation understanding of the pilot and to
support mutual communication and understanding.
1
In this paper, we use the example of a pilot assistance system, but the concept applies to assistance
and collaboration systems in other domains.
Towards Cognitive Assistance and Teaming in Aviation 1023
These arguments call for a novel approach towards assistance functionality. To
understand the pilot, behavior needs to be tracked and put in relation to a pilot model
that captures the cognitive requirements of the task, the task dynamics, the individual
characteristics of the pilot, the operational context and the environmental circum-
stances. A special focus should be set on a prediction and enrichment of the pilot’s
situation understanding. We propose a so-called cognitive assistance that relies on an
approach of understanding the pilot and builds up the cognitive state of the pilot, the
situation and environmental factors to provide assistance appropriate in the form, the
timing and content to the current pilot’s situation and mental state.
2 Cognitive Assistance
2.1 Aspects to Consider for Cognitive Assistance
For a cognitive assistance approach, as we suggest it in this paper, it is necessary to
consider several types of information and put these into relation to provide the basis to
gain a good understanding of the pilot in the task context, operational context and the
environment.
Fig. 1. Cognitive assistance - information that needs to be coordinated for providing appropriate
assistance for the pilot
The Environment is the actual environment in which the operation takes place and
includes weather, terrain, obstacles or traffic. All these environmental elements may
create essential boundary conditions or constraints for the operation, but they typically
cannot be influenced by the pilot.
1024 N. Russwinkel et al.
By contrast, Operational Context comprises systems, services and other stake-
holder support enabling the current mission, first and foremost the aircraft and its
systems. Obviously, any system failures will, depending on their criticality, more or
less immediately result in a need for pilot action. Additionally, Operational Context
also involves Air Traffic Control as well as requests by the airline and covers the
infrastructure (such as airports and navigational aids) and its status.
In contrast to the “environment”, which typically cannot be influenced by pilot or
assistance, interactions with the operational context are feasible. System failures can be
recovered, and it is entirely possible to request a different runway than the one currently
in operation.
In combination, Operational Context and Environment characterize the situation at
hand, and can immediately be used to derive a potential objective need for action.
Accordingly, Task Context encompasses the actual pilot tasks at hand, i.e. the specific
procedural steps that are currently taken in the cockpit, involving both normal proce-
dures such as cockpit preparation, as well as potentially abnormal procedures such as
handling an engine fire.
Evidently, the Operator States could be a result of changes in the task, the
operational context or in the environment. To gain a good understanding about the pilot
in the situation and anticipating the next steps or evaluate deviation actions we need a
detailed approach that also considers cognitive mechanisms. Cognitive Assistance must
therefore “understand” all the above aspects in order to assist pilots in judging the
(useful) options they have, and therefore corresponds to a meaningful course of action
analysis and trade-off, which in turn results in a specific proposal of assistance, which is
ideally supplemented by a rationale the pilot can understand.
2.2 Understanding Others
How can we achieve a concept of automation assistance or even teaming that is able to
support the pilot appropriately through understanding of arising problems or challenges
in the evolving situation?
There are numerous theoretical approaches on how people understand others, but
only few of these consider that humans are normally in an interactive situation and
have limited time.
Newen [9] proposes the Person Model Theory (PMT) that considers interactive
situations. The theory posits that we understand others to a large amount based on the
situation they are in, which is called a situation model. This situation model might
include environment factors as well as operational context and task context. This
situation model, as well as the interdependence of personal models (individual factors
such as emotions, preference and capabilities – similar to the operator state) can be used
to observe, explain, or predict the behavior of the other person. This way we can find an
explanation of e.g. situational emotions, intentions and actions.
The core idea of this paper is that implementing an analogous approach to
understand the pilot in a situation would provide a wide range of possibilities to design
appropriate assistance to the pilot. Such a cognitive assistance would rely on an
understanding of what kind of information the pilot needs, what information has been
processed and explain divergent behavior or anticipate a possibly dangerous situation.
Towards Cognitive Assistance and Teaming in Aviation 1025
To achieve a modelling approach to account for this, we need a dynamic cognitive
model of the pilot capable of tracing the pilot during a task, while interacting with the
technical system, conducting a specific task, including environment events, checking
for the reactions of the pilot and also enriching this model with measurable information
about the individual pilot, such as surprise reactions (e.g. via ERPs – Event Related
Potentials), visual attention (eyetracking) and actions (input to the technical device)
providing information e.g. about the underlying situational understanding.
2.3 Cognitive Modelling of the Cognitive State of the Pilot
A first approach in this direction has been realized based on a study about pilots in a
simulator that have to handle various alerts [7]. A cognitive modelling approach was
implemented with the cognitive architecture ACT-R [2] that captures in which situa-
tions what kind of pilot interactions will probably be completed. The result of these
action(s) will lead to a new situation including an altered visual display, for which the
model predicts what kind of interactions the pilot will initiate next until the task goal is
reached. The model integrates information about the real interactions of the pilot as
well as information from the technical system, therefore matching the predictions with
the real events. This modelling approach is not deterministic, different possible next
steps are considered depending on the circumstances (processed information, envi-
ronmental or operational cues) or preferences. In addition, ERPs are measured to
capture attentional reactions of the pilot. This information can be accessed by the pilot
model e.g. testing whether the pilot reacts to an alert and therefore has processed this
information in contrast to inattentional deafness [4]. Such additional information about
the pilot naturally increases the reliability of the model predictions.
Emotions are especially interesting on such a cognitive assistance approach.
Understanding the emotional reaction of the pilot to an event is highly relevant, both
for understanding among humans, but also for cognitive assistance. This provides
information about the current understanding of a situation, in particular if an event has
been expected or not, and whether the pilot still feels in control of the situation.
There is much potential in relating surprise reactions to the underlying mental
representation of situational understanding. The question is what kind of situational
understanding would cause such a surprise reaction, e.g. automation surprise. There
have been some approaches to model expectations and surprise with cognitive archi-
tectures [8]. This line of research holds large potential for future assistance systems.
3 Discussion
3.1 Outlook Cognitive Assistance
In the future, technical systems, especially autonomous systems, will most likely
continue to increase in complexity. Already today, several autonomous or semiau-
tonomous systems seem to be too complex for a human operator to fully grasp the
whole system. Our approach for cognitive assistance aims at enhancing the effective
1026 N. Russwinkel et al.
and efficient communication between operator and semi-autonomous system in order to
enable a human operator to stay in the loop and make well-informed decisions.
A first example assistance system [7] shows how a situation model of the pilot
including information about surprise reactions as the consequence of events can already
predict pilot performance to a large degree. The underlying cognitive model provides
the basis to compare situation understanding between technical system and the pilot. If
both are deviating critically, some form of communication has to be initiated, con-
sidering that both views could be at least partially wrong. By addressing the divergence
in human and system perception, critical situations might be prevented or solved. This
would refer to the assistance proposal in Fig. 1.
In addition, the situation model could be more detailed, and cognitive mechanisms,
such as mental simulation of different situational explanations to find the best fitting
model, would be a good mechanism to include in such systems.
3.2 What We Need for Human Machine Teaming
For human-machine teaming, a mutual understanding and enabling an implicit com-
munication about divergent ideas, or the understanding of a situation, is even more
important.
Especially the representation of “what changes can I (as the system or pilot) cause
in this situation and would this help for the common goal” (called self-model), is
fundamental for real collaboration between technical system and pilot. For this, a more
elaborate cognitive model of the pilot is needed, as well as some kind of “self-model of
the technical system. The self-model is a representation of what the technical system
itself can achieve in the present situation, including the possible course of action. This
aspect is more relevant for the teaming approach, where the technical system must
consider how it can support the pilot in terms of collaborating towards a specific goal,
which then results in a specific assistance proposal with associated rationale.
References
1. Air Accident Investigation & Aviation Safety Board (AAIASB): Aircraft accident report:
helios airways flight HCY522, Boeing 737-31S, at Grammatiko, Hellas, on August 14 2005,
accident investigation report 11/2006, Ministry of Transport and Communications, Hellenic
Republic, Athens (2006)
2. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated
theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004)
3. Berberian, B.: Man-machine teaming: a problem of agency. IFAC-PapersOnLine 51(34),
118–123 (2019)
4. Dehais, F., Roy, R.N., Scannella, S.: Inattentional deafness to auditory alarms: inter-
individual differences, electrophysiological signature and single trial classification. Behav.
Brain Res. 360, 51–59 (2019)
5. Dekker, S.W.A., Woods, D.D.: MABA-MABA or abracadabra? Progress on human–
automation coordination. Cogn. Technol. Work 4(4), 240–244 (2002)
6. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors 37
(1), 32–64 (1995)
Towards Cognitive Assistance and Teaming in Aviation 1027
7. Klaproth, O., Halbrügge, M., Russwinkel, N.: ACT-R model for cognitive assistance in
handling flight deck alerts. In Proceedings of the 17th International Conference on Cognitive
Modeling, Montreal (2019)
8. Lindner, S., Russwinkel, N.: Modeling of proximity-based expectations. In: Reitter, D., Ritter,
F.E., (eds.) Proceedings of the 14th International Conference on Cognitive Modeling (2016)
9. Newen, A.: Understanding Others: The Person Model Theory (2014)
Evaluating the Impact of Phone Interface
Modality on Response Times to Stimuli
in Conventional and Semi-automated Vehicles
Sky O. Eurich1, Shivangi Agarwal1, and Francesca Favaro1,2(&)
1
RiSA2S Research Center, San Jose State University, San Jose, CA, USA
{sky.eurich,shivangi.agarwal,
francesca.favaro}@sjsu.edu
2
Department of Aviation and Technology, San Jose State University, San Jose,
CA, USA
Abstract. This study investigates response times to stimuli within a driving
environment in the presence of secondary non-driving tasks involving cell-
phone usage by the driver. Two interaction modalities are employed for text
reply while at the wheel: manual text replies and vocal text replies. The study
evaluates the impact of the two modalities on throttle response time after a traffic
light turns green (city scenario), as well as steering, braking, and throttle
response times in a semi-automated vehicle that transitions from automated
driving to manual conventional driving (highway scenario). Both scenarios were
tested with the two investigated interfaces in a human-in-the-loop simulator.
Moreover, the paper includes an analysis of time-to-reply to message, and
analyzes different trends in texting behavior when the two interfaces are
employed.
Keywords: Response times Human-in-the-loop simulation
Disengagement Driver distraction
1 Introduction
The National Highway Traffic Safety Administration (NHTSA) estimates that 94% of
US crashes are imputable to human error [1], with distraction related to cell phone
usage in the top 3 causes of incidents on the road [2]. The use of portable devices has
also been in the spotlight in relation to recent crashes of semi-autonomous vehicles,
where current regulations mandate the driver to monitor the outside environment at all
times [3]. With the recent advances in autonomy and the inclusion of autonomous
features in everyday vehicles, it is more important than ever to assess the capability of
drivers to monitor the surrounding environment and whether or not they are capable of
taking over control when needed, while at the same time being engaged in activities
other than driving [4]. New technologies that leverage voice commands for the inter-
action with portable devices claim to achieve a “safety improvement” by allowing the
driver’s focus to be kept on the road [5, 6]. Novel frontiers of voice commands are now
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1028–1033, 2020.
https://doi.org/10.1007/978-3-030-39512-4_156
Evaluating the Impact of Phone Interface Modality 1029
becoming standards for vehicles deployed on the market: through Apple “carplay” a
smartphone can be integrated within the vehicle, and voice commands allow the driver
to respond to texts, provide desired destination to navigation applications, as well as set
music and radio stations [6]. While voice commands have been advertised as a safety
improvement for reducing distraction by tech companies [5], scientific evidence of
safety improvements is still lacking in the literature.
This project thus aims at quantifying possible safety improvement when different
interfaces with portable devices (i.e., use of voice instead of touch) are engaged while
driving. To this end, we investigate how response times to specific stimuli are affected
by the two interface modalities (i.e., manual texting, and voice replies) for both con-
ventionally driven vehicles as well as semi-automated ones in a car-integrated, human-
in-the-loop simulator.
The structure of this paper is as follows: Sect. 2 presents the experimental setup and
methodology for the study; Sect. 3 examines the collected data and analyzes the
results; Sect. 4 concludes this work.
2 Experiment Setup
This study employed human-in-the-loop (HITL) simulation (see Fig. 1) with the aim of
measuring how response times to specific stimuli are affected by the two interface
modalities (i.e., manual texting, and voice replies) for both conventionally driven
vehicles as well as semi-automated ones.
Fig. 1. The HITL static simulator employed in the study, owned by FKA Silicon Valley, and
simulator consisting of a BMW 6 series, a projection wall providing 220-degree horizontal front
view and a split rear-projection wall providing the projection for side and rear-view mirrors. The
simulation environment uses the Linux-based simulation framework Virtual Test Drive (VTD) by
Vires Simulations technologie GmbH in version 2.1.0. Open standards like OpenDRIVE® and
OpenSCENARIO were used for road and scenario creation
1030 S. O. Eurich et al.
In particular, we quantify the effect of the two interfaces on response times to
external stimuli in two operating scenarios: a city scenario, and a highway scenario. In
the city scenario, participants manually drove the car in conventional mode, following
GPS instructions in a suburban environment. A non-driving task related to texting of a
message would then be introduced in conjunction with a traffic light turning red. While
replying to the text the participant needed to stop at the red light; subsequently, after the
light became green, the participant needed to resume driving while a second text
message was received and needed reply. In the highway scenario, the car was driving
autonomously. The participant was still asked to monitor the outside environment and
GPS instructions. After roughly two minutes, the GPS would prompt the driver to take
the next exit, at which point the car triggered a warning to prepare to regain manual
control of the vehicle. A structured disengagement of the autonomous technology
would then happen, prompting the driver to regain control of the vehicle while taking
the exit. A non-driving task relating to text reply would also be prompted in con-
junction with the vehicle reverting to manual control.
Response times were measured for each scenario twice, accounting for both the
manual and the vocal interface. In particular, for the city scenario, the response time to
the stimulus of the traffic light turning green in terms of acceleration input was mea-
sured. For the highway scenario, the response time as first input (steering, throttle, or
braking) after disengagement was investigated.
The study employed a population of 32 US licensed drivers. Each driver was tested
in each scenario for both interfaces, with a within-subject design of experiment with
repeated measures. Age of the drivers spanned between 18 and 48, with an average age
of 26.7 years, and the population equally split between male and female.
The study employed a Nexus 6 smartphone mounted on top of the CD player in the
center front of the vehicle through an iOttie cell phone car mount. The touch-screen
smartphone was running an Android OS Version 5, and was used for the texting task,
as well as to navigate the participants through the simulated city and highway envi-
ronments with GPS instructions.
3 Analysis of Results
3.1 City: Throttle Response to Traffic Light Turning Green
In the city scenario, throttle pedal response times to the traffic light changing from red
to green were measured. Mean response times for manual interface and the voice
interface were 2.03 s and 1.79 s, respectively. After checking for normality of the data,
an independent, two sample t test was performed using the R programming language.
The means of each group, manual and voice interface, were not found to be signifi-
cantly different (t(62) = 0.5662, p = 0.5733).
Due to the low sample size, as this was preliminary research, it is possible that no
significant difference was observed as the effect size may be quite small, on the order of
a second to two. That said, it is also possible that different cell phone interaction
Evaluating the Impact of Phone Interface Modality 1031
modalities have little or no effect on driver response time. In this scenario, drivers
reacted to the changing green light in about two seconds, whether they were responding
to a text message through manual typing or voice-recognition typing. This may be an
indication that runs counter to the supposed safety benefits that voice interfaces in cell
phones bring to driving.
3.2 Highway: Time to First Input After Disengagement
For the highway scenario, a separate analysis of steering, throttle, and braking response
time was executed. Steering was the recorded first input in the majority of the test
(62.5%). Mean response times for the manual and vocal interfaces are shown in
Table 1. Response times show similar values between the two interfaces and are in line
with literature [7]. Two-tailed t-test shows once more no statistically significant change
among the two interfaces (results reported in Table 2).
Table 1. Average response times for the highway scenario
Response time Manual [s] Vocal [s]
Throttle 1.9531 3.801
Braking 18.2134 16.9334
Steering 0.7747 0.7411
Table 2. Summary of t-test statistics for the highway scenario
Scenario Response time t p-value
Highway Throttle −1.1084 0.272
Highway Braking 0.71066 0.48
Highway Steering 0.36874 0.7136
3.3 Time-to-Reply and Texting Behavior
Finally, we analyzed whether there was a significant difference in the texting behavior
when using the two investigated interfaces. Figure 2 shows boxplots for the length of
time (here reported in ms) it took participants to finish replying to text message. When
replying to the first city message (p = 0.0009) and the highway message (p = 0.0474),
participants took significantly longer to reply when using the manual interface.
A comparison of the character distributions showed no statistically significant differ-
ence in the length of the texts. Moreover, we investigated the total time of completion
from when the text was first received to when the text was replied. Once more no
statistical significance was found.
1032 S. O. Eurich et al.
Fig. 2. Time-to-reply boxplots for city and highway scenario
4 Conclusions
One of the goals of the presented study was that of exploring whether a potential
“safety improvement” could be attributed to the use of vocal interfaces over the tra-
ditional manual interaction modality with a cellphone. The alarming statistics on tex-
ting while driving have prompted many US States to pass legislation to prohibit manual
use of cellphones while driving. The underlying rationale of lower distraction and
reduced impairment has not yet received scientific backup from the research commu-
nity. Our study did not show any statistically significant change in response times to
various maneuvers (acceleration from still at traffic light in conventional driving, and
control takeover from automated driving). While this effect could be due to a limited
sample size, it could also indicate that such potential “safety improvement” fails to
manifest itself. Additional data analysis in relation to eye tracking and lateral drift while
replying to texts is currently ongoing.
Acknowledgments. Funding for this research was provided by the US Department of Trans-
portation (grant 69A3551747127 managed by the Mineta Transportation Institute of San Jose,
CA).
References
1. U.S. Department of Transportation: Federal Automated Vehicle Policy: accelerating the next
revolution in roadway safety. National Highway Traffic Safety Administration, NHTSA
(2016)
2. National Highway Traffic Safety Administration, NHTSA: Distracted driving 2015 –
DOT HS 812 381 (2015)
Evaluating the Impact of Phone Interface Modality 1033
3. Favarò, F., Eurich, S., Nader, N.: Autonomous vehicles’ disengagements: trends, triggers, and
regulatory limitations. Accid. Anal. Prev. 110, 136–148 (2018)
4. Llaneras, R.E., Salinger, J., Green, C.A.: Human factors issues associated with limited ability
autonomous driving systems. In: Proceedings of the 7th International Driving Symposium on
Human Factors in Driver Assessment, Training and Vehicle Design, New York, 17–20 June
2013 (2013)
5. https://drivemode.com/
6. Apple CarPlay: the ultimate Co-Pilot. https://www.apple.com/ios/carplay/
7. Favarò, F.M., Eurich, S., Rizvi, S., Agarwal, S., Mahmood, S., Nader, N.: Analysis of
disengagements in semi-autonomous vehicles: drivers’ takeover performance and operational
implications (2019)
Design and Evaluation of Human-Friendly
Hand-Held Gaming Interface for Robot-
Assisted Intuitive Telerehabilitation
S. M. Mizanoor Rahman(&)
Department of Intelligent Systems and Robotics, Hal Marcus College of Science
and Engineering, University of West Florida, 11000 University Pkwy,
Pensacola, FL 32514, USA
[email protected]
Abstract. A testbed gaming interface operated through a hand-held mobile
device is designed and developed for robot-assisted telerehabilitation. In the
developed gaming interface, a patient needs to wear sensors (position and
force/pressure sensors) at different body parts that measure kinematics and
kinetics data during rehabilitation practices through some game-like activities.
The extent of rehabilitation progress is reflected through the magnitudes, pat-
terns and expressions (such as the graphical expressions) of the kinematics and
kinetics data. To do so, the kinematics and kinetics raw data are processed in
real-time, expressed in different ways and displayed through the interface (e.g.,
mobile phone) held in the patient’s hand. The patient can monitor the rehabil-
itation progress through the graphical data expressions via the interface, interact
with the display device and the rehabilitation results, and communicate the
results to the healthcare providers and the patient community. To make the
gaming interface human-friendly, prior knowledge of human factors involved in
the interface is considered when the interface is designed. The assessment
method for each human factor is developed, and the interface is implemented
and evaluated in a pilot study. The evaluation results show that the hand-held
gaming interface designed and developed through addressing the human factors
is effective to satisfy the human factor requirements in the human-device
interaction, and to make the interaction more human-friendly. The results can be
used to optimize and benchmark the performance of the interface and of the
rehabilitation system.
Keywords: Telerehabilitation Rehabilitation robot Gaming interface
Human-friendliness Performance evaluation
1 Introduction
Robot-based post-stroke rehabilitation has become an important topic of research and
development due to having manifold advantages of robot-based rehabilitation [1–5].
Hand-held gaming interfaces are the very latest addition to the robot-based rehabili-
tation where the interface can facilitate users (patients) to monitor rehabilitation per-
formance in real-time via the hand-held visual interfaces based on some game-like
activities performed during the rehabilitation cycles [6, 7]. On the other hand,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1034–1040, 2020.
https://doi.org/10.1007/978-3-030-39512-4_157
Design and Evaluation of Human-Friendly Hand-Held Gaming Interface 1035
telerehabilitation facility is a newly coined human-system interaction concept that can
keep patients connected with healthcare providers (therapists) and patient communities
[8–11]. Hand-held gaming interfaces can add value to the telerehabilitation concepts
through easing the monitoring and sharing the status of the rehabilitation performance
[7, 9, 10]. The usability of the gaming interfaces is of high importance, and the user
acceptance of the hand-held interfaces may depend on how the human factors asso-
ciated with the interfaces are identified and addressed while designing the gaming
interfaces [8, 9]. However, investigation of human factors involved in hand-held
gaming interfaces has not received priority yet except the few efforts where the human
factors associated with the gaming interface were identified and analyzed [12]. Thus,
efforts to design the actual gaming interfaces addressing the human factors have not
received much attention yet.
Hence, the objective of this paper is to develop and evaluate a hand-held gaming
interface to be used with robot-based telerehabilitation systems addressing some of the
human factors proposed previously in [12].
2 Development of the Experimental System
A mock prototype of hand-held gaming interface was developed. As shown in Fig. 1, a
healthy human user needs to wear an IMU (inertial measurement unit) in his/her wrist.
Then, the user can be asked to pick an object from a table surface and place it in
another location on the table surface with a specified orientation of the object main-
taining a targeted absolute hand velocity (speed). This can be treated as a simple game
that the user can perform. The gaming software (e.g., the Unity software) was used to
develop the required software facility to execute the gaming interface. The software
was uploaded into a mobile device, e.g. a mobile phone, a small tablet, etc. The
performance of the potential user in the game was expressed in term of his/her absolute
hand velocity while picking and placing the object in the specified orientation. For the
game, the user can be asked to pick the object and place it in another location on the
table surface. The user’s absolute velocity can be captured in real-time, processed
through the software, and expressed through a bar diagram. The bar diagram can be
displayed to the user through the hand-held mobile interface in real-time. The diagram
could also be displayed to the user through a large monitor placed in front of him/her
[9]. The pattern and magnitude of the diagram can change due to changes in the user’s
absolute velocity. Then, the user can be asked to repeat the picking and placing task,
but the user may need to maintain the same absolute velocity as he/she has produced
during the first trial.
The bar diagram was colorful that could be updated in every time step. The layout
of the gaming interface and its interconnection with the movement of the human hand
wearing an IMU sensor are shown in Fig. 2. The human can also wear an EMG at an
appropriate location on his/her arm and a force sensor at the fingers, which can measure
the manipulation and the grip forces respectively when the human picks and places the
object. For simplicity, the kinetics parameters such as the grip force (GF) and the
torque were not measured here, but such parameters could make the interface more
informative. The user may need to hold the mobile interface in one hand, and perform
1036 S. M. M. Rahman
Fig. 1. A healthy human wears an IMU (inertial measurement unit) in his/her wrist, picks an
object (left) and places it in another location at a specified orientation (aligning along the red line)
with a predefined absolute hand velocity (right), which is treated as the game of the user.
the game using another hand. The user may also need to share the gaming performance
to his/her friend in another room through the internet/cloud, which can resemble the
concepts of the telerehabilitation [12].
Fig. 2. The layout of the gaming interface to display the gaming performance and facilitate the
gaming operation in real-time through the hand-held interface device (right), and the
interconnection between the gaming activity (left) and the gaming interface (right).
3 Experimental Procedures
In total, 12 healthy human subjects were recruited. The subjects were undergraduate
and graduate students with different engineering majors. Ten subjects were male, and
two were female. The mean age was 23.46 years with standard deviations of 1:98.
The subjects gave informed consent. Each subject participated in the experiment
(game) separately as described in previous section. Each subject was separately
instructed about the detailed experiment procedures before he/she participated in the
experiment.
At the end of the experiment, the subject was asked to respond various question-
naires to assess the appropriateness of the human factors associated with the gaming
interface for the proposed telerehabilitation practice as identified in [12]. The subject
rated his/her feeling/perception about the ease of use, safety, haptic and proprioceptive
perception, maneuverability, haptic weight of the hand-held device, fatigue, self-
satisfaction, situation awareness, trust on the device, engagement and connectedness
with the device, display color, shape and size of the hand-held device, transparency of
Design and Evaluation of Human-Friendly Hand-Held Gaming Interface 1037
the information flow, promptness of communication with the distant person, and
intuitiveness and predictability of the performance using a 5-point Likert scale, where 1
and 5 indicated the least and the most favorable responses respectively [13–15]. Each
subject’s cognitive workload was assessed using the NASA TLX [16].
4 Experimental Results and Analyses
Figure 3 shows the mean (n = 12) evaluation scores of the pHFs evaluated by the
subjects while using the hand-held gaming interface during the experiment. The results
show that the pHFs are satisfactory in comparison with the highest obtainable score,
which was 5. The results thus prove that the developed hand-held gaming interface was
able to produce satisfactory pHFs for the proposed telerehabilitation system. Again, all
the pHFs criteria were not equally satisfactory. However, all the pHFs criteria were
satisfactory because the obtained evaluation scores 4 and above out of 5 could be easily
considered as the satisfactory results [17]. Analyses of variances (ANOVAs) showed
that variations in the evaluation scores between the subjects were not statistically
significant (p > 0.05 in each case), which indicates that the results can be used as the
general findings even though a small number of subjects participated in the experiment.
6
Mean evaluation
4
scores
2
0
Ease of use Safety Haptic, tactile Maneuverability Perceived haptic Fatigue
and weight of hand-
proprioceptive held device
perceptions
pHFs
Fig. 3. Mean (n = 12) evaluation scores (out of 5) of the pHFs (physical human factors)
evaluated by the subjects for using the hand-held gaming interface during the experiment.
Figure 4 shows the mean (n = 12) evaluation scores of the cHFs evaluated by the
subjects while using the hand-held gaming interface during the experiment. The results
show that the cHFs were also satisfactory in comparison with the highest obtainable
score, which was 5. The results thus prove that the developed hand-held gaming
interface was able to produce satisfactory cHFs for the telerehabilitation system [18].
Again, all the cHFs criteria were not equally satisfactory. However, all the cHFs criteria
were satisfactory because the obtained evaluation scores 4 and above out of 5 could be
easily considered as the satisfactory results. ANOVAs showed that variations in
evaluation scores between the subjects were not statistically significant (p > 0.05 in
each case), which indicates that the results can be used as the general findings even
though a small number of subjects participated in the experiment.
In addition, Fig. 5 shows the mean (n = 12) evaluation scores of the cognitive
workload values for each of the six NASA TLX terms evaluated by the subjects for
using the hand-held gaming interface during the experiment. The results show that the
1038 S. M. M. Rahman
Predictability of performance
Intuitiveness
Promptness of communication with distant person
Transparency of information flow
Perceived shape and size of the hand-held device
cHFs
Perceived color
Engagement and connectedness
Trust
Situation awareness
Self-satisfaction
0 1 2 3 4 5 6
Mean evaluation scores
Fig. 4. Mean (n = 12) evaluation scores (out of 5) of the cHFs (cognitive human factors)
evaluated by the subjects for using the hand-held gaming interface during the experiment.
40
Mean workload
values
20
0
Mental Physical Temporal Performance Effort Frustration
demand demand demand
NASA TLX Terms
Fig. 5. Mean (n = 12) evaluation scores of the cognitive workload values (out of 100) for each
NASA TLX term evaluated by the subjects for using the hand-held gaming interface.
workload scores were not very high, which indicates that the use of the hand-held
gaming interface did not create too much workload on the users. Cognitive workload
due to performance was slightly high because the users needed to pick the object and
place it in a specific location in a specific orientation and also needed to maintain
specified manipulation speed, which might be slightly difficult for the users, and thus
impacted the cognition of the users. There is no standard that can define the satisfactory
cognitive workload [16]. Note that the cognitive workload cannot be zero. It is assumed
that the average cognitive workload below 25% in any or all NASA TLX terms may be
treated as the satisfactory workload. ANOVAs showed that variations in the evaluation
scores of the NASA TLX terms between the subjects were not statistically significant
(p > 0.05 in each case), which indicates that the results can be used as the general
findings.
5 Conclusions and Future Work
A simple mock prototype hand-held gaming interface as a part of a proposed whole-
body robotic telerehabilitation system was developed addressing the human factor cues
identified previously. Healthy human subjects were recruited to use the interface and
perform some mock telerehabilitation practices (games). The appropriate assessment
Design and Evaluation of Human-Friendly Hand-Held Gaming Interface 1039
methods of all the identified human factor cues were determined, and the appropriate-
ness of the proposed hand-held gaming interface for the proposed telerehabilitation was
assessed. The results showed satisfactory human factors associated with the developed
hand-held gaming interface. The results can be used to optimize the gaming interface
design satisfying human requirements, and benchmark and share the telerehabilitation
performance for stroke patients. In the near future, intelligent software will be designed
using artificial intelligence (AI) and machine learning to perform the data acquisition,
computation and display in real-time. The system will be evaluated using real patients.
References
1. Rahman, S.M.M.: Design of a modular knee-ankle-foot-orthosis using soft actuator for gait
rehabilitation. In: Proceedings of the 14th Annual Conference on Towards Autonomous
Robotic Systems (TAROS 2013). Lecture Notes in Computer Science, Oxford, U.K., vol.
8069, pp. 195–209. Springer (2014)
2. Rahman, S.M.M., Ikeura, R.: A novel variable impedance compact compliant ankle robot for
overground gait rehabilitation and assistance. Procedia Eng. 41, 522–531 (2012)
3. Rahman, S.M.M.: A novel variable impedance compact compliant series elastic actuator for
human-friendly soft robotics applications. In: Proceedings of the 21st IEEE International
Symposium on Robot and Human Interactive Communication, pp. 19–24. IEEE Press
(2012)
4. Rahman, S.M.M.: A novel variable impedance compact compliant series elastic actuator:
analysis of design, dynamics, materials and manufacturing. Appl. Mech. Mater. 245, 99–106
(2013)
5. Yu, H., Rahman, S.M.M., Zhu, C.: Preliminary design analysis of a novel variable
impedance compact compliant actuator. In: Proceedings of 2011 IEEE International
Conference on Robotics and Biomimetics, pp. 2553–2558. IEEE Press (2011)
6. Rahman, S.M.M., Ikeura, R.: Improving interactions between a power assist robot system
and its human user in horizontal transfer of objects using a novel adaptive control method.
Adv. Hum.-Comput. Interact. 2012, 1–12 (2012). ID 745216
7. Valdés, B.A., Hilderman, C.G.E., Hung, C.T., Shirzad, N., Van der Loos, H.F.M.: Usability
testing of gaming and social media applications for stroke and cerebral palsy upper limb
rehabilitation. In: Proceedings of 2014 36th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society, pp. 3602–3605. IEEE Press (2014)
8. Rahman, S.M.M., Ikeura, R.: Cognition-based control and optimization algorithms for
optimizing human-robot interactions in power assisted object manipulation. J. Inf. Sci. Eng.
32(5), 1325–1344 (2016)
9. Rahman, S.M.M., Wang, Y.: Mutual trust-based subtask allocation for human-robot
collaboration in flexible lightweight assembly in manufacturing. Mechatronics 54, 94–109
(2018)
10. Kawai, Y., Honda, K., Kawai, H., Miyoshi, T., Fujita, M.: Tele-rehabilitation system for
human lower limb using electrical stimulation based on bilateral teleoperation. In: Proceedings
of 2017 IEEE Conference on Control Technology and Applications, pp. 1446–1451 (2017)
11. Rahman, S.M.M.: Evaluating and benchmarking the interactions between a humanoid robot
and a virtual human for a real-world social task. In: Proceedings of the 6th International
Conference on Advances in Information Technology. Communications in Computer and
Information Science, vol. 409, pp. 184–197 (2013)
1040 S. M. M. Rahman
12. Rahman, S.M.M.: Investigating human factors in the hand-held gaming interface of a
telerehabilitation robotic system. In: Proceedings of the 3rd International Conference on
Intelligent Human Systems Integration: Integrating People and Intelligent Systems, Modena,
Italy (2020)
13. Rahman, S.M.M., Ikeura, R.: Investigating the factors affecting human’s weight perception
in lifting objects with a power assist robot. In: Proceedings of 2012 21st IEEE International
Symposium on Robot and Human Interactive Communication, pp. 227–233 (2012)
14. Rahman, S.M.M., Ikeura, R.: Cognition-based variable admittance control for active
compliance in flexible manipulation of heavy objects with a power assist robotic system.
Robot. Biomim. 5(7), 1–25 (2018)
15. Rahman, S.M.M., Liao, Z., Jiang, L., Wang, Y.: A regret-based autonomy allocation scheme
for human-robot shared vision systems in collaborative assembly in manufacturing. In:
Proceedings of IEEE International Conference on Automation Science and Engineering,
pp. 897–902 (2016)
16. Rahman, S.M.M., Wang, Y.: Dynamic affection-based motion control of a humanoid robot
to collaborate with human in flexible assembly in manufacturing. In: Proceedings of ASME
Dynamic Systems and Controls Conference, p. V003T40A005 (2015)
17. Schmidtler, J., Bengler, K., Dimeas, F., Campeau-Lecours, A.: A questionnaire for the
evaluation of physical assistive devices (QUEAD): testing usability and acceptance in
physical human-robot interaction. In: Proceedings of 2017 IEEE International Conference on
Systems, Man, and Cybernetics (SMC), pp. 876–881 (2017)
18. Szi, B., Csapo, A.: An outline of some human factors contributing to mathability research.
In: Proceedings of IEEE Conference on Cognitive Infocommunications, pp. 583–586 (2014)
Capture of Intruders by Cooperative Multiple
Robots Using Mobile Agents
Yasushi Kambayashi1(&), Taichi Sekido2, and Munehiro Takimoto2
1
Department of Computer and Information Engineering, Nippon Institute of
Technology, 4-1 Gakuendai Miyashiro-Machi, Minamisaitama-gun,
Saitama 345-8510, Japan
[email protected]
2
Department of Information Sciences, Tokyo University of Science,
2641 Yamazaki, Noda 278-8510, Japan
[email protected]
Abstract. Detecting and capturing intruders is one of the most important
applications of multi-robot systems. In this paper, we introduce a novel control
system for chasing an intruder into a corner in a building by multiple small not-
so-agile mobile robots. We have applied our mobile software agent system to
this problem in order to mitigate slow movement of mobile robots. We can
station multiple small robots in a building, and make them cooperate using
mobile software agents that control mobile robots. Since software agents can
move instantly from a robot to another robot, it can migrate to the most suitably
located mobile robot and drive the intruder into a bay where human captors are
waiting. The cooperation of robots and software agents and humans is the key of
our system. In order to investigate the feasibility of our idea, we have imple-
mented a simulator, and demonstrated the usefulness of our system.
Keywords: Mobile agents Multi robots Intruder capture systems
1 Introduction
The advent of a combination of multiple robots and multiple software agents is real-
izing a new stage of cooperation of mobile robotic agents and humans. We have
engaged the study of mobile software agents, and produced several applications such as
multi-robot systems that cooperatively search objects [1] and cooperatively compose
formations [2, 3]. The persuasion is based on the finding that those systems are not only
feasible but also enjoy the energy saving [4].
In this paper, we focus our attention on detection and capture of intruder in a
building. Detecting and capturing an intruder is one of the most important applications
of multi-robot systems. We introduce a novel control system for chasing an intruder
into a corner in a building by multiple small not-so-agile mobile robots. With the
current robotics technologies and safety requirements, it is hard to obtain mobile robots
as agile as human being. In order to ameliorate the laggardness of mobile robots, we
have employed multiple small robots and mobile software agents. We can station
multiple small robot in a building, and make them cooperate using mobile software
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1041–1047, 2020.
https://doi.org/10.1007/978-3-030-39512-4_158
1042 Y. Kambayashi et al.
agents that control mobile robots. Since software agents can move instantly from a
robot to another robot, it can migrate to the most suitably located mobile robot to
capture the intruder.
In addition to the agility of mobile software agents, they can easily duplicate
themselves and control multiple mobile robots simultaneously to achieve highly
cooperative activities such as capturing an intruder. In this paper, we describe the
attributes of mobile software agents and explain how they can make multiple mobile
robots cooperate to capture an intruder.
The rest of this paper is organized as follows. The second section describes the
background. The third section describes the method for coordinating multiple mobile
robots using the mobile software agents. Our mobile agent system consists of two kinds
of mobile software agents, namely ant agents and pheromone agents. We describe each
of them and present the role and algorithm to implement the control system for multiple
robots’ coordination. The fourth section describes the experiments on the simulator,
and we conclude our discussion in the fifth section.
2 Background
We have previously presented a framework for controlling intelligent multiple robots
connected by wireless communication networks [2]. The framework makes the mul-
tiple mobile robots indirectly cooperate each other and accomplish highly coordinated
actions. The core of the framework consists of two kinds of mobile software agents,
namely pheromone agents and driving agents. The idea of pheromone agents is
inspired by the behaviors of social insects, i.e. ants. In the algorithm, agents that mimic
the behaviors of ants communicating with each other by an indirect communication
mediated by modifications of the environment called stigmergy. The behavioral sci-
entists Goss et al. found that ants exchanged information by laying down a trail of a
chemical substance called pheromone that is followed by other ants [5]. Dorigo et al.
have abstracted the behaviors of ants and applied the extracted algorithm to derive a
quasi-optimal solution for NP-complete problems [6]. Later, Dorigo et al. successfully
applied the idea to control and coordinate swarm robots [7]. Their SWARM-BOTS
project is today considered as the pinnacles of many multi-robot projects.
3 Mobile Robot Control
Captor robots are small mobile robots. They do not actively attack an intruder violently
but quietly chase and drive him or her into the designated capture location. Our system
consists of robots and two kinds of mobile software agents, namely ant agents and
pheromone agents. All the controls for the mobile robots are achieved through ant
agents. Each mobile robot has Wi-Fi capability. They are connected through wireless
LAN.
Each ant agent can freely move among the herd of mobile robots. Since the ant
agents control the robots, captor robots without ant agents just sit and sense the
environments quietly, while ant agents are hopping from a robot to another robot to
Capture of Intruders by Cooperative Multiple Robots Using Mobile Agents 1043
patrol the sensing area by checking the quietly sitting robots’ sensors. We assume the
captor robots have enough sensors such as optical cameras and ultrasonic sensors to
detect an intruder. Once an ant agent arrives at a captor robot who senses an intruder, it
drives the robot to hunt down the suspect and simultaneously dispatches pheromone
agents to rally nearby robots to bring the intruder to the bay. We describe the ant agents
and pheromone agents in the following sections.
3.1 Ant Agent
The ant agent (AA) has the following four capabilities. (1) It drives a robot on which it
resides depending on the specific state they can take. (2) It can migrate from a robot to
another robot through wireless LAN. (3) It has the IP addresses of all the participating
robots. (4) It creates pheromone agents that attract other mobile robots.
Without AAs, mobile robots cannot even move; they are just sitting quietly.
Therefore, at the initial phase, a number of AA migrate into the herd of the mobile
robots, and each AA looks for an idle robot (a robot without AA) that it can drive.
Once an AA arrives on a robot, it is in the “search” state. After that, AA changes its
state depending on the environmental conditions. The idea of changing state to make
AA perform different kinds of tasks is inspired from [8].
Initially, the AA is in the “search” state. In this state, the AA makes the robot
randomly walk to find a suspect. The robot has basic capability for collision avoidance
by ultrasonic sensors and electronic circuit. Each robot has basic capabilities to survive
by on-board controller without AA’s interference. The robot’s visual sensor can sense
objects in the fan-shaped area with 45° right and left from the centerline of its driving
direction. When the AA finds an intruder in its fan-shaped sensing area through robot’s
visual sensor, it changes its state to “chase”.
In the “chase” state, the AA drives its robot toward the intruder while it gives
intimidation by warning sound and flashing light, and tries to drive the intruder to the
designated capture location. Since the mobile robots are not as agile as human intruder,
it cannot capture or drive the intruder into the capture location alone. It needs fellow
robots to drive the intruder in cooperation with them. In order to obtain the cooperation
from other robots that do not sense the intruder, the chasing robot changes its state into
“attract” temporarily, and creates and dispatches pheromone agents.
In the “attract” state, AA creates and dispatches pheromone agents as many as it
can find other AAs in search state. A pheromone agent (PA) is the other kind of mobile
agents, and its purpose is to share information about the intruder with other robots in
search state within the wireless communication range. PA has a vector value that points
from the robot that creates the PA to the intruder, so that PA can tell another robot to
which direction it should move to find the intruder. After dispatching PAs, the AA
resume to the “chase” state, and continues to chase the intruder.
When an AA receives a pheromone agent from a chasing AA, the AA transits to be
in the “guided” state temporarily. In the guided state, the AA calculates its new
direction by synthesizing two vector values; one is pointing to the intruder from the
original robot that is given by the arriving PA and the other is pointing to the fellow
robot that creates the arriving PA. Upon completing the calculation of new direction, it
starts moving along the calculated vector value until it finds the intruder. When AA
1044 Y. Kambayashi et al.
finds the intruder, it changes its state from “guided” to “chase” and starts chasing the
intruder based on its own sensor information. If AA cannot find the intruder after
certain period, it changes its state to “search” again, and starts random walk. We
describe how a PA plays its role in the next section.
3.2 Pheromone Agent
The pheromone agent (PA) is a mobile agent that guides AA to move. Figure 1 shows
how a PA guides and makes a searching robot chase an intruder in cooperation with an
already chasing robot. AA1 finds a robot randomly walking that is controlled by AA2
while it is chasing the intruder. Since each robot has a visual sensor, AA1 can calculate
the intruders coordinate (xt ; yt ). Since mobile robots are not agile enough to capture an
intruder alone, it needs the other robot’s assistance. AA2 is driving the other robot. AA1
decides to make that robot participate in the chase. AA1 creates a pheromone agent
with the vector value that points to the intruder, and dispatches the PA to the robot
where AA2 resides. Upon receiving the PA, AA2 changes its state “search” to “guided”
and start calculating the new direction it should move.
(xt, yt) (xt, yt)
AA1 AA1
Direct to (xt, yt)
AA2 AA2
Fig. 1. PA guides AA’s movement.
Even though each mobile robot is slow and can move in laggard pace, by using the
ant agents and pheromone agents, the captor robot that finds an intruder can attract
other fellow robots as many as it can communicate with wireless LAN, and encircle the
intruder to drive it to the capture location.
4 Preliminary Experiments
In order to investigate the feasibility of our detection and capture system, we have
implemented our system on a simulator and conducted preliminary numerical experi-
ments. In the experiments, we have set the experimental field is 500 500 pixel
square field, and robots are randomly placed in the field. The capture location is set at
one end of the field. The range of communication network for each robot is 200 units
and the range of the sensor is same as the above. We have conducted 100 trials with
both with PA and without PA and averaged the duration to drive the intruder to the
capture location.
Capture of Intruders by Cooperative Multiple Robots Using Mobile Agents 1045
Figure 2 shows the results of the experiments. The vertical axis represents the
number of steps needed to drive the intruder into the capture location, and horizontal
axis represents the number of participating mobile robots. We can observe that PAs
contribute to capturing the intruder when the number of captor robots is small. Since
the mobile agents are just software and has almost no cost to produce many, we can
claim that the pheromone agent methodology is economically useful to save the
number of physical mobile robots.
Fig. 2. The relation between the number of robots and duration to capture an intruder.
Figure 3 shows the relationship between the communication range of the wireless
LAN and duration to drive an intruder to the capture location. We can observe setting
300 units provides the optimal case. Of course, this number has no meaning. However,
we have found there should be optimal Wi-Fi range for mobile robot control, and it
may not be very large. This finding encourages us to investigate the optimal strength of
radio by experimenting with real mobile robots.
Fig. 3. The relation between the communication range of the wireless LAN and time period to
capture an intruder.
1046 Y. Kambayashi et al.
5 Conclusion and Future Directions
We have proposed a mobile multi-robot system that detects and captures an intruder in
a building. We have applied our mobile software agent system to this problem to
mitigate slow movement of mobile robots. For the safety requirements, we cannot
make mobile robot fast enough to capture agile intruder. It is easy to lose the trail of the
intruders. The movement of mobile software agents, however, is instantaneous.
Therefore, if a mobile agent that has information about the intruder can migrate to
another more conveniently located robot, it makes easier to trace and capture the
intruder. In addition, we have tried to use pheromone agents to attract other idle robots
to participate into the persuasion.
In order to investigate the feasibility of our idea, we have implemented a simulator
for our mobile robot system to demonstrate the usefulness, and to make clear what
points we should make efforts to make the system practical. The natural direction
toward the next step is to implement a real robot system to perform further feasibility
studies. Then we must take account of more heterogeneous teams including diverse
robots and even humans. In order to implement detecting and capturing an intruder in
practice, we need to make a system involving humans. Therefore, we plan to produce
effective synergies of mobile robots and human through cooperation and collaboration
of multiple mobile software agents.
Acknowledgments. This work is partially supported by Japan Society for Promotion of Science
(JSPS), with the basic research program (C) (No. 17K01304), Grant-in-Aid for Scientific
Research (KAKENHI) and Suzuki Foundation.
References
1. Nagata, T., Takimoto, M., Kambayashi, Y.: Cooperatively searching objects based on mobile
agents. In: Transaction on Computational Collective Intelligence XI. LNCS, vol. 8065,
pp. 119–136. Springer, Heidelberg (2013)
2. Shintani, M., Lee, S., Takimoto, M., Kambayashi, Y.: A serialization algorithm for mobile
robots using mobile agents with distributed ant colony clustering. In: König, A., Dengel, A.,
Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds.) KES 2011, Part I. LNCS, vol. 6881,
pp. 260–270. Springer, Heidelberg (2011)
3. Kambayashi, Y., Yajima, H., Shyoji, T., Oikawa, R., Takimoto, M.: Formation control of
swarm robots using mobile agents. Vietnam J. Comput. Sci. 6(2), 193–222 (2019)
4. Takimoto, M., Mizuno, M., Kurio, M., Kambayashi, Y.: Saving energy consumption of multi-
robots using higher-order mobile agents. In: Nguyen, N.T., Grzech, A., Howlett, R.J., Jain, L.
C. (eds.) KES-AMSTA 2007. LNCS (LNAI), vol. 4496, pp. 549–558. Springer, Heidelberg
(2007)
5. Goss, S., Aron, S., Deneubourg, J.L., Pasteels, J.M.: Self-organized shortcuts in the Argentine
ant. Naturwissenschaften 76, 579–581 (1989)
6. Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperating
agents. Trans. Syst. Man Cyber. Part B 26(1), 29–41 (1996)
Capture of Intruders by Cooperative Multiple Robots Using Mobile Agents 1047
7. Dorigo, M.: SWARM-BOT: an experiment in swarm robotics. In: Proceedings of the 2005
IEEE Swarm Intelligence Symposium, pp. 192–200. IEEE Press (2005)
8. Takahashi, R., Takimoto, M., Kambayashi, Y.: Cooperatively transporting unknown objects
using mobile agents. In: 6th International Conference on Agents and Artificial Intelligence.
Revised Selected Papers LNAI (LNAI), vol. 8946, pp. 46–62. Springer, Heidelberg (2015)
Automation as Driver Companion: Findings
of AutoMate Project
Andrea Castellano1(&), Massimo Fossanetti2, Elisa Landini1,
Fabio Tango2, and Roberto Montanari1
1
RE:Lab Srl, Via A. Tamburini, 5, 42122 Reggio Emilia, Italy
{andrea.castellano,elisa.landini,
roberto.montanari}@re-lab.it
2
Centro Ricerche Fiat, Strada Torino, 50, 10043 Orbassano, Italy
{massimo.fossanetti,fabio.tango}@crf.it
Abstract. Driving automation is radically changing the role of the driver. The
proliferation of driving assistance systems is increasingly transforming driver’s
tasks from vehicle control operations to supervising activities. However, the
process of turning the driver into a passenger is far from being accomplished.
This paper describes an innovative interaction approach developed in the
framework of EU funded project AutoMate. The overarching aim of AutoMate
is to build a “TeamMate System”, in which the human and the automation
cooperate with each other to achieve a safe, pleasant and efficient driving.
Through an effective interaction, and by sharing perception, decision and action,
they can negotiate specific behaviors and maneuvers in order to build a team
based on trust. In order to measure the effectiveness of this concept, a driving
simulator experiment has been conducted: findings suggest that the concept of
Human-Machine Team can increase the trust in automation and improve the
efficiency in specific conditions.
Keywords: Highly automated vehicles Human-machine cooperation HMI
Human factors Cooperative driving Negotiation-based HMI
1 Introduction and Related Works
In recent years, vehicles are progressively integrating more sophisticated and reliable
Driving Assistance Systems (DASs). These systems have proven to offer greater safety
to the driver and to the surrounding traffic [1]. Several projects carried out by car
manufacturers, suppliers, universities and research institutes have developed prototypes
of automated vehicles (AVs). In addition, automotive industry has also been active in
many EU co-funded projects, such as HAVEit, SARTRE, HOLIDES, L3-PILOT. In
this scenario, although the technical development progressively proceeds and partial
automated driving features are already available, nevertheless, there will be circum-
stances in which such systems will no longer be able to work reliably. In such cases, the
driver must intervene and resume the vehicle control as quickly as possible with a high
take-over quality (i.e. avoiding significant speed variation and lane deviation). In
certain conditions, in fact, the automation could not be able to cope with complexity,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1048–1054, 2020.
https://doi.org/10.1007/978-3-030-39512-4_159
Automation as Driver Companion: Findings of AutoMate Project 1049
such as irregularity in road shape/surface and dense urban traffic. For example, a well-
known situation that may lead to a system disengagement is the roundabout, in which
shape peculiarity, high traffic flows and the absence of road markings could generate
unpredictable dangers. The situation understanding and prediction capabilities of
vehicles, in fact, are at the moment far less sophisticated than human driver capabilities.
Consequently, the automation is clearly overburdened in complex and unexpected
situations (e.g. if other traffic participants operate against traffic rules).
Since Level 5 (full automated driving without driver supervision) is far from being
accomplished, it is increasingly important to address Human Factors issues with auto-
mated systems; therefore, experts and designers are concentrating their attention on
Levels 3 and 4. As already stated by Wachtel, a non-trivial percent of drivers will be ill-
equipped to handle Take-Over Request (TOR) and thus, the introduction of AVs is to be
considered hubris, based on results and experiences from other industries [2]. In fact –
and inevitably – this future technology will cause out-of-the-loop problems [3], mode
confusion [4], and behavioral adaptation [5] issues that need urgent reconsideration in
order to maintain safe driving [6].
Stanton [7] describes the utopian vision to have an on-board “auto-driver”, similar
to the “autopilot” in aircraft, to assume the control of driving task, allowing humans to
work, rest or play. The problem arises when the human drivers are requested to monitor
their “auto-driver” to take-over the vehicle control when the artificial agent fails. In this
perspective, Eriksson and Stanton [8] state that, when the operational limits are
reached, the driver must receive support and guidance, necessary to re-enter into the
control loop [9]. The concept of maintaining a form of meaningful human control over
automated systems is not new [10]. This concept encompasses all forms of control, i.e.,
not only operational, but also tactical and strategical actions, of a human being over an
automated system.
2 The AutoMate Project in the Cooperative Framework
To deal with all these aspects, it becomes of paramount importance the question of how
these independent agents (the automated vehicle and the human driver) can interact
with each other in a simple and natural way, requiring the ability to learn when to
support and how to mediate joint actions in a collaboration. The European funded
project AutoMate addresses the topic of Human-automation teaming, exploring inter-
action design solutions to address the issues generated by the increasing automation.
The long-term vision of AutoMate is that cooperation among traffic participants can
improve efficiency and safety beyond the level attainable by only human drivers or
only machine-agents. The “TeamMate Car” built in this project considers the possi-
bility to mimic the driver’s attitudes, through the recognition and the online learning of
human intentions and capabilities. This concept conceives driver and automation as
members of a team, that understand and support each other in cooperatively pursuing
the goal of driving safely, efficiently and comfortably. Under this perspective, driver
and vehicle are considered as a joint system aimed at performing all relevant driving
tasks in order to achieve a safe and efficient driving in different traffic conditions.
1050 A. Castellano et al.
A crucial aspect to design the technological system as a driver companion is to
build the trust and ensure the transparency in decision making; this depends on how
well the agents interact, communicate and cooperate with humans both inside and
outside the vehicle. At any given moment in time, all relevant subtasks are carried out
by the joint system without leaving some subtasks unattended. Since one of the main
barriers in the introduction of highly automated vehicles is the users’ acceptance,
AutoMate is working on the development of technologies able to allow a safe and
human-like driving and a fluid, smooth and efficient interaction.
3 The “TeamMate” Multimodal HMI
The aim of a Human-Machine Interface (HMI) in a cooperative highly automated
system is to improve the trust and the acceptance by increasing the transparency of the
automation. Traditionally, Human-Machine Interfaces are designed with a warning-
based approach: the system is aimed at warning the driver in order to change his/her
behavior, or to inform him/her about imminent risks.
In AutoMate, the HMI is also used as a means to build the team: the traditional
warning-based approach cannot be considered as optimal when the system should
explain the reasons behind its choices: in order to do that, a dialogue based on
negotiation of specific tasks is needed. Therefore, the TeamMate system is designed to
make the driver aware of the limits and the supports needed, in order to increase the
driver’s situation awareness. This is the so-called “why layer” [11]: by explaining the
reason behind the request (e.g. a request of take-over), the vehicle can improve the
relation with the driver. This is particularly true in non-safety critical situations, i.e.
when the time of execution is not a relevant issue. The actual implementation of the
cooperation is given through a “Human to Automation (H2A) support in perception”:
in this modality, the HMI shows a 3D video embedded in the instrument cluster to
explain the limit of the automation and the support that is needed.
The HMI is designed as a multimodal tool, i.e. it uses different sensory channel to
maximize the interaction and avoid cognitive and physical overloads: for example, the
Take-Over Request is reinforced with a haptic vibration in the seat when the driver is
distracted, while the message is explained also with audio.
Fig. 1. TeamMate multimodal HMI
Automation as Driver Companion: Findings of AutoMate Project 1051
An incremental V&V process of the HMI has been performed in the project. The
tests were used to collect and validate significant requirements to drive the design: the
results are described in [12] (Fig. 1).
4 Experiment and Results
In order to assess the actual implementation of the TeamMate concept and technolo-
gies, an experiment at driving simulator has been performed.
4.1 Scenario and Methodology
The aim of the test was to measure the effectiveness and the quality of the interaction
between the human and the TeamMate system. The experiment has been performed in
a one-seat driving simulator based on SCANeR Studio 1.7, provided with automated
driving features and real car commands (i.e. driving seat, pedals, gearbox and force-
feedback steering wheel). The simulator’s default sensor configuration has been used in
each part of the experiment, in order to avoid undesired effects on the data. Twenty
participants with valid driving license have been recruited for the experiment. The user
sample was balanced for gender (9 males and 11 females), in order to avoid biases. The
users were 28,35 years old in average. The mean years of driving experience was 9,3
years; they travelled about 17.000 km/year in average. The test was designed as a
repeated measures experimental design, meaning that each participant performed the
scenario twice, in order to compare the results with a baseline, i.e. a deployment of the
scenario without the technologies developed in the project. The baseline was made by a
fully automated system, including all the limitations, in terms of sensors and control
capabilities, that could affect the efficiency (Fig. 2).
Fig. 2. Driving simulator test setup
1052 A. Castellano et al.
The scenario used was a peri-urban scenario, in which the car encountered several
roundabouts. It is described by the following story:
“The TeamMate car is driving in Automated Mode. When it approaches a roundabout, it
detects high traffic flows that can affect the efficiency, i.e. it evaluates that it may take some time
to enter the roundabout in Automated Mode. To speed up the manoeuvre, the car asks the
driver to check the available space and to provide a trigger to start the manoeuvre. The driver
checks the traffic and gives the confirmation to enter the roundabout. The TeamMate car
understands the feedback and enters the roundabout in Automated Mode”.
The main difference between the scenarios consisted in a minimum support (at
perceptual and decisional level) given by the driver to the automation: the car, rec-
ognizing an inefficiency deriving from the traffic flows at some roundabouts, asked the
driver to trigger the entering manouver, in order to execute (and potentially learn) a
more efficient behaviour. The hypotheses were that TeamMate car could be able to
improve the efficiency, the trust and the acceptance in highly automated vehicles, and
to balance the workload in the interaction, i.e. reducing it while avoiding a tendency
towards too low values. The indicator selected to measure the efficiency was the time to
enter the roundabout, calculated with driving simulator’s logs; other measures included
trust, acceptance and workload.
4.2 Results
In full automated modality (i.e. the baseline), with the sensor and the traffic configu-
ration considered in the experiment, the time to enter the roundabout was designed to
be constant at 27,12 s. In the TeamMate scenario, this time depended from the moment
in which the user gives the support in perception to the vehicle. The average time to
perform the roundabout, on a sample of 40 roundabouts (2 per each user), has been
measured in 17,83 s, with a reduction of 9,25 s.
Regarding the trust, the baseline score was +0,25, while the TeamMate score was
+0,47 (with a D of +0,25) on the Koerber scale [13]. In detail, the TeamMate has been
considered reliable (+0,60 than the baseline), and the state of the system was considered
clear (+0,55 than the baseline). However, the user found the TeamMate system not very
predictable, also in comparison with the baseline (TeamMate predictability = −0,25
than the baseline). For the acceptance, the experiment showed encouraging results, since
the overall acceptance of the TeamMate system was found to be higher than the baseline
(TeamMate acceptance = +0,97; baseline acceptance = +0,41; D of the increase =
+0,55 on the Van der Laan scale [14]). The users found that the TeamMate car can
allow them to do what they want (+0,75 against the −0,15 of the baseline; D = +0,90).
From qualitative data, emerged that the users appreciated the possibility ensured by the
TeamMate system of increase the effectiveness in terms of time needed to complete the
tasks, since they found the baseline, in full automation, too cautious and restrictive in the
driving behavior. Regarding the workload, measured with the NASA-TLX [15], the
results showed an increase of the performance with TeamMate car. The overall work-
load score for baseline system is 6,81, while the score for TeamMate system in 4,82. The
results, even if not particularly prominent, can be considered as a relevant tendency.
Automation as Driver Companion: Findings of AutoMate Project 1053
5 Conclusions and Future Works
The preliminary findings described in this paper show that cooperative driving concept
implemented in the project has been well accepted by the users. In particular, it showed
a good calibration of mental workload, avoiding both low and high peaks.
The next steps include further evaluations of the TeamMate concept in driving
simulators and real cars, already equipped with the HMI and the other technological
enablers developed in the project. Moreover, for the HMI, it is planned to test an
already developed “adaptive distributed HMI”, able to show the information directly on
the device the driver is looking at. This concept has been implemented as a mobile app
able to use the data from the simulator (and, potentially, the real vehicle) and the Driver
Monitoring System, through a CAN-gateway, to display the information on a smart-
phone placed in the central tunnel.
Acknowledgments. The paper has been written in the framework of AutoMate Project. This
project has received funding from the European Union’s Horizon 2020 Research and Innovation
programme, under Grant Agreement No. 690705.
References
1. Karush, S.: They’re working: Insurance claims data show which new technologies are
preventing crashes. Status report of the insurance institute for highway safety—highway loss
data institute, vol. 52, no. 5 (2012)
2. Buehler, M., Iagnemma, K., Singh, S. (eds.): The DARPA Urban Challenge: Autonomous
Vehicles in City Traffic. (Series Springer Tracts in Advanced Robotics), vol. 56. Springer,
Heidelberg (2009)
3. Wei, J., Snider, J., Kim, J., Dolan, J., Rajkumar, R., Litkouhi, B.: Towards a viable
autonomous driving research platform. In: Proceeding of the IEEE Intelligent Vehicles
Symposium, Gold Coast, Australia, June 2013, pp. 763–770 (2013)
4. Wachtel, J.: By what hubris? The readiness of the human operator to take over when the
automation fails or hands over control. In: Proceeding of DDI 2018, 6th International
Conference on Driver Distraction and Inattention, Gothenborg (2018)
5. Kaber, D.B., Endsley, M.R.: Out-of-the-loop performance problems and the use of
intermediate levels of automation for improved control system functioning and safety.
Process Saf. Prog. 16, 126–131 (1997)
6. Stanton, N.A., Marsden, P.: From fly-by-wire to drive-by-wire: Safety implications of
automation in vehicles. Saf. Sci. 24, 35–49 (1996)
7. Rudin-Brown, C.M., Parker, H.A.: Behavioural adaptation to adaptive cruise control (ACC):
implications for preventive strategies. Transp. Res. Part F Traffic Psychol. Behav. 7, 59–76
(2004)
8. Saffarian, M., De Winter, J.C.F., Happee, R.: Automated driving: human-factors issues and
design solutions. In: Proceedings of the Human Factors and Ergonomics Society 56th
Annual Meeting, pp. 2296–2300 (2012)
9. Poulin, C., Stanton, N.A., Cebon, D.: Response to: autonomous vehicles. Ingenia Online
(62) (2015)
1054 A. Castellano et al.
10. Eriksson, A., Stanton, N.A.: The chatty co-driver: a linguistics approach to human-
automation-interaction. In: Human Factors in Organizational Design and Management – XI
(1997)
11. Koo, J., et al.: Why did my car just do that? Explaining semi-autonomous driving actions to
improve driver understanding, trust, and performance. Int. J. Interact. Des. Manuf. (IJIDeM)
9(4), 269–275 (2015)
12. Castellano, A., et al.: Is your request just this? New automation paradigm to reduce the
requests of transition without increasing the effort of the driver. In: 25th ITS World
Congress, Copenhagen, Denmark (2018)
13. Körber, M.: Theoretical considerations and development of a questionnaire to measure trust
in automation. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.)
Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018):
Volume VI: Transport Ergonomics and Human Factors (TEHF), Aerospace Human Factors
and Ergonomics, 1st edn, pp. 13–30. Springer (2019)
14. Van der Laan, J.D., Heino, A., De Waard, D.: A simple procedure for the assessment of
acceptance of advanced transport telematics. Transp. Res.-Part C Emerg. Technol. 5, 1–10
(1997)
15. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of
empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental
Workload. Advances in Psychology (1988)
Applications and Future Trends
Development of a Human System Integration
Program in Military Context
Jari Laarni(&) and Marja Ylönen
VTT Technical Research Centre of Finland Ltd., Vuorimiehentie 3,
02044 Espoo, Finland
{Jari.Laarni,Marja.Ylonen}@vtt.fi
Abstract. Human Systems Integration (HSI) integrates the knowledge and
skills of different areas of Human and Organizational Factors in order to support
various levels of systems engineering and management. Our aim is to develop
the first comprehensive and systematic human-system integration program for
the Finnish Defence Forces. In the early phases of the project, we reviewed
lessons learned from the existing human-system integration programs and
interviewed experts both from the Defence Forces and the defence industry. The
objective of the project is to develop a detailed guidance for HSI program
execution and provide a checklist of human factors issues to consider during the
acquisition process. In addition, cost-benefit guidance will be provided for the
calculation of a project’s HSI costs, and a survey of human factors modelling
and simulation tools will be conducted. The preliminary draft version of the
program will be evaluated by applying it to a real-world acquisition scenario.
Keywords: Human and organizational factors Human-systems integration
Military acquisition
1 Introduction
There is a lot of evidence that military acquisition projects do not always proceed as
planned. One of the main reasons for the failure to meet the deadlines and to be on
budget in the design of large systems, is the lack of socio-technical systemic approach,
that refers to taking into account interconnections between human, organizational and
technical aspects simultaneously [1]. Furthermore, experiences from several military
acquisition projects have shown that a strong push for delivering systems and products
within budget, and on schedule, has either resulted in insufficient attention to human
factors considerations in organizations, or drift away from the human factors’ stan-
dards, which the organization still believes it is pursuing.
Human Systems Integration (HSI) combines the knowledge and skills of different
areas of Human Factors in order to support various levels of systems engineering and
management. The U.S Army was the first large organization that implemented the HSI
approach in system design. The Manpower and Personnel Integration (MANPRINT)
management and technical program was launched in 1986 [2]. Its aim was to improve
military systems and unit performance by focusing on the design of the total system
comprising both soldier performance and equipment. The MANPRINT covered the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1057–1062, 2020.
https://doi.org/10.1007/978-3-030-39512-4_160
1058 J. Laarni and M. Ylönen
following domains: manpower, personnel, training, human factors engineering, system
safety health hazards and soldier survivability [2]. The MANPRINT was defined as “a
comprehensive management and technical program which focuses on the integration of
human considerations into the system acquisition process to enhance human/system
design, reduce life cycle ownership costs, and optimize total system performance” ([3],
p. 1). This was accomplished “by ensuring that the human is fully and continuously
considered as part of the total system in the development and/or acquisition of all
systems” ([3], p. 1).
HSI activities support management and organization, operational processes and
product design. At the conceptual level, the HSI process inputs are all systems that are
associated with systems engineering and management. The output of the HSI process
is, in turn, systemic integration of humans, technology and organization. The HSI
model also emphasizes the application of human-related technologies and disciplines
and focusses on end-users in systems definition, development and deployment. There
are two integration steps for HSI contributions: in the first step, a common focus is
created for all HSI domains, and in the second step, HSI inputs are fully integrated with
the systems engineering and management processes. There are several clear benefits of
HSI in military system design: For example, HSI activities provide input to the design
of complex systems, and quality control at different phases of the design process.
Our aim is to develop the first comprehensive and systematic human-system
integration program for the Finnish Defence Forces. The proposed program consists of
the following seven elements: human-system engineering, including user-centered
design, environmental issues, manning and personnel selection, training, organizations,
leadership and safety culture, work safety and survivability. We will develop detailed
guidance for HSI program execution and provide a checklist of human factors issues to
consider during the acquisition process. In addition, cost-benefit guidance will be
provided for the calculation of a project’s HSI costs, and a survey of human factors
modelling and simulation tools will be conducted. The preliminary draft version of the
program will be evaluated by applying it to a real-world acquisition scenario.
2 Military Acquisition in Finland
The general process of military acquisition in Finland is the following: The Materiel
unit of the Ministry of Defence makes decisions on major acquisitions and steers the
materiel projects [4]. The Finnish Defence Forces prepares and executes acquisitions,
and the Defence Command Finland plays a directing role in acquisitions and materiel-
related work in accordance with the set guidelines and objectives [4].
Military acquisition policies in Finland are regulated by guidance of the Ministry of
Defense. This guidance also touches upon human-system integration issues such as
competence management and organization of training, but there is a real need for
guidance on HSI topics. The proposed HSI program should be integrated into the main
phases of the acquisition process: initial concept definition, requirements specification,
detailed design, implementation and operation. Regarding the life-cycle management,
also maintenance and decommissioning are relevant and should be considered.
Development of a Human System Integration Program in Military Context 1059
3 Key Findings
In the early phases of the project, we reviewed lessons learned from the existing
human-system integration programs and interviewed experts both from the Defence
Forces and the Finnish defence industry. Among the key findings from the interviews
are the following:
• Acquisition process is strongly regulated by the norms of the Defence Forces.
• Interviewees are not very familiar with the concept and program of HSI and its main
elements. Currently, there is trust in contractors’ ability to take into account HSI-/
HOF factors in an adequate way.
• Interviewees recognized soldiers as the best representatives of the end-users.
• Management of requirements is one of the most critical phases in the acquisition
process. Requirements are discussed in the working groups consisting of key
stakeholders. Therefore, also the identification of key stakeholders in the early stage
is crucial.
• Communication of requirements to stakeholders is relevant, since different stake-
holders may interpret them differently.
• Since there are differences between different military branches, interfaces between
different branches, systems and programs must be taken into account.
• Information gaps were identified between engineers and soldiers (i.e., end-users).
Due to these gaps the end results are not always optimal.
• Identification of cultural differences between nations, military branches, and
between engineers and soldiers would be relevant, because cultural misunder-
standings can create delays and extra costs in the acquisition projects.
• Consideration of HSI issues is not only a question of resources, but mostly a
question of organizational culture.
• Career development was considered as a bottleneck for competence development in
HSI topics. This is because military job rotation is continuous. Learning from
projects is not optimal, if one needs to change to another position at frequent
intervals.
4 HSI Program Development
4.1 Integration of HSI into Acquisition Process
A key requirement for a military HSI program is that it is well integrated into the
overall acquisition process. Based on our own observations, some basic HF activities
are practised but they are quite unfocused. Development of the HSI program and
systematic inclusion of human and organizational factors in the acquisition process and
documentation would support the acquisition. In addition, a continuous, timely and
effective dialogue between HSI experts and acquisition specialists is beneficial to
acquisitions.
Two kinds of integration of HSI into the military acquisition process are needed:
Since the HSI area is quite fragmented itself, it has to be integrated first. After that the
1060 J. Laarni and M. Ylönen
HSI activities can be integrated with other domains [5]. For example, there are complex
interactions between HSI activities that have to be settled before it can be started to
think about their integration with other domains and activities [5].
Integration into the overall acquisition process is needed to tackle difficult human
factors issues concerning the linking of human capabilities with the capabilities of
technical systems. It is the responsibility of the HSI process to bring human factors
issues into the acquisition process and, for example, monitor that user requirements are
taken into account in different stages of the project.
One of the key requirements for a unified HSI process is to start HSI activities as
early as possible in the acquisition process and continue them throughout the life-cycle
of the new systems. When HF issues are considered late in the project the required
changes may become costly, and it is possible that they are not optimal. There is also
danger that HSI issues are tackled in an inconsistent and fragmented manner, if they are
addressed in later phases of the project.
How should HSI overlap with the phases of the military acquisition process? One
possible approach is given by [6] by identifying three main goals for human factors
activities, feasibility, usability and reliability that have to be linked to project phases to
adequately manage human factors in the acquisition process (Fig. 1). Feasibility
indicates that a feasible and practicable role is defined for people; usability indicates
that human-system interfaces have to be designed for easy to use and learn; and thirdly,
the reliability of human performance has to be assured in order to reduce the risk of
human error and health hazards.
ENGINEERING OPERATION
Concept Design Requirements & Specifications Detailed Design Implementation Operation Decommissioning
Fig. 1. The overlap of HSI activities with the phases of system design (modified from [6]).
Furthermore, organizational aspects should be considered as a broader phenomenon
than in the current HSI literature, in which the organization is often understood as one
element among other factors. This understanding of organizations remains narrow.
Instead, organizational factors, such as organizational culture that refers to shared
norms, values, beliefs, practices as well as structures in the organizations, can be seen
as enabling or constraining human actions. Thus, an organization can be understood as
providing a framework within which humans act. In organizations people act in specific
positions and roles to which expectations and responsibilities are linked. People
internalize these expectations as norms that affect their action. Therefore, understanding
of mechanisms, through which organizations exert power over individuals and affect
their action and performance is needed. For instance, financial resources, organization’s
Development of a Human System Integration Program in Military Context 1061
mindset, including values and beliefs, and management’s support plays a crucial role in
adoption of HSI.
4.2 Guideline Development
Guidelines are linked to the different HSI elements and different phases of the acqui-
sition process. According to military experts, some of the most important guidelines are
the following:
• Manpower: Will it be possible to reduce the required manpower by suitable design
solutions?
• Personnel: Will new skills, abilities and knowledge bases be required to support the
new capabilities?
• Training: Are individual and/or team-level training requirements identified and
specified?
• Human Factors Engineering: Are HSI domains addressed in requirements specifi-
cation documents? Does the system place excessive physical and cognitive work-
load demands on the Soldier?
• System safety: What is the possibility of a human error caused by poor design,
insufficient feedback, or excessive cognitive load?
• Survivability: Does the system introduce new threats? Do the system’s character-
istics have an impact on its detectability by enemy forces? Does the system is able
to protect the troops from the enemy’s weapons? Does the system is able to prevent
or deter attack?
4.3 Benefits of Human-Systems Integration
There are several clear benefits of HSI in systems design, but the value of HF con-
siderations has not been always acknowledged, and the success of HSI activities has
shown to be difficult to evaluate. HSI is considered a costly process, and there is a
general lack of understanding of how HF could produce cost benefits [7]. Because of
big investments without certainty of later return, there are a lot of reservations against
HSI. Because of these reservations, one of our aim is to estimate the cost-benefit of HSI
in a precise fashion. We will follow the guidance provided by [7], according to which
there are two parts in the justification of HSI: first, it has to identify and specify the cost
effects of potential problems; and second, it has to demonstrate by which way HSI adds
value.
5 Conclusions and Recommendations
Some of the main guiding lights throughout the process are the following:
• HSI activities have to be fully integrated into different phases of the acquisition
process and throughout the system lifecycle.
• HSI activities have to be started as early as possible in the design process, and they
should continue throughout the life-cycle of the target system.
1062 J. Laarni and M. Ylönen
• Many of the HSI activities must be iterative by nature, and there must be a sys-
tematic procedure for aggregating and systematizing system evaluation data.
• Organizational aspects should be considered more broadly than it is currently made
in the HSI literature, and as relevant precondition for any HSI activity.
• An organizational unit is required that takes care of the execution of HSI activities,
and is responsible for the communication of HSI-related information prospects to
the whole organization.
• Detailed analyses of HSI costs and benefits are required, and these analyses should
be repeated iteratively throughout the acquisition process.
Acknowledgments. The research was funded by MATINE, The Finnish Scientific Advisory
Board of Defence.
References
1. Harvey, C., Stanton, N.A.: Safety in system-of-systems: ten key challenges. Saf. Sci. 70, 358–
366 (2014)
2. Booher, H.R.: Introduction: human systems integration. In: Booher, H.R. (ed.) Handbook of
Human Systems Integration. Wiley, Hoboken (2003)
3. U.S. Army: Manpower and personnel integration (MANPRINT) in the systems acquisition
process. Army Regulation 602-2. Department of the Army, Washington, D.C. (2001)
4. https://www.defmin.fi/en/tasks_and_activities/resources_of_the_defence_administration/
materiel_policy/materiel_policy_and_procurement
5. Madni, A.M.: Integrating humans with software and systems: technical challenges and a
research agenda. Syst. Eng. 13, 232–245 (2010)
6. Hamilton, W.I., Kazem, M.L.M., He, X., Dumolo, D.: Practical human factors integration in
the nuclear industry. Cogn. Technol. Work 15, 5–12 (2013)
7. Bruseberg, A.: Cost-benefit analysis for human factors integration: a practical guide. HFI
DTC/WP 2.7.2/3. BAE Systems (2009)
Beyond Confluence, Integration
and Symbiosis: Creating More Aware
Relationships in Smart Cities
H. Patricia McKenna(&)
AmbientEase, #541, 185-911 Yates Street, Victoria, BC V8V 4Y9, Canada
[email protected]
Abstract. The main objective of this paper is to explore the grand challenge of
human-technology symbiosis in the context of smart cities. A review of the
research literature for awareness in relation to the concepts of confluence,
integration, and symbiosis in smart cities, enables formulation of a conceptual
framework. The methodology for this work consists of an exploratory case study
involving the use of survey and in-depth interviews combined with an
explanatory correlational design. Data are gathered from diverse individuals in
small to medium to large-sized cities across several countries and analysis
consists of descriptive statistics for quantitative data and content analysis for
qualitative data. This paper is significant because it contributes to the research
literature for smart cities; advances a conceptual framework for heightening the
realization of human potential in smart urban environments; and highlights the
construct of awareness for exploring relationships with sensing, perception, and
interaction in smart cities.
Keywords: Aware people Aware technologies Confluence Human-
technology Symbiosis Integration Relationships Smart cities
1 Introduction
The main objective of this paper is to explore one of several “grand challenges for
living and interacting in technology augmented environments” identified by
Stephanidis et al. [1] as that of human-technology symbiosis. In the context of Intel-
ligent Human Systems Integration (IHSI) and Integrating People and Intelligent Sys-
tems (IPIS), the human-technology symbiosis grand challenge is addressed in this work
in relation to applications and future trends, through probing more closely the concepts
of confluence, integration, and symbiosis in smart cities. Gil-Garcia, Pardo, and Nam
[2] highlight the need to understand the interdependencies and components of smart-
ness such as technology, data, and associated aspects of the environment and society.
Konomi and Roussos [3] claim that, “we are now going beyond the last decade’s
conception of smart cities” and moving “towards a deeper level of symbiosis among
smart citizens, Internet of Things and ambient spaces.” Theoretically, this paper is
positioned at the intersection of interactivities and relationships between more aware
people and aware technologies. As such, through enhanced awareness, enabled through
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1063–1068, 2020.
https://doi.org/10.1007/978-3-030-39512-4_161
1064 H. P. McKenna
more aware technologies in combination with more aware people leveraging their
multi-sensorial capabilities, what is proposed in this work is a renewed, amplified, and
enhanced connection with human capabilities and potentials. This paper is significant
in that it contributes to the research literature for smart cities; addresses the Human-
Computer Interaction (HCI) grand challenge of human-technology symbiosis in the
context of smart cities; and advances a conceptual framework for heightening the
realization of human potential in smart urban environments, spaces, and regions.
This background and context gives rise to the main research question under
exploration in this work – How and why are people and technology relationships
important in smart cities?
2 Theoretical Perspective
A review of the research literature is provided for confluence, integration, and sym-
biosis in the context of smart cities, followed by a summary and conceptual framework.
2.1 Confluence
Human computer confluence (HCC) is articulated by Ferscha [4] in the form of a
research agenda where, based on European initiatives, HCC is described as an
“emerging symbiotic relationship between humans and ICT (Information and Com-
munication Technologies)” said to be “based on radically new forms of sensing, per-
ception, interaction and understanding.” As such, HCC “refers to an invisible, implicit,
embodied or even implanted interaction between humans and system components.” As
an emerging area with “some 200 researchers (European and international)”, Ferscha
[4] highlights for HCC a range of elements including: large scale socio-technical
systems; ethics and value sensitive design; augmenting human perception and cogni-
tion; empathy and emotion; experience and sharing; and the importance of the com-
puting interface as disappearing. Regarding the interface, Ferscha [4] claims that “the
nature and appearance of computing devices has changed” so as to be “hidden in the
fabric of everyday life, invisibly networked, and omnipresent” such that “‘everything’
has become the interface” from “things and objects of everyday use” to “the human
body, the human brain, the environment, or even the whole planet” such that “implicit
interaction replaces explicit interaction.” Van de Velde [5] claims that HCC “is not
about augmentation of specific human capabilities” including perception, and is instead
“about empowering humans to use their abilities in new ways.”
2.2 Integration
Zygiaris [6] developed a measurement system based on six layers of a smart city, one
of which is that of the open integration layer with apps to communicate and share data,
content, services, and information. It is worth noting that Farooq and Grudin [7] state
that, “integration implies partnership” identifying a series of everyday scenarios
involving examples of interaction on the one hand and examples of integration on the
other, concluding that “integration extends but doesn’t replace interaction.” Farooq
Beyond Confluence, Integration and Symbiosis 1065
et al. [8] argue that integration may “be dangerous and ethically questionable in life-
critical applications” while looking to “other ways to enhance human capabilities
through more powerful tools.” Lim, Kim, and Maglio [9] identified a series of chal-
lenges when developing “a framework for data use for smart cities” where focus is “on
the local in terms of operations management, preventive administration, network
development, and information diffusion” which included integration. The notion of
“ambient sensing”, as in, more dynamic and creative ways of sensing, is advanced by
McKenna [10] “as an integrating element for aware people and technologies.”
2.3 Symbiosis
Farooq and Grudin [7] seem to use the concepts of integration and symbiosis inter-
changeably, stating that, “it took us 50 years to reach integration or symbiosis”
claiming that, “we are now there”, understood in terms of an “interaction-integration
continuum.” Ferscha [4] speaks of HCC in terms of “the emerging symbiosis of
technology and society.” However, Farooq et al. [8], in the context of a panel dis-
cussion, make a distinction, arguing that symbiosis “implies independent entities with
their own goals that happen to help and rely on one another” while integration “implies
a potentially more intimate but less symmetrical relationship.”
2.4 Summary and Conceptual Framework
Table 1 provides a summary by author and year, from 2013 to 2019, for elements
associated with confluence, integration, and symbiosis. Confluence includes a new
research agenda involving people and the element of human empowerment, enabling
people “to use their abilities in new ways.” Integration includes the elements of
communicate, data use, ethical issues, intimate relationships, less symmetrical rela-
tionships, partnership, risk issues, sensing (ambient), and sharing. Symbiosis includes
the elements of independent entities, integration, and society and technology.
Table 1. Elements associated with confluence, integration, and symbiosis.
Author(s) Year Confluence Integration Symbiosis
Zygiaris 2013 Communicate, Share
Farooq & 2016 Partnership Integration
Grudin
Ferscha 2016 Research Society &
agenda technology
Van de 2016 Empowerment
Velde
Farooq 2017 Ethical issues; Risk issues; Intimate Independent
et al. relationships; Less symmetrical entities
relationships
Lim et al. 2018 Data use
McKenna 2019 Sensing (ambient)
1066 H. P. McKenna
This theoretical perspective enables formulation and operationalization of a con-
ceptual framework in this paper for heightening human capabilities and potentials in
smart cities, as depicted in Fig. 1. Framed through a people – technologies – cities
dynamic, the construct of awareness provides a lens for emerging aware applications
and trends in support of integration, confluence, and symbiosis for sensing, perception,
and interaction, characterized by partnerships and relationships as key for heightening
human capabilities and potentials in smart cities, with implications for the environment
and society, more generally.
Fig. 1. Conceptual framework for heightening human capabilities and potentials in smart cities.
The research question presented in the Introduction section is reformulated here as
a proposition under investigation in this work, as follows:
P1: People and technology relationships are important in smart cities as a mechanism
for understanding the interconnections between variables such as awareness and
sensing, awareness and perception, and awareness and interaction, in support of
extending human capabilities and potentials.
3 Methodology
Methodologically, the research design for this work consists of an exploratory case
study approach incorporating the use of survey and in-depth interviews for the gath-
ering of data from diverse individuals in small to medium to large-sized cities across
several countries. Data analysis techniques include the use of content analysis for
qualitative data and descriptive statistics in the form of an explanatory correlational
design for quantitative data. In parallel with this study, many group and individual
discussions were also conducted across multiple sectors in several Canadian cities.
The Real Statistics Software [11] Resource Pack add-in for Microsoft Excel,
containing a range of supplemental statistical functions and data analysis tools, is used
in this work for the correlating of awareness with the variables of sensing and
Beyond Confluence, Integration and Symbiosis 1067
interaction. Overall, data were analyzed for an n = 76 consisting of 41% females and
59% males, spanning the age ranges of people in their 20 s to their 70 s.
4 Findings and Discussion
Findings in this work are described in terms of relationships between awareness and
sensing, awareness and perception, and awareness and interactions.
Awareness and Sensing. Based on survey findings, using “heightening urban sensi-
bilities” as an indicator of awareness and feelings of “comfort” as an indicator of
sensing, when correlated as ordinal data, a Spearman correlation coefficient of .23 was
achieved. Creswell [12] claims that with correlations ranging from “.20 to .35, there is
only a slight relationship” such that “this size of coefficient may be valuable to explore
the interconnection of variables” while having “little value in prediction studies.”
Qualitatively, reference was made to the importance of human needs and values
combined with aware “tech systems to make the city a better place.”
Awareness and Perception. When asked, “How do you think people are becoming
more aware of smart cities and smart technologies”, it is worth noting that 100% of
survey respondents selected the option, “people generally are not aware of smart
cities.” Qualitatively, findings highlight the potential for aware technologies in urban
spaces to “assist in citizen/visitor education” and “awareness” while also providing
“mobile apps for orientation and information, not just navigation.” Smart city tech-
nologies are said to support elements such as “faster pattern recognition” and “better
resource allocation” with attention to the creation of interactive spaces that are “as
intuitive and frictionless as possible”, important for involving and engaging people.
Awareness and Interactions. Based on survey findings, using “heightening urban
sensibilities” as an indicator of awareness and “interactive public spaces” as an indi-
cator of interactions, a Spearman correlation coefficient of .81 was achieved. Creswell
[12] indicates that correlations in the .66 to .85 ranges are “very good” with “good
prediction.” The importance of “interactive urban displays” was highlighted by 50% of
survey respondents while qualitatively, “interactive kiosks” emerged as important.
5 Limitations and Future Directions
A key limitation of this work is the small sample size and this is mitigated by the
potential to extend this study more broadly to include more participants in more cities
and more countries. Although what Creswell would consider to be a good correlation,
in the .66 to .85 range [12] was not found in this work between awareness and the
variable of sensing where only a slight relationship emerged, a very good correlation
was found for awareness and interactions, along with some good prediction value. As
such, findings in this work identify spaces for the creating of more aware relationships
in support of explorations for enhancing human potential in smart cities as future
directions for research and practice.
1068 H. P. McKenna
6 Conclusion
In response to the human-technology symbiosis grand challenge for HCI [1], this work
opens novel spaces for discourse/debate on aware people and technologies through
development and operationalization of a conceptual framework for heightening human
capabilities and potentials in smart cities. Use and validation of the framework is
invited in exploring and creating more aware relationships in smart cities.
Acknowledgments. This work was supported in part by an Independent Scholars Grant (2019)
from the Canadian Academy of Independent Scholars (CAIS) through Simon Fraser University
(SFU), Vancouver, British Columbia, Canada.
References
1. Stephanidis, C., Salvendy, G. (Chairs), Antona, M., Chen, J.Y.C., Dong, J., Duffy, V.G.,
Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L.P., Guo, Y., Harris, D., Ioannou, A., Jeong,
K., Konomi, S., Krömker, H., Kurosu, M., Lewis, J.R., Marcus, A., Meiselwitz, G.,
Moallem, A., Mori, H., Nah, F.F.-H., Ntoa, S., Rau, P.-L.P., Schmorrow, D., Siau, K.,
Streitz, N., Wang, W., Yamamoto, S., Zaphiris P., Zhou, J. (Members of the Group):
Seven HCI grand challenges. Int. J. Hum.-Comput. Interact. 35(14), 1229–1269 (2019).
https://doi.org/10.1080/10447318.2019.1619259
2. Gil-Garcia, J.R., Pardo, T.A., Nam, T.: What makes a city smart? Identifying core
components and proposing an integrative and comprehensive conceptualization. Inf. Polity
20(1), 61–87 (2015)
3. Konomi, S., Roussos, G.: Enriching Urban Spaces with Ambient Computing, the Internet of
Things, and Smart City Design. IGI Global, Hershey (2017)
4. Ferscha, A.: A research agenda for human computer confluence. In: Gaggioli, A., Ferscha,
A., Riva, G., Dunne, S., Viaud-Delmon, I. (eds.) Human Computer Confluence Transform-
ing Human Experience Through Symbiotic Technologies, pp. 7–17. De Gruyter Open,
Warsaw, Berlin (2016)
5. Van de Velde, W.: Foreword. In: Gaggioli, A., Ferscha, A., Riva, G., Dunne, S., Viaud-
Delmon, I. (eds.) Human Computer Confluence Transforming Human Experience Through
Symbiotic Technologies, pp. 1–4. De Gruyter Open, Warsaw, Berlin (2016)
6. Zygiaris, S.: Smart city reference model: assisting planners to conceptualize the building of
smart city innovation ecosystems. J. Knowl. Econ. 4, 217–231 (2013)
7. Farooq, U., Grudin, J.: Human-computer integration. Interactions 23(6), 26–32 (2016).
https://doi.org/10.1145/3001896
8. Farooq, U., Grudin, J., Shneiderman, B., Maes, P., Ren, X.: Human computer integration
versus powerful tools. In: Proceedings of the 2017 CHI Conference Extended Abstracts on
Human Factors in Computing Systems, pp. 1277–1282. ACM, New York (2017). https://doi.
org/10.1145/3027063.3051137
9. Lim, C., Kim, K.-J., Maglio, P.P.: Smart cities with big data: reference models, challenges,
and considerations. Cities (2018). https://doi.org/10.1016/j.cities.2018.04.011
10. McKenna, H.P.: Ambient urbanities: a synthesis. In: McKenna, H.P. (ed.) Ambient Urbanities
as the Intersection Between the IoT and the IoP in Smart Cities. IGI Global, Hershey (2019)
11. Zaiontz, C.: Real Statistics Using Excel (2019). www.real-statistics.com
12. Creswell, J.W.: Educational Research: Planning, Conducting, and Evaluating Quantitative
and Qualitative Research, 6th edn. Pearson, Boston (2018)
A Potential Analysis of Cognitive Assistance
Systems in Production Areas
Jessica Klapper1(&), Bastian Pokorni2, and Moritz Hämmerle2
1
University of Stuttgart, Nobelstraße 12, 70569 Stuttgart, Germany
[email protected]
2
Fraunhofer Institute for Industrial Engineering, Nobelstraße 12,
70569 Stuttgart, Germany
{Bastian.Pokorni,Moritz.Haemmerle}@iao.fraunhofer.de
Abstract. A cognitive assistance system supports the person in information
gathering and preparation, decision-making, as well as performing work tasks
and coordinating people, machines, processes and products [1]. At present, a
large number of cyber physical systems are coming onto the market, which also
find their way into today’s factories. This paper discusses which technologies
are interesting today for research and companies. A short study was used to
reflect the current use of cognitive assistance systems in the production envi-
ronment and to determine the prospects for the future.
Keywords: Assistance systems Digitalization Survey Industry 4.0
Manufacturing Cyber-physical systems
1 Introduction
The challenges that are currently affecting companies are various. Demographic
change, diversity, globalization, flexibility requirements in order to serve individual
customer requirements and delivery dates [2].
Cognitive assistance systems offer the possibility of integrating digitalization into
the company and master upcoming challenges successfully. The advantage is that the
investments and the changes made by the assistance systems can be designed to be
particularly variable. The development challenge is to find the right assistance system
for the specific company and task [3].
To address this difficulty, a short study was carried out. The aim of the study is to
gain a snapshot of the status quo regarding the use of cognitive assistance systems and
to determine future potentials and the opportunities and barriers companies see in the
context. For the survey, an anonymous online questionnaire on LimeSurvey was cre-
ated and sent to manufacturing companies and advertised in newsletters. Within the
questionnaire there is a division of questions for (potential) users and (potential)
manufacturers of cognitive assistance systems. The questionnaire included 17 questions
that had been submitted for participation over a period of two months and had 144
participants. Of these, 125 participants were (potential) users of cognitive assistance
systems. Half of the participants belonged to the mechanical and plant engineering
industries or the automotive industry. 9% of the participants were business leaders,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1069–1073, 2020.
https://doi.org/10.1007/978-3-030-39512-4_162
1070 J. Klapper et al.
40% head of a business unit and 51% non-executive employees. The majority of 68%
of the interviewees came from the areas of production, assembly, logistics, industrial
engineering and work preparation.
2 Demarcation of Assistance Systems
The assistance systems in the production environment are differentiated into physical
and cognitive assistance systems.
Physical assistance systems assist people with demanding physical activities. They
compensate for physically declining abilities and the prevention of physical overuse
[4]. Often, collaborative robots are used [5].
Cognitive assistance systems offer the possibilities of a closer cooperation
between human and machine, with the aim of unifying the outstanding abilities of
humans with the special characteristics of machines as “best of two worlds” [4]. An
assistance system records the actions of the user and reprocessed information back to
them. The employee processes this information and uses it to solve his tasks. The
system requires a communication interface (frontend) [6]. This consists of an input
system for data acquisition from the operator side and an output system for information
transfer [3]. It also means that the human being is supported by the technical systems,
which in the ideal case corresponds exactly to their abilities and needs as well as the
requirements of the work context [4]. The cognitive assistance systems do not support
(or replace) the employee directly in the physical execution. Part of the survey were
cognitive assistance systems only.
3 Today’s Potential of Cognitive Assistance Systems
At the beginning of the survey, it was to be determined how many of the participants
are already using cognitive assistance systems or thinking about using them. The
results are very varying, almost as many companies are considering (32%) or using
assistance systems (36%). 15% of the 123 respondents have concrete plans.
Their motivations are particularly relevant to companies’ implementation plans for
cognitive assistance systems. One question was therefore asked about the benefits of
using them (N = 119).
Four out of five participants expect higher productivity (79%) through cognitive
assistance. Process control (69%), quality and cost-effectiveness (66%) are further
important features that a user hopes for. A little less often, the employee-related criteria
were selected. For example, only about 40% of companies hope for higher employee
satisfaction. Control of employees hardly matters. The question was posed to both
users and eighteen providers of assistance systems.
A Potential Analysis of Cognitive Assistance Systems in Production Areas 1071
4 Usage of Cognitive Assistance Systems
In addition to the general use of assistance systems, this survey also dealt in detail with
the technology, the area of application and the single-task assignment. Four divisions
have been defined as manufacturing, assembly, maintenance and logistics.
Furthermore eight technologies Virtual Reality (VR) [6], Augmented Reality
(AR) [7], Condition Monitoring [8], Radio Frequency Identification (RFID) [9],
Picking technologies, Image Processing [10], visualization and position detection are
identified that are particularly relevant for the use as a cognitive assistance system.
Based on the business units and technologies, a matrix query was performed. The
aim of this complex questioning method was to link the technologies and the divisions
in order to determine their potential for the use of cognitive assistance systems. The
results are shown in the Table 1.
Table 1. In which divisions do you use the following technologies as cognitive assistance
systems or is their use planned? (N = 119)
One in three respondents uses the AR applications in assembly and maintenance.
The possibilities of condition monitoring are mainly used in production.
One in two already uses picking technologies in logistics and increasingly in
assembly. Almost no company renounces the possibilities of visualization and
increasingly uses this in assembly, which at the same time, at 67%, is the highest
expression of the respondents in this question. One of the most important questions
about the use of cognitive assistance systems is the purchase criteria according to which
users select them.
1072 J. Klapper et al.
5 Implementation of Cognitive Assistance Systems
One of the most important question on this topic is the purchase criteria that lead to the
acquisition of a cognitive assistance system. To increase the validity of the answers,
participants who have not considered, planned or used it have not been asked this
question.
The results are that high compatibility is very important for the purchase of a
cognitive assistance system and ergonomics plays a role in the acquisition. The pur-
chase criterion data protection is less important at 65%. For only two out of three
companies, data protection is very important.
The price-performance ratio plays a minor role for the question of the purchase
criteria, the high costs of the assistance system are the most-mentioned reason in the
question of the obstacles. This result is attributable to the fact that the participants were
also interviewed about the obstacles without the planned use of assistance systems.
Every second participant indicates that the technical requirements are not
(yet) fulfilled. The problems of technology adaptation could be a cause for this, where
not the availability of new technological concepts is usually problematic, but the
internal diffusion of new technologies in the company [11]. Another criterion is
employee acceptance. 43% see this as an obstacle. Digital transformation only succeeds
if the fears and emotions of the employees are taken seriously [12].
6 Conclusion
Combining the knowledge gained about technology and field of application, the fol-
lowing application combinations have the highest potential:
• In production, RFID chips and Condition Monitoring have largely arrived.
• In the assembly division, visualization technologies and image processing are
supportive.
• Augmented reality in connection with condition monitoring already see respondents
as an established use case in maintenance.
• The possibilities of picking technologies are widely used in logistics.
The challenge of finding the right assistance system for the individual company
remains. The evaluation of current and future potentials shows that there is no clear
assignment or a clear tendency of the individual technologies and their possible
applications. A similar result can be found in the Technology Compendium “Interac-
tive Assistance Systems”. Again, the assessments of optical, acoustic and haptic
assistance systems were little differentiated [13]. It is furthermore important for
research and practice to provide the appropriate support here. Acquired know-how and
experiences should be communicated and developed within the appropriate rooms.
Modern methods of project management based on SCRUM can offer providers and
users of assistance systems the opportunity to achieve faster successful launches.
A Potential Analysis of Cognitive Assistance Systems in Production Areas 1073
Acknowledgments. This short study is funded within the project “Mittelstand 4.0 Kompe-
tenzzentrum Stuttgart” by the German Federal Ministry for Economic Affairs and Energy. The
survey was developed in collaboration with Robert Rothenberger the managing director of the
company memex GmbH from Dettingen/Teck, Germany. The company is a developer and
provider of utility films.
References
1. Kagermann, H., Wahlster, W., Helbig, J. (eds.): Deutschlands Zukunft als Produktionsstan-
dort sichern - Umsetzungsempfehlungen für das Zukunftsprojekt Industrie 4.0: Abschluss-
bericht des Arbeitskreises Industrie 4.0, Berlin (2012)
2. Adenauer, S.: Leistungsfähigkeit im Betrieb: Kompendium für den Betriebspraktiker zur
Bewältigung des demografischen Wandels, pp. 9–18
3. Merazzi, J., Friedel, A.: Einteilung und Bewertung von Montageassistenzsystemen. In: ZWF
Zeitschrift für wirtschaftlichen Fabrikbetrieb, pp. 413–416
4. Deutschland. Bundesministerium für Arbeit und Sozialordnung (ed.)
Forschungsbericht/Bundesminister für Arbeit und Sozialordnung, Referat L P 3, Bonn
Bundesministerium (2018)
5. Hold, P., Ranz, F., Sihn, W.: Konzeption eines MTM-basierten Bewertungsmodells für
digitalen Assistenzbedarf in der cyber-physischen Montage. In: Schlick, C. (ed.) Megatrend
Digitalisierung - Potenziale der Arbeits- und Betriebsorganisation, GITO, Berlin, pp. 295–
322 (2016)
6. Stahl, T., Völter, M., Efftinge, S., Haase, A. (eds.): Modellgetriebene Softwareentwicklung:
Techniken, Engineering, Management, 1st edn. dpunkt.verlag, s.l. (2012)
7. Kruse Brandão, T., Wolfram, G. (eds.): Digital Connection: Die bessere Customer Journey
mit smarten Technologien - Strategie und Praxisbeispiele. Springer Fachmedien Wiesbaden,
Wiesbaden (2018)
8. Lin, M.: Augmented Reality – The Next Big Thing (2018). https://imbstudent.donau-uni.ac.
at/mensch-maschine-interaktion/2018/02/26/augmented-reality-the-next-big-thing/. Acces-
sed 8 Nov 2018
9. Niggemann, O.: Selbstlernende Systeme zur Anlagenüberwachung. In: VDMA Verlag
GmbH (ed.) Am Puls der Maschine: Condition Monitoring, 8000th edn. VDMA Verlag
GmbH, Frankfurt, pp. 8–9 (2015)
10. Hierzer, R. (ed.): Prozessoptimierung 4.0: Den digitalen Wandel als Chance nutzen, 1st edn.
Haufe Gruppe, Freiburg, München, Stuttgart (2017)
11. Fraunhofer IOF, Industrielle Bildverarbeitung: Schlüsseltechnologie für Industrie 4.0 (2018).
https://www.vision.fraunhofer.de/de/technologien-anwendungen/anwendungsfelder/
industrie_4.html
12. Ganz, W., Warschat, J., Kremers, D., Leyh, J. (eds.) Neue Technologien schneller in
innovative Produkte umsetzen: Kurzstudie zu den Unterstützungspotenzialen der Arbeits-
gestaltung und Personalentwicklung für die Technologieadaption in Hightech-Unternehmen,
Fraunhofer-Verl., Stuttgart (2010)
13. Borgmaier, A., Borgmeier, A., Grohmann, A., Gross, S.F. (eds.): Smart services und Internet
der Dinge: Geschäftsmodelle, Umsetzung und Best Practices Industrie 4.0, Internet of
Things (ioT), Machine-to-Machine, Big Data, Augmented Reality Technologie, Hanser,
München (2017)
Identifying and Analysing Risk Factors
from a Sociotechnical System Perspective:
A Case Study
Amangul A. Imanghaliyeva(&)
Institute for Infrastructure and Environment, Heriot-Watt University,
Edinburgh EH144AS, UK
[email protected], [email protected]
Abstract. In the current paper, a set of 20 synthesised sociotechnical principles
[1] is used to examine how the multidisciplinary teams’ performance affects the
project design delivery. The paper aims to address the following research
objectives: (1) whether an approach based on the 20 sociotechnical principles
[1] is one that can lead to successful project design; and (2) identify and analyse
risk factors from a Sociotechnical System (STS) perspective. To this end, the
above objectives were achieved by analysing the team performance of students
participating in a collaborative design case project at Heriot-Watt University.
The case study findings confirmed the importance of the sociotechnical prin-
ciples [1] and risk factors associated with a collaborative design project. Overall,
the findings demonstrate the relevance of the sociotechnical principles for
addressing project design problems.
Keywords: Risk factors Sociotechnical principles Sociotechnical system
Collaborative design project Multidisciplinary team Team performance
Winning Losing
1 Introduction
Undergraduate students from the School of Energy, Geoscience, Infrastructure and
Society of Heriot-Watt University from various disciplines formed 22 multidisciplinary
teams to undertake a collaborative design project. The performance and effectiveness of
the teams were measured within two phases of data collection: (1) Week 1, which
involved a five-day project design task from which the observational study resulted and
voting results were obtained, and (2) Week 6, when a survey of 161 students and 4
semi-structured interviews were conducted. The independent measure was team per-
formance and effectiveness, as measured by a voting procedure applied to each team’s
project outputs. This helped to identify two categories of teams: winning teams and
losing teams. The dependent measure was the team performances based on 20
sociotechnical principles [1] and 18 risk factors, measured by the questionnaire survey
and the semi-structured interviews.
However, in order to fit within the Conference Proceedings’ page limit policy, the
current paper presents the data analysis and findings obtained from only Sect. 3
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1074–1081, 2020.
https://doi.org/10.1007/978-3-030-39512-4_163
Identifying and Analysing Risk Factors from a STS Perspective 1075
(Identification & Analysis of Risk factors affecting the Collaborative Design Projects) of
the questionnaire survey conducted in Week 6 of the Collaborative Design Project study.
2 Method
2.1 Participants
Phase 1 (Week 1): There were 220 students organised into 22 multidisciplinary teams
of 10 students. The team members were from various disciplines including Interior
Design (ID), Architectural Engineering (AE), Civil and Structural Engineering (CSE),
Construction Project Management and Quantity Surveying (CPMQS) and Geography,
Urban Planning and Property Development (UPPD).
Phase 2 (Week 6 - Questionnaire Survey): There were 22 teams with respondents
ranging from 4–10 students per team, with a total of 161 respondents, 108 of which
were male students and 53 female.
2.2 Materials
Phase 1 (Week 1): The design project teams needed to design a Construction Skills
College in the North East part of Edinburgh, UK. The design project had to address
infrastructure and landscaping, as well as public transport, out-of-hours community
use, and services deliveries.
Phase 2 (Week 6 - Questionnaire Survey): The questionnaire comprised three sec-
tions: (1) General background information; (2) Team performance and effectiveness
questionnaire and (3) Identification & Analysis of Risk factors affecting the Collabo-
rative Design Projects.
Section 3 of the questionnaire was designed [2] to identify and analyse the possible
occurrence of 18 risk factors that might negatively affect the project design. Respon-
dents were asked to select from among the five options (very low, low, medium, high
and very high) to indicate the significance of the18 risk factors.
18 risk factors were mapped onto the 20 sociotechnical principles [1] (see Table 1).
The mapping exercise was established to identify and analyse risk factors from the STS
perspective.
2.3 Procedure
Survey/Questionnaire Data Collection Procedure): Aweb link using the Heriot-Watt
University’s Vision service was created to distribute some of the questionnaires using
email. This was done in the following ways: (1) already identified potential participants
were reminded by emails to complete a survey following the web link; (2) additionally,
participants were invited to participate in ‘drop-n’ sessions five times a week to
complete the questionnaires for an incentive payment £3 for each completed survey.
1076 A. A. Imanghaliyeva
Table 1. Mapping 18 risk factors onto the 20 sociotechnical principles
№ Identification & analysis of 18 Lack of one or more Notes/explanations
risk factors affecting the of the 20
collaborative design project sociotechnical
principles [1]
1 Lack of support from ruling 2 Congruence Team members should ensure that
party support systems and sub-systems are
congruent with the primary project
design
2 Project scope changes 6 Designing design Team members should recognise
7 Experimentation that each project design process is
15 Uniqueness different and itself is an STS, which
should be treated as ‘experiments’
3 Project being cancelled due to 9 Responsibility Team members should be given
change in ruling party 20 Self-regulation responsible autonomy for carrying
out tasks, and they should also be
given optimised conditions under
which they become self-regulating
4 Lack of mutual trust and 1 Multifunctionalism Team members should ensure that
understanding among the 11 Boundaries boundaries in the project design
project’s parties process, including the lack of mutual
trust and understanding among
project participants, do not interfere
with the exchange of information
and learning. Team members should
draw from a diverse range of skills,
and they should be skilled in more
one function
5 Project funding problems 2 Congruence Team members should identify any
12 Constraints constraints and ensure that support
systems and sub-systems are
congruent with the basic project
design
6 Project site acquisition 2 Congruence Team members should identify any
problems 12 Constraints constraints and ensure that support
systems and sub-systems are
congruent with the basic project
design
7 Contractual disputes 10 Values Project parties should express one
another’s aspirations and needs
within their design criteria
8 Communication breakdown 5 Participation Team members should match
among the different project 11 Boundaries democratic teamwork structures with
parties democratic project design processes
and ensure any project design
boundaries do not interfere with the
exchange of information and
learning
(continued)
Identifying and Analysing Risk Factors from a STS Perspective 1077
Table 1. (continued)
№ Identification & analysis of 18 Lack of one or more Notes/explanations
risk factors affecting the of the 20
collaborative design project sociotechnical
principles [1]
9 Lack of commitment between 5 Participation Team members should be motivated
project parties 18 Job characteristics and interested to take forward the
project design
10 Shortage of specialised project 1 Multifunctionalism Team members should draw from a
experts 13 Multidisciplinarity diverse range of disciplines, and they
should be skilled in more than one
function
11 Shortage of technical skilled 13 Multidisciplinarity Team members should draw from a
project design team members diverse range of disciplines
12 Lack of joint risk mechanism by 4 Joint optimisation Team members should establish a
project parties 6 Designing design mechanism to prevent any issues,
unexpected, unplanned events,
deviations, disturbances,
interruptions that prevent meeting
the project deadlines
13 Lack of situational awareness 14 Resource flows Team members need to ensure that
among the project parties information is provided at the point
at which it is required
14 Lack of coordination and 5 Participation Team members should match
communication among the 11 Boundaries democratic teamwork structures with
project parties democratic project design processes
and ensure any project design
boundaries do not interfere with the
exchange of information and
learning
15 Inexperienced, inadequate 1 Multifunctionalism Team members should draw from a
project design team members 13 Multidisciplinarity diverse range of disciplines, and they
should be skilled in more than one
function
16 Losing critical project design 13 Multidisciplinarity Team members should draw from a
team members at crucial points diverse range of disciplines and
of designing the project skills
17 Insufficient time for project 12 Constraints Team members should identify any
planning obstacles that prevent project design
delivery
18 Additional needs requested by 10 Values Project parties should express one
project parties another’s aspirations and needs of
one another’s within their design
criteria
1078 A. A. Imanghaliyeva
3 Results
3.1 Possibility of Missing Sociotechnical Principles in Winning
and Losing Teams During the Collaborative Design Project Study
Table 2 shows that the losing teams were more likely to have a larger number of
missing sociotechnical principles [1] compared to the winning teams. Particularly for
‘multidisciplinarity’ (13), there was a gap of almost 27 points between the mean score
for the absence of this principle in losing teams (51.11) and that for the winning teams
(24.52), and similarly, for absence of ‘boundaries’ (11), the losing teams’ score was
significantly higher (50.97) than that the winning teams (24.71). The scores for the
likelihood of ‘congruence’ (2) and ‘constraints’ (12) being missing principles in losing
teams was significantly higher than for winning teams, by a margin of about 22 points.
The scores for four other principles, ‘multifunctionalism’ (1), ‘participation’ (5) ‘val-
ues’ (10) and ‘functional purposes’ (17) also differed significantly, with the losing
teams scoring significantly higher for likely absence of these critical principles than
winning teams, although by smaller margins of around 15. There were no statistically
significant differences between winning and losing teams in the scores for the
remaining sociotechnical principles.
Table 2. The Mann-Whitney U Tests for 20 sociotechnical principles derived from the 18 risk
factors affecting design projects via the mapping exercise.
№ 20 Sociotechnical Mann-Whitney U tests
principles Winning Losing U-Value Z Significance Result
teams teams
Mean Rank
1 Multifunctionalism 31.7 45.96 1.033 2.756 0.006 Reject the null
hypothesis
2 Congruence 26.65 49.58 1,199.5 4.412 0.000 Reject the null
hypothesis
3 Flexible 38.38 41.16 812.5 0.555 0.579 Retain the null
specification hypothesis
4 Joint optimisation 39.95 40.03 760.5 0.016 0.987 Retain the null
hypothesis
5 Participation 31.55 46.07 1,038 2.795 0.005 Reject the null
hypothesis
6 Designing design 40.98 39.29 726.5 0.337 0.736 Retain the null
hypothesis
7 Experimentation 43.27 37.65 837.5 −1.1 0.271 Retain the null
hypothesis
8 Flux 37.62 41.71 774 0.805 0.421 Retain the null
hypothesis
9 Responsibility 39.58 40.3 773 0.145 0.884 Retain the null
hypothesis
(continued)
Identifying and Analysing Risk Factors from a STS Perspective 1079
Table 2. (continued)
№ 20 Sociotechnical Mann-Whitney U tests
principles Winning Losing U-Value Z Significance Result
teams teams
Mean Rank
10 Values 31.77 45.9 1,030.50 2.737 0.006 Reject the null
hypothesis
11 Boundaries 24.71 50.97 1,263.50 5.034 0.000 Reject the null
hypothesis
12 Constraints 27.05 49.29 1,270 5.106 0.000 Reject the null
hypothesis
13 Multidisciplinarity 24.52 51.11 1,186.50 4.27 0.000 Reject the null
hypothesis
14 Resource flows 36.67 42.39 869 1.144 0.253 Retain the null
hypothesis
15 Uniqueness 39.95 40.03 760.5 0.016 0.988 Retain the null
hypothesis
16 Variance control 39.58 40.3 773 0.147 0.883 Retain the null
hypothesis
17 Functional 31.11 46.38 1,052.50 2.952 0.003 Reject the null
purposes hypothesis
18 Job characteristics 35.5 43.23 907.5 1.522 0.128 Retain the null
hypothesis
19 Simplicity and scale 39.73 40.2 768 0.093 0.926 Retain the null
hypothesis
20 Self-regulation 39.91 40.07 762 0.031 0.975 Retain the null
hypothesis
3.2 The Criticality of Missing Sociotechnical Principles in Losing Teams
During the Collaborative Design Project Study
Table 3 categorises the possibility of absence or missing of each sociotechnical prin-
ciple into three different levels (more critical, less critical and rarely critical). The top
eight risks, absence of ‘multidisciplinarity’ (13), ‘boundaries’ (11), ‘congruence’ (2),
‘constraints’ (12), ‘functional purposes’ (17), ‘participation’ (5), ‘mulifunctionalism’
(1), and ‘values’ (10), were identified to be highly or more critical, based on the mean
rating ranks of the losing teams. These top eight sociotechnical principles found to be
critical, and highly likely to be missing or absent and to have severe consequences on
the teamwork performance of a collaborative design project. Out of all the
sociotechnical principles, ‘multidisciplinarity’ (13) was found to be top in the lists of
high criticality (losing team = 51.11, Ranked 1st).
1080 A. A. Imanghaliyeva
Table 3. Ranking from highest to lowest the possibility of absence of 20 sociotechnical
principles derived from 18 risk factors affecting design projects via the mapping exercise.
№ 20 sociotechnical principles Losing teams
Most critical
1 13 Multidisciplinarity 51.11
2 11 Boundaries 50.97
3 2 Congruence 49.58
4 12 Constraints 49.29
5 17 Functional purposes 46.38
6 5 Participation 46.07
7 1 Multifunctionalism 45.96
8 10 Values 45.9
Less critical
9 18 Job characteristics 43.23
10 14 Resource flows 42.39
11 8 Flux 41.71
12 3 Flexible specification 41.16
13 9 Responsibility 40.3
14 16 Variance control 40.03
15 19 Simplicity and scale 40.2
Rarely critical
16 20 Self-regulation 40.07
17 4 Joint optimisation 40.03
18 15 Uniqueness 40.03
19 6 Designing design 39.29
20 7 Experimentation 37.65
4 Conclusion
The possible missing scores of sociotechnical principles were identified in the team
performance of winning and losing teams. The losing teams had significantly higher
scores for possible missing principles that the winning teams. The top eight principles
whose absence posed risks were ‘multifunctionalism’ (1), ‘congruence’ (2), ‘partici-
pation’ (5), ‘values’ (1), ‘boundaries’ (11), ‘constraints’ (12), ‘multidisciplinarity’
(13) and ‘functional purposes’ (17), which were identified as being more critical to or
frequently missing in team performance of losing teams. The lack of these
sociotechnical principles led to a high likelihood of risk occurrence, which had a severe
impact on successful delivery of the collaborative design project.
Acknowledgments. This work was financed by the Republic of Kazakhstan’s JSC ‘Centre for
International Programs’ which operates under the Ministry of Education and Science of the
Republic of Kazakhstan, and administers the Kazakhstan President’s Bolashaq Scholarship
Program, that granted a Ph.D scholarship to Amangul Imanghaliyeva to support this study.
Identifying and Analysing Risk Factors from a STS Perspective 1081
References
1. Imanghaliyeva, A.A., Thompson, P., Salmon, P., Stanton, N.A.: A synthesis of sociotechnical
principles for system design. In: International Conference on Applied Human Factors and
Ergonomics, pp. 665–676. Springer, Cham, July 2019
2. Bing, L., Akintoye, A., Edwards, P.J., Hardcastle, C.: The allocation of risk in PPP/PFI
construction projects in the UK. Int. J. Project Manag. 23(1), 25–35 (2005)
Experimental Learning for a Basic Technology
Acquisition of Moving Images Production
Akiko Kondo1(&) and Satoshi Togawa2
1
Faculty of Management and Information Science, Shikoku University,
123-1 Furukawa Ojin-cho, Tokushima 771-1192, Japan
[email protected]
2
Education Center for Information Processing, Shikoku University,
123-1 Furukawa Ojin-cho, Tokushima 771-1192, Japan
[email protected]
Abstract. The aim of this study is to evaluate experimental learning for a basic
technology acquisition of moving images production. In Japan, we are facing
serious problems such as low birthrate and longevity. Thus, it is difficult to
sustain their municipalities in rural regions. In this situation, it is important to
promote sustainable tourism. In addition, an impact of information sending of
moving images is extremely large. Many a lot of people can post and produce
moving images because of technological innovation. It is required to foster
human resources to be able to create an attractive moving image on a daily basis.
The learners could obtain the skills concerning that under this classroom. From
the results of the surveys and interviews, we quantitatively understand the effect
of education on the learners by using the rubric method and the experts measure
a status of skills that they have learned.
Keywords: Experimental learning Sightseeing Moving images Resolving
regional problem Student behavior
1 Introduction
1.1 Background and Aim
Japan is facing serious problems such as declining birth rate, an aging population, and
depopulation. Additionally, local municipalities are losing their vitality because of this
situation and the extreme concentration of the population and industries in the Tokyo
prefecture and surrounding prefectures which are Tokyo metropolitan area. In this
situation, it is important to promote sustainable tourism. Factors to promote tourism are
not only improving convenience of transport facilities but to let people know the
attractiveness of tourist spots and the regions.
Recently, an impact of information sending of moving images is extremely large.
A moving image is frequently used because of technological innovation. On the other
hand, an impact of a vast amount of information existing in society for receiver dis-
perses. In other words, it is necessary to consider a way of information transfer for
efficiently when we transmit information. Thus, a video creator must create attractive
moving images for every case.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1082–1087, 2020.
https://doi.org/10.1007/978-3-030-39512-4_164
Experimental Learning for a Basic Technology Acquisition 1083
The aim of this study is to evaluate experimental learning for a basic technology
acquisition of moving images production. Learners have experiences as follows in this
project. First, the learners learn a way of shooting procedure. Second, they learn a way
of edit. Third, they suggest a scenario and a proposal. Forth, they make a moving image
(which is included in shooting and edit). Fifth, the moving images which are created by
the learners are criticized by experts and learning members. Through this project, the
learners get advice from experts and they discuss among the member of participants.
We used surveys on the learners with and without this project. Through the results
of the surveys and interviews, we qualitatively understand the effect of education on the
learners. In addition, by using the rubric method, we quantitively measured the degree
to which the learners understood the problems and propose solution. In addition, the
experts measure a status of skills, individually. We propose a method to acquire a basic
technology. From these results, we clarify education effects of experimental learning
concerning reginal problems. This contribute to create learning programs concerning
regional issues in the future.
1.2 Existing Studies and Characteristics of This Study
Education effective and assessment method of education have studied over the years
with significant amount of literature begin available. As result of this, many studies
have been reviewed in previous papers. The ways of educational evaluation have much
achieved, too.
Dannella et al. [1] are mainly explain the rubric method which can give an effective
feedback to learners, and propose the way of use it. They analyze and marshal the
educational method for cultivating human resources with a well- education. Dannelle [2]
explain that learners can recognize their strong and weak points smoothly in case
evaluation criteria is provided for them before their educational process. Enesto et al. [3]
compare the effects of two different self-assessment tools; rubrics and scripts on self-
regulation, leaning and self- efficacy. Suzuki et al. [4] clarify the effects of telling learners
about the evaluate standards for an examination and the purpose of it Michita [5].
In this study, we measure a teaching effectiveness of experimental learning in order
to acquire skills of using special equipment concerning moving image under this
classroom. Learners evaluate their own learning effectiveness and teachers evaluate it,
too. The learners become an effective human resource base that can help to realize a
sustainable society for the next generation.
2 Process of Experimental Learning and Method
of a Teaching Effectiveness
In this study, we conduct and experimental learning classroom and learners learn about
moving images in their university educational program. First, the learners learn a way
of shooting procedure. Second, they learn a way of edit. Third, they suggest a scenario
and a proposal. Forth, they make a moving image (which is included in shooting and
edit). Fifth, the moving images which are created by the learners are criticized by
1084 A. Kondo and S. Togawa
experts and learning members. Under this project, the learners get advice from experts
and they discuss among the member of participants.
We quantitatively and qualitatively measured the degree to which the learners
acquire skills by using the rubric method and hearing surveys on the learners. The
reason why we use the rubric method is as follows [6]. We explore how learners learn
the issues and how teacher educate it under this classroom. This evaluation result
enables us to understand what learners should learn and acquire skills or knowledge
under this classroom as well as evaluation of educational effects. This program makes
learners to realize a goal of their own learning program. In other words, they can
understand what they should learn under this classroom. On the other hand, teacher can
help them toward to achieve their goals. It is called self-regulated learning.
2.1 Operation of Experimental Learning
Experimental learning process in this classroom is divided into three processes. There
are “acquiring skill of creating scenario”, “acquiring skill about operation method of
shooting moving image” and “acquiring skill of image editing”.
First, learners discuss and create a scenario based on a given topic. They also learn
the operation method of professional use of camera and a basis way of editing. They
should reflect these methods to their scenario, and they repeat and keep practicing for
achieving their goals. On these processes, they discuss to derive a best way which they
can do in order to deliver things to someone who wants to get the information. These
steps are influenced by each other. They reiterate these processes which are a way of
shooting, editing and discussing by their selves. This is an important and useful pro-
gress in order to draw their potential and ability.
2.2 Providing of Rubric Chart
We quantitatively evaluate the degree to which the learners understood this program’s
contents by using the rubric method with and without their learning. In addition, we
qualitatively understood the educational effect on the learners under this classroom
thorough the results of the surveys and interviews in a complementary way. They
accumulate knowledge and experiment about the method of operation of professional
use of camera and editing images. It might have been supposed that the evaluation
result of learning effects becomes better gradually on each question item, some results
do not change for the better. We force upon what they learned. They can learn by
providing the rubric chart which is one of a goal to reach for their learning.
We defined the rubric in order to grasp an educational effect by their selves, as
shown in Table 1. Learned skills of them are evaluated by teachers, too. Teachers
evaluate their acquired skills based on balanced sheet include in detailed items at every
phase. Additionally, their moving productions are evaluated by teachers through their
own eyes.
Experimental Learning for a Basic Technology Acquisition 1085
Table 1. Rubric of self-assessing for the program
1086 A. Kondo and S. Togawa
3 Evaluation Method
Previously discussed, we quantitatively evaluate the degree to which the learners
understood the things they should do by using the rubric method. In analysis of these,
we focused upon the students of the basis of moving images class. There are 33
students. Table 2 shows a growth before and after learning.
Table 2. Comparing of evaluation before and after learning by using rubric method
Question/ Basis of shooting Basis of editing Scenario and proposal/application of
Item shooting and editing
Operation Method of Operaton of Capturing Image Image Creating Directions Various Writing
method of professional professional of images editing editing scenario for editing and
preparation of use of camera (basic (next step; shooting finishing
professional use camera steps) like of movie
of camera setting connecting) images
Average 1.94 2.00 2.25 1.88 1.81 2.00 2.06 1.75 1.88 1.81
Disparsion 0.33 1.33 0.47 0.65 0.70 0.53 0.60 0.47 0.65 0.56
From the Table 2, each average value grows up to about two steps by comparison
with before and after learning. There is great variability among the dispersion values of
the question which is “Method of professional use of camera setting”. This reflects the
educational method cannot provide a way of acquiring method, averagely. Measure-
ments are different form individual concerning the skill they should acquire. From this
result, it is clarified that we must devise ways of program in order to learners can feel
rewarded for their effort.
Next, we analyze a degree of diremption concerning an effect of education based on
comparisons with learners and teachers. Comprehensive evaluation of an educational
effect and an item; “method of professional use of camera setting” are analyzed.
Concerning camera setting, we carried out a practical test which contents are separated
into 14 detailed items. The result of this test is calculated with weighted according to
the importance of a technic. Additionally, teachers also evaluate them, comprehen-
sively. Teachers’ evaluation is conducted based on 100 points, and learners evaluate
their skills based on the rubric chart which are 4–5 grades. Table 3 shows a result of
correlation coefficient of each evaluation value. From this result, it is clarified as
follows.
Table 3. Correlation coefficient of each evaluation value
Item of evaluation [a] [b] [c] [d]
[a] Total (self-evaluation) 1
[b] Method of professional use of camera setting (self- 0.67 1
evalutation)
[c] Total (evaluation by teachers) 0.48 0.58 1
[d] Method of professional use of camera setting 0.48 0.26 0.53 1
(evaluation by teachers)
Experimental Learning for a Basic Technology Acquisition 1087
Concerning a result of the self-evaluation, some learners evaluate that their
acquired skills are high level, and some are otherwise. Then, the value for the corre-
lation coefficient for comprehensive evaluation concerning the proficiency of each
learner under this classroom and their judge is at a relatively low, which is 0.48. This
means that we can see differs in different evaluation between learners and teachers
concerning learning achievement evaluation. We focus upon the item of a method of
professional use of camera setting because the deviation of this item is large. The value
for the correlation coefficient for comprehensive evaluation concerning this topic is
0.26. It is lower that other items in this learning contents. From the result, it is clarified
that there is a deference between the learning achievement evaluation by learners and
teachers. The scores of the average of comprehensive evaluation is 79.1 and of camera
setting is 78.8 out of 100, and these are not low scores.
4 Conclusion
The aim of this study was to valuate experimental learning for a basic technology
acquisition of moving images production. From the summary of this interviews, almost
all the learners responded that they satisfied with the process and contents of this
classroom. On the other hand, concerning the learning evaluation concerning method
of use of professional camera setting, there is a wide divergence in learners and
teachers. It is required that we improve the learning program as follows in the future.
Teachers construct a lecture program that learners can acquire their skills or basic
technology regarding moving images production, on the premise that learners realize
the fun of learning. To achieve the purpose can cultivate human resources who can
contribute to the development of the whole community.
Acknowledgments. This work was supported by JSPS KAKENHI Grant-in Aid for Young
scientists (B) Number 17K18293.
References
1. Dannelle, D.S., Antonia, J.L.: Introduction to Rubrics: An Assessment Tool to Save Grading
Time, Convey Effective Feedback and Promote Student Learning, 2nd edn. Stylus Pub Lic
(2012)
2. Weaver, M.R.: Do students value feed-back? Student perceptions of tutors’ written responses.
Assess. Eval. High. Educ. 31(3), 379–394 (2006)
3. Ernest, P., Jesus, A.T., Juan, A.H.: Rubrics and self-assessment scripts effects on self-
regulation, learning and self-efficacy in secondary education. Learn. Individ. Differ. 22, 806–
813 (2012)
4. Suzuki, M.: Effects of a rubric: values of a test, motivation for learning and learning strategies.
Jpn. J. Educ. Psychol. 59, 131–143 (2011)
5. Michita, Y.: Effect of question-asking training in a college lecture class on learners’ attitudes
and ability to ask questions. Jpn. J. Educ. Phycol. 59, 193–205 (2011)
6. Kondo, A., Togawa, S.: Experimental learning for resolving regional problems. In: Proceedings
of International Conference of Education, Research and Innovation, pp. 77–85 (2018)
Mechanical Fatigue Evaluation by Image
Recognition
Massimo Milani, Luca Montorsi, Luca Fontanili(&), Gabriele Storchi,
and Gabriele Muzzioli
Department of Sciences and Methods for Engineering, University of Modena
and Reggio Emilia, Via Giovanni Amendola, 2, 42122 Reggio Emilia,
Emilia-Romagna, Italy
{Massimo.Milani,Luca.Montorsi,Luca.Fontanili,
Gabriele.Storchi,Gabriele.Muzzioli}@Unimore.it
Abstract. The mechanical fatigue is an important contributor to the failure of
mechanical components. In order to avoid this condition, the phenomenon has to
be predicted and controlled during the design and the implementation of a
mechanical component. The mechanical fatigue can lead to maintenance, to
parts replacement, to extra-needs for lubricants and ancillary labor, and it is one
of the main factors of economic loss. Every mechanical component intended for
the force transmission is subject to mechanical fatigue. The analysis of the
system status during time enables the evaluation and characterization of the
fatigue influence on its behavior. In mechanical devices performing a work-
cycle with moving parts, the cyclic movements have to be “the same” during
time. The main target of this paper is to verify the mechanical behavior of two
different gripping blocks during long periods of high cyclic fatigue work. Both
blocks have moving parts and all the acquisitions are concentrated to capture
fatigue sign on devices work-cycle, mainly in terms of moving parts positioning.
To perform the kinematic analysis of both devices under test, and to verify and
quantify the degradation in their mechanical performance, a Motion Capture
System (VICON) has been combined with an intelligent tool for imaging
analysis (KINOVEA). In this way, the precision and the reliability of a free
imaging analysis software applied to cyclic working conditions have been
compared, on a bi dimensional plane, with data captured by a stereopho-
togrammetric system.
Keywords: Motion capture Mechanical fatigue Human-system integration
Work-cycle High cyclic fatigue work Image analysis
1 Introduction
The mechanical fatigue is an important contributor to the failure of mechanical com-
ponents [1]. In order to avoid this condition, the phenomenon has to be predicted and
controlled during the design and the implementation of a mechanical component. Every
mechanical component intended for the force transmission is subject to mechanical
fatigue. In mechanical devices performing a work-cycle with moving parts, the cyclic
movements have to be “the same” during time. The main target of this paper is to verify
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1088–1093, 2020.
https://doi.org/10.1007/978-3-030-39512-4_165
Mechanical Fatigue Evaluation by Image Recognition 1089
the mechanical behavior of two different gripping blocks during long periods of high
cyclic fatigue work. The first gripping block has been analyzed for 500.000 work
cycles using a Motion Capture System (VICON) [5] to acquire the position of some
important areas of the block, highlighted by reflector markers. The use of this
stereophotogrammetric system enables to reconstruct the marker trajectories during
various times of the campaign. Additionally, an intelligent tool for imaging analysis
(KINOVEA) [6] has been used to evaluate its reliability as in [2, 3] and [4]. This
software is based on the analysis of a bi-dimensional video recording to analyze the
body motion. The same reference system origin has been defined to compare both data
acquired by the two technologies. In this way, the precision and the reliability of a free
imaging analysis software applied to cyclic working conditions have been compared,
on a bi dimensional plane, with data captured by a stereophotogrammetric system. In
the second part of this work, the second gripping block has been analyzed to verify its
behavior during 100.000 work cycles. Motion Capture has analyzed the principal
cylinder movement while the video analysis, carried out with Kinovea, enabled to
evaluate the trajectories of the three little pistons with a radial movement compared to
the principal cylinder. The results obtained from the two campaigns have been studied
and compared to address the accuracy variation during time. This data have been
evaluated to quantify the mechanical fatigue.
2 Materials and Methods
2.1 First Gripping Block
Two self-aligning cylinders with double effects compose the first analyzed gripping
block. For the analysis of Block 1, a marker-set composed by 12 units has been used, as
depicted in Fig. 1.
Fig. 1. The twelve markers composing the used marker-set.
1090 M. Milani et al.
This are compact components with high gripping stability and fluency. The
working fluid is pressurized oil. During a work cycle, both cylinders fulfill a complete
up and down run. In the peak time, the two brackets rotate 90° and lower few mil-
limeters to execute the gripping action. Subsequently, the cylinder returns to its rest
position. The markers located on the block are named as follows:
• SC1, LC1: fixed reference markers;
• SC2, LC2: markers on the rotation axis of the two cylinders;
• SC3, LC3: markers on the extremities of the gripping beak;
• B1, B2, B3: system origins;
• BLOCCO1, BLOCCO2 and BLOCCO3: the first two markers are used as a ref-
erence for the third one that accomplishes a short up and down movement and it is
located on the pressure transductor.
2.2 Second Gripping Block
A principal piston that goes in and out the main cylinder composes the second block,
and in the maximum extension phase, three small pistons come out radially to grip the
outer ring. Subsequently for the release phase, these pistons return to the rest position
to complete the work-cycle. The analyzed movements of this unit are fewer than the
first block: the main piston and the three smallest one trajectories. Only one marker, on
a hexagonal 7 cm metal arm fixed to the block’s head, has been used to monitor the
position of the principal piston during the movement, as shown in Fig. 2. Since it was
not possible to fix the markers on the three smaller pistons, due to their dimensions,
their motion has been acquired using a high-resolution digital video camera and ana-
lyzed only via image analysis software.
MARKER
TARGET
Fig. 2. The markers used for measurement on the second gripping block.
2.3 Instruments
For the movement analysis, the VICON Motion Capture system and a free image
analysis software have been used to monitor the blocks movements. For the first one
the Mo Cap is used to verify the reliability of KINOVEAs, while for the second block
the two systems have been integrated to analyze the second block’s movements.
Mechanical Fatigue Evaluation by Image Recognition 1091
3 Results
For almost all acquisitions six work cycles are recorded for a total video duration of
30–40 s. First results are relative to the reliability of the free image analysis software
for the first gripping block. The markers’ trajectories measured with the Motion
Capture technique are compared to the ones measured with Kinovea. In Fig. 3, this
comparison has been reported and the error span of Kinovea technology added.
Kinovea follows a pattern in a selected sector of the image; in the test the spherical
markers were used. The center of the marker has a color while the remaining spherical
surface has a different color. In a static video, the trajectory jumps from a point to
another inside this single-color pattern. This behavior leads to problem in maintaining a
linear track. The evaluation has been carried out for all the measurements, but only the
vertical movement has been reported in this work. The analyzed marker is the SC2
(Short Component) for the first block before and after 500.000 work cycles.
Fig. 3. Vicon data compared to the same trajectory acquired with Kinovea. The upper graph,
named A, shows the trajectory for the first day of “work” while the lower one, named B, reports
the behavior after 26 days.
After addressing the accuracy of Kinovea, the data have been analyzed to compare
the positioning differences with the previous day and with the initial one. These data
have been carried out for every coordinate (X, Y, and Z) for the Short Component. In
the present work, only the measurement for the Z-axis are reported in terms of per-
centage difference with respect to the ones measured the first working day. This
coordinate showed the largest variation. Figure 4A shows the behavior for the Z values
of the SC. This is the cylinder with the shortest trajectory and the percentage error is
consistently higher than the Long Component. It is possible to find out that during the
300.000 cycles the gripping position is the same but after those work-cycles the
positioning became less accurate, but still acceptable. Subsequently, the second grip-
ping block has been analyzed to verify its behavior after 100.000 cycles. The mea-
surements include, as in the previous analysis, the delta percentage of the
malpositioning of the principal cylinder with respect the first day and the previous one,
and the trajectory of the three smaller cylinders during the first and the seventh days of
operation. In Fig. 4B, only the data about the comparison with the starting day for the
principal cylinder are reported, since they are the most significant ones.
1092 M. Milani et al.
A B
Fig. 4. (A) The Delta percentage between the measurement of the height (Z-axis) of the Short
Component in the various work-cycles and the same measurement in the first working day.
(B) The Delta percentage between the measurement of the height (Z-axis) of the Main Cylinder
in the various work-cycles and the same measurement in the first working day.
The last obtained data, as mentioned previously, represent the trajectories of the
three smaller cylinders with radial movements. Figure 5 shows their behavior at the
start and the end of their 100.000 work cycles.
Fig. 5. The traveled distance difference of the three smaller cylinder with radial movements,
between the first day and after 100.000 work-cycles. (A) represents the acquisition of the first
day, while (B) reports the last day.
Mechanical Fatigue Evaluation by Image Recognition 1093
4 Conclusions
The MoCap and Kinovea techniques enable to analyze and quantify the mechanical
fatigue phenomena. The results lead to a percentage error, calculated on the marker
movements for all the measurements, of 0.5% and the maximum deviations are limited
to 2–4%. It is possible to highlight that medium deviations vary by a few tenths of a
millimeter despite the thousands of cycles completed. Every one of the two gripping
blocks reacts well to the sollecitations, (vibrations and leaks of lubricants), preventing
the fatigue problem. This is possible if a well-designed block enables a good lubri-
cation of the gripping mechanism to fit perfectly the high- work cycle needed for the
function. The Kinovea software has been evaluated as reliable for the video analysis
with respect to a much more sophisticated technology based on stereophotogrammetry
like VICON Nexus. The comparison of the trajectories obtained by the two tech-
nologies makes it clear that Kinovea proves reliable measurements; furthermore, the
accuracy of the video-imaging technique can be enhanced by a higher resolution of the
video camera. Obviously, for precise measurements like those made during this project,
it is advisable to use a system as accurate as possible but Kinovea remains an excellent
and efficient video analysis system, considering also the low equipment cost and
complexity (almost any camera can be used).
References
1. Boninardi, M., D’Errico, F., Fabio, M.: Il problema dell’usura negli organi meccanici (2004)
2. Elrahim, R.M.A., et al.: Inter-rater and intra-rater reliability of Kinovea software for
measurement of shoulder range of motion. Bull. Fac. Phys. Ther. 21(2), 80 (2016)
3. Balsalobre-Fernández, C., et al.: The concurrent validity and reliability of a low-cost, high-
speed camera-based method for measuring the flight time of vertical jumps. J. Strength
Conditioning Res. 28(2), 528–533 (2014)
4. Elwardany, S.H., El-Sayed, W.H., Ali, M.F.: Reliability of Kinovea computer program in
measuring cervical range of motion in sagittal plane. Open Access Libr. J. 2(9), 1 (2015)
5. Francia, M., Milani, M., Montorsi, L.: An innovative approach to kinematic analysis of
multibody hydraulic actuation systems, pp. 313–320 (2016)
6. Kinovea. https://www.kinovea.org
Universal Access and Inclusive Dwelling
Design for a Family in Monte Sinahí,
Guayaquil, Ecuador
Jesús Rafael Hechavarría Hernández(&), Boris Forero,
and Robinson Vega Jaramillo
School of Architecture and Design, Catholic University of Santiago
de Guayaquil, Av. Carlos Julio Arosemena Km 1 ½, Guayaquil, Ecuador
{jesus.hechavarria,boris.forero,
robinson.vega}@cu.ucsg.edu.ec
Abstract. The present work is the result of a low-income dwelling inclusive
design for a family at Monte Sinahí. Students and professors of the Faculty of
Architecture and Design of the Catholic University of Santiago de Guayaquil in
one of the economically less favored settlements and exposed to high vulner-
ability risks in Guayaquil, a housing prototype is designed through a systemic
way with bioclimatic criteria and the use of ecomaterials. The proposal includes
universal accessibility required by a family consisting of an older adult with
motor disability and a young man with Down’s syndrome. This archetype is a
partial result of a winner project of the National Program of Financing for
Research - INEDITA in the area of Energy and Materials research of the
Institutional Modality, organized by the Secretariat of Higher Education, Sci-
ence, Technology and Innovation (SENESCYT) and the United Nations
Development Program (UNDP) in Ecuador.
Keywords: Universal access Inclusive design Low-income dwelling
Ecomaterials
1 Introduction
Monte Sinahí’s sector has been considered a priority in the investments that government
has developed due to the increasing number of dwellers and the poverty rates that
characterize them. However, the housing proposals that have been built do not have
previous studies of bioclimatic analysis, causing absence of comfort on its occupants [1].
As the research line of the Institute of Habitat, Design and Construction (IHADIC),
where the research of the professors of the Faculty of Architecture and Design of the
Catholic University of Santiago de Guayaquil is carried out, the habitat of Monte
Sinahí is constituted in the field of action to be congruent with that line. The expected
results will allow defining new strategies to contribute for the welfare of the residents in
the popular dwellings of Monte Sinahí. In recent years, multiple researchers have paid
attention to the absence of thermal comfort at popular settlements of Guayaquil [2],
identified some causes [3, 4] of this unfavorable situation and have made proposals [5]
and [6] to improve indoor thermal comfort conditions experienced by the occupants.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1094–1100, 2020.
https://doi.org/10.1007/978-3-030-39512-4_166
Universal Access and Inclusive Dwelling Design for a Family 1095
Main results obtained are related to the proposal of graphic and mathematical
methods to analyze the dimensional behavior of residential spaces in Guayaquil, [7],
the user’s perception of their home and the environment in popular settlements [2], the
thermal comfort effect analysis in low-cost settlements [5, 8], the systemic approach to
design as a fundamental element in multiobjective analysis and the use of bioclimatic
strategies in the design of popular dwellings [9]. On the other hand, the use of software
in the geometric development of houses and thermal modeling is a practice experienced
by teamwork [3], which favors the acquisition of new knowledge for the development
of the investigation.
2 Analysis of Climate Information
In the process of architectural design under bioclimatic criteria, historical data from
nearby meteorological stations belonging to the National Institute of Meteorology and
Hydrology (INAMHI) are collected. Among the data acquired are the outside tem-
perature, with minimum, maximum and average data, as well as relative humidity with
their respective averages and maximum and minimum values, see Tables 1 and 2.
Table 1. Outdoor temperature data in Monte Sinahí (°C).
Months J F M A M J J A S O N D
Minimum average 23.3 23.4 23.5 23.5 27.5 21.2 20.6 20.6 19.8 21.3 21.5 23.5
Average 26.7 26.5 27.1 28.0 22.5 26.1 25.6 25.3 25.4 25.3 25.8 27.4
Maximum average 31.4 31.0 31.6 32.7 32.1 30.7 30.7 30.8 31.2 30.9 31.5 32.4
Table 2. Relative humidity (RH) data in Mount Sinahí (%).
Months J F M A M J J A S O N D
Minimum average 43 55 54 44 48 54 51 52 49 45 49 46
Average 75 80 76 68 69 72 73 73 71 71 69 68
Maximum average 97 97 97 94 95 99 93 95 87 88 86 85
Subsequently, the work of collecting climate information in the geographical area
where Monte Sinahí is located is continued, consolidating data from other special
phenomena such as cloud coverage measured in octas, evaporation in millimeters, dew
point in degrees Celsius and the vapor tension in hectopascal. Also on prevailing
winds, its direction and speeds as well as percentages of calm; rainfall, total in the
month, maximum total in 24 h and total days in which rains occur. Climate charac-
terization of the study area starts from the climatic data that is gradually collected, and
whose analysis allows the development of Isothermas and Isohigras graphs to visualize
and determine the climatic reality of the area where the architectural project will be
implemented (Fig. 1).
1096 J. R. Hechavarría Hernández et al.
Fig. 1. Isotherms of the hourly behavior of temperatures at Monte Sinahí.
In the graphic, plotted lines from hot-humid season from December to May, shows
highest temperatures outside confort zone that runs through all day (center) and even
early hours of the night. And the other season, the hot-dry one, with lower temperature
averages but still stay high during the day. Confort zone shown in grey with range
between 23,5 and 28,5 °C for Guayaquil according to Tc = 17.8 + 0.31(Tm) [10] The
average hourly relative humidity behavior at Monte Sinahí is shown in Fig. 2. It is
observed that the highest concentration is recorded from Jan-uary to March in late night
and early morning between 01:00 and 08:00 h [11].
Fig. 2. Isohygras of the relative humidity behavior at Monte Sinahí.
Universal Access and Inclusive Dwelling Design for a Family 1097
3 Analysis of the Housing Information
In visits to Monte Sinahí for the recognition of the place there were several approaches
with spiritual and community leaders who helped in the selection of the family who
will be the benefited of the project, see Fig. 3.
Fig. 3. Meeting with the priests in Monte Sinahí.
The selected family complies with project required profile for the following rea-
sons: they have documents proving possession of the lot, current dwell is in a very bad
condition and it is a family of two members with an older disabled adult by amputation
in one leg and a young man with Down syndrome (Fig. 4).
Fig. 4. Family benefited from the project.
1098 J. R. Hechavarría Hernández et al.
4 Bioclimatic Project
Investigation begins with the analysis of the climatic information from databases
related to geographical Monte Sinahí area and gathering information at field. Then a
sustainable dwelling unit archetype can be proposed through bioclimatic design
methodology considering also the environmental impact reduction using ecomaterials
[12] (Fig. 5).
Fig. 5. Manufacturing process of ecomaterials to use in the archetype.
5 Inclusive Dwelling Design Proposal
Universal access is important during the archetype design process. In order to achieve
this, socio cultural activities and demands of inhabitants, as well as their special char-
acteristics must be considered. The project design meets these requirements, high-
lighting the availability of displacement and access to all spaces that make up the
dwelling (Fig. 6).
Fig. 6. Plan view of the proposed archetype model.
Universal Access and Inclusive Dwelling Design for a Family 1099
6 Discussion
Main strategy followed in designing process of the archetype is to cool down indoor
spaces most of the year using as a tactic the form factor and decrease heat gains through
roof and walls, in addition to shading using solar control devices as overhangs on main
and rear facades added to an inner garden that functions as an extraction element. The
use of materials and construction elements with high albedo index finishes and low
absorbency are techniques that stand out, as well as designed openings as part of roof
systems to favor natural cross ventilation. The use of PlasBam [13] as an ecomaterial to
be used on walls, floors and structural elements guarantees a good thermal performance
of the dwelling envelope.
Acknowledgments. The authors wish to thank to the Catholic University of Santiago de
Guayaquil and the Secretariat of Higher Education, Science, Technology and Innovation in
Ecuador - SENESCYT for the financial support of this research. Also to the United Nations
Development Program - UNDP for the excellence in the Project management and to Graphisoft
Ecuador due its professionalism in assistance and teaching of ARCHICAD. To priests Luis
Enriquez and Antonio Martinez for his guide and support and Mrs. Gloria and her son for their
trust and human quality.
References
1. Hechavarría, J., Vega, R., Forero, B.: Multi-objective optimization applied to the bioclimatic
design of dwellings with ecomaterials, pp. 506–511. Springer, Cham (2019)
2. Ricaurte, M., Hechavarría, J.: La percepción del usuario sobre su vivienda y el entorno en
programas de interés social en Durán, Ecuador, Rev. Científica Ecociencia, vol. 4, no.
6 (2017)
3. Dick, S., Hechavarría, J., Forero, B.: Systemic analysis of bioclimatic design of low-income
state-led housing program ‘Socio Vivienda’ at Guayaquil, Ecuador, pp. 647–651. Springer,
Cham (2019)
4. Ricaurte, V., Almeida, B., Hechavarría, J., Forero, B.: Effects of the urban form on the
external thermal comfort in low-income settlements of Guayaquil, Ecuador, pp. 447–457
(2020)
5. Forero, B., Hechavarría, J., Alcivar, S., Ricaurte, V.: Systemic approach for inclusive design
of low-income dwellings in popular settlements at Guayaquil, Ecuador, pp. 606–610.
Springer, Cham (2019)
6. Forero, B., Hechavarría, J., Vega, R.: Bioclimatic design approach for low-income dwelling
at Monte Sinahí, Guayaquil, pp. 176–185 (2020)
7. Almeida, B., Díaz, J., Bermeo, P., Hechavarría, J., Forero, B.: An alternative graphic and
mathematical method of dimensional analysis: its application on 71 constructive models of
social housing in Guayaquil, pp. 598–607. Springer, Cham (2020)
8. Yahia, M., Johansson, E., Thorsson, S., Lindberg, F., Rasmussen, M.: Effect of urban design
on microclimate and thermal comfort outdoors in warm-humid Dar es Salaam, Tanzania. Int.
J. Biometeorol. 62(3), 373–385 (2018)
9. Hechavarría, J., Forero, B., Bermeo, P., Portilla, Y.: Social inclusion: a proposal from the
University of Guayaquil to design popular housing with citizen participation. In: 2018 World
Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC),
1100 J. R. Hechavarría Hernández et al.
Albuquerque, NM, USA, pp. 1–5 (2019). https://doi.org/10.1109/weef-gedc.2018.8629778.
ISBN 978-1-5386-7764-3
10. Auliciems, A., Szokolay, S.: Thermal comfort. In: PLEA: Passive and Low Energy
Architecture International in Association with Department of Architecture, The University of
Queensland Brisbane 4072 (2007)
11. Gómez-Azpeitia, G.: Caracterización climática: Diagnóstico de Confort. Colima (2016)
12. Forero, B., Hechavarría, J., Vega, R.: Bioclimatic design approach for low-income dwelling
at Monte Sinahí, Guayaquil, pp. 176–185. Springer, Cham (2020)
13. IEPI (2014-26585): Structural boards of Guadua Thermo-Pressed cane (PLAS-BAM). Code
IEPI-01-02-01-2014-01-000282 (2014)
Integrated Safety Risk Assessment Between
Enterprises, Industries and Areas
Lu Zhang1(&), Yun Luo1, and Rui Liao2
1
School of Engineering and Technology,
China University of Geosciences (Beijing), Haidian District, Beijing, China
[email protected]
2
Sichuan Transportation Investment Industrial Co., Ltd.,
Chengdu, Sichuan, China
Abstract. Accidents in industries cause great damage to human life, economy
and environment. In order to regulate and supervise safety work effectively, it’s
needed to measure the risk level of enterprise, industry and area. First, a basic
framework of index system was established. Second, the method to determine
index weight and compute risk score was proposed. Finally, the risk evaluation
model for enterprise was proposed and applied in Taian city. This method will
provide a good means to estimate the actual risk level of enterprise, industry and
area year by year. By this method, the risk of enterprise, industry and area can be
compared directly and the managing cost of safety regulation and supervision
will be reduced.
Keywords: Integrated risk Risk assessment Safety engineering
1 Introduction
With the acceleration of economic development and urbanization in China, industry
accidents in different enterprises, industries and areas have become more and more
frequent and destructive. It has been a serious challenge to prevent from serious
accidents and protect people’s safety of life and wealth. China’s State Administration
of Work Safety, as known as Ministry of Emergency Management now, has promoted
the theories and methods of risk identification, assessment and control to enterprises
since 2016. As more and more enterprises established the long-term mechanism of risk
control gradually, it comes out new requirements that how to evaluate the “risk level”
of a whole enterprise, industry or area. Importantly, it’s needed by large group cor-
porations as well as government regulators. The safety management department in
large group corporations, such as State Grid Corporation of China and China National
Gold Group Corporation, need to regulate safety situations of hundreds of subordinate
enterprises. According to the law of the People’s Republic of China on work safety, the
emergency management departments of the local governments at various levels should
supervise over the whole situation of work safety within their administrative areas,
which contains the situation of enterprise, industry and area. Therefore, it’s important
and urgent to develop a method of risk assessment to evaluate enterprises, industries
and areas. Ma set up a warning index system for coal mine safety [1]. He proposed a
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1101–1106, 2020.
https://doi.org/10.1007/978-3-030-39512-4_167
1102 L. Zhang et al.
grading supervision model of work safety in enterprises [2]. According to the research
literatures above, the method of constructing index system is a feasible way to evaluate
the risk situation of enterprises, industries and areas. In this paper, a basic structure of
risk evaluation index system for enterprise is established. Then, AHP is used to
determine the weights of index system. Finally, a risk evaluation model is proposed and
applied.
2 The Risk Evaluation Index System for Enterprises
2.1 Basic Framework of Index System
The word of “risk” has many definitions in safety area. However, it’s always related to
accident (or event), possibility and consequence, based on International standard,
national standard [3] and research [4, 5]. In this work, the purpose of risk evaluation is
to estimate and evaluate the comprehensive level reflecting enterprise’s preventive
control measures to accident. If the ability to control hazards and prevent accidents is
stronger, then the risk for whole enterprise is smaller. Thus, the risk research object is
not a single equipment, or a kind of human behavior, or in a kind of particular
environment, but the whole enterprise production system comprised with all of them
above. In other words, it gives risk a unique character of externality. Both the factors
inside enterprise and outside enterprise are considered in this work. Therefore, the risk
evaluation index system for enterprises includes three following dimensions: Accident
Possibility, Consequence Severity and Society Sensibility.
The dimension of possibility measures the likelihood to which accident will occur
in an enterprise. The dimension of severity measures the extent to which accident might
cause once it occurs. The dimension of sensibility measures the possible influence to
outside (public, government, and so on), which is produced by accident [5]. It’s not
usually included in the calculation scope of accident loss. Based on indexes above,
combined with the characteristics of enterprise production and requirements of safety
supervision from government, the risk evaluation index system for enterprises is
established, as shown in Fig. 1. To ensure universality and practicability of index
system for enterprise, the weights and computing method of indexes need to
determined.
2.2 Determination of Index Weight and Computing Method
The purpose to construct index system is to effectively and quantitatively identify the
risk level of enterprise. However, there are different effects on risks from different
indexes. Therefore, it’s important to determine weights of risk indexes. As mentioned
above, this index system is mostly used for regulators and supervisors. It’s reasonable
to consider the preference of weights for decision makers. Therefore, we choose AHP
method to determine the weights of indexes because it takes the object as an entire
system and quantify the impact of each index on results by experts.
As for computing method, there is a complex situation when computing all of the
indexes. The common approach is to assign one value or one set of values to a decided
Integrated Safety Risk Assessment Between Enterprises, Industries and Areas 1103
ap1 The scale of enterprise
ap2 The number of hazards
Accident
Possibility (AP) ap3 The number of special equipment
ap4 The number of registered safety en-
gineering
Risk
Evalu-
ation cs1 The number of accidents occurred
Index
System cs2 The number of deaths and injured
for Consequence
enterp- Severity (CS)
cs3 The value of emergency supplies
rises
cs4 The number of emergency exercises
ss1 The number of illegal punishments by
Society government
Sensibility (SS)
ss2 The maximum number of people
affected by accident
Fig. 1. The risk evaluation index system for enterprises.
range for each index. However, the value of indexes between enterprises is variable,
which makes it difficult to decide fixed value range for each index. Thus, we put
forward a new computing method to value index by ranking and sectioning. The
computing steps are as follows.
Step 1. According to the influence of index on the result, the indexes can be divided
into two kinds: enhanced indexes and weakened indexes. Enhanced index has a pos-
itive effect on the result, while weakened index has a negative effect.
Step 2. Rank the single enhanced index in order of value collected from large to small
and divide equally into four sections. Determine the rank score based on the section
corresponding to the sample value.
Step 3. Rank the single weakened index in order of value collected from small to large
and divide equally into four sections. Determine the rank score based on the section
corresponding to the sample value.
Step 4. Repeat Step 2 and Step 3 until get all the rank scores on all indexes.
Based on the risk evaluation index system and calculation method above, the risk
evaluation model for enterprises can be calculated as
1104 L. Zhang et al.
X4 X4 X2
R¼ i¼1
wap
i api þ m¼1 m csm þ
wcs n¼1 n ssn :
wss ð1Þ
where R denotes the integrated risk of enterprise, wap i denotes the weight of the ith
index of accident possibility, api denotes the rank score of the ith index of accident
possibility, wcs
m denotes the weight of the mth index of consequence severity, csm
denotes the rank score of the mth index of consequence severity, wss
n denotes the weight
of the nth index of society sensibility, ssn denotes the rank score of the nth index of
society sensibility.
3 Case Study
In this part, we will provide cases to illustrate the index system and model proposed.
Taian city is located in Shandong Province, China. We gathered data from 12 enter-
prises, which include coal mines, chemical industries, manufacturing and construction
enterprises. Based on the formula (1), the integrated risk value of 12 enterprises were
calculated as Table 1 shown.
Table 1. Risk value of twenty-two enterprises in Taian.
CorpID Industry belonged Risk value
CM-01 Coal mine 213.74
CM-02 Coal mine 306.02
CM-03 Coal mine 144.10
CM-04 Coal mine 284.60
CE-01 Construction 267.43
CE-02 Construction 131.11
CI-01 Chemical industry 340.30
CI-02 Chemical industry 170.15
CI-03 Chemical industry 201.47
ME-01 Manufacturing 166.87
ME-02 Manufacturing 103.14
ME-03 Manufacturing 104.49
In this case, we can see the values at risk range from 103.14 to 340.30. Figure 2
shows the integrated risk values between 12 enterprises. Half of the enterprises had a
risk value between 100 and 200, 30% of them between 200 and 300 and only two of
them beyond 300. Meanwhile, once the risk of enterprise is acquired, the risk of
industry can be easily calculated through sum and average. Based on the results above,
the industry of coal mine has a risk value at 237.115, the industry of construction at
199.270, the industry of chemical at 237.307 and the industry of manufacturing at
124.833. The level of safety risk between different industries can be compared through
the average risk and so can the area too.
Integrated Safety Risk Assessment Between Enterprises, Industries and Areas 1105
Fig. 2. The integrated risk values with twelve enterprises.
4 Conclusions
Integrated safety risk evaluation is a complex system with many indexes involving
factors from enterprises, industries and areas. In this paper, a risk evaluation model for
enterprises, industries and areas was studied. The main conclusions are as follows:
(1) The basic framework of index system for enterprise is established. It includes
three dimensions: Accident Possibility, Consequence Severity and Society Sen-
sibility. Then the indexes based on these dimensions are confirmed.
(2) The AHP method is used to determine the weights of indexes. A new computing
method to value index by ranking and sectioning is proposed. This new method
can adapt the changes of values of indexes and assign risk score to a variable
range based on values collected. With indexes, weights and risk scores, the risk
evaluation model for enterprises is established. Furthermore, the risk level of
industry and area can be calculated based on results of enterprises.
(3) Application case shows that the index system and model is a feasible and effective
method to evaluate the actual risk level between enterprise, industry and area. By
using this method, regulators and supervisors from corporation and government
can easily find out which enterprise, or industry has a high-risk level and the
pertinence and effectiveness of management measures can be improved and
management cost can also be reduced.
1106 L. Zhang et al.
References
1. Ma, J.Q., Dai, H.: A methodology to construct warning index system for coal mine safety
based on collaborative management. Saf. Sci. 93, 86–95 (2017)
2. He, C., Liu, G.Z., Ren, Z.G., Wang, S.W.: Study on grading supervision model of work safety
in enterprises. J. Saf. Sci. Tech. 2, 84–90 (2011)
3. Wang, G., Pei, J.J.: Macro risk: a versatile and universal strategy for measuring the overall
safety of hazardous industrial installations in China. Int. J. Environ. Res. Public Health 16, 1–
13 (2019)
4. Luo, Y., Pei, J.J.: Risk Analysis and Safety Assessment. Chemical Industry, Beijing (2016)
5. Marvin, R.: Risk Assessment Theory, Methods, and Applications. Wiley, New York (2011)
Comparison Between ARIMA
and LSTM-RNN for VN-Index Prediction
Nguyen Trong Co, Hoang Huu Son(&), Ngo Thanh Hoang,
Tran Thi Phuong Lien, and Trinh Minh Ngoc
Academy of Finance, 58 Le Van Hien, Ha Noi, Vietnam
{nguyentrongco,sonhh,ngothanhhoang}@hvtc.edu.vn,
[email protected], [email protected]
Abstract. The VN-Index prediction problem is a part of a research project that
predicts macroeconomic indicators for developing economies to detect the risk
of inflation and economic crisis. In this study, given the records of VN-Index in
the Vietnam Stock Exchange (VSE), we compare two different VN-Index
forecast methods: the ARIMA econometric model and the Deep-learning
approach with the LSTM-RNN model. Empirical results shows that: (1) the
LSTM-RNN model is more accurate than the ARIMA model in the VN-Index
forecast problems, (2) when applying the LSTM-RNN model, the window size
would converge if it was larger than 15 days for LSTM-256 and 10 days for
LSTM-512. From such results, we conclude that the LSTM-RNN model based
on one-month data is optimal, which is appropriate according to the Wave
theory and the psychology of the market.
Keywords: VN-index Stock price prediction ARIMA LSTM-RNN
1 Introduction
The Vietnam Stock Exchange (VSE) is rather young and highly fluctuating with unique
investor behavior (Vo 2016). The majority of Vietnamese investors are small investors
with limited funds, which are allocated for various purposes; hence, they tend to rotate
their funds as quick as possible to maximize their profits. Moreover, in long-term
investments, they are easily affected by the changes in inflation rate, exchange rate, due
to the small scale of the economy. Therefore, short-term investments are preferred. In
addition, Vietnamese investors are mostly spontaneous investors, who lack the
knowledge in stock investment, ergo; their decisions are purely based on speculation,
sentiment or observation of a “shark” or the crowd. Moreover, the Vietnamese gov-
ernment imposes a weaker enforcement of stock market regulations, thus, it has less
control and adapts slowly to the changes of the markets.
Each stock market has its own unique index system that reflected the volatility of
trading sessions, e.g., NASDAQ and Dow Jones in the U.S, Nikkei in Japan, and
FTSE100 in the U.K. In Vietnam, VN-Index is the most popular one for these fol-
lowing reasons: first, the VN-index compares the current market capitalization to the
base one on the original date of July 28, 2000 shows the price fluctuation of all stock
listed and traded on HOSE. In fact, it shows the trend of more than 90% of trading
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1107–1112, 2020.
https://doi.org/10.1007/978-3-030-39512-4_168
1108 N. T. Co et al.
values and more than 80% market capitalization of the listed companies on the HOSE;
second, stock represented by the VN-Index must meet conditions on quality, transfer
rate and liquidity ratios. Another characteristic of VN-Index is its periodicity. The VN-
Index is under the influence of three-year cycles, which proves that the VSE is
repetitive and stays true to the Elliot Wave Theory.
Stock price prediction has become an emerging problem in the last two decades.
Among forecast methods, ARIMA (Auto Regression Integrated Moving Average) are
considered as the most popular and efficient forecasting techniques. This method only
needs a combination of past values and past errors of the series for prediction and does
not assume knowledge of underlying models or relationships like some other methods
[4]. In addition, as reported by Box and Jenkins, in short-term forecasting, ARIMA
proves to be prominent and even superior to more complicated structural models.
However, this approach has its own limitations. ARIMA is linear; hence, it is not
capable of solving the volatile and non-linear nature of various time series including the
stock market [5]. Also, according to Namini et al. [6], it is hard to produce results using
ARIMA without a standard deviation in errors.
In the past years, new forecasting approaches in artificial intelligence to predict the
stock market has held researches’ interest. Nelson et al. [7] proposed a model based on
LSTM using technical indicators to predict the stock market. Bao et al. [8] built a
framework to forecast the closing price of six different stocks. Despite being a powerful
method in forecasting stock price, LSTM has its limitation. As reported by Krauss et al.
[9], it was hard to train neural networks, thus leading to difficulty in LSTM.
There have been various papers on big stock markets, such as the ASE, yet no
paper on small, young and immature markets like the VSE. Therefore, by investigating,
analyzing and comparing the performance of ARIMA and LSTM in the VN-index
prediction, we hope to set an example in researches on the application of these two
forecast techniques in similar stock market.
This study is structured as follows. Section 2 discusses the background of ARIMA
and LSTM. Section 3 presents results on the application of each model in Vietnam
stock market prediction. Further discussion is included in Sect. 4. Finally, Sect. 5
concludes the paper.
2 Methodology
2.1 Dataset
VN-Index is gathered in the format of a time series of candles (open, close, high, low
and volume) in a granularity of a day. Date from 2000 to 2018 with 4,441 recorded
days (2000–2018) was collected from website1. As mentioned above, it is applied as
objective prediction because of its influence to other indexes and the strength of stock
market. This dataset is split into two parts: 95% for training set and 5% for testing
respectively.
1
https://www.cophieu68.vn/index.php.
Comparison Between ARIMA and LSTM-RNN for VN-Index Prediction 1109
2.2 ARIMA Model
Forecast model ARIMA (Autoregressive Integrated Moving Average), first introduced
by Box and Jenkins [10] in 1974, is a time series forecast model which uses latency
algorithms to provide an appropriate prediction model. The future value of a variable in
ARIMA model is a linear combination of past values and past errors. ARIMA is
available integrated on STATA 15 as follows:
In brief, ARIMA model is stimulated by three variables (p, d, q) in which: p: the
auto regressive variables, d: the number of difference, q: the moving average variable.
ARIMA is determined using the Box-Jenkins method with three following steps:
(1) Model identification: based on Dickey-Fuller test, correction function (ACF) and
partial auto correction function (PACF), shapes of ARIMA for VN-Index are
defined as ARIMA ARIMA(1,1,1), ARIMA(5,1,1), ARIMA(5,1,3), ARIMA
(10,1,2), ARIMA(13,1,1).
(2) Parameter estimation: for each ARIMA model, Train will be used to determine
these variables u and j based on training test.
(3) Model test: The stability will be tested in order to find the most appropriate model
for the problem.
(4) Analysis and forecast: When the model satisfies the test, the model will be
applied to the data set of 240 days (from record 4201 to record 4441). In order to
evaluate the accuracy of the model, we then calculate the mean squared error
(MSE).
2.3 Long Short Term Memory-Recurrent Neuron Network (LSTM- RNN)
An LSTM-RNN is an artificial neural network that is an extension of Recurrent Neural
network (RNN). RNN is capable of generating a continuous sequence by establishing a
loop connection, so the information of the current state is used in the next state.
A LSTM block is a multi-layers network and applied to replace memory states in RNN
that excels at remembering values for either long or short durations of time.
All data in the format of a time series of candles (open, close, high, low and
volume) are used for closed prediction in LSTM-RNN model. The data are designed to
work on windows sizes. The size represents a length of historic data. The number of
LSTM blocks for each recurrent network layers represent the complexity of network.
More LSTM blocks in each layer make the model more fit to training data, but the time
consumption should be considered.
LSTM model for VN-Index prediction is composed of a sequential input layer
followed by two LSTM layers and a dense layer with ReLU activation and then finally
a dense output layer with linear activation function. Two fully connection layers are
applied in order to optimize the output layer of RNN networks. The code of the Neural
Network implemented in Keras is as follows:
1110 N. T. Co et al.
model = Sequential()
model.add(LSTM(num_LSTM,input_shape=(layers[1], layers[0]), re-
turn_sequences=True))
model.add(LSTM(num_LSTM,input_shape=(layers[1], layers[0]), re-
turn_sequences=False))
model.add(Dense(16,init='uniform',activation='relu'))
model.add(Dense(1,init='uniform',activation='linear'))
The parameters of environmental implementation are described as: Python pro-
graming language version 3.7.2 computer with Centos 07 OS, CPU Core i7, RAM
8 GB, Graphic Card, GeForce-GTX-1070-Ti-GAMING-8G.
3 Results
3.1 ARIMA Models
The results of the accuracy recorded when applying the ARIMA models for the data set
is showed in Table 1. It is clear that the more days used for prediction increased, the
more the MSE decreased. With the window size = 1, stock price prediction based on
only the data of the day before resulted in a large MSE (2648.54); therefore, one-day
data was not reliable enough to predict stock prices of the following day and only
allowed us to see the trend of VN-Index. When the window size was increased
(window size = five (a week), = 10 (two weeks), = 13 (two and a half weeks), the
MSE was 200, which proved that the data ranging from one to two weeks were optimal
for VN-Index forecast models.
Table 1. MSE and RMSE of ARIMA models
Model MSE RMSE
ARIMA(1,1,1) 2648.543 51.464
ARIMA(5,1,1) 225.8521 15.028
ARIMA(5,1,3) 204.9193 14.315
ARIMA(10,1,2) 201.6777 14.201
ARIMA(13,1,1) 213.2943 14.605
3.2 LSTM-RNN Models
The graph a on Fig. 1a shows that the MSE converged at week 3 if only 256 LSTM
elements were used and week 2 for 512 elements. Even if the window size was
increased, the deviation would decrease slightly; therefore, a period of 15 days with
256 LSTM elements and that of 10 days with 512 LSTM elements are appropriate for
the stock prediction problem.
Comparison Between ARIMA and LSTM-RNN for VN-Index Prediction 1111
Fig. 1. Performance of LSTM - RNN models following window size: (a) MSE (b) Time
consuming
From the graph Fig. 1b, the calculation time was directly proportional to the
number of days used for prediction. If the window size is larger than 20 then the
calculation time soared in both case: 256 LSTMs and 512 LSTMs.
4 Discussion
In terms of accuracy, the best prediction result of VN-Index can be seen in LSTM-RNN
model from 15 days or more (with MSE = 121.43). The result is so far relevant to both
literature and practical review. LSTM-RNN is a neural network with nonlinear cal-
culation with a large number of parameters, thus, within larger data; the neuron net-
work model will become more realistic. On the other hand, ARIMA uses fewer
parameters and the linear hypothesis, which should only be applied in short-term
prediction. The good performance observed through Deep-learning approaches to the
prediction problem is due to the “iterative” optimization algorithm. By iterative, we
mean obtain the results several times and then select the optimal one, i.e., the iteration
that minimizes the errors. As a result, the iterations help in an under-fitted model to be
transformed to a model optimally fitted to the data.
In practice, many investors and brokers of big stock companies use the line EMA15
as an indicator of trend. EMA is the average line with exponential function - a line that
graphs the average value of the closing price for a certain time period. EMA15 is a
moving average line for 15 days. According to Granville’s rule and the actual trans-
action of investors on the VSE, when the closing price is above the EMA15 (blue line),
the stock follows an upward trend and should be bought. Conversely, when the closing
price is below the EMA15, then it follows the downward trend and should be sold.
Also, the psychology and actions of the investors are supposed to be the most
influential factors to the market. Therefore, EMA15 is close to the market and has a low
latency. Furthermore, the prediction of stock price of 15 days (or more) could be used
to predict and learn stock market trends.
1112 N. T. Co et al.
5 Conclusion
In this paper, we propose two methods used to forecast VN-Index in the form of time
series problem: the method of using ARIMA model and the LSTM-RNN model. From
the empirical data, the author finds out that over the prediction length of 15 days, the
LSTM-RNN model can help improve the forecast accuracy by extending historical data
and combining with many other data sources. However, using deep learning means
having to pay for the computational complexity and the time it takes to get information.
One limitation of the study is that it is only carried out with Blue Chip, which are
income stock issued by well-established companies with a long history of positive
finances and steady dividend payments. Further research might be open to apply with
other stock types like Growth Stocks, Cyclical Stocks, Defensive Stocks. Furthermore,
there are other prediction problems in finance and economics that can be formulated
using Deep-learning. Thus, the author might investigate the improvement achieved
through Deep-learning by applying these techniques to other problems and datasets
with various numbers of features.
References
1. Meyler, A., Kenny, G., Quinn, T.: Forecasting Irish inflation using ARIMA models (1998)
2. Yaziz, S., Azizan, N., Zakaria, R., Ahmad, M.: The performance of hybrid ARIMA-GARCH
modeling in forecasting gold price (2013)
3. Namini, S.S., Tavakoli, N., Namin, A.S.: A Comparison of ARIMA and LSTM in
Forecasting Time Series (2019)
4. Nelson, D.M., Periara, A.C., de Oliveira, R.A.: Stock market’s price movement prediction
with LSTM neural networks. In: Nelson, D.M.Q., Pereira, A.C.M., de Oliveira, R.A. (eds.)
Stock Market’s Price Movement Prediction International Joint Conference on Neural
Networks (IJCNN) (2017)
5. Bao, W., Yue, J., Rao, Y.: A deep learning framework for financial time series using stacked
auto encoders and long-short term memory, July 2017
6. Krauss, C., Fischer, T.: Deep learning with long short-term memory networks for financial
market predictions. Eur. J. Oper. Res. 270, 654–669 (2017)
7. Box, G., Jenkins, G.: Time Series Analysis: Forecasting and Control (1970)
8. Zhang, D., Zhang, X.: Study on forecasting the stock market trend based on stochastic
analysis method. Int. J. Bus. Manag. 4(6), 163–170 (2009)
9. Svoboda, M., Lukáš, L.: Application of Markov chain analysis to trend prediction of stock
indices. In: Proceedings of 30th International Conference Mathematical Methods in
Economics (2012)
10. Gupta, A., Dhingra, B.: Stock market prediction using hidden Markov. In: 2012 Students
Conference on Engineering and Systems (SCES). IEEE (2012)
E-material Formatting Application Prototype
2.0 Development Through Usability Testing
of Prototype 1.0
Kristine Mackare1(&), Anita Jansone1, and Raivo Mackars2
1
Faculty of Science and Engineering, Liepaja University, Liela Street 14,
Liepaja, Latvia
{kristine.mackare,anita.jansone}@liepu.lv
2
IK, Liepaja, Latvia
[email protected]
Abstract. The increase of digital device use is a reality. The idea about the
automated e-material formatting application development came as a possible
solution for the improvement of e-material reading comfort and text perception
from the screen. It is needed as there are no complete recommendations for e-
material formatting, and users are having complains after work behind screens.
After prototype 1.0 development, it is understandable that there is a possibility
of idea realization. Preliminary prototype usability testing was conducted in a
small group. It showed places for improvement. Prototype 2.0 is focused on
improvements. A correctly created prototype must give the possibility to con-
duct an alpha testing of the application in a larger user group.
Keywords: Application prototype E-material formatting Usability testing
1 Introduction
Increase of the digital device use is a reality, and it is topical in all age groups, from
small kids to the elderly population. Probably, it is even higher when technology
development and improvement. Devices are used in every edge of human life, espe-
cially at work and education sphere. According to statistics, at least half of the
worldwide population uses the internet and devices providing it [1].
Different e-materials is one of the most popular information exchange types and
knowledge gaining processes nowadays. Textual type of information performance in
education processes is the main of all e-materials. As well as about 80% of information
humans are gaining by visual perception and vision is the dominant sense [2].
The idea about the automated e-material formatting application development came
as a possible solution for the improvement of e-material reading comfort and text
perception from the screen. It is needed as there are no complete recommendations for
e-material formatting [3], and users are having complains after work behind screens.
Mostly, complains are related to visual comfort and visual clearness. Not only
discomfort and users complains is a problem, but also a long-term effect on users’
visual and ocular health is becoming a public health problem related to near work and
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1113–1118, 2020.
https://doi.org/10.1007/978-3-030-39512-4_169
1114 K. Mackare et al.
screen reading [4, 5]. It affects education quality and work performance. Users are
asking for user-centered designs and solutions.
Previous work includes broad theoretical and statistical research, users’ testing and
users’ research, development of the recommendations and the methodology of
appropriate e-material formatting, development of the application concept, and
development of the application prototype 1.0.
2 Prototype 1.0
2.1 Idea and Concept
The concept for application prototype based on developed recommendations for e-
material formatting by users’ individual factors and need. It represents three mine edges
of the app: user, interface, and database. It includes collaboration processes between
edges and the mine idea of the app – text formatting of documents based on user
groups. The more detailed description and visual representation of the concept can be
found in previous publications [6, 7].
The app collects information as a user’s data, processes it, and save them in the
database. Then the app recognizes the necessary data from all data in the database by
categories and uses necessary data for the chosen formatting level. It includes data
analyses based on the training database. The app categorizes data and provides deci-
sions using the decision tree method. After, the app decides for formatting recom-
mendations on what to suggest for each user. Finally, it makes document formatting
related to a selected recommendation based on specific developed recommendations of
parameters.
If needed, the user can make additional changes in text formatting. App collect data
about changes. After the use of new formatting in e-material, the app collects users’
feedback. The app uses all collected data for the learning process to improve person-
alized recommendations to create a user-centered design. To achieve a high level of
personalization of e-material formatting, AI and ML approach should be used. A more
detailed description of the application idea and working plan can be found in previous
publications [6, 8].
2.2 Prototype 1.0
The first developed version of the prototype gives the overview of application work to
follow implemented working schemes, relationships between main edges and the
collaboration process with the user, material, and database. All as described in concept
before. An application can be described from four sides: developers, e-material for-
matting users as readers, e-material creators and researchers.
For the possibility to give formatting recommendations, the application collects
necessary data from the user. Depending on the formatting personalization level, it
could be only general information or more detailed information about the user. The
solution-finding for e-material formatting follows data collecting. It is realized with the
use of step-by-step recommendations and so-called tree-scheme of users’ answers and
E-material Formatting Application Prototype 2.0 Development 1115
related combinations of possibilities. The application responds and gives a recom-
mendation of text formatting for e-materials. And it provides automatized formatting
according to suggested recommendations. After the user has tried new formatting of e-
material, the application provides a short questionnaire of user feedbacks.
An XML programming language was used for the first application prototype
development.
The application prototype is developed on Moodle type platform base but with
possible transformation and adaptation for a different environment and wider range of
use. As the application is developed to give access to the database to researchers, it
helps in user-habit research and allows to keep the application up to date to reach
application learning process. That is an essential part of nowadays user-centric designs
for users’ satisfaction. A more detailed description of the application prototype is
published previously [9, 10].
Developed prototype 1.0 gives the overview of the application work to follow
implemented working schemes, relationships between the main edges and the collab-
oration process with the user, material and database.
After prototype 1.0 development, it is understandable that there is a possibility of
idea realization. For the best realization of the final product analyze of existed proto-
type must be done as the first prototype mostly leaves room for improvements.
3 Usability Testing
By simplified definition of the usability it is “easy to use”. The good tone for the
application usability tells what usability testing should be done:
• as soon as possible,
• as often as possible
• and better to have one or small group testing at begging than the large group at the
end [11].
After the development of the first version of the prototype, authors have concluded
that there is a need for the first usability testing.
In addition, the focus should be as is the application:
• useful
• findable
• accessible
• usable
• desirable [12].
Preliminary prototype usability testing was conducted in a small group between
developers of the application and persons informed about the development, its idea and
purpose of the final product. The prototype 1.0 was presented for users and tested for
the application usability and e-material formatting of some regular Word documents.
After, feedbacks about the application was collected.
It showed that it is not only possible to make such an application but that there is
room for improvement.
1116 K. Mackare et al.
First, it showed the lack of full understandability of the problem and a need for
application from the programmer side. Also, there is a lack of experience chosen
programmer for this type of application program realization on high quality.
The secondary, inappropriate visual design of the prototype was shown. But it was
an early stage of development; that why visual appearance was not a priority. It needed
being ready for a try of e-material formatting function.
Thirdly, it showed a mistake in programming language choice. All four previously
analyzed programming languages, including chosen an XML programming language
for the first application prototype development, are entirely out of date for such
solutions.
It was followed by showing a need for improvement in three-scheme use of rec-
ommendations as well as a lack of possibility to modify all the most popular e-material
formats.
Also, during testing, it showed sensitive places in the application that should be
taken into consideration before the development of the final product. It includes a
specific of different versions of such accessible format as Word *.docx that can bring
errors in the formatted document.
Finally, it shows limitations of the existing approach and a need for adding Arti-
ficial Intelligence and Machine Learning aspects to reach a higher level of personal-
ization for e-materials that could be based on users’ needs.
4 Prototype 2.0
Prototype 2.0 is focused on the improvement of all parts, excluding Artificial Intelligent
and Machine Learning, as it is the next step of the application development.
The first decision was to change the view of the programmer on application by
extra educating about the background of the research and getting additional and more
experienced programmers for help.
Secondary, as the application will be developed to use primarily on the e-study
environment, in the second prototype has been used PHP7.3 programming language, as
it is the server language. It is a programming language which is used for interaction
between the browser and the server, which should be a perfect option for this appli-
cation development. Using PHP7.3 functions allows overwriting the XML code, thus
are changing and modifying the e-materials formatting, updating them to another
formatting for learner needs.
The third, in the prototype 2.0 focus, is to make it accessible and workable on all
most popular formats of e-materials, not only Word but including PDF. To achieve this
part, there is a need to take a more in-depth look into existing Word to PDF converters.
A possible solution is to take several steps to convert processes — research and
solution finding in progress.
The fourth significant upgrade is that the app is made in such a way so it could be
used with any e-material system not only Moodle. Moreover, as e-learning is getting
more and more popular on the web, beyond traditional education as schools and
universities, then authors concluded that the app must be able to format the web
materials that are viewable only in the browser, therefor edit HTML and CSS.
E-material Formatting Application Prototype 2.0 Development 1117
In theory, it is possible to develop the application like a web browser add-on, which
can modify any base code of the loaded webpage. Then it will be able to edit HTML
format by modifying its CSS parameters too. It will be improved in the final version of
the app [13].
The improved version of the application will de focus on visual appearance as
prototype 2.0 is planned to use for the alpha testing. It is vital not only for usability
testing but for future improvement of the final product.
As errors have been found in the popular format as Word *.docx, there is a need to
overview and test all different versions of accessible formats for the specific differences
in them. It will allow for catching up as many errors as possible and figure out possible
solutions or at least inform users about that error’s possibilities.
It is planned to implement machine learning artificial intelligence to suggest the
best parameters as fonts, sizes, and colors for each person, but it initially needs a huge
data amount of user choice preferences. So, alpha tests will allow collecting as much
data as possible. But for that application should have as little errors and problems as
possible.
A correctly created prototype must give the possibility to conduct an alpha testing
of the application in a larger user group.
5 Conclusions
The first testing was made in the early stage of application development. Was tested
quite raw application prototype and was expected to meet errors.
Usability testing is beneficial as it shows problems of the developed prototype and
places what must be improved.
The advantage of early-stage usability testing, even in a small group, is to find out
basic errors and problems that can kill the perspectives of the development process and
the final product.
E-material formatting application prototype 2.0 development through the prelimi-
nary usability testing of the prototype 1.0 have been very successful and allowed to find
a massive amount of problems and mistakes of the prototype 1.0. So, the developing
team is able to make improvements in the early stage of application development.
It’s a huge step closer to AI and ML approaches could be used in the application
and to the successful final product launching.
Acknowledgments. The article is written with the financial support of European Regional
Development Fund project Nr.1.1.1.5/18/I/018 “Pētniecības, inovāciju un starptautiskās sadar-
bības zinātnē veicināšana Liepājas universitātē”.
1118 K. Mackare et al.
References
1. Report from We Are Social and Hootsuite, Global Digital Statshot Q3 (2017). https://www.
slideshare.net/wearesocialsg/digital-in-2017-global-overview
2. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12,
97–136 (1980)
3. Mackare, K., Jansone, A.: Research of guidelines for designing e-study materials. In:
Proceedings of the 11th International Scientific and Practical Conference Environment,
Technology, Resources, ETR 2017 Rezekne, Latvia, 15–17 June 2017, vol. 2, pp. 90–96
(2017)
4. Mackare, K., Jansone, A.: Digital devices use for educational reasons and related vision
problems. In: Proceedings of the 4th International Conference on Lifelong Education and
Leadership for All, ICLEL 2018, Wroclaw, Poland, 03–05 July 2018, pp. 762–769 (2018)
5. Mackare, K., Jansone, A.: Habits of using internet and digital devices in education. In:
Proceedings of the 12th International Scientific Conference, SIE 2017, Rezekne, Latvia, 25–
26 May 2018, vol. V, pp. 348–356. Society Integration Education (2018)
6. Mackare, K., Jansone, A., Zigunovs, M.: E-material creating and formatting application. In:
Advances in Intelligent Systems and Computing 876, Human Systems Engineering and
Design: Proceedings of the 1st International Conference on Human Systems Engineering and
Design (IHSED2018): Future Trends and Applications, Reims, France, 25–27 October 2018,
pp. 135–140 (2018)
7. Mackare, K., Jansone, A.: The concept for e-material creating and formatting application
prototype. PEN, Periodicals Eng. Nat. Sci. 7(1), 197–204 (2019)
8. Mackare, K., Jansone, A.: Artificial Intelligence aspects in developed E-material formatting
application. In: Conference Proceedings of the International Conference on Artificial
Intelligence and Advanced Manufacturing of ACM 2019, AIAM 2019, Dublin, Ireland, 17–
19 October 2019 (2019)
9. Zigunovs, M., Jansone, A., Mackare, K.: E-learning material adaptive software development.
In: The 2nd International Conference of Innovation and Creativity, ICIC 2018, Liepaja,
Latvia, 5–7 April 2018 (2018)
10. Mackare, K., Jansone, A., Konarevs, I.: The prototype version for e-material creating and
formatting application. BJMC J. 7(3), 383–392 (2019)
11. What Does Usability Mean: Looking Beyond ‘Ease of Use’. https://www.wqusability.com/
articles/more-than-ease-of-use.html
12. User Experience Basics. https://www.usability.gov/what-and-why/user-experience.html
13. Mackare, K., Jansone, A., Mackars, R.: Use of Artificial Intelligence and Machine Learning
for personalization improvement in developed E-material formatting application. In:
Computing Conference 2020 Proceedings will be published in Springer series Advances
in Intelligent Systems and Computing, London, (Will be published. Approximate Publishing
Date: in 2020)
Use of CAD-CAM Technologies
in the Production of Furniture for Natural
Disaster Areas in Ecuador
Francesco Giuseppe Magnone1(&),
Víctor Gustavo Gómez Rodríguez2, Yoenia Portilla Castell3,
and Jesús Rafael Hechavarría Hernández1
1
Faculty of Architecture and Urbanism, University of Guayaquil,
Cdla. Salvador Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{francesco.magnone,jesus.hechavarriah}@ug.edu.ec
2
Faculty of Commercial Education and Sciences, Bolivariano Technological
Institute of Technology, Guayaquil, Ecuador
[email protected]
3
Higher Technological Institute of Administrative and Commercial Professional
Training, Guayaquil, Ecuador
[email protected]
Abstract. This article presents the experience developed in the Interior Design
career of the Faculty of Architecture and Urbanism of the University of
Guayaquil in the realization of furniture prototypes with CAD-CAM technology.
This process has allowed the development of new knowledge based on the
proposed methodology of the Italian Bruno Munari that uses design methods
with experimentation phases, achieving appreciable results in the form of pro-
totypes with innovative characteristics according to a serial production. The
market analysis allows us to know that not all user needs are met in terms of the
demand for furniture in natural disaster areas. The research on the technology
that is proposed to be used leads to the choice of CNC manufacturing machines
coupled to a CAD-CAM system.
Keywords: CAD-CAM CNC manufacturing Furniture production
Industrial design
1 Introduction
The composition of the Gross Domestic Product (GDP) of the Ecuadorian economy has
had significant changes. In 2004, the sector with the greatest contribution was oil and
mines, with 13.2%. In 2014, manufacturing obtained the highest contribution in GDP
with 11.8%. This demonstrates the growing influence of this sector in the Ecuadorian
economy and its influence on the productive development of the country. However, the
increase achieved in the last 10 years is 0.3%, which indicates that there are still
challenges to strengthen the participation of that sector in the economy. The sectors
with greater public investments and greater access to financing have achieved greater
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1119–1125, 2020.
https://doi.org/10.1007/978-3-030-39512-4_170
1120 F. G. Magnone et al.
contributions to GDP, among which are: construction (+2.4%) and mail and commu-
nications (+1.8%) [1].
Ecuador is a country that exports raw materials and imports finished products from
abroad, which means that the production of finished products is not fully utilized,
giving other countries the opportunity to take advantage of their raw material and then
market products that Ecuador It must matter. This situation causes that the import made
by Ecuador is superior to the export because the finished products have an added value
for using registered trademarks, wasting great potential in the development of national
production [2].
Design is a discipline partly scientific and partly technical, not forgetting the link
with the area of sociological and humanistic. Some authors define design as the con-
nection between Engineering and Art. Where Engineering deals with fulfilling a
specific function in response to a need and Art deals with the emotional part expressing
a message. The design is between these two categories, sometimes closer to engi-
neering and other times closer to an artistic expression according to the intrinsic
characteristics of each designer and the requirement of each project. Currently, a design
cannot be conceived without considering the manufacturing process due to the diversity
of options that are consistent with the product’s service destination and the quality
requirements that must be achieved. In the present work, an integration between the
design process and manufacturing technologies is achieved by applying the Bruno
Munari method in which the design is carried out in a series of necessary operations
arranged in a logical order dictated by experience.
On the national scene, furniture production is often done by hand, with strong
competition with imported products. That is why this document proposes original
products produced in Ecuadorian territory using native resources and a low cost pro-
duction process. The products will not only have the basic function dictated by stan-
dard use, but will also be a response to the needs of the users considering their
anthropometric measurements.
The use of CAD-CAM technologies has diversified in recent years due to the high
quality of the results obtained in multiple spheres of science and technology, among
which are: Architecture [3], Engineering [4] and health related areas [5–7]. In this
work, this technology is applied to the design and manufacture of furniture. Among the
relevant results obtained, the project entitled BOXED FURNITURE stands out.
The idea arises after the 7.8 Mw magnitude earthquake that hit the coasts of
northwestern Ecuador in 2016. The most affected province was Manabi; one of its
cantons, Pedernales (55,000 inhabitants), was declared a “disaster zone”. Thousands of
buildings were destroyed in 6 affected provinces and more than 829 were damaged,
including 281 schools. Additionally, 25,376 people were refugees in collective shelters
[8] (Fig. 1).
Knowing that the education system must be reinvented periodically to face the
changes of social and cultural paradigm [9], students of the Interior Design career of the
University of Guayaquil were involved in solving the problem. That offer students the
opportunity to explore problems and challenges that have real applications, increasing
the possibility of acquiring competences effectively needed to practice the profession of
interior designer.
Use of CAD-CAM Technologies in the Production of Furniture 1121
Fig. 1. Disaster zone in Ecuador after the earthquake of 2016.
2 Munari Methodology
Once the need has been identified and the type of product to be obtained has been
decided, as a proposed solution for the furniture deficit during periods of natural
disasters, the Munari methodology was followed to bring the initial idea closer to a real
and feasible solution. Now, the conceptual idea of the product becomes the design
problem to solve. The most important characteristics that will be an indispensable part
of the product are defined and, subsequently, the problem is divided into parts, which
will be solved one by one (Fig. 2).
Fig. 2. Munari conceptual model applied to furniture design.
1122 F. G. Magnone et al.
These components of the problem allow us to define the research guide to acquire
the necessary knowledge in the good performance of the design. The analysis of the
existing products in the market allowed to know the possible competitors, the level of
compliance of the existing products with respect to the control characteristics of the
product to be designed to implement: new functions, changes of use and to reduce:
design, manufacturing, transport and assembly times.
3 Proposed Solution
In the face of an existing problem, the strongest temptation is to provide an intuitive
solution, but with the infinity of variables that can influence, the result can be not
effective, very expensive or technically not viable. Considering the situation presented
by the inhabitants of the areas affected by the earthquake, A multidisciplinary team of
researchers contributed ideas for the development of prototypes characterized by being
quick to build, easy to transport and assemble, as well as deciding the productive
technology to use, for the sake of providing the industrial sector of Guayaquil with
prototypes according to a serial production considering the technological advances that
characterize the current historical period defined as Industrial Revolution 4.0. For this
purpose, it was selected to use machine tools with CNC technology for the quality
obtained in the product and the manufacturing speed of the cutting process. The design
was characterized by achieving a compact product, fast serial production, which can be
transported in packaging boxes to facilitate transport and storage and allow an
assembly without tools.
The elaboration of furniture in box, as a new method of industrialization on request,
allows the furniture to be sold disassembled, assembled and resistant to use, in addition
to solving the lack of storage space and the difficulty of transporting the product. The
objective was to meet a high demand for furniture by reducing the delivery times of the
product and the costs associated with production and transportation (Fig. 3).
Fig. 3. Scheme of the idea for the fulfillment of objectives.
Use of CAD-CAM Technologies in the Production of Furniture 1123
The following activities were carried out to achieve the proposed objectives:
• Analysis of the furniture market.
• Selection of CNC cutting technologies.
• Resistance tests during assembly and service destination.
• Material savings during the production of furniture components.
• Evaluation of aesthetic and functional impact.
4 Results
The CNC manufacturing machines used are coupled to a CAD-CAM system that
guarantees the integration of digital information with the orders that the machine tool
must fulfill to obtain the product by cutting the material.
The CNC Router cutting machine used has a 3 kwp motor and works as a milling
machine with interchangeable drill bit, capable of following a line drawn with a vector
program for real-time furniture production. In the process, a 6 mm thick drill bit was
used, achieving an adequate cutting regime considering the advance of the tool for a
variable speed of rotation up to 24000 rpm (revolutions per minute). The work area of
the machine allows cuts on boards of various materials derived from wood and plastics
whose dimensions are: 1300 2500 200 mm. Figure 4(a) shows the technology
used and in Fig. 4(b) the result achieved.
Fig. 4. CNC cutting technology. (a) Cutting machine. (b) Finished product.
A CNC laser machine (Rayjet 500–100 watts) was used to make the furniture
models at a 1:5 scale, due to the high precision and cleanliness of the cut. Figure 5(a)
shows the technology used and in Fig. 5(b) the result achieved.
1124 F. G. Magnone et al.
Fig. 5. CNC cutting technology. (a) CNC laser machine. (b) Finished product.
5 Discussion
The analysis of the market allowed us to understand that there is still no adequate
domain of CAD-CAM technologies, so the satisfaction of the users in terms of delivery
times and product quality are not fully met. In the majority of cases, furniture sold in
Ecuador is provided armed and there are no furniture that presents all the necessary
functions to solve the problem. This shows the opportunity to provide products that
meet the needs of users. Among the results achieved stand out 129 intellectual property
registers related to furniture design in the Interior Design career of the Faculty of
Architecture and Urbanism of the University of Guayaquil.
Acknowledgments. To the Faculty of Architecture and Urbanism of the University of
Guayaquil. To the Faculty of Commercial Education and Sciences, Bolivariano Technological
Institute of Technology. Al Higher Technological Institute of Administrative and Commercial
Professional Training. To all the people who have contributed in the development of this
research.
References
1. Central Bank of Ecuador. https://www.bce.fin.ec/index.php/banco-central-del-ecuador.
Accessed 10 Sept 2019
2. Ecuador - Directory of Companies and Establishments 2017. https://anda.inec.gob.ec/anda/
index.php/catalog/672/related_materials. Accessed 10 Sept 2019
3. Goldschmidt, G.: Architecture. Ref. Modul. Neurosci. Biobehav. Psychol. (2020)
4. González-Henríquez, C.M., Sarabia-Vallejos, M.A., Rodriguez-Hernandez, J.: Polymers for
additive manufacturing and 4D-printing: materials, methodologies, and biomedical applica-
tions. Prog. Polym. Sci. 94, 57–116 (2019)
5. Roe, P., Rungcharassaeng, K., Kan, J.Y., Putra, A.: Adhesive residue on the CAD-CAM
surgical guide sleeve: a technical report. J. Prosthet. Dent. 121(5), 746–748 (2019)
6. Unsal, G.S., Turkyilmaz, I.: Improved reconstruction of an implant-retained auricular
prosthesis using CAD/CAM technology. J. Dent. Sci. 14(3), 328–329 (2019)
7. Reddy, S., Macmillan, A.: Virtual surgical planning. Facial Trauma Surg, pp. 483–492 (2020)
Use of CAD-CAM Technologies in the Production of Furniture 1125
8. Ecuador: Llamamiento - Terremoto ocurrido el 16 de abril de 2016. Periodo: Abril a Julio de
2016 - Ecuador | ReliefWeb. https://reliefweb.int/report/ecuador/ecuador-llamamiento-
terremoto-ocurrido-el-16-de-abril-de-2016-periodo-abril-julio-de. Accessed 10 Sept 2019
9. Matos, D., Terroso, M., Sampaio, J.: The growing path in search of an industrial design
identity. Procedia CIRP 84, 353–356 (2019)
A Five-Factor KMS Success Model
Gabriel Nyame(&) and Zhiguang Qin
School of Information and Software Engineering, University of Electronic
Science and Technology of China, No. 2006, Xiyuan Ave, West Hi-Tech Zone,
Chengdu 611731, Sichuan, People’s Republic of China
[email protected]
Abstract. We propose a KMS success model using five critical factors that
must be present and effective. The factors are based on past and current
Knowledge Management (KM) practices and review of Knowledge Manage-
ment Systems (KMS) success literature. DeLone and McLean’s Model is pivotal
to our model, while Trist’s suggestion in 1967 of the socio-technical perspective
of organizations remains valid in explaining these factors. Management and
leadership, knowledge sharing and transfer capability, knowledge architecture
and infrastructure, knowledge innovation capability, and knowledge access
control policy are the factor requirements and antecedents for KMS success. The
paper analyzes the interconnectedness and relationships between the factors, and
justifies factor presence and effectiveness contingent on tools for measuring
KMS success in general or specifics.
Keywords: Knowledge Management System Knowledge innovation
capability Knowledge access control Knowledge architecture and
infrastructure Factor presence and effectiveness
1 Introduction
Practitioners and researchers in the IS discipline in general and KMS, in particular,
elevate and still recognize knowledge as an essential intangible resource (e.g. [1]). The
success and competitiveness of organizations are obtained from possessed unique
knowledge [2, 3]. Other works such as [4–6] ascribed an organization’s sustainable
competitive edge in terms of its effective management of knowledge. [7] and [8]
described KM as a vehicle to integrating an organization’s capabilities in ways that lead
to creativity, innovation, and performance.
KMS defines all those knowledge architecture and technologies (hardware and
software) that enable people in and across an organization to capture, create, store,
share, disseminate, and reuse knowledge [9]. Not in all instances of KMS adoption and
diffusion in organizations might lead to KMS success. [10] pointed out that only about
30% of KM initiatives deployed by organizations could reap full net benefits. The view
of the author suggests how careful and tactical the approach to KM initiatives should
be. It is essential, however, to note that the design and implementation of KMS are
context-specific and dependent on those business requirements that KMS is envisaged
to address.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1126–1131, 2020.
https://doi.org/10.1007/978-3-030-39512-4_171
A Five-Factor KMS Success Model 1127
The intriguing nature of socio-technical KMS impacts organizational learning,
knowledge capabilities, and knowledge value creation. However, an aspect in the KMS
literature that remains incoherent is the failure to ensure the presence support and
effectiveness of the five critical factors during the design and implementation of KMS.
In this study, we uncover these five factors in a model and acknowledge the DeLone
and McLean IS model, the social system theory, the organizational capability theory,
and the knowledge-based theory.
2 KMS: The DeLone and McLean’s IS Success Model
KMS is an activity-based and knowledge management support system that captures,
creates, stores, transfers, shares, and utilizes knowledge for competitive advantage [11–
13]. It brings out the inherent tacit knowledge within individuals and teams together
with explicit knowledge for organization-wide learning that increases organic growth
and innovation capabilities.
User roles, tasks, and responsibilities generally impact the role of KMS. As roles
are associated with tasks and tasks defined based on stated responsibilities, [14] raised
concerns about the need to have an appropriate fit between tasks and technology. Thus,
to harness, integrate, and apply knowledge, KMS facilitates knowledge transfer,
sharing, collaboration, and collectivism.
Measuring KMS success mostly pivoted on DeLone and McLean’s IS success
model introduced in 1992 and updated in 2003 by [15, 16]. In the updated model,
context is vital in the specification and application of the D&M IS success model. Thus,
many past studies applied model contextually in different organizational settings.
Similarly, we recognize the D&M IS success model in the context of factor presence
support and effectiveness. In essence, the updated model articulated security construct
as a metric measure for only the information quality dimension. We, however, argue
that system security, knowledge security, and service security respectively characterize
KMS quality, knowledge quality, and KMS service quality (Ref. D&M IS success
model, 2003, p. 12). Hence, our proposed model includes knowledge access control
policy, and antecedently guarantees practical take-off in an attempt to measure KMS
success.
Many developers and designers are careful not to restrict knowledge-sharing or
transfer capability. However, [17] explained the need for businesses to include security
mechanisms to avoid misuse and unauthorized access to knowledge resources. From an
access control perspective, rules are attached to roles assigned to users in accessing
knowledge repositories. It is thus justifiable to include in our model a security factor or
policy (e.g., role-based access control (RBAC) policy).
3 The Proposed Five-Factor Model
Our five-factor model preludes KMS success measurement, and these factors are
critical to any contextual KMS success measurement. We use the terms presence
support and effectiveness in KMS context to mean sustainability and success
1128 G. Nyame and Z. Qin
respectfully. While the former expresses a phenomenon of a long-term effect of
knowledge leveraging strategy, the latter emphasizes the achievement of expected
knowledge-oriented results. Thus, our discussion highlights the need to sustain these
factors while ensuring the quality success of each factor. Figure 1 shows the five-factor
model, and we discuss each of these factors and indicate the roles played by the
resource types. We emphasize that the relationships between the factors do not depict
causal effects but rather illustrating existential contexts within which the factors must
be present and productive.
Fig. 1. A five-factor model for KMS success
Knowledge Architecture and Infrastructure. Knowledge architecture refers to an
information-based framework that enables knowledge managers/workers to transform
unconnected data in a connected fashion and into usable/reusable knowledge. It is a
fundamental requirement for KMS solution development [18]. Mostly, tacit and
explicit knowledge in their complex forms gets scattered across an organization. Such
heterogeneous information needs to be identified, organized, and made accessible for
reuse. Thus, organizations need to institutionalize comprehensive and robust knowl-
edge architecture that leverages knowledge more effectively.
Knowledge infrastructure refers to all knowledge-related elements that facilitate
knowledge creation, capture, organization, access, and use/reuse of knowledge assets.
It is purely a socio-technical phenomenon and capability in its own right. It encom-
passes an organization’s culture, structure, information technology (hardware and
software), knowledge (tacit and explicit), and physical environment (internal and
external). Thus, it is critical for organizations to ensure the presence support and
effectiveness of stronger knowledge architecture and infrastructure that lowers risks
associated with unconnected data, eliminates time loss, and reduce the high cost
involved in operationalizing knowledge platforms.
A Five-Factor KMS Success Model 1129
Management and Leadership. Besides the vital role of senior managers and func-
tional leaders, they are the conduit to lower-level business operations. Intension and
attitudes towards knowledge-sharing are managerially related and require support,
commitment, and motivation from senior management [19, 20]. Importantly, the ability
of managers and leaders to excellently sustain the shared value of knowledge sharing
depends on the presence of support from senior management significantly.
As trust in senior management and leaders increases, knowledge sharing capability
also increases [19]. Hence, we emphasize that the presence (such as support in all
forms, commitment, motivation, and trust.) and effectiveness (such as KM initiatives,
ease of adoption, and ease of use) of senior management and leaders are inevitable to
KMS success.
Knowledge Innovation Capability. Innovation diffusion theory approaches KM
effectiveness in the context of innovation capability. The presence effect of this factor
in a KMS environment is to enhance the rate at which new knowledge is given
recognition and accepted for use for further increased innovation. Is the question worth
asking: In what shared vision will people willingly contribute knowledge to the system
and consciously participate in knowledge-sharing? For this critical question, [21]
identified the leverage of knowledge-based capabilities as being contingent on building
new knowledge that reinforces organizational capability.
The success effect of knowledge innovation capability relates to the unique abilities
of an organization towards knowledge innovation. By implication, people and teams
dynamically influence knowledge innovation capability across time and space. Hence,
the presence and success effects of knowledge innovation capability (1) integrates
innovation culture (2) increases knowledge innovation intention and behavior (3) im-
pacts adaptive system strategies, and (4) reinforces knowledge entrepreneurship in the
organization.
Knowledge Access Control Policy. Access control is a security-based mechanism
that regulates the ability of subjects (e.g., users or programs) to access valuable
organizational objects (e.g., knowledge or database assets, files, printers, or computers).
A subject assigned with some privileges requests accesses to use an object. Access is
granted or denied based on identity verification, authentication, and authorization
control principles such as RBAC policy. For quality and secured KMS, a trust-based
access control policy is necessary to address security related issues while at the same
time promoting rigorous knowledge sharing.
For fear of knowledge leakage, access control capability in KMS is the surest way to
enhance the knowledge integrity, confidentiality, and quality for an organization’s
competitiveness. Therefore, access control models such as RBAC deployed in KMS
need to be effective in the right context and goal for quality security enforcement.
Hence, we advocate that the presence support and effectiveness of knowledge access
control policy remains an important consideration as far as an organization’s intel-
lectual capital is a concern.
Knowledge-Sharing and Transfer Capability. A transfer and sharing capability
relate to the capacities of the source unit (i.e., willingness and enthusiasm to share or
transfer) and the recipient unit (i.e., disposition or predisposition to acquire, absorb, and
1130 G. Nyame and Z. Qin
apply) to mutually enhance knowledge innovation capability. Whether tacit or explicit
knowledge, the success of the knowledge sharing (i.e., knowledge-sharing effective-
ness) is contingent on the source entity’s capacity to identify and contribute possessed
knowledge and the recipient entity’s predisposition to adopt and re-create knowledge.
Also, the persistent interaction at every knowledge-sharing level (i.e., knowledge-
sharing presence) regarding tasks, routines, and procedures for each knowledge stage
in its life cycle facilitates knowledge growth and innovation. We, therefore, argue that
whether or not there are blurriness, stickiness, or barriers [22], knowledge sharing and
transfer capability must be present and effective at each knowledge stage and unit level
to increase knowledge innovation.
4 Limitations and Future Work
A significant limitation is that the extent of factor presence support and effectiveness is
indeterministic, and can vary among organizations in terms of both KM strategy and
KMS design. Moreover, the factors are not exhaustible. Our model does not consider
other inhibiting factors that can affect the existential contexts assumed in the model.
Our future work will test the model in order to evaluate the model’s applicability in the
KMS environment.
5 Conclusion
With varying organizational contexts and knowledge needs, several factors influence
the implementation and quality use of KMS. This paper provides initial insights into
five critical factors affecting KMS initiatives and proposes the need for the presence
and effectiveness of these five factors. Segregating each factor’s presence support and
effectiveness from others can jeopardize the KMS initiative. Hence, our model stands
to offer practitioners and researchers the platform to test the model and possibly extend
the dimensions due to the dynamic nature of KMS.
References
1. Hwang, H.G., Chang, I.C., Chen, F.J., Wu, S.Y.: Investigation of the application of KMS for
diseases classifications: a study in a Taiwanese hospital. Expert Syst. Appl. 34(1), 725–733
(2008)
2. Shrafat, F.D.: Examining the factors influencing knowledge management system
(KMS) adoption in small and medium enterprises SMEs. Bus. Process Manag. J. 24(1),
234–265 (2018)
3. Grant, R.M.: The knowledge-based view of the firm: Implications for management practice.
Long Range Plan. 30(3), 450–454 (1997)
4. Xu, J., Quaddus, M.: A six-stage model for the effective diffusion of knowledge management
systems. J. Manag. Dev. 24(4), 362–373 (2005)
5. Storey, C., Kahn, K.B.: The role of knowledge management strategies and task knowledge in
stimulating service innovation. J. Serv. Res. 13(4), 397–410 (2010)
A Five-Factor KMS Success Model 1131
6. Mesquita, L.F., Anand, J., Brush, T.H.: Comparing the resource-based and relational views:
knowledge transfer and spillover in vertical alliances. Strat. Manag. J. 29(9), 913–941 (2008)
7. Handzic, M., Ozlen, K.: The impact of KMS sophistication on decision support in varying
contexts. Front. Artif. Intell. Appl. 238, 349–357 (2012)
8. Lin, T.-C., Chen, C.-J.: Validating the satisfaction and continuance intention of e-learning
systems: combining tam and is success models. Int. J. Distance Educ. Technol. 10(1), 44–54
(2012)
9. Alavi, M., Leidner, D.: Knowledge management systems: issues, challenges, and benefits.
Commun. Assoc. Inf. Syst. 1(1), 7 (2018)
10. Ambrosio, J.: Knowledge management mistakes. Computerworld 34(27), 44 (2000)
11. Svetsky, S., Moravcik, O., Tanuska, P., Sakal, P.: The IT support of knowledge management
and transmission within community of researchers. In: Proceedings of the International
Conference on Intellectual Capital, Knowledge Management and Organisational Learning,
ICICKM (2018)
12. Mafereka, M., Winberg, S.: Analysis and development of an online knowledge management
support system for a Community of Practice (2017)
13. Chew, K.C.: An exploratory case study. Knowledge management: managing organizational
knowledge assets by aligning business strategy, knowledge strategy, and knowledge
management strategy. Dissertation Abstracts International. A, The Humanities and Social
Sciences (2009)
14. Lin, T.C., Huang, C.C.: Understanding knowledge management system usage antecedents:
an integration of social cognitive theory and task technology fit. Inf. Manag. 45(6), 410–417
(2008)
15. DeLone, W., McLean, E.: The quest for the dependent variable. Inf. Syst. Res. 3(1), 60–95
(1992)
16. DeLone, W.H., McLean, E.R.: The DeLone and McLean model of information systems
success: a ten-year update. J. Manag. Inf. Syst. 19(4), 9–30 (2003)
17. Memon, N., Daniels, T.: Special issue on secure knowledge management. Inf. Syst. Front.
9(5), 449 (2007)
18. Pan, S.L., Scarbrough, H.: A socio-technical view of knowledge sharing at Buckman
laboratories. J. Knowl. Manag. 2(1), 55–66 (1998)
19. Renzl, B.: Trust in management and knowledge sharing: the mediating effects of fear and
knowledge documentation. Omega 36(2), 206–220 (2008)
20. Chiu, C.M., Hsu, M.H., Wang, E.T.G.: Understanding knowledge sharing in virtual
communities: an integration of social capital and social cognitive theories. Decis. Support
Syst. 42(3), 1872–1888 (2006)
21. Gold, A.H., Malhotra, A., Segars, A.H.: Knowledge management: an organizational
capabilities perspective. J. Manag. Inf. Syst. 18(1), 185–214 (2001)
22. Paulin, D., Suneson, K.: Knowledge transfer, knowledge sharing and knowledge barriers-
three blurry terms in KM. In: Proceedings of the European Conference on Knowledge
Management, ECKM (2011)
A Study on Understanding of Visitor Needs
in Art Museum: Based on Analysis of Visual
Perception Through Eye-Tracking
Taeha Yi, Mi Chang, Sukjoo Hong, Meereh Kim, and Ji-Hyun Lee(&)
Graduate School of Culture Technology, KAIST, 291 Daehak-ro, Yuseong-gu,
Daejeon 34141, Republic of Korea
{yitaeha,rosechang,sukjoo.hong,meereh,
jihyunlee}@kaist.ac.kr
Abstract. This study aims to examine the art museum experience of visitors in
detail through eye tracking in the aspect of the visitor-centered approach, which
is important in the contemporary art museum. To achieve this goal, we con-
ducted the eye-tracking experiment and in-depth interview to grasp the interest
and needs of visitors. We suggest the possibility of deriving their interests and
needs by studying the gaze data (e.g. duration and the number of fixations) of the
visitors in the museum.
Keywords: Art museum Visitor studies Eye-tracking Museum experience
1 Introduction
The contemporary art museums have been developed as a visitor-centered museum to
prepare art education, participatory programs, mobile guide application, etc. For this
reason, observing and examining those who visit museums to grasp their needs has
become an important research area. Many researchers have proposed solutions and
studies that focused on recognizing the art exhibition experience of visitors in order to
enhance visitors’ experience in the museum. For example, [1] observed visitors to
figure out the characteristics of appreciation or behavior when they were looking
around in the art museum. These visitor studies help us understand the characteristics
of visitors in the museum. However, there are lacks of study on observing visitors not
just focusing on the exhibition experience, but considering the whole art museum
experience from entering the entrance to shopping art products. In addition, the recent
trend of visitor studies is utilizing cutting-edge technologies to analyze visitors,
because the new technologies improve the data collection with high accuracy [2].
In this paper, we will investigate visitor needs by observing the overall museum
experience and conducting an in-depth interview. Our research team chose the eye-
tracking device to examine the visitor’s gaze movement because it keeps track of what
they are looking at and interested in [3]. By recording the vision of visitors wearing the
mobile eye-tracker enables us to monitor the path of visitors and identify how much time
they spend to see a specific object while appreciating the art museum. This enables us to
understand the museum experience of visitors by observing them deeply in seconds.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1132–1137, 2020.
https://doi.org/10.1007/978-3-030-39512-4_172
A Study on Understanding of Visitor Needs in Art Museum 1133
2 Related Works
The previous studies have asked visitors what is their needs, feelings, or satisfaction by
interviews [4] and surveys [5] in order to analyze the experience of the visitor on the
museum. Based on the literature to understand the previous research about experience
attributes and visitor satisfaction, in-depth interviews were conducted for identifying
the impacts of different exhibition designs on satisfaction with museum experience [6].
However, most of these studies mainly focused on qualitative researches that could
create subjective data and does not offer statistical representation. Furthermore, there
are studies to analyze the visitors’ behavior while art appreciating [7] had an obser-
vation study to track the visitors’ viewing behavior. These visitor behavior studies only
focus on the behavior of appreciating the artwork.
In addition, there have been precedent researches by using a portable eye tracker to
analyze visitor’s physical response during art appreciation. The eye movement
behavior on unique artwork gives information about the types of visitors identified
from the salience analysis gathered from the device [8]. Not only the painting also
installation art is assessed by the diverse, the embodied, and mobile eye tracker to
capture the reactions [9]. On the other hand, partial artworks classified into a different
type of genre in a real-world setting are used to analyze the correlation between the
factors derived from eye movement patterns [10]. Visitor experience in terms of the
context of the museum journey, also, is studied in order that curators take advantage of
the analysis from eye movement patterns [11]. From single artwork to overall appre-
ciation, eye-tracking studies focused on the responses on each artwork. The eye
movement from the entrance to exit needs to be taken into account, thereby under-
standing the whole behavior of visitors.
3 Methodology
For the purpose of this study, we set three phases: (1) pre-survey, (2) eye-tracking
experiments, and (3) in-depth interview. All process of this research was approved by
the KAIST Institutional Review Board for bioethics. First of all, the participants fill out
the Consent form about the experiment. If they agree with participating, we collect their
demographic and basic information through the pre-survey. Secondly, before starting
the experiment, the calibration process is performed to calculate their gaze position.
After calibration, they are browsing the museum from the entrance to the exit. When
the participants finish their appreciation, third, they are moving to the room for the
interview. The participants will watch the recorded eye movement video and conduct
an in-depth interview asking the reason for the attention of each element.
3.1 Experimental Environment
Seven adults from the KAIST participated in this study with a mean age of 30.14 years
(SD = 2.85 years). We selected the only person who is majored in engineering to
eliminate the differences in prior knowledge of the participants. All participants had
correct visual acuity of 1.0 or higher without wearing glasses so as not to be
uncomfortable wearing eye trackers.
1134 T. Yi et al.
Table 1. Elements of Lee Ungno museum
Space (label) Elements
Entrance (E) Exhibition title (E1), Brochure locker (E2), Wall text of ticket info (E3),
Brochures (E0)
Lobby (L) Wall image of exhibition (L1), Wall text of museum (L2), Program banner (L3),
Brochures (L0)
Ticket desk (T) Info staff (T1), Ticket info (T2), Museum regulation (T3), Exhibition poster (T4),
Brochure locker (T5), Brochure (T0)
Exhibition – Section 1 Exhibition text (FE1), ‘Juksa and Goam’ (FE2*4), ‘To the global stage’
(FE) (FE5*8), ‘In Paris’ (FE9*13), ‘From abstract letter to People series’
(FE14*17), Old newspapers (FE18), ‘Marco Polo’ (FE19*21)
Exhibition – Section 2 Section text (SE1), ‘People series’ (SE2*4), ‘Symbiosis, Coexistence’ (SE5*6),
(SE) ‘Have your own people print!’ (SE7*8), Corridor: outside (SE9), Corridor:
pictures (SE10)
Exhibition – Section 3 Section text (TE1), Paintings (TE2*3), ‘Landscape’ (TE4*5), ‘Abstract Letter’
(TE) (TE6)
Exhibition – Section 4 Section text (FOE1), ‘People series’ with 3D printing (FOE2*3), Illustration of
(FOE) Lee Ungno (FOE4*5), ‘Yami, the hungry cat’ (FOE6*7), ‘Text and audience’
(FOE8*9), Interview video of Lee Ungno (FOE10), Wall image of Lee Ungno
(FOE11), Exit (FOE12), Drone video of museum (FOE13)
Cafeteria (C) Menu (C1), Menu Explanation (C2), Café staff (C3), Display (C4)
Art shop (A) Discount Info (A1), Art Merchandises (A2*21, except A18), Portrait of Lee
Ungno (A18)
We chose Lee Ung-no Museum for this experiment. It is considered as a suitable
test site because it has all the elements of the art museum. Our research team set the
elements located in each space as shown in Table 1. For example, in Exhibition-
Section 1, 21 elements are set up. A mobile eye tracker, the Pupil Labs 120 Hz
monocular was used and the participants were asked to wear it with an eco-bag which
includes a light-weight laptop on their shoulders.
3.2 Analysis
Our research team analyzes eye-tracking data and interviews. The eye-tracking data is
analyzed by two parameters. First is the number of fixations, which elements the
viewers focused on. Second is the fixation duration that represents the relative
engagement with the elements. Longer average fixation duration means higher levels of
engagement [12]. We can use the above data to identify which elements the visitors pay
attention to in the museum or not. To the next step, through the in-depth interviews, we
extract what needs the visitors have in each element of the museum.
4 Result
The art museum is divided into two categories, exhibition, and additional spaces. First,
exhibition spaces mean the rooms for displaying artworks. Second, additional spaces
include the entrance, lobby, cafeteria, and the art shop. The elements that the visitors
showed interest by spaces are summarized in Fig. 1 and details online [13].
A Study on Understanding of Visitor Needs in Art Museum 1135
Fig. 1. Result of eye-tracking analysis (left parameter means counts and right parameter means
seconds in the graph)
1136 T. Yi et al.
Exhibition Spaces. There are four sections of the exhibition spaces. First, there are the
main artworks in each section. For example, in Section 1, a painting of People series
received the most attention from the participants, and they mentioned in the interviews
that everyone was memorable (P01, “I think this piece was very eye-catching.”).
Second, there were elements that the majority of visitors did not recognize. For
example, Two artworks of ‘To the global stage’ (FE7, 8) in Section 1 and artwork of
‘Abstract Letter’ (TE6) in Section 3 were appreciated by only two participants. This
shows that the places where the artworks are displayed are difficult for visitors to visit
naturally. (P06, “I couldn’t recognize them (FE7, 8) at all.”)
In addition, six of the seven participants commented that the description of the work
was not available in the pamphlet or wall text, making it difficult to understand the
artworks. (P02, “I was sorry for not knowing what this work meant.”) For example, the
oriental paintings (TE2) in Section 3 received the most attention, but participants
mentioned the lack of information about the work. (P03, P04, “It was good, but I
couldn’t satisfy my curiosity.”) The participants wanted information about the work
they enjoyed. (P02, “Why artist draw these works?”; P07, “I am just do googling to
find the information about them.”)
Additional Spaces. The visitors usually looked at the pamphlets at the entrance,
quickly passed through the lobby and bought tickets at the ticket desk. In addition,
most participants had a lot of time to visit the art shop after seeing the exhibition. In
case of the art shop, we can see which merchandises are focused (such as, postcards
(A2) and books (A5)) or not (such as, cables (A8) and portrait (A18)) through this
experiment. (P01, “I want to buy some souvenirs.”; P02, “Even I don’t buy something, I
have visited the art shop in the museum.”)
5 Discussion and Conclusion
This study derives the interests and needs of visitors by examining their art museum
experiences through eye-tracking experiment. We find out three findings: (1) museum
can recognize which elements are appealing or not to visitors, (2) most visitors want the
explanatory descriptions, and (3) museum can find out particular spaces where visitors
did not perceive. Another important point is that the elements the visitors focus on do not
necessarily lead to their interest. For example, in Section 1, the old newspaper (FE18)
showed a higher than average fixation, but the visitor complained that the element was
difficult to see due to small letters. In other words, by combining quantitative analysis
through eye tracker and qualitative analysis through interviews, it is possible to grasp the
elements and detailed needs that the viewers pay attention to in the art museum.
This study has limitations. First, the discussion about generalization is difficult to
carry out, because the number of experiment participants was seven. However, it shows
the possibility of carrying out the museum experience of visitors through eye-tracking.
Second, in the case of the cafeteria, little attention was given to the participants. None
of the participants took a break to drink coffee or spend time sitting in the cafeteria. It
shows that we need to consider the experimental aspects to be more natural.
A Study on Understanding of Visitor Needs in Art Museum 1137
In the future, we plan to develop a mobile application that complements infor-
mation about elements that visitors do not recognize in the exhibition (such as, exhi-
bition map) and their needs (such as, providing artwork information). We expect that
this study will be a stepping stone to an art museum where visitors can more easily
appreciate and be satisfied.
Acknowledgments. This work was supported by the National Research Foundation of Korea
(NRF) grant funded by the Korea government (MSIT) (No. NRF-2019R1A2C1007042) and also,
supported by Institute for Information & communications Technology Promotion (IITP) grant
funded by the Korea government (MSIT) (R7124-16-0004, Development of Intelligent Inter-
action Technology Based on Context Awareness and Human Intention Understanding). We
thank the Lee Ungno Museum for supporting this research to conduct visitor studies.
References
1. Smith, L.F., Smith, J.K., Tinio, P.P.: Time spent viewing art and reading labels. Psychol.
Aesthet. Creativity Arts 11(1), 77–85 (2017)
2. Yalowitz, S.S., Bronnenkant, K.: Timing and tracking: unlocking visitor behavior. Visitor
Stud. 12(1), 47–64 (2009)
3. Mokatren, M., Kuflik, T., Shimshoni, I.: Exploring the potential of a mobile eye tracker as an
intuitive indoor pointing device: a case study in cultural heritage. Future Gener. Comput.
Syst. 81, 528–541 (2018)
4. Packer, J.: Beyond learning: exploring visitors’ perceptions of the value and benefits of
museum experiences. Curator: Mus. J. 51(1), 33–54 (2008)
5. Edwards, R.W., Loomis, R.J., Fusco, M.E., McDermott, M.: Motivation and information
needs of art museum visitors: a cluster analytic study. Ilvs Rev. 1(2), 20–35 (1990)
6. Chen, C.C.: Exploring the attractive attributes of exhibition experience in museum. In: 15th
International Conference on Service Systems and Service Management (ICSSSM), pp. 1–6.
IEEE (2018)
7. Carbon, C.C.: Art perception in the museum: how we spend time and space in art
exhibitions. i-Perception. 8(1), 2041669517694184 (2017)
8. Walker, F., Bucker, B., Anderson, N.C., Schreij, D., Theeuwes, J.: Looking at paintings in
the Vincent Van Gogh Museum: eye movement patterns of children and adults. PLoS ONE
12(6), e0178912 (2017)
9. Pelowski, M., Leder, H., Mitschke, V., Specker, E., Tinio, P.P., Vaporova, E., Gerger, G.,
Bieg, T., Husslein-Arco, A.: Capturing aesthetic experiences with installation art: an
empirical assessment of emotion, evaluations, and mobile eye tracking in Olafur Eliasson’s’
Baroque, Baroque!’. Front. Psychol. 9, 1255 (2018)
10. Heidenreich, S.M., Turano, K.A.: Where does one look when viewing artwork in a museum?
Empir. Stud. Arts 29(1), 51–72 (2011)
11. Tseng, Y.C., Tang, A.H., Shih, Y.H., Liang, S.F.: Using eye movement data and visit
contexts to understand the experience of museum visitors. In: Extended Abstracts of the
2018 CHI Conference on Human Factors in Computing Systems, p. LBW598. ACM (2018)
12. Albert, W., Tullis, T.: Measuring the user experience: collecting, analyzing, and presenting
usability metrics. Morgan Kaufmann (2013)
13. Detail of eye-tracking analysis data. https://www.dropbox.com/s/erpkiykfobagm4z/Data-
analysis.xlsx?dl=0
Analysis of Art Museums’ Visitor Behavior
and Eye Movements for Mobile Guide
App Design
Mi Chang1, Taeha Yi1, Po Yan Lai1, Jun Hee Lee2,
and Ji-Hyun Lee1(&)
1
Graduate School of Culture Technology,
KAIST, Daejeon 305-701, Republic of Korea
{rosechang,yitaeha,pylai,jihyunlee}@kaist.ac.kr
2
Department of Consumer Science, Seoul National University,
Gwanak-ro, Gwanak-gu, Seoul, Republic of Korea
[email protected]
Abstract. Over the past decade, the growing prevalence of smartphone has
motivated museums to bring out the guide service in the form of a mobile
app. However, not many museums are actively embracing cutting-edge tech-
nology. Therefore, this study aims at proposing a design blueprint of a visitor-
centered art museum guide app that guarantees its usability. For integrated
approach of quantitative and qualitative analysis, we conducted the eye-tracking,
survey, and in-depth interview by selecting five art museum mobile apps. We
then analyzed which apps are the most efficient by analyzing the number of page
steps and the time through the required task of the mobile app, as well as the
problem area for each app. Through the results of this experiment, it contributes
to identifying and solving the existing problems of app-aided exhibition services
and provides insights into the digital strategies of the visitor-centered museum.
Keywords: Visitor-centered museum Mobile app design User experience
design Eye-tracking analysis
1 Introduction
The guide plays a crucial role in the museum visiting experience, especially of art. As
Samis and Michaelson [1] noted, contemporary artwork and objects remain obscure
and hard to understand for the visitor to understand. In order to enhance the art
appreciation experience, museums around the world started to offer visitors a portable
device, on which pre-recorded commentaries about the collections are stored, since the
1950s [2]. Instead of joining a group tour of a fixed schedule, the individual can be self-
guided around the gallery at their own pace and interest. What known as the audio
guide, one of the earliest attempts of the museum to provide customized service for the
mass, has hence become the dominating means to guide the visitors.
Over the past decade, the growing prevalence of smartphone has motivated
museums to bring out the guide service in the form of a mobile app. Not only because
mobile devices can support personalized multimedia contents and location-aware
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1138–1144, 2020.
https://doi.org/10.1007/978-3-030-39512-4_173
Analysis of Art Museums’ Visitor Behavior and Eye Movements 1139
navigation, but also of its ability to encourage interactions and connections among the
exhibitions, the visitors, and the organizations [3]. For example, utilizing the aug-
mented reality (AR) display functions on the mobile device enables more satisfying
navigation and recommendation services through the app [4]. Although mobile tech-
nology enables highly personalized service, not all museums take full advantage of the
media.
Surprisingly, not many museums are actively embracing cutting-edge technology.
By surveying the world’s top 100 visited art museums [5], we discovered about one-
third of them have never developed a mobile app. The rest have developed apps for
guide service, specific exhibition or other purposes, or have supported third-party apps
for guide service. However, only about half of the officially developed guide app still
exists and is in use. Since app development and maintenance cost resource and effort
[6], this might why many museums are still struggling to adopt the technology,
especially when it is not clear whether they can be benefited from the app.
Yet, according to Sayre [7], several variables would significantly contribute to the
overall success of mobile service in museums. These variables consider the hardware,
the software, and most importantly, the visitors. While the technology infrastructure
and hardware support are the fundamentals, usable system, and valuable contents are
what make the visitors feel the apps pay off for them. Therefore, this study aims at
proposing a design blueprint of a visitor-centered art museum guide app that guarantees
its usability. This paper presents a usability evaluation on five top downloaded and
reviewed museum apps, on which our blueprint will be designed based.
Museum researchers have developed a set of methods, criteria, and tools for
usability evaluation of museums’ digital services. Since the early 2000s, researchers
have been accessing usability of museums’ websites by a single or mixed approach of
case study, observation, interview, questionnaire, field or laboratory experiments, and
heuristic evaluation [8, 9]. These criteria and methods have been continually employed
to access other digital systems in the museum, including augmented reality device [10],
and native mobile apps [3]. Meanwhile, researchers from other domains have started to
use eye-tracking technology, which provides more accurate and manipulatable data for
analysis, in place of the traditional pen-and-paper observation for mobile app design
and evaluation [11, 12].
Consequently, our research takes an integrated approach of quantitative and
qualitative analysis. We tracked the eye movement of the subjects while they were
carrying out a set of usability-related tasks. The experiment was followed by a survey
and an interview to understand the user behaviors. Our study revealed the essential
elements and design requirements to build a museum guide app. Moreover, it con-
tributes to identifying and solving the existing problems of app-aided exhibition ser-
vices and provides insights into the digital strategies of the visitor-centered museum.
2 Methodology
The process of this study consists of four phases as in Fig. 1. In phase 1, we selected five
mobile guide apps from the list of top visited art museums in the world [5] based on the
number of installations and reviews to ensure the representativeness. The museums
1140 M. Chang et al.
included the Louvre Museum, Rijksmuseum, National Gallery Singapore (NGS), Los
Angeles County Museum of Art (LACMA), and National Museum Modern and Con-
temporary Art (MMCA), Korea. In phase 2, we conducted two experiments to study if
the visitors’ needs are satisfied in different use cases. Following the experiments, we
analyzed the number of page steps and the duration in phase 3. We also analyzed the
problem area based on the results from interviews and surveys. Lastly, based on the
analysis results, we will present the design direction of the art museum mobile app.
Fig. 1. Research process
2.1 Experimental Environments
Eight people participated in the study. Participants were 50% females and 50% males
and ranged in age from 26 to 33. We used the app Hawkeye – User Testing from
Hawkeye Labs on iPhone XR to experiment. The data we retrieved from the app
included the number the page steps, the duration of time spent on each page, the heat
map, and the gaze plot for problem area analysis. The KAIST Institutional Review
Board approved all process of this research for bioethics.
In the first experiment, we designed three tasks which are related to looking up
information about the museum, the exhibition and the artwork respectively. Partici-
pants were asked to execute the tasks on a prototype while their eye movements were
being tracked. The participants responded to a usability survey after completing the
tasks. The second one is the overall usability evaluation. We tracked the participants’
eye movements when they were freely using the app; then, we conducted an in-depth
interview with them regarding the app’s usability. The whole process repeated for each
app. Based on their behaviors and the interviews, we analyzed the problem areas app
by app.
Analysis of Art Museums’ Visitor Behavior and Eye Movements 1141
3 Experimental Results
Table 1 shows the ratings regarding how easy it is to find information and how satisfied
user is with the mobile app. App 1 to App 5 represents Louvre, Rijksmuseum, NGS,
LACMA, and MMCA, respectively. In overall, Louvre and LACMA scored high on
providing the exhibition information, while NGS scored the lowest (Fig. 2).
Table 1. Subjective rating of the prototype system
Statement Mean
App 1 App 2 App 3 App 4 App 5
It was easy to find time to open the 5 2.25 1.75 4.87 4.5
museum. (Task #1)
It was easy to find the museum ticket 4.62 3.25 1.5 4.37 4.37
information. (Task #2)
It was easy to find information on the 3.5 4.12 2.62 3.87 3.62
artworks (exhibition) in the museum.
(Task #3)
While using this app, I wanted to go to art 4.37 4.12 2.25 4 2.87
museum
I want to use this app in actual museum 4.25 4.25 2.12 3.87 2.87
Fig. 2. Number of page steps and duration based on eye-tracking
Figure 1 and Table 1 show the correlation between eye-tracking results and user
satisfaction survey. In the case of Louvre, the number of page steps and the time for
1142 M. Chang et al.
required tasks are generally low, and the overall score is high in user satisfaction.
Second, Rijksmuseum received high marks on the appreciation of the artwork, but
looking up basic information about the museum was uneasy for the participants. Third,
it was difficult to find information on NGS mobile app. It took a lot of time and
required numerous pages to complete all three tasks. Fourth, LACMA has a UI that
provides information efficiently, but the labeling is not properly done for artwork
appreciation. Lastly, MMCA was providing information efficiently, but the satisfaction
with the app was low.
Table 2. Problem area
No Problem area Number of participants who encountered
the problem
Mobile App 1 (Louvre)
1 Finding artwork 3
2 Navigation on the map 2
3 UI design 1
4 Hard to find the menu 1
5 Description of the artwork 1
Mobile App 2 (Rijksmuseum)
1 Proper exhibition Information 5
2 UI design 2
3 Finding artwork 1
Mobile App 3 (NGS)
1 Proper exhibition Information 2
2 Finding artwork 2
3 Uncomfortable overall 1
4 Finding out basic exhibition 1
information
5 Finding location and artworks inside the 1
museum
6 UI design 1
Mobile App 4 (LACMA)
1 Description of the artwork 2
2 Finding artwork 2
3 Navigation on the map 1
4 UI design 1
5 (Nothing found to be problematic) 1
Mobile App 5 (MMCA)
1 Description of the artwork 5
2 UI design 2
3 Finding artwork 1
Analysis of Art Museums’ Visitor Behavior and Eye Movements 1143
Additionally, Table 2 summarizes the problem areas mentioned in the user survey
for each app. A page that is considered a problem area has a larger area in the heat map
and has more records in the gaze plot than pages that are not problematic. It is aberrant
that it was easy to find information on MMCA’s mobile app, but there was a lack of
work description and dissatisfaction with UI design.
4 Conclusion
We analyzed five art museum mobile apps through the integration of qualitative and
quantitative analysis. Through three tasks and free use, we tracked the eye movements
and conducted surveys and interviews. Our study shows the problems of the existing
art museum mobile app and gives new insights into developing a mobile guide app for
the visitor-centered museum. However, we have not verified our findings in actual
design yet. To overcome this limitation, we confirmed a collaboration with Lee Ungno
Museum in Daejeon, Korea, to continue the research in an actual museum context. In
the future, we will implement and evaluation the findings of this paper by designing a
visitor-centered smartphone application for Lee Ungno Museum.
Acknowledgments. This work was supported by the National Research Foundation of Korea
(NRF) grant funded by the Korea government (NRF-2019R1A2C1007042 & NRF-2019-Global
Ph.D. Fellowship Program) and also, supported by Institute for Information & communications
Technology Promotion (IITP) grant funded by the Korea government (MSIT) (R7124-16-0004,
Development of Intelligent Interaction Technology Based on Context Awareness and Human
Intention Understanding).
References
1. Samis, P., Michaelson, M.: Creating the Visitor-Centered Museum. Routledge, London
(2016)
2. Bradburne, J.M.: Foreword. In: Tallon, L., Walker, K. (eds.) Digital Technologies and the
Museum Experience, p. 264. AltaMira Press, Plymouth (2008)
3. Economou, M., Meintani, E.: Promising beginnings? Evaluating museum mobile phone
apps. In: Rethinking Technology in Museums Conference Proceedings (2011)
4. Wang, C.S.: An AR mobile navigation system integrating indoor positioning and content
recommendation services. World Wide Web 22, 1241–1262 (2019)
5. The Art Newspaper: Special Report on Art’s Most Popular Exhibition and museum visitor
figures 2018 (2019)
6. Forbes, T.: Native or Not? Why a Mobile Web App Might Be Right for Your Museum.
https://mobileappsformuseums.wordpress.com/2011/08/05/native-or-not-why-a-mobile-
web-app-might-be-right-for-your-museum/
7. Sayre, S.: Bring it on: ensuring the success of BYOD programming in the museum
environment. In: MW2015: Museums and the Web 2015 (2015)
8. Cunliffe, D., Kritou, E., Tudhope, D.: Usability evaluation for museum web sites. Mus.
Manag. Curatorsh. 19, 229–252 (2001)
9. Kabassi, K.: Evaluating websites of museums: state of the art. J. Cult. Herit. 24, 184–196
(2017)
1144 M. Chang et al.
10. Othman, M.K., Petrie, H., Power, C.: Measuring the usability of a smartphone delivered
museum guide. Procedia - Soc. Behav. Sci. 97, 629–637 (2013)
11. Tupikovskaja-Omovie, Z., Tyler, D.J., Dhanapala, S., Hayes, S.: Mobile app versus website:
a comparative eye-tracking case study. Int. J. Soc. Behav. Educ. Econ. Bus. Ind. Eng. 9,
3251–3258 (2015)
12. Cho, H., Powell, D., Pichon, A., Kuhns, L.M., Garofalo, R., Schnall, R.: Eye-tracking
retrospective think-aloud as a novel approach for a usability evaluation. Int. J. Med. Inform.
129, 366–373 (2019)
Al-Maqta Canal of Abu Dhabi, UAE: A Study
of Waterfront Landscapes and Flow
in Manmade Canals
Mohamed El Amrousi and Mohamed Elhakeem(&)
Abu Dhabi University, Abu Dhabi, United Arab Emirates
[email protected]
Abstract. Urban renewal is important to regenerate the living city in Abu Dhabi
were recently public space along waterfronts is emerging along with cafes,
restaurants and hotels to represent as new kind of mitigation between public and
semi-public spaces. Waterfront urban assemblages in Abu Dhabi offer a rela-
tively sanitized experience than those associated with conventional public spaces
in historic cities. For example Al-Bateen marina and the Al-Maqta Canal are new
waterfront developments shaped by concrete breakwaters that re-define the
connection of public space to waterfronts. While pedestrian access and walka-
bility in Abu Dhabi are still in their infantile stages, new projects such as the new
Al-Maqta Canal Project and Al-Bateen Marina promises to create new venues for
pedestrian and outdoor activities. Although these spaces replace earlier water-
front spaces such as old wharfs and fishing hubs had a more traditional attire,
these new open spaces display an urban strategy of creating public spaces along
waterfronts in oil rich Gulf States. When taken together such waterfront devel-
opments address the complex urban whole that reflects the multi-ethnic nature of
Gulf state cities. This paper studies the strategies and effects of architectural
assemblages along waterfronts in Abu Dhabi and how they shape public space
and urban landscape. In addition, we use a 2D-hydrodynamic water surface
model to simulate the flow pattern in the new Al-Maqta Canal.
Keywords: Urban landscapes Shorefront developments Public space
Abu Dhabi 2D-hydrodynamic water surface model
1 Introduction
Much of the urban fabric of Abu Dhabi is changing; especially during this past decade,
Abu Dhabi has begun to invest heavily in mega projects as part of its re-branding from
a transit Gulf State city to one aiming to develop a sustainable community and spaces
of belonging for many of its predominately expatriate community. Urban renewal is
important to regenerate the living city especially that the existing stock of open spaces
has not kept up with urban expansion and population growth. The Abu Dhabi 2030
plan outlined a series of principles emphasizing a focus on cultural identity and sus-
tainability. However, the challenge for policy makers, planners and architects is to
retain an urban identity while creating mega projects that adhere to forms from the
wider spectrum of architecture in the 21st Century while avoiding mimicry and the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1145–1151, 2020.
https://doi.org/10.1007/978-3-030-39512-4_174
1146 M. E. Amrousi and M. Elhakeem
creation of an urban fabric characterized by unfamiliarity [1]. Abu Dhabi’s shorefront
developments in the new Al-Bateen Wharf and the Bain Al-Jessrain area can be
classified as developing urban enclaves along waterfronts that proactively incorporate
natural scenery, which requires pedestrian path-making, pavements and cycling tracks,
planting and digging of artificial canals. Creating natural environments are important
for metropolitan areas to support ecosystems, public health, social interactions, aes-
thetic needs, and to mitigate urban heat-island effects. Abu Dhabi has a promising
program of preserving and planting mangrove trees that are bushes and trees that grow
at the intersection of land and sea, at the mouth and surrounding areas of tidal lagoons,
estuaries, and rivers, and they are partially emerged under the high-water tide. Land-
scapes, including water canals, contribute to a sense of place, history, identity, and
community. They have also been found to stimulate local economies, contributing
significantly to urban and economic revitalization [2]. Abu Dhabi’s has little heritage
remaining intact with the exception of Qasr Al-Hosn in the city center, other heritage
are at distance in Dalma Island in the form of buildings from coral stone, and have flat
roofs, supported by mangrove poles and palm matting interwoven from reed mesh, and
arched stucco carved screens [3]. The historical vacuum allowed designers and urban
planners to construct an urban fabric characterized as hybrid or eclectic. As outlined by
Kingston Health, the notion of authenticity within the context of the living city war-
rants revisiting the past within a framework of identity and multiple sources of local
and global influences [4]. This paper studies the strategies and effects of architectural
assemblages along waterfronts in Abu Dhabi and how they shape public space and
urban landscape. In addition, we use a 2D-hydrodynamic water surface model to
simulate the flow pattern in the new Al-Maqta Canal in relation to the main Canal in
Abu Dhabi.
Fig. 1. The new Al-Bateen Wharf mixed use waterfront development.
2 Public Space and Waterfront Development–Al Bateen
Wharf
Orientation toward waterfronts brings is important and cultivates respect for local
heritage however there is a gradual replacement of older activities with new more
touristic ones. Al-Bateen Wharf is one of the older sections of Abu Dhabi that has
Al-Maqta Canal of Abu Dhabi, UAE: A Study of Waterfront Landscapes and Flow 1147
currently been redesigned, leaving the old shipyard in the background overshadowed
by the new Edition Hotel, cafes and waterfront promenades. The Al-Bateen Wharf
redesign houses at its heart the Edition hotel that creates a series of walkways along its
circumambulating its waterfronts. The Abu Dhabi hotel has been designed by Lebanese
architecture and engineering firm Khatib & Alami it forms part of Abu Dhabi’s Marina
mixed-use development (Fig. 1). Emerging waterfront assemblages offer a relatively
sanitized experience than those associated with conventional public spaces. However
they represent a new space for interaction in Gulf State cities that were designed to
cater to automobile urbanism. These new spaces offer jogging trails, tennis courts, and
nature reserves more-moderate-income communities struggle to keep up with the high
end leisure clusters offered by the new developments in the city [5].
Fig. 2. Pedestrian areas and cycling paths emerging along Abu Dhabi’s shorefronts.
Urban renewal is important to regenerate the living city, the Corniche developments
of public spaces represents one of the few areas were pedestrian access and walkability
can be witnessed amongst automobile oriented urbanism. Walkability is an important
component of any living city especially that the stock of open spaces has not kept up
with population growth and urban expansion. Much of the water fronts developments
are engineered and offer a sanitized experience however they still encourage the belief
that the city has the ability to change its urban fabric focus on more inclusive spaces.
There are different methods for beach nourishment and improvement of walkability one
of the most important is the right choice of materials. An important principle is to use a
different material for the construction of the roads associated with waterfronts, a
material other than asphalt (Fig. 2). The quality of the path itself is, of course, essential
to walkability. If we wish to encourage walking we need to deal with more than
connectivity, land use patterns, safety, and quality of the path itself. A safe, continuous
path network in a monotonous physical setting will not invite pedestrians [6]. Fur-
thermore, it is essential to replace asphalt areas with sidewalks pavement materials that
allow the development of a variety of other uses such as the bicycle, pedestrians,
playgrounds and staging areas. In this context, the interface between city and water,
1148 M. E. Amrousi and M. Elhakeem
waterfronts are one of the most complex and challenging landscapes in the city. For
contemporary waterfronts, it is crucial to understand the structure of urban waterfronts
and their role for integration of different social groups within the existing city structure,
because they foster social interactions and contribute to a sense of place and communal
identity.
Fig. 3. The new Al-Maqta Canal, Abu Dhabi.
Fig. 4. The Ritz Carlton hotel, Abu Dhabi.
3 Bain Al-Jessrain and the New Al-Maqta Canal
The new Al-Maqta Canal mixed use project will create new landscapes and public
spaces along a 2.4 km waterfront, that is located between Al-Maqta and Mussafah
bridges (Fig. 3). The mixed use project highlights its public facilities such as a new
Aquarium, cultural center, cinema, family entertainment center, and areas dedicated for
indoor and outdoor sports in addition to running and cycling tracks. The new project
fronts the Sheikh Zayed Grand Mosque and the Ritz Carlton Hotel, which are on the
other side of the main canal, and is adjacent to the Shangri-La Hotel and Souk Qaryat
Al-Beri. The Al-Maqta waterfront development manifests an ultra-modern style that
Al-Maqta Canal of Abu Dhabi, UAE: A Study of Waterfront Landscapes and Flow 1149
stands in contrast to the post-modern Arabian/Islamic themed buildings especially the
Grand Mosque, and Ritz Carlton (Fig. 4). When juxtaposed the otherness of these
mega projects reflects the diversity of forms and architectural styles emerging in Gulf
cities in the 21st century. From Postmodern—Arabian themed Ritz Carlton, Souk
Qaryat Al-Beri, Shangri-La Hotel to the ultra-modern deconstructivism styled Edition
Hotel these lavish assemblies of transnational architectures proclaim a radically new
and different image of the city that caters for expat/transnational communities. The
displacement myriad traditions through eclectic designs creates architectural collages
and contrasting styles that are interwoven by waterfronts to represent narratives of
hybrid traditions that effectively create a third space beyond heritage and continuity.
They can also be understood as part of a wider matrix of simulacra’s that replicate
features of building traditions and contemporary architecture [7].
4 Model Description
Due to the complex geometry of Al-Maqta Canal junction with the main canal, a 2D
hydrodynamic model was used to investigate the flow pattern in the canal. In this study,
we chose the 2D Finite Element Surface Water Modelling System (FESWMS), which
is part of the commercially available Surface water Modelling System (SMS) software
package (version 12.1) developed by the Federal Highway Administration [8].
FESWMS solves the differential forms of the continuity and the momentum equations
in the stream wise and transverse directions using the Galerkin method of weighted
residuals providing water depth and depth-averaged velocity magnitude in x and y di-
rections at each node in the grid.
The model inputs are: the canal bathymetry, Manning’s coefficient n = 0.025, eddy
viscosity m = 0.3 m2/s, flow rate entering the main canal Q = 1000 m3/s, and flow
depth exiting the main canal y = 8 m at the junction with Al-Maqta Canal. Figure 5
shows the simulations of the velocity distribution of the Al-Maqta Canal and Fig. 5
shows a close-up of the velocity vectors at the junction of Al-Maqta Canal with the
main canal. The water velocity was in the range of 0.03 to 0.6 m/s. It can be seen from
the figure that a complex circulation flow pattern has been developed at the junction.
This circulation pattern is important because it helps in controlling sediment deposition,
aquatic growth, and sulphide formation. Nonetheless, it can be seen from Fig. 5 that
there is no circulated flow in the inner part of Al-Maqta canal and the water is almost
stagnant. This can increase salinity, sediment, organic and inorganic materials in the
canal, which may affect water quality and marine life. Therefore, an artificial flow
circulation mechanism may be needed in the canal to improve water quality.
1150 M. E. Amrousi and M. Elhakeem
Fig. 5. Simulation of the velocity and the circulation flow pattern in Al-Maqta Canal.
5 Conclusion
In post-industrial cities such as Abu Dhabi, the position of water and waterfronts in the
city context has radically changed. The new waterfront developments in the form of
public gardens redefine public space with an identity that is more inclusive and can be
enjoyed by multi-ethnic communities. In response to urban regeneration pressures in
the living city, as well as a quest for shared public space, the success of the Al- Maqta
Canal as a public space lies in its accessibility. This suggests the outlines of an
alternative model for supporting commercial activity that is oriented towards canals and
brings communities closer together. Abu Dhabi’s 2030 Master Plan’ highlights the
importance of sustainability in its broader sense; Environmental, Cultural, Economic…
etc., which includes sustaining healthy communities through waterfront landscapes and
public spaces that include various sport, and cultural activities. Such emerging land-
scapes and public spaces in Arab cities are necessary and when mitigated they represent
an intermediate intervention of government and the private sector to improve the
walkability of the city. Finally, from hydraulic viewpoint, an artificial flow circulation
mechanism may be needed in the canal to improve water quality because water is
almost stagnant in Al-Maqta Canal.
References
1. Frampton, K.: The catalytic city: between strategy and intervention. In: Identification and
Documentation of Modern Heritage. van Oers, R., Haraguchi, S. (eds.) UNESCO World
Heritage Centre, pp. 72–78 (2003)
2. Ellin, N.: Canalscape: practicing integral urbanism in metropolitan phoenix. J. Urban Des. 15
(4), 599–610 (2010)
Al-Maqta Canal of Abu Dhabi, UAE: A Study of Waterfront Landscapes and Flow 1151
3. King, G.: The islamic architecture of Dalma Island. In: Damluji, S. (ed.) The Architecture of
the UAE, pp. 151–178. Garnet Publishing, Reading (2006)
4. Health, K.: Vernacular Architecture and Regional Design: Cultural Process and Environ-
mental Response, Elsevier-Oxford (2009)
5. Banerjee, T.: Public space, beyond invented streets and reinvented places. APA J. Winter 67
(1), 9–24 (2001)
6. Speck, J.: Walkable city: how downtown can save America, one step at a time (2013)
7. Olroyd-Robinson, K.: The urban architecture of Al-Bastakiyyah. In: Damluji, S. (ed.) The
Architecture of the UAE, pp. 179–204, Garnet Publishing, Reading (2006)
8. Froehlich, D.: User’s manual for FESWMS Flo2DH: two-dimensional depth-averaged flow
and sediment transport model. Release 3 (2002)
A Discussion of User Experience
on a Panoramic Scooter Riding Video Service
Fei-Hui Huang(&)
Department of Marketing and Distribution Management, Oriental Institute
of Technology, Banciao. 58, New Taipei City 22061, Taiwan R.O.C.
[email protected]
Abstract. This paper compares the user experiences (UXs) while riding a
scooter on the road to watching a panoramic scooter riding video. The aim of
this study is to determine whether using a panoramic scooter riding video ser-
vice produces positive UXs for the riding experience as riding on a real scooter.
Data were collected from an experiment involving a total of 34 individual
scooter commuters. The participants were asked to watch a panoramic scooter
riding video and to complete a user experience questionnaire (UEQ). The results
verified that using the panoramic scooter riding video service may not be able to
create scooter riding experiences for users. In order to enable widespread use of
panoramic scooter riding video service in real life applications, a number of
suggestions are proposed for the design of related services to strengthen the
advantages of panoramic scooter riding video in simulated two-wheeler ride
experiences.
Keywords: User experience Technology application Panoramic video
Two-wheelers
1 Introduction
For improving air quality and public health, the Taiwanese government proposes an
eco-friendly environmental protection policy to replace scooters with electric two-
wheelers (E2Ws) in the Taiwanese two-wheeler market. Providing E2W trial ride
services has a positive impact on the user’s acceptance level of the E2W product [1–3].
Direct product or service experiences may create stronger associations in memory,
given its inherent self-relevance [4]. Experiential benefits generally correspond to
product-related attributes, and the experience of using the product or service, and
satisfying needs such as sensory pleasure, variety, and cognitive stimulation [5].
Because Taiwan has the highest density of scooter users, this study set up a scooter
riding service. To consider universal use, this study focuses on scooter riding video
service. Three major types of video content are realistic images or videos, including
three-dimensional (3D) digital, normal video, and 360° panoramic format representa-
tions. The 3D format is created digitally through computer vision software, the navi-
gation is continuous, and it must be connected to a computer. “The 3D scene
specification is divided into three parts: geometrical, behavioral, and aural” [6].
Compared to 3D, videos provide highly realistic imagery and are comparatively easy to
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1152–1158, 2020.
https://doi.org/10.1007/978-3-030-39512-4_175
A Discussion of User Experience on a Panoramic Scooter Riding Video Service 1153
produce. Due to the advancement of low-cost camera equipment, videos are convenient
to watch and share. The 360° panoramic format is based on videos of real situations
and content is limited to a 360° view of each photogram. Panoramic format is gaining
popularity due to smartphones are widely accepted and used around the world. Also,
the panoramic format is cheaper than 3D format [7]. In other words, panoramic videos
are a new and rapidly growing approach. The panoramic videos can display the power
of sight, sound, and motion in an entirely new way. Also, the panoramic videos allows
viewers to sense action from all angles and directions.
In this experimental study, the research goal is to analyze the effectiveness of
applying 360° panoramic formats in eliciting positive user experiences (UXs) and to
compare these to UXs evoked in physical scooter ride settings. We adopted the user
experience questionnaire (UEQ) [8], which is based on attractiveness, pragmatic
quality, and hedonic quality. We investigated paths through which panoramic video
technologies impacted users’ UXs and usage intentions. The panoramic video service
presented actual situations of riding scooters on the road, which allowed us to make
valid comparisons. The responses from participants were collected to understand user’s
UXs both in using panoramic scooter riding video service and riding a scooter on the
road.
2 Methods
Thirty-four individual scooter commuters took part in the experiment and completed
surveys. The median age of participants was 22.5 years (min: 21, max: 46), with 38.2%
females (n = 12) and 61.8% males (n = 22). Their median riding experience was 3.8
years. For each participant, two paper-and-pencil questionnaires for user experience
evaluations were used. One pre-experimental subjective rating was designed to explore
participants’ riding experiences. It contained three sections—(1) personal information:
three items designed to collect socio-demographic data on gender, age, and riding
experiences; (2) UX: twenty-six items designed to measure attractiveness, pragmatic
quality, and hedonic quality. The items were adopted from the UEQ to measure the
subjective response to scooter usage, this was assessed by using a 7-point semantic
differential scale; (3) satisfaction: one item was designed to collect quantitative data on
what users perceived had self-satisfaction, in riding a scooter, this was assessed by
using a 7-point Likert scale ranging from extremely agree to extremely disagree. One
post-experimental subjective rating was designed to explore participants’ viewing
experience; it contained two sections—(1) UX: twenty-six items were adopted from the
UEQ to collect quantitative data; (2) satisfaction: two items were designed to collect
quantitative data on whether the user perceived that watching the panoramic scooter
riding video achieved the purpose of having a realistic riding experience and had an
enhanced willingness to continue watching panoramic videos, this was assessed by
using a 7-point Likert scale ranging from extremely agree to extremely disagree.
1154 F.-H. Huang
3 Results
3.1 Descriptive Statistics
The items of the UEQ are scaled from −3 (most negative answer) to +3 (most positive
answer). The UEQ scales show the mean of the attractiveness, perspicuity, and effi-
ciency factor for riding a scooter are in the range of +0.8 to +1.2. A value near +1
represents a positive response and is a nice impression. The scales’ mean of the
efficiency factor for watching scooter riding video, and the dependability, stimulation,
and novelty factors for riding a scooter are in the range of +0.6 to +0.1. These values
represent a slightly positive impression. Finally, the scales’ mean of the attractiveness,
perspicuity, dependability, stimulation, and novelty factors for watching scooter riding
video are in range of −0.2 to −0.7. These values represent a slightly negative
impression. The error bars represent the 5% confidence intervals for the scale means,
i.e. the probability that the true value of the scale mean lies outside this interval is less
than 5%. With regard to satisfaction, approximately 17 participants (50%) satisfied with
the experiences of riding a scooter (X = 4.77, r = 1.21) and approximately 1 partic-
ipant (2.91%) satisfied with the experiences of watching the panoramic scooter riding
video (X = 2.29, r = 1.22). With regard to usage intention, approximately 26 partic-
ipants (76.5%) intended to continue riding the scooter (X = 5.62, r = 1.44) and
approximately 9 participants (26.5%) were willing to watching a panoramic scooter
riding video for scooter ride experiences (X = 3.50, r = 1.56).
3.2 Factor Analysis and Reliability Analysis
Please refer to Table 1 for factor loadings and scale reliabilities. The internal consis-
tency of the attractiveness, efficiency, perspicuity, and stimulation scores were high.
However, the dependability and novelty scores had low internal consistency (Cron-
bach’s / on the pooled values of riding a scooter: efficiency, / ¼ :46; dependability,
/ ¼ :67; novelty; / ¼ :09. Cronbach’s / on the pooled values of watching a
panoramic scooter riding video: Perspicuity, / ¼ :52). Therefore, these items were not
included in the subsequent analyses. The sampling for riding a scooter (Kaiser-Meyer-
Olkin (KMO) value ¼ :611, Bartlett’s test Chi-sq ¼ 261:258, p ¼ :000) and using VR
(KMO value ¼ :781, Bartlett’s test Chi-sq ¼ 378:311, p ¼ :000) is adequate.
To ensure that the four UEQ dimensions were distinct, a factor analysis using
principal component extraction and oblimin rotation was conducted. For riding a
scooter, a Cattell’s screen plot showed four clear factors emerging, explaining 36.9%,
17.8%, and 11.8% of the variance, respectively; for watching a panoramic scooter
riding video, the factors explained 48.1%, 18.7%, and 8.7% of the variance, respec-
tively. In addition, the factor loading should be >0.6 [9, 10]. See Table 1, it is clear that
items A2, A6, D1, and S1 did not load appropriately onto the attractiveness,
dependability, and stimulation scales. These items were therefore not included in the
subsequent analyses.
A Discussion of User Experience on a Panoramic Scooter Riding Video Service 1155
Table 1. Factor loadings and reliabilities for UEQ measures.
Construct Item Factor loading Cronbach’s /
Riding Viewing Riding Viewing
Attractiveness A1: annoying/enjoyable 0.791 0.801 0.80 0.92
A2: good/bad 0.479 0.778
A3: unlikable/pleasing 0.643 0.819
A4: unpleasant/pleasant 0.658 0.840
A5: attractive/unattractive 0.766 0.824
A6: friendly/unfriendly 0.839 0.499
Dependability D1: unpredictable/predictable 0.950 0.562 0.67 0.78
D2: obstructive/supportive 0.673 0.723
D3: secure/not secure 0.819 0.870
D4: meets expectations/dose 0.643 0.765
not meet expectations
Stimulation S1: valuable/inferior 0.376 0.780 0.75 0.86
S2: boring/exiting 0.805 0.858
S3: not interesting/interesting 0.868 0.831
S4: motivating/demotivating 0.662 0.626
3.3 Wilcoxon Signed-Rank Test
The Wilcoxon signed-rank test revealed that there were statistically significant differ-
ence for perceived attractiveness, dependability, stimulation, satisfaction, and usage
intention (P ≦ 0.05). These results indicate that:
• Users are perceived more attractiveness when riding a scooter (Mdn = 4.8) than
when watching the video (Mdn = 3.6) (Z = −3.9, p = 0.00)
• Users are perceived more dependability when riding a scooter (Mdn = 4.7) than
when watching the video (Mdn = 3.7) (Z = −2.15, p = 0.031)
• Users are perceived more stimulation when riding a scooter (Mdn = 4.7) than when
watching the video (Mdn = 3.3) (Z = −3.84, p = 0.00)
• Users are perceived more satisfaction when riding a scooter (Mdn = 4.5) than when
watching the video (Mdn = 2.0) (Z = −4.67, p = 0.00)
• Users are perceived more intention to willingness riding a scooter (Mdn = 6.0) than
watching the video (Mdn = 4.0) (Z = −4.63, p = 0.00)
3.4 Multiple Regression Analysis
The results of a multiple regression analysis indicated that the variables precisely
predicted whether the users intended to use a panoramic scooter riding service
(F(2,33) = 109.83, p < 0.000, Adjusted R2 = 0.87). These variables included attrac-
tiveness (b = 1.58, t = 8.97, p = .000) and stimulation (b = −0.75, t = −3.36,
p = .002). All the variables significantly contributed to the prediction, p < .05.
1156 F.-H. Huang
4 Discussion
In this study, a quantitative analysis of the UXs while riding a scooter and UXs while
watching the panoramic scooter riding video was carried out using an UEQ scale. The
results revealed that riding a scooter created a positive impression concerning per-
spicuity and efficiency and a slightly positive impression concerning attractiveness,
dependability, stimulation, and novelty. It means that riding a scooter may allow most
of the riders to perceive positive UXs, with involved positive attractiveness, pragmatic,
and hedonic qualities. On the other hand, the results revealed that watching the
panoramic scooter riding video created a slightly positive impression concerning effi-
ciency, and a slightly negative impression concerning attractiveness, perspicuity,
dependability, stimulation, and novelty. It means that watching the panoramic scooter
riding video may allow most of the users to perceive slightly negative UXs.
In this study, the perspicuity, efficiency, and novelty scales of the UEQ were
excluded because these scales had a low internal consistency. Furthermore, A2, A6,
D1, and S1 were excluded because these items did not load appropriately onto the
attractiveness, dependability, and stimulation scales. The results of the analysis with
each factor of the UEQ revealed that there were significant differences in attractiveness,
dependability, and stimulation between the riding of a scooter and watching a
panoramic scooter riding video. Compared to watching the panoramic scooter riding
video, participants tended to have higher agreement levels related to attractiveness,
dependability, and stimulation for the riding of a scooter. The results verified that most
of the participants may experience a significantly higher attractiveness, dependability,
and stimulation feeling in riding a scooter, than in watching the panoramic scooter
riding video. In addition, the results of comparing the satisfaction found that there was
significant difference between riding a scooter and watching the panoramic scooter
riding video. These results verified that most of the participants had a high agreement
level of perceived satisfaction on riding a scooter. Moreover, there were no individual
difference in these factors. Although the results showed that the service cannot create
UXs related to riding a scooter, the panoramic scooter riding video service is somewhat
creating a slightly positive impression in terms of efficiency. Considering the cost and
easy to use, the factor of efficiency can be strengthened for designing a useful
panoramic two-wheeler riding video service for users to have positive user experiences.
Here, efficiency is the degree to which using a panoramic scooter riding video service is
possible to use the scooter fast and efficient. This indicated that the user interface of the
service should be designed making the users perceived the interface looks organized
and then increasing the interaction between user and system. Finally, it may allow the
users using the scooter fast and efficient after using the panoramic scooter riding video
service. Other factors, including attractiveness, perspicuity, dependability, stimulation,
and novelty should be improved in the video contents design.
The influence of immersive video services on viewers’ experiences and usage
intentions in using a panoramic scooter riding service was investigated. The results
revealed that the experiences may lead 26.5% of the participants, to have the intention
to be frequent users in riding a scooter. Significant support was found for the model and
for the goodness of fit. Results indicate that the model was successful in predicting
A Discussion of User Experience on a Panoramic Scooter Riding Video Service 1157
continuance usage intentions towards the panoramic scooter riding service, accounting
for 81% of the variance. Two of the model’s predicted relationships were attractiveness
and stimulation all contributing uniquely to users’ continuance usage intentions
towards the panoramic scooter riding service. These two scales can be defined as:
(1) attractive is the degree to which the general impression towards the panoramic
scooter riding service; and (2) stimulation is the degree of interesting and exciting to
use the panoramic scooter riding service. Of these factors, attractive positively influ-
ences the user’s usage intentions, while stimulation negatively influences them. To
increase the panoramic scooter riding video system creating positive UXs, it is rec-
ommended that designers consider enhancing user attraction. For example, selecting a
special riding scenery and in special time point is suggested.
5 Conclusion
The main aim of this study was to use the items and scales of the UEQ to investigate
the factors of attractiveness, pragmatic quality, and hedonic quality that influences a
users’ acceptance of a panoramic scooter riding video service and a users’ willingness
to use the panoramic video service to experience the riding of a scooter. Results of the
experiments and surveys provide evidence of the UEQ for increasing the understanding
of the public acceptance of panoramic scooter riding video services. Furthermore, the
study successfully compared UXs of riding a scooter to a panoramic scooter riding
video service for experiencing scooter riding. The results verified that using a
panoramic scooter riding video service may not create related scooter riding experience
for users. Providing people with an experience service in watching panoramic scooter
riding videos can yield slightly positive influences on users’ continuance usage
intentions. In order to enable widespread use of panoramic scooter riding video service
in real life applications, it is desirable to provide the user with a sense of natural
interaction and a special riding scenery and contents on the video.
References
1. Huang, F.H.: Exploring the environmental benefits associated with battery swapping system
processes. Adv. Environ. Biol. 9(26), 87–93 (2015). SI
2. Huang, F.H.: Measuring user experience of using battery swapping station. In: International
Conference on Applied Human Factors and Ergonomics, pp. 656—664. Springer, Cham
(2017)
3. Huang, F.H.: System acceptability evaluation of battery swapping system for electric two
wheelers. In: Advances in Ergonomics Modeling, Usability & Special Populations, pp. 325–
337. Springer, Cham (2017)
4. Millar, M., Thomas, R.: Discretionary activity and happiness: the role of materialism. J. Res.
Pers. 43(4), 699–702 (2009)
5. Nicolao, L., Irwin, J.R., Goodman, J.K.: Happiness for sale: do experiential purchases make
consumers happier than material purchases? J. Consum. Res. 36(2), 188–198 (2009)
6. Steuer, J.S.: Defining virtual reality: dimensions determining telepresence. J. Commun. 42
(4), 73–93 (1992)
1158 F.-H. Huang
7. Walczak, K.: Beh-VR: modeling behavior of dynamic virtual reality contents. In:
International Conference on Virtual Systems and Multimedia, pp. 40—51. Springer,
Heidelberg, October 2006
8. Rauschenberger, M., Schrepp, M., Cota, M.P., Olschner, S., Thomaschewski, J.: Efficient
measurement of the user experience of interactive products. How to use the user experience
questionnaire (UEQ). example: Spanish language version. Int. J. Artif. Intell. Interact.
Multimedia 2(1), 39–45 (2013)
9. Hair, J.F., Ringle, C.M., Sarstedt, M.: PLS-SEM: indeed a silver bullet. J. Mark. Theory
Pract. 19(2), 139–150 (2011)
10. Latan, H., Ghozali, I.: Partial Least Squares: Concept. Technique and Application SmartPLS,
BP UNDIP (2012)
Application Trend of Interactive Multimedia
in Art Museums
Yongbin Wang1 and Jian Yu2(&)
1
Guangdong University of Finance and Economics, Guangzhou, China
2
Guangzhou Academy of Fine Arts, Guangzhou, China
[email protected]
Abstract. This research focuses on interactive multimedia design that promotes
the cognition of works in art museums. As an important contributor to social
aesthetic education, art museums should employ modern science and technology
to vividly display and profoundly interpret the connotations of art. Three
methods were used in this research as follows: first, the relations of humans with
art museums and digital media art were summarized; second, the behavioral
habits of target groups in specific art museums were observed; third, the
importance of multimedia interactive technology in the information dissemi-
nation of art museum works was summarized. This research evaluated the
practical significance of various theoretical results from three perspectives and
explored the penetration of these theories in the development of interactive
application design in art museums.
Keywords: Interactive multimedia Digital media technology Information
dissemination
1 Introduction
With the upgrading of digital media technology, new ways of information dissemi-
nation have been shown, such as interactive multimedia that has become a positive
trend in media development. Digital media show such general features as technological
innovation, rich perceptual experience, free choice of time and space, and exchange of
subject and object of information dissemination. However, interactive multimedia is
not the simple superposition of media forms or the evolution of technology but a
variety of attempts and practices taken by art museums that play their advantages under
the influence of digital media technology and market to maximize the effect of infor-
mation dissemination [1, 2].
Art museums tend to show their value when they help art workers transmit their
information, knowledge, etc. to the audience more quickly and completely and achieve
their purposes of social dissemination. In general, art information that can provide
channels for emotional catharsis, resonate with thoughts or solve daily problems is
more acceptable to the audience. Interactive multimedia can improve the quality of
information dissemination through multiple digital media devices and thus improve
information efficiency so that information can be kept for a longer time in the minds of
the audience through abstract formal language and spiritual perception. Interactive
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1159–1163, 2020.
https://doi.org/10.1007/978-3-030-39512-4_176
1160 Y. Wang and J. Yu
multimedia presents complex theoretical knowledge in the form of simple device
interaction and stimulates the interest of audience, whose attention is the focus that art
museums should be concerned with in information dissemination. The application of
digital media technology embodies the different expressions presented by interactive
multimedia, extends the size of space through visual illusion and even creates multi-
dimensional space so that the artworks in the space become vivid with new expres-
sions. For example, at the exhibition Legends of the Giant Dinosaurs in Hong Kong,
the multimedia image area used 180-degree spectra vision and curved screen to display
the living environment of the dinosaur era (see Fig. 1). This is essentially a phantom
image generated by reflection and transmission technology, presentation models, and
media of sound, light, color and image under the principles of optics, and its form is an
imaging process in which the video projection image is suspended in mid-air in a real
scene. Another example is real-time video synthesis technology. Multiple cameras need
to be placed in the display space to collect the activities of audience to the computer in
an all-round way. The computer separates the images of audience from the background
and then synthesizes the audience and the real-time video images to achieve immersive
display effects. As a result, the audience can have an immersive experience at the
battlefield, through the mysterious and silent primitive forest, or in the vast outer space,
with completely subverted concept of time and space [3, 4].
Fig. 1. The Legend of Hong Kong Giant Dinosaur exhibits interactive media design
2 Method
2.1 Interactive Multimedia Improves the Information Dissemination
Efficiency of Artworks: From Static Monotonousness to Dynamic
Interaction
With the development of modern science and technology, art museums have changed
their information dissemination methods, with increasing dissemination speed, preci-
sion and means. After all, the audience is an uncertain set of individuals, who have
increasingly higher requirements for the contents and methods of information dis-
semination by art museums. How to make the audience deeply interested? According to
Application Trend of Interactive Multimedia in Art Museums 1161
the interviews and investigations on thousands of women by the Applied Social
Research Lab of Columbia University, there are three major types of contents that can
attract their attention, namely emotional catharsis and ideological resonance, escaping
from reality and killing time, explaining life and understanding ways to solve everyday
problems. Despite the audience (women) of this report, if it is widely applied, there is a
question whether the goal of all the audience is to satisfy themselves. Art museums
certainly cannot estimate and judge the needs of each individual, but they can focus on
every individual through the integration of interactive media and use various care to
replace the numbness and visual aesthetic fatigue brought about by monotonousness.
2.2 Interactive Multimedia Shifts Information Dissemination of Artworks
from Extensive Repeated Infusion to Precise Customization
To interpret interactive media, the term “interactive” seems to mean that the two parties
in the dissemination relationship are no longer one-on-one and the dissemination
process is not linear development section by section but a structure of winding, jux-
taposition, entanglement and even network. Linear metaphors are replaced by con-
nected resonances and explicit functions and structures are weakened, leading to the
loss of personalization. According to this research, the integration of interactive media
tends to precise customization in art museums, which will break the inherent thinking
of monotonous traditional media and allow multiple media to share the entire process
of information dissemination. Customization is reflected in the optimal expression of
information on art creation, which considers the preferences and needs of the audience
and creates a combination mode with multiple forms, platforms and aspects. In this
way, not only can the audience share the sources and basic contents of art information,
but they can choose the angle, method and time to accept the information according to
the characteristics of the information or their own preferences to achieve the optimal
use of information resources and the high-quality access to the mass media. For
example, the apriority brought about by traditional media to people can be used to
achieve new visual effects for the carrier of video images. For example, in the exhi-
bition hall of Mystic Manor at Hong Kong Disney, designers use traditional oil
painting and porcelain as the carriers of dynamic images so that the audience can see
moving oil painting or story-telling porcelain [5, 6].
2.3 Interactive Multimedia Shifts Information Dissemination of Artworks
from Acquisition of Information to Experience of Feelings
Interactive multimedia does not call for the replacement of traditional monotonous
media with new integrated media. Traditional media has its own advantages and has
been used for a long time so that the audience has an emotional dependence on it and
many people are willing to pay for their own feelings. The traditional media of books,
for example, can be completely replaced by e-books in terms of the practicality of the
media (except handmade books), but book centers and exhibits are still overcrowded by
the audience regardless of age, job and sex. Different groups of audience said in the
interview, “It is a kind of feeling to turn pages”, “It is feelingless to click on e-books”,
“Although it sounds a little sensational, I really like the sound of paper rubbing against
1162 Y. Wang and J. Yu
each other”. “Feelings” reflect not only the recollection of past life but also the thinking
of real life. It is the contrast of reality with past and ideal life that resonates most [7].
2.4 Results
Whether the audience pays attention to the information dissemination by the interactive
multimedia in art museums often depends on the following factors: whether it attracts
attention, whether the information can continue to arouse their interest, whether the
information is useful, and whether they will take actions because of such information.
If it attracts no attention at all, let alone dissemination; if the information does not
match the knowledge and beliefs of the audience, dissemination cannot be sustained;
even though the audience accepts the information but takes no action, dissemination
does not achieve the desired effects. Therefore, a successful dissemination process may
be rapid or tortuous and complicated. In the research on the effects of information
dissemination in art museums, Aristotle focuses on disseminators to see how they
persuade others, while Cartwright prefers to study information receivers, who are
interested in their changes in the dissemination process. According to this research, it is
easier to draw conclusions from the audience in art museums. The way of interactive
multimedia is the channel and way of dissemination, which are realistically presented
by digital media technology. In the face of endless information stream in art museums,
they can combine with each other to determine the quantity and length of information
stream to get the best dissemination effects of artworks.
3 Conclusion
The application trend of interactive multimedia in art museums includes information
sources of digital media, channels as well as quality and storage time of information
dissemination, which in fact discusses how to constitute effective information and what
kind of artistic information the audience will choose to receive. Emotional appeals are
more likely to change the attitude than logical appeals. Information is no doubt the
content of dissemination in art museums, but not all information needs to be dissem-
inated or acquired, and digital media technically supports and realizes this interactive
expression. Interactive multimedia is both the integration of information itself and the
integration of media and information.
Acknowledgments. This paper is sponsored by the characteristic specialty construction project
of colleges and universities in guangdong province. Supported by the youth innovation talents
project (humanities and social sciences) of guangdong education department, the project title is
“research on media convergence and media industry development”, project number:
2016WQNCX079; Guangdong provincial philosophy and social sciences planning joint project
in 2018: “research on the application of energy-saving methods in traditional buildings and
modern environmental art design in guangfu” (GD18XYS03); Art project of humanities and
social sciences research planning foundation of ministry of education, restoration and con-
struction of lingnan city culture: research on environmental art design method based on
geospatial information system (GIS), no. 15YJA760037.
Application Trend of Interactive Multimedia in Art Museums 1163
References
1. Raskin, J.: The Humane Interface, translated by Shi Yuanchun, pp. 1–8. China Machine Press
(2003)
2. Zettl, H.: Image, Sound, Motion, translated by Zhao Miaomiao, pp. 17–30, 207–216, Beijing
Broadcasting Institute Press (2003)
3. Carey, J.W.: As the Transmission of Culture, translated by Ding Wei, Huaxia Publishing
House (2005)
4. Dernie, D.: Advanced Course of British Exhibition Design, translated by Han Wei, pp. 18–40.
Shanghai People’s Fine Arts Publishing House (2007)
5. Coper, A.: The Way of Interaction Design, translated by Ding Quangang and Jiang fang,
pp. 169–188. Electronic Industry Press (2006)
6. Gombrich, E.H.: Art and Illusion, translated by Yang Chengkai, li Benzheng and Fan
Jingzhong, pp. 154–217. Guangxi Fine Arts Publishing House (2012)
7. Martin, D., Jacobus, L.A.: An Introduction to the Art, translated by Bao Huiyi and Huang
Shaoting, pp. 395–432. Shanghai Social Science Press (2011)
Design Criteria in Vernacular Architecture
as a Proposal for Low-Income Dwelling
for Urban Parishes of the Babahoyo
Canton, Ecuador
Julio Franco Puga1(&), Bryan Colorado Pástor1,
Jesús Rafael Hechavarría Hernández1, and Maikel Leyva2
1
Faculty of Architecture and Urbanism, University of Guayaquil,
Cdla. Salvador Allende Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{julio.francop,bryan.coloradopa,
jesus.hechavarriah}@ug.edu.ec
2
Babahoyo Technical University, Babahoyo, Los Ríos, Ecuador
[email protected]
Abstract. The objective of this study is to establish the fundamental criteria to
consider in the vernacular architecture for houses in the canton of Babahoyo in
Ecuador. The decision-making process is based on a methodology conceived
under a systemic approach where the specific needs of the urban parishes of
Barreiro, El Salto and Camilo Ponce are considered. A holistic approach is
promoted, through the qualitative method developed by interviews, surveys and
workshops. The Quality Function Deployment (QFD) methodology was used in
the specific case of the proposal as sustainable groups in the short, medium and
long term. For the quantitative verification of the QFD methodology, it was
decided to follow a mathematical method based on cognitive neutrophic maps to
represent the behavior and functioning of complex systems and express them
graphically using the Mental Modeler computer platform. This flexible com-
puting technique allows us to model how users make decisions based on their
knowledge of the subject at different levels of abstraction. It is debated whether
this housing design with vernacular proposals generates comfort and is attractive
for new users called non-native and for native or traditional users. As a pre-
liminary conclusion of the study, it is obtained that low-income dwelling with
characteristics of vernacular architecture incurs practical solutions to the housing
deficit in the urban parishes of Barreiro, Salto and Camilo Ponce, obtaining a
high acceptance rate by users.
Keywords: Decision-making Low-income dwelling Vernacular architecture
1 Introduction
In Ecuador, during 2013, the 45% of 3.8 million Ecuadorian households lived in
inadequate housing. This number accounted for 36% of households suffering from
qualitative deficits and 9% of households suffering quantitative déficits. The 1.37
million homes with qualitative deficit reside in homes whose tenure is insecure, built
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1164–1170, 2020.
https://doi.org/10.1007/978-3-030-39512-4_177
Design Criteria in Vernacular Architecture as a Proposal 1165
with inadequate materials, lacking basic health services, or with over-crowding prob-
lems. The 342 000 households with quantitative deficits share their home with one or
more homes, or live in makeshift housing units [1]. All these deficits are caused by lack
of access to credit, lack of urban infrastructure, inadequate employment and unplanned
development; what generates a situation of poverty and exclusion causing a con-
struction of empirical houses, informal settlements for the need to have a house.
The current problem of the Babahoyo canton within the Province of Los Ríos, lies
in the deficient housing according to the current population quantity maintained by the
canton. A proposal of public housing policy focuses mainly on the allocation of
subsidies to demand and the implementation of construction projects. However,
without a public land policy that regulates the use and market it will be very difficult to
reduce the supply gap and the demand for housing and achieve an efficient, equitable
and orderly development of the city [2]. For a good redistribution of resources (terri-
tory), a qualitative and multidisciplinary research is carried out in such a way that the
social and habitability conditions presented by the population with low economic
resources can be examined; and in this way establish the spatial and constructive
requirements that can be implemented [3].
In general, in the process of designing low-income dwelling, the human being is not
considered, only the cost of materials and labor are of interest [4]. Currently, special
interest is given to this criterion where the priority is the user over the economic and
political aspects [5]. It is for this reason that many researchers have dedicated efforts to
give this sector of the population design options congruent with decent housing [6–9].
Vernacular architecture is defined as that architecture given by tradition in which its
construction materials and technological designs are native, simple and efficient in
response to the conditions of their environment [10, 11]. The World Tourism Orga-
nization indicates that vernacular architecture is a reference of local territorial culture
and defines this constructive model of architecture as a resource for tourism devel-
opment [12]. That is why it is proposed to use the methodology (QDF) to strengthen
community organization, so that citizens thus link and engage together in the decision
making process. “This seeks to generate economic, administrative and social sustain-
ability of the new communities” [13], so that the design proposals of vernacular
architecture are sustainable and the inhabitants of Babahoyo to resort not to self-
construction of low-quality housing, the occupation of homes without services, or the
residence shared between several homes.
2 Methodological Framework
This study aims to determine the priority of maintaining the reliability of low-income
dwelling design. The method used was Importance Performance Analysis (IPA) and
Quality Function Deployment (QFD) [14]. The level of importance of the user is
described in the importance classification diagram that is divided into four quadrants
with the following explanation:
Quadrant A: An area that contains factors that users consider important, but with very
low levels of user satisfaction. In this area designers make continuous improvements to
1166 J. Franco Puga et al.
improve performance. In this case the users of the urban parishes Barreiro, Salto and
Camilo Ponce were part of an educational workshop in which they were asked to
determine the area of influence of their home and describe that part of it seemed less
satisfactory.
Quadrant B: An area that contains factors that users consider important, with high
levels of satisfaction. As result of workshops conducted with users of various ages, it
was concluded that 85% of respondents opted for an area that everyone considers
equally satisfactory to develop their activities. The so called “zaguán” or front hall
which was for recreational use and rest of the inhabitants and in turn allowed inter-
action between neighbors who organized various activities such as parties or children’s
games.
Quadrant C: An area that contains factors that users consider less important and, in
fact, are less important. The opinion of the housewives showed that the back yard, the
garden and sometimes the dining room are the least used residential spaces during the
day.
Quadrant D: An area that contains factors that users consider less important and feel
overvalued. In sections of work with the users of each parish it was determined that the
dining room space has been overrated. In previous proposals where a design was made
considering the minimum dimensions of the houses, the dining room shares functions
with the living room, which is used as the new dining room and family gathering place.
The proposal of the users consists in recovering the space of the dining room with a
new design respecting the idiosyncrasies of the inhabitants.
2.1 TOPSIS Method: Options and Criteria
The criteria to consider in the TOPSIS method are established to understand the
complexity of the low-income dwelling designs using the fuzzy cognitive maps
modeling and to support decision making process. Initially relevant indicators are
selected (Table 1).
After the analysis carried out with the users, the following proposals were made for
each territory:
• Barreiro Parish, double height housing with ventilated spaces.
• Parroquia El Salto, 1-storey house with better distribution of spaces to avoid over-
crowding.
• Camilo Ponce Parish, redesign of interior housing at least 2 floors.
Among the suggestions to consider in the use of construction materials the bamboo
cane stands out for its easy treatment and obtaining in the coastal region. The wood
used as a permanent building material should not have bark or large holes or too many
knots. It must be dry wood and not be crooked or broken. It should be stored hori-
zontally in a dry place with good air circulation. Light hollow bricks are usually not
strong enough to be used in the construction of these properties, the solid clay bricks is
Design Criteria in Vernacular Architecture as a Proposal 1167
Table 1. CFM relevant indicators (Nodes) and their meanings.
Concept Concept description
Materials The choice of construction materials should be made taking into account
especially their suitability, their local availability and the money they are
willing to invest
Bioclimatic The bioclimatic design is applied in the construction of buildings for
designs harmony between the natural environment and the local climate,
guaranteeing thermal conditions and comfort inside. Among the benefits
are: conventional energy savings, money savings, environmental
protection and improvement of indoor living conditions
Cost Using inexpensive materials and saving 50% of the total price of a house
achieving a better result between the cost-benefit ratio in the architectural
designs
Duration The live of the house is a measure of the duration and the capacity to
endure the climatic and exogenous conditions of the environment.
Focused on the resistance of the materials and their subsequent reuse in
time facing the climatic conditions of the environment
Capacity The definition of the number of people members of the same family
nucleus that can live in the house in good conditions
Socio-economic Oriented to the degree of economic income and lifestyle that families lead
level to maintain their home
Earthquake The design of the house must withstand possible earthquake disasters by
resistance adopting and applying a building code with an appropriate seismic design
and construction standards
preferable. It was also considered that the bioclimatic designs must be houses those are
cool in the day and at night. The approximate cost of a low-income dwelling is $10,000
developed in a land of 24 to 36 m2. The estimate duration of the house is 20 to 40 years
with a capacity between 5 to 8 people, since the families of most communities are
numerous in a medium/low socio-economic level. The durability of materials used have
the highest degree of durability so that the houses do not present future pathological
problems and using resistant materials and look beautiful.
2.2 TOPSIS Method: Weighting and Development
For the development of the TOPSIS method, by using the multiple criteria of the users,
7 alternatives were obtained that received a value or weight set by the user on a
maximum score of 16 points (Fig. 1).
The following values were obtained as a result: Materials (3), Bioclimatic condi-
tions (1), Cost (3), Duration (2), Capacity (3), Socioeconomic level (1), Earthquake
resistant (3). The TOPSIS method and full calculation result is shown below (Fig. 2).
1168 J. Franco Puga et al.
Fig. 1. Indicators and decision matrix.
Fig. 2. TOPSIS method calculation result.
3 Conclusion
As a final result, it is obtained that the development of a proposal of low-income
dwelling must contemplate an iterative analysis in the tendency of the decision making
of the beneficiaries generating proposals according to the current needs, analyzed under
Design Criteria in Vernacular Architecture as a Proposal 1169
the critical look of the professionals. The QFD methodology proved to be effective tool
by quantifying qualitative criteria that improve the decision-making process with which
housing proposals can be obtained that meet the user’s requirements. Vernacular
architecture proved to be more economical and efficient, so it will be used as a
structural model in the social housing program, together with a design in which the
fundamental expression of identity within the community and its relations with the
territory prevails.
Acknowledgments. To the Programa of Master in Architecture with mention in Territorial
Planning and Environmental Management. Faculty of Architecture and Urbanism of the
University of Guayaquil.
References
1. MIDUVI: Ministerio de Desarrollo Urbano y Vivienda. Subsecretaría de Vivienda Dirección
de Gestión y Ejecución de Vivienda PROYECTO: K014 MIDUVI - Programa Nacional de
Vivienda Social – PNVS (2017)
2. García-Ubaque, C., Henao-Trujillo, C.A., Vaca-Bohórquez, M.: Soil management instru-
ments for social housing: case analysis. DYNA 81(184), 217 (2014). https://doi.org/10.
15446/dyna.v81n184.37899
3. Avila, M., Orellana, M., Toledo, J., Tello, M., Córdova, F.: Qualitative architecture.
A multidisciplinary approach to the provision of social housing. Case study Cuenca –
Ecuador, pp. 210–222 (2019). https://doi.org/10.1007/978-3-030-01406-3_18
4. Hechavarría, J., Bermeo, P., Forero, B., Portilla, Y.: Social inclusion: a proposal from the
University of Guayaquil to design popular housing with citizen participation. In: 2018 World
Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC),
Albuquerque, NM, USA, pp. 1–5 (2019)
5. Forero, B., Hechavarría, J., Vega, R.: Bioclimatic design approach for low-income Dwelling
at Monte Sinahí, Guayaquil. In: Di Bucchianico, G. (eds.) Advances in Design for Inclusion,
AHFE 2019. Advances in Intelligent Systems and Computing, vol. 954. Springer, Cham
(2020). https://doi.org/10.1007/978-3-030-20444-0_17
6. Ricaurte, V., Almeida, B., Hechavarría, J., Forero, B.: Effects of the urban form on the
external thermal comfort in low-income Settlements of Guayaquil, Ecuador. In: Charytonow-
icz, J., Falcão, C. (eds.) Advances in Human Factors in Architecture, Sustainable Urban
Planning and Infrastructure, AHFE 2019. Advances in Intelligent Systems and Computing,
vol. 966. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-20151-7_42
7. Hechavarría, J., Vega, R., Forero, B.: Multi-objective optimization applied to the bioclimatic
design of dwellings with ecomaterials. In: Ahram, T., et al. (eds.) Human Systems
Engineering and Design, IHSED 2018. AISC, Chap. 77, vol. 876, pp. 1–6. Springer (2019).
https://doi.org/10.1007/978-3-030-02053-8_77
8. Dick, S., Hechavarría, J., Forero, B.: Systemic analysis of bioclimatic design of low-income
state-led housing program “socio vivienda” at Guayaquil, Ecuador. In: Ahram, T., et al.
(eds.) Human Systems Engineering and Design, IHSED 2018. AISC, Chap. 99, vol. 876,
pp. 1–5. Springer (2019). https://doi.org/10.1007/978-3-030-02053-8_99
9. Forero, B., Hechavarría, J., Alcivar, S., Ricaurte, V.: Systemic approach for inclusive design
of low-income dwellings in popular settlements at Guayaquil, Ecuador. In: Ahram, T., et al.
(eds.) Human Systems Engineering and Design, IHSED. AISC, Chap. 93, vol. 876, pp. 1–5.
Springer (2019). https://doi.org/10.1007/978-3-030-02053-8_93
1170 J. Franco Puga et al.
10. Heidari, A., Sahebzadeh, S., Dalvand, Z.: Natural ventilation in vernacular architecture of
Sistan, Iran; classification and CFD study of compound rooms. Sustainability 9(6), 1048
(2017). https://doi.org/10.3390/su9061048
11. Samir, H., Klingmann, A., Mohamed, M.: Examining the potential values of vernacular
houses in the Asir Region of Saudi Arabia, pp. 27–38 (2018). https://doi.org/10.2495/
IHA180031
12. Jureniene, V.: Interaction between cultural/creative tourism and tourism/cultural heritage
industries. In: Tourism - From Empirical Research Towards Practical Application. InTech
(2016). https://doi.org/10.5772/62661
13. Etxezarreta, A., Cano, G., Merino, S.: Las cooperativas de viviendas de cesión de uso:
experiencias emergentes en España. CIRIEC-España, Revista de Economía Pública, Social y
Cooperativa (92), 61 (2018). https://doi.org/10.7203/CIRIEC-E.92.9266
14. Sedayu, A.: The priority of maintaining the reliability of sustainable construction at the
Ampel Mosque Surabaya. MATEC Web Conf. 195, 06008 (2018). https://doi.org/10.1051/
matecconf/201819506008
Consumer Experience of a Disruptive
Technology: An O2O Food Delivery App Case
Jaehye Suk, Yeon Ji Yang, Yun Jik Jeong, Muzi Xiang,
and Kee Ok Kim(&)
Department of Consumer and Family Sciences,
Sungkyunkwan University, Seoul, Korea
[email protected]
Abstract. The online to offline (O2O) market is rapidly expanding into a
shared-economy industry and penetrating every aspect of consumer life. Food
delivery applications are representative O2O services and are growing with the
usage of smartphones. The purpose of this study is to leverage users’ insights of
a food delivery application to help create a more user-focused environment.
Topic modeling identified key experiences of application usage and a network
analysis highlighted the relationships among these experiences. These analysis
prove that technology applied to food services is an incredibly disruptive as it
penetrates into our daily lives by providing convenience, speed, and accu-
racy – the main factors of disruptive technologies. This study implies that time-
starved contemporary consumers need such all-in-one platforms with on-
demand simplicity, utility, and reliable information which will, ultimately,
empower and grow our capabilities without disrupting the rhythm of our daily
lives.
Keywords: O2O App Consumer experience Online reviews Text-mining
Topic modeling Network analysis
1 Introduction
The online to offline (O2O) service was introduced by Alex Rampell, CEO of Trialpay,
and it combines the advantages of online and offline commerce using information and
communication technology (ICT) [1]. O2O is a system in which consumers place
orders for goods or services online and then receive them at an offline outlet [2].
Korea’s O2O market size in 2015 was approximately 13.2 billion dollars, and it is
estimated to increase to about 950.7 billion dollars in 2020 [3].
Among the O2O platform services, food delivery services are expanding quickly
among urban consumers by replacing physical shopping with digital solutions and
providing time and cost efficiencies. Baedal Minjok, operated by the Seoul-based start-
up Woowa Brothers, is the leading O2O food delivery app in Korea. It offers an all-in-
one service platform for completing search, order, and pay activities. This study
focuses on consumers’ opinions about the app using text mining. The purpose of this
study is to understand users’ positive and negative experiences with the app and
support app developers to be in tune with user-friendly mobile applications.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1171–1177, 2020.
https://doi.org/10.1007/978-3-030-39512-4_178
1172 J. Suk et al.
2 Background
O2O services span various fields, including food services, car rentals, hotels, and real
estate. The food service industry has adopted O2O transactions more proactively
compared to other industries, because the apps offer a low-cost way for businesses to
attract consumers, promote their products, and facilitate contact with consumers, fea-
tures that are especially attractive to small-scale restaurants [2].
According to a survey in 2018 [3], seven in 10 Koreans have used a food delivery
app, and the proportion of orders placed via such apps is increasing, from 24.9% in
2017 to 34.7% in 2018. The real-time connectivity of food delivery apps on smart-
phones provides busy consumers with speed and convenience. The main drivers of the
growth in food delivery app market include high internet penetration, a higher standard
of living, and an increase in mobile usage [4].
The front runner in the food delivery app market in Korea is Baedal Minjok, backed
by major investors, including Goldman Sachs, venture capitals, and Korean internet
giant Naver [5]. It was selected as the most popular app of 2018 on Google Play and
Apple’s App Store. Food orders, sales, and net profits of service providers using the
app have all increased, making them mutually compatible.
Leveraging insights from consumer reviews on a food delivery app could create a
user-focused app development environment, and this switch toward mobile users could
expand the food delivery market in scope. While consumer feedback is valuable to app
developers, the sheer volume of daily reviews can be overwhelming. Consumer
reviews are essential for extracting insights into what they want in the future and how
an app can be improved.
3 Methods
3.1 Text Data
Text data were collected from online reviews of a food delivery app, called Baedal
Minjok, posted on Google Play from September 1, 2017 to February 28, 2019 by using
the R 3.5.3 version. The first dataset comprised 21,138 reviews and the tidy data was
reduced to 13,495 through a pre-processing procedure for cleaning and preparing the
data to facilitate analysis. A lemmatization process was also carried out to reduce the
different forms of a word to one, i.e., the process of grouping inflected forms together
as a single base form [6].
3.2 Topic Modeling
To understand consumers’ experiences with the app, topic modeling was performed
using the Latent Dirichlet Allocation (LDA) package in the R 3.5.3 version. Topic
modeling algorithms are used to discover a set of topics from a large collection of
documents, where a topic is a distribution of terms associated with a single theme [7].
LDA is one of the most common algorithms for topic modeling [8]. It is a generative
probabilistic model of a corpus, and the basic idea is that documents are represented as
Consumer Experience of a Disruptive Technology: An O2O Food Delivery App Case 1173
random mixtures over latent topics, where each topic is characterized by a distribution
over words [9].
3.3 Network Analysis
Graphs provide a better way of dealing with abstract concepts like relationships and
interactions. Network analysis offers an intuitively visual way of thinking about rela-
tionships and interactions among entities with graphs [10]. The tidytext package in R
3.5.3 was used to extract tokenized text dataset by pairs of adjacent words. The ggraph
and the igraph packages in R 3.5.3 were used to visualize the results of network
analysis.
4 Results
The 13,495 reviews were text mined by topic modeling and network analysis. Topic
modeling identifies key issues from user reviews by classifying them, while network
analysis adds detailed stories in addition to the results from topic modeling. Positive
and negative experiences with the app were revealed through these text mining pro-
cesses. This study focuses more on the negative experiences than the positive ones. The
topic modeling analysis results of 10 topics and the top-rated 15 keywords in each topic
are listed in Table 1.
Among the 10 topics, six topics in 2, 4 through 6, 9, and 10 reveal positive
experiences with the app. They include tangible benefits such as various events’
information on first orders, informing arrival time, recommendations for menus or
restaurants, one-person menus, and payments. On the other hand, negative experiences
with the app are revealed in four topics in 1, 3, 7, and 8. These include reviews for the
purpose of advertising, lack of updates, inconveniences with respect to minimum
orders, and errors or problems with networks and logins.
To further examine users’ negative experiences, the Intertopic Distance Map along
with top-rated 30 keywords in each of the four negative topics is shown in Fig. 1.
Topic 1 reveals difficulties in referencing accurate reviews since they are exposed to
reviews that are for advertising purposes. Topics 3 and 8 consist of keywords that
indicate errors in the app. Topic 3 consists of errors in payment that are critical issues
for users. Topic 8 includes network problems in relation to the app. A problem with
login is one of the fatal negative experiences from a user’s standpoint and, therefore,
should be thoroughly resolved. Topic 7 includes inconveniences with lazy updates, and
this signals contemporary consumers’ need for speed and accuracy.
To understand the complexity of the interrelationships among keywords in the user
reviews, the results of the network analysis are depicted in Fig. 2. These clearly reveal
major issues along with minor but critical ones for understanding users’ insights on the
app. Keywords related to the major functions of the app are positioned in the middle
and widely interconnected among them. This proves that food service technology is
one of the most powerfully disruptive technologies penetrating our daily lives by
providing convenience, speed, and accuracy. Along with these pivotal benefits from the
app, more conveniences from coupon, event, discount, and first order are appreciated
1174 J. Suk et al.
Table 1. Results of topic modeling
Topic 1 Topic 2 Topic 3 Topic 4 Topic 5
1 Review Food Delete Order Coupons
2 Consumer Use Full payment Call Menu
3 Service Delivery Call Photo Registration
company
4 Restaurant Event Necessary Necessary Neighborhood
5 Advertising Discount Modification Search Recommendation
6 Center Various Card Category Choice
7 Write First order Inconvenience List Apply
8 Add Patronize Pay Preparation Restaurant
9 Rating Benefits Download Inconvenience Customized
coupons
10 Delivery Area Execute Invisible Distance
company
11 Arrange Unsuccessful Contact Text message Ranking
12 Improvement Neighborhood Automatic Expected arrival Friend
time
13 Function Commission Kakao Request Operate
14 F2F Payment Neat Setting Phone number Assistance
15 Inconvenience Consumption Confirmation Method Decision
Topic 6 Topic 7 Topic 8 Topic 9 Topic 10
1 Speedy Search Error Useful Use
2 Convenience Address Network Live alone The best
3 Payment Update Problem Card payment Join
4 Possible Restaurant Login Type Accumulation
5 Discount Information Cancel Concern Mange
coupons
6 Famous Number Connect Neighborhood Member
restaurant
7 Accurate Cash System Confirmation Payment
8 Price Function Confirmation Cash payment Business trip
9 Suitable Location Occur Insufficiency One-person menu
10 Category Input Access Planned Naver pay
11 Local restaurant Amount Phone Choice Ten thousand
won
12 Phone Name Weird Discount coupons Saved points
13 Transfer Inconvenience Notification Famous restaurant Neighborhood
14 Nice pay Relieved Done More Simple
15 Login Minimum Loading Category Delivery charge
Consumer Experience of a Disruptive Technology: An O2O Food Delivery App Case 1175
Fig. 1. Intertopic Distance Map (IDM)
by users. The app also suits those who live alone and supports them to stay better than
others or have famous restaurant experiences without physical movement. The two
negative issues that popped up in the network analysis are errors with respect to system
and logins. Login errors are severe negative experiences, and users clearly reveal their
wish for an improved and flawless system for login.
Fig. 2. Results of network analysis
1176 J. Suk et al.
5 Conclusion
This study applies topic modeling and network analysis to analyze online reviews of a
food delivery app to reveal detailed stories about users’ experiences with the app and
give insights into the app’s positive and negative aspects to developers.
The results of this study indicate that consumers enjoy benefits of speed, conve-
nience, and accuracy when using the app. These are the main factors behind the power
of disruptive technologies that create the world we live in and change how we behave.
All-in-one convenience that comes from search, order, and pay on one platform lays
the foundation for core positive experiences among the app users. The major negative
experiences come from system errors, login errors, or fake reviews for advertising
purposes. These results indicate that even though app developers continue to make
efforts to improve the app, consumers still experience problems, such basic errors with
network, system, and login.
Food delivery apps are providing third-party services, since the food made by a
restaurant is picked up and dropped off by the delivery company, and, therefore, the
company has two clients—the provider and the user who uses the delivery service. App
developers, especially in the O2O business, need to focus on the quality of services for
both users and suppliers. Alliances and collaborations between providers and devel-
opers are necessary for providing efficient and enjoyable user experiences and, in turn,
for ultimate success in the related industry. To create a user-centered development
environment, app developers’ focus must shift toward the app users by providing all-in-
one platforms with on-demand simplicity, utility, and reliable information which will,
ultimately, empower and grow our capabilities without disrupting the rhythm of our
daily lives.
References
1. Yao, Z., Kim, E., Hong, T.: A study on impact of intention to use O2O commerce of China:
focusing on moderating effects of cultural values. J. China Stud. 19(1), 153–172 (2016)
2. Lee, S.W., Sung, H.J., Jeon, H.M.: Determinants of continuous intention on food delivery
apps: extending UTAUT2 with information quality. Sustainability 11, 3141 (2019)
3. Korea Internet Security Agency and KT Economic Research Institute: Top ten internet issues
report in 2017 (2016)
4. Sahin, M.: Food & delivery apps market research 2019. Mobile Marketing News & Trends
(2019)
5. Sohn, J.Y.: Food delivery apps thrive in Korea, where 7 in 10 orders via mobile. The Korea
Herald (2018)
6. Man, Y., Gai, C., Lyu, M.R., Jiang, J.: Experience report: understanding cross-platform app
issues from user reviews. In: 27th IEEE International Symposium on Software Reliability
Engineering, ISSRE 2016, Ottawa, ON, Canada, pp. 138–149 (2016)
7. Blei, D.M., Lafferty, J.D.: Dynamic topic models. In: Proceedings of the 23rd International
Conference on Machine Learning, Pittsburg, PA, USA, pp. 113–120 (2006)
Consumer Experience of a Disruptive Technology: An O2O Food Delivery App Case 1177
8. Silge, J., Robinson, D.: Text Mining with R: A Tidy Approach (2019)
9. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–
1022 (2003)
10. Saddler, J.: Introduction to Network Analysis with R. https://www.jessesadler.com/post/
network-analysis-with-r/
A Group Travel Recommender System Based
on Collaborative Filtering and Group
Approximate Constraint Satisfaction
JinLu He, IlYoung Choi, and JaeKyeong Kim(&)
School of Management and AI Research Center, KyungHee University,
Seoul, Korea
{hejinlu,choice102,jaek}@khu.ac.kr
Abstract. In this research, a modified CF (Collaborative filtering) procedure is
suggested to decide a destination for group travel planning service. Furthermore,
a composite travel product includes hotel, and attraction in addition to the
selection of a destination. For such a purpose, individual user’s personal
requirements are represented by constraints, and a group constraints are con-
structed based on group member’s constraints. A two-phase group approximate
satisfying constraint procedure is suggested to automate group decision making
procedure for selecting their trip packages to improve group tourists’ conve-
nience and their satisfaction. A group travel recommendation is suggested with
an illustrative example and an experimental design is suggested to validate the
suggested procedure.
Keywords: Group recommender systems Approximate constraint
satisfaction Travel recommender system Collaborative filtering
1 Introduction
In the smart tourism, travelling group internet sites like TripAdvisor have contributed
much in selecting tourist’s destinations as well as for planning trips (Choi et al. 2015;
2019). However, due to the wealth of travel information on the travel community
website, it is difficult for visitors to make plans that suit them. In order to handle these
difficulties and better suggest qualified services to customers, travel recommender
systems have been suggested to give personalized trip planning service by analyzing
users’ past historic travelling record. Collaborative filtering (CF) is recognized as a
very successful recommending techniques (Schafer et al. 2001). It gives recommending
services to users based on other users’ preferences on items, giving more importance
weights on those from other similar users. A recommender system helps users to get
information, services, or products by analyzing suggestions from the other users.
However, travel recommendation systems have mainly focused on recommending
personalized trip planning service to individual user rather than group tourists intending
to travel together (Gretzel et al. 2015; 2016) Existing recommender systems are not
easy to aggregate individual tourists’ preferences into a group’s taste properly.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1178–1183, 2020.
https://doi.org/10.1007/978-3-030-39512-4_179
A Group Travel Recommender System 1179
Aggregating individual tourists’ tastes into a group taste or adding individual
tourist’s recommendation lists into a group recommendation usually give a result in
dissatisfaction of a part of group tourists while it satisfies the majority of group tourists
(Kim et al. 2010). A group trip recommender system is proposed to improve the group
recommendation effectiveness as well as the individual tourist’s satisfaction. In this
research, a modified CF-procedure and a group satisfying constraint filtering process is
suggested to decide destination, traveling duration, and travelling cost. Individual
tourist’s requirements are represented by constraints, and a group approximate satis-
fying constraint procedure is suggested to automate group decision making process for
selecting group trip packages to improve group tourists’ satisfaction and convenience.
GRecTr (which stands for a group recommender system for travel planning service) is
suggested to offer recommending service sets for group tourists through a two-phase
recommending process. An overall procedure is explained step by step with an illus-
trative example.
2 A Group Recommendation Procedure
The proposed method is composed of two phases as shown in Fig. 1. The first phase
analyzes a group of users’ preferences for a destination and identifies who is neighbors
or not, that is, other group users that exhibit similar behavior to the target group. After
finding the neighborhood of the target group of users, an initial top-n recommended
destination list is chosen. The second phase compares the recommended destination
lists based on the limitations of the target group (such as transportation, hotels and their
prices). The constraints for the target group of users are provided by each user in that
group, and the best items which meet the constraints for that group of users is then
recommended.
Fig. 1. Overall procedure
1180 J. He et al.
2.1 Phase 1: CF-Based Filtering
At this phase, individual member’s profiles are added into a group profile. InGRecTr,
the user profile is represented by preference ranking matrix R ¼ rij , where
i ¼ 1; 2; . . .; m; j ¼ 1; 2; . . .; n; rij 2 ½1; R, the total number of users is represented by
m, and the total number of destinations is represented by n, rij is the user i’s rating
value on destination j (R = highest, 1 = lowest). The next step is to calculate the
similarity between the group profiles, and find the target group of neighbors. The
process follows the typical nearest neighbor algorithm. Once we get the target group’s
similarity score between target group and all the other groups, we will choose several
groups as neighbors. The next step is to eventually get top N recommended value. For a
given set of generated recommend set, GRecTr access frequency of the highest des-
tination choice (Schafer et al. 2001).
2.2 Phase 2: Group Approximate Constraint Creation Procedure
In the first phase, a list of recommended destinations will be generated for the
advantageous destinations of the target group. However, even at the same destination,
visitors will have different experiences. Therefore, it is considered to be meaningful to
give users with custom trip planning services (Choi et al. 2019). Users directly enter
their restrictions on travel products or services (e.g., transportation, type of accom-
modation). These limitations are not absolute standards. For example, we can ignore
the $30 gap, but we must not ignore the huge difference of $60. In addition, user
constraints are independent of each other, and users have independent constraints that
are more important than other constraints and do not allow comparisons between
constraints.
Based on these assumptions, GRecTr derives group tourists’ preferences from
individual tourist interests. GRecTr monitors the activities of members and learns the
vehicles, hotels, prices and other information of interest to members, which is repre-
sented as an individual member’s constraint set. GRecTr creates a group constraint set
representing a group’s general characteristics by aggregating individual member’s
constraints and derive a common set into a group preference.
2.3 Phase 3: Group Approximate Constraint-Based Filtering
After performing group constraints based on the constraints of individual members, we
matched the constraints of group visitors with an electronic directory which contains
the information about the candidate destination lists. If constraint c of group user i is
defined as a relationship such as “flight = American airlines”, or “price 500”, then
for constraint c, that of electronic directory, e (c, h) is defined as an error function to
return satisfaction error, that is, the degree of dissatisfaction. Then, the focus of user i is
given pfci;1 ; ci;2 ; . . .; ci;p g. Therefore, if s < t, satisfaction with ci, s must be more
important than satisfaction with ci. That is to say, the importance of the constraint by
strict linear order. However, this kind of satisfying constraint often is too rigid. For
example, it is assumed that the price limit is $2000 or less, and the three travel
packages (P1, P2, and P3) respectively are the price of $2000, $2001 and $2100.
A Group Travel Recommender System 1181
Although the e (price at $2000 or less, P1) > e (price of $2000 or less, P2), users will
be P1 and P2 is regarded as the same (for just $1), but he/she will P1 and P3 as
different in order to solve the rigid, we adopt the concept of “difference interval”. (Choi
et al. 2015). The N destination lists chosen in phase 2 were re-calculated and ranked
based on the approximate satisfying constraint method. As a result, either M product
packages (M < N) roughly meet the constraints of group tourists, or N product
packages are reordered to target group tourists.
2.4 Phase 4: Individual Profile-Based Filtering
If the recommendation process is stopped after phase 3, members of a group may be
ignored for being less active or expressing their opinions less frequently. In planning a
group travel together, considering each individual member is a very important issue.
The phase 4 of GRecTr is composed of two steps: calculation of the degree of dis-
satisfaction of each member about the candidate product package set, and filtering of
product packages which result in higher degree value of dissatisfaction of each
member.
The first step is to calculate each member in a group and candidate destinations not
satisfaction. GRecTr uses total sum of e(c, h), which is the return error of satisfaction
error function, namely, the dissatisfaction degree constraint c and h electronic directory
entry. That is, an error function value of individual user s is the sum of error function
values of all constraints. The second step in this phase is to eliminate some targets from
the candidate recommendation set based on the user’s error function value to minimize
the dissatisfaction of individual members in the group. If there is an error function
value is lower than those of the members of the predefined threshold, it is deleted from
the candidate set.
3 An Illustrative Example
We present an example to better understand the proposed method. Assume that group
G1 is collecting travel information for vacations to Europe from online travel agencies.
We suppose there are 23 tourists, divided into 5 groups, 9 destinations, all destinations
totaling 20 flights 27 hotels, 27 activities, 6 constraints. Table 1 shows rating infor-
mation of groups for 9 destination.
Table 1. Group profile
Mad Lis Ath Cop War Lon Par Rom Zür
G1 3.67 2.67 3.00 3.00 3.00 3.67
G2 3.67 3.50 3.25 2.75 3.75 2.80 4.00
G3 3.67 3.67 4.50 3.33 3.50 3.50 4.00 2.50
G4 2.67 4.00 3.50 3.00 4.33 2.00 3.67 2.00
G5 3.00 3.33 4.50 2.50 3.00 2.75 3.50 3.00
1182 J. He et al.
3.1 Phase 1: CF-Based Filtering
The similarity degree between target group G1 and others are represented by Pearson-r
correlation measures. We find that G3 and G4 have higher similarity degrees. Thus, the
neighbors of target group G1 becomes G3 and G4. Based on neighbor G3 and G4, Mad,
Lis, Cop, War, and Rom are chosen as a candidate destinations.
3.2 Phase 2: Group Approximate Constraint Creation Procedure
GRec_Tr creates a group constraint set which represents a group’s overall demands by
adding individual group member’s constraints and derive a common set into a group
preference. The airplane price-related constraint C_1 of individual tourists are
“price 650”, “price 550”, “price 640”, and “price 700”. Then the con-
straint C_1 for target group G1 is derived as “price 550”. Table 2 shows the con-
cerns of target group G1 and their indifference interval per each constraint.
Table 2. Constraint set for target group G1
No Constraints Indifference interval
C_1 Airfare is $550 or less $100
C_2 One stop fligh (N = 1) None
C_3 Hotel price is less than $175 $35
C_4 Hotel grade is 3-star or above None
C_5 Activity price is $75 or less $10
C_6 Activity is located within 5 km from the hotel 2 km
3.3 Phase 3: Group Approximate Constraint-Based Filtering
G1’s main concern is assumed to be the price of airplane and assumed that a difference
of $100 is not large enough to discriminate between flights. The dissatisfaction level e
is calculated for the constraint and the candidate recommendation set to generate a
recommendation set. For flight, for example, we include e(flight price $550,
F_Mad001) = 12, and we exclude F_Lis001, F_Lis002, F_War002 and F_War001
from candidate recommendation set. The other constraints are compared to the can-
didate recommendation set in the same way. Finally, A_Rom003 is excluded from the
candidate set. The reason is the location of A_Rom003 is beyond the indifference level
from the recommended hotel.
3.4 Phase 4: Individual Profile-Based Filtering
To respect inactive members’ preferences, the candidate set is suffering filtering pro-
cess based on travel-group compatibility measures. As a result, Cop are removed.
Therefore, it is recommended {destination = Rom (Rome), hotel = H_Rom001 (Hotel
Sol Levante), flight = F_Rom001(Russian Airlines), and activity = A_Rom001(Papal
Audience)} to the target group G1.
A Group Travel Recommender System 1183
4 Conclusion
This research suggests a GRec_Tr (a group recommender system for travel planning
service) to offer recommending service sets for group tourist members through a two-
phase recommending procedure, a modified CF and group approximate satisfying
constraint procedure. Compared to existing researches, which usually handles the
average preference of all group members. GRec_Tr suggests candidate recommenda-
tions after removing the recommendations which would be not satisfied by inactive
members. Consequently, tourists’ traveling expertise would be reinforced and their
satisfaction level will be higher. To do experiments, we develop a web-based recom-
mendation system and a benchmark recommendation system that mimics an online
travel information system. Furthermore, we plan to survey the tourists’ satisfaction
level of the recommendation results from GRec_Tr.
References
Choi, I.Y., Kim, J.K., Ryu, Y.U.: A two-tiered recommender system for tourism product
recommendations In: 2015 48th Hawaii International Conference on System Sciences,
pp. 3354–3363, January 2015
Choi, I.Y., Kim, J.K., Ryu, Y.U.: A recommender system based on personal constraints for smart
tourism city. Asia Pac. J. Tourism Res. (2019, in press). https://doi.org/10.1080/10941665.
2019.1592765
Gretzel, U., Sigala, M., Xiang, Z., Koo, C.: Smart tourism: foundations and developments.
Electron. Markets 25(3), 179–188 (2015)
Gretzel, U., Zhong, L., Koo, C.: Application of smart tourism to cities. Int. J. Tourism Cities 2(2),
35–46 (2016)
Kim, J.K., Kim, H.K., Oh, H.Y., Ryu, Y.U.: A group recommendation system for online
communities. Int. J. Inf. Manag. 30(3), 212–219 (2010)
Schafer, J.B., Konstan, J.A., Riedl, J.: E-commerce recommendation applications. Data Min.
Knowl. Discov. 5(1/2), 115–153 (2001)
Consumer’s Information Privacy and Security
Concerns and Use of Intelligent Technology
Seonglim Lee(&), Jaehye Suk, Hee Ra Ha, Xiao Xi Song,
and YuanZhou Deng
Department of Consumer and Family Sciences, Sungkyunkwan University,
Seoul, Korea
[email protected]
Abstract. Using the data from the 2017 Survey on Information Security, we
seek to identify specific security and privacy concerns of consumers that are
significantly associated with consumers’ use of AI and IoT services. The major
findings were as follows. First, the common security and privacy concerns that
affected use of both intelligent technologies, AI and IoT, were “excessive col-
lection of personal information”, “unlawful use of personal information by
hacking”, and “massive spread of damage due to increased connectivity”.
Second, additionally, use of IoT services was negatively associated with con-
sumer’s concerns on privacy invasion, management vulnerability due to mul-
tiple connectivity, malware and ransomware attack, and operational failures and
interruptions. Third, consumers who concerned on unlawful collection of per-
sonal information by hacking and uncertainty on responsible entity for security
breach were less likely to use AI services. These results suggested that con-
sumers prioritize security concerns over privacy concerns in using intelligent
technology.
Keywords: Security concern Privacy concern Intelligent technology IoT
AI
1 Introduction
Intelligent technologies have been spread into consumers’ daily life. Recent emergence
of intelligent technology brings consumers convenience and efficiency that they have
never experienced. Performing services through intelligent technology involves various
inter-connected devices and players such as devices provider, network provider, service
developer, platform providers, and etc. Sensitive private data are transmitted among
them in the process of collecting, storing, analyzing, and utilizing them, which
increases serious security vulnerabilities [1]. Therefore, with the rise of intelligent
technologies such as Artificial Intelligence and Internet of Things, the issue of data
security and information privacy has become central [2, 3]. Devices for AI and IoT
technology continuously collect massive data about individuals and their surrounding
environment, and these collected private information can be used to track, profile, and
predict people. Security breaches and privacy violations are more likely to occur with
more serious consequences in the era of intelligent technology than before [4–6].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1184–1189, 2020.
https://doi.org/10.1007/978-3-030-39512-4_180
Consumer’s Information Privacy and Security Concerns 1185
As privacy and security concerns hinder consumers from using intelligent tech-
nology [1, 7], it is necessary to enhance understanding of consumers’ security concerns
as well as privacy concerns for the widespread application of intelligent technologies.
We seek to identify specific security and privacy concerns of consumers that are
significantly associated with consumers’ use of AI and IoT services and to suggest the
implications for the policies to enhance the utilization of the Intelligent technology.
Privacy concern includes loss of control over personal information, unauthorized
collection, access, and use of personal information, and erroneous inferences [8, 9].
Security concern is referred to malware attacks and its damaging consequences [1].
2 Methods
2.1 Sample
The analysis was based on 2017 Survey on Information Security (Individual) con-
ducted by Korean Internet & Security Agency, which was designed to track individ-
ual’s experiences and perceptions of invasion of personal information and information
security by specific services among a nationally representative sample of Korean
population aged between 10 and 59 years old. Among 4,000 subjects participated in the
survey the sample for analysis was restricted to 3,986 smartphone users who are
supposed to use intelligent technology. The descriptive characteristics of the sample are
shown in Table 1.
Table 1. Descriptive statistics of sample (N = 3986)
Variables Freq (%) Variables Freq (%)
Gender Age
Female 1918 (48.11) 10s 452 (11.33)
Male 2068 (51.98) 20s 736 (18.46)
Education 30s 858 (21.52)
High school 2235 (56.06) 40s 977 (24.51)
College 1751 (43.94) 50s 964 (24.18)
Note: weight values
2.2 Measures
The dependent variables were whether to use of IoT and AI technologies which were
measured as a dummy variable, users are measured as “1” and non-users as “0”.
Questions about information privacy concerns consisted of five items indicating
excessive collecting, using without consent, profiling, leakage, and sharing to third
parties. Questions about security concerns consisted of eight items indicating increased
possibility of privacy invasion, unlawful collection of personal information, unlawful
use of personal information by hacking, increased vulnerability due to multiple con-
nectivity, attack by malware and ransomware, operational failures and interruptions,
1186 S. Lee et al.
Table 2. Frequency of privacy and security concerns (N = 3986)
Variables Freq (%)
Privacy Concerns
Excessive collection of personal information 1945 (48.30)
Use of personal information without consumer consent 1957 (49.10)
Profiling and inferencing consumers 1737 (43.58)
Leakage of personal information 1574 (39.50)
Sharing and transmitting personal information to third parties without consumer consent 740 (18.57)
Security Concerns
Unlawful use of personal information by hacking 2244 (56.29)
Unlawful collection of personal information 2623 (65.81)
Responsibility for Security breach 1140 (28.60)
Increased possibility of privacy invasion 1929 (48.40)
Increased vulnerability due to multiple connectivity 2236 (56.10)
Attack by malware and ransomware 1807 (45.34)
Operational failures and interruptions 1477 (37.05)
Massive spread of damage due to increased connectivity 508 (12.75)
Note: weight values
massive spread of damage due to increased connectivity, and responsibility for security
breach. The frequency of privacy and security concerns was presented in Table 2.
Subjects were asked to choose items as their primary privacy or security concerns. Each
item was converted into a dummy variable measured as “1” if it was a concern and “0”
if it was not a concern.
2.3 Analysis
Binomial logit analysis was conducted to examine the effects of consumers’ infor-
mation privacy concerns and security concerns on the usage of IoT and AI technolo-
gies, with the two dichotomous dependent variables of whether to use IoT and AI
services as a dependent variable. The final model included variables that were sig-
nificantly associated with the use of IoT and AI technologies.
3 Results
Only about 3.16% used IoT technology, while 9.66% used AI technology among the
sample. The results of binomial logit analysis were presented in Tables 3 and 4. The
common security and privacy concerns that affected use of both AI and IoT services
were “excessive collection of personal information”, “unlawful use of personal infor-
mation by hacking”, and “massive spread of damage due to increased connectivity”.
Additionally, use of IoT services was negatively associated with consumer’s concerns
on privacy invasion, management vulnerability due to multiple connectivity, malware
Consumer’s Information Privacy and Security Concerns 1187
Table 3. Results of logit analysis of usage of IoT technology
Variables Coefficient (SE)
Constant 1.10 (2.51)
Gender a) Male 0.26 (0.19)
Education a) College 0.40 (0.23)
Age a) 20s 0.59 (0.36)
30s 0.83 (0.37)*
40s 0.08 (0.38)
50s -1.06 (0.48)*
Region a) Seoul and Kyengky 0.54 (0.62)
Busan and Kyengnam -0.53 (0.68)
Daegu and Kyengbuk -1.41 (0.83)
Gwangju and Jenra -1.15 (0.75)
Daejeon and Chungcheng 0.70 (0.62)
Gangwon 1.53 (0.65)*
Log of household Income -0.22 (0.33)
Privacy concerns
Excessive collection of personal information -0.73 (0.20)***
Security concerns
Increased possibility of privacy invasion -1.96 (0.80)*
Unlawful use of personal information by hacking -0.32 (0.19)
Increased vulnerability due to multiple connectivity -1.67 (0.79)*
Attack by malware and ransomware -1.81 (0.79)*
Operational failures and interruptions -1.75 (0.81)*
Massive spread of damage due to increased connectivity -2.07 (0.84)*
LR chi2 137.63***
Pseudo R2 0.12
a)
Reference categories are female, less than college education, age of 10s, Jeju; *p < .05, **p < .01,
p < .001
***
and ransomware attack, and operational failures and interruptions. Consumers who
concerned on unlawful collection of personal information by hacking and uncertainty
on responsible entity for security breach were less likely to use AI services. The results
showed that the use of intelligent technologies was not significantly associated with
usage of personal information without consent, profiling and inference, and transmit-
ting personal information to the third party without consent, all of which represent
major privacy concerns.
1188 S. Lee et al.
Table 4. Results of logit analysis of usage of AI
Variables Coefficient (SE)
Constant -1.25 (1.19)
Gender a) Male -0.01 (0.11)
Education a) College 0.21 (0.14)
Age a) 20s 0.07 (0.17)
30s -0.32 (0.20)
40s -0.70 (0.20)***
50s -1.68 (0.24)***
Region a) Seoul and Kyengky -0.80 (0.27)**
Busan and Kyengnam -1.58 (0.30)***
Daegu and Kyengbuk -1.72 (0.33)***
Gwangju and Jenra -2.19 (0.35)***
Daejeon and Chungcheng -0.35 (0.27)
Gangwon -0.11 (0.32)
Log of household Income 0.19 (0.20)
Privacy concerns
Excessive collection of personal information -0.32 (0.12)**
Security concerns
Unlawful collection of personal information -0.38 (0.14)**
Unlawful use of personal information by hacking -0.53 (0.14)***
Responsibility for Security breach -0.43 (0.15)**
Massive spread of damage due to increased connectivity -0.44 (0.20)*
LR chi2 256.29***
Pseudo R2 0.10
a)
Reference categories are female, less than college education, age of 10s, Jeju; *p < .05, **p < .01, ***p < .001
4 Conclusion
This study identifies specific security and privacy concerns of consumers that are
significantly associated with consumers’ use of AI and IoT services. The results show
that the privacy concerns and the security failures have a negative impact on the use of
intelligent technology. Intelligent technology needs to reduce the anxiety and concerns
raised by consumers. The results indicated that the use of intelligent technologies was
not significantly associated with usage of personal information without consent, pro-
filing and inference, and transmitting personal information to the third party without
consent, all of which represent major privacy concerns, suggesting that consumers
prioritize security concerns over privacy concerns in using intelligent technology.
References
1. Yang, J.S., Lee, H.J., Park, M.W., Eom, J.H.: Security threats on national defense ICT based
on IoT. Adv. Sci. Technol. Lett. 97, 94–98 (2015)
2. Roman, R., Zhou, J., Lopez, J.: On the features and challenges of security and privacy in
distributed internet of things. Comput. Netw. 57(10), 2266–2279 (2013)
3. Talwana, J.C., Hua, H.J.: Smart world of internet of things (IoT) and its security concerns. In:
2016 IEEE International Conference on Internet of Things and IEEE Green Computing and
Communications and IEEE Cyber, Physical and Social Computing and IEEE Smart Data,
pp. 240–245. IEEE (2016)
Consumer’s Information Privacy and Security Concerns 1189
4. Baruh, L., Popescu, M.: Big data analytics and the limits of privacy self-management. New
Media Soc. 19(1), 57–596 (2015)
5. Kshetri, N.: Big data’s impact on privacy, security and consumer welfare. Telecommun.
Policy 38(11), 113–1145 (2014)
6. Lee, S., Ha, H.R., Oh, J.H., Park, N.: The impact of perceived privacy benefit and risk on
consumers’ desire to use internet of things technology. In: Yamamoto S., Mori H. (eds.)
Human Interface and the Management of Information, Information in Applications and
Services, HIMI 2018. Lecture Notes in Computer Science, vol. 10905, pp. 609–619. Springer,
Cham (2018)
7. Alkhatib, K., Alaiad, A.: The influences of privacy, security, and legal concerns on online
banking adoption: a conceptual framework. In: Online Banking Security Measures and Data
Protection, pp. 113–126. IGI Global (2017)
8. Smith, H.J., Milberg, S.J., Burke, S.J.: Information privacy: measuring individuals’ concerns
about organizational practices. MIS Q. 20(2), 167–196 (1996)
9. Porambage, P., Ylianttila, M., Schmitt, C., Kumar, P., Gurtov, A., Vasilakos, A.V.: The quest
for privacy in the internet of things. IEEE Cloud Comput. 3(2), 36–45 (2016)
PEST Analysis Based on Fuzzy Decision Maps
for the Ordering of Risk Factors in Territorial
Planning of the Vinces Canton, Ecuador
Carlos Luis Valero Fajardo
and Jesús Rafael Hechavarría Hernández(&)
Faculty of Architecture and Urbanism, University of Guayaquil. Cdla. Salvador
Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{carlos.valerofa,jesus.hechavarriah}@ug.edu.ec
Abstract. The inadequate choice of the actions of growth and development in
the Canton Vinces in the province of Los Ríos in Ecuador has increased the risk
for the habitants in the face of natural phenomena. In the present work, a study is
made of the environmental problems and risk situations that exist in the territory.
In support of the decision-making process during territorial planning, a PEST
analysis based on fuzzy decision maps is applied in order to obtain an ordering
of the risk factors before the most unfavorable situations. The proposal identifies
PEST factors and subfactors that allow forming a hierarchical structure of the
model, compares the importance between the criteria to derive the vector of
local weight using the eigenvalue approach, the influence of the model is ana-
lyzed through the cognitive map diffuse, the steady state matrix is obtained to
then normalize the local weight vector obtaining as a result the normalization of
the global weight.
Keywords: Territorial planning Fuzzy decision maps Risk factors PEST
1 Introduction
The urban territorial extension of the canton Vinces located in province of Los Ríos is
prone to flooding because its origin of alluvial geological formation and the proximity
to the natural effluent river Vinces. The 54% of the territory of the Vinces Canton
belongs to the alluvial formations of the Pichilingue geological formation, where is
registeris areas prone to flooding permanently through runoff and natural processes of
accumulation of dragging or alluvial sedimentation materials. The over-flow in the
tributary isdue to the accumulation of sediments which causes the lost hydraulic
capacity. This causes Vinces river downstream to form deltas because of the accu-
mulation of material and the reduction of the course of river before becoming estuaries.
On the other hand, changes in land use, the absence of studies assessing risk
situations and poor urban planning caused by disorderly urban growth have increased
the danger to the population.
The management as an instrument or strategy to achieve sustainable development
in terms of plurisectorial, horizontal policy is currently used [1]. Territorial planning
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1190–1194, 2020.
https://doi.org/10.1007/978-3-030-39512-4_181
PEST Analysis Based on Fuzzy Decision Maps 1191
denotes the need to implement a tool that allows an effective and efficient analysis of
environmental problems and risk situations, in order to identify the risk factors that
support decision-making and appropriate actions within the Canton Vinces. In this
paper, a PEST [2], analysis based on diffuse decision maps to obtain an ordering of the
risk factors in the most unfavorable situations of the Vinces canton is applied.
2 Methodological Framework
The design of this research is based on a methodical, exploratory, descriptive,
deductive analysis; and its object of study is Territorial Planning and Environmental
Management. The synthetic analytical method was used, by means of which the
analysis of the risk indicators against natural phenomena of the inhabitants of the
Canton Vinces was carried out and then generate a synthesis of the results that allow
the assessment of the factors. The technique for assessing the factors was the PEST
(Political, Economic, Social and Technological) matrix [3]. The assessment of risk
factors it was done by development of diffuse decision maps which helped in obtaining
an ordering of risk factors in the most unfavorable situations.
2.1 Theoretical Framework
The PEST technique allows the integration of different variables that identify Political,
Economic, Social, Technological, present factors. For this reason, this technique can be
adapted to different case studies, everything will depend on the approach taken by the
researcher [4].
The PEST analysis technique articulated with diffuse cognitive maps, could give
rise to a diffuse casual model, aimed at identifying risks against natural phenomena of
the inhabitants of the Canton Vinces. The diffuse model used uses feedback to rep-
resent chance and can be combined with theoretical tools of cognitive maps, fuzzy
logic, neural networks, semantic networks, expert systems and nonlinear dynamic
systems [5].
A diffuse cognitive map is nothing more than a representation of the information
that structures knowledge about a sector or field of action and seeks to relate the
concepts found during the exploration of the object of study.
Cognitive Maps are a knowledge representation tool. These maps are composed of
entities called concepts, and relationships between the different concepts that make up
the map. For the design of this map, the knowledge of the experts is used, which
reflects their knowledge in the structuring of the Map. This allows to create models of
complex systems where an exact mathematical model is not possible to use to the
complexity of the system.
2.2 Analysis PEST
Political and Normative Scenario. In this scenario, the risk factors incorporated in
the environmental legislation that regulates the Canton Vinces are recognized.
1192 C. L. Valero Fajardo and J. R. Hechavarría Hernández
Ecuadorian environmental legislation, through the Unified Text of the Secondary
Environmental Law (TULAS in Spanish), establishes regulations (maximum permis-
sible limits) for different uses of water. Despite the existence of the regulations, the
aforementioned in the identification of environmental problems and risk situations
related to air and water pollution disavow the non-rigid application of the standard-
ization registered in Ecuadorian environmental legislation.
Productive Economic Scenario. In this scenario, the risk factors that orbit the pro-
ductive economic sector of the Canton Vinces are recognized. The economy and
production are linked in a sector economically dependent on agricultural production,
land communication routes are a very important factor and a critical image of the
quality of its tracks is presented. Quality of the tracks: 12% of asphalted roads, 13% of
dirt roads, and 51% of ballast tracks. The roads in good condition at the date of the
lifting of the information are detailed in this way, 11% of the paved roads, 5% land, 8%
of the ballast roads.
Social and Cultural Stage. In this scenario are recognize the risk factors that would
limit the social and cultural development of the Canton Vinces. It is distinguished that
there are 4 factors that due to their high percentage within the population are presented
as warning indicators in the social part. 62% of the Canton Vinces does not enjoy the
potable water service, 86.35% of the population has an absence of a sewage system, a
high level of digital illiteracy is denoted in 94.5% of the inhabitants.
Technological Scenario. In this scenario are recognize the factors that would limit the
technological development of the Canton Vinces. The limitation in access to tech-
nology is a critical condition. The 9.3% of homes have access to fixed telephony and
2.9% have access to the Internet.
3 Results
The following table details 18 risk factors in the territorial planning of the Canon
Vinces that are managed.
The result of the analysis carried out in the development of this research is pre-
sented with the ordering of the identified risk factors; which are recorded in Table 1
and the design of the diffuse cognitive map in Fig. 1. This will allow to analyze the
influence of the guide model for decision making and application of actions in the
Territorial Planning of the canton Vinces.
The comparison of the importance between the criteria to derive the local weight
vector using the own value approach, was a direct influence for the guide model of
decision making and application of actions through the diffuse cognitive map, resulting
from the integration of the knowledge acquired through the research process carried out
in this work.
PEST Analysis Based on Fuzzy Decision Maps 1193
Table 1. Management of risk factors in Territorial Planning of canon Vinces.
Risk factor Description
X1 Territorial planning
X2 Threat to biodiversity
X3 Lost of human lives
X4 Economic losses
X5 Proliferation of diseases
X6 Droughts
X7 Floods
X8 Climate change
X9 Water contamination
X10 Air pollution
X11 Non-compliance with regulations
X12 Low coverage of the drinking water network
X13 Low sewer network coverage
X14 Scattered Populación
X15 Low agricultural production
X16 Low Quality on the tracks
X17 Digital Illiteracy
X18 Devaluation of cultural identity
Fig. 1. Diffuse cognitive map
1194 C. L. Valero Fajardo and J. R. Hechavarría Hernández
4 Discussion
In conclusion, the PEST analysis based on diffuse decision maps for the management
of risk factors in the territorial planning of the canton of Vinces gives an improvement
in decision making and choice of actions during Territorial Planning. Through the
investigation is recognized that the inefficient identification of risk factors against
natural phenomena, together with the inefficient normalization of urban growth and the
spatial segregation of the population are causes of the increased risk of flooding in the
urban area of the canton Vinces.
The floods have as an effect, damage to the infrastructure due to excessive humidity
that sometimes causes the collapse of these infrastructures, leading to the existence of
human and economic losses. A causative factor of the increased risk of flooding in the
urban area of Canton Vinces is deforestation.
The deforestation cause desert soils and due to urban growth there is a change in
land cover. The conversion of natural soils into urban lands means the removal of
natural vegetation cover and its replacement with asphalt, cement or other industrial
materials. Everything discussed above is the cause of global warming of the planet.
On the other hand, as a result of the floods, pathogenic transmissions occur which
result in different types of diseases that lead to a high percentage of human losses. The
increase in the risk of flooding in urban areas, has the effect of mobility problems,
which is followed by the suspension of work activities which results in the loss of
habitability in the canton Vinces. The construction of the city automatically implies a
change in the original ecological and environmental systems.
Acknowledgments. To the Programa of Master in Architecture with mention in Territorial
Planning and Environmental Management. Faculty of Architecture and Urbanism of the
University of Guayaquil.
References
1. Cabeza, A.: Ordenación del territorio en América Latina. Scr. Nova. Rev. Electrónica Geogr.
y Ciencias Soc. 6, 105–132 (2002)
2. Colorado, B., Fois, M., Leyva, M., Hechavarría, J.: Proposal of a technological ergonomic
model for people with disabilities in the public transport system in Guayaquil, pp. 831–843.
Springer, Cham (2020)
3. Leyva, M., Hechavarría, J., Batista, N., Alarcón, J., Gómez, O.: A framework for PEST
analysis based on fuzzy decision maps. Espacios 39(16), 13 (2018)
4. Zambrano, C., Hechavarría, J., Leyva, M.: Multicriteria analysis in the proposed environ-
mental management regulations for construction in Aurora, Guayas, Ecuador, pp. 101–113
(2020)
5. Groumpos, P.: Fuzzy cognitive maps: basic theories and their application to complex systems,
pp. 1–22 (2010)
Model for Urban Consolidation of Informal
Human Settlements Based on Cooperation
Systems and Human Participation
in Guayaquil, Ecuador
María Milagros Fois1,3(&), Karen Sellan2, Karla Moscoso1,
and Maria Ruiz1
1
Faculty of Architecture and Urbanism, University of Guayaquil. Cdla.
Salvador Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{maria.foisl,karla.moscosog,maria.ruizri}@ug.edu.ec
2
Architect of the Municipaly of Guayaquil Calle Sucre entre calle Chile y calle,
Pedro Carbo, Ecuador
[email protected]
3
Faculty of Engineering, Industry, and Construction, University Laica Vicente
Rocafuerte, Av de las Américas, Guayaquil 090514, Ecuador
[email protected]
Abstract. This work is the product of an investigation that aims to develop a
model that allows the consolidation of the informal human settlement Monte
Sinahí, located in the northeast of the city of Guayaquil - Ecuador. The structure
of the methodological process is carried out in two phases: in Phase 1 it consists
in carrying out an exhaustive process of research and selection of methodology
based on participatory methods, application in informal urban contexts of Latin
America. Phase 2 consists on the design of diagnostic processes and urban
model for the consolidation of informal human settlements in Guayaquil, based
on the study of participatory activities carried out.
Keywords: Urban model Participatory design Urban planning Social
architecture
1 Introduction
The morphological structures of Latin American cities mostly present similar urban
patterns, including the accelerated expansion of the peripheries, residential occupation
of areas destined for agricultural or reserve use, which are defined as a series of
processes typical of the informal urbanization [1]. Guayaquil, since its inception was
considered one of the main ports of Ecuador, limited by the Guayas River and the
different estuaries that surround it, due to this condition, it has suffered migratory waves
countryside – city [2], which has generated that its consolidation is linked to the
constant illegal occupation of the land, producing a dispersed and fragmented city [3].
In 2000, Mount Sinahí emerged as an informal settlement on agricultural land,
while in 2010 population growth and urban expansion accelerated due to the approval
of the Law on Land Tenure Legalization (Law 88). This law allowed the National
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1195–1200, 2020.
https://doi.org/10.1007/978-3-030-39512-4_182
1196 M. M. Fois et al.
Government to intervene in the territory, with the power to initiate a process of
planning, adjudication and legalization of housing in the company of the Municipality
of Guayaquil. Currently, it is considered the largest informal human settlement in the
country with approximately 133,000 inhabitants and a territorial extension of 781 ha.
From this article, it is intended to generate a model of urban consolidation through
participatory design tools and urban land planning tools, the latter, stipulated from the
legal field, established by the National Constituent Assembly of the Republic of
Ecuador from the year 2016.
This work is developed based on a methodology in which different variables are
analyzed to identify how the study sector is developed, generating results ranging from:
the identification of legal and urban deficiencies, determination of the desired urban
image and established, developed participatory workshops and finally the delivery of
projects to the regulatory body. Thus concluding with an applicable system for the
future process of consolidation of informal human settlements.
2 State of the Art
The phenomenon of uncontrolled urban expansion around the peripheries of cities in
Latin America begins with a low density in the possession of the territory, which is
exacerbated as the lots or land are densifying outside of all urban regulations approved
by the regulators of municipal councils. Commonly these areas of land expansion have
been legally decreed for agricultural or ecological conservation uses, which will then be
sold illegally, which results in the conservation of rural to urban land use within the peri-
urban territory being understood as a irreversible problem by local governments [4].
In relation to the above, it is necessary to initiate a consolidation process whose
function is to strengthen the urban structures between the formal city and the new
informal areas settled in the peripheries of the same, thus ensuring coherence between
the use of the soil and public space.
It is necessary that this consolidation process includes an interdisciplinary partic-
ipatory design methodology used as a mechanism to meet the user’s needs, through the
collection of information, which will allow to design based on the results obtained.
Robert Chambers, defines participatory design as a tool that comes from an
anthropometric methodology that aims to identify the needs and patterns of a com-
munity that is detailed in his article entitled “The Origin and Practice of Participatory
Evolution”, methods Providers of the Rapid Rural Assessment (RRA) used in the
1970s for qualitative research [5].
Among the contributions of architects and urban planners, the following authors
and their methodologies stand out:
Christopher Alexander, architect and mathematician, proposed a method that makes
a design procedure available to the inhabitants through the identification of patterns.
The method is divided into two parts: the first in the identification of patterns, con-
sidering the scales and degrees of complexity, the second a set of combinable rules,
studying activities and places.
Jan Gelh, urban architect proposes, analysis sheet as tools to study the spaces the
behavior of the human being in the public space, based on five parameters that identify
the problems in relation to: the heritage value of the study sector, the public space
equity, mobility and accessibility, informal trade, the human scale [6].
Model for Urban Consolidation of Informal Human Settlements 1197
Randolh Hester, landscape architect, in his article “12 Steps for Participation”,
proposes strategies for work within communities, through the analysis of human
mobility and participation proposed the Sacred Plan of Manteo, North Carolina - States
United, urban movement that generated optimal results for the local economy.
Henry Sanoff, landscape architect, proposes in his book GAMES a methodology
based on a series of cards for urban and architectural evaluation using tools such as
observation, workshops that will identify problems and make proposals together with
the community [7].
The importance of participatory design is reflected in the creation of a link between
professionals and locals, constituting a fundamental axis in the consolidation processes
for the construction or reconstruction of the habitat. Generating local empowerment
will benefit citizen participation in decision-making by breaking with the traditional
schemes of conceiving architecture and urbanism.
3 Methodology
The methodological process was carried out through the analysis of the physical,
spatial, social and economic variables that are developed within the territory of study,
the same that belongs to the informal human settlement Monte Sinaí, located northeast
of the city of Guayaquil, which has a territorial extension of approximately 778 ha.
The methodological structure carried out for the realization of this research is
developed in two phases and three stages explained below:
Phase 1. Documentary research on participatory methodologies for urban consol-
idation processes: stage 1 consists in carrying out an exhaustive process of research and
selection of methodology based on results parameters, application in informal urban
contexts of Latin America, impact in the communities and modes of application. Once
the methodologies have been selected, step 2 will proceed and consists of carrying out
an evaluation procedure for the identification of methods that can be extrapolated to the
reality of Guayaquil and the evaluation and estimation of the operational levels.
Phase 2. Proposal: it consists of the design of diagnostic processes and urban model
for the consolidation of informal human settlements in Guayaquil, based on the study
of existing participatory methods. Stage 3 or final was established with the objective of
evaluating and analyzing the new methodology by applying it to a real case study,
involving academic, governmental and private entities.
4 Results
In Ecuador, territorial actions can be promoted by public and private entities.
Responsible for interventions within the urban soil, in the case of the city of Guayaquil,
is the Very Illustrious Municipality of Guayaquil (MIMG), while private initiatives
may be represented by the leaders of the cooperatives, inhabitants of the sector, private
companies, among others. The model that was obtained as a result of the research
(Development Model) is based on the interest of the inhabitants of the Human
Settlement.
1198 M. M. Fois et al.
The following section establishes the structured results according to the method-
ology implemented, using as a legal basis for the approach of the Model for urban
consolidation of informal human settlements based on cooperation systems and human
participation in Guayaquil, Ecuador, the article 4 of section 14-B of the LOOTUGS
where the land consolidation treatment is stipulated [8].
The model is subdivided into 3 stages, the first one is where the need is established
as an initiative of the inhabitants to acquire rights associated with the city (IP), in the
case of Monte Sinaí, the community carries out it. The second stage is the realization of
an urban project, which can contemplate a partial plan or an intervention polygon,
according to Title III of the LOOTUGS, Land use planning and management, Chapter
2, Section 3: Instruments of Urban land planning [8]. The third stage is the delivery,
evaluation and approval by the regulatory entity. Once the process has begun and the
interest of the community has been established (E1), stage 2 begins, which is subdi-
vided into subprocesses:
1. Establish the initial condition of the land in relation to the legal process (current
state), taking as a case study the largest Informal Human Settlement in the country,
Mount Sinai, the specific sector of study refers to the immediate context of the 1-EB
and 2-MS tracks located in the Ebenezer and Monte Sinaí Cooperatives respectively
located in the largest Informal Human Settlement in the country, Monte Sinaí.
A. Legal: Current status.
Monte Sinaí was included in the urban land of Guayaquil in 2000, it has 3 of 11
neighborhoods approved by the Illustrious Municipality of Guayaquil and the
remaining 8 are in the process of legalization, but not consolidation.
B. Physics: current state
Basic physical-spatial studies are carried out for the analysis of the site,
including the following as shown in Table 1:
1. Determination of the desired urban image and established by the consecutive
variables:
Traditional characteristics of the inhabitants, interconnections that promote the mix
of uses. Building Regulations in relation to what exists (height, withdrawals, vol-
umetric standards, COS and CUS).
2. Development of participatory workshops as a generator of an interactive urban
design. The following parameters were consider for the workshops:
A. Information from the community that participates.
Profile of people to participate where age, socioeconomic profile, interests, rela-
tionship with the territory, provenance, among others are identified. Working hours
of the people to participate.
B.Workshops (5)
Presentation of proposals and their respective process of elaboration, establishment
of logistics, letters of commitment of those involved and by final determination of
the working groups. Work table for urban design with a schedule established on
weekends during the course of a month. Proposal of the preliminary proposal and
discussion of the details. Socialization of the proposal and signing of documentation
by the community representatives. Proposal delivery to the regulatory entity for
consideration and subsequent approval.
Model for Urban Consolidation of Informal Human Settlements 1199
Table 1. Site analysis. Source: self-made.
Table 1. Site analysis. Source: self-made.
Analysis Mappings Description
Natural Condition: To- Image 1: Topography mapping
and: typological analysis of the
pography, soil type, study sector. Source: self made
hydrography, climate
and protected areas.
Social Condition: Densi- Image 2: Typological analysis
of the study sector. Source: self
ty, housing, land use, made
development process.
Economic Condition: Image 3: Land use mapping of
the study sector. Source: self
State services and inter- made
connections.
Interconnection Condi- Image 4: Bus route mapping
within the study site. Source:
tion: Infrastructure and self made
traf-fic.
5 Conclusions
The purpose of these conclusions is to establish the importance of creating models of
integration of informal human settlements into urban plots based on complex participa-
tory parameters that affect comprehensively and not just morphologically. The problems
of the sector in general are a consequence of the lack of regulation, integration and
consolidation of informal settlements. The proposed model is based on the initiative of the
inhabitants of the sector and integrates the collaboration between educational entities of
3rd and 4th level, as developers of urban activations and, finally, they involve municipal
regulatory entities for the approval of urban proposals. The creation of consolidation
models for informal human settlements is a broad field of study, particularly in the case of
the city of Guayaquil, where planning is an early stage process. In 2016, with the approval
of the LOOTUS, new instruments and soil treatments were created for the affectation of
management and management of the Ecuadorian cities, with this step the processes of
regulation, legalization and acquisition of rights began in Guayaquil through settlement
consolidation mechanisms.
1200 M. M. Fois et al.
Acknowledgments. To the Faculty of Architecture and Urbanism of the University of
Guayaquil for the great support, not only with the assaying time for making this research possible
but specially for the financial aid.
References
1. Zambrano, C., Hechavarría, J., Leyva, M.: Multicriteria analysis in the proposed environ-
mental management regulations for construction in Aurora, Guayas, Ecuador. In: Ahram, T.
(eds.) Advances in Artificial Intelligence, Software and Systems Engineering. AHFE 2019.
Advances in Intelligent Systems and Computing, vol. 965. Springer, Cham (2020) https://doi.
org/10.1007/978-3-030-20454-9_10
2. Hechavarría, J., Bermeo, P., Forero, B., Portilla, Y.: Social Inclusion: A proposal from the
University of Guayaquil to design popular housing with citizen participation. In: 2018 World
Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC),
Albuquerque, NM, USA, pp. 1–5 (2019). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=
&arnumber=8629778&isnumber=8629581
3. Forero, B., Hechavarría, J., Alcivar, S., Ricaurte, V.: Systemic approach for inclusive design
of low-income dwellings in popular settlements at Guayaquil, Ecuador. In: Ahram, T., et al.
(eds.) Human Systems Engineering and Design, IHSED 2018. AISC, vol. 876, pp. 1–5.
Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-030-02053-8_93
4. Bazant, J.: Procesos de expansion y consolidacion urbana de bajos ingresos en las periferias.
Bitacora 13, 17 (2008)
5. Gehl, J., Svarre, B.: How to Study Public Life. Island Press, Washington, D.C. (2010)
6. Hidalgo, D., Delgado, A.: Diseño Participativo como herramienta para el desarrollo de
proyectos de comunidades rurales del sector Samborondon. Casos: San Jose y Barraca
Central. Universidad Espiritu Santo-Ecuador, Guayaquil (2019)
7. Sanoff, H.: Desing Games. Tokyo (2017)
8. Lootugs, A.N.: Ley Organica de Ordenamiento territorial uso y gestión de suelo. Quito (2016)
Systemic Approach to the Territorial Planning
of the Urban Parish La Aurora,
Daule, Ecuador
Cyntia Alava Portugal(&), Jesús Rafael Hechavarría Hernández,
and Milagros Fois Lugo
Faculty of Architecture and Urbanism, University of Guayaquil. Cdla.
Salvador Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{cyntia.alavapo,jesus.hechavarriah,
maria.foisl}@ug.edu.ec
Abstract. The territorial planning of the urban parish La Aurora, in the canton
Daule, has been characterized by the agricultural production of rice and other
primary products. In recent years, this contribution to the economy has been
affected by the increase in urban development that has considerably reduced the
agricultural sector. The inadequate territorial planning has caused, among other
negative aspects, an important environmental impact product of to urban
development and vehicular chaos due to the migration of the upper middle class
to the closed urbanizations from La Puntilla to La Aurora. The present work
proposes a territorial planning under a systemic approach where the interaction
between the urban development, economic growth and environmental pollution
is analyzed. The objective is search of an adequate urban development with
alternatives for a modern agricultural development and initiatives for integral
management of natural resources that allow sustainable agricultural production.
Keywords: Systemic approach Territorial planning Sustainable agriculture
Environmental impact
1 Introduction
Ecuador, a country located on the equator in South America, has a population that
exceeds 16 million inhabitants, with an area of 256,370 km2, and where the Land Use
is a planning instrument and a high public policy to the category of constitutional norm
[1]. The Territorial Planning and Environmental Management of the urban parish La
Aurora, of the canton Daule, has been marked by agricultural land and its remarkable
rice production, as well as, an important interregional connection especially of primary
products, the same as in the Nowadays it is affected because it is not enough to cover
production costs [2]. “Ecuador stands out for its potential to implement infrastructure
due to its new constitution and subsequent legislation, as well as its prosperous
economy, the growth of its urban centers and the increase in environmental quality
expectations of its growing middle class” [3].
While it is true that agriculture was the basis of the economy of this canton, it is
urban development that has displaced the agricultural sector, for which it is of great
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1201–1205, 2020.
https://doi.org/10.1007/978-3-030-39512-4_183
1202 C. Alava Portugal et al.
importance to study the interaction between urbanization, economic growth and
environmental pollution, since the lack of participation in the planning has resulted in
the growth leaving without alternatives a main route, the same that is congested gen-
erating chaos. It should be noted that the conurbation between Guayaquil, Daule and
Samborondón, is due to the migration of the upper middle class to the urbanizations
closed in the urban route of La Puntilla and La Aurora, whose model causes envi-
ronmental, economic and social impacts. There are about 50 thousand people, more
than 90% of Guayaquil, who have migrated to that area that became a satellite city for
the set of several urbanizations with all services (Fig. 1).
Fig. 1. Population growth in the city of Guayaquil, 2010–2018
Accelerated urban growth makes it difficult to carry out land use planning, which is
an important measure to minimize losses in crop production. The idea of urban
planning today takes into account the agenda of almost all countries where the increase
in urban population has created a series of new demands that force the public power to
adopt concrete measures to meet these demands. The urban environment is a place
where different social actors live in constant conflict, especially if we take into account
an accentuated economic relevance factor such as urban land [4].
Accordingly, it is necessary to raise awareness when updating development plans
and territorial planning (PDOT) in order to make improvements when it comes to
planning. Currently, sustainable urban development is being sought, promoting green
urbanizations, as well as proposing alternatives for modern agricultural development
and initiatives for the integral management of natural resources such as soil and water
that are key to sustainable agricultural production. Faced with this challenge, territorial
planning, within the paradigm of Sustainable Development, seeks a rational organi-
zation of the territory according to its capacity, in such a way that it allows the
responsible consumption of resources, the optimal development of human activities
and the conservation of nature.
The configuration of cities is the product of a social process created and maintained
by the complex interrelation of economic, social, political and cultural factors,
Systemic Approach to the Territorial Planning 1203
however, in recent times, cities have sought their own limits under the framework of
urban aspects such as the growth of infrastructure, functional size, its expansion and
relationship with the rural environment, irregular settlements and social housing [5].
2 Methodology
The parish La Aurora of the canton Daule of the province of Guayas. Daule has an area
of 7,793.28 ha. and it is located to the north with Santa Lucía, to the south with
Guayaquil, to the east with Salitre (Urbina Jado) and Samborondón and to the west with
Nobol and Lomas de Sargentillo. Geographically it is between the coordinates: Latitude
1° 52 ′S Longitude W 79° 59′00′ ′W. It has a population of 120,326 inhabitants, 87,508
in urban areas and 32,818 inhabitants in rural areas (INEC, 2010) (Fig. 2).
Fig. 2. Geographic location of the urban parish La Aurora. Source: GAD Daule
To study the case, we can take as an example the city of Amsterdam, which under
the criteria of governance, cultural and lifestyle, its channels were redesigned, giving
way to a new layout for its territorial ordering in response to the population increase.
likewise this demand generates the resolution of the traffic, using as a means of
transport the bicycle and a garden city with green spaces.
3 Results
In Fig. 3 we can see in red the urbanized area of the La Aurora parish of the Daule
canton since 2015 and how it has been expanding rapidly to the present (2019). Many
of the rural agricultural lands (TAR) dedicated to the cultivation of rice and livestock,
have already been urbanized, as well as part of the protected area (hills) that in some
cases have become quarries.
1204 C. Alava Portugal et al.
Fig. 3. Map of the La Aurora parish of the Daule canton, expansion of the urban area. (A) Rice
crops in the floodplain year 2015 (Source: MAE-MAGAP). (B) Urbanized agricultural land
(TAU) year 2019 (Source: Google Earth).
4 Conclusions
The process of significant change in the morphology of the parish La Aurora, began
especially about 15 years ago with the urbanization of livestock farms in Samborondón
and continues until today. In this period the lands around the city have been absorbed
by the expanding urban area. The institutional efforts for the planned urban develop-
ment of La Aurora have not proved adequate for the rapid growth of private urban-
izations. The implications of urbanization in the study area require immediate attention.
Integrated urban land use planning is the best option. Ecological security must be
guaranteed through the management of agricultural land, the designation of resources
and areas of sensitive land, the establishment of buffer zones, the protection of cultural
resources and the conservation of open spaces. On the other hand, farmers must par-
ticipate in the elaboration and execution of a rural development program based mainly
on agricultural processing companies. Integrated development planning will create
mutually beneficial relationships between urban and rural areas, economic development
and urban development. Modern agriculture is characterized by incorporating science
and technology to be more efficient, which helps in saving economic resources and
time so that there is commercial openness for the growth of the economy and improves
the productive profile.
References
1. Gonzalez, M.J., Chuquiguanga, C.: Development and territorial planning in Ecuador at
cantonal scale planning zone 6 case study. Revista de la Facultad de Arquitectura y
Urbanismo de la Universidad de Cuenca. 7(13), 160–179 (2018)
Systemic Approach to the Territorial Planning 1205
2. Zambrano, C., Hechavarría, J., Leyva, M.: Multicriteria analysis in the proposed environ-
mental management regulations for construction in Aurora, Guayas, Ecuador. In: Ahram, T.
(eds.) Advances in Artificial Intelligence, Software and Systems Engineering. AHFE 2019.
Advances in Intelligent Systems and Computing, vol. 965. Springer, Cham (2020)
3. Serra-Llobet, A., Hermida, M.A.: Opportunities for green infrastructure under Ecuador’s new
legal framework. Landsc. Urban Plan 159, 1–4 (2017)
4. Araujo, M.E.: Las competencias urbanísticas y la planificación metropolitana en el estado
autonómico español y en el federalismo brasileño”. Boletín Mex. Derecho Comp. 46(138),
879–907 (2013)
5. Villegas, E., Contreras, D., Cifuentes, J., Fernández, L.: J. Technol. 14(2) (2015). Universidad
El Bosque
Systemic Approach to Strategic Tourism
Planning in the Cantonal Capital of Bahía de
Caráquez, Sucre, Ecuador
Milton Zambrano(&) and Jesús Rafael Hechavarría Hernández
Faculty of Architecture and Urbanism, University of Guayaquil. Cdla.
Salvador Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{milton.zambranoch,jesus.hechavarriah}@ug.edu.ec
Abstract. Bahía de Caráquez is a tourist destination since the 80s for its
excellent cuisine and shrimp boom. It is currently a city abandoned by massive
migration because of two devastating earthquakes above 7 on the Richter scale
in the years 1998 and 2016 respectively. This reality has caused precarious
conditions due to recurrent seismic movements in recent years. The coastal
territory has a rich diversity in natural resources, bordering islands and estuaries
with various tourist attractions that have a high economic value. This area is
influenced by changes in land and sea, so that its coasts are highly vulnerable to
environmental phenomena. The objective of the present investigation is to
design a strategic plan for the renovation of the tourist infrastructure under
earthquake-resistant criteria, in addition to the reactivation and potentialization
of gastronomic tourism in Bahía de Caráquez of Canton Sucre beach. The
systemic approach will make it possible to define the guidelines of the strategic
plan for the development and socioeconomic reactivation through the quanti-
tative and qualitative analysis of the collected statistical data. The spatial and
descriptive analysis of the data obtained in the field proposes a functional and
sustainable city model for both local residents and visitors through the gentrified
tourism conversion model. The proposal of infrastructures and tourist nodes will
contemplate the location of geological faults as well as the identification of
critical and favorable points for the establishment of poles of socioeconomic
development.
Keywords: Strategic planning Systemic approach Tourism Ecuador
1 Introduction
The cantonal capital of the Sucre canton is Bahía de Caráquez, a small city located in
the center of the Ecuadorian coast in the Province of Manabí on the shores of the
Pacific Ocean. As the name implies, it is a small bay located at the mouth of the most
important river in this province, the Chone River. All geographical factors give it an
impressive scenic variety. This is how Bahía de Caráquez enters Manabi gastronomy,
distinguished by its variety and national and international appreciation. The preparation
of the different dishes is very special and a large part of them are based on seafood,
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1206–1212, 2020.
https://doi.org/10.1007/978-3-030-39512-4_184
Systemic Approach to Strategic Tourism Planning 1207
bananas, peanuts, cheese and fine herbs, thus becoming an impressive gastronomic
proposal that attracts locals and foreigners.
The theoretical model of the life cycle of a destination developed by Butler [1] is
the frame of reference that is applied here for the analysis of the evolution of tourist
spaces. Although its model is simple, it is the most used. It studies the behavior of the
demand and the carrying capacity of the tourist centers, considering the positive
relationship between the increase in the number of visitors and the tourist development
of the area in terms of accommodation, organization and ownership of tourist attrac-
tions and services; sociocultural, economic and environmental impacts; visitor satis-
faction, among others [2]. In accordance with the technological approach to
infrastructure, a constructive system model was used. To reinforce the renovation,
seismic heatsink structure reinforcement technology was used with the partially
encapsulated steel column beam construction system connected by end plate [3].
A sustainable tourism is one that in its social dimension includes the issues that
frame the well-being of the local population, the conservation of cultural heritage and
the participation of the community in the tourism planning and development process
[4]. Regarding the economic dimension, it takes into account aspects related to the
long-term viability of tourism as an economic activity. It is about taking advantage of
and consolidating tourism employment, the direct and indirect effects of the activity;
Identify and differentiate the destination by a memorable experience and a brand of its
own [5].
2 Strategic Plan Proposal
In the present work, the proposal of a strategic plan focused on the renovation of
infrastructures under parameters of earthquake-resistance and tourism-gastronomic
reactivation is carried out. Therefore, the information that identifies the tourist areas
with the risks due to the frequency of earthquakes must be analyzed. The Fig. 1 shows
the frequency with which earthquakes occur in the Ecuador in recent years.
Fig. 1. Seismic risk areas in Ecuador. Source: Ecuadorian building code.
1208 M. Zambrano and J. R. Hechavarría Hernández
The seismic risk areas in Ecuador were considered as base information (Fig. 2).
Fig. 2. Seismicity rate of the last 17 years. Source: Geophysical Institute of the National
Polytechnic School.
Among the specific objectives an organized structure of tourism-gastronomic
management is planned and guidelines are established for the implementation of these
construction systems. In addition, a map of geological faults is generated contemplating
existing infrastructure and tourist nodes.
Within the territorial planning there are scales of intervention of the realities, in
which the strategic planning has a capacity for immediate action when acting on specific
problems and managing it dynamically in the territory in accordance with the continuous
action of intrinsic actors. It is known as micro planning, since its actions alter realities in
favor of society and its development. They work together with macro planning by
articulating with it naturally by generating expansion gaps in the planning phase.
The resilience to earthquakes demonstrates the level of development of a society
[6]; According to this, a socioeconomic development plan must be executed imme-
diately, so the promotion of a new technological-gastronomic approach brings inno-
vation and a timely response to the problem of infrastructure deficit for tourism
purposes and guarantee gastronomic tourism to reach the socioeconomic development
within strategic planning.
3 Methods
Your cover letter is acceptable, but your current condition shows the opposite. In order
to develop the technological-gastronomic approach, the article is structured with a first
review section of the existing literature in relation to the development and management
of tourist destinations. Methodologically, different municipal tourism plans, reports and
policies, and statistical data published by both the Ministry of Tourism of Ecuador and
Zonal Coordination 4 - Pacific are analyzed.
Systemic Approach to Strategic Tourism Planning 1209
Quantitative/qualitative methodologies are used with the method of rapid data
collection (Interviews, finds, background, among others), and the method of spatial and
descriptive analysis of the data obtained in the field. To analyze the parameters of the
transformation of Bahía de Caráquez, hemerographic sources and public documents are
taken into account, which allow us to understand their evolution and urban changes.
Subsequently, data from secondary sources on tourism activity, use of beaches,
accommodation, occupancy and population are analyzed. The observation and the
opinion of neighbors involved allow to interpret in depth the situation of the coastal
area and the natural disasters that have occurred. Their choice follows criteria of
representativeness and knowledge of the tourism, social and urban sector: represen-
tatives of the platform of neighbors affected by the earthquake, responsible for hotels
and restaurants.
Also interviews with tourists, chosen at random, offer information about their
perception of the attractions of the coastal area. In this case, the questions are aimed at
the satisfaction of public infrastructure and services, and the offer of accommodation
and gastronomic resources. All the mentioned sessions took place in tourist nodes
(Puerto Amistad, Hotel la Piedra, Parque Sixto, Durán Ballén and Basic Education
Center Pío Mon-tufar) in the months of August to December 2019, with a maximum
duration of sixty minutes, time to collect the opinions of the participants and consider
an eventual saturation of the information. Finally, the study of documents, strategic
plans and uses of the local administration and studies of other academics [7] on the
satisfaction of the residents were discussed.
In the methodological procedures to express the results, spatial analysis was used
with the ARCGIS tool, to generate a vulnerability plan highlighting geological faults,
hotel and restaurant infrastructures and tourist nodes. According to the descriptive
analysis, the guidelines for design of earthquake-resistant construction systems [8] and
organized structure of tourism-gastronomic management were proposed. A qualitative
methodology is presented according to the foundation of environmental governance
management [9] (Fig. 3).
Fig. 3. Map of seismogenic knots near the city of Manta and Bahía de Caráquez. Source:
Geophysical Institute of the National Polytechnic School.
1210 M. Zambrano and J. R. Hechavarría Hernández
To analyze the parameters of the transformation, hemerographic sources and public
documents are taken into account, which allow the evolution and urban changes to be
tracked. Subsequently, data from secondary sources are analyzed in terms of tourism
activity, use of beaches, accommodation, employment and population. The participant
observation and the opinion of the neighbors involved allow to interpret in depth and
detail the situation of the coastal area and the natural disasters that have affected it from
individual and semi-structured interviews. Their choice follows criteria of representa-
tiveness and knowledge of the tourism, social and urban sector. The representatives of
the platform of neighbors affected by the earthquake, responsible for hotels and
restaurants. The questions encompass aspects related to the current tourism situation;
with the social, cultural and urban effects of tourism activity; loss of identity; the
changes that favor tourists and economic revitalization; and the sociodemographic
repercussions. Also interviews with tourists, chosen randomly, offer information about
their perception of the attractions of the coastal area. In this case, the questions are
aimed at your satisfaction in terms of public infrastructure and services, and the offer of
accommodation and gastronomic resources (Fig. 4).
Fig. 4. Map of urban post-earthquake voids.
Systemic Approach to Strategic Tourism Planning 1211
4 Results
The identification of areas of seismogenic origin is required as a first instance, since this
information is of paramount importance for mapping the area most prone to telluric
movements around the study area. The collection of data is importance to specify the
monitoring of main tectonic tendencies; earthquake distribution, seismicity rate and a
fault plane solution for major earthquakes; In addition, the type of faults that prevail
and characterize these areas. These results are effective for the evaluation of the seismic
threat of the study area for the anti-seismic structures of socioeconomic development
projects. Urban areas with tourist attraction are undergoing major transformations. The
purpose is to project a functional and sustainable city model for both local residents and
visitors, which opts for the gentrified tourism conversion model.
5 Discussion
The revival of tourism with a technological-gastronomic approach in the coastal area
through the design of a strategic plan for the renovation of infrastructure under
earthquake-resistance parameters, a multidisciplinary framework is presented where
several actors participate in order to reactivate tourism in Bahía de Caráquez, such as:
citizen participation, private sector investment incentives, public policy management,
local and national interest to reactivate this vulnerable sector. The promotion of a new
technological-gastronomic approach brings innovation and a timely response to the
problem of infrastructure deficit for tourism purposes and the rise of gastronomic
tourism to achieve socioeconomic development and which is within the scope of
strategic planning.
Acknowledgments. To the Program of Master in Architecture with mention in Territorial
Planning and Environmental Management. Faculty of Architecture and Urbanism of the
University of Guayaquil.
References
1. Butler, R.: Tourism and the environment: a geographical perspective. Tour. Geogr. 2(3), 337–
358 (2000). https://doi.org/10.1080/14616680050082553
2. Butler, R.: The concept of a tourist area cycle of evolution: implications for management of
resources. Can. Geogr./Le Géogr. Can. 24(1), 5–12 (1980). https://doi.org/10.1111/j.1541-
0064.1980.tb00970.x
3. Zhao, G., Zhang, Y., Cao, F., Di, H.: Experimental research on sismic performance of PEC
column-steel beam frame connected by endplate. J. Build. Struct. 50–57 (2017)
4. Bibri, S.: Backcasting in futures studies: a synthesized scholarly and planning approach to
strategic smart sustainable city development. Eur. J. Futur. Res. 6(1), 13 (2018). https://doi.
org/10.1186/s40309-018-0142-z
5. Ritchie, J., Lewis, J. (eds.): Qualitative Research Practice. A Guide for Social Science
Students and Researchers, 336 Seiten. Sage, London/Thousand Oaks/ New Delhi (2003).
ISBN 0–7619-7110-6
1212 M. Zambrano and J. R. Hechavarría Hernández
6. Alhamidi, Pakpahan, V.H., Simanjuntak, J.: Analysis of tsunami disaster resilience in Bandar
Lampung Bay Coastal Zone. In: IOP Conference Series: Earth and Environmental Science,
vol. 158, p. 012037 (2018). https://doi.org/10.1088/1755-1315/158/1/012037
7. Sellares, M.A., Azpelicueta, M.C., Sánchez-Fernandesz, M.: Turismo sostenible: lugareños
frente a turistas. El caso de la ciudad de Barcelona. Holos 3, 331 (2015). https://doi.org/10.
15628/holos.2015.2809
8. Fnais, M.S., Abdelrahman, K., E-Hady, S., Abdel-monem, E.: Seismicity and seismotectonics
of the Jeddah area, Saudi Arabia, pp. 219–232 (2013). https://doi.org/10.2495/ERES130181
9. Crespi-Vallbona, M., Mascarilla-Miró, Ó.: La transformación y gentrificación turística del
espacio urbano. El caso de la Barceloneta (Barcelona). EURE (Santiago), 44(133), 51–70
(2018). https://doi.org/10.4067/s0250-71612018000300051
Cognitive Rehabilitation for Autism Children
Mental Status Observation Using Virtual
Reality Based Interactive Environment
Tamilselvi Dhamodharan1, Manju Thomas1,
Sathiyaprakash Ramdoss2, Karthikeyan JothiKumar1,
SaiNaveenaSri SaravanaSundharam1(&),
BhavaniDevi Muthuramalingam1, NilofarNisa Hussainalikhan1,
Sugirtha Ravichandran1, VaibhavaShivani Vadivel1, Pavika Suresh1,
Sasikumar Buddhan3, and Ajith Madhusudanan3
1
Cognitive Science Research Group, Department of Information Technology,
Thiagarajar College of Engineering, Madurai 625 015, India
[email protected],
[email protected]
2
Department of Doctoral Studies and Advance Programs, College of Education,
Ashland University, Ashland, OH 44805, USA
3
Kewaunee Labway India Pvt. Ltd., Bengaluru, India
Abstract. Proposed Cognitive Rehabilitation Therapy is to observe autism
children mental status using virtual reality based interactive environment.
Interactive environment is designed to analyze three levels of cognitive
impairment decreasing functioning such as attention, reasoning, emotion, social
behavior, decision making and language understanding. All the three levels
were designed for mild autism children to have virtual walk through using HMD
(Head Mount Device). We modeled a house and the behavior of the autism
children were analyzed with day to day activities with the simple query to
observe children language understanding. Observing the 5 autism children (3
boys and 2 girls in the age between 7 to 11) with the support of therapist gives
promising improvement over the behavior in the virtual interactive environment
and enhance their cognitive rehabilitation. Each level the therapist repeat the
level, wherever children not responding. Observation results gives the children
interact with virtual world with happy and positive emotions. In all the levels we
incorporated children native language (Tamil Language) for the instruction,
query and appreciation while they interact well. In future it is proposed to track
their face, eye ball movement and predict the mental status of the children in
detail to improve their overall mental status of autism children.
Keywords: Autism children Rehabilitation Virtual reality Mental state
observation Assessment
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1213–1218, 2020.
https://doi.org/10.1007/978-3-030-39512-4_185
1214 T. Dhamodharan et al.
1 Introduction
Autism is diagnosed when a child or adult has abnormalities in a ‘triad’ of behavioural
domains: social development, communication, and repetitive behaviour/obsessive
interests. Autism is a spectrum disorder and children have very different abilities and
skills [1, 2]. Theoretical backbone of cognitive rehabilitation originates from Luria’s
theory of brain organization, which emphasizes integrated brain function [3]. In the
rehabilitation of children with autism therapeutic issues (e.g. eye contact, joint atten-
tion, turn taking, reading mental states and emotions) are usually addressed in con-
strained teaching sessions [4]. Autism children feel comfortable in predictable
environment, and enjoy interacting with computers [5, 6]. Computer-based interactive
learning used in therapy and education have improve the students learning as well as
skills [7, 8]. Virtual Environment research in autism includes some of the most
quintessential aspects of the human experience: from understanding others’ intentions
(theory of mind) to managing everyday tasks (multitasking).
Children with autism fail to develop the ability to think about mental states in the
normal way, and thus fail to understand behavior in terms of mental state [9]. Theory of
mind, executive function of autism children mind have direct affinity with VR Tech-
nologies [10]. The immersive environment allows the student to train and develop
different social situations in a structured, visual and continuous manner [11]. ToMM
(Theory of Mind Machines) develops between 2 and 4 years of age, and allows pretend
play [12], understanding of false belief [13], and understanding of the relationships
between mental states [14]. Three commonly found traits involve abnormal response
for autism children are input stimuli, lack of human interaction, and the inability to
interact with people and objects for their needs [15].
Virtual environment are interactive, virtual image displays enhanced by special
processing such as auditory and haptic, to convince users that they are immersed in a
synthetic space [16–19]. Virtual environments can be simplified to the level of input
stimuli tolerable by autism children [20]. Visual and auditory effects make the autism
children to interact emotionally [21, 22] . Virtual Reality environment immerses autism
children in synthetic world will exhibit different behaviors [23]. Computers play a vital
role in children language therapy [24, 25]. VR environment, haptic technology added
with computers in the imaginary world plays a major role in the rehabilitation therapy
such cognitive, behavioral and social interaction for the autism children [26]. Tech-
nologies can support for human therapist and replace the therapy also.
Autism children immersive into virtual environments and the task created will
stimulate the visual sensory information and create the movement automatically and
change the behavior of children [27, 28]. Fear, anxiety and basic day to day activities
were handled with the help of VR to improve the psychology based activities designed
to improve the therapy, learning and playing activity of children [29–31]. VR empower
the autism children cognitive disabilities [32]. HMD (Head Mount Device) along with
VR changes in real time in response to the children movement. Iwata says, bodily
movement is coordinated with visual images [33]. A more lifelike model can be
experienced along with the walking movements [34]. Additional devices like HMD
used should be evaluated for physiological effects of patients [35].
Cognitive Rehabilitation for Autism Children Mental Status Observation 1215
The proposed VR environment for autism children rehabilitation for the mental
state observation was designed with 3 levels. First level designed for Attention and
Reasoning. Second level with emotion and applied behavior analysis. Third level with
decision making and language understanding. Autism children observed for the mental
state were assessed by the therapist. Mild and Medium level autism children were taken
for pilot study and based on the cognitive and behavior state of the mental state, trials in
the VR environment was given and improvement is assessed by the therapist for the
children cognitive rehabilitation.
2 Proposed Methodology
Figure 1 shows the proposed method to observe the mental state of autism children for
3 levels. Mild and medium level autism children were considered for the rehabilitation
trial. The virtual environment is created with 3 levels. Autism children interact with
Virtual Environment through HMD or Leap motion device based on the convenient to
interact for the 3 levels.
Fig. 1. VR environment for cognitive, occupational and evidence based therapy
2.1 First Level – Cognitive Behavioral Therapy
The proposed first level concentrates on the autism children observation. The lack of
communication skills and difficulty in handling day to day routine tasks were designed
in the VR environment as shown in Fig. 2. The above tasks were created in VR and
interaction through the environment by the children using HMD/Leap motion device.
Based on the children behavior the mental state of the children were observed.
Attention, Recognition and Reasoning during interaction act as a measure to identify
the levels of cognitive therapy. Based on the levels, the therapy was repeated as per the
therapist recommendations. 3 mild level and 2 medium level children were considered
for the 3 levels of therapy.
Once the children performance in the first level shown the remarkable results and
enhancement in learning while going of cognitive behavior therapy, we can make the
children to enter in to the second level of Therapy. Mild autism children interact with
1216 T. Dhamodharan et al.
Fig. 2. Cognitive behavior therapy for autism children
VR using HMD. Medium level children find difficult to coordinate with HMD. Leap
motion device with hand free movement is given for the medium level autism children.
2.2 Second Level – Occupational and Music Therapy
Second level music therapy will stimulate the children emotions and interest to deal
with musical instruments and occupational therapy for the children to apply their
knowledge. While the children walkthrough using leap motion device, wire free
environment makes them to touch the various instruments designed in VR environ-
ment. While hearing the music they can touch and create the sounds in different forms.
Occupational Therapy various color balls in the environment are randomly distributed
and children can apply the color, shape identified in the first level and apply the
learning in second level and make the similar group of color balls or shapes. Mild and
medium children struggles and takes time to do the task in their own. Repeated trials
will shows the significant improvement over the second level.
2.3 Third Level Decision Making and Language Understanding
Third level VR environment is designed to make the autism children to take decision
and language understanding in Tamil and English Language. A house model was
created in VR and make the child to explore their day to day life things used and the
purpose with the voice added. It improves the language understanding and do the task
as per the instructions given in the VR environment.
3 Conclusion and Future Works
Based on the observation of 5 autism children at the 3 levels, mild level children are
interested in all 3 levels and do well the activities designed in the VR environment.
Medium level autism children one child hesitate to use HMD Device and interact only
through Leap motion device. In the second level, music therapy some children play the
Cognitive Rehabilitation for Autism Children Mental Status Observation 1217
instruments randomly and one mild level children only hear the music and doesn’t
initiate interaction in the VR environment. Third level language description medium
level autism children find difficulty in English language. All the observations are made
in the presence of therapist. In future, the proposed work will be integrated in the
Avatar character in Augmented Reality for all the three levels to initiate the interaction,
mimic the activates and encourage the children to do the task. If the children hesitate,
Avatar will engage them with common songs and actions and decrease the distraction
and create interest for the student to interact with Avatar and perform the above 3 levels
of activities. This will promisingly increase the mental state of autism children for all
three levels and improve their life skills.
Acknowledgement. Work is funded by DST – CSRI (Department of Science and Technology –
Cognitive Research Initiative) Project (Sanction No. DST/CSRI/2017/304 dated 12.4.2018). Our
sincere thanks to Sparks Vidhyalaya Autism Special School for observing the Autism Children
observation during observation studies.
References
1. American Psychiatric Association: DSM-IV Diagnostic and Statistical Manual of Mental
Disorders, 4th edn. American Psychiatric Association, Washington DC (1994)
2. I.C.D-10: International classification of diseases, 10th edn. World Health Organization,
Geneva (1994)
3. Luria, A.R., Yudovich, F.Ia.: Speech and the development of mental processes in the child.
In: Kovasc, O., Simon, J. (eds.) Penguin, Harmondsworth (1971)
4. Howlin, P., Baron-Cohen, S., Hadwin, J.: Teaching Children With Autism to Mind-Read.
Wiley, Hoboken (1999)
5. Moor, D.: Computers and people with autism. Communication 20–21 (1998)
6. Powell, S.: The use of computers in teaching people with autism. Autism on the agenda.
Papers from a National Autistic Society Conference, London (1996)
7. Hershkowitz, V.: How adults with autism utilized their computers. Advocate - Newsletter of
the Autism Society of America Inc., November–December 1997
8. Hershkowitz, V.: Computer based therapy for individuals with autism. Adv. Mag. (2000)
9. Baron-Cohen, S., Tager-Flusberg, H., Cohen, D.J. (eds.): Understanding Other Minds:
Perspectives from Autism. Oxford University Press, New York (1994)
10. Rajendran, G.: Virtual environments and autism: a developmental psychopathological
approach. J. Comput. Assist. Learn. 29(4), 334–347 (2013)
11. Lorenzo, G., Lledó, A., Pomares, J., Roig, R.: Design and application of an immersive
virtual reality system to enhance emotional skills for children with autism spectrum
disorders. Comput. Educ. 98, 192–205 (2016)
12. Leslie, A.M.: Pretence and representation: the origins of theory of mind. Psychol. Rev. 94,
412–426 (1987)
13. Wimmer, H., Perner, J.: Beliefs about beliefs: representation and constraining function of
wrong beliefs in young children’s understanding of deception. Cognition 13, 103–128
(1983)
14. Wellman, H.: Children’s Theories of Mind. Bradford/MIT Press, Cambridge (1990)
15. Rutter, M.: Diagnosis and definition of childhood autism. J. Autistic Child. Schizophr. 8,
1938–1961 (1978)
1218 T. Dhamodharan et al.
16. Hodges, L.F., Kooper, R., Meyer, T.C., Rothbaum, B.O., Opdyke, D., De Graaff, J.J.,
Willford, J.S., North, M.M.: Virtual environments for treating the fear of heights. IEEE
Comput. 28(7), 27–34 (1995)
17. Durlach, N.I., Mavor, A.S. (eds.): Virtual Reality: Scientific and Technological Challanges.
National Academy Press, Washington DC (1995)
18. Burdea, G.C., Coiffet, P.: Virtual Reality Technology. Wiley, Hoboken (2003)
19. Stanney, K.M. (ed.): Handbook of Virtual Environments: Design, Implementation and
Applications. Lawrence Erlbaum, Mahwah (2002)
20. Strickland, D.: Virtual reality for the treatment of autism. In: Riva, G. (ed.) Virtual Reality in
Neuro-Psycho-Physiology, pp. 81–86 (1997)
21. Park, D., Youderian, P.: Light and number: ordering principles in the world of an autistic
child. J. Autism Child. Schizophr. 4, 313–323 (1974)
22. Grandin, T., Acariano, M.: Emergence: Labeled Autistic. Arena, Novata (1986)
23. Gregory, R.L.: Seeing by exploring. In: Ellis, S.R. (ed.) Pictorial Communications in Virtual
and Real Environments, pp. 328–337. Taylor and Francil, London (1991)
24. Colby, K.M.: Computer aided language development in non-speaking children. Arch. Gen.
Psychiatry 19, 641–652 (1968)
25. Zarr, M.: Computer-aided psychotherapy: machine helping therapist. Psychiatr. Ann. 24(1),
42–46 (1994)
26. Barfield, W., Hendrix, C.: Factors affecting presence and performance in virtual environ-
ments. In: Interactive Technology and the New Paradigm for Healthcare, pp. 21–28. IOS
Press, Washington D.C. (1995)
27. Wann, J.P., Turnbull, J.D.: Motor skill learning in cerebral palsy: movement, action, and
computer-enhanced therapy. Bailliere’s Clin. Neurol. 2(1), 15–28 (1993)
28. VREPAR Project. http://www.ehto.be/ht_projects/vrepar/movement.html
29. Strickland, D.: A virtual reality application with autistic children. Presence: Teleoperators
Virtual Environ. 5(3 Summer), 319–329 (1996)
30. Buckert-Donelson, A.: Heads up products: virtual worlds ease dental patients. VR World
3(3), 6 (1995)
31. Gourlay, D., Lun, K.C., Lee, Y.N., et al.: Virtual reality for relearning daily living skills. Int.
J. Med. Infomatics 60, 225–261 (2000)
32. Greenleaf, W.J.: Rehabilitation, ergonomics, and disability solutions using virtual reality
technology. In: Interactive Technology and the New Paradigm for Healthcare, pp. 415–422.
IOS Press, Washington D.C. (1995)
33. Kenyon, R.V., DeFanti, T.A., Sandin, D.J.: Visual requirements for virtual environment
generation. J. Soc. Inform. Disp. 3, 211–214 (1995)
34. Rowley, T.W.: Problems of head mounted display. In: IEE Colloquium on Real World
Visualization – Virtual World – Virtual Reality, pp. 12/1–4. IEE, London (1991)
35. Nakajima, R., Nomura, J., Fukushima, S., Ojika, T.: A relax/refresh system applying virtual
reality. In: Proceedings: 3rd IEEE International Workshop on Robot and Human
Communication, pp. 145–150. IEEE, NewYork (1994)
Eye Control System Development
and Research of Effects of Color of Icons
on Visual Search Performance Based
on the System
Jiaqi Cui1, Yafeng Niu1(&), Chengqi Xue1, Xijiang Cai1, Yi Xie2,
Bingzheng Shi3, and Lincun Qiu1
1
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], {nyf,ipd_xcq}@seu.edu.cn,
[email protected], [email protected]
2
Science and Technology on Electro-Optic Control Laboratory,
Luoyang 471023, China
[email protected]
3
Shanghai Academy of Spaceflight Technology, Shanghai 201109, China
[email protected]
Abstract. Traditional interactive approach which has been developed very
mature has evolved from mouse control to many other forms. Eye controlled, as
one of new interaction form, will help people with physical disability to
manipulate devices as normal users. Meanwhile, because of its natural and high-
efficiency, eye controlled system could be the future trend not only for the
disabled but also ordinary people. This study proposed an eye controlled system
frame to help researchers and designers to explore eye control system interface
design because eyes are the only real-time input medium for eye control system.
This study firstly analyzes and summarize the difference of eye-control system
and traditional interface through surveys and literature reviews. Then according
to the uniqueness of eye-controlled interaction, an eye control interactive logic
algorithm is developed through the Tobii API combined with the Unity API.
Experiments investigated the effects of color combination of icon on visual
search performance were done based on the proposed system.
Keywords: Eye controlled system Color Icon Interface design
1 Introduction
The development and advancement of technology have made the interactive approach
evolved from the well-known mouse-controlled approach to many other forms, such as
voice interaction, gesture interaction, brainwave interaction, and eye-control interac-
tion. In order to pursue convenience and neutrality of information, traditional inter-
action methods are increasingly unable to satisfy us. The study of eye control
technology could not lack the analysis of advantages and disadvantages of eye control.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1219–1224, 2020.
https://doi.org/10.1007/978-3-030-39512-4_186
1220 J. Cui et al.
Eye is the most important tool for people to obtain information, about 80%–90% of
the information can be obtained through the visual channel. And the visual channel has
the characteristics that other channels cannot have: directness, naturalness, bidirec-
tionality and so on [1]. Although the application of eye-controlled interaction still has
many limitations, once it is rationally utilized and developed, it will greatly shorten the
difficulty for people to obtain information and further improve human-computer
interaction.
2 Related Work
2.1 Eye Controlled System Interface Design Guidelines
Eye controlled technology needs new approach that is different from the conventional
interface driven by the keyboard and mouse. In order to design a more suitable
interface for eye controlled system, it is important to understand the differences
between eye gaze and traditional user interface design. Design guidelines of eye
controlled system were created for designers in Nell Chitty’s paper [2]. One major
differences the author mentioned is that the target in eye controlled interface must be
larger in comparison to the traditional interface to offset the lack of precision which eye
gaze causes [3]. Larger targets are easier to hit [4]. As for the layout of interface, the
targets are better when spaced in the center and upper area of the screen. They should
be placed sufficiently scattered to avoid Midas contact. There are other guidelines such
as text size and background color could also be used in eye controlled system devel-
opment (Figs. 1 and 2).
Fig. 1. Targets in eye controlled system must be larger.
Fig. 2. Targets in eye controlled system are better placed in the center part of the screen.
Eye Control System Development and Research of Effects of Color of Icons 1221
The system proposed in this paper is based on these guidelines. Using these rules
could help developers and designers avoiding repetitive work or impossible hypothesis.
2.2 Eye Controlled System Development
The early eye movement equipment mainly starts with the simplest text input to realize
the communication between the disabled and the outside world.
Hansen of the Technical University of Denmark and Skovsgaard of the University
of Copenhagen have developed a StarGazer [5] system by means of eye control
interaction. By translating and scaling continuously, the user can more accurately
obtain the location he or she is interested in. The system effectively expands the letter
interval and reduces the interference of noise.
Simple communication is realized at the same time, the deeper communication of
users is also gradually reflected. Spanish eye movement researchers have designed a
browser for the disabled [6] where users can move the cursor up and down by moving
their eyeballs up and down and select buttons by blinking their eyes. In the input
interface, the intrusive system of EOG improves the accuracy of users and greatly
improves the efficiency of using the browser (Fig. 3).
Fig. 3. Frame of proposed system
3 System Development
3.1 Framework of Eye Controlled System
1. Module 1: Gaze input
The gaze input module is also called channel input module, which is specifically
referred to eye control interaction. The module is mainly composed of a Tobii eye
tracker and a computer equipped with a WINDOWS 10 system. Before the sight is
input, the eye tracker needs to be placed in a fixed position in advance for subse-
quent eye tracking and sight calibration.
2. Module 2: Obtaining gaze point through eye tracker
This module mainly converts the user’s input information into actual parameters
with clear meanings - also can be said that we need or may need experimental
parameters, such as the position of the eye gaze point, the size of the pupil and the
1222 J. Cui et al.
distance between the two eyes; these factors are not only affected It is also disturbed
by other factors such as eye trackers and other factors such as ambient light.
3. Module 3: Sight calibration
The line-of-sight calibration module is primarily a calibration software provided for
the Tobii Eye Tracker and can be calibrated based on software knowledge.
4. Module 4: Real-time feedback
The real-time monitoring and feedback module is a unique module in the eye-
controlled interactive system. Because of the instability of the single-channel input
of the eye control system, if the eye tracker fails to capture the gaze properly, data
loss occurs. The system needs to be immediately detected and given some feedback
to the user. At this point the system can open the calibration path, allowing the user
to perform a recalibration operation.
3.2 Software and Hardware Development Environment
This research mainly uses Tobii X2-30 eye tracker, which has the advantages of,
portability, and easy assessment of sight. It is convenient for us to build interactive
scenes in the process of studying eye movement interaction. The use of Tobii’s native
interactive model and SDK development kit gives us the possibility to use the eye
tracker for secondary development. The relatively high precision Tobii eye tracker
guarantees the key point of eye movement interaction accuracy.
3.3 System Function
The eye control interaction logic algorithm is developed by Tobii API combined with
Unity API. The algorithm includes the redefinition of gaze interaction and eye
movement interaction. The logical results of gaze interaction mainly include interface
conversion, switching of different directories under the same interface and switching of
different UI at the same location.
4 Experiment Methods
Based on the proposed system, developers and researchers could use their own
equipment to find out the index they would apply in their system or research. In this
study, an experiment aims to find out the effect of saturation of icon on visual per-
formance would be set as an example.
Pre-experiment aims to find out icons with same comprehensibility. Experiment 1
was conducted to investigate the visual performance of different color saturation of icon
on dark background. In this study, color saturation was the only variable.
4.1 Subjects
10 females and 10 male college aged between 23–26 participated in the experiment. All
of them had normal or corrected to normal vision.
Eye Control System Development and Research of Effects of Color of Icons 1223
4.2 Stimuli
The stimuli in the experiment consisted of 4 (different types of icons) 4 (different
order) 4 (levels of color saturation, S1: 0%–25%, S2: 26%–50%, S3: 51%–75%, S4:
76%–100%). The reason for selecting four levels of saturation was that it is nearly
impossible to prove a saturation with a specific figure is the best. The background color
used in the experiments was grey of which the value of R is 29, G is 30 and B is 31,
corresponding to dark mode in Mac Os interface design.
4.3 Procedure
In experiments 1, subjects would complete participants had to gaze on “Start” on the
screen for 1 s to start the trial. Subjects were asked to remember the target icon which
was shown on the center of screen for 2 s. The target icon is white so that participants
could not be affected by the color they saw. After the target disappeared, a cross mark
in the center of the screen would be presented for 1 s to make sure that subject’s sight
would focus on the center of the screen. Then, the stimulus array was presented in the
center. Subjects searched for the target item and gazed on it for 2 s. the search time and
accuracy rate would be recorded by computer (Figs. 4 and 5).
Fig. 4. Background color in S1 interval.
Fig. 5. Background color in S4 interval.
1224 J. Cui et al.
5 Conclusions
In this study, Tobii eye tracker is used to acquire eye information and with Unity
programming, the interface of eye control system is created. It is also necessary to
consider humanization research in human-computer interaction, such as whether the
execution logic of the interface system conforms to the habits of most users, whether
the layout and color matching in the interface design are suitable.
Acknowledgments. This work was supported jointly by National Natural Science Foundation
of China (No. 71801037, 71871056, 71471037), Science and Technology on Electro-optic
Control Laboratory and Aerospace Science Foundation of China (No. 20165169017), SAST
Foundation of China (SAST No. 2016010), Equipment Pre research & Ministry of education of
China Joint fund, Fundamental Research Funds for the Central Universities of China
(No. 2242019k1G023).
References
1. Jacob, R.J.K., Karn, K.S.: Eye tracking in human-computer interaction and usability research:
ready to deliver the promises. Mind 2(3), 4 (2003)
2. Kumar, M., Winograd, T.: GUIDe: gaze-enhanced UI design. In: Extended Abstracts
Proceedings of the 2007 Conference on Human Factors in Computing Systems, CHI 2007,
San Jose, California, USA, 28 April–3 May 2007. DBLP (2007)
3. Räihä, K.-J., Aula, A., Majaranta, P., et al.: Static visualization of temporal eye-tracking data.
In: Proceedings of Human-Computer Interaction - INTERACT 2005, IFIP TC13 International
Conference, Rome, Italy, 12–16 September 2005. Springer (2005)
4. Komogortsev, O., Holland, C., Camou, J.: Adaptive eye-gaze-guided interfaces: design &
performance evaluation. In: CHI 2011 Extended Abstracts on Human Factors in Computing
Systems. ACM (2011)
5. Hansen, D.W., Skovsgaard, H., Hansen, J.P., et al.: Noise tolerant selection by gaze-
controlled pan and zoom in 3D. In: Proceedings of the 2008 Symposium on Eye Tracking
Research & Applications, pp. 205–212. ACM (2008)
6. Lledo, L.D., Ubeda, A., Lanez, E., et al.: Internet browsing application based on
electrooculography for disabled people. Expert Syst. Appl. 40(7), 2640–2648 (2013)
How to Improve Manufacturing Process
Implementing 5S Practices: A Case Study
Beata Mrugalska(&), Monika Konieczna,
and Magdalena K. Wyrwicka
Faculty of Engineering Management, Poznan University of Technology,
ul. Strzelecka 11, 60-965 Poznan, Poland
{beata.mrugalska,magdalena.wyrwicka}@put.poznan.pl,
[email protected]
Abstract. This paper presents the practical implementation of 5S method,
which is used to improve the quality and safety in organization. 5S, which
comes from the Japanese words (translated in English: sort, set, shine, stan-
dardize and sustain), is a practice based on Lean Management system which
brings many advantages to the organizations and employees. The practice shows
that the activities of 5S methods can be effectively and easily implemented to
industry. The main aim of this paper is to describe the implementation of 5S
method at a chosen workplace and present its positive impact on quality, safety
and ergonomic improvement.
Keywords: Lean Management Manufacturing process 5S method
1 Introduction
Nowadays, business environment forces enterprises to search constantly for the new
opportunities and chances in order to fulfill the customers’ demands as well as being
more competitive. Providing the safe and ergonomic environment at the workplace is
one of the basic task of employer [1]. In order to be more competitive on the market,
organizations have to develop continuously, so they search for new sources of quality
improvement and decide to implement different methods, techniques and tools [2, 3].
For example, they introduce rules, concepts and tools of lean [4, 5]. A special feature of
this philosophy is the indication of the direction of improvements based on defining
what is an added value and what is waste taking into account human factor [6]. 5S is
one of Lean Management methods, based on continuous development, which helps to
increase productivity, reduce non-value activities and at the same time improve quality.
This tool, more and more widely used in various organizations is no longer just a good
practice, but a significant instrument for managing the company [7]. 5S is an acronym
from the Japanese words which describe five phases. In order to get all possible benefits
it is essential that activities in each stage are performed in correct way, but also that
they are constantly maintained and improved [5]. Therefore, 5S method is the process
of reorganization of the whole organization in order to improve different aspects, but
mostly quality and safety at the workplace.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1225–1232, 2020.
https://doi.org/10.1007/978-3-030-39512-4_187
1226 B. Mrugalska et al.
2 Implementation of 5S Method at the Workplace
The 5S tool is recognized as an easy improvement system performance, however, in
order to achieve it the training to the staff is required. Therefore, the first issue before
implementing 5S in the organization is to let all the employees understand the meaning
and significance of this method. They should be able to think about potential
improvements which can help them in performing daily work and at the same time
understand how their actions will influence the whole production process [9]. The main
goal of 5S is to make and also sustain, tidiness and discipline at the workplace. The
implementation of 5S method is one of the basis in the creation of such working place
which helps in environmental activities and continuous improvement in order to
increase effectiveness of organization. The advantages of its implementation are
increased productivity, reduced lead time, downtime and time of repairing machines,
cycle time, time of dies, elimination of unnecessary activities, such as searching for
tools and equipment, increased quality by reduction of the amount of mistakes and
disadvantages, standardization of processes and activities, maintained savings by
reducing stock in warehouses, increased safety at the workplace and the ergonomics at
workplace.
5S system reduces waste, increases effectiveness and helps the employees to work
efficient by eliminating unnecessary activities, such as searching for tools, waiting for
starting the machine or changing the dies. The important aspect of the implementation
of 5S method is that it can be used in different enterprises from various branches and
every employee can take part in it. However, its implementation can meet some barriers
such as [10]: lack of interest of top management and their inability to promote team
working initiatives, resistance from the employees, no time for implementation, lack of
support from management and training, problems with communication between
employees and lack of availability of resources.
Therefore, it is important to early define the barriers of 5S implementation. The
management should create such an atmosphere in which the employees can discuss the
method, propose new ideas and solutions in order to improve the process. The 5S
method should be used in enterprise as a tool for better organization at the workplace
and employees should feel that the discipline and the application of the adopted rules
will improve the quality and safety of their work. It consists of 5 action steps which
should be taken into account [11, 12]:
(1) Sort - the first stage of 5S method and the main goal of it is to collect all
unnecessary items which are not needed to perform daily activities. This stage
enables identification of unneeded and unused equipment, old work instructions,
data and information which are not needed at the moment. This way of cleaning
and tidying helps to maintain cleaned and ergonomic workplace. According to 5S
method all items, tools and equipment which are not needed at the moment should
be thrown away or taken from the workplace and marked with red label. This
label can be used to explain the reason of throwing the item, date and other
information.
(2) Set - is based on organization of all elements of work station such as materials,
tools, instructions, documents in the way to be easily found and put back in the
How to Improve Manufacturing Process Implementing 5S Practices 1227
right place after using. The implementation of appropriate visual control tools
allows to eliminate many types of waste. The basic tool in this step is visual
control, so all solutions used in the work environment, are aimed at a quick and
clear presentation of the way of working, the status of currently performed
activities and control such as clearly defined storage areas (lines, plates, labels,
storage areas), color marking, arrow marking, marking of maximum and mini-
mum levels for production materials, for finished products and goods, and so-
called shadow boards. On the shadow board, each tool is placed on a designated
place, which allows to identify any of them and implement the routine of taking
and putting back each tool.
(3) Shine - helps to establish the frequency and methods of cleaning the workplace
and also to set up the standard of right cleaning and needed resources. All
employees should be involved in this process in order to take care about own
workplace. The main idea of this stage is to identify the sources and reasons of
dirtiness and then eliminate them, keep regular maintenance and clean the
workplace. This stage of 5S method should be included in the scope of daily
duties of the operator and performed by him personally.
(4) Standardize - aims to set standards for implementation of the first three steps of
the 5S approach. This step is also defined as a key technique to achieve
repeatability of implemented changes improving the workplace. The main idea is
to create detailed instructions for performing all activities in the scope of the first
three steps of 5S method. The standards and procedures must be clear, simple and
easily understandable what leads to need of engagement of the employees who
perform these tasks. They are able to create these documents understandably,
introduce the changes on spot and constantly improve them.
(5) Sustain - the last step, which is often considered as the most difficult one, requires
development of new habits which are often a long-term process that needs a lot of
effort. Such changes refer to organizational culture which requires an involvement
of the organization’s management. The main issue is more about creating habits
for observing the implemented improvement solutions as well as the ability to
work in accordance with the created and accepted standards. The most important
tools supporting the fifth step of the method include job audits. Such audits are
aimed primarily at identifying departures in the functioning of workplace from
assumed standards and, in the case of non-compliance, determining the necessary
corrective actions. If they are performed regularly and according to a set schedule
and a specific audit card, they allow for quick discovery of irregularities and
determination of the necessary activities for their elimination on time indicating
responsible people for taking them.
3 Case Study
The investigation and implementation of 5S method was done in a milling workstation
in Polish production company.
1228 B. Mrugalska et al.
3.1 Sort
Sort is the stage of identifying objects at the workplace and in its surroundings. It refers
to two main categories: items necessary for work and unnecessary items. All unnec-
essary and useless items must be removed from the workplace and its surroundings. In
order to facilitate the definition of the necessary tools, a strategy of red labels can be
applied (Fig. 1).
Fig. 1. Exemplary 5S red tag for workstation and its practical implementation.
In order to use the tags appropriately, it is needed to establish a standard that could
clearly define:
– a red label – for items not used in the last 30 days,
– no red label – for items used in the last 30 days.
Another important task is to determine the designated place where these elements
with a red card will be stored. It facilitates access to these items if necessary. After-
wards, the result of red tagging should be documented to let the organization measure
the introduced improvements and savings due to this process.
3.2 Set in Order
Set in order refers to the placement of objects on the stand and in its surroundings to
improve the appearance of the workplace. It is important that the employee is free to
move around his workstation and locate needed items such as tools or instruments near
How to Improve Manufacturing Process Implementing 5S Practices 1229
the place of their use. Before 5S implementation on its shelves there were containers
with joint elements such as screws, nuts, rivets etc. as well as other company’s tools
and equipment. They were chaotically arranged and therefore, the access to them was
difficult. The labels were also missing. The solutions are provided in Fig. 2.
Fig. 2. Arranged stillage and place for tools and equipment for milling workstation [13].
During the processing of the details at the milling machine, depending on the type
of material and its dimensions, waste (shavings and filings) is generated. Therefore,
labeling was introduced to keep everything in its designed location and arrange its
storage area (Fig. 3).
Fig. 3. Labelled containers for waste storage [13].
Moreover, a cork board was purchased and hang over the milling machine. It
resulted from the fact that at this workstation documentation or technical drawings were
sometimes lost or found at various places in the plant. Now all documents such as
product, control and technical drawings are sorted out and have their designated place.
1230 B. Mrugalska et al.
3.3 Shine
Cleanliness of the work area ensures more comfortable and safer workplace. Everyone
is responsible for it and to be able to perform correctly it is necessary to assign clean
zones and schedules, methods, cleaning tools and materials. In the analyzed work-
station it was achieved through the combination of the cleaning function and defect
detection. Each cleaning of the machine was automatically recognized as an inspection.
Its systematical cleaning aimed at identifying and locating problems such as leak. It
was also noticed that the dirty tools or instruments can have an influence on quality.
For example, filings getting under the caliper overestimated the measured values.
Therefore, the employee had to pay attention to the cleanliness of tools and instruments
at the workplace and after each use, they should be cleaned.
A cleaning equipment organizer was also introduced. It resulted from the fact that
the employee had to borrow the cleaning tools from another workstation to be able to
clean up. It happened that after finishing the cleaning, the employee simply forgot
about giving it back. The arranged place is shown in Fig. 4.
Fig. 4. Cleaning equipment organizer [13]
An arranged cleaning corner was located behind the milling machine next to the
bookcase. There were a vacuum cleaner, a broom, a scoop, a washing liquid, a con-
tainer with dry cloths and a waste bucket. Instead of using a shadow table, it was
decided to describe the objects and place the subtitles directly above the given element.
The machine had covers to counteract the possibility of touching the hand or other
part of the worker’s body, or other undesirable elements with its rotating parts. It
helped to protect the environment against splashing of coolant and chips during
material cutting (Fig. 5).
3.4 Standardize
Standardization allows the first three steps to be an everyday and integral part of the
work. It means that they are included to specific job responsibilities that will be
implemented. In order to support the daily checklists, instructions, control charts and
guidelines were introduced. They facilitated to remember to perform the activities.
How to Improve Manufacturing Process Implementing 5S Practices 1231
Fig. 5. Prevention of splashing of coolant and chips to maintain the floor cleanness [13]
The suggested list of cleanness and maintenance of the machine, including a field for
comments, date, maintenance intervals and a responsible person is shown in Table 1.
Table 1. Cleanness control sheet
Scope of cleanness Details of cleanness Date and time Responsible Remarks
(underline in the column) person
Cleaning the machine
Cleaning the surroundings
Cleaning the machine
Cleaning the surroundings
3.5 Sustain
Sustain requires self-discipline to maintain consistent standards of quality, safety and
cleanliness. In order to achieved it, the effective use of P-D-C-A cycle is needed.
Moreover, the awareness of 5S should be built through retraining. As self-discipline is
a human dependable factor, it can be increased through appropriate system of rewards
and recognition of efforts of staff. For example, the introduction of bonus for fulfillment
of duties, may increase the motivation of employees to sustain self-discipline and fulfill
the pre-defined assumptions.
4 Conclusions
The implementation of 5S practices enhances operation effectiveness for a better and
safer working environment. It is a simple, understandable system, and it can be quite
easily accepted by employees. The implemented order and cleanliness in a short time
can give measurable benefits. The working environment becomes more effective and at
the same time more ergonomic and safe. However, before the implementation of this
1232 B. Mrugalska et al.
method some barriers may occur. That is why, it is important to explain employees how
their actions will contribute to the functioning of the whole organization and what are
the potential benefits. The presented aspects of the implementation of 5S method show
that this method is the first step to understanding and implementing broadly understood
Lean Management philosophy, and at the same time can provide a solid basis for
implementation successive management systems that improve safety, quality and
environmental aspects.
References
1. Henderson, s, Stuckey, R., Keegel, T.: Current and ceased users of sit stand workstations: a
qualitative evaluation of ergonomics, safety and health factors within a workplace setting.
BMC Public Health 18(1), 1374 (2018)
2. Górny, A.: Use of quality management tools to identify ergonomic non-conformities in
human activities. In: Karwowski, W., Trzcieliński, S., Mrugalska, B. (eds.) Proceedings of
the AHFE 2019 International Conference on Human Aspects of Advanced Manufacturing,
and the AHFE International Conference on Advanced Production Management and Process
Control, AHFE 2019, AISC, vol. 971, pp. 319–329. Springer, Cham (2020)
3. Smętkowska, M., Mrugalska, B.: Using Six Sigma DMAIC to improve the quality of the
production process: a case study. Proc.-Soc. Behav. Sci. 238, 590–596 (2018)
4. Bhasin, S.: Lean Management Beyond Manufacturing. Springer, New York (2015)
5. Wyrwicka, M.K., Mrugalska, B.: Mirages of lean manufacturing in practice. Proc. Eng. 182,
780–785 (2017)
6. Jasińska, S., Żurek, M., Wyrwicka, M.K.: Analiza efektywności wdrożenia Lean Manufac-
turing. Studium przypadku. Ekonomia i Zarządzanie 1(7), 311–327 (2015)
7. Kamińska, A.: Praktyczne zastosowanie 5s w wybranych fabrykach branży motoryzacyjnej.
Studia Oeconomica Posnaniensia 4(1), 134–144 (2016)
8. Veres, C., Marian, L., Moica, S., Al-Akel, K.: Case study concerning 5S method impact in
an automotive company. Proc. Manuf. 22, 900–905 (2018)
9. Omogbai, O., Salonitis, K.: The implementation of 5S lean tool using system dynamics
approach. Proc. CIRP 60, 380–385 (2017)
10. Singh, A., Ahuja, I.S.: Review of 5S methodology and its contributions towards
manufacturing performance. Int. J. Process Manag. Benchmarking 5(4), 408–424 (2015)
11. Fabrizio, T., Tapping, D.: 5S for the Office: Organizing the Workplace to Eliminate Waste.
Productivity Press, New York (2006)
12. Visco, D.: 5S Made Easy: a Step-by-Step Guide to Implementing and Sustaining Your 5S
Program. Productivity Press, New York (2017)
13. Nowaczewski, M.: Zastosowanie wybranych metod i narzędzi zarządzania jakością w
przedsiębiorstwie produkcyjnym. Unpublished bachelor thesis. Supervisor: Beata Mrugalska
(2017)
The Initial Stage of Development of a New
Computer Program for the Processing
of Psychophysiological Tests
Jelena Turlisova(&) and Anita Jansone(&)
Faculty of Science and Engineering, Liepaja University,
Liela Street 14, Liepaja, Latvia
[email protected], [email protected]
Abstract. Nowadays technologies tend to facilitate not only the learning pro-
cess itself but also analyzing the results of this process, in order to create a
detailed correction plan for every learner. Educational and study processes must
be appraising in the terms of the perspectives in diverse electronic technology
usage and development tendencies to interest contemporary youth in the
learning process. The aim of this study is to depict the meaning of the computer
programs in the processing of psychophysiological tests.
Keywords: Computer program Computer assessment Visual perception
Visual – motor integration
1 Introduction
Visual perception and understanding of the world are both active and coordinated.
Therefore, in order to ensure a prosperous learning process, visual signals must be
productively conceived, interpreted and integrated with the incoming information from
other human senses [1]. Today’s available study tools and technologies are extremely
diverse and still in continuous development process, but overall, the practice of text and
textual information displayed on the screen remain irreplaceable. Visual perception is a
common definition used to describe visual information processing techniques such as
visual analysis, visual-spatial and visual-motor integration, and hearing-vision inte-
gration. Visual-motor integration (VMI) describes the level of visual perception’s
coordination and body movements. The process itself embraces touch (sensory per-
ception) activating muscles for the smooth and productive performance of the given
task [2].
In the previous study, the Image J computer program was developed to process and
obtain the results of standard psychophysiological test DTVP-2 [3]. This program
allows analyzing the eye-hand coordination test results of the DTVP-2 test (Exercises
1-16). This program, in order to process the test results, needs the children’s
drawings/test pages to be scanned first and converted from PDF format to JPG format
and marked with colored dots in the beginning and end of the work field. Each task has
its Macros (.txt format). The obtained results enable us to estimate the degree of camber
of the line and its deviation from the optimal direction. In exercise from 1 to 6, the
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1233–1237, 2020.
https://doi.org/10.1007/978-3-030-39512-4_188
1234 J. Turlisova and A. Jansone
Macros-1 is used. Macros for each exercise can look up for colored dots in the image,
draw a line between these points on one side and the other, then it summarizes pixels
and divides with the length of the line to obtain a scale-independent variation coeffi-
cient. Results obtained from Macros enable the evaluation of the line incline and the
deviations from the optimal direction. The higher the number, the higher is inclination
from the desired line direction drawn by the child. Both vertical and horizontal lines
can be processed this way. It should be noted that, before using the drawing, it needs to
be prepared, that is, small red ovals must be put at the beginning and the end of the
particular drawing, and it has to be saved with these changes. This is necessary for the
program to work correctly. Images representing the results obtained can also be saved
to be used in further research and work however, the images should be renamed
differently in order not to lose output data. During the study and based on the results
obtained, the author concluded that this program can be used by various specialists who
use psychophysiological tests to screen and evaluate visual - motor and visual per-
ception both in preschool institutions, schools and other educational institutions [4].
Significantly, the computerized assessment provides considerably more accurate
and faster interpretation of results, which in turn allows professionals to devote more
time for the necessary correction work and development of specialized e-study mate-
rials. The technology world is rapidly developing and updating; therefore, the purpose
of this study is to develop a new computer program for psychophysiological tests,
based on the existing one.
2 The Purpose of the Program
The computer program designed to display on the screen of an electronic tablet (tablet
PC) the initial data of psychomotor tests, accept the drawing and its subsequent pro-
cessing with scoring and saving the results for later comparison. The program can be
very useful for assessing the development of psychomotor skills of children and
especially children with developmental problems.
The new computer program provided with moderate sound effects that help to fix
the child’s attention. The program has two options: to work on Android and Windows,
based on Microsoft Visual Studio 2017.
The program provides the following functions:
1. Enter the program by either log in or register. In the latter case, client data request
and password registration are performed. It is understood that the administrator
enters the program. It can be either a teacher or a doctor if the work is with a group
of children or a single user. All user data is stored in the program database. The
possibility of remote data storage on the server is also provided.
2. Choice of the contractor. This can be either a registered customer or a new one is
added. In single-user mode, the administrator as a client registers himself.
3. Choosing a series of tests. In the initial version, a series of tests is one, but a choice
is provided for further development.
4. Filing the initial conditions of the next test from the selected series to the screen,
accounting for the time spent and fixing the end of the test.
The Initial Stage of Development of a New Computer Program 1235
5. Processing of test results and their safety until the end of a series of tests.
6. Displaying the test results one by one with highlighting errors and, if this option is
enabled, comparing them with previous results stored in the database. This allows
you to visually see the evolution of customer results.
7. Adding and removing tests to a series of available in the program.
8. Drawing mode without stylus pen with reduced resolution.
The test tasks are based on work (DTVP-2) [1] in processing convenient for use on
the tablet screen. The program provides several groups of tests, at the moment they are
combined in one series.
2.1 The First Group of Tests
These are the tasks of connecting two points on the screen with a straight line. Two
points - the initial A and the final B, are located first horizontally, then vertically. The
test subject leads the stylus pen from point A to point B in a straight line either
horizontally or vertically. The drawing is considered complete when the stylus pen
crosses the location line of point B, either along the X coordinate or Y, depending on
the vertical or horizontal location. At this point, the program calculates deviations from
the straight line using several parameters.
These are the tasks of connecting two points on the screen with a straight line. Two
points - the initial A and the final B, are located first horizontally, then vertically. The
test subject leads the stylus pen from point A to point B in a straight line either
horizontally or vertically. The drawing is considered complete when the stylus pen
crosses the location line of point B, either along the X coordinate or Y, depending on
the vertical or horizontal location. At this point, the program calculates deviations from
the straight line using several parameters.
The first parameter is the total integral deviation I.
I ¼ S=x; ð1Þ
where S is the total area between the ideal straight line and the line drawn by the
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P
subject. It is calculated by the formula S x0 ðy y0 Þ2 , where y is the current
coordinate of the line, y0 is the coordinate of the ideal line. x is the total line length. All
units here and below are in pixels. In tests with vertical lines, the x and y coordinates
naturally change places. This parameter is quite general and describes the accuracy of
the task as a whole.
The second parameter is the maximum deviation of the coefficient of the direction
of the line kmax from the ideal (=0).
For each pair of points on the line drawn by the subject, kmax, the direction
coefficient is calculated using the formula, where
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
kmax ¼ max> x½ð ððy y0Þ2 ÞÞ=10ÞÞ ð2Þ
10- this is x − x0 since the points x are taken with an interval of ten pixels to
reduce small random errors (“pixel noise”). This parameter characterizes well the
1236 J. Turlisova and A. Jansone
degree of concentration of the subject, then how diligently he performs the task. If he
draws the line diligently and constantly corrects the direction, the spread to and,
consequently, to the max, will be small (<0.1). For a careless task, k max will be > 0.2.
The third parameter is D. It is calculated by the formula:
D ¼ I1 = I2; ð3Þ
where I1 is the integral deviation mentioned above for the first half of the line
x < xmax/2, and I2 for the second half of line x > xmax/2. This parameter charac-
terizes the stability of attention. Usually, I2 > I1, since towards the end of the test the
attentiveness weakens and the test tends to finish the task more quickly, which means
that D < 1.
Time can be considered as the fourth parameter, however, this parameter is very
variable. Here you need to keep in mind that the first time you complete the task, it will
always be more since the subject simply gets acquainted with the situation. The exe-
cution time can be taken into account when the number of tests performed is more than
2–3, and even then with great care.
Here is an example (Fig. 1) of one such task, executed on paper by the old method
(ImageJ) [3].
Fig. 1. The pattern for connecting dots.
Two stages of the drawing are visible. First, the subject incorrectly assessed the
direction to the finish point, then corrected it, but he searched and already at the end
completed the task. Below, the calculation of the coefficients is still according to the old
method
2.2 The Second Group of Tests
These are tests for repeating shapes (Fig. 2. such example).
Fig. 2. An example of a test for repeating shapes based on an open path.
The Initial Stage of Development of a New Computer Program 1237
Shapes can be between lines, in a frame, or without them. At the beginning of the
test, a figure is displayed that needs to be repeated a certain number of times. These are
more complex tests that require the use of complex matrix algorithms close to artificial
intelligence. One of the main questions here is whether the task can be considered
completed if the figure is similar to the original, but has a different scale, smaller or
larger. If geometric similarity solves, then image similarity algorithms can be decisive,
says the same Pearson cross-correlation. However, if the violation of scale can also be
considered an error, then the simple method shown in (Fig. 3) can give better results.
Fig. 3. A simple technique for comparing images based on open paths.
In this case, after the test is completed, the picture is not scaled, but the starting
point of the curve is simply combined with the starting point of the template, and the
endpoint of the curve is connected to the endpoint of the template. The standard
deviations and error fields S1 and S2 are calculated according to the procedure
described above for the first parameter of the linear test. More complex figures can only
be compared using matrix methods based on artificial intelligence technologies. I will
dwell on them in the next article.
Acknowledgments. The article is written with the financial support of European Regional
Development Fund project Nr.1.1.1.5/18/I/018 “Pētniecības, inovāciju un starptautiskās sadar-
bības zinātnē veicināšana Liepājas universitātē”.
References
1. Hammill, D.D., Pearson, N.A., Voress, J.K.: Measures visual perception and visual-motor
integration. In: Developmental Test of Visual Perception, 2nd edn (1993)
2. Schwartz, S.: Visual Perception: A Clinical Orientation, 4th edn, pp. 229–2414. McGraw Hill
Companies, New York (2009)
3. Turlisova, J.: Master work. Eye – hand coordination interaction with visual function (Acu –
rokas koordinxcijas saistība ar redzes funkcijām), 13 June 2018
4. Turlisova, J., Jansone, A.: E-studies and mastering of educational material for people with
visual perception and visual – motor integration problems – topical issues and perspectives.
In: Proceeding of ICLEL 2018 (2018)
Experimental Study on Dynamic Map
Information Layout Based on Eye Tracking
Jiapei Ren, Haiyan Wang(&), and Junkai Shao
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
[email protected], [email protected],
[email protected]
Abstract. In the GIS system interface, the map module occupies more than
90% of the interface size, which is the focus of the GIS interface design. There
are a large number of targets, target signs, target trajectories and other infor-
mation on the map. The layout of various information is crowded and over-
lapped, making it difficult for users to search and identify information. The
purpose of this paper is to explore the impact of different presentation forms of
target signs on user search efficiency. This paper used 8 kinds of overlap and 4
kinds of target sign layout display to design the experiment. This paper analyzed
the search performance of subjects, and gaze plot based on eye tracking data.
Keywords: Map information layout Eye movement experiment Map
cognitive experiment Dynamic layout
1 Introduction
The Geographic information system (GIS) situational system is based on digital maps
to display geographical environment information, target motion situation and resource
allocation and other related data in real time [1]. However, the excessive data causes
the information density of the map interface to be high, and the layout in the situation
map is crowded. Frequent overlap makes it difficult for users to quickly find the
information they want to know. The situation map needs to solve the problem of
information presentation through dynamic interactive layout.
Visual search is an important method of obtaining visual information behavior and
information processing. Eye tracking technology is widely used in web interface
design, map usability research and so on. Nielsen [2] concluded in the eye movement
experiment that the user’s attention distribution tends to be close to the “F” type when
browsing the web. In 2008, Owens [3] used eye movement to study the subject’s search
efficiency at different locations on a web page. Jing [4] used eye tracking technology to
found that the layout has cognitive guidance for the display information of the
numerical control interface.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1238–1243, 2020.
https://doi.org/10.1007/978-3-030-39512-4_189
Experimental Study on Dynamic Map Information Layout 1239
2 Overlap Rate
The overlap rate refers to the ratio of the area of overlap between graphics to the total
area of the graphics. As shown in Fig. 1, rectangles are randomly distributed in the
interface, and the rectangles overlap each other. The overlap area is the interface pixel
areas - the sum of the white (color value #ffffff) pixel area - the sum of the light gray
(#bdbdbd) pixel area. The total area of graphics is the interface pixel areas - the sum of
the white (color value #ffffff) pixel area.
Target overlap rate ¼ overlap area/total area of graphics ð1Þ
Fig. 1. Overlap rate calculation case
3 Materials and Methods
3.1 Objective and Participants
There are a lot of information on the map, and various information overlap each other,
making it difficult for users to search and identify information. This paper aims to
explore the influence of different layouts of signs on the search efficiency of users under
different overlap rate of graphics.
A total of 22 participants from the School of Mechanical Engineering were
enrolled. The average age of the subjects was 22 to 27 years old. The ratio of male to
female subjects was 1:1.
3.2 Design
This test uses a two-factor multi-level test design. Variable 1 has a graph overlap rate of
eight levels (0, 1.6%, 4.9%, 7.5%, 10.6%, 13.5%, 16.9%, 19.7%), and variable 2 is four
kind of sign layout (the sign is presented below the target, the sign is not aligned with
the target line, the sign is connected to the target and distributed in right and left
alignment and the sign is connected to the target and distributed up and down aligned.).
The experimental material is shown in Fig. 2.
1240 J. Ren et al.
Fig. 2. Different overlap rates and layout
3.3 Procedure
At the beginning of the experiment, the computer screen presents the experimental
instruction. After reading, press the space bar to enter the experimental stage. Firstly,
the cross vision guiding center appears in the center of the blank screen, and then the
material picture appears. After finding the rounded rectangle, check the corresponding
number and enter the last two digits on the keyboard. The stimulation interval is
1000 ms to eliminate visual residue, the entire experimental procedure was about
12 min.
4 Results
4.1 Behavior Data
4.1.1 Descriptive Statistics
The statistical analysis results are as follows: the average correct rate of the test is over
90%, indicating that the test has maintained a high correct rate. The reaction time of
layout 3 is the shortest, and the reaction of layout 1 is the longest. The order of the
reaction time from short to long is: layout 3 < layout 2 < layout 4 < layout 1. The
overlap rate level 1 (0) has the shortest reaction time, and the overlap rate level 8
(19.7%) has the longest reaction time (Table 1).
Table 1. Descriptive statistics of behavior data.
Layout Correct rate(avg) Reaction time(avg)
1 0.9549 2.770
2 0.9792 2.628
3 0.9948 2.371
4 0.9818 2.670
4.1.2 Analysis of Variance
Since the difference in correct rate is small, the deep interaction effect analysis is no
longer carried out. The results of reaction time showed that the overlap rate P < 0.001;
layout P = 0.004; overlap rate layout P < 0.001, the overlap rate, layout, and the
interaction between the two are significantly different.
Experimental Study on Dynamic Map Information Layout 1241
Layout 3 compared with layout 1, p = 0.003, compared with layout 2, p = 0.004,
compared with layout 4, p = 001. There are significant differences between layout 3
and other layout forms, and the reaction time is lower than other layouts. There is no
significant difference between the other three layouts. The overlap rate level of 8 is
significantly different from other overlap rates (except for level 2), and the reaction time
is higher than other layout forms. When the overlap rate level is 1, there is a significant
difference in the reaction rate with the 2, 3, 6, 7, and 8 overlap rate levels, and the
reaction time is lower than other layout forms (Table 2).
Table 2. Analysis of variance of reaction time in different layouts.
(I)Layout (J) Layout Standard error Sig
3 1 .092 .003
2 .062 .004
4 .061 .001
4.2 Eye Movement Data
4.2.1 Descriptive Statistics
Fixation count is the number of gaze points in a region of interest or in an area of
interest (AOI) group; total fixation duration calculates the sum of the durations of all
gaze points in an AOI.
Layout 3 has the least fixation count and layout 4 has the most fixation count. The
order of the fixation counts under the four layouts from small to large is: layout
3 < layout 1 < layout 2 < layout 4. In the case of the overlap rate levels 1, 2, 5, 6, and 8,
the fixation count of the layout 3 is small.
Layout 3 has the shortest total fixation duration, and Layout 2 has the longest Total
fixation duration. The order of total fixation duration under the four layouts from small
to large is: layout 3 < layout 1 < layout 4 < layout 2. At the overlap rate levels of 3, 5,
and 6, the total fixation duration of layout 3 is small. At the overlap rate levels 1, 2, 4,
and 7, the total fixation duration of layout 1 is small (Table 3).
Table 3. Descriptive statistics of eye movement data.
Layout Fixation count (avg) Total fixation duration (avg)
1 7.913 1.544
2 8.285 1.753
3 7.740 1.504
4 8.366 1.558
4.2.2 Analysis of Variance
The results of the variance analysis of fixation count showed that the overlap rate and
overlap rate layout P < 0.001, layout P = 0.028. total fixation duration analysis
showed that the overlap rate layout and overlap rate layout P < 0.001. Therefore, the
1242 J. Ren et al.
overlap rate, layout, and interaction between the two have a significant impact on
fixation count and total fixation duration.
The results of variance analysis of fixation count show that there is a significant
difference between layout 3 and layout 2, 4 and no significant difference between the
other three layouts. The results of total fixation duration show that there is a significant
difference between layout 2 and layout 1, 3 and no significant difference between the
other three layouts. Total fixation duration of layout2 is higher than 1, 3.
4.3 Visual Graphic Analysis
Figure 3 is one of the test results of the gaze plot sequence. The subject used random
search strategy, the subject first randomly searches for the target graph, then fix on the
target and further checks the corresponding number. The line in layouts 2, 3, and 4
have a significant guiding effect on the search direction of the subject, and this effect is
consistent with expectations.
Fig. 3. Visual graphic diagram under different layouts
5 Discussion
Layout 1 performs better when the overlap rate is low. When the overlap rate reaches
19.7%, the reaction time of layout 1 is significantly increased. It has also been proved
that a small amount of overlap does not affect the visual search. Therefore, when the
overlap rate is low, layout 1 can be used as the default state. Since layout 2 cannot
completely avoid the overlap of the signs, so layout 2 is not recommended.
Layout 3 performs better at various overlap rates than various other layout indi-
cators. When searching for graphics, the graphics part of the layouts 3 and 4 are
relatively less interfered, the subjects can quickly search for the graphics, and the
layout 3 signs are aligned left and right, the sign layout is more compact, graphics and
signs and the distance is closer. Therefore in most cases, the search efficiency of layout
3 is better than layout 4, and the result is consistent with Fitts’ law [5]. And according
to the literature, the human eye’s visual law is from top to bottom, from left to right,
and the left and right movement speed is fast [6]. After the layout 3 searches for the
target, when the sign is positioned, the eye moves in the left and right direction, and the
speed is also faster than the up and down movement of the layout 4.
Experimental Study on Dynamic Map Information Layout 1243
6 Conclusion
In this paper, eye tracking technology is used to evaluate the layout design of map
module graphics and signs in GIS interface. Under the task scenario developed in this
experiment, there are significant differences between different layout forms. When there
is no overlap or a small amount of overlap between the targets, you can choose layout 1
and layout 3; when the targets overlap rate becomes high, the first recommendation is
to use layout 3, followed by Recommended layout 4. Since layout 2 has an ordinary
effect at different overlap rates, layout 2 is not recommended. In addition, in this
experiment, when the graph overlap rate increases to a certain extent, the search effi-
ciency drops sharply. Therefore, after the target overlap rate on the map exceeds 20%, a
certain design method should be adopted to reduce the graphics overlap rate.
Acknowledgments. The authors would like to gratefully acknowledge the reviewers’ com-
ments. This work was supported jointly by National Natural Science Foundation of China
(No. 71871056, 71471037) and Equipment Pre research & Ministry of education of China Joint
fund.
References
1. Wang, Y., Yao, Y.: Design of situation display system based on geographical information
system. J. Shipboard Electron. Countermeas. 29(4), 77–79 (2006)
2. F-Shaped pattern for reading Web content (original study). http://www.nngroup.com/articles/
f-shaped-pattern-reading-web-content
3. Owens, J.W., Chaparro, B.S., Palmer, E.M.: Text advertising blindness: the new banner
blindness? J. Usability Stud. 6(3), 172–197 (2011)
4. Jing, L., Shulan, Y., Wei, L.: Cognitive characteristic evaluation of CNC interface layout
based on eye-tracking. J. Comput.Aided Des. Comput. Graph. 7, 20 (2017)
5. Soukoreff, R.W., MacKenzie, I.S.: Towards a standard for pointing device evaluation,
perspectives on 27 years of Fitts’ law research in HCI. Int. J. Hum.Comput. Stud. 61(6), 751–
789 (2004)
6. Mccormick, E.J., Sanders, M.S.: Human factors in engineering and design (1982)
Research on Readability of Adaptive
Foreground in Dynamic Background
Maoping Chi and Lei Zhou(&)
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
{chi,zhoulei}@seu.edu.cn
Abstract. Head-up display is ancillary information presentation devices that
are widely used on aircraft, and the use of HUD increases flight adaptability and
safety. However, in some special scenarios, the color of the background envi-
ronment interferes with the color of the head-up display, reducing readability.
Here we propose a method of adaptively adapting the color of HUD information
according to the background environment to enhance readability. By simulating
the process of reading information in a dynamic background, the subject asked
to complete the task of identifying numbers in a string in a dynamic background.
The experimental result is that the dynamically changing information color has a
certain interference to the information recognition. In future research, adaptive
matching methods are implemented to reduce the interference caused by color
for better readability.
Keywords: Human factors Color Head-up-display Dynamic background
Readability
1 Introduction
Head up display is mainly used to present auxiliary flight information on the fighter. Its
main advantage is to reduce the number of times the pilot looks down at the dashboard
information during the flight and keep looking up [1]. Since the display content of the
head-up display and the external environment have color and brightness fusion and
superposition [2], the display effect of the information is unstable, which will affect the
performance of the pilot to complete the task, so in the aspect of the head-up display
interface Color coding is an important research direction. In this paper, aiming at the
color of the head-up display, this paper proposes an idea of adaptively adjusting the
foreground color of the scene when the background field environment changes, and
studies the character foreground color under the dynamic background.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1244–1249, 2020.
https://doi.org/10.1007/978-3-030-39512-4_190
Research on Readability of Adaptive Foreground in Dynamic Background 1245
2 Related Work
2.1 Interface Color Study of Head-up Display
Most of the current HUD interface colors are basically single green with wavelengths
between 500 and 560 nm. On the basis of the green interface, Xie et al. studied the
character brightness and line width of the usual display [3]. Wang Haiyan et al. used
the measurement method of cursor acuity to find that the response of green (120° 99%
33%) characters correctly recognized in various environments is shorter (Kang Haiyan
literature). In order to explore the possibility of other colors, Derefeldt et al.’s study of
the color coding of fighter interfaces pointed out that multi-color coding can improve
the reaction time in complex environments, and pilots get better state perception [4].
Xiong et al. studied the ergonomics of the HUD interface using multiple colors in a
simulated flight platform, and obtained two colors in combination with a single color or
three colors, and the magenta logo key information and the subject green form per-
formance are optimal [5].
2.2 Research on Adaptive Regulation of Head-up Display
Due to changes in the external market environment, Yoon et al. proposed that when the
highest frequency hue value in the background color and the HUD foreground color
hue are within 30°, the HUD color has been adapted to the background environment,
and the conclusion is that the color change HUD The interface is better readability [6].
Zhi-Shan et al. used this theory to improve the HUD interface. The results show that
the user’s required PJND value is higher in low-light environments, and the PJND
values are not much different for different chroma environments. Brightness adaptive
design of HUD interface [7]. In color correlation theory, complementary colors and
contrast colors are often used to enhance information contrast to highlight important
information. This method is very common in graphic design. However, it is rarely used
in a dynamic environment.
3 Experimental Design and Discussion
The experiment simulates the external dynamic scene of the pilot in the flight mission
to determine the type of randomly appearing characters to measure the performance of
the foreground character color while remaining unchanged and adaptive according to
the environment. The experiment selected four typical scenes, three character colors,
and two adaptive methods.
3.1 Experimental Materials
The experimental variables are set to the color of the character and the background
environment. The color of the characters is divided into two groups, and one set of
colors remains unchanged in the dynamic environment, including red, green, and blue.
The other set is an adaptively varying foreground color, experimented with
1246 M. Chi and L. Zhou
complementary colors with dynamic background color averages and 2 sets of contrast
colors. The background environment selects the typical scene as shown in Fig. 1.
Figure 1a and b simulate different time and varied sky, Fig. 1c simulates the calm blue
sky, and Fig. 1d simulates the large green forest. Special scenes. Superimpose seven
random characters consisting of uppercase English and numbers 4, 5, and 6 in front of
the dynamic background, where the number is at most one [9]. Figure 2a shows the
experimental material for one of the cases. Two groups of white on black and black on
white were added as a control group, as shown in Fig. 2b and c.
Fig. 1. Typical scene.
Fig. 2. Experimental material case.
3.2 Experimental Environment and Subjects
The experiment used unity 3d to implement the experimental program. The presenta-
tion of the experimental video and the recording of the experimental data are performed
by the programming of unity 3d. The raw data is exported and the experimental data is
summarized and summarized by excel, and the statistical theory is used for data
analysis.
The experiment was performed on the iMac to ensure that the display color
deviation was within an acceptable range, the brightness of the control display was
150 cd/m2, and the illumination in the control room was 500 lx.
The resolution of the experimental video was 1024*768. During the experiment,
the subjects were undergraduate and graduate students of 10 universities. They were
not exposed to similar experiments. They were between 22–26 years old and had no
color blindness and color weakness. After correction, the visual acuity was above 1.0.
Research on Readability of Adaptive Foreground in Dynamic Background 1247
3.3 Experimental Design and Process
According to the setting of experimental variables, the performance difference of dif-
ferent variables was reflected by testing the reaction time, the correct rate and the
number of reactions. 26 experimental video materials were compiled into unity3d and
presented randomly. Record the test response and the correct rate and number of
reactions by the preparation of the program. Participants read the experimental
instructions, press A to start the experiment, and ask the participants to feedback the
appearing string. If there is no number in the string, press keyboard 0. If 4, 5, 6 digits
appear, the corresponding keyboard button will be pressed. Feedback.
(a) Experimental results
Through the calculation and analysis of the correct rate of the test, the correct rate is
calculated according to the following formula (1). The correct number of times of
the test reaction is ncorrect, the number of errors is nincorrect, and the number of
unreacted is nmiss. The result of the T-test is obtained by keeping the character color
still and the correct rate according to the environmental adaptive condition is
P > 0.05 (P = 0.625), so the character color change has no significant influence on
the accuracy of the task judgment.
P ¼ ðncorrect nincorrect 2 nmiss Þ=ntotal : ð1Þ
(b) Through the calculation and analysis of the reaction of the subjects, the values of
the subjects were normalized by using the formula 2 Min-max normalization, and
the difference in reaction between the subjects was eliminated. As shown in
Table 1, the white character on the black background has the lowest response time.
As shown in Table 2, in the context of dynamic changes, the average value after
normalization of the test reaction is shown in the figure. In different scenarios, the
response of green characters is the shortest and most stable. Complementary and
contrasting adaptive character colors are not stable, and blue does not perform well
in all backgrounds (Figs. 3 and 4).
T ¼ ðRT RTMin Þ=ðRTMax RTMin Þ: ð2Þ
Fig. 3. Average response time in different background.
1248 M. Chi and L. Zhou
Fig. 4. Average response rime for different backgrounds with different fore-colors.
(c) By mathematically counting the number of responses to the stimulus, the
number of stimuli controlling each background was 19, and the average number of
responses of the character color in a set of experiments was 18.96 (standard
deviation: 0.38, deviation 0.2%). The average number of response times for char-
acter color adaptation was 19.39 (standard deviation: 0.84, deviation 2.0%). It can
be seen that the adaptation of the character color increases the number of reactions
of the subject, and the number of reactions of each subject fluctuates greatly.
4 Conclusion
This experiment got the conclusion that the green characters have better stability in the
HUD interface, and the other colors are lower when the subjects respond. Although
color adaptation meets the requirements for color contrast, it increases the extra load of
reading the information of the subject.
Acknowledgments. This work was supported by the Science and Technology on Avionics
Integration Laboratory and Aeronautical Science Fund (No. 20165569019) and the National
Natural Science Foundation of China (No. 71871056, No. 71471037).
References
1. Yang, X.: Design of head-up display. J. Fuzhou Univ. (Nat. Sci. Edn.) 04, 29–32 (2000). (in
Chinese)
2. Harding, T.H., Rash, C.E., Lattimore, M.R., Statz, J., Martin, J.S.: Perceptual issues for
color helmet-mounted displays: luminance and color contrast requirements. In: SPIE
Defense+Security (2016)
3. Xie, J., Wang, X., Lu, J., Chai, X., Wang, L.: Study on character brightness and line width of
head-up display. Electro-Opt. Control 21(08), 68–72 (2014). (in Chinese)
Research on Readability of Adaptive Foreground in Dynamic Background 1249
4. Derefeldt, G., Skinnars, Ö., Alfredson, J., Eriksson, L., Andersson, P., Westlund, J.,
Berggrund, U., Holmberg, J., Santesson, R.: Improvement of tactical situation awareness with
colour-coded horizontal-situation displays in combat aircraft. J. Disp. 20, 171–184 (1999)
5. Xiong, D., et al.: The effect of one-color and multi-color displays with HUD information in
aircraft cockpits. In: Man-Machine-Environment System Engineering. Springer, Singapore
(2016)
6. Yoon, H.J., Park, Y., Jung, H.Y.: Background scene dominant color based visibility
enhancement of head-up display. In: International Conference on Systems Engineering. IEEE
Computer Society (2017)
7. Zhi-Shan, W., et al.: A new method to evaluate the readability of head up display. In: IEEE
International Conference on Computer & Communications. IEEE (2017)
8. Wang, H., Shao, J., Gao, Y., et al.: Color design of helmet aiming interface. J. Electro-Opt.
Control 23, 64–67+76 (2016)
9. Gabbard, J.L., Swan, J.E., Hix, D.: The effects of text drawing styles, background textures,
and natural lighting on text legibility in outdoor augmented reality (2006)
Research on Interaction Design of Children’s
Companion Robot Based on Cognitive
Psychology Theory
Tianmai Zhang and Wencheng Tang(&)
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
{zhangtianmai,tangwc}@seu.edu.cn
Abstract. This paper aims to explore the interactive design methods for chil-
dren’s companion robots by applying cognitive psychology theories. First of all,
the researchers conduct an investigation, which analysis from the perspective of
cognitive psychology. In combination with the users’ aesthetic requirements,
habits and physiological laws, the user demands in product design are reason-
ably analyzed. Then use cognitive psychology to observe the using process of
children. Design information is obtained by investigating perception, thinking
and operational process of users. Finally, combined with the user model of
design object groups, a targeted interaction mode is developed and verified by
the product design practice. The article transforms the traditional design thinking
of “machine-oriented” into “people-oriented”, which has guiding significance
for the future interactive design of other robot products.
Keywords: Interaction design Cognitive psychology Companion robot
1 Introduction
Nowadays, the “left-behind” and “feeding by generations” children have become the
mainstream of Chinese society [1]. More and more children are lacking in parenting
during their growth. The “lack of love” environment will cause psychological trauma to
children’s psychology [2]. In this context, children’s companion robots came into
being. They are not only developing, accelerating and promoting children’s intellectual
level, social ability and logic thinking, but also create a solid foundation for children’s
physical and mental development. They can also be indispensable partners for chil-
dren’s healthy growth.
2 Current Status of Children’s Companion Robot
Companion robots are one of the mainstream research directions in the home robot
industry. Accompanying objects include children, the elderly and even pets. In the field
of companion robots, most manufacturers are targeting children [3]. Modern life is fast-
paced and stressful. Parents are busy with work and neglect their companionship with
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1250–1255, 2020.
https://doi.org/10.1007/978-3-030-39512-4_191
Research on Interaction Design of Children’s Companion Robot 1251
their children. At the same time, children have a natural interest in robots. In fact, with
the help of currently available technologies and equipment, on the one hand, the current
technology is easier to meet the needs of children. On the other hand, adult compan-
ionship requirements for robots do not seem to be particularly strong. It is foreseeable
that the future smart companion machine market is bound to become the focus of
global robotics competition.
For the interactive interface of the product, the interface layout generally includes a
side function bar and a content display area, and some products have a navigation bar
portion. The structure of the interface is generally a 3-level page structure, and some of
the more complex ones have a 4-level page structure. The interactive action is mainly
by touch, and the gesture is generally by click and swipe.
3 Application of Cognitive Psychology
Human cognition is a psychological process. The ability of the human brain to process,
store, and extract information, perception, memory, attention, thinking, and imagina-
tion are all considered cognitive abilities [4]. Cognitive ability refers to the ability of
the human brain to process, store, and extract information. It is the intelligence we
generally talk about, such as observation, memory, imagination and so on. People
mainly rely on cognitive ability to understand the objective world and acquire a variety
of knowledge.
3.1 Human Cognitive Process
According to cognitive psychology theory, the entire cognitive process of a person can
be summarized into three steps [5]:
(1) Sensors receive information under environmental stimuli;
(2) The brain processes the information;
(3) Effector output corresponding reaction.
The sensor mainly refers to the sensory system of the body, including vision, touch,
hearing, smell, and force. Effectors are mainly including muscles, glands, and organs
that respond to brain commands. Responses are expressed in terms of expressions,
language, actions, and behaviors (shown in Fig. 1).
Fig. 1. Human cognitive process
1252 T. Zhang and W. Tang
3.2 Factors Affecting Cognitive Ability
The performance of a user’s cognitive ability is influenced by many factors. Therefore,
the factors affecting the potential cognitive ability need to be fully considered in the
process of product design. On the one hand, it will help to further understand the
cognitive ability of the target user group. On the other hand, it helps to establish a more
realistic user model to predict product usage scenarios.
Fig. 2. Cognitive model [6]
As shown in Fig. 2, human behavior, performance, and achievement are just rep-
resentations. What hid in the hidden layers are the cognitive ability and cognitive
psychology of human beings. The environment (including the family environment,
school environment, social environment) will have an important impact on the ability
of people to form and the formation of psychology.
4 User Analysis
The interactive subject of the child companion robot is the child. At present, most of
the research on interaction design of children’s companion robots is carried out from
two aspects: children’s physical and mental development characteristics and the
application of interactive technology. According to Piaget’s theory of children’s cog-
nitive development [7], the development of children’s cognition is characterized by
stage and incompleteness, as shown in Fig. 3.
From that theory we can easily find that children from 0 to 2 years old are in the
stage of perceptual movement, and the interaction between children and the outside
world is from instinct and unconscious. Between the ages of 2 and 11 years, children
gradually complete the transition from figurative thinking to simple abstract thinking.
After age 11, children begin to master advanced abstract thinking skills.
Perception mainly includes vision, hearing, touch, force, object perception, time
and spatial perception. It mainly refers to the ability to exercise and behavior including
movement. Children’s perception and motion development show different character-
istics at different ages, as shown in Fig. 3.
Research on Interaction Design of Children’s Companion Robot 1253
Fig. 3. Children development process of perception
Therefore, companion robot products designed for children tend to be highly
experiential and interactive, and need to meet the child’s physical interaction needs. In
addition, children show a certain degree of gradual and progressive in the cognitive
process due to the development of their perception and athletic ability. The interaction
design of the robot should take this feature into consideration and make appropriate
design choices.
5 Interactive Mode Design
Through the cognitive theory model and the cognitive characteristics of children, we
find that establishing multi-channel information interaction and forming rich sensory
and motor stimuli have great influence on children in the stage of growth and devel-
opment, which is of great significance. Next, we combine the analysis of children’s
cognitive information channels with design practice to illustrate several ways of
interacting with children.
5.1 Image Interaction
Children are very sensitive to color and like bright and bright colors, especially red,
yellow, green, orange and blue. It is possible to distinguish between various figures by
the age of 2, and enter adult vision when you are 6 years old. Our children’s companion
robots use high purity and saturation in their design. The robot’s LCD interactive
interface displays a clear-cut expression or cartoon pattern. Through intuitive image
interaction, children are connected to the robot through image recognition and image
perception (shown in Fig. 4).
1254 T. Zhang and W. Tang
Fig. 4. The robot’s LCD interactive interface
5.2 Voice Interaction
Providing appropriate auditory feedback with visual graphics and tactile perception
during the interaction process can help children learn the basic concepts of everyday
things, such as whistle, wind, raindrops, drumming, and so on. The child’s auditory
development can be exercised by accompanying the movement of the robot to emit
sound signals of different orientations. In addition, intelligent voice interaction is a new
generation of interactive mode based on voice input. By developing a speech recog-
nition feedback system, the robot can make corresponding feedback through the child’s
speech. The use of auditory feedback and speech recognition enables the robot to
effectively express its role positioning, making the interactive context more vivid and
interesting.
5.3 Behavioral Interaction
Behavioral interaction refers to the transmission of meaning through the movements
and postures of the limbs. The robot understands the meaning of actions and behaviors
by tracking the child’s limb movement trajectory and recognizing the child’s posture
characteristics, and makes corresponding intelligent feedback. By simulating different
life situations, the robot allows children to experience simple actions such as “on-off”
and “push-pull” to form a learned action experience. As children’s comprehensive
cognitive ability increases, setting challenging interactions can not only exercise
children’s hands-on ability, but also improve children’s intelligence and logical
thinking ability.
Our companion robot behavioral interaction mode is non-touch type, which can
realize different movements such as turning and climbing through a 360-degree rotating
front wheel and two 55-degree rotating track mechanisms (shown in Fig. 5). In addi-
tion, the robot can make a 55-degree lifting action of the rear left and right crawler
wheels by cooperating with the child’s left & right hands lifting movements, thereby
bringing an interesting interactive experience.
Research on Interaction Design of Children’s Companion Robot 1255
Fig. 5. Behavioral interaction
6 Conclusion
This paper aims to explore the interactive design methods for children’s companion
robots by applying cognitive psychology theories. First of all, the researchers conduct
an investigation, which analysis from the perspective of cognitive psychology. In
combination with the users’ aesthetic requirements, habits and physiological laws, the
user demands in product design are reasonably analyzed. Finally, combined with the
user model of design object groups, a targeted interaction mode is developed and
verified by the product design practice. The article transforms the traditional design
thinking of “machine-oriented” into “people-oriented”, which has guiding significance
for the future interactive design of other robot products.
Since children are still in the early stages of thinking development, most of their
minds are still thinking in images and not good at abstract thinking (such as reasoning
and judgment). The interactive system of robots should be designed from the perspective
of children so that they can use most of the functions of the robot without special training.
Through the three-dimensional of interaction design (images, voices and actions), the
robot interaction with children is smoother and more natural than before.
References
1. Jingzhong, Y., Lu, P.: Differentiated childhoods: impacts of rural labor migration on left-
behind children in China. J. Peasant Stud. 38(2), 355–377 (2011)
2. Hetherington, E.M., Parke, R.D.: Child Psychology: A Contemporary Viewpoint. McGraw-
Hill, New York (1976)
3. Belpaeme, T., Baxter, P., Greeff, J.D., Kennedy, J., Zelati, M.C.: Child-robot interaction:
perspectives and challenges. In: International Conference on Social Robotics. Springer (2013)
4. Merikle, P.M., Smilek, D., Eastwood, J.D.: Perception without awareness: perspectives from
cognitive psychology. Cognition 79(1), 115–134 (2001)
5. Carroll, J.B.: A theory of cognitive abilities: the three-stratum theory. Hum. Cogn. Abil. 631–
655 (1993)
6. Dong, H., Ning, W.N., Hou, G.H.: Measuring cognitive capability: a literature review based
on inclusive design. Industrial Engineering & Management (2016)
7. Kamii, C., Devries, R.: Physical knowledge in preschool education: implications of Piaget’s
theory (1978)
Strategies for Accessibility to the Teodoro
Maldonado Hospital in Guayaquil. A Design
Proposal Focused on the Human Being
Josefina Avila Beneras(&), Milagros Fois Lugo,
and Jesús Rafael Hechavarría Hernández
Faculty of Architecture and Urbanism, University of Guayaquil,
Cdla. Salvador Allende, Av. Delta y Av. Kennedy, Guayaquil, Ecuador
{josefina.avilab,maria.foisl,
jesus.hechavarriah}@ug.edu.ec
Abstract. Urban planning is a mechanism that allows cities to achieve sus-
tainable development to the extent that it provides the citizen with quality of life.
The present study is a territorial analysis and diagnosis process is carried out to
improve pedestrian accessibility in the public space immediately to Teodoro
Maldonado Hospital in Guayaquil through the development of a mobility plan
for the establishment of design strategies. A mixed methodology is proposed
under a systemic approach where six components are considered in the Jan Gehl
methodology on five areas of action: Preservation of Heritage, Sustainable
Mobility, Equity and Diversity, Urban Design at the Human Scale, Develop-
ment Economic and Cultural. The proposal considers a higher level of impor-
tance (70%) for pedestrian circulation than the road system, following the
parameters established in NACTO (National Association of transport officials of
the city).
Keywords: Urban planning Human factors Mobility Accessibility
strategies
1 Introduction
Transport is associated with environmental problems, economic losses, population
health and social inequities [1]. The Teodoro Maldonado Carbo hospital, in the city of
Guayaquil-Ecuador, provides daily care in an average of 6000 members with a cov-
erage of 50,000 inhabitants. Considering mobility and accessibility to this flow of
people daily implies risks and dangers for passers-by and especially for people with
special abilities that require even greater mobility due to their condition, according to
Sierra, “provide for the mobility dynamics of cities to small scale, by neighborhoods,
segments, crossings [2].
Urban planning should be oriented to seek equity in the distribution of resources
(public works, equipment, infrastructure, among others [3], which is why it is proposed
to achieve an equitable distribution of these resources to compensate for socioeconomic
inequality and socio-spatial that presents the case study [4] and [5].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1256–1262, 2020.
https://doi.org/10.1007/978-3-030-39512-4_192
Strategies for Accessibility to the Teodoro Maldonado Hospital 1257
In the work, a study of accessibility to the public space of the hospital facilities is
presented, using a mixed methodology. The diagnosis is made based on six compo-
nents for the sector in general and for the specific sector the methodology of the
architect - urbanist Gehl [6] is used. Based on the conclusions, the proposal for a
strategic mobility plan is presented based on the design process of the NACTO
(National Association of city transport officials) urban streets guide that uses qualitative
and quantitative methods-two.
2 Methodology
The proposal of the strategic mobility plan is presented in two phases, which are
described below:
Phase 1: It begins with stage 1 called “Diagnosis” it is composed of 4 methods that
allow the analysis process:
• (M-1) with a coverage diameter of 1000 m as seen in image 3, based on compo-
nents specified in the Senplades processes for a general analysis;
• (M-2) the diagnosis of the specific sector based on the Jan Gelh methodology is
developed using a smaller radius of action 260 m;
• (M-3) is the survey process as a complement to the diagnosis of the specific sector,
it works by applying a survey of 382 samples with 5% error, and 95% confidence
level, which its purpose is to know what people think and the feasibility of the
project;
• (M-4) to complete the diagnostic stage is established in SWOT matrix format,
exposing the advantages and disadvantages found in the sector.
Stage 2 establishes (M-5) the strategies for the improvement in mobility and
accessibility of the immediate sector to health equipment Level III Teodoro Maldonado
Carbo Hospital based on the conclusions of the diagnosis of the specific sector and its
relationship with the results of the study of the general sector, resulting in the approach
of the strategies to be used. Once the strategies for the Special Mobility Plan of the
sector have been established, the structuring criteria of said plan are determined.
Phase 2: As a secondary phase of the investigation, stage 3 is established, (M-6)
which raises the proposal for the development of the content of the Special Mobility
Plan, projecting strategic interventions based on two variables: the first will consider
the Pedestrian with an importance of 70% and the second the road system with a ratio
of 30%, following the parameters established in NACTO.
Jan Ghel’s methodology is complemented, with the use of the NACTO guide,
(National Association off City Transportation Officials) that is based on qualitative and
quantitative methods with which a clear vision of the entire streets is described in a
range of action from the hospital and a basic road map with the use of the guide
(Fig. 1).
1258 J. Avila Beneras et al.
Fig. 1. Methodology diagram.
3 Development
Phase 1: Stage 1-Diagnosis
M1: Within stage 1 the diagnosis was established, which was made according to
Senplades methodology, selecting the parameters related to the development of a
PEMU: (i) land use, (ii) road (iii) public transport and (iv) infrastructure.
The following is a summary of what was obtained about the study sector in phase 1
of the investigation: (ia) Land uses in the sector are divided into: 1.03% commercial,
1.65% equipment, 2.47 use of green area, 0.72% industrial, 80.42% residential com-
ponent. 13.71% mixed use (residential-commercial); (ib) The limits of Teodoro Mal-
donado Carbo Hospital are: North with the secondary collector track V4: Av. Ernesto
Albán Mosquera, South with V4 collector track: Dr. Leónidas Ortega Moreira, this
with a V3 primary arterial route: Av. 25 de Julio, West via V4 secondary: Av. 1 around
tracks V4 and V5; (iia) The study sector has: mass transportation through the Metrovia,
3 metrovía stations on Av. 25 de Julio, a terminal station called July 25 to the south,
transportation of urban buses, there are 21 line signs (type signs); The coverage of the
services is established as follows: 100% paved streets, 100% coverage of drinking
water and sewerage, presence of electrical substation in the citadel La Pradera (200mt
distance).
M2: The methodological sheet of the urban architect Jan Gehl is used to assess the
state of public space, with the B-Mobility Sustainability card whose objective is to
combat the progressive invasion of the car and provide more sustainable and friendly
solutions for people.
Through the use of the described card, the following observations were obtained:
• The study area is made up of routes: Arterial Primary V3, Secondary Collector V4
and V5. With Av. 25 de Julio being the primary arterial route V3.
• All streets are paved and has mass transportation through Metrovia Transportation
and Urban Buses
• There are 3 Metrovía stations on Av. 25 de Julio that covers the study sector and a
terminal station called 25 de Julio del sur.
Strategies for Accessibility to the Teodoro Maldonado Hospital 1259
• There is a public transport network system that congestes Av. 1 avenue
• There is a very defined pedestrian access circuit to the hospital (Fig. 2).
Fig. 2. Parameter chart for diagnosis.
M4: Tool detailing the advantages and disadvantages that the sector presents in the
swot: Strengths, Opportunities, Weaknesses and Threats based on the conclusions
drawn from the site analysis.
In order to establish the strategies and criteria, the Special Urban Mobility Plan
(PEMU) is taken into account, in which interventions are planned based on two
variables, that of prioritizing pedestrians and following the parameters established in
the methodology. NACTO (National Association of city transportation officials).
Strategies
1. Delimitation of parking lots for private vehicles and public bus parking, as an
improvement strategy for road traffic in the surrounding sector of the Hospital.
2. Redesign of the roadway as a strategy for safe pedestrian mobility.
3. Incorporation of green areas with urban furniture, as a strategy of good use of public
space.
4. Reorganization of informal merchants to determine specific areas of product sales
and contribute to the availability of space to pedestrians.
5. Sectorize the intervention in specific urban actions (A, B and C).
Criteria: On Av. Ernesto Albán Mosquera and Dr. Leonidas Ortega: Incorporation
of tree vegetation: considering the current size of the sidewalks between 2 m and
2.30 m by selecting the species of semi-deciduous tree Zebra Tree (Erythrina indica
picta) with a height ranging from 8 to 12 m [3]. With an 80 cm protection grid
anchored in such a way that it is at the same level as the floor.
1260 J. Avila Beneras et al.
Corresponding to Av. 1 on the east sidewalk (emergency entrance and laboratory): •
Delimitation of public bus parking lots: with their respective signage. • Redesign of the
roadway: expanding from the current dimension of 3 m to 3.50 m depending on the
high flow of users generated by the Teodoro Maldonado Hospital. • Incorporation of
street furniture: The furniture has been arranged semicircularly with respect to the
factory line (enclosure). • Reorganization of informal merchants: with support of the
location of the kiosks for the food service.
Corresponding to Av. 1 on the west sidewalk: • Incorporation of green areas with
urban furniture: conditioning the existing spaces destined for this function, the vege-
tation to be used is the species Olivo Negro (Bucida buceras), an evergreen tree 12 m
high. The furniture arranged on this sidewalk will have a circular shape with wooden
seats plastic in a concrete base.
Phase 2: Stage 3 - Urban Design Proposal (M5)
Through the study of accessibility to the public space of the facilities of the Teodoro
Maldonado Carbo Hospital, an urban design project is proposed in the immediate roads
to the equipment and as a result of the strategies explained in the previous section the
following proposal is expressed, consisting of 3 interventions listed below (Fig. 3):
Fig. 3. Representation of interventions.
• Intervention A: The use of existing infrastructure for the improvement of the cir-
cuits in zone C is proposed.
• Intervention B: A reconfiguration will be done in the s/n route, admission to
external consultation and pharmacy, in which the discontinuous widening of the
sidewalks is proposed emphasizing the pedestrian access to the hospital in the same
way as creating spaces for the inclusion of informal traders (Fig. 4).
• Intervention C: It is planned to relocate the existing bus stops on Av. 1 avenue, so
that bus stops associated with the main flows related to the hospital can be created,
as well as the spaces for the proposal of parallel parking for the private vehicle and
widening sidewalk (Fig. 5).
Strategies for Accessibility to the Teodoro Maldonado Hospital 1261
Fig. 4. Intervention B in the street adjacent to the income of external consultation.
Fig. 5. Intervention B, on both sides of the avenue of Av. 1.
4 Discussion
The analysis carried out based on the methodology used allowed a design that prior-
itizes to people who come to the Hospital for some need and. Ample spaces are created
that provide the comfort required for these cases. The proposed strategies were carried
out so that there is a road and pedestrian adequate circulation in the vicinity of the
Hospital. The redesign of the roadway, the implementation of green areas and the
reorganization and location of informal merchants in the streets vicinity of the hospital;
will allow an improvement in the quality of life of people arriving at this establishment.
Acknowledgments. To the Programa of Master in Architecture with mention in Territorial
Planning and Environmental Management. Faculty of Architecture and Urbanism of the University
of Guayaquil.
1262 J. Avila Beneras et al.
References
1. Pástor, B.A.C., Lugo, M.M.F., Vázquez, M.L., Hernández, J.R.H.: Proposal of a technolog-
ical ergonomic model for people with disabilities in the public transport system in Guayaquil.
In: Ahram, T., Falcão, C. (eds.) Advances in Usability and User Experience, AHFE 2019.
Advances in Intelligent Systems and Computing, vol. 972. Springer, Cham (2020)
2. Sierra, I.: Ciudades para las personas, Escenarios de vida. Editorial, Diaz de Santos, vol.
105 (2015)
3. Bazant, J.: Planeación Urbana estratégica, métodos y técnicas de análisis. Editorial Trillas,
Mexico (2014)
4. Llop, J.M., Vivanco, L.: El derecho a la ciudad en el contexto de la agenda urbana para
ciudades intermedias en Ecuador (2017)
5. Sánchez, G.: Planeación moderna de ciudades. Editorial Trillas, México (2008)
6. Ghel, J.: La dimensión humana en el espacio público, recomendaciones para el análisis en el
diseño. Ministerio de vivienda y Urbanismo de Chile, PNDU (2017)
Fatigue Measurement of Task:
Based on Multiple Eye-Tracking Parameters
and Task Performance
Hanyang Xu, Xiaozhou Zhou, and Chengqi Xue(&)
School of Mechanical Engineering, Southeast University,
Nanjing 211189, China
{220174254,zxz,ipd_xcq}@seu.edu.cn
Abstract. Human error can be reduced by monitoring fatigue during work and
taking measures so that accidents can be effectively prevent. In this paper,
participants accumulated fatigue during the process of using the experimental
program integrating the open source SDK, and their task performance and
PERLCLOS were collected and calculated by the experimental program.
Finally, a neural network model was established to describe the relationship
between various parameters and fatigue. Through experiments, it is found that
PERCLOS and task performance can well reflect fatigue state in tasks of
searching. This model can detect the fatigue degree of different people in
specific application scenarios.
Keywords: Fatigue PERCLOS Neural network Saccade Task
performance
1 Introduction
The accident can be caused by human, machine, environment and management. With
the improvement of the reliability of equipment technology, human error has become
the main reason of accidents. Once human suffered from the physiological or psy-
chological load in activities, fatigue was generated [1]. Due to the complex factors of
fatigue, only defining it according to specific tasks can make it more valuable for
research. In studies on the relationship between eye movement parameters and fatigue,
the commonly used parameters include PERCLOS, SPV (Saccade Peak velocity),
duration and frequency of blinking, pupil diameter, etc. [2–4]. Meanwhile, studies have
shown that eye movement parameters can objectively reflect the fatigue status of
practitioners from a physiological perspective [5, 6]. Therefore, the fatigue study based
on eye parameters is of great significance to reduce man-made accidents [7].
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1263–1269, 2020.
https://doi.org/10.1007/978-3-030-39512-4_193
1264 H. Xu et al.
2 Paper Preparation
2.1 Participants
Sixteen participants were recruited for the experiment (mean age 24 years old, From 22
to 26 years old), they are all graduate students of industrial design or mechanical
engineering. All participants have normal vision or corrected vision. Six of the par-
ticipants were given a pre-test, and after a five-day interval all participants were given a
formal test. They were told to follow a normal and regular schedule of sleep and rest
during the test. The whole fatigue tests did not influence on their work or life.
2.2 Technique
In this study, Unity and C# was used to build the experimental program. The program
structure of the pre-experiment and experiment was roughly similar, except that the
formal experiment had time limitation. In the experiment, one group of icons were
randomly displayed in the screen, which contains five kind of icons, 225 icons in total
(45*5), as shown in the Fig. 1.
Fig. 1. Experiment program: the matrix of icons
Six groups of icons will be used in this study, with five icons in which have similar
complexity to ensure consistent response time. All the icons are semantically
Fatigue Measurement of Task: Based on Multiple Eye-Tracking Parameters 1265
ambiguous to ensure that the experiment is random and special to each participant, five
groups of them was shown in the Fig. 2. As shown in the Fig. 3, participants was told
to click on only one kind of these symbols in a specified amount of time, When the
participant was ready, press the key A then they can entered the experiment which
shown in the Fig. 1. The click path, amount, time and time stamp are recorded by
programs. Meanwhile, the program will gradually increase the participants’ cognitive
load and leads them to accumulate fatigue. The number and time of icon cancellation
indirectly reflects the participants’ reaction time, which can be used to calculate fatigue
status over a period of time.
Fig. 2. Groups of icons
Fig. 3. The instruct before experiment
2.3 Pre-experiment
Each participant will conduct the experiment in active period (9:00–10:00) and the tired
period (15:30–17:00 PM or 40 min after dinner). As shown in the Fig. 4, all the groups
of icons will be used in six rounds of pre-experiment only once. The random algorithm
keeps each groups of icons appear with equal probability and without repetition. There
was no time limit for each round of experiments, and a break between them, which was
controlled by the participants.
1266 H. Xu et al.
Fig. 4. The process of pre-experiment
In this part, participants won’t wear the Diskablis Eye Tracker, and their click
paths, reaction times and PERCLOS were recorded to optimize the program of main
experiment and verify that the next stage of the experiment had a controlled effect on
the SDK.
2.4 Experiment
As shown in the Tables 1 and 2, the results of the pre-experiment indicated that the
reaction time, break time and error rate of participants in the tired period were sig-
nificantly higher than those in the active period, so the main experiment would be
conducted in the former.
Table 1. Duration(s) and error rate of each icon set in different of period
Period Parameters Icon Set1 Icon Set2 Icon Set3 Icon Set4 Icon Set5 Icon Set6 Average
Morning Duration 36.667 41.467 47.367 50.650 48.583 42.100 44.427
Error rate 0.363% 0.727% 2.488% 0.708% 0.727% 0.708% 0.954%
Evening Duration 39.893 50.550 52.800 47.533 54.505 47.217 48.689
Error rate 2.488% 3.530% 2.143% 1.072% 5.777% 3.810% 3.137%
Table 2. Time on task(s), total time(s) and break time(s) in different of period.
Period Time on task Total time Break time
Morning 266.833 275.700 8.867
Evening 292.133 316.750 24.617
The process of the main experiment is similar to that of the pre-experiment.
According to the Fig. 5, six sets of icon will be used three times respectively, which
means a total of 18 rounds of experiment. The order of each icon set was adjusted
according to the average response time in the pre-experiment (each set of icon still had
a random click target).
Fatigue Measurement of Task: Based on Multiple Eye-Tracking Parameters 1267
Fig. 5. The process of main experiment
There are six time threshold in the main experiment (t1, t2,… t6), which are 80% of
the average reaction time of each set of icons. Participants will be informed to complete
the experiment as soon as possible without knowing the time threshold, otherwise the
current group of experiments may end with no warning, and start a new group of
experiments without being informed. In this part, participants will wear Diskablis eye
tracker, whose saccade distance, saccade time and front view image will be recorded,
while the derived data will be retained for further studies.
3 Results
Nie et al. [8] found in their study that fatigue was obvious when PERCLOS exceeded
6%, that is, the time of closing eyes of participants reached 3.5 s/min. At this time, P80
was selected as the calculation standard of PERCLOS with the best effect. The formula
is show as follows:
PERCLOS ¼ ½dmax ðdmax dmin Þ 80%=Ti ð1Þ
In formula (1), dmax and dmin means the maximum and distance of eyelid closure
per unit time, and Ti means the time spent in each round of experiments. The fatigue
state of participants was defined by the PERCLOS parameters from two-thirds of the
experimental data. SPSS was used to analyze the correlation between univariates.
PERCLOS was strongly correlated with fatigue state (Pearson correlation coeffi-
cient = 0.714) and moderately correlated with task performance (Pearson correlation
coefficient = −0.450), but only weakly correlated with SPV (Pearson correlation
coefficient = −0.167). The inter-group reliability of PERCLOS is not available
(Cronbach Alpha = 0.141).
The click score was selected as the task performance in this study. The initial value
of clicks score was 0, a correct click adds one point, and an error subtracted one until
the end of the current task. The results showed that task performance was normally
correlated with fatigue state (Pearson correlation coefficient = −0.384), but there was
no correlation between task performance and SPV in this situation (Pearson correlation
coefficient = 0.043). The average weight of each parameter obtained by using Radial
basis function neural network to analyze the data of each participant was shown in the
Table 3.
1268 H. Xu et al.
Table 3. The weight of PERCLOS, click score and SPV
Parameters Weight
PERCLOS 0.522
Click score 0.351
SPV 0.127
4 Discussion
The results show that the fatigue description model established in this study is reliable.
The low correlation between SPV and fatigue may be due to the fact that there is no
linear correlation between SPV and fatigue, and since PERCLOS is based on image
calculation, the error of the experimental instrument will also lead to this result, and the
relationship between SPV and fatigue can be further described in further studies.
Fatigue is a very complex phenomenon, and there must be more potential relationship
between various parameters of fatigue. Therefore, the fatigue described based on a
single index of PERCLOS may be different from the fatigue described by neural
network. In practical application, the fatigue can be divided into several important
grades and fed back to the user by fault-tolerant design in the system.
5 Conclusion
This study confirms the availability of eye movement parameters to measure the fatigue
levels in specific tasks. The method used in this study can also be used in other specific
tasks to derive the specific relationships among PERCLOS, task performance and
fatigue in a simulated environment. More complex neural network models and more
physiological indicators should be adopted to make the method more applicable in the
further research.
Acknowledgments. This study was supported by the National Natural Science Foundation of
China (No. 71901061), the Science and Technology on Avionics Integration Laboratory and
Aeronautical Science Fund (No. 20185569008) and the National Natural Science Foundation of
China (No. 71871056, No. 71471037).
References
1. Worm-Smeitink, M., Gielissen, M., Bloot, L., van Laarhoven, H.W.M., van Engelen, B.G.M.,
van Riel, P., Bleijenberg, G., Nikolaus, S., Knoop, H.: The assessment of fatigue:
psychometric qualities and norms for the checklist individual strength. J. Psychosom. Res.
98, 40–46 (2017)
2. May, J.G., Kennedy, R.S., Williams, M.C., et al.: Eye movement indices of mental workload.
Acta Psychol. 75(1), 75–89 (1990)
Fatigue Measurement of Task: Based on Multiple Eye-Tracking Parameters 1269
3. Di Stasi, L.L., Renner, R., Staehr, P., Helmert, J.R., Velichkovsky, B.M., Cañas, J.J., et al.:
Saccadic peak velocity sensitivity to variations in mental workload. Aviat. Space Environ.
Med. 81(4), 413–417 (2010)
4. Van Orden, K.F., Jung, T.P., Makeig, S.: Combined eye activity measures accurately estimate
changes in sustained visual task performance. Biol. Psychol. 52(3), 221–240 (2000)
5. Wilson, G.F.: An analysis of mental workload in pilots during flight using multiple
psychophysiological measures. Int. J. Aviat. Psychol. 12(1), 3–18 (2002)
6. van Drongelen, A., Boot, C.R.L., Hlobil, H., Smid, T., van der Beek, A.J.: Risk factors for
fatigue among airline pilots. Int. Arch. Occup. Environ. Health 1(90), 39–47 (2017)
7. Marandi, R.Z., Madeleine, P., Omland, O., et al.: Eye movement characteristics reflected
fatigue development in both young and elderly individuals. Sci. Rep. 8(1), 13148 (2018)
8. Nie, B., Huang, X., Chen, Y., Li, A., Zhang, R., Huang, J.: Experimental study on visual
detection for fatigue of fixed-position staff. Appl. Ergon. 65, 1–11 (2017)
Emotional Data Visualization for Well-Being,
Based on HRV Analysis
Akane Matsumae1(&), Ruiyao Luo2, Yun Wang3, Eigo Nishimura1,
and Yuki Motomura1
1
Faculty of Design, Kyushu University, 4-9-1 Shiobaru, Minami, Fukuoka,
Fukuoka, Japan
[email protected]
2
Politecnico di Milano, Milan, Italy
3
Beihang University, Beijing, China
Abstract. Managing negative emotions and promptly relieving anxiety are two
main focuses in the prevention of mental illness. This paper indicates a
biofeedback-training experience that helps an experiencer to manage his/her
emotions. The authors have defined the following major problems to approach
this bio-feedback design procedure. The first one is how to grasp human
emotions using existing technologies in exhibition circumstances. The second
one is what kind of manifestation should be used to create emotional resonance.
For the first problem, Heart Rate Variability (HRV) is adopted to obtain anxiety
and stress levels. For the second problem, the authors took a nature-inspired
design approach. The results are visualized on a wall composed of 64 bionic
scales. A device system guides the experiencer into a meditative state, and
contributes to spurring thoughts on emotional management, and thus achieving
the purpose of preventing mental illness.
Keywords: Bio-feedback design Heart rate variability Well-Being
Meditation
1 Introduction
Between 2005 and 2015, diagnosed cases of depression increased by 18% globally. At
the same time, depression has been confirmed to be one of the biggest contributors to
loss of life. In 2015, at least 788,000 people around the world committed suicide
because of depression. Low treatment and cure rates are typical characteristics of
mental illness. There are various theories trying to explain this situation. Greg Miller
says “The human brain is complex and difficult to study, which has impeded the
development of drug treatments for mental illnesses” [1].
Considering the fact that contemporary medical researchers studying the human
brain remain in a primary stage (many of the mental illness models in use today were
developed decades ago), the World Health Organization cites prevention as the best
treatment for mental illness. The proper management of negative emotions is the main
focus of prevention. In response to this situation, the authors hope to create an inter-
active emotional-visualization experience that leads people to pay more attention to
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1270–1276, 2020.
https://doi.org/10.1007/978-3-030-39512-4_194
Emotional Data Visualization for Well-Being, Based on HRV Analysis 1271
their mental health issues. Further, we hope to help people better manage their emo-
tions to achieve the purpose of prevention.
In this paper, we describe an HRV analysis algorithm using real-time ECG signals
obtained by BiTalino, with an acceptable time complexity and detection accuracy. At
the same time, a visualization installation is introduced in line with the needs of actual
practice. Moreover, the real-time monitoring of HRV may play a crucial part in dis-
ciplines such as high-performance sports that could benefit from real-time detection of
threatening heart conditions.
2 Methods
The authors approached the problem via interactive installation design. In this section,
HRV analysis, emotion features extraction and processing steps are described.
2.1 HRV Analysis
Heart rate variability (HRV) is a method of measuring the degree of change in a
continuous pulse signal. Researchers have found that, by analyzing heart rate vari-
ability in ECG signals, it is possible to effectively identify changes in mood. This
method offers a very good detection effect, especially regarding stress levels.
The association between emotions and HRV is actually the association of emotions
with autonomic nerves. The autonomic nervous system (ANS) is divided into two
systems: the sympathetic nervous system (SNS) and the parasympathetic nervous
system (PNS). Overall systems maintain a relative balance under normal conditions.
Sympathetic and parasympathetic nerves can develop disorders when people are suf-
fering from long-term stress. The heart is regulated by the autonomic nervous system in
addition to its own rhythmic discharge. When the conductivity of the sympathetic
nervous system cell membranes increases (sympathetic excitation), the membrane
potential of cardiac pacemaker cells also increases [2].
Depolarization occurs when the membrane potential of cardiac pacemaker cells
reaches a certain threshold which causes the interval between two heartbeats to become
shorter. In contrast, parasympathetic excitation causes the interval to become longer
[3]. The method of studying the change in intervals between two heartbeats is HRV, so
we can use HRV to infer whether the tester is experiencing anxiety.
2.2 Statistical Analysis
The BiTalino platform is used as a means of simple ECG acquisition. This device
allows various sampling rates in signal acquisition, fixed to 1000 Hz in our study. After
receiving the sampling data from the platform, a band-pass filter is applied. (Flow =
5 Hz, Fhigh = 15 Hz). In this case we use a finite impulse response (FIR) filter. This
procedure allows the removal of baseline noise and the avoidance of high-frequency
noise and artifacts. The voltage signals are segmented and annotated to the P, Q, R, S,
T points widely used in electrocardiography (Fig. 1). These points correspond to dif-
ferent stages of atrial/ventricular depolarization/re-polarization and are used to identify
1272 A. Matsumae et al.
waves and intervals that are relevant to ECG analysis. HRV assessment has tradi-
tionally relied on the analysis of the tachogram, giving potential information captured
by the time series. The tachogram indicates the temporal series of RR heartbeat
intervals, which contains the most relevant structural information of the HRV.
Fig. 1. QRS wave and RR interval
The first step of HRV assessment consists of R-peak detection. For this purpose, we
apply an algorithm based on the widely used QRS complex detection algorithm (Pan
and Tompkins, 1985). Furthermore, before the detection process, a derivative and
squaring filter is applied to the original signal to amplify the peak feature for a better
and faster detection rate. Once the R-peaks are located, several HRV features are
extracted from the RR-interval series:
SDNN (ms): Standard Deviation of Normal to Normal. Null variance would
indicate identical consecutive RR intervals [4].
SDANN (ms): Standard deviation of the averages of NN intervals in all 5-minute
segments of the entire recording.
NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in
the entire recording.
SD1 (ms) and SD2 (s) ellipse sub-axis estimators: Nonlinear features are derived
from the Poincaré plot, where each point is formed by the coordinates (RRi and
RRi + 1). Each data point represents a pair of successive beats; the x-axis is the current
RR interval, while the y-axis is the previous RR interval.
LF and HF: The elementary components inside the Low-Frequency (LF - 0.04 to
0.15 Hz) and High-Frequency (HF - 0.15 to 0.40 Hz) bands refer to autonomic seg-
ments of HRV. LF and HF can refer to the current mental state [5].
3 Interaction Design
3.1 Procedure
Participants begin the process of interacting with the device as they enter the room. At this
time the participant has the identity of the observer. After entering the process, a cushion
Emotional Data Visualization for Well-Being, Based on HRV Analysis 1273
is placed in the experience area. The visual image of the cushion is zen meditation, which
can give the experiencer the suggestion of sitting still. Once seated, the sensor attached to
the experiencer’s skin will start to capture ECG data. At the same time, the scale wall
(Fig. 2) will create an opening movement, suggest that the experience process has begun.
The device then starts the process of HRV analysis. The ECG signal will be extracted and
mapped to multiple motion latitudes of the scale wall to give feedback to the experiencer.
Fig. 2. “Scale wall” visual variations
The overall trend of the movement is a regular rhythm. If emotional pressure on the
experiencer is found after analyzing the data, the movement of the scale will change
from regular to irregular. Through the two different feedback signals of regularity and
irregularity on the scale wall, the user is finally guided to a calm, meditative state to
achieve the purpose of emotional management. Finally, when the data reaches a certain
threshold, (that is, when the experience is calm enough), the scale wall will enter a calm
state, suggesting that the experience has ended, and will no longer react to the user’s
actions. At this point, the experience process is over.
3.2 Flake Design
When it comes to the sense of order, a very striking example of animal scales can be
found in nature. Different types of scales have different shapes and arrangements. In
order to find a suitable combination for the device, it was necessary to analyze biological
scales present in nature. Below are some representative scales of fish and reptiles.
In order to reflect the core design concept of “taming a beast”, the authors refer-
enced the scales of pangolins when designing the flakes. The scales of pangolins are
narrow and sharp, with a longitudinal projection at the tail. Although the impression of
pangolin itself is not fierce, the shape of its scale is closer to the feeling of a “dragon’s
scale” in people’s minds. On the arrangement of the scales, this design refers to the
western diamond spotted rattlesnake and rainbow trout. In the overall movement pat-
tern of the scales, the design refers to the movement of the scales that occurs when a
chameleon changes color. Inspired by the articulation between the snake scales and the
1274 A. Matsumae et al.
body, the mechanical part consists of a hinge mechanism at the upper end of the scale
to connect with the underlying substrate, and supports the rotational movement of the
scale on the corresponding axis (Fig. 3).
Fig. 3. Flake mechanical design
4 Discussions
For this paper, a comparative analysis of eight ECG samples was conducted. Some of
the datasets are referred from PhysioNet, which provides an ECG database from
meditation experts. Other data are from our daily experiments performed on volunteers.
One of the results is shown in Fig. 4. The green lines indicate the LF and HF spectrum
during normal activity, while red lines indicate the others. By comparing the data, as
the meditation process deepens, the tester generally shows a relatively higher spectrum
in both LF and HF bands.
Fig. 4. LF and HF power comparison between meditation and normal activities
Emotional Data Visualization for Well-Being, Based on HRV Analysis 1275
A similar result was found in S. A. Matzner’s research [6]: “In the time domain, the
variance of the heart rate during meditation was significantly higher than before
meditation for all subjects”. However, there is no clear evidence indicating that HF and
LF have a positive correlation with meditation procedures. Nevertheless, this result is
sufficient to suggest that a deep meditation state is accompanied by a spectrum increase
in HF and LF bands. Therefore, it can be used as an indicator to determine the degree of
chaos on the scale wall.
5 Conclusion
In recent studies in the bio-medical fields, there has been considerable research related
to HRV analysis, and many inspiring research results have been produced. HRV is also
continuously being used in the fields of sports research, rehabilitation, etc., and is
expected to be a valuable research direction in the future.
Because of the weak correlation between the signal and the output, the testers
cannot directly describe the correspondence at the physical level. However, the HRV
biofeedback training system not only effectively determines the emotional state of the
experiencer, but also offers a brand-new interactive experience.
The real-time processing method described in this paper has an efficient elimination
effect on noise generated during ECG signal acquisition, which makes it possible to
collect and analyze signals in more complex environments, such as at exhibitions and
during athletic activities. It provides a relatively wide application range for this
algorithm.
Meditation is an effective means of emotional adjustment, but its application is not
extensive due to the need for experience and long periods of training, and different
individuals show disparate effects. The process described in this paper uses the
aforementioned HRV analysis method to convey mental signals in the meditation
process to the experiencer, which can guide them to enter meditation as soon as
possible and maintain their mental health.
Acknowledgments. We are grateful to Mitsuo Tsuda for his useful and constructive advice,
Genki Fujita for his valuable technical support.
References
1. Miller, G.: Why is mental illness so hard to treat? Science 80(338), 32–33 (2012)
2. Berntson, G.G., Thomas Bigger, J., Eckberg, D.L., Grossman, P., Kaufmann, P.G., Malik, M.,
Nagaraja, H.N., Porges, S.W., Saul, J.P., Stone, P.H., Van Der Molen, M.W.: Heart rate
variability: origins methods, and interpretive caveats. Psychophysiology 34(6), 623–648
(1997)
3. Shaffer, F., McCraty, R., Zerr, C.L.: A healthy heart is not a metronome: an integrative review
of the heart’s anatomy and heart rate variability. Front. Psychol. 5, 1–19 (2014)
4. Acharya, U.R., Joseph, K.P., Kannathal, N., Lim, C.M., Suri, J.S.: Heart rate variability: a
review. Med. Biol. Eng. Comput. 44, 1031–1051 (2006)
1276 A. Matsumae et al.
5. Rosenberg, W.V., Chanwimalueang, T., Adjei, T., Jaffer, U., Goverdovsky, V., Mandic, D.P.:
Resolving ambiguities in the LF/HF ratio: LF-HF scatter plots for the categorization of mental
and physical stress from HRV. Front. Physiol. 8, 1–12 (2017)
6. Matzner, S.A.: Heart rate variability during meditation. In: ECE 510 Statistical Signal
Processing, pp. 1–4 (2003)
A Consumer-Centric Approach to Understand
User’s Digital Experiences
Yeon Ji Yang, Jaehye Suk, Kee Ok Kim(&), Hyesun Hwang,
Hyewon Lim, and Muzi Xiang
Department of Consumer and Family Sciences, Sungkyunkwan University,
Seoul, Korea
[email protected]
Abstract. Life-friendly payment platforms, offered by all-in-one applications,
have transformed the way in which consumers interact with financial and non-
financial services. This study investigates user experiences in the digital finan-
cial sphere via an innovative payment application, called PAYCO, from a
consumer-centric perspective to provide practical insights to application
developers into the domain requirements and opportunities, and thereby,
improve user experience. From March 1st, 2016 to February 28th, 2019, 11,586
user-reviews of the application were collected from the Google Play website.
The results of this study revealed that the reviews shifted from negative to
positive and from those related to the major functions of the application to
multiple functions of a life-supporting payment platform. These reviews col-
lectively facilitate toward building a robust framework that addresses the
changing consumer expectations in the payment space. It is envisaged that the
adoption of this approach will help enhance the quality of human life.
Keywords: All-in-one payment application Mobile application Topic
modeling analysis Network analysis User experience
1 Introduction
Fintech provides the facility of making financial transactions via smartphone applica-
tions or through a company’s website. Payment systems benefit the most from Fintech
that allow convenient and speedy monetary transactions [1]. Powerful mobile-app-
based payments are revolutionizing the consumer experience in transactions. The
transformation of consumers’ engagement with non-bank payment apps will raise their
expectations to the same level as their expectations from any other supply side in digital
economy.
Innovation in app development should be grounded in end users’ experiences. The
challenge of what to focus on and how to develop and deliver new services requires a
consistent framework, through which a developer can evaluate present and future
technologies [2]. When consumer needs and feedback serve as that framework, inno-
vation will be set to deliver growth and value in human life in the year ahead.
Launched in August 2015, PAYCO has become one of the leading non-bank
mobile-payment providers with all-in-one fintech applications in Korea. The company
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1277–1283, 2020.
https://doi.org/10.1007/978-3-030-39512-4_195
1278 Y. J. Yang et al.
aims to transform the app into a life-friendly payment platform by offering financial and
non-financial services. The purpose of this study is to understand consumers’ digital
experiences with the innovation-driven payment application from a consumer-centric
perspective to provide practical insights to application developers into the domain
requirements and opportunities, and thereby, improve the related industry.
2 Methods
2.1 Data
Data were collected from online reviews posted on the Google Play website for the
three years from March 1, 2016 to February 28, 2019 using R 3.3.3. The total number
of collected texts was 11,691. Texts, not related to the services provided by PAYCO,
were excluded and words with the same meaning in the text were unified into the same
words. The final tidy dataset was composed of 11,586 reviews. The number of online
reviews sharply increased during the three years, from 1,261 in 2016 and 1,441 in 2017
to 7,459 in 2018.
2.2 Topic Modeling
Topic modeling was applied to understand consumers’ main concerns regarding the
payment app, and six to seven topics were extracted using the Latent Dirichlet Allo-
cation (lda) package in R 3.5.3. Topic models extend and build on classical methods in
natural language processing [3]. The lda model is a Bayesian mixture model for
discrete data where topics are assumed to be uncorrelated [4].
2.3 Network Analysis
Network analysis offers an intuitively visual way of thinking about relationships and
interactions among entities such as keywords from online reviews. Bigram analysis is
one of the commonly used to conduct the relationship between words in a text corpus
[5]. The tidytext package in R 3.5.3 was used to extract a tokenized text dataset by pairs
of adjacent words. In addition, the ggraph and igraph packages in R 3.5.3 were used to
visualize the results of network analysis.
3 Results
3.1 Topic Extraction
First, the topic modeling analysis for 2016 results in seven topics and the top rated
fifteen keywords in each topic are listed in Table 1. The seven topics are revealing
three consumer issues, named as improvements, unstable system, and privacy. Topics 1
and 4 indicate users’ wish for improvements such as demanding more various offline
A Consumer-Centric Approach to Understand User’s Digital Experiences 1279
partner shops and overcoming system errors or customer service call errors. Topics 2,
3, and 5 are revealing negative user experiences caused by the unstable system such as
transportation card charging and recognition errors in Topic 2, inconveniences due to
the authentication errors when joining the application in Topic 3, and barcode or
fingerprint recognition errors in Topic 5. Privacy issues are shown in Topics 6 and 7
with those keywords as consumers’ concerns about personal information steal when
logging in and issues of certificate errors or certificate security when using banking-
related functions such as remittance and deposit.
Table 1. Topic modeling results in 2016
Improvement Unstable system Privacy
Topic 1 Topic 4 Topic 2 Topic 3 Topic 5 Topic 6 Topic 7
1 Partner Discount Transportation Registration Usage Card Remittance
card
2 Shop Coupons Charging Card Offline Check Card Account
3 Update Authentication Card Password Fingerprint Usage Bank
4 Usage Identification Error Error Accumulate Event Usage
5 Camera First payment Bus Join Convenience Login Authentication
store
6 Functions Customer Usage Inconvenient Support Steal Amount
7 Shopping Call Recognition Input Phone type The best Function
8 Need Improvement Credit card Complete Execute Install Deposit
9 Variety Center Balance Button Galaxy Private Virtual
information
10 Offline Cancel Subway Bank Method Delete The worst
11 Service Delivery Usim Information Wallet Add Security
12 Download Weird Pay later Way Barcode Benefits Certificate
13 Login Person Delete Number Doesn’t work Error Internet
14 Connect Upgrade Apply Authentication Touch The latest Error
15 Order Mobile Doesn’t work Member Site Bank Shopping
Second, the topic modeling analysis for 2017 yielded six topics and the fifteen top
keywords in each topic are listed in Table 2. These six topics depict three consumer
issues: unstable system, benefits, and method of usage. Topics 1, 3, and 5 indicate
users’ negative experiences caused by an unstable system, including the installation
and authentication errors when users install and join the application, transportation card
charging or recognition errors, and requests for inspection or refunds. Topic 5 contains
mixed responses about payment service. Topic 2 is regarding the benefits of using the
app, such as coupons or a 3% discount. Topics 4 and 6 are related to extended usage of
the app, including partner shops in games, bookstores, and food according to the users’
lifestyles and activities with friends, such as gift giving or going to movie theaters.
1280 Y. J. Yang et al.
Table 2. Topic modeling results in 2017
Unstable system Benefits Method of usage
Topic 1 Topic 3 Topic 5 Topic 3 Topic 4 Topic 6
1 Authentication Transportation Card Discount Partner Charging
card
2 Installation Charging Registration Usage Shop Purchase
3 Update Error Convenient Error Bookstore Cafe
4 Delivery Input Coupons 3 percent Food Friend
5 Join Wallet Remittance Kakao Simple Amount
6 Error Login Offline Fingerprint Order Fee
7 Shopping Complete Account Add Credit card Gift card
8 Identification Inspection Bank Home Convenience Gift
shopping store
9 Delivery app Prepayment Error Weekend Partnership Movie
theater
10 Clothing Recognition Way Charging Lifestyle Game
11 Mobile Refund Password Ten thousand Wallet Implement
won
12 Benefits Balance Check card Bank Person Remittance
charge
13 Doesn’t work Inquiry Samsung Account Customer Culture
pay
14 Member Keyboard Event Support Center Movie
15 Delete Reservation Choice Accumulation Game Free
Third, the topic modeling analysis for 2018 yielded seven topics. All seven topics
reveal functional benefits of the app with minor negative experiences. The major
functions of the app are included in Topic 1, bank-related functions in Topic 5, and
further functions, such as ordering food or books, in Topic 3. Benefits of using the app
are revealed in Topic 4 with partner shops, Topic 6 with gift giving and Topic 7 with
discounting or coupons. Inconveniences from the automatically charged minimum
amount pop up in Topic 2 (Table 3).
Table 3. Topic modeling results in 2018
Topic 1 Topic 2 Topic 3 Topic 4 Topic 5 Topic 6 Topic 7
1 Remittance Usage Card Discount Functions Shopping Coupons
2 Free Charging Event Accumulation Account Gift Partner
3 Magazine Transportation Security Friend Registration Mobile Shop
card
4 Remittance Automatic Delivery Gift shop Bank Error Offline
charge
5 Fee Gift card Convenience Variety Inquiry Purchase Benefits
store
6 Credit rating Game Food Gifticon Samsung pay Simple Wallet
7 Information Amount People Add Authentication Internet Necessary
(continued)
A Consumer-Centric Approach to Understand User’s Digital Experiences 1281
Table 3. (continued)
Topic 1 Topic 2 Topic 3 Topic 4 Topic 5 Topic 6 Topic 7
8 3 percent Inconvenience Inquiry Cafe Certificate Products Useful
9 Management Cash Fortune Recommendation Improvement Authentication Lifestyle
10 Ticket Interconnect Bookstore Type Breakdown Company Worry
11 Frequent use Minimum Order Start Fingerprint Fun Amazing
12 Reservation Culture Delivery app Download Movie Parents Criteria
13 Installation Google play Join Clothing Update Hot deal Utilize
14 Ten thousand Credit card Method Coffee Inquiry Credit Card Discount
won
15 Times Save Check card Satisfaction Bankbook Birthday Trust
3.2 Interrelationships of Keywords
Figure 1 shows the result of the network analysis for 2016. Most of words are closely
linked to the word usage. The connection between the words usage and transportation
card shows that charging and recognition errors of transportation cards are important
issues for users. The connection between usage and offline indicates that users use this
app at offline partner shops. The word convenience linked to usage in the center of the
network connects to various benefits, such as remittance, event, (accumulate) points,
and (first payment) discount. Those words linked with usage clearly indicate positive
functional experiences among users with the app. Those words that are not associated
with usage starting from private information, such as steal, doubt, login, and limit
usage, indicate users’ severe concerns about their private information.
Fig. 1. Result of network analysis in 2016
The result of the network analysis in 2017 reveals the origins of positive and
negative user experiences with the app, as shown in Fig. 2. The major positive
experiences with the app come from points that could be accumulated with various
convenient usages or orders, and they could be used for a 3% discount or coupons in
convenient stores or in offline partner shops. The major negative experiences come
1282 Y. J. Yang et al.
from a charge of 10,000 won that requires prepayment with respect to the trans-
portation card, which in turn links to inconvenient card registration, installation,
updates, and app registers.
Fig. 2. Result of network analysis in 2017
The result of the network analysis in 2018 reveals extended benefits of using the
app, as shown in Fig. 3. The 3% discount with coupons and accumulation of points
through shopping or gift giving are the most highly appreciated among the app users.
Other major benefits of using the app come from convenient usage for remittance with
functions of transportation card, automatic charging, checking accounts, and notifica-
tion services.
Fig. 3. Result of network analysis in 2018
The results from the topic modeling and from the network analysis for the three
years of the time span reveal that users’ experiences with the app have shifted from
negative to positive and from those related to the major functions of the app to those
A Consumer-Centric Approach to Understand User’s Digital Experiences 1283
related to an all-in-one platform offering financial and non-financial services. The
network analysis with user reviews on the app adds detailed stories of user experiences
to those from the topic modeling analysis.
4 Conclusion
In this study, topic modeling and network analysis were applied to analyze consumers’
reviews of the simple payment application PAYCO during the past three years. The
results suggest that consumers’ digital experiences with the app have become more
positive. Negative experiences caused by system errors or privacy issues have quickly
disappeared, and the benefits from extended financial and non-financial services
through the app are clearly appreciated by users.
These positive experiences reveal fintech’s advances in the payment space and lead
to a new norm for consumer expectations by offering enhanced convenience and utility.
Fintech has clearly led the simple payment services to become the most intensive users
of personal data. Protection issues regarding consumer privacy and responsible use of
consumer data become even more pivotal in relation to the functioning of fintech
services.
This study shows that text mining with topic modeling and network analysis can
help app developers to understand users’ daily experience with an app. Bridging the
gap between users and app developers and, further, between social scientists and
computer scientists is challenging. A technical perspective and a human perspective
need to interact intimately to deliver growth and value to society.
The limitations of this study include a lack of Korean dictionary for identifying and
extracting words or removing stop words, and an agreed upon selection criteria for the
number of topics. To gain valid and valuable insights into consumer experience with
mobile apps, more-robust data-cleaning and text-mining procedures are needed.
References
1. Bates, R.: Banking on the future: an exploration of fintech and the consumer interest.
A Report for Consumers International (2017)
2. Sahn, B.: Customer experience at the centre of innovation agenda. FinTech Futures. https://
www.fintechfutures.com/2018/02/customer-experience-at-the-centre-of-innovation-agenda/
3. Hornik, K., Grün, B.: Topicmodels: an R package for fitting topic models. J. Stat. Softw. 40
(13), 1–30 (2011)
4. Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and
unlabeled documents using EM. Mach. Learn. 39(2–3), 103–134 (2000)
5. Qasim, M.: Sustainability and wellbeing: a text analysis of New Zealand parliamentary
debates, official yearbooks and ministerial documents, No. 19/01 (2019)
Research on Design Skills for Personnel
Evaluation Systems and Educational Programs
Toshiya Sasaki(&)
X Design Academy, 4-1-35, Kudankita, Chiyoda-Ku, Tokyo 102-0073, Japan
[email protected]
Abstract. In this research, we researched the next generation design skills and the
current evaluation system, and developed a prototype for the evaluation system.
Keywords: Design thinking Service design Computational design Human
centered design Design skills
1 Introduction
In recent years, businesses using data and artificial intelligence are spreading into
society. With the trend of this era, the role of design in organizations and the required
design skills are diversified. The purpose of this study is to organize design skills and
use them as indicators for organizational personnel evaluation systems and educational
programs.
2 Approach
In this study, we investigate design skills and repeat hypothesis verification in per-
sonnel evaluation systems and educational programs based on prototypes created based
on hypotheses (Fig. 1).
Fig. 1. Approach
3 Definition of “Design Skills”
Design skills are the ability to think and practice to create the strategy, structure and
appearance of services and products in IT companies.
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1284–1288, 2020.
https://doi.org/10.1007/978-3-030-39512-4_196
Research on Design Skills for Personnel Evaluation Systems and Educational Programs 1285
4 Investigation of Design Skills
The concept of design was organized from the viewpoint of design definition [1],
design type, and design process [2] (Fig. 2). The elements of 31 design skills obtained
from the survey of design skills were classified into 10 design skills [3].
Fig. 2. Organizing design concepts
5 Hypotheses and Prototypes
Since the skills required for each member differ depending on the organization, a
typical designer pattern was created [4]. We thought that it could be used as an index
for personnel evaluation by setting necessary skills for each pattern. The “Design Skill
Sheet” and The “Designer Pattern Sheet” were created as prototypes. In the “Design
Skill Sheet”, when the proficiency level of 31 skill elements is input, the proficiency
level of 10 design skills is visualized. This tool is supposed to be used for diagnosing
skill proficiency when performing personnel evaluations and for setting goals (Fig. 3).
The “Designer Pattern Sheet” visualizes the proficiency level of skills required for each
designer pattern within six career levels. It is assumed that career levels, required skills,
skill proficiency levels, etc. will be arranged by the organization and used when
grasping the required skills from patterns (Fig. 4).
1286 T. Sasaki
Fig. 3. Design Skill Sheet
6 Hypothesis Testing
A workshop to visualize design skills was held for eight IT company designers. As a
result of the workshop, the following problems were found.
• It is difficult to understand the explanation of each skill element
• It is difficult to match language recognition among members
• Perception of proficiency varies from person to person
Research on Design Skills for Personnel Evaluation Systems and Educational Programs 1287
Fig. 4. Designer Pattern Sheet
In the prototype for personnel evaluation system, it will be considered to make it
easy to understand the explanation of each skill and element, and to make it easy to
match the proficiency level.
7 The Next Deployment
Regarding education programs, the necessary skills are organized at each step of the
design process, and the priority of skills required for the members of the target orga-
nization is considered and the education programs are examined.
References
1. Ministry of Economy: Trade and Industry/Patent Office Study Group Considering Industrial
Competitiveness and Design, Declaration of “Design Management” (2018)
2. Norman, D.A.: The Design of Everyday Things: Revised and Expanded Edition (2015)
1288 T. Sasaki
3. Human Centered Design Organization, HCD Competence Map (2018)
4. Merholz, P., Skinner, K.: Org Design for Design Orgs - Building and Managing In-House
Design Teams (2016)
Author Index
A Basapur, Santosh, 573
Abdennadher, Slim, 208 Beggiato, Matthias, 932
Acosta-Vargas, Gloria, 875 Bellotti, Andrea, 260
Acosta-Vargas, Patricia, 853, 875 Benelli, Elisabetta, 241
Afanasyev, Vladimir, 307 Bengler, Klaus, 73
Agarwal, Shivangi, 1028 Berberoğlu, Mehmet, 646
Ajdari, Alireza, 409 Bernal-Bazalar, Michael, 697
Alava Portugal, Cyntia, 1201 Berrú, Julio César Ortíz, 421
Alexandris, Christina, 276 Bertacchini, Alessandro, 803
Alkuino, Glenn, 871 Bettelli, Valentina, 816
Altamirano, Ernesto, 683 Bielecki, Konrad, 1007
Alvarez-Merino, José, 683 Bilyk, Rostyslav, 439
Amasifén-Pacheco, Diego, 711 Bimenyimana, Christian Ildegard, 667
Ambeck-Madsen, Jonas, 260 Bogg, Adam, 938
Amrousi, Mohamed El, 1145 Bolock, Alia El, 208
Andajani, Sari, 753 Borghi, Guido, 104
Angulo-Baca, Alejandra, 697 Boronenko, Marina, 365, 398
Antopolskiy, Sergey, 260 Boronenko, Yurii, 398
Arcoraci, Andrea, 34 Bozhikova, Violeta, 480
Asai, Haruna, 111 Bradley, Mike, 20
Ascari, Luca, 260 Brechmann, André, 159
Aston, Jeremy, 118 Bromfield, Mike, 938
Astudillo, Catalina, 392 Buddhan, Sasikumar, 1213
Avanzini, Pietro, 260 Burgio, Paolo, 653
Avila Beneras, Josefina, 1256 Burov, Oleksandr, 282, 359
Buss, Peter, 404
B Butturi, Maria Angela, 474
Baba, Akira, 385
Bacciaglia, Antonio, 796 C
Bachechi, Chiara, 468 Cadzow, Scott, 15
Baghramyan, Aleksandr, 172 Cai, Xijiang, 1219
Baier, Ralph, 1007 Caira-Jimenez, Manuel, 690
Balugani, Elia, 474 Calogero, Angela Lucia, 214
Barbi, Silvia, 803 Campa, Francesco, 214
Bardales, Z., 660 Camplone, Stefania, 835
© Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1289–1295, 2020.
https://doi.org/10.1007/978-3-030-39512-4
1290 Author Index
Canales-Ramos, Lourdes, 746 Dovramadjiev, Tihomir, 480
Cano-Lazarte, Mercedes, 740 Dudek, Marius, 966
Caon, Maurizio, 66
Castellano, Andrea, 653, 1048 E
Castelli, Andrea, 266 Echevarria-Cahuas, Jose, 780
Castro-Blancas, Audy, 719 Elhakeem, Mohamed, 1145
Celi Costa, Rosana María, 510 Etinger, Darko, 486
Ceruti, Alessandro, 796 Eurich, Sky O., 1028
Cespedes-Blanco, Carlos, 719 Evreinov, Grigori, 882
Chaiklieng, Sunisa, 372, 753
Chang, Mi, 1132, 1138 F
Chatelais, Bénédicte, 177 Fahimipirehgalin, Mina, 341
Chavez-Soriano, Pedro, 746 Fandialan, Ervin, 871
Chen, Chun Hsien, 183 Farfan-Quintanilla, Zhelenn, 690
Chen, Jianfeng, 453, 532 Farooq, Ahmed, 882
Chen, Xuebo, 415 Favaro, Francesca, 80, 945, 1028
Chen, Yiyan, 766 Fei, Dingzhou, 271, 433
Chen, Yu, 539 Ferrara, Giannina, 190
Cheng, Jianxin, 136 Ferrara, Marinella, 909
Cheung, Virginia, 129 Filieri, Jurji, 241
Chi, Maoping, 1244 Finke, Jan, 153
Choi, IlYoung, 1178 Fleming, Lorraine, 335
Chung, Jae-Eun, 865 Flemisch, Frank, 1007
Clark, Jed, 20 Fois Lugo, Milagros, 1201, 1256
Co, Nguyen Trong, 1107 Fois, María Milagros, 1195
Colorado Pástor, Bryan, 1164 Fontanili, Luca, 1088
Colucciello, Alessia, 260 Forero, Boris, 546, 1094
Contreras-Choccata, Denilson, 760 Fossanetti, Massimo, 1048
Crespo-Martínez, Esteban, 392, 889 Franco Puga, Julio, 1164
Cucchiara, Rita, 104 Franke, Katharina, 973
Cuffie, Brandon, 165 Freire, Rui Pedro, 118
Cui, Jiaqi, 1219
G
D Gagné, Christian, 177
Dalpadulo, Enrico, 789, 816 Galvez-Acevedo, Oscar, 740
Davtyan, Kristina, 80 Garay-Osorio, Angela, 711
de Antonio, Angélica, 254 Garcia-Montero, Diana, 704
de S. Dunck, Danillo A., 896 Geniola, Viola, 835
Del Vecchio, Maria, 260 Gherardini, Francesco, 816
Delaigue, Pierre, 66 Ghonaim, Ahmed, 208
Dell‘Orfano, Vittorio, 266 Giacobone, Gian Andrea, 59, 823, 829
Demierre, Marc, 66 Giraldi, Laura, 46
Demirci, Hatice Merve, 646 Goh, Seng Yuen Marcus, 605
Deng, YuanZhou, 1184 Gomes, Neil, 561
Desimoni, Federico, 468 Gómez Rodríguez, Víctor Gustavo, 1119
Dhamodharan, Tamilselvi, 524, 1213 González Calle, María José, 510
Di Caprio, Debora, 235 Gonzalez, Mario, 875
Di Salvo, Andrea, 34 Grasso, Giorgio M., 41
Díaz, Sergio, 653 Gremillion, Gregory, 999
Dimova, Rozalina, 480 Grigoletto, Alessia, 214
Djanatliev, Anatoli, 92 Grunstein, Daniel, 27
Dolgikh, Yana, 282 Grunstein, Ron, 27
Author Index 1291
Gu, Rongrong, 427 J
Guan, Cheng, 319 Jansone, Anita, 1113, 1233
Guaña-Moya, Javier, 875 Jaramillo, Félix, 546
Jaramillo, Robinson Vega, 546
H Jeong, Yun Jik, 1171
Ha, Hee Ra, 1184 Jinno, Yasuyoshi, 503
Hafez, Wael, 627 Jost, Jana, 153
Hains, Alexandre, 177 JothiKumar, Karthikeyan, 1213
Hambardzumyan, Larisa, 172 Jung, Minji, 865
Hämmerle, Moritz, 1069
Han, Ye, 453 K
Hara, Tatsunori, 294 Kaindl, Hermann, 9
Harvey, Eric R., 201 Kambayashi, Yasushi, 1041
Hashimoto, Yoshihiro, 111, 143, 301 Kanenishi, Kazuhide, 677
He, Da, 532 Karandey, Vladimir, 307
He, JinLu, 1178 Kedari, Santosh, 325, 348
Hechavarría Hernández, Jesús Rafael, 546, Kedari, Sharat, 325, 348
1094, 1119, 1164, 1190, 1201, 1206, Khaled, Omar Abou, 66
1256 Khymych, Iryna, 439
Heilemann, Felix, 1014 Kim, Ahyoung, 865
Hengeveld, Bart, 810 Kim, JaeKyeong, 1178
Herbert, Cornelia, 208 Kim, Jenna, 190
Hernández, Daniel López, 1007 Kim, Jisun, 20, 980
Herzberger, Nicolas Daniel, 1007 Kim, Kee Ok, 1171, 1277
Hidalgo, Jairo, 446 Kim, Meereh, 1132
Hidalgo, Paula, 875 Kirks, Thomas, 153
Hlazunova, Olena, 282 Kiseleva, Elizaveta, 365, 398
Ho, Bach Q., 294 Klapper, Jessica, 1069
Hoang, Ngo Thanh, 1107 Klaproth, Oliver W., 1021
Holzbach, Markus, 916 Kolbachev, Evgeny, 288
Hong, Sukjoo, 1132 Kondo, Akiko, 677, 1082
Hoose, Sebastian, 153 Konieczna, Monika, 1225
Hosokawa, Youichi, 301 Krems, Josef, 932
Hounsou, Joël Toyigbé, 667 Krylova-Grek, Yuliya, 359
Hrynkevych, Olha, 439 Kuuseok, Ahto, 619
Hu, Jun, 810 Kyzenko, Vasyl, 282
Hu, Lingling, 313
Hua, Jenna, 190 L
Huanca, Lucio Leo Verástegui, 421 La Fleur, Claire, 999
Huang, Fei-Hui, 1152 Laarni, Jari, 1057
Huber, Bernd, 92 Lafond, Daniel, 177, 201
Hummels, Caroline, 810 Lai, Po Yan, 1138
Hussainalikhan, NilofarNisa, 1213 Landini, Elisa, 653, 1048
Hwang, Hyesun, 633, 1277 Langdon, Pat, 20
Laßmann, Paula, 3
I Latessa, Pasqualino Maietta, 214
Ihara, Masayuki, 588 Lauer, Tim, 973
Ilapakurti, Anitha, 325, 348 Lavrov, Evgeniy, 282, 359
Imanghaliyeva, Amangul A., 580, 1074 Lavrova, Olga, 282
Imbesi, Silvia, 823, 829, 860 Leali, Francesco, 789, 816
Inomae, Goro, 588 Lee, Ji-Hyun, 1132, 1138
Isaeva, Oksana, 365, 398 Lee, Jun Hee, 1138
Ishizaka, Alessio, 474 Lee, Seonglim, 1184
1292 Author Index
Lee, Subin, 600 Martin, Bruno, 201
Lee, Tony, 129 Martinez-Castañon, Jose, 740
Lee, Yu Lim, 865 Mascia, Maria Teresa, 816
Leikas, Jaana, 594 Masters, Matthew, 959
Leon-Chavarril, Claudia, 733 Matsumae, Akane, 1270
Leung, Michael, 129 Mazzeo, Elena, 266
Leyva, Maikel, 1164 McKenna, H. Patricia, 1063
Li, Cun, 810 Medina, Marvin, 871
Li, Fan, 183 Metcalfe, Jason S., 999
Li, Xian, 248 Meyer, Ronald, 1007
Li, Xu, 633 Mikros, George, 276
Li, Yangxu, 294 Milani, Massimo, 1088
Li, Zhigang, 415 Mincolelli, Giuseppe, 59, 823, 829, 860
Liao, Rui, 1101 Miranda Zamora, William Rolando, 498
Lien, Tran Thi Phuong, 1107 Miwa, Tetsushi, 301
Lim, Hyewon, 633, 1277 Miyake, Yutaka, 385
Lin, Shuhao, 190 Mogrovejo, Pedro, 392, 889
Lindner, Sebastian, 925, 966 Mohamed, Mona A., 842
Lisi, Giuseppe, 111, 143, 301 Montalvo, Lourdes, 492
Liu, Jie, 453, 532 Montanari, Roberto, 1048
Liu, Jing, 848 Montorsi, Luca, 1088
Liu, Jinjing, 640 Montorsi, Monia, 803
Liu, Wenfeng, 766 Moscoso, Karla, 1195
Liverani, Alfredo, 796 Motomura, Yuki, 1270
Loch, Frieder, 341 Mouzakitis, Alex, 20
Lolli, Francesco, 474 Mrugalska, Beata, 1225
Lommerzheim, Marcel, 159 Mugellini, Elena, 66
Lucifora, Chiara, 41 Mukaihira, Koki, 503
Lüdtke, Andreas, 52 Müller, Juliane, 952
Luján-Mora, Sergio, 254, 853 Muncie, Helen, 986
Luo, Ruiyao, 1270 Mund, Dennis, 959
Luo, Yun, 1101 Murray, Acklyn, 335
Lye, Sun Woh, 183, 221, 605 Muthammal, Swashi, 524
Lytvynova, Svitlana, 359 Muthuramalingam, BhavaniDevi, 1213
Muzzioli, Gabriele, 1088
M
Mackare, Kristine, 1113
Mackars, Raivo, 1113 N
Madhusudanan, Ajith, 1213 Nader, Nazanin, 945
Magnone, Francesco Giuseppe, 1119 Naghdbishi, Hamid, 409
Maier, Thomas, 3 Nah, Ken, 600, 640, 848
Malca-Ramirez, Carlos, 683 Nakajima, Hiroshi, 588
Maldonado-Maldonado, Juan Manuel, 392 Naranjo, Katherine, 546
Maldonado-Matute, Juan Manuel, 510 Neto, Guilherme F., 896
Manzoni, Luigi, 214 Neubauer, Catherine, 999
Maradiegue, F., 660 Ngoc, Trinh Minh, 1107
Marano, Antonio, 835 Niermann, Dario, 52
Marcano, Mauricio, 653 Nishigaki, Masakatsu, 385, 503
Marchenkova, Anna, 260 Nishimura, Eigo, 1270
Marchi, Michele, 823, 829 Niu, Yafeng, 1219
Marois, Alexandre, 201 Nsabimana, Thierry, 667
Author Index 1293
Nuñez-Salome, Luis, 683 Q
Nyame, Gabriel, 1126 Qin, Zhiguang, 1126
Qiu, Lincun, 1219
Quispe-Huapaya, Maria, 780
O
O-Donoghue, Jim, 20
R
Odumuyiwa, Victor, 667
Radlmayr, Jonas, 73
Oetting, Andreas, 992
Rahman, S. M. Mizanoor, 612, 1034
Ogata, Wakaha, 503
Raisamo, Roope, 882
Ohki, Tetsushi, 503
Rajendran, Gopi Krishnan, 524
Ohki, Tetushi, 385
Ramar, Ramalakshmi, 524
Okui, Norihiro, 385
Ramdoss, Sathiyaprakash, 1213
Orliyk, Olena, 359
Ramírez, Jaime, 254
Ota, Jun, 294
Ramirez-Valdivia, Cesar, 780
Ottenstein, Maia, 561
Ramos-Palomino, Edgar, 704
Rao, Zhihong, 453, 532
P Rauh, Nadine, 932
Palomino-Moya, Jose, 733 Ravichandran, Sugirtha, 1213
Panukhnyk, Olena, 439 Raymundo, Carlos, 719, 780
Parisi, Stefano, 916 Raymundo-Ibañez, Carlos, 660, 690, 697, 711,
Park, In-Hyoung, 865 733, 740, 746, 760
Parkes, Andrew, 938 Reichelt, Florian, 3
Pasetti, Chiara, 909 Ren, Jiapei, 1238
Pasko, Nadiia, 282 Revell, Kirsten, 20, 980
Patterson, Wayne, 335 Richardson, Joy, 20, 980
Patwardhan, Viraj, 561 Rivas-Zavaleta, Carlos, 719
Paul, Gunther, 9 Rivera, Luis, 660, 711, 719, 760, 780
Pavlidis, Evgeni, 959 Rizvi, Syeda, 945
Peeters, Thomas, 195 Roberts, Aaron, 20
Perconti, Pietro, 41 Rognoli, Valentina, 916
Perelman, Brandon S., 999 Rollo, Federica, 468
Pérez, Joshue, 653 Roman-Ramirez, Luz, 704
Perez, Moises, 690, 697, 733, 740, 746 Rossi, Emilio, 835
Perez-Paredes, Maribel, 711 Ruiz, Maria, 1195
Petrenko, Sergiy, 359 Russwinkel, Nele, 159, 1021
Petruccioli, Andrea, 816 Rutitis, Didzis, 379
Pinheiro, Jean-Philippe, 221
Pini, Fabio, 789 S
Pini, Stefano, 104 Saariluoma, Pertti, 594
Pisaniello, John Dean, 404 Salvador-Acosta, Belén, 875
Plebe, Alessio, 41 Salvador-Ullauri, Luis, 853
Po, Laura, 468 Sanchez Chero, Jose Antonio, 498
Pokorni, Bastian, 1069 Sanchez Chero, Manuel Jesus, 498
Politis, Ioannis, 20 Sánchez, Fernanda, 546
Popadynets, Nazariy, 439 Santos-Arteaga, Francisco J., 235
Popov, Boris, 307 SaravanaSundharam, SaiNaveenaSri, 1213
Popova, Olga, 307 Sasaki, Toshiya, 1284
Portilla Castell, Yoenia, 1119 Sato, Keiichi, 573
Pourbafrani, Mahsa, 461 Scataglini, Sofia, 195
Prezenski, Sabine, 159 Schaefer, Kristin E., 999
1294 Author Index
Schmidl, Paul, 92 Tingey-Holyoak, Joanne, 404
Schulte, Axel, 925, 952, 959, 966, 1014 Tito, P., 660
Schwerd, Simon, 925 Tkachenko, Oleksii M., 359
Sekido, Taichi, 1041 Togawa, Satoshi, 677, 1082
Sellan, Karen, 1195 Toni, Rita, 214
Sellitto, Miguel Afonso, 474 Toselli, Stefania, 214
Seto, Edmund, 190 Tremblay, Sébastien, 201
Shao, Junkai, 248, 313, 1238 Trignano, Claudia, 266
Shen, Jie, 726 Tripi, Ferdinando, 214
Shevchenko, Svitlana, 359 Tsuchiya, Takashi, 503
Shi, Bingzheng, 1219 Tupot, Marie Lena, 567
Shiomi, Yuya, 385 Turlisova, Jelena, 1233
Shuhaiber, Ahmed, 903
Sidorova, Elena, 288
U
Sippl, Christoph, 92
Üyümez, Bilal, 992
Skrypchuk, Lee, 20
Son, Hoang Huu, 1107
Song, Xiao Xi, 1184 V
Sorochak, Oleg, 439 Vachon, François, 201
Sosa-Perez, Valeria, 733 Vadivel, VaibhavaShivani, 1213
Sotelo-Raffo, Fernando, 690, 704 Valavani, Christina, 276
Sotelo-Raffo, Juan, 697, 760 Valero Fajardo, Carlos Luis, 1190
Soto, Billy, 546 van der Aalst, Wil M. P., 461
Sou, Ka Lon, 605 Van Goethem, Sander, 195
Stanton, Neville A., 20, 980 van Zelst, Sebastiaan J., 461
Stephane, Lucas, 165 Vaneeva, Polina, 288
Stimm, Dominique, 3 Varotti, Davide, 214
Stock, Timothy J., 567 Vecchiato, Giovanni, 260
Stoeva, Mariana, 480 Vega Jaramillo, Robinson, 1094
Stoppa, Marcelo H., 896 Velasquez-Vargas, Arelis, 746
Storchi, Gabriele, 1088 Vergnano, Alberto, 214
Suarez, Diego S., 889 Vernaleken, Christoph, 1021
Sudo, Fumiya, 143 Verwulgen, Stijn, 195
Suggaravetsiri, Pornnapa, 372, 753 Vezzani, Roberto, 104
Sugimoto, Ayaka, 385 Vignati, Arianna, 228
Suhir, Ephraim, 9 Viktoriia, Yazina, 439
Suk, Jaehye, 1171, 1184, 1277 Villanueva, Edwin, 492
Suresh, Pavika, 1213 Vogel-Heuser, Birgit, 341
Volkova, Tatjana, 379
T Vuppalapati, Chandrasekar, 325, 348
Takahashi, Kenta, 503 Vuppalapati, Jayashankar, 325, 348
Takimoto, Munehiro, 1041 Vuppalapati, Rajasekar, 325, 348
Talwar, Palak, 86
Tan, Shi Yin, 183 W
Tang, Simon, 129 Wang, Fenghong, 766
Tang, Wencheng, 1250 Wang, Haiyan, 248, 1238
Tango, Fabio, 653, 1048 Wang, Huajie, 136
Tempesta, Antonio, 214 Wang, Xia, 433
Tenemaza, Maritzol, 254 Wang, Yifan, 453, 532
Terenzi, Benedetta, 228 Wang, Yongbin, 1159
Ter-Sargisova, Viktoria, 172 Wang, Yun, 1270
Thomas, Manju, 1213 Wang, Zhangyu, 129
Thompson, Simon, 20 Wasser, Joscha, 1007
Tin, Man Lai-man, 517 Watanabe, Hiroshi, 588
Author Index 1295
Wee, Hong Jie, 221 Ylönen, Marja, 1057
Wiersma, Ben, 404 Yu, Dehua, 552
Wu, Zhen, 124, 773 Yu, Jian, 1159
Wu, Zheng, 99
Wyrwicka, Magdalena K., 1225 Z
Zacherl, Larissa, 73
X Zambrano, Milton, 1206
Xiang, Muzi, 1171, 1277 Zarabia, Omar, 254
Xie, Yi, 1219 Zelensky, Vladimir, 365
Xu, Hanyang, 1263 Zeng, Xiang, 319
Xu, Hong, 605 Zhai, Binhong, 124, 773
Xue, Chengqi, 313, 773, 1219, 1263 Zhang, Chi, 99, 773
Zhang, Lu, 1101
Y Zhang, Qiang, 129
Yandún, Marco, 446 Zhang, Tianmai, 1250
Yang, Yeon Ji, 1171, 1277 Zhang, Tongtong, 319
Yang, Zining, 129 Zhao, Shifeng, 726
Yarlequé, Cristhian Aldana, 421 Zhou, Jia, 415
Yeo, Harim, 633 Zhou, Lei, 319, 1244
Yi, Taeha, 1132, 1138 Zhou, Wuzhong, 427
Yin, Goudong, 99 Zhou, Xiaozhou, 1263
Yin, Guodong, 124 Zhu, Yanfei, 773
You might also like
- Fusion of Neural Networks Fuzzy Systems and Genetic Algorithms - Industrial ApplicationsNo ratings yetFusion of Neural Networks Fuzzy Systems and Genetic Algorithms - Industrial Applications297 pages
- Using Machine Learning & Deep Learning With Arcgis Imagery: Kannan Jayaraman GistecNo ratings yetUsing Machine Learning & Deep Learning With Arcgis Imagery: Kannan Jayaraman Gistec28 pages
- Dzone Rc251 GettingstartedwithtensorflowNo ratings yetDzone Rc251 Gettingstartedwithtensorflow5 pages
- Applied Computer Vision and Soft Computing With - Swati V - Shinde, Darshan V - Medhane, Oscar Castillo - 2024 - CRC Press - 9781003359456 - Anna's ArchiveNo ratings yetApplied Computer Vision and Soft Computing With - Swati V - Shinde, Darshan V - Medhane, Oscar Castillo - 2024 - CRC Press - 9781003359456 - Anna's Archive333 pages
- Industrial Revolution of The 21st CenturyNo ratings yetIndustrial Revolution of The 21st Century74 pages
- Machine Vision and Augmented Intelligence - Theory and Applications100% (1)Machine Vision and Augmented Intelligence - Theory and Applications681 pages
- Advanced Technologies For Industrial Applications-Springer (2023)No ratings yetAdvanced Technologies For Industrial Applications-Springer (2023)105 pages
- Renewable Energy: RD&D Priorities: Insights From IEA Technology Programmes0% (1)Renewable Energy: RD&D Priorities: Insights From IEA Technology Programmes224 pages
- 2013 Book SelectedTopicsInNonlinearDynamNo ratings yet2013 Book SelectedTopicsInNonlinearDynam481 pages
- Awasthi K. Mathematical Modeling For Intelligent Systems 20220% (1)Awasthi K. Mathematical Modeling For Intelligent Systems 2022259 pages
- Quantum Technology and Optimization Problems: Sebastian Feld Claudia Linnhoff-PopienNo ratings yetQuantum Technology and Optimization Problems: Sebastian Feld Claudia Linnhoff-Popien234 pages
- (Mohamed Ibnkahla) Cooperative Cognitive Radio Net PDF100% (2)(Mohamed Ibnkahla) Cooperative Cognitive Radio Net PDF342 pages
- Online Creativity in Engineering Education PG 528 2021100% (1)Online Creativity in Engineering Education PG 528 20211,043 pages
- Intelligent Electrical Systems A Step Towards Smarter Earth 2021No ratings yetIntelligent Electrical Systems A Step Towards Smarter Earth 2021375 pages
- Simscape Modeling of A Custom Closed-Volume TankNo ratings yetSimscape Modeling of A Custom Closed-Volume Tank8 pages
- Application of Artificial Neural Networks in Geoinformatics PDFNo ratings yetApplication of Artificial Neural Networks in Geoinformatics PDF230 pages
- Multi Sensor Fusion and Integration Seminar ReportNo ratings yetMulti Sensor Fusion and Integration Seminar Report23 pages
- Verification Techniques and Tools For Reliable Cyber-Physical SystemsNo ratings yetVerification Techniques and Tools For Reliable Cyber-Physical Systems40 pages
- Dokumen - Pub Wireless Ai Wireless Sensing Positioning Iot and Communications 9781108597234No ratings yetDokumen - Pub Wireless Ai Wireless Sensing Positioning Iot and Communications 9781108597234624 pages
- Cai, Baoping - Bayesian Networks in Fault Diagnosis Practice and Application-World Scientific Publishing (2019) PDFNo ratings yetCai, Baoping - Bayesian Networks in Fault Diagnosis Practice and Application-World Scientific Publishing (2019) PDF418 pages
- (Studies in big data 30) Ashour, Amira_ Bhatt, Chintan_ Dey, Nilanjan_ Hassanien, Aboul Ella_ Satapathy, Suresh Chandra - Internet of things and big data analytics toward next-generation intelligence-.pdf100% (1)(Studies in big data 30) Ashour, Amira_ Bhatt, Chintan_ Dey, Nilanjan_ Hassanien, Aboul Ella_ Satapathy, Suresh Chandra - Internet of things and big data analytics toward next-generation intelligence-.pdf545 pages
- Jacak Intelligent Robotic Systems. Design, Planning, and Control100% (8)Jacak Intelligent Robotic Systems. Design, Planning, and Control321 pages
- Advances in Intelligent Robotics and Collaborative Automation2No ratings yetAdvances in Intelligent Robotics and Collaborative Automation2363 pages
- Computational Electromagnetics and Model-Based Inversion - A Modern Paradigm For Eddy-Current Nondestructive Evaluation (PDFDrive)No ratings yetComputational Electromagnetics and Model-Based Inversion - A Modern Paradigm For Eddy-Current Nondestructive Evaluation (PDFDrive)451 pages
- 2018 Book IntelligentHumanSystemsIntegraNo ratings yet2018 Book IntelligentHumanSystemsIntegra791 pages
- Advances in Artificial Intelligence Software and Systems Engineering 1st Edition by Tareq Ahram 3030513270 9783030513276 PDF Download100% (4)Advances in Artificial Intelligence Software and Systems Engineering 1st Edition by Tareq Ahram 3030513270 9783030513276 PDF Download75 pages
- Advances in Simulation and Digital Human Modeling CompressNo ratings yetAdvances in Simulation and Digital Human Modeling Compress354 pages
- Advances in Human Factors in Architecture Sustainable Urban Planning and Infrastructure Proceedings of the AHFE 2020 Virtual Conference on Human Factors in Architecture Sustainable Urban Planning and Infrastructure 16 20 July 2020 USA Jerzy Charytonowicz instant downloadNo ratings yetAdvances in Human Factors in Architecture Sustainable Urban Planning and Infrastructure Proceedings of the AHFE 2020 Virtual Conference on Human Factors in Architecture Sustainable Urban Planning and Infrastructure 16 20 July 2020 USA Jerzy Charytonowicz instant download150 pages
- Intelligent Human Systems Integration Proceedings of the 1st International Conference on Intelligent Human Systems Integration IHSI 2018 Integrating People and Intelligent Systems January 7 9 2018 Dubai United Arab Emirates 1st Edition Waldemar Karwowski downloadNo ratings yetIntelligent Human Systems Integration Proceedings of the 1st International Conference on Intelligent Human Systems Integration IHSI 2018 Integrating People and Intelligent Systems January 7 9 2018 Dubai United Arab Emirates 1st Edition Waldemar Karwowski download116 pages
- Progress in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2019, Volume 1 Chhabi Rani Panigrahi downloadNo ratings yetProgress in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2019, Volume 1 Chhabi Rani Panigrahi download92 pages
- Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2020 Aboul Ella Hassanien downloadNo ratings yetProceedings of the International Conference on Advanced Intelligent Systems and Informatics 2020 Aboul Ella Hassanien download161 pages
- Progress in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2019, Volume 2 Chhabi Rani Panigrahi instant downloadNo ratings yetProgress in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2019, Volume 2 Chhabi Rani Panigrahi instant download91 pages
- Trends and Applications in Information Systems and Technologies 2021No ratings yetTrends and Applications in Information Systems and Technologies 2021602 pages
- 2019 Bookmatter ComputationalIntelligenceTheor PDFNo ratings yet2019 Bookmatter ComputationalIntelligenceTheor PDF11 pages
- 2022 - Scaling Up Renewable Energy in Africa - Accompanying ChartsNo ratings yet2022 - Scaling Up Renewable Energy in Africa - Accompanying Charts31 pages
- Construction Technology 2 Notes - Copy (1) 2023No ratings yetConstruction Technology 2 Notes - Copy (1) 202321 pages
- Men's and Boys' Knitted and Woven Beachwear and Sports Shirt FabricsNo ratings yetMen's and Boys' Knitted and Woven Beachwear and Sports Shirt Fabrics4 pages
- Modellingturbulenceinengineering Andtheenvironment Second-Moment Routes To ClosureNo ratings yetModellingturbulenceinengineering Andtheenvironment Second-Moment Routes To Closure4 pages
- CFD Modelling of A Two-Phase Jet AeratorNo ratings yetCFD Modelling of A Two-Phase Jet Aerator13 pages
- Oracle Fusion Cloud SCM - 25B Release UpdateNo ratings yetOracle Fusion Cloud SCM - 25B Release Update53 pages
- Finance Allied Workers Union V Bank of Baroda (U) Limited 2025 UGHCCD 42 (26 February 2025)No ratings yetFinance Allied Workers Union V Bank of Baroda (U) Limited 2025 UGHCCD 42 (26 February 2025)7 pages
