Pedro Luis Mateo Navarro, profile picture
Uploaded byPedro Luis Mateo Navarro
3,350 views

PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

The document outlines a PhD thesis by Pedro Luis Mateo Navarro focused on enhancing software quality through improved user interface testing and user experience evaluation. It introduces two main approaches: one dissecting user-system interactions into components for targeted assessment, and another that evaluates the interaction process as a whole, alongside methodologies for verifying system responses and user inputs during runtime. Key solutions include the Open HMI Tester for GUI testing and the Carim model for analyzing mobile user experiences with contextual data.

Downloaded 159 times
Enhancing Software Quality and Quality of Experience through User Interfaces Pedro Luis Mateo Navarro Advisors: Dr. Diego Sevilla Ruiz, Dr. Gregorio Mart´ınez P´erez Murcia, July 2014 Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 1
Agenda Enhancing Software Quality and Quality of Experience through User Interfaces I. Motivation, Goals, and Context II. Solutions Proposed in this PhD Thesis III. Conclusions and Further Work IV. Contributions Summary Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 2
I Motivation, Goals, and Context
Table of contents I. Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 4
Quality of Software and Interaction Quality of Software is a broad concept... encompasses many disciplines: from technical to psychological can be achieved in many ways: checking functional requirements of software evaluating users perceived quality etc. This thesis focuses on User Interfaces... ...a field in which there is too much work to do Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 5
User interfaces The link between the human and the machine... users interact with the interface → they can see/touch/feel it system provide responses through the interface mean of communication User interfaces are essential; guaranteeing their quality is critical Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 6
Software Testing Processes to guarantee software and interfaces quality... a system or application works as expected? a system or application works within its context? guarantee that the requirements are met Testing is essential before a software is released Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 7
Human-Computer Interaction (HCI) Assessment HCI assessment ⇔ discipline within Software Testing Study of a conversational process involving: user → has a task to accomplish; provides input system → processes data to elaborate a response context → surrounds them; affects behavior and response Interaction has changed in the past years Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 8
What is Good-Quality Software? Quality software ⇔ requirements are met The software fulfills: functional requirements → the behavior is the expected non-functional requirements → structural quality Quality can also be measured in terms of Usability and Interaction Experience Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 11
Software Usability & Quality of Experience (QoE) Usability: How easy it is to use a software + how easy it is to learn how to use that software assure software effectiveness (is it useful?) and efficiency assure user satisfaction QoE: Subjective measure of users experiences with a software based on user perception and user feelings encompasses user behaviors + attitudes + emotions dynamic → changing context/circumstances Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 12
Achieving Good-Quality Software? Different ways to achieve software quality We try to narrow the problem... focusing on user interfaces analysis of user-system interaction in real-time How can we assess interaction? Since interaction = user inputs + system outputs then... ...we posed 2 different approaches Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 13
Achieving Software Quality through User Interfaces Approach 1: Decompose interaction into its components main components ⇒ input + output components quality is enhanced separately analysis/testing processes → more focused, more effective Approach 2: Assess interaction process as a whole whole dialog → keeps totality of the process conversational nature → cause-effect relationships analysis of external factors ⇒ context Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 14
Table of contents I. Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 15
Goal of this PhD Thesis Improve methods for software testing and interaction analysis design better tools → +effective, +robust design easy-to-integrate tools → +adaptable, +lightweight ease the process of achieving software quality Too abstract/ambitious ⇒ we defined 4 main Goals (2 Blocks) Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 16
Research Goals in Block 1 Block 1: assure quality of interaction components separately Goal 1.1: Ease the validation of system response automation of GUI testing processes → simulate expert actions ease the integration into diverse scenarios Goal 1.2: Allow verification of user inputs at runtime protect GUI data using rules/constraints → prevent malfunction dynamic and interactive verification process Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 17
Research Goals in Block 2 Block 2: assure quality of user-system interaction Goal 2.1: Describe multimodal interaction for its assessment generic and dynamic description of interaction allow instrumentation, analysis, and comparison Goal 2.2: Allow analysis of user experiences in mobile scenarios combine interaction/context data with users subjective data generic description of mobile context and users perceived quality Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 18
Table of contents I. Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 19
Context of this research C´atedra SAES-UMU: research initiative created by SAES and the University of Murcia improve development processes promote the use and development of open-source tools Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 20
SAES SAES (Cartagena, Spain) is the only Spanish company specialized in submarine electronics and acoustics Besides funding our work, SAES provides us with: real problems to be solved realistic quality requirements to be achieved real scenarios in which integrate and use the developed methods Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 21
Research Internship at TLabs Telekom Innovation Laboratories = Deutsche Telekom + TU Berlin (Berlin, Germany) Central research institute of Deutsche Telekom: more user-centered approaches real problems in interaction with current technology Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 22
II Solutions Proposed in this PhD Thesis
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 24
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 25
Goal 1.1 & Problems Validating System Response Block 1: assure quality of interaction components separately → Goal 1.1: Ease the validation of system response Problems with GUI output validation: expensive/cumbersome processes → automation high dependency on experts → encapsulation + simulation changes during GUI design/development → robust testing difficult integration of testing tools diverse testing environments → adaptable architecture code-intrusiveness → non-invasive tools Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 26
Solution: Open HMI Tester (OHT) OHT: a framework to develop GUI-testing tools → Capture/replay approach GUI Testing ⇒ uses test cases to check requirements 1 Capture → create test cases with interaction experts interact with parts to be tested GUI introspection → test cases created automatically 2 Replay → execute test cases to validate app response functional & regression testing expert simulation: GUI events → +tolerance to changes Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 27
HMI TESTER PRELOAD MODULE TESTED APPLICATION dep. on data-model dep. on OS dep. on GUI comm comm preloading control event consumer event executor logic recording control playback control logic data-model adapter user interface preloading action EVENT DATA EVENT DATA GUI EVENTS Open architecture divided into 2 elements: HMI Tester → recording/playback control Preload Module → deployed into the target application → event capture and execution Adaptable modules + LIB Preload ⇔ Easy integration
How does it work? OHT example video: Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 29
OHT Conclusions basis to develop GUI testing tools based on GUI introspection adaptable architecture + adaptable recording/execution processes direct validation of system response → no model, no test oracles we still need visual validation of results Open-source implementation... cross-platform (C++) + agnostic to GUI platform + extensible ...used in SAES → Qt and ILOG versions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 30
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 31
Goal 1.2 & Problems Verifying User Input Block 1: assure quality of interaction components separately → Goal 1.2: Allow verification of user inputs at runtime In Runtime Verification (RV) a program is monitored at runtime to determine if it satisfies correctness properties Problems of using RV to verify GUI input: GUI properties are simpler → +effective method complex languages and formal logic → +options formal specification of acceptable behavior → lightweight method aspect-oriented bindings are static → dynamic languages feedback to users? → interactive verification Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 32
Solution: S-DAVER (Script-based Verification) S-DAVER: lightweight framework to verify user inputs in GUI → verification layer between GUI and business logic Based on fundamental pillars of RV... GUI data verified when produced → runtime efficient verification → while user interaction ...but: RV verifies app behavior = S-DAVER verifies data rules written in scripting languages → no formalisms included in separate files can be changed at runtime → dynamic approach lightweight + easy-to-integrate → smooth verification process rules loaded and verified transparently Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 33
verification layer business logic 1 2 rules logsrule loading interaction extraction rule checking feedback 3 gui dep. on rules dep. on GUI Independent/autonomous verification layer: 1 works while user interaction → no perceived overhead 2 only changed data is verified → efficiency 3 if rules fulfilled then go to business logic else do interactive feedback Adaptable architecture ⇒ GUI platform + rules language
How does it work? S-DAVER example video: verification layer rules logs 1 4 rule search rule check 2 3 Event: FocusLost Widget: le_bufferSize le_bufferSize change was invalid. Rule: "le_bufferSize <= 20" Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 35
S-DAVER Conclusions lightweight, easy-to-integrate verification solution trust relationship GUI ⇔ business logic interactive verification experience no guarantee of the internal correctness of rules Open-source implementation... cross-platform + agnostic to GUI platform and verification language + extensible ...tested in SAES → Qt + Lua version Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 36
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 37
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 38
Goal 2.1 & Problems Assessing Multimodal Interaction Block 2: assure quality of user-system interaction → Goal 2.1: Describe multimodal interaction for its assessment Problems with multimodal interaction assessment: many representations → abstract + generic different modalities analyzed separately → equivalences static parameters or average metrics dynamic nature of dialog process? → stepwise description conversational nature of dialog? → interaction turns Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 39
Solution: PALADIN (Multimodal Interaction Model) PALADIN: a generic, runtime multimodal interaction model → dynamic description of multimodal interaction Dialog = sequence of alternative user/system turns interaction described stepwise relationship data⇔time → runtime model allows dynamic analysis of interaction + cause-effect relationships system turn user turn ends system feedback starts info presentation starts system turn ends user turn ends user starts exploring user starts transferring action user turn delay feedback presentation delay exploring transferring Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 40
Generic description of multimodal interaction: model → relationships between data model → same metrics + same structure → comparison unified criteria to describe multimodal interaction agnostic to system and modalities used
Model with 2 levels of abstraction: generic metrics → main aspects of communication specific metrics → peculiarities of modalities Multimodal Interaction ⇒ described as a single flow of actions
PALADIN Integration and Usage 1 PALADIN model instance = 1 interaction episode Instances intended to be created automatically: 1 interaction extraction → automatic user-system instrumentation 2 PALADIN instantiation → helping framework 3 interaction analysis, comparison, transformation, and decision PALADIN instances 1. Interaction tracking 2. Model instantiation 3. Interaction analysis INSTANTIATION FRAMEWORK HCI EXTRACTOR INTERACTION ANALYZER DECIDER PALADIN implementation app Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 43
PALADIN Conclusions generic description of multimodal interaction + stepwise multimodal interaction = single, seamless flow of actions unified criteria to analyze and compare different systems Open-source implementation... model design + model implementation (Ecore + Java) instrumentation tool + instantiation framework (Java + Android) ...tested with real users in TLabs What about mobile interaction? What about quality of experiences? Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 45
Table of contents II. Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 46
Main Problems when Assessing Mobile User Experiences Block 2: assure quality of user-system interaction → Goal 2.2: Allow analysis of user experiences in mobile scenarios Problems analyzing mobile user experiences: many interaction and context data → select useful ones low standardization in context definition → generic context incorporate user ratings into interaction analysis no robust methodologies combining... quantitative data evaluating performance qualitative data assessing usability/QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 47
Solution: CARIM (Context & Ratings Interaction Model) CARIM = PALADIN + context data + users subjective ratings combines mobile interaction data runtime instrumentation of mobile interaction ModalityTurnDialog System TurnUser Turn Metacommunication Dynamic Context Physical environment Social context Location/time Device Communication Trial User data Ratings Mood Attitude PALADIN (base) CARIM (new) Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 48
Incorporating Context Data Generic definition of the surrounding context: parameter-based 6 dimensions user is the subject extensible CARIM deviceuser location/time communication social context human-computer interaction dynamic parameters static parameters user ratings environment PALADIN Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 49
Incorporating User Ratings User ratings = users quality perception + 2 influencing factors Quality perception: AttrakDiff questionnaire → pragmatic, hedonic, attractiveness mini version (10 pairs of opposite adjectives) Factors → high influence on quality perception user mood → attitude towards technology use → 5-item questionnaire Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 50
How does it work? CARIM experiment scenario + video: Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 51
CARIM Conclusions unified criteria to assess QoE from the perspectives of... user-system interaction its surrounding context quality perception of the users basis for context- and QoE-based decision we are still asking too much data to users Open-source implementation... model design + model implementation (Ecore + Java) instrumentation tool + instantiation framework (Java + Android) ...tested in our labs with real users Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 52
III Conclusions and Further Work
Table of contents III. Conclusions and Further Work 6 Conclusions 7 Future Lines of Work Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 54
Conclusions - Goals Achieved (i) 2 Approaches to assess interaction: Block 1: Decompose interaction into its components Block 2: Assess interaction as a whole We outlined 4 main goals Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 55
Conclusions - Goals Achieved (ii) Block 1: assure quality of interaction components separately → Goal 1.1: Ease the validation of system response expensive, hard-to-integrate testing processes OHT ⇒ adaptable architecture for developing GUI testing tools implement GUI testing on a robust basis validate app response using a battery of test cases test software with precision and tolerance to changes Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 56
Conclusions - Goals Achieved (iii) Block 1: assure quality of interaction components separately → Goal 1.2: Allow verification of user inputs at runtime no effective GUI data verification + no feedback S-DAVER ⇒ lightweight, runtime, interactive verification framework independent layer to verify GUI data dynamic rules unified interactive feedback Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 57
Conclusions - Goals Achieved (iv) Block 2: assure quality of user-system interaction → Goal 2.1: Assess and compare multimodal interaction static, no abstract description of modalities PALADIN ⇒ generic, runtime, stepwise interaction model 1 model ⇔ 1 description ⇔ many scenarios seamless description of modalities runtime interaction model Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 58
Conclusions - Goals Achieved (v) Block 2: assure quality of user-system interaction → Goal 2.2: Allow analysis of mobile user experiences no methods combining quantitative + qualitative CARIM ⇒ runtime model = interaction + context + user ratings measure and analyze QoE from different perspectives combined + structured data into model generic definition of context Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 59
Table of contents III. Conclusions and Further Work 6 Conclusions 7 Future Lines of Work Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 60
Future Work (i) Support automatic validation of GUI output OHT ⇒ still need visual validation of results graphical computing → simulate visual validation Ensure correctness and consistency of specifications S-DAVER ⇒ no method to guarantee rules correctness explore alternatives → CLP? Outline Block 1 concepts into a multimodal context from only-GUI to multimodal scenario validation of multimodal input/output Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 61
Future Work (ii) Enhance/extend design of interaction models PALADIN & CARIM ⇒ limitations in design fix potential errors/new emerging needs instrumentation of new modalities → gestures incorporate new dimensions of interaction task: interaction issues ⇔ problem being solved user emotions: interaction issues ⇔ user state Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 62
Future Work (iii) Exploit the full potential of the interaction models PALADIN & CARIM ⇒ structured data model transformation/code generation decision systems based on live data → adaptation continuous authentication of users Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 63
Future Work (iv) Study interaction in multi-person, concurrent interfaces multiple users interacting with the system concurrently e.g., smart home, tabletop devices reconsider turn-based approach and interaction models Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 64
IV Contributions Summary
Table of contents IV. Contributions Summary 8 Scientific Publications 9 Software Contributions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 66
JCR Journals 1 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. OpenHMI-Tester: An Open and Cross-Platform Architecture for GUI Testing and Certification, International Journal of Computer Systems Science and Engineering (IJCSSE), Special Issue on Open Source Certification, 2010. 2 Pedro Luis Mateo Navarro, Stefan Hillmann, Sebastian M¨oller, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Runtime Model Based Framework for Automatic Evaluation of Multimodal Interfaces, Journal on Multimodal User Interfaces (JMUI), 2014. In press. 3 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. A Context-aware Interaction Model for the Analysis of Users QoE in Mobile Environments, International Journal of Human-Computer Interaction (IJHC), Taylor & Francis, 2014. In press. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 67
JCR Journals under Review 4 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Lightweight Framework for Dynamic GUI Data Verification Based on Scripts, Journal on Software Testing, Verification and Reliability, 2014. Minor revision. Solution JCR journal OHT S-DAVER minor revision PALADIN CARIM Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 68
Other Journals 1 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Aplicaci´on de Open HMI Tester como framework open-source para herramientas de pruebas de software, Revista Espa˜nola de Innovaci´on, Calidad e Ingenier´ıa del Software (REICIS), 2009. 2 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Proposal for Automatic Testing of GUIs based on Annotated Use Cases, Advances in Software Engineering journal, Special Issue on Software Test Automation, vol. 2010, 2010. 3 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. CARIM: Un Modelo de Interacci´on para el An´alisis de la QoE en Entornos M´oviles, FAZ: Revista de Dise˜no de Interacci´on, 2014. In press. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 69
Conferences and Workshops (i) 1 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Context-aware Model for QoE Analysis in Mobile Environments, XIV International Congress on Human-Computer Interaction, Madrid (Spain), September 2013. 2 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Context-aware Model for the Analysis of User Interaction and QoE in Mobile Environments, CENTRIC 2012, Lisbon (Portugal), November 2012. 3 Pedro Luis Mateo Navarro, Stefan Schmidt. Model-based Measurement of Human-Computer Interaction in Mobile Multimodal Environments, aMMI (Assessing Multimodal Interaction), NordiCHI 2012, Copenhagen (Denmark), October 2012. 4 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Towards Software Quality and User Satisfaction through User Interfaces, Fourth IEEE International Conference on Software Testing, Berlin (Germany), March 2011. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 70
Conferences and Workshops (ii) 5 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Verificaci´on de Datos en la GUI como un Aspecto Separado de las Aplicaciones, PRIS 2010, Jornadas de Ingenieria del Software y Bases de Datos (JISBD 2010), Valencia (Spain), September 2010. 6 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Open HMI Tester: un Framework Open-source para Herramientas de Pruebas de Software, PRIS 2009, Jornadas de Ingenieria del Software y Bases de Datos (JISBD 2009), San Sebastian (Spain), September 2009. 7 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Automated GUI Testing Validation guided by Annotated Use Cases, MOTES 09 - Model-based Testing Workshop, L¨ubeck (Germany), September 2009. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 71
Table of contents IV. Contributions Summary 8 Scientific Publications 9 Software Contributions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 72
Main Software Contributions OHT Testing framework (C++, Qt4) http://www.catedrasaes.org/wiki/OHT S-DAVER Verification framework (C++, Qt4) + Rules (Lua/Chaiscript) http://www.catedrasaes.org/wiki/GuiVerification PALADIN Interaction model (EMF, Java) + HCI Extractor 8 (Java, Android) http://www.catedrasaes.org/wiki/MIM CARIM: Interaction model (EMF, Java) + HCI Extractor 11 (Java, Android) http://www.catedrasaes.org/wiki/Carim Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 73
Main Software Contributions — Validation Tools validated into different scenarios: OHT SAES developments S-DAVER and internal processes PALADIN experiments in TLabs CARIM experiments in C´atedra-SAES lab All tools are open-source, downloadable, and ready-to-use Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 74
Acknowledgements www.catedrasaes.org Supported by: cátedrasaeslabs Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 75

More Related Content

PPTX
Android Project Presentation
byLaxmi Kant Yadav
 
PPT
Sign language translator ieee power point
byMadhuri Yellapu
 
PPTX
Face detection and recognition
byPankaj Thakur
 
PPTX
Natural language processing
byYogendra Tamang
 
PPTX
Introduction to Keras
byJohn Ramey
 
PPT
Natural Language Processing
byYasir Khan
 
PDF
Lecture: Word Sense Disambiguation
byMarina Santini
 
PPTX
NLP_KASHK:Minimum Edit Distance
byHemantha Kulathilake
 
Android Project Presentation
byLaxmi Kant Yadav
 
Sign language translator ieee power point
byMadhuri Yellapu
 
Face detection and recognition
byPankaj Thakur
 
Natural language processing
byYogendra Tamang
 
Introduction to Keras
byJohn Ramey
 
Natural Language Processing
byYasir Khan
 
Lecture: Word Sense Disambiguation
byMarina Santini
 
NLP_KASHK:Minimum Edit Distance
byHemantha Kulathilake
 

What's hot

PDF
Placement management system
byMehul Ranavasiya
 
PPTX
Voice based email for blinds
byArjun AJ
 
PPT
Natural language processing
byprashantdahake
 
PDF
Crop Recommendation System to Maximize Crop Yield using Machine Learning Tech...
byIRJET Journal
 
PDF
Spam Email identification
byPartnered Health
 
PPTX
Unit 4
byMayura shelke
 
PDF
Speech Recognition Using Python | Edureka
byEdureka!
 
PPTX
NLP
byGirish Khanzode
 
PDF
Natural language processing (NLP) introduction
byRobert Lujo
 
PPTX
Embedded c c++ programming fundamentals master
byHossam Hassan
 
PDF
Hand Gesture Recognition using Neural Network
byBhagwat Singh Rathore
 
PPT
Speech recognition system
byRipal Ranpara
 
PDF
GPT-2: Language Models are Unsupervised Multitask Learners
byYoung Seok Kim
 
PPT
Automatic Speaker Recognition system using MFCC and VQ approach
byAbdullah al Mamun
 
PDF
Natural language processing and its application in ai
byRam Kumar
 
PDF
6 shallow parsing introduction
byThennarasuSakkan
 
PPTX
IEEE Posix Standards
byHussain Biedouh
 
PPTX
Partial redundancy elimination
byIshucs
 
PDF
Rain project
bySarfarajHussain7
 
PPTX
Information retrieval dynamic indexing
byNadia Nahar
 
Placement management system
byMehul Ranavasiya
 
Voice based email for blinds
byArjun AJ
 
Natural language processing
byprashantdahake
 
Crop Recommendation System to Maximize Crop Yield using Machine Learning Tech...
byIRJET Journal
 
Spam Email identification
byPartnered Health
 
Unit 4
byMayura shelke
 
Speech Recognition Using Python | Edureka
byEdureka!
 
NLP
byGirish Khanzode
 
Natural language processing (NLP) introduction
byRobert Lujo
 
Embedded c c++ programming fundamentals master
byHossam Hassan
 
Hand Gesture Recognition using Neural Network
byBhagwat Singh Rathore
 
Speech recognition system
byRipal Ranpara
 
GPT-2: Language Models are Unsupervised Multitask Learners
byYoung Seok Kim
 
Automatic Speaker Recognition system using MFCC and VQ approach
byAbdullah al Mamun
 
Natural language processing and its application in ai
byRam Kumar
 
6 shallow parsing introduction
byThennarasuSakkan
 
IEEE Posix Standards
byHussain Biedouh
 
Partial redundancy elimination
byIshucs
 
Rain project
bySarfarajHussain7
 
Information retrieval dynamic indexing
byNadia Nahar
 

Viewers also liked

PPTX
Prepare your Ph.D. Defense Presentation
byChristian Glahn
 
PPT
PhD Defense - A Context Management Framework based on Wisdom of Crowds for So...
byAdrien Joly
 
PPTX
An Information Retrieval Based Approach for Measuring Service Conceptual Cohe...
byPooyan Jamshidi
 
PPT
PhD Defense: Computer-Based Support and Feedback for Collaborative Chat Conve...
byTraian Rebedea
 
PPT
Powerpoint presentation M.A. Thesis Defence
byCatie Chase
 
PPT
Dissertation oral defense presentation
byDr. Naomi Mangatu
 
PPTX
Thesis Powerpoint
byneha47
 
PDF
My Thesis Defense Presentation
byDavid Onoue
 
PPTX
Thesis Power Point Presentation
byriddhikapandya1985
 
PPTX
Power Point Thesis
byRobert Riche
 
PPTX
Languages and frameworks for specifying test artifacts
byZoltan Micskei
 
PDF
Public PhD defense
byJose Luis Santos Odriozola
 
PPTX
Single SignOn and Context Management Solutions
bymanikrane
 
PPTX
Mobile Learning Unit - Communicating with Your Mobile Device
byRoss University School of Medicine
 
KEY
Brian J King - Thesis Defense Presentation (Prior to giving) - Commercial Med...
byBrian King
 
PPTX
Narratives of Learning: The Portfolio Approach. PhD - Public Defense Presenta...
byRoss University School of Medicine
 
PPTX
Egységtesztek automatikus generálása forráskódból
byZoltan Micskei
 
PPTX
VEDA Climate Change Solutions Ltd - Improving Rural Livelihoods Through Carbo...
byCCAFS | CGIAR Research Program on Climate Change, Agriculture and Food Security
 
PDF
Public defense PhD
byAnnemarie Devis
 
PPTX
Defesa doutorado 2013
byJuan "Pg" Waldir Mendoza Cortez
 
Prepare your Ph.D. Defense Presentation
byChristian Glahn
 
PhD Defense - A Context Management Framework based on Wisdom of Crowds for So...
byAdrien Joly
 
An Information Retrieval Based Approach for Measuring Service Conceptual Cohe...
byPooyan Jamshidi
 
PhD Defense: Computer-Based Support and Feedback for Collaborative Chat Conve...
byTraian Rebedea
 
Powerpoint presentation M.A. Thesis Defence
byCatie Chase
 
Dissertation oral defense presentation
byDr. Naomi Mangatu
 
Thesis Powerpoint
byneha47
 
My Thesis Defense Presentation
byDavid Onoue
 
Thesis Power Point Presentation
byriddhikapandya1985
 
Power Point Thesis
byRobert Riche
 
Languages and frameworks for specifying test artifacts
byZoltan Micskei
 
Public PhD defense
byJose Luis Santos Odriozola
 
Single SignOn and Context Management Solutions
bymanikrane
 
Mobile Learning Unit - Communicating with Your Mobile Device
byRoss University School of Medicine
 
Brian J King - Thesis Defense Presentation (Prior to giving) - Commercial Med...
byBrian King
 
Narratives of Learning: The Portfolio Approach. PhD - Public Defense Presenta...
byRoss University School of Medicine
 
Egységtesztek automatikus generálása forráskódból
byZoltan Micskei
 
VEDA Climate Change Solutions Ltd - Improving Rural Livelihoods Through Carbo...
byCCAFS | CGIAR Research Program on Climate Change, Agriculture and Food Security
 
Public defense PhD
byAnnemarie Devis
 
Defesa doutorado 2013
byJuan "Pg" Waldir Mendoza Cortez
 

Similar to PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

PPTX
Software Testing Strategies ,Validation Testing and System Testing.
byTanzeem Aslam
 
PPT
Software Testing
byEcaterina Moraru (Valica)
 
PPT
Software testing
bythaneofife
 
PPTX
Testing software development
byEr. Nawaraj Bhandari
 
PPTX
UNIT-1-SQE-Dr.K.Srinivas-CSE.pptx
bySrinivas Kanakala
 
PPTX
User interface testing By Priyanka Chauhan
byPriyanka Chauhan
 
PPTX
Software testing incorporating an Artificial Intelligence function
byJean-Antoine Moreau
 
PDF
Lionel Briand ICSM 2011 Keynote
byICSM 2011
 
PPT
Software Engineering (Testing Overview)
byShudipPal
 
PPTX
Manual Testing Types Used in Software Testing
byseojayeshts
 
PPTX
Testing strategies -2
byDivya Tiwari
 
PDF
interacting-with-ai-2023---module-2---session-3---handout.pdf
bycniclsh1
 
PDF
2015-11-11 research seminar
byifi8106tlu
 
PPT
Software Testing 1198102207476437 4
bySiddhartha Parida
 
PPTX
Manual Testing Test Case To Learn Part-2.pptx
byASHUTOSH TRIVEDI
 
PDF
Software Engineering Research: Leading a Double-Agent Life.
byLionel Briand
 
PPT
Massimo Rossello Zen And The Art Of Testing
bytarvos
 
PDF
Gui path oriented test generation algorithms paper
byIzzat Alsmadi
 
PPTX
Software testing lecture software engineering
byaabd9934
 
PDF
Presentation
bySATYALOK
 
Software Testing Strategies ,Validation Testing and System Testing.
byTanzeem Aslam
 
Software Testing
byEcaterina Moraru (Valica)
 
Software testing
bythaneofife
 
Testing software development
byEr. Nawaraj Bhandari
 
UNIT-1-SQE-Dr.K.Srinivas-CSE.pptx
bySrinivas Kanakala
 
User interface testing By Priyanka Chauhan
byPriyanka Chauhan
 
Software testing incorporating an Artificial Intelligence function
byJean-Antoine Moreau
 
Lionel Briand ICSM 2011 Keynote
byICSM 2011
 
Software Engineering (Testing Overview)
byShudipPal
 
Manual Testing Types Used in Software Testing
byseojayeshts
 
Testing strategies -2
byDivya Tiwari
 
interacting-with-ai-2023---module-2---session-3---handout.pdf
bycniclsh1
 
2015-11-11 research seminar
byifi8106tlu
 
Software Testing 1198102207476437 4
bySiddhartha Parida
 
Manual Testing Test Case To Learn Part-2.pptx
byASHUTOSH TRIVEDI
 
Software Engineering Research: Leading a Double-Agent Life.
byLionel Briand
 
Massimo Rossello Zen And The Art Of Testing
bytarvos
 
Gui path oriented test generation algorithms paper
byIzzat Alsmadi
 
Software testing lecture software engineering
byaabd9934
 
Presentation
bySATYALOK
 

Recently uploaded

PDF
Integrare AI e Automazione con UiPath Agent Builder
byUiPathCommunity
 
PDF
n8n - Next Generation Workflow Automation with Open Source
byDaisuke Masuda
 
PDF
AWS Community Day Indonesia 2025 - Attack the Cloud Before Attackers Do Build...
byMuhammad Yuga Nugraha
 
PDF
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
PDF
Igalia and WebKit: Status update and plans (2025)
byIgalia
 
PPTX
Stop Worrying, Start Wiring: Azure Serverless for the AI Era
byMuralidharan Deenathayalan
 
PDF
CymruSec 2025 - Wales' Inaugural Cyber Summit
byRay Bugg
 
PPTX
Autonomous Convoy Routing via Drone Swarms and Multi-Modal Threat Detection
byIvan Ruchkin
 
PDF
Energia Nowa and byteLAKE Partner to Develop Intelligent Services for Energy ...
bybyteLAKE
 
PDF
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
PDF
API Days Australia - API Management in the AI Era
byNilesh Gule
 
PPTX
ScalaF: Functional Refactoring and Automated Code Transformations
byhelloshrikha
 
PDF
AI Demands More: Help Ensure Network Readiness With ThousandEyes
byThousandEyes
 
PPTX
Introduction to WordPress Workshop - WCEH 2025
byShanta Nathwani
 
PDF
Memo No.3006141 dt.25.10.2025- SOP for Grant of Online Change Of Land Use
byRaghu Ram
 
PDF
Linux Day Prato 2025: NixOS spiegato ammiocuggino - NixOS for Human Beings
byPeter Bittner
 
DOCX
Formula 3 - MACO using general limit.docx
byMarkus Janssen
 
PDF
The State of Adventure Capital - Grant Gregory | Cantos
byRazin Mustafiz
 
PDF
Transcript: Ready, set, go: Pre-fall sales trends and data-driven insights - ...
byBookNet Canada
 
PDF
Spec Driven Development with Kiro: Ensuring Quality and Traceability in the A...
byHaim Michael
 
Integrare AI e Automazione con UiPath Agent Builder
byUiPathCommunity
 
n8n - Next Generation Workflow Automation with Open Source
byDaisuke Masuda
 
AWS Community Day Indonesia 2025 - Attack the Cloud Before Attackers Do Build...
byMuhammad Yuga Nugraha
 
FME Realize in the Real World - The Power of Spatial Computing in Action
bySafe Software
 
Igalia and WebKit: Status update and plans (2025)
byIgalia
 
Stop Worrying, Start Wiring: Azure Serverless for the AI Era
byMuralidharan Deenathayalan
 
CymruSec 2025 - Wales' Inaugural Cyber Summit
byRay Bugg
 
Autonomous Convoy Routing via Drone Swarms and Multi-Modal Threat Detection
byIvan Ruchkin
 
Energia Nowa and byteLAKE Partner to Develop Intelligent Services for Energy ...
bybyteLAKE
 
Immer vorne mit dabei sein - Roaming leicht gemacht
bypanagenda
 
API Days Australia - API Management in the AI Era
byNilesh Gule
 
ScalaF: Functional Refactoring and Automated Code Transformations
byhelloshrikha
 
AI Demands More: Help Ensure Network Readiness With ThousandEyes
byThousandEyes
 
Introduction to WordPress Workshop - WCEH 2025
byShanta Nathwani
 
Memo No.3006141 dt.25.10.2025- SOP for Grant of Online Change Of Land Use
byRaghu Ram
 
Linux Day Prato 2025: NixOS spiegato ammiocuggino - NixOS for Human Beings
byPeter Bittner
 
Formula 3 - MACO using general limit.docx
byMarkus Janssen
 
The State of Adventure Capital - Grant Gregory | Cantos
byRazin Mustafiz
 
Transcript: Ready, set, go: Pre-fall sales trends and data-driven insights - ...
byBookNet Canada
 
Spec Driven Development with Kiro: Ensuring Quality and Traceability in the A...
byHaim Michael
 

PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

  • 1.
    Enhancing Software Qualityand Quality of Experience through User Interfaces Pedro Luis Mateo Navarro Advisors: Dr. Diego Sevilla Ruiz, Dr. Gregorio Mart´ınez P´erez Murcia, July 2014 Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 1
  • 2.
    Agenda Enhancing Software Qualityand Quality of Experience through User Interfaces I. Motivation, Goals, and Context II. Solutions Proposed in this PhD Thesis III. Conclusions and Further Work IV. Contributions Summary Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 2
  • 3.
  • 4.
    Table of contents I.Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 4
  • 5.
    Quality of Softwareand Interaction Quality of Software is a broad concept... encompasses many disciplines: from technical to psychological can be achieved in many ways: checking functional requirements of software evaluating users perceived quality etc. This thesis focuses on User Interfaces... ...a field in which there is too much work to do Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 5
  • 6.
    User interfaces The linkbetween the human and the machine... users interact with the interface → they can see/touch/feel it system provide responses through the interface mean of communication User interfaces are essential; guaranteeing their quality is critical Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 6
  • 7.
    Software Testing Processes toguarantee software and interfaces quality... a system or application works as expected? a system or application works within its context? guarantee that the requirements are met Testing is essential before a software is released Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 7
  • 8.
    Human-Computer Interaction (HCI)Assessment HCI assessment ⇔ discipline within Software Testing Study of a conversational process involving: user → has a task to accomplish; provides input system → processes data to elaborate a response context → surrounds them; affects behavior and response Interaction has changed in the past years Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 8
  • 11.
    What is Good-QualitySoftware? Quality software ⇔ requirements are met The software fulfills: functional requirements → the behavior is the expected non-functional requirements → structural quality Quality can also be measured in terms of Usability and Interaction Experience Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 11
  • 12.
    Software Usability &Quality of Experience (QoE) Usability: How easy it is to use a software + how easy it is to learn how to use that software assure software effectiveness (is it useful?) and efficiency assure user satisfaction QoE: Subjective measure of users experiences with a software based on user perception and user feelings encompasses user behaviors + attitudes + emotions dynamic → changing context/circumstances Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 12
  • 13.
    Achieving Good-Quality Software? Differentways to achieve software quality We try to narrow the problem... focusing on user interfaces analysis of user-system interaction in real-time How can we assess interaction? Since interaction = user inputs + system outputs then... ...we posed 2 different approaches Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 13
  • 14.
    Achieving Software Qualitythrough User Interfaces Approach 1: Decompose interaction into its components main components ⇒ input + output components quality is enhanced separately analysis/testing processes → more focused, more effective Approach 2: Assess interaction process as a whole whole dialog → keeps totality of the process conversational nature → cause-effect relationships analysis of external factors ⇒ context Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 14
  • 15.
    Table of contents I.Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 15
  • 16.
    Goal of thisPhD Thesis Improve methods for software testing and interaction analysis design better tools → +effective, +robust design easy-to-integrate tools → +adaptable, +lightweight ease the process of achieving software quality Too abstract/ambitious ⇒ we defined 4 main Goals (2 Blocks) Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 16
  • 17.
    Research Goals inBlock 1 Block 1: assure quality of interaction components separately Goal 1.1: Ease the validation of system response automation of GUI testing processes → simulate expert actions ease the integration into diverse scenarios Goal 1.2: Allow verification of user inputs at runtime protect GUI data using rules/constraints → prevent malfunction dynamic and interactive verification process Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 17
  • 18.
    Research Goals inBlock 2 Block 2: assure quality of user-system interaction Goal 2.1: Describe multimodal interaction for its assessment generic and dynamic description of interaction allow instrumentation, analysis, and comparison Goal 2.2: Allow analysis of user experiences in mobile scenarios combine interaction/context data with users subjective data generic description of mobile context and users perceived quality Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 18
  • 19.
    Table of contents I.Motivation, Goals, and Context 1 Motivation 2 Goals of this PhD Thesis 3 Context of this PhD Thesis Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 19
  • 20.
    Context of thisresearch C´atedra SAES-UMU: research initiative created by SAES and the University of Murcia improve development processes promote the use and development of open-source tools Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 20
  • 21.
    SAES SAES (Cartagena, Spain)is the only Spanish company specialized in submarine electronics and acoustics Besides funding our work, SAES provides us with: real problems to be solved realistic quality requirements to be achieved real scenarios in which integrate and use the developed methods Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 21
  • 22.
    Research Internship atTLabs Telekom Innovation Laboratories = Deutsche Telekom + TU Berlin (Berlin, Germany) Central research institute of Deutsche Telekom: more user-centered approaches real problems in interaction with current technology Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 22
  • 23.
    II Solutions Proposed inthis PhD Thesis
  • 24.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 24
  • 25.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 25
  • 26.
    Goal 1.1 &Problems Validating System Response Block 1: assure quality of interaction components separately → Goal 1.1: Ease the validation of system response Problems with GUI output validation: expensive/cumbersome processes → automation high dependency on experts → encapsulation + simulation changes during GUI design/development → robust testing difficult integration of testing tools diverse testing environments → adaptable architecture code-intrusiveness → non-invasive tools Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 26
  • 27.
    Solution: Open HMITester (OHT) OHT: a framework to develop GUI-testing tools → Capture/replay approach GUI Testing ⇒ uses test cases to check requirements 1 Capture → create test cases with interaction experts interact with parts to be tested GUI introspection → test cases created automatically 2 Replay → execute test cases to validate app response functional & regression testing expert simulation: GUI events → +tolerance to changes Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 27
  • 28.
    HMI TESTER PRELOAD MODULE TESTEDAPPLICATION dep. on data-model dep. on OS dep. on GUI comm comm preloading control event consumer event executor logic recording control playback control logic data-model adapter user interface preloading action EVENT DATA EVENT DATA GUI EVENTS Open architecture divided into 2 elements: HMI Tester → recording/playback control Preload Module → deployed into the target application → event capture and execution Adaptable modules + LIB Preload ⇔ Easy integration
  • 29.
    How does itwork? OHT example video: Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 29
  • 30.
    OHT Conclusions basis todevelop GUI testing tools based on GUI introspection adaptable architecture + adaptable recording/execution processes direct validation of system response → no model, no test oracles we still need visual validation of results Open-source implementation... cross-platform (C++) + agnostic to GUI platform + extensible ...used in SAES → Qt and ILOG versions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 30
  • 31.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 31
  • 32.
    Goal 1.2 &Problems Verifying User Input Block 1: assure quality of interaction components separately → Goal 1.2: Allow verification of user inputs at runtime In Runtime Verification (RV) a program is monitored at runtime to determine if it satisfies correctness properties Problems of using RV to verify GUI input: GUI properties are simpler → +effective method complex languages and formal logic → +options formal specification of acceptable behavior → lightweight method aspect-oriented bindings are static → dynamic languages feedback to users? → interactive verification Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 32
  • 33.
    Solution: S-DAVER (Script-basedVerification) S-DAVER: lightweight framework to verify user inputs in GUI → verification layer between GUI and business logic Based on fundamental pillars of RV... GUI data verified when produced → runtime efficient verification → while user interaction ...but: RV verifies app behavior = S-DAVER verifies data rules written in scripting languages → no formalisms included in separate files can be changed at runtime → dynamic approach lightweight + easy-to-integrate → smooth verification process rules loaded and verified transparently Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 33
  • 34.
    verification layer business logic 1 2 rules logsruleloading interaction extraction rule checking feedback 3 gui dep. on rules dep. on GUI Independent/autonomous verification layer: 1 works while user interaction → no perceived overhead 2 only changed data is verified → efficiency 3 if rules fulfilled then go to business logic else do interactive feedback Adaptable architecture ⇒ GUI platform + rules language
  • 35.
    How does itwork? S-DAVER example video: verification layer rules logs 1 4 rule search rule check 2 3 Event: FocusLost Widget: le_bufferSize le_bufferSize change was invalid. Rule: "le_bufferSize <= 20" Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 35
  • 36.
    S-DAVER Conclusions lightweight, easy-to-integrateverification solution trust relationship GUI ⇔ business logic interactive verification experience no guarantee of the internal correctness of rules Open-source implementation... cross-platform + agnostic to GUI platform and verification language + extensible ...tested in SAES → Qt + Lua version Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 36
  • 37.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 37
  • 38.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 38
  • 39.
    Goal 2.1 &Problems Assessing Multimodal Interaction Block 2: assure quality of user-system interaction → Goal 2.1: Describe multimodal interaction for its assessment Problems with multimodal interaction assessment: many representations → abstract + generic different modalities analyzed separately → equivalences static parameters or average metrics dynamic nature of dialog process? → stepwise description conversational nature of dialog? → interaction turns Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 39
  • 40.
    Solution: PALADIN (MultimodalInteraction Model) PALADIN: a generic, runtime multimodal interaction model → dynamic description of multimodal interaction Dialog = sequence of alternative user/system turns interaction described stepwise relationship data⇔time → runtime model allows dynamic analysis of interaction + cause-effect relationships system turn user turn ends system feedback starts info presentation starts system turn ends user turn ends user starts exploring user starts transferring action user turn delay feedback presentation delay exploring transferring Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 40
  • 41.
    Generic description ofmultimodal interaction: model → relationships between data model → same metrics + same structure → comparison unified criteria to describe multimodal interaction agnostic to system and modalities used
  • 42.
    Model with 2levels of abstraction: generic metrics → main aspects of communication specific metrics → peculiarities of modalities Multimodal Interaction ⇒ described as a single flow of actions
  • 43.
    PALADIN Integration andUsage 1 PALADIN model instance = 1 interaction episode Instances intended to be created automatically: 1 interaction extraction → automatic user-system instrumentation 2 PALADIN instantiation → helping framework 3 interaction analysis, comparison, transformation, and decision PALADIN instances 1. Interaction tracking 2. Model instantiation 3. Interaction analysis INSTANTIATION FRAMEWORK HCI EXTRACTOR INTERACTION ANALYZER DECIDER PALADIN implementation app Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 43
  • 45.
    PALADIN Conclusions generic descriptionof multimodal interaction + stepwise multimodal interaction = single, seamless flow of actions unified criteria to analyze and compare different systems Open-source implementation... model design + model implementation (Ecore + Java) instrumentation tool + instantiation framework (Java + Android) ...tested with real users in TLabs What about mobile interaction? What about quality of experiences? Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 45
  • 46.
    Table of contents II.Solutions Proposed in this PhD Thesis 4 Block I Validating System Output Verifying Users Input 5 Block II Evaluating Multimodal Interaction Evaluating Mobile QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 46
  • 47.
    Main Problems whenAssessing Mobile User Experiences Block 2: assure quality of user-system interaction → Goal 2.2: Allow analysis of user experiences in mobile scenarios Problems analyzing mobile user experiences: many interaction and context data → select useful ones low standardization in context definition → generic context incorporate user ratings into interaction analysis no robust methodologies combining... quantitative data evaluating performance qualitative data assessing usability/QoE Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 47
  • 48.
    Solution: CARIM (Context& Ratings Interaction Model) CARIM = PALADIN + context data + users subjective ratings combines mobile interaction data runtime instrumentation of mobile interaction ModalityTurnDialog System TurnUser Turn Metacommunication Dynamic Context Physical environment Social context Location/time Device Communication Trial User data Ratings Mood Attitude PALADIN (base) CARIM (new) Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 48
  • 49.
    Incorporating Context Data Genericdefinition of the surrounding context: parameter-based 6 dimensions user is the subject extensible CARIM deviceuser location/time communication social context human-computer interaction dynamic parameters static parameters user ratings environment PALADIN Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 49
  • 50.
    Incorporating User Ratings Userratings = users quality perception + 2 influencing factors Quality perception: AttrakDiff questionnaire → pragmatic, hedonic, attractiveness mini version (10 pairs of opposite adjectives) Factors → high influence on quality perception user mood → attitude towards technology use → 5-item questionnaire Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 50
  • 51.
    How does itwork? CARIM experiment scenario + video: Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 51
  • 52.
    CARIM Conclusions unified criteriato assess QoE from the perspectives of... user-system interaction its surrounding context quality perception of the users basis for context- and QoE-based decision we are still asking too much data to users Open-source implementation... model design + model implementation (Ecore + Java) instrumentation tool + instantiation framework (Java + Android) ...tested in our labs with real users Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 52
  • 53.
  • 54.
    Table of contents III.Conclusions and Further Work 6 Conclusions 7 Future Lines of Work Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 54
  • 55.
    Conclusions - GoalsAchieved (i) 2 Approaches to assess interaction: Block 1: Decompose interaction into its components Block 2: Assess interaction as a whole We outlined 4 main goals Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 55
  • 56.
    Conclusions - GoalsAchieved (ii) Block 1: assure quality of interaction components separately → Goal 1.1: Ease the validation of system response expensive, hard-to-integrate testing processes OHT ⇒ adaptable architecture for developing GUI testing tools implement GUI testing on a robust basis validate app response using a battery of test cases test software with precision and tolerance to changes Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 56
  • 57.
    Conclusions - GoalsAchieved (iii) Block 1: assure quality of interaction components separately → Goal 1.2: Allow verification of user inputs at runtime no effective GUI data verification + no feedback S-DAVER ⇒ lightweight, runtime, interactive verification framework independent layer to verify GUI data dynamic rules unified interactive feedback Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 57
  • 58.
    Conclusions - GoalsAchieved (iv) Block 2: assure quality of user-system interaction → Goal 2.1: Assess and compare multimodal interaction static, no abstract description of modalities PALADIN ⇒ generic, runtime, stepwise interaction model 1 model ⇔ 1 description ⇔ many scenarios seamless description of modalities runtime interaction model Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 58
  • 59.
    Conclusions - GoalsAchieved (v) Block 2: assure quality of user-system interaction → Goal 2.2: Allow analysis of mobile user experiences no methods combining quantitative + qualitative CARIM ⇒ runtime model = interaction + context + user ratings measure and analyze QoE from different perspectives combined + structured data into model generic definition of context Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 59
  • 60.
    Table of contents III.Conclusions and Further Work 6 Conclusions 7 Future Lines of Work Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 60
  • 61.
    Future Work (i) Supportautomatic validation of GUI output OHT ⇒ still need visual validation of results graphical computing → simulate visual validation Ensure correctness and consistency of specifications S-DAVER ⇒ no method to guarantee rules correctness explore alternatives → CLP? Outline Block 1 concepts into a multimodal context from only-GUI to multimodal scenario validation of multimodal input/output Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 61
  • 62.
    Future Work (ii) Enhance/extenddesign of interaction models PALADIN & CARIM ⇒ limitations in design fix potential errors/new emerging needs instrumentation of new modalities → gestures incorporate new dimensions of interaction task: interaction issues ⇔ problem being solved user emotions: interaction issues ⇔ user state Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 62
  • 63.
    Future Work (iii) Exploitthe full potential of the interaction models PALADIN & CARIM ⇒ structured data model transformation/code generation decision systems based on live data → adaptation continuous authentication of users Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 63
  • 64.
    Future Work (iv) Studyinteraction in multi-person, concurrent interfaces multiple users interacting with the system concurrently e.g., smart home, tabletop devices reconsider turn-based approach and interaction models Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 64
  • 65.
  • 66.
    Table of contents IV.Contributions Summary 8 Scientific Publications 9 Software Contributions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 66
  • 67.
    JCR Journals 1 PedroLuis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. OpenHMI-Tester: An Open and Cross-Platform Architecture for GUI Testing and Certification, International Journal of Computer Systems Science and Engineering (IJCSSE), Special Issue on Open Source Certification, 2010. 2 Pedro Luis Mateo Navarro, Stefan Hillmann, Sebastian M¨oller, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Runtime Model Based Framework for Automatic Evaluation of Multimodal Interfaces, Journal on Multimodal User Interfaces (JMUI), 2014. In press. 3 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. A Context-aware Interaction Model for the Analysis of Users QoE in Mobile Environments, International Journal of Human-Computer Interaction (IJHC), Taylor & Francis, 2014. In press. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 67
  • 68.
    JCR Journals underReview 4 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Lightweight Framework for Dynamic GUI Data Verification Based on Scripts, Journal on Software Testing, Verification and Reliability, 2014. Minor revision. Solution JCR journal OHT S-DAVER minor revision PALADIN CARIM Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 68
  • 69.
    Other Journals 1 PedroLuis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Aplicaci´on de Open HMI Tester como framework open-source para herramientas de pruebas de software, Revista Espa˜nola de Innovaci´on, Calidad e Ingenier´ıa del Software (REICIS), 2009. 2 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Proposal for Automatic Testing of GUIs based on Annotated Use Cases, Advances in Software Engineering journal, Special Issue on Software Test Automation, vol. 2010, 2010. 3 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. CARIM: Un Modelo de Interacci´on para el An´alisis de la QoE en Entornos M´oviles, FAZ: Revista de Dise˜no de Interacci´on, 2014. In press. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 69
  • 70.
    Conferences and Workshops(i) 1 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Context-aware Model for QoE Analysis in Mobile Environments, XIV International Congress on Human-Computer Interaction, Madrid (Spain), September 2013. 2 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. A Context-aware Model for the Analysis of User Interaction and QoE in Mobile Environments, CENTRIC 2012, Lisbon (Portugal), November 2012. 3 Pedro Luis Mateo Navarro, Stefan Schmidt. Model-based Measurement of Human-Computer Interaction in Mobile Multimodal Environments, aMMI (Assessing Multimodal Interaction), NordiCHI 2012, Copenhagen (Denmark), October 2012. 4 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Towards Software Quality and User Satisfaction through User Interfaces, Fourth IEEE International Conference on Software Testing, Berlin (Germany), March 2011. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 70
  • 71.
    Conferences and Workshops(ii) 5 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Verificaci´on de Datos en la GUI como un Aspecto Separado de las Aplicaciones, PRIS 2010, Jornadas de Ingenieria del Software y Bases de Datos (JISBD 2010), Valencia (Spain), September 2010. 6 Pedro Luis Mateo Navarro, Gregorio Mart´ınez P´erez, Diego Sevilla Ruiz. Open HMI Tester: un Framework Open-source para Herramientas de Pruebas de Software, PRIS 2009, Jornadas de Ingenieria del Software y Bases de Datos (JISBD 2009), San Sebastian (Spain), September 2009. 7 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez P´erez. Automated GUI Testing Validation guided by Annotated Use Cases, MOTES 09 - Model-based Testing Workshop, L¨ubeck (Germany), September 2009. Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 71
  • 72.
    Table of contents IV.Contributions Summary 8 Scientific Publications 9 Software Contributions Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 72
  • 73.
    Main Software Contributions OHT Testingframework (C++, Qt4) http://www.catedrasaes.org/wiki/OHT S-DAVER Verification framework (C++, Qt4) + Rules (Lua/Chaiscript) http://www.catedrasaes.org/wiki/GuiVerification PALADIN Interaction model (EMF, Java) + HCI Extractor 8 (Java, Android) http://www.catedrasaes.org/wiki/MIM CARIM: Interaction model (EMF, Java) + HCI Extractor 11 (Java, Android) http://www.catedrasaes.org/wiki/Carim Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 73
  • 74.
    Main Software Contributions— Validation Tools validated into different scenarios: OHT SAES developments S-DAVER and internal processes PALADIN experiments in TLabs CARIM experiments in C´atedra-SAES lab All tools are open-source, downloadable, and ready-to-use Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 74
  • 75.
    Acknowledgements www.catedrasaes.org Supported by: cátedrasaeslabs Pedro LuisMateo Navarro () PhD Thesis Defense Murcia, July 2014 75