Explicating and Reasoning with Model Uncertainty by Marsha Chechik (ECMFA'14 Keynote)
Download as PPTX, PDF2 likes1,552 views
The document discusses the challenges of model uncertainty in model-based software engineering (MBSE) and proposes the development of a toolbox aimed at managing uncertainty during the modeling process. The toolbox will help modelers express, manage, and reduce uncertainty, facilitating progress in software development despite incomplete information and conflicting stakeholder opinions. It details various techniques for representing uncertainty in models and emphasizes the need for transformations and verification processes that accommodate uncertain elements.
![Classifying Natalie’s Uncertainty
[Aughenbaugh06] classification
15
Is
inherently
unknowable?
In the world
(aleatory,
“gambler”)
In the knowledge
(epistemic,
“science”)
Yes
(irreducible
uncertainty)
No
(reducible
uncertainty)
Location of
uncertainty
Uncertainty about the
content of the model.](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-11-320.jpg)
![How to Represent Uncertainty?
Ask users
• Goals: model uncertainty explicitly, keep syntax familiar
21
[MiSE’13]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-17-320.jpg)
![Representing Uncertainty
Partial Models
• Points of uncertainty
(“May elements”)
explicated using
syntactic annotations
22
Solver
SolverException
+ effect : String
Unsure if it
should be an
inner class.
Unsure if we
need this field.
[ICSE12]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-18-320.jpg)
![Representing Uncertainty
Partial Models
• Points of uncertainty
(“May elements”)
explicated using
syntactic annotations
Propositional variables:
“the element exists”
23
Solver
SolverException
+ effect : String
X
Y
[ICSE12]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-19-320.jpg)
![Transforming Models with Uncertainty
33
Natalie should be able
to use model
transformations
[MODELS’13]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-29-320.jpg)
![Transforming Models with Uncertainty
34
We need to lift Natalie’s transformations so that
they can apply to models with uncertainty
Existing transformation techniques
do not support this!
Natalie should be able
to use model
transformations
[MODELS’13]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-30-320.jpg)
![Case Study
• Object–relational mapping (ORM)
• “Translate a class diagram to a relational database
schema.”
• Classic benchmark for model
transformation research
• Triple graph grammar with 5 layered graph rules [Varro06]
58
(Image from [Varro06])](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-54-320.jpg)
![Case Study
• Input model: the Ecore metamodel
• ORM for Ecore is important: cf. CDO and Teneo
• Manually flattened inheritance hierarchy and adapted to the
metamodel in [Varro06]
• Resulting model had 65 model elements:
• 17 classes, 17 associations, 6 generalization links, 25 attributes
• Manually injected points of uncertainty to create partial models
with increasing numbers of concretizations
59](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-55-320.jpg)
![Verification: Lifting
• Checking syntactic properties [ICSE’12]:
• Language independent
• SAT-based
• Checking semantic properties:
• Depends on language semantics
• Requires custom adaptation of verification technique
• Examples:
• Goal satisfaction in i* models [RE’12, RE’14]
• MTSA: Model checking and other analyses over MTSs [ASE’08]
65](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-61-320.jpg)
![Case Study: Checking Properties
• Why? Triangulate experimental results (random inputs) with
observations from a real-word scenario
• Case study details:
• Bug from the UMLet bugzilla
• Realistic bug fixes
• Uncertainty: which combination of actions that fix the bug
to choose from?
• We checked two properties from the literature [SMSJ03]
• Resulting partial model: 27,261 elements, 220
concretizations
68
[SMSJ03] R.V. Der Straeten, T. Mens, J. Simmonds, and V. Jonckers. “Using Description Logic to
Maintain Consistency between UML Models”. In Proc. of UML’03, pp 326-340, 2003](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-64-320.jpg)
![Uncertainty-Reducing Refinement
• Systematically incorporate
new information…
• … Manually
• Edit the uncertainty
annotations
• … Using automated
transformations
Verify that a
transformation is always
uncertainty reducing.
72
Unsure if it
should be an
inner class.
X v Y
Unsure if
we need
this field.
You know what? I
want to get rid of
effect.
[FASE’12, VOLT’12]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-68-320.jpg)
![Inspired by: Modal Transition Systems (MTS)
LTS Extension:
– Required transitions
– Possible transitions
– the rest is proscribed (prohibited)
– MTS toolbox – MTSA [ASE’08]
request?
request
reply
[Larsen/Thompson, 1988]](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-76-320.jpg)
![• Vagueness in modeling [Hermann]
• Incompleteness in databases
• Representing null values in db [Fagin, etc.]
• Local Closed World Assumption [Cortes-Calabuig, etc.]
Other Similar Concepts](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-84-320.jpg)
![Some Current and Future Work
• Front-end for modeling uncertainty [MiSE’13]
• Extending the uncertainty toolbox to handle full MAVO models
• Lifting other techniques that require semantics to handle
uncertainty
• E.g., non-functional analysis
• Understanding origins and models of uncertainty
• Mining developer conversations to infer uncertainty in collaborative
design [RSSE’14]
• Building uncertainty dashboard to help the design process
• Uncertainty patterns
• Bottom-up: useful for tool building
• Top down: detecting uncertainty in workflows and conversations
90](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-86-320.jpg)
![Some Current and Future Work
• More complex modeling of uncertainty
• Combining “our” uncertainty with environment uncertainty
• Handling richer qualitative models (fuzzy logic? probabilities?)
• Using explicit uncertainty modeling to bridge the gap with
informal modeling/sketching
• Bran Selic’s “formality slider”
• Uncertainty all the way down to code
• Uncertainty as a meta-aspect, to be handled using language
workbenches
• Lifting programming languages, e.g., Java
• Symbolic execution of partial models
• Lifting analyses to other set-handling formalisms
• E.g., transformations in product lines [ICSE’14]
• Code analyses?
91](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-87-320.jpg)
![Bibliography I
[MODELS13] M. Famelis, R. Salay, A. Di Sandro, M. Chechik.
“Transformation of Models Containing Uncertainty”. In Proc. of
MODELS’13, 2013.
[ICSE12] M. Famelis, M. Chechik, and R. Salay. “Partial Models:
Towards Modeling and Reasoning with Uncertainty”. In Proc. of
ICSE’12, 2012.
[ICSE14] R. Salay, M. Famelis, J. Rubin, A. Di Sandro, M. Chechik.
“Lifting Model Transformations to Product Lines”. In Proc. of ICSE’14,
2014.
[FASE12] R. Salay, M. Famelis, and M. Chechik. “Language
Independent Refinement using Partial Modeling”. In Proc. of FASE’12,
2012.
93](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-89-320.jpg)
![Bibliography II
[Varro06] D. Varro, S. Varro-Gyapay, H. Ehrig, U. Prange, and G.
Taentzer. “Termination Analysis of Model Transformations by Petri
Nets”. In Proc. of ICGT’06, pages 260–274, 2006.
[Classen09] A. Classen, P. Heymans, P. Schobbens, A. Legay, and
J. Raskin. “Model Checking Lots of Systems: Efficient Verification of
Temporal Properties in Software Product Lines”. In Proc. of ICSE'10,
pages 335-344, 2010.
[Svahnberg05] M. Svahnberg, J. Van Gurp, and J. Bosch. “A Taxonomy
of Variability Realization Techniques”. Software: Practice and
Experience, 35(8):705-754, 2005.
94](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-90-320.jpg)
![Example: Lifting Transformations for
Product Lines
100
1. Determine applicability
2. Transform graph
3. Transform formula
1. Determine applicability
2. Transform graph
3. Transform presence conditions
Transforming Partial Models Transforming Product Lines [ICSE’14]
May
elements
May
formula
Feature
Presence
conditions](https://image.slidesharecdn.com/marshachechikkeynote-140730184518-phpapp01/85/Explicating-and-Reasoning-with-Model-Uncertainty-by-Marsha-Chechik-ECMFA-14-Keynote-96-320.jpg)
Explicating and Reasoning with Model Uncertainty by Marsha Chechik (ECMFA'14 Keynote)
- 1. EXPLICATING AND REASONING WITH MODEL UNCERTAINTY Marsha Chechik University of Toronto July, 2014 Acknowledgements: work done in collaboration with R. Salay, M. Famelis, A. DiSandro and many others 1
- 2. Why Do Engineers Build Models? • To understand • …the interesting characteristics of an existing or desired (complex) system and its environment • To predict • …the interesting characteristics of the system by analysing its model(s) • To communicate • …their understanding and design intent (to others and to oneself!) • To specify • ...the implementation of the system (models as blueprints) Source: Bran Selic’s Modeling Course
- 3. Software Engineering Models Requirements Architecture Behaviour Static Design Use CasesDeployment Concepts StructureAbstraction
- 4. Model-Based Software Engineering (MBSE) • An approach to system and software development in which software models play an indispensable role • Based on two time-proven ideas: switch (state) { case‘1:action1; newState(‘2’); break; case‘2:action2; newState(‘3’); break; case’3:action3; newState(‘1’); break;} (2) AUTOMATION S1 S3 S2 e1/action1 e2/action2 e3/action3 switch (state) { case‘1:action1; newState(‘2’); break; case‘2:action2; newState(‘3’); break; case’3:action3; newState(‘1’); break;} (1) ABSTRACTION S1 S3 S2 e1/action1 e2/action2 e3/action3 Realm of modeling languages Realm of tools Source: Bran Selic’s Modeling Course
- 5. Model-Based Software Engineering (MBSE) Toolbox (partial list!) Support for: Development, analysis, refinement, transformation of models + Communication, documentation 9
- 6. 10 This is Natalie. Natalie is a modeler. Natalie faces uncertainty in her everyday work
- 7. Alternative Designs 11 Hmm, I don’t know which one, yet.
- 8. Conflicting Stakeholder Opinions 12 What do I do until they decide?
- 9. Incomplete Information 13 I don’t know everything about this, yet.
- 10. Uncertainty in Software Development 14 Uncertainty about the content of the model. Many design alternatives Conflicting stakeholder opinions Incomplete information
- 11. Classifying Natalie’s Uncertainty [Aughenbaugh06] classification 15 Is inherently unknowable? In the world (aleatory, “gambler”) In the knowledge (epistemic, “science”) Yes (irreducible uncertainty) No (reducible uncertainty) Location of uncertainty Uncertainty about the content of the model.
- 12. MBSE Toolbox for Natalie? Existing tools do not apply Options: • Wait until uncertainty gets resolved (how long?) • Make a provisional decision and run with it (need backtracking!) We propose: • Defer resolution of uncertainty but incorporate uncertainty handling into the development process to allow progress • Lift standard MDE tools to handle models with uncertainty • Add new tools: • Help Natalie express her uncertainty • Help Natalie determine what she does not yet know • Help Natalie manage and reduce uncertainty
- 13. • Motivation: • MBSE toolbox • Sources of uncertainty • Goal: MBSE toolbox for uncertainty-aware models • Creating the toolbox • Modeling uncertainty • Transforming models containing uncertainty • Reasoning about models • Uncertainty-reducing refinement • Tool support • Putting this into context 17
- 15. Representing Uncertainty: Example 19 Solver SolverException + effect : String Unsure if it should be an inner class. Unsure if we need this field. Notation means “inner class” in UML
- 16. Some Decisions to Make Options for assigning values to states of confidence 20 Quantitative Examples: 0, 0.3, 0.75 Good for capturing likelihood of events Hard to capture one’s own uncertainty Qualitative Examples: yes/no/don’t know strongly … strongly agree disagree Good for capturing one’s own level of uncertainty Less precise than quantitative
- 17. How to Represent Uncertainty? Ask users • Goals: model uncertainty explicitly, keep syntax familiar 21 [MiSE’13]
- 18. Representing Uncertainty Partial Models • Points of uncertainty (“May elements”) explicated using syntactic annotations 22 Solver SolverException + effect : String Unsure if it should be an inner class. Unsure if we need this field. [ICSE12]
- 19. Representing Uncertainty Partial Models • Points of uncertainty (“May elements”) explicated using syntactic annotations Propositional variables: “the element exists” 23 Solver SolverException + effect : String X Y [ICSE12]
- 20. Representing Uncertainty 24 Solver SolverException + effect : String X Y Solver SolverException Solver SolverException Solver SolverException + effect : String Solver SolverException + effect : String x=F, y=F x=T, y=F x=F, y=T x=T, y=T 4 concretizations: 4 ways to resolve uncertainty.
- 21. Representing Uncertainty Partial Models • Points of uncertainty (“May elements”) explicated using syntactic annotations • Restrictions to the set of concretizations can be captured in the “May formula” 25 Solver SolverException + effect : String X Y X v Y
- 22. Representing Uncertainty 26 Solver SolverException + effect : String X Y Solver SolverException Solver SolverException Solver SolverException + effect : String Solver SolverException + effect : String x=F, y=F x=T, y=F x=F, y=T x=T, y=T X v Y
- 23. -(S)securityRelated : (SV)Types1 -numOfPassengers : Int Vehicle CarTimeMachine Boat (M)Hovercraft (MS)OtherVehicles -numOfDoors : Int (V)C1 (M) (M) (M) (M) (MS) (SV)Types1 Int P1:ClassDiagram ? ? ? ? ? ? ? More Complex Design Uncertainty Not sure which security attributes - but know some will be needed. Not sure which class will hold the numOfDoors attribute Not sure if we will have Hovercraft or which class it specializes. Not sure what other vehicle types there will be
- 24. • May • Not sure if a model element should occur • annotations: M – maybe E - exists (default) • Abs • Not sure how many elements exist that play a particular role • annotations: S - set, P - particular (default) • Var • Not sure whether the element will be merged with another • annotations: V – variable, C - constant (default) • OW • Not sure if the model is complete • annotations: M - maybe, E - exists (default) Capturing Partial Information with MAVO M – maybe E – exists (default) S – set P – particular (default) V – variable C – constant (default) inc – incomplete comp – complete (default)
- 25. A MAVO Class Diagram Not sure which security attributes - but know some will be needed. -(S)securityRelated : (SV)Types1 -numOfPassengers : Int Vehicle CarTimeMachine Boat (M)Hovercraft (MS)OtherVehicles -numOfDoors : Int (V)C1 (M) (M) (M) (M) (MS) (SV)Types1 Int P1:ClassDiagram Not sure which class will hold the numOfDoors attribute Not sure if we will have Hovercraft or which class it specializes. Not sure what other vehicle types there will be ? ? ? ? ? ? ? P1:MAVO(ClassDiagram)
- 26. • Benefits • Partial model refinement = uncertainty reduction • Formal: allows analysis and automation • Semantics-independent – applies to models of different types • Use instead of concrete models … … to allow deferral of decisions without delaying progress Partial Models A partial model represents the entire set of possible concrete models
- 28. • Motivation: • MBSE toolbox • Sources of uncertainty • Goal: MBSE toolbox for uncertainty-aware models • Creating the toolbox • Modeling uncertainty • Transforming models containing uncertainty • Reasoning about models • Uncertainty-reducing refinement • Tool support • Putting this into context 32
- 29. Transforming Models with Uncertainty 33 Natalie should be able to use model transformations [MODELS’13]
- 30. Transforming Models with Uncertainty 34 We need to lift Natalie’s transformations so that they can apply to models with uncertainty Existing transformation techniques do not support this! Natalie should be able to use model transformations [MODELS’13]
- 31. Reminder: Model Transformations With Graph Rewriting (“Classical”) 35 class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 Negative Application Condition Left Hand Side Right Hand Side EncapsulateVariable refactoring: Make fields private and add getter methods unless they belong to some inner class Example rule:
- 32. Graph Rewriting: Example Input Model 36 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Match
- 33. Graph Rewriting: Example Input Model 37 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC NAC also matches! ABORT !
- 34. Graph Rewriting: Example Input Model 2 38 SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Match
- 35. Graph Rewriting: Example Input Model 2 39 SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Delete
- 36. Graph Rewriting: Example Input Model 2 40 SolverException - effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Add
- 37. Graph Rewriting: Example Input Model 2 41 SolverException - effect : String + getEffect() : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Add
- 38. Graph Rewriting: Example Input Model 2 42 SolverException - effect : String + getEffect() : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC No more LHS matches. Stop.
- 39. And Now: Transforming Models With Uncertainty 43 Natalie wants to apply the rule to an input with uncertainty
- 40. Why Transforming Models with Uncertainty is Hard? 44 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Match??? X Y X v Y
- 41. Why Transforming Models with Uncertainty is Hard? 45 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Match??? X Y X v Y
- 42. Why Transforming Models with Uncertainty is Hard? 46 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHSLHSNAC Should we delete? Should we add? X Y X v Y
- 43. Goal: “Lifting” 47 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y ?
- 44. Goal: “Lifting” 48 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException ?
- 45. Goal: “Lifting” 49 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException Solver SolverException -effect : String +getEffect() : String Solver SolverException + effect : String Solver SolverException
- 46. Goal: “Lifting” 50 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException ( X∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X∧ Y ∧ ¬a ∧ b) v ( X∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Ya b
- 47. Goal: “Lifting” 51 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y ( X∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X∧ Y ∧ ¬a ∧ b) v ( X∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Ya b
- 48. Lifting Technique 52 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y (a) Find Match Step 1: Determine applicability
- 49. Lifting Technique 53 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y (a) Find Match (b) Make sure the rule applies to at least one concretization (requires solving a SAT problem) Step 1: Determine applicability
- 50. Lifting Technique 54 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y Solver X Step 1: Determine applicability Step 2: Transform graph SolverException + effect : String Y (a) Copy over unchanged parts
- 51. Lifting Technique 55 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y Solver SolverException + - effect : String +getEffect() : String X Ya b Step 1: Determine applicability Step 2: Transform graph (a) Copy over unchanged parts (b) Perform additions and deletions Added and deleted elements become Maybe
- 52. Lifting Technique 56 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHSLHSNAC Solver SolverException + effect : String X Y X v Y Step 1: Determine applicability Step 2: Transform graph Step 3: Transform formula ( X∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X∧ Y ∧ ¬a ∧ b) v ( X∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Ya b Constrain Maybe elements to ensure each that each concrete model is correctly transformed.
- 53. Lifting Technique Summary 57 Solver SolverException + effect : String X Y X v Y ( X∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X∧ Y ∧ ¬a ∧ b) v ( X∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Ya b Step 1: Determine applicability (Henshin + Z3 SMT Solver) Step 2: Transform graph (Hanshin) Step 3: Transform formula (Java + Z3 input strings)
- 54. Case Study • Object–relational mapping (ORM) • “Translate a class diagram to a relational database schema.” • Classic benchmark for model transformation research • Triple graph grammar with 5 layered graph rules [Varro06] 58 (Image from [Varro06])
- 55. Case Study • Input model: the Ecore metamodel • ORM for Ecore is important: cf. CDO and Teneo • Manually flattened inheritance hierarchy and adapted to the metamodel in [Varro06] • Resulting model had 65 model elements: • 17 classes, 17 associations, 6 generalization links, 25 attributes • Manually injected points of uncertainty to create partial models with increasing numbers of concretizations 59
- 56. Setup And Results • RQ: How does lifting scale with increasing uncertainty? • Varied: number of concretizations of input • Measured: time to complete the ORM transformation • Ran on Intel Core i7-2600 3.40GHz×4core, 8GB RAM, Ubuntu-64 12.10. • Runtime does not increase dramatically. Approach scales. 60 # concretizations 1 24 48 108 144 192 256 Time (seconds) 32.6 32.8 32.7 32.9 32.6 33.0 48.4
- 57. • Motivation: • MBSE toolbox • Sources of uncertainty • Goal: MBSE toolbox for uncertainty-aware models • Creating the toolbox • Modeling uncertainty • Transforming models containing uncertainty • Reasoning about models • Uncertainty-reducing refinement • Tool support • Putting this into context 61
- 58. Verification 62 Example property: “Every inner class has at least one attribute” Natalie’s favorite analysis technique How can you apply it to partial models?
- 59. Verification: the Naïve Approach 63 Example property: “Every inner class has at least one attribute” Enumerate:
- 60. Verification: Lifting 64 Example property: “Every inner class has at least one attribute” Natalie’s favorite analysis technique, LIFTED • Applies directly to the partial model • Does not enumerate concretizations • Computes result using three-valued logic …all concretizations …some but not all …none Property holds for…
- 61. Verification: Lifting • Checking syntactic properties [ICSE’12]: • Language independent • SAT-based • Checking semantic properties: • Depends on language semantics • Requires custom adaptation of verification technique • Examples: • Goal satisfaction in i* models [RE’12, RE’14] • MTSA: Model checking and other analyses over MTSs [ASE’08] 65
- 62. Scalability of Verification • Q: Is verification of models with uncertainty feasible? • How does it compare with checking each model individually? • What about comparison with the classical verification? • Q: How does the level of uncertainty affect this feasibility? Experiments: with random inputs and real case studies Random inputs: varying size of the model and level of uncertainty speedup = time for set of conventional models time to check partial model 66
- 63. Results 67
- 64. Case Study: Checking Properties • Why? Triangulate experimental results (random inputs) with observations from a real-word scenario • Case study details: • Bug from the UMLet bugzilla • Realistic bug fixes • Uncertainty: which combination of actions that fix the bug to choose from? • We checked two properties from the literature [SMSJ03] • Resulting partial model: 27,261 elements, 220 concretizations 68 [SMSJ03] R.V. Der Straeten, T. Mens, J. Simmonds, and V. Jonckers. “Using Description Logic to Maintain Consistency between UML Models”. In Proc. of UML’03, pp 326-340, 2003
- 65. Results 69
- 66. Results of Evaluation • Is there a speedup? - Yes, it is consistently faster than reasoning with the set. • How is speedup affected by changing model size and levels of uncertainty? - Speedup decreases with model size - Speedup increases with level of uncertainty - No slowdowns! 70 Reasoning with Partial models vs Reasoning with a set of conventional models
- 67. Property-Based Refinement 71 SATSolver SolverException + effect : String X Y X v Y Property Φ1: “Every inner class has at least one attribute” Goal: Create a partial model of the concretizations for which the property holds. (X v Y) Λ Φ1 We just need to update the May formula:
- 68. Uncertainty-Reducing Refinement • Systematically incorporate new information… • … Manually • Edit the uncertainty annotations • … Using automated transformations Verify that a transformation is always uncertainty reducing. 72 Unsure if it should be an inner class. X v Y Unsure if we need this field. You know what? I want to get rid of effect. [FASE’12, VOLT’12]
- 69. -numOfDoors : Int CarTimeMachine Boat (MS)OtherVehicles -lastLegalEntry : Timestamp -lastIDNumber : (V)IDType -previousIDNumber : (V)IDType -numOfPassengers : Int Vehicle Truck (MS) (V)IDType Timestamp P2:ClassDiagram Int A MAVO Refinement Not sure which security attributes - but know some will be needed. -(S)securityRelated : (SV)Types1 -numOfPassengers : Int Vehicle CarTimeMachine Boat (M)Hovercraft (MS)OtherVehicles -numOfDoors : Int (V)C1 (M) (M) (M) (M) (MS) (SV)Types1 Int P1:ClassDiagram Decision Not sure which class will hold the numOfDoors attribute Decision Not sure if we will have Hovercraft or which class it specializes. Decision: remove it Not sure what other vehicle types there will be Partial Decision Refines P1:MAVO(ClassDiagram)
- 70. Making the Toolbox... MMINT: Model Management INTeractive • A platform for developing (typed) model management tools • Model management environment: • Diff, refine, merge, etc. Available at https://github.com/adisandro/MMINT CSP SAT/SMT Henshin Alloy
- 71. Making the Toolbox... Uncertainty-aware MMINT: Model Management INTeractive • A platform for developing (typed) model management tools • Model management environment: • Diff, refine, merge, etc. Available at https://github.com/adisandro/MMINT MU-MMINT – lifting of model management operators + new ones CSP SAT/SMT Henshin AlloyMU-MMINT (pronounced “Moomin”)
- 72. • Motivation: • MBSE toolbox • Sources of uncertainty • Goal: MBSE toolbox for uncertainty-aware models • Creating the toolbox • Modeling uncertainty • Transforming models containing uncertainty • Reasoning about models • Uncertainty-reducing refinement • Tool support • Putting this into context 76
- 74. Back to Uncertainty Modeling • Our choice: • Qualitative, possibilistic uncertainty modeling • Keeping concrete syntax familiar • Encoding an exact set of potential classical models • Refinement – reducing the set of possibilities 78 Explicit uncertainty modeling with Partial Models + May formulas
- 75. Representation of Sets Uncertainty Product Lines Metamodels Megamodels 79 ConcretizationsPartial model ProductsProduct Line InstancesMetamodel MembersMegamodel
- 76. Inspired by: Modal Transition Systems (MTS) LTS Extension: – Required transitions – Possible transitions – the rest is proscribed (prohibited) – MTS toolbox – MTSA [ASE’08] request? request reply [Larsen/Thompson, 1988]
- 77. reply? request? request reply MTS = Set of LTS request? reply? request reply request request reply reply LTS
- 78. MTS = Set of LTS request? request reply reply? request? request reply request? reply? Refinement+- request reply request request reply reply LTS
- 79. Our Modeling of Uncertainty We generalize MTSs in two ways: 1. Make partiality modeling language-independent • Benefit: consistent applicability across all phases of SE • Cost: reasoning is only about syntactic properties 2. Use different types of partiality for different types of uncertainty • Benefit: methodological principles for expressing uncertainty
- 80. Aside: Complete w.r.t Scope and Span • Scope – what is of interest (and what is not) • Span – level of abstraction Valid call sequences of public methods of class SocketPolitical Map of Africa sdkfj sdkfj sdkfj sdkfjsdkfj sdkfj sdkfj sdkfj sdkfj
- 81. Some Similar Concepts • Non-determinism • Only in behavioral models • Corresponds to omitted information • Underspecification • Specification – saying what is known/required • Everything else about a given scope and span is unspecified (uncertainty is implicit) • With “our” uncertainty – can say what we do not know • Including statements about a given scope and span • Uncertainty is explicit 85 a a
- 82. Some More Similar Concepts • Incompleteness • Often same as underspecification • But can also include knowledge about the state of completeness about the model w.r.t. a given scope and span • “Our” uncertainty model can explicitly state which parts are incomplete 86
- 83. Uncertainty is a Meta-Property! • Allows the to capture: uncertainty of the specification uncertainty in the variability model uncertainty in a meta-model uncertainty in parts of a non-deterministic model 87
- 84. • Vagueness in modeling [Hermann] • Incompleteness in databases • Representing null values in db [Fagin, etc.] • Local Closed World Assumption [Cortes-Calabuig, etc.] Other Similar Concepts
- 85. Summary 89
- 86. Some Current and Future Work • Front-end for modeling uncertainty [MiSE’13] • Extending the uncertainty toolbox to handle full MAVO models • Lifting other techniques that require semantics to handle uncertainty • E.g., non-functional analysis • Understanding origins and models of uncertainty • Mining developer conversations to infer uncertainty in collaborative design [RSSE’14] • Building uncertainty dashboard to help the design process • Uncertainty patterns • Bottom-up: useful for tool building • Top down: detecting uncertainty in workflows and conversations 90
- 87. Some Current and Future Work • More complex modeling of uncertainty • Combining “our” uncertainty with environment uncertainty • Handling richer qualitative models (fuzzy logic? probabilities?) • Using explicit uncertainty modeling to bridge the gap with informal modeling/sketching • Bran Selic’s “formality slider” • Uncertainty all the way down to code • Uncertainty as a meta-aspect, to be handled using language workbenches • Lifting programming languages, e.g., Java • Symbolic execution of partial models • Lifting analyses to other set-handling formalisms • E.g., transformations in product lines [ICSE’14] • Code analyses? 91
- 88. 92 Questions?
- 89. Bibliography I [MODELS13] M. Famelis, R. Salay, A. Di Sandro, M. Chechik. “Transformation of Models Containing Uncertainty”. In Proc. of MODELS’13, 2013. [ICSE12] M. Famelis, M. Chechik, and R. Salay. “Partial Models: Towards Modeling and Reasoning with Uncertainty”. In Proc. of ICSE’12, 2012. [ICSE14] R. Salay, M. Famelis, J. Rubin, A. Di Sandro, M. Chechik. “Lifting Model Transformations to Product Lines”. In Proc. of ICSE’14, 2014. [FASE12] R. Salay, M. Famelis, and M. Chechik. “Language Independent Refinement using Partial Modeling”. In Proc. of FASE’12, 2012. 93
- 90. Bibliography II [Varro06] D. Varro, S. Varro-Gyapay, H. Ehrig, U. Prange, and G. Taentzer. “Termination Analysis of Model Transformations by Petri Nets”. In Proc. of ICGT’06, pages 260–274, 2006. [Classen09] A. Classen, P. Heymans, P. Schobbens, A. Legay, and J. Raskin. “Model Checking Lots of Systems: Efficient Verification of Temporal Properties in Software Product Lines”. In Proc. of ICSE'10, pages 335-344, 2010. [Svahnberg05] M. Svahnberg, J. Van Gurp, and J. Bosch. “A Taxonomy of Variability Realization Techniques”. Software: Practice and Experience, 35(8):705-754, 2005. 94
- 91. I don’t know vs I don’t care • Knowledge vs relevance scale 95 I care I don’t care I know Specification (classical) Out of scope I do not know Uncertainty Out of scope
- 92. MAVO • M – default is exists. Explicit only about uncertainty of existance • A • V • O – by default, the world is closed unless considered open 96
- 94. “Plural” Models “Singular” model m Plural model p “Deconstruct” D : p {m} “Extract” E: p x r m Lifted Transformation T : p1 p2 “Construct” C: {m} p 98 “Project” P : p1 x r p2 |p1| > |p2| The intent behind each abstraction determines pragmatics
- 95. Sets of Models – A Useful Abstraction? • Unified way of looking at set abstractions • Reuse methods from one abstraction to another • Chain abstractions? • Finding connections between fields: principled but practical outcomes 99
- 96. Example: Lifting Transformations for Product Lines 100 1. Determine applicability 2. Transform graph 3. Transform formula 1. Determine applicability 2. Transform graph 3. Transform presence conditions Transforming Partial Models Transforming Product Lines [ICSE’14] May elements May formula Feature Presence conditions