Download to read offline
T5_Test_08182022T5_Test_08182022T5_Test_08182022
- 2. BGSV Embedded Academy(BEA) Technical Competence T1: Automotive Basics (Sensor, SW, Mobility Solution) T2: Automotive SW Architecture (AUTOSAR) T3: Embedded Programming T5: Test Overview Methodological Competence M1: SW Development Lifecycle M3: Clean Code Process Competence P1: Requirements Engineering P2: Design Principles P3: Review P4: Safety & Security BGSV EMBEDDED ACADEMY Purpose: Develop basic general embedded competence Focused Program to Develop Embedded Competence Classroom training, Online Self-learning, Live Demo
- 3. Disclaimer This slideis a part of BGSV Embedded Academy (BEA) program and only used for BEA training purposes. This slide is Bosch Global Software Technology Company Limited’s internal property. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution as well as in the event of applications for industrial property rights. This slide has some copyright images and text, which belong to the respective organizations.
- 4.
- 5. Birthday pen selling Birthdaypen selling
- 6. Airbag Deployment andCalibration Chassis Systems Control | BGSV/ESS_CC-PS | Aug/10/2019 © Robert Bosch GmbH 2017. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights
- 7. Why is Testingnecessary? What do software faults cost? 7 Why is Testing necessary?
- 8. Internal | BGSV| 22/10/2010 | © Bosch Global Software Technologies Company Limited 2010. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. Robert Bosch Engineering & Business Solutions Vietnam Company Limited 8 Hospital Revives Its "Dead" Patients Eighty-five hundred people at St. Mary's Mercy thought they were still alive. But the hospital's computers were telling them they were not. In October, Cathy Uhl gave a blood sample as part of a routine physical examination, but less than two months later—according to the hospital that processed her laboratory results—she was dead. Fortunately, Uhl and some 8,500 other patients who received treatment at St. Mary's Mercy Medical Center in Grand Rapids, Mich., between Oct. 25 and Dec. 11 were able to read of their unfortunate "demise" in the bills sent out by the hospital's flawed patient-management software system. THE BEA NEWS www.BEAnews.com THE BOSCH’S FAVOURITE NEWSPAPER - Since 2011
- 9. Internal | BGSV| 22/10/2010 | © Bosch Global Software Technologies Company Limited 2010. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. Robert Bosch Engineering & Business Solutions Vietnam Company Limited 9 A Bug and a Crash It took the European Space Agency 10 years and $7 billion to produce Ariane 5, a giant rocket capable of hurling a pair of three-ton satellites into orbit with each launch and intended to give Europe overwhelming supremacy in the commercial space business. All it took to explode that rocket less than a minute into its maiden voyage last June was a small computer program trying to stuff a 64-bit number into a 16-bit space. Self-destruction was triggered automatically because aerodynamic forces were ripping the boosters from the rocket. http://www.around.com/arian e.html THE BEA NEWS www.BEAnews.com THE BOSCH’S FAVOURITE NEWSPAPER - Since 2011
- 10. Why is Testingnecessary? What do software faults cost? huge sums Ariane 5 ($7billion) Mariner space probe to Venus ($250m) American Airlines ($50m) very little or nothing at all minor inconvenience no visible or physical detrimental impact software is not “linear” small input may have very large effect 10 Why is Testing necessary?
- 11. Why is Testingnecessary? 11 Why is Testing necessary?
- 12. A person makes anerror ... … that creates a fault in the software ... … that can cause a failure in operation Why is Testing necessary? 12 Why is Testing necessary?
- 13. Why is Testingnecessary? Causes of software defects Error (mistake): a human action that produces an incorrect result Fault (defect, bug): a manifestation of an error in software also known as a defect or bug if executed, a fault may cause a failure Failure: deviation of the software from its expected delivery or service (found defect) A human being can make an error (mistake), which produces a defect (fault, bug) in the code, in software or a system, or in a document. If a defect in code is executed, the system will fail to do what it should do (or do something it shouldn’t), causing a failure Failure is an event, fault is a state of the software, caused by an error 13 Why is Testing necessary?
- 14. Why is Testingnecessary? Why do faults occur in software? software is written by human beings who know something, but not everything who have skills, but aren’t perfect who do make mistakes (errors) under increasing pressure to deliver to strict deadlines no time to check but assumptions may be wrong systems may be incomplete if you have ever written software ... 14 Why is Testing necessary?
- 15. Why is Testingnecessary? Why do faults occur in software? Communication Software complexity Programming errors Changing requirements Poorly documented code Software development tools Environmental conditions 15 Why is Testing necessary?
- 16. Why is Testingnecessary? Why not just "test everything"? How many testcases do we need to test the below statement? UWORD a_uw, b_uw; a_uw = b_uw; 16 Why is Testing necessary?
- 17. Why is Testingnecessary? Exhaustive testing? What is exhaustive testing? when all the testers are exhausted when all the planned tests have been executed exercising all combinations of inputs and preconditions How much time will exhaustive testing take? infinite time not much time impractical amount of time 17 Why is Testing necessary?
- 18. Why is Testingnecessary? How much testing is enough? it’s never enough when you have done what you planned when your customer/user is happy when you have proved that the system works correctly it depends on the risks for your system Why is Testing necessary? 18
- 19. Checking the Softwareis OK!!?? What? The process of executing a program with the intent of finding errors. The process of executing a software system to determine whether it matches its specification and executes in its intended environment. The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of system or component. 19 What is Testing?
- 20. Why is Testing? Tocheck if there are any errors in a part or a product. To gain the confidence in the correctness of the product. To ensure the quality and satisfaction of the product. Why? 20 What is Testing?
- 21. 21 Internal | RBEI/ECT| 10/01/2011 | © Robert Bosch Engineering and Business Solutions Limited 2011. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. What is Testing Definition as per ISTQB standard glossary The process consisting of all lifecycle activities both static and dynamic, Concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects Objectives of testing Finding defects Gaining confidence about level of quality Providing information for decision making Preventing defects Testing and Debugging Testing (done by the tester) Debugging (done by the developer)
- 22. Verification – arewe building the product correctly?* – The set of activities to ensure that software correctly implements specific functions Validation – are we building the correct product?* – The set of activities to ensure that the developed software is traceable to customer requirements V&V 22 V&V
- 23.
- 24.
- 25. Seven Testing Principles Principle1 – Testing shows presence of defects Testing can show the presence of defects, but cannot prove there are no defects. Principle 2 – Exhaustive testing is impossible Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases Principle 3 – Early testing Testing early as possible Principle 4 – Defect clustering Most of defects are found in a small number of module Principle 5 – Pesticide paradox Test cases also need to be updated Principle 6 – Testing is context dependent Testing is done differently in different contexts. Principle 7 – Absence-of-errors fallacy Testing is useless if the system does not meet the user’s requirements Prioritise tests so that, whenever you stop testing, you have done the best testing in the time available. 25 Seven Testing Principles
- 26. Cost of fixingfaults Source: https://hkrtrainings.com/istqb-tutorial 26
- 27.
- 28. V Model inASPICE Source: https://evav.omnex.com/7-levers/automotive-spice 28
- 29. Test Levels Courtesy :Practical Software Testing , illene Burnstein Test Levels Unit testing Integration testing System testing Acceptance testing 29
- 30. Test Levels Unit Test Integration SystemTest Acceptance A minimal software item for which a separate specification is available. lowest level tested in isolation Separately testable usually done by programmer Stubs, Drivers and Simulators also known as component, module, program testing Unit Test 30
- 31. Test Levels Unit Test Integration SystemTest Acceptance more than one (tested) component communication between components integration strategy: big-bang vs incremental (top- down, bottom-up, functional) done by designers, analysts, or independent testers test non-functional aspects such as performance Integration Test 31
- 32. Test Levels Unit Test Integration SystemTest Acceptance last integration step functional functional requirements and requirements-based testing business process-based testing non-functional as important as functional requirements often poorly specified must be tested often done by independent test group System Test 32
- 33. Test Levels Unit Test Integration SystemTest Acceptance Final stage of validation customer (user) should perform or be closely involved customer can perform any test they wish, usually based on their business processes Use the product in a realistic way in its operational environment Focuses on building confidence rather than finding faults Alpha and Beta testing Acceptance Test 33
- 34. Fundamental Test Process Source:https://hkrtrainings.com/istqb-tutorial 34
- 35.
- 36.
- 37. Advantages of TestDesign Techniques Why need test techniques? Exhaustive testing (use of all possible inputs and conditions) is impractical must use a subset of all possible test cases must have high probability of detecting faults Need thought processes that help us select test cases more intelligently test case design techniques are such thought processes What is a testing technique? a procedure for selecting or designing tests based on a structural or functional model of the software successful at finding faults 'best' practice a way of deriving good test cases a way of objectively measuring a test effort 37
- 38. Test Design Techniques Threetypes of systematic technique Static (non-execution) examination of documentation, source code listings, etc. Functional (Black Box) based on behaviour / functionality of software Structural (White Box) based on structure of software Input Output Input Output 38
- 39. Test Design Techniques Static TestDesign Techniques Dynamic Walk Through Technical Review Inspection Specificatio n based Structure d based Experienc e based Statement Condition Decision MCD Error Guessing Exploratory State Transition Control Flow Classification tree Equivalence Partitioning Boundary value analysis 39
- 40. Static Testing Static techniquesdo not execute code!! 40 Benefits: Start early in life cycle, early feedback on quality issues can be established By detecting defects at an early stage, rework costs are most often low Rework effort is reduced, development productivity figures are likely to increase To be used before any tests are executed on the software Can be used to “test” any form of document including source code, design document and models, functional specifications and requirement specifications
- 41. Static Testing 41 Informal review widelyviewed as useful and cheap (but no one can prove it!) A helpful first step for chaotic organisations. Technical review or Peer review includes peer and technical experts, no management participation. Normally documented, fault-finding. Can be rather subjective Walkthrough author guides the group through a document and his or her thought processes, so all understand the same thing, consensus on changes to make Inspection formal individual and group checking, using sources and standards, according to generic and specific rules and checklists, using entry and exit criteria, Leader must be trained & certified, metrics required
- 42. Test Design Techniques Static TestDesign Techniques Dynamic Walk Through Technical Review Inspection Specificatio n based Structure d based Experienc e based Statement Condition Decision MCD Error Guessing Exploratory State Transition Control Flow Classification tree Equivalence Partitioning Boundary value analysis 42
- 43. Dynamic Testing 43 Specification-based (black-box)testing techniques A procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional of a component or system without reference to its internal structure. View the software as a black-box with inputs and outputs Have no knowledge of how the system or component is structured inside the box Concentrate on what the software does, not how it does Are appropriate at all levels of testing where a specification exists Input Output
- 44. Dynamic Testing 44 Structure-based (white-box)testing techniques A procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system Use the internal structure of the software to derive test cases Require knowledge of how the software is implemented, that is, how it works Can be used at all levels of testing, especially where there is good tool support for code coverage Input Output
- 45. Dynamic Testing 45 White boxor Black box testing techniques?
- 46. Test Design Techniques Static TestDesign Techniques Dynamic Walk Through Technical Review Inspection Specificatio n based Structure d based Experienc e based Statement Condition Decision MCD Error Guessing Exploratory State Transition Control Flow Classification tree Equivalence Partitioning Boundary value analysis 46
- 47. Dynamic Testing –Specificationbased – Equivalence Partitioning Equivalence partitioning (EP) Divide (partition) the inputs, outputs, etc. into areas which are the same (equivalent) Assumption: if one value works, all will work One from each partition better than all from one 0 1 100 101 Valid Invali d Invali d 47
- 48. Dynamic Testing –Specificationbased – Boundary Value Analysis Boundary Value Analysis (BVA) Faults tend to be found near boundaries Good place to look for faults Test values on both sides of boundaries 0 1 100 101 Valid Invali d Invali d 48
- 49. Dynamic Testing –Specificationbased – Equivalence Partitioning Why do both EP and BVA? If you do boundaries only, you have covered all the partitions as well technically correct and may be OK if everything works correctly! if the test fails, is the whole partition wrong, or is a boundary in the wrong place - have to test mid-partition anyway testing only extremes may not give confidence for typical use scenarios (especially for users) boundaries may be harder (more costly) to set up 49
- 50. Dynamic Testing –Specificationbased – Boundary Value Analysis 50 What are the equivalence partitioning & boundary values of below function?
- 51. Dynamic Testing –Specificationbased – Boundary Value Analysis 51 What are the equivalence partitioning & boundary values of below function?
- 52.
Editor's Notes
- #1 Section covering general information and data about the Bosch group.
- #5 Objective: Create the awareness of Everybody can make mistake, although problems are quite easy and do not need any complex competence to solve. Even making mistake, people still believe that they are doing right. 2. Bring the mindset and habit of Independently raising voice. People are different and respect other views. What: Using a story to let people trial the real experience of solving a basic but still have the confusion for the right outcome. How: Storyline: Today is the birthday of a seller. He’s very happy and want to sell a pen with the original price: 15$. A customer comes and gives him 50$ for buying 1 pen. As he does not have enough change, he needs to go to his neighbor to exchange the money to smaller price (e.g. 10$, 20$, 20$). Next day, he realizes that the money is fake. Question: Is he profitable or lost? What is the exact amount of profit / loss? Note: Trainer must give full clarity for all open points from participant to make no doubt for the information. Method: Using moderation skill to let people go through: Individual round (without any discussion with others) Individual answers … Still have multiple rounds but we do not have enough time for moving deeper. Summary: See objective from beginning.
- #6 Objective: Create the awareness of Impact we have when a software in automotive is failed (especially for safety relevant feature). Accuracy of milliseconds which are nothing for an enterprise software / web app could be a big issue for automotive software. Humanity of the work we are doing. 2. Bring the mindset and habit of Independently raising voice. How: Using the video and interactive conversation let people raise their ideas about their understanding for the video.
- #8 One of the funny sample to show that the faults cost very little or nothing at all.
- #9 One of the popular sample to show that the faults cost huge sums.
- #14 Trainer to ask participants for their real experience in the past.
- #16 UWORD: unsigned int 16, value from 0 65535 If test everything: How many testcases are required? Input: b_uw Output: a_uw Result 0 0 Passed 1 1 Passed … 65535 65535 Passed
- #19 There are various definition of What is Testing. It could be simple definition like this one.
- #21 Or the academic definition, based on ISTQB
- #22 Or, using the concept of V&V.
- #23 There are arguments for several years that what is really the scope of Verification? As per this model, Verification is the left part of the V.
- #24 However, there is also the concept of Verification is up to Software Test. Validation is from System Test and above.
- #26 Cost of fixing faults (bugs) is not linear. Finding bugs earlier in the development process translates to a lower cost per bug.
- #28 As per concept in ASPICE (which was learned in previous sessions for software development life cycle), there are multiple levels of software test.
- #29 However, in general it would come to these 4 levels.
- #30 To make it easy for understanding, unit test is testing for the minimal software item (could be a function, a public method, a file, …).
- #31 Integration test is testing for the multiple units (sometimes we call it as component / sub-system).
- #32 System test is to test for the whole system (multiple components / sub-systems).
- #33 Then we have the acceptance test for the full test of the whole product (multiple system).
- #34 Phases of Test Process: Details could be referred here http://tryqa.com/what-is-fundamental-test-process-in-software-testing/ Trainer just need to briefly go through the steps of these activities.
- #37 If exhaustive testing (use of all possible inputs and conditions) is impractical, what should we do? We need to Reduce the effort as much as possible. Have high probability of detecting faults.
- #40 Basically, static techniques do not execute code (Review)!
- #41 Definition could be bypassed as it is not important at this time. Participants can read later.
- #47 This concept is to reduce the number of testcases.
- #48 Together with the reduction, we look for the testcase that could have highest probability to detect the faults.
- #50 Equivalence Partition: [10..99] [100..199] [200..127] ? Defect here.
- #51 Equivalence Partition: x y [10..99] 1 [100..199] 2 [200..255] 3 [0..9] y Boundary values x y 10,99 1 100,199 2 200,255 3 0,9 y