Vaidyanathan Ramalingam Testing Checklist Conference Speech
2 likes560 views
The document outlines five key leadership lessons for the test and QA profession, emphasizing the differences between waterfall and agile testing approaches, the importance of a testing checklist, the economics of trade-offs in testing, the software testing ecosystem, and the significance of root cause analysis (RCA). It draws from the speaker's extensive experience in the field and promotes knowledge sharing in software testing practices. The content is non-commercial and aims to enhance quality and efficiency in testing methodologies.
![5. RCA (Root Cause Analysis) 2 of 2
feedback
(I) Requirement Analysis & Architecture Phase Defects RCA
with Corrective & Preventive Actions
RCA Scope: Requirement Documents / RCA on: Requirement & Architecture RCA by: Sys. Analyst/Architects/Product
Architecture Defects owner; Source: Review comments, Defects
(II) Design & Development Phase/Iterations/Sprints Defects RCA
with Corrective & Preventive Actions
RCA on: Design Issues; Unit & Module Test RCA by: System Architects & Developers
RCA Scope: Design, Source Code
Defects Source: Review comments, Defects
(III) Testing Phase/Iterations/Sprints Defects RCA
with Corrective & Preventive Actions
RCA Scope: Test Cases/Automation suite of RCA by: Testers
RCA on: Product / Application Defects
New & Old Features / Maintenance Fixes Source: Defects & Customers Feedback
(IV) SDLC [Software Development Life Cycle] Defects RCA with Corrective & Preventive Actions
RCA by: Quality Engineers; Source: Defects (in-phase/post release)
RCA Scope: To analyze root causes on slipped defects & where/how the defects got customer
originated/injected.
Outcome: Report on systematic improvements w.r.t. SDLC phases/iterations/sprints/release
Iteration /sprint/release wise feedback to previous SDLC phases. Teams to conduct iterative/in-phase/Post Release RCA.
Thus team can improve/prevent on defects slippage in next iteration/sprints/release.](https://image.slidesharecdn.com/vaidyanathanramalingamtestingchecklistconferencespeech-13096966628601-phpapp02-110703074822-phpapp02/85/Vaidyanathan-Ramalingam-Testing-Checklist-Conference-Speech-9-320.jpg)
Vaidyanathan Ramalingam Testing Checklist Conference Speech
- 1. Leadership Lessons for the Test & QA Profession Vaidyanathan Ramalingam (Vaidya) Director Engineering (Test) Huawei Technologies India Pvt. Ltd., Bangalore, INDIA http://www.siliconindia.com/events-speakers/software-testing-Bangalore-SoftecBangalore_2011.html
- 2. 5 Key Lessons! 1. Waterfall Testing Vs Agile Testing 2. Testing Checklist – 5 W & 2 H 3. Trade Off Economics in Testing 4. Software Testing Eco System 5. RCA (Root Cause Analysis) Huawei is a global telecommunications solutions provider with long-term partnerships with operators around the world. Huawei actively participates in 91 international standardization organization including ITU, 3GPP, 3GPP2, ETSI, IETF, OMA andIEEE. Huawei holds the world’s #1 position in PCT Patent applications (WIPO 2008) and is ranked among the top 3 in LTE essential patents. Huawei’s products and solutions have been deployed in over 100 countries and have served 45 of the world's top 50 telecom operators, as well as one third of the world's population. www.huawei.com *The content of this presentation is non commercial and based on self experience, interpretations/view points. The data provided may not be full, accurate and latest. The intention is knowledge sharing & to promote the software testing. The Product names, logos, brands, standard body/institute’s names & other trademarks referred within the presentation are the property of their respective trademark holders.
- 3. 1. Waterfall Testing Vs Agile Testing (1 of 2) Area Testing In Waterfall Model Testing in Agile Model (1) Test •Complete & Base-line (freeze) •Incremental Req. as stories Requirement •Separate Change Request / •To accommodate change/dynamic Req. Enhancement •Req. prioritized based on: business values of customers, early realization & feedback (2) Test •Detailed upfront test strategy •Initial test strategy Strategy •Covers e2e test scope • incremental changes during iterations/sprints (3) Test Team •Independent team /mindset •Collaborative team (dev & test) ; common goal; Agile mindset Structure •Test Manager: Project & •Test Manager: Resource Management (people •Test Planning & Estimation (input to backlogs) & lab) •Resource Management (people & lab) •Defined & Clear role •Scrum Master: Release/Scrum Management •Lack of role clarity & 100% buy-in on agile practices (4) Test Cases •One time TC dev. •Iterative/Sprint wise TCs dev. @ story level (Functional & Non (TCs) •Longer time Functional) Development •Ver. level contribution (partial •TCs for inter-relation between stories (within & previous iterations) & at the end) for acceptance •Story level acceptance TCs dev. with customers tests (5) Test •Automation behind manual •Iteration/Sprint based automation & in C.I. Automation test phase •Shortage of time (poor scope) leads to missing/detailed scenarios •Typically separate manual, •Automation suite quality deteriorates - Poor Focus on: automation & performance test – Scripting standard/review team – SDLC approach – Maintenance (Ver.2 Ver.) – No Synch.: code <-> automation •Lack of attention leads to: Random Failures, Longer time to run, GUI errors, where the issue: defect or automation suite? •Coverage metric
- 4. 1. Waterfall Testing Vs Agile Testing (2 of 2) Area Testing In Waterfall Model Testing in Agile Model (6) •Not strict •Strictly followed (early feedback/defects); back bone of agile Continuous •Relatively longer time •Efficient CI topology, architecture & timely automation suite Integration •Relatively less frequent builds are critical (C.I.) •CI pass rate metric (7) Test Lab •Required just before the testing •Required from initial iteration onwards / Test bed (due to phase approach) (8) Test •Longer, dedicated & multiple •All the stories TCs tested within the iteration/sprint Execution test cycles •Continuous product/application visibility to testers •Late product/application visibility •Customer demo at each iteration •Phase wise test cycles •Short test cycles for Non Functional Testing @ iteration end •Same Test engineers to iteratively test & be an expert in: – customer domain – test design – test automation / scripting – Non functional test – C.I. (9) Defect •Zero defect target @ end of Ver. •Early & iterative defect finding Finding •Relatively Defect backlog is •Less/zero iterative defect backlog larger •How much re-work metric •Test cycle wise # defects trend •Iteration wise # defects trend (much controlled) (high-low-high-low ….)
- 5. 2. Testing Checklist – 5 W & 2 H Why To TEST? To meet the Req., Architecture, Design, Code, Production Environment, Usability, Interoperability, Migration and their changes What to TEST? Customer/Domain Requirements, Test Strategy, Weak & Strong Area, Test Iteration/Sprint Scope or Focus, Buggy Module, When to TEST? Timeline asked for test iterations/sprints/phases/post release tests [R&D or Field tests) Who will TEST? Functional/Non Functional testers; integration/field testers; ext. certification bodies, customer reps. Where To TEST? Environment: Typical R&D Env, Integration (Platform/Component/Solution/Multiple Inter-operation Systems) How to TEST? Test Techniques, Automation, Script less Automation (Ref: Qualitia), Model Based Testing (Ref: Conformiq), Hypothesis Based Testing (Ref: STAG S/w) How Long to TEST? #Test Cycles, Release quality criteria, Iteration/ Sprints check point exit quality, Defects Trend, deadline/feedback based approach Note: The same approach can be applied for Test Automation also. Example: Why to automate? What to automate? When to automate?....
- 6. 3. Trade Off Economics in Testing •Test coverage scope •Iteration/Sprint & Release •Iteration/sprint wise plan quality goal •Test priorities •Design/Code changes •Automation scope / goal •Continuous reporting to •Late features / stories PMO & prioritization •Test Scope Risk mitigation •Defect Trend & Backlog •Quality Risk Mitigation •Test Estimation/planning & •Test phases/ iterations resource allocation •Handover to QA timeline •Test bed/lab need •Iteration/Sprint/Version •Quality based (defect level release dates trend/backlog) test cycles •Automation & Defect •Automation cost verification timeline •Cost Risk Mitigation •Time Risk Mitigation
- 7. 4. Software Testing Eco System Technology (DB, OS, Platform, Management / Client QA / Testing Development Program Office / IDE & Configuration/Topology, Capital / Budget (add/delete/modify/ Product Owner Standards & Protocols) enhancements) Human Resource (On-site, Competency Development Offshore, Customers’ site, Framework Vendor partners) Defect Management System Quality & Release Process & Project Tracking Tools Framework (SDLC or PDLC, Agile/Lean/ISO/CMMi etc) Test Automation & IT Support / Professional Tools Centre / CoE Services / Field & Tech Support Software Configuration Software Build & System Test Lab / Document Repository / Access to Customers / Management Continuous Integration Test bed / Third party Test Artifacts & Clients / Remote certification or Knowledge Management Environments / Cloud benchmark / Cloud Framework Production Environments / Test environment HQ (Needs collaboration with each stakeholders / elements to ensure QUALITY, COST, TIME-LINE & SCOPE based on your project need!)
- 8. 5. RCA (Root Cause Analysis) 1 of 2 Focus: within the version Focus: version to version RCA on defects must be used like RCA on defects should “not” be like “preventive health check-up” to live longer investigating “dead body” during & healthy (In-Phase Quality) postmortem (Post Release Quality) The software we build & test to have in-phase Often (in waterfall model) the RCA is done at RCA at every checkpoints/ iterations/ sprints to the end of the project to improve the “next” ensure no defect slippage to next stage release (not before death of the patient) Each phase/iteration/sprint to demand quality In this case the poor quality impact is already from previous one and ensure to next one made to S/w customers. (RCA learning in this Quality to be achieved continuously! case saves other patients, not the one who is already dead!)
- 9. 5. RCA (Root Cause Analysis) 2 of 2 feedback (I) Requirement Analysis & Architecture Phase Defects RCA with Corrective & Preventive Actions RCA Scope: Requirement Documents / RCA on: Requirement & Architecture RCA by: Sys. Analyst/Architects/Product Architecture Defects owner; Source: Review comments, Defects (II) Design & Development Phase/Iterations/Sprints Defects RCA with Corrective & Preventive Actions RCA on: Design Issues; Unit & Module Test RCA by: System Architects & Developers RCA Scope: Design, Source Code Defects Source: Review comments, Defects (III) Testing Phase/Iterations/Sprints Defects RCA with Corrective & Preventive Actions RCA Scope: Test Cases/Automation suite of RCA by: Testers RCA on: Product / Application Defects New & Old Features / Maintenance Fixes Source: Defects & Customers Feedback (IV) SDLC [Software Development Life Cycle] Defects RCA with Corrective & Preventive Actions RCA by: Quality Engineers; Source: Defects (in-phase/post release) RCA Scope: To analyze root causes on slipped defects & where/how the defects got customer originated/injected. Outcome: Report on systematic improvements w.r.t. SDLC phases/iterations/sprints/release Iteration /sprint/release wise feedback to previous SDLC phases. Teams to conduct iterative/in-phase/Post Release RCA. Thus team can improve/prevent on defects slippage in next iteration/sprints/release.
- 10. Knowledge Is Power Thank You! Feedback / Suggestions / Power Point Request to: [email protected] Vaidyanathan Ramalingam http://www.siliconindia.com/events-speakers/software-testing-Bangalore-SoftecBangalore_2011.html