SOFTWARE QUALITY
ASSURANCE
(BISE 3033)
CHAPTER 6
Software Testing
Integration Testing
Integration testing is an intermediate phase
between unit testing and system testing
Involves testing the modules in executable
combination (exe file)
Is an iterative process involving the testing of sub-
assemblies which are repeatedly integrated into
larger sub-assemblies culminating in the testing of
the system as a whole
Two aspects to consider:
The testing of interfaces between modules
The testing of the interface between the integrated
assembly and its environment
Integration Testing
Each of the following modules, shown below in the
application's control flow chart, has finally passed unit
testing standards. How do you plan to conduct integration
testing?
B C D E
F G H I J K
L M
User Interface Testing
User Interface Validation
paper prototypes made early are great
User Interface Evaluation
Expert / Heuristic Evaluation
Label Testing
is a "rose" still a rose by any other name?
User Interface Testing
User Interface Testing
Cognitive Walkthroughs
Usability Testing
hopefully the SRS specifically defines usability criteria
Random Testing
automated random key/mouse presses
What's the problem?
http://homepage.mac.com/bradster/iarchitect/color.htm
Dialog Boxes
Cognitive Walkthroughs
Participants
real end user - sits at keyboard and performs tasks
evaluator - takes notes and asks questions, mostly quiet
developer - probably hidden or watches video
Results
effectiveness
Will the user associate the next action with the appropriate
interface control?
Will the user notice an action is available?
If the correct action is performed, will the user see that
progress is being made toward solution of the task?
Heuristics
Simple and natural dialogue
Aesthetic and minimalist design
Speak the user's language
Minimize user memory load
Recognition rather than recall
Consistency
Feedback
Visibility of system status
User control and freedom
Clearly marked exits
Shortcuts and Flexibility
Good error messages
Help users recognize, diagnose, and recover from errors
modified from http://en.wikipedia.org/wiki/Heuristic_evaluation
Validation Testing
Alpha Testing
Beta Testing
Alpha Test Beta Test
What they do Improve the quality of
the product, integrate
Improve the quality of
customer input on the
the product and
complete product, and
ensure beta readiness.
ensure release
readiness.
When they happen Toward the end of a
Just prior to launch,
development process
sometimes ending
when the product is in
within weeks or even
a near fully-usable
days of final release.
state.
How long they last Usually very long and Usually only a few
see many iterations. weeks (sometimes up
It’s not uncommon for to a couple of months)
alpha to last 3-5x the with few major
length of beta. iterations.
Alpha Test Beta Test
Who cares about it Usually involves
product marketing,
Almost exclusively
support, docs, quality
quality/engineering
and engineering
(bugs, bugs, bugs).
(basically the entire
product team).
Who participates Normally performed by
(tests) test engineers, Tested in the “real
employees, and world” with “real
sometimes “friends customers” and the
and family”. Focuses feedback can cover
on testing that would every element of the
emulate ~80% of the product.
customers.
Alpha Test Beta Test
What testers should Plenty of bugs, crashes, Some bugs, fewer
expect missing docs and crashes, most docs,
features. feature complete.
How they’re addressed Much of the feedback
Most known critical collected is considered
issues are fixed, some for and/or implemented
features may change or in future versions of the
be added as a result of product. Only
early feedback. important/critical
changes are made.
What they achieve About methodology, About chaos, reality,
efficiency and regiment. and imagination. Beta
A good alpha test sets tests explore the limits
well-defined of a product by allowing
benchmarks and customers to explore
measures a product every element of the
against those product in their native
benchmarks. environments.
System Testing
A complex activity that aims to test the system as a
whole
Initially system testing is concerned with proving that
the system works correctly as a whole:
it is merely a higher level of integration testing
once this has been demonstrated to developers to their own
satisfaction, the process broadens in a number of
dimensions
First
the performance of the system rather than correctness
Second
test on the business system
Third
demonstrate technical correctness in the eyes of users
System Testing (cont..)
System testing involve a broad range of users i.e.
true users, their managers, system support
staff, computer operations staff and auditors
Many of the problems of system testing are not
technical problem, but problems of communication
and human nature.
Those problems must have been dealt with
continuously throughout design and development.
If not, system testing is likely to be not the finale of
development.
Regression Testing Retesting
Regression testing is a type of software Retesting is done to make sure that the
testing that intends to ensure that tests cases which failed in last
changes like defect fixes or execution are passing after the defects
enhancements to the module or against those failures are fixed.
application have not affecting
unchanged part.
Regression testing is not carried out on Retesting is carried out based on the
specific defect fixes. It is planned as defect fixes.
specific area or full regression testing.
In Regression testing, you can include In Retesting, you can include the test
the test cases which passed earlier. We cases which failed earlier. We can say
can say that check the functionality that check the functionality which was
which was working earlier. failed in earlier build.
Regression test cases we use are Test cases for Retesting cannot be
derived from the functional specification, prepared before start testing. In
the user manuals, user tutorials, and Retesting only re-execute the test cases
defect reports in relation to corrected failed in the prior execution.
problems.
Regression Testing Retesting
Automation is the key for regression You cannot automate the test cases for
testing. Retesting.
Manual regression testing tends to get
more expensive with each new release.
Regression testing is right time to start
automating test cases.
Defect verification is not comes under Defect verification is comes under
Regression testing. Retesting.
Based on the availability of resources Priority of Retesting over Regression
the Regression testing can be carried testing is higher, so it is carried out
out parallel with Retesting. before regression testing.
And the glue that holds it all
together…
The Test Plan
who
what
when
where
how
Test Plan Considerations
What are the critical or most complex modules?
make sure they get integration tested first
probably deserve white-box attention
Where have you had problems in the past?
What training is required?
conducting formal reviews
use of testing tools
defect report logging
IEEE 829 - Standard for
Software Test Documentation
Recommends 8 types of testing documents:
1. Test Plan
next slide
2. Test Design Specification
expected results, pass criteria, ...
3. Test Case Specification
test data for use in running the tests
4. Test Procedure Specification
how to run each test
5. Test Item Transmittal Report
reporting on when components have progressed from one stage of
testing to the next
6. Test Log
7. Test Incident Report
for any test that failed, the actual versus expected result
8. Test Summary Report
management report
Test Plan Contents (IEEE 829 format)
1. Test Plan Identifier 11. Test Deliverables
2. References 12. Remaining Test Tasks
3. Introduction 13. Environmental Needs
4. Test Items 14. Staffing and Training Needs
see next slide
15. Responsibilities
5. Software Risk Issues
16. Schedule
6. Features to be Tested
17. Planning Risks and
7. Features not to be Tested Contingencies
8. Approach 18. Approvals
9. Item Pass/Fail Criteria 19. Glossary
10. Suspension Criteria and
Resumption Requirements
http://en.wikipedia.org/wiki/Test_Plan
Test Items
Requirements Specification
Design
Modules
User/Operator Material
the user interface
User Guide
Operations Guide
Features
response time, data accuracy, security, etc
System Validation
alpha and beta testing
The V-Model
The V-Model of software development illustrates
when testing activities should take place
The test at each level exercise the corresponding
development activity-the same principles apply
no matter what software life cycle model used
The most important factor for successful
application of the V-Model is the issue of when
the test cases are designed
The test design activity always find defects in
whatever the tests are designed against
26
The V-Model (cont..)
For example:
Designing acceptance test cases will find defects
in the requirement
Designing system test cases will find defects in the
functional specification
Designing integration test cases will find defects in
the design
Designing unit test cases will find defects in the
code
If test design is left until the last possible
moment, these defects will only be found
immediately before those tests would be run,
when it is more expensive to fix them
27
The V-Model (cont..)
Requirements test Acceptance
s test
Functions test
System test
s
Design test Integration test
s
Write test Code Unit Run
test
s test tests
28
Tool Support for Life-Cycle
Testing
Tool support is available for testing in every
stage of the software development life cycle
It helps to derive test inputs or test data
29
Progress….
Pre-Project Activities
how will we define quality
contract review
writing a plan for quality assurance
Development Activities
Formal Reviews, Peer Reviews, unit testing, integration testing
Test plan You are here
Infrastructure Needs and Management Tasks
Procedures and Work Instructions
Training
Configuration Control
Tracking Progress
Preventative Actions
CMM
Thank You
