Software Testing
Software engineering
processes
Course objectives
Become familiar with the software
engineering processes
Have an image of the main software
life cycles
Place the testing processes into the
right place
Describe few classes of software
projects and their specifics
References
I. Sommerwille, Software Engineering, 8th
edition, Addison Wesley, 2006 (chapter 4)
G. Everett, R. McLeod: Software Testing. Testing
Across the Entire Software Development Life
Cycle, Wiley&Sons ,2007 (chapter 2)
C. Kaner, J. Falk, H.Q. Nguyen: Testing computer
software, Second edition, Wiley & Sons, 1999
(chapter 3)
K. Benk, C. Andress: Extreme Programming
Explained. Embrace Change. 2nd edition,
Addison-Wesley Professional, 2004
S. McConnell, Rapid Development:Taming Wild
Software Schedules, Microsoft Press,1996
Software projects
There are a potpourri of software projects
No single structure or process that
optimally applies to the requirements and
environments for all sorts of projects
The discipline and the art of managing
projects makes the project success
What makes the software projects
different?
Software Project Management
Software Engineering
What is software engineering?
is an engineering discipline that is concerned
with all aspects of software production from
the early stages of system specification to
maintaining the system after it has gone into
use.[Sommerville, Software engineering, 8th
edition, 2007]
is the application of a systematic, disciplined,
quantifiable approach to the development,
operation, and maintenance of software[IEEE,
"Guide to the Software Engineering Body of
Knowledge" (February 6, 2004)]
Software Project Management
Typical Phases in Software
Requirement analysis and Specification
Design
Implementation
Testing
Maintenance
Life cycle:
Establishes the order these activities are
performed
Establishes the criteria for moving from one
phase to the other
Software Project Management
Software specification
The process of establishing what
services are required and the
constraints on the systems operation
and development.
Requirements engineering process
Feasibility study;
Requirements elicitation and analysis;
Requirements specification;
Requirements validation.
Software Project Management
Requirements Validation
Showing that the requirements define what the
customer wants
Checks:
Validity checks compared to stakeholder needs
Consistency checks related to possible conflicts
Completeness checks related to the coverage
Realism checks ensuring that they could be
implemented
Verifiability proved by the potential of writing
acceptance tests
Techniques:
Reviews
Prototyping
Test case generation
Software Project Management
Software design
The process of converting the system
specification into an executable
system.
Software design
Describes software structure that realises
the specification;
Describes data which is part of the system
Describes interfaces between the system
components
Describes the algorithms
Software Project Management
Design process activities
Chapter 4,10,11,12 Sommerville
Architectural design: sub-systems identified and
documented
Abstract specification: abstract specification of
the services and services constraints per sub
system
Interface design: Unambiguously describes how
the subsystems interact
Component design: allocation of services per
components
Data structure design: data structures used in
implementation being detailed
Algorithm design: algorithms being detailed
Software Project Management
10
Programming and
debugging
Translating a design into a program and
removing errors from that program.
Programming is a personal activity there is no generic programming
process.
Programmers carry out some program
testing to discover faults in the
program and remove these faults in the
debugging process.
Software Project Management
11
Software testing
Verification and validation (V & V) is intended to
show that a system conforms to its specification
and meets the requirements of the system
customer.
Involves checking and review processes,
system testing and acceptance testing.
System testing involves executing the
system with test cases that are derived
from the specification of the real data to
be processed by the system.
Acceptance testing involves executing
tests intended to asses the way the
customer requirements were implemented.
Software Project Management
12
Software evolution
Software is inherently flexible and can
change.
As requirements change through changing
business circumstances, the software that
supports the business must also evolve and
change.
Although there has been a demarcation
between development and evolution
(maintenance) this is increasingly irrelevant
as fewer and fewer systems are completely
new.
Software Project Management
13
Life cycle choices
Life-cycle: a particular way of performing
activities related to different software
projects phases
Main idea: keep the focus to the goal
Possible goals:
Improve development speed
Improve quality
Improve project tracking and control
Improve client relations
Minimize overhead
Minimize risks
Support software evolution
Software Project Management
14
Waterfall Project Phases
Software Project Management
15
Waterfall
First applied software model
Basis for many other models
Orderly progress from initial concept
to system testing and maintenance
Review at each phase end. Return to
previous phase if no acceptance.
Document driven
Software Project Management
16
Waterfall
Phases are discontinuous: do not
overlap
Suitable for stable product definitions
and technologies
Find errors at early stages
Minimizes planning overhead
Suitable even for un-experienced
teams
Software Project Management
17
Waterfall Drawbacks
Risks not explicitly addressed
The product is not visible until final stages
The stability of requirements is not a valid
assumption in software development
Discovering missing features at the end of
the cycle is very expensive
Lack of flexibility
Costly
Longer schedules
Software Project Management
18
Waterfall testing
Testing at the end of the lifecycle
No interaction between testing activities
and the development activities
Consequence: keep the testing and
development departments separate
Software testing is considered as being a
destructive process
Programmers should not test their own
programs
Programming organizations should not
test their own programs.
V-Model
20
V-Model testing
Start testing activities early in the lifecycle
Clearly defines four kind of testing
activities and their chaining:
Unit testing
Integration testing
System testing
Acceptance testing
Testing plans affects the development
plans
Risk reduction: moving to
iterations
*Reproduced from the IBM testing
course
Spiral models
Spiral
Boehm 1988
(http://www.computer.org/portal/cms_docs_computer/co
mputer/homepage/misc/Boehm/r5061.pdf)
First major iterative model: planning spread across
iterations
Enhance waterfall by early risk identifications
Process is represented as a spiral rather than as a
sequence of activities with backtracking.
Each loop in the spiral represents a phase in the process.
Each spiral iteration consists in a major risk elimination.
No fixed phases such as specification or design - loops in
the spiral are chosen depending on what is required.
The last spiral iteration may be a waterfall cycle
Software Project Management
24
Spiral
The result of each iteration are more refined
prototypes until the released version
Advantages:
Early addressing of risks: as costs increases the
risk decreases
The management control is at least as much as
in the waterfall case, improved by the existence
of checkpoints at the end of each spiral
iteration
Disadvantages
Rather complicated to manage
Can be difficult to define milestones to indicate
if you are ready to move to the next iteration
Software Project Management
25
RUP phase model
Phaseiteration
Inception
Elaboration
Construction
Software Project Management
Transition
27
RUP phases
Inception - Define the project scope, gain
agreement on project objectives, baseline
the product Vision
Elaboration - Address key technical risks,
produce an evolutionary prototype,
baseline the Architecture
Construction - Iteratively and
incrementally develop an operationaly
complete product
Transition - Deliver the product into the
live end-user environment
Software Project Management
28
RUP good practice for business
driven development
Adapt The Process More processes is not
necessarily better
Balance Competing Stakeholder Priorities Manage often conflicting requirements
Collaborate Across Teams - Proper team
organization and the setting up of effective
collaborative environments.
Demonstrate Value Iteratively - Develop
software iteratively
Elevate Level Of Abstraction - Use
component-based architectures
Focus Continuously On Quality Verify software quality
Software Project Management
29
Practices and Anti-patterns
Practices:
Ensure team ownership of quality for the product.
Test early and continuously in step with
integration of demonstrable capabilities.
Incrementally build test automation.
Anti-patterns (compare this with the V-model)
To peer-review all artifacts and complete all unit
testing before integration testing.
To conduct in-depth peer-review of all
intermediate artifacts, which is counter
productive because it delays application testing
and hence identification of major issues.
RUP disciplines
Business Modeling: business processes modeled as business cases
Requirements: identification of actors and the development of system
use cases
Analysis & Design: a design model is created using architectural
models, component models, object models and sequence models
Implementation: The components are implemented and structured
into sub-systems
Test: iterative process carried out in conjunction with the
implementation. System testing is performed after
implementation
Deployment: a release is created, distributed and installed on users
systems
Configuration & Change Management: controls change to, and
maintains the integrity of, a projects artifacts.
Project Management: Provide a framework for managing software
intensive projects. Provide practical guidelines for planning, staffing,
executing, and monitoring projects.
Environment: Focuses on the activities and tools necessary to
configure the process for a project
RUP
Software Project Management
32
RUP
Software Project Management
33
RUP-Test Discipline
Finding and documenting defects in software
Generally advising about the perceived software
quality
Proving the validity of the assumptions made in
design and requirement specifications through
concrete demonstration
Verifying the software product functions as designed
Validating that the requirements have been
implemented appropriately
Focuses primarily on the evaluation or assessment of
quality realized through a number of core practices
Acts in many respects as a service provider to the
other disciplines
Agile Development
Customer centered: www.agilemanifesto.org
Extreme Programming XP
SCRUM
Highly iterative
Adapts to requirements changes
Focus on team communication
Good for small/skilled teams
Pair programming
Refactoring
Test driven development
Active customer involvement
May not scale for large projects
Software Project Management
35
Manifesto for Agile Software Development
We are uncovering better ways of
developing
software by doing it and helping others
do it.
Through this work we have come to
value:
Individuals and interactions over
processes and tools
Working software over comprehensive
documentation
Customer collaboration over contract
negotiation
Responding to change over following a
Kent Becks assumptions
If code reviews are good, we'll review code all the time (pair
programming).
If testing is good, everybody will test all the time (unit
testing), even the customers (functional testing).
If design is good, we'll make it part of everybody's daily
business (refactoring).
If simplicity is good, we'll always leave the system with the
simplest design that supports its current functionality (the
simplest thing that could possibly work).
If architecture is important, everybody will work defining
and refining the architecture all the time (metaphor).
If integration testing is important, then we'll integrate and
test several times a day (continuous integration).
If short iterations are good, we'll make the iterations really,
really shortseconds and minutes and hours, not weeks
and months and years (the Planning Game).
Software Project Management
37
XP Promises(Beck)
To programmers:
They will be able to work on things that really matter, every day.
They won't have to face scary situations alone.
They will be able to do everything in their power to make their system
successful.
They will make decisions that they can make best, and they won't
make decisions they aren't best qualified to make.
To customers and managers:
They will get the most possible value out of every programming week.
Every few weeks they will be able to see concrete progress on goals
they care about.
They will be able to change the direction of the project in the middle
of development without incurring exorbitant costs.
In short, XP:
reduce project risk,
improve responsiveness to business changes,
improve productivity throughout the life of a system,
add fun to building software in teams
Software Project Management
38
XP phases
Architectural Spike, where a prototype is created
to validate the primary concerns of an iteration:
reduces risk
Release planning phase, where the customer
writes stories, the programmers estimate them,
and the customer chooses the order in which
stories will be developed;
Iteration phase, where the customer writes tests
and answers questions, while the programmers
program. Versions are generated at few weeks.
Release phase, where the Programmers install
the software, and the Customer (hopefully)
accepts the result.
Software Project Management
39
XP
http://www.extremeprogramming.org/map/project.ht
ml
Software Project Management
40
XP Iterations
J. Shore, S. Warden: The Art Of Agile Development
Software Project Management
41
XP testing
Test early. This principle states that you must not wait
with testing until the entire system is assembled. Instead,
run test cases as soon as a unit is implemented, and
assemble your system out of carefully tested units.
Test first. Write test cases before implementing the unit.
This is useful because test cases can serve as
specifications. Any program feature without an
automated test simply doesn't exist.(K. Beck)
Test often. At the minimum, run your tests with each
release of the system. Better yet, run your tests with every
change.
Have others test. Always have someone independent
test your program, and be open to criticism
Mostly perform Unit tests and Acceptance testing
Agile vs RUP:
http://www.agiledata.org/essays/rup.html
SCRUM
SCRUM: eight players rugby team with
different roles acting together for a specific
goal
Agile methodology
Iteration based: 1-4 weeks iterations
(Sprints)
Small teams: 7-9 people
SCRUM: practices and roles
http://www.infoq.com/minibooks/
scrum-xp-from-the-trenches
Software Project Management
43
http://scrumtraininginstitute.com/home/stream_download/scru
mprimer
Software Project Management
44
SCRUM Roles
Product Owner: business representative
identifying product features
translating these into a prioritized list
deciding which should be at the top of the list for the next Sprint
continually re-prioritizing and refining the list.
Team: programmers, architects, testers, etc
builds the product that the Product Owner indicates
cross-functional it includes all the expertise necessary to deliver the
potentially shippable product each Sprint
self-organizing with a very high degree of autonomy and
accountability
Scrum Master
helps the product group learn and apply Scrum to achieve business
value
help the Team and Product Owner be successful
cannot be the same with the product owner
he/she is not the leader of the team
The Scrum Master is NOT the project manager!
Software Project Management
45
Product backlog
Product vision: product owner building the
product backlog
customer features but also technical tasks
user stories or use cases
continuously updated by the Product Owner
items having a business value estimate
helping maximizing the ROI
Release Backlog: the subset of the
Product Backlog that is intended for the
current release
Software Project Management
46
Sprint backlog
Creating the current sprint plan from the product backlog
Sprint planning meeting part one: Product Owner the Team and the Scrum
Master
Focuses on what are the PO priorities
Provides the team with insight about the goal and the context of the
sprint
Establishes what a done item mean (e.g. coded to standards, reviewed,
implemented with unit test-driven development (TDD), tested with 100
percent test automation, integrated, and documented)
Sprint planning meeting part two: the Team
focuses on detailed task planning for how to implement the items that
the Team decides to take on
the Team selects the items from the Product Backlog they commit to
complete by the end of the Sprint, starting at the top of the Product
Backlog
It is updated every day with the remaining time for each task
Sprint Burndown chart: estimate of how much work (measured in person
hours) remains until the Teams tasks are finished.
It is a downward sloping graph that is on a trajectory to reach zero effort
remaining by the last day of the Sprint
Software Project Management
47
Daily Scrum
Short (15 minutes or less) meeting that happens every
workday at an appointed time.
Everyone on the Team attends.
To keep it brief, it is recommended that everyone
remain standing.
It is the Teams opportunity to synchronize their work
and report to each other on obstacles.
No management involved.
Each member of the Team reports three (and only
three) things to the other members of the Team:
what they were able to get done since the last meeting;
what they are planning to finish by the next meeting;
any blocks or impediments that are in their way.
Software Project Management
48
Sprint review
After the Sprint ends;
The Team and the Product Owner review the
Sprint;
An inspect and adapt activity for the product;
Product Owner learns what is going on with the
product and with the Team;
The Team learns what is going on with the
Product Owner and the market;
Attended by: Product Owner, Team members, and
Scrum Master, plus customers, stakeholders,
experts, executives, and anyone else interested.
Software Project Management
49
Sprint retrospective
Inspect and adapt regarding the process.
Its the main mechanism for taking the
visibility that Scrum provides into areas of
potential improvement, and turning it into
results.
Its an opportunity for the Team to discuss
whats working and whats not working, and
agree on changes to try.
The Team and Scrum Master will attend, and
the Product Owner is welcome but not
required to attend.
Software Project Management
50
Release sprint
Final sprint to complete the release
Integrate some elements, polish, etc
Sign of some development weakness
as ideally at the end of each sprint
the product should be good enough
for use
Software Project Management
51
Common system types
Business Systems
Mission-Critical Systems
Embedded Life-Critical Systems
Internet site
Embedded software
Avionics Software
Intranet site
Games
Embedded software
Inventory management
Internet site
Medical devices
Games
Packaged software
Operating Systems
Management Information Systems
Software tools
Packaged software
Payroll systems
Web services
Source: McConnel, Code Complete
Software Project Management
53
Life cycle model
Business Systems
Mission-Critical Systems
Agile development (Extreme
Programming, Scrum, timebox
development, and so on)
Staged delivery
Evolutionary prototyping
Evolutionary delivery
Embedded Life-Critical Systems
Staged delivery
Spiral development
Evolutionary delivery
Spiral development
Software Project Management
54
Project planning and
management
Business Systems
Mission-Critical Systems
Embedded Life-Critical Systems
Incremental project planning
Basic up-front planning
Extensive up-front planning
As-needed test and QA planning
Basic test planning
Extensive test planning
Informal change control
As-needed QA planning
Extensive QA planning
Formal change control
Rigorous change control
Software Project Management
55
Requirements and design
Business Systems
Mission-Critical Systems
Informal requirements specification Semiformal requirements
specification
Design and coding are combined
Embedded Life-Critical Systems
Formal requirements
specification
As-needed requirements reviews
Formal requirements inspections
Architectural design
Architectural design
Informal detailed design
Formal architecture inspections
As-needed design reviews
Formal detailed design
Formal detailed design
inspections
Software Project Management
56
*Coding, Testing and
Deployment*
Business Systems
Mission-Critical Systems
Embedded Life-Critical Systems
Pair programming or individual
coding
Pair programming or individual
coding
Pair programming or individual
coding
Informal check-in procedure or no
check-in procedure
Informal check-in procedure
Formal check-in procedure
As-needed code reviews
Formal code inspections
Developers test their own code
Developers test their own code
Developers test their own code
Test-first development
Test-first development
Test-first development
Little or no testing by a separate
test group
Separate testing group
Separate testing group
Separate QA group
Informal deployment procedure
Formal deployment procedure
Software Project Management
Formal deployment procedure
57
