International Software Testing Qualifications Board

Certified Tester Foundation Level

Performance Testing
Sample Exam Answers

Version 2018

Provided by
American Software Testing Qualifications Board
and
German Testing Board
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
1 B B is correct. Reproducible results is a key factor in PTFL- 1
performance test execution. The results should be 1.1.1
statistically identical when run on the same unchanged K2
system. A is not correct because tests are sometimes
quite complex to create. C is not correct because tests
generally should not be run in the actual production
environment, but rather a representative equivalent. D
is not correct because the test results may show that the
system does not meet expectations.
2 D D is correct. This is a description of spike testing. A is PTFL- 1
a description of load testing. B is a description of stress 1.2.1
testing. C is a description of scalability testing. K2
3 A A is correct. This is a description of load testing. B is a PTFL- 1
description of stress testing. C is a description of 1.2.1
scalability testing. D is a description of spike testing. K2
4 D D is correct. While this can occur at any time, it should PTFL- 1
start with unit testing. A is not correct because this 1.3.1
occurs at system or SIT. B is not correct because this K1
occurs during SIT. C is not correct because this is done
during integration testing
5 C C is correct. Testing through the API makes the most PTFL- 1
sense when the UI is likely to change, but the full 1.4.1
communication to and from the UI must be tested. A is K2
not correct because this is crowd testing and these
testers would be using the real UI. B is not correct
because the communications protocol level is one level
below the API level. D is not correct because many test
instances can be created with API testing.
6 D D is correct. With a memory leak, response time will PTFL- 1
degrade over time (the time length is unknown though 1.5.1
until you test it). A is an indication of underlying K2
performance issues such as bad design, under-powered
hardware, network latency, etc. B is an indication of
resource pool or queue or stack issues as well as invalid
timeout settings. C is an indication that some of the
resources may become saturated, but only at high loads
7 A A is correct. A high latency means that there are delays PTFL- 1
occurring in the network which may be due to 2.1.1
inadequate bandwidth. B is not true because low K2
latency is a good thing. C is not true because this is

Version 2018 Page 2 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)

trackable with the right tools. D is not true because if

there is variability, that needs to be understood as part
of the performance tests.
8 A A is correct. Aggregated results help to provide the big PTFL- 1
picture and to observe consistent trends. Aggregating 2.2.1
the results helps to identify and understand the K2
importance of outliers and to know which ones are
interesting vs just test aberrations. B is not correct
because aggregated results will eliminate the
information on the outliers. C is not correct because
aggregation allows you to see the big picture rather than
focusing on the individual results which may be due to
transient system noise. It is unlikely to ever reach the
point where “all” variations are understood. D is not
correct because this type of reporting will be too difficult
for stakeholders to understand and may focus on the
highly unlikely situations.
9 C C is correct. The log analysis tools scan the server logs PTFL- 1
for particular error types (high usage, memory errors, 2.3.1
etc.) and report on those. A is not correct as these are K2
general system monitoring tools. B is not correct as
these are performance testing tools. D is not correct
because the performance test results aren’t written to
the logs, rather the system activities during the
performance tests are written to the logs.
10 C C is correct. This is a description of a failure that has PTFL- 1
resulted from a spike test. A is not correct as this is a 2.4.1
result of an endurance test. B is not correct because K1
this is an example of resource degradation over time. D
is not correct because this is the result of a scalability
test.
11 A A is correct. Risk identification and analysis is done at PTFL- 1
the test planning stage. This information is then used 3.1.1
for the analysis and design of the tests. B, C and D are K2
incorrect because this is too late in the process.
12 C C is correct. This should occur during the test PTFL- 1
implementation and execution activities. A and B are 3.1.1
incorrect because this is when the test cases are K2
planned and designed. D is incorrect because this is too
late.

Version 2018 Page 3 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
13 B B is correct according to the syllabus. Mobile and PTFL- 1
embedded environments are particularly prone to 3.2.1
connectivity issues particularly when embedded K2
software is used in the IoT devices. A and D are
incorrect because these environments are not prone to
connection dropping. C is incorrect because other than
#7, the other environments are not prone to connection
dropping.
14 D D is correct. The virtualized environment uses shared PTFL- 1
resources across different applications, including 3.2.1
memory. The dynamic/cloud-based environment are K2
designed to dynamically scale, so while memory leaks
may occur, they are less of a risk because the
environment will scale to compensate. Client/Server
and browser-based environments are prone to memory
leaks, particularly since a lot of this code is written in C
and C++. Mobile environments are prone to leakage
and are particularly at risk due to the limited available
memory. The same is true of the embedded
environments (and these also tend to be written in
languages that have less protection against leaks).
Mainframes tend to run older code, often written in the
higher level languages that provide their own memory
management.
15 A A is correct. C and C++ require the programmer to do PTFL- 1
their own memory management which sometimes 3.2.1
results in memory leaks (where memory is allocated, but K2
not de-allocated). B, C and D would have the same
issues regardless of the language used.
16 D D is correct. Since performance is critical, the risks must PTFL- 1
be analyzed as early as possible and then repeatedly as 3.3.1
the system is assembled since risks may change both in K4
impact and likelihood. A is not correct because it ignores
looking at the risks during development. B is not correct
because it leaves out the requirements and
development and at the completion of development. C
is not correct because it leaves out the requirements and
development options for analysis.
17 C C is correct. The best way to do the testing is to first do PTFL- 1
a technical analysis because this can be done at the 3.4.1
design time. Once implemented, full testing will be K4

Version 2018 Page 4 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
needed to verify that the solution does indeed deliver the
performance required. A is not correct because this
leaves out the static testing. B is not correct because
this would bypass any issues that might occur at the UI
end and doesn’t consider the database loading. D is not
correct because the network is less likely an issue
regarding the data handling for such a large amount of
data.
18 B B is correct. Scalability is a technical objective for PTFL- 1
performance. A is not correct because this is a user- 4.1.1
based objective. C is not correct because this is also a K4
user-based objective and not a very clear one at that. D
is not correct because this is primarily a robustness test
rather than a performance test.
19 A A is correct. Because data retrieval time is critical to the PTFL- 1
project, you need to know who can access that data (so 4.1.2
you can get the user profiles created), what data they K4
will access (so you can be sure you have the right data
available) and how often will they access it (so you can
set up an operational profile). B is not correct because
this is more related to security than performance testing
and is not the most critical performance objective. C is
not correct because the data storage may be something
to investigate if the data retrieval proves slow, but we
don’t need to know it to conduct the tests. D is not
correct because that is after retrieval and the scenario
just discusses presenting the information to the
requestor.
20 B B is correct. This is information that should be conveyed PTFL- 1
to the business stakeholders. A is not correct because 4.1.3
this is information that should be shared with the K4
technical stakeholders. C is not correct because it is
incomplete. If testing will be conducted in production,
mitigation plans must also be made so that any impact
to production is understood. D is not correct because
this is a list of risks but doesn’t include a list of how the
tests will help mitigate these risks.
21 A A is correct. This is information that should be conveyed PTFL- 1
to the technical stakeholders because it shows how the 4.1.3
tests will be conducted. B is not correct because this is K4
information that should be shared with the business

Version 2018 Page 5 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
stakeholders. C is not correct because it is incomplete.
If testing will be conducted in production, mitigation
plans must also be made so that any impact to
production is understood. D is not correct because this
is a list of risks but doesn’t include a list of how the tests
will help mitigate these risks.
22 B B is correct. This is an example of a web protocol. A is PTFL- 1
an example of a database protocol. C is an example of 4.2.1
a web service protocol. D is an example of a network K2
protocol.
23 A A is correct. This is an example of a database protocol. PTFL- 1
B is an example of a web protocol. C is an example of a 4.2.1
web service protocol. D is an example of a network K2
protocol.
24 B B is correct. This is referred to as think time. A, C and PTFL- 1
D are incorrect. 4.2.2
K2
25 B B is correct. The discrete transactions can be nested PTFL- 1
together to provide response information for a series of 4.2.2
transactions that would normally be performed together K2
while still allowing measurement of the discrete
transactions as well. A is not correct because parent
and child transactions may not accomplish anything
meaningful in terms of system usage. C is not correct
because it will not affect the reporting time, particularly
if the discrete transactions are tracked as well. It may
increase reporting time. D is not correct because this
has nothing to do with bypassing network
communications.
26 B B is correct. At this point, we need to know how many PTFL- 1
users of this type will be accessing the system. We 4.2.3
know the concurrency because we can take the number K4
of users and the accesses per shift to figure this out. A
is not needed for this instance because we already have
the information. C and D are considerations for setting
up the load profile but not needed for the operational
profile.
27 C C is correct. See the calculations below. PTFL- 1
4.2.4
K4

Version 2018 Page 6 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
Total transactions should be 2500 doctors x 10
accesses x 6 record updates: 150,000 transactions per
day. Hourly load:

7 hours of 500 doctors = 21,000 transactions

11 hours of 1000 doctors = 66,000 transactions
3 hours of 1500 doctors = 27,000 transactions
3 hours of 2000 doctors = 36,000 transactions

Doctors per hour in 24 hour clock:

1:00 - 500
2:00 - 500
3:00 - 500
4:00 - 500
5:00 - 500
6:00 - 500
7:00 - 1000
8:00 - 1000
9:00 – 1000
10:00 - 1000
11:00 - 1000
12:00 - 1000
13:00 - 1000
14:00 - 2000
15:00 - 2000
16:00 - 2000
17:00 - 1000
18:00 - 1000
19:00 - 1000
20:00 - 1000
21:00 - 1500
22:00 - 1500
23:00 - 1500
24:00 – 500
28 B B is correct. The highest concurrent usage is when 2000 PTFL- 1
doctors are working from 2:00-5:00 in the afternoon. 4.2.5
Each doctor has only one access per hour according to K4
the question and that access has up to 6 updates. A, C
and D are incorrect.

Version 2018 Page 7 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
29 C C is correct. This is a scalable method because it PTFL- 1
bypasses the UI so you can run many of these tests at 4.2.6
the same time without requiring the client. A is not K2
correct because this scripting can be quite difficult
without using a tool to capture the protocol level
communication. B is not correct because the UI is
bypassed. D is not correct because data correlation is
more difficult because the correlation done at the UI is
bypassed.
30 D D is correct. Scripts should contain their own checks PTFL- 1
that the process has worked and verifying via the 4.2.6
database directly is a more efficient way to do it (and K2
more maintainable) than having the script do the
verification through the application (which may also be
wrong) as this adds to the maintenance requirements of
the script. A is not correct because no errors doesn’t
mean that it worked – it could just mean that the error
handling is also broken. B is not correct because this
could be a very time-consuming approach. C is not
correct because the application may have a defect that
causes it to not find the new users.
31 A A is correct. The problem is most likely with the system PTFL- 1
identifier that is given to that particular user when they 4.2.7
login. When they try to do another transaction, that K3
unique identifier needs to be passed to the system so it
can provide access to the transaction. B is not correct
because there is no indication that this is the problem.
C is not correct because the question indicates that the
doctor performs multiple lookups per access. D is not
correct because if this were happening, it would log in
as the previous user, not error out on the login.
32 C C is correct. This is a symptom of caching and can be PTFL- 1
avoided by either varying the data or by clearing the 4.2.7
cache. A is not correct and is highly unlikely as systems K3
tend to get slower, not faster, unless there is a caching
issue. B is not correct because it’s not the user’s
information that is being re-used but rather the search
results. D is not correct for the same reason.

Version 2018 Page 8 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
33 A A is correct. The less like production the performance PTFL- 1
testing system is, the less reliable are the results and 4.2.8
this increases the risk of making incorrect assumptions. K2
B is not correct. It could happen, but not necessarily. C
is not correct because there is no reason to think the
results would be easier to understand with a different
configuration. D is not correct because the project risk
would actually be increased in this case.
34 B B is correct. A load generator, particularly if you are PTFL- 1
running multiples on one machine, may slow down 4.2.8
enough that it is no longer maintaining the desired load K2
on the system. A and C will not occur if the tool is
configured properly. D is not an issue as the load
generator is not usually used to generate the
performance results – it just creates the load on the
system so the performance tests can be conducted.
35 B B is correct. The system must have started up and PTFL- 1
achieved a steady state for valid steady state 4.3.1
performance metrics to be gathered. A is not correct K2
because the goal of the ramp up to is to get the number
of users to the point desired for the test, not to exceed
that number. C is not correct because ramp up is
increasing load, not decreasing it getting ready for shut
down. D is not correct because this has nothing to do
with ramp up and should be tested before any official
tests are started.
36 C C is correct. This is achieved with spike tests which PTFL- 1
force the system to transition between a low load to a 4.3.1
high load in a short period of time. K2
37 C A is correct but this is also a problem with the PTFL- 1
requirements being unclear. Does it have to be less 4.4.1
than 3 seconds 100% of the time? Or is it acceptable if K4
it’s 95%? As stated though, the system has failed the
test and the requirements should be revisited.
C might be a good result if the requirements had been
better defined. Right now, they say that 100% must be
less than 3 seconds. Realistically though, the 90% time
should be investigated to see if there are a few outliers
that are bringing it down or if it is always at the 5 second
mark. More tuning should be attempted though because
this is a critical system and performance could be life or

Version 2018 Page 9 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version
 Foundation Level Performance Testing – Sample Exam

Answer Explanation / Rationale Learning # of

Question Objective Points
(LO)
death impacting. B is not correct because the goal didn’t
stay 85% of the time was acceptable. D is not correct
because tuning to get to below 3 seconds 100% of the
time will probably not be possible.

38 D D is correct. This is what a load generator tool does. A PTFL- 1

is a network analysis tool. B might be a browser testing 5.1.1
tool or might not exist. C is a monitoring tool. K2
39 A A is correct. This is the description of a pay-as-you-go PTFL- 1
tool. These are often cloud-based tools but not all cloud- 5.1.1
based tools are pay-as-you-go. B, C and D are not K2
correct as these are not descriptions of pay-as-you-go
tools.
40 C C is correct. At this point you know that your testers are PTFL- 1
able to code what is needed (which eliminates B and D). 5.2.1
The project schedule is not really a factor because any K4
tool will take time to acquire and integrate, but the
developers can code in it, so that’s the main schedule
factor. What you need to know is can this tool simulate
the necessary number of virtual users and, of course,
how much will it cost.

Version 2018 Page 10 of 10 9 December 2018

© International Software Testing Qualifications Board ISTQB public release version