CISA® Official Review Manual 28th Edition | Chapter 3

Figure 3.23—Data File Controls

Method Description
Before and after Computer data in a file prior to and after a transaction is processed can be recorded and reported.
image reporting The before and after images make it possible to trace the impact that transactions hâve on computer
records.
Maintenance Control procedures should be in place to ensure that ail error reports are properly reconciled and
error reporting corrections are submitted on a timely basis. To ensure séparation of duties (SoD), error corrections
and handling should be reviewed properly and authorized by personnel who did not initiate the transaction.
Source Source documentation should be retained for an adéquate time period to enable retrieval,
documentation reconstruction or vérification of data. Policies regarding the rétention of source documentation should
rétention be enforced. Originating departments should maintain copies of source documentation and ensure that
only authorized personnel hâve access. When appropriate, source documentation should be destroyed
in a secure, controlled environment.
Internai and Internai and external labeling of removable storage media is impérative to ensure that the proper data
external labeling are loaded for Processing. External labels provide the basic level of assurance that the correct data
medium is loaded for processing. Internai labels, including file header records, provide assurance that
the proper data files are used and allow for automated checking.
Version usage For processing to be correct, it is critical that the proper version of a file and the correct file are used.
For example, transactions should be applied to the most current database, while restart procedures
should use earlier versions.
Data file security Data file security Controls prevent unauthorized access by unauthorized users that may hâve access to
the application to alter data files. These Controls do not provide assurances relating to the validity of
data but ensure that unauthorized users who may hâve access to the application cannot alter stored
data improperly.
One-for-one Individual documents agréé with a detailed listing of documents processed by the computer. It is
checking necessary to ensure that ail documents hâve been received for processing.
Prerecorded Certain information fields are preprinted on blank input forms to reduce initial input errors.
input
Transaction logs Ail transaction input activity is recorded by the computer. A detailed listing, including date of input, time
of input, user ID and terminal location, can then be generated to provide an audit trail. It also permits
operations personnel to détermine which transactions hâve been posted. This will help to decrease the
research time needed to investigate exceptions and decrease recovery time if a System failure occurs.
File updating Proper authorization for file updating and maintenance is necessary to ensure that stored data are
and correct, up to date and safeguarded adequately. Application programs may contain access restrictions
maintenance in addition to the overall System access restrictions. The additional security may provide levels of
authorization authorization and an audit trail of file maintenance.
Parity checking Data transfers in a computer System are expected to be made in a relatively error-free environment.
However, when programs or vital data are transmitted, additional Controls are needed. Transmission
errors are controlled primarily by error-detecting or correcting codes. The former is used more often
because error-correcting codes are costly to implement and are unable to correct ail errors. Generally,
error détection methods such as a check bit and redundant transmission are adéquate. Redundancy
checking is a common error-detection routine. A transmitted block of data containing one or more
records or messages is checked for the number of characters or patterns of bits contained in it. If the
numbers or patterns do not conform to predetermined parameters, the receiving device ignores the
transmitted data and instructs the user to retransmit. Check bits are often added to the transmitted
data by the télécommunications control unit and may be applied either horizontally or vertically.
These checks are similar to the parity checks normally applied to data characters within on-premises
equipment. A parity check on a single character generally is referred to as a vertical or column check,
and a parity check on ail the équivalent bits is known as a horizontal, longitudinal or row check. Use of
both checks greatly improves the possibilities of detecting a transmission error, which may be missed
when either of those checks is used alone.

©ISACA. AH Rights Reserved. | 221

 CISA® Official Review Manual 28th Edition | Chapter 3

The Controls built into an application represent the Even with the most reliable and accurate data sources,
management design of Controls on how a business improperly configured, constructed and prepared
process should be run. While an application contains reports are still a significant risk. Report design
the rules for the business, the data that are the outcome and génération spécifications, templates and création/
of the processing are stored in the database. An entity change request processes are critical System output
may hâve the best Controls built into the application, Controls.
but if management personnel directly update data in the • Reports generated from the System—These
database, then the benefit of the best Controls in the represent the data that management relies on for
application will be overridden. business decisions and review of business results.
Therefore, ensuring the integrity of data in reports is
However, in some situations, enterprises may hâve to
key for the reliability of information in information
carry out direct updates to a database. For example,
Systems. An IS auditor should validate that the reports
if, due to a Systems outage, transactions cannot be
are accurate and provide correct représentation of the
processed in real time, it is not practical to insist that
source data.
the backlog should be entered through the application
(front end) when the System becomes available, before IS auditors need to apply an assessment approach
the transactions of the subséquent days are entered or in validating reports depending on the situation
processed. In such cases, management may décidé to (more évaluation is required when the organization
directly update the backlog transactions in the database has undergone a System change or when evaluating
(back end). customized reports against standard reports of a widely
used application). Methods to validate reports include the
An IS auditor should ensure that there are Controls in
following:
place to ensure that such direct back-end data fixes
• Report distribution—Output reports should be
are supported by authorization of the business for
distributed according to authorized distribution
completeness and accuracy and are processed subject
parameters. Operations personnel should verify that
to computer operations Controls. The important point
reports are complété and delivered according to
to remember is that, in any enterprise, the quality of
schedule. Ail reports should be logged prior to
application Controls is only as good as the quality of
distribution. In most environments, processing output
Controls around direct back-end data fixes.
is spooled to a buffer or print spool on completion of
job processing, where it waits for an available printer.
3.4.2 Output Controls
Controls over access to the print spools are important
Output Controls provide assurance that the data delivered to prevent reports from being deleted accidentaily
to users will be presented, formatted and delivered in a from print spools or directed to a different printer.
consistent and secure manner. In addition, changes to the output print priority can
delay printing of critical jobs. Access to distributed
Output Controls include:
reports can compromise confidentiality; therefore,
• Logging and storage of negotiable, sensitive and
physical distribution of reports should be controlled
critical forms in a secure place—Negotiable,
adequately. Reports containing sensitive data should
sensitive or critical forms should be properly logged
be printed under secure, controlled conditions. Secure
and secured to provide adéquate safeguards against
output drop-off points should be established. Output
theft, damage or disclosure. The form log should be
disposai should also be secured to ensure that no
routinely reconciled to hâve inventory on hand and
unauthorized access can occur. Electronic distribution
any discrepancies should be properly researched.
should also be considered, and logical access Controls
• Computer génération of negotiable instruments,
should be put in place.
forms and signatures—The computer génération
• Balancing and reconciling—Data processing
of negotiable instruments, forms and signatures
application program output should be balanced
should be properly controlled. A detailed listing of routinely to the control totals. Audit trails should
generated forms should be compared to the physical
be provided to facilitate the tracking of transaction
forms received. One should properly account for ail
processing and the réconciliation of data.
exceptions, rejections and mutilations. • Output error handling—Procedures for reporting
• Report accuracy, completeness and timeliness—
and controlling errors contained in the application
Often reports are generated using third-party data program output should be established. The error
analysis and reporting applications (e.g., Essbase).
report should be timely and delivered to the

222 | ©ISACA. Ail Rights Reserved.

 CISAC Official Review Manual 28th Edition | Chapter 3

originating department for review and error

correction.
• Output report rétention—A record rétention
schedule should be adhered to firmly. Any governing
legal régulations should be included in the rétention
policy.
• Vérification of receipt of reports—To provide
assurance that sensitive reports are properly
distributed, the récipient should sign a log as evidence
of receipt of output.
An IS auditor should be aware of existing concerns
regarding record-retention policies for the enterprise and
address legal requirements. Output can be restricted to
particular IT resources or devices (e.g., a particular
printer).

©ISACA. AH Rights Reserved. | 223

 CISA® Official Review Manual 28th Edition

Page intentionally left blank

224 | ©ISACA. AH Rights Reserved.

 CISA® Official Review Manual 28th Edition | Chapter 3

Part B: Information Systems 3.5.1 Testing Classifications

Implémentation The following tests relate, to varying degrees, to the
approaches that can be performed based on the size and
IS implémentation is when the System is installed and complexity of the modified System:
moved into the production environment after appropriate • Unit testing—The testing of an individual program or
System and users’ acceptance testing. This is the stage at module. Unit testing uses a set of test cases that focus
which: on the control structure of the procédural design.
• End users are notified and trained. These tests ensure that the internai operation of the
• System testing occurs. program performs according to spécification.
• Data entry or conversions occur. • Interface or intégration testing—A hardware or
• Postimplementation reviews occur. software test that évaluâtes the connection of two
or more components that pass information from one
3.5 System Readiness and area to another. The objective is to take unit-tested
Implémentation Testing modules and build an integrated structure dictated by
design. The term intégration testing is also used to
Intégral to IS implémentation is the proper sélection of refer to tests that verify and validate the functioning
testing méthodologies, the development of testing plans of the application under test with other Systems, in
fully traceable to requirements and the acquisition of which a set of data is transferred from one System to
essential resources to successfully complété testing. Once another.
completed, testing provides confidence to stakeholders • System testing—A sériés of tests designed to ensure
that a System or System component opérâtes as intended that modified programs, objects, database schéma,
and delivers the benefits realization as required at the etc., which collectively constitute a new or modified
start of a project. System, function properly. These test procedures are
often performed in a nonproduction test/development
An IS auditor should understand the application
environment by software developers designated as a
of varions forms of testing. An IS auditor should
test team. The following spécifie analyses may be
also understand how QA monitoring and évaluation
carried out during System testing:
contribute to the quality of an organization’s
■ Recovery testing—Checking the system’s ability
internai processes (e.g., project management, software
to recover after a software or hardware failure
development process or IT service) and the quality of
■ Security testing—Making sure the modified/new
the final products produced by these processes (e.g., the
System includes provisions for appropriate access
System implemented or software developed).
Controls and does not introduce any security holes
Testing is an essential part of the Systems development that might compromise other Systems
process that vérifiés and validâtes that a program, n Load testing—Testing an application with large
subsystem or application performs the functions for quantities of data to evaluate its performance
which it has been designed. Testing also détermines during peak hours
whether the units being tested operate without any ■ Volume testing—Studying the impact on the
malfunction or adverse effect on other components of the application by testing with an incrémental volume
System. of records to détermine the maximum volume of
records (data) that the application can process
The variety of Systems development méthodologies and
■ Stress testing—Studying the impact on the
organizational requirements provide for a large range of
application by testing, with an incrémental number
testing schemes or levels. Each set of tests is performed
of concurrent users/services on the application,
with a different set of data and under the responsibility
to détermine the maximum number of concurrent
of different people or functions. An IS auditor plays a
users/services that the application can process
préventive or détective rôle in the testing process.
■ Performance testing—Comparing the System
performance to other équivalent Systems using
well-defined benchmarks
• Final acceptance testing—Performed after the
System staff is satisfied with the System tests.
Acceptance testing occurs during the implémentation
phase. During this testing phase, the defined methods

©ISACA. AH Rights Reserved. | 225

 CISA0 Official Review Manual 28th Edition | Chapter 3

of testing to apply should be incorporated into the Although acquired Systems are tested by the vendor
enterprise’s QA methodology. QA activities should prior to distribution, these Systems and any subséquent
proactively encourage adéquate levels of testing to changes should be tested thoroughly by the end user and
be performed on ail software development projects. the System maintenance staff. These supplémentai tests
Final acceptance testing has two major parts: quality help ensure that programs function as designed by the
assurance testing (QAT), focusing on technical vendor and the changes do not interact adversely with
aspects of the application, and UAT, focusing on existing Systems. In the case of acquired software, after
functional aspects of the application. QAT and UAT attending to the changes during testing by the vendor,
hâve different objectives and, therefore, should not be the accepted version should be controlled and used for
combined. implémentation. In the absence of Controls, the risk of
introducing malicious patches/Trojan horse programs is
QAT focuses on the documented spécifications and the
very high.
technology employed. It vérifiés that the application
works as documented by testing the logical ‘design and Some enterprises rely on integrated test facilities
the technology itself. It also ensures that the application (ITFs). Test data usually are processed in production-
meets the documented technical spécifications and like Systems. This confirms the behavior of the new
deliverables. QAT is performed primarily by the IT application or modules in real-life conditions. These
départaient. The participation of the end user is minimal conditions include peak volume and other resource-
and on request. QAT does not focus on functionality related constraints. In this environment, IS performs tests
testing. with a set of fictitious data whereas client représentatives
use extracts of production data to cover the most possible
UAT supports the process of ensuring that the System
scénarios and some fictitious data for scénarios that
is production ready and satisfies ail documented
would not be tested by the production data. In some
requirements. The methods include:
enterprises that use a subset of production data in a test
• Définition of test strategies and procedures
environment, such production data may be altered to
• Design of test cases and scénarios
scramble the data so that the confidential nature of the
• Execution of the tests
data is obscured from the tester. This is often the case
• Use of the results to verify System readiness
when the acceptance testing is done by team members
Acceptance criteria are defined éléments that a who, under usual circumstances, would not hâve access
deliverable must meet to satisfy the predefined needs to such production data.
of the user. A UAT plan must be documented for the
After acceptance testing is complété, certification and
final test of the completed System. The tests are written
accréditation processes are performed. These should
from a user’s perspective and should use the System in a
be done after the System has been implemented and
manner as close to production as possible. For example,
in operation for enough time to produce the evidence
tests may be based around typical, predefined business
needed for these processes. This includes evaluating
process scénarios. If a new business process has been
program documentation and testing effectiveness and
developed to accommodate the new or modified System,
results in a final decision for deploying the business
it should also be tested at this point. A key aspect of
application System. For information security issues, the
testing should also include testers seeking to verify that
évaluation process includes reviewing security plans, the
supporting processes integrate into the application in
risk assessments performed and test plans, and results in
an acceptable fashion. Successful completion generally
an assessment of the effectiveness of the security Controls
enables a project team to hand over a complété integrated
and processes to be deployed. Generally involving
package of application and supporting procedures.
security staff and the business owner of the application,
Ideally, UAT should be performed in a secure testing this process provides some degree of accountability to
or staging environment. A secure testing environment, the business owner regarding the State of the System
in which both source code and exécutable code are that they will accept for deployment. See section 3.7.5
protected, helps to ensure that unauthorized or last- Certification/Accreditation for more information.
minute changes are not made to the System without going
When the tests are completed, an IS auditor should issue
through the standard System maintenance process. The
an opinion to management as to whether the System
nature and extent of the tests will be dépendent on the
meets the business requirements, has implemented
magnitude and complexity of the System change.
appropriate Controls and is ready to be migrated to
production. This report should specify the deficiencies

226 | ©ISACA. AH Rights Reserved.

 CISA* Official Review Manual 28th Edition | Chapter 3

in the System that need to be corrected and should and comparing the results. The purpose of parallel
identify and explain the risk that the enterprise is taking testing is to détermine whether the new application
by implementing the new System. See section 5.12.6 performs in the same way as the original System and
Security Testing Techniques for more information. meets end-user requirements.
• Sociability testing—Tests to confirm that the new
Other Types of Testing or modified System can operate in its target
Other types of testing include the following: environment without adversely impacting existing
• Alpha and beta testing—The two phases of testing Systems. This should cover the platform that will
that software undergoes before being considered perform primary application processing and interfaces
finished. The first stage, called alpha testing, is often with other Systems and, in a client server or web
performed only by users within the enterprise who development, those that perform changes to the
are developing the software (i.e., Systems testing). desktop environment.
The second stage, called beta testing, a form of UAT,
generally involves a limited number of external users.
3.5.2 Software Testing
This involves real-world exposure, sending the beta Test plans identify the spécifie portions of the System to
version of the product to independent test sites or be tested and may include a categorization of types of
offering it free to interested users. deficiencies that can be found during the test. Categories
• Pilot testing—A preliminary test that focuses on of such deficiencies may be System defects, incomplète
spécifie and predetermined aspects of a System. It requirements, designs, spécifications or errors in the
is not meant to replace other testing methods, but to test case itself. Test plans also specify severity levels
provide a limited évaluation of the System. POCs are of problems found and guidelines on identifying the
early pilot tests—usually over intérim platforms and business priority. The tester détermines the severity of
with only basic functionalities. the problem found during testing. Based on the severity
• White box testing—A test that assesses the level, the problem may be fixed prior to implémentation
effectiveness of software program logic. Specifically, or may be noted for correction following implémentation.
test data are used in determining procédural accuracy The project sponsor, end-user management and the
or conditions of a program’s spécifie logic paths (i.e., project manager décidé early in the test phase on the
applicable to unit and intégration testing). However, severity définitions.
testing ail possible logic paths in large information
Systems is not feasible and would be cost prohibitive; Test plans also identify test approaches, such as the
therefore, white box testing is used on a select basis following two reciprocal approaches, to software testing:
only. • Bottom up—Testing begins with atomic units, such
• Black box testing—An integrity-based form as programs or modules, and works upward until
of testing associated with testing components a complété System testing has taken place. The
of an information system’s functional operating advantages include:
effectiveness without regard to any spécifie internai ■ There is no need for stubs or drivers.
program structure. It is applicable to intégration ■ Testing can be started before ail programs are
(interface) and UAT processes. complété.
• Function/validation testing—Similar to System ■ Errors in critical modules are found early.
• Top down—Testing follows the opposite path, either
testing but often used to test the functionality of the
System against the detailed requirements to ensure in depth-first or breadth-first search order. The
that the software that has been built is traceable to advantages include:
■ Tests of major functions and processing are
customer requirements (i.e., Are we building the right
conducted early.
product?).
■ Interface errors can be detected sooner.
• Régression testing—The process of rerunning a
■ Confidence in the System is increased because
portion of a test scénario or test plan to ensure
programmers and users see a working System.
that changes or corrections hâve not introduced new
errors. The data used in régression testing should be Generally, most application testing of large Systems
the same as the data used in the original. follows a bottom-up testing approach that involves
• Parallel testing—The process of feeding test data
into two Systems—the modified System and an
alternative System (possibly the original System)—

©ISACA. Ail Rights Reserved. | 227

 CISA° Official Review Manual 28th Edition | Chapter 3

ascending levels of intégration and testing (e.g., unit or insertions, délétions and updates to these relations.
program, subsystem/integration, System): Database software generally provides varions built-
• Conduct and report test results—Describe in automated procedures for checking and ensuring
resources implied in testing, including personnel referential integrity. Referential integrity checks
involved and information resources/facilities used involve ensuring that ail references to a primary
during the test and actual versus expected test results. key from another table (i.e., a foreign key) exist
Results reported and the test plan should be retained in their original table. In nonpointer databases
as part of the System’s permanent documentation. (e.g., relational), referential integrity checks involve
• Address outstanding issues—Identify errors and making sure that ail foreign keys exist in their original
irregularities from the actual tests conducted. When table.
such problems occur, the spécifie tests in question
must be redesigned in the test plan until acceptable Data Integrity in Online Transaction Processing
conditions occur when the tests are redone. Systems
In multiuser transaction Systems, it is necessary to
3.5.3 Data Integrity Testing manage parallel user access to stored data typically
Data integrity testing is a set of substantive tests controlled by a DBMS and deliver fault tolérance.
that examines accuracy, completeness, consistency and Of particular importance are four online data integrity
authorization of data presently held in a System. requirements known collectively as the ACID principle:
It employs testing similar to that used for input • Atomicity—From a user perspective, a transaction
control. Data integrity tests indicate failures in input or is either completed in its entirety (i.e., ail relevant
Processing Controls. Controls for ensuring the integrity of database tables are updated) or not at ail. If an error or
accumulated data in a file can be exercised by regularly interruption occurs, ail changes made up to that point
checking data in the file. When this checking is done are backed out.
against authorized source documentation, it is common • Consistency—Ail integrity conditions in the database
to check only a portion of the file at a time. Because are maintained with each transaction, taking the
the whole file is regularly checked in cycles, the control database from one consistent State into another
technique is often referred to as cyclical checking. consistent State.
• Isolation—Each transaction is isolated from other
Two common types of data integrity tests are relational transactions, so each transaction accesses only data
and referential integrity tests: that are part of a consistent database State.
• Relational integrity tests—Performed at the data • Durability—If a transaction has been reported back
element and record-based levels. Relational integrity to a user as complété, the resulting changes to the
is enforced through data validation routines built into database survive subséquent hardware or software
the application or by defining the input condition failures.
constraints and data characteristics at the table
définition in the database stage. Sometimes it is a 3.5.4 Application Systems Testing
combination of both.
• Referential integrity tests—Define existence Testing the effectiveness of application Controls involves
relationships between entities in different tables of analyzing computer application programs, testing
a database that needs to be maintained by the computer application program Controls or selecting and
DBMS. It is required for maintaining interrelation monitoring data process transactions. Testing Controls
integrity in the relational data model. Whenever by applying appropriate audit procedures is important to
two or more relations are related through referential ensure their functionality and effectiveness. Methods and
constraints (primary and foreign key), it is necessary techniques for each category are described in figure 3.24.
that references be kept consistent in the event of

228 | ©ISACA. Ail Rights Reserved.

 Cl SA* Official Review Manual 28th Edition | Chapter 3

Figure 3.24—Testing Application Systems

Technique Description Advantages Disadvantages

Snapshot Records flow of designated Vérifiés program logic Requires extensive knowledge
transactions through logic paths of the information Systems (IS)
within programs environment
Mapping Identifies spécifie program logic • Increases efficiency by Cost of software
that has not been tested identifying unused code
and analyzes programs during • Identifies potential exposures
execution to indicate whether
program statements hâve been
executed
Tracing and • Tracing shows thetrail of Provides an exact picture of Requires extensive amounts of
tagging instructions executed during sequence of events, and is computer time, an intimate
an application. effective with live and simulated knowledge of the application
• Tagging involves placing transactions program and additional
an indicator on selected programming to execute trace
transactions at input and routines
using tracing to track them.
Test data/deck Simulâtes transactions through • Copies of the actual master • Difficult to ensure that the
real programs files or dummy data may be proper program is checked
used as test data • Risk of not including ail
• Source code review is transaction scénarios
unnecessary • Requires good knowledge of
• Can be used on a surprise application Systems
basis • Does not test the actual
• Provides objective review master file and master file
and vérification of program records
Controls and edits
• Initial use can be limited to
spécifie program functions
minimizing scope and
complexity
• Requires minimal knowledge
of the IS environment
Base-case • Uses test data sets Comprehensive testing • Extensive effort to maintain
System developed as part of a vérification and compliance data sets
évaluation comprehensive testing of testing • Close coopération required
programs among ail parties
• Vérifiés correct System
operations before
acceptance and periodic
revalidation
Parallel Processes production data Vérifiés new System before Added Processing costs
operation through existing and newly discontinuing the old one
developed programs at the same
time, compares results and
vérifiés changed production prior
to replacing existing procedures
Integrated Créâtes a fictitious file in the Periodic testing does not require • Needs careful planning
testing facility database with test transactions separate test process. • Test data must be isolated
processed simultaneously with from production data
live data

©ISACA. AH Rights Reserved. | 229

 CISA® Official Review Manual 28th Edition | Chapter 3

Figure 3.24—Testing Application Systems (cont.)

Technique Description Advantages Disadvantages

Parallel Processes production data Eliminâtes need to préparé test Programs must be developed
simulation using computer programs that data
simulate application program
logic
Transaction Use audit software to screen and • Independent of production Cost of development and
sélection select transactions input to the System maintenance
programs regular production cycle • Controlled by the auditor
• Requires no modification to
production Systems
Embedded audit Software embedded in host Provides sampling and • High cost of development
data collection computer applications screens. productions statistics and maintenance
It selects input transactions • Auditor independence issues
and generates transactions
during production. Usually, it is
developed as part of System
development. Types include:
• Systems control audit review
file (SCARF) - Auditor
détermines reasonableness
of tests incorporated into
normal processing. It
provides information for
further review.
• Sample audit review file
(SARF) — Randomly selects
transactions to provide
représentative file for
analysis
Extended Gathers ail data that hâve been Records are put into one Adds to data storage costs
records affected by a particular program convenient file. and overhead, and to System
development costs

To facilitate the évaluation of application System tests, IS Auditor’s Rôle in Information Systems Testing
an IS auditor may want to use generalized audit
Testing is crucial in determining that user requirements
software (GAS). This is particularly useful when spécifie
hâve been validated, the System is performing as
application control weaknesses are discovered (e.g., ones
anticipated and internai Controls work as intended.
that affect updates to master file records and certain error
Therefore, it is essential that an IS auditor be involved
conditions on spécifie transaction records). Additionally,
in reviewing this phase and perform the following:
GAS can be used to perform certain application control
• Review the test plan for completeness; indicate
tests, such as parallel simulation, in comparing expected
evidence of user participation, such as user
outcomes to live data.
development of test scénarios and/or user sign-off of
Automated Application Testing results; and consider rerunning critical tests.
• Reconcile control totals and converted data.
Test data generators can be used to systematically • Review error reports for their précision in recognizing
generate random data that can be used to test programs. erroneous data and resolution of errors.
The generators work by using the field characteristics, • Verify cyclical processing for correctness (month-end,
layout and values of the data. In addition to test data year-end processing, etc.).
generators, there are interactive debugging aids and code • Verify accuracy of critical reports and output used by
logic analyzers available to assist in the testing activities. management and other stakeholders.

230 | ©ISACA. Ail Rights Reserved.

 CISA0 Official Review Manual 28th Edition | Chapter 3

• Interview end users of the System for their The objective of such a project is to develop and
understanding of new methods, procedures and establish the support structure that will exist for the
operating instructions. new technical infrastructure. The main goals are to
• Review System and end-user documentation to accomplish the following:
détermine its completeness and verify its accuracy • Provide appropriate support structures for first-,
during the test phase. second- and third-line support teams.
• Review parallel testing results for accuracy. • Provide a single point of contact.
• Verify that System security is functioning as designed • Provide rôles and skills définitions with applicable
by developing and executing access tests. training plans.
• Review unit and System test plans to détermine
Often the project sponsor’s organization opérâtes and
whether tests for internai Controls are planned and
supports a legacy solution and will implement a new
performed.
System environment based on new System architecture.
• Review the UAT and ensure that the accepted
The existing support procedures and the organizational
software has been delivered to the implémentation
units will hâve to maintain the future System to provide
team. The vendor should not be able to replace this
the appropriate level of support for the new platform and
version.
for the old one.
• Review procedures used for recording and following
through on error reports. To achieve significant success in updating staff on
changes to the business process and introducing new
3.5.5 System Implémentation software, it is necessary to address some important
questions, such as:
Implémentation is initiated only after a successful testing
• How can the existing support staff be involved in
phase. The System should be installed according to the
the setup of the new project without neglecting the
enterprise’s change control procedures.
currently running System?
An IS auditor should verify that appropriate signoffs • What is the gap of knowledge/skills that must be
hâve been obtained prior to implémentation and perform addressed in the training plan?
the following: • How large is the différence between the current
• Review the programmed procedures used for legacy environment operation and the operation of the
scheduling and running the System and System new platform?
parameters used in executing the production schedule.
Generally, a transition project should conform to the
• Review ail System documentation to ensure its
following guidelines:
completeness and confirm that ail recent updates from
• There should be a smooth transition from the existing
the testing phase hâve been incorporated.
platform to the new platform, without any négative
• Verify ail data conversions to ensure that they are
effect on users of the System.
correct and complété before implementing the System
• There should be maximum employment of the
in production.
existing support staff to operate the new System
environment and keep the effort of new hires at a
Implémentation Planning
minimum level.
After it is developed and ready for operation, the new
System delivered by the project will need an efficient A primary challenge is to manage the phases from build
support structure. It is not enough to set up rôles for a to integrate, to migrate and for the phasing-out of the
support structure and name people to fulfil 1 these rôles. existing System and the phasing-in of the new one. The
Support personnel will need to acquire new skills. The migration cannot be accomplished via a single event.
workload has to be distributed such that the right people Instead, a step-by-step transition of the affected services
support the right issues; thus, new processes hâve to must take place. Further, the implemented processes for
be developed while respecting the specificities of IT a legacy environment might be different from what may
department requirements. Additionally, an infrastructure be implemented with the new platform and any changes
dedicated to support staff has to be made available. For must be communicated to users and System support staff.
these and other reasons, setting up a support structure
normally is a project in itself and requires planning, a
methodology and good practices adaptation from past
expériences.

©ISACA. AH Rights Reserved. | 231

 CISA° Official Review Manual 28th Edition | Chapter 3

Implémentation Plan/Knowledge Trans fer Plan These processes provide systematic, consistent and
unambiguous control on attributes of IT components
In accordance with good practices, the transfer
comprising the System (hardware, software, firmware
should follow the shadowing and relay-baton method.
and network connectivity, including physical connecting
Shadowing gives staff the opportunity to become
media wire, fiber and radio frequency). Knowledge of
accustomed to the System by observation. The relay-
the configuration status of computing environments is
baton approach is the most suitable concept to transfer
critical to System reliability, availability and security and
knowledge and responsibility in a transparent way. The
achieving timely maintenance of these Systems. Changes
metaphor of the relay-baton expresses exactly what must
to IT Systems must be carefully assessed, planned, tested,
be achieved (i.e., knowledge is transferred in small
approved, documented and communicated to minimize
portions).
any undesirable conséquences to the business processes.
TrainingPlan An IS auditor should be aware of the tools availabié for
After the rôles and responsibilities are defmed, they will managing configuration, change and release management
be documented in the form of a chart to allow for a clear and of the Controls in place to ensure SoD between
and easy-to-read overview. development staff and the production environment.
Tools for managing configuration, change and release
For example, a staff training plan should show ail of the management include ManageEngine Service Desk,
required training in tenus of: SysAid and ServiceNow.
• Content
• Scheduling information 3.6.1 Configuration Management
• Duration Systems
• Delivery mechanism (classroom and/or web-based)
• Train-the-trainer concept Because of the difficultés associated with exercising
control over System and programming maintenance
The plan should consider the rôle définitions and skill
activities, more enterprises are implementing
profiles for the new to-be structure and the results of
configuration management Systems. In many cases,
the gap analysis. The plan considers that the staff who
regulatory requirements mandate these levels of control
need to be trained must still run the current System, so
to provide a high degree of reliability and repeatability
that detailed coordination with the daily business tasks is
in ail associated System processes. In a configuration
maintained.
management System, maintenance requests must be
The following list gives an example of work tasks formally documented and approved by a change control
defmed to fulfill the overall project goal: group (e.g., configuration control boards). In addition,
• Collate existing support structure documentation. careful control is exercised over each stage of the
• Review the existing IT organization model. maintenance process via checkpoints, reviews and sign-
• Defme the new support organization structure. off procedures.
• Defme the new support processes.
Configuration management involves procedures
• Map the new process to the organization model.
throughout the System hardware and software life cycle
• Exécuté the new organization model.
(from requirements analysis to maintenance) to identify,
• Establish support functions.
defme and baseline software items in the System
• Develop communications material for support staff.
and provide a basis for problem management, change
• Conduct briefing and training sessions.
management and release management. This process is
• Review mobilization progress.
usually facilitated with a configuration management
• Transfer to the new organization structure.
database (CMDB), which documents information on the
• Review the preceding items.
hardware and software assets within the organization.

3.6 Implémentation Configuration and The process of checking out prevents or manages
simultaneous code edits, with hardware, network and
Release Management
System architects reviewing and approving the changes
The effective and efficient development and maintenance or updates to the hardware asset and inventory tracking
of complicated IT Systems requires that rigorous Systems.
configuration, change and release management processes
Checking in is the process of moving an item to the
be implemented and adhered to within an enterprise.
controlled environment. When a change is required (and

232 | ©ISACA. Ail Rights Reserved.

 CISAC Official Review Manual 28th Edition | Chapter 3

supported by a change control form), the configuration evidence of management’s commitment to careful control
manager checks out the item. After the change is made, over the maintenance process.
it can be checked using a different version number.
The process of checking out also prevents or manages 3.7 System Migration, Infrastructure
simultaneous code edits. With hardware, network and Deployment and Data Conversion
System architects review and approve the changes or
updates to both the hardware asset and the inventory New software applications tend to be more
tracking Systems. comprehensive and integrated than older applications.
Furthermore, enterprises rely increasingly on data
For configuration management to work, management
warehouses, models and simulation for decision making;
support is critical. The configuration management
thus, importing data from old (and legacy) Systems into
process is implemented by developing and following
the new application is crucial. Data format, coding,
a configuration management plan and operating
structure and integrity are to be preserved or properly
procedures. This plan should not be limited to just the
translated. A migration scénario must be set up and
software developed but should also include ail System
a rollback plan needs to be in place. There are many
documentation, test plans and procedures.
direct (old to new application) and indirect (using intérim
Commercial software products are often used to automate repositories) strategies and tools. Data conversion is
some processes. Such tools should allow control to be a one-time task in many development projects. The
maintained for applications software from the outset of importance of correct results is critical, and success
System analysis and design to running live. Configuration dépends on the use of good practices by the development
management tools support change management and team because the programmed input checks under
release management through the: development will not be available for the conversion.
1. Identification of items affected by a proposed Source data must be correctly characterized, and the
change to assist with impact assessment (functional, destination database must accommodate ail existing
operational and security) data values. Resulting data should be carefully tested.
2. Recording of configuration items affected by Steps for the conversion that are developed in the test
authorized changes environment must be recorded so they can be repeated on
3. Implémentation of changes in accordance with the production System.
authorization records
An IS auditor should ensure that any tools and techniques
4. Registering of configuration item changes when
selected for the process are adéquate and appropriate,
authorized changes and releases are implemented
data conversion achieves the necessary objectives
5. Recording of baselines that are related to releases
without data loss or corruption and any loss of data is
(with known conséquences) to which an enterprise
minimal and formally accepted by user management.
would revert if an implemented change fails
6. Preparing a release to avoid human errors and
3.7.1 Data Migration
resource costs
A new version of the System (or builds) should be A data conversion (also known as data porting) is
built only from the baselined items. The baseline required if the source and target Systems use different
becomes the trusted recovery source for these Systems field formats or sizes, file/database structures or coding
and applications. These baselines should be built or schemes. For example, a number may be stored as text,
based on industry recognized sources or benchmarks. floating point or binary-coded décimal.
For example, the Center for Internet Security (CIS) Conversions are often necessary when the source and
provides benchmarks that are configuration baselines target Systems are on different hardware and/or OS
and practices to configure a System securely. It is platforms, and different file or database structures (e.g.,
important to use baselines not only for recovery but also relational database, fiat files, or Virtual storage access
to ensure that Systems are implemented with the least method) are used.
amount of functionality necessary so as not to introduce
unnecessary vulnerabilities into the environment. The objective of data conversion is to convert existing
data into the new required format, coding and structure
From an IS audit perspective, effective use of while preserving the meaning and integrity of the data.
configuration management software provides important The data conversion process must provide some means,
such as audit trails and logs, to allow for the vérification

©ISACA. Ail Rights Reserved. | 233

 CISA® Official Review Manual 28th Edition | Chapter 3

of the accuracy and completeness of the converted data. • Conflicts and contention between legacy and migrated
This vérification of accuracy and completeness may be operations
performed through a combination of manual processes, • Data inconsistencies and loss of data integrity during
System utilities, vendor tools and one-time-use spécial the migration process
applications.
The data model and the new application model should
A large-scale data conversion can potentially become a be stored in an enterprise repository. Using a repository
project within a project because considérable analysis, allows a simulation of the migration scénario and
design and planning will be required. Among the steps traceability during the project. An enterprise repository
necessary for a successful data conversion are: enables an overview of the reengineering and data
• Determining which data should be converted using migration process (e.g., which modules and entities are
programs and which data, if any, should be converted in which stage, such as in service or already migrated).
manual ly These models will be modified in the course of the
• Performing any necessary data cleansing ahead of processes described in the following sections.
conversion
• Identifying the methods to be used to verify the Refîning the Migration Scénario
conversion, such as automated file comparisons, To détermine the scope of the implémentation project,
comparing record counts and control totals, module analysis should be undertaken to identify the
accounting balances and individual data items on a affected functional modules and data entities. The
sample basis plan for the implémentation project should be refined
• Establishing the parameters for a successful based on this information and an analysis of business
conversion (e.g., Is 100-percent consistency between requirements.
the old and new Systems necessary, or will some
différences within defined ranges be acceptable?) The next step is to develop a migration plan. This is a
• Scheduling the sequence of conversion tasks detailed listing of tasks for the production deployment
• Designing audit trail reports to document of a new System. Within this plan, decision points are
the conversion, including data mappings and defined to make go or no-go decisions. The following
transformations processes require decision points:
• Designing exception reports to record any items that • Support migration process—A support process
cannot be converted automatically to administer the enterprise repository must be
• Establishing responsibility for verifying and signing implemented. Because this repository should be used
off on individual conversion steps and accepting the after completion of the project to manage the software
overall conversion components of the new architecture, this process
• Developing and testing conversion programs, should be capable of supporting future development
including functionality and performance processes. The enterprise repository administration
• Performing one or more conversion dress rehearsals and report génération support the migration by
to familiarize personnel with the sequence of events supporting the reverse engineering of changes in the
and their rôles, and testing the conversion process end legacy architecture and facilitating the création of
to end with real data impact analysis reports.
• Controlling the outsourcing of the conversion process • Migration infrastructure—The project develops
with a proper agreement covering nondisclosure, data spécifications for the infrastructure of the migration
privacy, data destruction and other warranties project. This approach ensures consistency and
• Running the actual conversion with ail necessary increases confidence in the functionality of the
personnel onsite or able to be contacted fallback scénario. The migration project team
complétés a high-level analysis of the legacy and new
A successful data migration delivers the new System on data models to establish links between them that will
time, on budget and with the required quality. The data be refined later. The migration infrastructure is the
migration project should be carefully planned and use basis for specifying the following components:
appropriate méthodologies and tools to minimize the risk ■ Data redirector (temporary adapters)—Good
of: practices suggest the staged deployment of
• Disruption of routine operations applications to minimize the end-user impact
• Violation of the security and confidentiality of data of their implémentation and limit the risk by
having a fallback scénario with minimum impact.

234 | ©ISACA. AH Rights Reserved.

 CISA° Official Review Manual 28th Edition | Chapter 3

For this reason, an infrastructure component is The first component consists of:
needed to handle distributed data on different • Unload components to execute the unloading of the
platforms within distributed applications. The data from the new data structures
design of a data redirector on the new architecture • Transfer components for the data conversion
corresponds to service-oriented architectures and • Load components to execute the loading of the data
should cover features such as access to the not- into the legacy data structures
yet-migrated legacy data during run time, data
The second component consists of:
consistency due to the usage of standards such
• A log component to log the data modifications within
as Open Group for Unix Systems eXtended
the new data model during runtime within the service
Architecture (X/Open XA) interface and a
layer
homogeneous new architecture.
• Transfer components for the data conversion
■ Data conversion components—The need to
• Load components to execute the load of the data into
create an enterprise data model to eliminate
the legacy data structures
data redundancies and inconsistencies often
is identified. For this reason, infrastructure Another important considération is the new system’s
components to transform the legacy data model data structure. This can be determined by reading the
to the new data model must be provided. These software user guides, analyzing the ERDs, understanding
components can be described as follows: the relationships between data éléments and reviewing
• Unload components to copy the data (either as définitions of key terms (e.g., entity and record) in the
is or suitably modified to align with the data new System.
format of the target System) in legacy databases
Next, it is important to review the decisions on how
that hâve been identified for migration
business processes should be conducted in the new
• Transfer components to execute the data
System. Changes are identified, and the output of this
transfer from the legacy System to the new
exercise is a table of new data terminology against
System
current définitions of data éléments. In this step, the
• Load components to execute the load of the
project team identifies how current data are defined in
data into the new database
the new System. Following this step, a data cleanup
Software packages that support data migration, such as is completed to eliminate inconsistencies in the current
ERP and document management software, should be database, if possible, and duplications of data sets
acquired as soon as the software évaluation is done. The are discovered and resolved. The rules of conversion
data conversion plan should be based on the available are defined and documented, with the objective of
databases and migration tools provided by the selected ensuring that the business processes executed in the
vendor(s). new System yield results that maintain data integrity and
relationships.
The decision on which method to use for data conversion
has to be made as part of the implémentation project Data conversion rules are programmed by the software
and should be based on transaction volume and change development team. Data conversion scripts are created
degree of the data model. to convert the data from the old database to the new
database. These are tested on a discrète sélection of
Fallback (Rollback) Scénario data that are carefully selected to include ail cases.
Not ail new System deployments go as planned. To This process is referred to as program or unit testing.
mitigate the risk of downtime for mission-critical Following the sign-off of data conversion scripts by
Systems, good practices dictate that the tools and programmers, the scripts are run on a test copy of the
applications required to reverse the migration are production database. The values of data are verified by
available prior to attempting the production cutover. executing assessments including business process tests.
Some or ail of these tools and applications may need to Users and developers complété cycles of testing until
be developed as part of the project. conversion scripts are fine tuned. After testing has been
completed, the next step is to promote the converted
Components hâve to be delivered that can back out ail database to production.
changes and restore data to the original applications in
case of nonfunctioning new applications. Two types of The key points to be taken into considération in a data
components should be considered as part of a fallback conversion project are to ensure:
contingency plan. • Completeness of data conversion

©ISACA. Ail Rights Reserved. | 235

 CISA° Official Review Manual 28th Edition | Chapter 3

• Integrity of data
• Storage and security of data under conversion Figure 3.25—Parallel Changeover
• Consistency of data
• Continuity of data access
The last copy of the data before conversion from the old Module n

platform and the first copy of the data after conversion to Module n1 ।
the new platform should be maintained separately in the Module 2 1

archive for any future reference.

Module 1 ,

3.7.2 Changeover (Go-Live or Cutover) । i Module m

Techniques ] Module m-1

« Module 2
Changeover refers to an approach to shift users from
Module 1
using the application from the existing (old) System to
the replacement (new) System. This is appropriate only Rollout Schedule
after testing the new System with respect to its program
and relevant data. Changeover is sometimes called the
go-live technique because it enables the start of the
new System. This approach is also called the cutover Phased Changeover
technique because it helps in cutting out from the older In this approach, the older System is broken into
System and moving over to the newer System. deliverable modules. Initially, the first module of the
This technique can be achieved in three different ways. older System is phased out using the first module of
the newer System. Then, the second module of the older
Parallel Changeover System is phased out using the second module of the
newer System, and so forth until reaching the last module.
This technique includes running the old System, then
Thus, the changeover from the older System to the newer
running both the old and new Systems in parallel and,
System takes place in a preplanned, phased manner. See
finally, fully changing over to the new System after
figure 3.26.
gaining confidence in the working of the new System.
With this approach, the users must use both Systems
during the period of overlap. This minimizes the risk Figure 3.26—Phased Changeover
of using the newer System and, at the same time, helps
in identifying problems, issues or concerns that the user
Old Subsystem
cornes across in the newer System in the beginning. (Modules) New Subsystem

After a period of overlap, the user gains confidence and

Module n Module n
assurance in relying on the newer System. At this point,
Module n-1 Module n-1
the use of the older System is discontinued and the new
Module 3 Module 3
System becomes totally operational. Note in figure 3.25
Module 2 Module 2
that the number (m and n, respectively) of modules in the
Module 1 Module 1
new and old Systems may be different.
Rollout Schedule

Some of the risk that may exist in the phased changeover

includes:
• Resource challenges (both on the IT side—to be
able to maintain two unique environments, such
as hardware, OSs, databases and code; and on the
operations side—to be able to maintain user guides,
procedures and policies, définitions of System terms,
etc.)

236 | ©ISACA. AH Rights Reserved.

 CISA0 Official Review Manual 28th Edition | Chapter 3

• Extension of the project life cycle to cover two 3.7.3 System Change Procedures and the
Systems Program Migration Process
• Change management for requirements and
customizations to maintain ongoing support of the Following implémentation and stabilization, a System
older System enters the ongoing development, or maintenance, stage.
This phase continues until the System is retired. The
Abrupt Changeover phase involves those activities required to either correct
errors in the System or enhance the capabilities of the
In this approach, the newer System is changed over from System.
the older System on a cutoff date and time, and the
older System is discontinued after changeover to the new In this regard, an IS auditor should consider the
System takes place. See figure 3.27. following:
• The existence and use of a methodology for
Figure 3.27—Abrupt Changeover authorizing, prioritizing and tracking System change
requests from the user
• Whether emergency change procedures are addressed
Old Subsystem
(Modules)
in the operations manuals
New Subsystem
• Whether change control is a formai procedure for the
Module n Module n user and the development groups
Module n-1
• Whether the change control log ensures ail changes
Module n-1
shown were resolved
Module 3 Module 3
• The user’s satisfaction with the turnaround—
Module 2 Module 2 timeliness and cost—of change requests
• The adequacy of the security access restrictions over
Module 1 Module 1
production source and exécutable modules
Rollout Schedule
• The adequacy of the enterprise’s procedures for
dealing with emergency program changes
• The adequacy of the security access restrictions over
Changeover to the newer System in volves four major the use of the emergency logon IDs
steps or activities: The IS auditor should examine a sélection of changes on
1. Conversion of files and programs; test running on test
the change control log to:
bed
• Détermine whether changes to requirements resulted
2. Installation of new hardware, OS, application System
in appropriate change-development documents, such
and the migrated data
as program and operations documents.
3. Training employées or users in groups
• Détermine whether changes were made as
4. Scheduling operations and test running for go-live or
documented.
changeover
• Détermine whether current documentation reflects the
Some of the risk items related to changeover include: changed environment.
• Asset safeguarding • Evaluate the adequacy of the procedures in place for
• Data integrity testing System changes.
• System effectiveness • Review evidence (test plans and test results) to ensure
• System efficiency that procedures are carried out as prescribed by
• Change management challenges (depending on the organizational standards.
configuration items considered) • Review the procedures established for ensuring
• Duplicate or missing records (possible existence of exécutable and source code integrity.
duplicate or erroneous records if data cleansing is not • Review production exécutable modules and verify
done correctly) that there is only one corresponding version of the
program source code.
• Check the technical Controls of the change
management tool.
Additionally, an IS auditor should review the
overall change management process for possible

©ISACA. AH Rights Reserved. | 237

 CISA° Official Review Manual 28th Edition | Chapter 3

improvements in acknowledgment, response time, • Practical sessions on how to use the System
response effectiveness and user satisfaction with the • Remédiai computer training (if needed)
process. • Online sessions on the web or on physical media

Critical Success Factors It is important to hâve a library of cases or tests,

including user errors and the System response to those
Critical success factors of planning the implémentation errors. The training administrator should record student
include the following: information in a database or spreadsheet, including
• To avoid delays, the appropriate skilled staff must student feedback for improving the training course.
attend workshops and participate for the entire project
duration. 3.7.4 System Software Implémentation
• The documentation needed for carrying out the work
needs to be ready at project initiation. System software implémentation involves identifying
• Decision-makers must be involved in ail steps to features, configuration options and Controls for
ensure that ail necessary decisions can be made. standard configurations to apply across the enterprise.
Additionally, implémentation involves testing the
End-User Training software in a nonproduction environment and obtaining
some form of certification and accréditation to place the
The goal of a training plan is to ensure that the end user
approved OS software into production.
can become self-sufficient in the operation of the System.
One of the most important keys in end-user training 3.7.5 Certification/Accréditation
is to ensure that training is considered and a training
project plan is created early in the development process. Certification is a process by which an assessor enterprise
A strategy can be developed that takes into considération performs a comprehensive assessment against a standard
the timing, extent and delivery mechanisms. of management and operational and technical Controls in
an IS. The assessor examines the level of compliance in
The training should be piloted using a cross-section of
meeting certain requirements, such as standards, policies,
users to détermine how best to customize the training to
processes, procedures, work instructions and guidelines
the different user groups. Following the pilot, the training
—requirements made in support of accréditation. The
approach can be adjusted as necessary, based on the
goal is to détermine the extent to which Controls
feedback received from the pilot group.
are implemented correctly, operating as intended and
Separate classes should be developed for individuals who producing the desired outcome with respect to meeting
will assist in the training process. These train-the-trainer the system’s security requirements. The results of a
classes also provide useful feedback for improving the certification are used to reassess the risk and update the
content of the training program. System security plan, thus providing the factual basis for
an authorizing official to render an accréditation decision.
The timing of the delivery of training is very important.
If training is delivered too early, users will forget Accréditation is the official management decision (given
much of the training by the time the System goes into by a senior official) to authorize operation of an IS and
production. If training is delivered too late, there will not to explicitly accept the risk to the enterprise’s operations,
be enough time to obtain feedback from the pilot group assets or individuals, based on the implémentation of
and implement the necessary changes into the main an agreed-on set of requirements and security Controls.
training program. Training classes should be customized Security accréditation provides a form of QC and
to address skill level and needs of users based on their challenges managers and technical staff at ail levels to
rôles within the enterprise. implement the most effective security Controls possible
in an IS, given mission requirements, and technical,
To develop the training strategy, an enterprise must name
operational and cost/schedule constraints.
a training administrator. The training administrator will
identify users who need to be trained with respect to their By accrediting an IS, a senior official accepts
spécifie job functions. Considération should be given to responsibility for the security of the System and is fully
the following format and delivery mechanisms: accountable for any adverse impact to the enterprise
• Case studies if a breach of security occurs. Thus, responsibility
• Role-based training and accountability are core principles that characterize
• Lecture and breakout sessions accréditation.
• Modules at different expérience levels

238 | ©ISACA. AH Rights Reserved.

 CISA® Official Review Manual 28th Edition | Chapter 3

3.8 Postimplementation Review • Projected cost versus benefits or ROI measurements

• Recommendations that address any System
Projects should be formally closed to provide accurate inadequacies and deficiencies
information on project results, improve future projects • Plan for implementing any recommendations
and allow an orderly release of project resources. • Assessment of the development project process:
The closure process should détermine whether project ■ Were the chosen méthodologies, standards and
objectives were met or excused and should identify techniques followed?
lessons learned to avoid mistakes and encourage ■ Were appropriate project management techniques
répétition of good practices. In contrast to project used?
closure, a postimplementation review typically is carried ■ Is the risk of operating the System within
out several weeks or months after project completion, acceptable risk levels?
when the major benefits and shortcomings of the solution
implemented will be realized. The review is part of a Not ail lessons associated with a given project may be
immediately évident on completion. It is good practice
benefits realization process and includes an estimate of
for the project development team and a sélection of
the project’s overall success and impact on the business.
end users to perform a second, joint review after the
A postimplementation review is also used to détermine project has been completed and the System has been
whether appropriate Controls were built into the System. in production for a sufficient time period, to assess its
It should consider the technical details and the process effectiveness and value to the enterprise.
that was followed in the course of the project, including
the following: Closing a project is a formai process that focuses on
capturing lessons learned for future use. Figure 3.28
• Adequacy of the System:
■ Does the System meet user requirements and summarizes five steps that should be included in the
business objectives? closing of a project.
■ Hâve Controls been adequately defïned and
implemented?

Figure 3.28—Project Closeout Steps

Step Action
1 Assign responsibility for any outstanding issues to spécifie individuals and identify the related budget for
addressing these issues (if applicable).
2 Assign custody of contracts, and either archive documentation or pass it on to those who need it.
3 Conduct a postimplementation review with the project team, development team, users and other stakeholders to
identify lessons learned that can be applied to future projects. Include the following information:
• Content-related criteria, such as:
■ Fulfillment of deliverable targets and any additional objectives
■ Attainment of project-related incentives
■ Adhérence to the schedule and costs
• Process-related criteria, such as:
■ Team dynamics and internai communication
■ Relationships between the project team and external stakeholders
4 Document any risk that was identified in the course of the project, including risk that may be associated with
proper use of the deliverables, and update the risk register.
5 Complété a second postimplementation review after the project deliverables hâve been completed for long enough
to realize the true business benefits and costs, and use the results of this review to measure the project’s overall
success and impact on the business.

It is important to note that, for a postimplementation and design phase and collected during each stage of
review to be effective, the information to be reviewed the project. For example, the project manager might
should be identified during the project feasibility establish certain checkpoints to measure effectiveness

©ISACA. AH Rights Reserved. | 239

 CISA6 Official Review Manual 28th Edition | Chapter 3

of software development processes and accuracy of collected before the project begins and after the project is
software estimâtes during the project execution. Business implemented (figure 3.29).
measurements should also be established up front and

Figure 3.29—Measurements of Critical Success Factors

Productivity • Dollars spent per user

• Number of transactions per month
• Number of transactions per user
Quality • Number of discrepancies
• Number of disputes
• Number of occurrences of fraud/misuse détection
Economie value • Total processing time réduction
• Monetary value of administration costs
Customer • Turnaround time for customer question handling
service • Frequency of useful communication to users

It is also important to allow enough business cycles to be business objectives in a cost-effective manner and control
executed in the new System to realize the new System’s integrity still exists.
actual ROI.
An IS auditor should perform the following functions:
A postimplementation project review should be • Détermine if the System objectives and requirements
performed jointly by the project development team were achieved. During the postimplementation
and appropriate end users. Typically, the focus of this review, careful attention should be paid to the end
type of internai review is to assess and critique the users’ use, trouble tickets, work orders and overall
project process, whereas a postimplementation review satisfaction with the System. This review indicates
has the objective of assessing and measuring the value whether the System’s objectives and requirements
that the project has on the business. Alternatively, an were achieved.
independent group that is not associated with the project • Détermine if the cost benefits identified in the
implémentation (internai or external audit) can perform a feasibility study are being measured, analyzed and
postimplementation review. accurately reported to management.
• Review program change requests performed to assess
3.8.1 IS Auditor's Rôle in the type of changes required of the System. The
Postimplementation Review type of changes requested may indicate problems
in the design, programming or interprétation of user
An IS auditor performing a postimplementation review requirements.
should be independent of the System development • Review Controls built into the System to ensure that
process. Therefore, an IS auditor involved in Consulting they are operating according to design. If enterprise
with the project team on the development of the System asset management (EAM) was included in the System,
should not perform this review. Unlike internai project this module should be used to test key operations.
team reviews, postimplementation reviews performed by • Review operators’ error logs to détermine if
an IS auditor tend to concentrate on the control aspects of there are any resource or operating problems
the System development and implémentation processes. inhérent within the System. The logs may indicate
It is important that ail audit involvement in the inappropriate planning or testing of the System prior
development project be thoroughly documented in the to implémentation.
audit work papers to support an IS auditor’s findings and • Review input and output control balances and reports
recommendations. This audit report and documentation to verify that the System is processing data accurately.
should be reused during maintenance and changes to
validate, verify and test the impact of any changes made
to the System. The System should periodically undergo a
review to ensure that the System is continuing to meet

240 | ©ISACA. Ail Rights Reserved.

 CISAC Official Review Manual 28th Edition | Chapter 3

Case Study • Wonderwheels management implemented plans to

upgrade to a new database package. The upgrade
Wonderwheels is a major national retailer specializing in is underway; however, it is taking longer than
outdoor sports, hunting, fishing and camping, including anticipated.
a wide variety of all-terrain vehicles (ATVs), which Regarding the database upgrade, sizeable customizations
inspired its name. The enterprise has operations currently were anticipated and are being carried out with a phased
based in the United States and has a long-term business approach of partial deliverables. These deliverables are
plan to expand its retail centers to selected parts of the released to users for pilot usage on real data and actual
As part of ongoing current operations, management has projects. Concurrently, design and programming of the
asked an internai IS auditor to review the enterprise next phase are ongoing. In spite of positive initial test
readiness for complying with requirements for protecting results, the internai audit group has voiced that it has
cardholder information. This is meant to be a high-level not been included in key compliance decisions regarding
overview of where the firm stands and not a point-by- the configuration and testing of the new System. In
point review of its compliance with the spécifie standard addition, operational transactions are often queued, or
(which would be undertaken as a separate engagement hang during execution, and, with increasing frequency,
later in the year). data are corrupted in the database. Additional problems
hâve shown up—errors already corrected hâve started
During the initial assessment, the IS auditor learned the occurring again, and functional modifications already
following information: tested tend to présent other errors. The project, already
• POS register encryption—The retailer uses wireless late, is now in a critical situation.
POS registers that connect to application servers
located at each store. These registers use wired 1. Which of the following présents the MOST
équivalent protection (WEP) encryption. significant risk to the retailer?
• POS local application server locations—The POS
application server, usually located in the middle of A. Database patches are severely out of date.
the customer service area at each store, forwards ail B. Wireless POS registers use WEP encryption.
sales data over a frame relay network to database C. Crédit cardholder information is sent over the
servers at the Wonderwheels corporate headquarters, Internet.
with strong encryption applied to the data. The data D. Aggregate sales data are mailed to a third party.
are then sent over a Virtual private network (VPN) to
the crédit card processor for approval of the sale. 2. Based on the case study, which of the following
• Enterprise database locations—Enterprise Controls is the MOST important to implement?
databases are located on a protected, screened subset
of the enterprise local area network. A. POS registers should use two-factor
• Sales data distribution—Weekly aggregated sales authentication, with enforced complex passwords.
data, by product line, are copied as-is from the B. Wireless access points should use Media Access
enterprise databases to magnetic media and mailed to Control (MAC) address filtering.
a third party for analysis of buying patterns. C. The current ERP System should be patched for
• Current ERP System compliance—The current compliance with the GDPR.
State of the enterprise ERP System is such that it D. Aggregate sales data should be anonymized and
may be out of compliance with newer laws and encrypted prior to distribution.
régulations. During the initial assessment, the IS
auditor determined that the ERP System does not
adhéré to the EU General Data Protection Régulation
(GDPR).
Additionally, Wonderwheels database software has not
been patched in over two years, due to a few factors:
• Vendor support for the database package was dropped
because it was acquired by a competitor and its
remaining business was refocused to other software
services.

©ISACA. AH Rights Reserved. | 241

 Cl SA" Official Review Manual 28th Edition | Chapter 3

3. In the preliminary report to management, regarding

the State of the database upgrade, which of the
following is MOST important for the IS auditor to
include?

A. Internai audit should be included among the

steering committee approvals.
B. There is a possibility that the new database may
not be compatible with the existing ERP solution.
C. An ERP upgrade and/or patch is required to ensure
updated database compatibility.
D. Internai audit should be able to review the
upgraded database to ensure compliance with
Payment Card Industry Data Security Standard
(PCI DSS).

4. To contribute more directly to help address the

problems around the database upgrade, the IS auditor
should:

A. Review the validity of the functional project

spécifications as the basis for an improved
software baselining définition.
B. Propose to be included in the project team as a
consultant for QC of deliverables.
C. Research the problems further to identify root
causes and define appropriate countermeasures.
D. Contact the project leader, discuss the project
plans and recommend redefining the delivery
schedule using the PERT methodology.

Answers on page 244

242 | ©ISACA. AH Rights Reserved.

 CISA’ Official Review Manual 28th Edition | Chapter 3

Page intentionally left blank

©ISACA. Ail Rights Reserved. I 243

 CISA6 Official Review Manual 28th Edition | Chapter 3

Chapter 3 Answer Key numbers. This présents the most significant

risk and should be addressed.
Case Study
3. A. If internai audit is part of the steering
1. A. Unpatched database servers are located on a
committee, then it will hâve a say in the
screened subnet; this mitigates the risk to the
compliance and security-related Controls to be
enterprise.
included in production releases.
B. Use of WEP encryption présents the most
B. Ensuring database compliance is an operational
significant risk because WEP uses a fixed secret
responsibility and not an audit responsibility.
key that is easy to break. Transmission of crédit
C. Compatibility with existing architecture must be
cardholder information by wireless registers is
a function of the database implémentation project
susceptible to interception and présents a very
team as a whole, which can include internai audit
serious risk.
and also includes operations. Therefore, it is not
C. Sending crédit cardholder data over the Internet is
the best answer choice.
less of a risk because strong encryption is being
D. Although it is important that the upgraded
used.
database solution be compliant with ail régulations
D. Because the sales data being sent to the third party
affecting the enterprise, such a review should not
are aggregate data, no cardholder information
be limited to one régulation. Therefore, it is not
should be included.
the best choice of those answers provided.

2. A. According to the case study, it is unclear

4. A. Functional project spécifications should be
whether the POS registers already use two-factor
executed by users and Systems analysts, and not
authentication. It is known that aggregate sales
by the auditor.
data are copied onto other media as-is, without any
B. To propose to be project consultant for quality
Controls, for external distribution.
would not bring about an essential contribution,
B. According to the case study, it is unclear whether
because quality is a formai characteristic; whereas,
the wireless access points use MAC address
in the current case, the problem is substantial
filtering. It is known that aggregate sales data
System instability.
are copied onto other media as-is, without any
C. The only appropriate action is additional
Controls, for external distribution.
research, even if the apparently technical
C. Compliance with the GDPR, although important,
nature of the problem renders it unlikely that
is not the most important due to the current
the auditor may find it alone.
operations being only in the United States, and the
D. To contact the project leader and redesign the
potential for expansion into the EU is a long-term
schedule of deliveries would not solve the
vision for the enterprise.
problem. Furthermore, the définition of real causes
D. It is unclear whether sales data are secure and
may sensibly alter the project environment.
free of personally identifiable information, such
as crédit card information and Social Security

244 | ©ISACA. Ail Rights Reserved.

 CISA° Official Review Manual 28th Edition | Chapter 4

Chapter 4

Information Systems Operations and

Business Resilience
OverView
Domain 4 Exam Content Outline..............................................................................................................246
Learning Objectives/Task Statements.......................................................................................................246
Suggested Resources For Further Study...................................................................................................246
Self-Assessment Questions.......................................................................................................................247
Chapter 4 Answer Key..............................................................................................................................250

Part A: Information Systems Operations

4.1 IT Components.................................................................................................................................. 253
4.2 IT Asset Management........................................................................................................................274
4.3 Job Scheduling and Production Process Automation........................................................................ 274
4.4 System Interfaces...............................................................................................................................276
4.5 End-User Computing and Shadow IT................................................................................................277
4.6 Systems Availability and Capacity Management.............................................................................. 279
4.7 Problem and Incident Management................................................................................................... 286
4.8 IT Change, Configuration and Patch Management........................................................................... 290
4.9 Operational Log Management........................................................................................................... 294
4.10 IT Service Level Management........................................................................................................ 298
4.11 Database Management..................................................................................................................... 300

Part B: Business Resilience

4.12 Business Impact Analysis................................................................................................................ 309
4.13 System and Operational Resilience.................................................................................................311
4.14 Data Backup, Storage and Restoration........................................................................................... 313
4.15 Business Continuity Plan................................................................................................................. 320
4.16 Disaster Recovery Plans.................................................................................................................. 335

Case Study
Case Study................................................................................................................................................ 345
Chapter 4 Answer Key............................................................................................................................. 348

©ISACA. Ail Rights Reserved. | 245