Monitoring and Information Systems

In this chapter, perhaps more than in any other, it would be helpful if we could consider everything at

once. How is it possible to discuss monitoring without specifying what is to be controlled? On the other

hand, how is it possible to specify a control system without understanding what aspects of a project

are subject to measurement and how the measurement is to be accomplished? As a matter of fact,

one could just as easily argue that evaluation, the primary subject of Chapter 12, should precede both

monitoring and control. The placement of these chapters is arbitrary, and readers may feel free to read

them in any order. Irrespective of the order in which one considers these subjects, however, their

interdependence is clear.

Our fundamental approach to evaluation and control of projects is that these activities are, at base,

the opposite sides of project selection and planning. The logic of selection, such as by the Project

Portfolio Process, or any of the methods described in Chapter 2, dictates the components to be

evaluated, and the details of planning expose the elements to be controlled. The ability to measure is

prerequisite to either. Thus, all of the project’s objectives as delineated in the project selection process

must be examined and measures for each included in the monitoring system. If a weighted factor

model is used, the measures can usually be integrated into a single evaluation measure. For a

continuously operating project selection system, monitoring the critical project measures as the

project proceeds through its life cycle, such as by the Project Management Offi ce as described in

Chapter 5, is required so projects can be terminated, if necessary, and new projects initiated. Much

more will be said about this in Chapters 12 and 13.

Many more evaluative measures are needed, for example, for the maintenance of a risk management

system. Not only must the project performance be monitored, but the environment within which the

project exists must also be observed and recorded. Monitoring is collecting, recording, and reporting

information concerning any and all aspects of project performance that the project manager or others

in the organization wish to know. In our discussion it is important to remember that monitoring, as an

activity, should be kept distinct from controlling (which uses the data supplied by monitoring to bring
actual performance into approximate congruence with planned performance), as well as from

evaluation (through which judgments are made about the quality and effectiveness of project

performance).

First we expand on the nature of this link between planning and control, including a brief discussion

of the various aspects of project performance that need to be monitored. We also examine some of

the problems associated with monitoring a project. Finally, we report on several computer software

packages that can greatly increase the speed and effectiveness of project monitoring.

This book is addressed to practicing PMs as well as students of project management. Students resist

the idea that PMs do not have immediate access to accurate information on every aspect of the

project. But PMs know it is not always easy to fi nd out what’s going on when working on a project.

Records are frequently out of date, incomplete, in error, or “somewhere else” when needed. A hospital

executive of our acquaintance carried out a project that was designed to generate a major

improvement in profi tability by altering the patient mix. The hospital’s accounting system could not

report on the results of the project until six months later.

Throughout the chapter, our primary concern is to ensure that all parties interested in the project have

available, on a timely basis, the information needed to exercise effective control over the project and

the uncertainties that impact on it. The other uses for monitoring (e.g., auditing, learning from past

mistakes, or keeping senior management informed), important as they are, must be considered

secondary to the control function when constructing the monitoring system. The key issue, then, is to

create an information system that gives project managers the information they need to make

informed, timely decisions that will keep project performance as close as possible to the project plan.

One fi nal note: In this chapter, we frequently refer to a “project monitor,” a “project controller,” or

even to the “group” or “offi ce” responsible for monitoring. These individuals and groups do in fact

exist on most large projects. On a small project, it is likely that the person in charge of monitoring is

the same person as the project controller—and the same person as the PM. That is, when we refer to
the project monitor and controller, we are referring to roles needed in project management, not

necessarily to different individuals.

10.1 THE PLANNING-MONITORING-CONTROLLING CYCLE

Throughout this book we have stressed the need to plan, check on progress, compare progress to the

plan, and take corrective action if progress does not match the plan. The key things to be planned,

monitored, and controlled are time (schedule), cost (budget), and performance (specifi cations).

These, after all, encompass the fundamental objectives of the project.

There is no doubt that some organizations do not spend suffi cient time and effort on planning and

controlling projects. It is far easier to focus on doing, especially because it appears to be more effective

to “stop all the talk and get on with the work.” We could cite firm after firm that incurred great expense

(and major losses) because the planning process was inadequate for the tasks undertaken.

• A major construction project ran over budget by 63 percent and over schedule by 48 percent

because the PM decided that, since “he had managed similar projects several times before, he

knew what to do without going into all that detail that no one looks at anyway.”

• A large industrial equipment supplier “took a bath” on a project designed to develop a new

area of business because they applied the same planning and control procedures to the new

area that they had used (successfully) on previous, smaller, less complex jobs.

• A computer store won a competitive bid to supply a computer, fi ve terminals, and associated

software to the Kansas City offi ce of a national fi rm. Admittedly insufficient planning made

the installation signifi cantly late. Performance of the software was not close to specifi ed

levels. This botched job prevented the fi rm from being invited to bid on more than 20 similar

installations planned by the client.

The planning (budgeting and scheduling) methods we propose “put the hassles up front.” They require

a signifi cantly greater investment of time and energy early in the life of the project, but they signifi

cantly reduce the extent and cost of poor performance and time/cost overruns. Note that this is no

guarantee of a trouble-free project, merely a decline in the risk of failure.

It is useful to perceive the control process as a closed-loop system, with revised plans and schedules

(if warranted) following corrective actions. We delay a detailed discussion on control until the next

chapter, but the planning-monitoring-controlling cycle is continuously in process until the project is

completed. The information fl ows for such a cycle are illustrated in Figure 10-1. Note the direction of

the fl ows, information fl owing from the bottom toward the top and authority fl owing from the top

down.

It is also useful to construct this process as an internal part of the organizational structure of the

project, not something external to and imposed on it or, worse, in confl ict with it. Finally, experience

tells us that it is also desirable, though not mandatory, that the planning monitoring- controlling cycle

be the normal way of life in the parent organization. What is good for the project is equally good for

the parent fi rm. In any case, unless the PM has a smoothly operating monitoring/control system, it

will be diffi cult to manage the project effectively.

Designing the Monitoring System

The first step in setting up any monitoring system is to identify the key factors to be controlled. Clearly,

the PM wants to monitor performance, cost, and time but must defi ne precisely which specific

characteristics of performance, cost, and time should be controlled and then establish exact

boundaries within which control should be maintained. There may also be other factors of importance

worth noting, at least at milestones or review points in the life of the project. For example, the number

of labor hours used, the number or extent of process or output changes, the level of customer

satisfaction, and similar items may be worthy of note on individual projects.

Figure 10-1 Project authorization and expenditure control system information flow. Source: Dean

(1968).

But the best sources of items to be monitored are the project action plan and the risk management

plan. The action plan describes what is being done, when, and the planned level of resource usage for

each task, work package, and work element in the project. Monitoring the risks found in the risk

management plan keeps the PM and project team alert to specific risks and thus lowers the probability

of surprises. The monitoring system is a direct connection between planning and control. If it does not

collect and report information on some significant element of the plan, control can be faulty or missing.

The action plan furnishes the key items that must be measured and reported to the control system,

but it is not suffi cient. For example, the PM might want to know about changes in the client’s attitudes

toward the project. Information on the morale of the project team might be useful in preparing for

organizational or personnel changes on the project. These two latter items may be quite important,

but are not reflected in the project’s action plans.

Unfortunately, it is common to focus monitoring activities on data that are easily gathered— rather

than important—or to concentrate on “objective” measures that are easily defended at the expense
of softer, more subjective data that may be more valuable for control. Above all, monitoring should

concentrate primarily on measuring various facets of output rather than intensity of activity. It is crucial

to remember that effective PMs are not primarily interested in how hard their project teams work.

They are interested in achieving results.

The measurement of project performance usually poses the most diffi cult data gathering problem.

There is a strong tendency to let project inputs serve as surrogate measures for output. If we have

spent 50 percent of the budget (or of the scheduled time), we assume we have also completed 50

percent of the project or reached 50 percent of our performance goal. If the item being referenced is

a small work unit, it does not make a significant difference if we are wrong. If, however, the reference

is to a task or to the entire project, the assumption of input/ output proportionality (hereafter, the

“proportionality rule”) is often seriously misleading.

Further, it is common to specify performance to a level of precision that is both unnecessary and

unrealistic. For example, a communications software project specifi ed that a telephone “information”

system had to locate a phone number and respond to the querier in 5 seconds or less. Is 5.1 seconds

a failure? Does the specifi cation mean 5 seconds or less every time, or merely that response times

should average 5 seconds or less? Is the specifi cation satisfi ed if the response time is 5 seconds or

less 90 percent of the time?

The monitoring systems we describe in this chapter, however, focus mainly on time and cost as

measures of performance, not specifi cations. While we are most certainly concerned with keeping

the project “on spec,” and do consider some of the problems of monitoring output, the subject is not

fully developed here because the software designed to monitor projects is not constructed to deal

with the subject adequately. The matter will get more attention in Chapter 12 when auditing is

discussed.

Given all this, performance criteria, standards, and data collection procedures must be established for

each of the factors to be measured. The criteria and data collection procedures are usually set up for

the life of the project. The standards themselves, however, may not be constant over the project’s life.
They may change as a result of altered capabilities within the parent organization or a technological

breakthrough made by the project team; but, perhaps more often than not, standards and criteria

change because of factors that are not under the control of the PM. For example, they may be changed

by the client. One client who had ordered a special piece of audio equipment altered performance

specifi cations signifi cantly when electronic parts became available that could fi lter out random

noises.

Standards may also be changed by the community as a response to some shift in public policy—witness

the changes in the performance standards imposed on nuclear power installations or automotive

exhaust systems. Shifts in the prime rate of interest or in unemployment levels often alter the

standards that the PM must use for making project-related decisions. The monitoring process is based

on the criteria and standards because they dictate, or at least constrain, the set of relevant measures.

Next, the information to be collected must be identifi ed. This may consist of accounting data,

operating data, engineering test data, customer reactions, specifi cation changes, and the like. The

fundamental problem is to determine precisely which of all the available data should be collected. It

is worth repeating that the typical determinant for collecting data too often seems to be simply the

ease with which they can be gathered. Of course the nature of the required data is dictated by the

project plan, as well as by the goals of the parent organization, the needs of the client, and by the fact

that it is desirable to improve the process of managing projects.

Closely monitoring project work is often justifi ed with the argument that keeping close track of

progress will reduce the amount of crashing required near the end of the project. Partovi et al. (1993)

tested this argument and found that it was not strictly true. They examined five monitoring policies:

no monitoring, monitoring at random times, equal interval monitoring, monitoring more frequently at

the start of the project, and monitoring more frequently late in the project’s life. They found that there

was no signifi cant difference in the amount of crashing effort expended, whatever monitoring protocol

was used, but that high-frequency monitoring late in the project life was the best policy to prevent

project lateness.
Perhaps the most common error made when monitoring data is to gather information that is clearly

related to project performance but has little or no probability of changing signifi cantly from one

collection period to the next. Prior to its breakup, the American Telephone and Telegraph Company

used to collect monthly statistics on a very large number of indicators of operating effi ciency. The

extent of the collection was such that it fi lled a telephone-book-sized volume known as “Ma Bell’s

Green Book.” For a great many of the indicators, the likelihood of a signifi cant change from one month

to the next was extremely small. When asked about the matter, one offi cial remarked that the mere

collection of the data kept the operating companies “on their toes.” We feel that there are other, more

positive and less expensive ways of motivating project personnel. Certainly, “collect everything” is

inappropriate as a monitoring policy.

Therefore, the fi rst task is to examine the project plans in order to extract performance, time, and cost

goals. These goals should relate in some fashion to each of the different levels of detail; that is, some

should relate to the project, some to its tasks, some to the work packages, and so on. Data must be

identifi ed that measure achievement against these goals, and mechanisms designed that gather and

store such data. If at least some of the data do not relate to the work unit level, no useful action is apt

to be taken. In the end, it is the detailed work of the project that must be altered if any aspect of

project performance is to be changed.

Similarly, the process of developing and managing projects should be considered and steps taken to

ensure that information relevant to the diagnosis and treatment of the project’s organizational infi

rmities and procedural problems are gathered and collected. A reading of the fascinating book The

Soul of a New Machine (Kidder, 1981) reveals the crucial roles that organizational factors, interpersonal

relationships, and managerial style play in determining project success. Also of value is a paper by

Mekhilef et al. (2005) that develops a method to find and analyze the ways in which individuals

introduce dysfunction into decision processes. The authors study development projects, but their

methods seem applicable to any type of project.

Given that we know what type of data we want to collect, the next question is how to collect this

information. At this point in the construction of a monitoring system, it is necessary to define precisely
what pieces of information should be gathered and when. In most cases, the PM has options.

Questions arise. Should cost data be gathered before or after some specific event? Is it always

mandatory to collect time and cost information at exactly the same point in the process? What do we

do if a specific item is difficult to collect because the data source (human) fears reporting any

information that might contribute to a negative performance evaluation? What do we do about the

fact that some use of time is reported as “hours charged” to our project, and we are quite aware that

our project has been charged for work done on another project (but for the same customer) that is

over budget? Are special forms needed for data collection? Should we set up quality control

procedures to ensure the integrity of data transference from its source to the project information

system? Such questions merely indicate the broad range of knotty issues that must be handled.

A large proportion of all data collected takes one of the following forms, each of which is suitable for

some types of measures.

1. Frequency counts A simple tally of the occurrence of an event. This type of measure is often

used for “complaints,” “number of times a project report is late,” “days without an accident,”

“bugs in a computer program,” and similar items. The data are usually easy to collect and are

often reported as events per unit time or events as a percent of a standard number. Even with

such simple counts, data may be diffi cult to collect. Items such as “errors” or “complaints”

often go unreported by individuals or groups not particularly eager to advertise

malperformance.

2. Raw numbers Dates, dollars, hours, physical amounts of resources used, and specifi cations

are usually reported in this way. These numbers are reported in a wide variety of ways, but

often as direct comparisons with an expected or standard number. Also, “variances” are

commonly reported either as the difference between actual and standard or as the ratio of

actual to standard. Differences or ratios can also be plotted as a time series to show changes

in system performance. When collecting raw project data, it is important to make sure that all

data are collected from sources that operate on the same time intervals and with the same

rules for data collection.

3. Subjective numeric ratings These numbers are subjective estimates, usually of a quality, made

by knowledgeable individuals or groups. They can be reported in most of the same ways that

objective raw numbers are, but care should be taken to make sure that the numbers are not

manipulated in ways only suitable for quantitative measures. (See Chapter 2 for comments on

measurements.) Ordinal rankings of performance are included in this category.

4. Indicators When the PM cannot measure some aspect of system performance directly, it may

be possible to fi nd an indirect measure or indicator. The speed with which change orders are

processed and changes are incorporated into the project is often a good measure of team effi

ciency. Response to change may also be an indicator of the quality of communications on the

project team. Of course this measure may vary with the complexity of the change. Therefore,

when using indicators to measure performance, the PM must make sure that the link between

the indicator and the desired performance measure is as direct as possible and is not affected

by (or is corrected for) other variables.

5. Verbal measures Measures for such performance characteristics as “quality of team member

cooperation,” “morale of team members,” or “quality of interaction with the client” frequently

take the form of verbal characterizations. As long as the set of characterizations is limited and

the meanings of the individual terms consistently understood by all, these data serve their

purposes reasonably well.

Figure 10-2 Number of bugs found during test of Datamix program.

After data collection has been completed, reports on project progress should be generated. These

include project status reports, time/cost reports, and variance reports, among others. Causes and

effects should be identified and trends noted. Plans, charts, and tables should be updated on a timely

basis. Where known, “comparables” should be reported, as should statistical distributions of previous

data if available. Both help the PM (and others) to interpret the data being monitored. Figures 10-2

and 10-3 illustrate the use of such data. Figure 10-2 shows the results of a count of “bugs” found during

a series of tests run on a new piece of computer software. (Bugs found were fixed prior to subsequent

tests.) Figure 10-3 shows the percent of the time a computer program retrieved data within a specified

time limit. Each point represents a series of trials.

The PM can fi t a statistical function to the data shown in Figure 10-2 and make a rough estimate of

the number of tests that will have to be run to fi nd some predetermined number of additional bugs

in the program. By fitting a curve (formally or “by eyeball”) to the data in Figure 10-3, the PM can

estimate the cost and time (the number of additional trials and adjustments) required to get system

performance up to the specifi ed level. (Curve and distribution fi tting is easily done by use of Crystal

Ball®.)

Figure 10-3 Percent of specifi ed performance met during repeated trials.

The nature of timeliness will be amplifi ed next, but it is important that the PM make sure that the

AON/AOA and Gantt charts in the project war room (office) are frequently updated. Monitoring can

serve to maintain high morale on the project team as well as to alert team members to problems that

will have to be solved.

The purpose of the monitoring system is to gather and report data. The purpose of the control system

is to act on the data. To aid the project controller, it is helpful for the monitor to carry out some data

analysis. Signifi cant differences from plan should be highlighted or “flagged” so that they cannot be

overlooked by the controller. The methods of statistical quality control are very useful for determining

what size variances are “significant” and sometimes even help in determining the probable cause(s) of

variances. Where causation is known, it should be noted. Where it is not known, an investigation may

be in order. (It is also useful to remember that some things are more easily fixed than understood, in

which case investigations may not be cost-effective.) The decisions about when an investigation should

be conducted, by whom, and by what methods are the prerogative of the project controller, although

the actual investigation may be conducted by the group responsible for monitoring.

The Emanon Aircraft Company example presented in Chapter 7 is a case in point. While the study team

collected and analyzed a great deal of cost information during the process of finding the problem, the

method used for the analysis was actually quite simple. The team compared forecast or estimated

cost, F(t), with actual cost, A(t), for each batch of output from the manufacturing system. This analysis

was done for each cost center. The ratio of actual cost to estimated cost was calculated and plotted as

a time series, as in Figure 10-4.

Figure 10-4 Ratio of actual material cost to estimated material cost. Emanon Aircraft Company.

*Actual data were not used in constructing Figure 10-4, but the fi gure refl ects the consultants’

findings.

* Note that A(t)/F(t) 1 when the cost forecast for a cost center is greater than actual. In this case, the

cost involved was “material cost.” Though careful statistical analysis was not necessary in this specific

case, standard quality control techniques have wide application to project management (see any book

on statistical quality control—Evans et al., 1993, for example). Time series analysis can often give the

PM an early warning of problems.

At base, this provides a management by exception reporting system for the PM. But management by

exception has its fl aws as well as its strengths. It is essentially an “after-the-fact” approach to control.

Variances occur, are investigated, and only then is action taken. The astute PM is far more interested

in preventing problems than curing them. Therefore, the monitoring system should develop data

streams that indicate variances yet to come. Obviously, such indicators are apt to be statistical in

nature, hinting at the likelihood of a future problem rather than predicting it with certainty. An example

would be a trend in the data showing a system heading out of control. Interested readers are referred

to the “2-5-7 Rule” (see Shafer et al., 1998, Chapter 3). The PM may waste time and effort trying to

deal with trouble that will not actually occur. This may be frustrating, but the costs of dealing with
some nonproblems is usually minor when compared to the costs of dealing with real problems too

late.

In creating the monitoring system, some care should be devoted to the issues of honesty and bias. The

former is dealt with by setting in place an internal audit. The audit serves the purpose of ensuring that

the information gathered is honest. No audit, however, can prevent bias. In general, data are biased

by those who report them, advertently or inadvertently. Science has long known that the observation

of a system affects how the system performs. The controller must understand this fact of life. The fi rst

issue is to determine whether the possibility of bias in the data matters signifi cantly. If not, nothing

need be done. Bias fi nding and correcting activities (cf. Chapter 7) are worthwhile only if data with

less or no bias are required.

The issue of creating an atmosphere that fosters honesty on a project is widely ignored, but it is of

major importance. A set of instructions to the PM on how to do this is not beyond the scope of this

book, but if such instructions exist, we do not know of them. We do, however, have some advice to

offer. The PM can tolerate almost any kind of behavior except dishonesty. Projects are vulnerable to

dishonesty, far more vulnerable than the ongoing operations of the parent organization. Standard

operations are characterized by considerable knowledge about expected system performance. When

the monitoring system reports information that deviates from expectations, it is visible, noteworthy,

and tends to get attention. In the case of many projects, expectations are not so well known. Deviations

are not recognized for what they are. The PM is often dependent on team members to call attention

to problems. To get this cooperation, the PM must make sure that the bearer of bad news is not

punished; nor is the admitter-to-error executed. On the other

hand, the hider-of-mistakes may be shot with impunity—and then sent to Siberia.

There is some tendency for project monitoring systems to include an analysis directed at the

assignment of blame. This practice has doubtful value. While the managerial dictum “rewards and

punishments should be closely associated with performance” has the ring of good common sense, it

is actually not good advice. Instead of motivating people to better performance, the practice is more
apt to result in lower expectations. If achievement of goals is directly measured and directly rewarded,

tremendous pressure will be put on people to understate goals and to generate plans that can be met

or exceeded with minimal risk and effort.

Project Management in Practice

Using Project Management Software to Schedule the Olympic Games

The XV Olympiad in Calgary involved nearly 2000 athletes from 57 countries in 129 competitive events,

attracted over 1,500,000 spectators, was covered by over 5000 journalists, and was run by a staff of

600 professionals complemented by 10,000 volunteers.

For those 600 responsible for organizing, planning, scheduling, coordinating, and handling the

information requirements for the 16-day extravaganza, the task was overwhelming. The top managers

of the organizing committee thus turned to a Computer Based Project Planning and Scheduling (CBPPS)

system for scheduling and managing the 30,000 tasks organized into 50 projects.

The goal for the Calgary Games was to provide the best games ever, but within the budget. The

philosophy employed was to let each project manager plan his/her own project but meet fi rm

completion dates and budget limits. This made a lot of additional work for the upper managers since

each project’s reports and needs were different from every other project’s. However, two major

features of the project helped make this a success: (1) Knowing that the Games would happen on the

scheduled date regardless of whether they were ready or not, and (2) Being such a high-visibility,

challenging project that demands exceptional focus on the task.

To schedule the entire Winter Games, the 129-event, 16-day Olympics was broken down into 15-

minute periods, except for short-track speed skating which was segmented into 1-minute intervals.

There was a printout for every day by venue, minute by minute, and a complete set of drawings of

every site, building, and room. Meticulous scheduling was necessary to ensure that the 2500 or so

competitors, members of royalty, and government officials were at the right place at the right time.
Support staff, including medical and security personnel, were also carefully scheduled for each event

as crowds shifted from competition to competition. Transportation—600 buses—also had to be

scheduled, oftentimes on short notice. The biggest concern was the weather, and sure enough, the

Chinook winds forced the rescheduling of over 20 events, some of them twice!

Yet, the Calgary Games were the best yet, and organized better than ever before. Moreover, as

compared to the budget overruns of many other cities, this Olympiad was completed under budget!

Source: R. G. Holland, “The XV Olympic Winter Games: A Case Study in Project Management,” PM

Network, November 1989.

Project Management in Practice

Drug Counseling Program

A social service agency applied for and received funding for a special project to counsel male drug

addicts between 18 and 24 years of age, and to secure full-time employment for each client (or part-

time employment for clients who were still in school). To qualify for the program, the addicts must

have been arrested for a crime, but not be classed as “repeat offenders.” Further, the addict must be

living with at least one member of his family who is a parent or guardian. Among other conditions

placed on the grant, the agency was asked to develop a measure of effectiveness for the counseling

program that was acceptable to the funding agency.

The primary measure of effectiveness adopted by most drug programs was “rate of recidivism.” A

recidivistic incident is defined as any re-arrest for a drug-related crime, or any behavior that resulted

in the individual reentering the social service system after completing the program and being

discharged.

While a “re-arrest” is most surely recidivistic, there were several cases in which former clients

contacted the agency and asked to be re-admitted to the program. These voluntary re-admissions

resulted when a former client either began to use drugs again or was fearful that he would begin again.

It seemed to the agency professionals that voluntary re-admissions were successes, not failures.
A new measure of effectiveness was developed to replace “rate of recidivism.” It was composed of

scores on three different measures, combined with equal weighting.

1. Number of successive weeks of “clean urines.”

2. Number of successive months of satisfactory employment (or schooling) experience.

3. Number of successive months of satisfactory behavior at home.

Scores on the second and third measures were based on interviews with employers, teachers, and

parent(s).

Source: S. J. Mantel, Jr. Consulting project