Training Professionals to Detect Deception

Joey F. George
College of Business
Florida State University
Tallahassee, FL 32306-1110
Voice: 850-644-7449
Fax: 850-644-8225
[email protected]

David P. Biros, Lt Col

USAF Chief Information Office
Washington, DC 20330-1155
Voice: 703-601-3555
Fax: 703-601-4076
[email protected]

Judee Burgoon
Eller College of Business and Public Administration
University of Arizona
Tucson, AZ 85721
Voice: 520-621-5818
Fax: 520-621-2641
[email protected]

Jay F. Nunamaker, Jr.

Eller College of Business and Public Administration
University of Arizona
Tucson, AZ 85721
Voice: 520-621-4475
Fax: 520-621-3918
[email protected]

February 15, 2003

 Training Professionals to Detect Deception

Although dramatic threats to security, such as viruses, get most of the press,
simple deception can also be a serious threat. A common use of deception for breaching
security is social engineering, a technique often used by infamous hacker Kevin Mitnick
(Littman, 1996). Social engineering occurs when one individual poses as someone else
in order to get information merely by asking for it. This practice can also be used for
other purposes, such as getting unsuspecting users to download malicious software,
which, if installed, can be used for tasks like launching denial of service attacks from the
user’s machine (see, for example, http://www.cert.org/incident_notes/IN-2002-03.html).

One reason that social engineering works as well as it does is that people are not
very good at detecting deception. Deception detection is generally little more successful
than chance, i.e., people are right only about half the time (Feeley & deTurck, 1995;
Miller & Stiff, 1993). To try to improve these odds, researchers have worked to uncover
reliable indicators of deception (Buller & Burgoon, 1994; Zuckerman & Driver, 1985).
These include increased blinking, higher voice pitch, increased self-grooming, more
passive statements, more negative statements, and more distancing of the storyteller from
the story told. Other cues that have emerged more recently include pauses, response
latencies, eye contact, and message duration (deTurck & Miller, 1985; Vrij, 2000). The
research question is whether training people to identify these cues helps improve their
accuracy in detecting deception.

While training seems like the ideal way to improve detection accuracy, the
findings from past research on the effectiveness of training are mixed (Frank & Feeley, in
press). The study described here was an attempt to develop and test a training program
for deception detection for rank-and-file military members. The training provided a basic
understanding of deception and specific information about cues for detecting deception.
We present the study design and procedures, followed by a brief summary of findings
related to accuracy in detecting deception, to illustrate how tests of such training systems
can be conducted.

Study Design and Procedures

The study was conducted in fall 2002 on a large USAF facility in the US. A total
of 125 officers participated as subjects, although the total number participating per
session varied. Subjects were randomly assigned to either the control or to one of three
treatment groups. For one treatment, traditional lectures were used for all three training
sessions. For the second treatment, lectures were used for the first and third sessions, but
in the second session, a specially created system called Agent99 was used exclusively.
For the third treatment, lectures were used for the first and third sessions, but for the
second session, a combination of lecture and Agent99 was used. All lectures in all
treatments were supported with Powerpoint presentations. The control received no
training, but control subjects completed the same measurement instruments as the
experimental subjects. A pre-session was used to collect baseline data on all subjects in
all four groups. The instructors were one doctoral student from a US business school and

2
three USAF officers completing their masters’ degrees at the Air Force Institute of
Technology.

The training curriculum was developed jointly by the authors and their respective
research teams. The basis for the curriculum was a set of three Powerpoint presentations,
each on a different topic: deception detection generally, cues used to detect deception,
and heuristics for decision making that are susceptible to deception. Each presentation
was designed to last for one hour. The second lecture, on cues, also included deceptive
communication examples used by the instructors to illustrate the cues. These examples
were either text only, audio only, or video with audio. Most examples came from past
studies of deception detection, consisting of experimental subjects trying to deceive their
interviewers. Other examples were specifically created and recorded for this study. The
lectures, delivered by the training instructors, were also videotaped. The instructors pilot
tested all training materials, including Agent99, weeks before the study began.

Agent99 is a multifaceted information system, but the only parts we used were the
ones designed to support training. (For details on the system and its development, see
Lin, et al., (2003).) Agent99 allows users to access training materials in any format, in
whatever order users decide to access them in, through a web browser interface. The
version tested had only some functionalities operational and was used for the cues session
only. It was populated with the same Powerpoint presentation and examples used in the
traditional lecture format. In addition, the system included the videotaped delivery of the
cues lecture. For the treatment that used a combination of lecture and Agent99, subjects
began the session with a live lecture and then were allowed to view examples using
Agent99. Great care was taken to ensure that subjects in all three treatments had access
to the exact same content for the cues material.

The basic procedures for the training sessions were as follows: Subjects reported
to a classroom at the USAF facility. They began by completing a battery of instruments,
including a knowledge pre-test and a deception detection accuracy pre-test. All data were
collected on-line. The knowledge pre-test consisted of 12 multiple choice questions
derived from the day’s training content. The deception accuracy pre-test consisted of 6
examples, two each in text, audio, and audio-video formats. Half were deceptive and half
were not. Subjects were asked to judge if each example was deceptive or not. Subjects
were then trained, except for control subjects, who were given a one hour break.
Afterwards, all subjects completed a knowledge post-test, made up of the same questions
as the pre-test but in a different order, and a deception detection accuracy post-test,
similar to the pre-test but consisting of different examples. Subjects then completed
additional instruments and were dismissed.

Findings

Table 1 provides the results of the knowledge tests for the control group and the
combined treatment groups for all three sessions. Each knowledge test had 12 questions,
and results reported indicate the number of questions answered correctly. Table 2
provides the results of the deception detection accuracy tests for the control group and the

3
combined treatment groups for all three sessions. Each accuracy test had six examples,
and the results reflect the number of examples evaluated correctly.

Table 1: Means and standard deviations (in parentheses) for knowledge pre-tests and post-tests

Control (N = 29) Treatments (N = 86)

Pre-test Post-test Pre-test Post-test
General 5.07 (1.60) 5.14 (1.64) 5.55 (1.69) 8.81 (1.64)
Cues 4.07 (1.49) 4.38 (1.59) 5.57 (1.63) 7.73 (2.30)
Heuristics 5.41 (2.23) 4.93 (2.30) 6.01 (1.97) 8.74 (2.07)

Table 2: Means and standard deviations (in parentheses) for accuracy pre-tests and post-tests

Control (N = 29) Treatments (N = 85)

Pre-test Post-test Pre-test Post-test
General 2.89 (1.11) 3.72 (1.25) 3.11 (1.29) 3.67 (1.01)
Cues 4.10 (1.23) 2.97 (0.78) 4.39 (1.11) 3.65 (0.86)
Heuristics 3.86 (1.22) 3.38 (1.37) 3.72 (1.14) 3.39 (1.02)

For the knowledge tests, performance was measured by taking the difference
between pre-test and post-test scores within each session. Independent t-tests showed
that the treatment groups differed from the control group for all three sessions (general:
t(113)=-8.921, p < .000; cues: t(113)=-4.54, p < .000; heuristics: t(113)=-7.536, p < .000)
For each session, the control group did not improve, while the training session groups
did.

For the detection accuracy tests, performance was also measured by taking the
difference between pre-test and post-test scores within the session. There were no
statistically significant differences between the treatment groups and the control group on
deception detection accuracy. The general trend, although not statistically significant, is
toward an improvement in deception detection accuracy for the control group (an
improvement of 0.48 between the pre-test of the first session and the post-test of the last
session) and for the combined treatment groups (an improvement of 0.28 for the same
comparison). If all subjects are combined, the average improvement in deception
detection performance between the pre-test of the first session and the post-test of the last
session is 0.333, which is statistically significant (t(113)=2.048, p < .043) (see Figure 1).

Discussion

Treatment groups improved their understanding of deception, vis-à-vis the control

group, as shown by their knowledge test performance, but there were no differences
between the treatment groups and the control on their deception detection accuracy.
However, overall, subjects participating in the study improved their detection accuracy.
It may be that mere exposure to the accuracy tests improved performance, possibly
through heightening subject lie bias (Feeley & Young, 1998). The subtleties exposed by
comparisons made within and between groups, and on both knowledge and judgment
tests, illustrates the value of the training design employed in this investigation. Tests that
lack pre- to post-test comparisons, control groups, or both types of knowledge gains

4
(cognitive and judgmental) may fail to adequately discern what a given training
curriculum and tool provide.

Figure 1: Deception detection accuracy scores for control vs. combined treatment groups and for the
combined sample

Judgment Test Scores

5
4
3
Control
2
Treatments
1
Overall Overall
0
Treatments
9/17
9/17 Control
pre 10/1
post 10/1
pre 10/15
post 10/15
pre
post

5
References

Buller, D. B., & Burgoon, J. K. (1994). Deception: Strategic and nonstrategic communication.
In J. A. Daly & J. M. Wiemann (Eds.), Strategic interpersonal communication (pp. 191-223).
Hillsdale, NJ: Erlbaum.

deTurck, M.A., & Miller, G. R. (1985). Deception and arousal: Isolating the behavioral
correlates of deception. Human Communication Research, 12, 181-201.

Feeley, T.H., & deTurck, M.A. (1995). Global cue usage in behavioral lie detection.
Communication Quarterly, 43, 420-430.

Feeley, T.H., & Young, M.J. (1998). Humans as lie detectors: Some more second
thoughts. Communication Quarterly, 46(2).

Frank, M., & Feeley, T. H. (in press). To catch a liar: Challenges for research in lie
detection training. Journal of Applied Communication Research, .

Lin, M., Cao, J., Crews, J., Burgoon, J. and Nunamaker, J.F. Jr. (2003). AGENT99
trainer: A web-based multimedia training system for deception detection.
Working paper, CMI, University of Arizona, Tucson, AZ.

Littman, J. (1996). The Fugitive Game. Boston: Little, Brown & Co.

Miller, G.R., & Stiff, J.B. (1993). Deceptive communication. Newbury Park, CA: Sage
Publications, Inc.

Vrij, A. (2000). Detecting lies and deceit: The psychology of lying and its implications
for professional practice. Chichester: John Wiley and Sons.

Zhang, D.; Zhao, J. L.; Zhou, L.; Nunamaker, J. F., Can E-Learning Replace Traditional
Classroom Learning?— Evidence and Implication of the Evolving E-Learning
Technology, CACM.

Zuckerman, M., & Driver, R.E. (1985). Telling lies: Verbal and nonverbal correlates of
deception. In A.W. Siegman & S. Feldstein (Eds.), Nonverbal communication: An
integrated perspective (129-147). Hillsdale, NJ: L. Erlbaum.