THE GLOBAL VILLAGE PLAYGROUND: A QUALITATIVE CASE STUDY OF

DESIGNING AN ARG AS A CAPSTONE LEARNING EXPERIENCE

Mary Jo Dondlinger, B.A., M.A.

Dissertation Prepared for the Degree of

DOCTOR OF PHILOSOPHY

UNIVERSITY OF NORTH TEXAS

May 2009

APPROVED:

Scott J. Warren, Major Professor

Kathleen Whitson, Minor Professor
Cathleen Norris, Committee Member
Jeff Allen, Interim Chair of the Department of
Learning Technologies
Herman L. Totten, Dean of the College of
Information
Michael Monticino, Interim Dean of the Robert
B. Toulouse School of Graduate
 Dondlinger, Mary Jo. The Global Village Playground: A qualitative case study of

designing an ARG as a capstone learning experience. Doctor of Philosophy

(Educational Computing), May 2009, 183 pp., 30 tables, 7 illustrations, references list,

131 titles.

The Global Village Playground (GVP) was a capstone learning experience

designed to address institutional assessment needs while providing an integrated,

contextualized, and authentic learning experience for students. In the GVP, students

work on simulated and real-world problems as a design team tasked with developing an

alternate reality game that makes an impact on the United Nations Millennium

Development Goals. The purpose of this study was to evaluate the effectiveness of the

design of the GVP as a capstone experience. The research design follows a qualitative

case study approach to gather and analyze data collected from the instructors and

students participating in the pilot implementation of the GVP. Results of the study show

predominantly favorable reactions to various aspects of the course and its design.

Students reported to have learned the most through interactions with peers and through

applying and integrating knowledge in developing the alternate reality game that was the

central problem scenario for the course. What students demonstrated to have learned

included knowledge construction, social responsibility, open-mindedness, big picture

thinking, and an understanding of their relationship to the larger society and world in

which they live. Challenges that resulted from the design included the amount of

necessary to build consensus and then develop an overarching game concept, the

tension between guided and directed instruction, and the need to foster greater

interdependence among students while encouraging them to become more self-

directed.
 Copyright 2009

by

Mary Jo Dondlinger

ii
 ACKNOWLEDGEMENTS

I am deeply grateful for the efforts of several inspiring, supportive and

innovative people without whom I could not have completed this project.

The students who participated in this study from whom I learned much

more than I taught.

My co-instructor for her wisdom, dedication to students, and willingness to

take risks.

My committee, who guided and encouraged me, particularly committee

chair, Dr. Scott J. Warren, for his compassion, creativity, and confidence.

My fellow students and researchers, Chris Bigenho, Anjum Najmi, Julie

McLeod, and especially Douglas Wilson for his open mind, positive energy, and

laborious efforts.

Most importantly, my husband, Russell L. Dondlinger, and children,

Charlene and Taylor Dondlinger, who have supported and inspired me more than

words can adequately express.

iii
 TABLE OF CONTENTS

ACKNOWLEDGEMENTS ..................................................................................... iii

LIST OF TABLES ................................................................................................. vi

LIST OF ILLUSTRATIONS ................................................................................. viii

CHAPTER 1: INTRODUCTION ............................................................................ 1

Introduction ................................................................................................ 1
Statement of the Problem .......................................................................... 4
Solution ...................................................................................................... 7
Purpose of the Study ................................................................................. 8
Research Questions .................................................................................. 9
Definition of Terms ..................................................................................... 9
Limitations ............................................................................................... 11
Overview of Dissertation .......................................................................... 12

CHAPTER 2: LITERATURE REVIEW ................................................................ 14

Introduction .............................................................................................. 14
Characteristics of Effective Digital Games ............................................... 15
Learning Outcomes Attributed to Educational Games ............................. 22
Learning Theories Supporting Use of Games for Instruction ................... 26
Alternate Reality Games .......................................................................... 30
Learning by Designing Digital Games ...................................................... 33
Summary ................................................................................................. 35

CHAPTER 3: DESIGN METHODOLOGY .......................................................... 36

Introduction .............................................................................................. 36
Learning Communities ............................................................................. 37
Identification of the Learning Objectives .................................................. 40
Alignment of Objectives with Instruction .................................................. 44
Overview of Resulting Course Design ..................................................... 53
Summary ................................................................................................. 57

iv
CHAPTER 4: RESEARCH METHODOLOGY .................................................... 58

Introduction .............................................................................................. 58
Philosophy of Inquiry................................................................................ 59
Educational Evaluation ............................................................................ 61
Context of this Study ................................................................................ 65
Data Collection ........................................................................................ 73
Data Analysis ........................................................................................... 74
Summary ................................................................................................. 78

CHAPTER 5: RESULTS ..................................................................................... 79

Introduction .............................................................................................. 79
Analysis Procedures ................................................................................ 80
Description of Themes ............................................................................. 82
Reaction Theme ...................................................................................... 87
Learning Theme ....................................................................................... 97
Transfer Theme ..................................................................................... 113
Results Theme....................................................................................... 130
Summary ............................................................................................... 153

CHAPTER 6: CONCLUSIONS & IMPLICATIONS............................................ 154

Introduction ............................................................................................ 154

Conclusions ........................................................................................... 154
Implications ............................................................................................ 165
Directions for Future Research .............................................................. 169

REFERENCES ................................................................................................. 172

v
 LIST OF TABLES

Table 3.1 Objectives & activities within the framework of the design process .... 46

Table 5.1 Amount of text assigned to categories within each theme .................. 80

Table 5.2 Passage/Character mean weights for themes and categories............ 86

Table 5.3 Codes in Instructional Design category .............................................. 88

Table 5.4 Codes in Instructors/Learning Environment category ......................... 91

Table 5.5 Codes in Overall Reaction category ................................................... 92

Table 5.6 Codes in Technology category ........................................................... 93

Table 5.7 Codes in Prior Experience category .................................................. 94

Table 5.8 Codes in Hybrid/ALCE category ......................................................... 95

Table 5.9 Codes in Peers/Students category ..................................................... 96

Table 5.10 Codes in Self-Reflection category .................................................... 99

Table 5.11 Codes in Teamwork category ......................................................... 102

Table 5.12 Codes in Game Scenario category ................................................. 104

Table 5.13 Codes in Prior Knowledge category................................................ 107

Table 5.14 Codes in Course Content category ................................................. 109

Table 5.15 Codes in Technology category ....................................................... 110

Table 5.16 Codes in Motivation category ......................................................... 112

Table 5.17 Codes in Instructors category ......................................................... 113

Table 5.18 Codes in Knowledge Construction category ................................... 115

vi
Table 5.19 Codes in Respect for Others category ............................................ 122

Table 5.20 Codes in Individual/Personal Values category ................................ 125

Table 5.21 Codes in Open-mindedness category............................................. 126

Table 5.22 Codes in Social Responsibility category ......................................... 129

Table 5.23 Codes in Instructional Methods category ........................................ 134

Table 5.24 Codes in Students category ............................................................ 140

Table 5.25 Codes in Curriculum & Assessment category ................................. 145

Table 5.26 Codes in Technology category ....................................................... 147

Table 5.27 Codes in Epistemology category .................................................... 149

Table 5.28 Codes in Course Format category .................................................. 152

Table 5.29 Codes in Institutional category ........................................................ 152

vii
 LIST OF ILLUSTRATIONS

Figure 4.1 Learn Lab classroom configuration.................................................... 67

Figure 5.1 Composition of themes ...................................................................... 85

Figure 5.2 P/C mean percentage of text in Reaction theme by category ............ 87

Figure 5.3 P/C mean percentage of text in Learning theme by category ............ 97

Figure 5.4 P/C mean percentage of text in Transfer theme by category .......... 114

Figure 5.5 P/C mean percentage of text in Results theme by category ............ 130

Figure 5.6 Structure of the game and corresponding core perspectives .......... 136

viii
 CHAPTER 1

INTRODUCTION

In the information age, the need to develop in learners the higher order

thinking skills that translate into real-world problem-solving ability is more urgent

than ever before (Secretary's Commission on Achieving Necessary Skills, 1991;

The Safflund Institute, 2007). As early as 1991, the Secretary of Labor’s

Commission on Achieving Necessary Skills found that basic skills in reading,

writing, and mathematics were the “irreducible minimum for anyone who wants to

get even a low-skill job” but those skills were not a guarantee to either a career or

access to higher education. Employer surveys continue to emphasize “thinking

skills . . . [that] permit workers to analyze, synthesize, and evaluate complexity”

are requisite to success in the global workplace (p. 14).

As global economic pressures mount, so does the need to produce a

competitive and talented workforce. Additionally, the accountability movement in

American education is driving educational institutions at all levels to examine

what learning should occur at their institutions, devise means to measure that

learning, and seek to continually improve the processes that have an impact on

this learning (U.S. Department of Education, 2002). Data from a recent national

survey initiated by the American Association of Colleges & Universities indicates

that employers are dissatisfied

1
with assessment test-scores, grade-point-averages, institution ratings, and other

numerically quantified scores of completer success. Instead, they call for

“faculty-evaluated internships and community-learning experiences” as well as

“essay tests, electronic portfolios of student work, and comprehensive senior

projects” which provide means for students to develop “real-world skills” as well

as demonstrable products of student performance in problem-solving and

readiness for the workplace (Peter D. Hart Research Associates, 2008, p. 1).

More specifically, the employers surveyed called for undergraduate learning

experiences that foster the following:

• Engagement with big questions

• Critical and creative thinking about complex problems
• Active involvement in diverse communities and real world challenges
• Application of knowledge and skills in diverse settings and innovative
ways (Peter D. Hart Research Associates, 2008)

In light of this report, the focus of instruction needs to become one that allows

large-scale problem solving and compels a deliverable product that can then be

evaluated by agencies outside of academia.

In his recent book, A Whole New Mind, Daniel H. Pink (2006) compellingly

argues that competitive success in the conceptual age requires a new mindset,

one infused with creative or artistic thinking.

The past few decades have belonged to a certain kind of person with a
certain kind of mind—computer programmers who could crank code,
lawyers who could craft contracts, MBAs who could crunch numbers. But
the keys to the kingdom are changing hands. The future belongs to a very
different kind of person with a very different kind of mind—creators and

2
 empathizers, pattern recognizers, and meaning makers. These people—
artists, inventors, designers, storytellers, caregivers, consolers, big picture
thinkers—will now reap society’s richest rewards. (Pink, 2006, p. 1)

While the sequential, detail- and text-oriented thinking vital to the occupations of

the information age is still important, Pink asserts that simultaneous, big picture,

context-oriented thinking is requisite to success in the conceptual age. Thus,

creating a problem-solving experience wherein students engage in the process of

designing is a potential means to foster this way of thinking.

However, as a recent National Science Foundation (NSF) sponsored

report suggests, educational systems continually have to do more with less.

Although employers are demanding these additional skills, learning institutions

have to instill those skills without adding additional credit hours or courses to

their programs (Safflund Institute, 2007). The means to achieving this end then is

through changing instructional strategies in existing courses, and providing that

vital added value through communications technologies, simulations, and other

forms of digital media.

In community colleges, assessment of student learning outcomes takes

place at the program level for career and technical programs and at the

department or discipline level for an academic curriculum. However, enrollment

at community colleges in general education curricula for the purpose of transfer

to a four-year university has increased dramatically over the past decade and a

half (Blom, 2001; Hoachlander, Sikora, & Horn, 2003; Sturtz, 2006). Along with

these increases are parallel pressures to assess student learning in general

3
education programs at an institutional level rather than at the department or

discipline levels at which it has been assessed in the past (Southern Association

of Colleges and Schools, 2007). In career and technical programs, which are

largely performance-based, measuring student attainment of learning outcomes

is not as difficult as it can be in academic programs which are based on

intellectual competencies: the distinction being between observable performance

and unobservable thinking patterns. Moreover, career and technical programs

usually culminate in a summative learning experience such as an internship,

practicum, service learning project, or problem-based capstone course. Such

program capstones provide not only a summative application of the knowledge

and skills attained throughout a program, but also a venue through which to

assess the effectiveness of the program. However, another difficulty in assessing

student learning in academic transfer or general education curricula is that such

programs neither require students to complete the curriculum in any particular

order nor culminate in a capstone experience at the end. While courses in some

disciplines are sequenced by prerequisites (e.g. English, math, and science), the

curriculum as a whole is not sequenced, making it difficult to identify a course or

set of courses that could be designated as capstones.

Statement of the Problem

The Texas Higher Education Coordinating Board (THECB) has designated

core perspectives, the recommended learning goals of the core curriculum

(Texas Higher Education Coordinating Board, 1999). The task at hand for a

4
large, urban, community college in the southwest is to find a means to measure

it. To this end, the college assembled an Institutional Capstone Committee (ICC)

with a five-year timeline for developing an institutionalized means of assessing

attainment of the THECB core perspectives in students who complete the core

curriculum at the college.

The ICC determined that an electronic portfolio to be evaluated by

members of that committee would serve this purpose. The use of portfolios to

assess student learning outcomes that result from engagement in courses as

well as entire programs of study has become a popular approach (Barrett, 2007;

Juniewicz, 2003). However, the proposed portfolio presents a number of

challenges rather than solutions to the issue at hand: assessing attainment of

core perspectives in order to improve instructional processes.

Assessment vs. Experience

The proposed portfolio is a capstone assessment rather than a capstone

experience. It artificially layers an additional product on top of student

coursework, rather than creating a capstone experience, integrating the learning

from the multiple disciplines that comprise the core curriculum. The college could

compel completion of the portfolio as a requirement for graduation; however,

research has shown that for a portfolio assessment to be effective, the means by

which it is implemented must provide scaffolding and feedback to learners

throughout the portfolio creation process (Segers, Gijbels, & Thurlings, 2008;

Van Tartwijk, Driessen, Van Der Vleuten, & Stokking, 2007). In other words, a

5
means to provide this scaffolding and feedback or to integrate the assessment

into student experience must be developed as well.

Validity of Results

Moreover, if the portfolio is implemented solely as mandated requirement

for graduation, it is not likely to yield the desired results. On the one hand,

students not wishing to take on this additional academic task could instead

complete the core curriculum at another institution where there is no such

mandate, an option readily available in a large, multi-college district. On the other

hand, students could opt not to complete the transfer core in its entirety, and

simply complete as many courses as possible within the institutional mandate

and then transfer directly to the university. Thus, those students who chose to

complete the portfolio would not likely constitute a representative sample of core

completers. Rather, they would represent a group that either is not aware that

other options exist or that atypically rises to the challenge of additional work.

Core Perspectives and Quality Improvement

While the items to be included in the portfolio address some aspects of

some of the THECB core perspectives, they do not fully address all of them. For

example, samples of student writing from a given sociology course might

demonstrate the second perspective, “stimulate a capacity to discuss and reflect

upon individual, political, economic, and social aspects of life in order to

understand ways in which to be a responsible member of society“ (Texas Higher

6
Education Coordinating Board, 1999), it does not necessarily do so. Moreover,

the goal of such an assessment is to measure both the achievement of outcomes

and to identify areas of weakness so that they can be improved. If better

designed, the portfolio could help identify which perspectives students are not

attaining well, but it does not illuminate what areas in the core curriculum need

further improvement.

Solution

One approach to addressing these issues is to develop a problem-based,

capstone experience that allows students to put knowledge gained across the

core curriculum to work as they develop solutions to problems (Albanese &

Mitchell, 1993). The Global Village Playground (GVP) was a capstone learning

experience designed to address these institutional assessment needs while

providing a more integrated, contextualized, and authentic learning experience

for students than the creation of a portfolio alone. In the GVP, students work on

simulated and real-world problems as a design team tasked with developing an

alternate reality game (ARG, see definition in Definition of Terms) (Martin &

Chatfield, 2006) that makes an impact on the United Nations Millennium

Development Goals (United Nations, 2005). This design project was intended to

simulate a real-world work scenario in which students collaborate to create a

deliverable product that meets the specifications of a client agency. The

scenario compels students to interact with authentic work problems, such as

communicating effectively with members of small and large groups, managing a

7
project timeline, and solving problems collaboratively. It also requires students to

engage with global issues and devise strategies that will address them, central

goals of the core perspectives (Texas Higher Education Coordinating Board,

1999), as well as the conceptual age thinking described briefly above (Pink,

2006).

Purpose of the Study

The underlying premise of the GVP as an instructional design solution is

that immersing community college students nearing completion of the general

academic transfer program in a large-scale, collaborative design project as a

capstone experience will have a perceived impact on their attainment of the

THECB core perspectives (Texas Higher Education Coordinating Board, 1999).

The purpose of this study was to evaluate the effectiveness of the design of the

GVP as a capstone experience. The research design follows a qualitative case

study approach to gather and analyze data collected from the instructors and

students participating in the pilot implementation of the GVP. Although the GVP

was designed to provide a means to evaluate the academic transfer studies

program as a whole, this study focused specifically on evaluating the design of

the GVP, the effectiveness of the problem-based instructional methods, and the

tensions that arise from implementing them.

8
 Research Questions

The effectiveness of many educational innovations is evaluated by student

achievement of the learning outcomes targeted by the instructional design

intervention. This study does not ignore that precedent or its urgency. However, it

follows a design-based research approach, focusing first on evaluating the pilot

implementation in order to improve the design prior to a more general evaluation

of its impact on student achievement. More specifically, this study addresses the

following research questions:

1. What are student and instructor reactions to the course/course

design?

2. What aspects of the design were most conducive to student

learning?

3. To what extent did the course promote attainment of the

overarching program objectives (core perspectives) and/or
advance conceptual age thinking?

4. What challenges or tensions arose from the design?

While this study is not intended as an evaluation of the entire academic transfer

program, it should inform institutional stakeholders about the viability of this

particular capstone as a means to that end.

Definition Of Terms

Alternate reality game (ARG): a relatively new genre of multi-player online

games, the ARG distributes game challenges among a variety of online

media, as opposed to game formats that make use of a single, stand

alone game world. For example, game play might begin with examination

9
 of a web site that directs players to a YouTube video, which in turn takes

them to an online news article. Each of these sites requires the player to

piece together parts of an overarching narrative puzzle that moves from

place to the next. Because they make use of spaces created specifically

for the game as well as “real” phenonmena (such as the news article

mentioned above), ARGs blur the boundaries between reality and fiction,

creating realities that are alternate to but not entirely separate from every

life.

Augmented reality game: Similar to alternate reality games, augmented reality

games distribute the game space among a variety of media. However,

augmented reality games differ in that they include on-the-ground settings

in the play space in addition to virtual sites.

Case study: A research design that investigates a single, bounded instance of a

phenomenon in its entirety. Case study research designs typically include

qualitative methods of data collection and analysis but may use mixed

methods approaches, blending qualitative with quantitative measures.

Game: A game is an interactive system characterized by an artificial conflict or

win scenario that is bounded by rules and a quantifiable outcome (Salen &

Zimmerman, 2004). In order to attain a “win,” players must follow the rules

as they overcome obstacles in pursuit of game goals.

Implementation: The enactment or execution of an instructional design for a

specified period of time.

10
Problem-based learning (PBL): An instructional methodology within the social

constructivist learning paradigm, the central component of problem-based

learning is an authentic, ill-structured problem which is posed to groups of

learners who develop a socially negotiated problem solution. In PBL

environments, the instructor’s role is to facilitate the problem-solving

process, guide learners to resources, and enable knowledge construction

by posing cognitively-challenging questions rather than providing definitive

answers.

Limitations of This Study

This study does pose limitations to conclusions that may be drawn from it

and applied to the body of knowledge regarding teaching and learning. First, the

author and researcher for this study was also the primary designer of the

instructional design solution, the Global Village Playground, as well as one of the

two instructors who taught the course. These multiple roles provide additional

insight into the research questions, but they also compromise claims to objective

distance from the case under study.

The number of participants in the study also limits the assertions that can

be made from it. Although eight students enrolled in the course, only six

completed the class. Two students withdrew around the fourth week of the

semester. Moreover, the course was designed as a capstone for the academic

transfer program at the college of implementation. However, three of six

participants were technical program students in the Interactive Simulation and

11
Game Design program (ISGT). Consequently, those participants had not been

exposed to the full range of general education courses that other participants had

experienced.

Additionally, this study does not intend to “control for” prior knowledge and

skills attained throughout the array of general education courses, in order to

make claims about what learning occurred in this course. The nature of a

capstone course requires that learners integrate and build upon prior knowledge.

At best, conclusions can be drawn about what learning goals the institution might

need to place more instructional emphasis on in all courses; however, the

methods of data collection and analysis for this research design to not lend

themselves to such conclusions.

Finally, because this research design does not compare the GVP with

other capstone course designs, the results cannot support claims that this

specific design scenario is the best or the only means to engage students in a

meaningful capstone experience. Conclusions will be limited to assertions

regarding the relative success or failure of problem or project based methods as

the foundation for meaningful capstone experiences.

Overview of the Dissertation

The next chapter reviews the relevant literature on designing games for

learning. An understanding of what makes an effective educational game is

imperative to understanding the knowledge and skills required and acquired in

the process of designing one. The review also explores the learning theories

12
supporting the use of games for instruction and the learning outcomes that have

been attributed to digital learning games. Finally, it examines an emerging digital

game genre, the alternate reality game (ARG), and the relatively nascent

research on learning by designing digital games.

The third chapter provides a description of the GVP capstone design,

beginning with the front-end analysis of the learning outcomes, the alignment of

those outcomes with the problem scenario, and the selection of course content

based on this specific problem scenario. The chapter also details the

collaborative process employed in developing course materials and the resulting

course design, which differed somewhat from the original intentions of the

primary instructional designer.

The fourth chapter explains the research methods used to evaluate the

design, including the primary means of data collection and analysis as well as

their appropriateness for this study. The fifth chapter presents the results of the

study, aligning assertions grounded in the data with the research questions that

framed this evaluation. The final chapter explores the implications of the results

and discusses directions for future research.

13
 CHAPTER 2

LITERATURE REVIEW

Introduction

The purpose of the Global Village Playground (GVP) design was to

immerse community college students who are nearing completion of the

academic core curriculum in a large-scale, collaborative design project as a

capstone experience, which situates them in the role of game designers,

developing an alternate reality game (ARG) that makes an impact on the United

Nations Millennium Development Goals (UN MDGs), the “big questions” with

which this design engages learners. The use of games to promote learning is not

a new instructional strategy (Aguilera & Mendiz, 2003); however, the surge of

interest in digital games and simulations for learning recently stems in part from

their ability to captivate players to voluntarily engage in game play for hours on

end. Coupled with this high level of engagement, digital learning games are also

thought to situate learning in contexts that better reflect the real world and to

immerse players in challenging learning experiences. But what characteristics

make a digital game engaging? What can players learn from digital games?

What learning theories support the use of games for learning? And finally, can

learners acquire similar or additional knowledge, skills, and abilities by designing

digital games that they do from playing them? In order to address the research

14
questions for this study, these questions and the research literature that offers

answers to them first need to be explored.

Characteristics of Effective Digital Games

Much research has examined the motivating quality of video games;

however, not all researchers entirely agree on the source of this motivation.

Some attribute the compelling nature of games to their narrative context (Dickey,

2005; Fisch, 2005; Waraich, 2004), others find that motivation is linked to goals

and rewards within the game itself or intrinsic to the act of playing (Amory,

Naicker, Vincent, & Adams, 1999; Denis & Jouvelot, 2005; Dickey, 2006;

Jennings, 2001). Nevertheless, all find that motivation to play is a significant

characteristic of educational video games and that effective game design

considers both intrinsic and extrinsic rewards for play. Denis and Jouvelot (2005)

distinguish between the two and their absence as follows:

Intrinsic motivation pushes us to act freely, on our own, for the sake of it;

extrinsic motivation pulls us to act due to factors that are external to the

activity itself, like reward or threat; amotivation denotes the absence of

motivation. (p. 462)

These authors see motivation as the interplay between desire and pleasure—the

desire to be competent and the pleasure one feels when one is. They argue that

competence, autonomy, and relatedness are factors that affect motivation.

“Motivation also leads to the activation of efficient cognitive strategies for long-

15
term memory issues like monitoring, elaborating or organizing information. On

the opposite side, resignation and amotivation have negative results on

memorization and personal development” (p. 463).

Amory, Naicker, Vincent, & Adams (1999) examined four different game

types and analyzed elements that players liked most. In this study, students

rated a number of game qualities including the “the fun aspect, sounds and

graphics, type of game, game story and use of technology”; “the importance of

some skills [logic, memory, visualisation, and mathematics, reflexes, and

problem solving]”; “whether the game was easy to play, addictive, too long,

challenging, confusing, too difficult, illogical, difficult to play or manoeuvre and if

their performance increased with continuous play” (p. 314). Adventure and

strategy games were found to be the most stimulating and rated the highest, a

finding which suggests that players preferred or were more motivated to play

games with objectives that require higher order thinking skills, including

visualization strategies that nurture creative problem solving and decision-

making. (p. 317).

On a similar note, Dickey (2006) argues that a narrative context that

promotes “challenge, fantasy, and curiosity” and that provides feedback for

players is one that promotes intrinsic motivation for play (p. 2). She also finds

that “Strategies of design that lead to engagement may include role-playing,

narrative arcs, challenges, and interactive choices within the game as well as

interaction with other players“ (p. 1). In another study, Waraich (2004) agrees

16
that narrative is essential to motivation but cautions that “intrinsic rewards are

based on a high congruence between the material being taught and the

motivational techniques used” (p. 98). Dissonance between the two can

decrease learning.

Narrative Context

Disagreement on the source of motivation aside, a general consensus that

narrative context is an important element of effective digital game design does

exist. Several studies support this finding and deal most prominently with

narrative as a significant design element. In two studies on game-like

environments for learning, Michele Dickey (2006) finds that 3-D learning

environments not only provide a narrative context for situating and

contextualizing learning, they also enable an understanding of spatial

relationships rather than linear ones. In an article on the design of Murder on

Grimm Isle, a game created to cultivate argumentative writing skills, Dickey

(2006) concludes that spatial and narrative contexts offer learners “a cognitive

framework for problem-solving because the narrative storyline in games provides

an environment in which players can identify and construct causal patterns which

integrates what is known (backstory, environment, rules, etc.) with that which is

conjectural yet plausible within the context of the story“ (p. 2). She presents

similar findings in case studies of two 3-D environments for courses in business

computing and 3-D modeling, arguing that contextual elements such as a first

person symbolic perspective and 3-dimensional representations of space

17
increase learners’ sense of presence and consequently their interaction and

collaboration. “This [narrative] context builds on learners’ real-world knowledge

by providing a visual metaphor, or perhaps more aptly stated, a visual narrative

of the course content” (Dickey, 2005, p. 444).

In a study of a variety of design elements on game environments for

instruction in computer science architecture, Waraich (2004) focuses mainly on

narrative. This empirical study analyzed the role of both narrative context and

game goals as features for motivating and conceptualizing learning in a 2-D ILE

(interactive learning environment). The mixed methods design of the study

revealed quantifiable knowledge gains in the ILE over traditional instruction.

Waraich concludes that “For any learning task to be meaningful to the learner

they must have both a sufficient context for the learning and motivation to

perform the tasks that will help them to learn. We believe that game based

learning environments that incorporate a strong narrative can meet these

requirements if the learning tasks are appropriately designed and tightly coupled

with the narrative” (p. 98).

In what is largely a theoretical discussion rather than a research study,

Fisch (2005) makes a similar observation. Although narrative context does

motivate learning, for an educational game to be effective, the learning content

must align with the narrative plotline. According to Fisch’s analysis, “research on

lectures and textbook readings has suggested that seductive details do not work;

children exposed to such material tend to remember the appealing elements but

18
not the intended educational content” (p. 57). He finds that a “far more powerful

approach is to place the educational content at the heart of engaging game play,

so that children employ the targeted academic skills and knowledge as an

integral part of playing the game” (p.58). Fisch also maintains that selecting

appropriate media as well as providing feedback and scaffolding within and

outside of the game are essential to effective educational game design.

Goals and Rules

Another significant element of effective video game design is a system of

objectives, goals, and rules of play (Salen & Zimmerman, 2004; Waraich, 2004;

Zagal, Nussbaum, & Rosas, 2000). Although they are integrated within a

narrative context, goals and rules are not subordinate to context; they are equally

important elements of it. In an overview of initiatives in educational games,

Jenkins, Klopfer, Squire, and Tan (2003) describe the design and testing of three

video game prototypes: Supercharged!, a game on electromagnetism;

Environmental Detectives, an environmental science game; and Revolution, a

game for American history. Each has a narrative structure that students follow to

determine their objectives or goals. Playing the role of a charged particle in

Supercharged!, a scientist in Environmental Detectives, and a soldier,

revolutionary, or townsperson in Revolution, each game has distinct objectives

and a variety of rules frame the play. For example, the laws of electromagnetism

provide the rules in Supercharged! Players must master the rules of the game to

accomplish their objectives.

19
 Swartout and van Lent (2003) further elaborate on goals in effective video

games, finding that goals of different levels help motivate learners to continue

play. “Game designers often seek to keep players engaged by creating three

levels of goals: short-term (collect the magic keys), lasting, perhaps, seconds;

medium-term (open the enchanted safe), lasting minutes; and finally, long-term

(save the world), lasting the length of the game” and that the “interplay of these

levels, with the support of the environment, is crafted to draw players into the

storyline of the game” (p.34). This design concept is similar to Gee’s (2003)

achievement principle which states that “for learners of all levels of skill there are

instrinsic rewards from the beginning, customized to each learner’s level, effort,

and growing mastery and signaling the learner’s ongoing achievement” (p. 67).

Interactivity and Multisensory Cues

Interaction between the player (or players) and the game environment is

another game element embedded in the narrative context and game objectives.

Effective games weave objects and characters into a game environment that

provide feedback and hint structures for successful game play (Fisch, 2005).

Moreover, the degree of user control over the game environment further

constitutes the level of interactivity. Swartout and van Lent (2003) deem that the

best games are “highly interactive, deliberately generating tension between the

degree of control the story imposes and the player’s freedom of interaction” (p.

34), reasoning that in games with complete freedom of interaction, the playing

experience can be boring and unchallenging. On the other hand, when the

20
plotline imposes too much control, the player becomes a passive observer rather

than an active participant. Providing a balance to these extremes by “cleverly

exploiting the narrative to shape the players’ experience,” effective game design

gives players “the perception they have free will, even though at any time their

options are actually quite limited” (p. 34). James Paul Gee (2003) calls this

concept the regime of competence principle which states that the player/learner

“gets ample opportunity to operate within, but at the outer edge of, his or her

resources, so that at those points things are felt as challenging but not

‘undoable’” (p. 71).

In an overview of the design process and the various elements of

multiplayer games, including rules and goals, props and tools, Zagal, Nussbaum,

and Rosas (2000) also examine the role of interactivity as a critical element in

effective games, proposing that game designers should consider the extent to

which the game rules, props, and tools affect stimulated and natural social

interaction. “Stimulated (or forced) social interaction occurs when the rules of a

game encourage the players to interact socially” while “natural social interaction

occurs when the players spontaneously decide to interact. The game rules do

not enforce this type of activity; it just happens” (p. 451). Such interactions might

depend on cooperation, competition, or a combination of both. They might also

require synchronicity or coordination, types of interactions, which are determined

by player composition in the game. The article also includes a model for

analyzing player composition and social interaction in game design.

21
 A study by Salzman, Loftin, Dede, and McGlynn (1996) further confirms

that multisensory cues are a significant component of successful game-like

environments. These researchers conclude that “multisensory cues can engage

learners, direct their attention to important behaviors and relationships, help them

understand new sensory perspectives, prevent errors through feedback cues,

and enhance ease of use” (p. 2). While learning outcomes afforded by gaming

media will be discussed in greater depth later in this review, another significant

finding in this study is that multisensory interactions “can help learners

understand complex phenomena,” particularly for students with “severely limited

or inaccurate mental models of science concepts” (p. 2).

Learning Outcomes Attributed To Educational Games

21st Century Skills

In a historical review of the research on video game design, Aguilera and

Mendiz (2003) maintain that “arguments in favor of the cognitive importance of

video games are based on a number of studies indicating that many video games

are conducive to the development of specific skills: attention, spatial

concentration, problem-solving, decision-making, collaborative work, creativity,

and, of course, ICT skills” (p. 8). Many of these skills are earmarked as those

necessary to successfully participate in the global, knowledge based economy of

the 21st century. Employing rather cursory case studies of specific games and

anecdotal comments from young video game players as evidence of his

22
assertions, Marc Prensky (2006) contrasts the nature of digital immigrants (those

who have recently migrated to the use of digital technology) to that of digital

natives (those who have grown up with it). Although Prensky is not an

educational researcher, he is a widely acclaimed speaker and writer on how

complex video games teach digital natives in ways not offered by traditional

instruction. The most significant of his ideas are his description of complex

videos games and the 21st century skills that game play can impart.

Schrier (2006) designed an augmented reality game (see definition in

Chapter 1) designed specifically to foster 21st century skills, including

“interpretation, multimodal thinking, problem solving, information management,

teamwork, flexibility, civic engagement, and the acceptance of diverse

perspectives.” The research design employed a mixed-methods approach which

included pre- and post attitudinal surveys, interviews, and video taped learning

sessions and yielded results which suggest that problem-based learning

augmented with game-like design features can indeed encourage the

development of these skills. Since the Schrier’s research did not employ an

experimental design comparing the augmented reality treatment to a control

group, generalizations about learning gains in such the game-like environment

over a traditional classroom were not supported by this study.

23
 Deduction and Hypothesis Testing

The results of a variety of studies suggest that video games and game-like

environments are conducive to deductive reasoning and hypothesis testing

(Aguilera & Mendiz, 2003; Gee, 2003; Jenkins et al., 2003; Klopfer & Yoon, 2005;

Lunce, 2006; Salzman, Dede, & Loftin, 1999). In a qualitative analysis of both

what and how students learned playing Civilization III in a interdisciplinary

history, humanities, and social studies course Barab and Squire (2004) found

that game play promoted deep learning, hypothesis testing, strategizing, and

appropriating content (history) as a tool for play. Squire, Barnett, Grant and

Higginbotham (2004) established that students in an experimental group who

played the simulation-game, Supercharged!, better mastered the abstract and

conceptual knowledge related to electromagnetism than those in the control

group who learned through guided discovery-based science methods. The

researchers attribute these learning gains to replay for testing new hypotheses

afforded by the simulation game.

Complex Concepts and Abstract Thinking

Other studies build on the findings of Squire, Barnett, Grant, and

Higginbotham concerning mastery of abstract and conceptual knowledge through

game play (Aguilera & Mendiz, 2003; Gee, 2003; Lunce, 2006; Prensky, 2006).

Writing about technology in general rather than games specifically, Kelly (2005)

argues that technology applications including video games promote mastery of

complex concepts. In a qualitative case study of the game-like computer-

24
modeling environment, StarLogo, Klopfer and Yoon (2005) discovered that

students who “struggle with understanding the dynamics of complex systems”

were able to better comprehend these systems after working with StarLogo.

Visual and Spatial Processing

Because most complex video games are situated in 2- or 3-dimensional

environments, it is no surprise that research has found increased spatial

development in video game players. According to Aguilera and Mendiz (2003),

“adolescents with medium- or long-term experience playing video games show

greater visual capacity, motor activity, and spatial abilities-reflexes and

responses” (p. 6). Using game engines to render and then explore the effects of

architectural designs, Burrow and More (2005) observed that the capabilities of

game-engines “allow participants to experience the spatial design in ways that

are not predetermined by the designer” (p. 35). The objective of their project was

to explore the relationship between architectural design elements and

atmosphere, analyzing both the atmosphere produced by the architectural design

and the impact of atmosphere on the design. Burrow and More argue that this

focus “emphasizes critical thinking on the nature of space and its representation,

from the visual and sonic through lighting, textures, and sound, to the nature of

the movement through space, and its interactivity” (p. 38).

25
 Learning Theories Supporting Digital Games for Instruction

Situated Cognition

In a symposium on learning theories for the analysis of educational video

games, Halverson, Shaffer, Squire, and Steinkuehler (2006) assert that situated

cognition provides a meaningful framework for the study of games, given their

ability to situate learning in an authentic context and engage players in a

community of practice. According to these and other scholars, the authentic,

situated context affords greater content mastery and transfer of learning to real

world settings than traditional classroom learning (Aldrich, 2003; Dickey, 2005,

2006; Heinich, Molenda, Russell, & Smaldino, 1999; Klopfer & Yoon, 2005;

Pearce, 1997; Schrier, 2006). According to Lave and Wenger (1991) and

Bransford et al (2003), situating learning in relevant environmental contexts

provides learners with cognitive scaffolds that are expected to increase levels of

learning, engagement, and transfer to future work. Methods that anchor

instruction to meaningful and authentic contexts have been found to better allow

learners to understand and transfer complex concepts than instructional methods

which neglect to convey how, when, and where a concept can be applied in

future situations that the learner might encounter (Cognition and Technology

Group at Vanderbilt, 1990, 1993). Such methods are also thought to immerse

learners in a community of practice, wherein they perform the roles of a

practitioner rather than a learner—functioning, for example, as a scientist rather

than a student in order to solve scientific problems, as opposed to the often

26
decontextualized student challenges of completing an assignment or passing a

test (Brush & Saye, 2001). Indeed, technological advances in computer-

generated media have allowed the creation of immersive virtual environments

that graphically represent reality more closely than ever and to pre-program

these environments for almost instantaneous feedback based on parameters

observed in reality. This learning affordance (Gibson, 1977) allows users to

manipulate variables, adjust their actions or behaviors, and experiment with

various processes or procedures.

Constructivism

Several researchers find that learning with well-designed video games

adheres to constructivist principles (Dede, Nelson, Ketelhut, Clarke, & Bowman,

2004; Dickey, 2005, 2006; Gee, 2003; Schrier, 2006). In an article describing the

multi-user virtual world, SciCtr, Margaret Corbit (2005) underscores the merits of

a constructivist approach for analyzing game-like environments. In SciCtr,

students create virtual science worlds, such as a rainforest or a desert, that other

learners can visit and explore. According to Corbit, these worlds, the paths to

navigate through them, and content embedded in them are constructed by the

developer/learner through meticulous research and thoughtful design. Dede,

Nelson, Ketelhut, Clarke, and Bowman (2004) outline both constructivist and

situated learning design principles present in effective video games including

GST (guided social constructivist design), EMC (expert modeling and coaching)

and LPP (legitimate peripheral participation). These authors employ such

27
principles in evaluating game design and apply their findings to future iterations

of the design.

Problem-Based Learning

A subset of constructivist learning theory, problem-based learning (PBL)

approaches can also provide a useful framework for understanding the value of

games for learning. The central feature of PBL environments is an authentic, ill-

structured problem, which is posed to groups of learners who develop a socially

negotiated problem solution. This approach framed the designs and research

related to River City and Taiga, learning game designs for science inquiry, both

of which document learning gains over other more traditional approaches to

instruction (Barab & Squire, 2004; Nelson, Ketelhut, Clarke, Bowman, & Dede,

2005). The PBL approach was also applied in Murder on Grimm Isle and

Anytown, game-like environments designed to further writing practice (Dickey,

2006; Warren, Stein, Dondlinger, & Barab, 2009). As cited earlier, Dickey (2006)

finds that the “narrative in games provides a cognitive framework for problem-

solving” (p. 2). According to Warren (2006), the narrative in combination with

additional scaffolding from non-player characters serving as pedagogical agents

facilitate the problem-solving process and guide learners’ knowledge

construction. PBL was the underlying learning theory that drove the design of

The Door, an alternate reality game (ARG) for a post-secondary computer

applications course (Warren & Dondlinger, 2008a). While the research on this

design is discussed in more detail later, what is unique about The Door,

28
compared to other learning game designs reviewed here, is the application of

PBL and game elements for learning at the post-secondary level. The majority of

the literature on games for learning studies their impact in K-12 settings.

Constructionism

Designing and developing video games, rather than playing them, applies

a contructionist approach to learning with games (Robertson & Good, 2005;

Robertson et al., 2004). El-Nasr and Smith (2006) view game modding—the

development of new modules in an existing game using toolkits packaged with

the game—as a constructivist method of learning. The constructionist approach

to learning involves two activities: “the mental construction of knowledge that

occurs with world experiences” and the creation of “products that are personally

meaningful” (p. 2). The theory proposes that whatever the product, a birdhouse,

computer program, or robot, the “design and implementation of products are

meaningful to those creating them and that learning becomes active and self-

directed through the construction of artifacts” (p. 2). Steiner, Kaplan, &

Moulthrop (2006) concur with this view and contend that when “working to

develop designs, test technology, and suggest revisions, children as design

partners improve the technologies they consume as well as gain educational

benefits from the experience” (p. 137). Burrow and More (2005) apply

constructionist techniques in an architecture course by having students render

their designs with a game-engine thereby exploring spatial relationships as well

29
as atmosphere, lighting, and other environmental conditions in a 3-dimensional

simulation of their architectural designs.

Alternate Reality Games

While the research literature indicates promising results in terms of the

educational merit of designing a digital game, developing the fully integrated,

stand-alone simulation or game worlds typical of contemporary video games is

costly and time consuming. Moreover, it requires a great deal of technical

proficiency with game-engines, modeling tools, and other high-end computer

software. However, a fairly new genre of game, the alternate reality game or

ARG, distributes game challenges, tasks, and rewards across a variety of media,

both digital and real. Harnessing media with intuitive usability, such as MySpace,

web logs, podcasts, and YouTube, an ARG leverages tools that digital age

learners use as part of their daily lives. Thus, development of the game design

requires little technical proficiency and imposes minimal expense to financially

strapped students, instructors, and educational institutions at any level.

What is an ARG? As described by the International Game Developers

Association (Martin & Chatfield, 2006), “Alternate Reality Games take the

substance of everyday life and weave it into narratives that layer additional

meaning, depth, and interaction upon the real world” (p. 6). CNET staff writer,

John Borland (2005), depicts them as “an obsession-inspiring genre that blends

real-life treasure hunting, interactive storytelling, video games and online

community” (para. 4). Thus, an ARG provides players with an immersive digital

30
experience that arguably better simulates the reality of information distribution

(Baudrillard, 1994) than closed system games modeled on discrete parameters

for the purpose of focused instructional goals. This is not to say that such media

are unworthy as instructional products, but merely to point out that ARGs

simulate information distribution and the skills necessary to seek, locate and

evaluate that information in a highly meaningful way. Controlled by the narrative

storyline, players are given new clues and directed to increasingly complex

puzzles as the game progresses. Moreover, these games compel players to seek

resources in a variety of places, evaluate the relevance of those resources, and

apply them to solving the task at hand. Such skills are invaluable in a knowledge-

based and information-saturated workplace, which requires not only the ability to

find information, but also to evaluate its validity, authority, and applicability (Peter

D. Hart Research Associates, 2008).

Although the ARG as a game genre has emerged quite recently, Web

sites such as unfiction.com and ARGNet have links to several past and current

ARGs, such as Cathy’s Book and iamtryingtobelieve.com. Some ARGs have

served a marketing function, such as ilovebees.com that supported the release of

Microsoft’s Halo 2 video game. Others have an educational focus; Hexagon

Challenge and Never Rest Game are billed as instructional ARGs and claim to

“address decision-making skills, after-action report generation, and adaptation to

performance” (Bogost, 2007). Yet others, while not explicitly educational, deal

with social, economic, or environmental justice and aim “to change the way

31
people think, and feel, and live” (Strickland, 2007). Jane McGonigal, who

masterminds ARGs for their capacity to construct “collective intelligence,”

maintains that the purpose of her 2007 ARG, World Without Oil, was to “play our

way to a set of ideas about how to manage that crisis [a dramatic decrease in oil

availability]” (cited in Strickland, 2007, p. 1). McGonigal observed that players not

only generated strategies for coping with a peak oil crisis, they also changed their

real world behavior: planting trees or converting their cars to run on biodiesel

(Strickland, 2007). Thus, the simulated problem presented through the ARG

yielded practical solutions and prompted real world applications of the knowledge

constructed in the simulated play space.

Furthermore, recent research on the use of The Door ARG to transform a

large enrollment class at a university in Texas met with favorable results

including statistically significant gains on posttest achievement when compared

with existing computer aided instruction (t(63)=3.898, p=.0001) (Warren &

Dondlinger, 2008c). Perhaps more significant, however, was the sense of power

that students reported to have gained from their experience with The Door ARG

(Warren, Dondlinger, & McLeod, 2008). Although they reported a great deal of

frustration with this learner-directed instructional design, they also marveled at

how much they learned, particularly the resourcefulness to seek out answers and

solutions to problems since neither were served up to them via the closed-

system learning management system (LMS) to which they were accustomed or

32
from the instructor who consistently redirected them to their peers and to other

resources for answers to their questions (Warren, Dondlinger et al., 2008).

Learning By Designing Digital Games

Despite its success in terms of empowering learners to be full participants

in their learning experiences, The Door ARG as a course redesign project was

created by instructional design faculty and revised based on student feedback.

Indeed, the most common approach to creating educational games is to design

for learners to play by including imposed instructional goals that result from

professional analysis (Dondlinger, 2007). The design for approach has been

employed by many designers of instructional game spaces (Hays, 2005; Squire,

2006; Warren & Dondlinger, 2008b). However, some learning game designers

have progressed to a second stage wherein the learners become co-participants

through regular feedback to the designers, an approach in which the space and

activities are designed with the learners as revision to the product takes place

(Barab, Warren, & Ingram-Goble, 2008). Designers of The Door implemented

this strategy. While more responsive to learners needs than the designed for

approach, it does not center the learner as the core designer of the learning and

play activities in the simulated space. For example, The Door attempted to be

responsive to learner feedback through the iterative nature of the design based

research process (Barab & Squire, 2004) by incorporating the daily, formative,

and summative feedback that the researchers collected each semester over two

33
years into revisions of the game. While useful for improving the experience over

time, this was not efficient for developing the simulation game curriculum.

Further, it did not employ the students as designers of the experience. Based on

the qualitative data from the study, designers and students perceived that this

disenfranchised them from what the designers now view as a necessary

participatory role in the design and development process and that would have

provided them with important buy-in (Warren, Dondlinger, & Whitworth, 2007).

However, another approach exists. This approach has students

themselves design and employ technological tools as a means of engaging

others in the problem-solving process in which the problem is the development of

the game itself, a method supported by the research of Kolodner (2002) with the

Learning by Design approach to problem-based learning; it also supports Squire

et al’s (2005) proposal that instructional designers should move learners from the

role of user to that of designer of the learning experience itself. However, since

developing the 3-D worlds typical of many simulations and games is costly, time-

consuming, and technically complicated, a much more viable alternative is to

engage students in the production of an alternate reality game.

While the research on learners designing games as part of their learning

program has yet to be instituted widely, a few studies have indicated that the

process of designing games or simulations can encourage higher order thinking

(McLester, 2005; Robertson & Good, 2005; Robertson et al., 2004; Steiner et al.,

2006). According to El-Nasr & Smith (2006), “during the design process, skills

34
such as analysis, synthesis, evaluation, and revision must be used, providing

opportunities for learning content and metacognitive skills such as planning and

monitoring” (p. 2). Such a strategy combines elements of both constructionist and

constructivist approaches, as well as problem-based and situated learning

models. Further, the literature from a design based learning and research

approach to the development of other forms of learning environments ranging

from the use of multi-user virtual environments (MUVE) has shown promise for

objective achievement in the areas of science education (Barab et al., 2007;

Dede, Ketelhut, & Ruess, 2006; Liu, 2003), writing practice (Warren, Barab, &

Dondlinger, 2008), and blended forms of instruction that leverage simulation

games to target general computer skill literacy (Warren & Dondlinger, 2008a).

Summary

The proposed instructional design solution, The Global Village Playground

(GVP), intends to examine how a game designed by students can be

accomplished. This project engages sophomore students at a large, urban

community college in an institutional capstone experience through which learners

design and develop an ARG based on the theme of global sustainability and

centered on the United Nations Millennium Development Goals (United Nations,

2005). Facilitated by a team of faculty, administrators, and community leaders,

students work together to create a coherent game narrative, research content,

organize activities, and develop the distributed game “world.”

35
 CHAPTER 3

DESIGN METHODOLOGY

Introduction

Games and simulations require specific design processes that take

advantage of the technological systems (e.g., the Unreal 3 game engine) and the

learning principles to be imparted through play (i.e. attention, perception, and

memory in the simulated context). The design of the Global Village Playground

(GVP) as a course centered on the process of designing a game followed a

similar process. In this case, the course design process was driven by

consideration of the activities involved in the process of designing a game, which

includes a phase of analysis or planning, followed by a design phase, and then

development of the game itself. While articulated here as a linear process, like

most creative processes each phase is much more circular or iterative in nature

than a written account of the process can express. In general, the process began

with an analysis of the learning objectives to be accomplished by the design,

followed by the design of the course itself, and finally development of the course

activities and means of assessing student attainment of the course objectives

(Heinich et al., 1999). However, in the process of developing instructional

activities, the shape of the overall instructional design concept changed rather

substantially, which then reshaped the instructional activities in development.

36
Each in turn were adapted and refined during the implementation of the course,

further blurring the distinctions among the various phases of the process:

analysis, design, development, and implementation. Thus, while each phase was

neither finite nor discrete, they are necessarily presented here in a sequential

fashion that belies their overlapping and mutually inclusive nature. The remainder

of this chapter details the elements of this design concept, including the learning

objectives, underlying design theory, alignment of objectives with academic

content, development of the instructional activities, and means of assessment.

Learning Communities

As with many instructional design frameworks, this began with an analysis

of the needs of the students, who would be served by the design and

development of the learning product. Although the primary impetus for creating

this capstone course emerged from the need to provide evidence to the state that

completers of the academic core curriculum attained the state-recommended

core perspectives, integrating those learning goals into a meaningful experience

for students through which they could earn course credit, a mandated

assessment product (a final portfolio) added on top of their coursework, was a

learner-centered imperative to the designer. However, since the general

education courses that comprise the academic transfer program are not

sequenced, students in their final semester might have any combination of the

required courses remaining to complete. Consequently, the capstone course

could not summarily replace any single core course requirement; it needed to

37
provide participating students the option to obtain credit for a variety of core

courses.

Like many community colleges, this institution already had an existing

integrated learning initiative, or learning community program, that could provide

the means to offer a capstone experience rather than a mere capstone

assessment. The term, learning community, refers to a variety of different

conceptions in different contexts. However, a Learning Community in community

colleges typically refers to a team-taught course that combines two or more

courses from different disciplines into one, integrated and themed learning

experience. For example, the learning community, Romantic Music and

Literature, combines music theory and English so that students “explore

connections between the Romantic revolutionary movement in the works of

several British poets and in that of selected musical composers,” while students

in Bye, Bye, Miss American Pie “study the political, social, and literary impact of

popular songs from Bill Haley to Kanye West” to receive credit for English and

government (Richland College, 2009b).

Learning communities benefit students in a number of ways. First, they

make the connections between two or more disparate disciplines more

transparent for students. Rather than struggling to make those connections

themselves individually, students in a learning community explore and construct

those connections together. Secondly, instead of managing course materials,

assignments, and deadlines for separate courses, students get an “all-inclusive

38
package” of sorts. Most significantly, students are better able to develop a

community of learners in such a course. Instead of spending three hours a week

in one course with the students enrolled in it and another three hours a week with

a different group of students in another course, students spend six hours a week

with the same group, more deeply exploring the content of both courses and the

connections between them.

Learning community courses have been found to have a positive impact

on student retention and success, particularly for entering students. Initiatives at

Cuyahoga Community College and Valencia Community College linking

developmental studies with college level courses in learning communities have

documented increases in success and retention (Lincoln, 2009). A study tracking

student progress at thirteen community colleges found that low-income,

academically unprepared freshmen who were placed in learning communities

were “significantly more likely to persist to sophomore year” than those who were

not (McHugh Engstrom & Tinto, 2008). While this research has focused on the

impact of learning communities for entering students, this integrated approach

can also provide a means to create a capstone experience in which student

efforts are focused on a large-scale project aimed at promoting the higher-order

outcomes identified as necessary to students’ future success:

• Engagement with big questions

• Critical and creative thinking about complex problems
• Active involvement in diverse communities and real world challenges
• Application of knowledge and skills in diverse settings and innovative
ways (Peter D. Hart Research Associates, 2008).

39
The student project along with other individual products, such as portfolios, could

also be used to assess attainment of the broader program or institutional learning

outcomes.

Identification of the Learning Objectives

Conceptual Age Thinking

In addition to the knowledge and skills identified in the AAC&U (American

Association of Colleges and Universities) report cited above, the central problem

scenario, designing an ARG, was informed by Pink’s (2006) call for “a whole new

mind,” ways of thinking valued in what he designates the conceptual age. Pink

distinguishes the conceptual age from the information age, identifying the

principal occupations and characteristic thinking patterns of each. The

information age was dominated by “computer programmers who could crank

code, lawyers who could craft contracts, MBAs who could crunch numbers”

(Pink, 2006, p. 1). These occupations require left-brained, sequential patterns of

thinking, specializing in text and attention to details. Those who were more right-

brained and simultaneous, who specialize in context and see the big picture,

weren’t as necessary in the information age job market. However, Pink argues

that the conceptual age will belong to “creators and empathizers, pattern

recognizers, and meaning makers . . . artists, inventors, designers, storytellers,

caregivers, consolers, big picture thinkers” (Pink, 2006, p. 1). Left-brained

thinkers will still be needed, but in a global marketplace, those who can add

value to an enterprise will be those with a flair for design or the personal touch

40
that sets an organization’s products or services apart from others. As Pink puts it,

“mere survival today depends on being able to do something that overseas

knowledge workers can’t do cheaper, that powerful computers can’t do faster,

and that satisfies one of the nonmaterial, transcendent desires of an abundant

age” (p. 51).

Pink designates these qualities as high concept and high touch. High

concept involves “the ability to create artistic and emotional beauty, to detect

patterns and opportunities, to craft a satisfying narrative, and to combine

seemingly unrelated ideas into a novel invention” (p. 52). It’s no longer enough to

offer something that’s merely functional and reasonably priced. What sets a

product or service apart is a unique design or even a story behind the product or

service being offered--in short, a concept rather than thing itself. High touch

involves “the ability to empathize, to understand the subtleties of human

interaction, to find joy in one’s self and to elicit it in others, and to stretch beyond

the quotidian, in pursuit of purpose and meaning” (p. 52). The attention to

customer satisfaction that rules the marketplace today is evidence of this quality.

Those that can identify and cater to human needs, sometimes a niche need, are

those that are most competitive.

Thus, while problem-based learning has shown great promise as an

instructional methodology for fostering higher-order critical thinking skills

(Albanese & Mitchell, 1993), the goals of the GVP were not problem-solving

alone. In order to engage learners in conceptual age patterns of thinking, the

41
central problem scenario was a design problem: designing and developing an

ARG. Immersing students in such a design problem was expected to engage

them in both critical and conceptual age thinking.

Institutional Goals

In the case of the Global Village Playground (GVP), a primary impetus for

creating this capstone course emerged from the need to provide evidence to the

state that completers of the academic core curriculum also attained the state-

recommended core perspectives (Texas Higher Education Coordinating Board,

1999). These core perspectives consequently became the learning objectives

targeted by the design:

1. Establish broad and multiple perspectives on the individual in relationship

to the larger society and world in which he or she lives, and to understand
the responsibilities of living in a culturally and ethnically diversified world;

2. Stimulate a capacity to discuss and reflect upon individual, political,

economic, and social aspects of life in order to understand ways in which
to be a responsible member of society;

3. Recognize the importance of maintaining health and wellness;

4. Develop a capacity to use knowledge of how technology and science

affect their lives;

5. Develop personal values for ethical behavior;

6. Develop the ability to make aesthetic judgments;

7. Use logical reasoning in problem solving; and

8. Integrate knowledge and understand the interrelationships of the scholarly

disciplines.

42
These perspectives are not the narrowly focused, cognitive outcomes typically

measured by objective and/or standardized tests. Additionally, they include the

affective values and attitudes vital to developing a global citizenry, perspectives

sought by business and industry in an increasingly global marketplace. One of

the core competencies identified by business leaders in the previously cited

AAC&U sponsored study was “global knowledge;” this competency was

immediately followed by skills in self-direction and in writing (Hart Research

Associates, 2008). The GVP project contextualizes these competencies as the

central activities of the design process to affect measurable performance of these

vital skills.

Efforts at the college to address the need to assess these intellectual

competencies in the past had yielded proposed “capstone” assessments, such

as an electronic portfolio of student work. The use of portfolios to assess student

learning outcomes that result from engagement in courses as well as entire

programs of study has become a popular approach (Barrett, 2007; Juniewicz,

2003; Pullman, 2002). However, in this case, the approach provided only an

assessment, artificially layering an additional product on top of student

coursework, rather than creating a capstone experience, integrating the learning

from the multiple disciplines that comprise the core curriculum. For a portfolio

assessment to be effective, the means by which it is implemented must provide

scaffolding and feedback to learners throughout the portfolio creation process

(Segers et al., 2008; Van Tartwijk et al., 2007). Such considerations drove both

43
the rationale for creating a capstone experience as well as the learning outcomes

assessed through The Global Village Playground design effort.

Alignment of Objectives with Instruction

Project Content

Given that the learning objectives were broad intellectual competencies

related to global perspectives and higher order thinking, the GVP project content

needed to compel learners to engage with global issues. The social constructivist

(Duffy & Cunningham, 1996) and problem-based (Barrows, 1986) approaches

upon which the course was methodologically based would address the higher

order thinking skills by engaging learners in devising a solution to a large-scale,

ill-structured problem, namely the development of an ARG. However, learner

development of global perspectives would not be fostered by method alone; the

content of the game they were to develop needed to center on global issues. In

2000, the United Nations (2005) proposed eight goals for the new millennium and

challenged the world to meet these goals by the year 2015. The eight Millennium

Development Goals aim to achieve the following:

1. Eradicate extreme poverty & hunger

2. Achieve universal primary education
3. Promote gender equality and empower women
4. Reduce child mortality
5. Improve maternal health
6. Combat HIV/AIDS, malaria and other diseases
7. Ensure environmental sustainability
8. Develop a global partnership for development

44
Since these goals align well with several of the core perspectives that are the

learning objectives of the course, these goals were selected to constitute both

the content and the problems to be solved by this student-designed ARG. In

other words, students were to ensure that the game’s narrative, challenges, and

rewards align with these overarching goals. For example, a game challenge

might compel player guilds to collect cans of food and take them to a designated

person at a specified food bank who then gives the players their next game clue.

Instructional Activities

Traditional learning game development is driven by the creative process of

professional instructional designers (Crawford, 2003; Salen & Zimmerman,

2004); in contrast, the instructional activities for a designed-by-students ARG are

driven by the design process and are mainly student led. This process involves

creating a coherent game narrative, researching necessary informational and

contextual content, structuring the game challenges and rewards, as well as

developing the distributed game world. In the case of the GVP project, the

process would require students to perform the following:

• Devise collaborative solutions to global problems

• Deploy research, productivity, and creativity skills and tools

• Create written design documents, such as narrative outlines, character

profiles, research reports

• Deliver presentations to share information, pitch ideas, evaluate progress

• Communicate effectively in small and large groups of diverse membership

45
 • Construct game spaces using a variety of technologies

The design process used for this project was divided into three general phases:

Analysis, Design, and Development. The objectives for each phase and the

activities that correspond to each are outlined in Table 3.1.

Table 3.1

Objectives and Activities within the Framework of the Design Process

Analysis Phase Design Phase Development Phase
Objectives Activities Objectives Activities Objectives Activities
Explore UN* Research and Evaluate Achieve Assess design Discussion,
MDGs** and discussion solutions and consensus phase progress metacognitive
identify places information through social and products; reflection
in the world to gathered in negotiation evaluate individual
potentially analysis phase progress
impact them and select best
alternatives

Devise Research, Create game Oral Devise quantifiable Create

culturally discussion, tasks/ presentations, outcomes (win database
viable oral & written challenges and written design scenarios) for and/or
solutions to presentations rewads for documents game tasks and spreadsheet
problems that task develop means to tools to track
underlie UN completion track player player
MDGs progress progress and
measure
impact of
game on
MDGs

Identify Reading, oral Generate Storyboard Develop game Set up blogs,

elements of and written overarching presentations, narrative and social
compelling presentations game written design character personas networking
narrative narrative, documents spaces, digital
construct videos, e-mail
characters, accounts
develop
“setting”

Explore media Research and Select best Written design Construct Create
(text, audio, discussion tools to convey documents interactive game dedicated
visuals, etc), story, spaces, websites,
technological disseminate puzzles/challenges discussion
tools, and their challenges, boards, wikis
affordances embed clues
* United Nations
** Millennium Development Goals

46
This table is necessarily oversimplified for illustrative purposes, but it serves to

demonstrate how the instructional activities are driven by the objectives of the

design process. The activities involved in this process subsequently drove the

selection of courses from the transfer program curriculum that formed this

integrated learning community experience.

Selection of Courses

Identification of courses for which students could earn credit proceeded

from an analysis of the tasks that students would perform throughout the process

of designing and developing an ARG (Annett, Duncan, Stammers, & Gray, 1971;

Kirwan & Ainsworth, 1992). This task analysis led to the identification of five

subject areas from the program curriculum upon which the capstone could be

based: composition, literary studies, speech communications, humanities, and

computer applications. Descriptions of these tasks or activities and their

relationship to these areas of study follow.

Composition

The process of designing a game of any genre involves the production of

design documents, which provide a record of the game’s narrative structure,

levels of play, artistic assets, and other technical features. Production of these

documents engages designers’ writing skills, including consideration of purpose,

audience, unity, and coherence, the fundamental elements of written

composition. Moreover, since the intention of this game was to address social

47
issues, such as poverty, health, or gender equity (the UN MDGs), student-

designers would need to research potential causes of these issues in their socio-

cultural contexts: AIDS in Malawi or malaria in Burundi, for example. Research

and documentation are principal competencies in postsecondary English

composition courses.

Literary Studies

Because the project scenario involves development of a narrative to

contextualize the ARG, student-designers explore various dimensions of literary

analysis, including narrative composition, character development, setting, and

plot organization. Students not only read literary works, they also create an

interactive narrative similar to those found in simulation games like America’s

Army. Creating the game narrative requires student-designers to synthesize

literary pieces with other communications media, make connections between

literature and life, and evaluate constructs of reality and fiction. For example,

after reading and discussing various literary works and identifying the elements

of a compelling narrative, students create the overarching narrative plotline of the

game including where game clues or puzzles will be stashed and revealed. Clues

might well be embedded in existing narratives, such as the trajectory of

Huckleberry Finn’s raft down the mighty Mississippi or the species of Faulkner’s

“Rose for Emily.” Student-designers also study archetypal literary characters,

such as the evil mastermind, the valiant hero, the comic foil, or the wise fool

(Campbell, 1972; Jung, 1958), as they develop characters to further the plotline.

48
Speech Communications

Design and development of a collaborative project such as an ARG requires

extensive use of communication, formal and informal, small and large group, in-

class and online. Students generate ideas from small group brainstorming

sessions and pitch them to the rest of the design team (persuasive

presentations), research content and inform the class about it (expository

presentations), and prepare visual media that allows their audience to better

understand the ideas and concepts conveyed. For example, after developing an

idea in a small group, students pitch a storyboard mockup of an overarching

narrative, including profiles of characters and setting and how both might be

connected to potential game challenges and rewards. Likewise, after researching

gender inequality in a region of the world, the group presents to rest of class how

the gender gap might be bridged in way that works within that culture without

further aggravating gender inequalities.

Humanities

In addition to literary or written materials, an ARG makes use of visual and

auditory media such as music, art, graphics, and even performance. Moreover,

since students must tie the game narrative (plot, tasks, and rewards) to the UN

Millennium Development Goals, the course involves the study of cultural

production (art, music, literature, drama) from a diverse array of continents,

nations, and cultures. For example, much like the popular novel, The DaVinci

49
Code, employed classical paintings and accompanying hints via secret

messages and documents about what the paintings revealed, students might

likewise post a series of visual artifacts relating to child mortality on an obscure

South Atlantic island to a blog maintained by a fictional international

correspondent whose comments about the images hint at what players should

infer from them and what actions they are to take in response to their inferences.

Computer Applications

Students use productivity applications such as Microsoft’s Office suite to

prepare documents, present ideas, and organize course and simulation-game

materials. They leverage communication/collaboration technologies to share

documents, generate ideas, and search for information. They also develop the

game space, distributed across the Internet, with Web 2.0 tools, such as web

logs, podcasts, social networking sites like MySpace, interactive and static web

pages, and even the 3-D digital world, Linden Labs’ Second Life. For example,

the television show Lost has an accompanying alternate reality game called The

Lost Experience that provides players with multiple videos uploaded to web sites

for fictional airlines, companies like TheHansoFoundation.org, and related

campaigns from fictional characters. Through these game media, players

investigate what really happened to Oceanic Flight 815. The videos have been

reposted to real YouTube spaces with links to social networking sites and

message boards such as the creative design team’s Web site, thefuselage.com.

Such spaces are centered on deciphering the clues embedded within the videos

50
and images on related sites and support players in their goal of unraveling the

mystery that surrounds the show. In player created spaces, players discuss and

build digital homages to the show’s setting in the virtual world, and some even

role-play the individual characters.

Means of Assessment

In a framework driven by the process of designing an ARG, assessment of

student learning outcomes at various levels occurs in unique ways. Hickey,

Zuiker, Taasoobshirazi, Shafer, & Michael (2006) identify five levels of

assessment (immediate, close, proximal, distal, and remote), their primary

orientation, and relevant timescales in order to differentiate how they ideally

should be assessed given the domain knowledge they represent. The authors

show that shifting the summative function of measuring individual student

knowledge gains at the distal and proximal levels to a more formative function

increases distal and remote level standardized assessment scores. Their

argument is that socio-cultural practices at the classroom level of assessment—

that is, the immediate, close, and proximal levels--can have an impact on distal

and remote assessments when they function formatively for the benefit of

students, as opposed to benefiting the teacher. Indeed, the guiding hypothesis of

these authors’ work is that “focusing on directly collective participation at the

immediate and close levels (and in practice, ignoring individual knowledge

acquisition) will have a greater indirect impact on individual knowledge at the

51
proximal level and beyond, compared to formative assessment practices that

focus more directly on individual knowledge” (Hickey et al., 2006, p. 188).

Although the GVP is not focused on standardized assessments, a key

feature of its design is that it shifts state-mandated learning outcomes for a post-

secondary general education program (the core perspectives) to a classroom

context wherein the objectives of the course are those of the entire program: the

set of general education courses that comprise a community college’s academic

transfer curriculum. What this shift of program-level outcomes to the course-level

does is to shift measures of course learning outcomes from distal to proximal-

level. Thus, what are summative assessments at the course-level (a presentation

given for evaluation of speech outcomes or a research paper composed for a

humanities course) can function in a more formative manner because they are

contextualized as products in the process of engaging in a large-scale design

problem. Thus, they become situative, socio-cultural practices inherent to the

process of designing a product, rather than summative assessments of individual

knowledge or ability in giving speeches or composing research papers.

Contextualizing these assessment products (student presentations and

written compositions) as the central activities of the design process, the principal

designer of the GVP hoped to promote attainment of course-level competencies

(such as writing, speaking, and listening) as well as the state core perspectives

when those artifacts were assembled into student portfolios. The institution might

also evaluate a product of “collective participation,” the ARG itself and the design

52
documents, which would accompany it, providing an alternative way to assess

program effectiveness.

Overview of Resulting Course Design

The previous sections of this chapter detail the analysis of needs and

theoretical framework that supported the design of the GVP capstone experience

as conceptualized by the principal designer and author of this dissertation.

Following that analysis, the course was then developed collaboratively by the

designer and her co-instructor. This process involved some modification to the

overarching design.

A Learning Community Experience

As discussed previously, the GVP made use of the learning community

approach, integrating multiple course subjects into one class. This approach

gave students some enrollment options depending on what courses they had left

take. Although preliminary analysis identified computer applications as a subject

area for the GVP, institutional constraints precluded it as a course option for

enrolling students. The computer applications course in the academic transfer

curriculum includes a lab requiring a greater number of weekly contact hours

than the other courses identified for this learning community. Thus, the course

options were limited to composition, literature, humanities, and speech

communications. Students enrolling in this capstone thus had the option of

53
completing it for any two of those courses depending upon what courses

remained in their individual programs of study.

At this institution, ALCE (A Learning Community Experience) courses are

team taught, in large part because accreditation requirements dictate that

courses be taught by faculty credentialed in the discipline areas for which

students will receive credit. The GVP was no exception. The principal designer is

credentialed to teach all of the course subjects except speech, and thus identified

a co-instructor who fulfilled this requirement. Instructional materials for the course

were then developed collaboratively by the two instructors.

Hybrid Delivery Format

Given their respective workloads and schedules, the instructors chose a

hybrid delivery format for the flexibility it would provide them and students. The

course entailed six contact hours per week, which were divided equally into face-

to-face and online instruction. The class met for three hours, one evening each

week, accompanied by three hours of online coursework. Online activities

included multimedia presentations of course content, interactive discussions

among students, peer evaluation, and self-reflective activities. Including an online

component, which required students to make use of telecommunications media

for collaboration, made sense in a course aimed at fostering 21st Century skills.

54
 The “Learn, then Apply” Approach

The co-instructor had difficulty reconciling her instructional philosophy with

the central problem-scenario, designing an ARG, since she had little experience

with games for learning. However, she was able to see the connections between

such collaborative projects and speech communications, as well as those

between literary elements and the narrative structures that underlie ARGs.

However, she was uncomfortable having students explore literature and art (a

central component of the humanities course) entirely through the process of

designing a game. Thus, the two instructors negotiated a compromise between

her epistemological need to “cover some material” and wholly contextualizing

student learning within the game design scenario of the course. Following a

“learn, then apply” approach, course activities were sequenced to allow some

presentation of course content during the first four weeks of the semester,

followed by three weeks of student exploration of additional course content, and

finally an application of that content to development of the game during the last

eight weeks.

The Learn Phase

During the first four weeks of the course, instructors led discussions on art

and architecture, archetypes and symbols, and elements of effective narratives.

The principal texts for this phase of the course included Joseph Campbell’s The

Hero with a Thousand Faces and Tom Stoppard’s Rosencrantz and Guildenstern

are Dead. Students prepared and gave presentations, as well. For example, they

55
developed a superhero alter-ego for themselves and gave a speech of

introduction, introducing themselves by describing their “character,” an activity

combining literary, speech communications, and game design features. Student

teams also gave presentations on the stages in the hero’s journey that they read

about in Campbell, providing examples of these stages from literature, art, and

film.

The Explore Phase

During the next phase of the course, student teams worked through what

came to be called the culture project. Preliminary work on this project began

during the last weeks of the learn phase with students identifying the major works

of literature, art, or architecture of a selected culture. They first prepared

proposals justifying their selection of these works, followed by an annotated

bibliography of resources regarding these works. The entire class then read the

proposed literature selections, and each team gave an informative presentation

to the class. The presenting team also led an online discussion about the culture

explored.

The Apply Phase

The last eight weeks of the course were dedicated to applying concepts

learned about archetypal narratives and symbols, as well as other cultures to the

design and development of an ARG. Although this was a project involving the

entire class, students broke into smaller brainstorming teams working on different

56
levels of the game, then pitched ideas to the rest of the class, discussed

alternatives, and built consensus regarding the overall narrative and game

structures. Class meetings would close with students assigning tasks to

individuals to complete before the next class. Online communication tools

provided a means to share ideas and document progress. The next class

meeting would begin with each student giving a walk-through of what he or she

had completed, followed by peer evaluation or discussion. Then the process

(break out, come back together, assign tasks) was repeated.

Summary

Design of the GVP began with front-end analysis of the learning outcomes

desired by employers, state agencies, and the institution where the GVP was

implemented. Selection of course content, instructional methods, and the central

problem-scenario were governed by these needs and supported by the research

literature. The development of the course proceeded collaboratively with both

course instructors, yielding a capstone learning experience that was mutually

agreeable to both instructors’ beliefs about teaching and learning. The next

chapter describes the methods of research used to evaluate the effectiveness of

this innovative instructional design.

57
 CHAPTER 4

RESEARCH METHODOLOGY

Introduction

This study examined the Global Village Playground (GVP) in a naturalistic

classroom context. Given the intricacy of the design and the blended modes of

instructional delivery, the GVP is laden with a complexity that is not conducive to

an experimental design that controls for discreet variables. Moreover, the primary

researcher of this study was both the designer of the GVP and a co-instructor of

this team-taught course. Because of these factors and the intimate size of the

class, this study took a qualitative case study approach to the evaluation of the

pilot implementation of the course design. This approach allowed a deeper

analysis of the complexities of the design and participant experiences with them,

including how well elements of the design met their learning needs or presented

challenges to them. The following research questions framed the study:

1. What were student and instructor reactions to the course/course

design?

2. What aspects of the design were most conducive to student

learning?

3. To what extent did the course promote attainment of the

overarching program objectives (core perspectives) and/or
advance conceptual age thinking?

4. What challenges or tensions arose from the design?

58
 Philosophy of Inquiry

A researcher’s belief about the nature of knowledge and how it comes to

be known underlie the methods of inquiry used to discover knowledge and the

conclusions drawn through analysis. Consequently, it is imperative to identify the

researcher’s ontological and epistemological assumptions prior to proceeding

with the inquiry. Ontological suppositions about “what is” and epistemological

convictions regarding “how it can be known” have a direct bearing on “what it

means,” the interpretations or generalizations that are drawn from the results.

Thus, these underlying perspectives must be consistent with the methods

employed to seek and make sense of the phenomenon explored through

research (Bernstein, 1983, 1993).

The positivist approach to studying phenomena has provided the

foundation of scientific inquiry for several centuries (Denzin & Lincoln, 2003;

Hollis, 1994). This approach supposes a reality outside of the human mind that

can be observed, measured, and understood through the human senses. The

tools employed in empiricist methods of inquiry allow a researcher to draw

conclusions about this observable reality that are deemed to be objective, that is

devoid of the subjective biases of the researcher (Bernstein, 1976, 1983).

Depending on the degree to which confounding factors are controlled, the

measured phenomenon is open to interpretations that “generalize” to other

contexts. An opposing perspective, the relativist stance, presumes that no

59
objective reality exists outside of the human mind, or if it does, it cannot be

understood objectively because it is measured and analyzed through tools of

human construction, including methods of inquiry (Denzin & Lincoln, 2003, 2008;

Guba & Lincoln, 1989). At a minimum, this perspective maintains that

interpretations of observed phenomena are limited by the subjective biases of the

researcher and to the specific context in which they were observed.

The constructivist, also called contextualist or hermeneutic perspective,

finds that reality is socially constructed through negotiation among groups of

people. As such, it is neither devoid of the subjective biases of the members of

the group nor divorced from the socio-cultural and political context in which it is

situated (Bernstein, 1983; Guba & Lincoln, 1989). Constructivist inquiry does not

attempt to control for the influences of subject or context, but rather to account

for them. Such methods acknowledge the influences on interpretation, but

present them fully so that others may draw their own inferences about the

usefulness of the observed phenomena based on their own situational contexts.

According to Denzin and Lincoln (2008), “terms such as credibility, transferability,

dependability, and confirmability replace the usual positivist criteria of internal

and external validity, reliability, and objectivity” (p. 35). As designer, instructor,

and researcher of the GVP, the constructivist perspective guided the curricular

design, the methods of instruction, the process of inquiry used to evaluate it, as

well as the claims made regarding its effectiveness.

60
 Educational Evaluation

According to Gall, Gall, and Borg (2007), educational evaluation is “the

process of making judgments about the merit, value, or worth of educational

programs” (p. 559). These authors further find that evaluation research plays a

key role in educational research and development (R&D), “an industry-based

development model in which the findings of research are used to design new

products and procedures, which are then systematically field-tested, evaluated,

and refined until they meet specified criteria of effectiveness” (p.589). This study

follows an educational R&D approach. The GVP was designed based on

research findings identifying skills necessary to academic and professional

success (Peter D. Hart Research Associates, 2008; Pink, 2006), as well as

findings supporting the use of instructional strategies to facilitate gains in the

acquisition of those skills (Albanese & Mitchell, 1993; Warren, Barab et al., 2008;

Warren & Dondlinger, 2008c). This front-end analysis on the part of the principal

designer involved “judgments about the merit, value and worth” of these research

findings throughout the process of selecting appropriate instructional methods,

materials, and technologies. Nevertheless, this designer does not view her

judgments to be infallible; she seeks the perceptions of other participants in the

implementation of this design in order to better judge its merits and refine the

design.

61
 Levels of Evaluation

Kirkpatrick (1994) identifies four levels of evaluation, which represent a

sequence of ways to assess the effectiveness of instructional programs. He

argues that “Each level is important. As you move from one level to the next, the

process becomes more difficult and time-consuming, but it also provides more

valuable information. None of the levels should be bypassed simply to get to the

level that the [evaluator] is interested in” (p. 21). A description of each of these

levels follows:

1. Reaction elicits the general reaction of the participants to the instructional

program. Kirkpatrick calls this the “customer satisfaction” measure and

asserts that even compulsory programs should attempt to elicit favorable

reactions. If participants do react favorably, the program might well be

abandoned for another that does.

2. Learning seeks to evaluate what and how much learning resulted from

participation in an instructional program. According to Kirkpatrick, learning

is “the extent to which participants change attitudes, improve knowledge,

and/or increase skill” (p. 22).

3. Behavior is defined as “the extent to which change in behavior has

occurred” as a result of participation in a program (p. 23). Kirkpatrick

cautions against skipping the first two levels of evaluation and relying

solely on measures of behavior change because many factors might

prevent participants from performing differently even though they have

62
 attained a change in attitude, knowledge, or skill. For example, the

organizational climate might discourage the change in behavior even

when learning (a change in attitude, knowledge, or skill) has occurred.

4. Results aims to evaluate the broader impact on an organization that an

instructional program creates. For example, a program on servant

leadership might be expected to yield fewer complaints about mid-level

managers or an increase in morale at an organization.

While Kirkpatrick’s model applies specifically to training rather than educational

programs, the sequence of levels provides a meaningful way to conceptualize

the evaluation of an educational R&D. The goals of each level can be adapted to

the uniqueness of an educational setting. Indeed, these levels shaped the

principal research questions for this study and consequently the themes to be

explored through the inquiry. An explanation of those themes and their

relationship to Kirkpatrick’s levels of evaluation is provided in the next chapter.

Fourth Generation Evaluation

Guba and Lincoln’s (1989) Fourth Generation Evaluation is a qualitative

approach to educational evaluation. This comprehensive methodology seeks “full

participative involvement, in which the stakeholders and others who may be

drawn into the evaluation are welcomed as equal partners in every aspect of

design, implementation, interpretation, and resulting action of an evaluation” (p.

11). The GVP emerged as a capstone experience that arose from the “claims,

concerns, and issues” of institutional stakeholders in need of a means to

63
evaluate the effectiveness of their academic transfer program. However,

comprehensively evaluating that program was neither the charge to the designer

nor the purpose of this study. The designer/researcher championed what she

presumed to be the student perspective in proposing an engaging capstone

experience for students in lieu of an additional mandatory assessment piled on

top and outside of the existing course load required for completion of this

program. Although evaluation of the entire program is outside the realm of the

designer/researcher’s control, those in charge of that process welcomed this

approach because of their dedication to meeting student needs. To a certain

extent, this study is situated between an emerging evaluation of an academic

transfer program and the specific, bounded case of the pilot implementation. The

results may shape the approach the institution takes in evaluating the program.

Results notwithstanding, this study focused specifically on obtaining

participant perceptions of their experiences in the pilot implementation. The

claims, issues, and concerns of institutional, business and industry, as well as

state-level stakeholders informed the design of the GVP. The purpose of this

evaluation was to gather and analyze the claims, issues, and concerns of

participants in the pilot concerning the effectiveness of the design: their

reactions, learning, values or behaviors, and the results or challenges

experienced within the course pilot. This input will help us evaluate the capstone

experience itself and inform refinements to the design so that it might better meet

the needs and goals of all.

64
Case Study Research

The research design follows a qualitative case study approach, which

investigates “a contemporary phenomenon within its real-life context especially

when the boundaries between the phenomenon and context are not clearly

evident” (Yin, 2003, p. 13). In this case, the phenomenon is student perceptions

of their learning experiences within the context of the pilot implementation of the

GVP. Given the nature of the GVP as a capstone experience, the boundaries

between student perceptions (phenomenon) and the GVP (context) will not be

clearly evident. Students’ prior and concurrent learning experiences will

necessarily come to bear on their perceptions as they are expected to be drawn

upon in a capstone environment. Nevertheless, a case must be defined by some

form of the limits or boundaries, so that it in can be explored in its totality (Stake,

1995). These boundaries might be location or setting, a specified duration of

time, or an attribute that defines the participants in the study (Yin, 1993). The

boundaries of this case include the duration of the semester-long

implementation, all of the individuals who participated in the course, as well as

the class settings, both online and on the ground.

Context of This Study

Setting

The setting for this research study was a sixteen-week course at a large,

urban, community college in the southwest United States. The course was an

65
integrated, learning community experience wherein students elected to enroll for

credit in two of four possible disciplines: speech, literature, humanities, or

composition. The course design blended face-to-face class meetings with online

learning and communication tools, in keeping with current practices in a global

workplace wherein problems are solved and projects are developed across

expansive geographical distances via various digital telecommunications media.

Institution

The institution where this course was implemented is one of seven,

separately accredited colleges in a large multi-college district. Of the seven

colleges, this is the largest institution with enrollments exceeding 16,000 students

during the implementation semester, the highest achieved by this college. The

student body is “internationally and ethnically diverse, speaking over 90 first

languages,” and enrollment is approximately 42% Anglo, 21% African-American,

19% Hispanic, and 15% Asian (Richland College, 2009a). The college is

dedicated to providing high quality instruction and support services for students

and has received state and national quality awards for their efforts. The college’s

vision is “to be the best place to teach, learn, and build sustainable local and

world community” (Richland College, 2008). Core institutional values are

integrity, mutual trust, wholeness, fairness, mindfulness, cooperation, diversity,

responsible risk-taking, joy, and considerate, meaningful communication

(Richland College, 2008).

66
Classroom

Class meetings were held in an innovatively designed classroom, dubbed

a Learn Lab. These learning spaces make use of moveable furniture and multi-

media technology designed to promote collaboration. Figure 4.1 depicts the

overall configuration of this learning space. A Learn Lab features rectangular

tables, which accommodate two learners to a side, arranged into pods of four

tables that seat eight students. Each lab has four pods configured into an X, so

that the space has no front or back. Moreover, the tables that make up each pod

can be separated into smaller units accommodating four or two students.

Figure 4.1. Learn Lab classroom configuration.

The room also has expansive whiteboards, complemented by mobile racks of

mini-white boards. Called huddle boards, because learners can huddle around

67
them, these boards have clips that allow them to be hung for display and

discussion with the rest of the class after learners have completed their “huddle.”

Wide lens cameras installed above the whiteboards allow users to capture

images produced on the whiteboards and huddle boards, so they may be

distributed digitally. The room also features three data projectors and display

screens, one of which is an interactive, multi-touch screen or Walk and Talk

board.

Participants

Participants included the six students who completed the course, as well

as the two instructors who taught the course, one of whom is the primary

researcher and author of this study. Descriptions of each participant, who have

been assigned pseudonyms, follow.

Karen, an education major, was a returning student in her early 40s—a

factor that had a strong influence on her perceptions. “One frame of reference is

that I am the eldest person in the class, with the most life experience.” She is a

mother and a grandmother who frequently expressed heartfelt devotion to her

family. Karen loves to travel and intends to “see as much of the planet” as she

can while she’s “here on it.” In fact, she took advantage of an unanticipated

opportunity to go to Prague during the semester of this study. Karen is open-

minded, demonstrated a passion for learning, and was willing to take risks and

just try new things. Indeed, she had “never set foot” on the campus of the

institution where the GVP was offered. She attended another college in this multi-

68
college district but chose to travel to this institution because she was captivated

by this innovative course.

Kevin, a Caucasian student in his early 20s, was poised to be the first

graduate from the college’s new Interactive Simulation and Game Technology

program (ISGT), where he worked part-time as an assistant to the program

coordinator. He was very much viewed as a leader among his peers in ISGT, two

of whom were participants in the GVP. Kevin began his academic career

studying metaphysics and engaged deeply in class discussions. Although Kevin

is soft-spoken, he holds strong convictions. He confessed, “sometimes there

[are] certain topics that I tend to shut down on because there’s no way I’m going

to change my view on it.” Throughout the class, he did seem to challenge only

his fellow ISGT peers and seemed uncomfortable taking an oppositional stance

with the other participants. Kevin’s primary passion is drawing. The instructors

often observed him sketching ideas being discussed in class rather than taking

written notes. His sketches often illuminated concepts for the other participants.

Les, a Hispanic student in his early 20s, was also an ISGT major and an

artist. As he describes himself, “I’m more of an artist person…I come from a

background where I just kind of design.” Les was more practical than Kevin,

however, and seems to ascribe this quality to family influences, “my dad’s a civil

engineer so I kind of grew up…discussing buildings and little technical things.”

Les was quieter than the other participants. Although he found that the course

environment allowed them “to have our thoughts and ideas said more openly,”

69
Les tended to be a listener and thinker, rather than a talker. He explained in his

interview, “I did listen things out. And I try to manage where I can talk and where

I shouldn’t. ‘Cause you know, if I don’t have credibility on the subject what good

does that [speaking] do me?” The instructors found Les to be highly engaged, but

also very cautious, introspective, and courteous.

Nick, a Caucasian student in his early 30s, was an introspective

participant as well. He was more vocal in class discussions than Les, but much

preferred one-on-one interactions to larger group discussions. Nick developed a

close friendship with Karen early in the semester. But a bad experience on a

team project with another participant later soured his experience for the

remainder of the semester. Nick was widely read and could converse at length

on a variety of subjects and make connections between seemingly disparate

phenomena with ease and clarity. He has a passion for food: how it is produced,

processed, and prepared. Nick was a chef for a number of years before returning

to college, preparing to transfer to a baccalaureate program. He indicated that he

was recovering from substance abuse, a struggle that likely played a role in his

social anxiousness and uncertainty. The class learned much from Nick,

particularly about engineered food, pharmaceutical colonialism, and systemic

ways of thinking.

Adam, a Caucasian student in his early 20s, was the only student other

than Nick who was an academic transfer major, “going for an associate of

sciences degree.” Adam was an enthusiastic participant in in-class activities.

70
However, he displayed many of the characteristics attributed to Generation Y

(Armour, 2005, November 8). He had a great deal of difficulty self-regulating,

frequently blurting out irrelevant or tangentially related comments while others

were speaking and fidgeting unremittingly. These behaviors distracted and

annoyed the other participants to such an extent that they virtually ostracized him

from the group by the end of the semester. He didn’t seem to notice this,

however. His demeanor was always positive, even bubbly. He contributed terrific

ideas when he was focused. Unfortunately, he was unable to follow through on

them. When he made it to class, he was usually very late and rarely had

prepared what had been assigned to him by either his teammates or the

instructors.

Michael was the other ISGT major, and like Les and Adam was in his early

20s. The only African-American participant, Michael shared some of his cultural

heritage in class discussions. He described himself as “a very strong opinionated

person” who keeps his “opinion lax.” He was observed to be quite accepting of

others viewpoints, yet resolved in his own convictions. Michael engaged readily

in class discussions. He often took a leadership role in the development of the

game, drawing others back to task, and even proposing the overarching symbol

or organizing metaphor for the game: The Four Horsemen of the Apocalypse. He

was not as productive outside of class between class meetings. He spoke at

length in his interview about his need to be more disciplined, particularly when

others were relying on him to get things done.

71
 Trina was the co-instructor for the course. She has over 30 years of

experience teaching speech communications, English (both composition and

literature), and psychology in the community college setting. She was actively

involved in the Study Abroad and Honors programs at the college, and has won

the college Excellence in Teaching award. She described her instructional role as

“a real cheerleader and encourager,” a role which I feel she executes with

enthusiasm, insight, and deep care for students. Her instructional philosophy is

quite humble: “I don’t teach them anything. If the student isn’t ready, I can’t

teach. My job is to inspire and make them receptive, and then own their …

learning. I also see my job as helping them make connections between other

disciplines.” Trina did not have much experience teaching with technology, but

wants to learn more. She regrets that she “didn’t come along as fast with the

technology because [she] had to teach another course that was a totally brand

new” during the GVP semester. A testament to her willingness to take risks and

experiment with new approaches, she described teaching the GVP as “jumping

over a cliff into the darkness,” a risk she was willing to take “to get an opportunity

to teach with Mary Jo.”

Mary Jo was the other instructor for the course, the primary designer of

the GVP, the principal researcher for this study, and author of this dissertation.

Her view of her role as an instructor is nearly identical to Trina’s: encourager and

enabler of learning. Mary Jo has taught English and humanities in the community

college setting for over ten years. Unlike Trina, however, her experience teaching

72
with technology includes designing, developing, and teaching online courses, as

well as integrating technology into face-to-face instruction.

Data Collection

The text of the case for this study was created using data collected from a

variety of sources, a practice in keeping with Merriam’s (1988) assertion that

multiple sources of data inform interpretations of the case. The primary method

of collection was semi-structured interviews conducted with students and

instructors near the end of the implementation. In addition to the interviews,

course documents, student posts and responses in online discussion boards and

web logs were also collected.

• Observations of sessions of the course were made by the researchers

who later conducted interviews with course participants. These

observations served to orient the researchers to the dynamics of

participants and course activities. They also provided participants with a

degree of familiarity and comfort with the researchers.

• Interviews were conducted by non-participant researchers. All participants,

including the instructors, were interviewed. Student interviews followed a

semi-structured approach, which asked them to tell what they learned

about each of the core perspectives and identify what they learned about

them in other courses, in this particular course, and whether the course

project (developing an ARG) had an impact on what they learned about

each perspective.

73
 • Course Documents were collected, including the course syllabus,

assignment sheets, scoring rubrics, student papers, media presentations,

portfolios, and game design documents.

• Online Discussions & Weblogs were collected and archived, capturing

both student posts and responses to those posts by other students and

the instructors.

As indicated above, the primary source of data for analysis was the transcript of

interviews with participants. Other forms of data documented students’ active

involvement in the learning activities of the GVP but did not necessarily elicit their

perceptions of those activities. As such, these data sources served to triangulate

the interview data, and provided further grounding for assertions made about

student perceptions. One follow up interview was conducted with a student

participant after the data collection period had ended because analysts agreed

that passages in the transcript were overtly led by the interviewer and required

clarification. However, this student’s responses were the only ones that the

researchers found to require this member check.

Data Analysis

Interpretations derived from qualitative analysis, as with any inquiry

approach, must be grounded in the data collected. Grounding assertions from

within the data allows researchers to make interpretations that can be considered

representative of the observed phenomena. Evidence for interpretations in this

study were obtained from multiple sources, principally each participant involved

74
in the pilot implementation. Additional sources of data in the forms of discussion

logs, web logs, course documents, and researcher observations served to

triangulate the findings. In order to further assure the credibility of the

interpretations, three researchers analyzed the data, identifying the emergent

patterns within it, and constructing consensus regarding these interpretations.

In order to systematically analyze this data, a constant-comparison

approach to interpretation was taken by the research team (Glaser & Strauss,

1967; Strauss & Corbin, 1998). The constant-comparison method began with

identifying codes, a process of assigning labels or meanings to units of text. As

codes were identified, the researchers compared these codes to each other,

identifying similarities, differences, and patterns among them. By constantly

comparing the codes to the text, the researchers continued to refine the labels of

the codes and the meanings they assigned to them. As a result of this process,

assertions made about the data were supported or grounded in the data (Miles &

Huberman, 1994).

According to Strauss and Corbin (1998), coding procedures for the

constant-comparison method of analysis involve three phases: open, axial, and

selective coding. During open coding, the researchers code the textual data,

capturing properties discovered in the text. They also build categories of these

codes by constantly comparing the codes to the text and to each other,

identifying commonalities among codes and grouping similar codes together.

Axial coding takes a more focused pass through data. In this phase, the

75
researchers examine whether the codes and categories that emerged through

open coding, adequately capture the essence or axis of the data they represent.

As such, axial coding analyzes the codes and categories themselves and

continues the refinement of labels as well as the generation of new categories,

subcategories, and codes. By the final phase, selective coding, the researchers

have identified the principal themes among the categories. During selective

coding, examples are selected from the text that illustrate these themes and

further inform comparisons or contrasts within and among them.

In this study, analysis of the data included coding the transcripts of

interviews all six students and the two instructors who participated in the course.

Combined, these transcripts include 1260 passages, 35,660 words, and 195,242

characters, or 91 pages of text, including interviewer questions. Participant

responses were coded line by line; interviewer questions were not coded, except

in instances where the interviewee responded with an affirmation or negation,

such as “mmm hmm” or “not really,” of a statement offered by the interviewer.

Although identification of codes and categories was guided by the “themes” or

levels of evaluation (Kirkpatrick, 1994) outlined earlier in this chapter, the

researchers worked in concert during open coding to identify emergent codes

and categories within each theme and to construct a mutual understanding of the

text, themes, categories, and codes. Emergent codes were constantly compared

to previously identified codes, collapsed into categories representative of their

76
similarities, and refined as additional codes and categories emerged. This

process prompted further refinement of the themes as well.

Since many passages within the text informed multiple themes, the text

was segmented into four separate documents representing each theme prior to

axial coding. Most of the text was duplicated in more than one theme; several

passages were included in all four themes. All of the data collected from student

interviews was coded at least once--in other words, in at least one theme. Data

from instructor interviews was included only in the fourth theme pertaining to

overall results of the evaluation. Categories and codes in this theme were

generated first from student interview data, and instructor interviews were coded

using those codes and categories. A few additional codes that informed the

tensions and successes of student experiences with the course emerged from

the instructor data and were added to the existing categories. Passages that

pertained exclusively to instructors’ perceptions and approaches to teaching and

learning were not coded, but were used to construct the instructor descriptions

given above.

All phases of the coding were completed by three researchers. Open

coding was conducted in concert with at least two researchers present. This

coding was cross-checked later by a third researcher when only two researchers

had been present. Disagreements in the assignment of codes were discussed

until consensus was achieved among the three researchers. After open coding,

the researchers separately performed the axial coding. Disagreements in the

77
assignment of codes and the “fit” of codes or categories were discussed until

consensus was achieved among all three researchers. Selective coding was

conducted by the principal researcher (author) and verified by at least one other

researcher.

Summary

This qualitative case study was an evaluation of an educational research

and development, namely the GVP. The researcher approached this evaluation

from the constructivist-interpretive inquiry approach, using a constant-

comparison method to build interpretations of participant perspectives on their

experiences with the GVP. In order to establish “credibility, transferability,

dependability, and confirmability,” the interpretive process of coding was

completed with two other researchers. Thus, the interpretive claims presented in

the following chapter can be said to have been validated by multiple coders and

are therefore grounded in the data collected.

78
 CHAPTER 5

RESULTS

This chapter presents the results of the study, beginning with the coding

procedures and methods used to interpret their weight or strength relative to the

total text. These procedures are followed by an overview of each of the themes

of analysis, which were derived from Kirkpatrick’s (1994) levels of evaluation and

the research questions asked in this study:

1. What were student and instructor reactions to the course design?

2. What aspects of the design were most conducive to student

learning?

3. To what extent did the course promote attainment of the

overarching program objectives (core perspectives) and/or
advance conceptual age thinking?

4. What challenges or tensions arose from the design?

The remainder of the chapter presents the categories and codes within each

theme, discussing their relative weights within the total text, theme, and category

in which they’re situated. So that the reader might distinguish among the labels

for codes, categories, and themes more easily, codes are italicized, categories

are bolded, and themes are bolded and italicized.

79
 Analysis Procedures

After segmenting the data into the four themes, the researchers coded

each theme line by line, continuing to compare the data with the codes,

generating additional codes, and refining the code and category labels. A total of

1107 passages and 35,410 words comprised of 191,693 characters were coded,

yielding 27 categories and 157 unique codes. The amount of text that comprises

the categories within each theme is presented in Table 5.1.

Table 5.1

Amount of Text Assigned to Categories within Each Theme

Theme Categories Codes Passages Characters
Reaction Overall Reaction 3 25 1892
Prior Experiences 4 17 2019
Technology 3 16 2416
Course Format 3 13 2208
Instructional Design 6 45 7750
Instructors/Learning Environment 3 27 3216
Peers/Other Students 3 13 1511
Theme Totals 25 156 21,012
Learning Prior Knowledge 3 60 11,575
Self-Reflection 7 107 20,859
Motivations 5 15 3100
Instructors 3 6 776
Course Content 7 37 8387
Teamwork 6 89 12,669
Game Design Scenario 7 54 13,132
Technology 4 38 5834
Theme Totals 42 406 76,332
Behavior Individual or Personal Values 7 31 9138
Respect for Others 6 55 11,851
Social Responsibility 4 16 2849
Open-mindedness 4 26 6491
Knowledge Construction 5 82 16,898
Theme Totals 26 210 47,227
Results Curriculum & Assessment 10 60 9797
Instructional Methods 15 104 17,694
Course Format 8 34 4624
Technology 7 31 5637
Students 13 64 11,651
Epistemology 7 30 5730
Institution 4 12 1432
Theme Totals 64 335 56,565
Text Totals 27 157 1107 201,136

80
In order to determine how much of the text each category represents, both the

percentage of characters and the percentage of passages for each category

were calculated and averaged. While the percentage of characters gives a fair

depiction of how much of the entire text each category represents, it does not

account for how often a category or code occurs within the text. For example,

one or two interviewees might respond at length about a given aspect of their

experience while others do not address that same topic at all. This could yield a

substantive amount of text that occurs only once or twice within all of the data.

Conversely, however, calculating only the percentage of occurrences—or

passages—does not account for how much text comprises each category and

code. Some codes, for example, occur repeatedly throughout the text, but

responses are brief. Compare the number of passages and characters for

Overall Reaction and Open-mindedness in Table 5.1 above. Each has an

approximately equal number of passages, 25 and 26 respectively. However, the

amount of text (number of characters) for each is quite different: 1,892 for

Overall Reaction and 6,491 for Open-mindedness. Thus, Open-mindedness

accounts for a little more than three times as much text as Overall Reaction. To

best capture both the number of occurrences and amount of discourse attributed

to each category and code, both the percentage of passages and the percentage

of characters were calculated and then averaged. This statistic, presented as the

81
Passage/Character Mean percentage, was used to further analyze and interpret

the text.

Description Of Themes

As explained previously, analysis of the interview transcripts was shaped

in part by the research questions, which were shaped in turn by the overall

purpose of evaluation research. The preliminary “themes” to be explored in this

evaluation study parallel the four levels of evaluation that Kirkpatrick (1994)

describes in Evaluating Training Programs: reaction, learning, behavior, and

results. The first theme in this analysis, Reaction, includes categories and codes

that pertain to students’ reactions to various aspects of the GVP, such as the

instructional design, the hybrid format of the course, use of technology, the

learning environment created by the instructors, and their interactions with other

students. The Learning theme shifts away from Kirkpatrick’s level somewhat.

Rather than providing evidence of what or how much learning occurred, this

theme captures how learning occurred in the course: what motivated students to

learn, what learning preferences students possessed, what role the content

presented in the course played in students’ learning, and what impact the game

design scenario had on their learning. The behavior level of evaluation assesses

the extent to which participants change how they perform as a result of

participating in a given training program. In this study, it was not possible to

examine changes in learner behavior after completion of the course, nor was it

82
part of the overall purpose of evaluating a pilot implementation, which was

focused on collecting and interpreting data to inform refinements of the design

prior to further implementation. Instead, this theme assembles evidence in

student interviews that suggest changes in thinking, values, attitudes, and

behaviors as a result of their experiences in the GVP. A further discussion of this

theme follows in the next paragraph. The final level, Results, attempts to

determine the wider impact that a training program has on an organization, which

is often difficult to evaluate. In the case of the GVP, this level or theme

specifically identifies what worked well and what didn’t, the tensions and

successes that occurred in this semester-long implementation.

Although a more detailed description of each theme, along with its

corresponding codes and categories is provided later, the author finds it

imperative to further define the third theme prior to proceeding. Throughout the

process of constantly comparing the text with the emerging codes, categories,

and even themes, the label for and understanding of this theme underwent more

reconstruction and refinement than any other theme. At one point, the

researchers called it Values, but soon found that the text informing this theme

also included patterns of thinking valued in the conceptual age (Pink, 2006).

However, labeling the theme Conceptual Age Thinking seemed to limit the

category to thinking patterns alone, excluding many of the personal, as well as

the broader socio-cultural values that individual students described in their

interviews. Thus, the “working label” became an accretive one, containing

83
several descriptors separated by slashes:

Behavior/Values/Attitudes/Conceptual Age Thinking. The principal

researcher resolved to call this theme, Empathic Thinking, purely for the

purpose of reporting the results here. This label seemed to capture both the

cognitive and affective learning that students reported in their interviews.

Ultimately, however, that label didn’t capture the spirit of this level of evaluation.

The behavior level evaluates whether the learning that occurs in an

instructional program has an impact on the learner’s performance of his or her

job. In other words, the learning level evaluates whether, what, and how much

participants learned, while the behavior level evaluates whether they apply what

they have learned to the performance of their respective jobs. In essence, this

level of evaluation examines transfer of learning from the context of the

classroom to the actual job. Once again, in the case of the GVP, assessing

learner performance after the course ended was not within the scope of this

research study. Instead, what this theme is about is whether students, in their

self-reports during interviews, provide evidence that they have to some degree

internalized the values, attitudes, and patterns of thinking that were the

overarching goals of the design, such that those outcomes might govern their

behavior or performance in the future. Consequently, this theme was labeled

Transfer. This label accounts for the cognitive and affective learning that might

have occurred. It also broadens the label of the theme from the specific goals of

this course—fostering conceptual age thinking and the various values and

84
attitudes represented by the core perspectives—to the goals of this particular

level of evaluation: application of learning to new contexts. The author did not

want to further limit the theme to Knowledge Transfer only, as many of the

codes and categories within it relate to learning that had a perceived impact on

students’ internalizing and reconstructing the value structures that ultimately

govern their behavior.

A total of 157 codes were assigned to 27 categories within the four

themes of analysis. The seven categories in the Reaction theme account for the

smallest portion of the total coded text (12.5 %) while the eight categories in the

Learning theme account for the most (35.8%). Figure 5.1 visually represents the

magnitude of each portion of the overall text captured by each theme and

similarly displays the weights of the categories that comprise them.

Reaction

Learning

Transfer

Results

0 5 10 15 20 25 30 35 40

Figure 5.1. Composition of themes.

85
The Results theme contained the most codes, more than twice the number of

codes in the Reaction and Transfer themes, in large part because the

researchers identified separate codes related to tensions and successes within

each category. Table 5.2 presents the P/C Mean percentage for each theme,

along with those of the categories within each. It also presents the P/C Means of

each category relative to its respective theme, as opposed to the overall text.

Table 5.2

Passage/Character Mean Weights for Themes and Categories

Theme Categories # P/C Mean % P/C Mean %
Codes of Theme of Total Text
Reaction Instructional Design 6 32.86 3.96
Instructors/Learning Envir. 3 16.31 2.02
Overall Reaction 3 12.52 1.60
Technology 3 10.88 1.32
Prior Experience 4 10.25 1.27
Hybrid/ALCE 3 9.42 1.14
Peers/Students 3 7.76 0.96
Theme Totals 25 100.00 12.27
Learning Self-Reflection 7 26.84 10.02
Teamwork 6 19.26 7.17
The Game 7 15.25 5.70
Prior Knowledge 3 14.97 5.59
Course Content 7 10.05 3.76
Technology 4 8.50 3.17
Motivation 5 3.88 1.45
Instructors 3 1.25 0.46
Theme Totals 42 100.00 37.31
Behavior Knowledge Construction 5 37.41 7.90
Respect for Others 6 25.64 5.43
Individual Values 7 17.06 3.67
Open-mindedness 4 13.06 2.79
Social Responsibility 4 6.83 1.43
Theme Totals 26 100.00 21.23
Results Instructional Methods 15 31.16 9.10
Students 13 19.85 5.79
Curric. & Assessment 10 17.62 5.15
Technology 7 9.61 2.80
Epistemology 7 9.54 2.78
Course Format 8 9.16 2.69
Institution 4 3.06 0.90
Theme Totals 64 100.00 29.19
Text Totals 27 157

86
Detailed descriptions of each of the themes, including the categories within them

and the codes that make up each of the categories follow in the remaining

sections of this chapter.

Reaction Theme

Based on the P/C Mean percentage, the Reaction theme represents the

smallest portion of the total text, 12.5%. This theme includes seven categories

comprised of 25 unique codes, and 156 passages made up of 21,012 characters

of text. The categories and codes that underpin them paint a picture of students’

reactions to various aspects of their experiences in GVP, including elements of

the problem-based instructional design, the hybrid or blended format of the

course, the learning environment created by the instructors, and their interactions

with other students. Figure 5.2 shows the P/C Mean percentage of each category

within this theme, as opposed to its percentage of the entire text.

Reaction Theme

7.76
9.42
Instructional Design
32.86
Intructors/Learning Env
10.25 Overall Reaction
Technology
Prior Experience
10.88 Hybrid/ALCE
Peers/Students
16.31
12.52

Figure 5.2. P/C mean percentage of text in Reaction theme by category.

87
Of the seven categories, Instructional Design was by far the largest,

representing almost a third of the text in this theme. The Learning Environment

fostered by the instructors was also a significant category of students’ reaction to

the course, representing over 16% of this theme. Data capturing students’

Overall Reaction was the third strongest category, while Technology, Prior

Experience, and the Hybrid and learning community (ALCE) formats of the

course were roughly equal at about 10% each. The remainder of this section

details the codes that underpin each of these categories.

Instructional Design

By far the largest category in this theme, Instructional Design contained

6 codes representing 32.86% of the data in this theme and 4.05% of the Total

Text analyzed in this study. Table 5.3 displays each of the codes and their

relative weights within the category, the Reaction Theme, and the total coded

text.

Table 5.3

Codes in Instructional Design Category

% of % of % of
Pass Char Category Theme Total Text
Activities/Tasks 11 1696 23.16 7.56 0.92
Course Sequence 5 1717 16.63 5.69 0.65
Course Structure/Flow 7 1290 16.10 5.31 0.64
Teamwork/Comm. 8 969 15.14 4.87 0.60
Connections among tasks 7 1122 15.02 4.91 0.60
Game Scenario 7 956 13.95 4.52 0.55
6 45 7750 100.00 32.86 3.96

88
 Within this category, the Activities/Tasks code represents the largest

segment of the data. Students enjoyed the discussions and preferred the team

assignments to more traditional instructional methods. As one student expressed

it, “I do like [group projects] better than just having assigned readings out of a

textbook.” Another commented that the “class was really talkative and everybody

had their own ideals and kind of fueled the fire for discussion.” Related to that is

a comment made by Kevin coded as Teamwork/Communication, “I like classes

where you have discussions and kind of learn people’s opinions. Learning

through that…learning through other people, your peers.”

The Course Structure/Flow code captures reactions such as Kevin’s

observation, ”I think, it was more structured in the very beginning.” For Nick, the

course needed a bit more structure: “there were times when the sort of free form

flow of the class sometimes didn’t seem as organized or pointed or driven.”

Course Sequence differs from Course Structure in that this code specifically

captures students’ reactions to the course content introduced in the first few

weeks of the semester followed by the development of the game later on. As

Nick observed, “the ARG itself didn’t come ‘til the end. It almost kind of feels like

an afterthought.” Michael, a student in the Interactive Simulation and Game

Technology (ISGT) program, asserted that “game development should always

start with as much time as possible cause you’ll work through a concept for

months. It’s hard to get people to agree on one thing in just a couple months and

then get a final product done just from that.”

89
 In the Game Scenario code were comments such as Karen’s, “So it [the

ARG design project] has kept me engaged. …to me, it’s much more interesting

than just writ[ing] a paper or read[ing] a chapter.” While Karen and others found it

engaging, Nick had some difficulty with it. He stated, “specific to the ARG...one of

things that was frustrating to me was that it was a combination of individual work

and group work, but they never, at least for me, they never came together.” Nick

later admitted that he saw the Connections Among Tasks, another code in this

category, but that they seemed “superficial” because development of the ARG

was given short shrift by the sequence of activities in the course. This perception

is explored more fully in the Results theme. For others, the Connections Among

Tasks were more transparent. As Les reported, “after they [instructors] kind of

talked about it a little I kind of got the idea of how this was going to work. And

with that we just kind of took the time to, um, you know, put the pieces together.”

Instructors/Learning Environment

Student reactions to the instructors and the learning environment fostered

by them is the second largest category, accounting for 16.3% of the text in this

theme. Over half of the text in this category was assigned to the code, Positive

Energy/Encouragement from the instructors (See Table 5.4). As Les describes it,

“it was always positive energy…always positive. So it really encouraged

everybody to really get involved.” Karen adds, “I felt it was like a safe learning

environment in that I didn’t feel any sort of superiority kind of attitude or things

like that from the instructors. So I felt that they were interested in what I could

90
learn, making sure that I did learn.” The Open-ness of the Course was also a

strong code in this category. Kevin “liked the openness of it,” and Michael felt

“the openness of the way it is, really is a good way to learn.” The Professional

Atmosphere code captures comments such as this one by Les: “It is a classroom.

We know how to act and stuff. But it’s been very open, well, professional.”

Table 5.4

Codes in Instructors/Learning Environment Category

% of % of % of
Code Passages Characters Category Theme Total
Text
Positive
Energy/Encouragement 14 1736 52.92 8.62 1.06
Open-ness of Course 8 922 29.15 4.76 0.59
Professional Atmosphere 5 558 17.93 2.93 0.36
3 27 3216 100.00 16.31 2.02

Overall Reaction

This category accounts for 12.5% of the data in the Reaction theme, and

contains three codes documenting students’ overall reaction to the course:

Enjoyable/Fun, Frustrating, and Engaging/Challenging. Data describing the

composition of each code within the category is presented in Table 5.5. The

Enjoyable/Fun code captures a little over half of the text in this category.

Example passages from this code include the following:

Michael: “I did like the class. It was enjoyable.”

Les: “So, actually, I wanted more of it. I enjoyed it. I did enjoy it.”

Nick: “It was fun,” and “I enjoyed certain aspects of it.”

91
 Adam: “It has been a very good learning experience.”

The code, Frustrating, contained fewer passages and characters, but was

characterized by statements, such as “The class was fun and frustrating” and “I

don’t like open-ended things, so that’s frustrating. Whether it has anything to do

with this class or not in particular.” Kevin found that “it was hard to keep track of

everything we needed to do.” Statements coded as Engaging/Challenging

include Karen’s descriptors of the course as “definitely more engaging” than

other courses, “very thought provoking and…very challenging.”

Table 5.5

Codes in Overall Reaction Category

Code Passages Characters % of % of % of
Category Theme Total
Text
Enjoyable/Fun 13 951 51.13 6.43 0.82
Frustrating 7 562 28.85 3.58 0.46
Engaging/Challenging 5 379 20.02 2.50 0.32
3 25 1892 100.00 12.52 1.60

Technology

The Technology category contains three codes for student reactions to

the use of technology in the course (see Table 5.6). Overall, this category

accounts for under 11% of the Reaction theme, but documents mostly favorable

reactions to technology for learning. In the Usefulness code, Les reflects, “we did

use a lot of these projectors that were fantastic. We were able to illus[trate]…I

mean, we’ve got 3 screens everywhere. It’s great.” Adam concurs, ”I love

technology and I think it’s cool that classes are starting to get to use technology

92
more.” Most of the students were familiar and comfortable with using technology.

The most indicative passage in the Comfort/Familiarity code is Kevin’s assertion

that technology is “how I think, so I like using the technology.” Nevertheless, the

two passages coded as Too Many Technologies came from Kevin. “With the

different technologies, I think we were being introduced to so much, so many

different types of communication over the Internet that it was hard for us to figure

out which one everyone else was using.” However, it seems that his reaction has

less to do with technology itself, and more to do with connecting and

communicating with others when there were too many venues for doing so.

Table 5.6

Codes in Technology Category

% of % of % of
Code Passages Characters Category Theme Total
Text
Usefulness 10 1918 70.94 7.77 0.93
Comfort/familiarity 4 156 15.73 1.65 0.22
Too many technologies 2 342 13.33 1.45 0.18
3 16 2416 100.00 10.88 1.32

Prior Experience

Four codes underpin this category, which comprises 10.25% of this

theme. Student experience in Other Classes, the code which dominates this

category, contains comments such as “Well, other classes are kind of in the

same classical setting. Where the professor talks and you just kind of do essays

and such.” Two students talked about their Preconceptions About the Course.

Les stated, “I already came into it knowing that we would make this alternate

93
reality game, but I wasn’t really sure what it was about.” Another student said, “I

kind of imagined taking a class that I was going to be participating in an ARG

while I was actually designing an ARG, so that we were, you know, like learning

something about making an ARG while we were sort of playing one.” The

Technology code captures students’ prior experiences with technology, as

opposed to their experiences with it in this course. Karen reported to have had

“some knowledge of technology, but not a lot,” while Kevin stated that “Almost all

my classes have a computer in front of me.” The fourth code in this category

contains statements about students’ prior Life Experiences. For example, Karen

stated that she “felt comfortable with that [exploring other cultures] already

because I have a…I love to travel.”

Table 5.7

Codes in Prior Experience Category

% of % of % of
Code Pass Char Category Theme Total Text
Other Classes 6 914 40.28 4.10 0.50
Preconceptions About Course 4 659 28.08 2.85 0.34
Technology 5 296 22.04 2.31 0.30
Life Experiences 2 150 9.60 1.00 0.13
4 17 2019 100.00 10.25 1.27

Hybrid/ALCE

This course was a six-credit hour, learning community course (ALCE) that

met three hours each week accompanied by three hours of online work each

week. While these features did influence this instructional design, the

researchers coded student reactions to them as a separate category in order to

94
more fully capture and isolate students’ reactions to each. The Instructional

Design category pertains specifically to the problem-based instructional

methods, the developing an ARG course scenario, and the sequence of

activities. Those design components differ from what are essentially delivery

components in this category. For the most part, students did not react favorably

to the hybrid delivery mode as captured by the code F2F (face to face) Once Per

Week. As Les expresses it, “I think it would have done better if it…met twice a

week instead of the one, because it didn’t give you a whole lot of time for face

time” which is discussed more fully in the Results theme. They generally

responded favorably that it was a 6-hour Course. Kevin affirms that “it doesn’t

hurt that it’s 6 credits or multiple classes” Students also liked that it was Multi-

disciplinary. Les observed that “you cover a lot more subjects that a lot of other

classes covered and they kind of reinforce your learning from those classes

and…kind of add more to it.”

Table 5.8

Codes in Hybrid/ALCE Category

% of % of % of
Code Passages Characters Category Theme Total Text
F2F Once/Week 8 1428 63.11 5.96 0.72
6-hr Course 3 356 19.60 1.81 0.22
Multi-disciplinary 2 424 17.29 1.65 0.20
3 13 2208 100.00 9.42 1.14

95
 Peers/Students

The least significant category in this theme in terms of quantifiable weight,

the Peers/Students category represents only 7.76% of this theme and less than

1% of the total text. However, that weight is misleading in that students highly

valued learning from each other through collaborative projects. Those reactions

are documented in other categories and themes. What this category uniquely

captures are the types of relationships that students developed from those

instructional activities. For example, the Working Relationships code includes

Michael’s recognition that “having to network with people really becomes

important in that class, especially when we started working on that final game

project that we are doing.” Related to that code, however, Nick expresses

Frustrations, “I guess my other experience with everybody else in the class has

been mostly negative, I guess. [laughs] Um, I don’t know, I haven’t really been

able to develop much of a working relationship with ‘em.” Despite this frustration,

however, even Nick developed a close friendship with Karen. The other students

did, as well. As Les describes it, “We’ve kind of developed friendships…these

friendship type of relationships.”

Table 5.9

Codes in Peers/Students Category

% of % of % of
Code Pass Char Category Theme Total Text
Working Relationships 7 930 57.70 4.46 0.55
Frustrations 3 365 23.62 1.83 0.23
Friendship 3 216 18.69 1.48 0.19
3 13 1511 100.00 7.76 0.96

96
 Learning Theme

The Learning theme represents the largest segment of the total text,

37.31%. This theme includes seven categories comprised of 42 unique codes,

and 406 passages made up of 76,332 characters of text. The categories and

codes in this theme reveal students’ perceptions of how learning occurred in the

GVP, whether it was through presentation of content by instructors, the use of

technology, or through the central problem scenario, designing an ARG. This

theme also identifies what motivated students to learn, what learning preferences

students possessed, and what role knowledge gained prior to this course played

in students’ learning. Figure 5.3 below shows the P/C mean percentage of each

category within this theme. Table 5.2, presented previously in this chapter,

displays the P/C mean percentages of the total text.

Learning Theme

1.25
3.88
8.50
26.84 Self-Reflection
Teamwork
10.05
Game Scenario
Prior Knowledge
Course Content
14.97 Technology
Motivation
19.26
Instructors
15.25

Figure 5.3. P/C mean percentage of text in Learning theme by category.

97
Of the eight categories, Self-Reflection was by far the largest, representing

almost 27% of the text in this theme. Indeed, Self-Reflection is the largest

category in the entire text (10.02% of total text), comprising almost as much text

as the whole Reaction theme (12.27%). Teamwork was also a strong category

in this theme. While it represents fewer characters of text than the Game

Scenario (12,669 and 13,132 respectively), it had far more occurrences in the

text (89 and 54 respectively) giving Teamwork a higher P/C mean weight (7.17%

of total text) than student references to the Game Scenario (5.70%). Prior

Knowledge weighed in very closely behind the Game Scenario category. In

fact, it seems from these figures to have been more important than the Course

Content, a finding that I consider a success of the design for reasons that I will

discuss later. The data suggests that Technology, Motivation, and Instructors

played a less significant role in this theme with a combined weight (13.63) less

than that of Prior Knowledge alone.

Self-Reflection

Generally speaking, this category captures how students feel they learn.

Of course, this whole theme aims to capture that construct, but at the theme

level, the goal is also to determine what aspects of the design best facilitated

students’ learning. The text and codes in this category focus on students’ self-

reflections about how they like to learn, as well as what internal and external

factors promote or hinder their learning. Table 5.10 details the codes for this

category along with their theme and text weights.

98
Table 5.10

Codes in Self-Reflection Category

% of % of % of
Code Pass Char Category Theme Total Text
Peer Interaction 27 3519 21.05 5.63 2.09
Personal Interests/Values 19 3835 18.07 4.85 1.81
Self-Awareness 18 3084 15.80 4.24 1.58
Self-Regulation 11 4260 15.35 4.15 1.56
Learning Preferences 17 2774 14.59 3.91 1.46
Self-Evaluation/Assessment 11 3055 12.46 3.36 1.26
Inst-Student Interaction 4 332 2.66 0.71 0.26
7 107 20859 100.00 26.84 10.02

Peer Interaction was the strongest code in this category. Despite

frustrations with working in teams, most of students in the GVP prefer learning

from peers to learning from instructors. As Michael expresses it, “discussion

helps you develop your own opinion as well as learn from the others. And you

don’t learn so much from what just the teacher is teaching you. You learn from

your fellow classmates.” Les also felt he learned better from peers, noting “we

just kind of learned from each others’ experiences, and at the same time we just

understood the subject better.” Nick liked learning from peers more than listening

to instructors; however, he preferred one-on-one conversations. He confesses, “if

there’s more than one person that I’m talking to, I get frazzled easily. I have a

tendency to clam up.” As the data above shows, Instructor-Student Interaction

was not nearly as important to students as the Peer Interaction fostered by the

learning environment the instructors facilitated.

99
 Students’ Personal Interests also played a strong role in their learning.

Karen enjoyed the multi-disciplinary nature of the course, “because maybe

there’s one area that I’m more interested in than others, so it hasn’t been a whole

semester devoted to something I’m not interested in.” The Self-Awareness code

captures students’ identification of psychological and in some cases,

demographic attributes that had an effect on their learning. For example, Michael

mused, “I realize that I’m sometimes too much of a passive person.” For Karen,

“One frame of reference is that I am the eldest person in the class.” Self-

Regulation was also a strong code. As Les reflects, “I try to manage, like, where I

can talk and where I shouldn’t, ‘cause you know, if I don’t have credibility on the

subject what good does that do me?” Karen’s comment, “I do feel like I can self-

teach a lot of things,” fell into this code as well. Nick acknowledges that he has

“kind of a limited patience. [He] like[s] to work on things at [his] pace, whether

that’s fast or slow depends on the project.”

Learning Preferences was somewhat of a “catch-all” code. It included

Kevin’s assertion, “I’m not one of those people that do very well with online

classes. I like using the technology too. But I would rather have that as a

supplement instead of half the class.” Les’s statement, “My preference is sort of

talking about it and having some sort of visual representation,” was included in

this code as well. Although not as strong as Self-Awareness, the Self-Evaluation

code was still significant. Text in this code depicts students’ recognition of

strengths or weaknesses in their abilities. Nick, for example, avers, “I have a

100
tendency to, uh, at least to analyze what I do, and try not to make, you know,

rationalizations in order to justify poor reasoning. Sometimes it’s easier than

others.” When asked what he learned about his ability to make aesthetic

judgments, Adam stated that he preferred the objectivity of science and admitted,

“I don’t make aesthetic judgments that often because it’s hard for me to really pin

down what I find as beauty.”

Teamwork

The six codes in the Teamwork category depict not students’ reactions to

working in teams, but their perceptions of how it impacted their learning. Table

5.11 shows weights of the codes in this category. Participation/Roles was the

most heavily weighted code in this category. According to Kevin, this involved

“finding that niche that everyone has and trying to work that out. That’s another

thing with the organization of the group, trying to find everyone’s strength.” At

times, participation didn’t occur as needed. Nick encountered this problem: “I had

a project with another fellow in the class, [laughs] mainly he was absent so I

ended up doing the project primarily on my own.”

Michael summed up the Communication/Discussion code when he

observed, “we have one person who has a very affirmative personality about

what he wants and what he thinks. We have another person who is the complete

opposite view but is also very affirmative about what she thinks.” Karen, who

missed class for two weeks to travel abroad, regretted that she and her project

101
partner “had agreed before [she] left to communicate in a certain way and that

didn’t happen.”

Table 5.11

Codes in Teamwork Category

% of % of % of
Code Passages Characters Category Theme Total
Text
Participation/Roles 24 3375 26.80 5.17 1.92
Communication/Discussion 22 3226 25.09 4.82 1.80
Strategies/Processes 14 2262 16.79 3.21 1.19
Frustrations 14 1495 13.77 2.70 1.00
Common Interests/Goals 11 1242 11.08 2.17 0.81
Interdependence 4 1069 6.47 1.19 0.45
6 89 12669 100.00 19.26 7.17

Nevertheless, students developed some new Strategies/Processes to deal

with collaborative work. According to Michael, “when I usually work for the team, I

thought ‘I work on my part, you work on your part, and then we come together at

the end.’ It didn’t really work that well in certain situations, and this class was one

of those certain situations.” Students found that they needed strategies for

working together throughout the collaborative process. Indeed, one of the

principal Frustrations reported by students was working independently on group

projects. As Karen expresses it, “I have been frustrated when I’m working on my

own to do my own work that I’m supposed to bring back.” Kevin puts this

Frustration a little differently saying that “having a project that you’re the only one

working on, or having a project that no one can agree on how to do, so nothing

gets done.”

102
 These frustrations seemed to be mitigated when students found Common

Interests/Goals. As Nick puts it, “I feel like Karen and I worked, and still do, work

well together. I feel that we both have the same sort of goals.” Interdependence

was also a mitigating factor for some. Although Les talked about

Interdependence at length, Nick states it more succinctly, “with the group

projects, it was always important to me to, in fact it was more important to me

actually, to participate or to have more participation and to put more effort into

group projects than it was individual projects.”

Game Scenario

Obviously, collaborative teamwork was a substantial part of student work

on the central problem scenario for the course--developing an ARG, the Game

Scenario that labels this category. While Teamwork is a separate category in

this theme, it was an integral part of what we are referring to as the Game

Scenario. The codes in this category illustrate the impact that this overarching

design component had on their learning. Some of the codes seem to indicate

what they learned rather than how they learned. However, these codes do not

capture what or how much students learned, but rather what they attribute to

having learned from engaging in the development of an ARG aimed at impacting

the UN MDGs. Table 5.12 shows the codes in this category and their respective

weights.

103
Table 5.12

Codes in Game Scenario Category

% of % of % of
Code Pass Char Category Theme Total Text
Applying & Integrating Knowledge 26 6122 47.38 7.21 2.70
Developing Narrative/Art 9 2646 18.41 2.84 1.06
Health Issues 7 2142 14.64 2.27 0.85
Global Awareness 6 940 9.13 1.35 0.50
Self-Efficacy 3 592 5.03 0.76 0.28
UN MDGs 2 379 3.29 0.49 0.18
Environmental Issues 1 311 2.11 0.33 0.12
7 54 13132 100.00 15.25 5.70

Applying & Integrating Knowledge was by far the strongest code,

representing nearly half of the text in this category (47.38%). To a certain extent,

the multi-disciplinary nature of the course compels learners to integrate

knowledge, but text in this code specifically attributes this cognitive task to the

Game Scenario. For example, Kevin declares, “we definitely had to do it for the

ARG…because you have to have the lit--the writing and literature--and then the

technology aspect and integrate them…with different people.” They drew from

more than just the course content, as well. As Adam points out, “you know, all of

us come from different family backgrounds and all that stuff and we’re all teaming

up together to make this one game, and we’re pulling ideas and locations from

different parts of the world.”

Developing Narrative/Art for the game was another application of

knowledge and skills that they attributed to the Game Scenario. For Les, it was

crucial to the effectiveness of the game narrative that they rectify “gaps between

the story and come up with something that would be reasonable and logical, in a

104
series of…in a timeline.” Nick shared Les’s concern about both narrative and art,

wondering, “Is it going to look good?” and “worrying about how its going to come

together, worried that’s its not going to be, you know, cogent…or coherent.”

Ironically and delightfully, Health Issues is a fairly strong code in this

category. One of the core perspectives regards the importance of maintaining

health and wellness. When asked what they learned about that, students

generally replied that it wasn’t covered in this class. Yet because the game

scenario was coupled with the MDGs, a few of which address issues of global

health, students learned a remarkable number things about Health Issues. For

example, Les explains, “since we were going to do sort of the Four Horsemen of

the Apocalypse that kind of says a little bit of it. ‘Hey, you should probably eat

well and kind of avoid all of these toxic materials that are, that may be in your

food’.” Les refers here to the Four Horsemen (an application of archetypes and

symbols which they studied earlier in the semester), which framed the levels of

the game they designed. It was student research on famine (one of the

Horsemen/game levels) that led to extensive discussions of toxic preservatives

and genetically modified organisms (GMOs) in food production.

Global Awareness was also a fairly significant code although surprisingly

not as strong as Health Issues. Again, students attributed this learning to what

they explored while developing the game. According to Kevin, “I think in just

these past few weeks that we’ve learned more about what we are because of the

ARG…about us as a whole and how our contributions make a bigger difference.”

105
Karen confirms this, “I feel much more connected to the world as whole. I’ve

researched some things that I had not researched before or spent time looking

into, so I feel like more of an actual global citizen than I did before.” The

remaining codes, while interesting and directly attributable to the development of

the game, did not represent a substantial portion of the category or the overall

text.

Prior Knowledge

From a design standpoint, the strength of the Prior Knowledge category

illustrates a vital component of the GVP as a capstone course. While the

instructors of the course might not look favorably on Other Classes (a code within

this category) carrying a heavier text weight than the content of this course, it

doesn’t actually do so. This category encompasses Prior Knowledge from Life

Experience, as well as Reading/Media that students engaged with outside of

class. As Table 5.13 shows, the Other Classes code accounts for 3.28% of the

text, slightly less than the Course Content category, a discussion of which

follows this section. In a capstone, one expects students to draw upon prior

knowledge from a variety of sources and experiences, bringing them to bear on

the construction of new knowledge in the capstone. This also appeared to be the

case with the GVP.

106
Table 5.13

Codes in Prior Knowledge Category

% of % of % of
Code Passages Characters Category Theme Total
Text
Other Classes 37 6464 58.76 8.79 3.28
Prior Life Experience 18 3767 31.27 4.68 1.75
Reading/Media 5 1344 9.97 1.50 0.56
3 60 11575 100.00 14.97 5.59

Other Classes represents a larger portion of the category than other

sources of prior knowledge. Adam talked about his pursuit of science, “I have

always been into science and technology. I mean, heck, I’m going for an

associates in science. I’ve taken…chem[istry] for instance, but, I think that

science and technology…I’ve been able to apply them in my life.” Les recounted

at length the impact that his Project Development class had on this course, but

also affirmed that “any class or any experience kind of builds upon” prior

knowledge. Statements in the Life Experience code include Michael’s revelation

that he learned much from “just going through life dealing with everything I’ve

had to deal with growing up,” and Les’s reflection, “my dad’s a civil engineer so I

kind of grew up on, kind of knowing how schematics look.” In the Reading/Media

code is Adam’s rather apologetic statement, “Well, this isn’t exactly related to the

class, but I‘ve learned that after watching enough CNN and a lot of other news

channels, I’ve learned a lot more about the politics of this country and obviously a

little bit about the world.” Nick was well read, and often sought out books that

weren’t assigned for class, but connected to something discussed in it. “I read on

107
a pretty broad sort of spectrum,” says Nick. “I just picked up Edward O. Wilson’s

book, Consilience,” which he talked about at length in the interview.

Course Content

The impact of the content and activities in the early part of the course was

worthy of note, representing over 10% of this category and 3% of the text overall.

What seems to have had the most influence, at least on the core perspectives

about which students were explicitly asked, was their culture project. The Other

Cultures code represents almost 40% of this category (see Table 5.14). The

students seem to have learned much from this three-week project, which asked

them to explore and then present to the class, significant works of literature, art,

and architecture of another culture. Michael comments:

I’d have to say a time when I was successful was when we were doing our
cultural projects. Me and my partner were assigned the Southern
Americas. That’s not my greatest area of forte. I’m more of the European,
the Asian, and the upper Americas. So I would have to say, I really had to
come out on my own on that one and learn things I never even knew.
Like, I didn’t know that there were this many tribes in Brazil, or I didn’t
even know Brazil had a specific form of marshal arts that was all their
own. I kinda of thought that was just a developed style from somewhere
else, so that one project taught me a lot.

Adam shared similar perceptions, “Me and my partner, Nick, we decided to do

the country Japan. I love the country Japan because I watch a lot of Anime and

play a lot of video games that come from Japan, and so I thought ‘Hey! It would

be cool to learn about the history...and so basically we were looking up historical

and cultural aspects of it.”

108
 The discussions and presentations on Architecture/Art led by the instructor

early in the class also made a prominent impression. According to Les, “I

remember the architectural thing being the thing I like the most and the

explanation behind Gothic architecture and the reason why they were so tall and

just the time period they were set in and the reason why they were...and the

explanation behind gargoyles, and stuff like that.” Text in the Multiple Disciplines

code is typified by another comment from Les: “This [class] kind of brought a lot

of them together. Architecture and religion and all of this stuff kind of tied in

together.”

Table 5.14

Codes in Course Content Category

% of % of % of
Code Passages Characters Category Theme Total Text
Other Cultures 13 3435 38.05 3.85 1.44
Architecture/Art 10 2536 28.63 2.89 1.08
Multiple Disciplines 6 831 13.06 1.28 0.48
Research/Documentation 3 766 8.62 0.87 0.33
Writing/Grammar 3 429 6.61 0.65 0.24
Archetypes/Symbols 1 233 2.74 0.28 0.10
Literature 1 157 2.29 0.23 0.08
7 37 8387 100.00 10.05 3.76

Technology

Students’ reactions to technology were reported in the previous theme. In

this category, codes represent how they learned with technology. Along the way,

they acquired some new Tools, which was the largest segment of this category.

According to Kevin, who is a technology guru, “Mary Jo definitely had a lot of

109
different resources that we used that I’ve never used before.” Nick “thought about

how we have used those sort of Web 2.0 tools…that’s been fun.” Students not

only learned about new tools, they saw how they might Apply/Use them in Other

Contexts. Karen discovered that she was “not too old to get involved and learn it

[technology] and make it part of everyday life.” Adam stated, “I’ll use it

[Powerpoint] in the future because it is a handy little tool.” They also saw how

technology afforded Clearer Communication. Les was thrilled that they “were

able to use graphics and video and everything . . . and then we kind of talked

about it. And used those to sort of illustrate our presentations better.”

Adam argued that “if it’s good technology and it’s working correctly it can help

make the learning easier.” Technology also afforded Active Involvement. As Les

puts it, “we would show something on the screen and maybe one of us had a

reference to what we were talking about and we kind of just talked about it a little

bit more.” For them, it was “more active other than just having a plain old text,

standard, Powerpoint presentation.”

Table 5.15

Codes in Technology Category

% of % of % of
Code Pass Char Category Theme Total Text
Tools 16 1985 38.06 3.27 1.22
Apply/Use in Other Contexts 9 1478 24.51 2.08 0.77
Clearer Communication 7 1294 20.30 1.71 0.64
Active Involvement/Interaction 6 1077 17.13 1.44 0.54
4 38 5834 100.00 8.50 3.17

110
 Motivation

The Motivation category is relatively insubstantial with respect to the

entire text and even this theme. Indeed, the researchers initially identified

Motivation as a code within the Self-Reflection category, rather than a category

itself. However, we soon discovered that students reported different sources of

motivation and wanted to capture those differences, particularly with respect to

the design of the course. Ultimately, if we had left Motivation as a code in Self-

Reflection that category would represent 11.47% of the total text. Table 5.16

provides descriptive data of the codes in this category.

The data suggests that what most motivated students was the Relevance

of the learning tasks and content to their immediate lives. Nick felt that “it’s hard

to motivate myself to do tasks that I don’t see…I can only seem to see long term

benefits for. If I don’t have a short-term benefit, it’s hard for me to uh, you know,

aside from the pleasure of completion, it’s hard for me to get involved in that.”

Personal Interests/Goals were also a motivation. For Nick those were fused the

Relevance or Connections to Life. “I guess that was kind of fused with, uh, my

interest in the task or my ability to get involved in the task.” Les’s “driving force is

inspiration.” If he is inspired by Personal Interests, he’s motivated to Do Well.

“And with that motivation, I’m able to go into other classes such as this one and

when something get’s assigned, say for a schematic or something for a military

vehicle, I’ve done that before. So, I’m able to, say, take this assignment and

111
make it look nice because I’ve done that before. And I’ll make it as best as I

possibly can.”

Table 5.16

Codes in Motivation Category

% of % of % of
Code Pass Char Category Theme Total
Text
Relevance/Connections to Life 5 937 31.78 1.23 0.46
Personal Interests/Goals 3 945 25.24 0.99 0.37
Grades 4 456 20.69 0.79 0.29
Doing Well 1 594 12.91 0.51 0.19
Reliability/Interdependence 2 168 9.38 0.36 0.13
5 15 3100 100.00 3.88 1.45

Instructors

As discussed earlier, students felt they learned more from each other than

from the instructors. For them, the learning environment that the instructors

created was highly important, in that it allowed them to learn better from their

peers. Part of that role was Focusing Discussions to relevant learning topics. Les

comments that despite the open-ness of the course, “I wouldn’t say we

sidetracked or anything. It was still on the same relevant topic.” Karen found that

a real time-saver, “because I haven’t had to wade through other things. It has

been very focused.” Nick felt the Support Outside of Class to be particularly

helpful. “I think the strongest guidance that I got was talking with Mary Jo in her

room [office]. When I would have questions outside of class, then she was

always available to talk to me for that. She helped me out quite a bit.” Kevin’s

description of Trina’s straightforward teaching style was also telling. “She just

112
kind of laid it out there, and I think I learned more from that than half a semester

in a normal English class. ‘Cause it was just straightforward.”

Table 5.17

Codes in Instructors Category

% of % of % of
Code Pass Char Category Theme Total Text
Support Outside of Class 2 289 35.29 0.44 0.16
Focusing Discussions 2 272 34.19 0.42 0.16
Straightforward Instruction 2 215 30.52 0.39 0.14
3 6 776 100.00 1.25 0.46

Transfer Theme

As explained earlier, the Transfer theme assembles evidence from

student interviews that suggests that they have to some degree internalized the

values, attitudes, and patterns of thinking that were the overarching goals of the

design: the core perspectives and conceptual age thinking. The Transfer theme

represents 21.23% of the total text. The theme includes five categories

comprised of 26 codes and includes 210 passages made up of 47,227

characters of text. The categories and codes in this theme relate to patterns of

thinking, as well as personal and socio-cultural values or responsibilities. Figure

5.4 below shows the P/C Mean Percentage of each category within this theme.

Table 5.2, presented previously in this chapter, displays the P/C Mean

Percentages in this theme in relation to the total text.

113
 Transfer Theme

6.83

13.06

37.41 Knowledge Construction

Respect for Others
Individual Values
17.06 Open-mindedness
Social Responsibility

25.64

Figure 5.4. P/C mean percentage of text in Transfer theme by category.

The Knowledge Construction category represents the largest segment of this

theme at 37.41%. a finding which is promising with respect to the goals of the

design. Indeed, this category represents 7.9% of the total text, the third largest

category in overall text. The Respect for Others category is also strongly

represented at 25.64% of the theme and 5.43% of the text overall. Individual

Values is the third largest category in the theme, followed by Open-mindedness

and Social Responsibility.

Knowledge Construction

This category contains five codes that represent a sequence of thinking

patterns indicative of building or constructing knowledge. The first is synthesizing

existing knowledge, in this case coded as Integrating Disciplines, followed by

114
Making Connections to New Ideas, then Applying to New Contexts in order to

Create/Construct Something New. A phase of this process that occurs

throughout is evaluating new constructions; however, we have placed that code

at the end of the sequence as it represents the highest level of cognitive skills.

Table 5.18 shows each of these codes and their weights within the text, but I’ve

chosen not to sort them by weight in order to preserve the sequence of the

thinking pattern. Integrating Disciplines, the first category in the sequence, is the

strongest code in the category. Although Applying to New Contexts has a slightly

higher weight than the second code in the sequence, Making Connections to

New Ideas, they differ only by a fraction of a percent. Likewise, Evaluating

Against Norms is larger than Creating/Constructing Something New although the

weight difference is more substantial than the difference between the two

previous codes.

Table 5.18

Codes in Knowledge Construction Category

% of % of % of
Code Pass Char Cat Theme Text
Integrating Disciplines 21 5540 29.20 10.87 2.33
Making Connections to New Ideas 17 3612 21.05 7.87 1.67
Applying to New Contexts 18 3536 21.44 8.03 1.69
Creating/Constructing Something New 12 994 10.26 3.91 0.79
Evaluating Against Norms 14 3216 18.05 6.74 1.43
5 82 16898 100.00 37.41 7.90

115
Before discussing each of these codes separately, the following passage from

Les’s interview is provided as an example of how one student proceeded through

this sequence:

Integrating It was the background information with archetypes. We

Disciplines covered archetypes and storylines… a lot of those concepts
come from other… from folklore, stories from different
religions, different countries, and nations and so on.
Making So after that, we also had more projects that involved
Connections to researching other cultures and with that you kind of learn
New Ideas how they believed in this and how history kind of led ‘em to
what they are today
Applying to New and how you can use a lot of that information to
Contexts understand how … what they believe in, what they
don’t like, kind of how they … interact with other
societies
Creating/ and at the same time use a lot of those references or
Constructing something. And kind of mix it together and make it a
Something New game.

He doesn’t arrive at Evaluating Against Norms in the passage above, but does so

in a later passage:

Self-Reflection I’m more of an artist person and . . . I come from a

background where I just kind of design,
Integrating and we’ve kind of used my own designs and art kind
Disciplines of things. And I’ve even textured a couple things, a
couple units and such. And with that motivation,
Making I’m able to go into other classes such as this one and
Connections to when something get’s assigned, say for a schematic
New Ideas or something for a military vehicle, I’ve done that
before.
Applying to New So, I’m able to, say, take this assignment and make it
Contexts look nice because I’ve don’t that before.
Creating/ And I’ll make it as best as I possibly can,
Constructing
Something New
Evaluating show it to other people, and hey, if they like it, perfect. If they
Against Norms don’t, if they want to suggest something, by all means I’ll go
ahead and fix it. . . And make it as best as I possibly can.

116
Of course, Les was not the only student who talked about Integrating Disciplines.

Kevin’s assertion, “We definitely had to do it [integrate scholarly disciplines] for

the ARG,” provided in the discussion of the Applying & Integrating Knowledge

code in the Game Scenario category of the Learning Theme was also coded as

Integrating Disciplines in this category and theme. In part, the strength of this

code derives from the fact that students were specifically asked to tell about what

they learned about “integrating knowledge and understanding the

interrelationships of the scholarly disciplines,” one of the core perspectives.

Adam’s musings on this perspective also illustrate the degree to which

students engaged in this type of thinking.

Well, the integration of knowledge is important because depending on how

you view something, you will figure out after awhile, or at least I have that,
everything is interconnected to a degree. Because, for instance, all the
branches of science, as just one random example, all come from
mathematics. From mathematics, then you get, then you get physics.
From physics, then you get chemistry. From chemistry, you get
biochem[istry]. And you get biology, and then you get anatomy, and then
it all breaks down from there.

Indeed, when asked about his ability to make aesthetic judgments, Adam initially

demurred (as discussed in Self-Reflection), but then Applied & Integrated

Knowledge from his background in science to an aesthetic depiction of the

universe. “For instance, I’ve always found it kind of interesting that, depending on

how you want to look at it, the string theory almost practically says that the

universe is kind of a cosmic symphony.”

117
 Adam takes those thoughts a step further in comments subsequently

coded as Making Connections to New Ideas. “Within science, you know, there’s

kind of an artistry to things, like for instance, there is a mathematical equation, I

want to say it’s E, no. I forget now. It’s like 5, the divine constant. Because like,

it’s like the length of the spirals on a seashell—or something crazy like that-- I

mean it’s everywhere. So you know, some people see beauty in that, and so

that’s kind of an artsy kind of thing. So everything connects in its own interesting

way.” Nick, the avid reader, made many connections to new ideas. As mentioned

earlier, he talked at some length about the book, Consilience. A passage from

that discussion fell into this code. “What Consilience or whatever, keeping that in

mind, you’re able to see natural processes and social processes as part of big,

interconnected, I don’t know, inextricable processes. Processes that are mutually

inextricable. I mean, they’re all dependent. So I mean, I see a lot of worth in that.”

Students also talked about taking new ideas and Applying [them] to New

Contexts, the development of the game, for example. As Karen puts it, “because

of the research to develop it. I mean it’s just all tied into the research to be able to

put all the pieces of that ARG together.” Les saw how the activities involved in

developing the game applied to work outside of the academic realm. He

expressed an appreciation for “the way that it prepares you to go into real life

settings.”

What distinguishes Applying to New Contexts from the code,

Creating/Constructing Something New, is that passages in the former discuss

118
how ideas or skills might apply while those in the latter more explicitly mention

the generation of “something new,” be that a new idea, image, or even the game.

As Kevin points out, each phase of the course, even the early weeks prior

development of the game, involved the creation of new artifacts. “So you have to

concentrate on different sections and combine them into whatever we’re doing,

the proposals, the work that we did on the ARG.” While the previous passage

denotes a more global perspective on the entire course, another passage in this

code represents a more localized view. In this passage, Les talks about

modifying images for the game so that they convey a specific message to the

player: “take a photo and Photoshop a photo and kind of change a few things

here and there and make it look like it was actually supposed to be, like it’s

supposed to be related to the subject.” Creating Something New occurs in simply

taking a photo, but it extends beyond that to modifying or “Photoshopping” it so

that it might better fit within the larger creation: the game itself.

In that passage, Les exercised evaluative thinking as he discusses

creating images. As mentioned earlier, Evaluating Against Norms occurs

throughout the process of Knowledge Construction. Because of that, it was

often difficult to separate text that might fall in this code from that which might

belong to other codes in this category. To a certain extent, that is why the codes

in this category were not ordered by rank. The weights in this category are not as

telling as they are in other categories because much of the text could be coded in

more than one way. Nevertheless, examples of students Evaluating Against

119
Norms are present. For example, Kevin recounts, “I built the website, so I had to

kind of view other people’s judgments of what professionalism looks like and

what certain sites look like to try to trick them into believing that these companies

were someone else.” What he means by “someone else” is someone other than

him or that “these companies” were real, rather than the fictitious constructions of

students in a class. As discussed earlier in the code, Developing Narrative/Art,

Nick worried about the coherence of the narrative, “the kind of, thematic flow,”

which provides another example of Evaluating Against Norms. For him, the

player experience was very important: “I just want it to be strong . . . I want the

player, you know, to experience being strongly directed. Rather than…wondering

what’s going to happen next…passively.”

Respect for Others

Of the six codes in the Respect for Others category, Different Cultures

was the strongest, which correlates with the finding in the Learning theme that

the Other Cultures code was a significant part of the Course Content category.

The following passage from Les typifies passages in this code, “You really need

to understand the history of the people and…kind of their background so you can

respect their ideals and their beliefs and such.” The insights of Michael and

Adam quoted earlier in the Other Cultures code for Course Content also fell in

this category and code.

The Opinions/Perspectives code contains statements in which students

express respect or appreciation for other people’s opinions. Kevin reflects that

120
the course made him, “a little more aware, more accepting, I guess. Just listening

to people. I’ve been fairly good at that, but sometimes there [are] certain topics

that I tend to shut down on because there’s no way I’m going to change my view

on it. But I learned a bit more to at least listen and understand their point of view

so I can understand them more.”

The Social Negotiation/Consensus code contains passages in which

students express an understanding of the process of social negotiation toward

building of consensus. As Karen expresses it, “the downside to [working

collaboratively)], is if you have very oppositional opinions, and for whatever

reasons have trouble reaching a compromise. And so that, you know, is gonna’

happen everywhere. And that’s part of learning to work things out with another

person.” Karen’s statement about social negotiation actually followed one in

which she expresses the Value of Collaboration. “I think the biggest benefit of

collaboration is just other thoughts. I mean just the, you know, everybody comes

from a different place. And so to have more than one set of ideas.” This code is

not about students liking or disliking collaborative work, it’s about their seeing the

value in it, whether they like it or not. Nick did not like collaborative work, but he

did recognize the value of it: “I mean, you know, I recognize group work is

important to both the scholarly disciplines and like the general marketplace…but I

just personally hate it.”

Differing from a respect for the Opinions/Perspectives of others,

Strengths/Talents/Goals codes statements regarding the respect of these

121
qualities in others. As Les puts it, “this in itself, to understand what everybody,

the other students, what they do better and then use those assets to you know,

give ‘em a certain job.” Kyle expresses this respect similarly, “a lot of times you

just have to find out what people are good at. Because sometimes they don’t

really know themselves until they start trying things.” In addition to a respect for

Other Cultures, students expressed respect for Differences Within [their own]

Culture. Karen expressed it regarding age differences, “Actually, it reminds me,

again, because these…my classmates are my kids ages…and even though I

respect my adult children, it reminds me of being that age and how much I did

have to offer. So, it just kind of brings back to mind that, yes, you really have to

listen, to even young adults because they do know things that I don’t know.”

Other passages in this code relate to differences in personality, religion, and

gender.

Table 5.19

Codes in Respect for Others Category

% of % of % of
Code Pass Char Category Theme Total Text
Different Cultures 13 3051 24.69 6.33 1.35
Opinions/Perspectives 11 2346 19.90 5.10 1.08
Social Negotiation/Consensus 10 2095 17.93 4.60 0.97
Value of Collaboration 8 1724 14.55 3.73 0.79
Strengths/Talents/Goals 8 1182 12.26 3.16 0.66
Differences w/i Culture 5 1453 10.68 2.73 0.59
6 55 11851 100.00 25.64 5.43

122
 Individual/Personal Values

This category contains seven codes representing individual or personal

values, as opposed to values or thinking patterns associated with relating to

others. The strongest code in this category deals with the importance of

Exercising Judgment or Being Informed rather than blindly accepting things at

face value. It includes statements such as this one made by Karen, “I have way

too much trust in what is at the WalMart and the Tom Thumb and, you know, so

I’ve been made very aware.” Similarly, Adam finds that “it’s very important to

know what is going on in the world because otherwise, you know, you don’t know

what you need to look out for.” Statements conveying the importance of a Work

Ethic made up the second largest code in this category. Les expresses it this

way, “Oooh. Well. You’ve gotta push yourself always.” A comment made by

Michael in this code was his response to the question related to the importance

of integrating the scholarly disciplines. “Well, I’ve learned that discipline on the

scholarly level is very essential when it comes to getting a lot of work done on

this level done. When in high school people hold your hand. And in middle

school people really hold your hand. And in elementary school people get paid

to hold your hand. But in college, you’re on your own.” For him, “scholarly

discipline” meant self-discipline.

The code Creativity/Self-Expression/Emotion is another rather accretive

code label. Interestingly, many of the passages in this code were given in

123
response to the question about using logical reasoning in problem solving. Take

the following dialogue between Karen and her interviewer, for example:

Karen: I think something can be emotion driven and that’s been some of
our discussions back and forth.

Interviewer: OK. So you would tend to depend more on your instincts…

Karen: yes.

Interviewer: …and emotions than a logical decision?

Karen: Yes.

Interviewer: OK.

Karen: Yeah. Especially for creative things.

Michael responded to this question thus, “In this class I’ve learned that logic

really doesn’t help except when you are doing that which is illogical. Art, writing,

a lot of these things are illogical. That’s just my personal view, but for writing,

writing is the expression of the writer. It is not meant to be logical to everyone.”

Only two passages were coded as Logic/Rationalization. Nick’s statement, “I

wouldn’t say I’m always logical. But I have a tendency at least to analyze what I

do, and try not to make uh, you know, rationalizations in order to justify poor

reasoning,” quoted earlier in the Learning theme as an example of Self-

Evaluation also fit here in this category.

An appreciation for Wholeness, balancing mind, body, and spirit, was

another fairly strong code. For Karen, this involved balancing logic and emotion.

She feels that her experience in the course “has reinforced that I come from an

124
emotional place [laughs] rather than a logical place, which I’m sure my

classmates would agree with, but that makes for good … I think you need that

balance.” For Michael, this related to self-discipline, as well. “I personally think of

wellness as a serene state of being. Others may not view it that way, but that is

the way I view it. So, my wellness learning . . . I’d have to say, is just getting

everything together, not putting things off until the end. Keeping things organized

really helps to improve your wellness.” While not as strong a code,

Reliability/Responsibility contained passages related to being a reliable member

of a team. As Nick expresses it, “I’m a lot more dependable if I’m afraid I’m going

to disappoint somebody, in a way that I’m afraid it’s going to hurt them in some

way.”

Table 5.20

Codes in Individual/Personal Values Category

% of % of % of
Code Pass Char Category Theme Total
Text
Exercising Judgment/Being Informed 8 2192 24.90 4.23 0.91
Work Ethic 3 2351 17.70 3.20 0.72
Creativity/Self-Expression/Emotion 6 1254 16.54 2.76 0.58
Wholeness 4 1528 14.81 2.57 0.56
Reliability/Responsibility 4 889 11.32 1.89 0.40
Honesty/Avoiding Plagiarism 4 624 9.87 1.61 0.34
Logic/Rationalization 2 300 4.87 0.79 0.16
7 31 9138 100.00 17.06 3.67

Open-mindedness

This category represents unique codes that differ from those in the

previous three categories. Knowledge Construction deals specifically with

125
thinking patterns showing a progression through the process of constructing

knowledge. To an extent, Open-Mindedness is part of that process, but it’s more

of an attitude or value than a progressive pattern of thinking. It must be present

for knowledge construction to occur, but differs from it, as well. In some ways,

Open-mindedness is an Individual/Personal Value; however, it also deals in

subtle ways with relationships outside of the individual. Notwithstanding, it is not

the same as Respect for Others, be that their opinions, cultural background, or

strengths. This category is more about being open to new ideas rather than

people, including technologies, works of artistic expression, and even re-

constructions of previously held knowledge constructs. Table 5.21 details the

codes in this category.

Table 5.21

Codes in Open-mindedness Category

% of % of % of
Code Pass Char Category Theme Total Text
Technologies 9 2232 34.50 4.51 0.96
New Ideas/Perspectives 9 1519 29.01 3.75 0.78
Reevaluating Old Ideas 5 2047 25.38 3.36 0.73
Aesthetic Appreciation 3 693 11.11 1.45 0.31
4 26 6491 100.00 13.06 2.79

The Technologies code captures passages expressing an appreciation for

or open-ness to new technologies. According to Adam, “without science there is

no technology cause, those are what give you the formulas and other things that

you need to learn how to build it…and so I think that I’ve gotten truly an

appreciation of just how useful all that is because we’re using a lot of

126
technology—you know we’re using computers, we’re using Websites—all that

stuff to put our game together which couldn’t have been done even 20 years

ago.” The New Ideas/Perspectives code is represented by passages such as this

one from Michael, “It’s really hard to be just one thing and not open yourself up to

other things, ‘cause you feel limited. Your view is very limited. You don’t get to

experience a lot of new things.”

The Re-evaluating Old Ideas code doesn’t contain very many passages,

but contains much more text than New Ideas/Perspectives. Responses in this

code include reflections such as this one made by Nick:

Oh, earlier when we were talking about Wooten Hall, I guess I called it a
big ugly box. Uh, we talked about some architecture and I know that’s
actually a style [laughs]. I happen to, you know, NOT like it very much, but
uh, but it’s a style, it represents something, it’s not just a cheap efficiency.
So, I get it in that way. It’d been really easy, you know, for me to just call
buildings like that big ugly boxes without recognizing that the architects,
builders had, I guess values that they were both using and showing by
building it. Or designing it.

While Nick is talking about the architecture and aesthetic judgments, his

comment was coded as Re-Evaluating Old Ideas because he is doing just that.

Much of what was presented to students on art and architecture challenged their

pre-existing notions that art is simply a matter of personal taste. Thus when

asked about their ability to make aesthetic judgments, most responded that they

don’t make them well. In the Aesthetic Appreciation code, for example, Adam

states, “I don’t make aesthetic judgments that often because it’s hard for me to

really pin down what I find as beauty…but I felt for instance that some of the

architecture we were looking at was rather interesting. So I mean I guess it has

127
helped me a little bit with figuring out, you know, with expanding my capacity to

make those judgments, but like I said I don’t do it very often.” I wonder if their

claims about their capacity to make aesthetic judgments would have been more

strongly stated prior to our lengthy discussion of it. Nevertheless, these

responses are a strong indication that previous conceptions about art and beauty

were perturbed by discussions in the GVP, and that casually dismissing

something as “ugly” rejects a wealth of ideas along with it. Their humble

comments about their abilities to make aesthetic judgments reflect an open-

mindedness to the ideas represented in objects of art that go deeper than pretty

vs. ugly.

Social Responsibility

This category contains four codes related to Social Responsibility. One

of the interview questions asked students to tell what they learned about

themselves in relation to the larger society and world in which they lived. The Self

in Society code contains mostly responses to that question, although not every

response to that question fell into this code. Some may have fallen into Respect

for Others. Nevertheless, as Karen expressed it, “I feel much more connected to

the world as whole. So I feel like more of an actual global citizen than I did

before.” And Les reflected that “it really makes me realize that not everybody has

this opportunity as we have.” Kevin’s response was, “I think understanding how

small we are. And that kind of puts it in more of a perspective for me.”

128
 Despite an understanding of the relative “smallness of self” in relation to

society and world, the Need for Action or responsibility to take action was also

present. Karen found herself “reminded that it’s very easy to just focus on

yourself and your family. And I have my own children and grandchildren and I

can’t stop there. Even if it’s not a huge impact, I can make a small impact on the

rest of the world.” Kevin concurs that even though we are individually small, “our

contributions make a bigger difference.” For one student, social responsibility

was to Do No Harm, which the researchers felt was different than taking action

and thus coded it separately. Adam puts it this way, “I will quote what I learned in

my ethics class to be the agreed upon universal ethical principal and that is

basically, do not do anything that harms others or you know, restricts their ability

to do something that they want to do.” Overall the code Protecting Planet was

quite small, but characterized by statements such as this one: “we also have a

responsibility to make sure that we protect the planet ‘cause everyone uses it.”

Table 5.22

Codes in Social Responsibility Category

% of % of % of
Code Passages Characters Category Theme Total Text
Self in Society 10 1490 57.40 3.96 0.82
Need for Action 3 588 19.69 1.34 0.28
Protecting Planet 2 348 12.36 0.84 0.18
Do No Harm 1 423 10.55 0.69 0.15
4 16 2849 100.00 6.83 1.43

129
 Results Theme

The Results theme examines what worked well and what did not, the

tensions and successes that occurred in this semester-long implementation. The

Instructional Methods category comprises the largest percentage of this theme,

with Students and Curriculum & Assessment representing the second and

third most important categories. Figure 5.5 shows the weights of each category

within this theme.

Results Theme

3.06
9.16

31.16
Instructional Methods
9.54
Students
Curric & Assessment
Technology
9.61
Epistemology
Course Format
Institution
17.62 19.85

Figure 5.5. P/C mean percentage of text in Results theme by category.

In relation to the entire text, this theme is the second largest of the four themes,

representing almost a third (29.19%). However, this is the only theme that

included data from the instructor interviews, which did have an impact on its

weight within the overall study.

130
 Instructional Methods: Tensions

The code Sequence/Time to Design ARG was the strongest tension in the

Instructional Methods category. Others concurred, but as Michael puts it,

“Personally, I would have to say there’s only one thing I would change about the

curriculum for this class, just because I’m also going into game design and I

know the effort it takes to just design a game. It can take years to design games.

When the game is thrown right at the end, and we also have all this other stuff

that we also needed to take care of that really does kind of throw it off.” Although

this was not as strong a code, the Time Necessary for Consensus seems to be

the underlying cause for the need for more time to design. According to Michael,

“Game development should always start with as much time as possible cause

you’ll work through a concept for months. It’s hard to get people to agree on one

thing in just a couple months and then get a final product done just from that.

Because everyone does have different views and if you expand it out to an entire

semester, it allows for more development.” Kevin, comparing the GVP to his

experiences in his Project Development class in the ISGT program, observes “in

the other class we had the entire semester, which is still not very long, but you

can take two weeks to argue about everything, get it out of everyone’s system,

and when that’s done you have the rest of the semester to work on, toward your

goal.”

The tension between Guided vs. Directed Instruction was the second

strongest code in the Tensions part of this category. Nick felt that he “require[d]

131
more direction than most people do,” adding much later that “there were times

when the sort of free form flow of the class sometimes didn’t seem as organized

or pointed or driven.” The ill-structured nature of a problem-based environment

often causes this sort of cognitive conflict in students who are accustomed to

directed instruction. Both Nick and Kevin thought this intentional aspect of the

design was a lack organization on the part of the instructors. Kevin postulates, “I

figured since it was the first class, it was a little--I don’t want to say disorganized--

it was just learning as we went on how we would do things.” That emergent

quality can be perturbing to students. Indeed, Kevin also notes, “Well, I think, it

was more structured in the very beginning. We kind of knew what we needed to

have done.” Then he adds later that he “didn’t like it as much towards the end,

because it was less discussion based.” While he perceived the later part of the

course to be less “discussion based,” the tension here is that while highly

discussion based, it was student led. This contrasted with the early weeks of the

semester when discussions were led by the instructors. The other instructor for

the class, Trina, admitted that her “straight speech classes are much more

structured and lock stepped,” differentiating how she teaches speech under

normal circumstances from how she taught speech in this learning community.

She compares the two formats in this comment that begins with how she

accustomed to teaching, “There’s more of a tangible agenda that they have to

do. This is more open-ended and kind of figuring, getting our bearing as we go.”

It presented some cognitive conflict to her, as well as the students.

132
 The Encouraging vs. Forcing code is a tension largely between or within

the instructors rather than students. Both instructors perceived their primary role

to be that of facilitating and encouraging rather than dictating what students

should do. Trina, the other instructor, believes, “first and foremost” that “positive

reinforcement is better than punishment, but it’s gotten down almost now that at

the end of the semester, it’s going to be punishment.” Although Trina does not

prefer to punish or force students, at the time she was interviewed near the end

of the semester, it was crunch time. She asserted that “this has to be done. We

have a project; we have a deadline. It’s much like anything I think that happens in

business and industry. The store has to open on Monday, that’s what we’ve

advertised. So, we have to get this done.” We differed on that approach.

Although the author feels that her role as an instructor is not to “harp on students

or nag them,” she found that “some of them really want that.” When she asked

them about using Twitter to remind them about work they needed to complete

outside of class, they responded favorably to the idea. Unfortunately, it was too

late in the semester to determine if such an approach would have had an impact.

Much of the text in the Student Expectations code was discussed earlier

with respect to Preconceptions About the Course (in Reaction theme, Prior

Experience category). In some instances, their expectations deal with the

Sequence of the course, as illustrated in Les’s statement, “I did like that we were

going to develop this, but I was kind of surprised that we were actually going to

develop this a little late in the semester. I was kind of hoping that we were going

133
to do this a little earlier.” This statement is a tension between what Les expected

to happen and what actually happened rather than a tension between how the

course was sequenced and the time it takes to design a game. For Nick, the

expectation was that “I kind of imagined taking a class that I was going to be

participating in an ARG while I was kind of like uh, while I was actually designing

an ARG, so that we were, you know, like learning something about making an

ARG while we were sort of playing one…and I thought the class was going to be

structured that way.”

Table 5.23

Codes in Instructional Methods Category

% of % of % of
Code Pass Char Category Theme Total Text
Sequence/Time to Design ARG 10 1465 8.95 2.79 0.82
Guided vs. Directed Instruction 10 1243 8.32 2.59 0.76
Encouraging vs. Forcing 9 1409 8.31 2.59 0.76
Student Expectations 5 1209 5.82 1.81 0.53
Time Necessary for Consensus 5 950 5.09 1.59 0.46
Totals for Tensions Codes 39 6276 36.49 11.37 3.33
Attainment of Core Perspectives 18 3518 18.60 5.80 1.69
Transfer/Relevance to Real World 7 1899 8.73 2.72 0.79
Instructors/Interaction w/ them 10 1247 8.33 2.59 0.76
Open Learning Environment 9 1024 7.22 2.25 0.66
Personal Responsibility for Learning 4 1074 4.96 1.55 0.45
Encouraging Learning/Risk-taking 4 885 4.42 1.38 0.40
GVP vs. Traditional Instruction 5 556 3.98 1.24 0.36
Building/Creating Something New 4 471 3.25 1.01 0.30
Strengthening Prior Knowledge 3 593 3.12 0.97 0.28
Empowerment from Design 1 151 0.91 0.28 0.08
Totals for Successes Codes 65 11418 63.52 19.79 5.77
Category Totals 104 17694 100.00 31.16 9.10

134
 Instructional Methods: Successes

Overall, there were more successes with respect to Instructional

Methods, than there were tensions. The text in codes related to Successes

represent 63.5% of this category. Within Successes, the strongest code is

Attainment of Core Perspectives. However, it is important to couch this result

within two caveats. The first is that the majority of the text in this category comes

from the author’s own description of the game that students designed given in

the interview conducted by a non-participant of the GVP. In the interview, I was

asked to articulate which of the core perspectives students seemed to engage

with or demonstrate understanding of in the game that they had developed.

Although this code is highly skewed by my description of the game, all three data

analysts agreed that this text indicates Attainment of Core Perspectives. Indeed,

while student responses were not re-coded here, virtually all of the text in the

Transfer theme, which is exclusively student interview data, could also be coded

as Attainment of Core Perspectives. The second caveat is that this code perhaps

should be placed in the Curriculum & Assessment category since it deals with

assessment. However, because the majority of the text here is a description of

the game, the researchers felt it was a direct Success of the course design or

Instructional Methods. As expressed in the interview, “I think that the

Millennium Development Goals gave them a target or something to reach for.

Something to make them think BIG.”

135
 Instead of reiterating the entire description of the game, Figure 5.6 depicts

the overall structure of the game and the corresponding core perspectives

demonstrated in its development. The villain in the game was a large, multi-

national corporation, Dreigund, Inc., modeled after the Four Horsemen of the

Apocalypse. Each “horseman” provided a symbolic frame for the four major

divisions or business enterprises of the corporation.

Figure 5.6. Structure of the game and corresponding core perspectives.

Each of the divisions represents a level of the game as well. The ultimate win

scenario at the end of the game, will require players to realize that they,

themselves, contribute to the evils enacted by the company if they do not take

136
some sort of action to prevent them. What students wanted the player to

understand at the end of this game is that abdicating responsibility and simply

blaming greedy corporate executives for the problems in the world is ultimately

how those problems continue to happen.

Transfer/Relevance to Real World was the second strongest success of

the Instructional Methods. Once again, as Les observes, “The way that [the

course project] prepares you to go into real life settings. It’s basically the same

thing. Same concepts.” The Instructors/Interaction with Them was also a

success. When asked about a time she felt successful in the course, Karen

recounts, “The first thing that comes to mind was when Mary Jo was first talking

about ARGs…and she was linking, so I was actually understanding the concept.

So I felt like, OK, maybe I can do this, maybe I do have something to contribute.”

Directly related to that, as demonstrated earlier, was the Open Learning

Environment that the instructors created. As Trina describes it, “We set up the

environment, we set up the banquet table and the students come to the banquet

table and either starve, as some of them have, one of them especially, or take

little nibbles, or gorge themselves.” Michael confirms the success of this

approach, “So this kind of class the way it’s setup, the openness of the way it is,

really is a good way to learn.”

Despite the tension among students between Guided and Directed

instruction, the emphasis on Encouraging [rather than] Forcing learning, did yield

some acknowledgement among students of their Personal Responsibility for

137
Learning. This code was not as strong as we, the instructors, might have liked,

but it was still present. As Michael reflects, “in college, you’re on your own. No

one is there to tell you, ‘Hey, get up! You’ve got an assignment [to get] done. Or,

‘Hey. You’ve got homework due tomorrow.’ Or, ‘Hey. You’ve got this project due

at the end of the month. You might as well get started on it now.’ But, that’s

scholarly discipline that I started to learn more so from this class than any of the

others.” The italics in the last line are the author’s emphasis, but serve to

highlight the role that guided instruction played in encouraging a Personal

Responsibility for Learning. The GVP vs. Traditional Instruction code captures

passages cited previously in which students compare traditional learning

activities, “that stagnant textbook, read Chapter 12” to what they did in the GVP.

Students: Tensions

Despite the successes of the Instructional Methods, and even students

appreciation for Teamwork as documented in the Learning theme, the course

did cause some tensions among students, intended and unintended. The Lack of

Leadership among students was the strongest tension. When asked to tell about

a time that he struggled in the course, Kevin expressed that the group was

thinking in “too many [different] directions.” The tension for him was that “there

was no leader. I think that came as a huge problem because we got so many

different ideas that there was no one to say ‘we’re going to go with this one. And

that’s final.’” Students also struggled with completing Individual Tasks for Group

Project when they were away from the group between class meetings. Karen

138
captures this tension well when she comments, “So we’re in class and we define

assignments and tasks and then sometimes I feel a little lost trying to complete or

make things work together.” Nick also expressed this tension, “I guess, one of

things that was frustrating to me was that it [developing the ARG] was a

combination of individual work and group work, but they never uh, at least for me,

they never came together.” This tension was due in large part to Communication

Challenges, the third strongest tensions code in this category. As Kevin

speculates, “I think communication. It was really the break down that ended

hurting us the most.” However, it wasn’t a break of communication during in-class

sessions. As will be discussed later with respect to the Course Format category,

communication broke down outside of class between class meetings.

Non-participation/Accountability to Group was also a tension between

students. I’d venture to say this is always an issue with collaborative learning.

Overall, only two students, Kelly and Nick had bad experiences with this in the

early part of the semester with their partners on the culture project. Nick “ended

up doing the project primarily on [his] own.” For Kelly, the result was a Conflict

Between Students rather than Non-participation, “we didn’t get a grade that I was

happy with…and I was not happy with the presentation.” Trina felt that much of

the tension among students was due to the Student Self-Regulation Challenge.

She argued, “I think it’s really about a lack of discipline. That it’s much more fun

to email or go on Facebook when you’re kind of having fun with it. I’d rather be

on e-bay than answering my students’ email. And, they’d rather be on YouTube

139
than answering discussion questions.” While the author, too, observed that

students had difficulty, “managing themselves and the tasks they had to do,”

she’s not as convinced that they preferred to be on Facebook or YouTube. Near

the end of the semester when she noted that students were not completing their

individual game development task outside of class because they had gotten

stuck on some detail that they weren’t sure about, she asked them why they

didn’t call someone, or email, or post a question in the wiki or the discussion

board. The response was, “Are we allowed to do that?” It simply had not

occurred to them that, once they had been assigned an individual group task, it

would be appropriate to seek help or input from the others. The interviews

indicate that they thought they were entirely on their own. The process

proceeded more smoothly after that revelation, but discovery of the

misperception came rather late in semester.

Table 5.24

Codes in Students Category

% of % of % of
Code Pass Char Category Theme Total Text
Lack of Leadership/Too Many Ideas 10 1789 15.49 3.07 0.90
Individual Tasks for Group Project 8 1834 14.12 2.82 0.82
Communication Challenges 8 1150 11.19 2.21 0.65
Non-participation/Accountability 5 735 7.06 1.40 0.41
Conflict Between Students 4 761 6.39 1.27 0.37
Student Self-Regulation Challenge 5 522 6.15 1.21 0.36
Difficulty Organizing Group/Work 2 546 3.91 0.78 0.23
Giving Up Control/Taking Risks 1 337 2.23 0.45 0.13
Totals for Tensions Codes 43 7674 66.54 13.21 3.87
Learning From Peers 9 1547 13.67 2.71 0.79
Synergy/Close Relationships 5 906 7.79 1.55 0.45
Personal Responsibility to Group 4 766 6.41 1.27 0.37
Peer Teaching/Sharing Life Experiences 2 392 3.24 0.65 0.19
Roles/Others' Strengths as Assets 1 366 2.35 0.47 0.14
Totals for Successes Codes 21 3977 33.47 6.65 1.94
Category Totals 64 11651 100.00 19.85 5.79

140
 Students: Successes

Once again, despite the tensions among students, successes were

present as well. In this category, however, the tensions outweighed the

successes at a ratio of 66.5 to 33.5 percent. The greatest success in the

Students category was Learning from Peers, which represents almost 14% of

this category. As discussed earlier, students found that they “don’t learn so much

from what just the teacher is teaching you. You learn from your fellow

classmates.” They also found that they developed Synergy/Close Personal

Relationships with each other. This was in keeping with the friendships and close

working relationships that were coded in the Reaction theme. Despite occasional

lapses in participation or accountability to the group, students did feel Personal

Responsibility to Group. This code represents 6.41% of the category, compared

to 7.06% in the Non-participation code. In other words, this tension and this

success are fairly equal. The remaining codes in the Successes section of this

category are Peer Teaching/Sharing Life Experiences, which is related to

Learning from Peers, and the way that students were able to assign Roles that

made use of each Others’ Strengths as Assets in the design process.

Curriculum & Assessment: Tensions

As happened with the Students category, there were considerably more

tensions in the Curriculum & Assessment category than successes. In this

category, the strongest tension was Contextualizing within the ARG, that is, a

141
course design in which all of the assignments or assessments are fully

contextualized within the process of designing an ARG. As discussed in the third

chapter, which described the course design, this was the original intent. While

this code contains the author’s statements regarding those original intentions, it

also includes student comments about this tension. Michael recommends that,

“for the curriculum of this class, if there is a final project to design the game, the

class should be wrapped around that instead of putting it at as the last thing we

need to worry about. “ Nick concurs, saying that “it seemed like a lot of the

projects weren’t tied together,” observing that he “wasn’t really sure how the ARG

really related to the first part of the class.” He later added that he “saw the

connections” between the first part of the class and the ARG project later, “but

the connections felt superficial.”

Although students did see how the ARG project related to the disciplines

included in the course (speech, writing, literature, and humanities), Trina did not

make that connection. According to her, “it’s going to take some convincing for

me this next semester to say, spending all these weeks designing a game to

what end? What’s the point here? What are they bringing out of this?” For Trina,

this tension was in large part due to the difficulty in Meeting Course Objectives

that often occurs in learning communities. Both she and the author were

concerned with how well students attained course-level objectives, particularly for

writing. As Trina put it, “I would be hard-pressed to take my [students’] English”

papers “to an invisible or nonexistent committee that grilled me about, ‘well, did

142
you meet this objective?’…our writing program asks you to do this, this, and this.

And there are some real holes.”

As co-instructor, I agree that there were holes in the instruction, but

wonder if the writing assignments had been better contextualized within the

game scenario, we might have better fostered them. As it turned out, students

were individually trying to complete their course research papers at the same

time that they were collaboratively working on their respective parts of the game

design documents. The result was that students did not execute either writing

activity was particularly well. Arguably, had they been combined into a research

supported game design document, the writing may have been better. At a

minimum, students’ efforts could have been more focused.

Another significant tension was Communicating Expectations. As stated in

the author’s interview, “I think that problem-based methods require much clearer

communication of expectations, that is, clear articulation of how the instructor will

tell if students ‘got it’ or not.” While most students found the scoring rubrics for

their presentations and writing assignments to be adequate, one student

struggled a bit with what he perceived to be the subjective nature of evaluating

speech and writing. According to Nick, “I haven’t always felt that my assignments

or my grade were as well defined as I would like them. In some of the speeches

that I had given, I just get a grade. It’s a subjective assessment. So like, I don’t

know what the difference between a 94 and a 95 is.” This statement is perplexing

because all students were given substantial feedback including extensive

143
comments and a break down of the grade provided on the scoring rubric.

Nevertheless, those rubrics and the expectations that they convey might need to

be explained more fully prior to the assignments in future iterations of the course.

The Covering Material code is indicative of another tension with problem-

based learning methods. Although Trina professes “all of the courses I teach are

problem-based,” her conception of what this means was quite different than the

author’s. Her approach to teaching relies heavily on covering material, as

opposed to presenting an ill-structured problem and allowing students to discover

material as they work toward solutions to that problem. As she stated in her

interview, “I’m not comfortable with what little we covered in speech. I’m not

comfortable with the lack of diversity of literature that my other classes get.”

Further, she questioned, “How is that [the game project] reinforcing Speech? It

does bring group dynamics into the equation, which is part of Speech. And we

heavily drilled them in Humanities.”

What her statements suggest is that while students were engaged in

practicing the group dynamics that they would have studied in a textbook and

perhaps been “heavily drilled” on in her other speech courses, Trina did not see

the value of practice and discovery as opposed to covering that material

explicitly. It was because of this epistemological frame, that the course took on a

“Learn Then Apply” sequence during the first part of the semester, so that the

instructors might cover some material first. It was believed that students would

then apply the covered material to development of the game. What seemed to

144
happen as a result, however, was that they perceived their activities with

developing the game (the writing and presentations associated with game

development) to be added work that was less important, “superficial” even, or “an

afterthought.”

Table 5.25

Codes in Curriculum & Assessment Category

% of % of % of
Code Pass Char Category Theme Total Text
Contextualizing C&A w/I ARG 13 2132 21.71 3.82 1.12
Meeting Course-level Objectives 7 1424 13.10 2.30 0.67
Communicating Expectations 5 1629 12.48 2.19 0.63
Covering Material/Content 6 740 8.78 1.55 0.45
Assessing Core Perspectives 3 541 5.26 0.93 0.27
"Learn then Apply" Approach 2 461 4.02 0.71 0.20
Totals for Tensions Codes 36 6927 65.35 11.5 3.34
Integrating Disciplines 8 1375 13.68 2.41 0.70
Enjoyed the Course 10 900 12.93 2.29 0.68
Content Establishes Foundation 3 401 4.55 0.80 0.24
Increased Substance 3 194 3.49 0.62 0.18
Totals for Successes Codes 24 2870 34.65 6.12 1.80
Category Totals 60 9797 100.00 17.62 5.15

Curriculum & Assessment: Successes

Nevertheless, the course did achieve some successes in terms of

Curriculum & Assessment. As noted earlier, it appeared to foster attainment of

the core perspectives. That was due in part to the ARG scenario, but it is also a

by-product of learning communities, which are particularly well-suited for

Integrating Disciplines. As Trina exclaims, “that’s why I love and believe in these

learning communities is that they get something better than… They see this

connection, they see this whole… holistic rather than lock-stepped into their little

145
discreet courses.” The students found that to be “the best part of it” as well,

“because you have to connect a lot of different things.” They attribute Increased

Substance to this feature of learning communities, finding that “I’ve gotten more

of the meat and more substance” and “more of a broader base” than in courses

focused on a single content area. They also saw how the Content Established a

Foundation for developing the game. The goal of the Learn Then Apply approach

in Trina’s own words was “laying a good slab foundation for archetype, myths,

stories, story narratives, story boarding.” Although she laments that she didn’t

see how developing the game applied those concepts, the students made the

connections to it. Michael reports that he “can understand the way things were

setup this way.” Les even admits that he enjoyed the early semester activities. “I

can’t say that I didn’t actually enjoy the stuff beforehand. That it would actually

kind of set the foundation, the bricks and all that sort of stuff that lead to the

game, so you’re able to develop it better.” Their regret was that there wasn’t as

much time to work on the game as a result.

Technology: Tensions

The Technology category represents less than 10% of this theme, but is

a category with more successes than tensions. Only one student expressed the

perception of Technology as an Impediment to Learning. Kevin felt that they

“were being introduced to so much, so many different types of communication

over the internet that it was hard for us to figure out which one everyone else was

using.” As explained earlier, this was less an issue with the technology than with

146
communication. Students were given a variety of tools to enable communication,

but they never really self-organized around a single tool or set of tools that

worked well for their unique dynamic. One technology found to be an impediment

to their learning was that much of their individual writing was assigned as blog

posts. Although it was carefully explained that these writing activities were “mini-

essays” and would be evaluated on their effectiveness as compositions, students

did not appear to take them as seriously as pieces of writing given the informality

of the weblog context.

Table 5.26

Codes in Technology Category

% of % of % of
Code Pass Char Category Theme Total Text
Tech as Impediment to Learning 7 1060 20.69 1.98 0.58
Limitations of Learn Lab 3 382 8.23 0.79 0.23
Totals for Tensions Codes 10 1442 28.92 2.77 0.81
Tech as Organizer/Motivator/Comm 7 1403 23.73 2.28 0.66
Learn Lab Enhancing Learning 7 1386 23.58 2.27 0.66
Tech as Tool to Improve Learning 3 738 11.38 1.10 0.32
Co-Learning w/ Students 2 428 7.02 0.68 0.20
Increased Confidence w/ Tech 2 240 5.35 0.51 0.15
Totals for Successes Codes 21 4195 71.08 6.84 1.99
Category Totals 31 5637 100.00 9.61 2.80

Although students reported thoroughly enjoying meeting in the Learn Lab,

Kevin pointed out one of its Limitations, namely that there’s only one computer

station. Kevin observes is that “you have much more focus…on one person…on

one task. Than when you have multiple peers (on their own work stations) and

people are kind off in their own world. So it’s a good.” However, he found that

147
“people had to have laptops to be working with technology. Only one person

could be working with” the technology in the room. He was “used to collaboration

where everyone has a computer and everyone has access to the same

technology so you can work faster. This was a little bit slower.”

Technology: Successes

The successes with technology outweighed the tensions. Student

comments about Tech as Organizer/Motivator/Communicator have largely been

previously cited, particularly those passages in which they found the technology

they learned to use in the course to be useful to their everyday lives. Issues of

access aside, they found the Learn Lab to Enhance Learning rather than detract

from it. As an instructor, Trina was captivated by the affordances of the Learn

Lab. She finds that “the image is the message and you have access right away if

there’s a question about an archetype or there’s a wonderful YouTube video on

that…or, let’s go to the dictionary.com. The moment isn’t lost, we can go right to

it. And, it deepens and enriches the discourse. So, that has been really

wonderful.” Adam’s comment on Technology as Tool to Improve Learning is quite

astute, “I think the more you use technology the more you can do because at

least if it’s good technology and it’s working correctly it can help make the

learning easier.”

148
 Epistemology

This category largely captures the epistemological differences Between

Instructor & Design or Between Instructors. Not only is this category less

significant than others based on coding, its strongest weights are the instructors’

perspectives rather than the students’ and those tensions are largely explained in

the Curriculum & Assessment tensions. As the principal designer of the

course, it was important to the author that course activities be mutually agreeable

to the respective epistemological viewpoints of both instructors and that Trina

“buy into it.” As stated in my interview, “the very best parts of the course were

those that we were able to build together: the Superhero speech assignment, for

example,” which adapted an assignment that Trina uses in her speech courses to

the unique context of the GVP. However, at the end of the course, Trina was still

“not sure that the weeks and the weeks and the weeks that we have spent

developing a game has efficacy in terms of the long-term strategies.”

Table 5.27

Codes in Epistemology Category

% of % of % of
Code Pass Char Category Theme Total Text
Between Instructor & Design 9 1516 28.23 2.68 0.78
Between Instructors 4 861 14.18 1.36 0.39
Between Students 4 615 12.03 1.14 0.33
Between Student & Design/Methods 4 514 11.15 1.05 0.31
Within Student/Learning Preferences 2 611 8.66 0.84 0.24
Totals for Tensions Codes
Value of Opinion w/ Respect to Design 4 809 13.73 1.31 0.38
Instructors' Values Align w/ Design 3 804 12.02 1.16 0.34
Totals for Successes Codes 7 1613 25.74 2.47 0.72
Category Totals 30 5730 100.00 9.54 2.78

149
 Course Format: Tensions

Most of the tensions among students were aggravated by the course

format, the blended or Hybrid/Half Online format that included only One Meeting

Per Week. On the one hand, the hybrid format might well have worked if the

class met more frequently. As Les points out, “I think it would have done better if

it…met twice a week instead of the one.” It is not quite clear if it was less

frequent meetings or reduced class time that was the critical issue. Much later in

his interview, Les puts it this way, “I just think the class needs to be offered more

often...instead of being just once a week, I think it needs to be one of those

consistent ones that at least be two a week or three times a week or something.”

It might well have simply been too long a period of time in between. However,

some students simply did not like that it was Hybrid/Half Online. Kevin states that

he thoroughly enjoys using the technologies that afford distance education, such

as the learning management system and the course wiki, but he “would rather

have that as a supplement instead of half the class.” The reason he gave for this

preference was “you had a lot of ‘you need to do this’ and ‘come back with this’,”

which leaves one wondering what he expected might occur in class. He seems to

feel that class time could have been devoted to preparing or completing what

was “homework” online.

As Trina observed, “there’s still brick and mortar in their concept.”

Students seemed to feel that they could “show up three hours a week in the class

and that’s all [they] need to put into it.” They exhibited a degree of Reluctance to

150
Work Online, but only Kevin expressed that he was “not one of those people that

do very well with online classes.” On the other hand, more of them commented

that the course left Inadequate Face Time, particularly for developing the game.

Had they realized that they should have been communicating with each other

outside of class during this phase of the course, their perceptions might have

been different. With respect to getting clarification from her classmates on

individual tasks for group work, Karen acknowledges it’s “not that I think that

there hasn’t been opportunities, there has been opportunity,” but they didn’t

make the best use of that opportunity.

Course Format: Successes

Although they struggled a great deal with Working Together while

Separated, once students discovered that they should be communicating with

each other outside of class, they expressed feeling successful at Working

Together while Separated. They also felt that size of class, “stood out in that it

was such a small group,” largely because it was better than “having 20 or 30

other people trying to get their ideas in.” Above all, they liked that it was a 6-

Credit Class, which was seen as “a very good way to get your credits done.”

151
Table 5.28

Codes in Course Format Category

% of % of % of
Code Pass Char Category Theme Total Text
One Mtg. Per Week 6 1093 20.64 1.86 0.54
Hybrid/Half Online 7 862 19.62 1.81 0.53
Reluctance to Work Online 7 858 19.57 1.80 0.53
Inadequate Face Time 5 336 10.99 1.04 0.31
Class Size 3 277 7.41 0.69 0.20
Totals for Tensions Codes 28 3426 78.23 7.20 2.11
Working Together While Separated 2 948 13.19 1.14 0.33
Class Size 2 160 4.67 0.44 0.13
6-Credit Class 2 90 3.91 0.38 0.11
Totals for Successes Codes 6 1198 21.78 1.95 0.57
Category 34 4624 100.00 9.16 2.69

Institutional Tensions & Successes

Institutional Tensions & Successes contained passages from

instructors only and represented such a small portion of the text that they aren’t

discussed here. However, Table 5.29 reports the number of passages and

characters in this category, as well at the codes and their weights within the

category, theme, and total text.

Table 5.29

Codes in Institutional Category

% of % of % of
Code Passages Characters Category Theme Total Text
Load 2 521 26.52 0.76 0.22
Retention 3 279 22.24 0.69 0.20
Time 3 259 21.54 0.68 0.20
Teaching Partner 4 373 29.69 0.93 0.27
4 12 1432 100.00 3.06 0.90

152
 Summary

The Learning Theme comprises the most overall text coded in this study,

followed by the Results, Transfer, and Reaction themes. The following were the

top five categories with respect to the overall text:

1. Self-Reflection, in Learning Theme (10.02%)

2. Instructional Methods, in Results Theme (9.10%)

3. Knowledge Construction, in Transfer Theme (7.90%)

4. Teamwork, in Learning Theme (7.17%)

5. Students, in Results Theme (5.79%)

6. Game Scenario, in Learning Theme (5.70%)

7. Prior Knowledge, in Learning Theme (5.59%)

8. Respect for Others, in Transfer Theme (5.43%)

9. Curriculum & Assessment, in Results Theme (5.15%)

10. Instructional Design, in Reaction Theme (3.96%)

The next chapter draws conclusions from these results, assesses their

implications with respect to evaluating the design, and presents directions for

future research.

153
 CHAPTER 6

CONCLUSIONS & IMPLICATIONS

Introduction

Having examined the codes and categories within each theme as well as

their relationships to the total text analyzed in this study, this chapter returns to

the research questions to draw conclusions about the effectiveness of the design

of the Global Village Playground (GVP). This includes a discussion of the

implications for modifying the design given the emergent tensions identified in the

previous chapter following these conclusions. Finally, areas in need of further

research exploration will be identified as directions for future research.

Conclusions

Reactions to the GVP

The first research question focused on identifying student and instructor

reactions to the course and the course design. Overall, students reacted

favorably to the GVP. They found the course activities, predominantly student

presentations, team projects and class discussions, to be effective ways to learn.

They also found their experience to be “enjoyable,” particularly the “open” and

“safe” learning environment fostered by the instructors. They seemed to prefer

learning in this way when compared with traditional methods of instruction,

154
findings in keeping with scholarship on student engagement (Chickering &

Gamson, 1987; Chickering & Gamson, 1991; Koljatic & Kuh, 2001; Kuh, Pace, &

Vesper, 1997). They also enjoyed learning with technology. However, they

reacted more favorably to the technology in the classroom than the hybrid design.

They enjoyed the communication tools that allowed them to keep in touch outside

of class, but did not like learning on their own at a distance as much as face to

face contact. This conclusion, too, aligns with prior research on student

engagement, particularly the literature on satisfaction and engagement with

distance learning environments (Conrad, 2002; Hassenplug & Harnish, 1998).

The students also indicated that the early part of the course did not seem

as well connected to later parts of the course as it could be. They wished that

developing the ARG, the central problem scenario for course, came earlier, but

they did respond favorably to the activities in the first several weeks of the course

nevertheless. Indeed, in some ways, they seemed to prefer those weeks because

assignments and activities were more instructor-directed than the activities in the

game development phase of the course. Although the instructors actually guided

class discussions rather than led them, in the later part of the course, student

work was largely student led. Instructors merely listened in and coached, and

student tasks and activities were identified by the students rather than assigned

by the instructors. This caused students some cognitive conflict, a response often

associated with problem-based learning methods (Savery & Duffy, 1995).

155
 Nevertheless, students liked the multi-disciplinary nature of the learning

community and responded favorably regarding their peers as well. The close

personal relationships that developed among students are the very foundation of

learning communities (McHugh Engstrom & Tinto, 2008), and these relationships

flourished despite the reduction in face-to-face contact of the hybrid delivery of

the course.

Impact of the Design on Student Learning

The second research question pertained to the impact of various aspects

of the course design on how students learned. This particular group of students

was very self-reflective. Both the nature of the course, which engaged them with

big questions, and the open-ness of the learning environment seemed to foster

this reflective vein. They reflected most on how well they learned from peers

through discussions in class led by both instructors and students, a promising

finding with respect to constructivist learning environments (Duffy & Cunningham,

1996; Jonassen, 1999). Students gave as many if not more presentations to the

class than the instructors, particularly in the early part of the class. Their personal

interests were given voice in such a multi-disciplinary class, and when they were

able to find common ground with their peers, they seemed particularly successful.

They also seemed to acquire an appreciation for differing perspectives and

located the source of their own developing conceptions, perspectives, and views

as this peer interaction. Although relying on peers to contribute to team tasks

presented frustrations, students learned some strategies for coping with those

156
frustrations, and saw how their own self-discipline was important when others

were relying on them. They were most motivated by the relevance of the learning

tasks, a finding that aligns with prior research on problem-based learning,

particularly the recent research on The Door, problem-based ARG (Warren,

Dondlinger et al., 2008). They were also highly motivated by personal interests

and goals, which is closely connected to relevance in that one relates most easily

to those things relevant to his or her personal interests.

While students seemed to prefer the more structured nature of the first part

of the course, they learned a great deal from the game scenario; the data

suggests that they learned more from that central scenario than through the

activities dedicated exclusively to course content. This finding is in keeping with

theories of situated learning, which posit that learners learn more and transfer

further knowledge acquired through authentic contexts and tasks (Bransford et

al., 2003; Lave, 1988; Lave & Wenger, 1991; Vanderbilt, 1990, 1993). Moreover,

the game scenario compelled them to bring their own prior knowledge and

personal interests to bear on the project while they integrated and applied what

they discovered to creating a rich learning and play experience for others. They

seemed particularly concerned with making the game a meaningful experience

for players, wanting the game to be coherent and seamless although not without

its challenges. Of the activities prior to developing the game, the group culture

project seemed to have the most impact on their learning. The course was

sequenced such that students could apply what they explored through this project

157
to the actual game. Although students did apply much of their prior knowledge

and some of what they learned in the course to the development of the game,

they didn’t use as much of they learned about other cultures in the game itself.

Levels of the game were “located” in the different countries student groups

explored, but they didn’t incorporate aspects of those cultures into the game

narrative. In other words, they didn’t use the art, architecture, or literature as

pieces of the design puzzle.

With respect to technology, they developed an appreciation for how

technology afforded clearer communication in presentations. They seemed to

learn from the online discussion boards while they lasted during the first part f the

class. However, they did not make as much use of the tools for communication

during the game phase. This is likely because they didn’t realize it was “allowed”

or that they should have been doing so as reported in the Results chapter.

Core Perspectives and Conceptual Age Thinking

The third research question explores the extent to which the course

promoted attainment of the overarching program objectives, the core

perspectives, and advanced conceptual age thinking. The Knowledge

Construction category and sequence of codes that comprise it strongly suggest

that the course was successful in fostering both the synthesis or big picture

thinking vital to the conceptual age and the last of the core perspectives,

integrating knowledge from the scholarly disciplines. The other categories, Social

Responsibility, Open-mindedness, as well as Individual or Personal Values

158
suggest that students achieved a measure of the empathic thinking needed in the

conceptual age and several of core perspectives, specifically those related to the

relationship between self and society, the responsibilities of living in a culturally

and ethnically diverse world, and the development of values for ethical behavior.

Students learned to appreciate other perspectives and opinions, work

through conflicts to build consensus, identify each other’s strengths, talents and

goals in order to benefit from these strengths rather than vilify weaknesses. They

learned to value being informed and critically evaluate rather than blindly accept

things at face value. They developed an appreciation for self-expression,

emotion, and creativity as opposed to relying solely on logical rationalizations,

evidence of conceptual age thinking (Pink, 2006). They came to appreciate the

beauty of the ideas that an object might express even if the object itself did not

seem particularly beautiful. In short, they came to appreciate wholeness and

found that achieving balance among mind, body, and spirit to be important—

another conceptual age ideal, namely perceiving the relationships among things

simultaneously and understanding the “big picture” (Pink, 2006). They also

expressed and demonstrated an open-mindedness about new technologies,

particularly the tools they acquired for collaboration, which suggests development

of an appreciation of the importance of communicating at a distance. They also

saw how they might apply and use these technologies in their personal lives, to

become better organized and improve communication, vital 21st century skills

(Kelly, 2005; Safflund Institute, 2007; Schrier, 2006). They demonstrated a

159
marked willingness to re-evaluate old ideas as they acquired new information--a

continuation of knowledge construction and further evidence of synthesis or

conceptual age thinking.

The expressed a new awareness of themselves in relation to the larger

society—even world. They felt “more connected to the world as whole” and “more

of a global citizen,” learning outcomes vital in a global marketplace (Peter D. Hart

Research Associates, 2008). Moreover, the term, “citizen,” denotes both an

empathy for fellow citizens as well as an imperative for action (Pink, 2006). The

idea that one is complicit with those who perpetuate evil in the world if one does

nothing to stop them, which was the overarching message in the game that

students developed, is indicative of some key core perspectives as well as

conceptual age thinking (Pink, 2006; Texas Higher Education Coordinating

Board, 1999). The data suggests that students find this imperative to pertain to

protecting the planet, preserving justice, and following the mantra to “do no

harm.”

Challenges and Tensions

Despite these successes, several challenges or tensions emerged from

the design. The strongest of these tensions was the amount of time necessary to

develop the game. On the one hand, part of this tension was the reduced face

time resulting from the hybrid design. However, much of that was alleviated once

students realized that they could and should be communicating with each other at

a distance. They thought they were to work on their own on their individual tasks

160
for the game without assistance from peers; thus, when they were uncertain

about how to proceed, they stopped working until the next class meeting. It didn’t

occur to them to simply call someone, post a question in the discussion board, or

send an email. On the other hand, face-to-face time is necessary to build

consensus, particularly when a relatively large group is working on a single

project. Getting everyone on the same page, establishing a common vision or

concept is critical to this effort, and that process takes time. Since the game was

not completed, clearly more development time was needed. Whether this was

due to the course sequence or the hybrid nature of the course is less certain.

Problem-based Learning and Instruction

The tension between guided as opposed to directed instruction was also

strong. Students seemed to learn the most from developing the game, a project

that was much less instructor-directed, but preferred learning with more direction.

This finding is not unusual among students accustomed the directed instruction

typified in an education system focused on preparing students for standardized

tests (Kelly, 2005; Ladd, 2008; Wasley, 2008). This preference for more direction

was also aggravated by the hybrid delivery mode as online learning requires

more self-direction. However, the ill-structured nature of the game development

project itself also perturbed them. They enjoyed the creativity, the application of

knowledge, discovery of new ideas, and the relevance to their emerging and

future values. This caused some psychological vertigo or cognitive conflict

(Savery & Duffy, 1995). The project also caused the other instructor some

161
internal conflict. While she prefers the role of encouraging rather than forcing

students to learn, she sought ways to compel students to complete assigned

tasks, in large part due to institutional pressures to retain students and ensure

their successful completion of the course. As co-instructor, I was not insensitive

to these pressures; however, I am reticent to regulate students too closely.

Learning to adapt to challenging situations and to become self-directed are vital

skills—ones that cannot be fostered through continuous intervention by

instructors (Grabinger, 1996; Jonassen, 1999). Finding the appropriate balance

between directing students and allowing them to develop these skills themselves

can be difficult. However, students recognized the need to become more self-

directed and take personal responsibility for their learning from the game design

scenario. This recognition came “more so from this class than any of the others”

according to class participants and is critical skill called for in the AAC&U poll,

which informed the design (Peter D. Hart Research Associates, 2008). Moreover,

the impact of developing the game on the broader learning goals, the core

perspectives and conceptual age thinking was demonstrable (Pink, 2006; Texas

Higher Education Coordinating Board, 1999).

Project Leadership

The strongest tension among students when designing the game was the

lack of leadership, which further illuminates the tension between directed and

guided instruction. At one point in the game development phase, students

discussed their need for a leader, a designated person who would serve as final

162
arbiter to keep things going. I concurred and encouraged them designate one.

However, they ultimately appointed me rather than a peer. None of them wanted

the responsibility, and they all wanted more direction. I continued to guide their

process but did not dictate or direct them. I simply served in the capacity they

described, which was as final arbiter in concept disputes. Their uneasiness with

the lack of direction was compounded by the dwindling time and impending end

of the semester; being told what to do speeds things up. Consequently, this was

a source of tension among students. For the most part, other tensions among

students related to self-regulation, participation, and individual accountability, the

usual challenges that occur in group projects (Johnson & Johnson, 1994).

Despite these challenges, students learned from each other and preferred that

mode to learning from the teacher. With the exception of Adam, who frequently

missed class, rarely delivered on his assigned tasks, and annoyed the others

when he was present, the group was very forgiving, empathized with each other,

and developed strong, learning community bonds.

Situating Learning in Context

Tensions pertaining to curriculum and assessment were also fairly strong

and tightly connected to the sequence of the course. While students enjoyed the

early part of the class, they felt that the connections between the first and second

part were “superficial.” (Lauzon, 1999; Orey & Nelson, 1997). Indeed, they didn’t

end up applying what they had learned from their culture projects to the game

beyond establishing the locations of levels. Trina was disappointed that students

163
didn’t fold the archetypes that students explored in their culture projects to

development of the game. Had the culture project been contextualized within

game development from the start they might well have been applied better.

However, since this project came so much earlier in the course, much of what

students presented to each other were distant memories by the time they were

immersed in game development. In fact, a plausible reason that students were

unable to finish the game is that they were generating new material for an

overarching game concept that they had developed much later. Had the game

concept been underway while students were presenting the information they had

discovered while researching other cultures, they likely would have used more of

that content in the game. Contexualizing the culture project within the game

development project would still satisfy the need to “cover some material,” but

would allow students to immediately identify how that content applies and should

promote greater transfer to the new context, the game.

Format of Course Delivery

The tensions regarding course format, namely the hybrid delivery, have

already been addressed, but deserve more attention. Clearly, the hybrid mode

had an impact on time necessary to build consensus and to develop the game.

However, that might well have been more of an issue with the sequence of the

course and the fact that course activities weren’t connected directly to game

development. Another interpretation might be that student dependence on

directed instruction made them dislike the hybrid format rather than the hybrid

164
detracted from the meeting time necessary the develop the game. Clearly, self-

direction played a role in the tension with the hybrid format as well. However, it

was not an aversion to technology that kept students from engaging with each

other online. Indeed, a greater contributor to the tension was that the course only

met once a week. It’s quite possible that if the class met more frequently, rather

than for more hours (twice a week for an hour and a half, for example) student

perceptions might have been dramatically different. The length of time between

meetings allowed students to put off tasks for a time, in some instances

forgetting about them. More frequent class meetings could make course tasks

more routine and increase student accountability to each other. Indeed, if game

development occurred throughout the semester, students might have been more

involved and engaged with the online communication from the start.

Implications

Begin Game Development Sooner

Although a great deal of the attainment of the core perspectives and

conceptual age thinking that were the goals of the course can be attributed to

both the course content and the game design scenario that followed later, game

development must begin at the onset of the course for the project to be more fully

effective. Much learning occurred in the course, but that learning (of the core

perspectives) came at the expense of some of the course level competencies,

particularly reading and writing. Beginning game development in the first week

165
need not detract from other course activities. Indeed, it could enhance them by

providing a situated context for them. For example, if students had already begun

to establish an overarching game concept when they gave their culture

presentations (which followed the informative speech requirements mandated by

speech curriculum), the discussion that immediately ensued following each

presentation could have been an evaluation of what parts of the cultural

production of that culture applied to the game concept. It would also have better

situated the proposals and research bibliographies that were components of this

project to a more relevant context than the class itself (Brown, Collins, & Duguid,

1989; Lauzon, 1999). As the course was sequenced, these assessments were

perceived to be busywork, hoops to jump through in order to get to development

of the game (Hickey et al., 2006).

Foster More Interdependence

One of the course successes, perhaps a success of the hybrid design,

was the self-direction that it obliged students to develop, an aptitude vital to life-

long learning and an asset in a global knowledge economy (Peter D. Hart

Research Associates, 2008). While the hybrid format did decrease the amount of

face-to-face contact, results are inconclusive regarding whether or not students

would have developed this self-direction if the class meet six hours a week

instead of three. Nevertheless, students should not feel hindered by the delivery

mode of a course. Clearly the class needs to meet more frequently, and

instructors need to more clearly communicate to students that they are expected

166
to be working together while apart. Nevertheless, more frequent meetings would

likely enhance personal accountability to the group since it’s more difficult to

procrastinate on assignments when one has to face those who are expecting it in

person.

Using the technology with which students routinely communicate might

also promote greater interdependence (Bonk & Zhang, 2008). While half of the

students in the GVP routinely logged into the course management system

because they were using it for other courses, the other half did not do so

habitually. Moreover, email reminders were not particularly effective with

students who rely more heavily on their cell phones and instant messengers.

However, web tools such as Twitter or the new SMS function in Google mail

could be leveraged to reach students on their mobile devices and keep them

more instantly in touch. The caution in using such tools, however, is to employ

them in a way that enables students to support one another rather than increase

their reliance on the instructor (Johnson & Johnson, 1994). Engaging students’

personal interests and prior experiences by beginning game development earlier,

so that they’re not forced or directed to communicate with each other, but are

eager to do so, might also encourage more interdependence.

Enable Group Self-organization

Engaging students in the central problem scenario and fostering

interdependence sooner could also allow students groups to better self-organize.

As several students indicated in their interviews, identifying each other’s

167
strengths takes time. Promoting concept development and greater

interdependence earlier could better allow these strengths to emerge, so that

students can assign and shift roles with greater facility (Brush & Saye, 2001;

Johnson & Johnson, 1994). For example, one student might take on a leadership

role for one phase of the process and then pass the torch to another later on, so

that no one student bears the all of the responsibility throughout. In this pilot of

the GVP, student leaders emerged in each of the small group projects early on,

but the class had more difficulty appointing those roles during the game

development project because their concept for the game and their familiarity with

each other’s skills and abilities relative to the entire group were still emerging.

Encourage Play

One student, Nick, indicated that a preconception he held prior to the

course was that the course itself would be structured like an ARG so that

students were “sort of playing one while they were designing one.” Although the

course was not promoted in a way that led to such a preconception, the designer

did consider creating a mini-ARG to orient students to the genre. However, since

a principal tenet underlying this application of games for learning was to avoid

the assumptions inherent to designing for students, the designer/instructor

walked through an existing ARG with students in class and tasked them with

exploring other ARGs and discussing their features in the online discussion

board. Nevertheless, another principle that drove the design, specifically this

application of games to learning, was the finding from the research on The Door

168
ARG that engaging with the problem-based game tasks encouraged a sense of

playfulness—a willingness to explore and take risks (Warren, Dondlinger et al.,

2008). The structure and sequence of the GVP did not allow this quality to

emerge. However, enhancing the online part of the class with game elements

might go a long way to encouraging this willingness to play while also orienting

students to the ARG genre, content for the course, and even to each other as

they begin concept development.

Directions for Future Research

Although this pilot implementation of the GVP met with some success, a

direction for future research is to compare these results with those from an

implementation in which course assignments are contextualized within the game

development process, which takes place throughout the whole semester. The

goals of such a redesign would be to enhance learner attainment of the course

level competencies by situating the assessment of them in a more relevant

context of practice. Although the number of participants in this study limits

conclusions to be drawn through comparisons to future iterations of the design,

evaluation of student perceptions of the experiences in future designs is an

important next step.

Designing an ARG for students to play prior to developing one is another

direction for future research. In fact, this design approach, which combines

elements of The Door with those of the GVP, is currently underway. It should be

noted, however, that playing an ARG first and developing one later is not the

169
approach that the findings from this study suggest. These results indicate that

game development should begin early and that course assignments should be

contextualized fully into the game development process. A “play then develop”

approach might produce results similar to these results from the “learn then

apply” approach deployed in this course. However, a comparison of findings

between the different iterations of these curricular designs using ARG elements,

including an iteration of the GVP in which game development occurs throughout

the semester, will provide further insight into the outcomes associated with

games designed for, designed with, and designed by learners.

Another area for future exploration is the course format. Comparing the

use of the distance learning components to the frequency and duration of class

meetings is an area ripe for further research. Examining them both when game

development begins earlier and course assignments are fully contextualized in

the problem scenario will enable us to make better assertions about the role of

distance communication tools and student self-reliance, assertions that aren’t

obfuscated by the time necessary to build consensus or develop a game

concept.

Finally, a comparison of the GVP to other capstone course designs is

another area for future research. At least one other capstone learning community

has been developed and implemented at the college where this study was set.

This other learning community employed more traditional methods of instruction.

Comparing outcomes between these two capstone experiences may illuminate

170
the efficacy of problem or project-based methods in fostering attainment of the

overarching objectives of the academic transfer program.

171
 REFERENCES

Aguilera, M., & Mendiz, A. (2003). Video games and education: Education in the
face of a "parallel school." ACM Computers in Entertainment, 1(1), 10-10.

Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A review of

literature on its outcomes and implementation issues. Academic Medicine,
68(1), 52-81.

Aldrich, C. (2003). Simulations and the future of learning. San Francisco: Pfeiffer.

Amory, A., Naicker, K., Vincent, J., & Adams, C. (1999). The use of computer
games as an educational tool: Identification of appropriate game types
and game elements. British Journal of Educational Technology, 30(4),
311-321.

Annett, J., Duncan, K. D., Stammers, R. B., & Gray, M. J. (1971). Task analysis.
London: HMSO.

Armour, S. (2005, November 8). Generation Y: They've arrived at work with a

new attitude [Electronic Version]. USA Today. Retrieved February 28,
2009 from http://www.usatoday.com/money/workplace/2005-11-06-gen-
y_x.htm.

Barab, S. A., & Squire, K. (2004). Design-based research: Putting a stake in the
ground. . Journal of the Learning Sciences, 13(1), 1-14.

Barab, S. A., Warren, S. J., & Ingram-Goble, A. (2008). Academic play spaces. In
R. Fertig (Ed.), Handbook of research on effective electronic gaming in
education. Hershey, PA: Idea Group Reference.

Barab, S. A., Zuiker, S., Warren, S. J., Hickey, D., Ingram-Goble, A., & Kwon, E.-
J. (2007). Situationally embodied curriculum: Relating formalisms and
contexts. Science Education, 91(5), 750-782.

Barrett, H. C. (2007). Researching electronic portfolios and learner engagement:

The REFLECT initiative. Journal of Adolescent & Adult Literacy, 50(6),
436.

172
Barrows, H. S. (1986). A taxonomy of problem based learning methods. Medical
Education, 20, 481-486.

Baudrillard, J. (1994). Simulacra and simulation. Ann Arbor: University of

Michigan Press.

Bernstein, R. J. (1976). The restructuring of social and political theory (6th ed.).
Philadelphia: University of Pennsylvania Press.

Bernstein, R. J. (1983). Beyond objectivism and relativism: Science,

hermeneutics, and praxis. Philadelphia: University of Pennsylvania Press.

Bernstein, R. J. (1993). The new constellation: The ethical-political horizons of

modernity/postmodernity. Cambridge, MA: MIT Press.

Blom, K. (2001). VET in the United States of America. Research Digest, 7, 5.

Bogost, I. (2007, September 1). An instructional alternate reality game.

Retrieved March 9, 2008, 2008, from
http://www.watercoolergames.org/archives/000842.shtml

Bonk, C., & Zhang, K. (2008). Empowering online learning. San Francisco:
Jossey-Bass.

Borland, J. (2005, February 28). Blurring the line between games and life. CNET
News.com. Retrieved February 19, 2008, from http://ecoustics-
cnet.com.com/Blurring+the+line+between+games+and+life/2100-1024_3-
5590956.html

Bransford, J., Vye, N., Bateman, H., Brophy, S., & Roselli, B. (2003). Vanderbilt's
AMIGO project: Knowledge of how people learn enters cyberspace.
Retrieved April 20, 2004, from
http://www.vanth.org/mmedia/vanth0103/vanth0103cd/papers/AmigoWFig.
pdf

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture
of learning. Educational Researcher, 18(1), 32-42.

Brush, T., & Saye, J. (2001). The use of embedded scaffolds with hypermedia-
supported student-centered learning. Journal of Educational Multimedia
and Hypermedia, 10(4), 333-356.

173
Burrow, A., & More, G. (2005). Architectural designers and the interactive
audience. Paper presented at the 2nd Australasian Conference on
Interactive Entertainment, Sydney, Australia.

Campbell, J. (1972). The hero with a thousand faces. Princeton, NJ: Princeton
University Press.

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in
undergraduate education [Electronic Version]. AAHE Bulletin. Retrieved
March 29, 2006 from
http://www.csueastbay.edu/wasc/pdfs/End%20Note.pdf.

Chickering, A. W., & Gamson, Z. F. (1991). Development and adaptations of the

seven principles for good practice in undergraduate education. New
Directions for Teaching and Learning (Winter), 75-81.

Cognition and Technology Group at Vanderbilt. (1990). Anchored instruction and

its relationship to situated cognition. Educational Researcher, 19(6), 2-10.

Cognition and Technology Group at Vanderbilt. (1993). Anchored instruction and

situated cognition revisited. Educational Technology, 33(3), 52-70.

Conrad, D. (2002). Engagement, excitement, anxiety, and fear: Learners’

experiences of starting and online course. American Journal of Distance
Education, 16(4), 205-222.

Corbit, M. (2005). Moving into cyberspace: Game worlds for learning. Knowledge
Quest, 34(1), 18-22.

Crawford, C. (2003). The art of interactive design. San Francisco, CA: No Starch
Press.

Dede, C., Ketelhut, D., & Ruess, K. (2006). Designing for motivation and usability
in a museum-based multi-user virtual environment. Retrieved March 11,
2006, from
http://www.gse.harvard.edu/~dedech/muvees/documents/AELppr.pdf

Dede, C., Nelson, B., Ketelhut, D. J., Clarke, J., & Bowman, C. (2004). Design-
based research strategies for studying situated learning in a multi-user
virtual environment. Paper presented at the 6th International Conference
on Learning Sciences, Santa Monica, California.

174
Denis, G., & Jouvelot, P. (2005). Motivation-driven educational game design:
applying best practices to music education. Paper presented at the 2005
ACM SIGCHI International Conference on Advances in Computer
Entertainment Technology, Valencia, Spain.

Denzin, N., & Lincoln, Y. (Eds.). (2003). The discipline and practice of qualitative
research (2nd ed.). Thousand Oaks, CA: Sage Publications.

Denzin, N., & Lincoln, Y. (Eds.). (2008). Collecting and interpreting qualitative
materials (2nd ed.). Los Angeles, CA: Sage Publications.

Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning:

Two case studies of Active Worlds as a medium for distance education.
British Journal of Educational Technology, 36(3), 439-451.

Dickey, M. D. (2006). "Ninja looting" for instructional design: The design

challenges of creating a game-based learning environment. Paper
presented at the ACM SIGGRAPH 2006 Conference, Boston,
Massachusetts.

Dondlinger, M. J. (2007). Educational video game design: A review of the

literature [Electronic Version]. Journal of Applied Educational Technology,
4, 1-11. Retrieved July 19, 2007 from
http://www.eduquery.com/jaet/index.htm.

Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the

design and delivery of instruction. In D. H. Jonassen (Ed.), Handbook of
research for educational communications and technology. NY: Macmillan.

El-Nasr, M. S., & Smith, B. K. (2006). Learning through game modding. ACM
Computers in Entertainment, 4(1), 7.

Fisch, S. M. (2005). Making educational computer games "educational." Paper

presented at the 2005 Conference on Interaction Design and Children,
Boulder, Colorado.

Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An

introduction (8th ed.). Boston: Pearson Longman.

Gee, J. P. (2003). What video games have to teach us about learning and
literacy. New York: Palgrave MacMillan.

175
Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford (Eds.),
Perceiving, acting, and knowing: Toward on ecological psychology (pp.
67-82). Hillsdale, NJ: Erlbaum and Associates.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory:

Strategies for qualitative research. New York: Aldine Publication
Company.

Grabinger, R. S. (1996). Rich environments for active learning. In D. H. Jonassen

(Ed.), Handbook of research for educational communications and
technology (pp. 665-692). New York: Macmillan.

Guba, E. G., & Lincoln, Y. S. (Eds.). (1989). Fourth generation evaluation (2nd
ed.). Newbury Park, CA: Sage Publications.

Halverson, R., Shaffer, D., Squire, K., & Steinkuehler, C. (2006). Theorizing
games in/and education. Paper presented at the 7th International
Conference on Learning Sciences, Bloomington, Indiana.

Hassenplug, C. A., & Harnish, D. (1998). The nature and importance of

interaction in distance education credit classes at technical institutes.
Community College Journal of Research and Practice, 22(6), 591-606.

Hays, R. T. (2005). The effectiveness of instructional games: A literature review

and discussion (Technical Report No. 2005-004). Orlando, FL: Naval Air
Warfare Center Training Systems Division.

Heinich, R., Molenda, M., Russell, J., & Smaldino, S. (1999). Instructional media
and technologies for learning (6th ed.). Upper Saddle River, New Jersey:
Prentice Hall.

Hickey, D., Zuiker, S. J., Taasoobshirazi, G., Shafer, N. J., & Michael, M. A.
(2006). Balancing varied assessment functions to attain systemic validity:
Three is the majic number. Studies in Educational Evaluation, 32, 180-
201.

Hoachlander, G., Sikora, A. C., & Horn, L. (2003). Community college students:
Goals, academic preparations, and outcomes. Education Statistics
Quarterly, 5(2).

Hollis, M. (1994). The philosophy of social science: An introduction. Cambridge,

UK: Cambridge University Press.

176
Jenkins, H., Klopfer, E., Squire, K., & Tan, P. (2003). Entering the education
arcade. Computers in Entertainment, 1(1), 17.

Jennings, M. (2001). Best practices in corporate training and the role of

aesthetics: Interviews with eight experts. Paper presented at the 2001
ACM SIGCPR Conference on Computer Personnel Research, San Diego,
California, United States.

Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone:

Cooperative, competitive, and individualistic learning (4th ed.). Needham
Heights, MA: Allyn and Bacon.

Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M.

Reigeluth (Ed.), Instructional design theories and models: A new paradigm
of instructional theory (Vol. 2, pp. 215-239). Mahwah, NJ: Lawrence
Erlbaum Associates, Publishers.

Jung, C. (1958). Psyche and symbol. Princeton, NJ: Princeton University Press.

Juniewicz, K. (2003). Student portfolios with a purpose. Clearing House: A

Journal of Educational Strategies, Issues and Ideas, 77(Nov-Dec), 73.

Kelly, H. (2005). Games, cookies, and the future of education. Issues in Science
& Technology, 21(4), 33-40.

Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San

Francisco: Berrett-Koehler.

Kirwan, B., & Ainsworth, L. K. (1992). A guide to task analysis. London: Taylor &
Francis.

Klopfer, E., & Yoon, S. (2005). Developing games and simulations for today and
tomorrow’s tech savvy youth. Tech Trends, 49(3), 33-41.

Koljatic, M., & Kuh, G. (2001). A longitudinal assessment of college student

engagement in good practices in undergraduate education. Higher
Education, 42, 351-337.

Kolodner, J. (2002). Facilitating the learning of design practices: Lessons learned

from an inquiry into science education. Journal of Industrial Teacher
Education, 39(3), 9-40.

177
Kuh, G., Pace, C. R., & Vesper, N. (1997). The development of process
indicators to estimate student gains associated with good practices in
undergraduate education. Research in Higher Education, 38(4), 435-454.

Ladd, H. F. (2008). Rethinking the way we hold schools accountable [Electronic

Version]. Education Week. Retrieved January 22, 2008 from
http://www.edweek.org/ew/articles/2008/01/23/20ladd.h27.html.

Lauzon, A. C. (1999). Situating cognition and crossing borders: Resisting the

hegemony of mediated education. British Journal of Educational
Technology, 30(3), 261.

Lave, J. (1988). Cognition in practice: Mind, mathematics, and culture in

everyday life. Cambridge, UK: Cambridge University Press.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral

participation. Cambridge, UK: Cambridge University Press.

Lincoln, C. (2009). Achieving the community-college dream [Electronic Version].

Change. Retrieved January 20, 2009 from
http://www.changemag.org/Archives/Back%20Issues/January-
February%202009/full-achieving-dream.html.

Liu, M. (2003). Enhancing learners' cognitive skills through multimedia design.

Interactive Learning Environments, 11(1), 23-39.

Lunce, L. (2006). Simulations: Bringing the benefits of situated learning to the

traditional classroom. Journal of Applied Educational Technology, 3(1),
37-45.

Martin, A., & Chatfield, T. (2006). Alternate Reality Games White Paper - IGDA
ARG SIG. Mt. Royal, New Jersey: International Game Developers
Association.

McHugh Engstrom, C., & Tinto, V. (2008). Learning better together: The impact
of learning communities on the persistence of low-income students.
Opportunity Matters: A Journal of Research Informing Educational
Opportunity Practice and Programs, 1(1), 5-21.

McLester, S. (2005). Student gamecraft. Technology & Learning, 26(4), 20.

Merriam, S. B. (1988). Case study research in education: A qualitative approach.

San Francisco: Jossey-Bass.

178
Miles, M., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.).
Thousand Oaks, CA: Sage Publications, Inc.

Nelson, B., Ketelhut, D. J., Clarke, J., Bowman, C., & Dede, C. (2005). Design-
based research strategies for developing a scientific inquiry curriculum in
a multi-user virtual environment. Educational Technology, 45(1), 21-28.

Orey, M. A., & Nelson, W. A. (1997). The impact of situated cognition:

Instructional design paradigms in transition. In C. R. Dill & A. J.
Romiszowski (Eds.), Instructional Design Paradigms. Englewood Cliffs,
NJ: Educational Technology Publications.

Pearce, C. (1997). The interactive book. Indianapolis: Macmillan.

Peter D. Hart Research Associates. (2008). How should colleges assess and
improve student learning: A survey of employers conducted on behalf of
the Association of American Colleges and Universities. Washington, D.C.:
Association of American Colleges and Universities.

Pink, D. H. (2006). A whole new mind: Why right-brainers will rule the future.
New York: Riverhead Books.

Prensky, M. (2006). Don't bother me mom--I'm learning. St. Paul, MN: Paragon
House.

Pullman, G. (2002). Electronic portfolios revisited: The efolios project. Computers

and Composition, 19(2), 151.

Richland College. (2008). Thunderdocuments. Retrieved February 15, 2009,

from http://www.richlandcollege.edu/thunderdoc/

Richland College. (2009a). Campus profile. Retrieved February 15, 2009, from
http://www.richlandcollege.edu/about/profile.php

Richland College. (2009b). Learning communities. Retrieved January 10, 2009,

from http://www.richlandcollege.edu/schedules/spring/LC.php

Robertson, J., & Good, J. (2005). Story creation in virtual game worlds.
Communications of the ACM, 48(1), 61-65.

179
Robertson, J., Good, J., Keeker, K., Pagulayan, R., Sykes, J., & Lazzaro, N.
(2004). Children's narrative development through computer game
authoring: The untapped world of video games. Paper presented at the
2004 Conference on Interaction Design and Children: Building a
Community, Vienna, Austria.

Safflund Institute. (2007). Information technology workforce skills study. Boston,

MA: Boston Area Advanced Technological Education Connections.

Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals.
Cambridge, Mass.: MIT Press.

Salzman, M. C., Dede, C., & Loftin, R. B. (1999). VR's frames of reference: A
visualization technique for mastering abstract multidimensional
information. Paper presented at the SIGCHI Conference on Human
Factors in Computing Systems, Pittsburgh, Pennsylvania, United States.

Salzman, M. C., Loftin, R. B., Dede, C., & McGlynn, D. (1996). ScienceSpace:
Lessons for designing immersive virtual realities. Paper presented at the
Conference on Human Factors in Computing Systems, Vancouver, British
Columbia, Canada.

Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional

model and its constructivist framework. Educational Technology, 35, 31-
38.

Schrier, K. (2006). Using augmented reality games to teach 21st century skills.
Paper presented at the ACM SIGGRAPH 2006 Conference, Boston,
Massachusetts.

Secretary's Commission on Achieving Necessary Skills. (1991). What work

requires of schools: A SCANS report for America. Retrieved November
10, 2007. from http://wdr.doleta.gov/SCANS/.

Segers, M., Gijbels, D., & Thurlings, M. (2008). The relationship between
students' perceptions of portfolio assessment practice and their
approaches to learning. Educational Studies, 34(1), 35.

Southern Association of Colleges and Schools. (2007). Principles of

accreditation: Foundation for quality enhancement. Retrieved January 29,
2008. from http://www.sacscoc.org/principles.asp.

Squire, K. (2006). From content to context: Videogames as designed experience.

Educational Researcher, 35(8), 19-29.

180
Squire, K., Barnett, M., Grant, J. M., & Higginbotham, T. (2004).
Electromagnetism Supercharged!: Learning physics with digital simulation
games. Paper presented at the 6th International Conference on Learning
Sciences, Santa Monica, California.

Squire, K., Giovanetto, L., Devane, B., & Durga, S. (2005). From users to
designers: Building a self-organizing game-based learning environment.
Tech Trends, 49(5), 34-42, 74.

Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Steiner, B., Kaplan, N., & Moulthrop, S. (2006). When play works: Turning game-
playing into learning. Paper presented at the 2006 Conference on
Interaction Design and Children, Tampere, Finland.

Strauss, A. L., & Corbin, J. (1998). Basics of qualitative research: Techniques

and procedures for developing grounded theory (2nd ed.). Thousand
Oaks, CA: Sage.

Strickland, E. (2007, July 10). Play peak oil before you live it. Retrieved March
8, 2008, 2008, from
http://www.salon.com/tech/feature/2007/07/10/alternative_reality_games/

Sturtz, A. J. (2006). The multiple dimensions of student swirl. Journal of Applied

Research in the Community College, 13(2), 151-158.

Swartout, W., & van Lent, M. (2003). Making a game of system design.
Communications of the ACM, 46(7), 32-39.

Texas Higher Education Coordinating Board. (1999). Core curriculum:

Assumptions and defining characteristics. Retrieved January 17, 2008,
from
http://www.thecb.state.tx.us/AAR/UndergraduateEd/fos_assumpdef.cfm

The Safflund Institute. (2007). Information technology workforce skills study.

Boston, MA: Boston Area Advanced Technological Education
Connections.

U.S. Department of Education. (2002). Strategic Plan, 2002-2007. Retrieved

March 10, 2004, from http://www.ed.gov/about/reports/strat/plan2002-
07/index.html

United Nations. (2005). UN Millenium Development Goals. Retrieved November

10, 2007, 2007, from http://www.un.org/millenniumgoals/

181
Van Tartwijk, J., Driessen, E., Van Der Vleuten, C., & Stokking, K. (2007).
Factors influencing the successful introduction of portfolios. Quality in
Higher Education, 13(1), 69.

Vanderbilt Cognition and Technology Group. (1990). Anchored instruction and its
relationship to situated cognition. Educational Researcher, 19(6), 2-10.

Vanderbilt Cognition and Technology Group. (1993). Anchored instruction and

situated cognition revisited. Educational Technology, 33(3), 52-70.

Waraich, A. (2004). Using narrative as a motivating device to teach binary

arithmetic and logic gates. Paper presented at the 9th annual SIGCSE
Conference on Innovation and Technology in Computer Science
Education, Leeds, United Kingdom.

Warren, S. J. (2006, April 7-11, 2006). Using digital learning environment

scaffolds to improve student writing in a PBL-style instructional space.
Paper presented at the American Educational Research Association
Annual Meeting, San Francisco, CA.

Warren, S. J., Barab, S. A., & Dondlinger, M. J. (2008). A MUVE towards PBL
writing: Effects of a digital learning environment designed to improve
elementary student writing. Journal of Research on Technology in
Education, 41(1), 113-140.

Warren, S. J., & Dondlinger, M. J. (2008a). Data-driven decision-making about

curricular redesign: Using PBL and game design to transform instruction in
a large enrollment course. Paper presented at the Annual Meeting of the
American Educational Research Association, New York, NY.

Warren, S. J., & Dondlinger, M. J. (2008b). Issues with designing games for
learning In R. Ferdig (Ed.), The handbook of research on effective
electronic gaming in education. Hershey, PA: IGI Global.

Warren, S. J., & Dondlinger, M. J. (2008c). Objective achievement in a PBL

course: Findings from a study on the first iteration of a problem-based
learning course game. Paper presented at the Association for Educational
Communications and Technology Annual Convention, Orlando, FL.

Warren, S. J., Dondlinger, M. J., & McLeod, J. (2008). Power, play and PBL in
postsecondary learning: Leveraging design models, emerging
technologies, and game elements to transform large group instruction.
Paper presented at the Annual Meeting of the American Educational
Research Association, New York, NY.

182
Warren, S. J., Dondlinger, M. J., & Whitworth, C. (2007). Distributed online
multimedia learning games: The challenge of instructional design in closed
systems. Paper presented at the Association for Educational
Communications and Technologies, Anaheim CA.

Warren, S. J., Stein, R., Dondlinger, M. J., & Barab, S. A. (2009). A look inside a
MUVE design process: Blending instructional design and game principles
to target writing skills. Journal of Educational Computing Research, 40(3).

Wasley, P. (2008). Tests aren't best way to evaluate graduates' skills, business
leaders say in survey [Electronic Version]. The Chronicle of Higher
Education. Retrieved January 23, 2008 from
http://chronicle.com/daily/2008/01/1340n.htm.

Yin, R. K. (1993). Application of case-study research. Newbury Park, CA: Sage.

Yin, R. K. (2003). Case Study Research: Design and Methods (3rd ed. Vol. 5).
Thousand Oaks, CA: Sage.

Zagal, J. P., Nussbaum, M., & Rosas, R. (2000). A model to support the design
of multiplayer games. Presence, 9(5), 448-462.

183