International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 37

Balancing Fun and Learning

in a Serious Game Design
Christopher Franzwa, Department of Electrical and Computer Engineering, Rowan
University, Glassboro, NJ, USA
Ying Tang, Department of Electrical and Computer Engineering, Rowan University,
Glassboro, NJ, USA
Aaron Johnson, Department of Electrical and Computer Engineering, Rowan University,
Glassboro, NJ, USA
Talbot Bielefeldt, Clearwater Program Evaluation, Eugene, OR, USA

ABSTRACT
This article presents the underlying philosophy of Sustain City, an educational serious game system that
engages students, particularly prospective and beginning science and engineering students, in a series of
engineering design challenges. Various strategies implemented in Sustain City for achieving a balance of fun
and learning are discussed, including narrative-learning synthesis, supplementing the player’s actions with
feedback, and the development of a sufficient guidance system. The evaluation of Sustain City deployment
is also presented. The assessment confirms the values of the serious games in promoting students’ interests
and learning in STEM fields.

Keywords: Game Design, Game Mechanics, Narrative Learning, Problem Solving, Serious Games

INTRODUCTION but the wide range of learning styles and vary-

ing degrees of proficiency between students
A number of recent reports (Computing Re- make it difficult for classic teaching methods
search Association, 2005) make it clear that the to be universally effective. Even in modest
United States is losing ground on key indica- class sizes, curriculum requirements and short
tors of innovation and progress because of its class times do not allow instructors to provide
deteriorating performance in teaching math and individualized instruction. Additionally, such a
science. Unfortunately, difficulty with early system often emphasizes rote memorization that
concepts in mathematics and sciences can dis- stifles students who are naturally adept at science
suade students from pursuing further education and serves as a barrier to those who may still
in Science, Technology, Engineering, and Math succeed if granted extra help. Although steps
(STEM) fields (Lantz, 2009). Student frustration have been taken to enhance school systems in
is commonly to blame for poor performance, STEM areas, improvement would require new

DOI: 10.4018/ijgbl.2014100103

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
38 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

equipment and training that may be beyond the healthcare training (Hoffman, 2006; Wattana-
skill set of current teachers. In fact, less invasive soontorn et al., 2012). A comprehensive review
ways to augment current school systems must of the current development as well as social and
be considered to be able to make wide-spread technical barriers to its mass implementation is
changes feasible. documented in (Freitas, 2010; Sandford et al.,
Video games have, since the 90s, become 2006; Ulicsak, 2010).
a mainstream form of entertainment and a Serious game design has room for extensive
multi-billion dollar industry. With improving growth which can be informed by recent changes
hardware and software, development and avail- in curricula as well as trends in academia. Echo-
ability of games have become inexpensive and ing general concerns with the current state of our
accessible (Mayo, 2009). The success of video school systems, many educational groups have
games is largely attributed to their ability to begun advocating curricular changes for STEM
synthesize elements of environment and story subjects. In a report created by the President’s
with simulations that can provide real-time Council of Advisors on Science and Technology
visualized responses. Video games are fun and (PCAST), the blame for poor retention rates is
embody real world situations in which play- split between high performing and low perform-
ers explore, learn and solve problems (Barab ing students. Higher performing students cite
et al., 2009). The consideration of games in “uninspiring” introductory courses as a factor in
education is made evident by recent and grow- choosing different majors while lower perform-
ing development in “serious games,” defined ing students struggle with mathematics due to
by design that takes into account “(i) serious insufficient assistance (President’s Council of
aspects that determine the pedagogical objec- Advisors on Science and Technology, 2012).
tives such as the transmission and/or acquisition Issues such as student interests and instructional
of knowledge, know-how, or information; (ii) feedback should be considered when developing
and fun aspects which focus on the motivation any STEM serious game. While the educational
and the management of end users’ frustration.” value of games has long been recognized, there
(Hocine and Gouaich, 2011). Taking advantage is significant resistance to their adoption in
of the tremendous revolution that interactive formal education. One problem is the amount
technologies have brought to consumers, of instructional time that must be devoted to
“serious games” offer a number of strong training and practice to allow games to have
learning-enhancement capabilities, allowing for a significant effect on student learning. There
the realization of virtual worlds that can adapt is a tendency for serious games to develop an
to and assist students in ways that the typical all-inclusive learning system that largely leaves
classroom environment cannot (Torrente et al., the instructors without the flexibility needed to
2011). In standard textbook-driven lecturing create their own curriculum. As indicated in
and study, visual or hands-on learners are left (Wilson, 2009; Arnab et al., 2012), considerable
to find their own ways of perceiving the ever benefit would be gained from aligning games
more complex concepts as they wade through with standards and curricula. Although serious
a course. Currently, even hands-on approaches games may be powerful tools, they should not be
to learning, such as lab experiments, are limited considered as a form of self-contained teaching.
by budgetary and safety constraints. Virtual en- A truly effective solution relies on an instruc-
vironments allow infinite potential for scientific tor for focus and guidance (Egenfeldt-Nielsen,
exploration, offering scenarios that are simply 2010). The best practices for using games in the
impossible in reality. Vivid examples can be classroom promote a strong interconnection
found in many domains, such as science and between instructors and software such that the
engineering discovery (Barab et al., 2009; Mott instructors remain the driving force behind the
et al., 2006; Sandford et al., 2006), military education (Wilson, 2009).
training (Smith, 2009; Zielke et al., 2009) and

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 39

Motivated by these general remarks, this the science and engineering process (Herbert,
paper presents such an approach that guides 2003). The resulting curriculum for pre- or
serious game design to address immediate needs early-engineering classes starves students of
of students with in-game references and help, context, leaving them lost in complex concepts
without compromising the role of the instruc- without any perceived purpose or connection to
tor in directing exploration. Aiming to meet engineering applications. Some institutions con-
the educational needs of students and supply struct their engineering programs in a guided,
tools for instructors, a series of serious games interconnected fashion; however early courses
under the umbrella theme of a “Sustain City” that build up math and science foundations, are
have been produced (Tang et al., 2012). Not always presented separately. Bordogna (1998)
seeking to replace or interrupt current curricula, expressed the need for curricular integration
each game focuses on particular fundamental as follows:
science and engineering concepts and serves as
a replacement for a traditional laboratory activ- Most curricula require students to learn in
ity. For example, Solaris One aims to enhance unconnected pieces - separate courses whose
understanding of basic thermodynamic laws relationship to each other and to the engineering
intuitively as well as mathematically. Gridlock process are not explained until late in a bac-
offers an in-depth exploration of logic-circuit calaureate education, if ever. … The content
design, providing students with a realistic ap- of the courses may be valuable, but this view
plication of logic systems as well as the tools to of engineering education appears to ignore
construct and analyze them. Powerville, a game the need for connections and for integration -
exploring the root meaning behind Sustain City, which should be at the core of an engineering
provides an exploration of alternative energies education.
and their impacts on a modern city. Textbooks
always provide example-based questions, but In Sustain City, a game experiment in a
students can be lost in wording. Here we present course builds upon concepts gained through
exactly what those textbooks intend, but in a game experiments performed in parallel or
more visual and interactive way. Although the previous courses. Students are thus in a bet-
common assumption is that today’s students ter position to see the interconnection of their
are well integrated with electronics and games, curricular courses and appreciate the integrated
we can’t assume base knowledge of common content value. This idea is exemplified in Fig-
software, thus design emphasis has also been ure 1. With the sustainable city as a broader
placed on scalable difficulty and usability. context, a series of games that individually
The rest of the paper is organized as have a focus on particular fundamental science
follows. Section 2 gives the background and and engineering concepts are deployed as a
overview of Sustain City with an emphasis replacement of traditional laboratory settings
on its pedagogy. The seamless integration of of courses at different levels of Project Lead
learning with game mechanics is elaborated in the Way (PLTW) curriculum. Eventually, stu-
depth in Section 3. The assessment results are dents integrate all game modules in their senior
then presented in Section 4, followed by our capstone project, resulting in a fully-functional
conclusion in Section 5. eco-city. A sustainable city is a city designed
to grow and evolve in all aspects of human life
without compromising the well-being of future
OVERVIEW OF SUSTAIN CITY generations. In action, Sustain City is our answer
to the issues of curricular integration. It is not
Recent reports have indicated that many STEM only a platform that offers expandable, modular
classes rely heavily on textbooks but are weak game design, but also a motivational learning
on examples, such that students are exposed to environment that emphasizes contextualized
encyclopedias of facts without ever engaging in

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
40 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

learning where students take on new roles Thus, it is obvious that much can be learned
(e.g., environmental engineer, traffic engineer, from existing smash hit titles that have reached
or power engineer) and explore what and how millions across the globe. Unfortunately, the
their knowledge and skills come together in differentiation between design techniques is far
engineering a future ecologically healthy city. greater than desired. When designing a game for
Using environmental sustainability as an au- entertainment, there are essentially “no rules.”
thentic and engaging context for teaching core If a game mechanic or part of a design is not
subjects promotes students’ 21st century skills fun for the player, it can be removed or heav-
(Church and Skelton, 2010). ily modified. No regard must be made toward
propriety of content, accessibility, or depth of
understanding. Serious games must consider
INTEGRATION OF GAME entertainment as a secondary factor, attempting
MECHANICS WITH LEARNING to design engaging content as a loose article
that fits around educational content.
The biggest challenge of serious game develop-
There are however a number of design
ment is the integration of learning content with
principles that can be shared equally. As the
core mechanics of games. Too much emphasis
video game entertainment industry continues
on learning tends to “suck the fun out” of games.
to seek a larger audience, more emphasis has
The opposite is also true: too much focus on
been placed on player guidance, ensuring that
game play “sucks out the learning” (Thomas
a game can teach any user how to play, regard-
and Young, 2010).
less of skill sets. In this regard, entertainment
Serious games, in principle, hope to lever-
games exhibit basic educational concepts that
age the interactive, entertaining medium of vid-
can be emphasized in serious game design.
eo games to more effectively educate students.

Figure 1. Curricular integration and alignment

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 41

Additionally, entire overarching gameplay designed to fill this role. A city is an exquisite
concepts can be borrowed and integrated with combination of interacting systems (infrastruc-
educational content, taking advantage of the tures) that can be designed and analyzed using
massive research and work that has been placed multidisciplinary engineering and scientific
into the entertainment industry. principles. With the future sustainable city as
This section presents several strategies a broader context and the city infrastructures as
deployed in Sustain City, including the infu- the themes, Sustain City consists of a series of
sion of narrative into educational material, narrative games (Figure 2 - the screenshots from
rewarding mechanism, and scaffolding for Sustain City for different game theme environ-
game navigation. ments) that provide students an opportunity to
learn what it means to be a scientist, engineer,
Narrative-Learning Synthesis or mathematician who helps design and main-
tain an eco-city. Demonstration, explanation,
As stated in the Introduction, students who and practice in different aspects of Sustain
struggle in STEM courses do so not necessar- City help students experience the interconnec-
ily because of their innate abilities or intellect. tion between their courses as a progression of
Instead, they are struggling with conceptual increasing design complexity.
transformation (Chen, 2007). Students develop Such an approach can be best illustrated
their own beliefs about the physical world from through the design of one of the virtual worlds,
personal experiences or from previous school- Solaris One, to promote a deep understanding
ing, in which an oversimplified definition or of curriculum content. The instructional goal of
approach might have been taken. When they this narrative game is to demonstrate the basic
encounter new knowledge in STEM courses, laws of thermodynamics and their applications
they try to fit it into the existing schema of in the real world. Rather than relying on dry
their understanding. Students often develop representations of thermodynamic topics that
misconceptions when a concept does not fit could result in a boring lab, Solaris One takes a
logically or runs counter to their schema. What sci-fi approach, tasking the student as a power
the students need in such cases is transforma- engineer who must go on a dangerous expedi-
tive learning – the expansion of consciousness tion into space to fix a failing “next-generation”
through contextual understanding, critical re- solar power plant. The power plant orbits the
flection on assumptions, and validated meaning earth attached to a rogue asteroid that has been
by assessing reasons. fitted with fields of solar panels. Due to a recent
On the other hand, the reach of education unexpected solar storm, the parabolic dish that
in a classroom is only as far as the walls per- gathers solar energy and transmits it back to the
mit. While instructors try to strike a balanced earth begins to malfunction. It is up to the student
presentation of challenging concepts, facts, to fix a number of broken components such as
and learning strategies, students always feel blown fuses and broken energy transfer devices.
that there are too many detailed, progressively Repetition, as one of the most basic key
complex theories with few “real” examples to learning techniques, can be very powerful but
relate. It is a constant challenge for instructors tends to result in disinterest from students who
to keep students of different backgrounds, either grasp a subject immediately or get bored
academic strengths, and learning tendencies of the same concept presented in the same for-
engaged in a meaningful exploration of the mat over and over again. Careful game design
relationship between abstract ideas and practical ensures that repetition is hidden behind varia-
applications in the context of the real world. tion such as a changing setting/environment
Thus, it is beneficial to supplement instruction or tweaks to the gameplay structure (Coyne,
with pedagogical tools to bring knowledge and 2003). In Solaris One, repetition is manifested
concepts into contextual reality. Sustain City is as a constant demand for players to understand

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
42 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Figure 2. Screenshots from Sustain City: (a) Overview of Sustain City; (b) Power Ville; (c)
Gridlock; (d) Solaris One

the same concepts of heat transfer in solving playing Games would lose their appeal and not
problems; while variation is introduced through be able hold player attention with their repetitive
degrees of difficulty and increasing levels of gameplay. Solaris One relies substantially on the
challenge. idea of layers/levels for high fidelity. First, the
The entire gameplay is broken into three game and/or mini-games are presented in terms
mini-tasks as shown in Figure 3. The first stage of a flowing story, each of which is integrated
of the game finds the player aboard an old Saturn into the environment with logical reasons for
V rocket that has been retrofitted with an entirely its existence within the game world. This can
new engine system. To successfully launch the be thought of in terms of an adventure such
rocket and exit the earth’s atmosphere, players that each game is completed with the incentive
must correctly route the control system’s energy to discover new, unknown stories and/or mini-
into five different nodes. The nodes represent games. Second, all mini-games in Solaris One
one of the two rocket engines mounted on are designed around an overarching goal of
the rocket. The second part of the game takes restoring power to the space station. Complet-
place when the rocket lands at the space sta- ing a mini-game not only provides the access
tion. To patch the power system, players must to the next level of challenge, but also adds to
manufacture fuses that require metal ore mined the final goal so that the eco-city can once again
from the asteroid. To obtain the ore, they must receive the power from the solar power plant.
first fix the mining drill’s cooling system pipes. When players go through layers of challenge,
Finally, they have to place fuses made out of they have to transcend levels of knowledge to
different materials (with unique heat transfer achieve higher understanding. They are then
coefficients) into fuse slots that will both endure getting closer to the truth, an understanding of
the heat created by the standard wattage and the way things really are.
correctly break with overcurrent.
The narrative design of a game can, on its Supplemental Feedback
own, serve as enough incentive to drive a player
forward. Without a strong narrative, games such It is natural for students to exert extra effort
as Mass Effect, Final Fantasy, and other Role- for a better grade or for superiority based on

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 43

Figure 3. Layers of sophisticated gameplay in Solaris One

in-class competition. Placing students into a reward system in place for those that excel.
game environment should preserve or enhance For each mini-game, monetary rewards are
this aspect of learning. When the outcome of offered depending on performance. Regardless
gameplay is only to pass or fail an objective, of performance, a flat fee is supplied to every
then performing the bare minimum provides player for completing each game. Bonus cash
identical results to over-performing. In these is rewarded to those who complete more dif-
situations, there is no incentive for the player ficult problems, complete problems faster, or
to improve or to seek higher skill levels. To have lower numbers of attempts before success.
ensure such situation does not occur, a game Without some outlet to spend the cash, the
must be designed such that there are various concept of a cash system becomes somewhat
degrees of accomplishment, that directly affect meaningless, devolving into a simple scoring
the game world and, by extension, the player’s method. On the asteroid, the primary form of
experiences. Such concepts have been present movement for the player is a rover (Figure 4)
in video games since their inception. Examples that is initially provided with very little speed
range from the simple “High Score” system to and no special features beyond surface move-
special unlockable components that change the ment. With the monetary bonuses, players can
way the game can be played. On the opposite purchase upgrades or perks to improve their
side of the concept has always been a punishment gaming experience. Upgrades can be perfor-
system such as the classic “Game Over” screen mance based (such as extra speed, thrusters,
or losing points in some way. Special caution or better brakes for the rover), aesthetic (such
should be taken in punishing the player within as changing the color of the rover or adding
the realm of a serious game as punishments parts), or additional features (such as vertical
may merely serve to frustrate or overwhelm boosts or hovering).
the student with negative feedback. Rather than Power Ville is a game in Sustain City de-
punish for poor performance, it is usually a good signed to educate students about four different
idea to withhold reward or supply additional methods of energy generation (coal, wind, solar
help instead. Experiential consequentiality and and nuclear) and their impact on the environ-
various rewarding mechanisms are threaded ment. In Power Ville the player takes the role
throughout Sustain City. of a consulting engineer to investigate what
In Solaris One there are no instances form of electricity generation the city should
of punishing the player; however, there is a pursue within the constraints of the city budget

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
44 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Figure 4. The rover in Solaris One

and power requirements. Over the course of the The player’s final decision is presented
game, the player talks with different experts on back to the mayor in the form of a report con-
the available forms of energy to the city and taining not only the selected solution, but also
learns the various pros and cons of each. With the results from other in-game activities. This
all the information collected, the player then solution is also evaluated by a pre-programmed
uses a simulation tool as shown in Figure 5 to scoring method to assess its quality. Figure 7
determine the environmental impact of each presents two possible outcomes. When a player
form of energy and its feasibility as a power ranks coal as the power source with the least
source. Soon after the simulation, the player is environmental impact, his or her preliminary un-
provided a design tool as seen in Figure 6 for derstanding was criticized as a one-star decision.
a hybrid energy system solution that provides On the other hand, the correct understanding
sufficient power for the city within the given of the environmental impact of the four power
budget. The tool scopes the city into different sources was rated as a four-star decision.
areas that can be powered by different sources
of the player’s choice. Whenever the player Player Guidance
makes such a decision, the information tally
on the left is dynamically updated with the It is always a challenge in any game to teach a
amount of power that can be generated versus player how to play and to guide them through the
the demand and the amount of budget required game world. When dealing with an education-
versus available. The updates clearly show how based game, the challenge becomes greater. It
the decision helps or hinders the optimization must guide students with various backgrounds
goal. The ability to use a portion of each type (e.g., highly self-motivated students vs. the
of power generation instead of a single type ones lack motivation and prior knowledge)
for the final solution has many benefits. First, from the start to the end of the game without
there is less likelihood of a student guessing a direct instructor interference. Sustain City takes
correct answer to a problem with more options. the approach of incremental learning where
Further, choosing partial power production is a learning concepts are built slowly over layers
more realistic situation as many cities buy power of game play such that students can ease into
from neighboring areas as an alternative or all aspects of a topic, avoiding frustrations that
supplement to building their own power supply. might dissuade them from learning.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 45

Figure 5. The simulation tool in Power Ville

Figure 6. The power generation design in Power Ville

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
46 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Figure 7. Examples of players’ solutions in Power Ville

Although Solaris One can, as a whole, The first step in guiding a player through
be viewed as a layered learning experience, any difficulty in game navigation is to identify
it is better to look at one of the mini-games if, when, and what problem the player has. Not
to better explain the concept. The pipe-game all students are alike. Some of them are highly
within Solaris One as shown in Figure 8 is one self-motivated, and the freedom to discover a
of two main events on the asteroid portion of realm of knowledge on their own is enough to
the game. The pipe game asks the student to foster growth. Others prefer coaching and a
construct pipe structures that obey given sets of more structured approach. For students who lack
rules such as length or heat transfer rates. Each motivation and prior knowledge in particular,
chamber is unique and creates situations such practice alone in a game with poor strategies,
as an extreme cold chamber where the player unproductive ideas, and very little guidance will
must move the liquid in the pipe with the low- get them nowhere in problem solving. Instead,
est heat loss possible, or an extremely hot one they will end up frustrated and give up on the
where players wish to guard the liquid from task. In Sustain City, a series of progressive
the temperature and aim for maximum insula- question prompts are designed to pinpoint where
tion. To introduce the interface and concepts, the student breakdowns occur in the context of
there are a series of challenges that increase problem-solving in-game. Those questions are
in difficulty, never introducing more than one closely tied to the goal, knowledge and facts of
new unique game mechanic at a time. At its problem solving stages and administered within
simplest, the game merely asks the player to the context of the game. If students answer those
play around with the interface, watching the questions correctly, no references are forced
result. Regardless of players’ actions, the next onto them while there are still options for help
level will progress when they choose to move should they choose. If students fail, additional
on. Next, the game simply asks the player to resources, such as tutorials and live videos of
complete a pipe path, ignoring any parameters experts solving similar problems, are made
for heat loss or pipe material. The levels continue clear to them and students have to go through
in this way until equations are introduced into the resources to progress to the next level. One
the player’s choices. By this stage, the natural vivid example of the approach can be seen in
progression of the levels should allow players Gridlock, a content-specific game in digital
to solve the problem on their own. electronics that invites students to investigate

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 47

Figure 8. A section of the pipe game within Solaris One

solutions to automatic traffic light control for improvement. Such feedback is automatically
a 4-way intersection. An automatic traffic light updated whenever players submit their answers
is a typical engineering invention that made to question prompts. The KWS method (What
the lives of common people safer and more I Know-What I Want to Know- What I have
convenient. For the development of the future Solved) is a structured metacognitive tool in
eco-city, its design inevitably appears in the our games to help students be aware of their
agenda of the city master plan and becomes the own learning process. The more they know
essential task of this game module. For students themselves, the more they can control such
to design a full-functioning traffic light control, matters as goals, dispositions, and attention,
they have to realize from the design specifica- and the better they become successful learners.
tion that it is a typical sequential circuit, from In our initial design, KWS was a three-column
which they have to apply their knowledge of chart structure as shown in Figure 10 to activate
finite state machine to nail down the design. students’ prior knowledge by recalling what
Soon after witnessing a serious accident at the students know about a problem (K), to motivate
intersection and being called for investigating students to read/think by asking what they want
a solution, players encounter the first set of to know (W), and finally to review what part
questions to ensure their understanding of the of the problem has been resolved and what
design specification as shown in Figure 9. is yet to be solved (S). This type of interven-
The question prompts also provide another tion is usually implemented in the traditional
active service to players by instantly giving classroom environment with facilitation from
them information about their current knowledge instructors. However, in a virtual game envi-
of the problem (What players know), their ronment, students are often left alone with the
weakness in understanding (What players do responsibility to explore and to figure out prob-
not know) or, rather, where they should seek lems themselves. The lack of guidance makes

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
48 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Figure 9. Sample questions prompt in Gridlock

it difficult to implement such an intervention fails some questions, the game immediately
because not all students are motivated to use it informs the player what went wrong as shown
without facilitation. in Figure 11. In the meantime, the system,
Can our game system take the role of the based on their answers and other information
facilitator? The answer to this question helped us gathered from their gameplay, selects the guid-
offer a new implementation of KWS – instead of ance tailored to their specific needs from an
asking students to write down what they know, instructional support standpoint. An example
don’t know, and have solved, the game connects of such help can be seen in Figure 12.
KWS with the question prompts. When a player

Figure 10. Original KWS structure in Sustain City

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 49

Figure 11. KWS in Gridlock

Figure 12. The roadmap in Gridlock

The specific learning information is pre- rials (Tang et al., 2011). Depending on game
sented in the format of a learning roadmap, content, a road map in Sustain City might be
providing study guides that endow students a task list, such as the one in Power Ville that
with the capability to find relevant information guides students to navigate through game as-
and to capture key concepts in the study mate- signments and retrieve important information

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
50 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

(Figure 13); or it might be a set of suggestions the fun aspect of a game. Rather than leading
designed to lead students through a problem a player with visual indicators, such as lines on
solving process by directing attention to key the ground, an AI companion can be used that,
ideas and suggesting the application of proper using the same algorithm mentioned before,
skills, such as the one in Gridlock (Figure 12). guides the player in a more personable and
In-game mobility is another issue in game interactive manner. For example, in Solaris
design as users might be non-gamers who do not One, a floating droid that matches the sci-fi
understand the conventions. Gamers understand setting is planned to be introduced to serve as
that any object on the screen may have meaning a guide while engaging the player.
in the game, and approaching it or rolling the
mouse over it may activate its functions. Non-
gamers, on the other hand, can be paralyzed ASSESSMENT
without explicit navigation clues. The resulting
To date, PowerVille and Gridlock have been
frustration immediately replaces any fun that
extensively piloted in both pre-engineering and
the players were having. In particular, Sustain
engineering classrooms. A mixed methodology
City is designed as a first-person game where a
was utilized for thorough evaluation and assess-
player experiences actions through the eyes of
ment that triangulated understanding for how
the protagonist. In other words, there is no “me”
the serious game system with metacognitive
in any game scene that is most likely expected by
interventions impacted student interests and
non-gamers. Manual and automated tools have
learning. Three research questions as listed
been put in place in Sustain City that helps to
below were used through the development of our
resolve this issue, each with a slightly different
evaluation plan shown in Table 1 that guided the
goal. As shown in Figure 13, a help menu is used
project assessment. Methods included surveys
to introduce the navigation through keyboard
and focus group interviews to determine:
and mouse (Figure 13(a)); simple arrows or lines
to show the player which direction they should
be heading if they get lost (Figure 13(b)); and a • To what extent is the metacognitive and
mini-map to show where the player is (Figure problem-solving content in the serious
13(c)). Techniques that are used for AI path- games useful to student learning;
finding, such as the node-based A* algorithm • To what extent do the serious games
(He et al., 2012), can also be used to trace the with metacognitive interventions play in
quickest route from a player to a destination. fostering student interests in engineering
The inclusion of pathfinding is intended as the problem-solving;
next step for Sustain City. • To what extent is the student learning
Player guidance in terms of navigation improved by the serious game experience
can also be used as an opportunity to enhance in general.

Figure 13. Navigation tools in Sustain City

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 51

Table 1. Evaluation Plan Matrix

Research Questions Evaluation Questions Evaluation Measures

To what extent is the metacognitive and 1. Frequency of game tools being used by students Surveys of the utility and
problem- solving content in the serious games 2. Open-ended questions on why students like/dislike a usability of game tools
useful to student learning? particular tool

To what extent does the serious games 1. The realism of games in delivering real-world 1. Focus group interviews
with metacognitive interventions play in engineering problems 2. Surveys of student interests in
fostering student interests in engineering 2. How fun and interesting does the problem-solving game learning
problem-solving? process in game compared to working out of a textbook/
lab instruction

To what extent is the student learning 1. What do student reflections from student game experience Surveys of student conceptual
improved by the serious game experience reveal about their learning learning
in general?

Utility and Usability of question prompts were connected with KWS

Metacognitive Interventions to facilitate student reflection, more students
started to appreciate their value (i.e., only
One instrument used for evaluation sought to 38% students felt KWS was not helpful). “The
determine how players perceived the useful- question prompts let me know that I was on the
ness of the various metacognitive tools built right track and gave me confidence in what I
into the games. In particular, students were was doing,” students wrote, “they reiterate the
surveyed about how helpful the tools were and concepts from the lectures needed to beat the
how often they used them during the games. game.” Although students perceived chatting
Two open-ended questions were also designed as an important tool, it was not used as often
in the survey to have students select the most/ as we expect. In students own comments, “it
least useful tools and provide their justification. serves no purpose since we were in the same
As shown in Charts 1 and 2, students had room” and “lab partners are in the room while
few problems figuring out how to use the tools, [game] lab is in progress.”
but of the three core supports, Road Map was the Over the course of development from
most popular, since Road Map, in their words, Power Ville (the first game in the series) to
“kept reminding me what I did and what I have Gridlock (the second game), the most dramatic
to get accomplished”, and “hints necessary change in student responses was the increased
concepts”. When KWS was used alone as a value attributed to KWS in Gridlock. It went
three-column chart in Power Ville, there was from the least-valued resource in Power Ville
not a direct purpose that required responses to the most-valued resource in the later game
from students to the intervention; thus students (Chart 1). Students were less likely to select
were not focused on the learning modeled in Road Map to retain, and more likely select Chat
such support as deeply as we had hoped (i.e., to omit. In terms of usability and helpfulness,
75% students felt KWS was not helpful). Once median values for KWS were the same across

Table 2. Attitude toward games, median values

Compared to Working this Problem out of a Textbook: (1-3: Less/Same/More)

Interest in Fun of the Access to Amount Career
Problem Process Resources Learned Interest
Power Ville (N=127) 2 3 2 2 2
Gridlock (N=62) 2 2 2 2 2

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
52 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Chart 1. Percentage of students selecting resources to keep or retain

*KWS includes Question Prompts in Gridlock

Chart 2. Use of and attitude toward metacognitive tools, median values

*KWS includes Question Prompts in Gridlock

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 53

the two games (Chart 2). The prediction of number in Binary, Octal, and Hexadecimal)
future use was less for Gridlock, presumably had relatively low correlation with the total
because the revised KWS with topic-specific score (.31 pre and .14 post) in 2011, but showed
prompts is not something that can generalize no such characteristic in 2012. All items were
unless it is programmed into a game or simula- retained for analysis.
tion. The students may be interpreting KWS as In 2011, the survey was given to a game-
a game-specific strategy rather than a general using class and to a comparison class. There
metacognitive approach that can be applied to was no significant difference between the
problem solving in general. group means at pretest, but the advantage of
the treatment group at the end of the term was
Interests and Motivation significant (t [37df]=1.89, p<.05). The survey
was repeated in 2012 with another class that
Student attitude towards the game scenario and used the game. While there was no comparison
learning environment was also measured (Table group in 2012, the results were similar to those
2). Students were asked to compare Power Ville for the treatment group from the year before
to learning material out of a book in terms of (Table 3).
interest, fun, resources, learning, and inclina-
tion towards engineering. Overall, students felt
the learning experience with Power Ville was OVERALL FINDINGS
comparable to covering the same material in a
book, with one exception: It was more fun to Careful consideration should be taken in inte-
play the game. Some students also applauded grating any technique meant to boost fun in a
the effort to present content in a problem con- serious game. If the fun is prioritized too greatly,
text as they commented ” The more realistic a player could become either distracted or halt
the scenario, the better. I enjoy working on progress completely in pursuit of a reward.
real-world problems because I feel it better Based on general play-testing and student feed-
prepares me than simply designing a circuit back, we recommend three concrete concepts
that performs some random function”, and “I around which to design a game that promotes
thought the movement in the game encouraged fun and minimizes frustration while maintaining
me a little more to make it work”. a balance between fun and learning:

Conceptual Understanding • The educational content should be woven

into a narrative that exists outside of norms,
An efficacy test of Gridlock was conducted in a including unusual settings and story;
low-division college level digital logic course at • Player guidance is most effectively imple-
Rowan University. Surveys measuring student mented by slowly stacking core concepts
understanding of core concepts were given to rather than providing all instruction at once;
game-using and control sections to rate stu- • Supplemental feedback can be realized
dents’ proficiency on core concepts. Response by rewarding the player with some kind
options on each item ranged from zero to five of spendable virtual currency or “points”.
(unfamiliar to proficient).
The 19-item Digital Logic survey used for
As many serious games are at least in
this assessment had high internal consistency
some degree simulation, it is far too common
(α=.92 pre and .86 post in fall 2011, α=.93 pre
that they present predictable settings such as a
and .94 post in fall 2012). Principal Component
hospital room for medical education or a labora-
Factor analysis showed a single factor explain-
tory for science. In focus groups we found that
ing 80% of variance. No other factor had an
students responded far more enthusiastically
eigenvalue greater than 1. Item 5 (Present a

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
54 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Table 3. Conceptual understanding after use of Gridlock

Survey at Course Start Survey at Course End

Year Group N Mean SD N Mean SD Change
Fall 2011 Game 35 .75 .57 18 3.95* .67 3.20
Control 26 .89 .81 21 3.62 .91 2.73
Fall 2012 Game 55 .59 .72 51 3.95 .71 3.36
*Group difference significant (t-test [37df] = 1.89, p < .05). Because the survey was anonymous, the statistic tests the
differences between groups at time 1 and time 2 rather the change in individuals over time.

to the sci-fi setting of Solaris One than to the methodology for spending the currency. We
straight-forward city structure of Sustain City found that it was effective to rely on purchas-
despite the considerably more challenging able upgrades that affect gameplay, enhancing
content. Unless the environment is an essential the performance of a rover the player pilots.
component to the educational topic, this strategy Although it can provide a huge boost to the
can be implemented without taking away from fun content of a serious game, the addition of a
the core content. Thus, there is no impact on reward system also runs the biggest risk of dis-
the learning component. The only barrier in a tracting the player. In our case, it is still possible
practical sense is that skilled artists are required for a student to remain inside the “fun” section
to construct such an environment and story, of the game without ever proceeding, however
potentially raising budgetary costs. the environment and story proved compelling
Player guidance is usually implemented enough that this was never the experience.
through text based instruction, or some kind of
bulleted guide. When referring to guidance, we
can refer to navigating a game’s environment CONCLUSION
and story as well as teaching core educational
This paper presents an approach to serious
content. Although it may be enticing to provide
games design that attempts to balance fun
large guides that provide the student with all in-
and education that are so often considered
formation necessary, we found that such guides
separately. The successful response to the many
were largely ignored. In fact, even if smaller
design challenges is made evident through
snippets were provided, the student would skip
three game examples, Power Ville, Gridlock,
quickly through them to arrive at the gameplay.
and Solaris One, where each subsequent game
It is far more beneficial and enjoyable to the
evolves to meet student feedback and experi-
student when the game slowly introduces con-
ence. An evaluation of the two games, Power
cepts through example and player experience,
Ville and Gridlock was conducted in several
forcing the player to tackle smaller tasks that
pre-engineering and engineering classrooms,
naturally stack to accomplish larger ones and
where over 200 students worked with the games
providing guides as a backup.
as the replacement of traditional laboratory
Players react very strongly to forms of
activities. Results from student surveys and
reward that exist within a virtual world. The
focus group interviews showed that the games
mobile and casual gaming market preys on the
were effective in promoting student learning.
player’s desire to accumulate “coins” or other
The game system was shown to be easy to work
types of virtual currency often in exchange for
with, useful and more beneficial than working
real world currency. Providing the player with
with the same problem solving in paper and
the ability to accumulate virtual cash is a far
pencil. In the future, we intend to implement
better alternative to a letter grade or a percent-
more dynamic systems. For instance, it is an
age. The key design decisions weigh on the

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 55

advantage to embed path-finding into the games Egenfeldt-Nielsen, S. (2010) Experiential eLearning:
that guides players throughout open-world An ID Model for Serious Games, Serious Games
Interactive, Retrieved from http://www.egenfeldt.
environments. Another challenge is to conduct
eu/papers/overview_serious_games.pdf
rigorous assessment in gameplay in order to of-
fer students personalized learning experiences Hays Blaine Lantz. Jr. (2009) Science, Technology,
[Johnson et al., 2014]. Engineering, and Mathematics (STEM) Education
What Form? What Function? What is STEM Educa-
tion, Retrieved From http://stem.afterschoolnetwork.
org/sites/default/files/stemeducation-whatformfunc-
ACKNOWLEDGMENT tionarticle.pdf
This work is supported in part by the National Herbert, B. E. (2003, November). The role of scaf-
Science Foundation grant #OCI-1041306 and folding student metacognition in developing mental
#EEC-0935089. models of complex, Earth and environmental systems.
In Proceedings of the DFG-NSF International Work-
shops on Research and Development in Mathematics
and Science Education, Washington, DC.
REFERENCES
Hocine, N., & Gouaich, A. (2011, May). A Survey of
Agent Programming and Adaptive Serious Games,
Arnab, S., Berta, R., Earp, J., de Freitas, S., Popesci,
Proc. of 10th Int. Conf. on Autonomous Agents
M., Romero, M., Stanescu, I., & Usart, M. (2012).
and Multiagent Systems (AAMAS 2011), Yolum,
Framing the adoption of serious games in formal
Tumer, Stone and Sonenberg (eds.), May, 2–6, 2011,
education. Electronic Journal of e-Learning, 10(2),
Taipei, Taiwan
159-171.
Hoffman, E. (2006). Games for Health 2006: Pulse!!
Barab, S. A., Gresalfi, M., & Arici, A. (2009). Why
First-Person Healthcare System Simulation. Seri-
educators should care about games. Teaching for the
ous Games Source. Retrieved from http://www.
21st Century, 67(1), 76-80.
gamecareerguide.com/news/11421/news/102373/
Bordogna, J. (1998, April). Next generation Engi- sgs_feature_pulse_first_person_.php
neering: Innovation through integration, Proc. of
Johnson, A., Tang, Y., & Franzwa, C. (2014, April)
NSF Eng. Educ. Innovator’s Conf.
kNN-based Adaptive Virtual Reality Game System,
Chen, J. C. (2007, June) Application of Transfor- To appear in Proceedings of IEEE International
mative Learning Theory in Engineering Education, Conference on Networking, Sensing, and Con-
Proceedings of 1st International Conference on trol, Miami, FL, April 7-9, 2014. doi::10.1109/
Research in Engineering Education, Honolulu, HI. ICNSC.2014.6819669

Church, W., & Skelton, L. (2010, May) Sustainability Mayo, M. (2009). Video Games: A Route to
education in K-12 classroom. Journal of Sustain- Large-Scale STEM Education? [PubMed]. AAAS
ability Education, 1. Science Journal, 323(5910), 79–82. doi:10.1126/
science.1166900 PMID:19119223
Coyne, R. (2003). Mindless repetition: Learning from
computer games. Design Studies, 24(3), 199–212. Mott, S. Lee, and J. Lester (2006, July) Probabilistic
doi:10.1016/S0142-694X(02)00052-2 goal recognition in interactive narrative environ-
ments, Proceedings of the 21st national conference
Cyberinfrastructure for Education and Learning for on artificial intelligence, Boston, Massachusetts.
the Future. A Vision and Research Agenda (2005),
Technical report, Computing Research Association. President’s Council of Advisors on Science and
Retrieved from http://archive.cra.org/reports/cyber- Technology. (2012) Report to the President: Engage
infrastructure.pdf to Excel: Producing One Million Additional College
Graduates with Degrees in Science, Technology, En-
de Freitas, S. (2010) Learning in Immersive worlds: gineering, and Mathematics, Retrieved From http://
A review of game-based learning, Jisc, Retrieved www.whitehouse.gov/sites/default/files/microsites/
From http://www.jisc.ac.uk/media/documents/pro- ostp/pcast-executive-report-final_2-13-12.pdf
grammes/elearninginnovation/gamingreport_v3.pdf

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
56 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014

Sandford, R., Ulicsak, M., Facer, K., & Rudd, T. Wilson, L. (2009). Best Practices for Using Games
(2006) Teaching with Games: Using commercial and Simulations in the Classroom: Guidelines for
off-the-shelf computer games in formal education. K-12 Educators. Software & Information Industry
Bristol. Futurelab. EA Futurelab. Retrieved from Association.
http://www2.futurelab.org.uk/resources/documents/
project_reports/teaching_with_games/TWG_report. Xiangguang, H., Yaya, W., & Yanyan, C. (2012,
pdf August) Researching on AI path-finding algorithm
in the game development International Symposium
Smith, R. (2009). The long history of gaming in mili- on Instrumentation & Measurement, Sensor Network
tary training. Simulation Gaming, 41, 183 898–920. and Automation 2, 484-486
Thomas, J. M., & Young, R. M. (2010). Annie: Ying, T., Sachin, S., Henry, J., Jahan, K., & Hargrove,
Automated generation of adaptive learner guid- S. (2011). Interactive and Collaborative Games
ance for fun serious games. IEEE Transactions on Promoting Metacognition for Science and Engineer-
Learning Technologies, 4(3), 329–343. doi:10.1109/ ing Design. In M. Zhou & H. Tan (Eds.), Advances
TLT.2010.32 in Computer Science and Education Applications
- Communications in Computer and Information
Torrente, J., Blanco, A., Moreno-Ger, P., & Fernan- Science (Vol. 202, pp. 405–412). Springer Berlin
dez-Manjon, B. (2012). Designing serious games for Heidelberg; doi:10.1007/978-3-642-22456-0_58
adult students with cognitive disabilities (Lecture
Notes in Computer Scirence 7666). Ying, T., Sachin, S., Kauser, J., Henry, J., & Hargrove,
S. K. (2012). Sustain City – A Cyberinfrastructure-
Ulicsak, M. (2010) Games in Education: serious Enabled Game System for Science and Engineering
games, Futurelab Retrieved From http://media. Design. Journal of Computational Science Educa-
futurelab.org.uk/resources/documents/lit_reviews/ tion, 1(3), 57–65.
Serious-Games_Review.pdf
Zielke, M. A., Evans, M. J., Dufour, F., Christopher, T.
Wattanasoontorn, V., Garc’ıa Hern’andez, R. J., & V., Donahue, J. K., & Johnson, P. et al. (2009). Seri-
Sbert, M. (2012) Serious games for e-health care. ous games for immersive cultural training: Creating
Asian-European Workshop on Serious Game and a living world. [PubMed]. IEEE Computer Graph-
Simulation, the 25th Annual Conference on Computer ics and Applications, 29(2), 49–60. doi:10.1109/
Animation and Social Agents doi:10.1007/978-981- MCG.2009.30 PMID:19462634
4560-32-0_9

Christopher Franzwa received a B.S. degree from Rowan University, Glassboro, NJ in 2013.
Currently, he is enrolled as an Electrical and Computer Engineering Master’s student at Rowan
University. His research interests include computer vision, virtual/augmented reality, serious
game design, and GPU computing.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 International Journal of Game-Based Learning, 4(4), 37-57, October-December 2014 57

Ying Tang received the B.S. and M.S. degrees from the Northeastern University, P. R. China, in
1996 and 1998, respectively, and Ph. D degree from New Jersey Institute of Technology, Newark,
NJ, in 2001. Dr. Tang is a Professor of Electrical and Computer Engineering at Rowan University,
Glassboro, NJ. Her research interests include discrete event systems and visualization, Petri
nets and applications, virtual/augmented reality, green manufacturing design and optimization,
intelligent learning game systems and artificial intelligent. Dr. Tang has led several research and
education projects funded by National Science Foundation, US Department of Transportation,
US Navy, the Charles A. and Anne Morrow Lindbergh Foundation, the Christian R. and Mary
F. Lindback Foundation and industry firms. Her work has resulted in 4 book chapters, and over
100 journal and conference proceedings articles. Dr. Tang is a member of the IEEE Systems,
Man and Cybernetics, and the IEEE Robotics and Automation.

Aaron Johnson received his B.S. degree from Rowan University in 2012. He is currently a M.S.
student of Electrical and Computer Engineering at Rowan University. His research interests
include machine learning, serious game programming and development, web server processing
and UI back end programming.

Talbot Bielefeldt is a program evaluator who works on numerous educational initiatives of

government agencies and private foundations. His own research has focused on educational
technology, classroom observation, and professional development. Mr. Bielefeldt runs Clearwater
Program Evaluation in Eugene, Oregon. He holds an MS in Educational Policy and Manage-
ment from the University of Oregon.

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.