Session 2 - Learning, Design and Technology in Schools, Skills 
Training and Higher Education 
Developmental evaluation and the experience API 
Professor Robert Fitzgerald 
www.assess2learn.com.au 
www.assess2learn.com.au
Learning, design and technology: Developmental 
evaluation and the experience API 
Professor Robert Fitzgerald 
Faculty of Education, Science, Education and Technology, University of Canberra 
Rethinking Assessment and Learning, GLOBAL MINDSET, Australian Technology Park, 
Sydney, 29 October, 2014
INNOVATION RESEARCH EDUCATION
INSPIRE: Research, Education & Innovation 
http://www.ideo.com http://www.tpck.org 
• Design Thinking & STEM education 
• Next practice learning & design 
• Mastery pedagogies 
• Augmented reality 
• Location based education services 
• ICT4D – Pakistan & Cambodia 
• Google Glass & Wearables 
• National mentoring for Maths & 
Science teachers 
• The Experience API
Educational challenges 
• The “new normal” of increasing demand, diminishing resources, 
increased expectations and constant change 
• Widespread consensus that graduates are not well prepared for the 
world of work and subsequent lifelong learning (AAEG, 2011; Candy, 
1991; Candy, Crebert & O’Leary, 1994; Oliver, 2011; Shah & Nair, 
2011) 
• John Seely Brown argues we need to radically shift the focus of 
education from the development of skills & knowledge, to the 
cultivation of mindsets and dispositions (Thomas & Brown, 2011)
Source: OECD Education
Technology futures 
• Emerging forms of interactive media and information communication 
technology are reshaping almost every aspect of our work and 
social life; Boundaries are blurring 
• New practices and literacies are emerging from digital 
communications and culture that challenge our traditional ideas 
about the form and function of communication, learning and 
education 
• Technology is more than just a ‘tool’ but an evocative ‘object to think 
with’ and an engine of social and cultural change; Change is 
ecological not additive 
• Design thinking: New opportunities for participation and interaction 
arise from communities of interest where users are active designers 
& content creators re-mixing, re-purposing and re-distributing 
content
A lesson: ICT benefits depend on “capital” 
Capital = skills, interests, attitudes, resources 
s 
c 
I 
e 
n 
c 
e 
The capital gap 
Source: www.oecd.org
The barriers to transformation are not "conceptual, technical, or 
economic. The primary barriers are psychological, political, and 
cultural. We now have all the means necessary to implement effective 
educational models that can prepare all students for a future very 
different from the immediate past. Whether we have the professional 
commitment and the societal will to actualize such a vision remains to 
be seen.” 
Dede et al (2013) Educause Review
Feedback - Learning, change & evaluation 
• We know that feedback is the most effective way of improving 
student learning (Hattie, 2009) 
• Forms of Evaluation - Formative, Summative & Developmental 
(Patton, 2011) 
• Summative and formative evaluation are most often used to 
examine established programs 
• Developmental Evaluation (DE) is well-suited to emergent or 
developing systems 
• DE is a design-based research tool for understanding innovation 
and change in educational systems 
• Salient features 
– engage in simulations and/or rapid reconnaissance 
– deploy tools that develop maps of the territory; and 
– allow for revised and emergent modelling (Patton, 2011)
Technology Assisted Developmental 
Evaluation 
• ‘What? So What? Now What? (Gamble, 2008, p. 47) 
• Learning & change should be expansive and transferable and not 
limited to contemporaneous testing (Engeström, 2006; Wiggins & 
McTighe, 2011, p. 5) 
• The theoretical perspectives draws on Vygotsky (1978), Activity 
Theory (Engestrom, 2001), Variation Theory (Marton & Tsui, 2004) 
and the concept of Communities of Practice (Lave & Wenger, 1991) 
• Towards a Technology Assisted Developmental Evaluation (TADE) 
framework that helps teachers and institutions with their data work in 
ways that support educational decision making in a practical and 
accessible form (Leonard, Fitzgerald & Bacon [under review]) 
• “A system is not the sum of its parts, but rather, the product of the 
interaction of the parts” (Russell Ackoff)
Bernhardt, V. (2004). Data analysis for continuous school improvement (2nd ed.). Larchmont, NY: Eye on Education.
Learning everywhere 
Source: tincanapi.com/overview
INSPIREx: A data-driven educational environment 
based on Experience API 
Source: www.brightcookie.com
INSPIREx learning ecology 
Source: www.brightcookie.com
Personalised learning & The Quantified Self 
Source: economist.com
Source: www.cnet.com/news/lumosity-vs-elevate-brain-training-apps
Data for the learner 
• Quantified Self goes to school 
– self-knowledge through self-tracking 
– designing self-experiments 
• What does an increasing personalisation of education and learning 
mean for learners, teachers, institutions? 
• Implications for research – develop repeated measures and single 
subject designs 
• What changes are required when we shift from data about the 
learner to data for the learner? 
• How do we develop hindsight, oversight, foresight, insight in these 
new data rich environments? 
• What happens when learners tell ‘us’ when they are ready for 
learning?
Rob Fitzgerald 
robert.fitzgerald@canberra.edu.au 
Twitter: @rfitzgerald 
INSPIRE 
Website: inspire.edu.au 
Facebook: www.facebook.com/inspiredu2 
Twitter: @inspiredu2

Learning, design and technology developmental evaluation and the experience api

  • 1.
    Session 2 -Learning, Design and Technology in Schools, Skills Training and Higher Education Developmental evaluation and the experience API Professor Robert Fitzgerald www.assess2learn.com.au www.assess2learn.com.au
  • 2.
    Learning, design andtechnology: Developmental evaluation and the experience API Professor Robert Fitzgerald Faculty of Education, Science, Education and Technology, University of Canberra Rethinking Assessment and Learning, GLOBAL MINDSET, Australian Technology Park, Sydney, 29 October, 2014
  • 3.
  • 6.
    INSPIRE: Research, Education& Innovation http://www.ideo.com http://www.tpck.org • Design Thinking & STEM education • Next practice learning & design • Mastery pedagogies • Augmented reality • Location based education services • ICT4D – Pakistan & Cambodia • Google Glass & Wearables • National mentoring for Maths & Science teachers • The Experience API
  • 7.
    Educational challenges •The “new normal” of increasing demand, diminishing resources, increased expectations and constant change • Widespread consensus that graduates are not well prepared for the world of work and subsequent lifelong learning (AAEG, 2011; Candy, 1991; Candy, Crebert & O’Leary, 1994; Oliver, 2011; Shah & Nair, 2011) • John Seely Brown argues we need to radically shift the focus of education from the development of skills & knowledge, to the cultivation of mindsets and dispositions (Thomas & Brown, 2011)
  • 8.
  • 9.
    Technology futures •Emerging forms of interactive media and information communication technology are reshaping almost every aspect of our work and social life; Boundaries are blurring • New practices and literacies are emerging from digital communications and culture that challenge our traditional ideas about the form and function of communication, learning and education • Technology is more than just a ‘tool’ but an evocative ‘object to think with’ and an engine of social and cultural change; Change is ecological not additive • Design thinking: New opportunities for participation and interaction arise from communities of interest where users are active designers & content creators re-mixing, re-purposing and re-distributing content
  • 10.
    A lesson: ICTbenefits depend on “capital” Capital = skills, interests, attitudes, resources s c I e n c e The capital gap Source: www.oecd.org
  • 11.
    The barriers totransformation are not "conceptual, technical, or economic. The primary barriers are psychological, political, and cultural. We now have all the means necessary to implement effective educational models that can prepare all students for a future very different from the immediate past. Whether we have the professional commitment and the societal will to actualize such a vision remains to be seen.” Dede et al (2013) Educause Review
  • 12.
    Feedback - Learning,change & evaluation • We know that feedback is the most effective way of improving student learning (Hattie, 2009) • Forms of Evaluation - Formative, Summative & Developmental (Patton, 2011) • Summative and formative evaluation are most often used to examine established programs • Developmental Evaluation (DE) is well-suited to emergent or developing systems • DE is a design-based research tool for understanding innovation and change in educational systems • Salient features – engage in simulations and/or rapid reconnaissance – deploy tools that develop maps of the territory; and – allow for revised and emergent modelling (Patton, 2011)
  • 13.
    Technology Assisted Developmental Evaluation • ‘What? So What? Now What? (Gamble, 2008, p. 47) • Learning & change should be expansive and transferable and not limited to contemporaneous testing (Engeström, 2006; Wiggins & McTighe, 2011, p. 5) • The theoretical perspectives draws on Vygotsky (1978), Activity Theory (Engestrom, 2001), Variation Theory (Marton & Tsui, 2004) and the concept of Communities of Practice (Lave & Wenger, 1991) • Towards a Technology Assisted Developmental Evaluation (TADE) framework that helps teachers and institutions with their data work in ways that support educational decision making in a practical and accessible form (Leonard, Fitzgerald & Bacon [under review]) • “A system is not the sum of its parts, but rather, the product of the interaction of the parts” (Russell Ackoff)
  • 14.
    Bernhardt, V. (2004).Data analysis for continuous school improvement (2nd ed.). Larchmont, NY: Eye on Education.
  • 15.
    Learning everywhere Source:tincanapi.com/overview
  • 16.
    INSPIREx: A data-driveneducational environment based on Experience API Source: www.brightcookie.com
  • 17.
    INSPIREx learning ecology Source: www.brightcookie.com
  • 18.
    Personalised learning &The Quantified Self Source: economist.com
  • 19.
  • 20.
    Data for thelearner • Quantified Self goes to school – self-knowledge through self-tracking – designing self-experiments • What does an increasing personalisation of education and learning mean for learners, teachers, institutions? • Implications for research – develop repeated measures and single subject designs • What changes are required when we shift from data about the learner to data for the learner? • How do we develop hindsight, oversight, foresight, insight in these new data rich environments? • What happens when learners tell ‘us’ when they are ready for learning?
  • 21.
    Rob Fitzgerald [email protected] Twitter: @rfitzgerald INSPIRE Website: inspire.edu.au Facebook: www.facebook.com/inspiredu2 Twitter: @inspiredu2

Editor's Notes

  • #5 The design
  • #6 Learning in action
  • #7 Inspired by design thinking, service design and “jams”, we use friendly competition to turn ideas into concrete designs, prototypes and plans of action
  • #13 Understood as an expansive activity, learning is about acquiring the tools of the learner’s socio-cultural context and ‘growing into the intellectual life of those around them’ (Vygotsky, 1978, p. 88) The use of those tools though, only makes sense when used for activity, that is the interaction between the learner and their context. In this formation, ‘learning’ is not an isolated product or ‘performance’, but rather is the integration of concept, learner, and their community. Learning connects ‘upwards, downwards and sideways’ (Engeström, 2006).