CIRTL Class Meeting 9: Teaching as research

The College Classroom – Spring 2015
Class Meeting 9: Teaching as Research
Dave Gross
dgross@
biochem.umass.edu
Thursday, March 26, 2015
1:00-2:30p ET, 12:00-1:30p CT, 11:00a-12:30p MT, 10:00-11:30a PT
Peter Newbury
pnewbury@ucsd.edu
@polarisdotca

Vocabulary check: SoTL and TAR
Teaching as research - collegeclassroom.ucsd.edu2
research done by you
in your class to inform how
you will teach in the future
carefully
designed experiment
to explore learning
paper
in journal
Scholarship ofTeaching and
Learning (SoTL),
Education Research
teaching as
research

Teaching as Research (TAR)
At the core of improving teaching and learning is the need
to accurately determine what students have learned as a result of
teaching practices. This is a research problem, to which STEM
instructors can effectively apply their research skills and ways of
knowing. In so doing, STEM instructors themselves become the
agents for change in STEM teaching and learning.
Teaching-as-Research involves the deliberate, systematic,
and reflective use of research methods to develop and implement
teaching practices that advance the learning experiences and
outcomes of students and teachers.
CIRTL Network [1]

Some of your Education journals
Institution of Chemical Engineers Education for Chemical Engineers
American Chemical Society Journal of Chemical Education
Academy of Management Academy of Management Learning &
Education (AMLE) (Journal)
American Geophysical Union Journal of Geoscience Education (under
NationalAssociation of GeoscienceTeachers)
INFORMS INFORMSTransactions on Education
IOP Science Physics Education
Certified Hospitality Educator International Journal of Hospitality
Management
American Society for Cell Biology Life Sciences Education
Society for Neuroscience Journal of Undergraduate Science Education

Categories of Educational Research[2]
Action or practitioner research
Experimental
'Cause and effect' research

Categories of Educational Research[2]
Theoretical research
Action or practitioner research
Evaluative Experimental
'Cause and effect' research
Case study
Systematic review
Exploratory
Comparative
Grounded theory
Ethnography

What is the value of course-specific learning goals?
Simon & Taylor (2009)
 3 classes (A = computer literacy Fa07, B = computer
literacy Sp08, C = microbiology Sp08)
 Last week of course (Wk 13): students asked to
complete up to five copies of,“For me, the use of
learning goals in this course is . . .”
 Comments put into categories using content-analysis
based coding
A B C total
Comments 225 252 120 597
Students 59 76 51 186
(Evaluative)

What is the value of course-specific learning goals?
Simon & Taylor (2009)
Research informs the
instructors about
 how students are
responding
 what’s working (or not)
 what to use again (or
not)
Helps instructors become
better educators.

Active learning increases student performance
in science, engineering and mathematics[4]
Meta-analysis of 225 research studies that explored the
impact of active learning:
Active learning engages students in the process of learning
through activities and/or discussions in class,as opposed to
passively listening to an expert.It emphasizes higher-order
thinking and often involves group work.
(Freeman et al., pp 8413-8414)
(Systematic review)

Active learning increases student performance
in science, engineering and mathematics[4]
 established active learning
has positive impact on
learning
 advances field of education
research: no need to
continue to replicate study
(in STEM)

What can you study?
Students’ attitudes
 what are they initially?
(e.g., what are students’ views on academic integrity?)
 how did they change after you did X?
(e.g., what do students think about learning goals?)
Students’ knowledge and skills
 what are students able to do now that they couldn’t do
before taking the course?
 are students thinking more like experts?
(e.g. what questions do they ask in lab/discussion?)

Common education research tools
Quantitative Qualitative
Teaching as research - collegeclassroom.ucsd.edu
What is happening? Why is it happening?
 Multiple choice tests (pre- and post-testing?)
 Attitude surveys (Likert SD, D, N,A, SA scale)
 mini-writing (muddiest point, think-pair-share, etc.)
 document students’
 answers on a worksheets  responses to a question
 opinions/views  peer instruction votes  questions
12

Can You Solve This?
As you watch this video,
 Play along with the Derek’s
puzzle
 Watch from an educator’s
point-of-view: when did
learning happen?
www.youtube.com/watch?v=vKA4w2O61Xo
See you back here in 5 minutes.

Can You Solve This?
Beware of confirmation bias: the
tendency to search for, interpret,
or recall information in a way that
confirms one's beliefs or
hypotheses.[8]
(Educator’s point-of-view:When did learning happen?)

How do you think the students felt?
Suppose you were a student in one of the active learning
classes included in the meta-analysis by Freeman et al.[4]
Do you think that research would have any impact on your
experiences and/or success in that class?
A) yes
B) no

How do you think the students felt?
Suppose you were a student in one of the Biology or
Computer Science classes asked to complete the sentence,
“For me, the use of learning goals in this course is . . .” for
the study by Simon andTaylor about the value of course-
specific learning goals.
Do you think that experiment would have any impact on
your experiences and/or success in that class?
A) yes
B) no

Imagine this…
Professor Smith teaches 2 sections of the same course.
Section 1 is an active classroom using peer instruction
(clickers). Section 2 is a traditional lecture. Both Sections
take the same final exam. Professor Smith analyzes the
students’ grades to investigate the impact of active learning.
Do you think this experiment had any impact on the success
and/or experiences of students in Section 1, Section 2?
A) yes, yes
B) yes, no
C) no, yes
D) no, no

If you suspect your research could impact students physical
well-being, privacy, or success, you should speak with your
IRB. The IRB may insist you apply for approval or
suggest you apply for an exemption from IRB review.
Humans are involved in TAR
Mission of the Institutional Review
Board (IRB):
1. Protection of human subjects from
physical harm.
2. Protection of students’ privacy and
students’ success.

IRB at your institution?
Take 5 minutes to find
1. the name of the organization that handles human
subjects research approval (it’s probably not the “IRB”)
2. what research activities are exempt from IRB approval?
3. which exemption(s) do you think apply to teaching-
as-research? (you’re going to copy-and-paste that
exemption onto our whiteboard later so keep the
window/tab open)
See you back here in 5 minutes.

Check one:
“exceptions” “exemptions”

Use the textbox tool to copy-n-paste
an exemption applicable to TAR

UC San Diego IRB
grants exemptions to…
(1) Research conducted in established or commonly accepted educational
settings, involving normal educational practices, such as (i) research on
regular and special education instructional strategies, or (ii) research on
the effectiveness of or the comparison among instructional techniques,
curricula,or classroom management methods.[5]
Things a scholarly, reflective instructor
might and should do anyway.

Involve your Center for Teaching
Before you apply for an exemption, check with your local
Center forTeaching. (At UC San Diego, we have a blanket
exemption for projects studying active learning, as long as
the Center is involved.)

A full inquiry cycle of TAR
1. Learning foundational knowledge
2. Creating objectives for student learning
3. Developing an hypothesis for practices to achieve the
learning objectives
4. Defining measures of success
5. Developing and implementing teaching practices
within an experimental design
6. Collecting and analyzing data
7. Reflecting, evaluating, and iterating
www.cirtl.net/CoreIdeas/teaching_as_research

Teaching as Research case study
Watch for the steps in the inquiry cycle in Dave’s
Biochem/Chem 471 research project. (Perhaps keep the
CIRTL webpage open and visible?)
At the end, together we’ll try to identify the inquiry cycle
in Dave’s project.

The course
Teaching as Research - collegeclassroom.ucsd.edu26
Biochem/Chem 471
Elementary Physical Chemistry
• Upperclass majors
• Several prerequisites
AY
#
students
#
sections
# meetings/
week
# minutes/
week
Active Exam score
2007-8 79 1 3 150 N 63.3 ± 15.8a
2008-9 88 1 3 150 N 70.5 ± 14.0a
2009-10 78 1 3 150 N 72.4 ± 15.1
2011-12 111 3 1 75
Y, clickers,
peer
77.0 ± 14.8b
2012-13 133 2 2 100 Y,TBL 75.8 ± 15.1b

Evolution of Chem/Biochem 471
Fall 04 Half exams in class (78%) & Final (22%)
Standard lecture format
Fall 06 Midterm and final (25% ea)
Online OWL quizzes (20%)
Online OWL homework (30%)
Fall 07 Two exams (40%) & Final (20%)
Online OWL quizzes (10%)
Online OWL homework (25%)
Online OWLBook examples (5%)

Results - assessments
0
4
8
12
16
1 2 3 4
Percentchange
GPA quartile
Exam 1
Exam 2
Exam 3
45
55
65
75
85
Exam 1 Exam 2 Exam 3
Examaverage
Pre
Post

Results – use of online materials
1
10
100
1000
20 40 60 80 100
Numberofvideoaccesses
Exam average
1st quartile
2nd quartile
3rd quartile
4th quartile
0.0
0.1
0.2
0.3
0.4
0.5
-2 -1 0 1 2
Fractionofvideoaccesses
Week from exam
Top quartile
2nd quartile
3rd quartile
4th quartile

Timeline
Fall 04
Fall 06
Standard class
Introduce online homework
and online quizzes
Fall 07 Introduce online interactive textbook
Spring 12 Multiple sections, active classroom,
collaborative learning
Spring 13 Team-based classroom, peer evaluations

Timeline
Fall 04
Fall 06
Standard class
and online quizzes
Spring 12 Flip class, active classroom,
Fall 01 TEACHnology Fellow
TBL Fellow
Blended Learning Fellow
NationalAcademies Summer InstituteSummer 10

Timeline
32
Fall 04
Fall 06
Standard class
and online quizzes
Spring 12 Flip class, active classroom,
Fall 01 TEACHnology Fellow
TBL Fellow
Blended Learning Fellow
NationalAcademies Summer InstituteSummer 10
Hypothesize improved outcomes
Summer 11
Accepted with revisions, CBE
Life Science Education
Summer 13
Summer 13
Spring 15

Timeline
Hypothesize improved outcomes with the flipped class
Plan with M. Graham (data needed, analysis)
Fall 10
Accepted with revisions, CBE Life Science Education
Summer 11
Spring 12
Spring 15
Collect flipped data, initial analysis
Presentations (SABER - feedback)Summer 12
Spring 13 Collect round 2 data, obtain IRB approval, recruit data miners (GPA, SAT, OWL)
Revisit plan with M. GrahamFall 12
Summer 13 Recruit statistician, do deep analysis
Summer 14 Manuscript submission (3 X)
1. foundational knowledge 2. objectives 3. hypothesis for practices 4. defining success
5. designing and teaching 6. collect & analyze data 7. reflect/evaluate/reiterate/(share)

Next class meeting (our last one!)
Thursday, April 10
Watch the blog and email for information about
 Growth and Fixed Mind
 planning the first day of class
 microteaching project and presentations
Before next week, all class meeting resources will be available on the
blog and you’ll have access to session recordings if you need them.

References
1. Center for the Integration of Research,Teaching and Learning (2010) Teaching as Research.
www.cirtl.net/CoreIdeas/teaching_as_research
2. Lambert, M. (2012). A Beginner's Guide to DoingYour Education Research Project.Thousand Oaks, CA: SAGE
Publications Inc. via Tomorrow’s Professor cgi.stanford.edu/~dept-ctl/tomprof/posting.php?ID=1233
3. Simon, B., &Taylor, J. (2009). What is theValue of Course-Specific Learning Goals? J. College ScienceTeaching, 39, 2,
52-57.
4. Freeman, S., Eddy, S.L. McDonough, M., Smith, M., Okoroafor, N., Jordt,. H. &Wenderoth, M.P. (2014)Active
learning increases student performance in science, engineering, and mathematics. PNAS 111, 23, 8410–8415.
5. Exemption from IRB Review, UCSD Human Research Protections Program, retrieved from
irb.ucsd.edu/Exempt_forms.shtml 24/2/ 2014.
6. Adams,W.K., Perkins, K.K., Podolefsky, N.S., Dubson, M., Finkelstein, N.D., &Wieman, C.E. (2006) A new
instrument for measuring student beliefs about physics and learning physics:The Colorado Learning Attitudes about Science
Survey (CLASS). Phys. Rev. ST Phys. Educ. Res. 2, 010101.
7. Confirmation bias (n.d.). In Wikipedia. Retrieved February 16, 2015, from
http://en.wikipedia.org/wiki/Confirmation_bias

CIRTL Class Meeting 9: Teaching as research

More Related Content

What's hot

Viewers also liked

Similar to CIRTL Class Meeting 9: Teaching as research

More from Peter Newbury

Recently uploaded

CIRTL Class Meeting 9: Teaching as research