Research Methods

in Education
Eighth edition

Louis Cohen, Lawrence Manion and

Keith Morrison
Contents

List of figures xiv 2 Mixed methods research 31

List of tables xvi 2.1 Introduction 31
List of boxes xix 2.2 What is mixed methods research? 32
List of contributors xxi 2.3 Why use mixed methods research? 33
Preface to the eighth edition xxii 2.4 The foundations of mixed methods
Acknowledgements xxv research 34
2.5 Working with mixed methods
approaches 38
PART 1 2.6 Stages in mixed methods research 48
2.7 Conclusion 48
The context of educational research 1
3 Critical educational research 51
1 The nature of enquiry: setting the field 3 3.1 Critical theory and critical educational
1.1 Introduction 3 research 51
1.2 The search for understanding 3 3.2 Criticisms of approaches from critical
1.3 Conceptions of social reality 5 theory 54
1.4 Paradigms 8 3.3 Participatory research and critical
1.5 Positivism 10 theory 55
1.6 The assumptions and nature of science 10 3.4 Feminist research 58
1.7 The tools of science 12 3.5 A note on post-­colonial theory and queer
1.8 The scientific method 13 theory 63
1.9 Criticisms of positivism and the scientific 3.6 Value-­neutrality in educational
method 14 research 63
1.10 Post-­positivism 16 3.7 A summary of three major paradigms 65
1.11 Alternatives to positivistic and
post-­positivist social science: naturalistic 4 Theory in educational research 68
and interpretive approaches 17 4.1 What is theory? 68
1.12 A question of terminology: the normative 4.2 Why have theory? 71
and interpretive paradigms 19 4.3 What makes a theory interesting? 71
1.13 Phenomenology, ethnomethodology, 4.4 Types of theory 72
symbolic interactionism and 4.5 Where does theory come from? 76
constructionism 20 4.6 Questions about theory for researchers 77
1.14 Criticisms of the naturalistic and 4.7 Conclusion 77
interpretive approaches 23
5 Evaluation and research 79
1.15 Postmodernism and post-­structuralist
5.1 Similarities and differences between
perspectives 24
research and evaluation 79
1.16 Subjectivity and objectivity in educational
5.2 Evaluation research and policy making 82
research 25
5.3 Research, evaluation, politics and policy
1.17 The paradigm of complexity theory 27
making 83
1.18 Conclusion 29

vii
contents

6 The search for causation 87 8 Ethics in Internet research 144

6.1 Introduction 87 8.1 What is Internet research? 144
6.2 Causes and conditions 87 8.2 What are key ethical issues in Internet
6.3 Causal inference and probabilistic research? 144
causation 88 8.3 Informed consent 145
6.4 Causation, explanation, prediction and 8.4 Public and private matters 146
correlation 92 8.5 Confidentiality and anonymity 148
6.5 Causal over-­determination 94 8.6 Ethical codes for Internet research 149
6.6 The timing and scope of the cause and the 8.7 Conclusion 152
effect 95
9 Choosing a research project 153
6.7 Causal direction, directness and
9.1 Introduction 153
indirectness 96
9.2 What gives rise to the research
6.8 Establishing causation 96
project? 153
6.9 The role of action narratives in
9.3 The importance of the research 156
causation 98
9.4 The purposes of the research 157
6.10 Researching causes and effects 99
9.5 Ensuring that the research can be
6.11 Researching the effects of causes 101
conducted 158
6.12 Researching the causes of effects 103
9.6 Considering research questions 160
6.13 Conclusion 107
9.7 The literature search and review 161
9.8 Summary of key issues in choosing a
PART 2 research topic or project 162
Research design 109 10 Research questions 165
10.1 Why have research questions? 165
7 The ethics of educational and social 10.2 Where do research questions come
research 111 from? 165
7.1 Introduction 111 10.3 What kinds of research question are
7.2 Ethical principles and the nature of ethics there? 166
in educational research 112 10.4 Devising your research question(s) 167
7.3 Sponsored research 114 10.5 Making your research question
7.4 Regulatory contexts of ethics 115 answerable 169
7.5 Choice of research topic and research 10.6 How many research questions should I
design 120 have? 172
7.6 Informed consent 122 10.7 A final thought 172
7.7 Non-­maleficence, beneficence and human
dignity 127 11 Research design and planning 173
7.8 Privacy 128 11.1 Introduction 173
7.9 Anonymity 129 11.2 Approaching research planning 174
7.10 Confidentiality 130 11.3 Research design and methodology 175
7.11 Against privacy, confidentiality and 11.4 From design to operational planning 177
anonymity 130 11.5 A framework for planning research 177
7.12 Deception 132 11.6 Conducting and reporting a literature
7.13 Gaining access and acceptance into the review 181
research setting 134 11.7 Searching for literature on the Internet 183
7.14 Power and position 136 11.8 How to operationalize research
7.15 Reciprocity 137 questions 185
7.16 Ethics in data analysis 137 11.9 Distinguishing methods from
7.17 Ethics in reporting and dissemination 139 methodologies 186
7.18 Responsibilities to sponsors, authors and 11.10 Data analysis 186
the research community 141 11.11 Presenting and reporting the results 186
7.19 Conclusion 141 11.12 A planning matrix for research 188

viii
 contents

11.13 Managing the planning of research 194 PART 3

11.14 A worked example 196
Methodologies for educational research 285
11.15 Ensuring quality in the planning of
research 201 15 Qualitative, naturalistic and
ethnographic research 287
12 Sampling 202
15.1 Foundations of qualitative, naturalistic
12.1 Introduction 202
and ethnographic inquiry 288
12.2 The sample size 203
15.2 Naturalistic research 292
12.3 Sampling error 209
15.3 Ethnographic research 292
12.4 Statistical power and sample size 211
15.4 Critical ethnography 294
12.5 The representativeness of the sample 212
15.5 Autoethnography 297
12.6 The access to the sample 213
15.6 Virtual ethnography 299
12.7 The sampling strategy to be used 214
15.7 Phenomenological research 300
12.8 Probability samples 214
15.8 Planning qualitative, naturalistic and
12.9 Non-­probability samples 217
ethnographic research 301
12.10 Sampling in qualitative research 223
15.9 Reflexivity 302
12.11 Sampling in mixed methods research 224
15.10 Doing qualitative research 303
12.12 Planning a sampling strategy 225
15.11 Some challenges in qualitative,
12.13 Conclusion 226
ethnographic and naturalistic
13 Sensitive educational research 228 approaches 320
13.1 Introduction 228
16 Historical and documentary research 323
13.2 What is sensitive research? 228
JANE MARTIN
13.3 Sampling and access 230
13.4 Ethical issues in sensitive research 233 16.1 Introduction 323
13.5 Effects of sensitive research on the 16.2 Some preliminary considerations: theory
researcher 236 and method 323
13.6 Researching powerful people 237 16.3 The requirements and process of
13.7 Researching powerless and vulnerable documentary analysis 325
people 240 16.4 Some problems surrounding the use of
13.8 Asking questions 242 documentary sources 325
13.9 Conclusion 243 16.5 The voice of the past: whose account
counts? 326
14 Validity and reliability 245 16.6 A worked example: a biographical
14.1 Defining validity 245 approach to the history of education 328
14.2 Validity in quantitative research 246 16.7 Conclusion 332
14.3 Validity in qualitative research 247
14.4 Validity in mixed methods research 250 17 Surveys, longitudinal, cross-­sectional and
14.5 Types of validity 252 trend studies 334
14.6 Triangulation 265 17.1 Introduction 334
14.7 Ensuring validity 267 17.2 What is a survey? 334
14.8 Reliability 268 17.3 Advantages of surveys 334
14.9 Reliability in quantitative research 268 17.4 Some preliminary considerations 336
14.10 Reliability in qualitative research 270 17.5 Planning and designing a survey 337
14.11 Validity and reliability in interviews 271 17.6 Survey questions 340
14.12 Validity and reliability in experiments 276 17.7 Low response, non-­response and missing
14.13 Validity and reliability in data 341
questionnaires 277 17.8 Survey sampling 345
14.14 Validity and reliability in observations 278 17.9 Longitudinal and cross-­sectional
14.15 Validity and reliability in tests 279 surveys 347
14.16 Validity and reliability in life histories 283 17.10 Strengths and weaknesses of longitudinal,
14.17 Validity and reliability in case studies 284 cohort and cross-­sectional studies 349

ix
contents

17.11 Postal, interview and telephone 21 Meta-­analysis, systematic reviews and

surveys 352 research syntheses 427
17.12 Comparing methods of data collection in HARSH SURI
surveys 357
21.1 Introduction 427
18 Internet surveys 361 21.2 Meta-­analysis 428
18.1 Introduction 361 21.3 Systematic reviews 430
18.2 Advantages of Internet surveys 361 21.4 Methodologically inclusive research
18.3 Disadvantages of Internet surveys 362 syntheses 431
18.4 Constructing Internet-­based surveys 363 21.5 Conclusion 439
18.5 Ethical issues in Internet-­based
22 Action research 440
surveys 367
22.1 Introduction 440
18.6 Sampling in Internet-­based surveys 372
22.2 Defining action research 441
18.7 Improving response rates in Internet
22.3 Principles and characteristics of action
surveys 372
research 443
18.8 Technological advances 374
22.4 Participatory action research 444
19 Case studies 375 22.5 Action research as critical praxis 445
19.1 What is a case study? 375 22.6 Action research and complexity theory 448
19.2 Types of case study 377 22.7 Procedures for action research 448
19.3 Advantages and disadvantages of case 22.8 Reporting action research 452
study 378 22.9 Reflexivity in action research 453
19.4 Generalization in case study 380 22.10 Ethical issues in action research 454
19.5 Reliability and validity in case studies 381 22.11 Some practical and theoretical matters 454
19.6 Planning a case study 382 22.12 Conclusion 456
19.7 Case study design and methodology 384
23 Virtual worlds, social network software
19.8 Sampling in case studies 386
and netography in educational research 457
19.9 Data in case studies 387
STEWART MARTIN
19.10 Writing up a case study 388
19.11 What makes a good case study 23.1 Introduction 457
researcher? 389 23.2 Key features of virtual worlds 457
19.12 Conclusion 390 23.3 Social network software 458
23.4 Using virtual worlds and social media in
20 Experiments 391 educational research 458
20.1 Introduction 391 23.5 Netography, virtual worlds and social
20.2 Randomized controlled trials 391 media network software 459
20.3 Designs in educational experiments 401 23.6 Opportunities for research with virtual
20.4 True experimental designs 402 worlds, social network software and
20.5 Quasi-­experimental designs 406 netography 461
20.6 Single-­case ABAB design 408 23.7 Ethics 463
20.7 Procedures in conducting experimental 23.8 Guidelines for practice 464
research 409 23.9 Data 465
20.8 Threats to internal and external validity in 23.10 Conclusion 467
experiments 411
20.9 The timing of the pre-­test and the post-­
test 412 PART 4
20.10 The design experiment 413
Methods of data collection 469
20.11 Internet-­based experiments 415
20.12 Ex post facto research 418 24 Questionnaires 471
20.13 Conclusion 425 24.1 Introduction 471
24.2 Ethical issues 471
24.3 Planning the questionnaire 472

x
 contents

24.4 Types of questionnaire items 475 26.12 Reliability and validity in

24.5 Asking sensitive questions 489 observations 560
24.6 Avoiding pitfalls in question writing 490 26.13 Conclusion 562
24.7 Sequencing questions 492
24.8 Questionnaires containing few verbal 27 Tests 563
items 493 27.1 Introduction 563
24.9 The layout of the questionnaire 493 27.2 What are we testing? 563
24.10 Covering letters/sheets and follow-­up 27.3 Parametric and non-­parametric tests 565
letters 495 27.4 Diagnostic tests 565
24.11 Piloting the questionnaire 496 27.5 Norm-­referenced, criterion-­referenced and
24.12 Practical considerations in questionnaire domain-­referenced tests 565
design 498 27.6 Commercially produced tests and
24.13 Administering questionnaires 501 researcher-­produced tests 567
24.14 Processing questionnaire data 504 27.7 Constructing and validating a test 568
27.8 Software for preparation of a test 583
25 Interviews 506 27.9 Devising a pre-­test and post-­test 583
25.1 Introduction 506 27.10 Ethical issues in testing 584
25.2 Conceptions of the interview 507 27.11 Computerized adaptive testing 585
25.3 Purposes of the interview 508
25.4 Types of interview 508 28 Using secondary data in educational
25.5 Planning and conducting interviews 512 research 586
25.6 Group interviewing 527 28.1 Introduction 586
25.7 Interviewing children 528 28.2 Advantages of using secondary data 587
25.8 Interviewing minority and marginalized 28.3 Challenges in using secondary data 588
people 531 28.4 Ethical issues in using secondary
25.9 Focus groups 532 data 589
25.10 Non-­directive, focused, problem-­centred 28.5 Examples of secondary data analysis 589
and in-­depth interviews 533 28.6 Working with secondary data 589
25.11 Telephone interviewing 535 28.7 Conclusion 592
25.12 Online interviewing 538 29 Personal constructs 593
25.13 Ethical issues in interviewing 540
RICHARD BELL

26 Observation 542 29.1 Introduction 593

26.1 Introduction 542 29.2 Strengths of repertory grid technique 594
26.2 Structured observation 545 29.3 Working with personal constructs 595
26.3 The need to practise structured 29.4 Grid analysis 599
observation 550 29.5 Some examples of the use of the repertory
26.4 Analysing data from structured grid in educational research 600
observations 550 29.6 Competing demands in the use of the
26.5 Critical incidents 551 repertory grid technique in research 604
26.6 Naturalistic and participant 29.7 Resources 605
observation 551
26.7 Data analysis for unstructured observations 30 Role-­play and research 606
and videos 555 CARMEL O’SULLIVAN

26.8 Natural and artificial settings for 30.1 Introduction 606

observation 555 30.2 Role-­play pedagogy 607
26.9 Video observations 556 30.3 What is role-­play? 608
26.10 Timing and causality with observational 30.4 Why use role-­play in research? 610
data 558 30.5 Issues to be aware of when using role-­
26.11 Ethical considerations in play 612
observations 558 30.6 Role-­play as a research method 616

xi
contents

30.7 Role-­play as a research method: special 35.2 A conversational analysis 688

features 616 35.3 Narrative analysis 694
30.8 A note of caution 617 35.4 Autobiography 698
30.9 How does role-­play work? 617 35.5 Conclusion 700
30.10 Strategies for successful role-­play 618
36 Analysing visual media 702
30.11 Examples of research using role-­play 623
36.1 Introduction 702
30.12 A note on simulations 626
36.2 Content analysis 704
31 Visual media in educational research 628 36.3 Discourse analysis 705
31.1 Introduction 628 36.4 Grounded theory 706
31.2 Who provides the images? 630 36.5 Interpreting images 707
31.3 Photo-­elicitation 630 36.6 Interpreting an image: a worked
31.4 Video and moving images 633 example 708
31.5 Artefacts 634 36.7 Analysing moving images 712
31.6 Ethical practices in visual research 636 36.8 Conclusion 713
37 Grounded theory 714
37.1 Introduction 714
PART 5 37.2 Versions of grounded theory 715
Data analysis and reporting 641 37.3 Stages in generating a grounded theory 717
37.4 The tools of grounded theory 717
32 Approaches to qualitative data analysis 643 37.5 The strength of the grounded theory 721
32.1 Elements of qualitative data analysis 643 37.6 Evaluating grounded theory 721
32.2 Data analysis, thick description and 37.7 Preparing to work in grounded theory 722
reflexivity 647 37.8 Some concerns about grounded theory 722
32.3 Ethics in qualitative data analysis 650
32.4 Computer assisted qualitative data analysis 38 Approaches to quantitative data analysis 725
(CAQDAS) 650 38.1 Introduction 725
38.2 Scales of data 725
33 Organizing and presenting qualitative 38.3 Parametric and non-­parametric data 727
data 657 38.4 Descriptive and inferential statistics 727
33.1 Tabulating data 657 38.5 Kinds of variables 728
33.2 Ten ways of organizing and presenting 38.6 Hypotheses 730
data analysis 661 38.7 One-­tailed and two-­tailed tests 732
33.3 Narrative and biographical approaches to 38.8 Confidence intervals 733
data analysis 664 38.9 Distributions 733
33.4 Systematic approaches to data analysis 665 38.10 Conclusion 737
33.5 Methodological tools for analysing
39 Statistical significance, effect size and
qualitative data 666
statistical power 739
34 Coding and content analysis 668 39.1 Introduction 739
34.1 Introduction 668 39.2 Statistical significance 739
34.2 Coding 668 39.3 Concerns about statistical significance 742
34.3 Concerns about coding 673 39.4 Hypothesis testing and null hypothesis
34.4 What is content analysis? 674 significance testing 744
34.5 How does content analysis work? 675 39.5 Effect size 745
34.6 A worked example of content analysis 680 39.6 Statistical power 749
34.7 Reliability in content analysis 684 39.7 Conclusion 752
35 Discourses: conversations, narratives and 40 Descriptive statistics 753
autobiographies as texts 686 40.1 Missing data 753
35.1 Discourse analysis and critical discourse 40.2 Frequencies, percentages and
analysis 686 crosstabulations 754

xii
 contents

40.3 Measures of central tendency and 43.2 What to look for in factor analysis
dispersal 762 output 826
40.4 Taking stock 765 43.3 Cluster analysis 828
40.5 Correlations and measures of 43.4 A note on structural equation
association 765 modelling 833
40.6 Partial correlations 772 43.5 A note on multilevel modelling 836
40.7 Reliability 774
44 Choosing a statistical test 839
41 Inferential statistics: difference tests 776 44.1 Introduction 839
41.1 Measures of difference between 44.2 Sampling issues 839
groups 776 44.3 The types of data used 841
41.2 The t-­test 777 44.4 Choosing the right statistic 841
41.3 Analysis of Variance 781 44.5 Assumptions of tests 841
41.4 The chi-­square test 789
45 Beyond mixed methods: using Qualitative
41.5 Degrees of freedom 792
41.6 The Mann-­Whitney and Wilcoxon Comparative Analysis (QCA) to integrate
tests 794 cross-­case and within-­case analyses 847
41.7 The Kruskal-­Wallis and Friedman BARRY COOPER and JUDITH GLAESSER

tests 797 45.1 Introduction 847

41.8 Conclusion 801 45.2 Starting from a ‘quantitative’ stance 848
45.3 Starting from a ‘qualitative’ stance 850
42 Inferential statistics: regression analysis
45.4 Qualitative Comparative Analysis
and standardization 802
(QCA) 850
42.1 Regression analysis 802
45.5 QCA: sufficiency 852
42.2 Simple linear regression 803
45.6 Conclusion 853
42.3 Multiple regression 805
42.4 Standardized scores 814 Bibliography 855
42.5 Conclusion 817 Index 907
43 Factor analysis, cluster analysis and
structural equation modelling 818
43.1 Conducting factor analysis 818

xiii