Punch-3777-Ch-02:Punch-3777-Ch-02.

qxp 11/18/2008 3:13 PM Page 14

14 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

2 Theory and Method in

Education Research

2.1 Methodological theory

2.2 Substantive theory
2.3 Description versus explanation
2.4 Theory verification − theory generation
2.5 Question method − connections
2.6 Prespecified versus unfolding: structure in research questions,
design and data
Chapter summary
Further reading
Exercises and study questions

14
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 15

THEORY AND METHOD IN EDUCATION RESEARCH 15

LEARNING OBJECTIVES
After studying this chapter you should be able to:

● Describe what is meant by methodological theory and substantive theory

● Define paradigms, and describe the difference between paradigm-driven

research and pragmatic research

● Understand the difference between description and explanation

● Describe the difference between theory verification and theory generation

research

● Explain the logical priority of research questions over research methods

● Describe the essential differences between prespecified and unfolding research

The term ‘theory’ is used in many different ways in the literature, which can
create difficulty for the beginning researcher. In this chapter, I focus on two
main uses of theory – methodological theory and substantive theory. Both are
important, and both are discussed below, methodological theory in section 2.1
and substantive theory in section 2.2. Both lead on to the other sections in this
chapter. Thus methodological theory leads to question-method connections.
Substantive theory leads first to description and explanation, and then to the-
ory verification and theory generation. The final section of the chapter deals
with the issue of structure in planning a piece of research. As noted in Chapter 1,
these three themes – description–explanation, question-method connections,
and prespecified–unfolding – run through other chapters in the book.

2.1 METHODOLOGICAL THEORY

Methodological theory, as used here, means theory about method. Whereas
substantive theory is about substance or content, methodological theory is
about method – about what lies behind the approaches and methods of inquiry
that might be used in a piece of research.
Methods of inquiry are based on assumptions – assumptions about the
nature of the reality being studied, assumptions about what constitutes knowl-
edge of that reality, and assumptions about what therefore are appropriate
ways (or methods) of building knowledge of that reality. Very often these
assumptions are implicit. A point of contention in research methods training
has been whether or not such assumptions should be made explicit in a piece
of postgraduate research.
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 16

16 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

These assumptions constitute the essential idea of what is meant by the term
‘paradigm’ in the research methodology and philosophy of science literature. In
general paradigm means a set of assumptions about the world, and about what
constitute proper techniques and topics for inquiring into that world. Put simply,
it is a way of looking at the world. It means a view of how science should be
done, and is a broad term encompassing elements of epistemology, theory and
philosophy, along with methods. Some writers use the term metatheory similarly,
to describe ideas about conceptions of science: ‘Different thinkers, especially
philosophers (of science) have suggested different ideas of what a scientist should
and can do. Such thoughts about what is scientifically possible and what is not,
are called metatheories’ (Higgs, 1995: 3). But paradigm (and inquiry paradigm)
is the more common term, and will be used here.
More formally, Denzin and Lincoln (1994: 107–9) describe a paradigm as:

... a setof basic beliefs (or metaphysics) that deals with ultimates or first
principles. It represents a worldview that defines, for its holder, the nature of
the world, the individuals place in it, and the range of possible relationships
to that world and its parts ...

They point out that inquiry paradigms define what they are concerned with, and
what falls within and outside the limits of legitimate inquiry, and that inquiry
paradigms address three fundamental questions, which reflect the assumptions
noted above:

1 The ontological question: What is the form and nature of reality and, there-
fore, what is there that can be known about it?
2 The epistemological question: What is the relationship between the knower
and what can be known?
3 The methodological question: How can the inquirer go about finding out
what can be known?

In simpler language, paradigms tell us:

• what reality is like (ontology),

• what the relationship is between the researcher and that reality (epistemology),
and
• what methods can be used for studying the reality (methodology).

These three interrelated questions illustrate the connections between methods

and the deeper underlying philosophical issues. Methods are ultimately based
on, and derive from, paradigms. Conversely, paradigms have implications for
methods. This point became clear during methodological developments of the
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 17

THEORY AND METHOD IN EDUCATION RESEARCH 17

past 40–50 years. At this point, therefore, a brief sketch of some historical
background on methods and paradigms in education research is necessary.
Beginning in the 1960s, the traditional dominance of quantitative methods,
as the way of doing empirical research in education, was challenged. That chal-
lenge accompanied a major growth of interest in using qualitative methods,
and this in turn produced a split in the field, between quantitative and qualita-
tive researchers. A prolonged quantitative-qualitative debate ensued, some-
times described as the ‘paradigm wars’.1
Much of that debate was characterised by either–or thinking. Some thought
that only quantitative approaches should be used in research. Others were just as
emphatic that only qualitative approaches are appropriate. More recently, how-
ever, there have been moves towards a détente, and an increased interest in the
combination of the two approaches (Bryman, 1988, 1992; Hammersley, 1992;
Tashakkori and Teddlie, 2003). While education research was a very prominent
arena for these developments and debates, the same methodological changes
occurred, though unevenly, across most areas of empirical social science research.
The full story of these developments, and debates, is much more complex than
this. I have focused only on one main dimension of it, the quantitative-qualitative
distinction, because these remain the two central methodological approaches in
education research today, and because this distinction is a central organizing prin-
ciple for this book. A major consequence of these developments is that qualitative
research methods have moved much more into the mainstream of education (and
social science) research, compared with their marginalized position of 40 years
ago. A further, more recent development, has been the combination of the two
methods in what is now called ‘mixed-methods research’ (see Chapter 13). As a
result, the field of research methodology for education (and social science) is now
bigger and more complex than it used to be. The context in which research oper-
ates is now also more complex (see Chapter 3).
Because of the connections between methods and paradigms, the history
briefly outlined above also has a deeper level, a level that is not just about the
quantitative-qualitative debate, or about research methods, but about para-
digms themselves. On this deeper level, a major rethinking began some time
ago, and is on-going. It has brought a questioning of all aspects of research (its
purposes, its place and role, its context and conceptualisations of research itself)
as well as the techniques it uses. It has also brought the development of new per-
spectives, and of new approaches to data and to the analysis of data, within
qualitative research especially. Prominent features of that rethinking are the
detailed critique of positivism, and the emergence and articulation of several
different paradigms, as alternatives to positivism. As a result, paradigm issues
are in a state of change and development, and many matters are still contested.
It is the development of qualitative methods which has exposed the many
different paradigm possibilities, and the situation has become very compli-
cated. Thus Denzin and Lincoln (1994: 109) identify four main alternative
inquiry paradigms underlying qualitative research (positivism, post-positivism,
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 18

18 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

critical theory et al., constructivism), but more detailed examples and classifications
of paradigms are given by Guba and Lincoln (1994). Morse (1994: 224–5) has
this classification of paradigms with associated qualitative research strategies:
philosophy – phenomenology; anthropology – ethnography; sociology–symbolic
interactionismgrounded theory; semiotics – ethnomethodology and discourse
analysis. Janesick (1994: 212) has a more detailed list of paradigm-related
qualitative research strategies, noting that it is not meant to include all possi-
bilities: ethnography, life history, oral history, ethnomethodology, case study,
participant observation, field research or field study, naturalistic study, phe-
nomenological study, ecological descriptive study, descriptive study, symbolic
interactionist study, microethnography, interpretive research, action research,
narrative research, historiography and literary criticism. And examples of
metatheories (or paradigms) considered by writers in the philosophy of education
are logical empiricism (and post-empiricism), critical rationalism, critical theory,
phenomenology, hermeneutics and systems theory.
This can be confusing and daunting territory for the beginning researcher,
partly because of philosophy and partly because of terminology. Faced with these
complications, some of the literature seems to be simplifying and converging. In
one version of this convergence, the main paradigm positions are positivism and
interpretivism, in another they are positivism and constructivism. Thus:

• positivism (associated with quantitative methods), and

• either interpretivism or constructivism (associated with qualitative
methods).

These associations – positivism with quantitative methods and interpretivism–constructivism

with qualitative methods – are generally true, but they are not necessary asso-
ciations. It is more accurate to say that positivism is likely to be associated with
quantitative methods, and interpretivism and constructivism are likely to be
associated with qualitative methods.
These terms are defined slightly differently by different writers, but their
main nature-of-reality ideas are as follows:

• Positivism – the belief that objective accounts of the world can be given, and
that the function of science is to develop explanations in the form of univer-
sal laws, that is, to develop nomothetic knowledge.
• Interpretivism – concentrates on the meanings people bring to situations and
behaviour, and which they use to understand their world (O’Donoghue,
2007: 16–17).
• Constructivism – realities are local, specific and constructed; they are socially
and experientially based, and depend on the individuals or groups holding
them (Guba and Lincoln, 1994: 109–11).
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 19

THEORY AND METHOD IN EDUCATION RESEARCH 19

These ideas are nicely illustrated by the story shown in Box 2.1.

The umpire in baseball stands close-up behind the pitcher. His job is to call
BOX 2.1 pitches as ‘strikes’ or ‘ball’. These calls are crucial in shaping the course of a
Three baseball
umpires game, and its outcome. Three baseball umpires were discussing the problem
of subjectivity in their calls. After a long and lively debate. Umpire A
attempted closure, saying:
‘Well, I call ‘em as I see ‘em. You can’t do more than that.’
‘No’ said Umpire B, “I call ‘em as they are – that’s the way to do it.’
‘You’ re both wrong,’ said Umpire C. ‘They ain’t nothing til I call ‘em, then
that’s what they are.’
These views roughly equate to the views from three main paradigms. Thus:
Umpire A in an interpretivist (specifically, a symbolic interactionist – see
section 7.4)
Umpire B is a positivist
Umpire C is a constructivist

In this book, question-method connections are discussed, and it is stressed that

there needs to be compatibility and integrity in the way the research questions
and research methods fit together in a study. This is shown in the top line in the
diagram below. Paradigms expand that, because paradigms have implications
both for the sorts of research questions asked and the methods used to answer
them. This is shown in the bottom line in the diagram.

questions methods

paradigms questions methods

What does all this methodological theory mean for the individual, planning
and executing a piece of research? Broadly, there are two main ways planning a
research project can proceed:

1 Paradigm-driven approach – one way is to begin with a paradigm, articulate

it and develop research questions and methods from it.
2 Pragmatic approach – the other way is to begin with research questions that
need answers and to choose methods for doing so.
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 20

20 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

The questions may come from any source – the literature, existing substantive
theory, the media, personal experience, and so on. But very often, especially in
a professional field such as education, they will come from practical and pro-
fessional issues and problems. The starting point here is not a paradigm.
Instead, the starting point is a problem that needs solution or a question that
needs answers. This is a pragmatic approach.
As noted, this has sometimes been a contentious issue in higher-degree research
programmes. Some university departments have taken the view that paradigm
issues are paramount, and insist that research should not be allowed to proceed
until it has articulated its paradigm position. I believe this insistence is not well
placed, because paradigm-driven research is not the only way to proceed, and
because I see a big role for a more pragmatic, applied and professional approach to
research in education. I have no objection to paradigm-driven research. My objec-
tion is only to the view that all research must be paradigm-driven. I take a similar
view with respect to the philosophical issues involved in paradigm debates. I think
we should be aware of the issues involved, and of the areas of debate. These are
indicated in several places throughout the book. But we can proceed to do
research, and to train researchers, mindful of those debates yet not engulfed by
them, and without necessarily yet being able to see their resolution. In other
words, we can acknowledge the connections of methods to these deeper issues,
and discuss them from time to time as they arise, without making them the major
focus of this book, or of our research. This is to take the pragmatic approach
noted, consistent with the view that not all questions for social research are driven
by paradigm considerations, and that different sorts of questions require different
methods for answering them. Both of these points are elaborated in later chapters.
To choose the pragmatic approach is to start by focusing on what we are trying
to find out in research, and then to fit methods in with that. The important topic
of question-method connections is discussed in section 2.5.

2.2 SUBSTANTIVE THEORY

By substantive theory I mean theory about a substantive issue or phenomenon,
some examples of which are shown below. Substantive theory is content-based
theory, and is not concerned with methods. Its purpose is to explain some phe-
nomenon or issue of interest – it is explanatory theory. But because explanation
requires description (see section 2.3), substantive theory both describes and
explains. An explanatory theory not only describes but also explains the phe-
nomenon of substantive interest. Theory, in this sense, is a set of propositions that
together describe and explain the phenomenon being studied. These propositions
are at a higher level of abstraction than the specific facts and empirical general-
izations (the data) about the phenomenon. They explain the data by deduction, in
the if–then sense. This is the model of scientific knowledge shown in Figure 2.1.
The model also describes the role of the hypothesis in Chapter 4 (section 4.7)
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 21

THEORY AND METHOD IN EDUCATION RESEARCH 21

Figure 2.1 The structure of scientific knowledge (nomothetic view)

Theory Explanatory theory

Data Empirical Empirical Empirical

(level 2) generalization 1 generalization 2 generalization 3

Data
(level 1) Discrete facts Discrete facts Discrete facts

Some examples of substantive theories from different areas of research relevant

to education are attribution theory, reinforcement theory, various learning theories
and personal construct theory (from psychology); reference group theory and
social stratification theory (from sociology); the theory of vocational personalities
and career anchors (from occupational sociology); various leadership theories
(from management and administration), and theories of children’s moral develop-
ment and of teacher career cycles.
Thus an important question in developing and planning research is ‘What is
the role of (substantive) theory in this study?’ This question is sometimes con-
sidered more appropriate for doctoral level research than for masters level
research. This seems to be because a common criterion among universities for
the award of the doctorate centres on the ‘substantial and original contribu-
tion to knowledge’ a study makes, and the ‘substantial’ part of that criterion is
often interpreted in terms of its contribution to substantive theory.

2.3 DESCRIPTION VERSUS EXPLANATION

In Chapter 1 a brief description of the scientific method was given, stressing
that it has the two central parts of data and theory, and that the objective of
scientific inquiry is to build explanatory theory about its data. In this view, the
aim is to explain the data, not just to use the data for description. This distinc-
tion between description and explanation is particularly relevant to the pur-
poses of a piece of research.
The description–explanation distinction is easy to understand on one level, and
difficult to understand on another.2 Fortunately, it is on the easier level where the
practical value of the distinction lies. Description and explanation represent two
different levels of understanding. To describe is to somehow draw a picture of
what happened, or of how things are proceeding, or of what a situation or person
or event is like. To explain, on the other hand, is to account for what happened,
or for how things are proceeding, or for what something or someone is like. It
involves finding the reasons for things (or events or situations), showing why and
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 22

22 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

how they have come to be what they are. Description is a more restricted purpose
than explanation. We can describe without explaining, but we can not really
explain without describing. Therefore explanation goes further than description.
It is more than just description – it is description plus something else.
Description focuses on what is the case, whereas explanation focuses on
why something is the case. Science as a method of building knowledge has, in
general, pursued the objective of explanation, not just of description. There is
a good reason for this. When we know why something happens, we know
much more than just what happens. It puts us in a position to predict what will
happen, and perhaps to be able to control what will happen.
Thus explanatory knowledge is more powerful than descriptive knowledge.
But descriptive knowledge is still important, since explanation requires descrip-
tion. To put it around the other way, description is a first step towards explana-
tion. If we want to know why something happens, it is important to have a
good description of exactly what happens. There are often clues to explanation
in a full description, and it is hard to explain something satisfactorily until you
understand just what the something is (Miles and Huberman, 1994: 91).
This distinction comes up mainly when the purpose of a piece of research is
being considered. Is the purpose to describe, to explain or both? Descriptive
studies are sometimes given a lower status than studies that aim to explain.
That is why we sometimes hear the expression ‘it is only a descriptive study’.
But while this judgement may sometimes have merit, it has to be made care-
fully. There are situations where a thorough descriptive study will be very valu-
able. Two examples of such situations are:

• when a new area for research is being developed, and exploratory studies are
planned – it is very sensible then to focus on systematic description as the
objective of the research;
• careful description of complex social processes can help us to understand
what factors to concentrate on for later explanatory studies.

Whether description or explanation is the appropriate purpose for a piece of

research depends on the particular situation. Here, as elsewhere, blanket rules
are not appropriate. Rather, each research situation needs to be analysed and
understood in its own context. It is useful to raise this question of whether the
objective of a study is description and/or explanation, especially during the
planning stages of research. A good way to do it is to ask ‘why’ about the
things being studied, as well as ‘what’.
Thus explanation is the central focus of substantive theory. The essential
idea is to explain whatever is being studied, with the explanation being
couched in more abstract terms than the terms used to describe it.3 We will
return to this idea of theory in two places later in the book. The first is in
Chapter 4 (section 4.7), where we consider the role of hypotheses in relation to
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 23

THEORY AND METHOD IN EDUCATION RESEARCH 23

research questions. There we will see that theory stands behind the hypothesis,
in an inductive–deductive relationship with it (Nagel, 1961; Brodbeck, 1968).
Studies that use this approach are theory verification studies. The second is in
Chapter 9, where we discuss grounded theory analysis in studies that aim to
develop theory. These are theory generation studies.

2.4 THEORY VERIFICATION − THEORY

GENERATION
This distinction between theory verification and theory generation research is
important. A project that has explanation as its objective can set out to test
theory, or to build theory – to verify theory, or to generate it. For Wolcott
(1992), this is the distinction between ‘theory first’ and ‘theory after’. In the-
ory-first research, we start with a theory, deduce hypotheses from it and design
a study to test these hypotheses. This is theory verification. In theory-after
research, we do not start with a theory. Instead, the aim is to end up with a the-
ory, developed systematically from the data we have collected. This is theory
generation.
Quantitative research has typically been more directed at theory verification,
while qualitative research has typically been more concerned with theory gener-
ation. While that correlation is historically valid, there is no necessary connec-
tion between purpose and approach. That is, quantitative research can be used
for theory generation (as well as for verification) and qualitative research can be
used for theory verification (as well as for generation), as pointed out by vari-
ous writers (for example, Hammersley, 1992; Brewer and Hunter, 1989).
However, while the connection is not necessary, it is nonetheless likely that the-
ory generation research will more often be qualitative. Research directed at the-
ory generation is more likely when a new area is being studied, and exploration
of that new area is more likely to use the less structured fieldwork techniques of
qualitative research.
Is theory verification research better than theory generation research? This
book does not favour one research purpose over the other, since both are
needed and both have their place. Either purpose can be appropriate in a
research project, and sometimes both will be appropriate. It depends on the
topic, the context and practical circumstances of the research, and especially
on how much prior theorizing and knowledge exists in the area. As with other
aspects of a project, the researcher needs to consider the alternatives, select
among them according to sensible criteria, and then articulate that position.
Theory generation research was given new legitimacy in social science by the
development of grounded theory. Grounded theory is an explicit theory genera-
tion research strategy, developed in reaction against the overemphasis on theory
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 24

24 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

verification research in the American sociology of that time. Glaser and Strauss
stated this clearly in their original grounded theory publication:

Verification is the keynote of current sociology. Some three decades ago, it was
felt that we had plenty of theories but few confirmations of them a position made
very feasible by the greatly increased sophistication of quantitative methods. As
this shift in emphasis took hold, the discovery of new theories became slighted
and, at some universities, virtually neglected. (Glaser and Strauss, 1967: 10)

Glaser and Strauss argued that the emphasis on verification of existing theories
kept researchers from investigating new problem areas, prevented them from
acknowledging the necessarily exploratory nature of much of their work,
encouraged instead the inappropriate use of verificational logic and rhetoric,
and discouraged the development and use of systematic empirical procedures
for generating as well as testing theories (Brewer and Hunter, 1989).
This gives us a useful general guideline for when each purpose might be
appropriate. When an area has lots of unverified theories, an emphasis on the-
ory verification research seems a good thing. On the other hand, when an area
is lacking in appropriate theories, it is time for the emphasis to shift to theory
generation. Also, when research is directed mostly at the verification of exist-
ing theories, looking at new problem areas is discouraged, and the logic and
techniques (usually quantitative) of verification research are seen as more
important. When it is important to look at new areas in research, theory gen-
eration appeals as the appropriate purpose. This aspect of grounded theory
research is taken up again in Chapter 7 (section 7.5).
The description–explanation distinction fits in with the structure of scientific
knowledge shown in Figure 2.1. In line with the conception of science given in
Chapter 1, we can distinguish three levels of knowledge. At the lowest level,
there are discrete facts. At the next level there are empirical generalizations
which group those facts together. At the highest level are theories, whose func-
tion is to explain the generalizations. This structure is summarized in the dia-
gram shown. The first two levels (facts and empirical generalization) focus on
description, while the third level focuses on explanation.
This model of the structure of scientific knowledge comes primarily from a
positivistic perspective, and stresses a nomothetic view of knowledge. It can be
contrasted with an ideographic view of knowledge, a more appropriate aspira-
tion for education research in the eyes of many qualitative researchers.4 But
while acknowledging its nomothetic bias, this model is very useful as a starting
point in learning about social science research. Much research is based on this
model, and it can often help in organizing an individual project. It is clear and
easy to understand, so the researcher who wishes to diverge from this model
can see where and why the divergence occurs. In other words, when
researchers argue about how research should proceed and contribute to
knowledge, this model helps to see what the argument is about.
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:13 PM Page 25

THEORY AND METHOD IN EDUCATION RESEARCH 25

There is another reason for stressing this model here. It shows the hierarchical
structure of knowledge, with higher levels of abstraction and generality at the top
and lower levels at the bottom. This is similar to the hierarchical structure that
links data indicators to variables and concepts, and which is central both to the
concept–indicator model behind grounded theory coding in qualitative research,
and to latent trait measurement theory in quantitative research. These topics are
described in Chapters 9 and 11 respectively. This hierarchical structure of increas-
ing levels of abstraction and generality, shown here with respect to scientific
knowledge in general, and shown in later chapters with respect to concept–data
links in both quantitative and qualitative research, is thus fundamental to much
empirical research. An example of it is indicated in Example 2.1.

EXAMPLE 2.1 THE HIERARCHICAL STRUCTURE

OF KNOWLEDGE

A well-known example of this way of structuring Knowledge is

Durkheim’s work on the social aetiology of suicide, described in Durkhelm
(1951) and summarized in Greenwood (1968). Durkheim (1951) and sum-
marizes in Greenwood (1968) Durkheime theorizes ‘upwards’ from a series
of empirical generalizations to law of sucicide.5

2.5 QUESTION METHOD − CONNECTIONS

The principle here is that the matching or fit between the research questions
and research methods should be as close as possible. A very good way to do
that is for methods to follow from questions.
Different questions require different methods to answer them. The way a ques-
tion is asked has implications for what needs to be done, in research, to answer it.
Quantitative questions require quantitative methods to answer them, and quali-
tative questions require qualitative methods to answer them. In today’s research
environment, with quantitative and qualitative methods often alongside each
other, the matching of questions and methods is even more important. Since this
book deals directly with both approaches, it is inevitable that this issue should be
a recurrent concern. The wording of questions is also important, since some
wordings carry methodological implications. Thus questions that include ‘vari-
ables’, ‘factors that affect’ and ‘the determinants or correlates of’, for example,
imply a quantitative approach, while questions that include ‘discover’, ‘seek to
understand’ ‘explore a process’ and ‘describe the experiences’ imply a qualitative
approach. Creswell (1994: 71) links these last four terms to grounded theory,
ethnography, case study and phenomenology respectively.
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 26

26 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

An example of different research questions and their implications for meth-

ods is given by Shulman (1988: 6–9). He takes the study of reading, suggests
four different types of questions, and shows the methods that would be
required to answer each.

1 A first question might be: What makes some people successful readers and other
unsuccessful? (Or, how can we predict what sorts of people will have difficulty
learning to read?) Such questions would be answered using a quantitative cor-
relational study that examined relationships between variables.
2 A second question might be: What are the best possible methods for teaching
reading to youngsters, irrespective of their backgrounds or attitudes? This
question would involve a quantitative experimental study comparing different
teaching methods.
3 A third question might be: What is the general level of reading performance
across different age, sex, social or ethnic groups in the population? This would
require a quantitative survey of reading performance and reading practices.
4 A fourth set of questions might be quite different from the previous ones:
How is reading instruction carried on? What are the experiences and percep-
tions of teachers and students as they engage in the teaching and learning of
reading? How is this complex activity accomplished? Here, a qualitative case
study involving observation and interview might be used, perhaps using the
perspective of ethnomethodology.

Shulman goes on to suggest philosophical and historical questions as well.

Other examples of question-method corrections are indicated in Example 2.2.

EXAMPLE 2.2 QUESTION METHOD − CONNECTIONS

• Shulman (1988: 6–9) shows connections between questions and meth-

ods with the topic of reading research in education; similar examples
are notes by Seidman (1991: 5)
• Marshall and Rossman (1989: 78) show, in a table, the links between
research purposes, research questions, research strategy and data col-
lection techniques.
• Maxwell (1996: 81–5) adapts a table from LeCompte and Preissle
(1993) to show the links between ‘What do I need to know?’ and ‘What
kind of data will answer the questions?’ and illustrates these links with
actual research questions.
• Maxwell (1996: 80) gives the example of a mismatch between question
and method, whereby, in a study of how historians work, the ‘right
answer’ is found to be to the ‘wrong question’.
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 27

THEORY AND METHOD IN EDUCATION RESEARCH 27

A good way to achieve a fit between questions and methods is to ensure that
the methods we use follow from the questions we seek to answer. In other
words, the content of the research (the research questions) has a logical prior-
ity over the method of the research. To say that content precedes method is
simply to say that we first need to establish what we are trying to find out, and
then consider how we are going to do it. On a practical level, this is often a
good way to get a research project off the ground. Sometimes it is difficult to
know where and how to start, in planning research. If so, asking ‘What are we
trying to find out?’ usually gets our thinking going, and ensures that we start
with the content, not with the method. Putting questions before methods is
also a good defence against overload when developing a research proposal. To
delay consideration of methods until it is clear what the questions are helps in
managing the inevitable complications that accompany a full examination of
the possibilities for research in any area. It helps in keeping the question devel-
opment stage systematic, and under control. It also helps achieve good ques-
tion-method fit, a central criterion in the validity of research.
I am stressing this point here as a counter to a previous unfortunate ten-
dency in education and social science research. In Chapter 1, the term
‘methodolatry’ was used:

I use the term methodolatry, a combination of method and idolatry, to describe a

preoccupation with selecting and defending methods to the exclusion of the
actual substance of the story being told. Methodolatry is the slavish attachment
and devotion to method that so often overtakes the discourse in the education
and human service fields. (Janesick, 1994: 215)

Methodolatry thus means putting method before content. It is first learning the
research method, then finding research questions that can fit into the method.
It is looking for research questions guided by methods.
This is a danger when we place too much stress on the teaching of research
methods, for their own sake. Because of this danger, this book concentrates on
the logic and rationale behind empirical research and its methods. Once this
logic is mastered, we can focus on research questions, and then fit the tech-
niques and methods to the questions. In my opinion, the best sequence of
learning activities for research is to start by learning the logic of research, then
to focus on identifying and developing the research questions, and then to fit
methods and techniques to the questions.
I am using the concept of methodolatry to argue for minimizing the direct
influence of methods on research questions, which we can do by first getting
the research questions clear, and then focusing on the methods required to
answer them. But methods can also indirectly influence research questions, by
constraining what can be studied. There are limits as to what can be designed
in research, and to what data can be obtained and analyzed. While taking this
into account, the advice is nonetheless to focus on questions first, as much as
possible. In the above example, after showing how different methodological
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 28

28 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

approaches fit different questions, Shulman emphasizes the same point:’ we are
advised to focus first on our problem and its characteristics before we rush to
select the appropriate method’ (1988: 15). Thus, when misfit between the
parts becomes apparent during the planning of the research, it is a matter of
adapting the parts to each other. Since I see methods primarily as tools, for use
in answering research questions, I think it is better to adapt design and meth-
ods to questions, rather than to adapt the questions to the design and methods.
Question-method fit is an aspect of conceptual clarity in a piece of research.
Conceptual clarity involves the precise and consistent use of terms, internal
consistency within an argument and logical links between concepts, especially
across different levels of abstraction. The pre-empirical question development
work described in Chapter 4 is directed at this conceptual clarity. Developing
specific research questions is a good way of achieving clarity and matching
questions and methods.
The different paradigms and strategies within qualitative research open up
many new and different types of research questions. Thus ethnographic ques-
tions might focus on cultural and symbolic aspects of behaviour; grounded the-
ory questions might focus on understanding social processes, and how people
manage different types of situations; a conversation analysis study might focus
on conversational structure and on the role of conversation in taken-for-granted
everyday activities; discourse analysis questions might focus on the way an
institution presents itself to the world, the symbols and language it uses, and the
connection of those with its ideology, knowledge, power and so on. Paradigm
can thus be important in generating research questions. Within qualitative
research especially, the range of questions of interest is now very broad. But it
remains important, even with this broader range of questions, that the methods
we use should follow from and fit in with the questions we seek to answer.

2.6 PRESPECIFIED VERSUS UNFOLDING:

STRUCTURE IN RESEARCH QUESTIONS,
DESIGN AND DATA
How much the research questions, design and data should be preplanned in a
piece of research, and how much they should emerge as the research develops,
is another recurring theme in this book.
There is a continuum we can set up for thinking about research questions
and methods, with the dimension of interest being the amount of prespecified
structure in the research strategy that is used. The central comparison is
between research that is prespecified (or preplanned, or prefigured, or prede-
termined) on the one hand, and research that is unfolding (or emerging, or
open-ended) on the other. Prespecified here refers to how much structure is
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 29

THEORY AND METHOD IN EDUCATION RESEARCH 29

Figure 2.2 Prespecified versus unfolding: the timing of structure

Prespecified research questions • • General guiding questions

Tightly structured design • • Loosely structured design
Prestructured data • • Data not prestructured

Qualitative
research

Quantitative
research

introduced ahead of the empirical work, as opposed to during the empirical

work. This continuum applies to three main areas – to research questions, to
research design and to data.
Miles and Huberman (1994: 16) discuss this idea in the context of qualita-
tive research under the heading of ‘tight versus loose’. Those terms are equiva-
lent to the terms used here – tight means prespecified and loose means
unfolding. The key questions are: To what extent are the research questions,
the design and the data focused, specified and structured ahead of the actual
empirical work? To what extent does the focus in the research questions, and
the structure in the design and the data, unfold and emerge as the empirical
work proceeds? The continuum of possibilities is shown in Figure 2.2. This
diagram shows that quantitative research typically falls towards the left hand
end of the continuum, whereas qualitative research typically occupies a much
greater range along the continuum.
The word structure, as used here, means showing what the different parts of
the research are, how they connect with each other, what will be done in the
research, and in what sequence. It means knowing what we are looking for,
and how we are going to get it – knowing what data we will want, and how
they will be collected. It also means knowing what structure the data will have
and how they will be analysed.
At the extreme left hand end of the continuum, everything is prespecified – the
research questions, the design and the data. It is all worked out in advance, a set of
steps is laid down, and the researcher proceeds through those steps. At the other
end, we can envisage a project where very little structure is determined in advance,
with a relatively open-ended and unstructured approach to the research questions,
the design and the data. The strategy is that these will unfold as the study is carried
out. Let us see what this contrast means for each of the three areas.

1 Research questions: at the left-hand end of the continuum, specific research

questions are set up in advance to guide the study. It is quite clear, in
advance, exactly what questions the study is trying to answer. At the right-
hand end, only general questions are set up in advance. The argument is
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 30

30 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

that, until some empirical work is carried out, it is not possible (or, if possi-
ble, not sensible) to identify the specific research questions. They will only
become clear as the research unfolds, and as a specific focus for the work is
developed. Wolcott (1982) describes this contrast as ‘looking for answers’
versus ‘looking for questions’. As we will see in Chapter 5, there is often a
close connection between the research questions and the conceptual frame-
work in a study. The issue described here in terms of research questions
applies to conceptual frameworks as well – they can be developed and spec-
ified in advance of the research, or they can emerge as the research proceeds.
The more tightly developed and prespecified the research questions are, the
more likely it is that there will be a well-developed conceptual framework as
well.
2 Design: at the left-hand end, the design is tightly structured. The clearest
examples come from quantitative research – experimental studies and non-
experimental quantitative studies with carefully developed conceptual
frameworks. Research questions, design and conceptual framework come
together here, since a tightly structured design requires that variables be
identified, and that their conceptual status in the research be made clear. At
the right-hand end, the design is indicated in general terms only (for exam-
ple, as in an unfolding case study, or an ethnography). Like the research
questions, it will take detailed shape as the research progresses, and as the
specific focus for the study is developed.
3 Data: at the left-hand end, data are structured in advance. A very clear
example is quantitative data, where measurement is used to give the data
numerical structure. Using numbers is the most common way of structuring
data in advance, but there are other ways as well. Whether it is numerical or
other categories, the point is that those categories are pre-established, or set
up a priori. At the right-hand end, the data are unstructured at the point of
collection. No pre-established categories or codes are used. The structure of
the data, the categories and codes, emerge from the data, during the analysis –
they are developed a posteriori. Thus the comparison is between starting
with categories for the data, and getting to them during the analysis of the
data – between pre-coding the data and post-coding the data. This point
about data has implications for instrumentation in data collection, not only
in quantitative research, but in qualitative research as well.

The continuum shown in Figure 2.2 can now be described more accurately.
It is really about when in the research process the structure is introduced. The
structure can be introduced in the planning or pre-empirical stage, when the
research is being set up, before data are collected; or, it can be introduced in
the execution stage of the research, as the study is being carried out, as data
are being collected. Either way, structure is needed. A research project will be
difficult both to report and to understand, and will lack credibility as a piece
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 31

THEORY AND METHOD IN EDUCATION RESEARCH 31

of research, without structure in its research questions, its design, especially in

its data, and also in its report. So this contrast is not about having structure or
not having structure, but about when in the research process the structure
occurs. In other words, this continuum is about the timing of structure in the
research – whether that structure is introduced ahead of the empirical research,
or is introduced during and as a result of the empirical research. Another way
of saying this is that the structure might be imposed on the research (and espe-
cially on the data) ahead of the empirical processes; or that the structure might
emerge during the research, in terms of the specific questions on which to
focus, the design to follow and the categories to use in representing the data.
The possibilities along this continuum represent different possible research
styles. As the diagram shows, there is a correlation between these styles, on the
one hand, and the typical quantitative and qualitative research approaches, on
the other. The typical quantitative study is much more likely to have specific
research questions, a clear conceptual framework and design for its variables, and
to use measurement as its way of structuring the data. It is harder to talk about
typical qualitative studies, and they may cover a wider range along the contin-
uum. Many of them fall towards the right-hand end, with general rather than spe-
cific questions set up in advance, with a general design and with data not coded at
the point of collection. This is well captured by Miles and Huberman (1994: 17),
discussing field research as a central part of the qualitative approach:

The conventional image of field research is one that keeps prestructured

designs to a minimum. Many social anthropologists and social phenomenolo-
gists consider social processes to be too complex, too relative, too elusive, or
too exotic to be approached with explicit conceptual frames or standard
instruments. They prefer a more loosely structured, emergent, inductively
'grounded' approach to gathering data: The conceptual framework should
emerge from the field in the course of the study; the important research ques-
tions will become clear only gradually; meaningful settings and actors cannot
be selected prior to fieldwork; instruments, if any, should be derived from the
properties of the setting and its actors views of them.

This general correlation between style and approach also extends to theory
verification versus theory generation research, the distinction discussed in
section 2.4. Theory verification research, by definition, is more likely to have
clear-cut research questions leading to hypotheses, a tightly structured design
and pre-established categories for data. Theory generation research, by con-
trast, will more likely use an approach where specific research questions
unfold as the study develops, and where codes and categories for the data are
empirically derived.
It is not a question of which strategy is best, since a large part of the answer
to this question is ‘it depends’. The question interacts with the overall
approach to the research. Is it a quantitative study, a qualitative study or one
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 32

32 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

that combines the two approaches? If quantitative, it is more likely to be

towards the left-hand end of the continuum in Figure 2.2. If qualitative, there
is likely to be a greater range of possibilities. Nor is it a dichotomous choice
between two extreme positions – it is a continuum. For clarity, the description
in this section has been given in terms of the ends of the continuum. In prac-
tice, there are many points along the continuum, and any study may combine
elements of either strategy, the prespecified one or the unfolding one.
How much predetermined structure is desirable in a project is a matter for
analysis in each particular situation. Structure is necessary. But the timing of the
structure – when is the appropriate point to introduce that structure – depends
on such factors as the topics and goals of the research, the availability of rele-
vant knowledge and theory about the topic, and the researcher’s familiarity
with the situation being studied (Miles and Huberman, 1994). Other factors to
be considered are the preferred style of the research, the resources (including
time) available to the researcher, and to what extent the researcher is interested
in explanation versus interpretation. Depending on these factors, there can be
merit in either approach. As Huberman and Miles (1994) point out, what is
required is a careful analysis of each situation where research is proposed. The
research strategy should then be custom-built, as far as is possible, on the basis
of that analysis.
The discussion in this section has treated research questions, design and data
together. Subsequent chapters deal with questions, design and data separately,
before bringing them back together in Chapters 13, 14 and 15. Without wish-
ing to advise against exploratory unfolding studies, it is worth noting some of
the benefits in having at least a reasonable level of specificity in the research
questions. For example, they give guidance during initial data collection,
thereby saving time and resources and helping to avoid confusion and over-
load, an especially valuable benefit for the beginning researcher. In addition,
research questions that are at least reasonably focused make it easier to com-
municate about the research, which can be important in the presentation of a
research proposal. Brewer and Hunter (1989) point out that, once a study is
published, it is irrelevant whether the research questions initiated the study or
emerged from it – but it can matter at the proposal stage. Finally, it is very
often the case that the researcher does have knowledge about the proposed
research problems, even in a relatively unexplored area (‘experiential data’ and
‘experiential knowledge’ – see Strauss, 1987 and Maxwell, 1996). There is
great benefit in getting that knowledge out on to the table, and working care-
fully to develop research questions in advance of the empirical work is a good
way to do that.
Developing specific research questions to a point where they are stable, and
connecting them to the design, data collection and data analysis parts of the
research, requires careful work. The question being considered here is whether
that work is done in advance of the research or during it. That brings us back
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 33

THEORY AND METHOD IN EDUCATION RESEARCH 33

to fitting the various parts of a project together, as discussed in section 2.5.

This fitting together can be done ahead of the research, or during the research,
but either way it needs to be done. Just as section 2.1 of this chapter stressed
the pragmatic benefits of ‘questions first – methods later’ in maximizing that
fit, so this section stresses the pragmatic benefits of beginning with research
questions that are at least reasonably well developed.
To summarize: There is a continuum of possibilities, which is about prespeci-
fying versus unfolding structure in the research. It applies to research questions,
design and data. The issue is structure and its timing – when in the research is
structure introduced? Prespecified research does it ahead of the empirical proce-
dures. Unfolding research does it during them. As a general rule, at least a rea-
sonable level of specificity in the initial research questions is desirable, though
various factors need to be taken into account in particular situations. Chapter 4
will describe a model of research where considerable effort is invested in devel-
oping research questions ahead of the empirical work. But that is not the only
model, and when research questions come later, they still require both the ana-
lytical development described in Chapters 4 and 5, and the matching with meth-
ods, design and data described in section 2.5 of this chapter. Before dealing with
these topics, we need to consider the context and ethics of education research.

CHAPTER SUMMARY
Methodological theory: theory about method; the paradigms that underlie
research methods
Paradigms: ontology – what reality is like; epistemology – the relationship
between the researcher and reality; methodology – how we can study the reality
Substantive theory: theory about content; focus on explanation – explanatory
theory, theory that explains content
Description: focus on ‘what’
Explanation: focus on ‘why’
Theory verification research: research to test an existing theory; uses hypotheses –
the relationship between theory and hypothesis
Theory generation: research to develop a new theory
Question-method connections: importance of a close fit; questions first – methods
later
Prestructured research: tightly structured research questions, design, data
Unfolding research: general questions, general strategy and design, unstructured data
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 34

34 INTRODUCTION TO RESEARCH METHODS IN EDUCATION

FURTHER READING
Anfara and Mertz. (2006) Theoretical Frameworks in Qualitative Research. Thousand
Oaks, CA: Sage.

Babbie, E. (1992) The Practice of Social Research. 6th edn. Belmont, CA: Wadsworth.

Bailey, K.D. (1987) Methods of Social Research. 3rd edn. New York: Free Press.

Creswell, J.W. (1994) Research Design: Qualitative and Quantitative Approaches.

Thousand Oaks, CA: Sage.

Lewins, F. (1992) Social Science Methodology. Melbourne: Macmillan.

Little, D. (1991) Varieties of Social Explanation: An Introduction to the Philosophy of

Social Science. Boulder, CO: Westview.

Marshall, C. and Rossman, G.B. (1989). Designing Qualitative Research. Newbury

Park, CA: Sage.

Maxwell, J.A. (1996) Qualitative Research Design: An Interactive Approach.

Thousand Oaks, CA: Sage.

Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis. Thousand Oaks,
CA: Sage.

Neuman, W.L. (1994) Social Research Methods: Qualitative and Quantitative

Approaches. Boston: Allyn and Bacon.

Wolcott, H.F. (1994) Transforming Qualitative Data: Description, Analysis, and

Interpretation. Thousand Oaks, CA: Sage.

EXERCISES AND STUDY QUESTIONS

1 What is a paradigm? What are the three main dimesions of paradigms?
2 What were the ‘paradigm war’?
3 How are paradigms and methods connected?
4 What is a paradigm driven approach to research? What is a pragmatic
approach to research? How do they differ?
5 What would a description of the climate of (say) a London winter look
like? What would an explanation of that climate look like? How are they
different?
Punch-3777-Ch-02:Punch-3777-Ch-02.qxp 11/18/2008 3:14 PM Page 35

THEORY AND METHOD IN EDUCATION RESEARCH 35

6 For what sorts of topics and research questions would prestructured

research be appropriate?
7 For what sorts of topics and research questions would unfolding research
be appropriate?

NOTES
1 The ‘paradigm wars’ were especially vigorous in the field of educational research. A
good record of those ‘wars’, including the moves towards reconciliation and
detente, can be found in a series of articles in The Educational Researcher, begin-
ning in the 1970s).
2 The ‘difficult’ level is about precise definitions of the two terms, and about philo-
sophical investigations into the concept of explanation – see, for example, Little
(1991), Lewins (1992).
3 Explanation itself is a complex philosophical concept. Another form of it is the
‘missing links’ form. Here, an event, or empirical generalization, is explained by
showing the links that bring it about. Thus the relationship between social class and
scholastic achievement might be explained by using cultural capital (Bourdieu,
1973) as the link between them. Or the relationship between social class and self-
esteem might be explained by using the parent-child relationship as the link between
them (Rosenberg, 1968: 54–82).
4 A nomothetic view sees generalized knowledge, universal laws and deductive expla-
nations, based mainly on probabilities derived from large samples, and standing
outside the constraints of everyday life. An ideographic view sees nomothetic
knowledge as insensitive to local, case-based meanings, and directs attention rather
to the specifics of particular cases. It prefers to see knowledge as local and situated
(Denzin and Lincoln, 1994: 99–104). The ideographic view thus points towards
understanding and interpretation as important goals of research, alongside descrip-
tion and explanation.
5 Note also Atkinson’s (1978) critique of that work, focusing on how suicide rates are
constructed and what they mean.