Ref - Step-By-Step Guide To Critiquing Research - Part 2
Ref - Step-By-Step Guide To Critiquing Research - Part 2
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
RESEARCH METHODOLOGIES
regarding the inclusion of a literature review before the data theory or conceptual framework. The literature review is
collection period. Similarly, there is debate over whether the therefore carried out after the data have been collected. The
conceptual framework should precede the period of data aim of this approach is to explore concepts embedded in the
collection and data analysis. data, thereby allowing theory to be generated from the data
When critiquing qualitative studies, the reviewer must rather than vice versa (Robinson, 2002).
decide whether the researcher has rationalized his/her chosen Similarly, in phenomenological investigations the literature
approach. A major premise of grounded theory, for example, review may be delayed until the data analysis is complete.
is that data are collected in isolation from any predetermined This ensures that the findings reflect participants’ experiences
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
and are truly grounded in the data. This is congruent with provide a comprehensive and balanced account of previous
the philosophical orientation of phenomenology, which work, identifying, where appropriate, the relevant themes,
views the subjective experience of participants as central to conceptual models and theoretical frameworks that provide
the methodology (Burns and Grove, 2001). The conceptual a sound background to the research.
frameworks or themes that emerge from the study may
then be supported by evidence gleaned from a subsequent Research question
examination of the literature. In qualitative research, a research question that reflects the
The ethnographic approach attempts to examine the identified phenomenon of interest is used to direct the
experiences of the person in the context of his/her course of the research. A research hypothesis is never used
natural world and explores the topic of study through the in qualitative research, unlike quantitative research (Connell
perceptions of the subjects of study. The literature review in Meehan, 1999). Depending on the qualitative approach
ethnographic studies is used to demonstrate knowledge of adopted, e.g. grounded theory, the research question may
previous work in the area, as well as frameworks used in the be modified as new data bring new direction to the
analysis of data. phenomenon of interest. Such modifications should be
According to Meadows (2003), existing literature provides explained and justified by the researcher when they occur.
both the basis for research and the context for interpreting
findings. When critiquing qualitative research studies it is Theoretical framework
necessary to appraise the literature review in the context Many qualitative studies are described as inductive/atheoretical
of the particular methodology used. If the literature review or theory-generating research. This means that the purpose
is appropriate only after the period of data collection, then of the study is to develop theory not test it. Therefore, the
the researcher needs to identify how this process is to be researcher does not use an existing or known theory to direct
achieved and in what way the literature is going to be used to the study. This approach, which is heavily based on the work
determine similarities with or differences from the research of Glaser and Strauss (1967), is known as grounded theory.
findings. Conversely, if the literature review is carried out Ethnography and phenomenology are also classed as theory
before data collection and analysis, then it is required to generating. It is important that the researcher indicates this
in the study and justifies the adoption of such a stance, e.g.
where little is known about the phenomenon under study or
Table 2. Characteristics of qualitative research where existing theories do not seem to provide the answer
(Cronin and Rawlings-Anderson, 2004).
Truth There are multiple truths – generalization is not sought Some qualitative studies use known theories to ‘frame’
their studies (McKenna, 1997). This provides boundaries or
Purpose Concerned with discovery and description although
verification is also possible
parameters for the study and guides all stages, including the
literature review, data collection, analysis and presentation of
Context There is attention to the social context in which events findings (Parahoo, 2006).
occur and have meaning An important point for qualitative descriptive/exploratory
Emphasis There is an emphasis on understanding the social world research is that there are some limits or boundaries to
from the point of view of the participants in the study what is being studied. For example, if the researcher
– an emic perspective wished to study an aspect of patients’ postoperative pain
it would be critical to establish how the particular focus
Approach The approach is primarily inductive
was determined. This is sometimes done by exploring the
Relationship between There is integration between researcher and literature and identifying the main themes or concepts,
researcher and participant participant – interaction is valued which are then used to focus data collection and/or data
Sample Usually small in number but consists of those who are analysis and presentation of the findings.
able and willing to describe the experience
Methodology: research design
Data Elicits ‘soft data’, i.e. words
Design in qualitative research incorporates a range of
Data collection The major data collection techniques include approaches within what is often referred to as the
interviewing, participant observation, examination of naturalistic, interpretive or constructivist world view. The
personal documents and other printed materials important point here is that such a view of the world
Procedures and tools for data gathering are subject to incorporates a set of beliefs about knowledge and how
ongoing revision in the field situation this knowledge is developed. Qualitative research therefore
comprises a set of characteristics that reflect this world view
Analysis Analysis is presented for the most part in a narrative
(Table 2). However, it is also important to note that the
rather than numerical form, but the inclusion of some
quantitative measures and numerical expressions is
main qualitative approaches do differ in their disciplinary
not precluded in qualitative research or philosophical origins, hence the focus and manner in
which they undertake sampling, data collection and analysis
Rigour Credibility, transferability (fittingness), dependability, will vary (Table 3). For the reviewer it is essential that the
confirmability, goodness researcher outlines and justifies the chosen approach in
order to establish coherence and congruence.
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
RESEARCH METHODOLOGIES
Phenomenology/
Generic qualitative research Grounded theory hermeneutics Ethnography
Origin Broad term for all qualitative Symbolic interactionism Various schools of Cultural anthropology
research and may have its origins and social sciences philosophy
in any of the other disciplines
Aims Description of the issue under Generation of theory from the data Description/interpretation/ Direct description of a
study Modification/extension of existing understanding/meaning of group, culture or community
theory the lived experience/
phenomenon under study
Sample Purposive/convenience Initial sampling of people able to give Purposive Purposive, non-probability,
information on the topic. This is followed criterion-based
by theoretical sampling where further
sampling is guided by the analysis and
emerging theory
Data Interviews often semi-structured Interviews, participant observation, Unstructured, formal, Observation, interviews and
collection diaries and other documents, multiple interviews, examination of documents in
researcher’s own experience written texts, e.g. diaries the field
Data Generic data analysis tools, Constant, comparative analysis – data Data analysis tools vary Description, analysis and
analysis e.g. Miles and Huberman (1994) collection and analysis are linked from depending on school of interpretation of the culture
the beginning of the research philosophy adopted
Findings Description of the phenomenon Description of the emergent theory, Description/interpretation The culture as experienced by
under study incorporating the cultural processes of the phenomenon under its members is presented
and meanings study
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
that ethical principles are being adhered to and that include use of an expert panel or member checking
participants are protected from potential sources of harm (verifying with participants).
(Burns and Grove, 1999). It is important to note that within Several computer-assisted packages are available to assist
qualitative research, ethical issues often arise at different the qualitative researcher during analysis, e.g. NUD*IST
stages in the study and may be discussed when they occur (Non-numerical Unstructured Data Indexing, Searching
rather than under a specific heading. and Theorising), Ethnograph and NVivo (Robson, 2002).
There are inherent advantages to these packages in terms
Data collection of handling large amounts of data and assisting with coding
In a qualitative study any number of strategies can be and organizing the material. However, the rationale for how
adopted when collecting data, including non-numerical and why a particular tool was chosen should be evident.
questionnaires with open-ended questions, interviews Although data analysis is central to qualitative research, it
(semi-structured and unstructured), participant observation, is often poorly delineated in research publications. Very few
written texts such as diaries or emails, and historical or offer sufficient detail to determine the emergence of the
contemporary documents. The researcher should outline findings from the raw data, with the result that readers are
the rationale for the chosen method of data collection asked to ‘accept’ what they see. According to Thorne and
and offer sufficient information of the process. If using a Darbyshire (2005), the obligation to show the data that led
particular approach, such as grounded theory, it should be to the findings is a reasonable one.
evident from the discussion that the researcher has adhered
to the processes inherent in the methodology (Table 3). Rigour (trustworthiness)
Interviews are by far the most common method of Unlike the quantitative (positivist) paradigm that seeks to
data collection and are mainly either semi-structured or examine objective, measurable data and causal relationships
unstructured (Holloway and Wheeler, 2002). If a semi- between variables, qualitative research or naturalistic inquiry
structured interview format is selected it should be evident concerns itself with processes and meanings that cannot
how the themes or questions were derived. In unstructured always be experimentally examined. Socially constructed
interviews the initial opening question should be presented realities and relationships between the researcher and what
and clearly linked to the purpose of the study. Interviews is being studied are essential components of qualitative
are more frequently conducted face to face, but online or inquiry (Denzin and Lincoln, 2003). Proponents of
telephone interviews are also used. They can be undertaken qualitative approaches emphasize the value-laden nature
with individuals or groups, such as focus groups, and can be of naturalistic inquiry; a commonly heard criticism is that
one-off or multiple. The rationale for each of these decisions qualitative research is subjective, anecdotal and subject to
should be clearly presented. researcher bias (Koch and Harrington, 1998).
Although traditionally associated with grounded theory, Pilkington (2002) suggests that because qualitative
‘data saturation’ is often referred to by some qualitative methods are aimed at primarily understanding human
researchers as a point where they claim no new information experiences and ultimately theory development, alternative
will arise from further sampling. Thorne and Darbyshire criteria are required for ensuring the scientific merit of
(2005) suggest that some researchers use the concept of qualitative research studies. As quantitative studies are
data saturation as a convenient stopping point, and it may concerned with the generalizability and reproducibility
be pertinent to assess whether the study being evaluated, of findings, the concepts of reliability and validity are
particularly if it is a small-scale descriptive study, could have seen as appropriate criteria to use when evaluating the
achieved this. adequacy or robustness of quantitative research. There is
much discussion about the applicability of validity and
Data analysis reliability to qualitative research (Koch and Harrington,
In qualitative research the process by which data analysis is 1998; Tobin and Begley, 2004; Hoye and Severinsson,
undertaken is fundamental to determining the credibility 2007). The challenge to alternative paradigms or qualitative
of the findings. Essentially it involves the transformation of approaches is to produce plausible, robust research and to
raw data into a final description, narrative, or themes and demonstrate rigour.
categories. There is considerable variation in how this is Rigour is the means of demonstrating the plausibility,
undertaken, depending on the research question and the credibility and integrity of the qualitative research process.
approach taken (Vishnevsky and Beanlands, 2004). The rigour, or trustworthiness, of a study may be established
Some researchers use generic data analysis tools whereas if the reader is able to audit the actions and developments
others use less structured and more creative approaches. of the researcher (Koch, 2006). According to Burns and
What is important is that the process is described in Grove (2001), the critique of qualitative research requires an
sufficient detail to enable the reader to judge whether the appraisal of the rigour in documentation, procedural rigour,
final outcome is rooted in the data generated (Holloway and ethical rigour:
and Wheeler, 2002). The researcher should demonstrate ■ Rigour in documentation ensures there is a correlation
understanding of concurrent data collection and analysis, between the steps of the research process and the study in
the processes of organizing and retrieving data, as well question, commencing with the phenomenon of interest
as the steps in coding and thematic analysis. In addition, and following through to the recommendations and
verification strategies, if used, should be presented. Examples implications for practice.
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
RESEARCH METHODOLOGIES
■ Procedural rigour refers to appropriate and precise data inquiry and must be reflected in the entire study. Goodness
collection techniques and incorporates a reflective/critical needs to be evident in the philosophical background and
component in order to reduce bias and misinterpretations. study design, providing explicit explanations regarding
■ Ethical rigour describes how confidentiality issues and the the study context, data collection and management and
rights of participants are dealt with during the research the interpretation and presentation process. Goodness,
process. therefore, is a principle that should be present during
The most common criteria used to evaluate qualitative all stages of the research study and explicit in the final
research studies are credibility, dependability, transferability written report.
and confirmability (Table 2). Other terms such as goodness
and fruitfulness may also be used (Lincoln and Guba, 1985). Findings and discussion
The researcher needs to identify the criteria used and the As stated above, findings from qualitative studies can be
reader should be able to clearly follow each step of the represented as a narrative (story), themes, description of
research process: the phenomenon under study or an interpretive account
■ Credibility refers to the faithfulness to the description of the of the understanding or meaning of an experience.
phenomenon in question (Koch and Harrington, 1998). It Regardless of how the final outcome is presented, the
addresses the issue of whether there is consistency between researcher should discuss the findings in the context of
the participants’ views and the researcher’s representation what is already known.
of them. Koch (2006), asserts that credibility may be For many this will involve further literature review related
enhanced by the researcher describing and interpreting to the final outcome. However, the reviewer should beware
his/her experiences as researcher, and also by consulting of exaggerated claims as to the significance of the research
with participants and allowing them to read and discuss the and implications for practice, and further research should
research findings. Credibility may also be demonstrated by be located in the study’s findings. Moreover, the researcher
prolonged engagement, observation and audit trails. should relate the findings of the study back to the original
■ Dependability (auditability) is an integral component of research purpose, and illustrate whether or not it has been
rigour and involves the researcher giving the reader adequately addressed (Thorne and Darbyshire, 2005).
sufficient information to determine how dependable the
study and the researcher are. A study may be deemed Conclusions, implications and recommendations
auditable when another researcher can clearly follow the The researcher should conclude by placing the findings
trail used by the investigator and potentially arrive at the in a context that indicates how this new information is of
same or comparable conclusions. A research study may interest, and its implications for nursing. These conclusions
be shown to be dependable by producing evidence of a should reflect the study’s findings and ideally should offer
decision trail at each stage of the research process. According recommendations as to how they may be developed.
to Koch (2006), this provides the reader with evidence of
the decisions and choices made regarding theoretical and References
methodological issues throughout the study and entails An accurate list of all the books, journal articles, reports and
discussing explicitly the reasons for such decisions. It is also other media referred to in the study should be included in a
necessary for each stage of the research to be traceable and reference list at the end of the study (Polit and Beck, 2006).
clearly documented. For the reader who has an interest in the topic that was
■ Transferability (fittingness) refers to whether or not findings studied, the reference list can be a good source of further
can be applied outside the context of the study situation. reading.
When critiquing qualitative research, a study can be
deemed to have met the criterion of transferability when Conclusion
the findings can ‘fit’ into other contexts and readers can As with a quantitative study, critical analysis of a qualitative
apply the findings to their own experiences. Transferability study involves an in-depth review of how each step of the
is also enhanced when the results are meaningful to research was undertaken. Because of the subjective nature of
individuals not involved in the research study. qualitative research it is often regarded as more difficult to
■ Confirmability requires the researcher to demonstrate how critique. However, an evidenced-based profession cannot
conclusions and interpretations have been reached. It is accept any research at face value and needs to be able to
concerned with establishing that findings are clearly derived determine the strengths and limitations of studies when
from the data (Tobin and Begley, 2004). Confirmability is reviewing the available literature on a topic. BJN
usually established when credibility, transferability and
dependability are achieved.
■ Goodness is another criterion against which the
trustworthiness and authenticity of qualitative research Beauchamp T, Childress J (2001) Principles of Biomedical Ethics. 5th edn. Oxford
can be measured.When critiquing the rigour of qualitative University Press, Oxford
studies the issue of goodness may be seen as an integral Burns N, Grove S (1999) Understanding Nursing Research. 2nd edn.WB Saunders,
Philadelphia
component of the research process and an indicator of the Burns N, Grove S (2001) The Practice of Nursing Research: Conduct, Critique and
robustness of the study. Tobin and Begley (2004) suggest Utilisation. WB Saunders, Philadelphia
Connell Meehan T (1999) The research critique. In: Treacy P, Hyde A (eds)
that goodness is an overarching principle of qualitative Nursing Research and Design. UCD Press, Dublin: 57–74
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.
Coughlan M, Cronin P, Ryan F (2007) Step-by-step guide to critiquing Holloway I, Wheeler S (2002) Qualitative Research in Nursing. 2nd edn.
reasearch. Part 1: quantitative research. Br J Nurs 16(11): 658–63 Blackwell, Oxford
Cronin P, Rawlings-Anderson K (2004) Knowledge for Contemporary Nursing Hoye S, Severinsson E (2007) Methodological aspects of rigor in qualitative
Practice. Mosby, Edinburgh nursing research on families involved in intensive care units: a literature
Denzin N, Lincoln Y (eds) (2003) Strategies of Qualitative Inquiry. 2nd edn. Sage, review. Nurs Health Sci 9(1): 61–8
Thousand Oaks, CA Koch T (2006) Establishing rigour in qualitative research: the decision trail.
Fossey E, Harvey C, McDermott F, Davidson L (2002) Understanding and J Adv Nurs 53(1): 91–100
evaluating qualitative research. Aust N Z J Psychiatry 36(6): 717–32 Koch T, Harrington A (1998) Reconceptualising rigour: the case for reflexivity.
Glaser B, Strauss A (1967) The Discovery of Grounded Theory: Strategies for J Adv Nurs 28(4): 882–90`
Qualitative Research. Aldine, Chicago Lincoln Y, Guba E (1985) Naturalistic Inquiry. Sage, Thousand Oaks, CA
McKenna H (1997) Nursing Theories and Models. Routledge, London
Meadows K (2003) So you want to do research 1: an overview of the research
process. Br J Community Nurs 8(8): 369–75
KEY POINTS Miles M, Huberman A (1994) Qualitative Data Analysis. 2nd edn. Sage,
Thousand Oaks, CA
■ Qualitative and quantitative studies are fundamentally different approaches Parahoo K (2006) Nursing Research: Principles, Process and Issues. 2nd edn.
Palgrave Macmillan, Houndmills, Basingstoke
to research and therefore need to be approached differently with regard to Pilkington F (2002) Scientific merit and research ethics. Nurs Sci Q 15(3):
critiquing. 196–200
Polit D, Beck C (2006) Essentials of Nursing Care: Methods, Appraisal and
■ Despite differences in philosophies and terminology, the various qualitative Utilization. 6th edn. Lippincott Williams and Wilkins, Philadelphia
Robinson J (2002) Choosing your methods. In: Tarling M, Crofts L (eds) The
methods have many similarities that can be categorized together. Essential Researcher’s Handbook for Nurses and Health Care Professionals. 2nd
edn. Ballière Tindall, Edinburgh: 64–81
■ Phenomena are not simply a number of different variables that can be Robson C (2002) Real World Research. 2nd edn. Blackwell, Oxford
studied independently: they are more complex than the sum of their parts Ryan-Wenger N (1992) Guidelines for critique of a research report. Heart
Lung 21(4): 394–401
and should be studied holistically. Smith L (1992) Ethical issues in interviewing. J Adv Nurs 17(1): 98–103
Thorne S, Darbyshire P (2005) Land mines in the field: a modest proposal for
■ Qualitative studies are perceived as being more difficult to critically evaluate improving the craft of qualitative health research. Qual Health Res 15(8):
1105–13
than quantitative studies; nevertheless, nurses need to be able to critique Tobin G, Begley C (2004) Methodological rigour within a qualitative
qualitative research in order to identify best practice. framework. J Adv Nurs 48(4): 388–96
Vishnevsky T, Beanlands H (2004) Qualitative research. Nephrol Nurs J 31(2):
234–8
British Journal of Nursing.Downloaded from magonlinelibrary.com by 130.113.086.233 on January 28, 2015. For personal use only. No other uses without permission. . All rights reserved.