Scribd
Upload
103 views2 pages

How To Critically Review

This document provides guidance on how to critically review research papers. It introduces the READER method for evaluating papers based on relevance, educational value, applicability, discrimination, evaluation, and reaction. Key aspects to examine are the research question, study design and methods, sample selection, and statistical analysis. Different types of studies like diagnostic tests, clinical trials, qualitative research, reviews, and surveys require examining specific aspects like blinding, randomization, search strategies, and sample representativeness. The goal is to judge the validity and reliability of a study's methods and conclusions.

Uploaded by

Andika Saputra
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
103 views2 pages

How To Critically Review

This document provides guidance on how to critically review research papers. It introduces the READER method for evaluating papers based on relevance, educational value, applicability, discrimination, evaluation, and reaction. Key aspects to examine are the research question, study design and methods, sample selection, and statistical analysis. Different types of studies like diagnostic tests, clinical trials, qualitative research, reviews, and surveys require examining specific aspects like blinding, randomization, search strategies, and sample representativeness. The goal is to judge the validity and reliability of a study's methods and conclusions.

Uploaded by

Andika Saputra
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

i

School of Postgraduate Medical and Dental Education

How to:

Critically Review a Paper


Glyn Elwyn
It may be difficult to acquire expertise in critically reading primary research papers, for several reasons. Time pressures deter access to papers that seem over long or complex, and often trainees (and trainers) rely on review papers which provide a carefully 'sanitised' view of research. There may also be an unwillingness or a lack of confidence in challenging the authority of the journal selection process and with it, the decisions of the peer reviews undertaken by experts. Using the acronym READER (Macauley, 1994) can be very helpful in considering what to read? Relevance: is it relevant to your needs. Educational: would it change what you do? Applicability: can the findings be applied? Discrimination: is it valid? Evaluation: check the quality. Reaction: how do you feel about it?

Keeping up to date with recent developments is an important task in any field. But it is beyond anyones ability to read everything that is published. Developing the ability to judge at high speed the quality of the papers you scan is a core skill. The editorial filter and the peerreview processes are not foolproof.

Introduction
It is said that clinicians read for roughly 30 minutes a week - a practised skim through the most relevant journal perhaps. Proponents of evidence-based medicine suggest a more purposeful approach by using digests of other journals, such as the Evidence Based Medicine Journal, or a search of the relevant databases for answers to structured, answerable clinical questions.

Is it worth reading this paper?


Reality is never as tidy. We all flick through journals, and/or read papers for journal clubs, examinations, and seminars. We sometimes settle on something that appears interesting but how can we decide if the paper in front of us is worth the effort? There are two key questions that you should be asking:

Why was the paper written?


Why was the paper written? What question was the researcher trying to answer? Other questions will follow of course, such as: Am I interested in this area and is it relevant to my work? Most papers follow the IMRAD structure (Introduction, Methods, Results and Discussion) and your answers will appear in the Introduction. The research question to be answered or the hypothesis to be refuted will be in this section, along with a summary of the background to the study (previous work and why there is a need for further inquiry). If this is missing, it is tempting to reject it at this stage.

How did the work try to answer the research questions?


How did the work try to answer the research questions? Go to the Methods section and look carefully for a description of the type of study involved.

There are many possible research designs, randomised controlled trials (RCTs), cohort studies, surveys, focus groups, observational studies and so on. Only some study designs suit some research questions. This is an important discriminator of the wheat from the chaff as the following sections, which cover various types of paper, will outline.

Diagnostic or screening tests


Sorting out who has and has not got disease x is one of the most difficult tasks in medicine. Firstly, we have to agree what we mean by normality (this is always a relative, not an absolute concept in biomedical science). Secondly, we need (but rarely achieve) tests that are definitely positive and definitely negative. Tests that dont fail to pick up problems on the one hand but also only identify problems where problems exist. They also have to be correctly negative: that they dont miss problems where they definitely exist. These qualifications are known as sensitivity and specificity. If you are reading a paper about a diagnostic procedure make sure that the work is comparing that test against a gold standard (as accurate a test as possible) in all subjects and in an appropriately wide spectrum of the population.

Clinical Trials
The best method for this type of questions is a randomised controlled trial. Matching sample groups are randomly allocated to experimental and control arms. The avoidance of bias during sample selection, randomisation and measurement is a critical part of the design. Explicit descriptions of true randomisation procedures and double-blinded out-come measurements (if feasible) on a prior determined intention to treat basis should be available. Full account should be given for subject dropouts. If you are reassured that these requirements are in place, then start checking (with help if needed) the statistical measures employed. Be wary of papers that draw conclusions entirely from subgroup analysis.

Qualitative studies
This type of research is finding increasing acceptance in clinical journals although there are many still who regard the seemingly open-ended approach to research questions difficult to accept. Appraising research that draws its data from observations and interviews is not easy. Methods which triangulate data (use more than one method to collect and analyse material) improve validity. A well described, transparent and validated process of content analysis helps a reader understand how conclusions have been drawn. The great advantage of this type of research is its ability to get close to the things that are difficult to quantify in numbers such as attitudes, constructs and issues that may be far more important than the easily measurable.

A review paper
In the 1970s, Cochrane suggested that a fully evaluated register of clinical trials should be available, but it is only recently that the far-sightedness of his vision is being recognised. Systematic reviews (rigorous, comprehensive search and analysis of published work on a specific question) and meta-analysis (pooling of numerical data from comparable RCTs) have now transformed most journals approaches to the review article. Non-systematic reviews are falling out of favour. When experts review a research topic is often difficult to judge whether or not they have bothered to do a comprehensive search for relevant papers or if they have simply used data to suit their conclusions. Often their review method is not transparent. Essays and discussion papers are different. We expect a point of view in editorials; bias is automatically built in. Work that claims to be an exhaustive objective overview of the literature should be exactly that. Procedures such as search strategies, data sources, inclusion and exclusions for studies located and the analysis process should be explicit and transparent. Meta-analysis goes a step further and pools the results of comparable RCTs. Make sure you check that the studies have homogeneity of trial design and outcome measures. The inherent danger is to compare oranges with lemons.

Further Reading
Chalmers, I. and Altman, D.G. (Eds) (1995). Systematic Reviews. London: BMJ Publishing. Clarke, R. (1998). Critical Reading for the reflective practitioner: a guide for primary care. Oxford: Butterworth-Heinemann. Cochrane, A.L. (1971). Effectiveness and efficacy: random reflections on health services. Nuffield Provincial Hospitals Trust. Greenhalgh, T. (2000). How to read a paper. London: BMJ Publishing Group. Jones, R. (1995). Critical Reading for Primary Care. Oxford: Oxford University Press. Macauley, D. (1994). READER: an acronym to aid critical reading by general practitioners. British Journal General Practice, Feb. 994. Mays, N. and Pope, C. (Eds) (1999) 2nd Edition. Qualitative research in health care. London: BMJ Publishing. Sackett, D., Scott Richardson, W. Rosenberg, W. and Haynes, R.B. (1997). Evidence Based Medicine. How to practise and teach EBM. New York: Churchill Livingstone.

A survey paper
Surveys ask a representative sample of people to answer specific questions, either by questionnaire or by various types of interviews. Take care to look at the derivation of the sample: were they randomly chosen from an appropriate population. Were they all approached and questioned in exactly the same way? Arriving at the correct questions to ask and constructing reliable instruments is not an easy task, so beware of tools that have never been piloted or validated. One of the main problems (with postal methods particularly) is the low response rates often reported. The findings of a study with a 60% positive outcome could be reviewed if the non-responders all disagreed and simply could not be bothered to reply.

Glyn Elwyn is Professor in General Practice with the Centre for Health Science Research at Cardiff University.

School of Postgraduate Medical and Dental Education


Wales College of Medicine, Cardiff University, Heath Park , Cardiff CF14 4XN Tel: +44 (0)29 2074 3160 Fax: +44 (0)29 2075 4966 E-mail: [email protected]

You might also like