Project Methodology 5

Health Evidence Bulletins - Wales (logo)

Appraisal Checklists

Health Evidence Bulletins - Wales: Questions to assist with the critical appraisal of a systematic review
[including at least one randomised controlled trial] (Type I evidence);
[including at least one non-randomised intervention study] (Type III evidence); or
[of observational studies] (Type IV evidence)

Adapted from the CASP questions (taken from Oxman AD et al. Users' guides to the medical literature. VI How to use an overview. Journal of the American Medical Association. 1994; 272(17): 1367-1371) and Barker, JM. Project for the enhancement of the Welsh Protocols for Investment in Health Gain. Project Methodology. Cardiff: Duthie Library. UWCM, 1996. HEBW systematic review definition is  based on the criteria used by NHS CRD for Database of Abstracts of Reviews of Effects  (DARE.)

Paper details Authors:

A/ What is this review about, and does it meet the systematic review criteria?

  Yes Can't tell No
1. Is the review relevant to the needs of the Project? continue   discard
2. Did the review address a clearly focused issue?
in terms of:
  • the population studied
  • the intervention given
  • the outcomes considered.
3. Did the authors look for the appropriate sort of papers?
Did the studies address the review's question and have an appropriate study design?
4.  Were the important, relevant studies included?      
  •  Databases searched,  reference list follow-up,
  • Personal contacts, unpublished work
  • Non-English publications
  • Are the inclusion, exclusion criteria stated?
  • Has a funnel plot been included to check for publication bias?


5.  Did the authors assess the quality (rigour) of the included studies?      

Is it worth continuing?


B/ What are the findings and can they be trusted?

  Yes Can't tell No
6. If the results of the review have been combined, was this reasonable?
  • Were the studies sufficiently similar in design and results?
  • Are the results of included studies clearly displayed?
  • Are the reasons for any variation in the results discussed?


7. What is the overall result of the review?
Include a numerical result with the confidence limits if available.

C/ Are the results relevant locally/to me?

  Yes Can't tell No
8. Can the results be applied to the local population?
  • Cultural and/or sociodemographics differences?
  • Genetic differences?
  • Differences in medical practice?
9. Were all important outcomes considered?

Consider reliability.  Are outcome measures objective (i.e. a direct measure of change) or subjective (e.g. self reported).

10. Is any cost information provided?   N/A  
11. Accept for further use as Type I, III or IV evidence?   Refer to Team Leader  



To be classified as a systematic review, the review should meet the criteria used by the NHS CRD for the Database of Reviews of Effectiveness (DARE): 
1)  Are inclusion/exclusion criteria reported that address the review question?
2) Is there evidence of a substantial effort to search for all the relevant research literature;
3) Is the validity of the included studies adequately assessed?;
4) Is sufficient detail of the individual studies presented?;
5) Are the primary studies summarised appropriately?
  [accessed 23.09.03] 

next page

Methodology Home