Project Methodology 5

Health Evidence Bulletins - Wales (logo)

Appraisal Checklists

Health Evidence Bulletins - Wales: Questions to assist with the critical appraisal of an observational study eg cohort, case-control, cross-sectional. (Type IV evidence)
Sources used: Critical Appraisal Skills Programme (CASP, Anglia and Oxford RHA) questions and Polgar A, Thomas SA. Chapter 22. Critical evaluation of published research in Introduction to research in the health sciences. 3rd edition. Melbourne: Churchill Livingstone, 1995; Undertaking systematic reviews of research on effectiveness. University of York: NHS Centre for Reviews & Dissemination, 2001; Weightman AL, Barker, JM, Lancaster J. Health Evidence Bulletins Wales Project Methodology 3. Cardiff: UWCM, 2000.

Paper details Authors:

A/ What is this paper about?

  Yes Can't tell No
1. Is the study relevant to the needs of the Project?      
2. Does the paper address a clearly focused issue?

in terms of
  • the population studied?
  • (case-control study only) Is the case definition explicit and confirmed?
  • the outcomes considered?
  • are the aims of the investigation clearly stated?

B/ Do I trust it?

  Yes Can't tell No
3. Is the choice of study method appropriate?      
4. Is the population studied appropriate?
  • (cohort study) Was an appropriate control group used – ie were groups comparable on important confounding factors?
  • (case-control study) Were the controls randomly selected from the same population as the cases?
5. Is confounding and bias considered?
  • Have all possible explanations of the effects been considered?
  • (cohort study) Were the assessors blind to the different groups?
  • (cohort study) Could selective drop out explain the effect?
  • (case-control study) How comparable are the cases and controls with respect to potential confounding factors?
  • (case-control study) Were interventions and other exposures assessed in the same way for cases and controls?
  • (case-control study) Is it possible that overmatching has occurred in that cases and controls were matched on factors related to exposure?
6. (Cohort study) Was follow up for long enough?
  • Could all likely effects have appeared in the time scale?
  • Could the effect be transitory?
  • Was follow up sufficiently complete?
  • Was dose response demonstrated?

C/ What did they find?


  Yes Can't tell No
7. Are tables/graphs adequately labelled and understandable?      
8. Are you confident with the authors' choice and use of statistical methods, if employed?      
9. What are the results of this piece of research?
Are the authors' conclusions adequately supported by the information cited?

D/ Are the results relevant locally?

  Yes Can't tell No
10. Can the results be applied to the local situation?
Consider differences between the local and study populations (eg cultural, geographical, ethical) which could affect the relevance of the study.
11. Were all important outcomes/results considered?      
12. Is any cost-information provided?      
13. Accept for further use as Type IV evidence?   Refer to Team Leader  


Draft Statement (if appropriate):
(Remember to include the relevant target group (age range, sex etc.); the measured outcomes/benefits with quantitative information if available; and the health gain notation)

next page

Methodology Home