Project Methodology 5

Health Evidence Bulletins - Wales (logo)

Appraisal Checklists

Health Evidence Bulletins - Wales: Questions to assist with the critical appraisal of an economic analysis (Type IV evidence)
Sources used: Greenhalgh T. How to read a paper: Papers that tell you what things cost (economic analyses). British Medical Journal 1997; 315: 596-599; Drummond MF et al. Users’ guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A.. Are the results valid? Journal of the American Medical Association 1997; 277: 1552-1557; O’Brien BJ et al. Users’ guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? Journal of the American Medical Association 1997; 277: 1802-1806.

Paper details Authors:

A What is this paper about?

  Yes Can't tell No
1. Is the study relevant to the needs of the Project?      
2. Is the analysis based on a study that answers a clearly defined clinical question about an economically important issue?
  • Costs and benefits (outcomes) should be compared for two or more strategies

B/ Do I trust it?

3. Whose viewpoint are costs and benefits being considered from?
  • The patient, the hospital, the drug company, the treasury, and/or society in general?
  Yes Can't tell No
4. Have the interventions being compared been shown to be clinically effective?
  • Is there evidence of effectiveness from one or more randomised controlled trials or other compelling studies?
5. Are the interventions sensible and workable in the settings where they are likely to be applied? eg
  • Is the ownership of a necessary piece of expensive equipment assumed?
  • Is likely compliance and/or the requirement for additional time considered?
6. Which method of analysis was used, and was this appropriate?
  • eg Cost-benefit, cost-utility (see Greenhalgh, 1997)

Method used..................



C/ What did they find?

7. How were costs and benefits measured?
  Yes Can't tell No
8. Were incremental rather than absolute benefits considered?
  • ie Consider cost per patient who benefits rather than cost per individual treatment eg where NNT = 10, one patient will benefit per 10 treated.
9. Was the "here and now" given precedence over the distant future?
  • Where future, rather than immediate, health benefits are assumed most analyses use a discount figure of around 5% per year.
10. Was a sensitivity analysis performed?      
11. Were "bottom line" aggregate scores overused?
  • Are unfamiliar units used with little explanation of the consequences in plain language?
12. What are the results of this piece of research?

Are the authors' conclusions adequately supported by the information cited?


D/ Are the results relevant locally?

  Yes Can't tell No
13. Can the results be applied to the local situation?

Consider differences between the local and study populations (eg cultural, geographical, ethical) which could affect the relevance of the study.

14. Accept for further use as Type IV evidence?   Refer to Team Leader  


Draft Statement (if appropriate):
(Remember to include the relevant target group (age range, sex etc.); the measured outcomes/benefits with quantitative information if available; and the health gain notation)

next page

Methodology Home