Health Evidence Bulletins - Wales
Team Leader: Dr. Ronan Lyons

Date of Completion: 09.01.98


The original Protocols for Investment in Health Gain were written in the early 1990s to suggest areas where the introduction, or more widespread use, of certain practices could lead to worthwhile improvements in health for the people of Wales. The documents also highlighted current practices which were of questionable value. This revision has been prepared by reviewing the earlier Protocol for Investment in Health Gain: Injuries to provide some clear, updated statements with a precise indication of the strength of the evidence and its sources for each statement; and to introduce new statements covering subjects of important current interest. The original Injury Protocol looked at the prevention, diagnosis, treatment and rehabilitation of injuries. This Bulletin has focussed only on the prevention of unintentional injuries as the evidence base is stronger in this area. Self harm and suicide will be covered in the Mental Health Bulletin.

In keeping with the original Protocols, these revised documents are designed to assist Health Authorities in developing local strategies and in purchasing high quality health care. It is anticipated, however, that they will be of value to all health professionals in keeping abreast of the huge and increasing body of medical literature and can provide an agenda for future action in a wide variety of settings.

The statements represent a methodical summary of the evidence in this area found through a formal literature search across a wide range of sources. The evidence has been critically appraised using internationally accepted methods3 compiled into this document under the direction of a public health physician, and reviewed by a team of experts who are directly involved in patient care. The information is also available in paper copy. Information on the methodology adopted (including a copy of the documentation), the variety of formats in which the document is issued and details of other publications in the series, is available on request.

The convention used in this document to indicate the type of evidence is:

‘Type I evidence’ - at least one good systematic review
(including at least one randomised controlled trial).
‘Type II evidence’ - at least one good randomised controlled trial
‘Type III evidence’ - well designed interventional studies without randomisation
‘Type IV evidence’ - well designed observational studies
‘Type V evidence’ - expert opinion; influential reports and studies

The health gain notation (used to indicate the potential benefit to health) is:

‘beneficial’ - effectiveness clearly demonstrated (1)
‘likely to be beneficial’ - effectiveness not so firmly established (2)
‘trade-off between beneficial and adverse effects’ - effects weighed according to individual circumstances(3)
‘unknown’ - insufficient/inadequate for recommendation (4)
‘unlikely to be beneficial’ - ineffectiveness is not as clearly demonstrated as for 6 (5)
‘likely to be ineffective or harmful’ - ineffectiveness or harm clearly demonstrated (6)

Although the statements are deliberately brief, statistically significant quantitative information has been provided where possible. This is usually given as reduction in risk


Injury prevention and the nature of evidence

The gold standard for assessing whether an intervention works is the randomised controlled trial whereby similar groups are randomly allocated to receiving an intervention or not, and differences in outcome measured. Randomised controlled trials are widely used in medicine when a promising new treatment is being tried but it is uncertain whether the treatment is beneficial or not. If it is obvious that the new treatment is better it is unethical to carry out a trial to prove this fact, as this would involve a breach of the ethical principle of equipoise in experimental studies with human beings. It is also unethical to carry out trials if the exposure is thought to be harmful.

In the case of injury epidemiology, the outcome (injury) usually follows so quickly on the exposure (e.g. car crash) that there is little doubt as to what caused the injury. Thus, common sense and logic often elucidate cause and effect, rather than relying on the demonstration of a statistically significant association. For example, anyone who has trapped a finger in a door knows it is painful, but a statistician might require the pain to be demonstrated on 6 out of 7 occasions before declaring that the association between trapping a finger and pain was statistically significant and not due to chance.

Because of these ethical considerations, randomised controlled trials to demonstrate the effectiveness of injury prevention measures are often not possible and, before and after, or other designs are more appropriate. By valuing evidence from randomised controlled studies more highly than observational studies, there is a danger that interventions with limited effectiveness might be judged to be more worthy than interventions based on observation or common sense. This is not to say that the randomised controlled design is not the best design, or that more of these trials should not be carried out when logistically and ethically feasible.

To look at evidence only from randomised controlled trials (ideally from systematic reviews and/or meta-analyses of these trials) would exclude convincing evidence from laboratory and observational studies and give a misleading impression on the science of injury prevention. Thus, where this evidence was not available, other forms of study were also considered. However, because many observational studies are poorly indexed in the literature and much of the evidence is to be found in the grey literature, it is clear that not all useful studies have been identified. Where several studies show the same conclusions, the best example is included in the references. The Project Methodology does not include carrying out systematic reviews of observational studies.

It should be noted that interventions which try to educate or change behaviour are not usually standardised. The attitude of the facilitator and intensity and duration of a programme are all likely to be important in ensuring success but these will differ considerably from study to study. The lack of success reported in many studies may reflect inadequate rather than ineffective interventions. In such circumstances meta-analysis will not be helpful.

Dr Ronan Lyons, Team Leader. September 1998


1  Welsh Health Planning Forum. Cardiff: Welsh Office NHS Directorate, 1992

2  Health Evidence Bulletins. Health Evidence Bulletins - Wales: Mental Health. Cardiff: Welsh Office, 1998

3  Barker J, Weightman AL and Lancaster J. Project for the Enhancement of the Welsh Protocols for Investment in Health       Gain: Project Methodology 2. Cardiff: Duthie Library UWCM, 1997.

4  See Contributors

5  Contact: Protocol Enhancement Project Office, Sir Herbert Duthie Library, UWCM, Heath Park, Cardiff CF4 4XN.

6  This table is based on the Bandolier system and the classification of a systematic review used by the NHS Centre for Reviews and Dissemination. See p.18 in ref.3 or

7 This notation is modified from the tables used in Enkin M, Kierse MJNC, Renfrew M and Neilson J. A guide to effective care in pregnancy and childbirth 2nd ed. Oxford: Oxford University Press, 1995 pp. 389-39


This initiative is funded by the Wales Office of Research and Development for Health and Social Care.


Contents Home

Health Evidence Bulletins: Wales, Duthie Library, UWCM, Cardiff CF14 4XN. e-mail: