What can occupational health professionals learn from the Cochrane Collaboration?

Introduction

The call to use evidence to improve the quality of health services is not new. In the seventies, even though he was not the first, Archie Cochrane clearly formulated the idea.1 Later this was taken over by Sackett who introduced the term evidence-based medicine.2;3 In turn, these concepts have been taken over by the Cochrane Collaboration, an organization dedicated to gathering, summarizing and disseminating systematic reviews of the effects of health care interventions.4 In 2004 the Finnish Institute of Occupational Health decided to join the Cochrane Collaboration, which resulted in the registration of the occupational health field with the Collaboration.

The Cochrane Collaboration uses a highly structured format both for gathering and summarizing evidence. Until now, we have not been used to systematically searching and summarizing evidence in questions of occupational health. Therefore, we have to compromise between the Cochrane format and the needs and demands of occupational health practice. In this article we would like to share our experiences in working with the highly structured format of the Cochrane Collaboration.

The format of a Cochrane systematic review

Thus far, Cochrane systematic reviews have dealt only with the effectiveness of interventions. In a systematic review, there should always be a clearly formulated question underlying a review preferably formatted according to the PICO acronym.5 The P stands for the population or type of patients at which the intervention is targeted. Then, obviously, the Intervention of interest should be defined. Next, there has to be a definition of the Comparison group against which the outcome in the intervention group will be compared. Finally, you have to state in advance what the Outcome is in which you are interested or by which you will measure the effectiveness. This will result in questions like: ‘Is advice for manual material handling effective in preventing or treating back pain in workers?'

A systematic review requires the definition of clear inclusion and exclusion criteria for the studies. This does not only relate to the PICO features, but also to the types of study design. In addition, the assessment and reporting of methodological quality is a required aspect of every Cochrane Review.

A comprehensive search strategy is a logical result of using a structured approach. The inclusion criteria have to be transformed into search words that can be used to locate studies of interest in electronic databases such as Medline. Next, one has to select manually the studies that fulfill all inclusion criteria.

The methods for summarizing the studies can be either quantitative as in a meta-analysis, or qualitative in the form of levels of evidence.6

Well posed questions

In occupational health care, as in other preventive specialties, there is a tendency to justify preventive activities as good in themselves. The line of thinking then is that if there are work-related health risks, then everything should be done to try to avoid them. However, all interventions use resources, and some interventions can do more harm than good. An example of this is debriefing in post-traumatic stress syndrome. Debriefing has long been considered an intervention that could prevent stress symptoms. Good evaluation studies with well posed questions and systematic reviews of these studies have nevertheless shown that in fact the opposite is true. Debriefing can actually induce post-traumatic stress syndrome and does not prevent it.7 Well posed questions can help to avoid confusion about prevention. It forces us to think about how the intervention that we apply might work, and what kind of outcome we would like to decrease or increase.8

Don't we have enough evidence of work-related health risks?

Do we really need evaluation research to prove that occupational health interventions are effective? One can easily infer that if there is a strong relation between a risk at work and a health outcome, then taking away the risk will also reduce the health outcome. This may well be true, but since there are many different ways by which we can eliminate risks, we would be interested in the most efficient way of doing so. For example, in the case of asbestos is it better to ban it, to impose taxes, or to provide subsidies for using alternatives? In the case of noise, we have been applying interventions for years already, and there is only a very slow decline in occupational noise-induced hearing loss. This can simply imply that our methods are not very effective. Rigorous evaluation might shine a new light on what works and what doesn't.

What kind of evidence to use?

A common criticism of the Cochrane Collaboration is that it is too restrictive in the use of research designs, most often restricted to randomized controlled trials. Especially in the area of occupational health it can be very difficult or even impossible to use a randomized design. This raises many interesting questions. If a randomized design is not appropriate for studying occupational health, then what is?

We think that the study design is not related to the setting but rather to the type of intervention that is applied. Clinical interventions are usually applied at the individual level and yield only minor to moderate effects. The use of methodologically weaker designs can easily lead to bias that completely distorts more moderate results. This is a strong incentive to randomize individuals, since randomization has been shown to best prevent bias. However, interventions that aim at changing the work environment, such as lessening exposure to noise are usually applied at the group level. Even though it is possible to randomize groups, this is more difficult. In the case of regulations or policy changes as an intervention, the level of application is a whole country or an entire population. Even though in theory it would be possible to randomize there, in practice this is impossible. In these cases, it is obvious that we would need research designs that differ from randomized controlled trials.

Are all interventions focusing on occupational health different from those in the clinical setting? This implies that we need more and better knowledge about occupational health interventions. It can easily be shown that we apply a wide variety of interventions at different levels. Many return-to-work interventions are individually directed and do not differ much from an ordinary clinical intervention. We feel that randomized designs are needed to evaluate these interventions. 9;10 Interventions aimed at decreasing exposure can be randomized at the workplace level. In a study advising employers about reducing wood dust exposure, researchers were able to use a cluster randomized design. They were surprised to find much smaller reductions in wood dust than they anticipated.11 We feel that this should be a strong incentive to use this design in the evaluation of environmental interventions. The question then remains, how to evaluate policy measures and national regulations. The time-series design could be feasible here. However, there is little experience in conducting them and it is not easy to summarize their outcomes over studies.

Is there an alternative to the use of evidence?

If it is not evidence that substantiates our decisions in practice or policy, what would be the alternative? We can see three alternatives: costs and competition, experts, and theory.

A rapidly emerging alternative in many European countries is commercial interests. Health services are being shaped according to client demands, and providers are becoming most interested in selling services that will deliver the highest profits and are the easiest to sell. Most health services are already cost driven and not outcome driven.12 For example, it is much easier to sell pre-employment examinations than physical conditioning for back pain. Evidence could be a means to counterbalance the pressure to increase the profitability of occupational health services at the expense of professional expertise.

It has been shown that occupational physicians rely heavily on the use of experts if they need information to underpin their decisions in practice.13 However, in a comparison of expert advice to answers from the literature, the experts turned out to be wrong in about half of the cases.14 Only if the experts said that they could base their answers on evidence from the literature, was the outcome much better. Therefore, the option of using experts instead of evidence from the literature does not seem to be viable.

Another alternative is to use theory to support practice. In occupational health, we have theoretical constructs, such as the hierarchy of controls that state that it is always preferable to eliminate the source of the health hazard than to use ventilation or personal protection.15 However, it is not clear if this is also the most efficient approach in practice.16 Evidence from evaluation research should be used to further shape occupational health theory.

Conclusion

The systematic approach of the Cochrane Collaboration can be applied to a great part of occupational health interventions. Applying the Cochrane methodology to the area of occupational health raises many interesting questions either about the methodology or about the alternatives. Finding answers to these questions is a challenge for all those involved in occupational health.