Prioritization Series
Ensuring relevance for Cochrane reviews: evaluating processes and methods for prioritizing topics for Cochrane reviews

https://doi.org/10.1016/j.jclinepi.2012.01.001Get rights and content

Abstract

Objective

The purpose of this study was to assess the presence and effectiveness of existing systems of prioritization for Cochrane review topics and to explore methods of improving those systems.

Study Design and Setting

We surveyed groups of Cochrane review authors and recorded any evidence of their use of priority-setting processes or policies. To evaluate the effectiveness of the policies we encountered, we assessed them using two frameworks from the literature: “Accountability for Reasonableness” (1) and Sibbald’s 2009 framework (2) for successful priority setting. We then held two workshops with the subject groups to discuss our findings and their implications.

Results

Of the 66 groups surveyed, 29 had a system in place to inform the selection or prioritization of topics for Cochrane reviews. Fifteen groups used a more comprehensive structured approach that eventually resulted in a list of ranked priority titles for authoring, updating, or disseminating Cochrane reviews. Most groups involved researchers, practitioners, and patients in their prioritization processes.

Conclusion

Groups within The Cochrane Collaboration currently use a range of different priority-setting systems, some of which are more detailed than others. These differences often reflect the nature of The Cochrane Collaboration itself: given the topic breadth, history, and variety of international contexts present in the organization, a single unified system would not always be appropriate. All Cochrane entities, however, should have or develop strategic plans to improve the inclusiveness and transparency of their own prioritization processes, increase the number of finished prioritized reviews, and make more effective use of feedback from end users to increase the likelihood of producing reviews that have positive effects on health outcomes.

Introduction

What is new?

Key findings:

  1. The current priority-setting strategies across the Cochrane Collaboration have great potential to improve the relevancy of the finalized Cochrane reviews published in The Cochrane Library.

What this adds to what was known?
  1. The present study depicts the gaps in priority-setting strategies across the Cochrane Collaboration as at 2008 and how they could be improved.

What is the implication, what should change now?
  1. Strategic plans of Cochrane entities should address how to improve the inclusiveness and transparency of topic selection processes and put an appeal and enforcement mechanism in their process.

  2. Priority-setting approaches should be compared and contrasted with other systematic review organizations.

  3. There is a need for reporting guidelines for research priority-setting projects.

Systematic reviews of intervention studies are considered to be one of the best sources of information for health decision making and policy making, especially when addressing questions about the overall effectiveness of treatments or interventions. When compared with individual studies, properly conducted systematic reviews offer several clear advantages: they provide a higher level of confidence in their results, they present the available evidence more efficiently, they are less likely to provide misleading information to end users, and they are accompanied by a greater degree of constructive context [1]. The Ministerial Summit on Health Research in Mexico City (58th World Health Assembly) endorsed systematic reviews as an important method of consolidating research findings, especially from clinical trials [2].

The Cochrane Collaboration (www.cochrane.org) is an international organization dedicated to the exchange of information between clinicians, health program decision makers, researchers, and consumers of health care services. One of the primary tasks of the organization is the preparation, maintenance, and dissemination of systematic reviews of the effects of interventions on health and related outcomes. These reviews are prepared primarily by volunteer or funded authors who work through one of the 52 Cochrane Review Groups, each of which focuses on a specific disease or health topic. While these groups form the editorial bases of the Cochrane Collaboration, the organization also contains Fields and Networks, which explore alternate dimensions of health care encompassing multiple diseases (such as the Child Health Field or the Primary Care Field) [3]. The work of the Cochrane Collaboration is based on 10 key principles: collaboration, building on the enthusiasm of individuals, avoiding duplication, minimizing bias, keeping up to date, striving for relevance, promoting access, ensuring quality, continuity, and enabling wide participation. As an organization striving to promote transparency, equity, and fairness, The Cochrane Collaboration has a critical need to regularly evaluate whether its current review topic prioritization processes serve these objectives.

To ensure the relevancy and usefulness of systematic Cochrane reviews to end users, it is crucial that those reviews address important and clinically relevant questions. Historically, the Cochrane model has relied on review topics being “supply driven”: The Cochrane Library contains over 4,000 reviews, most of which were proposed by clinician authors and approved by the editorial team of the relevant Cochrane Review Group. Although this model was generally successful, its reliance on researcher interest sometimes resulted in imbalanced or incomplete review subject coverage. To address this issue, many of the groups have begun to adopt a more systematic approach to identify priority topics for Cochrane reviews, using various strategies to ensure that relevant and important topics are addressed [4], [5], [6], [7], [8], [9].

Until recently, the priority-setting processes were implemented on a per-group basis. In 2006, however, the Cochrane Collaboration’s Steering Group recognized the need for a more strategic approach to improve overall prioritization, resulting in the creation of the Cochrane Prioritisation Fund. This fund financed five initiatives—one of which resulted in the creation of this study—to explore prioritization in the production and updating of Cochrane reviews (www.cochrane.org/admin/cc_funding_initiatives.htm#priorfund).

The focus of this specific initiative was a comparison of current methods and processes being used for topic prioritization across the Collaboration, with the aim of evaluating how successful these processes are and suggesting ways in which they could be improved.

Section snippets

Collecting data on prioritization processes across the Collaboration

In 2008, we solicited information on prioritization policies from the contact representatives of a variety of Cochrane Review Groups, Fields, and Networks. Using a combination of e-mail communication and in-person interviewing, we asked each representative whether their group had made any recent use of such policies when writing, updating, or disseminating Cochrane reviews and whether they had undertaken any projects on prioritizing titles. We extracted our data from dialogues with the contact

Collected data on prioritization processes across the Collaboration

Of the 66 Cochrane entities (Table 3) that we contacted, 52 responded (78% response rate). Of these, 29 (about 56%) had an existing process in place to inform the selection or prioritization of topics for Cochrane reviews. These processes included the following:

  • (1)

    editorial processes in which the editors were asked for priority topics;

  • (2)

    a survey of consumers, reviewers, and editors of the review group;

  • (3)

    comparisons of the topics of finished Cochrane reviews with the trials in the trial register of the

Discussion

Most of the Cochrane entities we surveyed used different strategies to both improve the selection of topics for Cochrane reviews and ensure that the relevance of the evidence provided in The Cochrane Library was maintained. These initiatives helped to identify topics that are more relevant for different stakeholders. They also raised awareness about the Cochrane Collaboration among stakeholders who were not yet directly involved in the Collaboration.

The currently available information is not

Conclusion

The results of this study show that current priority-setting strategies across the Cochrane Collaboration are fragmented but have great potential. Making priority setting a priority would improve the relevancy of Cochrane reviews published in The Cochrane and thus increase the impact of its reviews on health and related outcomes. Cochrane entities should address how to improve the inclusiveness and transparency of the priority-setting processes.

Further research is needed to improve methods for

References (15)

There are more references available in the full text version of this article.

Cited by (39)

  • A common framework of steps and criteria for prioritizing topics for evidence syntheses: a systematic review

    2020, Journal of Clinical Epidemiology
    Citation Excerpt :

    Our review findings can guide the work of researchers, funders, policymakers, and other stakeholders seeking to prioritize topics or questions to be addressed in systematic reviews. As Nasser et al. noted, there is no best practice for priority setting due to differences in context, purpose, and topic breadth of each priority setting [13]. Individuals or groups funding or conducting systematic reviews can select the process that best fits their needs.

  • Beginning with high value care in mind: A scoping review and toolkit to support the content, delivery, measurement, and sustainment of high value care

    2019, Patient Education and Counseling
    Citation Excerpt :

    They additionally address adaption to changing healthcare context, capacity, and processes [203,204]. Additional insights on key processes (e.g. prioritization [205–209] and public engagement [210]) are available in the broader literature. Overall, potential users (n = 36) were enthusiastic about the toolkit.

View all citing articles on Scopus
View full text