The purpose of this study was to assess the presence and effectiveness of existing systems of prioritization for Cochrane review topics and to explore methods of improving those systems.
Study Design and Setting
We surveyed groups of Cochrane review authors and recorded any evidence of their use of priority-setting processes or policies. To evaluate the effectiveness of the policies we encountered, we assessed them using two frameworks from the literature: “Accountability for Reasonableness” (1) and Sibbald’s 2009 framework (2) for successful priority setting. We then held two workshops with the subject groups to discuss our findings and their implications.
Results
Of the 66 groups surveyed, 29 had a system in place to inform the selection or prioritization of topics for Cochrane reviews. Fifteen groups used a more comprehensive structured approach that eventually resulted in a list of ranked priority titles for authoring, updating, or disseminating Cochrane reviews. Most groups involved researchers, practitioners, and patients in their prioritization processes.
Conclusion
Groups within The Cochrane Collaboration currently use a range of different priority-setting systems, some of which are more detailed than others. These differences often reflect the nature of The Cochrane Collaboration itself: given the topic breadth, history, and variety of international contexts present in the organization, a single unified system would not always be appropriate. All Cochrane entities, however, should have or develop strategic plans to improve the inclusiveness and transparency of their own prioritization processes, increase the number of finished prioritized reviews, and make more effective use of feedback from end users to increase the likelihood of producing reviews that have positive effects on health outcomes.
Introduction
What is new?
Key findings:
•
The current priority-setting strategies across the Cochrane Collaboration have great potential to improve the relevancy of the finalized Cochrane reviews published in The Cochrane Library.
What this adds to what was known?
•
The present study depicts the gaps in priority-setting strategies across the Cochrane Collaboration as at 2008 and how they could be improved.
What is the implication, what should change now?
•
Strategic plans of Cochrane entities should address how to improve the inclusiveness and transparency of topic selection processes and put an appeal and enforcement mechanism in their process.
•
Priority-setting approaches should be compared and contrasted with other systematic review organizations.
•
There is a need for reporting guidelines for research priority-setting projects.
Systematic reviews of intervention studies are considered to be one of the best sources of information for health decision making and policy making, especially when addressing questions about the overall effectiveness of treatments or interventions. When compared with individual studies, properly conducted systematic reviews offer several clear advantages: they provide a higher level of confidence in their results, they present the available evidence more efficiently, they are less likely to provide misleading information to end users, and they are accompanied by a greater degree of constructive context [1]. The Ministerial Summit on Health Research in Mexico City (58th World Health Assembly) endorsed systematic reviews as an important method of consolidating research findings, especially from clinical trials [2].
The Cochrane Collaboration (www.cochrane.org) is an international organization dedicated to the exchange of information between clinicians, health program decision makers, researchers, and consumers of health care services. One of the primary tasks of the organization is the preparation, maintenance, and dissemination of systematic reviews of the effects of interventions on health and related outcomes. These reviews are prepared primarily by volunteer or funded authors who work through one of the 52 Cochrane Review Groups, each of which focuses on a specific disease or health topic. While these groups form the editorial bases of the Cochrane Collaboration, the organization also contains Fields and Networks, which explore alternate dimensions of health care encompassing multiple diseases (such as the Child Health Field or the Primary Care Field) [3]. The work of the Cochrane Collaboration is based on 10 key principles: collaboration, building on the enthusiasm of individuals, avoiding duplication, minimizing bias, keeping up to date, striving for relevance, promoting access, ensuring quality, continuity, and enabling wide participation. As an organization striving to promote transparency, equity, and fairness, The Cochrane Collaboration has a critical need to regularly evaluate whether its current review topic prioritization processes serve these objectives.
To ensure the relevancy and usefulness of systematic Cochrane reviews to end users, it is crucial that those reviews address important and clinically relevant questions. Historically, the Cochrane model has relied on review topics being “supply driven”: The Cochrane Library contains over 4,000 reviews, most of which were proposed by clinician authors and approved by the editorial team of the relevant Cochrane Review Group. Although this model was generally successful, its reliance on researcher interest sometimes resulted in imbalanced or incomplete review subject coverage. To address this issue, many of the groups have begun to adopt a more systematic approach to identify priority topics for Cochrane reviews, using various strategies to ensure that relevant and important topics are addressed [4], [5], [6], [7], [8], [9].
Until recently, the priority-setting processes were implemented on a per-group basis. In 2006, however, the Cochrane Collaboration’s Steering Group recognized the need for a more strategic approach to improve overall prioritization, resulting in the creation of the Cochrane Prioritisation Fund. This fund financed five initiatives—one of which resulted in the creation of this study—to explore prioritization in the production and updating of Cochrane reviews (www.cochrane.org/admin/cc_funding_initiatives.htm#priorfund).
The focus of this specific initiative was a comparison of current methods and processes being used for topic prioritization across the Collaboration, with the aim of evaluating how successful these processes are and suggesting ways in which they could be improved.
Section snippets
Collecting data on prioritization processes across the Collaboration
In 2008, we solicited information on prioritization policies from the contact representatives of a variety of Cochrane Review Groups, Fields, and Networks. Using a combination of e-mail communication and in-person interviewing, we asked each representative whether their group had made any recent use of such policies when writing, updating, or disseminating Cochrane reviews and whether they had undertaken any projects on prioritizing titles. We extracted our data from dialogues with the contact
Collected data on prioritization processes across the Collaboration
Of the 66 Cochrane entities (Table 3) that we contacted, 52 responded (78% response rate). Of these, 29 (about 56%) had an existing process in place to inform the selection or prioritization of topics for Cochrane reviews. These processes included the following:
(1)
editorial processes in which the editors were asked for priority topics;
(2)
a survey of consumers, reviewers, and editors of the review group;
(3)
comparisons of the topics of finished Cochrane reviews with the trials in the trial register of the
Discussion
Most of the Cochrane entities we surveyed used different strategies to both improve the selection of topics for Cochrane reviews and ensure that the relevance of the evidence provided in The Cochrane Library was maintained. These initiatives helped to identify topics that are more relevant for different stakeholders. They also raised awareness about the Cochrane Collaboration among stakeholders who were not yet directly involved in the Collaboration.
The currently available information is not
Conclusion
The results of this study show that current priority-setting strategies across the Cochrane Collaboration are fragmented but have great potential. Making priority setting a priority would improve the relevancy of Cochrane reviews published in The Cochrane and thus increase the impact of its reviews on health and related outcomes. Cochrane entities should address how to improve the inclusiveness and transparency of the priority-setting processes.
Towards systematic reviews that inform health care management and policymaking
J Health Serv Res Policy
(2005)
World Health Organization. 58th World Health Assembly. Ministerial Summit on Health Research. April 2005 Available at:...
Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Introduction. In: Higgins JPT, Green S, Eds. Cochrane...
B.S. Buckley et al.
Prioritising research: patients, carers and clinicians working together to identify and prioritise important clinical uncertainties in urinary incontinence
Neurourol Urodyn
(2010)
J. Doyle et al.
Global priority setting for Cochrane systematic reviews of health promotion and public health research
J Epidemiol Community Health
(2005)
There are more references available in the full text version of this article.
The project aimed to rapidly identify priority topic uncertainties as a first step to identify future systematic review questions of pertinence to key international fecal incontinence (FI) stakeholders (patients, carers, health care professionals, policy makers and voluntary, community, or social enterprise representatives). The paper’s aim is to share our methods, experience, and learning with other groups planning to deliver a rapid priority setting exercise.
An evidence gap map incorporated three evidence streams: emerging evidence identified through horizon scanning; existing evidence identified through systematic searches of bibliographic databases; and FI stakeholder insights collected through an international survey. The evidence gap map was presented during an online workshop with stakeholders, where they shared their expertize to expand, refine, and rank topic uncertainties using ideation techniques, focus group discussions, consensus techniques, and online polling.
The multistep methods used to deliver this priority setting exercise resulted in identification of broad priority topic uncertainties. The methods appear to have high acceptability and engagement with participants but await full evaluation.
This project successfully followed robust methodology, building upon frameworks from published priority setting and evidence gap mapping projects while incorporating strong patient and public involvement components.
Eliciting the research priorities of people affected by a condition, carers and health care professionals can increase research value and reduce research waste. The Cochrane Multiple Sclerosis and Rare Disease of CNS Group, in collaboration with the Cochrane Neurological Sciences Field, launched a priority setting exercise with the aim of prioritizing pressing questions to ensure that future systematic reviews are as useful as possible to the people who need them, in all countries, regardless of their economic status.
Sixteen high priority questions on different aspects of MS were developed by members of a multi-stakeholder priority setting Steering Group (SG). In an anonymous online survey translated into 12 languages researchers, clinicians, people with MS (PwMS) and carers were asked to identify and rank, 5 out of 16 questions as high priority and to provide an explanation for their choice. An additional free-text priority research topic suggestion was allowed.
The survey was accessible through MS advocacy associations’ social media and Cochrane web pages from October 20, 2020 to February 6, 2021. 1.190 responses (86.73% of all web contacts) were evaluable and included in the analysis. Responses came from 55 countries worldwide, 7 of which provided >75% of respondents and 95% of which were high and upper-middle income countries. 58.8% of respondents live in the EU, 23% in the Americas, 8.9% in the Western Pacific, 2.8% in the Eastern Mediterranean and 0.3% in South Eastern Asia. About 75% of the respondents were PwMS. The five research questions to be answered with the highest priority were: Question (Q)1 “Does MRI help predict disability worsening of PwMS?” (19.9%), Q5 “What are the benefits and harms of treating PwMS with one disease-modifying drug compared to another?” (19.3%), Q3 “Does multidisciplinary care by teams of different social and health professionals improve health outcomes and experiences for PwMS?” (11.9%), Q16 “Does psychological health affect disease progression in PwMS?” (9.2%) and Q10 “What are the benefits and harms of exercise for PwMS?” (7.2%). The multivariable logistic regression analysis indicated a significant influence of geographic area and income level on the ranking of Q1 and a marginal for Q16 as top a priority after accounting for the effect of all other predictors. Approximately 50% of the respondents indicated that they had an important additional suggestion to be considered.
This international collaborative initiative in the field of MS offers a worldwide perspective on the research questions perceived as pivotal by a geographically representative sample of multiple stakeholders in the field of MS. The results of the survey could guide the prioritization of research on pharmacological and non-pharmacological interventions which could be meaningful and useful for PwMS and carers, avoiding the duplication of efforts and research waste. High quality systematic reviews elicited by priority setting exercises may offer the best available evidence and inform decisions by healthcare providers and policy-makers which can be adapted to the different realities around the world.
2020, Journal of Allergy and Clinical Immunology: In Practice
Pediatric asthma remains a public health challenge with enormous impact worldwide.
The aim of this study was to identify and prioritize unmet clinical needs in pediatric asthma, which could be used to guide future research and policy activities.
We first identified unmet needs through an open-question survey administered to international experts in pediatric asthma who were members of the Pediatric Asthma in Real Life Think Tank. Prioritization of topics was then achieved through a second, extensive survey with global reach, of multiple stakeholders (leading experts, researchers, clinicians, patients, policy makers, and the pharmaceutical industry). Differences across responder groups were compared.
A total of 57 unmet clinical need topics identified by international experts were prioritized by 412 participants from 5 continents and 60 countries. Prevention of disease progression and prediction of future risk, including persistence into adulthood, emerged as the most urgent research questions. Stratified care, based on biomarkers, clinical phenotypes, the children's age, and demographics were also highly rated. The identification of minimum diagnostic criteria in different age groups, cultural perceptions of asthma, and best treatment by age group were priorities for responders from low-middle-income countries. There was good agreement across different stakeholder groups in all domains with some notable exceptions that highlight the importance of involving the whole range of stakeholders in formulation of recommendations.
Different stakeholders agree in the majority of research and strategic (eg, prevention, personalized approach) priorities for pediatric asthma. Stakeholder diversity is crucial for highlighting divergent issues that future guidelines should consider.
Our review findings can guide the work of researchers, funders, policymakers, and other stakeholders seeking to prioritize topics or questions to be addressed in systematic reviews. As Nasser et al. noted, there is no best practice for priority setting due to differences in context, purpose, and topic breadth of each priority setting [13]. Individuals or groups funding or conducting systematic reviews can select the process that best fits their needs.
The objective of the study was to systematically review the literature for proposed approaches and exercises conducted to prioritize topics or questions for systematic reviews and other types of evidence syntheses in any health-related area.
A systematic review. We searched Medline and CINAHL databases in addition to Cochrane website and Google Scholar. Teams of two reviewers independently screened the studies and extracted data.
We included 31 articles reporting on 29 studies: seven proposed approaches for prioritization and 25 conducted prioritization exercises (three studies did both). The included studies addressed the following fields: clinical (n = 19; 66%), public health (n = 10; 34%), and health policy and systems (n = 8; 28%), with six studies (21%) addressing more than one field. We categorized prioritization into 11 steps clustered in 3 phases (preprioritization, prioritization, and postprioritization). Twenty-eight studies (97%) involved or proposed involving stakeholders in the priority-setting process. These 28 studies referred to twelve stakeholder categories, most frequently to health care providers (n = 24; 86%) and researchers (n = 21; 75%). A common framework of 25 prioritization criteria was derived, clustered in 10 domains.
We identified literature that addresses different aspects of prioritizing topics or questions for evidence syntheses, including prioritization steps and criteria. The identified steps and criteria can serve as a menu of options to select from, as judged appropriate to the context.
Cochrane systematic reviews require more methodological support from Cochrane Review Groups (CRGs) than is customarily received by authors from journals; CRGs must therefore prioritize reviews to conserve resources. The TRIP database provided a data set of questions to guide prioritization for the acute respiratory infections (ARIs) CRG.
We extracted the ARI searches from the TRIP database (2010 to 2017) that contained at least one disease and one clinical management term, (defined as a “search”), and tabulated these by frequency.
There were 314,346 ARI searches from which we inferred 45,497 clinical questions, covering 365 topics. Two-thirds (30,541) of these addressed 20 clinical questions, of which treatment were the most frequent, followed by diagnosis, mortality, and prognosis. The five most frequent clinical questions were “Influenza + Vaccination” 4,989 (12.1%), “acute otitis media + antibiotics” 3,578 (8.7%), “common cold + vitamin C” 3,528 (8.6%), “meningitis + corticosteroids” 1,910 (4.6%), and “pneumonia + general treatment” 1,765 (4.3%). The 20 most frequent clinical questions were addressed by Cochrane reviews or protocols.
ARI questions are common and repeated often. Most may have been addressed by Cochrane reviews. The remainder form the basis of a priority list to assign resources for future Cochrane topics.
They additionally address adaption to changing healthcare context, capacity, and processes [203,204]. Additional insights on key processes (e.g. prioritization [205–209] and public engagement [210]) are available in the broader literature. Overall, potential users (n = 36) were enthusiastic about the toolkit.
To create a shared vision for the content, delivery, measurement, and sustainment of patient-centered high value care.
We performed a scoping review and translated findings into toolkit for system leaders. For our scoping review, we searched Medline, 2005-November 2015, for literature on patient-centered care (PCC) and its relationship to a high value care change model. We supplemented searches with key author, Google Scholar, and key website searches. One author reviewed all titles, abstracts, and articles for inclusion; another reviewed a random 20%. To develop our toolkit, we translated evidence into simple, actionable briefs on key topics and added resources. We then iteratively circulated briefs and the overall toolkit to potential users, making updates as needed.
In our scoping review, we found multiple interventions and measures to support the components of PCC and our change model. We found little on the overall effects of PCC or how PCC creates value. Potential users reported our toolkit was simple, understandable, thorough, timely, and likely to be globally useful.
Considerable evidence supports patient-centered high value care and a toolkit garnered enthusiasm.
The toolkit is ready for use, but needs comparison to other approaches.