a case study systematic review

• Open supplemental data
• Reference Manager
• Simple TEXT file

People also looked at

Study protocol article, a protocol for the use of case reports/studies and case series in systematic reviews for clinical toxicology.

• 1 Univ Angers, CHU Angers, Univ Rennes, INSERM, EHESP, Institut de Recherche en Santé, Environnement et Travail-UMR_S 1085, Angers, France
• 2 Department of Occupational Medicine, Epidemiology and Prevention, Donald and Barbara Zucker School of Medicine, Northwell Health, Feinstein Institutes for Medical Research, Hofstra University, Great Neck, NY, United States
• 3 Department of Health Sciences, University of California, San Francisco and California State University, Hayward, CA, United States
• 4 Program on Reproductive Health and the Environment, University of California, San Francisco, San Francisco, CA, United States
• 5 Cesare Maltoni Cancer Research Center, Ramazzini Institute, Bologna, Italy
• 6 Department of Research and Public Health, Reims Teaching Hospitals, Robert Debré Hospital, Reims, France
• 7 CHU Angers, Univ Angers, Poisoning Control Center, Clinical Data Center, Angers, France

Introduction: Systematic reviews are routinely used to synthesize current science and evaluate the evidential strength and quality of resulting recommendations. For specific events, such as rare acute poisonings or preliminary reports of new drugs, we posit that case reports/studies and case series (human subjects research with no control group) may provide important evidence for systematic reviews. Our aim, therefore, is to present a protocol that uses rigorous selection criteria, to distinguish high quality case reports/studies and case series for inclusion in systematic reviews.

Methods: This protocol will adapt the existing Navigation Guide methodology for specific inclusion of case studies. The usual procedure for systematic reviews will be followed. Case reports/studies and case series will be specified in the search strategy and included in separate sections. Data from these sources will be extracted and where possible, quantitatively synthesized. Criteria for integrating cases reports/studies and case series into the overall body of evidence are that these studies will need to be well-documented, scientifically rigorous, and follow ethical practices. The instructions and standards for evaluating risk of bias will be based on the Navigation Guide. The risk of bias, quality of evidence and the strength of recommendations will be assessed by two independent review teams that are blinded to each other.

Conclusion: This is a protocol specified for systematic reviews that use case reports/studies and case series to evaluate the quality of evidence and strength of recommendations in disciplines like clinical toxicology, where case reports/studies are the norm.

Introduction

Systematic reviews are routinely relied upon to qualitatively synthesize current knowledge in a subject area. These reviews are often paired with a meta-analysis for quantitative syntheses. These qualitative and quantitative summaries of pooled data, collectively evaluate the quality of the evidence and the strength of the resulting research recommendations.

There currently exist several guidance documents to instruct on the rigors of systematic review methodology: (i) the Cochrane Collaboration, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and PRISMA-P (for protocols) that offer directives on data synthesis; and (ii) the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) guidelines that establish rules for the development of scientific recommendations ( 1 – 5 ). This systematic review guidance is based predominantly on clinical studies, where randomized controlled trials (RCTs) are the gold standard. For that reason, a separate group of researchers has designed the Navigation Guide, specific to environmental health studies that are often observational ( 6 , 7 ). To date, systematic review guidelines (GRADE, PRISMA, PRISMA-P, and Navigation Guide) remove case reports/studies and case series (human subjects research with no control group) from consideration in systematic reviews, in part due to the challenges in evaluating the internal validity of these kinds of study designs. We hypothesize, however, that under certain circumstances, such as in rare acute poisonings, or preliminary reports of new drugs, some case reports and case series may contribute relevant knowledge that would be informative to systematic review recommendations. This is particularly important in clinical settings, where such evidence could potentially change our understanding of the screening, presentation, and potential treatment of rare conditions, such as poisoning from obscure toxins. The Cochrane Collaboration handbook states that “ for some rare or delayed adverse outcomes only case series or case-control studies may be available. Non-randomized studies of interventions with some study design features that are more susceptible to bias may be acceptable for evaluation of serious adverse events in the absence of better evidence, but the risk of bias must still be assessed and reported ” ( 8 ). In addition, the Cochrane Adverse Effects group has shown that case studies may be the best settings in which to observe adverse effects, especially when they are rare and acute ( 9 ). We believe that there may be an effective way to consider case reports/studies and case series in systematic reviews, specifically by developing specific criteria for their inclusion and accounting for their inherent bias.

We propose here a systematic review protocol that has been specifically developed to consider the inclusion and integration of case reports/studies and case series. Our main objective is to create a protocol that is an adaptation of the Navigation Guide ( 6 , 10 ) that presents methodology to examine high quality case reports/studies and case series through cogent inclusion and exclusion criteria. This methodology is in concordance with the Cochrane Methods for Adverse Effects for scoping reviews ( 11 ).

This protocol was prepared in accordance with the usual structured methodology for systematic reviews (PRISMA, PRISMA-P, and Navigation guide) ( 3 – 7 , 10 ). The protocol will be registered on an appropriate website, such as one of the following:

(i) The International Prospective Register of Systematic Reviews (PROSPERO) database ( https://www.crd.york.ac.uk/PROSPERO/ ) is an international database of prospectively registered systematic reviews in health and social welfare, public health, education, crime, justice, and international development, where there is a health-related outcome. It aims to provide a comprehensive listing of systematic reviews registered at inception to help avoid duplication and reduce opportunity for reporting bias by enabling comparison of the completed review with what was planned in the protocol. PROSPERO accepts registrations for systematic reviews, rapid reviews, and umbrella reviews. Key elements of the review protocol are permanently recorded and stored.

(ii) The Open Science Framework (OSF) platform ( https://osf.io/ ) is a free, open, and integrated platform that facilitates open collaboration in research science. It allows for the management and sharing of research project at all stages of research for broad dissemination. It also enables capture of different aspects and products of the research lifecycle, from the development of a research idea, through the design of a study, the storage and analysis of collected data, to the writing and publication of reports or research articles.

(iii) The Research Registry (RR) database ( https://www.researchregistry.com/ ) is a one-stop repository for the registration of all types of research studies, from “first-in-man” case reports/studies to observational/interventional studies to systematic reviews and meta-analyses. The goal is to ensure that every study involving human participants is registered in accordance with the 2013 Declaration of Helsinki. The RR enables prospective or retrospective registrations of studies, including those types of studies that cannot be registered in existing registries. It specifically publishes systematic reviews and meta-analyses and does not register case reports/studies that are not first-in-man or animal studies.

Any significant future changes to the protocol resulting from knowledge gained during the development stages of this project will be documented in detail and a rationale for all changes will be proposed and reported in PROSPERO, OSF, or RR.

The overall protocol will differentiate itself from other known methodologies, by defining two independent teams of reviewers: a classical team and a case team. The classical team will review studies with control groups and an acceptable comparison group (case reports/studies and case series will be excluded). In effect, this team will conduct a more traditional systematic review where evidence from case reports/studies and case series are not considered. The case team will review classical studies, case reports, and case series. This case team will act as a comparison group to identify differences in systematic review conclusions due to the inclusion of evidence from case reports/studies and case series. Both teams will identify studies that meet specified inclusion criteria, conduct separate analyses and risk of bias evaluations, along with overall quality assessments, and syntheses of strengths of evidence. Each team will be blinded to the results of the other team throughout the process. Upon completion of the systematic review, results from each team will be presented, evaluated, and compared.

Patient and Public Involvement

No patient involved.

Eligibility Criteria

Studies will be selected according to the criteria outlined below.

Study Designs

Studies of any design reported in any translatable language to English by online programs (e.g., Google Translate) will be included at the beginning. These studies will span interventional studies with control groups (Randomized Controlled Trials: RCTs), as well as observational studies with and without exposed groups. All observational studies will be eligible for inclusion in accordance with the objectives of this systematic review. Thereafter, only the case team will include cases reports/studies and case series, as specified in their search strategy. The case team will include a separate section for human subjects research that has been conducted with no control groups.

Type of Population

All types of studies examining the general adult human population or healthy adult humans will be included. Studies that involve both adults and children will also be included if data for adults are reported separately. Animal studies will be excluded for the methodological purpose of this (case reports/studies and case series) protocol given that the framework for systematic reviews in toxicology already adequately retrieves this type of toxin data on animals.

Inclusion/Exclusion Criteria

Studies of any design will be included if they fulfill all the eligibility criteria. To be integrated into the overall body of evidence, cases reports/studies and case series must meet pre-defined criteria indicating that they are well-documented, scientifically rigorous, and follow ethical practices, under the CARE guidelines (for Ca se Re ports) ( 12 , 13 ) and the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Case reports/studies and for Case Series ( 14 , 15 ) that classify case reports/studies in terms of completeness, transparency and data analysis. Studies that were conducted using unethical practices will be excluded.

Type of Exposure/Intervention

Either the prescribed treatment or described exposure to a chemical substance (toxin/toxicant) will be detailed here.

Type of Comparators

In this protocol we plan to compare two review methodologies: one will include and the other will exclude high quality case reports/studies and case series; these two review methodologies will be compared. The comparator will be (the presence or absence of) an available control group that has been specified and is acceptable scientifically and ethically.

Type of Outcomes

The outcome of mortality or morbidity related to the toxicological exposure, will be detailed here.

Information Sources and Search Strategy

There will be no design, date or language limitations applied to the search strategy. A systematic search in electronic academic databases, electronic grey literature, organizational websites, and internet search engines will be performed. We will search at least the following major databases:

- Electronic academic databases : Pubmed, Web of Sciences, Toxline, Poisondex, and databases specific to case reports/studies and case series (e.g., PMC, Scopus, Medline) ( 13 )

- Electronic grey literature databases : OpenGrey ( http://www.opengrey.eu/ ), grey literature Report ( http://greylit.org/ )

- Organizational websites : AHRQ Patient Safety Network ( https://psnet.ahrq.gov/webmm ), World Health Organization ( www.who.int )

- Internet search engines : Google ( https://www.google.com/ ), GoogleScholar ( https://scholar.google.com/ ).

Study Records

Following a systematic search in all the databases above, each of the two independent teams of reviewers (the classical team and the case team) will, respectively, upload separately and in accordance with the eligibility criteria, the literature search results to the systematic review management software, “Covidence,” a primary screening and data extraction tool ( 16 ).

All study records identified during the search will be downloaded and duplicate records will be identified and deleted. Thereafter, two research team members will independently screen the titles and abstracts (step 1) and then the full texts (step 2) of potentially relevant studies for inclusion. If necessary, information will be requested from the publication authors to resolve questions about eligibility. Finally, any disagreements that may potentially exist between the two research team members will be resolved first by discussion and then by consulting a third research team member for arbitration.

If a study record identified during the search was authored by a reviewing research team member, or that team member participated in the identified study, that study record will be re-assigned to another reviewing team member.

Data Collection Process, Items Included, and Prioritization if Needed

All reviewing team members will use standardized forms or software (e.g., Covidence), and each review member will independently extract the data from included studies. If possible, the extracted data will be synthesized numerically. To ensure consistency across reviewers, calibration exercises (reviewer training) will be conducted prior to starting the reviews. Extracted information will include the minimum study characteristics (study authors, study year, study country, participants, intervention/exposure, outcome), study design (summary of study design, comparator, models used, and effect estimate measure) and study context (e.g., data on simultaneous exposure to other risk factors that would be relevant contributors to morbidity or mortality). As specified in the section on study records, a third review team member will resolve any conflicts that arise during data extraction that are not resolved by consensus between the two initial data extractors.

Data on potential conflict of interest for included studies, as well as financial disclosures and funding sources, will also be extracted. If no financial statement or conflict of interest declaration is available, the names of the authors will be searched in other studies published within the previous 36 months and in other publicly available declarations of interests, for funding information ( 17 , 18 ).

Risk of Bias Assessment

To assess the risk of bias within included studies, the internal validity of potential studies will be assessed by using the Navigation Guide tool ( 6 , 19 ), which covers nine domains of bias for human studies: (a) source population representation; (b) blinding; (c) exposure or intervention assessment; (d) outcome assessment; (e) confounding; (f) incomplete outcome data; (g) selective outcome reporting; (h) conflict of interest; and (i) other sources of bias. For each section of the tool, the procedures undertaken for each study will be described and the risk of bias will be rated as “ low risk”; “probably low risk”; “probably risk”; “high risk”; or “not applicable.” Risk of bias on the levels of the individual study and the entire body of evidence will be assessed. Most of the text from these instructions and criteria for judging risk of bias has been adopted verbatim or adapted from one of the latest Navigation Guide systematic reviews used by WHO/ILO ( 6 , 19 , 20 ).

For case reports/studies and case series, the text from these instructions and criteria for judging risk of bias has been adopted verbatim or adapted from one of the latest Navigation Guide systematic reviews ( 21 ), and is given in Supplementary Material . Specific criteria are listed below. To ensure consistency across reviewers, calibration exercises (reviewer training) will be conducted prior to starting the risk of bias assessments for case reports/studies and case series.

Are the Study Groups at Risk of Not Representing Their Source Populations in a Manner That Might Introduce Selection Bias?

The source population is viewed as the population for which study investigators are targeting their study question of interest.

Examples of considerations for this risk of bias domain include: (1) the context of the case report; (2) level of detail reported for participant inclusion/exclusion (including details from previously published papers referenced in the article), with inclusion of all relevant consecutive patients in the considered period; ( 14 , 15 ) (3) exclusion rates, attrition rates and reasons.

Were Exposure/Intervention (Toxic, Treatment) Assessment Methods Lacking Accuracy?

The following list of considerations represents a collection of factors proposed by experts in various fields that may potentially influence the internal validity of the exposure assessment in a systematic manner (not those that may randomly affect overall study results). These should be interpreted only as suggested considerations and should not be viewed as scoring or a checklist . Considering there are no controls in such designs, this should be evaluated carefully to be sure the report really contributes to the actual knowledge .

List of Considerations :

Possible sources of exposure assessment metrics:

1) Identification of the exposure

2) Dose evaluation

3) Toxicological values

4) Clinical effects *

5) Biological effects *

6) Treatments given (dose, timing, route)

* Some clinical and biological effects might be related to exposure

For each, overall considerations include:

1) What is the quality of the source of the metric being used?

2) Is the exposure measured in the study a surrogate for the exposure?

3) What was the temporal coverage (i.e., short or long-term exposure)?

4) Did the analysis account for prediction uncertainty?

5) How was missing data accounted for, and any data imputations incorporated?

6) Were sensitivity analyses performed?

Were Outcome Assessment Methods Lacking Accuracy?

This item is similar to actual Navigation guidelines that require an assessment of the accuracy of the measured outcome.

Was Potential Confounding Inadequately Incorporated?

This is a very important issue for case reports/studies and case series. Case reports/studies and case series do not include controls and so, to be considered in a systematic review, these types of studies will need to be well-documented with respect to treatment or other contextual factors that may explain or influence the outcome. Prior to initiating the study screening, review team members should collectively generate a list of potential confounders that are based on expert opinion and knowledge gathered from the scientific literature:

Tier I: Important confounders

• Other associated treatment (i.e., intoxication, insufficient dose, history, or context)

• Medical history

Tier II: Other potentially important confounders and effect modifiers:

• Age, sex, country.

Were Incomplete Outcome Data Inadequately Addressed?

This item is similar to actual Navigation Guide instructions, though it may be very unlikely that outcome data would be incomplete in published case reports/studies and case series.

Does the Study Report Appear to Have Selective Outcome Reporting?

This item is similar to actual Navigation Guide instructions, though it may be very unlikely that there would be selective outcome reporting in published case reports/studies and case series.

Did the Study Receive Any Support From a Company, Study Author, or Other Entity Having a Financial Interest?

This item is similar to actual Navigation Guide instructions.

Did the Study Appear to Have Other Problems That Could Put It at a Risk of Bias?

Data synthesis criteria and summary measures if feasible.

Meta-analyses will be conducted using a random-effects model if studies are sufficiently homogeneous in terms of design and comparator. For dichotomous outcomes, effects of associations will be determined by using risk ratios (RR) or odds ratios (OR) with 95% confidence intervals (CI). Continuous outcomes will be analyzed using weighted mean differences (with 95% CI) or standardized mean differences (with 95% CI) if different measurement scales are used. Skewed data and non-quantitative data will be presented descriptively. Where data are missing, a request will be made to the original authors of the study to obtain the relevant missing data. If these data cannot be obtained, an imputation method will be performed. The statistical heterogeneity of the studies using the Chi Squared test (significance level: 0.1) and I 2 statistic (0–40%: might not be important; 30–60%: may represent moderate heterogeneity; 50–90%: may represent substantial heterogeneity; 75–100%: considerable heterogeneity). If there is heterogeneity, an attempt will be made to explain the source of this heterogeneity through a subgroup or sensitivity analysis.

Finally, the meta-analysis will be conducted in the latest version of the statistical software RevMan. The Mantel-Haenszel method will be used for the fixed effects model if tests of heterogeneity are not significant. If statistical heterogeneity is observed ( I 2 ≥ 50% or p < 0.1), the random effects model will be chosen. If quantitative synthesis is not feasible (e.g., if heterogeneity exists), a meta-analysis will not be performed and a narrative, qualitative summary of the study findings will be done.

Separate analyses will be conducted for the studies that contain control groups using expected mortality/morbidity, in order to include them in the quantitative synthesis of case reports/studies and case series.

If quantitative synthesis is not appropriate, a systematic narrative synthesis will be provided with information presented in the text and tables to summarize and explain the characteristics and findings of the included studies. The narrative synthesis will explore the relationship and findings both within and between the included studies.

Possible Additional Analyses

If feasible, subgroup analyses will be used to explore possible sources of heterogeneity, if there is evidence for differences in effect estimates by country, study design, or patient characteristics (e.g., sex and age). In addition, sensitivity analysis will be performed to explore the source of heterogeneity as for example, published vs. unpublished data, full-text publications vs. abstracts, risk of bias (by omitting studies that are judged to be at high risk of bias).

Overall Quality of Evidence Assessment

The quality of evidence will be assessed using an adapted version of the Evidence Quality Assessment Tool in the Navigation Guide. This tool is based on the GRADE approach ( 1 ). The assessment will be conducted by two teams, again blinded to each other, one that has the results of the case reports/studies and case series/control synthesis, the other without.

Data synthesis will be conducted independently by the classical and case teams. Evidence ratings will start at “high” for randomized control studies, “moderate” for observational studies, and “low” for case reports/studies and case series . It is important to be clear that sufficient levels of evidence cannot be achieved without study comparators. With regards to case reports/studies and case series, we classify these as starting at the lowest point of evidence and therefore we cannot consider evidence higher than low for these kinds of studies. Complete instructions for making quality of evidence judgments are presented in Supplementary Material .

Synthesis of Strength of Evidence

The standard Navigation Guide methodology will be applied to rate the strength of recommendations. The classical and case teams, blinded to the results from each other during the process, will independently assess the strength of evidence. The evidence quality ratings will be translated into strength of evidence for each population based on a combination of four criteria: (a) Quality of body of evidence; (b) Direction of effect; (c) Confidence in effect; and (d) Other compelling attributes of the data that may influence certainty. The ratings for strength of evidence will be “sufficient evidence of harmfulness,” “limited of harmfulness,” “inadequate of harmfulness” and “evidence of lack of harmfulness.”

Once we complete the synthesis of case reports/studies and case series, findings of this separate evidence stream will only be considered if RCTs and observational studies are not available. They will not be used to upgrade or downgrade the strength of other evidence streams.

To the best of our knowledge, this protocol is one of the first to specifically address the incorporation of case reports/studies and case series in a systematic review ( 9 ). The protocol was adapted from the Navigation Guide with the intent of integrating the case reports/studies and case series in systematic review recommendations, while following traditional systematic review methodology to the greatest extent possible. To be included, these case report/studies and case series will need to be well-documented, scientifically rigorous, and follow ethical practices. In addition, we believe that some case reports/studies and case series might bring relevant knowledge that should be considered in systematic review recommendations when data from RCT's and observational studies are not available, especially when even a small number of studies report an important and possibly causal association in an epidemic or a side effect of a newly marketed medicine. Our methodology will be the first to effectively incorporate case reports/studies and case series in systematic reviews that synthesize evidence for clinicians, researchers, and drug developers. These types of studies will be incorporated mostly through paper selection and risk of bias assessments. In addition, we will conduct meta-analyses if the eligible studies provide sufficient data.

This protocol has limitations related primarily to the constraints of case reports/studies and case series. These are descriptive studies. In addition, a case series is subject to selection bias because the clinician or researcher selects the cases themselves and may represent outliers in clinical practice. Furthermore, this kind of study does not have a control group, so it is not possible to compare what happens to other people who do not have the disease or receive treatment. These sources of bias mean that reported results may not be generalizable to a larger patient population and therefore cannot generate information on incidences or prevalence rates and ratios ( 22 , 23 ). However, it is important to note that promoting the need to synthesize these types of studies (case reports/studies and case series) in a formal systematic review, should not deter or delay immediate action from being taken when a few small studies report a plausible causal association between exposure and disease, such as, in the event of an epidemic or a side effect of a newly marketed medicine ( 23 ). In this study protocol, we will not consider animal studies that might give relevant toxicological information because we are focusing on study areas where a paucity of information exists. Finally, we must note that, case reports/studies and case series do not provide independent proof, and therefore, the findings of this separate evidence stream (case reports/studies and case series) will only be considered if evidence from RCTs and observational studies is not available. Case reports/studies and case series will not be used to upgrade or downgrade the strength of other evidence streams. In any case, it is very important to remember that these kinds of studies (case reports/studies and case series) are there to quickly alert agencies of the need to take immediate action to prevent further harm.

Despite these limitations, case reports/studies and case series are a first line of evidence because they are where new issues and ideas emerge (hypothesis-generating) and can contribute to a change in clinical practice ( 23 – 25 ). We therefore believe that data from case reports/studies and case series, when synthesized and presented with completeness and transparency, may provide important details that are relevant to systematic review recommendations.

Author Contributions

AD and GS the protocol study was designed. JL, TW, and DM reviewed. MF, ALG, RV, NC, CB, GLR, MD, ML, and AN significant improvement was made. AN and AD wrote the manuscript. GS improved the language. All authors reviewed and commented on the final manuscript, read and approved the final manuscript to be published.

This project was supported by the French Pays de la Loire region and Angers Loire Métropole, University of Angers and Centre Hospitalo-Universitaire CHU Angers. The project is entitled TEC-TOP (no award/grant number).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2021.708380/full#supplementary-material

1. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. (2008) 336:924–6. doi: 10.1136/bmj.39489.470347.AD

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020) . Cochrane (2020). Available online at: www.training.cochrane.org/handbook

Google Scholar

3. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. (2009) 62:e1–34. doi: 10.1016/j.jclinepi.2009.06.006

4. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. (2015) 4:1. doi: 10.1186/2046-4053-4-1

5. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 : elaboration and explanation. BMJ . (2015) 350:g7647. doi: 10.1136/bmj.g7647

PubMed Abstract | CrossRef Full Text

6. Woodruff TJ, Sutton P, Navigation Guide Work Group. An evidence-based medicine methodology to bridge the gap between clinical and environmental health sciences. Health Aff (Millwood). (2011) 30:931–7. doi: 10.1377/hlthaff.2010.1219

7. Woodruff TJ, Sutton P. The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect. (2014) 122:1007–14. doi: 10.1289/ehp.1307175

8. Reeves BC, Deeks JJ, Higgins JPT, Shea B, Tugwell P, Wells GA. Chapter 24: Including non-randomized studies on intervention effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane (2020). Available online at: www.training.cochrane.org/handbook

9. Loke YK, Price D, Herxheimer A, the Cochrane Adverse Effects Methods Group. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. (2007) 7:32. doi: 10.1186/1471-2288-7-32

10. Lam J, Koustas E, Sutton P, Johnson PI, Atchley DS, Sen S, et al. The Navigation Guide - evidence-based medicine meets environmental health: integration of animal and human evidence for PFOA effects on fetal growth. Environ Health Perspect. (2014) 122:1040–51. doi: 10.1289/ehp.1307923

11. Peryer G, Golder S, Junqueira DR, Vohra S, Loke YK. Chapter 19: Adverse effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020) . Cochrane (2020). Available online at: www.training.cochrane.org/handbook

12. Gagnier JJ, Kienle G, Altman DG, Moher D, Sox H, Riley D, et al. The CARE guidelines: consensus-based clinical case reporting guideline development. J Med Case Rep. (2013) 7:223. doi: 10.1186/1752-1947-7-223

13. Riley DS, Barber MS, Kienle GS, Aronson JK, von Schoen-Angerer T, Tugwell P, et al. CARE guidelines for case reports: explanation and elaboration document. J Clin Epidemiol. (2017) 89:218–35. doi: 10.1016/j.jclinepi.2017.04.026

14. Moola S, Munn Z, Tufanaru C, Aromataris E, Sears K, Sfetcu R, et al. Chapter 7: Systematic reviews of etiology and risk. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI (2020). doi: 10.46658/JBIMES-20-08. Available online at: https://synthesismanual.jbi.global

CrossRef Full Text

15. Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evidence Synthesis. (2020) 18:2127–33. doi: 10.11124/JBISRIR-D-19-00099

16. Covidence systematic review software, V.H.I. Covidence Systematic Review Software , V.H.I. Melbourne, CA. Available online at: www.covidence.org ; https://support.covidence.org/help/how-can-i-cite-covidence

17. Drazen JM, de Leeuw PW, Laine C, Mulrow C, DeAngelis CD, Frizelle FA, et al. Toward More Uniform Conflict Disclosures: The Updated ICMJE Conflict of Interest Reporting Form. JAMA. (2010) 304:212. doi: 10.1001/jama.2010.918

18. Drazen JM, Weyden MBVD, Sahni P, Rosenberg J, Marusic A, Laine C, et al. Uniform Format for Disclosure of Competing Interests in ICMJE Journals. N Engl J Med. (2009) 361:1896–7. doi: 10.1056/NEJMe0909052

19. Johnson PI, Sutton P, Atchley DS, Koustas E, Lam J, Sen S, et al. The navigation guide—evidence-based medicine meets environmental health: systematic review of human evidence for PFOA effects on fetal growth. Environ Health Perspect. (2014) 122:1028–39. doi: 10.1289/ehp.1307893

20. Descatha A, Sembajwe G, Baer M, Boccuni F, Di Tecco C, Duret C, et al. WHO/ILO work-related burden of disease and injury: protocol for systematic reviews of exposure to long working hours and of the effect of exposure to long working hours on stroke. Environ Int. (2018) 119:366–78. doi: 10.1016/j.envint.2018.06.016

21. Lam J, Lanphear BP, Bellinger D, Axelrad DA, McPartland J, Sutton P, et al. Developmental PBDE exposure and IQ/ADHD in childhood: a systematic review and meta-analysis. Environ Health Perspect. (2017) 125:086001. doi: 10.1289/EHP1632

22. Hay JE, Wiesner RH, Shorter RG, LaRusso NF, Baldus WP. Primary sclerosing cholangitis and celiac disease. Ann Intern Med. (1988) 109:713–7. doi: 10.7326/0003-4819-109-9-713

23. Nissen T, Wynn R. The clinical case report: a review of its merits and limitations. BMC Res Notes. (2014) 7:264. doi: 10.1186/1756-0500-7-264

24. Buonfrate D, Requena-Mendez A, Angheben A, Muñoz J, Gobbi F, Van Den Ende J, et al. Severe strongyloidiasis: a systematic review of case reports. BMC Infect Dis. (2013) 13:78. doi: 10.1186/1471-2334-13-78

25. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, et al. Clinical Practice Guidelines We Can Trust . Washington, D.C.: National Academies Press (2011).

Keywords: toxicology, epidemiology, public health, protocol, systematic review, case reports/studies, case series

Citation: Nambiema A, Sembajwe G, Lam J, Woodruff T, Mandrioli D, Chartres N, Fadel M, Le Guillou A, Valter R, Deguigne M, Legeay M, Bruneau C, Le Roux G and Descatha A (2021) A Protocol for the Use of Case Reports/Studies and Case Series in Systematic Reviews for Clinical Toxicology. Front. Med. 8:708380. doi: 10.3389/fmed.2021.708380

Received: 19 May 2021; Accepted: 11 August 2021; Published: 06 September 2021.

Reviewed by:

Copyright © 2021 Nambiema, Sembajwe, Lam, Woodruff, Mandrioli, Chartres, Fadel, Le Guillou, Valter, Deguigne, Legeay, Bruneau, Le Roux and Descatha. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Aboubakari Nambiema, aboubakari.nambiema@univ-angers.fr ; orcid.org/0000-0002-4258-3764

Introduction to Systematic Reviews

• Reference work entry
• First Online: 20 July 2022
• pp 2159–2177
• Cite this reference work entry

• Tianjing Li 3 ,
• Ian J. Saldanha 4 &
• Karen A. Robinson 5  

215 Accesses

A systematic review identifies and synthesizes all relevant studies that fit prespecified criteria to answer a research question. Systematic review methods can be used to answer many types of research questions. The type of question most relevant to trialists is the effects of treatments and is thus the focus of this chapter. We discuss the motivation for and importance of performing systematic reviews and their relevance to trialists. We introduce the key steps in completing a systematic review, including framing the question, searching for and selecting studies, collecting data, assessing risk of bias in included studies, conducting a qualitative synthesis and a quantitative synthesis (i.e., meta-analysis), grading the certainty of evidence, and writing the systematic review report. We also describe how to identify systematic reviews and how to assess their methodological rigor. We discuss the challenges and criticisms of systematic reviews, and how technology and innovations, combined with a closer partnership between trialists and systematic reviewers, can help identify effective and safe evidence-based practices more quickly.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

AHRQ (2015) Methods guide for effectiveness and comparative effectiveness reviews. Available from https://effectivehealthcare.ahrq.gov/products/cer-methods-guide/overview . Accessed on 27 Oct 2019

Andersen MZ, Gülen S, Fonnes S, Andresen K, Rosenberg J (2020) Half of Cochrane reviews were published more than two years after the protocol. J Clin Epidemiol 124:85–93. https://doi.org/10.1016/j.jclinepi.2020.05.011

Article   Google Scholar  

Berkman ND, Lohr KN, Ansari MT, Balk EM, Kane R, McDonagh M, Morton SC, Viswanathan M, Bass EB, Butler M, Gartlehner G, Hartling L, McPheeters M, Morgan LC, Reston J, Sista P, Whitlock E, Chang S (2015) Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 68(11):1312–1324

Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545. https://doi.org/10.1136/bmjopen-2016-012545

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JP, Oliver S (2014) How to increase value and reduce waste when research priorities are set. Lancet 383(9912):156–165. https://doi.org/10.1016/S0140-6736(13)62229-1

Clarke M, Chalmers I (1998) Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 280(3):280–282

Cooper NJ, Jones DR, Sutton AJ (2005) The use of systematic reviews when designing studies. Clin Trials 2(3):260–264

Djulbegovic B, Kumar A, Magazin A, Schroen AT, Soares H, Hozo I, Clarke M, Sargent D, Schell MJ (2011) Optimism bias leads to inconclusive results-an empirical study. J Clin Epidemiol 64(6):583–593. https://doi.org/10.1016/j.jclinepi.2010.09.007

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 91:23–30

Equator Network. Reporting guidelines for systematic reviews. Available from https://www.equator-network.org/?post_type=eq_guidelines&eq_guidelines_study_design=systematic-reviews-and-meta-analyses&eq_guidelines_clinical_specialty=0&eq_guidelines_report_section=0&s=+ . Accessed 9 Mar 2020

Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ (2016) Panel for updating guidance for systematic reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ 354:i3507. https://doi.org/10.1136/bmj.i3507 . Erratum in: BMJ 2016 Sep 06 354:i4853

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) (2019a) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester

Google Scholar  

Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (2019b) Standards for the conduct of new Cochrane intervention reviews. In: JPT H, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (eds) Methodological expectations of Cochrane intervention reviews. Cochrane, London

IOM (2011) Committee on standards for systematic reviews of comparative effectiveness research, board on health care services. In: Eden J, Levit L, Berg A, Morton S (eds) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC

Jonnalagadda SR, Goyal P, Huffman MD (2015) Automating data extraction in systematic reviews: a systematic review. Syst Rev 4:78

Krnic Martinic M, Pieper D, Glatt A, Puljak L (2019) Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol 19(1):203. Published 4 Nov 2019. https://doi.org/10.1186/s12874-019-0855-0

Lasserson TJ, Thomas J, Higgins JPT (2019) Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (1992) Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327(4):248–254

Lau J (2019) Editorial: systematic review automation thematic series. Syst Rev 8(1):70. Published 11 Mar 2019. https://doi.org/10.1186/s13643-019-0974-z

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100. https://doi.org/10.1371/journal.pmed.1000100

Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I (2016) Towards evidence based research. BMJ 355:i5440. https://doi.org/10.1136/bmj.i5440

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC (2018) Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods 9(4):602–614. https://doi.org/10.1002/jrsm.1287

Michelson M, Reuter K (2019) The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun 16:100443. https://doi.org/10.1016/j.conctc.2019.100443 . Erratum in: Contemp Clin Trials Commun 2019 16:100450

Moher D, Liberati A, Tetzlaff J (2009) Altman DG; PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269. W64

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4(1):1. https://doi.org/10.1186/2046-4053-4-1

NIHR HTA Stage 1 guidance notes. Available from https://www.nihr.ac.uk/documents/hta-stage-1-guidance-notes/11743 ; Accessed 10 Mar 2020

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D (2016) Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 13(5):e1002028. https://doi.org/10.1371/journal.pmed.1002028

Page MJ, Higgins JPT, Sterne JAC (2019) Chapter 13: assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ et al (eds) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester, pp 349–374

Chapter   Google Scholar  

Robinson KA (2009) Use of prior research in the justification and interpretation of clinical trials. Johns Hopkins University

Robinson KA, Goodman SN (2011) A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 154(1):50–55. https://doi.org/10.7326/0003-4819-154-1-201101040-00007

Rouse B, Cipriani A, Shi Q, Coleman AL, Dickersin K, Li T (2016) Network meta-analysis for clinical practice guidelines – a case study on first-line medical therapies for primary open-angle glaucoma. Ann Intern Med 164(10):674–682. https://doi.org/10.7326/M15-2367

Saldanha IJ, Lindsley K, Do DV et al (2017) Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol 135(9):933–940. https://doi.org/10.1001/jamaophthalmol.2017.2583

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 147(4):224–233

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355:i4919. https://doi.org/10.1136/bmj.i4919

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT (2019) RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366:l4898. https://doi.org/10.1136/bmj.l4898

Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S (2019) Chapter 2: determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

USPSTF U.S. Preventive Services Task Force Procedure Manual (2017). Available from: https://www.uspreventiveservicestaskforce.org/uspstf/sites/default/files/inline-files/procedure-manual2017_update.pdf . Accessed 21 May 2020

Whitaker (2015) UCSF guides: systematic review: when will i be finished? https://guides.ucsf.edu/c.php?g=375744&p=3041343 , Accessed 13 May 2020

Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J (2016) Churchill R; ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69:225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005

Download references

Author information

Authors and affiliations.

Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

Tianjing Li

Department of Health Services, Policy, and Practice and Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA

Ian J. Saldanha

Department of Medicine, Johns Hopkins University, Baltimore, MD, USA

Karen A. Robinson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tianjing Li .

Editor information

Editors and affiliations.

Department of Surgery, Division of Surgical Oncology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Steven Piantadosi

Department of Epidemiology, School of Public Health, Johns Hopkins University, Baltimore, MD, USA

Curtis L. Meinert

Section Editor information

Department of Epidemiology, University of Colorado Denver Anschutz Medical Campus, Aurora, CO, USA

The Johns Hopkins Center for Clinical Trials and Evidence Synthesis, Johns Hopkins University, Baltimore, MD, USA

Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Cite this entry.

Li, T., Saldanha, I.J., Robinson, K.A. (2022). Introduction to Systematic Reviews. In: Piantadosi, S., Meinert, C.L. (eds) Principles and Practice of Clinical Trials. Springer, Cham. https://doi.org/10.1007/978-3-319-52636-2_194

Download citation

DOI : https://doi.org/10.1007/978-3-319-52636-2_194

Published : 20 July 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-52635-5

Online ISBN : 978-3-319-52636-2

eBook Packages : Mathematics and Statistics Reference Module Computer Science and Engineering

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 17 October 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesise the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimise research b ias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinised by others.
• They’re thorough : they summarise all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomised control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective(s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgement of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

• Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2022, October 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 23, Issue 2
• Methodological quality and synthesis of case series and case reports
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-5502-5975 Mohammad Hassan Murad 1 ,
• Shahnaz Sultan 2 ,
• Samir Haffar 3 ,
• Fateh Bazerbachi 4
• 1 Evidence-Based Practice Center, Mayo Clinic , Rochester , Minnesota , USA
• 2 Division of Gastroenterology, Hepatology, and Nutrition , University of Minnesota, Center for Chronic Diseases Outcomes Research, Minneapolis Veterans Affairs Healthcare System , Minneapolis , Minnesota , USA
• 3 Digestive Center for Diagnosis and Treatment , Damascus , Syrian Arab Republic
• 4 Department of Gastroenterology and Hepatology , Mayo Clinic , Rochester , Minnesota , USA
• Correspondence to Dr Mohammad Hassan Murad, Evidence-Based Practice Center, Mayo Clinic, Rochester, MN 55905, USA; murad.mohammad{at}mayo.edu

https://doi.org/10.1136/bmjebm-2017-110853

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• epidemiology

In 1904, Dr James Herrick evaluated a 20-year-old patient from Grenada who was studying in Chicago and suffered from anaemia and a multisystem illness. The patient was found to have ‘freakish’ elongated red cells that resembled a crescent or a sickle. Dr Herrick concluded that the red cells were not artefacts because the appearance of the cells was maintained regardless of how the smear slide was prepared. He followed the patient who had subsequently received care from other physicians until 1907 and questioned whether this was syphilis or a parasite from the tropics. Then in 1910, in a published case report, he concluded that this presentation strongly suggested a previously unrecognised change in the composition of the corpuscle itself. 1 Sickle cell disease became a diagnosis thereafter.

Case reports and case series have profoundly influenced the medical literature and continue to advance our knowledge in the present time. In 1985, the American Medical Association reprinted 51 papers from its journal that had significantly changed the science and practice of medicine over the past 150 years, and five of these papers were case reports. 2 However, concerns about weak inferences and the high likelihood of bias associated with such reports have resulted in minimal attention being devoted to developing frameworks for approaching, appraising, synthesising and applying evidence derived from case reports/series. Nevertheless, such observations remain the bread and butter of learning by pattern recognition and integral to advancing medical knowledge.

Guidance on how to write a case report is available (ie, a reporting guideline). The Case Report (CARE) guidelines 3 were developed following a three-phase consensus process and provide a 13-item checklist that can assist researchers in publishing complete and meaningful exposition of medical information. This checklist encourages the explicit presentation of patient information, clinical findings, timeline, diagnostic assessment, therapeutic interventions, follow-up and outcomes. 3 Yet, systematic reviewers appraising the evidence for decision-makers require tools to assess the methodological quality (risk of bias assessment) of this evidence.

In this guide, we present a framework to evaluate the methodological quality of case reports/series and synthesise their results, which is particularly important when conducting a systematic review of a body of evidence that consists primarily of uncontrolled clinical observations.

Definitions

In the biomedical published literature, a case report is the description of the clinical course of one individual, which may include particular exposures, symptoms, signs, interventions or outcomes. A case report is the smallest publishable unit in the literature, whereas case series report aggregates individual cases in one publication. 4

If a case series is prospective, differentiating it from a single-arm uncontrolled cohort study becomes difficult. In one clinical practice guideline, it was proposed that studies without internal comparisons can be labelled as case series unless they explicitly report having a protocol before commencement of data collection, a definition of inclusion and exclusion criteria, a standardised follow-up and clear reporting of the number of excluded patients and those lost to follow-up. 6

Evaluating methodological quality

Pierson 7 provided an approach to evaluate the validity of a case report based on five components: documentation, uniqueness, objectivity, interpretation and educational value, resulting in a score with a maximum of 10 (a score above 5 was suggested indicate a valid case report). This approach, however, was rarely used in subsequent work and seems to conflate methodological quality with other constructs. For case reports of adverse drug reactions, other systems classify an association as definite, probable, possible or doubtful based on leading questions. 8 9 These questions are derived from the causality criteria that was established in 1965 by the English epidemiologist Bradford Hills. 10 Lastly, we have adapted the Newcastle Ottawa scale 11 for cohort and case–control studies by removing items that relate to comparability and adjustment (which are not relevant to non-comparative studies) and retained items that focused on selection, representativeness of cases and ascertainment of outcomes and exposure. This tool was applied in several published systematic reviews with good inter-rater agreement. 12–16

Proposed tool

The previous criteria from Pierson, 7 Bradford Hills 10 and Newcastle Ottawa scale modifications 11 converge into eight items that can be categorised into four domains: selection, ascertainment, causality and reporting. The eight items with leading explanatory questions are summarised in table 1 .

• View inline

Tool for evaluating the methodological quality of case reports and case series

For example, a study that explicitly describes all the cases who have presented to a medical centre over a certain period of time would satisfy the selection domain. In contrast, a study that reports on several individuals with unclear selection approach leaves the reader with uncertainty to whether this is the whole experience of the researchers and suggests possible selection bias. For the domain of ascertainment, self-report (of the exposure or the outcome) is less reliable than ascertainment using administrative and billing codes, which in turn is less reliable than clinical records. For the domain of causality, we would have stronger inference in a case report of an adverse drug reaction that has resolved with cessation of the drug and reoccurred after reintroduction of the drug. Lastly, for the domain of reporting, a case report that is described with sufficient details may allow readers to apply the evidence derived from the report in their practice. On the other hand, an inadequately reported case will likely be unhelpful in the course of clinical care.

We suggest using this tool in systematic reviews of case reports/series. One option to summarise the results of this tool is to sum the scores of the eight binary responses into an aggregate score. A better option is not to use an aggregate score because numeric representation of methodological quality may not be appropriate when one or two questions are deemed most critical to the validity of a report (compared with other questions). Therefore, we suggest making an overall judgement about methodological quality based on the questions deemed most critical in the specific clinical scenario.

Synthesis of case reports/series

A single patient case report does not allow the estimation of an effect size and would only provide descriptive or narrative results. Case series of more than one patient may allow narrative or quantitative synthesis.

Narrative synthesis

A systematic review of the cases with the rare syndrome of lipodystrophy was able to suggest core and supportive clinical features and narratively summarised data on available treatment approaches. 17 Another systematic review of 172 cases of the infrequently encountered glycogenic hepatopathy was able to characterise for the first time patterns of liver enzymes and hepatic injury in this disease. 18

Quantitative synthesis

Quantitative analysis of non-comparative series does not produce relative association measures such as ORs or relative risks but can provide estimates of prevalence or event rates in the form of a proportion (with associated precision). Proportions can be pooled using fixed or random effects models by means of the various available meta-analysis software. For example, a meta-analysis of case series of patients presenting with aortic transection showed that mortality was significantly lower in patients who underwent endovascular repair, followed by open repair and non-operative management (9%, 19% and 46%, respectively, P<0.01). 19

A common challenge, however, occurs when proportions are too large or too small (close to 0 or to 1). In this situation, the variance of the proportion becomes very small leading to an inappropriately large weight in meta-analysis. One way to overcome this challenge is to transform prevalence to a variable that is not constrained to the 0–1 range and has approximately normal distribution, conduct the meta-analysis and then transform the estimate back to a proportion. 20 This is done using logit transformation or using the Freeman-Tukey double arcsine transformation, 21 with the latter being often preferred. 20

Another type of quantitative analysis that may be utilised is regression. A meta-analysis of 47 published cases of hypocalcaemia and cardiac dysfunction used univariate linear regression analysis to demonstrate that both QT interval and left ventricular ejection fraction were significantly correlated with corrected total serum calcium level. 22 Meta-regression, which is a regression in which the unit of analysis is a study, not a patient, can also be used to synthesise case series and control for study-level confounders. A meta-regression analysis of uncontrolled series of patients with uveal melanoma treated with proton beam therapy has shown that this treatment was associated with better outcomes than brachytherapy. 23 It is very important, however, to recognise that meta-regression results can be severely affected by ecological bias.

From evidence to decision

Several authors have described various important reasons to publish case reports/series ( table 2 ). 7 24 25

Role of case reports/series in the medical literature

It is paramount to recognise that a systematic review and meta-analysis of case reports/series should not be placed at the top of the hierarchy in a pyramid that depicts validity. 26 The certainty of evidence derived from a meta-analysis is contingent on the design of included studies, their risk of bias, as well as other factors such as imprecision, indirectness, inconsistency and likelihood of publication bias. 27 Commonly, certainty in evidence derived from case series/reports will be very low. Nevertheless, inferences from such reports can be used for decision-making. In the example of case series of aortic transection showing lower mortality with endovascular repair, a guideline recommendation was made stating ‘We suggest that endovascular repair be performed preferentially over open surgical repair or non-operative management’. This was graded as a weak recommendation based on low certainty evidence. 28 The strength of this recommendation acknowledged that the recommendation might not universally apply to everyone and that variability in decision-making was expected. The certainty in evidence rating of this recommendation implied that future research would likely yield different results that may change the recommendation. 28

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach clearly separates the certainty of evidence from the strength of recommendation. This separation allows decision-making based on lower levels of evidence. For example, despite low certainty evidence (derived from case series) regarding the association between aspirin and Reye’s syndrome in febrile children, a strong recommendation for using acetaminophen over aspirin is possible. 29 GRADE literature also describes five paradigmatic situations in which a strong recommendation can be made based on low quality evidence. 30 One of which is when the condition is life threatening. An example of which would be using hyperbaric oxygen therapy for purpura fulminans, which is only based on case reports. 31

Guideline developers and decision-makers often struggle when dealing with case reports/case series. On occasions, they ignore such evidence and focus the scope of guidelines on areas with higher quality evidence. Sometimes they label recommendations based on case reports as expert opinion. 32 We propose an approach to evaluate the methodological quality of case reports/series based on the domains of selection, ascertainment, causality and reporting and provide signalling questions to aid evidence-based practitioners and systematic reviewers in their assessment. We suggest the incorporation of case reports/series in decision-making based on the GRADE approach when no other higher level of evidence is available.

In this guide, we have made the case for publishing case reports/series and proposed synthesis of their results in systematic reviews to facilitate using this evidence in decision-making. We have proposed a tool that can be used to evaluate the methodological quality in systematic reviews that examine case reports and case series.

• Gagnier JJ ,
• Altman DG , et al
• Grimes DA ,
• Abu-Zidan FM ,
• Schünemann HJ ,
• Naranjo CA ,
• Sellers EM , et al
• 9. ↵ The World health Organization-Uppsala Monitoring Centre . The use of the WHO-UMC system for standardised case causality assessment . https://www.who-umc.org/media/2768/standardised-case-causality-assessment.pdf ( accessed 20 Sep 2017 ).
• O’Connell D , et al
• Bazerbachi F ,
• Prokop L , et al
• Vargas EJ , et al
• Watt KD , et al
• Szarka LA , et al
• Hussain MT , et al
• Farah W , et al
• Leise MD , et al
• Malgor R , et al
• Barendregt JJ ,
• Lee YY , et al
• Freeman MF ,
• Newman DB ,
• Fidahussein SS ,
• Kashiwagi DT , et al
• Schild SE , et al
• Vandenbroucke JP
• Alsawas M , et al
• Matsumura JS ,
• Mitchell RS , et al
• Guyatt GH ,
• Vist GE , et al
• Domecq JP ,
• Murad MH , et al
• Mestrovic J , et al
• Alvarez-Villalobos N ,
• Shah R , et al
• Conboy EE ,
• Mounajjed T , et al
• Gottlieb MS
• Coodin FJ ,
• Uchida IA ,
• Goldfinger SE
• Lennox BR ,
• Jones PB , et al

Contributors MHM drafted the paper and all coauthors critically revised the manuscript. All the authors contributed to conceive the idea and approved the final submitted version.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Study Design 101

• Helpful formulas
• Finding specific study types
• Systematic Review
• Meta- Analysis
• Practice Guideline
• Randomized Controlled Trial
• Cohort Study
• Case Control Study
• Case Reports

A document often written by a panel that provides a comprehensive review of all relevant studies on a particular clinical or health-related topic/question. The systematic review is created after reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings.

• Exhaustive review of the current literature and other sources (unpublished studies, ongoing research)
• Less costly to review prior studies than to create a new study
• Less time required than conducting a new study
• Results can be generalized and extrapolated into the general population more broadly than individual studies
• More reliable and accurate than individual studies
• Considered an evidence-based resource

Disadvantages

• Very time-consuming
• May not be easy to combine studies

Design pitfalls to look out for

Studies included in systematic reviews may be of varying study designs, but should collectively be studying the same outcome.

Is each study included in the review studying the same variables?

Some reviews may group and analyze studies by variables such as age and gender; factors that were not allocated to participants.

Do the analyses in the systematic review fit the variables being studied in the original studies?

Fictitious Example

Does the regular wearing of ultraviolet-blocking sunscreen prevent melanoma? An exhaustive literature search was conducted, resulting in 54 studies on sunscreen and melanoma. Each study was then evaluated to determine whether the study focused specifically on ultraviolet-blocking sunscreen and melanoma prevention; 30 of the 54 studies were retained. The thirty studies were reviewed and showed a strong positive relationship between daily wearing of sunscreen and a reduced diagnosis of melanoma.

Real-life Examples

Yang, J., Chen, J., Yang, M., Yu, S., Ying, L., Liu, G., ... Liang, F. (2018). Acupuncture for hypertension. The Cochrane Database of Systematic Reviews, 11 (11), CD008821. https://doi.org/10.1002/14651858.CD008821.pub2

This systematic review analyzed twenty-two randomized controlled trials to determine whether acupuncture is a safe and effective way to lower blood pressure in adults with primary hypertension. Due to the low quality of evidence in these studies and lack of blinding, it is not possible to link any short-term decrease in blood pressure to the use of acupuncture. Additional research is needed to determine if there is an effect due to acupuncture that lasts at least seven days.

Parker, H.W. and Vadiveloo, M.K. (2019). Diet quality of vegetarian diets compared with nonvegetarian diets: a systematic review. Nutrition Reviews , https://doi.org/10.1093/nutrit/nuy067

This systematic review was interested in comparing the diet quality of vegetarian and non-vegetarian diets. Twelve studies were included. Vegetarians more closely met recommendations for total fruit, whole grains, seafood and plant protein, and sodium intake. In nine of the twelve studies, vegetarians had higher overall diet quality compared to non-vegetarians. These findings may explain better health outcomes in vegetarians, but additional research is needed to remove any possible confounding variables.

Related Terms

Cochrane Database of Systematic Reviews

A highly-regarded database of systematic reviews prepared by The Cochrane Collaboration , an international group of individuals and institutions who review and analyze the published literature.

Exclusion Criteria

The set of conditions that characterize some individuals which result in being excluded in the study (i.e. other health conditions, taking specific medications, etc.). Since systematic reviews seek to include all relevant studies, exclusion criteria are not generally utilized in this situation.

Inclusion Criteria

The set of conditions that studies must meet to be included in the review (or for individual studies - the set of conditions that participants must meet to be included in the study; often comprises age, gender, disease type and status, etc.).

Now test yourself!

1. Systematic Reviews are similar to Meta-Analyses, except they do not include a statistical analysis quantitatively combining all the studies.

a) True b) False

2. The panels writing Systematic Reviews may include which of the following publication types in their review?

a) Published studies b) Unpublished studies c) Cohort studies d) Randomized Controlled Trials e) All of the above

← Previous Next →

© 2011-2019, The Himmelfarb Health Sciences Library Questions? Ask us .

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .

• Methodology
• Open access
• Published: 01 February 2018

Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools

• Christian Kohl   ORCID: orcid.org/0000-0001-6524-8371 1   na1 ,
• Emma J. McIntosh 2   na1 ,
• Stefan Unger 1   na1 ,
• Neal R. Haddaway 3 ,
• Steffen Kecke 1 ,
• Joachim Schiemann 1 &
• Ralf Wilhelm 1  

Environmental Evidence volume  7 , Article number:  8 ( 2018 ) Cite this article

35k Accesses

149 Citations

31 Altmetric

Metrics details

A Correction to this article was published on 27 March 2018

This article has been updated

Systematic reviews and systematic maps represent powerful tools to identify, collect, evaluate and summarise primary research pertinent to a specific research question or topic in a highly standardised and reproducible manner. Even though they are seen as the “gold standard” when synthesising primary research, systematic reviews and maps are typically resource-intensive and complex activities. Thus, managing the conduct and reporting of such reviews can become a time consuming and challenging task. This paper introduces the open access online tool CADIMA, which was developed through a collaboration between the Julius Kühn-Institut and the Collaboration for Environmental Evidence, in order to increase the efficiency of the evidence synthesis process and facilitate reporting of all activities to maximise methodological rigour. Furthermore, we analyse how CADIMA compares with other available tools by providing a comprehensive summary of existing software designed for the purposes of systematic review management. We show that CADIMA is the only available open access tool that is designed to: (1) assist throughout the systematic review/map process; (2) be suited to reviews broader than medical sciences; (3) allow for offline data extraction; and, (4) support working as a review team.

Systematic reviews were first established in the field of healthcare to support evidence-based decision making [ 1 ]. Their use is continuously expanding into other disciplines, including social welfare, international development, education, crime and justice, Footnote 1 environmental management Footnote 2 (including the impact assessment of crop genetic improvement technologies [ 2 , 3 , 4 ]), software engineering [ 5 ] and food/feed safety assessment [ 6 ]. Systematic reviews and related systematic maps follow standardised and rigorous methodologies aiming to ensure comprehensiveness, minimise bias, and increase transparency [ 7 , 8 ]. Although seen as a “gold standard” when synthesising primary research, the central tenets of systematic review and map methodologies necessarily increase the complexity of the review processes and their resource requirements (i.e. time, money and personnel).

In order to support reviewers throughout the conduct of their syntheses, and to increase efficiency and maximise methodological rigour, software tools have been developed by a diverse set of providers to support review teams during the evidence synthesis process (the term evidence synthesis is used herein to cover both systematic reviews and systematic maps, which aim to characterise the available evidence-base rather than providing quantitative or qualitative answers to an impact or effectiveness question [ 8 , 9 ]).

Potential drawbacks associated with these tools include that: (1) they may not be open access (i.e. free to use, an important consideration for non-profit organisations in particular); (2) they may be targeted to a particular research discipline, meaning that their applicability in other disciplines may be restricted; (3) they may not support the entire evidence synthesis process; and, (4) they may have been developed solely for systematic reviews and may not support the conduct of systematic maps.

Here, we present the open access online tool CADIMA that was established by Julius Kühn-Institut (JKI) during a recently completed EU-funded project called GMO Risk Assessment and Communication of Evidence (GRACE). The project’s working agenda included: (1) the conduct of a number of systematic reviews and maps for the purposes of increasing the transparency and traceability of information on potential risks and benefits associated with the deliberate release of genetically modified crops [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ]; and, (2) the development of an open access online tool (CADIMA) to facilitate the conduct of systematic reviews and maps on agricultural and environmental questions. Due to the expertise available at the Collaboration for Environmental Evidence (CEE) and the overlap of topics covered by both institutions, a close collaboration between JKI and CEE was established to develop CADIMA.

Herein, we discuss how CADIMA compares with other available tools by providing a comprehensive summary of existing review management software, and also discuss possible future development of CADIMA. Existing reviews of available software and tools (e.g. [ 18 ]), have quickly become out of date since many new software packages have been recently released or are in development. In order to ensure the independence of the review reported in this manuscript and the assessment of how CADIMA compares to existing tools, the review part of this paper was solely conducted by EJM as she was/is not involved in the development of CADIMA.

Review of existing online tools

A series of searches was conducted for the purposes of comparing CADIMA with other available online tools to identify software packages designed to facilitate evidence synthesis. We excluded software that only supported isolated aspects of, rather than the majority of, the systematic review process (e.g. reference management in endnote, duplicate checking using the systematic review accelerator [ 19 , 20 ], screening in Abstrackr [ 21 ], meta-analysis in comprehensive meta-analysis (CMA), or data extraction and quantitative synthesis in RobotReviewer [ 22 ]). For more details on these and other tools, see the SR toolbox: http://systematicreviewtools.com/ .

The search strategy involved four approaches: (1) conducting online bibliographic database searches; (2) snowballing via general web searches (tracking backwards and forwards for studies via links in relevant websites); (3) screening targeted websites; and, (4) backwards and forwards citation searches of relevant publications (search methods are outlined in Additional file 1 ). Following the completion of the searches, 24 systematic review software packages were identified from across a wide range of disciplines (Table  1 ). Of these, two were excluded from the analysis; one has been discontinued (Slrtool [ 23 ]), and the developers of another product currently in development, DRAGON ONLINE ( https://www.icf.com/solutions-and-apps/dragon-online-tool-systematic-review ), did not respond to our request for further information.

The 22 remaining software packages were researched and trialed by EJM (where free access or free trials were available) and characterised according to a suite of features, including; the stages of the systematic review process supported, whether they are suitable for a team of reviewers, and their cost (Table  1 ). These features were chosen in part based on previous studies on user preferences for systematic review software functionality [ 24 , 25 ]. Developers were contacted when insufficient information was available online or in publications about a software package. Where no further information was available, the characteristic was marked as ‘Unavailable’.

Introduction to CADIMA

CADIMA is a client–server software application and was developed by using the interactive management framework Scrum ( http://www.scrumguides.org/ ) and the project management tool Redmine ( http://www.redmine.org/ ). The user interface of the CADIMA web application requires a web browser, such as Mozilla Firefox or Google Chrome. CADIMA is coded with the programming language PHP V5.5 using the Yii V1.1 framework with the Bootstrap CSS extension ( http://yiibooster.clevertech.biz/ ). The application runs on an Apache 2.4 web server and a Linux Ubuntu Server V14.04, and data are stored in a MySQL 5.5 database management system with a daily data backup stored for 6 months. CADIMA is permanently hosted and maintained by JKI and uses a SSL encrypted connection between the client and server.

The support provided by CADIMA mirrors the key steps of systematic reviews or systematic maps. CADIMA supports the following: (1) development of the review protocol; (2) management of search results (including the identification of duplicates); (3) management and conduct of the study selection process (including the performance of a consistency check); (4) management and conduct of on- and off-line data extraction; and, (5) management and conduct of the critical appraisal process. In addition, CADIMA ensures thorough documentation of the entire evidence synthesis process and allows for review results to be made publicly available: i.e. documents can be made accessible to third parties if agreed by the review team. The permanent maintenance and further development of CADIMA is guaranteed by JKI and user support is provided to review teams via email. Furthermore, users can participate in online workshops or experiment using a test website before creating a full review.

In the following pages, we briefly describe CADIMA’s main features, starting from the registration and customisation of a review and its team, to the conduct and documentation of the evidence synthesis process. In addition, we describe and summarise the different tasks within the review team and the information formats that are currently supported during the evidence synthesis process (see Table  2 ).

Registering with CADIMA and user roles

Users must register with the program in order to access the full functionality of CADIMA, which is free of charge. Footnote 3 By accepting CADIMA’s terms of service that regulate, besides others, the use of CADIMA and the handling of data (see Additional file 2 ), any registered user can initiate a new systematic review or map and can customise the review team. There are two different roles in a review team implemented in CADIMA. The ‘review coordinator’ manages the review and its team, and also performs more general tasks when compared to the one or more ‘review team members’ (see Table  2 ). Only the nominated members of the respective review team and the review coordinator can access the new evidence synthesis.

Structure of CADIMA

The menu structure of CADIMA mirrors the core steps and workflow of systematic reviews and systematic maps. This begins with the development of the review protocol (including the development of the review question), followed by the conduct of the literature search, study selection, data extraction, critical appraisal, data synthesis and the presentation of results. For each menu item, explanatory notes and submenus are provided. We now go on to explain the functionality of the different menu items in more detail.

Review protocol

At this stage, review authors are requested to detail information regarding the planned methods for the review, ensuring scientific rigour, transparency and repeatability. The input to CADIMA is provided by uploading remotely prepared blocks of text that correspond to key sections of a protocol. The overall format implemented in CADIMA resembles the draft of a protocol and has two major benefits: (1) it prevents important information from being unintentionally omitted; and (2) it facilitates peer-review of the protocol by ensuring that relevant information is included in the most appropriate section. Furthermore, CADIMA combines the respective text and generates one single document, which can then be formatted by the review team and submitted for peer-review.

Literature search

CADIMA is not a meta-search engine, such as PubMed or Scopus. Instead, CADIMA helps to structure and document the literature search by associating a search string with a search engine or further information source it was applied to, whilst the respective search results can be uploaded to CADIMA as RIS files. Following this, search results can be combined, duplicates removed and records screened (see below). In addition, to facilitate the study selection process at title/abstract stage, CADIMA highlights those reports where an abstract is missing.

Study selection

The study selection step includes the following key aspects: (1) definition of selection criteria; (2) automated calculation of a kappa-statistic to test inter-reviewer agreement Footnote 4 when applying the defined criteria; (3) screening of the records from the literature list according to the selection criteria at title, abstract and full text stage; and, (4) extraction of studies from eligible records (an important step that recognises the difference between a study [i.e. an independent unit of research] and an article [i.e. an independent unit of publication]). During the screening process, title, abstract and full text are displayed together with the selection criteria during each respective stage. Where records are independently assessed by more than one reviewer and inconsistencies between reviewers occur, they will be automatically identified by CADIMA and the respective reviewers asked to solve those conflicts.

Data extraction and critical appraisal

CADIMA is designed to encourage best practice in systematic reviewing, such as the requirement that reviewers specify their critical appraisal criteria prior to data extraction. Critical appraisal criteria can refer to a specific bias under assessment (i.e. the internal validity of a study) and/or the generalisability of a study (i.e. its external validity). In addition, the critical appraisal judgement system (i.e. whether a distinction will be made between low, medium, high and unclear risk, or only between low, high and unclear risk etc.) and items for data extraction (i.e. which data should be extracted) must be defined. The data extraction sheet will automatically be generated by CADIMA and the reviewer can mark those data that are needed to inform critical appraisal.

CADIMA allows users to conduct either on- or off-line extraction of data and meta-data, Footnote 5 by either directly entering information into CADIMA or by providing a download of the data extraction sheet as a spreadsheet file that can be uploaded once extraction is complete.

During critical appraisal, the appraisal criteria are used to assess the validity of included studies. CADIMA allows users to undertake critical appraisal online, while the extracted data relevant to the critical appraisal are shown together with the appraisal criteria. Where inconsistencies in coding decisions occur between two independent reviewers for one record, these will be automatically identified by CADIMA, and the respective reviewers are asked to resolve those conflicts.

Flexibility provided by CADIMA

CADIMA allows review steps to be modified and/or updated during the conduct of the review, with the exception of the selection criteria, since a change in the selection criteria would require the de novo performance of the consistency check and all previously extracted information would be lost. The core steps do not need to be undertaken in order: for example search results can still be entered once the selection process has started, and the selection process does not need to be completed in order to start the data extraction or critical appraisal steps.

To support data synthesis activities, CADIMA provides the completed data extraction sheet and the results from the critical appraisal, as spreadsheets that facilitate data transfer and preparation for quantitative synthesis. These files can then be used by the review team to perform statistical analyses within the software package of their choice, such as R ( https://cran.r-project.org/ ).

Presenting data and results

CADIMA facilitates thorough documentation of the review process, providing, besides others, the following information and data formats:

a flow diagram summarising the study selection process, satisfying PRISMA standards Footnote 6 (docx),

reference lists for each database (xlsx) and the final reference list after duplicate removal (xlsx and RIS),

the outcomes of the consistency check and study selection across the different stages (title, abstract and full text) including the reasons for exclusion (xlsx),

the results of the critical appraisal (xlsx),

the filled data extraction sheet (xlsx).

Furthermore, CADIMA offers the possibility of uploading results generated by the review team, to make synthesis results available to third parties, i.e. displaying the documents on the web site and enable external users to download them. These features encourage a higher level of transparency than is common in publish systematic reviews.

CADIMA and other types of evidence synthesis

CADIMA is also suitable for assisting in the process of conducting other forms of evidence synthesis, including systematic maps [ 8 , 9 ] and rapid reviews [ 26 ] since not all steps of a systematic review have to be completed within the program. Consequently, the data extraction sheet can be designed to house meta-data only, and the critical appraisal step can be skipped completely if deemed necessary by the review authors.

Review of existing tools

Of the 22 software packages identified as being suitable to support the systematic review or systematic map process, nine were advertised as suitable for users from any field of research, nine were designed for the health care and medical science sectors, three were designed primarily for software engineering and one for experimental animal studies (Fig.  1 ). The programs vary in terms of available support, and most offered graphical user interfaces (GUI), although four required prior knowledge of coding or software development to use. Web-based functions were available for 15 of the packages and seven involved downloadable applications. Most packages were designed for a team of reviewers, an important consideration given many guidelines require more than one reviewer to be involved with screening (e.g. [ 7 ]). However, two packages did not provide this functionality. Of the primary stages of the systematic review process we identified, most software packages had the capacity to address article screening (most enabling title and/or title and abstract screening in addition to full text screening) (Table  3 ).

Breakdown of the intended fields of research each of the 22 software packages were primarily designed for

Machine learning and text mining features for use during screening, data extraction or synthesis stages are in their infancy, with only 10 software packages currently supporting or planning to support their use. To date these approaches have been incorporated into these tools in various ways, for example by assisting with article screening (e.g. Rayyan and EPPI-Reviewer), data extraction (e.g. METAGEAR package for R), and risk of bias assessments (e.g. SyRF). For further information about how text mining approaches have been effectively applied to systematic reviews, and more information about their potential future applications, see [ 27 , 28 ]. Encouragingly, 16 software packages are freely available for non-commercial uses, and six are also open source. All of the software we assessed are available to use in English, although several lacked help documentation in English as they were designed primarily for use in another language (e.g. [ 29 ]). Furthermore, some programs have advanced capabilities to manage articles in other languages and other character sets (e.g. DistillerSR).

During trialing of the software packages (summarised in Table  1 ), several general issues were noted. Most software packages lacked customisability; this was often to ensure compliance with specific existing guidelines or protocols within a particular discipline area (e.g. the Kitchenham guidelines for systematic reviews in software engineering [ 5 ]). This limits the degree to which many of the software packages can be used between disciplines. Most of the software packages differ in the types of input files they accept, and many only accept one type of input file (e.g. PubMed output files). The most common file type is RIS. This is problematic in interdisciplinary studies when importing studies from a wide range of sources and grey literature databases, many of which do not provide standardised export features (e.g. Google Scholar https://scholar.google.co.uk/ , EU Joint Research Centre—Publications Repository http://publications.jrc.ec.europa.eu/repository , OECD iLibrary http://www.oecd-ilibrary.org/ ). To help address this, EPPI-Reviewer developers have designed a RIS converter to convert other file formats such as CSV files to RIS format ( http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2934 ).

Duplicate checking is an increasingly common feature (Table  3 ) that can provide valuable time savings, particularly if duplicate detection can be partially automated (e.g. EPPI-Reviewer). Automated import of abstracts and full-text PDFs is also an important time-saving feature in larger studies, but is not yet widely available (and is difficult when many studies are not open access, as in the field of conservation biology).

Discussion and outlook

There is increasing demand for information management systems which assist with the centralisation and management of the systematic review process, to improve efficiency and to facilitate teams of reviewers to collaborate. We have identified 22 software packages which provide this functionality, designed for users from a wide range of disciplines. There is a large degree of overlap between many of these software packages, however most have been developed with particular disciplines in mind and lack the customisability suitable for access and use by reviewers across disciplines. As a general observation, many developers appear to have developed these tools without an awareness of the full range of similar tools available (a point also noted in a recent systematic review [ 27 ]).

EJM (who was not part of the development team) trialled CADIMA and found it intuitive to use and noted it performed smoothly even with large datasets. A major benefit of CADIMA is the fact it is suitable for teams (vital for reviewers following certain guidelines e.g. [ 7 ]) and is free and well supported—an important consideration for students, small organisations and not-for-profits (even low monthly fees are barriers, as the typical review process can take over a year). CADIMA also offers greater security than traditional approaches to review management, such as Microsoft Excel, when it comes to sorting records and tracing included articles between different stages of the screening and data extraction process. The ability to export files and work offline easily with CADIMA was considered a great asset, although the linear structure of the application has so far precluded adjustments to review team membership between screening stages. The developers have taken this into consideration for future developments of the programme. As CADIMA combines many different stages of the review process in a single piece of software, it also has the advantage of enhancing transparency and replicability.

CADIMA is designed to provide important information to users in the form of prompts, which make the difference between a rigorous systematic review and a standard literature review, considerably reducing the barrier to entry for first time reviewers. These include protocol development prompts which mirror Collaboration for Environmental Evidence guidelines, and stages such as consistency checking. The structure and layout of CADIMA encourages users to document their methodology and screening criteria clearly, and also provides a location for record and methods to be hosted online, so that subsequent revisions can be undertaken easily.

Like CADIMA, the majority of software packages support teams of reviewers, require no prior coding knowledge and offer a range of help and support, facilitating rapid learning and working with a team of individuals with differing degrees of experience. A handful of tools are particularly designed to lead the user in a stepwise manner through the review process, including CADIMA with its inbuilt guidance and clear layout, and SESRA [ 29 ], which mirrors the stages in the Kitchenham and Charters guidelines [ 5 ]. Others, such as EPPI-Reviewer, do not follow this structured approach, and users design the stages according to their needs, meaning they must be familiar with both the software and systematic review methodology.

No single software package guides the reviewer through all stages of a systematic review or map project (from question formation to the exporting of project documentation), meaning stages such as literature searches or analysis and writing up of results are often expected to be managed separately. This is also true for CADIMA, which provides support for the majority of the stages we assessed (Table  3 ), excluding built-in searching and quantitative synthesis. Just over half of the software packages are integrated with one or more publication databases to allow for built-in searching, however this inevitably limited them to certain databases and their associated disciplines, such as PubMed (medical and healthcare evidence, https://www.ncbi.nlm.nih.gov/pubmed/ ) in the case of DistillerSR, SRDB.PRO, SWIFT-Review and SyRF.

The principal advantage of using software to assist in managing the review process is to increase efficiency of time consuming tasks, to allow for efforts to be concentrated on the most important tasks—namely synthesis and analysis. CADIMA facilitates the importing and exporting of the results of searching and synthesis to allow literature searches and statistical analysis to be conducted flexibly, using alternative software, and focuses on simplifying the tracking large numbers of review articles throughout the process.

Future developments of CADIMA

Based on the results of the conducted review and received user feedback, the following issues will be considered during the next round of development for CADIMA:

To facilitate the exchange between CADIMA and different reference sources, additional input formats will be catered for, rather than RIS files only;

Duplicates are detectable within CADIMA, but cannot be automatically removed in the current version. This can be quite time consuming in cases where many duplicates are identified. In such cases, review teams can automatically delete duplicates, for example by using EndNote and import the cleared list to CADIMA. In the future, an automated removal process will be implemented to CADIMA;

In order to speed up the study selection process at title/abstract stage, text mining approaches will be tested and potentially implemented in the event a demonstrably robust method is developed (currently the software RapidMiner Footnote 7 is used to trial the use of text mining during the selection process);

To increase the time savings offered by CADIMA, an automated upload of PDFs at full-text screening stage is planned;

Currently, the same reviewers have to participate during the study selection process at title, abstract and full text stage. In the future, the possibility will be provided that different reviewers can be involved during the respective stages; and

Due to the limitations associated with the conduct of a full systematic review, further evidence synthesis approaches, such as rapid reviews, are evolving in order to save resources and to provide a timely answer to a posed question [ 26 , 30 ]. This is especially important in the political context where time is a major consideration. A future goal for CADIMA is to allow people to customise their review, depending on the purpose of the synthesis and available resources.

CADIMA will continue to be developed to join several other software packages which make use of machine learning approaches to increase efficiency at the article screening stages of the systematic review process. This is an area that we believe will be of increasing interest to users, particularly for updating existing reviews (algorithms can be trained to identify relevant studies based on similarity to previously included studies) [ 31 ] and dealing with very large bodies of literature.

The use of new technology to assist the systematic review process is a rapidly developing area, demonstrated by the inclusion of three new or upgraded software packages expected to become live in 2017 in our review (plus another we were unable to find further information on; DRAGON ONLINE). Several other packages which came up in our search have been discontinued, suggesting security of funding, ongoing maintenance and continual improvement are essential considerations for the developers of these types of software packages to prevent them quickly becoming obsolete.

Conclusions

From a user perspective, we believe that CADIMA stands out in terms of ease of use, support for multiple users, support for on- or off-line data extraction, commitment to ongoing maintenance and financing, therefore meeting the criteria rated as most important by users of systematic review software in a recent study [ 25 ]. Many other free software packages require prior experience of software development and computer coding, or have limited capacity for ongoing maintenance. Aside from CADIMA, those that are continually updated and provide user-friendly graphical user interfaces, tend to be expensive for team reviews, making them less feasible options for small research teams or non-profit organisations.

Change history

27 march 2018.

The authors wish to update information about the software DistillerSR in Tables 1 and 3 which we were alerted to following the publication of this article. In addition to the analysis provided, DistillerSR does support protocol development (Pi) e.g. assistance to determine appropriate PICO elements, and critical appraisal (Cr) as ‘stages of the SR process supported’. This information was not originally included in the assessment due to a lack of clarity on the service providers’ website. No further updates to this manuscript will be possible for this or other software, in line with the general disclaimer below. General disclaimer: The review of systematic review support software represents an independent assessment by EJ McIntosh based on publicly available information on each software package. This assessment represents an attempt to best capture information located via service providers’ websites, in academic publications, user manuals and via free trials or software demonstrations. Occasionally, relevant information was not publicly available or may have been difficult to access or interpret. This assessment does not represent the views or opinions of any of the software developers or service providers. The review of software was completed in mid-2017, readers should visit the software providers’ websites (linked in Table 1) to check for updates, for further information and to seek clarification where necessary.

http://www.campbellcollaboration.org/ .

http://www.environmentalevidence.org/ .

Accessible via https://www.cadima.info/index.php/area/evidenceSynthesisDatabase .

http://handbook.cochrane.org/ (part 2, chapter 7.2.6).

Meta-data are descriptive information relating to where and how a study was performed.

http://www.prisma-statement.org/PRISMAStatement/FlowDiagram.aspx .

https://rapidminer.com/ .

Guyatt G. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268:2420–5.

Article   Google Scholar  

(EU) IR. Commission Implementing Regulation (EU) No. 503/2013 on applications for authorisation of genetically modified food and feed in accordance with Regulation (EC) No. 1829/2003 of the European Parliament and of the Council and amending Commission Regulations (EC) No. 641/2004 and (EC) No. 1981/2006. 2013, OJ L 157; 2013. p. 1–48.

Kohl C, Craig W, Frampton G, Garcia-Yi J, van Herck K, Kleter GA, Krogh PH, Meissle M, Romeis J, Spök A. Developing a good practice for the review of evidence relevant to GMO risk assessment. GMOs Integr Plant Prod. 2013;97:55–62.

Google Scholar  

Kohl C, Frampton G, Sweet J, Spök A, Haddaway NR, Wilhelm R, Unger S, Schiemann J. Can systematic reviews inform GMo risk assessment and risk management? Front Bioeng Biotechnol. 2015;3:113.

Kitchenham B, Charters S. Guidelines for performing systematic literature reviews in software engineering version 2.3. EBSE Tech Rep. 2007;1–65.

EFSA. Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA J. 2010;8(6):1637.

CEE. Guidelines for systematic review and evidence synthesis in environmental management. Version 4.2; 2013. p. 1–80.

James KL, Randall NP, Haddaway NR. A methodology for systematic mapping in environmental sciences. Environ Evid. 2016;5:7.

Bragge P, Clavisi O, Turner T, Tavender E, Collie A, Gruen RL. The global evidence mapping initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11:92.

Gathmann A, Priesnitz KU. What is the evidence on the inheritance of resistance alleles in populations of lepidopteran/coleopteran maize pest species: a systematic map protocol. Environ Evid. 2014;3:13.

Gathmann A, Priesnitz KU. How susceptible are different lepidopteran/coleopteran maize pests to Bt-proteins: a systematic review protocol. Environ Evid. 2014;3:12.

Priesnitz KU, Vaasen A, Gathmann A. Baseline susceptibility of different European lepidopteran and coleopteran pests to Bt proteins expressed in Bt maize: a systematic review. Environ Evid. 2016;5:27.

Meissle M, Naranjo SE, Kohl C, Riedel J, Romeis J. Does the growing of Bt maize change abundance or ecological function of non-target animals compared to the growing of non-GM maize? A systematic review protocol. Environ Evid. 2014;3:7.

Kostov K, Damgaard CF, Hendriksen NB, Sweet JB, Krogh PH. Are population abundances and biomasses of soil invertebrates changed by Bt crops compared with conventional crops? A systematic review protocol. Environ Evid. 2014;3:10.

Kostov K, Krogh PH, Damgaard CF, Sweet JB, Hendriksen NB. Are soil microbial endpoints changed by Bt crops compared with conventional crops? A systematic review protocol. Environ Evid. 2014;3:11.

Sweet J, Kostov K. What are the effects of the cultivation of GM herbicide tolerant crops on botanical diversity? A systematic review protocol. A systematic review protocol. Environ Evid. 2014;3:8.

Garcia-Yi J, Lapikanonth T, Vionita H, Vu H, Yang S, Zhong Y, Li Y, Nagelschneider V, Schlindwein B, Wesseler J. What are the socio-economic impacts of genetically modified crops worldwide? A systematic map protocol. Environ Evid. 2014;3:24.

Marshall C, Brereton P. Tools to support systematic literature reviews in software engineering: a feature analysis. In: International Symposium on Empirical Software Engineering and Measurement; 2014. p. 296–9.

Practice CfRiEB. The systematic review accelerator. Centre for Research in Evidence Based Practice; 2017.

Rathbone J, Carter M, Hoffmann T, Glasziou P. Better duplicate detection for systematic reviewers: evaluation of systematic review assistant-deduplication module. Syst Rev. 2015;4(1):6.

Wallace BC, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence-based practice center: abstrackr. In: Proceedings of the ACM International Health Informatics Symposium (IHI); 2012. p. 819–24.

Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc. 2016;23:193–201.

Barn BS, Raimondi F, Athappian L, Clark T. Slrtool: a tool to support collaborative systematic literature reviews. In: Proceedings of the 16th International Conference on Enterprise Information Systems (ICEIS-2014). Science and Technology Publications, Lda.; 2014. p. 440–7.

Hassler E, Carver JC, Hale D, Al-Zubidy A. Identification of SLR tool needs—results of a community workshop. Inf Softw Technol. 2016;70:122–9.

Marshall C, Brereton P, Kitchenham B. Tools to support systematic reviews in software engineering: a cross-domain survey using semi-structured interviews. In: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering—EASE ‘15; 2015. p. 1–6.

Collins A, Coughlin D, Miller J, Kirk S. The production of quick scoping reviews and rapid evidence assessments: a how to guide. London: Joint Water Evidence Group; 2015.

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5.

Thomas J, McNaught J, Ananiadou S. Applications of text mining within systematic reviews. Res Synth Methods. 2011;2:1–14.

Molléri JS, Benitti FBV. ARS—Uma abordagem para automatização de revisões sistemáticas da literatura em engenharia de software: Relatório Técnico. Itajaí, Brazil; 2013.

Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10.

Roll U, Correia RA, Berger-Tal O. Using machine learning to disentangle homonyms in large text corpora. Conserv Biol. 2017. https://doi.org/10.1111/cobi.13044 .

CADIMA. Quedlinburg, Germany: Julius Kühn-Institut; 2017.

Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation.

DistillerSR. Ottawa, Canada: Evidence Partners.

Glujovsky D, Bardach A, García Martí S, Comandé D, Ciapponi A. EROS: a new software for early stage of systematic reviews. Value Health. 2011;14:A564.

Thomas J, Brunton J, Graziosi S. EPPI-reviewer 4: software for research synthesis. EPPI-Centre Software. London: Social Science Research Unit, Institute of Education; 2010.

HAWC. Health Assessment Workplace Collaborative. 2013.

Shapiro A, Rusyn I. Health assessment workspace collaborative (HAWC) project overview; 2014.

Lajeunesse MJ. Facilitating systematic reviews, data extraction, and meta-analysis with the METAGEAR package for R. Methods Ecol Evol. 2015;7:323–30.

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A, Chalmers T, Smith H, Blackburn B, Silverman B, Schroeder B, Reitman D, et al. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5:210.

Review Manager (RevMan) Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration; 2014.

Fernández-Sáez AM, Genero Bocco M, Romero FP. SLR-Tool a tool for performing systematic literature reviews. In: ICSOFT 2010—Proceedings of the 5th International Conference on Software and Data Technologies. 2010; 2:157–66.

Bowes D, Hall T, Beecham S. SLuRp: a tool to help large complex systematic literature reviews deliver valid and rigorous results. In: Proceedings of the 2nd international workshop on Evidential assessment of software technologies—EAST ‘12; 2012. p. 33–6.

Fabbri S, Silva C, Hernandes E, Octaviano F, Di Thommazo A, Belgamo A. Improvements in the StArt tool to better support the systematic review process. In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering—EASE ‘16 2016. p. 1–5.

Howard BE, Phillips J, Miller K, Tandon A, Mav D, Shah MR, Holmgren S, Pelch KE, Walker V, Rooney AA, et al. SWIFT-Review: a text-mining workbench for systematic review. Syst Rev. 2016;5:87.

Systematic Review and Meta-Analysis Facility (Syrf). Edinburgh, UK: CAMARADES-NC3Rs; 2017.

Download references

Authors’ contributions

CK drafted the CADIMA part of the manuscript, EJM performed the review of software tools and drafted the associated parts of the manuscript, SU is responsible for the programming of CADIMA and all authors contributed to the final manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors wish to thank Simone Frenzel for her help during the development of CADIMA, GRACE team members for their input and Andrew Pullin for his support when establishing the collaboration between JKI and CEE.

Competing interests

The authors declare that they have no competing interests. The review of software packages was not conducted by the authors responsible for developing CADIMA to ensure the independence of this analysis.

Availability of data and materials

Not applicable.

Consent for publication

Ethics approval and consent to participate.

This work received funding by the EU-FP7 project: GMO Risk Assessment and Communication of Evidence (GRACE); Grant Agreement KBBE-2011-6-311957.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Christian Kohl, Emma J. McIntosh and Stefan Unger contributed equally to this work

Authors and Affiliations

Julius Kühn-Institut (JKI), Federal Research Centre for Cultivated Plants, Erwin-Baur-Strasse 27, 06484, Quedlinburg, Germany

Christian Kohl, Stefan Unger, Steffen Kecke, Joachim Schiemann & Ralf Wilhelm

School of Geography and the Environment, University of Oxford, South Parks Road, Oxford, OX1 3QY, UK

Emma J. McIntosh

Mistra EviEM, Stockholm Environment Institute, 10451, Stockholm, Sweden

Neal R. Haddaway

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Christian Kohl .

Additional information

A correction to this article is available online at https://doi.org/10.1186/s13750-018-0124-4 .

Additional files

Additional file 1..

Search strategy for identifying software programs listed in Table  1 .

Additional file 2.

CADIMA terms of service.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Kohl, C., McIntosh, E.J., Unger, S. et al. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ Evid 7 , 8 (2018). https://doi.org/10.1186/s13750-018-0115-5

Download citation

Received : 25 November 2016

Accepted : 05 January 2018

Published : 01 February 2018

DOI : https://doi.org/10.1186/s13750-018-0115-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Review management
• Managing systems
• Systematic review software
• Evidence synthesis
• Time management
• Rapid review
• Text mining

Environmental Evidence

ISSN: 2047-2382

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Evidence-Based Medicine
• Finding the Evidence
• eJournals for EBM

Levels of Evidence

• JAMA Users' Guides
• Tutorials (Learning EBM)
• Web Resources

Resources That Rate The Evidence

• ACP Smart Medicine
• Agency for Healthcare Research and Quality
• Clinical Evidence
• Cochrane Library
• Health Services/Technology Assessment Texts (HSTAT)
• PDQ® Cancer Information Summaries from NCI
• Trip Database

Critically Appraised Individual Articles

• Evidence-Based Complementary and Alternative Medicine
• Evidence-Based Dentistry
• Evidence-Based Nursing
• Journal of Evidence-Based Dental Practice

Grades of Recommendation

Critically-appraised individual articles and synopses include:

Filtered evidence:

• Level I: Evidence from a systematic review of all relevant randomized controlled trials.
• Level II: Evidence from a meta-analysis of all relevant randomized controlled trials.
• Level III: Evidence from evidence summaries developed from systematic reviews
• Level IV: Evidence from guidelines developed from systematic reviews
• Level V: Evidence from meta-syntheses of a group of descriptive or qualitative studies
• Level VI: Evidence from evidence summaries of individual studies
• Level VII: Evidence from one properly designed randomized controlled trial

Unfiltered evidence:

• Level VIII: Evidence from nonrandomized controlled clinical trials, nonrandomized clinical trials, cohort studies, case series, case reports, and individual qualitative studies.
• Level IX: Evidence from opinion of authorities and/or reports of expert committee

Two things to remember:

1. Studies in which randomization occurs represent a higher level of evidence than those in which subject selection is not random.

2. Controlled studies carry a higher level of evidence than those in which control groups are not used.

Strength of Recommendation Taxonomy (SORT)

• SORT The American Academy of Family Physicians uses the Strength of Recommendation Taxonomy (SORT) to label key recommendations in clinical review articles. In general, only key recommendations are given a Strength-of-Recommendation grade. Grades are assigned on the basis of the quality and consistency of available evidence.
• << Previous: eJournals for EBM
• Next: JAMA Users' Guides >>
• Last Updated: Jan 25, 2024 4:15 PM
• URL: https://guides.library.stonybrook.edu/evidence-based-medicine
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Cost-effectiveness of case management: a systematic review

Affiliation.

• 1 inav - Institute for Applied Health Services Research, Schiffbauerdamm 12, 10117 Berlin, Germany. Email: [email protected].
• PMID: 35852890
• DOI: 10.37765/ajmc.2022.89186

Objectives: In this time of aging and increasingly multimorbid populations, effective and efficient case management approaches play a crucial role in supporting patients who are navigating complex health care systems. Until now, no rigorous systematic review has synthesized studies about the cost-effectiveness of case management.

Study design: A systematic review was performed.

Methods: The bibliographic databases PubMed and CINAHL Plus were systematically searched using key blocks and synonyms of the terms case management, effectiveness, and costs. The methodological quality of the studies was assessed using the Consensus Health Economic Criteria list.

Results: A total of 29 studies were included. In 3 studies, the intervention was less effective and more costly than the control group and can therefore be considered not cost-effective. Two studies found that the intervention was less effective and less costly. A more effective and less costly intervention, and therefore a strong recommendation for case management, was found in 6 studies. In 17 studies, the intervention was more effective while being more costly. Nearly half of the studies met most of the quality criteria, with 16 or more points out of 19.

Conclusions: Existing studies often have adequate quality and, in many cases, show cost-effective or even cost-saving results. Case management appears to be a promising method to support patients facing complex care situations. However, variation among case management approaches is very high, and the topic needs further study to determine the most cost-effective way of providing such care coordination.

Publication types

• Systematic Review
• Research Support, Non-U.S. Gov't
• Cost-Benefit Analysis*

• Library Hours
• Strategic Plan
• Giving to the Libraries
• Jobs at the Libraries
• Find Your Librarian
• View All →
• Google Scholar
• Research Guides
• Textbook/Reserves
• Government Documents
• Get It For Me
• Print/Copy/Scan
• Renew Materials
• Study Rooms
• Use a Computer
• Borrow Tech Gear
• Student Services
• Faculty Services
• Users with Disabilities
• Visitors & Alumni
• Special Collections
• Find Information

Basics of Systematic Reviews

• About Systematic Review

Types of Reviews

Literature review.

Collects key sources on a topic and discusses those sources in conversation with each other

• Standard for research articles in most disciplines
• Tells the reader what is known, or not known, about a particular issue, topic, or subject
• Demonstrates knowledge and understanding of a topic
• Establishes context or background for a case or argument
• Helps develop the author’s ideas and perspective

Rapid Review

Thorough methodology but with process limitations in place to expeditethe completion of a review.

• For questions that require timely answers
• 3-4 months vs. 12-24 months
• Limitations - scope, comprehensiveness bias, and quality of appraisal
• Discusses potential effects that the limited methods may have had on results

Scoping Review

Determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview of its focus.

• Identify types of available evidence in a given field
• Clarify key concepts/definitions in the literature
• Examine how research is conducted on a certain topic or field
• Identify key factors related to a concept
• Key difference is focus
• Identify and analyze knowledge gaps

Systematic Review

Attempts to identify, appraise, and summarize all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question.

• clearly defined question with inclusion/exclusion criteria
• rigorous and systematic search of the literature
• thorough screening of results
• data extraction and management
• analysis and interpretation of results
• risk of bias assessment of included studies

Meta-Analysis

Used to systematically synthesize or merge the findings of single, independent studies, using statistical methods to calculate an overall or ‘absolute’ effect.

• Combines results from multiple empirical studies
• Requires systematic review first
• Use well recognized, systematic methods to account for differences in sample size, variability (heterogeneity) in study approach and findings (treatment effects)
• Test how sensitive their results are to their own systematic review protocol

For additional types of reviews please see these articles:

• Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019), Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J, 36: 202-222. https://doi.org/10.1111/hir.12276
• Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
• << Previous: About Systematic Review
• Next: Sources >>
• Last Updated: May 17, 2024 10:04 AM
• URL: https://libguides.utsa.edu/systematicreview
• Library Locations
• Staff Directory
• 508 Compliance
• Site Search
• © The University of Texas at San Antonio
• Information: 210-458-4011
• Campus Alerts
• Required Links
• UTSA Policies
• Report Fraud

• Open access
• Published: 14 May 2024

Protocol for a scoping review study on learning plan use in undergraduate medical education

• Anna Romanova   ORCID: orcid.org/0000-0003-1118-1604 1 ,
• Claire Touchie 1 ,
• Sydney Ruller 2 ,
• Victoria Cole 3 &
• Susan Humphrey-Murto 4  

Systematic Reviews volume  13 , Article number:  131 ( 2024 ) Cite this article

Metrics details

The current paradigm of competency-based medical education and learner-centredness requires learners to take an active role in their training. However, deliberate and planned continual assessment and performance improvement is hindered by the fragmented nature of many medical training programs. Attempts to bridge this continuity gap between supervision and feedback through learner handover have been controversial. Learning plans are an alternate educational tool that helps trainees identify their learning needs and facilitate longitudinal assessment by providing supervisors with a roadmap of their goals. Informed by self-regulated learning theory, learning plans may be the answer to track trainees’ progress along their learning trajectory. The purpose of this study is to summarise the literature regarding learning plan use specifically in undergraduate medical education and explore the student’s role in all stages of learning plan development and implementation.

Following Arksey and O’Malley’s framework, a scoping review will be conducted to explore the use of learning plans in undergraduate medical education. Literature searches will be conducted using multiple databases by a librarian with expertise in scoping reviews. Through an iterative process, inclusion and exclusion criteria will be developed and a data extraction form refined. Data will be analysed using quantitative and qualitative content analyses.

By summarising the literature on learning plan use in undergraduate medical education, this study aims to better understand how to support self-regulated learning in undergraduate medical education. The results from this project will inform future scholarly work in competency-based medical education at the undergraduate level and have implications for improving feedback and supporting learners at all levels of competence.

Scoping review registration:

Open Science Framework osf.io/wvzbx.

Peer Review reports

Competency-based medical education (CBME) has transformed the approach to medical education to focus on demonstration of acquired competencies rather than time-based completion of rotations [ 1 ]. As a result, undergraduate and graduate medical training programs worldwide have adopted outcomes-based assessments in the form of entrustable professional activities (EPAs) comprised of competencies to be met [ 2 ]. These assessments are completed longitudinally by multiple different evaluators to generate an overall impression of a learner’s competency.

In CBME, trainees will progress along their learning trajectory at individual speeds and some may excel while others struggle to achieve the required knowledge, skills or attitudes. Therefore, deliberate and planned continual assessment and performance improvement is required. However, due to the fragmented nature of many medical training programs where learners rotate through different rotations and work with many supervisors, longitudinal observation is similarly fragmented. This makes it difficult to determine where trainees are on their learning trajectories and can affect the quality of feedback provided to them, which is a known major influencer of academic achievement [ 3 ]. As a result, struggling learners may not be identified until late in their training and the growth of high-performing learners may be stifled [ 4 , 5 , 6 ].

Bridging this continuity gap between supervision and feedback through some form of learner handover or forward feeding has been debated since the 1970s and continues to this day [ 5 , 7 , 8 , 9 , 10 , 11 ]. The goal of learner handover is to improve trainee assessment and feedback by sharing their performance and learning needs between supervisors or across rotations. However, several concerns have been raised about this approach including that it could inappropriately bias subsequent assessments of the learner’s abilities [ 9 , 11 , 12 ]. A different approach to keeping track of trainees’ learning goals and progress along their learning trajectories is required. Learning plans (LPs) informed by self-regulated learning (SRL) theory may be the answer.

SRL has been defined as a cyclical process where learners actively control their thoughts, actions and motivation to achieve their goals [ 13 ]. Several models of SRL exist but all entail that the trainee is responsible for setting, planning, executing, monitoring and reflecting on their learning goals [ 13 ]. According to Zimmerman’s SRL model, this process occurs in three stages: forethought phase before an activity, performance phase during an activity and self-reflection phase after an activity [ 13 ]. Since each trainee leads their own learning process and has an individual trajectory towards competence, this theory relates well to the CBME paradigm which is grounded in learner-centredness [ 1 ]. However, we know that medical students and residents have difficulty identifying their own learning goals and therefore need guidance to effectively partake in SRL [ 14 , 15 , 16 , 17 ]. Motivation has also emerged as a key component of SRL, and numerous studies have explored factors that influence student engagement in learning [ 18 , 19 ]. In addition to meeting their basic psychological needs of autonomy, relatedness and competence, perceived learning relevance through meaningful learning activities has been shown to increase trainee engagement in their learning [ 19 ].

LPs are a well-known tool across many educational fields including CBME that can provide trainees with meaningful learning activities since they help them direct their own learning goals in a guided fashion [ 20 ]. Also known as personal learning plans, learning contracts, personal action plans, personal development plans, and learning goals, LPs are documents that outline the learner’s roadmap to achieve their learning goals. They require the learner to self-identify what they need to learn and why, how they are going to do it, how they will know when they are finished, define the timeframe for goal achievement and assess the impact of their learning [ 20 ]. In so doing, LPs give more autonomy to the learner and facilitate objective and targeted feedback from supervisors. This approach has been described as “most congruent with the assumptions we make about adults as learners” [ 21 ].

LP use has been explored across various clinical settings and at all levels of medical education; however, most of the experience lies in postgraduate medical education [ 22 ]. Medical students are a unique learner population with learning needs that appear to be very well suited for using LPs for two main reasons. First, their education is often divided between classroom and clinical settings. During clinical training, students need to be more independent in setting learning goals to meet desired competencies as their education is no longer outlined for them in a detailed fashion by the medical school curriculum [ 23 ]. SRL in the workplace is also different than in the classroom due to additional complexities of clinical care that can impact students’ ability to self-regulate their learning [ 24 ]. Second, although most medical trainees have difficulty with goal setting, medical students in particular need more guidance compared to residents due to their relative lack of experience upon which they can build within the SRL framework [ 25 ]. LPs can therefore provide much-needed structure to their learning but should be guided by an experienced tutor to be effective [ 15 , 24 ].

LPs fit well within the learner-centred educational framework of CBME by helping trainees identify their learning needs and facilitating longitudinal assessment by providing supervisors with a roadmap of their goals. In so doing, they can address current issues with learner handover and identification as well as remediation of struggling learners. Moreover, they have the potential to help trainees develop lifelong skills with respect to continuing professional development after graduation which is required by many medical licensing bodies.

An initial search of the JBI Database, Cochrane Database, MEDLINE (PubMed) and Google Scholar conducted in July–August 2022 revealed a paucity of research on LP use in undergraduate medical education (UGME). A related systematic review by van Houten–Schat et al. [ 24 ] on SRL in the clinical setting identified three interventions used by medical students and residents in SRL—coaching, LPs and supportive tools. However, only a couple of the included studies looked specifically at medical students’ use of LPs, so this remains an area in need of more exploration. A scoping review would provide an excellent starting point to map the body of literature on this topic.

The objective of this scoping review will therefore be to explore LP use in UGME. In doing so, it will address a gap in knowledge and help determine additional areas for research.

This study will follow Arksey and O’Malley’s [ 26 ] five-step framework for scoping review methodology. It will not include the optional sixth step which entails stakeholder consultation as relevant stakeholders will be intentionally included in the research team (a member of UGME leadership, a medical student and a first-year resident).

Step 1—Identifying the research question

The overarching purpose of this study is to “explore the use of LPs in UGME”. More specifically we seek to achieve the following:

Summarise the literature regarding the use of LPs in UGME (including context, students targeted, frameworks used)

Explore the role of the student in all stages of the LP development and implementation

Determine existing research gaps

Step 2—Identifying relevant studies

An experienced health sciences librarian (VC) will conduct all searches and develop the initial search strategy. The preliminary search strategy is shown in Appendix A (see Additional file 2). Articles will be included if they meet the following criteria [ 27 ]:

Participants

Medical students enrolled at a medical school at the undergraduate level.

Any use of LPs by medical students. LPs are defined as a document, usually presented in a table format, that outlines the learner’s roadmap to achieve their learning goals [ 20 ].

Any stage of UGME in any geographic setting.

Types of evidence sources

We will search existing published and unpublished (grey) literature. This may include research studies, reviews, or expert opinion pieces.

Search strategy

With the assistance of an experienced librarian (VC), a pilot search will be conducted to inform the final search strategy. A search will be conducted in the following electronic databases: MEDLINE, Embase, Education Source, APA PsycInfo and Web of Science. The search terms will be developed in consultation with the research team and librarian. The search strategy will proceed according to the JBI Manual for Evidence Synthesis three-step search strategy for reviews [ 27 ]. First, we will conduct a limited search in two appropriate online databases and analyse text words from the title, abstracts and index terms of relevant papers. Next, we will conduct a second search using all identified key words in all databases. Third, we will review reference lists of all included studies to identify further relevant studies to include in the review. We will also contact the authors of relevant papers for further information if required. This will be an iterative process as the research team becomes more familiar with the literature and will be guided by the librarian. Any modifications to the search strategy as it evolves will be described in the scoping review report. As a measure of rigour, the search strategy will be peer-reviewed by another librarian using the PRESS checklist [ 28 ]. No language or date limits will be applied.

Step 3—Study selection

The screening process will consist of a two-step approach: screening titles/abstracts and, if they meet inclusion criteria, this will be followed by a full-text review. All screening will be done by two members of the research team and any disagreements will be resolved by an independent third member of the team. Based on preliminary inclusion criteria, the whole research team will first pilot the screening process by reviewing a random sample of 25 titles/abstracts. The search strategy, eligibility criteria and study objectives will be refined in an iterative process. We anticipate several meetings as the topic is not well described in the literature. A flowchart of the review process will be generated. Any modifications to the study selection process will be described in the scoping review report. The papers will be excluded if a full text is not available. The search results will be managed using Covidence software.

Step 4—Charting the data

A preliminary data extraction tool is shown in Appendix B (see Additional file 3 ). Data will be extracted into Excel and will include demographic information and specific details about the population, concept, context, study methods and outcomes as they relate to the scoping review objectives. The whole research team will pilot the data extraction tool on ten articles selected for full-text review. Through an iterative process, the final data extraction form will be refined. Subsequently, two members of the team will independently extract data from all articles included for full-text review using this tool. Charting disagreements will be resolved by the principal and senior investigators. Google Translate will be used for any included articles that are not in the English language.

Step 5—Collating, summarising and reporting the results

Quantitative and qualitative analyses will be used to summarise the results. Quantitative analysis will capture descriptive statistics with details about the population, concept, context, study methods and outcomes being examined in this scoping review. Qualitative content analysis will enable interpretation of text data through the systematic classification process of coding and identifying themes and patterns [ 29 ]. Several team meetings will be held to review potential themes to ensure an accurate representation of the data. The PRISMA Extension for Scoping Reviews (PRISMA-ScR) will be used to guide the reporting of review findings [ 30 ]. Data will be presented in tables and/or diagrams as applicable. A descriptive summary will explain the presented results and how they relate to the scoping review objectives.

By summarising the literature on LP use in UGME, this study will contribute to a better understanding of how to support SRL amongst medical students. The results from this project will also inform future scholarly work in CBME at the undergraduate level and have implications for improving feedback as well as supporting learners at all levels of competence. In doing so, this study may have practical applications by informing learning plan incorporation into CBME-based curricula.

We do not anticipate any practical or operational issues at this time. We assembled a team with the necessary expertise and tools to complete this project.

Availability of data and materials

All data generated or analysed during this study will be included in the published scoping review article.

Abbreviations

• Competency-based medical education

Entrustable professional activity

• Learning plan
• Self-regulated learning
• Undergraduate medical education

Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45.

Article   PubMed   Google Scholar  

Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019;53(8):766–77.

Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77(1):81–112.

Article   Google Scholar  

Dudek NL, Marks MB, Regehr G. Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005;80(10 Suppl):S84–7.

Warm EJ, Englander R, Pereira A, Barach P. Improving learner handovers in medical education. Acad Med. 2017;92(7):927–31.

Spooner M, Duane C, Uygur J, et al. Self-regulatory learning theory as a lens on how undergraduate and postgraduate learners respond to feedback: a BEME scoping review : BEME Guide No. 66. Med Teach. 2022;44(1):3–18.

Frellsen SL, Baker EA, Papp KK, Durning SJ. Medical school policies regarding struggling medical students during the internal medicine clerkships: results of a National Survey. Acad Med. 2008;83(9):876–81.

Humphrey-Murto S, LeBlanc A, Touchie C, et al. The influence of prior performance information on ratings of current performance and implications for learner handover: a scoping review. Acad Med. 2019;94(7):1050–7.

Morgan HK, Mejicano GC, Skochelak S, et al. A responsible educational handover: improving communication to improve learning. Acad Med. 2020;95(2):194–9.

Dory V, Danoff D, Plotnick LH, et al. Does educational handover influence subsequent assessment? Acad Med. 2021;96(1):118–25.

Humphrey-Murto S, Lingard L, Varpio L, et al. Learner handover: who is it really for? Acad Med. 2021;96(4):592–8.

Shaw T, Wood TJ, Touchie T, Pugh D, Humphrey-Murto S. How biased are you? The effect of prior performance information on attending physician ratings and implications for learner handover. Adv Health Sci Educ Theory Pract. 2021;26(1):199–214.

Artino AR, Brydges R, Gruppen LD. Chapter 14: Self-regulated learning in health professional education: theoretical perspectives and research methods. In: Cleland J, Duning SJ, editors. Researching Medical Education. 1st ed. John Wiley & Sons; 2015. p. 155–66.

Chapter   Google Scholar  

Cleland J, Arnold R, Chesser A. Failing finals is often a surprise for the student but not the teacher: identifying difficulties and supporting students with academic difficulties. Med Teach. 2005;27(6):504–8.

Reed S, Lockspeiser TM, Burke A, et al. Practical suggestions for the creation and use of meaningful learning goals in graduate medical education. Acad Pediatr. 2016;16(1):20–4.

Wolff M, Stojan J, Cranford J, et al. The impact of informed self-assessment on the development of medical students’ learning goals. Med Teach. 2018;40(3):296–301.

Sawatsky AP, Halvorsen AJ, Daniels PR, et al. Characteristics and quality of rotation-specific resident learning goals: a prospective study. Med Educ Online. 2020;25(1):1714198.

Article   PubMed   PubMed Central   Google Scholar  

Pintrich PR. Chapter 14: The role of goal orientation in self-regulated learning. In: Boekaerts M, Pintrich PR, Zeidner M, editors. Handbook of self-regulation. 1st ed. Academic Press; 2000. p. 451–502.

Kassab SE, El-Sayed W, Hamdy H. Student engagement in undergraduate medical education: a scoping review. Med Educ. 2022;56(7):703–15.

Challis M. AMEE medical education guide No. 19: Personal learning plans. Med Teach. 2000;22(3):225–36.

Knowles MS. Using learning contracts. 1 st ed. San Francisco: Jossey Bass; 1986.

Parsell G, Bligh J. Contract learning, clinical learning and clinicians. Postgrad Med J. 1996;72(847):284–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Teunissen PW, Scheele F, Scherpbier AJJA, et al. How residents learn: qualitative evidence for the pivotal role of clinical activities. Med Educ. 2007;41(8):763–70.

Article   CAS   PubMed   Google Scholar  

van Houten-Schat MA, Berkhout JJ, van Dijk N, Endedijk MD, Jaarsma ADC, Diemers AD. Self-regulated learning in the clinical context: a systematic review. Med Educ. 2018;52(10):1008–15.

Taylor DCM, Hamdy H. Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. Med Teach. 2013;35(11):e1561–72.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalol H. Chapter 11: Scoping reviews. In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. JBI; 2020. https://synthesismanual.jbi.global. . Accessed 30 Aug 2022.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Venables M, Larocque A, Sikora L, Archibald D, Grudniewicz A. Understanding indigenous health education and exploring indigenous anti-racism approaches in undergraduate medical education: a scoping review protocol. OSF; 2022. https://osf.io/umwgr/ . Accessed 26 Oct 2022.

Download references

Acknowledgements

Not applicable.

This study will be supported through grants from the Department of Medicine at the Ottawa Hospital and the University of Ottawa. The funding bodies had no role in the study design and will not have any role in the collection, analysis and interpretation of data or writing of the manuscript.

Author information

Authors and affiliations.

The Ottawa Hospital – General Campus, 501 Smyth Rd, PO Box 209, Ottawa, ON, K1H 8L6, Canada

Anna Romanova & Claire Touchie

The Ottawa Hospital Research Institute, Ottawa, Canada

Sydney Ruller

The University of Ottawa, Ottawa, Canada

Victoria Cole

The Ottawa Hospital – Riverside Campus, Ottawa, Canada

Susan Humphrey-Murto

You can also search for this author in PubMed   Google Scholar

Contributions

AR designed and drafted the protocol. CT and SH contributed to the refinement of the research question, study methods and editing of the manuscript. VC designed the initial search strategy. All authors reviewed the manuscript for final approval. The review guarantors are CT and SH. The corresponding author is AR.

Authors’ information

AR is a clinician teacher and Assistant Professor with the Division of General Internal Medicine at the University of Ottawa. She is also the Associate Director for the internal medicine clerkship rotation at the General campus of the Ottawa Hospital.

CT is a Professor of Medicine with the Divisions of General Internal Medicine and Infectious Diseases at the University of Ottawa. She is also a member of the UGME Competence Committee at the University of Ottawa and an advisor for the development of a new school of medicine at Toronto Metropolitan University.

SH is an Associate Professor with the Department of Medicine at the University of Ottawa and holds a Tier 2 Research Chair in Medical Education. She is also the Interim Director for the Research Support Unit within the Department of Innovation in Medical Education at the University of Ottawa.

CT and SH have extensive experience with medical education research and have numerous publications in this field.

SR is a Research Assistant with the Division of General Internal Medicine at the Ottawa Hospital Research Institute.

VC is a Health Sciences Research Librarian at the University of Ottawa.

SR and VC have extensive experience in systematic and scoping reviews.

Corresponding author

Correspondence to Anna Romanova .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. prisma-p 2015 checklist., 13643_2024_2553_moesm2_esm.docx.

Additional file 2: Appendix A. Preliminary search strategy [ 31 ].

Additional file 3: Appendix B. Preliminary data extraction tool.

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Romanova, A., Touchie, C., Ruller, S. et al. Protocol for a scoping review study on learning plan use in undergraduate medical education. Syst Rev 13 , 131 (2024). https://doi.org/10.1186/s13643-024-02553-w

Download citation

Received : 29 November 2022

Accepted : 03 May 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s13643-024-02553-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 17 May 2024

Immune checkpoint inhibitors and pericardial disease: a systematic review

• Sarah E. Mudra 1   na1 ,
• Danny L. Rayes 1   na1 ,
• Ankit Agrawal 2 ,
• Ashwin K. Kumar 1 , 2 ,
• Jason Z. Li 1 ,
• Meredith Njus 1 ,
• Kevin McGowan 1 ,
• Kazi A. Kalam 1 ,
• Charalompos Charalampous 1 ,
• Mary Schleicher 3 ,
• Muhammad Majid 2 ,
• Alvena Syed 2 ,
• Abdullah Yesilyaprak 2 &
• Allan L. Klein 2  

Cardio-Oncology volume  10 , Article number:  29 ( 2024 ) Cite this article

Metrics details

Introduction

Despite the growing use of immune checkpoint inhibitors (ICI) in cancer treatment, data regarding ICI-associated pericardial disease are primarily derived from case reports and case series. ICI related pericardial disease can be difficult to diagnose and is associated with significant morbidity. We conducted a systematic review to further characterize the epidemiology, clinical presentation, and outcomes of this patient population.

A search of four databases resulted in 31 studies meeting inclusion criteria. Patients > 18 years old who presented with ICI mediated pericardial disease were included. Intervention was medical + surgical therapy and outcomes were development of cardiac tamponade, morbidity, and mortality.

Thirty- eight patients across 31 cases were included. Patients were majority male (72%) with a median age of 63. Common symptoms included dyspnea (59%) and chest pain (32%), with 41% presenting with cardiac tamponade. Lung cancer (81%) was the most prevalent, and nivolumab (61%) and pembrolizumab (34%) were the most used ICIs. Pericardiocentesis was performed in 68% of patients, and 92% experienced symptom improvement upon ICI cessation. Overall mortality was 16%.

This study provides the most comprehensive analysis of ICI-mediated pericardial disease to date. Patients affected were most commonly male with lung cancer treated with either Nivolumab or Pembrolizumab. Diagnosis may be challenging in the setting of occult presentation with normal EKG and physical exam as well as delayed onset from therapy initiation. ICI-associated pericardial disease demonstrates high morbidity and mortality, as evidenced by a majority of patients requiring pericardiocentesis.

Immune checkpoint inhibitors (ICIs) have revolutionized cancer treatment, but they are associated with a range of immune-related adverse events (irAEs), including pericardial disease [ 1 ]. Despite the growing use of ICIs, data regarding ICI-associated pericardial disease are primarily derived from case reports and case series [ 2 ].

The exact mechanism of irAEs is incompletely understood currently. Recent studies suggest that both cardiac muscle and tumor cells have several high frequency T-cell receptor sequences in common suggesting a shared antigen theory [ 3 ]. However, others suggest that the development of myopericarditis may be due to underlying predisposing conditions that increase risk of development of disease [ 3 ].

Overall, given that there are no clear guidelines for diagnosis and treatment, and that delayed diagnosis portends an increased risk of mortality, further understanding of this complicated phenomenon is needed. Therefore, given the relative increase in ICI usage, we attempted to ascertain updated epidemiologic, clinical, and outcomes related data regarding this patient population.

In order to identify articles related to ICI-mediated pericarditis, a comprehensive search of the databases: Ovid Embase, Ovid Medline, Cochrane Register of Clinical Trials, and Web of Science were completed on May 16th, 2023. Overall, we focused our search on results from 2010 to present. A total of 1169 citations were uploaded into Covidence, the software program used to manage the screening process. After Covidence removed 350 exact duplicates, 819 citations remained for title & abstract screening. Eventually, 31 studies were included in final analysis. Two independent researchers (AK, AA) assessed and screened data in a method consistent with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PRISMA) (Fig.  1 ) [ 4 ]. Conflict resolution during the screening process was undertaken by another reviewer (DR). Letters to the editors, review articles, animal studies, populations including pediatric patients, and articles in languages other than English were excluded.

Prisma diagram demonstrating included studies

We employed the patient intervention, control, and outcome framework in our qualitative systematic. Our inclusion criteria included patients > 18 years old who presented with ICI mediated pericardial disease. The intervention employed was medical  ±  surgical therapy. The outcomes were development of cardiac tamponade, morbidity, and mortality. To our knowledge, no observational studies have been done previously, therefore, we did not include any comparison groups for analysis.

Included data was compiled for analysis. We did not include a comparison group given the nature of the study. Categorical variables were described using proportions (%). Continuous variables were described with mean or median. Statistical analysis was completed via SPSS 23.0 software (IBM Corp., Armonk, New York).

Epidemiology/clinical characteristics

In total, 31 cases encompassing 38 patients were included. Most patients were males ( n  = 31, 72%) with a median age of 63 (IQR: 55–69). The majority of cases identified were from the United States of America ( n  = 13, 30%) or Asia ( n  = 11, 25%). The most common presenting symptoms were dyspnea ( n  = 26, 59%), chest pain ( n  = 14, 32%), and bilateral leg edema ( n  = 2, 5%). Electrocardiogram (EKG) findings of pericarditis were present in 8 (19%) patients, whereas 1 (2%) patient presented with a pericardial rub. Concomitant myocardial involvement was noted in 7% ( n  = 3) of patients. Eighteen (41%) patients presented with cardiac tamponade (Table  1 ).

The most common cancers necessitating ICI therapy were lung adenocarcinoma ( n  = 23, 63%), lung squamous cell cancer ( n  = 7, 18%), melanoma ( n  = 3, 8%), and renal cell carcinoma ( n  = 2, 5%). The most common ICIs were nivolumab ( n  = 23, 61%), pembrolizumab ( n  = 13, 34%), and ipilimumab ( n  = 4, 10%). The median number of ICI cycles prior to symptom onset was 4 (IQR: 2–6). Prior chemotherapy had been undertaken in 66% ( n  = 25) of patients, while 35% ( n  = 13) had undergone prior radiation (Table  1 ).

The most frequent finding on Chest X-ray was enlarged cardiac silhouette ( n  = 6, 14%). Computed tomography (CT) findings demonstrated a pericardial effusion in 21 (60%) patients with the predominant size being large ( n  = 10, 25%). A total of 4 (9%) patients underwent cardiac magnetic resonance imaging (CMR) testing. Of these 4, 2 (5%) had delayed hyperenhancement and 1 (2%) had positive T2 short tau inversion recovery. Transthoracic echocardiogram (TTE) was the most common overall modality used for confirmatory diagnosis ( n  = 29, 83%). The most common findings were tamponade ( n  = 13,30%) and large pericardial effusion ( n  = 7, 16%) (Table  2 ).

Medical management consisted primarily of corticosteroids ( n  = 26, 59%), colchicine ( n  = 8, 18%), and non-steroidal anti-inflammatory drugs (NSAIDs) ( n  = 6, 14%). Pericardiocentesis occurred in 68% ( n  = 26) of patients, whereas pericardial window occurred in 21% ( n  = 9). Median fluid drained was 540 mL (IQR: 400–1000). Cytology was obtained in 66% ( n  = 25) of patients with 25% ( n  = 11) being positive for malignancy. Overall, 35 (92%) patients had symptom improvement with cessation of ICIs. Resumption of ICI occurred in 15 (34%) patients. Of those that resumed ICI therapy, recurrence occurred in 47% ( n  = 7) of patients. Median follow up was 210 (IQR:46–495) days. Overall mortality was 16% ( n  = 7) (Table  3 ).

To our knowledge, this is the most comprehensive analysis of ICI-mediated pericardial disease reported to date.

The incidence of ICI mediated pericardial disease remains unknown. Prior systematic reviews and retrospective studies of patients treated with ICIs have reported varied incidence and prevalence rates [ 1 , 5 , 6 ]. Patients with ICI mediated pericarditis in these studies were often males with lung cancer, and nivolumab and pembrolizumab were the two most common ICIs used [ 1 , 7 , 8 ]. Interestingly, men likely have higher rates of irAEs due to the disparity in ICI treatment in males as compared to females [ 1 , 9 ]. Similarly, in our cohort, we found ICI-associated pericardial disease predominantly affects males, accounting for 72% of cases, with a median age of 63 years. Lung cancer (adenocarcinoma and squamous cell carcinoma) was the most common cancer in our cohort (81% of cases). In a study of 60 patients with advanced non-small cell lung caner receiving immunotherapy with either nivolumab or pembrolizumab, pericardial effusion was found to be a relatively common complication occurring in 7% of patients [ 5 ]. We observed that the PD-1 inhibitors nivolumab and pembrolizumab were the most commonly implicated ICIs, used in 61% and 34% of cases, respectively.

Understanding the timeframe for the development of pericardial effusion after ICI initiation is crucial for timely diagnosis and management. Our study revealed an average time to onset of pericardial effusion after 4 cycles of ICI therapy. Prior studies have reported ranges as wide as 1 to 12 months [ 1 , 8 , 9 , 10 , 11 ]. Sawada et al. reported a case of a 67-year-old man who developed pericardial effusion which resolved with corticosteroids after 94 cycles of Nivolumab [ 12 ]. Overall, this suggests that pericarditis may be a risk during any cycle of ICI therapy.

Dyspnea and chest pain were the most reported symptoms in our study. Remarkably, only 19% of patients exhibited ECG abnormalities, and a minority displayed physical exam abnormalities, such as pericardial rub (2%). These findings underscore the occult nature of pericardial disease in cancer patients undergoing immunotherapy. Given the poor sensitivity of ECG and physical examination in detecting ICI-mediated pericardial disease, our findings suggest the necessity of a low threshold for further diagnostic imaging when patients present with symptoms. The wide range of reported incidence in previous studies likely reflects the under-diagnosis of pericardial effusion, especially when limited to cases requiring drainage [ 8 ].

Currently, there are limited prognostic tools to suggest development of ICI-mediated pericardial disease, our findings highlight the importance of continued surveillance throughout the course of treatment as there can be a significant delay from therapy initiation and the development of pericardial disease [ 13 ]. Current data suggests baseline EKG and serial troponin measurement to monitor for development [ 14 , 15 , 16 ].

Despite the assistance of imaging, the diagnosis of ICI-mediated pericardial disease is complex. Causes of effusion including cancer progression, pseudo progression, or infection are other possibilities in this patient population complicating appropriate diagnosis [ 17 ]. Progression and pseudo progression can be particularly difficult to distinguish from ICI-mediated pericarditis, as patients may present with effusion and imaging evidence of tumor pseudo-growth due to inflammation from response to therapy [ 18 ].

Interestingly, of 25 patients who underwent cytological testing of their effusions, 11 tested positive for malignant cells. Similarly, Gong et al. reported 8 of 15 patients who underwent pericardiocentesis for ICI mediated pericardial effusion had malignant cells present on cytology [ 11 ]. This finding suggests that pericardial effusion due to ICI may have malignant cells, and that consideration of the overall clinical picture is necessary for accurate diagnosis and differentiation from the similar but distinct entities of malignant pericardial effusion and pseudo progression.

A substantial portion of patients in our study, 68%, required pericardiocentesis, with 41% experiencing cardiac tamponade. These rates mirror those reported in prior studies, underscoring the significant morbidity and mortality associated with pericardial effusion in ICI-treated patients [ 17 ]. Almost all patients, 92%, had symptomatic improvement with cessation of ICI. ICI therapy was resumed in 38% of patients, and half of these patients experienced recurrence of pericardial effusion. Over a median follow up of 210 days, the mortality rate amongst patients with pericardial effusion was 16%. Gong et al. found that patients treated with ICI who developed pericardial effusion had an increased risk of mortality (Hazard Ratio: 1.53) compared with those who did not [ 11 ]. Thus, the development of ICI mediated pericardial effusion may be a poor prognostic factor for survival [ 19 ].

Despite our best efforts, our review is not without several limitations. The entirety of our cohort was derived from case reports with inherent publication bias present. Our analysis may overestimate the severity of this condition as severe case are more likely to be reported. Additionally, given the lack of a control group, our results have limited generalizability.

ICIs are a novel therapy used for the management of several malignancies. While they are effective for treatment of malignancy, they possess certain cardiotoxic side effects including development of pericardial disease. Patients with ICI mediated pericarditis often present in life threatening cardiac tamponade and definitive diagnosis requires a combination of imaging and pericardial fluid analysis. Management is primarily cessation of ICI therapy coupled with a combination of NSAIDs, colchicine, and steroids. Overall if diagnosed early, mortality rates are low.

Data availability

Available upon reasonable request.

Abbreviations

Immune checkpoint inhibitors

transthoracic echocardiogram

computed tomography

Salem JE, Manouchehri A, Moey M, et al. Cardiovascular toxicities associated with immune checkpoint inhibitors: an observational, retrospective, pharmacovigilance study. Lancet Oncol. 2018;19(12):1579–89. https://doi.org/10.1016/S1470-2045(18)30608-9

Article   CAS   PubMed   PubMed Central   Google Scholar  

Palaskas N, Lopez-Mattei J, Durand JB, Iliescu C, Deswal A. Immune checkpoint inhibitor myocarditis: pathophysiological characteristics, diagnosis, and treatment. J Am Heart Assoc. 2020;9(2):e013757. https://doi.org/10.1161/JAHA.119.013757

Article   PubMed   PubMed Central   Google Scholar  

Ganatra S, Neilan TG. Immune Checkpoint inhibitor-Associated Myocarditis. Oncologist. 2018;23(8):879–86. https://doi.org/10.1634/theoncologist.2018-0130

Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71

Canale ML, Camerini A, Casolo G, et al. Incidence of Pericardial Effusion in patients with Advanced Non-small Cell Lung Cancer receiving immunotherapy. Adv Ther. 2020;37(7):3178–84. https://doi.org/10.1007/s12325-020-01386-y

Nso N, Antwi-Amoabeng D, Beutler BD, et al. Cardiac adverse events of immune checkpoint inhibitors in oncology patients: a systematic review and meta-analysis. World J Cardiol. 2020;12(11):584–98. https://doi.org/10.4330/wjc.v12.i11.584

Brumberger ZL, Branch ME, Klein MW, Seals A, Shapiro MD, Vasu S. Cardiotoxicity risk factors with immune checkpoint inhibitors. Cardio-Oncol Lond Engl. 2022;8(1):3. https://doi.org/10.1186/s40959-022-00130-5

Article   Google Scholar  

Palaskas N, Morgan J, Daigle T, et al. Targeted Cancer Therapies with Pericardial effusions requiring Pericardiocentesis focusing on Immune Checkpoint inhibitors. Am J Cardiol. 2019;123(8):1351–7. https://doi.org/10.1016/j.amjcard.2019.01.013

Article   PubMed   Google Scholar  

Varricchi G, Galdiero MR, Mercurio V, Bonaduce D, Marone G, Tocchetti CG. Pharmacovigilating cardiotoxicity of immune checkpoint inhibitors. Lancet Oncol. 2018;19(12):1545–6. https://doi.org/10.1016/S1470-2045(18)30622-3

Chahine J, Collier P, Maroo A, Tang WHW, Klein AL. Myocardial and pericardial toxicity Associated with Immune checkpoint inhibitors in Cancer patients. JACC Case Rep. 2020;2(2):191–9. https://doi.org/10.1016/j.jaccas.2019.11.080

Gong J, Drobni ZD, Zafar A, et al. Pericardial disease in patients treated with immune checkpoint inhibitors. J Immunother Cancer. 2021;9(6):e002771. https://doi.org/10.1136/jitc-2021-002771

Sawada R, Matsui Y, Uchino J, et al. Late-onset Pleural and Pericardial Effusion as Immune-related adverse events after 94 cycles of Nivolumab. Intern Med Tokyo Jpn. 2021;60(22):3585–8. https://doi.org/10.2169/internalmedicine.7219-21

Yousif LI, Screever EM, Versluis D, et al. Risk factors for Immune Checkpoint inhibitor-mediated Cardiovascular Toxicities. Curr Oncol Rep. 2023;25(7):753–63. https://doi.org/10.1007/s11912-023-01414-4

Johnson DB, Balko JM, Compton ML, et al. Fulminant myocarditis with combination Immune Checkpoint Blockade. N Engl J Med. 2016;375(18):1749–55. https://doi.org/10.1056/NEJMoa1609214

Spallarossa P, Sarocchi M, Tini G, et al. How to monitor Cardiac complications of Immune checkpoint inhibitor therapy. Front Pharmacol. 2020;11:972. https://doi.org/10.3389/fphar.2020.00972

Spallarossa P, Tini G, Sarocchi M, et al. Identification and management of Immune Checkpoint inhibitor-related myocarditis: use troponin wisely. J Clin Oncol off J Am Soc Clin Oncol. 2019;37(25):2201–5. https://doi.org/10.1200/JCO.18.02464

Article   CAS   Google Scholar  

Inno A, Maurea N, Metro G, Carbone A, Russo A, Gori S. Immune checkpoint inhibitors-associated pericardial disease: a systematic review of case reports. Cancer Immunol Immunother CII. 2021;70(10):3041–53. https://doi.org/10.1007/s00262-021-02938-z

Article   CAS   PubMed   Google Scholar  

Asai M, Kato Y, Kawai S, et al. Management of cardiac tamponade during nivolumab of lung cancer with intrapericardial bleomycin: case report. Immunotherapy. 2019;11(6):467–72. https://doi.org/10.2217/imt-2019-0003

Patel RP, Parikh R, Gunturu KS, et al. Cardiotoxicity of Immune Checkpoint inhibitors. Curr Oncol Rep. 2021;23(7):79. https://doi.org/10.1007/s11912-021-01070-6

Download references

Not Applicable.

Author information

Sarah E. Mudra and Danny L. Rayes contributed equally to this study.

Authors and Affiliations

Department of Internal Medicine, MedStar Georgetown University Hospital, MedStar Health, Washington, DC, USA

Sarah E. Mudra, Danny L. Rayes, Ashwin K. Kumar, Jason Z. Li, Meredith Njus, Kevin McGowan, Kazi A. Kalam & Charalompos Charalampous

Center for the Diagnosis and Treatment of Pericardial Diseases, Section of Cardiovascular Imaging, Department of Cardiovascular Medicine, Heart, Vascular, and Thoracic Institute, Cleveland Clinic, 9500 Euclid Ave., Desk J1-5, Cleveland, OH, 44195, USA

Ankit Agrawal, Ashwin K. Kumar, Muhammad Majid, Alvena Syed, Abdullah Yesilyaprak & Allan L. Klein

Floyd D. Loop Memorial Library, Cleveland Clinic, Cleveland, OH, USA

Mary Schleicher

You can also search for this author in PubMed   Google Scholar

Contributions

SM, DR, AA, AK, JZ, MN, KM, KK, CC wrote the main manuscript, edited the manuscript, and prepared the figures and manuscript for submission. MS, MM, AA, AY reviewed and edited the manuscript, and provided initial guidance for methodology. ALK edited the manuscript, reviewed the manuscript, provided guidance on methodology and oversaw all other authors.

Corresponding author

Correspondence to Allan L. Klein .

Ethics declarations

Ethical approval.

As this a systematic review no prior institutional review board and the ethics committee approval was obtained.

Competing interests

Allan Klein: research funding from Kiniksa Pharmaceuticals, Ltd; and have served on scientific advisory boards for Cardiol Therapeutics, Kiniksa Pharmaceuticals, Ltd., Swedish Orphan Biovitrum AB, and Pfizer, Inc. All author others have no disclosures.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1: Supplement: Supplemental References

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mudra, S.E., Rayes, D.L., Agrawal, A. et al. Immune checkpoint inhibitors and pericardial disease: a systematic review. Cardio-Oncology 10 , 29 (2024). https://doi.org/10.1186/s40959-024-00234-0

Download citation

Received : 06 March 2024

Accepted : 03 May 2024

Published : 17 May 2024

DOI : https://doi.org/10.1186/s40959-024-00234-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Immune Checkpoint inhibitors
• Pericarditis
• Pericardial Effusion
• Cardiac Tamponade

Cardio-Oncology

ISSN: 2057-3804

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

The Levels of Evidence and their role in Evidence-Based Medicine

Patricia b. burns.

1 Research Associate, Section of Plastic Surgery, Department of Surgery, The University of Michigan Health System

Rod J. Rohrich

2 Professor of Surgery, Department of Plastic Surgery, University of Texas Southwestern Medical Center

Kevin C. Chung

3 Professor of Surgery, Section of Plastic Surgery, Department of Surgery, The University of Michigan Health System

As the name suggests, evidence-based medicine (EBM), is about finding evidence and using that evidence to make clinical decisions. A cornerstone of EBM is the hierarchical system of classifying evidence. This hierarchy is known as the levels of evidence. Physicians are encouraged to find the highest level of evidence to answer clinical questions. Several papers published in Plastic Surgery journals concerning EBM topics have touched on this subject. 1 – 6 Specifically, previous papers have discussed the lack of higher level evidence in PRS and need to improve the evidence published in the journal. Before that can be accomplished, it is important to understand the history behind the levels and how they should be interpreted. This paper will focus on the origin of levels of evidence, their relevance to the EBM movement and the implications for the field of plastic surgery as well as the everyday practice of plastic surgery.

History of Levels of Evidence

The levels of evidence were originally described in a report by the Canadian Task Force on the Periodic Health Examination in 1979. 7 The report’s purpose was to develop recommendations on the periodic health exam and base those recommendations on evidence in the medical literature. The authors developed a system of rating evidence ( Table 1 ) when determining the effectiveness of a particular intervention. The evidence was taken into account when grading recommendations. For example, a Grade A recommendation was given if there was good evidence to support a recommendation that a condition be included in the periodic health exam. The levels of evidence were further described and expanded by Sackett 8 in an article on levels of evidence for antithrombotic agents in 1989 ( Table 2 ). Both systems place randomized controlled trials (RCT) at the highest level and case series or expert opinions at the lowest level. The hierarchies rank studies according to the probability of bias. RCTs are given the highest level because they are designed to be unbiased and have less risk of systematic errors. For example, by randomly allocating subjects to two or more treatment groups, these types of studies also randomize confounding factors that may bias results. A case series or expert opinion is often biased by the author’s experience or opinions and there is no control of confounding factors.

Canadian Task Force on the Periodic Health Examination’s Levels of Evidence *

Levels of Evidence from Sackett *

Modification of levels

Since the introduction of levels of evidence, several other organizations and journals have adopted variation of the classification system. Diverse specialties are often asking different questions and it was recognized that the type and level of evidence needed to be modified accordingly. Research questions are divided into the categories: treatment, prognosis, diagnosis, and economic/decision analysis. For example, Table 3 shows the levels of evidence developed by the American Society of Plastic Surgeons (ASPS) for prognosis 9 and Table 4 shows the levels developed by the Centre for Evidence Based Medicine (CEBM) for treatment. 10 The two tables highlight the types of studies that are appropriate for the question (prognosis versus treatment) and how quality of data is taken into account when assigning a level. For example, RCTs are not appropriate when looking at the prognosis of a disease. The question in this instance is: “What will happen if we do nothing at all”? Because a prognosis question does not involve comparing treatments, the highest evidence would come from a cohort study or a systematic review of cohort studies. The levels of evidence also take into account the quality of the data. For example, in the chart from CEBM, poorly designed RCTs have the same level of evidence as a cohort study.

Levels of Evidence for Prognostic Studies *

Levels of Evidence for Therapeutic Studies *

A grading system that provides strength of recommendations based on evidence has also changed over time. Table 5 shows the Grade Practice Recommendations developed by ASPS. The grading system provides an important component in evidence-based medicine and assists in clinical decision making. For example, a strong recommendation is given when there is level I evidence and consistent evidence from Level II, III and IV studies available. The grading system does not degrade lower level evidence when deciding recommendations if the results are consistent.

Grade Practice Recommendations *

Interpretation of levels

Many journals assign a level to the papers they publish and authors often assign a level when submitting an abstract to conference proceedings. This allows the reader to know the level of evidence of the research but the designated level of evidence does always guarantee the quality of the research. It is important that readers not assume that level 1 evidence is always the best choice or appropriate for the research question. This concept will be very important for all of us to understand as we evolve into the field of EBM in Plastic Surgery. By design, our designated surgical specialty will always have important articles that may have a lower level of evidence due to the level of innovation and technique articles which are needed to move our surgical specialty forward.

Although RCTs are the often assigned the highest level of evidence, not all RCTs are conducted properly and the results should be carefully scrutinized. Sackett 8 stressed the importance of estimating types of errors and the power of studies when interpreting results from RCTs. For example, a poorly conducted RCT may report a negative result due to low power when in fact a real difference exists between treatment groups. Scales such as the Jadad scale have been developed to judge the quality of RCTs. 11 Although physicians may not have the time or inclination to use a scale to assess quality, there are some basic items that should be taken into account. Items used for assessing RCTs include: randomization, blinding, a description of the randomization and blinding process, description of the number of subjects who withdrew or drop out of the study; the confidence intervals around study estimates; and a description of the power analysis. For example, Bhandari et al. 12 published a paper assessing the quality of surgical RCTs. The authors evaluated the quality of RCTs reported in the Journal of Bone and Joint Surgery (JBJS) from 1988–2000. Papers with a score of > 75% were deemed high quality and 60% of the papers had a score < 75%. The authors identified 72 RCTs during this time period and the mean score was 68%. The main reason for the low quality score was lack of appropriate randomization, blinding, and a description of patient exclusion criteria. Another paper found the same quality score of papers in JBJS with a level 1 rating compared to level 2. 13 Therefore, one should not assume that level 1 studies have higher quality than level 2.

A resource for surgeons when appraising levels of evidence are the users’ guides published in the Canadian Journal of Surgery 14 , 15 and the Journal of Bone and Joint Surgery. 16 Similar papers that are not specific to surgery have been published in the Journal of the American Medical Association (JAMA). 17 , 18

Plastic surgery and EBM

The field of plastic surgery has been slow to adopt evidence-based medicine. This was demonstrated in a paper examining the level of evidence of papers published in PRS. 19 The authors assigned levels of evidence to papers published in PRS over a 20 year period. The majority of studies (93% in 1983) were level 4 or 5, which denotes case series and case reports. Although the results are disappointing, there was some improvement over time. By 2003 there were more level 1studies (1.5%) and fewer level 4 and 5 studies (87%). A recent analysis looked at the number of level 1 studies in 5 different plastic surgery journals from 1978–2009. The authors defined level 1 studies as RCTs and meta-analysis and restricted their search to these studies. The number of level 1 studies increased from 1 in 1978 to 32 by 2009. 20 From these results, we see that the field of plastic surgery is improving the level of evidence but still has a way to go, especially in improving the quality of studies published. For example, approximately a third of the studies involved double blinding, but the majority did not randomize subjects, describe the randomization process, or perform a power analysis. Power analysis is another area of concern in plastic surgery. A review of the plastic surgery literature found that the majority of published studies have inadequate power to detect moderate to large differences between treatment groups. 21 No matter what the level of evidence for a study, if it is under powered, the interpretation of results is questionable.

Although the goal is to improve the overall level of evidence in plastic surgery, this does not mean that all lower level evidence should be discarded. Case series and case reports are important for hypothesis generation and can lead to more controlled studies. Additionally, in the face of overwhelming evidence to support a treatment, such as the use of antibiotics for wound infections, there is no need for an RCT.

Clinical examples using levels of evidence

In order to understand how the levels of evidence work and aid the reader in interpreting levels, we provide some examples from the plastic surgery literature. The examples also show the peril of medical decisions based on results from case reports.

An association was hypothesized between lymphoma and silicone breast implants based on case reports. 22 – 27 The level of evidence for case reports, depending on the scale used, is 4 or 5. These case reports were used to generate the hypothesis that a possible association existed. Because of these results, several large retrospective cohort studies from the United States, Canada, Denmark, Sweden and Finland were conducted. 28 – 32 The level of evidence for a retrospective cohort is 2. All of these studies had many years of follow-up for a large number of patients. Some of the studies found an elevated risk and others no risk for lymphoma. None of the studies reached statistical significance. Therefore, higher level evidence from cohort studies does not provide evidence of any risk of lymphoma. Finally, a systematic review was performed that combined the evidence from the retrospective cohorts. 27 The results found an overall standardized incidence ratio of 0.89 (95% CI 0.67–1.18). Because the confidence intervals include 1, the results indicate there is no increased incidence. The level of evidence for the systematic review is 1. Based on the best available evidence, there is no association between lymphoma and silicone implants. This example shows how low level evidence studies were used to generate a hypothesis, which then led to higher level evidence that disproved the hypothesis. This example also demonstrates that RCTs are not feasible for rare events such as cancer and the importance of observational studies for a specific study question. A case-control study is a better option and provides higher evidence for testing the prognosis of the long-term effect of silicone breast implants.

Another example is the injection of epinephrine in fingers. Based on case reports prior to 1950, physicians were advised that epinephrine injection can result in finger ischemia. 33 We see in this example in which level 4 or 5 evidence was accepted as fact and incorporated into medical textbooks and teaching. However, not all physicians accepted this evidence and are performing injections of epinephrine into the fingers with no adverse effects on the hand. Obviously, it was time for higher level evidence to resolve this issue. An in-depth review of the literature from 1880 to 2000 by Denkler, 33 identified 48 cases of digital infarction of which 21 were injected with epinephrine. Further analysis found that the addition of procaine to the epinephrine injection was the cause of the ischemia. 34 The procaine used in these injections included toxic acidic batches that were recalled in 1948. In addition, several cohort studies found no complications from the use of epinephrine in the fingers and hand. 35 , 36 , 37 The results from these cohort studies increased the level of evidence. Based on the best available evidence from these studies, the hypothesis that epinephrine injection will harm fingers was rejected. This example highlights the biases inherent in case reports. It also shows the risk when spurious evidence is handed down and integrated into medical teaching.

Obtaining the best evidence

We have established the need for RCTs to improve evidence in plastic surgery but have also acknowledged the difficulties, particularly with randomization and blinding. Although RCTs may not be appropriate for many surgical questions, well designed and conducted cohort or case-control studies could boost the level of evidence. Many of the current studies tend to be descriptive and lack a control group. The way forward seems clear. Plastic surgery researchers need to consider utilizing a cohort or case-control design whenever an RCT is not possible. If designed properly, the level of evidence for observational studies can approach or surpass those from an RCT. In some instances, observation studies and RCTs have found similar results. 38 If enough cohort or case-control studies become available, this increases the prospect of systematic reviews of these studies that will increase overall evidence levels in plastic surgery.

The levels of evidence are an important component of EBM. Understanding the levels and why they are assigned to publications and abstracts helps the reader to prioritize information. This is not to say that all level 4 evidence should be ignored and all level 1 evidence accepted as fact. The levels of evidence provide a guide and the reader needs to be cautious when interpreting these results.

Acknowledgments

Supported in part by a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (to Dr. Kevin C. Chung).

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 15 May 2024

Comparative efficacy and safety of alpha-blockers as monotherapy for benign prostatic hyperplasia: a systematic review and network meta-analysis

• Beema T Yoosuf   ORCID: orcid.org/0009-0001-3584-6212 1 ,
• Abhilash Kumar Panda 1 ,
• Muhammed Favas KT   ORCID: orcid.org/0000-0001-8068-6839 1 ,
• Saroj Kundan Bharti   ORCID: orcid.org/0000-0003-4221-0025 1 ,
• Sudheer Kumar Devana 2 &
• Dipika Bansal   ORCID: orcid.org/0000-0003-4520-3293 1  

Scientific Reports volume  14 , Article number:  11116 ( 2024 ) Cite this article

Metrics details

• Health care
• Medical research

Despite the availability of various drugs for benign prostatic hyperplasia (BPH), alpha(α)-blockers are the preferred first-line treatment. However, there remains a scarcity of direct comparisons among various α-blockers. Therefore, this network meta-analysis (NMA) of randomized controlled trials (RCTs) aimed to evaluate the efficacy and safety of α-blockers in the management of BPH. A comprehensive electronic search covered PubMed, Embase, Ovid MEDLINE, and Cochrane Library until August 2023. The primary endpoints comprised international prostate symptom score (IPSS), maximum flow rate (Qmax), quality of life (QoL), and post-void residual volume (PVR), while treatment-emergent adverse events (TEAEs) were considered as secondary endpoints. This NMA synthesized evidence from 22 studies covering 3371 patients with six kinds of α-blockers with 12 dose categories. IPSS has been considerably improved by tamsulosin 0.4 mg, naftopidil 50 mg and silodosin 8 mg as compared to the placebo. Based on the p-score, tamsulosin 0.4 mg had the highest probability of ranking for IPSS, PVR, and Qmax, whereas doxazosin 8 mg had the highest probability of improving QoL. A total of 297 adverse events were reported among all the α-blockers, silodosin has reported a notable number of TEAEs. Current evidence supports α-blockers are effective in IPSS reduction and are considered safer. Larger sample size with long-term studies are needed to refine estimates of IPSS, QoL, PVR, and Qmax outcomes in α-blocker users.

Similar content being viewed by others

Current management strategy of treating patients with erectile dysfunction after radical prostatectomy: a systematic review and meta-analysis

A Pilot retrospective analysis of alpha-blockers on recurrence in men with localised prostate cancer treated with radiotherapy

Impact of minimally invasive surgical procedures for Male Lower Urinary Tract Symptoms due to benign prostatic hyperplasia on ejaculatory function: a systematic review

Introduction.

Benign prostatic hyperplasia (BPH) is a ubiquitous urological disease that inevitably affects older men, occurring in up to 50% of men over 50 to 60 years, rising to 90% by age 80, and its predominance increases further with age 1 , 2 , 3 . BPH results from the noncancerous prostate gland enlargement induced by cellular hyperplasia of both glandular and stromal components 4 . Numerous sources of evidence reveal that in addition to ageing and family history, modifiable risk factors such as enlarged prostate, dyslipidemia, hypertension, hormonal imbalance, obesity, metabolic syndrome, diet, alcohol use, and smoking can collectively contribute to BPH 5 , 6 . Many individuals with BPH experience lower urinary tract symptoms (LUTS) in the form of irritative (frequency, nocturia and urgency) and obstructive urinary symptoms (hesitancy, intermittency, weak stream, incomplete bladder emptying and acute urinary retention (AUR)) 1 . LUTS correlated with BPH drastically compromises the quality of life (QoL), primarily disrupting sleep and daily activities 7 . Ipso facto, the intent of BPH treatment is to alleviate these troublesome and irritating symptoms 1 .

Pharmacological management of LUTS correlated with BPH has emerged over the last 25 years 6 . Existing medical therapy for BPH includes alpha-adrenergic receptor antagonists (α-blockers), anticholinergics, 5-alpha reductase inhibitors (5-ARIs) and phosphodiesterase inhibitors (PDE5-Is). Medical therapy is generally considered the initial treatment option for patients with moderate to severe LUTS while surgical approaches like transurethral resection of the prostate (TURP) are recommended for patients who had poor response to medical therapy or those with specific indications like refractory urinary retention, recurrent hematuria and those with severe bladder outlet obstruction leading to hydroureteronephrosis 4 . α-blockers are considered as the first-line drugs for treating BPH. Long-acting α-blockers, such as doxazosin, terazosin, tamsulosin, alfuzosin and silodosin, have been approved by the Food and Drug Administration (FDA) for the treatment of BPH 8 . They can mitigate symptoms by blocking endogenously secreted noradrenaline on smooth muscle cells in the prostate gland, thus reducing prostate tone and bladder outlet obstruction 9 .

Despite the fact that a large number of drugs are now available to treat BPH, α-blockers have a significant impact on improvement in International Prostate Symptom Score (IPSS), maximum flow rate (Qmax), post-void residual (PVR) and QoL 8 , 10 , 11 . Even though several clinical trials have been performed to explore the effectiveness of α-blockers for BPH, direct comparisons among these drugs are still lacking and there is conflicting information coming forward from meta-analysis 12 , 13 , 14 . For instance, a network meta-analysis (NMA) conducted on drug therapies for BPH assessed the effectiveness of multiple drug classes, instead of individual agents 15 . Furthermore, the most recently published NMA demonstrates merely IPSS, peak urine flow rate (PUF), and adverse events (AEs) among mono-drug therapies for LUTs related to BPH 16 . At present, none of the NMA have extensively evaluated the efficacy of these agents within the class in terms of the majority of outcomes as well as treatment-emergent adverse events (TEAEs). Therefore, the aim of the present study is to address the knowledge gap surrounding the comparative effectiveness of α-blockers for BPH based on available randomised controlled trials (RCTs) and rank these agents for clinical consideration.

This network meta-analysis (NMA) was executed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for NMA. We have applied frequentist network meta-analysis for its simplicity associated with the model formulation 17 . The protocol was registered in the Prospective Register of Systematic Reviews (CRD42022365398).

Literature searches

A comprehensive electronic search of PubMed, Ovid MEDLINE, EMBASE and the Cochrane library, was carried out to identify the eligible studies. Additionally, a manual search in Google Scholar was performed. The initial search strategy was developed in the PubMed database, and the search strings used for electronic searches consist of combinations of keywords and medical subject headings (MeSH) terms like “alpha-blockers”, “Alfuzosin”, “Tamsulosin”, “Doxazosin”, “Terazosin”, “Silodosin”, “Naftopidil”, “Benign prostatic hyperplasia” and “Randomised controlled trial”. A methodological search filter was adopted to identify RCTs, and the search was limited to English-language publications. This search strategy serves as a template for alternative search algorithms customized to different databases, such as EMBASE, Ovid MEDLINE, and the Cochrane Library. In addition, the reference lists of the selected studies and review articles were hand-searched for additional potentially pertinent studies.

Study eligibility

This systematic review and NMA sought studies that met the PICO (P—population, I—intervention, C—comparator, O—outcome) framework. RCTs that investigated the efficacy and safety of α-blocker in men aged 45 and above with LUTS related to BPH were included. However, monotherapy with α-blockers were eligible, including selective (i.e., terazosin and doxazosin) and uroselective (tamsulosin, silodosin, alfuzosin and naftopidil), with no restrictions on α-blocker dosage 18 . As the research question also explored placebo-controlled trials, therefore the placebo serves to be the comparator. The key outcomes of interest were IPSS, QoL, PVR and Q max. TEAEs are also evaluated in order to provide a comprehensive overview of these drugs. Reviews, editorials, case reports, conference abstracts, studies that deviated from the aimed outcomes or with incomplete results and articles published in non-English were excluded.

Study screening

Two reviewers (BY and AP) worked independently to screen citations and evaluate full-text records for eligibility. Initially, only the title and abstract were screened, and the full texts of presumably pertinent articles were subsequently assessed for ultimate inclusion. A cross-check has been performed at both stages to ensure full compliance with eligibility requirements. Disputes regarding the full-text articles were rectified through discussion with a third reviewer (DB).

Data extraction

Two reviewers (BY and AP) individually extracted the following information into a spreadsheet: study characteristics (Title, first author, publication year, country, duration of treatment), population (study setting, sample size, baseline demographics), characterization of interventions (drug name and dose), and outcomes (reduction in IPSS and PVR, improvement in QOL, Qmax). Disagreements among reviewers were resolved by discussion or, if necessary, communicating with a third reviewer (DB). If any imperative information about study outcomes was missing or unclear in the published studies, the authors were contacted to seek clarification or additional data.

Risk of bias

The methodological quality of each included RCT was critically appraised employing the revised Cochrane Risk of Bias Tool (ROB 2.0) 19 . This tool captures six main sources of bias, comprising random sequence generation, allocation concealment, missing outcome data, blinding, selective reporting, and other sources of bias. Each domain has been assigned a score of low, moderate to high.

Statistical analysis

To account for certain methodological and clinical heterogeneity across studies, and to acquire the optimal generalizability in the meta-analytical treatment effects, we adopted a random-effects model 20 . As all the efficacy outcomes are continuous data, the effect size was computed as standardised mean difference (SMD) along with 95% confidence intervals (CI), and the outcome data was compiled using direct and indirect evidence employing a frequentist approach.

Statistical analysis was carried out using the “netmeta” package of R Studio and data were analysed following the intention-to-treat approach. A network plot of interventions was used to visualise the evidence gathered and offered a succinct overview of its characteristics. Direct evidence has gathered by pair-wise meta-analysis, while indirect evidence was obtained through indirect comparisons. The treatments were ranked using p-scores derived from the surface under the cumulative ranking curve (SUCRA). Higher p- scores tend to indicate a higher probability of being the most effective treatment 21 . In order to evaluate inconsistency, both global and local approaches were utilized. Under the presumption of a full design-by-treatment interaction random effects model 22 , the Q test and the I 2 statistic are adopted to evaluate consistency 23 . The local approach distinguishes indirect from direct evidence (SIDE) using the back-calculation method. The comparison-adjusted funnel plot was utilized to evaluate small-study effects for each outcome with ≥ 10 studies, where the overall treatment effect for every comparison was estimated employing random-effect meta-analysis model 24 . All eligible drugs have been ordered from oldest to newest according to their international market authorisation dates. Furthermore, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings were deployed to assess the certainty of evidence in networks employing the Confidence In Network Meta‐Analysis (CINeMA) framework 25 .

Study selection

The literature search of across multiple databases yielded a total of 3019 potentially relevant citations (Table S9). After duplication screening, 2164 articles were found. Of these 2022 articles were removed after the initial title and abstract screening and retrieved 142 articles for full-text review. Finally, 22 RCTs (3271 participants) published from 2000 to 2023 were included (Fig.  1 ) 12 , 13 , 14 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 .

PRISMA flow chart of literature searches and results. ( PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses).

Study characteristics

The included RCTs comprised the currently used six kinds of α-blockers with different dose categories including Naftopidil 25 mg, 50 mg and 75 mg, Silodosin 8 mg, Tamsulosin 0.2 mg and 0.4 mg, Alfuzosin 2.5 mg and 10 mg, Doxazosin 2 mg, 4 mg and 8 mg, Terazosin 0.5 mg with a total 3,371 participants. Among the 22 included studies, 10 were multi-centric 26 , 29 , 31 , 32 , 34 , 36 , 39 , 40 , 41 , 42 , while 12 were single-centric 12 , 13 , 14 , 27 , 28 , 30 , 33 , 35 , 37 , 38 , 43 , 44 . Nine trials were conducted in Japan 26 , 30 , 31 , 32 , 33 , 36 , 39 , 40 , 43 , five in India 12 , 13 , 14 , 37 , 44 , two in Korea 41 , 42 and one each in China 34 , Indonesia 29 , Europe 27 , Philippines 28 , Egypt 35 and Turkey 38 . Most studies (91%) were published after 2005 and, over half of the studies (50%) involved more than 100 patients. A majority of trials (72.73%) had treatment durations of more than 4 weeks. The mean (SD) age of the patients was 65.3 (6.7) years (Table 1 ). According to IPSS, the symptoms of patients in the included trials varied from moderate to severe, with a baseline mean (SD) of 18.1 (4.6). The baseline mean (SD) value of QOL was 4.2 (0.8), Q max (ml/s) 10.2 (3.4), and PVR (ml) 49.0 (34.2).

In terms of study quality, 15 trials (68.18%) exhibited a low risk of bias, three trials (13.64%) had a moderate risk of bias, and four trials (18.18%) had a high risk of bias (Table 2 ).

Efficacy outcome

International prostate symptom score (ipss).

The NMA on IPSS included 22 RCTs with 6 interventions across 13 dose categories and 3271 participants (Fig.  2 a). The base-case estimates of the efficacy of α-blockers regimens on reducing IPSS are listed in Table 2 . Twenty-three comparisons estimated the treatment effect derived from direct evidence, 86 comparisons with indirect evidence and 18 comparisons with mixed evidence. Compared to the placebo, the NMA results found that three drugs had a significant effect on the reduction in IPSS, such as tamsulosin 0.4 mg (SMD: − 6.10; 95% CI: [− 8.74; − 3.47]), followed by naftopidil 50 mg (SMD: − 5.09; 95% CI: [− 8.29; − 1.89]) and silodosin 8 mg (SMD: − 3.63; 95% CI: [− 6.31; − 0.95]) (Fig.  3 a). The relative effectiveness was depicted using the league table (Table S1), all included α-blockers significantly reduce the IPSS compared to the placebo. Based on the p-score the highest-ranked treatment was tamsulosin 0.4 mg (0.89) and the lowest-ranked treatment was doxazosin 2 mg (0.22) (Table 3 ). Furthermore, the Q test of consistency showed substantial heterogeneity for this comparison (I 2 , 85.5%) (Appendix S1).

Network plot comparing individual α-blockers on international prostate symptom score (IPSS), quality of life (QoL), post-void residual volume (PVR) and maximum flow rate (Q max). The width of the edge is proportional to the number of trials comparing the two drugs, and the node represents the type of treatment. Tam = tamsulosin, Alfu = alfuzosin, Naf = naftopidil, Tera = terazosin, Dox = doxazosin, Sil = silodosin.

Forest plot of interventions as measured by the international prostate symptom score (IPSS), quality of life (QoL), post-void residual volume (PVR) and maximum flow rate (Q max). Tam = tamsulosin, Alfu = alfuzosin, Naf = naftopidil, Tera = terazosin, Dox = doxazosin, Sil = silodosin.

Quality of life (QoL)

13 RCTs including 6 interventions in 12 dose categories with 2,783 participants contributed to the comparison of the improvement in QoL (Fig.  2 b). Fourteen comparisons estimated the treatment effect derived from direct evidence, 58 comparisons with indirect evidence and 7 comparisons with mixed evidence. Compared to the placebo, none of the comparison reached statistical significance in improving QoL (Fig.  3 b). Doxazosin 8 mg has the highest probability of improving QoL, although the results were imprecise (Table S3). According to the pairwise comparisons, doxazosin 8 mg (− 1.35 [− 4.68; 1.98]) improves QoL compared to placebo (Table S2). Additionally, the Q consistency test showed a substantial heterogeneity for this evaluation (I 2 , 83.04%) (Appendix S1).

Post-void residual volume (PVR)

15 RCTs including 6 interventions in 10 dose categories with 2,761 participants contributed to the comparison of the reduction in PVR (Fig.  2 c). Fifteen comparisons estimated the treatment effect derived from direct evidence, 51 comparisons with indirect evidence and 11 comparisons with mixed evidence. Compared to the placebo, none of the comparisons showed statistical significance in reducing PVR (Fig.  3 c). Tamsulosin 0.4 mg and naftopidil 50 mg had the highest probability of improving PVR, with a p-score of 0.89, however, the results were imprecise (Table S5). According to the pairwise comparisons, tamsulosin 0.4 mg (− 15.99 [− 3.15; 35.12]) reduces the PVR compared to placebo; followed by naftopidil 50 mg (− 15.88 [− 34.73; 2.97]), doxazosin 2 mg (− 12.44[− 36.96; 12.07]) and doxazosin 4 mg (− 6.34 [− 28.27; 15.58]) (Table S4). Additionally, the Q consistency test showed a no heterogeneity for this evaluation (I 2 , 0%) (Appendix S1).

Maximum urinary flow rate (Qmax)

16 RCTs including 6 interventions in 13 dose categories with 3,114 participants contributed to the comparison of the improvement in Qmax (Fig.  2 d). Twenty comparisons estimated the treatment effect derived from direct evidence, 60 comparisons with indirect evidence and 15 comparisons with mixed evidence. Compared to the placebo, none of the comparisons showed statistical significance in improving Qmax (Fig.  3 d). Tamsulosin 0.4 mg has the highest probability of improving Qmax, with a p-score of 0.75 (Table S7). According to the pairwise comparisons, tamsulosin 0.4 mg (− 4.30 [− 9.37; 0.76]) reduces the Qmax compared to placebo; followed by terazosin 1 mg (− 3.99 [− 9.86; 1.89]), doxazosin 4 mg (− 3.39 [− 2.08; 8.86]) and naftopidil 75 mg (− 3.53 [− 3.03; 10.09]) (Table S6). Moreover, the Q consistency test showed a considerable heterogeneity for this evaluation (I 2 , 65.87%) (Appendix S1).

Safety outcomes

A total of 297 AEs was reported among the α-blockers (events/participants = 297/3009), silodosin (190/739) dominated with a notable number of AEs followed by tamsulosin (32/966), doxazosin (27/313), naftopidil (25/544), alfuzosin (20/416) and terazosin (3/31). The most prominent AEs included ejaculation dysfunction, dizziness and hypotension. The AEs associated with α-blockers have been listed in Table S8.

Evaluation of evidence quality

The degree of certainty of evidence for each outcome has been depicted in Figure S6, S7, S8, S9. About half of the comparison are moderate to low level confidence rating for IPSS vs placebo. Despite this, it was low for all other comparisons owing to imprecision and incoherence. However, the results of local and global approaches for IPSS showed inconsistent while all the other outcomes were found consistent. The quality scoring of the included studies is illustrated in the Figure S5. Furthermore, visual inspection of the comparison-adjusted funnel plots found the evidence of small-study effects for all outcomes (asymmetrical funnel plot) which indicates presence of potential publication bias (Figs. S1, S2, S3, S4).

Although there are several therapeutical options for BPH presently, pharmacological therapy has become standard care and is widely recommended by clinical guidelines 7 . American Urological Association (AUA) and Canadian Urological Association (CUA) guidelines recommend α- blockers as the first-line drug for BPH 45 , 46 . Despite their rapid onset of action, efficacy and modest frequency and intensity of adverse effects, α-blockers are considered as an excellent choice of therapy for BPH associated LUTS. The underlying mechanism of α-blockers is to inhibit the effect of norepinephrine produced endogenously on smooth muscle cells of the prostate; thereby reducing prostatic tone and consequently, urethral obstruction 47 , 48 . Several α-blockers have been approved by the FDA for the treatment of BPH, including terazosin, alfuzosin, doxazosin, tamsulosin and silodosin whereas naftopidil is only approved in Japan 49 , 50 , 51 .

Various clinical trials have been performed to investigate the effectiveness of α- blockers for BPH, however direct comparisons among many drugs are still lacking 12 , 13 , 14 . At present, none of the NMA have extensively evaluated the efficacy of these agents within the class in terms of the majority of outcomes (IPSS, QoL, PVR, Qmax) as well as TEAEs. This NMA focused on 22 RCTs, which included 3271 patients randomly assigned to 6 kinds of α-blockers or placebo with 12 dose categories. Our study revealed that among all the α-blocker monotherapy, tamsulosin 0.4 mg is more effective in improving the IPSS, PVR and Qmax, compared to a placebo, as well as the highest-ranked treatment option for these outcomes based on the rank test. Silodosin is considered to be having the highest selectivity for α1A adrenoreceptors in comparison to other α-blockers. In-vitro studies have shown that the affinity of silodosin and tamsulosin for α1A adrenoreceptors over α1B adrenoreceptors was 580-fold and 55-fold respectively. Based on this several clinical trials have also shown that silodosin has greater or comparable efficacy to tamsulosin. However, our NMA contradicts the above observations 13 , 44 and it clearly suggests the so called highly selective α-blocker, silodosin is not superior to tamsulosin in terms of clinical outcomes. This will help urologists in better counselling the BPH patients with regard to efficacy of different α-blockers. All the included α-blockers in our study showed a promising effect in reducing the IPSS. On the other hand, α-blockers did not significantly improve QoL, although they showed numerically better results. Even though, the pairwise comparison has shown that doxazosin 8 mg considerably improves QoL more than other α-blockers and is the highest-ranked treatment choice in the rank test.

Most guidelines routinely recommended using a symptom questionnaire to evaluate the patient's symptoms. IPSS, is the most ordinarily preferred scoring system, which is based on the American Urological Association Symptom Index (AUA-SI) 15 , 16 . It comprises eight questions, seven of which explore urinary symptoms and one on the overall quality of life 52 . All of the included α-blockers significantly reduced IPSS within the first 2 weeks of treatment. Controlled studies suggest that α-blockers often lower the IPSS by 30–40% 47 . In addition to their remarkable efficacy, α-blockers are the least expensive and well-tolerated of the drugs used to treat LUTS 16 , 53 .

The included studies validated the overall safety profile, with the proportion of AEs ranging mild to moderate. The most commonly reported AEs were ejaculation disorder, dizziness, diarrhoea, nasal congestion, drowsiness and postural hypotension. Moreover, for each of the aforementioned α-blockers, dizziness was reported. Wang et al. observed similar findings, stating that the most commonly reported AEs with α-blockers were ejaculation disorders, nasopharyngitis, and vasodilation effects such as asthenia, dizziness, headache and hypotension 15 . As compared to other α-blockers, silodosin elicits a notable number of AEs followed by tamsulosin and doxazosin and the most predominant adverse effects were ejaculation dysfunction, dizziness, and hypotension. In addition to corroborating our findings, investigations on those most recent drug treatments for LUTS also concurred that silodosin have a higher AE profile than the other therapies, exhibiting with a higher rate of ejaculation dysfunction 54 , 55 . However, α-blockers monotherapies are generally safe with relatively few AEs.

This is the first robust network meta-analysis purely focused on α-blockers, considering the majority of outcomes (IPSS, QoL, PVR, Qmax) along with TEAEs. In 2015, Yuan et al. performed a NMA of RCTs for evaluating the comparative effectiveness of monodrug therapies in BPH 16 . However, outcomes such as PVR and QoL were not considered. Moreover, numerous studies were published after 2015 (36.4%), resulting in the up-to-date comparison of interventions. Studies conducted by Lepor et al. found that when comparing different α-blockers, it is imperative to consider that efficacy and safety are dose-dependent. As a result, observed differences in efficacy and toxicity may be related to diverse levels of α1-blockade achieved rather than inherent pharmacological advantages of the specific drug 8 . We compared α-blockers in a dose-dependent way to benefit the comparative efficacy and safety at different dose levels. Furthermore, the selected studies had similar study designs, selection criteria, and patient characteristics with few exception (duration of treatment) thus, supporting exchangeability. Exchangeability across the trials were conceptually considered and the NMA findings were interpreted accordingly. These factors enhance the credibility of the comparisons generated. Besides, the overall quality of the studies selected was found satisfactory.

Although we performed a comprehensive systematic review and NMA of α-blockers, there are still constraints to consider when interpreting the findings. This review focused on four outcomes, but there were limited data available for QoL, PVR, and Qmax as compared to IPSS. The majority of comparisons for outcomes such as PVR and QoL exhibited low certainty of evidence with the CINeMA framework, predominantly implying the risk of bias from the open-label trials and imprecision owing to a relatively small number of trials. Secondly, α-blockers can minimize both storage and voiding LUTS, however, prostate size has no effect in short-term studies (≤ 1 year) 56 , 57 . The conventional clinical treatment for larger prostate size requires a prolonged treatment period 15 . Ipso facto, the limited duration in the included RCTs (50% of studies were ≤ 8 weeks and 45% ≤ 12 weeks) impede the estimation of long-term effects of α-blockers. Furthermore, this study assessed the efficacy and safety of six different kinds of α-blockers, including five drugs approved by the US FDA (terazosin, alfuzosin, doxazosin, silodosin, and tamsulosin) for BPH while naftopidil is only approved in Japan. As a result, the findings of naftopidil cannot be generalised. Furthermore, the majority of studies were conducted in Asian countries, which could impact the broader applicability of the results. The safety of different kinds of α-blockers was not evaluated using NMA due to a lack of information and the diversity of TEAEs. When interpreting the outcomes of this study, it is imperative to consider the imprecision, heterogeneity and incoherence inherent in the effect estimates.

All the included α-blockers showed reduction in IPSS whereas tamsulosin 0.4 mg outperforms the other α-blocker monotherapies in terms of improving IPSS, PVR, and Qmax. Moreover, larger sample sizes along with longer-term studies are required to refine our estimates of IPSS, QoL, PVR, and Qmax among α-blocker users. Silodosin elicits a notable number of AEs however, dizziness was a common AE observed for all α-blockers. Despite the advancing volume of evidence on the α-blocker, there remains a paucity of evidence demonstrating comparative safety in terms of serious and unexpected outcomes. Even though results provide a pragmatic evaluation of six different types of α-blockers that can aid in treatment decisions, direct head-to-head comparisons are required to validate these findings.

Data availability

The datasets gathered in the present study are considered for sharing upon reasonable requests to the corresponding author.

Park, T. & Choi, J. Y. Efficacy and safety of dutasteride for the treatment of symptomatic benign prostatic hyperplasia (BPH): A systematic review and meta-analysis. World J. Urol. 32 (4), 1093–1105 (2014).

Article   CAS   PubMed   Google Scholar  

Kim, J. H. et al. Efficacy and safety of 5 alpha-reductase inhibitor monotherapy in patients with benign prostatic hyperplasia: A meta-analysis. PLoS ONE. 13 (10), e0203479 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Zitoun, O. A. et al. Management of benign prostate hyperplasia (BPH) by combinatorial approach using alpha-1-adrenergic antagonists and 5-alpha-reductase inhibitors. Eur. J. Pharmacol. 883 , 173301 (2020).

Wu, Y. J., Dong, Q., Liu, L. R. & Wei, Q. A meta-analysis of efficacy and safety of the new α1A-adrenoceptor-selective antagonist silodosin for treating lower urinary tract symptoms associated with BPH. Prostate Cancer Prostatic Dis. 16 (1), 79–84 (2013).

Calogero, A. E., Burgio, G., Condorelli, R. A., Cannarella, R. & La Vignera, S. Epidemiology and risk factors of lower urinary tract symptoms/benign prostatic hyperplasia and erectile dysfunction. Aging Male. 22 (1), 12–19 (2019).

Article   PubMed   Google Scholar  

MacDonald, R. et al. Efficacy of newer medications for lower urinary tract symptoms attributed to benign prostatic hyperplasia: A systematic review. Aging Male. 22 (1), 1–11 (2019).

Sun, X. et al. Efficacy and safety of PDE5-Is and α-1 blockers for treating lower ureteric stones or LUTS: A meta-analysis of RCTs. BMC Urol. 18 (1), 30 (2018).

Lepor, H., Kazzazi, A. & Djavan, B. α-Blockers for benign prostatic hyperplasia: The new era. Curr. Opin. Urol. 22 (1), 7–15 (2012).

Zhang, J. et al. Alpha-blockers with or without phosphodiesterase type 5 inhibitor for treatment of lower urinary tract symptoms secondary to benign prostatic hyperplasia: A systematic review and meta-analysis. World J. Urol. 37 (1), 143–153 (2019).

Wang, X. H. et al. Systematic review and meta-analysis on phosphodiesterase 5 inhibitors and α-adrenoceptor antagonists used alone or combined for treatment of LUTS due to BPH. Asian J. Androl. 17 (6), 1022–1032 (2015).

Fusco, F. et al. Alpha-1 adrenergic antagonists, 5-alpha reductase inhibitors, phosphodiesterase type 5 inhibitors, and phytotherapic compounds in men with lower urinary tract symptoms suggestive of benign prostatic obstruction: A systematic review and meta-analysis of urodynamic studies. Neurourol. Urodyn. 37 (6), 1865–1874 (2018).

Perumal, C., Chowdhury, P. S., Ananthakrishnan, N., Nayak, P. & Gurumurthy, S. A comparison of the efficacy of naftopidil and tamsulosin hydrochloride in medical treatment of benign prostatic enlargement. Urol. Ann. 7 (1), 74–78 (2015).

Manohar, C. M. S. et al. Safety and efficacy of tamsulosin, alfuzosin or silodosin as monotherapy for LUTS in BPH—A double-blind randomized trial. Cent. Eur. J. Urol. 70 (2), 148–153 (2017).

CAS   Google Scholar  

Patil, S. B., Ranka, K., Kundargi, V. S. & Guru, N. Comparison of tamsulosin and silodosin in the management of acute urinary retention secondary to benign prostatic hyperplasia in patients planned for trial without catheter. A prospective randomized study. Cent. Eur. J. Urol. 70 (3), 259–263 (2017).

Wang, X. et al. Comparative effectiveness of oral drug therapies for lower urinary tract symptoms due to benign prostatic hyperplasia: A systematic review and network meta-analysis. PLoS ONE. 9 (9), e107593 (2014).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Yuan, J. Q. et al. Comparative effectiveness and safety of monodrug therapies for lower urinary tract symptoms associated with benign prostatic hyperplasia: A network meta-analysis. Medicine 94 (27), e974 (2015).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Shim, S. R., Kim, S. J., Lee, J. & Rücker, G. Network meta-analysis: Application and practice using R software. Epidemiol. Health. 41 , e2019013 (2019).

Capogrosso, P., Salonia, A. & Montorsi, F. Evaluation and Nonsurgical Management of Benign Prostatic Hyperplasia. Campbell-Walsh-Wein Urology 12th edn. (Elsevier, 2021).

Google Scholar  

Higgins, J. P. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343 , d5928 (2011).

Furukawa, T. A., Guyatt, G. H. & Griffith, L. E. Can we individualize the “number needed to treat”? An empirical study of summary effect measures in meta-analyses. Int. J. Epidemiol. 31 (1), 72–76 (2002).

Rücker, G. & Schwarzer, G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med. Res. Methodol. 15 , 58 (2015).

Higgins, J. P. et al. Consistency and inconsistency in network meta-analysis: Concepts and models for multi-arm studies. Res. Synth. Methods. 3 (2), 98–110 (2012).

Higgins, J. P., Thompson, S. G., Deeks, J. J. & Altman, D. G. Measuring inconsistency in meta-analyses. BMJ. 327 (7414), 557–560 (2003).

Chaimani, A., Higgins, J. P., Mavridis, D., Spyridonos, P. & Salanti, G. Graphical tools for network meta-analysis in STATA. PLoS ONE. 8 (10), e76654 (2013).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Papakonstantinou, T., Nikolakopoulou, A., Higgins, J. P., Egger, M. & Salanti, G. Cinema: Software for semiautomated assessment of the confidence in the results of network meta-analysis. Campbell Syst. Rev. 16 (1), e1080 (2020).

Okada, H. et al. A comparative study of terazosin and tamsulosin for symptomatic benign prostatic hyperplasia in Japanese patients. BJU Int. 85 (6), 676–681 (2000).

van Kerrebroeck, P., Jardin, A., Laval, K. U. & van Cangh, P. Efficacy and safety of a new prolonged release formulation of alfuzosin 10 mg once daily versus alfuzosin 25 mg thrice daily and placebo in patients with symptomatic benign prostatic hyperplasia. ALFORTI Study Group. Eur. Urol. 37 (3), 306–313 (2000).

Lapitan, M. C., Acepcion, V. & Mangubat, J. A comparative study on the safety and efficacy of tamsulosin and alfuzosin in the management of symptomatic benign prostatic hyperplasia: A randomized controlled clinical trial. J. Int. Med. Res. 33 (5), 562–573 (2005).

Rahardjo, D. et al. Efficacy and safety of tamsulosin hydrochloride compared to doxazosin in the treatment of Indonesian patients with lower urinary tract symptoms due to benign prostatic hyperplasia. Int. J. Urol. 13 (11), 1405–1409 (2006).

Yokoyama, T., Kumon, H., Nasu, Y., Takamoto, H. & Watanabe, T. Comparison of 25 and 75 mg/day naftopidil for lower urinary tract symptoms associated with benign prostatic hyperplasia: A prospective, randomized controlled study. Int. J. Urol. 13 (7), 932–938 (2006).

Ukimura, O. et al. Naftopidil versus tamsulosin hydrochloride for lower urinary tract symptoms associated with benign prostatic hyperplasia with special reference to the storage symptom: A prospective randomized controlled study. Int. J. Urol. 15 (12), 1049–1054 (2008).

Masumori, N. et al. Ejaculatory disorders caused by alpha-1 blockers for patients with lower urinary tract symptoms suggestive of benign prostatic hyperplasia: Comparison of naftopidil and tamsulosin in a randomized multicenter study. Urol. Int. 83 (1), 49–54 (2009).

Yokoyama, T. et al. Effects of three types of alpha-1 adrenoceptor blocker on lower urinary tract symptoms and sexual function in males with benign prostatic hyperplasia. Int. J. Urol. 18 (3), 225–230 (2011).

Zhang, K. et al. Effect of doxazosin gastrointestinal therapeutic system 4 mg vs tamsulosin 0.2 mg on nocturia in Chinese men with lower urinary tract symptoms: A prospective, multicenter, randomized, open, parallel study. Urology. 78 (3), 636–640 (2011).

Shelbaia, A., Elsaied, W. M., Elghamrawy, H., Abdullah, A. & Salaheldin, M. Effect of selective alpha-blocker tamsulosin on erectile function in patients with lower urinary tract symptoms due to benign prostatic hyperplasia. Urology. 82 (1), 130–135 (2013).

Yamaguchi, K. et al. Silodosin versus naftopidil for the treatment of benign prostatic hyperplasia: A multicenter randomized trial. Int. J. Urol. 20 (12), 1234–1238 (2013).

Kumar, S., Tiwari, D. P., Ganesamoni, R. & Singh, S. K. Prospective randomized placebo-controlled study to assess the safety and efficacy of silodosin in the management of acute urinary retention. Urology. 82 (1), 171–175 (2013).

Keten, T. et al. Determination of the efficiency of 8 mg doxazosin XL treatment in patients with an inadequate response to 4 mg doxazosin XL treatment for benign prostatic hyperplasia. Urology. 85 (1), 189–194 (2015).

Seki, N. et al. Non-inferiority of silodosin 4 mg once daily to twice daily for storage symptoms score evaluated by the International Prostate Symptom Score in Japanese patients with benign prostatic hyperplasia: A multicenter, randomized, parallel-group study. Int. J. Urol. 22 (3), 311–316 (2015).

Matsukawa, Y. et al. Comparison of silodosin and naftopidil for efficacy in the treatment of benign prostatic enlargement complicated by overactive bladder: A randomized, prospective study (SNIPER study). J. Urol. 197 (2), 452–458 (2017).

Chung, J. H. et al. Efficacy and safety of tamsulosin 0.4 mg single pills for treatment of Asian patients with symptomatic benign prostatic hyperplasia with lower urinary tract symptoms: A randomized, double-blind, phase 3 trial. Curr. Med. Res. Opin. 34 (10), 1793–1801 (2018).

Kwon, S. Y. et al. Comparison of the effect of naftopidil 75 mg and tamsulosin 0.2 mg on the bladder storage symptom with benign prostatic hyperplasia: Prospective, multi-institutional study. Urology 111 , 145–150 (2018).

Matsumoto, S., Kasamo, S. & Hashizume, K. Influence of alpha-adrenoceptor antagonists therapy on stool form in patients with lower urinary tract symptoms suggestive of benign prostatic hyperplasia. Low Urin. Tract. Symptoms. 12 (1), 86–91 (2020).

Pande, S., Hazra, A. & Kundu, A. K. Evaluation of silodosin in comparison to tamsulosin in benign prostatic hyperplasia: A randomized controlled trial. Indian J. Pharmacol. 46 (6), 601–607 (2014).

Lerner, L. B. et al. Management of lower urinary tract symptoms attributed to benign prostatic hyperplasia: AUA GUIDELINE PART I-initial work-up and medical management. J. Urol. 206 (4), 806–817 (2021).

Nickel, J. C., Méndez-Probst, C. E., Whelan, T. F., Paterson, R. F. & Razvi, H. 2010 update: Guidelines for the management of benign prostatic hyperplasia. Can. Urol. Assoc. J. 4 (5), 310–316 (2010).

La Vignera, S. et al. Pharmacological treatment of lower urinary tract symptoms in benign prostatic hyperplasia: Consequences on sexual function and possible endocrine effects. Expert Opin. Pharmacother. 22 (2), 179–189 (2021).

Yassin, A. et al. Alpha-adrenoceptors are a common denominator in the pathophysiology of erectile function and BPH/LUTS–implications for clinical practice. Andrologia. 38 (1), 1–12 (2006).

Florent, R., Poulain, L. & N’Diaye, M. Drug repositioning of the α(1)-adrenergic receptor antagonist naftopidil: A potential new anti-cancer drug?. Int. J. Mol. Sci. 21 (15), 5339 (2020).

Masumori, N. Naftopidil for the treatment of urinary symptoms in patients with benign prostatic hyperplasia. Ther. Clin. Risk Manag. 7 , 227–238 (2011).

Yu, Z. J. et al. Efficacy and side effects of drugs commonly used for the treatment of lower urinary tract symptoms associated with benign prostatic hyperplasia. Front. Pharmacol. 11 , 658 (2020).

de la Rosette, J. J. et al. EAU Guidelines on benign prostatic hyperplasia (BPH). Eur. Urol. 40 (3), 256–263 (2001) ( discussion 64 ).

Lepor, H. Alpha-blockers for the treatment of benign prostatic hyperplasia. Urol. Clin. North Am. 43 (3), 311–323 (2016).

Dahm, P. et al. Comparative effectiveness of newer medications for lower urinary tract symptoms attributed to benign prostatic hyperplasia: A systematic review and meta-analysis. Eur. Urol. 71 (4), 570–581 (2017).

Hwang, E. C., Gandhi, S. & Jung, J. H. New alpha blockers to treat male lower urinary tract symptoms. Curr. Opin. Urol. 28 (3), 273–276 (2018).

McVary, K. T. et al. Update on AUA guideline on the management of benign prostatic hyperplasia. J. Urol. 185 (5), 1793–1803 (2011).

Oelke, M. et al. EAU guidelines on the treatment and follow-up of non-neurogenic male lower urinary tract symptoms including benign prostatic obstruction. Eur. Urol. 64 (1), 118–140 (2013).

Download references

Author information

Authors and affiliations.

Department of Pharmacy Practice, National Institute of Pharmaceutical Education and Research (NIPER), S.A.S. Nagar, Mohali, Punjab, India

Beema T Yoosuf, Abhilash Kumar Panda, Muhammed Favas KT, Saroj Kundan Bharti & Dipika Bansal

Department of Urology, Postgraduate Institute of Medical Education and Research, Chandigarh, India

Sudheer Kumar Devana

You can also search for this author in PubMed   Google Scholar

Contributions

BTY contributed to the study concept and design, acquisition and interpretation of data, statistical analysis, drafted the manuscript and final revision and editing. AKP contributed to the acquisition and interpretation of data. MFKT contributed to the statistical analysis and interpretation of data. SKB contributed to the cross-checked the information and revision of the drafted manuscript. Dr. SKD contributed to the critical revision of the manuscript for important intellectual content and supervision. Dr. DB contributed to the study concept, critical revision of the manuscript for important intellectual content and supervision. All authors participated in the manuscript review process and endorsed the final version.

Corresponding author

Correspondence to Dipika Bansal .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Yoosuf, B.T., Panda, A.K., KT, M.F. et al. Comparative efficacy and safety of alpha-blockers as monotherapy for benign prostatic hyperplasia: a systematic review and network meta-analysis. Sci Rep 14 , 11116 (2024). https://doi.org/10.1038/s41598-024-61977-5

Download citation

Received : 27 February 2024

Accepted : 13 May 2024

Published : 15 May 2024

DOI : https://doi.org/10.1038/s41598-024-61977-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Benign prostatic hyperplasia
• International prostate symptom score
• Network meta-analysis
• Quality of life

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Search Menu
• Advance articles
• Editor's Choice
• Supplement Archive
• Cover Archive
• IDSA Guidelines
• IDSA Journals
• The Journal of Infectious Diseases
• Open Forum Infectious Diseases
• Photo Quizzes
• State-of-the-Art Reviews
• Voices of ID
• Author Guidelines
• Open Access
• Why Publish
• Advertising and Corporate Services
• Advertising
• Journals Career Network
• Reprints and ePrints
• Sponsored Supplements
• Branded Books
• About Clinical Infectious Diseases
• About the Infectious Diseases Society of America
• About the HIV Medicine Association
• IDSA COI Policy
• Editorial Board
• Self-Archiving Policy
• For Reviewers
• For Press Offices
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Supplementary data, breakthrough invasive fungal infections in patients with high-risk hematological disorders receiving voriconazole and posaconazole prophylaxis: a systematic review.

Potential conflicts of interest . S. F. D. has received honoraria for consultancy and research funding from AVIR Pharma Inc and Merck & Co. All other authors report no potential conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

• Article contents
• Figures & tables

Catherine-Audrey Boutin, Florence Durocher, Stéphanie Beauchemin, Daniela Ziegler, Claire Nour Abou Chakra, Simon Frédéric Dufresne, Breakthrough Invasive Fungal Infections in Patients With High-Risk Hematological Disorders Receiving Voriconazole and Posaconazole Prophylaxis: A Systematic Review, Clinical Infectious Diseases , 2024;, ciae203, https://doi.org/10.1093/cid/ciae203

• Permissions Icon Permissions

Primary antifungal prophylaxis with mold-active azoles is used to prevent invasive fungal infections in patients with high-risk hematological disorders; however, breakthrough infections occur, and the reasons for treatment failure are still not fully understood. To help inform clinical decisions, we sought to define microbiological, clinical, and pharmacological characteristics of proven and probable breakthrough invasive fungal infections (bIFIs) in patients with high-risk hematological disorders receiving voriconazole or posaconazole prophylaxis.

We performed a systematic review of the literature following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search strategy was last conducted on 19 April 2023.

We assessed 5293 studies for eligibility, and 300 were selected for data extraction. These studies described 1076 cases of bIFIs occurring under voriconazole (42.5%) or posaconazole (57.5%). The most commonly found pathogens were Aspergillus (40%), Mucorales (20%), Candida (18%), and Fusarium (9%) species. Mucorales were more frequent among voriconazole-emerging cases, whereas Aspergillus and Fusarium were more prevalent among posaconazole-emerging cases. Definitive, putative, or probable antifungal resistance was found in 31% of cases. Therapeutic drug monitoring showed subtherapeutic azole concentration in 32 of 90 (36%) cases. Infection-related mortality was reported in 117 cases and reached 35%.

In our systemic review, the most common bIFIs were aspergillosis, mucormycosis, candidiasis, and fusariosis. Antifungal resistance explains only a minority of cases. Subtherapeutic prophylaxis was frequent but rarely reported. Prospective studies are needed to better understand these infections and to establish optimal management.

Invasive fungal infections (IFIs) are associated with increased morbidity and mortality among patients with high-risk hematological disorders [ 1 , 2 ]. Primary antifungal prophylaxis is considered the best strategy to prevent IFI in this patient population and mold-active antifungal agents are preferred for some patients at the highest risk, including those undergoing remission-induction chemotherapy for acute leukemia or receiving allogeneic stem cell transplantation. The broad-spectrum triazoles voriconazole and posaconazole are the most studied and strongly recommended prophylactic agents in that context [ 3–5 ], although itraconazole and isavuconazole are also used in some centers [ 6 , 7 ]. However, primary antifungal prophylaxis with these agents is not completely effective, and breakthrough IFI (bIFI) occurs in 2%–5% of patients [ 8 , 9 ]. Reasons for antifungal prophylaxis failure are not fully understood but may include suboptimal pharmacokinetics or infections with azole-resistant species [ 9–13 ].

Managing bIFI is clinically challenging. There are no controlled trials specifically designed to address best practices in these clinical scenarios. Given the relative rarity of these infections and the scarcity of data, broad trends are difficult to discern from individual clinical trials or single-center experiences; guidelines are mainly based on expert opinion rather than evidence, with some recommending switching antifungal classes or advocating for combination therapies [ 3 , 13 ]. There remains a clinical need to better understand bIFI to establish and provide more evidence-based guidance.

In our present study, we sought to define microbiological, clinical, and pharmacological characteristics of bIFI in patients with high-risk hematological disorders to help inform clinical decisions. We conducted a systematic review targeting probable and proven bIFI in patients receiving mold-active triazole prophylaxis. We focused on voriconazole and posaconazole in accordance with current clinical practice guidelines. Our main objectives were to define the effect of different prophylactic agents on recovered pathogens and characterize the contribution of antifungal resistance and pharmacological failure to bIFI. In our systematic review, we present data on 1076 cases of bIFI.

Systematic Review Methodology

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines [ 14 ]. Our search strategy was developed with a professional librarian (D. Z.) and is available on Borealis Dataverse ( https://borealisdata.ca/dataset.xhtml?persistentId=doi:10.5683/SP3/ADOR5C ). Studies qualified as eligible if they described cases of probable or proven IFI breaking through voriconazole or posaconazole prophylaxis among patients with high-risk hematological disorders and provided enough microbiological information to distinguish between yeast, mold, and endemic fungi. Study selection was performed by independent reviewers using the Covidence platform ( www.covidence.org ). Basic mycological and clinical characteristics of each bIFI case were manually extracted to a database. The risk of bias and certainty was assessed by focusing on the reporting bias. A more extensive systematic review methodology can be found in the Supplementary Materials .

Definitions

We used European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group Education and Research Consortium (EORTC/MSG-ERC) criteria to classify IFI as possible, probable, or proven [ 15 ]. We followed bIFI definitions recently proposed by the MSG-ERC and the European Confederation of Medical Mycology [ 8 ]. Target trough plasma levels for voriconazole and posaconazole were 0.5 mg/L and 0.7 mg/L, respectively [ 16 ]. Further definitions for breakthrough periods, high-risk hematological disorders, and azole resistance can be found in the Supplementary Materials .

Statistical Analyses

We used Fisher exact test or χ 2 test for comparisons of categorical data displayed as proportions. We performed logistic regression for risk analysis using Stata 16 (StataCorp LLC, College Station, Texas). We produced other calculations and figures using Prism 10 (GraphPad Software, San Diego, California). A P value ≤.05 was considered statistically significant. We did not consider a meta-analysis appropriate to address the objectives of this review.

Included Studies and bIFI Cases

We retrieved and assessed for eligibility a total of 5293 entries and selected 300 studies for data extraction. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart ( Figure 1 ) presents an overview of the selection process, and a complete list of included studies is available in Supplementary Table 1 . We found and analyzed 1076 cases of bIFI in the selected studies. The basic characteristics of the included cases are shown in Table 1 .

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart. Abbreviations: CINAHL, Cumulative Index to Nursing and Allied Health; CRD, Centre for Reviews and Dissemination; EBM, evidence-based medicine; HTA, health technology assessment; IFI, invasive fungal infection; POSA, posaconazole; PPx, prophylaxis; VORI, voriconazole. This Figure was created using Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org .

Basic Characteristics of 1076 Cases of Breakthrough Invasive Fungal Infection

Abbreviations: EORTC/MSG-ERC, European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group Education and Research Consortium; HSCT, hematopoietic stem cell transplant; MDS, myelodysplastic syndrome.

a Cases from studies including well-defined high-risk hematological disorders but lacking information at the patient level, or cases from studies including unspecified high-risk hematological disorders.

b Cases from studies including proven/probable cases but lacking information at the patient level.

Clinical and Microbiological Characteristics of bIFI

A total of 1093 fungal pathogens were found when accounting for different organisms involved in 16 mixed infections ( Table 2 ). Overall, molds were markedly predominant (77%), whereas endemic fungi were rare (<1%). Relative frequencies of the most commonly found pathogens were Aspergillus species (40%), Mucorales (20%), Candida species (18%) and Fusarium species (9%). Altogether, these 4 groups accounted for 87% of cases.

Fungal Pathogens (n = 1093) From 1076 Cases of Breakthrough Invasive Fungal Infection a

a Considering 16 mixed infections with 2 or more pathogens.

b Includes A. flavus , A. glaucus , A. nidulans , A. oryzae , A. terreus , and A. versicolor .

c Includes Acremonium , Hormographiella , Macrophomina , Paecilomyces , Penicillium , Rasamsonia , Scopulariopsis , and Verticillium .

d Includes Alternaria , Bipolaris , Cladosporium , Curvularia , and Exserohilum .

e Includes C. famata , C. guilliermondii , C. kefyr , and C. tropicalis .

f Includes isolates reported as Blastoschizomyces and Geotrichum .

g Includes Malassezia and Rhodotorula .

The proportion of main pathogens differed significantly according to the prophylactic agent used ( Figure 2 A ). Mucorales were more frequent among voriconazole-emerging cases, whereas Aspergillus and Fusarium species were more prevalent among posaconazole-emerging cases. Among posaconazole cases, formulation was available for 309 cases and 313 pathogens (suspension, n = 258; tablet, n = 55), and the distribution of main pathogens was not statistically different between both groups (data not shown). In a subset of 706 infections, where the underlying disease was clearly defined, Mucorales and Candida species were more frequent among patients receiving stem cell transplants than those with acute leukemia ( Supplementary Figure 1 ). Other variables influenced the distribution of pathogens, including the risk category of reporting bias ( Supplementary Figure 2 ) and geographic origin ( Supplementary Figure 3 ). We performed a multiple logistic regression to account for confounding effects between variables ( Table 3 ). We found that geographic origin (North America and Asia Pacific) and voriconazole prophylaxis independently associated with a higher probability of finding Mucorales as the pathogen, while acute leukemia was associated with a lower probability. Voriconazole prophylaxis showed the strongest effect, with a risk almost 3 times higher than posaconazole.

Distribution of fungal pathogens according to antifungal agent (n = 1093). P values represent a χ 2 test for observed and expected frequencies.

Logistic Regression Assessing the Risk of Mucorales Infection (n = 1093)

Abbreviations: CI, confidence interval; OR, odds ratio.

The distribution of Aspergillus species also varied significantly under different antifungal agents. Among Aspergillus isolates identified at the species level (n = 61), non- fumigatus species were significantly more frequent in the voriconazole group ( Figure 2 B ). Similarly, the distribution of main yeast genera differed according to the prophylactic agent used, which appeared to be mostly driven by a higher relative frequency of Saprochaete species, including isolates reported as Blastoschizomyces and Geotrichum species, among posaconazole cases ( Figure 2 C ). For yeasts where echinocandins were not considered as a primary treatment or should be avoided according to recent international guidelines [ 17 ] (ie, Saprochaete and various Basidiomycetes including Cryptococcus , Malassezia , Rhodotorula , and Trichosporon species), the proportion was significantly higher for posaconazole than for voriconazole (32/159 [21.5%] vs 10/104 [9.6%]; P = .0157). Finally, we observed a disparity in Candida isolates identified at the species level (n = 122). While Candida glabrata was the dominant species with both antifungal agents, Candida parapsilosis was markedly more frequent among posaconazole-emerging cases ( Figure 2 D ).

Anatomical sites were available for 679 infections. Of these, 309 (45.5%) were localized lung infections, 316 (46.5%) were disseminated, 32 (4.7%) were sinus or sino-orbital infections, 5 (0.7%) were isolated brain infections, and 17 (2.5%) were found in other sites. A relationship between the anatomical site of 454 invasive mold infections and their associated pathogens is shown in Supplementary Table 2 . Of note, the dominant site for mucormycosis cases (42/105 [40%]) was pulmonary.

Antifungal susceptibility testing (AST) was reported in some cases as interpretive categories (resistant, susceptible, intermediate), minimum inhibitory concentration (MIC) data, or both. AST data were available for 120 of 1093 (11%) fungal pathogens, and 105 (10%) included data for the specific agent in use at the time of bIFI. Of these, 65 (62%) were resistant, and 40 (38%) were susceptible to the prophylactic agent. Among cases lacking susceptibility data for the antifungal agent, resistance was deemed highly probable (putative) in 153 infections. First, certain organisms were considered intrinsically resistant to voriconazole (Mucorales, n = 146) or both agents ( Aspergillus ustus complex, n = 5). Putative resistance was also considered in 2 pathogens, based on cross-resistance to other surrogate azole agents for which susceptibility data were reported. In addition, resistance was considered probable in 125 additional infections, considering published susceptibility profiles of such species, including Fusarium species (n = 75), Lomentospora prolificans (n = 5), Rasamsonia argillacea (n = 3), and C. glabrata (n = 42). Altogether, 343 (31%) pathogens were considered definitively resistant (65/1093 [6%]), putatively resistant (153/1093 [14%]), or probably resistant (125/1093 [11%]) to the prophylactic agent from which they were breaking through. Overall, resistant organisms were significantly more common among voriconazole-emerging pathogens (213/466 [45.7%]) compared to posaconazole (130/627 [20.7%]; P < .0001). The difference was largely driven by intrinsic resistance of Mucorales species in the voriconazole group ( Figure 3 A ). Cases with AST data consistently showed a higher frequency of resistance in this group ( Figure 3 B ).

Patterns of resistance according to antifungal agent. A , Distribution of resistance categories among 343 resistant pathogens. B , Susceptibility profile of 105 pathogens with antifungal susceptibility testing data. All putatively resistant pathogens in the voriconazole group (n = 148) represent intrinsically resistant organisms, among which 144 are Mucorales species; P values represent a χ 2 test for observed and expected frequencies.

Therapeutic drug monitoring data were available for only 90 of 1076 (8.4%) cases. Among these, 36% (32/90) were considered subtherapeutic. This proportion was not significantly different between posaconazole (24/67 [36%]) and voriconazole (8/23 [35%]) cases ( P = 1.00). Of 67 posaconazole cases with therapeutic drug monitoring data, 54 cases reported formulation (suspension, n = 24; tablets, n = 30). Subtherapeutic levels were much more frequent among cases with posaconazole suspension (71% vs 13%; P < .0001).

Management and Outcomes of bIFI

Initial therapeutic antifungal agents were reported for 244 of 1076 (23%) cases ( Table 4 ). An antifungal from a different class (polyenes, echinocandins, allylamines, and/or antimetabolites) was started in most cases (82%). An azole agent was used in 38% of cases, either as part of a combination therapy with an agent from a different class or alone. Upfront combination therapy was used in 27% of cases. Yeast infections were more often treated with class switch and monotherapy. Survival data were provided for 337 cases with infection-related mortality in 117 (35%). Attributable mortality was similar regardless of the prophylactic fungal agent used. However, it was lower in cases from studies at low risk of reporting bias compared to high-risk studies (30.4% vs 42.3%; P = .032, Fisher exact test).

Therapeutic Antifungal Agents Utilized in Treating Breakthrough Invasive Fungal Infection Cases (n = 244)

Abbreviation: IFI, invasive fungal infection.

This is the first systematic review of bIFI in patients with high-risk hematological disorders receiving voriconazole and posaconazole prophylaxis. To our knowledge, our systematic review is also the largest collection of published material on bIFI. We found that the most prevalent IFI was aspergillosis, followed by mucormycosis, candidiasis, and fusariosis. Among molds, usually clinically distinct from yeast infections, Mucorales represented over a quarter of pathogens. The relative abundance of mucormycosis and fusariosis contrasts significantly with large epidemiological studies that were not focused on patients receiving mold-active prophylaxis, where they accounted for less than 5%–10% of overall IFI [ 1 , 2 , 18 ]. Overrepresentation of mucormycosis was more evident among voriconazole-emerging cases. However, this is unsurprising considering the known intrinsic resistance to voriconazole in Mucorales. Interestingly, the magnitude of Mucorales abundance was significantly affected by reporting bias. The phenomenon was recognized in early postmarketing studies of voriconazole, including a highly cited report by Marty et al [ 19 ], which has increased awareness of the drug and may have contributed to reporting bias. Nevertheless, mucormycosis was about a quarter of bIFI cases among voriconazole-emerging cases, even in the subgroup of studies at low risk of reporting bias, which was significantly higher than for posaconazole. Our risk analysis also identified voriconazole as independently associated with Mucorales species. The present data confirm the association between voriconazole prophylaxis and mucormycosis on a large scale and estimate relative frequency compared to other fungal pathogens. This is very clinically significant, given the increased mortality (>50%) reported with invasive mucormycosis among patients with high-risk hematological disorders (especially in cases that occur under prophylaxis) [ 18 , 20–23 ], and because mucormycosis has specific treatment strategies that differ from many other fungal infections, including a central role for liposomal amphotericin B and consideration for aggressive surgical debridement [ 24 ].

In addition to voriconazole, we found that some geographic origins, including North America, were associated with the Mucorales species. However, it remains unclear whether this reflects reporting variabilities that persist even if the risk of bias was taken into account or true biologically meaningful differences. Such differences could stem from environmental or climatic factors, host variabilities, or medical practice divergence across regions. Importantly, such factors may vary greatly within much smaller geographic areas and large subdivisions, as presented here, are likely to provide an incomplete picture. Hematopoietic stem cell transplantation was also associated with a higher risk of mucormycosis, which could reflect the prolonged, profound, and multifaceted immunosuppressive state associated with this condition, especially with graft-versus-host-disease.

The relative abundance of non- fumigatus Aspergillus species also contrasted with most epidemiological reports on invasive aspergillosis. This could be explained by the selection of more resistant species (eg, Aspergillus calidoustus ); however, many of the species are not recognized as intrinsically resistant or more prone to develop acquired resistance (eg, Aspergillus versicolor , Aspergillus oryzae ) and most isolates were not tested for antimicrobial susceptibility. Overrepresentation of non- fumigatus Aspergillus species was also more evident among voriconazole-emerging cases, which could also reflect different in vitro and/or in vivo activity of both agents against specific species, although this was not substantiated with antifungal susceptibility data. Other variables not accounted for may also be relevant, including environmental and host factors. Importantly, culture-proven (with speciation) infections represented only 14% of aspergillosis cases, greatly limiting our ability to detail Aspergillus species distribution accurately.

Non- albicans species represented 87% of speciated Candida isolates. While it was not surprising that there were fewer Candida albicans isolates among yeasts compared to other Candida species that are more associated with azole resistance, such as C. glabrata , C. krusei , C. tropicalis and C. parapsilosis , the high relative frequency of C. parapsilosis among posaconazole cases was noteworthy and intriguing. Recently, azole-resistant C. parapsilosis has emerged as a global phenomenon [ 25 ]. It has been shown that some mechanisms leading to posaconazole resistance are distinct from those associated with fluconazole or voriconazole resistance [ 26 ]. Interestingly, in a large collection of clinical isolates from France, resistance to posaconazole was much more frequent (24%) than to voriconazole (2%) or even fluconazole (6%) [ 27 ]. An experimental exposure study partly supported the hypothesis that posaconazole resistance could emerge more rapidly than voriconazole resistance upon exposure to these specific drugs [ 28 ]. Sixteen cases of C. parapsilosis were included in our study, and AST data were provided for only 3 cases, which were all susceptible (wild type) to posaconazole. However, 1 isolate would have been considered resistant, according to the European Committee on Antimicrobial Susceptibility Testing, with an MIC of 0.12 mg/L [ 29 ]. The dominance of Saprochaete species among posaconazole cases was also striking. Saprochaete clavata and Saprochaete capitata are known for intrinsic resistance to echinocandins, but data on susceptibility to broad-spectrum azoles remain scarce [ 30 ]. In the French study, the MICs to voriconazole and posaconazole were generally low, but posaconazole MIC distribution for S. capitata was slightly higher than voriconazole [ 27 ]. Susceptibility data were provided for only 2 of the 22 cases in our data set, 1 each of S. clavata and S. capitata , and both were considered resistant. Additional data are required to confirm the association between posaconazole and these pathogens, as well as the underlying mechanisms.

Understanding the biological mechanisms of bIFI is of high clinical interest. Our present study found that about 20% of bIFI cases involved an organism resistant to the antifungal, with an additional 11% likely resistant. On the other hand, suboptimal dosing of the prophylactic agent was demonstrated in 3% (32/1076) of cases. About a third (34%) of cases displayed either resistance, subtherapeutic dosing, or both, leaving two-thirds of cases unexplained. However, many cases were not properly investigated for a cause of breakthrough. Indeed, in only a minority of cases, AST or therapeutic drug monitoring was performed or reported, providing only a partial understanding. Notably, when performed, more than one-third of azole dosing was below the recognized threshold, reaching more than two-thirds of cases with posaconazole suspension. This could be a contributing factor of bIFI in many cases and illustrates the challenging pharmacokinetics of long-term azole use secondary to drug interactions, hypermetabolic states, and variability in absorption and metabolizer phenotypes. While the need for routine therapeutic drug monitoring in the context of azole prophylaxis is out of the scope of this study, our data certainly militate for measuring drug levels in bIFI cases. Regarding the lack of resistance data, it should be emphasized that current clinical practice largely relies on culture-independent fungal diagnostic methods. At the same time, AST remains almost exclusively phenotypical (culture-dependent), creating a major gap in our ability to detect resistance. Finally, host factors and profound immunological deficiency have been speculated to explain a part of breakthrough infections, but the present study could not evaluate this.

Management of bIFI was characterized by a broad utilization of antifungal class switching, consistent with prevailing expert opinion. However, many cases included an azole as part of initial therapy, and combination therapy was also used in a significant proportion of cases. The present study could not inform on the efficacy of different therapeutic strategies but emphasized the heterogeneity of practices, which may reflect the lack of guidance or the complexity and diversity of clinical scenarios involving bIFI. Providing specific treatment options would fall outside the scope of this work; however, our data underscore the need for empirical coverage of mucormycosis in suspected breakthrough invasive mold infections, especially among patients receiving stem cell transplantation or voriconazole prophylaxis. It should also be emphasized that pulmonary and disseminated mucormycosis was more frequent than sino-orbital mucormycosis in our hematological population, which is consistent with large case series already published [ 31 , 32 ]. In addition, based on the relative abundance of the Saprochaete species and Basidiomycetes yeasts, we would advise against using empirical echinocandin monotherapy in breakthrough yeast fungemia, especially if it occurs under posaconazole prophylaxis.

Outcomes of bIFI are not well defined. The present study found that infection-related mortality was very high, reaching 35%. However, it was reported only in a minority of cases. Also, comparisons with nonbreakthrough IFI would be hazardous considering the multiplicity of different pathogens and the heterogeneity of data on attributable mortality in the literature.

Our study also has several limitations. Most importantly, as a large collection of cases, our study was not designed to provide incidence or efficacy data; any comparison of different antifungal agents in that regard would be hazardous. Also, selected cases represent only a fraction of all bIFIs, and the representativity of this sample is unknown. Probable and proven IFI likely represent only a small subset of all IFIs, since many fungal infections lack mycological evidence and are therefore classified as possible according to widely accepted criteria. By choosing not to include possible cases, we favored specificity and consistency of diagnoses and case findings, which certainly prevented some true cases from being captured in our review. Moreover, only a small proportion of probable and proven IFI are published, and those may be subjected to reporting bias. Despite our best efforts to design a comprehensive search strategy and a rigorous study selection process based on multiple reviewers, we recognize that we may have missed some studies. Incomplete data were a major problem across many variables of interest, including speciation, antifungal resistance, therapeutic drug monitoring, clinical management, and outcomes. Some other potentially relevant variables could not be examined because they were not reported with sufficient granularity to capture data at the patient level. Another limitation lies in the constantly evolving or lack of definitions for IFI and antifungal resistance. When including cases spanning decades, their homogeneity and the external validity of findings are likely to be affected. Finally, many of the included reports were retrospective and, therefore, subject to inherent limits of such study designs, including inconsistent or inaccurate data collection.

Furthermore, our study did not include 2 mold-active triazoles, itraconazole and isavuconazole, which should be noted in addition to the aforementioned limitations. Although some centers currently use these agents as prophylaxis, we chose to follow the most current guidelines from major societies in the fields of oncology and infectious diseases, which strongly favor prophylaxis with voriconazole and posaconazole [ 3–5 ]. A recent systematic review has described bIFI cases occurring in patients with high-risk hematological disorders while receiving isavuconazole prophylaxis. Of the 33 bIFI cases with microbiological data, aspergillosis, mucormycosis, and candidiasis accounted for 42%, 18%, and 24%, respectively, which is similar to our findings [ 33 ].

In this large collection of bIFI in patients with high-risk hematological disorders receiving voriconazole and posaconazole prophylaxis, we found that aspergillosis, mucormycosis, candidiasis, and fusariosis were the most common infections and were associated with high attributable mortality. Antifungal resistance and suboptimal drug exposure explain a proportion of cases, but many are still underinvestigated or unexplained. Prospective studies are needed to better define microbiological, pharmacological, and host characteristics associated with these infections and to establish optimal management. Better diagnostic tools, including culture-independent resistance-detection methods, would likely provide highly valuable insights in these cases.

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Author Contributions. Conceptualization: S. F. D. and D. Z. Data curation: S. F. D., C.-A. B., D. Z., and C. N. A. C. Formal analysis: S. F. D., C.-A. B., D. Z., and C. N. A. C. Funding acquisition: S. F. D. Investigation: S. F. D., C.-A. B., D. Z., F. D., and S. B. Methodology: S. F. D., C.-A. B., D. Z., C. N. A. C. Project administration: S. F. D. and S. B. Writing—original draft: S. F. D. and C.-A. B. Writing—review and editing: all authors.

Acknowledgments. The authors thank Kyle Vaughn Roerick, MA, for his thorough copy editing and insightful structural editing of the manuscript and supporting documents.

Data availability. The authors confirm that the data supporting the findings of this study are available within the article. Additional data are available from the corresponding author, S. F. D., upon reasonable request.

Financial support . This work was partly supported by an investigator-initiated research grant (to S. F. D.) from AVIR Pharma Inc. AVIR Pharma Inc holds a distribution agreement with Basilea Pharmaceutica Ltd for Cresemba (isavuconazole) in Canada.

Kontoyiannis   DP , Marr   KA , Park   BJ , et al.    Prospective surveillance for invasive fungal infections in hematopoietic stem cell transplant recipients, 2001–2006: overview of the Transplant-Associated Infection Surveillance Network (TRANSNET) database . Clin Infect Dis   2010 ; 50 : 1091 – 100 .

Google Scholar

Azie   N , Neofytos   D , Pfaller   M , Meier-Kriesche   HU , Quan   SP , Horn   D . The PATH (Prospective Antifungal Therapy) Alliance registry and invasive fungal infections: update 2012 . Diagn Microbiol Infect Dis   2012 ; 73 : 293 – 300 .

Patterson   TF , Thompson   GR , Denning   DW  3rd , et al.    Practice guidelines for the diagnosis and management of aspergillosis: 2016 update by the Infectious Diseases Society of America . Clin Infect Dis   2016 ; 63 : e1 – 60 .

Baden   LR , Swaminathan   S , Almyroudis   NG , et al.  NCCN guidelines version 2.2023 prevention and treatment of cancer-related infections. National Comprehensive Cancer Network, Inc, 2023. Available at: NCCN.org . Accessed 13 February 2024.

Ullmann   AJ , Aguado   JM , Arikan-Akdagli   S , et al.    Diagnosis and management of Aspergillus diseases: executive summary of the 2017 ESCMID-ECMM-ERS guideline . Clin Microbiol Infect   2018 ; 24 : e1 – 38 .

Lindsay   J , Othman   J , Kong   Y , et al.    SUBA-itraconazole for primary antifungal prophylaxis after allogeneic hematopoietic cell transplantation . Open Forum Infect Dis   2021 ; 8 : ofab502 .

Nguyen   MH , Ostrosky-Zeichner   L , Pappas   PG , et al.    Real-world use of mold-active triazole prophylaxis in the prevention of invasive fungal diseases: results from a subgroup analysis of a multicenter national registry . Open Forum Infect Dis   2023 ; 10 : ofad424 .

Cornely   OA , Hoenigl   M , Lass-Flörl   C , et al.    Defining breakthrough invasive fungal infection—position paper of the Mycoses Study Group Education and Research Consortium and the European Confederation of Medical Mycology . Mycoses   2019 ; 62 : 716 – 29 .

Jenks   JD , Cornely   OA , Chen   SC , Thompson   GR  3rd , Hoenigl   M . Breakthrough invasive fungal infections: who is at risk?   Mycoses   2020 ; 63 : 1021 – 32 .

Maschmeyer   G , Patterson   TF . Our 2014 approach to breakthrough invasive fungal infections . Mycoses   2014 ; 57 : 645 – 51 .

Biehl   LM , Vehreschild   JJ , Liss   B , et al.    A cohort study on breakthrough invasive fungal infections in high-risk patients receiving antifungal prophylaxis . J Antimicrob Chemother   2016 ; 71 : 2634 – 41 .

Lamoth   F , Chung   SJ , Damonti   L , Alexander   BD . Changing epidemiology of invasive mold infections in patients receiving azole prophylaxis . Clin Infect Dis   2017 ; 64 : 1619 – 21 .

Girmenia   C , Busca   A , Candoni   A , et al.    Breakthrough invasive fungal diseases in acute myeloid leukemia patients receiving mould active triazole primary prophylaxis after intensive chemotherapy: an Italian consensus agreement on definitions and management . Med Mycol   2019 ; 57 : S127 – 37 .

Page   MJ , McKenzie   JE , Bossuyt   PM , et al.    The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . BMJ   2021 ; 372 : n71 .

Donnelly   JP , Chen   SC , Kauffman   CA , et al.    Revision and update of the consensus definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer and the Mycoses Study Group Education and Research Consortium . Clin Infect Dis   2020 ; 71 : 1367 – 76 .

McCreary   EK , Davis   MR , Narayanan   N , et al.    Utility of triazole antifungal therapeutic drug monitoring: insights from the Society of Infectious Diseases Pharmacists: endorsed by the Mycoses Study Group Education and Research Consortium . Pharmacotherapy   2023 ; 43 : 1043 – 50 .

Chen   SC , Perfect   J , Colombo   AL , et al.    Global guideline for the diagnosis and management of rare yeast infections: an initiative of the ECMM in cooperation with ISHAM and ASM . Lancet Infect Dis   2021 ; 21 : e375 – 86 .

Pagano   L , Caira   M , Candoni   A , et al.    The epidemiology of fungal infections in patients with hematologic malignancies: the SEIFEM-2004 study . Haematologica   2006 ; 91 : 1068 – 75 .

Marty   FM , Cosimi   LA , Baden   LR . Breakthrough zygomycosis after voriconazole treatment in recipients of hematopoietic stem-cell transplants . N Engl J Med   2004 ; 350 : 950 – 2 .

Yang   N , Zhang   L , Feng   S . Clinical features and treatment progress of invasive mucormycosis in patients with hematological malignancies . J Fungi (Basel)   2023 ; 9 : 592 .

Muthu   V , Agarwal   R , Dhooria   S , et al.    Has the mortality from pulmonary mucormycosis changed over time? A systematic review and meta-analysis . Clin Microbiol Infect   2021 ; 27 : 538 – 49 .

Puerta-Alcalde   P , Monzó-Gallo   P , Aguilar-Guisado   M , et al.    Breakthrough invasive fungal infection among patients with haematologic malignancies: a national, prospective, and multicentre study . J Infect   2023 ; 87 : 46 – 53 .

Axell-House   DB , Wurster   S , Jiang   Y , et al.    Breakthrough mucormycosis developing on mucorales-active antifungals portrays a poor prognosis in patients with hematologic cancer . J Fungi (Basel)   2021 ; 7 : 217 .

Cornely   OA , Alastruey-Izquierdo   A , Arenz   D , et al.    Global guideline for the diagnosis and management of mucormycosis: an initiative of the European Confederation of Medical Mycology in cooperation with the Mycoses Study Group Education and Research Consortium . Lancet Infect Dis   2019 ; 19 : e405 – 21 .

Daneshnia   F , de Almeida Júnior   JN , Ilkit   M , et al.    Worldwide emergence of fluconazole-resistant Candida parapsilosis : current framework and future research roadmap . Lancet Microbe   2023 ; 4 : e470 – 80 .

Franconi   I , Rizzato   C , Poma   N , Tavanti   A , Lupetti   A . Candida parapsilosis sensu stricto antifungal resistance mechanisms and associated epidemiology . J Fungi (Basel)   2023 ; 9 : 798 .

Desnos-Ollivier   M , Lortholary   O , Bretagne   S , Dromer   F . Azole susceptibility profiles of more than 9,000 clinical yeast isolates belonging to 40 common and rare species . Antimicrob Agents Chemother   2021 ; 65 : e02615-20 .

Pinto e Silva   AT , Costa-de-Oliveira   S , Silva-Dias   A , Pina-Vaz   C , Rodrigues   AG . Dynamics of in vitro acquisition of resistance by Candida parapsilosis to different azoles . FEMS Yeast Res   2009 ; 9 : 626 – 33 .

The European Committee on Antimicrobial Susceptibility Testing. Breakpoint tables for interpretation of MICs for antifungal agents, version 10.0, 2020. Available at: http://www.eucast.org/astoffungi/clinicalbreakpointsforantifungals/ .

El Zein   S , Hindy   JR , Kanj   SS . Invasive Saprochaete infections: an emerging threat to immunocompromised patients . Pathogens   2020 ; 9 : 922 .

Jeong   W , Keighley   C , Wolfe   R , et al.    The epidemiology and clinical manifestations of mucormycosis: a systematic review and meta-analysis of case reports . Clin Microbiol Infect   2019 ; 25 : 26 – 34 .

Roden   MM , Zaoutis   TE , Buchanan   WL , et al.    Epidemiology and outcome of zygomycosis: a review of 929 reported cases . Clin Infect Dis   2005 ; 41 : 634 – 53 .

Ishida   K , Haraguchi   M , Kimura   M , et al.    Breakthrough fungal infection after isavuconazole primary prophylaxis in patients with hematologic malignancy and hematopoietic stem cell transplant: systematic review . Open Forum Infect Dis   2023 ; 10 : ofad500.2320 .

Author notes

• antifungal agents
• aspergillosis
• candidiasis
• systemic mycosis
• hematological diseases
• mucormycosis
• pathogenic organism
• voriconazole
• posaconazole

Supplementary data

Email alerts, more on this topic, related articles in pubmed, citing articles via, looking for your next opportunity.

• Recommend to your Library

Affiliations

• Online ISSN 1537-6591
• Print ISSN 1058-4838
• Copyright © 2024 Infectious Diseases Society of America
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.