Pearls of meta-analyses and systematic review in scientific evidence

Abstract

Scientific evidence develops bit by bit from case reports, case series; to larger case-control, case-cohort; and further escalate to randomized controlled trials. This echoed the importance of continue publishing World journal of Clinical Cases, where novel and advancing discoveries start from a single case. In contrast, at the other end of the realm of evidence synthesis, systematic review and meta-analysis represent distinct yet interconnected processes. Butorphanol in epidural labor analgesia has long been studied since 1989, and with 70 publications from MEDLINE searches. However, there was no meta-analysis, nor any systematic review published so far. The latest in-press article published by Tang et al. on the protocol for the systematic review and meta-analysis on the safety and effectiveness of butorphanol in epidural labor analgesia is encouraging. We believe the findings of this study will be valuable for clinical practice as well as for future research.

Key Words: Meta-analysis, Systematic review, Methodology, Research, Journal, Academic

Core Tip: Scientific evidence evolves from case reports to randomized controlled trials. Systematic reviews and meta-analyses are crucial but can be flawed due to limited information, the so called: “Garbage in, Garbage out”. In running the meta-analysis, mathematical pooling can be invalidated by study heterogeneity, poor study quality, and publication bias. Remedial measures include exploring heterogeneity, restricting meta-analysis to systematic review in case of insufficient information, and addressing different types of bias. Remember, the pearl of meta-analyses relies on quality over quantity in evidence synthesis.

INTRODUCTION

We read with interest the protocol for the higher level of evidence titled “Safety and effectiveness of butorphanol in epidural labor analgesia: A protocol for a systematic review and meta-analysis”[1]. This is an interesting topic since PubMed first available publications on butorphanol in epidural labor analgesia was back in 1989[2] and the latest one was in 2023[3]. Across the 34 years of scientific literature, there were 70 publications from MEDLINE searches, while none was meta-analysis nor any systematic review. Therefore, we are excited to see the higher level of evidence is ongoing under research on this topic.

Scientific evidence develops bit by bit from case reports[4], case series; to larger case-control, case-cohort; and further escalate to randomized controlled trials. This echoed the importance of continue publishing World journal of Clinical Cases, where novel and advancing discoveries start from a single case. In contrast, at the other end of the realm of evidence synthesis, systematic review and meta-analysis represent distinct yet interconnected processes. While systematic review entails a comprehensive examination of literature, meta-analysis involves the mathematical amalgamation of data. Despite their integral roles, not all systematic reviews and meta-analyses rest on foundations of high-quality evidence; rather, many are crafted based on limited information. This situation has the potential to breed inappropriate or premature mathematical pooling of data, consequently yielding conclusions that are invalid or unstable.

SITUATIONS INVALIDATING MATHEMATICAL POOLING

Tang et al[1] has properly followed the standard Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols[5]. However, authors should take caution that upon analysis several circumstances can render the mathematical pooling of results inappropriate or incorrect within the context of systematic reviews and meta-analyses.

Heterogeneity of studies

The presence of clinical or statistical heterogeneity arising from diverse factors such as studied population, intervention, comparator, or outcomes may challenge the appropriateness of meta-analysis. Severe statistical heterogeneity (I-squared statistic > 60%) warrants the abandonment of meta-analysis, with a necessity to explore heterogeneity superseding the pooling of results[6].

Poor quality of studies

The inclusion of results from poor-quality studies characterized by a high degree of bias poses challenges to drawing meaningful conclusions. Authors grapple with the need for heightened rigor in assessing bias, particularly when faced with inadequate information or limited expertise in bias evaluation.

Publication bias and sample size limitations

Selective publication of positive studies, exclusion of negative studies, and the constraint of limited studies or sample sizes pose significant threats to the validity of meta-analyses. Such limitations impede the use of essential tools like funnel plots, hindering the ability to gauge publication bias accurately[7].

CORRECTIVE MEASURES FOR VALIDITY AND RELIABILITY

Tang et al[1] stated in their methodology that a third author will settle any disputes that arise throughout the verification process, and that any differences between the two writers will be settled by discussion with a third author. This is a good practice upon researching the higher level of evidence. To ensure the integrity of meta-analyses, it is imperative to implement appropriate remedial measures when faced with situations that could compromise the accuracy and validity of pooled results. These measures include.

Exploration of heterogeneity

Authors should diligently explore heterogeneity through techniques like subgroup analysis, sensitivity analysis, and meta-regression to address unexplained heterogeneity, a pervasive threat to the reliability of meta-analyses.

Restriction of meta-analysis

In instances of insufficient information or a paucity of trials with smaller sample sizes, authors should consider restricting meta-analysis, placing greater emphasis on qualitative aspects of systematic review. Waiting for the publication of larger trials can forestall the hasty release of poor-quality meta-analyses.

Addressing bias

Confronting prevailing bias in included studies necessitates stratified analysis or the exclusion of low-quality studies to prevent the propagation of misleading results[8].

EMBRACING RESPONSIBILITY IN EVIDENCE SYNTHESIS

While meta-analysis stands as a potent tool for summarizing and synthesizing data, evaluating study quality, heterogeneity, potential bias, and other limitations remains crucial before embarking on this analytical endeavor. Notably, not all systematic reviews require a meta-analysis, emphasizing the responsibility incumbent upon researchers to wield this powerful technique judiciously. With the published protocol, we look forward to the final results output from this meta-analysis and the systematic review conclusions on butorphanol use in epidural labor analgesia. We believe the findings of this study will be valuable for clinical practice as well as for future research.

CONCLUSION

The pearl of meta-analyses relies on quality over quantity in evidence synthesis. Their validity and reliability hinge on the careful consideration of potential pitfalls and the diligent application of corrective measures. The pursuit of high-quality evidence should remain at the forefront of these processes, ensuring that the conclusions drawn are both valid and meaningful.