Journal-based CME Course: The Fragility of Statistically Significant Results in Randomized Clinical Trials for COVID-19

Activity ID

12656

Expires

March 18, 2025

Format Type

Journal-based

CME Credit

1

Fee

$30

CME Provider: JAMA Network Open

Description of CME Course

Importance  Interpreting results from randomized clinical trials (RCTs) for COVID-19, which have been published rapidly and in vast numbers, is challenging during a pandemic.

Objective  To evaluate the robustness of statistically significant findings from RCTs for COVID-19 using the fragility index.

Design, Setting, and Participants  This cross-sectional study included COVID-19 trial articles that randomly assigned patients 1:1 into 2 parallel groups and reported at least 1 binary outcome as significant in the abstract. A systematic search was conducted using PubMed to identify RCTs on COVID-19 published until August 7, 2021.

Exposures  Trial characteristics, such as type of intervention (treatment drug, vaccine, or others), number of outcome events, and sample size.

Main Outcomes and Measures  Fragility index.

Results  Of the 47 RCTs for COVID-19 included, 36 (77%) were studies of the effects of treatment drugs, 5 (11%) were studies of vaccines, and 6 (13%) were of other interventions. A total of 138 235 participants were included in these trials. The median (IQR) fragility index of the included trials was 4 (1-11). The medians (IQRs) of the fragility indexes of RCTs of treatment drugs, vaccines, and other interventions were 2.5 (1-6), 119 (61-139), and 4.5 (1-18), respectively. The fragility index among more than half of the studies was less than 1% of each sample size, although the fragility index as a proportion of events needing to change would be much higher.

Conclusions and Relevance  This cross-sectional study found a relatively small number of events (a median of 4) would be required to change the results of COVID-19 RCTs from statistically significant to not significant. These findings suggest that health care professionals and policy makers should not rely heavily on individual results of RCTs for COVID-19.

Disclaimers

1. This activity is accredited by the American Medical Association.
2. This activity is free to AMA members.

Register for this Activity