Saving Lives by Modifying the Process of Science: Estimated Historical Mortality Associated with the Failure to Conduct Routine Prospective Cumulative Systematic Reviews

Background

In 1992, Antman and colleagues published a pioneering study demonstrating that knowledge of the effectiveness, ineffectiveness, or harm associated with 15 interventions to reduce mortality among patients who had suffered acute myocardial infarctions (MIs) could have improved medical practice [1]. Effective interventions could have been, but were not adopted, ineffective and harmful interventions could have been, but were not, discontinued. Effective medical practices were delayed for years-even decades because ongoing systematic reviews of the state of knowledge were not conducted as new research emerged. The methodology of “living systematic reviews” (LSR) has recently been developed for the conduct of cumulative meta-analyses. To demonstrate the benefits of this approach, we use data from the Antman study to estimate the mortality likely to have resulted from the failure to use LSR [2].

Patient and Observation

The 2-year-old male child had consulted at the Medical Center of the company RVA for a prolonged cough and fever that had progressed for 4 weeks, night sweats and weight loss. His background reveals that he was born quietly, weighing 2,800kg. The child was breast fed and with cow’s milk. He is 3rd in the family and the vaccination schedule is well respected. Her father is employed in a state company and her mother is a housewife. The history of her illness dates back to 4 weeks of the consultation for the general signs mentioned above. He was treated successively with Clamoxyl, Bactrim, Cefuroclav and Palucur without improvement.

Methods

We estimate mortality associated with four interventions analyzed by Antman: three interventions that reduce mortalityintravenous vasodilators administered during hospitalization, and ß-blockers and aspirin administered during and after hospitalization, and one intervention that increases mortality- Class 1 anti-arrhythmic drugs [1,3]. Antman’s study reports the year in which cumulative meta-analysis first indicated benefit (or harm) for each intervention, and the year in which the intervention became routine in practice (or was eliminated because it was found to be ineffective or harmful) [1]. Routine practice was assessed by examination of reviews and texts focused on the intervention published each year. Published reviews and texts were classified as: recommending use routinely, recommending use in specific circumstances, recommending use rarely or never, or as experimental or not mentioned.

We used information from available studies about mortality from acute MI during and after hospitalization to estimate mortality attributable to failure to use (or use of) each intervention assessed. While the numbers of deaths associated with MI have changed over the study period, we use the number of deaths at the approximate Antman study period midpoint, i.e., 1980, to estimate attributable mortality. We use the method of population attributable risk (PAR) to estimate the number of deaths that might have been averted [4].

PAR = P_e (RR-1) / ((Pe (RR-1)) + 1),

where P_e is the prevalence of practice nonuse, and RR is the relative risk of death associated with nonuse of the practice. With 100% nonuse, the equation becomes PAR = (RR-1)/RR. In sensitivity analyses, we assess the benefits of partial adoption, i.e., P_e <100%, or changing other parameters. We estimate RR as the inverse of the odds ratio.

We also assessed research that may have been unnecessary had cumulative meta-analysis been used and the delay in adoption of demonstrably effective interventions by practitioners [4]. From Antman’s analysis, we report the number of RCTs that followed finding of effectiveness, the number of patients in these trials, and the delay between the year of cumulative meta-analysis finding and evidence of practice. Because conditions surrounding MI in the U.S. have changed since the publication of Antman’s study, we conducted a sensitivity analysis for one of the interventions reviewed-the use of aspirin. We varied the prevalence of use of the interventions assessed, included all (rather than only first) MIs within ICD code 410, and considered the possibility that effect sizes were half of what was found by Antman.

Mortality associated with MI

We used an estimate of the number of MI patients in the population with an acute MI (ICD code 410) in 1980. We use fatality rates for first and subsequent MIs from a synthesis of estimates of associated pre-hospital, in-hospital, and post-hospital mortality rates to reconstruct the number of first and subsequent MIs in 1980, in-hospital deaths, and deaths within 3 years post-discharge [5,6] (Table 1). We assume that the reporting of ICD 410 is for first MI and reconstruct subsequent MIs using proportions from other sources. (https://www.cdc.gov/heartdisease/facts.htm) In a sensitivity analysis, we assume ICD 410 includes first and subsequent MIs. Approximately 225,000 MI patients died before reaching the hospital, i.e., 49% of deaths (Table 1). There were an estimated 95,000 in-hospital deaths (15.2% of patients admitted to the hospital) and 137,000 (25.9% of patients discharged from the hospital) post-hospital deaths in 1980 (Table 1).

Table 1: Estimating In- and Post- Hospital MI Total Deaths in 1980.

Results and Discussion

Estimating mortality attributable to intervention nonuse/use

In the hospital setting, the annual number of deaths associated with the non-use of interventions that would have been available had cumulative meta-analyses been conducted range from 12,000 for the non-use of intravenous or oral ß-blockers to 41,000 for the non-use of intravenous vasodilators (Table 2).

In the post-hospital setting, the annual number of deaths attributable to failure to use secondary preventive measures that could have been available are 14,000 for the non-use of anti-platelet drugs (predominantly aspirin) and 26,000 for the non-use of oral ß-blockers (Table 2). The use of type I antiarrhythmic drugs was found to be harmful, with a summary odds ratio of 1.28 (Table 2). Their routine use is estimated to be associated with 39,000 deaths annually (Table 2).

Table 2: Annual Deaths Attributable to Non-Use of Interventions for Acute MI in A. Acute Treatment, and B. 3 Years Post-Discharge.

Failure to use prospective cumulative meta-analysis also resulted in a number of RCTs that were conducted after a significant effect could have been assessed, the enrollment of patients in these RCTs, and delays in the use of effective interventions (Table 2). The number of RCTs ranged from 2 (for aspirin and intravenous and oral ß-blockers in acute MI care) to 13 trials (for oral ß- blockers for hospital MI care). The number of patients enrolled in RCTS subsequent to a cumulative meta-analysis finding of benefit ranged from 296 (for aspirin in acute MI care) to 16,616 (for oral ß-blockers for hospital MI care). Delays in the use of effective interventions ranged from two years (for the use of aspirin in acute MI care) to 13 years (for the use of intravenous vasodilators in acute MI care).

Sensitivity Analyses

Table 3: Sensitivity Analyses for Aspirin Use In- and After-Hospitalization.

We modify several parameters in our analysis to determine effects on estimated deaths, with the example of aspirin use. First, we ask how many deaths would have occurred if use had been 50% instead of 0% (Table 3). Annual in-hospital deaths would have been approximately 13,000, and after-hospital 12,000, and after-hospital deaths would have been approximately 7,000 (Table 3). Then we ask what would have happened if both first and subsequent MIs were reported in ICD code 410 (Table 3). Annual in-hospital deaths would be approximately 13,000, and after-hospital deaths approximately 7,500 (Table 3). Finally, we ask what would have happened were the effect sizes reduced by 50%. Annual in-hospital deaths would have been approximately 6,600, and after-hospital deaths approximately 3,700 (Table 3).

Conclusions

The mortality costs of the failure to routinely conduct prospective cumulative systematic reviews, i.e., “living systematic reviews,” are very high. Even when underlying parameters are reduced, estimated annual deaths remain high. There are other costs-the financial, opportunity, and human costs of continued randomized trials when the basic question has already been answered, and the moral cost of not deploying the best available treatment.

The data sources for the present analysis are less than optimal and our analysis rests on unverifiable assumptions. Feinlieb noted in 1984, “Unfortunately, this country has no method or system for standardized complete reporting of new MIs and no incidence data representative of the national population”-a situation that has not changed [7]. Estimates of the incidence of MIs range between 450,000 and 600,000, indicating that our estimate is consistent with other reports. In addition, data are not available on interventions actually used or on patient adherence to treatments, particularly after discharge [7,8]. Moreover, there are likely interactions among interventions that might affect outcomes. Nevertheless, while the estimation of mortality in this analysis required many simplifying assumptions, the number of deaths from failure to use existing information and apply it is unarguably large.

The circumstances of MI have changed greatly since the period examined here. The incidence of MIs has declined, due partly to the interventions reviewed by Antman and because of changes in population behavior, e.g., smoking [9]. Our purpose here is not to portray the current state of MIs, but to use historical data to indicate how the standard practice of science may severely hinder effective knowledge and practice and lead to unnecessary harm and unneeded research [10,11]. The process of building knowledge and applying it in medicine and public health needs fundamental revision. Ongoing cumulative meta-analysis should be routine but requires a process of prioritization and systematic methods. The development of living systematic reviews establishes an essential foundation for this project [2,11].

References

• Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992) A comparison of results of meta-analyses of JAMA 268(2): 240-248.
• Elliott JH, Turner T, Clavisi O, James T, Julian PTH, et al. (2014) Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS med 11(2): e1001603.
• Lau J, Antman EM, Jimenez SJ, Kupelnick B, Mosteller F, et al. (1992) Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327(4): 248-254.
• Boston University School of Public Health (2018) Measuring Association in Case-Control Studies.
• McCarthy E (1981) Inpatient utilization of short-stay hospitals by diagnosis, United States, 1978. Vital Health Stat 13 55: 1-79.
• Law MR, Watt HC, Wald NJ (2002) The underlying risk of death after myocardial infarction in the absence of treatment. Arch Internal Med 162(21): 2405-2410.
• Feinleib M (1984) Changes in cardiovascular epidemiology since 1950. Bull N Y Acad Med 60(5): 449-464.
• National Institutes of Health.‘Morbidity & mortality: 2012 chart book on cardiovascular, myocardial infarction in the Cardiovascular Research Network (CVRN). The American journal of medicine 2017; 130: 317-327.
• Floyd KC, Yarzebski J, Spencer FA, Darleen L, James ED, et al. (2009) A 30-year perspective (1975–2005) into the changing landscape of patients hospitalized with initial acute myocardial infarction: Worcester Heart Attack Study. Circ Cardiovasc Qual Outcomes 2(2): 88-95.
• Reynolds K, Go AS, Leong TK, Denise MB, Andrea ECB, et al. (2017) Trends in incidence of hospitalized acute myocardial infarction in the Cardiovascular Research Network (CVRN). Am J Med 130(3): 317-327.
• Garner P, Hopewell S, Chandler J, Harriet ML, Holger JS, et al. When and how to update systematic reviews: consensus and checklist. BMJ 20: 354.