The frequency of medical reversals in a cross-sectional analysis of high-impact oncology journals, 2009–2018
BMC Cancer volume 21, Article number: 889 (2021)
Identifying ineffective practices that have been used in oncology is important in reducing wasted resources and harm. We sought to examine the prevalence of practices that are being used but have been shown in RCTs to be ineffective (medical reversals) in published oncology studies.
We cross-sectionally analyzed studies published in three high-impact oncology medical journals (2009–2018). We abstracted data relating to the frequency and characterization of medical reversals.
Of the 64 oncology reversals, medications (44%) represented the most common intervention type (39% were targeted). Fourteen (22%) were funded by pharmaceutical/industry only and 56% were funded by an organization other than pharmaceutical/industry. The median number of years that the practice had been in use prior to the reversal study was 9 years (range 1–50 years).
Here we show that oncology reversals most often involve the administration of medications, have been practiced for years, and are often identified through studies funded by non-industry organizations.
Medical reversals, which are practices employed outside of clinical trials that are later found to be no better than a prior or lesser standard of care in a randomized controlled trial (RCT), are present in all medical disciplines . Oncology is an important field to study the phenomenon of medical reversals, because it often includes the care of lethal and feared diseases, where patients and physicians may be willing to engage in treatments without support from prior RCTs, especially for individuals with advanced, metastatic or incurable cancers with few or no satisfactory treatment options .
If such practices are later validated, patients have benefit from early access to a promising intervention. If such practices are later refuted, costs and downsides include (a) opportunity costs, (b) side effects or off target effects of therapy, and (c) financial burden. Identifying ineffective practices that are being or have been used in oncology practice is especially important—not only to reduce wasted resources but to also reduce harm that often accompany ineffective practices.
We have previously characterized a group of almost 400 practices found in high impact factor medical journals, which are considered medical reversals , and of which approximately 6% were oncology practices. It is our purpose in this study to examine the prevalence of medical reversals in high impact factor oncology journals and to identify those specific practices that are being used but have been shown in RCTs to be ineffective.
We reviewed all articles published in high impact factor oncology journals during a 10-year period – Lancet Oncology, Journal of Clinical Oncology (JCO), and Journal of the American Medical Association (JAMA) Oncology. The journals were chosen based on impact factor, specifically the 5-year Hirsch index for medical journals (https://jcr.clarivate.com/JCRJournalHomeAction.action?), for the year 2018 and those journals that published an average of at least 20 RCTs per year. From these articles, we selected all RCTs under the heading of “Articles” in Lancet Oncology, “Original Reports” in JCO, and “Original Investigation” for JAMA Oncology. Selected articles needed to report the results in the overall study population. We excluded articles that reported stratified or subgroup analysis only or that did not analyze the outcome in randomized groups. We also excluded articles that did not test a medical intervention or that were pharmacodynamics studies.
We used articles published between January 1, 2009 and December 31, 2018. Because JAMA Oncology’s inception was April 2015, we only looked at articles between then and December 31, 2018 for this journal.
Data extraction and coding
For each RCT, we coded each medical practice being tested as novel or established at the time of the study. We defined established as being those practices having published proof of being used outside of clinical trials or were embodied in clinical guidelines that supported the use of the practice. We also coded articles as having positive, negative, or inconclusive results. A study was considered positive if it met its primary endpoint – significantly better for superiority trials and no worse for equivalence and noninferiority trials. Studies were coded as inconclusive if there was no clear benefit or harm from the intervention (e.g., progression free survival was better but quality of life was worse). Studies were considered negative if the study did not meet statistical significance in its primary endpoint. In studies that had both overall survival and progression-free survival as primary endpoints, overall survival was the outcome we used to determine whether the study met its endpoint.
For each final reversal study, we abstracted the funding source, year published, setting of the intervention, tumor type, and type of intervention. Funding source was categorized as industry only, non-industry only, or a combination of the two. The type of intervention was coded as medication, procedure, device, screening test, vitamins/supplements/food, behavioral therapy, radiation, or optimization. We considered optimization as a trial that compared different ways to administer an intervention (different doses, different orders). The intervention type “medication” was further categorized into cytotoxic, targeted, hormonal, radioactive, or other. We also created a variable that indicated roughly how long, at a minimum, the practice had been used outside of clinical trials. This documentation for all usage needed to come from the published scientific literature. We found the duration of use by abstracting either the date within the publication or the date of publication showing that the practice had been used (a more conservative estimate of duration of use) and finding the difference between this date and the date of reversal study publication. If the proof of evidence occurred during the same year as the study publication, the duration of the practice was coded as 1 year.
For all steps of study selection, two of four reviewers (D.H., A.H., T.C., J.G.) independently examined and abstracted information for each article. When there were differences in opinion between the two reviewers, adjudication first involved discussion between the two readers to see whether agreement could be reached. If disagreement persisted, a third reviewer (V.P.) adjudicated the discrepancy, as well as confirmed all potential reversals.
Studies that were both established and negative were included in our tentative list of oncology reversals. We used a two-step process to confirm the effectiveness of the practices on our tentative list of reversals . In step one, we looked for a systematic review/meta-analysis to confirm or refute the results of the tentative reversal. Reviews were identified through PubMed suggestions when looking at the article on PubMed, and if no relevant review was found, we searched Google Scholar using search terms relating to the article. Reviews needed to include the tentative reversal study and the conclusions needed to be based on the RCT results only. Newer reviews were prioritized over older reviews. If a tentative reversal had a meta-analysis that refuted the results of its findings, it was removed from the reversal list. The final reversal list included articles that either did not have a systematic review to confirm the results or had a review that confirmed the results of the trial. For the second step, three practicing oncologists reviewed all reversals to confirm that they had been used and that the results were negative (V.P., E.Y.C., and T.H.). Our methods are similar to prior analyses [3,4,5].
We calculated descriptive statistics on the final list of reversals in SAS software, version 9.4 (SAS Institute Inc., Cary, NC, USA). In accordance with 45 CFR §46.102(f), this study was not submitted for institutional review board approval because it involved publicly available data and did not involve individual patient data.
There were 5592 total articles in high-impact oncology journals during the 10-year time frame. Of those, 1542 were randomized controlled trials. Of the RCTs, we found 781 (50%) studies that reached positive conclusions, 548 (36%) studies that were negative, and 213 (14%) that were inconclusive. There were 1047 (68%) studies pertaining to novel medical practices and 495 (32%) pertaining to established practices. We identified 68 potential reversals, i.e., practices for which RCTs were coded as negative pertaining to established interventions. The process for how studies were selected is depicted in Fig. 1.
Of the potential reversals, 20 (31%) were confirmed by a systematic review and four were refuted. Three (5%) practices had a meta-analysis that was inconclusive as to whether it supported the practice or not, and 41 (64%) did not have a meta-analysis done on that particular medical practice. Thus, our final list of oncology reversals included 64 studies (4% of all RCTs) [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69]. These are all described in the Supplemental Table.
Thirty-two reversal studies (51%) were published in JCO, 28 (44%) were published in Lancet Oncology, and four (6%) were published in JAMA Oncology.
The breakdown of intervention type is as follows: 28 (44%) were medications, 10 (16%) were forms of radiotherapy or radiologic study, nine (14%) were procedures, six (9%) were optimization interventions, six (9%) were vitamins, supplements, or foods, four (6%) were behavioral therapies, and one (2%) was a screening test. For those that were medication interventions, 11 (39%) were targeted, six (21%) were cytotoxic, two (7%) were hormonal, none were radioactive, and nine (32%) were another type of medication.
The breakdown of tumor type is as follows: 18 (28%) were general/multiple, 13 (20%) were lung, 10 (16%) were breast, four (6%) were prostate, three (5%) were colorectal cancer, two (3%) were hepatocellular cancer, two (3%) were lymphoma, two (3%) were sarcoma, two (3%) were skin, two (3%) were endometrial, and six (10%) were another type of cancer.
As for funding, 14 (23%) were funded by pharmaceutical/industry only, 36 (59%) were funded by an organization other than pharmaceutical/industry, and 11 (18%) were a combination of industry and non-industry. Three studies either did not report funding or reported that there was no funding.
We found that the median number of years that the practice had been in use prior to the reversal study was 9 years (range 1–50 years). These characteristics are summarized in Table 1.
In our review of the three oncology journals the highest impact factor, we found 64 practices that have been used outside of clinical trials but were found to be ineffective in randomized trials. These practices represented the spectrum of cancer care, from prevention to screening to treatment (procedural or medication), and apply to all of the more common cancers. These practices constitute medical reversals. They were not replaced by better interventions; they were found to be no better than prior or lesser standards. In retrospect, their use failed to improve outcomes for patients. The implementation of practices prior to adequate evidence for their use is a central theme to the practices we identified. These practices are in addition to a list of medical reversals that we previously identified, which include oncology practices published on in general medical journals (e.g., New England Journal of Medicine, Lancet, and the Journal of the American Medical Association).
The reversals we found represented 4% of the oncology trials in the journals we examined. This percentage is similar to other studies that have found that 5% of oncology drugs receiving accelerated approval are withdrawn after post-marketing data are available . These percentages are small and suggests that most of the practices in oncology that are being used have evidence backing their efficacy. The list of reversals that we have compiled shows the work that still needs to be done in identifying and eliminating medical reversals. As with medicine in general, a constant evaluation of the effectiveness of existing practices is central to the progress in oncology.
The identification of medical reversals or low-value practices is an important first step in eliminating their use. Their identification can then lead to clinical decision support tools and better communication between healthcare providers and patients, including educating patients about the value and effectiveness of medical practices, which have been identified as other important steps in reducing the usage of low-value practices . The identification of these practices provides an opportunity for healthcare administrators, physicians and other healthcare providers to re-evaluate the care they provide, thus improving patient care and outcomes.
It is well acknowledged that the cost of cancer treatment is extremely high, and the financial toxicity is burdensome to patients and society at-large. Eliminating low-value care is key to easing the financial burden that comes with cancer treatment . The identification of medical reversals, which are a subset of low-value practices, is one way to reduce health care costs through less money spent on ineffective practices. The Choosing Wisely campaign , which is one entity that lists low-value practices, has identified 75 practices in oncology (as of May 4, 2020), but this list is not comprehensive or objective, and relies on professional organizations to report which practices should be on the list. Adherence to Choosing Wisely recommendations has been estimated to be associated with a $19 million cost reduction per quarter for Medicare patients treated at cancer centers in the southwest region of the US . The list we have assembled is objective and is comprehensive as far as what was published, and has the potential for notable savings if these practices were to be de-implemented.
An example of a practice failing to improve outcomes for patients is when first-generation EGFR inhibitors, such as erlotinib and gefitinib, were initially approved for 2nd and 3rd line metastatic non-small-cell lung cancer after a small improvement in survival endpoints. However, several later RCTs did not detect any survival advantage. It was not until additional studies were conducted that it was discovered that the subgroups with EGFR exon 19 and 21 abnormalities were the true responders . Until the later studies, patients with non-EGFR mutations were exposed to costly drugs with potential side effects. This illustrates why we need to develop a good understanding of the molecular basis of novel therapies and identify the appropriate subgroups in an effort to prevent medical reversals from occurring.
Most reversals we found had little research previously published on them before the practices were implemented, as evident by the low percentage of practices with a systematic review done on them. However, a recent example of when a practice was implemented based on a randomized trial and then later reversed is olaratumab for patients with sarcoma. The accelerated approval by the US Food and Drug Administration in 2016 was based on a small phase II, randomized study, which led to thousands of patients receiving this drug . But, when phase III trial data were released in 2019, olaratumab offered no overall survival benefit . The drug has since been pulled from the market by the drug manufacturer. It is increasingly appreciated that small, underpowered studies that find benefits, particularly for outcomes that are not primary endpoints, often produce false positive or exaggerated results, which can lead to the widespread adoption of practices that provide no benefit .
We did examine the funding source of each of the reversals and found that most studies (77%) were funded by a non-industry organization, either in-part or in-full. Only 41% of the reversal studies we examined were in part industry funded. In contrast, 78% of randomized trials in oncology were funded in part by the industry over a comparable time period . Industry funded trials are far more likely to be positive compared to trials funded by other entities and may be subject to methodological bias . The lower numerical percentage of medical reversals that are funded by industry, in light of the much higher percentage of total clinical trials funded by industry, may be an indication of the pharmaceutical’s priority to fund novel interventions or promote the use of certain products. It is notable that 9 out of 11 (82%) medical reversals regarding targeted therapies are industry funded. This may reflect the high cost of novel agents preventing non-industry funded trials from evaluating these drugs. The burden of identifying practices that are ineffective or of low-value seems to fall more heavily on non-industry organizations. However, costly interventions may be prohibitively expensive to investigate without industry support. How to effectively direct public funding and motivate industry to generate high quality evidence is a complex issue warranting further research.
The lag time between use of a medical practice and subsequent reversal was found to be a median of 9 years in our analysis. This is the first reversal study to date that has sought to capture this lag [81, 82]. Notably our results are recapitulated in the FDA’s enforcement of post-marketing commitments for drugs receiving accelerated approval. These requirements are often delayed or incomplete [72, 83]. A lag between practice onset and definitive studies constitute an important period of time where the use of a practice, ultimately found ineffective or detrimental, was perpetuated without robust clinical trials. We believe it is important for the field to study strategies to minimize this time, and elsewhere have offered a number of solutions .
There are four limitations to our analysis. First, our study is not an exhaustive list of all oncology reversals. Several notable examples of medical reversals were not included on our list because the results were either published in other journals or have not been published to date. For example, olaratumab in combination with doxorubicin has been approved for use in patients with advanced or metastatic soft-tissue sarcoma, but to date, publication on the failure of the trial to show improvement in the primary endpoint of overall survival has only been published in abstract form or in non-scientific publications. With a reported 60% of negative trials (in either efficacy or safety) not being reported on in the literature , it is likely that our list is missing other noteworthy medical reversals.
Second, we only looked at three oncology journals, and, as such, our findings may not represent oncology practices at-large. Publication bias may also limit the generalizability. We chose these journals because they were more likely to publish studies on high-quality randomized trials. Third, the initial screening of each study was done by people without medical degrees in oncology, and some practices may not have been added to the list of reversals (possibly reducing sensitivity). Related to this is the limitation of subjectivity by different reviewers on whether practices were established, but notably each potential reversal was reviewed by three practicing medical oncologists. It is natural that others may disagree with our characterizations, and we encourage further independent research on this topic. Fourth, we were not able to capture the strength of recommendations when practices were implemented. Some of these recommendations were strong, while others were weak because of lack of definitive evidence. Finally, we recognize that not all of the identified medical reversals were widely implemented. We have included all practices that had documented use outside of clinical trials and case reports because sometimes when practices are used, even at low frequency, their use is perpetuated.
In conclusion, we found that oncology reversals most often involve the administration of medications and are found in many oncology subspecialties. Further, we found that many of these reversals had been practiced for years and were often found to be a reversal because of studies funded predominately by non-industry organizations. Identification of medical reversals is an important step in being able to discontinue their use, which will help in directing resources towards practices that truly work. Our work also emphasizes the importance of robust governmental funding of cancer trials.
Role of the funding source
The funder had no role in the design, analysis, or writing of this manuscript. The corresponding author had full access to all data used in this study and had final responsibility for the decision to submit for publication.
Availability of data and materials
All data are publicly available.
Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88(8):790–8. https://doi.org/10.1016/j.mayocp.2013.05.012. Epub 2013 Jul 18.
Hahn EE, Gould MK. Drivers of low-value care. JAMA Intern Med. 2018;178(3):433. https://doi.org/10.1001/jamainternmed.2017.8528.
Herrera-Perez D, Haslam A, Crain T, Gill J, Livingston C, Kaestner V, et al. Meta-research: a comprehensive review of randomized clinical trials in three medical journals reveals 396 medical reversals. Elife. 2019;8:e45183. https://doi.org/10.7554/eLife.45183.
Haslam A, Prasad V, Livingston C. Medical reversals in low- and middle-income countries. Int J Health Plann Manag. 2020;35(2):631–8. https://doi.org/10.1002/hpm.2937.
Haslam A, Livingston C, Prasad V. Medical reversals in family practice: a review. Curr Ther Res Clin Exp. 2020;92:100579. https://doi.org/10.1016/j.curtheres.2020.100579.
Porceddu SV, Bressel M, Poulsen MG, Stoneley A, Veness MJ, Kenny LM, et al. Postoperative Concurrent Chemoradiotherapy Versus Postoperative Radiotherapy in High-Risk Cutaneous Squamous Cell Carcinoma of the Head and Neck: The Randomized Phase III TROG 05.01 Trial. J Clin Oncol. 2018;36(13):1275–83. https://doi.org/10.1200/JCO.2017.77.0941. Epub 2018 Mar 14.
Mok TS, Kim S-W, Wu Y-L, et al. Gefitinib plus chemotherapy versus chemotherapy in epidermal growth factor receptor mutation-positive non-small-cell lung cancer resistant to first-line gefitinib (IMPRESS): overall survival and biomarker analyses. J Clin Oncol. 2017;35(36):4027–34.
Chambers SK, Occhipinti S, Foley E, Clutton S, Legg M, Berry M, et al. Mindfulness-based cognitive therapy in advanced prostate cancer: a randomized controlled trial. J Clin Oncol. 2017;35(3):291–7. https://doi.org/10.1200/JCO.2016.68.8788.
Demeestere I, Brice P, Peccatori FA, Kentos A, Dupuis J, Zachee P, et al. No evidence for the benefit of gonadotropin-releasing hormone agonist in preserving ovarian function and fertility in lymphoma survivors treated with chemotherapy: final long-term report of a prospective randomized trial. J Clin Oncol. 2016;34(22):2568–74. https://doi.org/10.1200/JCO.2015.65.8864.
Brown KT, Do RK, Gonen M, Covey AM, Getrajdman GI, Sofocleous CT, et al. Randomized trial of hepatic artery embolization for hepatocellular carcinoma using doxorubicin-eluting microspheres compared with embolization with microspheres alone. J Clin Oncol. 2016;34(17):2046–53. https://doi.org/10.1200/JCO.2015.64.0821.
Van Hazel GA, Heinemann V, Sharma NK, Peeters M. SIRFLOX: randomized phase III trial comparing first-line mFOLFOX6 (plus or minus bevacizumab) versus mFOLFOX6 (plus or minus bevacizumab) plus selective internal radiation therapy in patients with metastatic colorectal cancer. J Clin Oncol. 2016;34(15):1723–31. https://doi.org/10.1200/JCO.2015.66.1181.
Leyland-Jones B, Bondarenko I, Nemsadze G, Smirnov V, Litvin I, Kokhreidze I, et al. A randomized, open-label, multicenter, phase III study of epoetin alfa versus best standard of care in anemic patients with metastatic breast cancer receiving standard chemotherapy. J Clin Oncol. 2016;34(11):1197–207. https://doi.org/10.1200/JCO.2015.63.5649.
Fallon M, Hoskin PJ, Colvin LA, Fleetwood-Walker SM, Adamson D, Byrne A, et al. Randomized double-blind trial of pregabalin versus placebo in conjunction with palliative radiotherapy for cancer-induced bone pain. J Clin Oncol. 2016;34(6):550–6. https://doi.org/10.1200/JCO.2015.63.8221.
Nicolaije K, Ezendam N, Vos C, et al. Impact of an automatically generated cancer survivorship care plan on patient-reported outcomes in routine clinical practice: longitudinal outcomes of a pragmatic, cluster randomized trial. J Clin Oncol. 2015;33(31):3550–9. https://doi.org/10.1200/JCO.2014.60.3399.
Pivot X, Manikhas A, Żurawski B, Chmielowska E, Karaszewska B, Allerton R, et al. CEREBEL (EGF111438): a phase III, randomized, open-label study of lapatinib plus capecitabine versus trastuzumab plus capecitabine in patients with human epidermal growth factor receptor 2–positive metastatic breast cancer. J Clin Oncol. 2015;33(14):1564–73. https://doi.org/10.1200/JCO.2014.57.1794.
Paulsen Ø, Klepstad P, Rosland JH, Aass N, Albert E, Fayers P, et al. Efficacy of methylprednisolone on pain, fatigue, and appetite loss in patients with advanced cancer using opioids: a randomized, placebo-controlled, double-blind trial. J Clin Oncol. 2014;32(29):3221–8. https://doi.org/10.1200/JCO.2013.54.3926.
Fernando HC, Landreneau RJ, Mandrekar SJ, Nichols FC, Hillman SL, Heron DE, et al. Impact of brachytherapy on local recurrence rates after sublobar resection: results from ACOSOG Z4032 (Alliance), a phase III randomized trial for high-risk operable non–small-cell lung cancer. J Clin Oncol. 2014;32(23):2456–62. https://doi.org/10.1200/JCO.2013.53.4115.
Kawaguchi T, Ando M, Asami K, Okano Y, Fukuda M, Nakagawa H, et al. Randomized phase III trial of erlotinib versus docetaxel as second-or third-line therapy in patients with advanced non–small-cell lung cancer: docetaxel and Erlotinib lung Cancer trial (DELTA). J Clin Oncol. 2014;32(18):1902–8. https://doi.org/10.1200/JCO.2013.52.4694.
Smith MR, Halabi S, Ryan CJ, Hussain A, Vogelzang N, Stadler W, et al. Randomized controlled trial of early zoledronic acid in men with castration-sensitive prostate cancer and bone metastases: results of CALGB 90202 (alliance). J Clin Oncol. 2014;32(11):1143–50. https://doi.org/10.1200/JCO.2013.51.6500.
Hoff PM, Saragiotto DF, Barrios CH, et al. Randomized phase III trial exploring the use of long-acting release octreotide in the prevention of chemotherapy-induced diarrhea in patients with colorectal cancer: the LARCID trial. J Clin Oncol. 2014;32(10):1006–11. https://doi.org/10.1200/jco.2013.50.8077.
Loprinzi CL, Qin R, Dakhil SR, et al. Phase III randomized, placebo-controlled, double-blind study of intravenous calcium and magnesium to prevent oxaliplatin-induced sensory neurotoxicity (N08CB/Alliance). J Clin Oncol. 2014;32(10):997–1005. https://doi.org/10.1200/jco.2013.52.0536.
Onsrud M, Cvancarova M, Hellebust TP, Tropé CG, Kristensen GB, Lindemann K. Long-term outcomes after pelvic radiation for early-stage endometrial cancer. J Clin Oncol. 2013;31(31):3951–6. https://doi.org/10.1200/jco.2013.48.8023.
Hollingworth W, Metcalfe C, Mancero S, et al. Are needs assessments cost effective in reducing distress among patients with cancer? A randomized controlled trial using the Distress Thermometer and Problem List. J Clin Oncol. 2013;31(29):3631–8. https://doi.org/10.1200/jco.2012.48.3040.
Gordon LI, Hong F, Fisher RI, et al. Randomized phase III trial of ABVD versus Stanford V with or without radiation therapy in locally extensive and advanced-stage Hodgkin lymphoma: an intergroup study coordinated by the Eastern Cooperative Oncology Group (E2496). J Clin Oncol. 2013;31(6):684–91. https://doi.org/10.1200/jco.2012.43.4803.
Soffietti R, Kocher M, Abacioglu UM, et al. A European Organisation for Research and Treatment of Cancer phase III trial of adjuvant whole-brain radiotherapy versus observation in patients with one to three brain metastases from solid tumors after surgical resection or radiosurgery: quality-of-life results. J Clin Oncol. 2013;31(1):65–72. https://doi.org/10.1200/jco.2011.41.0639.
Bruera E, Hui D, Dalal S, et al. Parenteral hydration in patients with advanced cancer: a multicenter, double-blind, placebo-controlled randomized trial. J Clin Oncol. 2013;31(1):111–8. https://doi.org/10.1200/jco.2012.44.6518.
Cruciani RA, Zhang JJ, Manola J, Cella D, Ansari B, Fisch MJ. L-carnitine supplementation for the management of fatigue in patients with cancer: an eastern cooperative oncology group phase III, randomized, double-blind, placebo-controlled trial. J Clin Oncol. 2012;30(31):3864–9. https://doi.org/10.1200/jco.2011.40.2180.
Grunfeld E, Julian JA, Pond G, et al. Evaluating survivorship care plans: results of a randomized, clinical trial of patients with breast cancer. J Clin Oncol. 2011;29(36):4755–62. https://doi.org/10.1200/jco.2011.36.8373.
Blohmer JU, Paepke S, Sehouli J, et al. Randomized phase III trial of sequential adjuvant chemoradiotherapy with or without erythropoietin Alfa in patients with high-risk cervical cancer: results of the NOGGO-AGO intergroup study. J Clin Oncol. 2011;29(28):3791–7. https://doi.org/10.1200/jco.2010.30.4899.
Nout RA, van de Poll-Franse LV, Lybeert ML, et al. Long-term outcome and quality of life of patients with endometrial carcinoma treated with or without pelvic radiotherapy in the post operative radiation therapy in endometrial carcinoma 1 (PORTEC-1) trial. J Clin Oncol. 2011;29(13):1692–700. https://doi.org/10.1200/jco.2010.32.4590.
Kocher M, Soffietti R, Abacioglu U, et al. Adjuvant whole-brain radiotherapy versus observation after radiosurgery or surgical resection of one to three cerebral metastases: results of the EORTC 22952–26001 study. J Clin Oncol. 2011;29(2):134–41. https://doi.org/10.1200/jco.2010.30.1655.
Steensma DP, Sloan JA, Dakhil SR, et al. Phase III, randomized study of the effects of parenteral iron, oral iron, or no iron supplementation on the erythropoietic response to darbepoetin alfa for patients with chemotherapy-associated anemia. J Clin Oncol. 2011;29(1):97–105. https://doi.org/10.1200/jco.2010.30.3644.
Wolf SL, Qin R, Menon SP, et al. Placebo-controlled trial to determine the effectiveness of a urea/lactic acid-based topical keratolytic agent for prevention of capecitabine-induced hand-foot syndrome: North Central Cancer Treatment Group Study N05C5. J Clin Oncol. 2010;28(35):5182–7. https://doi.org/10.1200/jco.2010.31.1431.
Kang YK, Lee SS, Yoon DH, et al. Pyridoxine is not effective to prevent hand-foot syndrome associated with capecitabine therapy: results of a randomized, double-blind, placebo-controlled study. J Clin Oncol. 2010;28(24):3824–9. https://doi.org/10.1200/jco.2010.29.1807.
Hauschild A, Weichenthal M, Rass K, et al. Prospective randomized multicenter adjuvant dermatologic cooperative oncology group trial of low-dose interferon alfa-2b with or without a modified high-dose interferon alfa-2b induction phase in patients with lymph node-negative melanoma. J Clin Oncol. 2009;27(21):3496–502. https://doi.org/10.1200/jco.2008.21.3892.
Goss G, Ferry D, Wierzbicki R, et al. Randomized phase II study of gefitinib compared with placebo in chemotherapy-naive patients with advanced non-small-cell lung cancer and poor performance status. J Clin Oncol. 2009;27(13):2253–60. https://doi.org/10.1200/jco.2008.18.4408.
Mills ME, Murray LJ, Johnston BT, Cardwell C, Donnelly M. Does a patient-held quality-of-life diary benefit patients with inoperable lung cancer? J Clin Oncol. 2009;27(1):70–7. https://doi.org/10.1200/jco.2008.17.5687.
Galimberti V, Cole BF, Viale G, Veronesi P, Vicini E, Intra M, et al. Axillary dissection versus no axillary dissection in patients with breast cancer and sentinel-node micrometastases (IBCSG 23-01): 10-year follow-up of a randomised, controlled phase 3 trial. Lancet Oncol. 2018;19(10):1385–93. https://doi.org/10.1016/s1470-2045(18)30380-2.
Negenborn VL, Young-Afat DA, Dikmans REG, Smit JM, Winters HAH, Don Griot JPW, et al. Quality of life and patient satisfaction after one-stage implant-based breast reconstruction with an acellular dermal matrix versus two-stage breast reconstruction (BRIOS): primary outcome of a randomised, controlled trial. Lancet Oncol. 2018;19(9):1205–14. https://doi.org/10.1016/s1470-2045(18)30378-4.
van den Bent MJ, Klein M, Smits M, Reijneveld JC, French PJ, Clement P, et al. Bevacizumab and temozolomide in patients with first recurrence of WHO grade II and III glioma, without 1p/19q co-deletion (TAVAREC): a randomised controlled phase 2 EORTC trial. Lancet Oncol. 2018;19(9):1170–9. https://doi.org/10.1016/s1470-2045(18)30362-0.
Coughlin GD, Yaxley JW, Chambers SK, Occhipinti S, Samaratunga H, Zajdlewicz L, et al. Robot-assisted laparoscopic prostatectomy versus open radical retropubic prostatectomy: 24-month outcomes from a randomised controlled study. Lancet Oncol. 2018;19(8):1051–60. https://doi.org/10.1016/s1470-2045(18)30357-7.
Herbst RS, Redman MW, Kim ES, Semrad TJ, Bazhenova L, Masters G, et al. Cetuximab plus carboplatin and paclitaxel with or without bevacizumab versus carboplatin and paclitaxel with or without bevacizumab in advanced NSCLC (SWOG S0819): a randomised, phase 3 study. Lancet Oncol. 2018;19(1):101–14. https://doi.org/10.1016/s1470-2045(17)30694-0.
Vilgrain V, Pereira H, Assenat E, Guiu B, Ilonca AD, Pageaux GP, et al. Efficacy and safety of selective internal radiotherapy with yttrium-90 resin microspheres compared with sorafenib in locally advanced and inoperable hepatocellular carcinoma (SARAH): an open-label randomised controlled phase 3 trial. Lancet Oncol. 2017;18(12):1624–36. https://doi.org/10.1016/s1470-2045(17)30683-6.
Tjan-Heijnen VCG, van Hellemond IEG, Peer PGM, Swinkels ACP, Smorenburg CH, van der Sangen MJC, et al. Extended adjuvant aromatase inhibition after sequential endocrine therapy (DATA): a randomised, phase 3 trial. Lancet Oncol. 2017;18(11):1502–11. https://doi.org/10.1016/s1470-2045(17)30600-9.
Seddon B, Strauss SJ, Whelan J, Leahy M, Woll PJ, Cowie F, et al. Gemcitabine and docetaxel versus doxorubicin as first-line treatment in previously untreated advanced unresectable or metastatic soft-tissue sarcomas (GeDDiS): a randomised controlled phase 3 trial. Lancet Oncol. 2017;18(10):1397–410. https://doi.org/10.1016/s1470-2045(17)30622-8.
Takahashi T, Yamanaka T, Seto T, Harada H, Nokihara H, Saka H, et al. Prophylactic cranial irradiation versus observation in patients with extensive-disease small-cell lung cancer: a multicentre, randomised, open-label, phase 3 trial. Lancet Oncol. 2017;18(5):663–71. https://doi.org/10.1016/s1470-2045(17)30230-9.
Dikmans RE, Negenborn VL, Bouman MB, et al. Two-stage implant-based breast reconstruction compared with immediate one-stage implant-based breast reconstruction augmented with an acellular dermal matrix: an open-label, phase 4, multicentre, randomised, controlled trial. Lancet Oncol. 2017;18(2):251–8. https://doi.org/10.1016/s1470-2045(16)30668-4.
Clive AO, Taylor H, Dobson L, Wilson P, de Winton E, Panakis N, et al. Prophylactic radiotherapy for the prevention of procedure-tract metastases after surgical and large-bore pleural procedures in malignant pleural mesothelioma (SMART): a multicentre, open-label, phase 3, randomised controlled trial. Lancet Oncol. 2016;17(8):1094–104. https://doi.org/10.1016/s1470-2045(16)30095-x.
Glover M, Smerdon GR, Andreyev HJ, Benton BE, Bothma P, Firth O, et al. Hyperbaric oxygen for patients with chronic bowel dysfunction after pelvic radiotherapy (HOT2): a randomised, double-blind, sham-controlled phase 3 trial. Lancet Oncol. 2016;17(2):224–33. https://doi.org/10.1016/s1470-2045(15)00461-1.
Stahel RA, Riesterer O, Xyrafas A, Opitz I, Beyeler M, Ochsenbein A, et al. Neoadjuvant chemotherapy and extrapleural pneumonectomy of malignant pleural mesothelioma with or without hemithoracic radiotherapy (SAKK 17/04): a randomised, international, multicentre phase 2 trial. Lancet Oncol. 2015;16(16):1651–8. https://doi.org/10.1016/s1470-2045(15)00208-9.
Le Tourneau C, Delord JP, Gonçalves A, et al. Molecularly targeted therapy based on tumour molecular profiling versus conventional therapy for advanced cancer (SHIVA): a multicentre, open-label, proof-of-concept, randomised, controlled phase 2 trial. Lancet Oncol. 2015;16(13):1324–34. https://doi.org/10.1016/s1470-2045(15)00188-6.
Badwe R, Hawaldar R, Nair N, Kaushik R, Parmar V, Siddique S, et al. Locoregional treatment versus no treatment of the primary tumour in metastatic breast cancer: an open-label randomised controlled trial. Lancet Oncol. 2015;16(13):1380–8. https://doi.org/10.1016/s1470-2045(15)00135-7.
Moss SM, Wale C, Smith R, Evans A, Cuckle H, Duffy SW. Effect of mammographic screening from age 40 years on breast cancer mortality in the UK age trial at 17 years' follow-up: a randomised controlled trial. Lancet Oncol. 2015;16(9):1123–32. https://doi.org/10.1016/s1470-2045(15)00128-x.
Henderson MA, Burmeister BH, Ainslie J, Fisher R, di Iulio J, Smithers BM, et al. Adjuvant lymph-node field radiotherapy versus observation only in patients with melanoma at high risk of further lymph-node field relapse after lymphadenectomy (ANZMTG 01.02/TROG 02.01): 6-year follow-up of a phase 3, randomised controlled trial. Lancet Oncol. 2015;16(9):1049–60. https://doi.org/10.1016/s1470-2045(15)00187-4.
Soria JC, Wu YL, Nakagawa K, Kim SW, Yang JJ, Ahn MJ, et al. Gefitinib plus chemotherapy versus placebo plus chemotherapy in EGFR-mutation-positive non-small-cell lung cancer after progression on first-line gefitinib (IMPRESS): a phase 3 randomised trial. Lancet Oncol. 2015;16(8):990–8. https://doi.org/10.1016/s1470-2045(15)00121-7.
Primrose J, Falk S, Finch-Jones M, Valle J, O'Reilly D, Siriwardena A, et al. Systemic chemotherapy with or without cetuximab in patients with resectable colorectal liver metastasis: the new EPOC randomised controlled trial. Lancet Oncol. 2014;15(6):601–11. https://doi.org/10.1016/s1470-2045(14)70105-6.
Judson I, Verweij J, Gelderblom H, Hartmann JT, Schöffski P, Blay JY, et al. Doxorubicin alone versus intensified doxorubicin plus ifosfamide for first-line treatment of advanced or metastatic soft-tissue sarcoma: a randomised controlled phase 3 trial. Lancet Oncol. 2014;15(4):415–23. https://doi.org/10.1016/s1470-2045(14)70063-4.
Bosset JF, Calais G, Mineur L, Maingon P, Stojanovic-Rundic S, Bensadoun RJ, et al. Fluorouracil-based adjuvant chemotherapy after preoperative chemoradiotherapy in rectal cancer: long-term results of the EORTC 22921 randomised study. Lancet Oncol. 2014;15(2):184–90. https://doi.org/10.1016/s1470-2045(13)70599-0.
Bath-Hextall F, Ozolins M, Armstrong SJ, Colver GB, Perkins W, Miller PS, et al. Surgical excision versus imiquimod 5% cream for nodular and superficial basal-cell carcinoma (SINS): a multicentre, non-inferiority, randomised controlled trial. Lancet Oncol. 2014;15(1):96–105. https://doi.org/10.1016/s1470-2045(13)70530-8.
Veronesi U, Orecchia R, Maisonneuve P, Viale G, Rotmensz N, Sangalli C, et al. Intraoperative radiotherapy versus external radiotherapy for early breast cancer (ELIOT): a randomised controlled equivalence trial. Lancet Oncol. 2013;14(13):1269–77. https://doi.org/10.1016/s1470-2045(13)70497-2.
Garassino MC, Martelli O, Broggini M, Farina G, Veronese S, Rulli E, et al. Erlotinib versus docetaxel as second-line treatment of patients with advanced non-small-cell lung cancer and wild-type EGFR tumours (TAILOR): a randomised controlled trial. Lancet Oncol. 2013;14(10):981–8. https://doi.org/10.1016/s1470-2045(13)70310-3.
Kreissman SG, Seeger RC, Matthay KK, London WB, Sposto R, Grupp SA, et al. Purged versus non-purged peripheral blood stem-cell transplantation for high-risk neuroblastoma (COG A3973): a randomised phase 3 trial. Lancet Oncol. 2013;14(10):999–1008. https://doi.org/10.1016/s1470-2045(13)70309-7.
van Hooft JE, Bemelman WA, Oldenburg B, Marinelli AW, Lutke Holzik MF, Grubben MJ, et al. Colonic stenting versus emergency surgery for acute left-sided malignant colonic obstruction: a multicentre randomised trial. Lancet Oncol. 2011;12(4):344–52. https://doi.org/10.1016/s1470-2045(11)70035-3.
Irani J, Salomon L, Oba R, Bouchard P, Mottet N. Efficacy of venlafaxine, medroxyprogesterone acetate, and cyproterone acetate for the treatment of vasomotor hot flushes in men taking gonadotropin-releasing hormone analogues for prostate cancer: a double-blind, randomised trial. Lancet Oncol. 2010;11(2):147–54. https://doi.org/10.1016/s1470-2045(09)70338-9.
Le Péchoux C, Dunant A, Senan S, et al. Standard-dose versus higher-dose prophylactic cranial irradiation (PCI) in patients with limited-stage small-cell lung cancer in complete remission after chemotherapy and thoracic radiotherapy (PCI 99-01, EORTC 22003-08004, RTOG 0212, and IFCT 99-01): a randomised clinical trial. Lancet Oncol. 2009;10(5):467–74. https://doi.org/10.1016/S1470-2045(09)70101-9.
Gridelli C, de Castro CJ, Dingemans AC, et al. Safety and efficacy of bevacizumab plus standard-of-care treatment beyond disease progression in patients with advanced non-small cell lung cancer: The AvaALL randomized clinical trial. JAMA Oncol. 2018;4(12):e183486. https://doi.org/10.1001/jamaoncol.2018.3486.
Scragg R, Khaw KT, Toop L, et al. Monthly high-dose vitamin D supplementation and cancer risk: A post hoc analysis of the vitamin D assessment randomized clinical trial. JAMA Oncol. 2018;4(11):e182178. https://doi.org/10.1001/jamaoncol.2018.2178.
Yap YS, Kwok LL, Syn N, et al. Predictors of hand-foot syndrome and pyridoxine for prevention of capecitabine-induced hand-foot syndrome: A randomized clinical trial. JAMA Oncol. 2017;3(11):1538–45. https://doi.org/10.1001/jamaoncol.2017.1269.
Pramanik R, Agarwala S, Gupta YK, et al. Metronomic chemotherapy vs best supportive care in progressive pediatric solid malignant tumors: A randomized clinical trial. JAMA Oncol. 2017;3(9):1222–7. https://doi.org/10.1001/jamaoncol.2017.0324.
Beaver JA, Howie LJ, Pelosof L, et al. A 25-year experience of US food and drug administration accelerated approval of malignant hematology and oncology drugs and biologics: A review. JAMA Oncol. 2018;4(6):849–56. https://doi.org/10.1001/jamaoncol.2017.5618.
Sypes EE, de Grood C, Clement FM, et al. Understanding the public’s role in reducing low-value care: a scoping review. Implement Sci. 2020;15(1):20. 2020/04/07. https://doi.org/10.1186/s13012-020-00986-0.
Gyawali B. Low-value practices in oncology contributing to financial toxicity. Ecancermedicalscience. 2017;11:727. https://doi.org/10.3332/ecancer.2017.727. Published 2017 Mar 16.
Wisely C. Clinician lists. 2020, https://www.choosingwisely.org/clinician-lists/#topic-area=Oncology
Rocque GB, Williams CP, Jackson BE, Wallace AS, Halilova KI, Kenzik KM, et al. Choosing wisely: opportunities for improving value in cancer care delivery? J Oncol Pract. 2017;13(1):e11–21. https://doi.org/10.1200/JOP.2016.015396.
Yang J, Zhou Q, Yan H, et al. A phase III randomised controlled trial of erlotinib vs gefitinib in advanced non-small cell lung cancer with EGFR mutations. Br J Cancer. 2017;116(5):568–74. https://doi.org/10.1038/bjc.2016.456.
Tap WD, Jones RL, Van Tine BA, et al. Olaratumab and doxorubicin versus doxorubicin alone for treatment of soft-tissue sarcoma: an open-label phase 1b and randomised phase 2 trial. Lancet. 2016;388(10043):488–97. https://doi.org/10.1016/s0140-6736(16)30587-6.
Tap WD, Wagner AJ, Schöffski P, Martin-Broto J, Krarup-Hansen A, Ganjoo KN, Yen CC, Abdul Razak AR, Spira A, Kawai A, Le Cesne A, Van Tine BA, Naito Y, Park SH, Fedenko A, Pápai Z, Soldatenkova V, Shahir A, Mo G, Wright J, Jones RL; ANNOUNCE Investigators. Effect of Doxorubicin Plus Olaratumab vs Doxorubicin Plus Placebo on Survival in Patients With Advanced Soft Tissue Sarcomas: The ANNOUNCE Randomized Clinical Trial. JAMA. 2020;323(13):1266–76. https://doi.org/10.1001/jama.2020.1707.
Button KS, Ioannidis JP, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365–76. https://doi.org/10.1038/nrn3475.
Kay A, Higgins J, Day A, Meyer R, Booth C. Randomized controlled trials in the era of molecular oncology: methodology, biomarkers, and end points. Ann Oncol. 2012;23(6):1646–51. https://doi.org/10.1093/annonc/mdr492.
Chopra SS. Industry funding of clinical trials: benefit or Bias? JAMA. 2003;290(1):113–4. https://doi.org/10.1001/jama.290.1.113.
Sutton D, Qureshi R, Martin J. Evidence reversal—when new evidence contradicts current claims: a systematic overview review of definitions and terms. J Clin Epidemiol. 2018;94:76–84. https://doi.org/10.1016/j.jclinepi.2017.10.004.
Martin J, Cheng D. Value-based critical care medicine: evidence reversal and choosing Wisely campaign in critical care. Critical Care Update. 2019;2019:50.
Zettler M, Nabhan C. Fulfillment of postmarketing requirements to the FDA for therapies granted oncology indications between 2011 and 2016. JAMA Oncol. 2018;4(7):993–4. https://doi.org/10.1001/jamaoncol.2018.0610.
Prasad VK. Malignant: how bad policy and bad evidence harm people with Cancer: Baltimore, Maryland: JHU Press; 2020.
Mlinarić A, Horvat M, Šupak SV. Dealing with the positive publication bias: why you should really publish your negative results. Biochem Med. 2017;27(3):447–52.
This project was funded by the Arnold Ventures. The funder had no involvement with the study design, data analysis, or manuscript preparation.
Ethics approval and consent to participate
In accordance with 45 CFR §46.102(f), this study was not submitted for institutional review board approval because it involved publicly available data and did not involve individual patient data.
Consent for publication
Vinay Prasad’s Disclosures. (Research funding) Arnold Ventures (Royalties) Johns Hopkins Press, Medscape (Honoraria) Grand Rounds/lectures from universities, medical centers, non-profits, and professional societies. (Consulting) UnitedHealthcare. (Speaking fees) Evicore. (Other) Plenary Session podcast has Patreon backers. All other authors have no funding to report.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Haslam, A., Gill, J., Crain, T. et al. The frequency of medical reversals in a cross-sectional analysis of high-impact oncology journals, 2009–2018. BMC Cancer 21, 889 (2021). https://doi.org/10.1186/s12885-021-08632-8