Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Development and validation of a nomogram to predict overall survival for patients with metastatic renal cell carcinoma

BMC Cancer volume 20, Article number: 1066 (2020) Cite this article

Abstract

Background

Heterogeneity of metastatic renal cell carcinoma (RCC) constraints accurate prognosis prediction of the tumor. We therefore aimed at developing a novel nomogram for accurate prediction of overall survival (OS) of patients with metastatic RCC.

Methods

We extracted 2010 to 2016 data for metastatic RCC patients in the Surveillance, Epidemiology, and End Results (SEER) database, and randomly stratified them equally into training and validation sets. Prognostic factors for OS were analyzed using Cox regression models, and thereafter integrated into a 1, 3 and 5-year OS predictive nomogram. The nomogram was validated using the training and validation sets. The performance of this model was evaluated by the Harrell’s concordance index (C-index), calibration curve, integrated discrimination improvement (IDI), category-free net reclassification improvement (NRI), index of prediction accuracy (IPA), and decision curve analysis (DCA).

Results

Overall, 2315 metastatic RCC patients in the SEER database who fulfilled our inclusion criteria were utilized in constructing a nomogram for predicting OS of newly diagnosed metastatic RCC patients. The nomogram incorporated eight clinical factors: Fuhrman grade, lymph node status, sarcomatoid feature, cancer-directed surgery and bone, brain, liver, and lung metastases, all significantly associated with OS. The model was superior to the American Joint Committee on Cancer (AJCC) staging system (7th edition) both in training (C-indices, 0.701 vs. 0.612, P < 0.001) and validation sets (C-indices, 0.676 vs. 0.600, P < 0.001). The calibration plots of the nomogram corresponded well between predicted and observed values. NRI, IDI, and IPA further validated the superior predictive capability of the nomogram relative to the AJCC staging system. The DCA plots revealed reliable clinical application of our model in prognosis prediction of metastatic RCC patients.

Conclusions

We developed and validated an accurate nomogram for individual OS prediction of metastatic RCC patients. This nomogram can be applied in design of clinical trials, patient counseling, and rationalizing therapeutic modalities.

Peer Review reports

Background

Kidney cancer is one of the most prevalent genitourinary malignancies. In 2018 alone, it accounted for 2.2% of new cancer cases and 1.8% of overall cancer deaths worldwide [1]. In the United States alone, new cases and deaths due to kidney and renal pelvis cancer in 2019 were 73,820 and 14,770, respectively [2]. RCC is the most common subtype of kidney neoplasms, accounting for 90–95% of these cancers [3]. Based on the SEER database, 16% of kidney cancer patients presented with metastatic tumors at diagnosis. Also, the 5-year survival rate of metastatic kidney cancer patients was only 13.0% during the 2010–2016 period [4]. The national Swedish kidney cancer register reported comparable findings, in which between 2005 and 2010, 15–23% of the RCC patients presented with metastatic forms of the disease at diagnosis [5].

Accurate prognostic models are invaluable in designing clinical trials, patient psychological management, and governing therapeutic modalities. Currently, the Memorial Sloan-Kettering Cancer Center (MSKCC) and the International Metastatic RCC Database Consortium (IMDC) models are the most widely used prognostic models for metastatic RCC. Karnofsky performance, level of serum lactate dehydrogenase, hemoglobin, and corrected calcium as well as time from diagnosis to initiation of treatment are the independent prognostic factors in the MSKCC model [6]. The efficiency of the MSKCC model was validated in an independent study of 353 participants, previously untreated for metastatic RCC [7]. In the era of targeted therapy, Heng et al. identified 645 patients with metastatic RCC treated with sunitinib, sorafenib, or bevacizumab plus interferon from the IMDC during 2004–2008, and proposed the IMDC model including Karnofsky performance status, serum hemoglobin, corrected serum calcium, time from diagnosis to treatment together with neutrophils and platelets [8]. The two models provide the median OS for each group after stratifying metastatic RCC patients based on the prognostic risk factors.

Clinical and pathological characteristics of metastatic RCC patients are highly heterogeneous. In addition, they exhibit highly variable survival time [9]. Besides the MSKCC and IMDC models, individual survival prediction can also be implemented by risk-scoring systems. Unfortunately, best to our knowledge, there is no risk-scoring system for metastatic RCC patients. Accordingly, we aimed at constructing a predictive nomogram for patients newly diagnosed with metastatic RCC by combining clinical and pathological characteristics derived from the SEER database. In addition, we further aimed at evaluating the discrimination, calibration, and clinical use of this model.

Methods

Data source

We used data in the publicly available SEER database (http://www.seer.cancer.gov), which covers approximately 28% of the US population. SEER provides data on patient demographics, primary tumor site, tumor stage, surgical treatment, patient survival among others. Relevant data was retrieved using SEER*Stat software (version 8.3.5).

Study population

To be included in this study, participants must have first presented with metastasis RCC at diagnosis. The RCC must have been identified using universal morphology codes (8050/3, 8260/3, 8310/3, 8317/3, 8318/3, and 8319/3) based on the International Classification of Diseases for Oncology codes (3rd edition). Besides, the RCC must have been the first and only primary disease. The diagnoses were confirmed by histological examination and there was complete follow-up data of the patients. Patients under 18 years at diagnosis, missing data on follow-up, race, marital status, Fuhrman grade, tumor size, tumor stage, lymph node status, metastasis, and surgery were all excluded from the study. Autopsy or death certificate cases were also excluded. In the end, 2315 metastatic RCC patients fulfilled the inclusion criteria, and were included in the final analyses. The flow diagram for patient selection is presented in Fig. 1.

Fig. 1
figure 1

Diagrammatic flow of patient selection

Full size image

Measurable variables

Demographic and clinical variables such as age at diagnosis, race (black, white, other), sex, marital status (married, unmarried), histologic subtype (clear cell renal cell carcinoma (CCRCC), papillary renal cell carcinoma (PRCC), chromophobe renal cell carcinoma (CHRCC), sarcomatoid renal cell carcinoma (SRCC), collecting duct renal cell carcinoma (CDRCC)), Fuhrman grade (grade I, grade II, grade III, grade IV), Tumor classification (T1, T2, T3, T4, TX), lymph node status (N0, N1, NX), sarcomatoid feature (yes, no, unknown), cancer-directed surgery (recommended and performed, recommended but not performed, not recommended), bone, brain, liver, and lung metastasis status, survival time and vital status were all captured. The AJCC Cancer Staging Manual (7th edition, 2010) was employed to evaluate tumor stages.

Ascertainment of the outcome

OS was the primary outcome of this study, defined as survival time between metastatic RCC diagnoses to death, attributed to any cause. OS was ascertained based on the code “vital status” in the SEER database.

Statistical analysis

Selected patients were randomly and equally stratified into training and validation sets. Preliminary descriptive statistics were performed to describe the baseline characteristics of the patients in both sets. Thereafter, normally distributed continuous variables were expressed as mean ± the standard deviation, whereas non-normal continuous variables were presented by medians (interquartile range). Categorical variables were summarized in frequencies and percentages. Univariable and multivariable Cox regression analyses were performed on the training set to obtain crude and adjusted hazard ratios (HRs), important in identifying prognostic factors significant for OS. Prognostic factors were determined by a backward stepwise process using the Bayesian information criterion. Schoenfeld residuals were used to assess the proportional hazards assumption of Cox regression models.

Nomograms are pictorial representations that quantify risks and the probability of clinical events by scoring the involved factors. They have been demonstrated to generate more precise predictions than the conventional AJCC staging system in several types of cancers [10, 11]. In this study, a nomogram for predicting 1, 3 and 5-year OS was derived based on the findings of multivariable Cox regression analysis.

Discrimination and calibration, important properties in evaluating the performance of the model [12, 13], were both assessed in our study. C-index was applied to evaluate the discriminative ability of the nomogram, which depicted the probability of the predicted risk was higher for a random patient having an event than for a random patient not having an event. After comparing the predicted probability of events for all possible pairs of patients, C-index is 0.5 if the model can not discriminate the patients with and without events. Conversely, C-index is 1 if the probability predicted by the model is always higher for patients with events than those without events [14]. The robust performance of the model was assessed using the original and optimism-corrected C-indices. Calibration plot, the best visual representation of the relationship between predicted risk and actual risk, was presented using bootstrap resampling method [14]. Calibration plots fall on a 45-degree diagonal line, reflecting excellent absolute risk estimates. NRI and IDI usually assess and quantify the refinement in risk prediction between new and old models [15]. NRI is based on reclassification tables composed of patients with and without events, and can quantify the correct reclassification in categories. NRI is calculated by adding the percentage of patients with events who are correctly reclassified to the percentage of patients without events who are correctly reclassified [14]. IDI reflects the improvement in sensitivity and specificity of a model. It is also an integrated difference in Youden’s indices [15]. IDI is calculated by adding the increased probability predicted by new model compared to old model for patients with events to the decreased probability predicted by new model compared to old model for patients without events [15]. Therefore, NRI and IDI were both employed to compare the discriminative ability between the new model and the AJCC staging system. Notably, even though NRI and IDI have become increasingly popular, they should be interpreted with caution [16]. IPA is a promising metric that combines discrimination and calibration in one value, thus improves interpretability by adjusting for the benchmark model [17, 18]. IPA was also reported in this study to reflect the performance of the model. DCA is a method for evaluating the benefits of a diagnostic test across a range of patient preferences for accepting risk of undertreatment and overtreatment to facilitate decisions about test selection and use [19]. Unlike the sensitivity, specificity, and area under the curve, DCA directly assesses the utility of clinical risk prediction models for decision making [20]. Herein, DCA was plotted to evaluate the clinical value of the nomogram by quantifying the net benefit in comparison with the AJCC staging system.

Statistical tests were performed using R software (version 3.5.2, http://www.r-project.org/). All tests were two-sided, with statistical significance set at P < 0.05.

Results

Patient baseline characteristics

Data for 2315 eligible metastatic RCC patients collected between 2010 and 2016 were extracted from the SEER database, and then randomly stratified equally into two sets, with 1158 and 1157 patients in the training and validation sets, respectively. The median age of the participants was 61 (54–68) years. Whites accounted for 83.8% of the patients. With regard to gender, 1629 (70.4%) patients were males. The majority of the patients (67.3%) were married. The most common histologic subtype was CCRCC (84.6%), with Grade III (40.0%) being the most common Fuhrman grade. The majority of the tumors were T3 (56.8%), with T2 and T4 accounting for the rest (16.5 and 12.7%, respectively). Concerning treatment, 85.6% of the patients underwent cancer-directed surgery. Overall, at diagnosis, 31.5, 9.9, 11.0, and 61.7% of the patients presented with bone, brain, liver, and lung metastases, respectively.

The median follow-up time of the cohort was 16 months, whereas the median OS time was 19 months (95% CI, 18–21 months). The median follow-up time for patients without events was 30 months. By the end of the survey, 1563 patients had died, of which 1467 from RCC and 96 from other causes. The demographic and clinical characteristics of the patients were summarized in Table 1.

Table 1 Demographics and clinical characteristics for the metastatic RCC patients
Full size table

Independent prognostic factors

Univariable and multivariable Cox regression analyses were performed to explore independent risk factors for OS. Crude and adjusted HRs were presented in Table 2. After adjustment for other risk factors, identified eight variables: Fuhrman grade, lymph node status, sarcomatoid feature, cancer-directed surgery, bone, brain, liver, and lung metastases were significantly associated with OS. Subgroup analyses (Table 3) also showed patients not recommended for nephrectomy were more likely to succumb to death than those who underwent nephrectomy, indicating the robustness of the model.

Table 2 Univariable and multivariable Cox regression analysis of OS for metastatic RCC patients
Full size table
Table 3 Subgroup analyses for patients who underwent or did not undergo nephrectomy
Full size table

Nomogram construction

The 1, 3 and 5-year predictive nomogram for OS were constructed only incorporating the significant prognostic factors. As shown in the nomogram (Fig. 2), cancer-directed surgery was the most significant factor for prognosis, followed by Fuhrman grade, brain metastasis, and sarcomatoid feature. Lymph node status, lung, and liver metastases moderately impacted on OS. Meanwhile, Bone metastasis had the least effect on OS.

Fig. 2
figure 2

Nomogram for 1, 3, and 5-year prediction of OS of patients with metastasis RCC a Classical application of the nomogram b. Each category of the prognostic variables was assigned a score on the Points scale. After summing up the score of each variable and locating the total score on the Total Points scale, a line was vertically drawn to the 1, 3, and 5-year survival probability scale and estimated survival probability at each time point could be obtained

Full size image

Classical application of the nomogram

The practical use of the nomogram was presented in Fig. 2. Here, a patient was diagnosed with Fuhrman grade IV, N1 stage metastatic RCC. He lacked sarcomatoid differentiation and displayed no bone metastasis. After assessment, he underwent nephrectomy. The nomogram scored 39 points for Fuhrman grade IV, 31 for N1 stage, 0 for no sarcomatoid feature, 27 for bone metastasis, and 0 for cancer-directed surgery, totaling 97 points. This corresponded with a 1, 3 and 5-year survival probability of 57, 20, and 8%, respectively.

Nomogram performance

Discrimination and calibration were employed to evaluate the performance of the model. The C-indices of the nomogram were 0.701 (95% CI, 0.682–0.720) for the training set and 0.676 (95% CI, 0.655–0.697) for the validation set. With respect to the AJCC staging system, the C-indices were 0.612 (95% CI, 0.591–0.633) for the training set and 0.600 (95% CI, 0.578–0.622) for the validation set, significantly lower than those of the nomogram (P < 0.001). Across the 1000 bootstrap resamples, the optimism-corrected C-indices of the nomogram were 0.698 (95% CI: 0.677–0.717) and 0.677 (95% CI: 0.656–0.697) for the training and validation sets, respectively, indicative of robustness in the performance of the model. Calibrations of the nomogram were assessed using actual and predicted estimates after bootstrapping with 1000 resamples. Calibration plots (Fig. 3) revealed good consistency between the predicted and the actual survival in both sets. Category-free NRI for 1, 3 and 5-year follow-up in the training set were 0.546 (95% CI: 0.376–0.655), 0.551 (95% CI: 0.368–0.679) and 0.468 (95% CI: 0.276–0.710), respectively, and 0.455 (95% CI: 0.307–0.610), 0.454 (95% CI: 0.305–0.615) and 0.356 (95% CI: 0.172–0.555), respectively, in the validation set. In addition, IDI for 1, 3 and 5-year follow-up in the training set were 0.082 (P < 0.001), 0.084 (P < 0.001) and 0.063 (P < 0.001), respectively, and 0.052 (P < 0.001), 0.057 (P < 0.001) and 0.048 (P < 0.001), respectively, in the validation set. The IPA estimates for the nomogram were greater than AJCC staging estimates in training set (19.6% vs. 7.8, 18.2% vs. 5.9 and 12.8% vs. 0.8%, for 1, 3 and 5-year IPA) and validation set (15.4% vs. 6.7, 15.9% vs. 5.7 and 9.1% vs. 2.9%, for 1, 3 and 5-year IPA). These results demonstrated the superior predictive power of the nomogram for OS over the AJCC staging system.

Fig. 3
figure 3

Calibration curves for predicting 1, 3 and 5-year OS of patients with metastasis RCC in training set a, b, c and validation set d, e, f. The nomogram-predicted probability of OS was plotted on the x-axis, with actual OS plotted on the y-axis. The calibration curves were visual representations of the relationship between the predicted and actual absolute risk

Full size image

Clinical use

DCA plots showed that our nomogram had greater net benefits in comparison with the AJCC staging system for predicting 1, 3 and 5-year OS of patients with metastatic RCC (Fig. 4), demonstrating its application in guiding clinical decision for metastatic RCC patients.

Fig. 4
figure 4

DCA for the nomogram and the AJCC staging system in the training set a, b, c and validation set d, e, f for 1, 3, and 5-year survival, respectively. Horizontal coordinates represented the threshold of probability, whereas the vertical coordinates represented the net benefit rate. The red and black dash lines represented DCA of the nomogram, and the AJCC staging system, respectively. The black solid line assumed all patients were alive, whereas the gray solid one with a negative slope assumed all patients were dead. DCA plot depicted the model with superior clinical application

Full size image

Discussion

It is well accepted that clinical and pathological heterogeneity of metastatic RCC highly influences the prognosis of patients with the cancer [9]. Therefore, accurate prognosis prediction of the tumor continually frustrates efforts against the cancer. The IMDC model is the most widely used prognostic prediction model for metastatic RCC patients receiving targeted therapy. It is composed of six clinical and laboratory factors [8]. Using this model, patients can be stratified into three risk groups, each with different median OS. The efficacy of IMDC has been validated in metastatic RCC patients receiving first-line [8], second-line [21], and third-line targeted therapies [22]. Such models aside, risk-scoring systems including nomograms can also predict survival probability of individual patients. This study comprised of data for patients from multiple centers, archived in the SEER database. The focus was on the newly diagnosed metastatic RCC patients for whom the MSKCC and IMDC models might be unsuitable. Meanwhile, the predictive model proposed in this study is a nomogram, demonstrated to predict the OS more precisely. Apparently, accurate OS prediction is one of the most concerned issues for metastatic RCC patients. Our nomogram meets all the criteria necessary for endorsement by the AJCC [23], and has the potential to supplement the AJCC staging system.

With an HR of 1.54, lymph node metastasis negatively affected the survival of metastatic RCC patients. Using 2530 RCC patients treated with nephrectomy, Karakiewicz et al. constructed a nomogram for survival prediction based on six independent factors, including tumor stage, lymph node status, metastatic status, tumor size, Fuhrman grade, and type of symptoms [24]. Pati et al. also reported that lymph node metastasis strongly impacted on progression-free and overall survival of metastatic RCC patients [25], consistent with the findings at Cleveland Clinic [7]. Fuhrman grade, as the most widely accepted prognostic grading system, is not only associated with the survival of local RCC but also metastatic types [24]. In this study, compared with grade I, Fuhrman grade IV was associated with a worse OS after adjusting other clinical variables (HR = 1.80, 95% CI: 1.06–3.06, P = 0.029). Sarcomatoid differentiation is another predictor of short survival for metastatic RCC patients. Considering the small percentage of patients with sarcomatoid features (< 6%), sarcomatoid differentiation was not included in the IMDC model for OS prediction, though it is associated with adverse oncologic outcomes [8]. However, nearly 20% of patients included in this study exhibited sarcomatoid differentiation, with patients not displaying this feature shown to have superior OS (HR = 0.61, 95% CI: 0.50–0.75, P < 0.001). Therefore, it was included in our model.

Based on a Nationwide Inpatient epidemiologic study, lung was the most common metastatic site for metastatic RCC, accounting for 45.2% of such cases. Bone and liver metastases followed with 29.5 and 20.3%, respectively [26]. For this study, 61.7, 31.5, 11.1, and 9.9% of the patients in our cohort presented with lung, bone, liver, and brain metastases, respectively (Table 1). These four clinical variables were all significantly associated with adverse survival, and our findings were consistent with the growing evidence that the presence of distant metastasis predicted worse clinical outcomes. In one phase III trial of 375 metastatic CCRCC patients who received sunitinib, lung and liver metastases were significantly associated with poor progression-free survival [27]. In a separate study involving 281 metastatic RCC patients administered with third-line targeted therapy, it was demonstrated metastases at diagnosis were associated with adverse effects on survival [28]. In a separate retrospective study of over 2000 metastatic RCC patients who received first-line targeted therapy, bone or liver metastasis were independent risk factors for shorter OS. Intriguingly, compared with other metastatic sites, both bone and liver metastases were associated with worse survival. Also, the incorporation of bone and liver metastases significantly increased the predictive performance of the IMDC model [29].

In the era of cytokines, two randomized controlled trials demonstrated prolonged survival with nephrectomy plus interferon, relative to interferon alone among metastatic RCC patients [30, 31]. Early retrospective studies also corroborated the role of cytoreductive nephrectomy [32, 33]. However, the role of nephrectomy in treating metastatic RCC in the era of targeted therapy remains controversial. For instance, in one randomized trial (CARMENA), sunitinib monotherapy against intermediate- to poor-risk metastatic RCC was not inferior to a combination modality of nephrectomy followed by sunitinib [33]. Though the survival advantage to metastatic RCC patients with favorable characteristics was not evaluated in CARMENA, a significant proportion of metastatic RCC patients did not benefit from nephrectomy. Multiple retrospective studies investigated the association between nephrectomy and clinical outcomes among metastatic RCC patients receiving targeted therapy. Most of them supported this population could receive survival benefits from nephrectomy [34,35,36,37]. Using the IMDC database, Heng et al. reported metastatic RCC patients who received targeted therapy and cytoreductive nephrectomy had longer OS compared to those on targeted therapy alone (20.6 vs. 9.6 months, P < 0.001). After adjusting for other variables, the HR for death was 0.60 (95% CI: 0.52–0.69, P < 0.001) [34]. Comparable findings were reported following analysis of the National Cancer Data Base [35]. Most investigations analyzing the cytoreductive nephrectomy benefits are inherently limited by their retrospective nature. Notably, under any circumstances, patient selection and timing of surgery critically impact on nephrectomy benefits in metastatic RCC patients [33, 35]. Most often, patients with poor health status are unlikely to benefit from nephrectomy [38]. On the other side, patients with good performance status, minimal symptoms related to metastases, a resectable primary tumor, and a limited burden of metastatic disease are likely to highly benefit from the surgery [38]. One systematic review also found that rigorously selected patients based on prognostic factors, with less metastasis, and those displaying favorable responses to initial systemic therapy could highly benefit from nephrectomy [39]. In this study, patients recommended for surgery and who indeed underwent the procedure were more likely to benefit from nephrectomy relative to non operated patients. Subgroup analyses validated the robustness of the nomogram. It is noteworthy that there is no “one size fits all” approach for managing cancers [33]. Stated differently, the current findings do not recommend cytoreductive nephrectomy to all patients, but rather, clinicians should adopt rigorous selection criteria for surgery. The findings of this study are only based on retrospective data in the SEER database, thus should be interpreted with caution. As such, more investigations should be performed to strengthen the reliability of this model before its clinical application.

Our findings notwithstanding, this study suffered several limitations. First, because of the retrospective nature, inherent selection bias associated with such studies, in particular treatment selection bias was unavoidable. Because more relatively healthy patients were commonly recommended for surgery, it was more likely they would exhibit better clinical outcomes. Second, other independent RCC prognostic factors such as MSKCC or IMDC risk stratification, level of serum albumin, and lactate dehydrogenase were not captured in the SEER database. Third, some potentially valuable variables such as type of targeted therapy, timing of surgery, and other comorbidities were also not captured in the SEER database.

Conclusions

In summary, we established a nomogram for survival prediction of metastatic RCC patients. It incorporated eight clinical factors: Fuhrman grade, lymph node status, sarcomatoid feature, cancer-directed surgery, bone, brain, liver, and lung metastases. The nomogram displayed superior predictive capability and higher clinical application than the conventional AJCC staging system.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available in the SEER repository (http://www.seer.cancer.gov).

Abbreviations

OS:

Overall survival

RCC:

Renal cell carcinoma

SEER:

Surveillance, Epidemiology and End Results

C-index:

Harrell’s concordance index

IDI:

Integrated discrimination improvement

NRI:

Net reclassification improvement

IPA:

Index of prediction accuracy

DCA:

Decision curve analysis

AJCC:

American Joint Committee on Cancer

MSKCC:

Memorial Sloan-Kettering Cancer Center

IMDC:

International Metastatic RCC Database Consortium

CCRCC:

Clear cell renal cell carcinoma

PRCC:

Papillary renal cell carcinoma

CHRCC:

Chromophobe renal cell carcinoma

SRCC:

Sarcomatoid renal cell carcinoma

CDRCC:

Collecting duct renal cell carcinoma

HR:

Hazard ratio

95% CI:

95% confidence interval

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424.

    Google Scholar 

  2. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA Cancer J Clin. 2019;69(1):7–34.

    Article  Google Scholar 

  3. Marconi L, de Bruijn R, van Werkhoven E, Beisland C, Fife K, Heidenreich A, et al. External validation of a predictive model of survival after cytoreductive nephrectomy for metastatic renal cell carcinoma. World J Urol. 2018;36(12):1973–80.

    Article  CAS  Google Scholar 

  4. SEER Cancer Stat Facts: Kidney and Renal Pelvis Cancer. Bethesda: National Cancer Institute. https://seer.cancer.gov/statfacts/html/kidrp.html. Accessed 12 Jan 2020.

  5. Thorstenson A, Bergman M, Scherman-Plogell AH, Hosseinnia S, Ljungberg B, Adolfsson J, et al. Tumour characteristics and surgical treatment of renal cell carcinoma in Sweden 2005-2010: a population-based study from the national Swedish kidney cancer register. Scand J Urol. 2014;48(3):231–8.

    Article  Google Scholar 

  6. Motzer RJ, Bacik J, Murphy BA, Russo P, Mazumdar M. Interferon-Alfa as a comparative treatment for clinical trials of new therapies against advanced renal cell carcinoma. J Clin Oncol. 2002;20:289–96.

    Article  CAS  Google Scholar 

  7. Mekhail TM, Abou-Jawde RM, Boumerhi G, Malhi S, Wood L, Elson P, et al. Validation and extension of the memorial Sloan-Kettering prognostic factors model for survival in patients with previously untreated metastatic renal cell carcinoma. J Clin Oncol. 2005;23(4):832–41.

    Article  Google Scholar 

  8. Heng DY, Xie W, Regan MM, Warren MA, Golshayan AR, Sahi C, et al. Prognostic factors for overall survival in patients with metastatic renal cell carcinoma treated with vascular endothelial growth factor-targeted agents: results from a large, multicenter study. J Clin Oncol. 2009;27(34):5794–9.

    Article  CAS  Google Scholar 

  9. Margulis V, Shariat SF, Rapoport Y, Rink M, Sjoberg DD, Tannir NM, et al. Development of accurate models for individualized prediction of survival after cytoreductive nephrectomy for metastatic renal cell carcinoma. Eur Urol. 2013;63(5):947–52.

    Article  CAS  Google Scholar 

  10. Liang W, Zhang L, Jiang G, Wang Q, Liu L, Liu D, et al. Development and validation of a nomogram for predicting survival in patients with resected non-small-cell lung cancer. J Clin Oncol. 2015;33(8):861–9.

    Article  Google Scholar 

  11. Chen S, Liu Y, Yang J, Liu Q, You H, Dong Y, et al. Development and validation of a nomogram for predicting survival in male patients with breast cancer. Front Oncol. 2019;9:361.

    Article  CAS  Google Scholar 

  12. Kattan MW, Vickers AJ. Statistical analysis and reporting guidelines for CHEST. Chest. 2020;158(1S):S3–S11.

    Article  Google Scholar 

  13. Kattan MW, Gerds TA. A framework for the evaluation of statistical prediction models. Chest. 2020;158(1S):S29–38.

    Article  Google Scholar 

  14. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: Users' guides to the medical literature. JAMA. 2017;318(14):1377–84.

    Article  Google Scholar 

  15. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.

    Article  Google Scholar 

  16. Hilden J, Gerds TA. A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index. Stat Med. 2014;33(19):3405–14.

    Article  Google Scholar 

  17. Kattan MW, Gerds TA. The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models. Diagn Progn Res. 2018;2:7.

    Article  Google Scholar 

  18. Kattan MW, Gerds TA. The c-index is not proper for the evaluation of t-year predicted risks. Biostatistics. 2019;20(2):347–57.

    Article  Google Scholar 

  19. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.

    Article  Google Scholar 

  20. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. 2018;74(6):796–804.

    Article  Google Scholar 

  21. Ko JJ, Xie W, Kroeger N, Lee J-L, Rini BI, Knox JJ, et al. The international metastatic renal cell carcinoma database consortium model as a prognostic tool in patients with metastatic renal cell carcinoma previously treated with first-line targeted therapy: a population-based study. Lancet Oncol. 2015;16(3):293–300.

    Article  Google Scholar 

  22. Wells JC, Stukalin I, Norton C, Srinivas S, Lee JL, Donskov F, et al. Third-line targeted therapy in metastatic renal cell carcinoma: results from the international metastatic renal cell carcinoma database consortium. Eur Urol. 2017;71(2):204–9.

    Article  Google Scholar 

  23. Kattan MW, Hess KR, Amin MB, Lu Y, Moons KG, Gershenwald JE, et al. American joint committee on Cancer acceptance criteria for inclusion of risk models for individualized prognosis in the practice of precision medicine. CA Cancer J Clin. 2016;66(5):370–4.

    Article  Google Scholar 

  24. Karakiewicz PI, Briganti A, Chun FK, Trinh QD, Perrotte P, Ficarra V, et al. Multi-institutional validation of a new renal cancer-specific survival nomogram. J Clin Oncol. 2007;25(11):1316–22.

    Article  Google Scholar 

  25. Patil S, Figlin RA, Hutson TE, Michaelson MD, Negrier S, Kim ST, et al. Prognostic factors for progression-free and overall survival with sunitinib targeted therapy and with cytokine as first-line therapy in patients with metastatic renal cell carcinoma. Ann Oncol. 2011;22(2):295–300.

    Article  CAS  Google Scholar 

  26. Bianchi M, Sun M, Jeldres C, Shariat SF, Trinh QD, Briganti A, et al. Distribution of metastatic sites in renal cell carcinoma: a population-based analysis. Ann Oncol. 2012;23(4):973–80.

    Article  CAS  Google Scholar 

  27. Motzer RJ, Bukowski RM, Figlin RA, Hutson TE, Michaelson MD, Kim ST, et al. Prognostic nomogram for sunitinib in patients with metastatic renal cell carcinoma. Cancer. 2008;113(7):1552–8.

    Article  CAS  Google Scholar 

  28. Iacovelli R, Farcomeni A, Sternberg CN, Carteni G, Milella M, Santoni M, et al. Prognostic factors in patients receiving third line targeted therapy for metastatic renal cell carcinoma. J Urol. 2015;193(6):1905–10.

    Article  Google Scholar 

  29. McKay RR, Kroeger N, Xie W, Lee JL, Knox JJ, Bjarnason GA, et al. Impact of bone and liver metastases on patients with renal cell carcinoma treated with targeted therapy. Eur Urol. 2014;65(3):577–84.

    Article  Google Scholar 

  30. Mickisch GHJ, Garin A, van Poppel H, de Prijck L, Sylvester R. Radical nephrectomy plus interferon-alfa-based immunotherapy compared with interferon alfa alone in metastatic renal-cell carcinoma: a randomised trial. Lancet. 2001;358(9286):966–70.

    Article  CAS  Google Scholar 

  31. Flanigan RC, Salmon SE, Blumenstein BA, Bearman SI, Roy V, Mcgrath PC, et al. Nephrectomy followed by interferon alfa-2b compared with interferon alfa-2b alone for metastatic renal-cell cancer. N Engl J Med. 2001;345:1655–9.

    Article  CAS  Google Scholar 

  32. Motzer RJ, Mazumdar M, Bacik J, Berg W, Amsterdam A, Ferrara J. Survival and prognostic stratification of 670 patients with advanced renal cell carcinoma. J Clin Oncol. 1999;17:2530–40.

    Article  CAS  Google Scholar 

  33. Mejean A, Ravaud A, Thezenas S, Colas S, Beauval JB, Bensalah K, et al. Sunitinib alone or after nephrectomy in metastatic renal-cell carcinoma. N Engl J Med. 2018;379(5):417–27.

    Article  CAS  Google Scholar 

  34. Heng DY, Wells JC, Rini BI, Beuselinck B, Lee JL, Knox JJ, et al. Cytoreductive nephrectomy in patients with synchronous metastases from renal cell carcinoma: results from the international metastatic renal cell carcinoma database consortium. Eur Urol. 2014;66(4):704–10.

    Article  Google Scholar 

  35. Hanna N, Sun M, Meyer CP, Nguyen PL, Pal SK, Chang SL, et al. Survival analyses of patients with metastatic renal cancer treated with targeted therapy with or without cytoreductive nephrectomy: a National Cancer Data Base study. J Clin Oncol. 2016;34(27):3267–75.

    Article  Google Scholar 

  36. de Groot S, Redekop WK, Sleijfer S, Oosterwijk E, Bex A, Kiemeney LA, et al. Survival in patients with primary metastatic renal cell carcinoma treated with sunitinib with or without previous cytoreductive nephrectomy: results from a population-based registry. Urology. 2016;95:121–7.

    Article  Google Scholar 

  37. Klatte T, Fife K, Welsh SJ, Sachdeva M, Armitage JN, Aho T, et al. Prognostic effect of cytoreductive nephrectomy in synchronous metastatic renal cell carcinoma: a comparative study using inverse probability of treatment weighting. World J Urol. 2018;36(3):417–25.

    Article  Google Scholar 

  38. Mason RJ, Wood L, Kapoor A, Basappa N, Bjarnason G, Boorjian SA, et al. Kidney Cancer research network of Canada (KCRNC) consensus statement on the role of cytoreductive nephrectomy for patients with metastatic renal cell carcinoma. Can Urol Assoc J. 2019;13(6):166–74.

    Google Scholar 

  39. Bhindi B, Abel EJ, Albiges L, Bensalah K, Boorjian SA, Daneshmand S, et al. Systematic review of the role of cytoreductive nephrectomy in the targeted therapy era and beyond: an individualized approach to metastatic renal cell carcinoma. Eur Urol. 2019;75(1):111–28.

    Article  Google Scholar 

Download references

Acknowledgments

Our sincere gratitude to the NIH-SEER program for high-quality data for researchers. We also thank our colleagues for the assistance they accorded us.

Funding

This study was funded by Yantai Science and Technology Program (grant no. 2020YD014). The funding agency did not participate in the development of study design, data collection or analysis, or preparation of this manuscript.

Author information

Author notes

  1. Wenwen Zheng and Weiwei Zhu contributed equally to this work.

Authors and Affiliations

  1. Department of Education, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China

    Wenwen Zheng

  2. Drug Clinical Trial Agency, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China

    Weiwei Zhu, Kangqi Li & Congxiao Lu

  3. Department of Urology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China

    Shengqiang Yu

  4. Department of Pharmacy, Yantai Yuhuangding Hospital, Qingdao University, No.20, Yuhuangdingdong Road, Yantai, Shandong, China

    Yuexia Ding, Qingna Wu, Qiling Tang, Quan Zhao & Chenyu Guo

Authors
  1. Wenwen Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengqiang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kangqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuexia Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingna Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qiling Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Quan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Congxiao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Chenyu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

WWZ and WWZ developed the study design, whereas SY, KL, YD, QW, and QT collected and analyzed the data. WWZ drafted the manuscript, whereas CL, QZ, and CG made appropriate revisions. Meanwhile, all authors read and approved the final manuscript.

Corresponding author

Correspondence to Chenyu Guo.

Ethics declarations

Ethics approval and consent to participate

Our study was exempted from institutional review board approval because of using the de-identified data in the SEER database.

Consent for publication

Not applicable.

Competing interests

The authors declared that they had no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, W., Zhu, W., Yu, S. et al. Development and validation of a nomogram to predict overall survival for patients with metastatic renal cell carcinoma. BMC Cancer 20, 1066 (2020). https://doi.org/10.1186/s12885-020-07586-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12885-020-07586-7

Keywords

BMC Cancer

ISSN: 1471-2407

Contact us