Development and validation of a model that includes two ultrasound parameters and the plasma D-dimer level for predicting malignancy in adnexal masses: an observational study
BMC Cancer volume 19, Article number: 564 (2019)
Pre-operative discrimination of malignant from benign adnexal masses is crucial for planning additional imaging, preparation, surgery and postoperative care. This study aimed to define key ultrasound and clinical variables and develop a predictive model for calculating preoperative ovarian tumor malignancy risk in a gynecologic oncology referral center. We compared our model to a subjective ultrasound assessment (SUA) method and previously described models.
This prospective, single-center observational study included consecutive patients. We collected systematic ultrasound and clinical data, including cancer antigen 125, D-dimer (DD) levels and platelet count. Histological examinations served as the reference standard. We performed univariate and multivariate regressions, and Bayesian information criterion (BIC) to assess the optimal model. Data were split into 2 subsets: training, for model development (190 observations) and testing, for model validation (n = 100).
Among 290 patients, 52% had malignant disease, including epithelial ovarian cancer (72.8%), metastatic disease (14.5%), borderline tumors (6.6%), and non-epithelial malignancies (4.6%). Significant variables were included into a multivariate analysis. The optimal model, included three independent factors: solid areas, the color score, and the DD level. Malignant and benign lesions had mean DD values of 2.837 and 0.354 μg/ml, respectively. We transformed established formulae into a web-based calculator (http://gin-onc-calculators.com/gynonc.php) for calculating the adnexal mass malignancy risk. The areas under the curve (AUCs) for models compared in the testing set were: our model (0.977), Simple Rules risk calculation (0.976), Assessment of Different NEoplasias in the adneXa (ADNEX) (0.972), Logistic Regression 2 (LR2) (0.969), Risk of Malignancy Index (RMI) 4 (0.932), SUA (0.930), and RMI3 (0.912).
Two simple ultrasound predictors and the DD level (also included in a mathematical model), when used by gynecologist oncologist, discriminated malignant from benign ovarian lesions as well or better than other more complex models and the SUA method. These parameters (and the model) may be clinically useful for planning adequate management in the cancer center. The model needs substantial validation.
An ultrasound examination performed by an experienced sonologist is considered the best diagnostic method for discriminating malignant from benign ovarian lesions [1,2,3]. Alternatively, optimal differential diagnosis of pelvic masses can be performed with predictive models that incorporate clinical and ultrasound variables [4,5,6,7]. Patients with tumors suspected for malignancy should be referred to a gynecologist oncologist (GO), because those treated in a referral center undergo cytoreduction to microscopic disease more often, and thus achieve significantly better overall survival [8,9,10]. Ovarian lesions considered indeterminate on an initial sonography should receive a “second step” evaluation, e.g. the patient might be referred to a GO . It means, that this consultant has to perform an accurate presurgical differential diagnosis of the pelvic mass to ensure patients are correctly assigned the optimal surgical approach and must plan additional imaging, surgical team, suitable operating time and postoperative care for individuals with epithelial ovarian cancer (EOC). In many institutions, ultrasound examination performed by non-radiologist – the GO, is considered a primary imaging modality for differential diagnosing of pathological masses in the pelvis and even for the staging [12,13,14,15,16].
All patients with active malignancies demonstrate some degree of coagulation activation, including activated thrombin and fibrin formation . Elevated plasma D-dimer (DD) levels were considered a prognostic factor of poor overall survival, independent of venous thromboembolism (VTE) [18, 19], and a potential biomarker for the preoperative differentiation of benign versus malignant ovarian masses [20,21,22].
The majority of clinical activity of the GO should be devoted to the management of patients with a gynecological cancer, thus this consultant should be equipped with simple, clinically useful diagnostic tools or experience for discriminating malignant from benign ovarian tumors. Some GO relay on pelvis/abdominal computed tomography  or diffusion-weighted magnetic resonance  only, while others perform comprehensive ultrasound assessment of pelvic masses, followed by ultrasound scanning of the pelvis and abdomen to diagnose and define distant disease [12, 15, 25].
We aimed to define key ultrasound and clinical variables and if different from previous studies, than to develop and validate a predictive model for calculating preoperative ovarian tumor malignancy risk in a gynecologic oncology referral center. We aimed to use ultrasound parameters, that would be relatively simple, not confusing and easily reproduced. We compared our model to a subjective ultrasound assessment (SUA) method and previously described models. We hypothesized that our set of variables or multivariable predictive model would outperform the SUA and other existing models in predicting the malignancy of adnexal masses.
This prospective, single-center, observational study included consecutive patients with ovarian tumors that underwent surgery within 60 days of the ultrasound examination in the Department of Gynecologic Oncology, in Gdynia Oncology Center, Poland. Exclusion criteria were: a prior bilateral oophorectomy, pregnancy, refusal to undergo ultrasonography.
All patients underwent transvaginal and transabdominal sonography (both during the same examination), according to a standardized gray-scale and Doppler protocol, before surgery, with high-quality ultrasound equipment. When patients exhibited multiple adnexal masses, the statistical analysis included only the mass with the most complex ultrasound morphology. When all masses had similar ultrasound morphology, we included only the most easily accessible to ultrasound examination.
The ultrasound protocol provided data on: the largest tumor diameter, locularity (multilocular if ≥2 locules, yes = 1, no = 0), solid areas (yes = 1, if any papillation or a solid tumor as described elsewhere , no = 0, if none), the presence of ascites (fluid outside the pouch of Douglas, yes = 1, no = 0), pelvic or abdominal metastases (any detected with ultrasound, e.g. carcinomatosis, yes = 1, no = 0); data on intratumoral vascularization included the qualitative Color Score (CS), a standardized terminology described elsewhere  (1 = no, 2 = minimal, 3 = moderate, 4 = very strong blood flow), and quantitative evaluations of the pulsatility index (PI), resistance index (RI), and peak systolic velocity (PSV), in areas with the highest blood flow velocity [26, 27]. When no blood flow or only venous flow was detected, the RI, PI, and PSV values were arbitrarily assigned values of 1, 1.2, and 10, respectively . Preoperative data included patient age, menopausal status (postmenopausal was defined as: more than 1 year of amenorrhea without a diagnosis of any endocrine disease that could influence menstrual cycles; receiving hormonal replacement therapy for menopausal symptoms; or ≥ 50 years of age, with a previous hysterectomy), and laboratory data: cancer antigen 125 (CA-125), platelet count (PLT), and plasma DD level. Age and laboratory data were modelled continuously - no threshold cut-off values were used. Blood tests were performed 1–14 days prior to surgery. The DD level was tested in citrate plasma. Briefly, venous blood was drawn from peripheral vessels, centrifuged, and the supernatant was diluted with a 0.11 M (3.2%) buffered solution of sodium citrate, where the component ratio was 9:1, respectively. The DD level was determined with immunoturbidometry, measured with an automatic coagulation analyser (STA R Evolution and STA Compact, Diagnostica STAGO) and a special kit of agents (i.e., latex particles coated with DD-specific murine monoclonal antibodies). We used second generation immunoradiometric assay kits for detecting CA-125 II (Roche Diagnostics). Kits included the OC125 antibody. Laboratory tests were performed as part of routine preoperative assessment.
The protocol was amended in December 2015. Subsequently, we collected additional ultrasound data, including the maximal diameter of the largest solid component, the number of papillary projections, and the presence/absence of more than 10 locules, acoustic shadows, blood flow in papillations, and an irregular cyst wall [26, 29].
Calculations of other predictive models were performed according to instructions provided elsewhere: the International Ovarian Analysis Group (IOTA): the Logistic Regression 2 (LR2) , Simple Rules (SR) , the Assessment of Different NEoplasias in the adneXa (ADNEX) , and the Simple Rules risk calculation (SRrisk) ; two versions of the risk-of-malignancy index (RMI): RMI3  and RMI4  (Additional file 1). All ADNEX model calculations included the CA-125 level.
The SUA was based on interviews, clinical and ultrasound (grey-scale and color Doppler) examinations. The examiner rated the level of diagnostic confidence as: certainly malignant; probably malignant; uncertain, but likely malignant; uncertain, but likely benign; probably benign; and certainly benign.
According to our institutional practice, an ultrasound scan was performed by a GO as a routine preoperative assessment. The examiner had 8–13 years of experience in ultrasound scanning; was blinded to histological reference diagnosis. All variables were recorded prospectively in a database (Excel, Microsoft Corporation), before surgery, and they were not changed thereafter. CA-125 results could be available to the examiner at the time of the ultrasound performance. All model’s calculations were performed after the study concluded; thus, the models played no role in the decision-making process. No imaging for VTE diagnosis was performed based solely on the finding that the plasma DD level was elevated.
The outcome was the histological diagnosis of the entire ovarian mass removed during surgery. Based on histology, tumors were classified according the World Health Organization classification of tumors . The reference test assessor was blinded to pre-surgery test details, but not to the general clinical impression and the CA-125 level.
Patients with early EOC received complete surgical staging, and advanced disease was treated with maximum debulking. Patients with benign tumors received individualized surgical treatment.
The study protocol was approved by the local Ethics Committee at Medical Council in Gdansk, Poland (KB – 36/17). This paper was written according to the standards for reporting of diagnostic accuracy (STARD) initiative  and transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) (Additional file 2) .
Statistical analysis was performed with R software v.3.4.3 . Mann-Whitney U test was used to check if there were significant differences in distributions between benign and malignant groups. The model was developed with the learning group, based on the original protocol. The model was validated and compared to other predictive models with the independent testing group, based on the amended protocol. The train and test sets were derived using a chronological split.
Before modelling, we selected variables in two stages. First, we reduced number of potential predictors based on subjective matter knowledge  and aims of our study (to select simple and easily reproduced parameters). Second, a univariable logistic regression model was constructed for each selected variable in the database of the learning group. Variables that achieved statistical significance (p < 0.05) in the univariable analysis were subsequently used in the multivariate model. This model used an algorithm of stepwise regression with backward selection. At each step, we removed the non-significant variable that caused the greatest reduction in the Bayes information criterion (BIC). The values of Akaike and Bayes (AIC/BIC) information criteria were reported since both serve valuable source of information . However, in case of any incompatibilities between AIC and BIC, value of BIC was taken into consideration as it generally performs better, without overfitting the model .
Calibration  of the predicted probability of malignancy calculated by the developed model was investigated using calibration curves and by the ratio between average predicted probability of malignancy and observed prevalence of malignancy . Plots were obtained using function from rms package . A ratio of < 1 or > 1 suggests general underprediction or overprediction of the risk, respectively. The calibration curves link the predicted risk with the observed outcome using a non-parametric logistic regression model based on local regression .
Sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV) and receiver operating characteristic (ROC) curves, and the areas under the curve (AUC) were used to compare the predictive efficacies of the different models and SUA. The cut-off 0.5 was used to calculate measures of the performance (sensitivity, specificity, PPV and NPV) for models. Predictive models that were based on logistic regression analysis (including our model and IOTA models: LR2, ADNEX, and SRrisk) were evaluated as the probability of malignancy with values that ranged from 0 to 1 (0–100%) – we did not use stratification into low or high risk, nor any predefined border values. RMI cut-off values and detailed description of all model’s calculations are presented in an Additional file 1. We used no threshold cut-off values for CA-125, DD, PLT values.
There were no indeterminate results of index test and other predictive models, but SR. The IOTA-SR was converted into a risk estimate, to create the SRrisk algorithm, based on definitions published previously . When the SUA was compared to the reference test and other predictive models, the first three levels of confidence were taken as malignant, and the last three were taken as benign. There were no indeterminate reference test results. In the statistical analysis, all benign epithelial and non-epithelial tumors were considered benign, and all epithelial malignant, borderline malignant, and non-epithelial malignant tumors were considered malignant.
There were no any missing data for the index test, nor for the reference standard.
The intended sample size was not determined before launching the study. However, we conducted the event per variable calculation to present the issue of the sample size and number of variables. According to recommendations, the suggested event per variable value should be more than 5–10 [39, 45].
From April 2012 to June 2017, we identified 307 eligible patients. Of these, 290 were included in the final analysis; 151 had malignant tumors (52%), and 139 had benign lesions (Fig. 1). The most common malignancy was EOC (72.8%), followed by metastatic cancer to the ovary (14.5%), borderline epithelial tumors (6.6%), non-epithelial malignancies (4.6%), and others (Fig. 1, Table 1).
Ovarian cancer, compared to benign disease, was associated with patient’s older age, post-menopause, and higher frequencies of the following tumor ultrasound features: the presence of solid areas, bilateral lesions, multilocularity, CS ≥2, the presence of ascites, and signs of metastases. Patients with malignancies had higher mean CA-125, plasma DD, and PLT count values than patients with benign lesions. The largest tumor diameters were similar between the benign and malignant groups (Table 2). Among patients with malignancies, the higher the stage, the higher the plasma DD level (p < 0.05) (Table 3). There were no differences in the median plasma DD level between patients with malignant epithelial primary (2.870, range, 0.138–28.618 μg/ml) and metastatic (2.828, range, 0.509–14.567 μg/ml) tumors.
The median time interval between the ultrasound examination and surgery was 1 day (range 0–60 days) for all included patients. No patient had clinical symptoms of VTE before surgery.
The learning group (N = 190, including 101 malignancies) was enrolled from April 2012 to May 2016. The testing group (N = 100, including 52 malignancies) was enrolled from June 2016 to June 2017. Data from both groups are presented in Additional file 3. The vascularization quantitative parameters (RI, PI, PSV) and the largest tumor diameter were excluded (learning group) before modelling based on subject matter knowledge  and lack of significant difference in univariate analysis, respectively. The following multivariate analysis was performed on learning group and included 11 clinical and ultrasound parameters from the protocol before amendment (Tables 4 and 5).
The analysis of the sample size and number of analyzed parameters revealed that the event per variable for the initial model was 8.1.
The learning group, model development
The multivariate analysis results are shown in Tables 4 and 5. In Table 5 results of AIC are reported until values between AIC and BIC are inconsistent. The final, optimal model had the lowest BIC, AUC was 0.956 (CI:0.932–0.981), sensitivity 91.1% (83.9–95.2), specificity 85.4% (76.6–91.3), PPV 87.6% (80.0–92.6), NPV 89.4% (81.1–94.3), and included three independent factors. Malignancy probability was calculated according to the mathematical formula:
where A, B and C represented the solid areas, the CS, the plasma DD level (μg/ml), respectively.
For example, a patient with a solid area in the tumor (score 1), a CS-3 (score 3), and a plasma DD level of 2.293 μg/ml had a risk of malignancy 0.9803 (range: 0–1); that multiplied by 100% gave a risk of 98.03%. We designed an internet web-based tool (http://gin-onc-calculators.com/gynonc.php), to facilitate calculations.
The testing group, model validation
In the testing group, models with the highest AUC values were: our model, the SRrisk and the ADNEX. The calculated performance indices are shown in Table 6 and Fig. 2. The SUA showed high values overall. Our model had the highest sensitivity and NPV. The LR2 had the highest specificity and PPV, and the lowest sensitivity. Our model and the RMI4 had the lowest specificity and PPV. Among all malignant tumors, the SR classified 52% as malignant, 1.9% as benign, and 46.1% as inconclusive. Among all benign tumors, the SR classified 83.3% as benign, 4.2% as malignant, and 12.5%, as inconclusive. Among patients with malignancies, 46% of tumors were described as without a blood flow (B5), and only 4% had CS-4; there were no any other SR benign features. Among patients with benign tumors 8% had an irregular multilocular-solid tumor with largest diameter > 100 mm (M4), and there were no any other SR malignant features. Details are presented in Additional file 4.
Overall, our model slightly overpredicted the risk of malignancy – ratio of predicted and observed risk, 1.123 (Additional file 5).
This study identified two simple ultrasound predictors: solid areas and blood flow (described with the CS) in the tumor; and one clinical predictor: plasma DD level, that could discriminate malignant from benign adnexal masses. We incorporated these parameters into a predictive model (also as the web-based tool) for calculating the probability of malignancy. Our model performance was comparable to that of the more detailed and complex models: SRrisk, ADNEX and LR2 (IOTA), and better than of the RMI3 and RMI4 models. The SUA performed by a GO was also of high clinical value.
In a systematic review, based on studies published before March 2008, Geomini et al. concluded that the RMI was the model of choice for preoperative assessments of adnexal masses . Later, Van Gorp et al. showed that the SUA was superior to the RMI . Another review and meta-analysis of studies published in 2008–2013, showed that an evidence-based approach for preoperatively characterizing any adnexal mass should incorporate the IOTA-LR2, the SR or SUA . Nunes et al. prospectively evaluated the IOTA-SR and performed a meta-analysis of studies (2008–2014) that utilized the model. They concluded that, in 76–89% of tumors, the SR protocol was accurate for diagnosing ovarian cancer. However, inconclusive cases required a “second- step” evaluation by an ultrasound expert . Similarly, in a systematic review and meta-analysis of studies published in 1990–2015, Meys et al. also recommended the SR for a first evaluation of adnexal masses, and an expert SUA or the LR2 model for inconclusive cases . In 2017, Westwood concluded that both the ADNEX and SR models offered higher sensitivity in assessing malignancy risk in adnexal masses than the RMI . In contrast to some of those studies, we noted that about half of malignant tumors in the testing (validation) group (24/52) were classified as inconclusive and another half as malignant according to SR. These results may be attributable to high proportion of patients without blood flow detected in Doppler mode – the positive B5 feature, which marked together with at least one M feature gave the inconclusive SR results. Detailed search of patients (in testing group) with malignancies and inconclusive SR results (n = 24) revealed that there were 11 patients with primary peritoneal cancer (with only minimal ovarian involvement), for whom a set of features: B5 (no blood flow) and M1 (irregular solid) or M2 (ascites) was the most frequent. Other patients with malignancies and inconclusive SR results had either none of SR features present (n = 3), or had some of M features and the positive B5. Thus it seems, that in our setting, the most important feature that classified malignant tumors as inconclusive SR was the B5 (color score 1) added to at least one M feature. The Doppler scanning was always performed according to standardized technique. Interestingly, the CS was a powerful, independent variable in our multivariate analysis in the learning group, and later, in the testing group; the CS was important as a part of the model, that performed well as compared to others. Performance of SR for benign tumors was good, with 83% of tumors being correctly classified as benign,
Many predictive models have considered solid areas important in predicting malignancy risk in adnexal masses [27,28,29, 47,48,49,50,51]. Both solid/papillary structures and vascularization are important predictors in the SR, SRrisk and LR models [30,31,32]. The Consensus Recommendation markedly emphasized the importance of detecting the presence or absence of any solid/papillary structure in the tumor .
Many predictive models consider vascularization, defined with the CS or as blood flow in a papillary projection, an important variable [29,30,31,32, 51]. Quantitative parameters were also considered important [28, 51, 52], but some considered the Doppler to be negligible . In our study, the CS was an independent predictive parameter. In contrast, RI, PI and PSV could be measured in only 40 and 13% of patients with malignant and benign tumors, respectively. Moreover, if measurable, there were no significant differences between both groups (Table 2). Consequently, these parameters were considered questionable in terms of clinical usefulness.
DD is a biomarker that globally indicates the activation of hemostasis and fibrinolysis. It is a degradation product of fibrin, which is produced when cross-linked fibrin is degraded by plasmin-induced fibrinolytic activity. As DD plasma levels are elevated after clot formation, the measurement of DD is routinely used in conjunction with clinical parameters in the initial assessment of suspected acute VTE . Elevated DD levels may also be observed in other clinical settings, such as cancer, pregnancy and infectious diseases or following trauma and surgery . Interestingly, a systemic activation of blood coagulation and procoagulant changes in the hemostatic system have frequently been observed in cancer patients, even in the absence of VTE. Moreover, coagulation activation, in particular thrombin generation and fibrin formation and dissolution, have been implicated in angiogenesis, tumor cell invasion, tumor progression, and metastatic spread. Tumor cells also possess strong procoagulant activities that induce local activation of the coagulation system and deposition of fibrin, which has an important role in the formation of tumor stroma and hematogenous spread of tumor cells . DD is routinely not considered a cancer marker, however giving the fact of our results and observations discussed above and described elsewhere , the DD might appear as a biomarker for cancer.
The mean DD level was 0.71 μg/ml in the general population of patients with cancer  and 4.1–5.4 μg/ml among patients with EOC; in EOC, 90% of patients had DD levels over the cut-off value [18, 56, 57]. When patients with EOC underwent DD level measurements and imaging to detect VTE, the incidences of 16–25% for deep venous thrombosis, 0–11% for pulmonary thromboembolism, and 93–100% for asymptomatic disease was shown [18, 56, 57]. We noted that the mean DD level was elevated among patients with ovarian cancer (even in stage I) compared to patients with benign lesions. None of our patients had symptomatic VTE.
Some studies showed that with a cut-off value of 0.5 μg/ml, the DD level had 77–92% sensitivity, 72–94% specificity, 59–91% PPV, and 82–95% NPV for discriminating between benign and malignant ovarian lesions [20, 22]. The DD level combined with CA-125 (with a cut-off 65 U/ml) provided a sensitivity, specificity, PPV, and NPV of 73, 100, 100, and 81%, respectively . The DD level seemed to outperform the CA-125 as a diagnostic biomarker, even for early stage EOC, where 73–83% of patients had elevated DD and 33–39% had elevated CA-125 [20, 22]. Interestingly, CA-125 in combinations with the red cell volume distribution width and the mean platelet volume (parameters usually measured as part of the whole blood cell count) may facilitate the early detection and differential diagnosis of ovarian cancer compared with benign ovarian tumors . Measurement of serum CA-125 is routinely used to aid diagnosis and in a follow-up of patients with EOC. However, its utility to detect early disease is questionable . Moreover, there is no overall survival advantage of early CA-125-directed retreatment for relapse . RMI cannot be calculated without CA-125 . The ADNEX model can be used with and without this variable . It was shown that serum CA-125 may not be needed in models with a binary outcome (benign vs malignant) . CA-125 is likely to be important for distinguishing between different types of malignant tumor . In the ADNEX study it was shown that serum CA-125 level was important for good discrimination between stage II-IV cancer and stage I and secondary metastatic cancer. Deriving a similar model without CA-125 level as a predictor mainly affected discrimination between stage II-IV cancer and other malignancies: validation AUCs decreased from 0.82 to 0.59 (stage II-IV cancer v metastatic cancer), from 0.87 to 0.76 (stage II-IV cancer v stage I cancer), and from 0.95 to 0.91 (stage II-IV cancer v borderline tumors) . Other models (e.g. SR, SRrisk, LR2) are based on ultrasound parameters only [30,31,32]. To our knowledge, this study was the first to show that, even in a multivariate analysis of various clinical and ultrasound variables, the DD level (not CA-125) was independently significant for differential diagnosing adnexal masses before surgery. Omission of CA-125 in our model with binary outcome (benign vs. malignant) was comparable to previous findings . Also, other authors concluded that CA-125 did not add any diagnostic information when an ovarian mass was examined using ultrasound techniques by an experienced examiner . Our model with DD level and without CA-125 had a comparable performance to ADNEX model calculated with CA-125.
This study had several strengths. It was performed prospectively, and it included consecutive, unselected patients diagnosed and treated in a regular center for gynecologic oncology. Additionally, the ultrasound was performed in a structured, planned manner, consistently for every patient. The index test was created with one set of patients (advanced statistical methods were used to select the most important variables), and it was validated with an independent set of patients, with a head-to-head comparison to results from other widely-used models, and to SUA. Other ultrasound-based models require many variables (6–12) to perform calculations. In contrast, our model was simpler, and it’s performance was comparable to other, more complex models. Moreover, we created a free-access, internet-based calculator to facilitate clinical use of our model.
This study also had some limitations. It was performed in a single institution, and the ultrasound was performed by a single examiner. Also, the patients had been referred to the gynecologic oncology department for surgery; thus, we did not include an observational arm. Our sample size was relatively small. IOTA models and the RMI were designed and validated on a larger number of patients and in many different clinical settings. Thus, our model lacked the generalizability of those models. However, our results could be useful for triaging patients at a referral center for gynecologic oncology. Another issue, is that there was no testing for VTE performed to compare it with DD levels. However, in order to test it, a comprehensive study should involve testing of every patient with Doppler ultrasound of lower extremities and pulmonary angio-computed tomography, and still would not cover testing for thrombosis in cancer tissue. A future study should involve a control group with adnexal mass and thrombosis, however this co-incidence might be incidental. There was no single patient who presented symptoms of VTE among all 290 included cases. One might note that an assessor of the reference test was not blinded to general clinical impression and CA-125 level, and consider this issue could lead to bias. However, this practice reflects clinical reality.
We developed a model with two simple ultrasound predictors (solid areas and vascularization) and the plasma DD level for discriminating malignant from benign adnexal masses. This simple tool might be useful in referral centers for gynecologic oncology. The performance of our model was comparable to other, highly effective, though more complex models. Moreover, our model could be used to complement a subjective assessment. The model needs substantial validation.
Assessment of different NEoplasias in the adnexa (IOTA model)
Akaike and Bayes information criteria
Area under the curve
Borderline malignant ovarian tumor
Cancer antigen 125
Epithelial ovarian cancer
International Federation of Gynecology and Obstetrics
International Ovarian Analysis Group
Logistic Regression 2 (IOTA model)
Negative predictive value
Positive predictive value
Risk of malignancy index (model modification 3)
Risk of malignancy index (model modification 4)
Simple Rules (IOTA model)
(SRR) Simple Rules risk calculation (IOTA model)
Subjective ultrasound assessment
Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of 'pattern recognition' and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol. 2001;18:357–65.
Van Gorp T, Veldman J, Van Calster B, Cadron I, Leunen K, Amant F, Timmerman D, Vergote I. Subjective assessment by ultrasound is superior to the risk of malignancy index (RMI) or the risk of ovarian malignancy algorithm (ROMA) in discriminating benign from malignant adnexal masses. Eur J Cancer. 2012;48:1649–56.
Meys EM, Kaijser J, Kruitwagen RF, Slangen BF, Van Calster B, Aertgeerts B, Verbakel JY, Timmerman D, Van Gorp T. Subjective assessment versus ultrasound models to diagnose ovarian cancer: a systematic review and meta-analysis. Eur J Cancer. 2016;58:17–29.
Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BW. The accuracy of risk scores in predicting ovarian malignancy: a systematic review. Obstet Gynecol. 2009;113:384–94.
Kaijser J, Sayasneh A, Van Hoorde K, Ghaem-Maghami S, Bourne T, Timmerman D, Van Calster B. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update. 2014;20:449–62.
Stukan M, Dudziak M, Ratajczak K, Grabowski JP. Usefulness of diagnostic indices comprising clinical, sonographic, and biomarker data for discriminating benign from malignant ovarian masses. J Ultrasound Med. 2015;34:207–17.
Westwood M. Tests in secondary care to identify people at high risk of ovarian cancer.2017 14/10/2017. Available from: https://www.nice.org.uk/guidance/dg31/documents/final-protocol.
Bristow RE, Chang J, Ziogas A, Campos B, Chavez LR, Anton-Culver H. Impact of National Cancer Institute Comprehensive Cancer centers on ovarian cancer treatment and survival. J Am Coll Surg. 2015;220:940–50.
Chang SJ, Bristow RE, Chi DS, Cliby WA. Role of aggressive surgical cytoreduction in advanced ovarian cancer. J Gynecol Oncol. 2015;26:336–42.
Wallace S, Kumar A, Mc Gree M, Weaver A, Mariani A, Langstraat C, Dowdy S, Bakkum-Gamez J, Cliby W. Efforts at maximal cytoreduction improve survival in ovarian cancer patients, even when complete gross resection is not feasible. Gynecol Oncol. 2017;145:21–6.
Glanc P, Benacerraf B, Bourne T, Brown D, Coleman BG, Crum C, Dodge J, Levine D, Pavlik E, Timmerman D, et al. First international consensus report on adnexal masses: management recommendations. J Ultrasound Med. 2017;36:849–63.
Testa AC, Ludovisi M, Mascilini F, Di Legge A, Malaggese M, Fagotti A, Fanfani F, Salerno MG, Ercoli A, Scambia G, et al. Ultrasound evaluation of intra-abdominal sites of disease to predict likelihood of suboptimal cytoreduction in advanced ovarian cancer: a prospective study. Ultrasound Obstet Gynecol. 2012;39:99–105.
Fischerova D, Cibula D. Ultrasound in gynecological cancer: is it time for re-evaluation of its uses? Curr Oncol Rep. 2015;17:28.
Weinberger V, Fischerova D, Semeradova I, Slama J, Dundr P, Dusek L, Cibula D, Zikan M. Prospective evaluation of ultrasound accuracy in the detection of pelvic Carcinomatosis in patients with ovarian Cancer. Ultrasound Med Biol. 2016;42:2196–202.
Fischerova D, Zikan M, Semeradova I, Slama J, Kocian R, Dundr P, Nemejcova K, Burgetova A, Dusek L, Cibula D. Ultrasound in preoperative assessment of pelvic and abdominal spread in patients with ovarian cancer: a prospective study. Ultrasound Obstet Gynecol. 2017;49:263–74.
Zikan M, Fischerova D, Semeradova I, Slama J, Dundr P, Weinberger V, Dusek L, Cibula D. Accuracy of ultrasound in prediction of rectosigmoid infiltration in epithelial ovarian cancer. Ultrasound Obstet Gynecol. 2017;50:533–8.
Lyman GH, Khorana AA. Cancer, clots and consensus: new understanding of an old problem. J Clin Oncol. 2009;27:4821–6.
Sakurai M, Satoh T, Matsumoto K, Michikami H, Nakamura Y, Nakao S, Ochi H, Onuki M, Minaguchi T, Yoshikawa H. High pretreatment plasma D-dimer levels are associated with poor prognosis in patients with ovarian cancer independently of venous thromboembolism and tumor extension. Int J Gynecol Cancer. 2015;25:593–8.
Man YN, Wang YN, Hao J, Liu X, Liu C, Zhu C, Wu XZ. Pretreatment plasma D-dimer, fibrinogen, and platelet levels significantly impact prognosis in patients with epithelial ovarian cancer independently of venous thromboembolism. Int J Gynecol Cancer. 2015;25:24–32.
Gadducci A, Baicchi U, Marrai R, Ferdeghini M, Bianchi R, Facchini V. Preoperative evaluation of D-dimer and CA 125 levels in differentiating benign from malignant ovarian masses. Gynecol Oncol. 1996;60:197–202.
Amirkhosravi A, Gt B, Desai H, Rivera-Amaya M, Coll E, Robles-Carrillo L, Faust P, Waters A, Meyer T, Reyes E, et al. Blood clotting activation analysis for preoperative differentiation of benign versus malignant ovarian masses. Blood Coagul Fibrinolysis. 2013;24:510–7.
Worasethsin P, Narkwichean A. D-dimer as a tumor marker in pre-operative assessment of adnexal masses. J Med Assoc Thail. 2013;96:1395–400.
National Comprehensive Cancer Network Clinical Practice Guidelines in Oncology. Ovarian Cancer Including Fallopian Tube Cancer and Primary Peritoneal Cancer. Version 2.2018.2018 Oct 07, 2018. Available from: www.nccn.org/professionals/physician_gls/pdf/ovarian.pdf.
Michielsen K, Dresen R, Vanslembrouck R, De Keyzer F, Amant F, Mussen E, Leunen K, Berteloot P, Moerman P, Vergote I, et al. Diagnostic value of whole body diffusion-weighted MRI compared to computed tomography for pre-operative assessment of patients suspected for ovarian cancer. Eur J Cancer. 2017;83:88–98.
Fischerova D. Ultrasound scanning of the pelvis and abdomen for staging of gynecological tumors: a review. Ultrasound Obstet Gynecol. 2011;38:246–66.
Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the international ovarian tumor analysis (IOTA) group. Ultrasond Obstet Gynecol. 2000;16:500–5.
Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol. 1990;97:922–9.
Alcazar JL, Errasti T, Laparte C, Jurado M, Lopez-Garcia G. Assessment of a new logistic model in the preoperative evaluation of adnexal masses. J Ultrasound Med. 2001;20:841–8.
Van Calster B, Van Hoorde K, Valentin L, Testa AC, Fischerova D, Van Holsbeke C, Savelli L, Franchi D, Epstein E, Kaijser J, et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ. 2014;349:g5920.
Timmerman D, Van Calster B, Testa AC, Guerriero S, Fischerova D, Lissoni AA, Van Holsbeke C, Fruscio R, Czekierdowski A, Jurkovic D, et al. Ovarian cancer prediction in adnexal masses using ultrasound-based logistic regression models: a temporal and external validation study by the IOTA group. Ultrasound Obstet Gynecol. 2010;36:226–34.
Timmerman D, Testa AC, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C, Paladini D, Van Calster B, Vergote I, Van Huffel S, et al. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet Gynecol. 2008;31:681–90.
Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W, Wynants L, Van Holsbeke C, Epstein E, Franchi D, et al. Predicting the risk of malignancy in adnexal masses based on the simple rules from the international ovarian tumor analysis group. Am J Obstet Gynecol. 2016;214:424–37.
Tingulstad S, Hagen B, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals. Obstet Gynecol. 1999;93:448–52.
Yamamoto Y, Yamada R, Oguri H, Maeda N, Fukaya T. Comparison of four malignancy risk indices in the preoperative evaluation of patients with pelvic masses. Eur J Obstet Gynecol Reprod Biol. 2009;144:163–7.
Tavassoli FA, Devilee P. Pathology and genetics of Tumours of the breast and female genital organs. Lyon: IARCPress; 2003. Available from: https://www.iarc.fr/wp-content/uploads/2018/07/BB4.pdf
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HC, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Radiology. 2015;277:826–32.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.
Team RC. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2017.
Steyerberg E. Clinical prediction models: a practical approach to development, validation, and updating. 1st ed. New York: Springer-Verlag; 2009.
Dziak JJ, Coffman DL, Lanza ST, Li R, Jermiin LS. Sensitivity and specificity of information criteria. Brief Bioinform. 2019. https://doi.org/10.1093/bib/bbz016. https://academic.oup.com/bib/advance-article-abstract/doi/10.1093/bib/bbz016/5380417?redirectedFrom=fulltext.
Xia Y, Sun J, Chen D-G. Statistical analysis of microbiome data with R. Singapore: Springer; 2018.
Collins GS, Altman DG. An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. BMJ. 2009;339:b2584.
Harrell Jr F. rms: Regression Modeling Strategies. R package version 5.1–2. Department of Biostatistics, Vanderbilt University2018.
Cleveland WS, Devlin SJ, Grosse E. Regression by local fitting: methods, properties, and computational algorithms. J Econ. 1988;37:87–114.
Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and cox regression. Am J Epidemiol. 2007;165:710–8.
Nunes N, Ambler G, Foo X, Naftalin J, Widschwendter M, Jurkovic D. Use of IOTA simple rules for diagnosis of ovarian cancer: meta-analysis. Ultrasound Obstet Gynecol. 2014;44:503–14.
Granberg S, Wikland M, Jansson I. Macroscopic characterization of ovarian tumors and the relation to the histological diagnosis: criteria to be used for ultrasound evaluation. Gynecol Oncol. 1989;35:139–44.
Sassone AM, Timor-Tritsch IE, Artner A, Westhoff C, Warren WB. Transvaginal sonographic characterization of ovarian disease: evaluation of a new scoring system to predict ovarian malignancy. Obstet Gynecol. 1991;78:70–6.
DePriest PD, Shenson D, Fried A, Hunter JE, Andrews SJ, Gallion HH, Pavlik EJ, Kryscio RJ, van Nagell JR Jr. A morphology index based on sonographic findings in ovarian cancer. Gynecol Oncol. 1993;51:7–11.
Lerner JP, Timor-Tritsch IE, Federman A, Abramovich G. Transvaginal ultrasonographic characterization of ovarian masses with an improved, weighted scoring system. Am J Obstet Gynecol. 1994;170:81–5.
Smolen A, Stachowicz N, Czekierowski A, Kotarski J. The estimation of the probability of tumor malignacy on the basis of test combination in the primary diagnosis of adnexal tumors. Ginekol Pol. 2010;81:254–61.
Berlanda N, Ferrari MM, Mezzopane R, Boero V, Grijuela B, Ferrazzi E, Pardi G. Impact of a multiparameter, ultrasound-based triage on surgical management of adnexal masses. Ultrasound Obstet Gynecol. 2002;20:181–5.
Ueland FR, DePriest PD, Pavlik EJ, Kryscio RJ, van Nagell JR Jr. Preoperative differentiation of malignant from benign ovarian tumors: the efficacy of morphology indexing and Doppler flow sonography. Gynecol Oncol. 2003;91:46–50.
Pabinger I, Ay C. Biomarkers and venous thromboembolism. Arterioscler Thromb Vasc Biol. 2009;29:332–6.
Ay C, Dunkler D, Pirker R, Thaler J, Quehenberger P, Wagner O, Zielinski C, Pabinger I. High D-dimer levels are associated with poor prognosis in cancer patients. Haematologica. 2012;97:1158–64.
Satoh T, Oki A, Uno K, Sakurai M, Ochi H, Okada S, Minami R, Matsumoto K, Tanaka YO, Tsunoda H, et al. High incidence of silent venous thromboembolism before treatment in ovarian cancer. Br J Cancer. 2007;97:1053–7.
Kawaguchi R, Furukawa N, Kobayashi H. Cut-off value of D-dimer for prediction of deep venous thrombosis before treatment in ovarian cancer. J Gynecol Oncol. 2012;23:98–102.
Qin YY, Wu YY, Xian XY, Qin JQ, Lai ZF, Liao L, Lin FQ. Single and combined use of red cell distribution width, mean platelet volume, and cancer antigen 125 for differential diagnosis of ovarian cancer and benign ovarian tumors. J Ovarian Res. 2018;11:10.
Ledermann JA, Raja FA, Fotopoulou C, Gonzalez-Martin A, Colombo N, Sessa C, Group EGW. Newly diagnosed and relapsed epithelial ovarian carcinoma: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2013;24(Suppl 6):vi24–32.
Rustin GJ, MEvd B. A randomized trial in ovarian cancer (OC) of early treatment of relapse based on CA125 level alone versus delayed treatment based on conventional clinical indicators (MRC OV05/EORTC 55955 trials). J Clin Oncol. 2009;27:1.
Timmerman D, Van Calster B, Jurkovic D, Valentin L, Testa AC, Bernard JP, Van Holsbeke C, Van Huffel S, Vergote I, Bourne T. Inclusion of CA-125 does not improve mathematical models developed to distinguish between benign and malignant adnexal tumors. J Clin Oncol. 2007;25:4194–200.
Van Calster B, Valentin L, Van Holsbeke C, Zhang J, Jurkovic D, Lissoni AA, Testa AC, Czekierdowski A, Fischerova D, Domali E, et al. A novel approach to predict the likelihood of specific ovarian tumor pathology based on serum CA-125: a multicenter observational study. Cancer Epidemiol Biomark Prev. 2011;20:2420–8.
Valentin L, Jurkovic D, Van Calster B, Testa A, Van Holsbeke C, Bourne T, Vergote I, Van Huffel S, Timmerman D. Adding a single CA 125 measurement to ultrasound imaging performed by an experienced examiner does not improve preoperative discrimination between benign and malignant adnexal masses. Ultrasound Obstet Gynecol. 2009;34:345–54.
We acknowledge Mirosław Dudziak for a clinical contribution, and Radosław Lenckowski and Urszula Śmiałek for providing detailed histological data. We thank to the Reviewers for their valuable review and comments, and the possibility to improve the manuscript.
There was no funding for conducting the research, preparing the manuscript, or the language editing.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Web calculator: http://gin-onc-calculators.com/gynonc.php.
Ethics approval and consent to participate
The study protocol was approved by the local Ethics Committee at Medical Council in Gdansk, Poland (KB – 36/17). Informed consent was obtained from all participants. It was a verbal consent, because all tests (imaging and laboratory) were performed as a routine clinical evaluations, according to institutional guidelines. There were no additional tests for the purpose of the study, performed on participants. All model’s calculations were performed after the study concluded; thus, the models played no role in the decision-making process. The above mentioned ethics committee approved this procedure.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Definitions and mathematical formulae for calculation of different predictive models. (DOC 38 kb)
TRIPOD checklist for the manuscript. (DOCX 90 kb)
Clinical and ultrasound parameters in the learning (N = 190) and testing groups (n = 100). (DOC 61 kb)
IOTA Simple Rules variables distribution in the testing group (N = 100) (malignant and benign groups compared with Fischer test). (DOCX 16 kb)
Calibration curves of our model validation (n = 100). The grey line represents the perfect model used for comparison; black line (—) represents calibration plot; dotted line (∙∙∙∙) represents smooth fit of calibration plot using lowest method . Predicted vs. observed risk 1.123. (PNG 7 kb)
About this article
Cite this article
Stukan, M., Badocha, M. & Ratajczak, K. Development and validation of a model that includes two ultrasound parameters and the plasma D-dimer level for predicting malignancy in adnexal masses: an observational study. BMC Cancer 19, 564 (2019). https://doi.org/10.1186/s12885-019-5629-x