Meta-analysis of quantitative diffusion-weighted MR imaging in the differential diagnosis of breast lesions

Breast cancer is the most common cancer in women worldwide [1]. Despite the improvement in the detection of breast cancer with the widespread application of mammography and ultrasound, differentiation between benign and malignant breast lesions remains a difficult diagnosis problem, especially in dense fibroglandular breasts. Numerous studies have been performed to assess the diagnostic performance of contrast-enhanced magnetic resonance imaging (MRI) in breast lesions [2–5]. Based on the enhancement pattern of lesions and morphologic changes, breast MRI offers an overall sensitivity of 90% and specificity of 72% in detecting breast lesions according to a published Meta-analysis [6]. Therefore, the classification of a lesion detected on MRI as benign or malignant remains a challenge [6, 7].

One of the latest advancements in MRI technology is the application of diffusion-weighted imaging (DWI) to offer quantitative evaluation of apparent diffusion coefficient (ADC). Quantitative DWI is a powerful imaging tool which provides unique information related to the diffusion of water molecules in the tissue and allows estimation of cellularity and tissue structure [8]. Restricted water movement in tumors with high cellularity usually leads to higher signal intensity and smaller ADC value [7, 9]. ADC is measured by acquiring the MR signal at least twice, typically with and without diffusion weighting by the following formula: ADC = [In (S₀/S_b)]/b, where S_b and S₀ are the signal intensities on the DW imaging with and without diffusion weighting, respectively [10]. Theoretically, the diffusion sensitivity is easily varied by changing the diffusion gradient factor known as the "b value" which is proportional to the gradient amplitude, the duration of the applied gradient and the time interval between the paired gradients [8].

The potential role of quantitative DWI in characterizing breast lesions has been reported [11–18]. However, the reported sensitivity and specificity of diagnosing malignant breast tumors ranged from 62.5% to 92.8% and 45.8% to 96.7%, respectively. Comparison of the diagnostic performance of breast quantitative DWI among the studies may have been compromised by differences in the patient characteristics, MR imaging techniques, and diagnostic criteria for malignancy in the studies. In this study, we pool a number of DWI studies of the breast to evaluate the diagnostic performance of DWI in breast lesion characterization.

With the computer search and manual cross-checking of reference lists, 65 abstracts (26 English and 39 Chinese) were retrieved. After reading the titles and abstracts, we found 25 eligible articles (15 English and 10 Chinese). After reading the full texts, we excluded 12 of the 25 relevant articles for the following reasons: The objective of studies was not to explore the diagnostic performance of 1.5 T MR DWI on breast lesions [31–34] (n = 4); Benign breast lesions in the study were all fibroadenomas [35] (n = 1); Researchers did not report data that could be used to construct or calculate TP, FP, TN and FN results [15, 17, 36] (n = 3); Researchers presented results from a combination of different MR series that could not be differentiated for assessment of single DWI [37] (n = 1); Data or subsets of data were presented in other articles [38, 39] (n = 2). Thirteen articles (5 English and 8 Chinese) fulfilled all inclusion and exclusion criteria and were selected for data extraction and data analysis [11, 13, 14, 16, 18, 40–47].

The forest plots of sensitivities and specificities from 23 subsets of data from all 13 studies are shown in Figure 1. A homogeneity test of sensitivity and specificity shows Q = 58.20 (P < 0.0001), I ² = 62.2% and Q = 61.44 (P < 0.0001), I ² = 64.2%, respectively. Therefore, notable heterogeneities are detected. The next step is the representation of sensitivity against 1-specificity from each study in a ROC space to explore the threshold effect. The pattern of the points in this plot is not a "shoulder-arm" shape (Figure 2). A Spearman rank correlation is performed as a further test for threshold effect. The Spearman correlation coefficient was equal to 0.097 (P = 0.66) and indicates that there should be other factors than threshold effect resulting in variations in accuracy estimates among individual studies.

The funnel plot in Figure 3 shows that the studies are distributed symmetrically about the combined effect size and yield a z-value of 4.46191 with corresponding P-value of 0.00001. The NFS is 92. This means that we would need to locate and include 92 "null" studies in order for the P to exceed 0.05. There is no significant publication bias, and actual combined effect size is equal to the theoretical combined effect size.

Having found notable heterogeneity beyond threshold effect and publication bias, we then focus on the subgroup of studies using maximum b = 1000 s/mm². A homogeneity test of sensitivity and specificity shows Q = 16.39 (P = 0.1271), I ² = 32.9% and Q = 15.81 (P = 0.1483), I ² = 30.4% respectively. The pooled weighted sensitivity and specificity with corresponding 95% CIs are 0.84 (0.80, 0.87) and 0.84 (0.79, 0.88) (Figure 4) respectively, and expressed as AUC of sROC curve is 0.9085 (Figure 5).

In this meta-analysis, we calculate an overall sensitivity of 0.84 (95% CI: 0.82, 0.87) and specificity of 0.79 (95% CI: 0.75, 0.82) from 13 studies fulfilling all inclusion and exclusion criteria. However, there is notable heterogeneity among individual studies. Therefore, it is critical to investigate the source of heterogeneity to determine the potential impact factors and to evaluate the appropriateness of statistical pooling of accuracy estimates from various studies.

Meta-Disc is performed to assess threshold effect from representation of accuracy estimates from each study in a ROC space, and Spearman correlation coefficient is computed between the log (SEN) and log (1-SPE). Lack of "shoulder-arm" shape of the points in the ROC space and mild Spearman correlation coefficient (0.097) indicate that there should be factors other than differences in cutoff points for malignancy causing variations in accuracy estimates across individual studies. Additionally, publication bias is an usual source of heterogeneity in meta-analysis. Publication biases might exaggerate clinical effects resulting in potentially erroneous clinical decision making. In systematic reviews or meta-analysis, a thorough literature search is crucial to identify all relevant studies. In this meta-analysis, the search of several electronic databases is supplemented by checking references of relevant studies in order to reduce publication bias [25]. Although only English and Chinese articles are searched, the funnel plot and NFS analysis indicate that there is no significant publication bias in our meta-analysis.

The parameters used in DWI sequences may affect the results of ADC calculation, and the diffusion gradient factor b is one of important parameters [8]. The variability in b values across studies makes the ranges and thresholds of ADC values difficult to interpret. Because b values used in the included studies varied, we needed to explore whether b values were the source of heterogeneity. Meta-analysis is performed again in the subgroup of studies using maximum b = 1000 s/mm² because more studies used b at 1000 s/mm² than any other value from Table 2. The results show that there is no significant heterogeneity, and the pooled sensitivity and specificity with 95% CIs are 0.84 (0.80, 0.87) and 0.84 (0.79, 0.88) respectively. The area under the curve of sROC is 0.9085. Compared with the original results from all 13 eligible studies, the specificity is improved. However, we could not determine effect of other b values to pooled accuracy estimates because of the relatively small number of included studies. Furthermore, sensitivity analysis is performed in the subgroup of b = 1000 s/mm². When the studies with the maximum variance of sensitivity and specificity are excluded in sequence, the characteristics of heterogeneity are not changed and the variances of the pooled accuracy estimates are not significant. The sensitivity analysis shows that this meta-analysis in the subgroup is stable.

A previously published meta-analysis for contrast-enhanced MRI including 44 studies reported the overall sensitivity of 0.90 (95% CI: 0.88, 0.92) and specificity of 0.72 (95% CI: 0.67, 0.77) in patients with breast lesions [6]. In comparison, our meta-analysis indicates that DWI has higher specificity but lower sensitivity. However, several differences and limitations in our meta-analysis should be noted. The significance of Peter et al. [6] lies in the diagnostic performance of MR imaging of small lesions detected at mammographic screening. Because a large proportion of these lesions are nonpalpable, they only include studies that enrolled at least one patient with nonpalpable lesion [6]. In our meta-analysis, we do not filter the studies using such criteria. In the included studies, Marini et al. selected lesions with diameter >1 cm [18]. Rubesova et al. selected lesions with diameter > 0.7 cm [14], and Tang et al. selected lesions with diameter ≤ 2 cm [43]. Lesion diameter ranged from 0.3 to 9 cm for other included studies. It is not certain that smaller lesions could be more easily missed in DW images since no included studies reported the number of nonpalpable lesions. Such variation might affect the overall estimates of the sensitivity and specificity. Because MR is one of the important methods for breast cancer diagnosing, DWI, having the advantages of a short examination time and no need to inject a contrast medium could be used in screening crowd to early detect nonpalpable lesions and to generate significant benefits to early tumor diagnosis and human health care needs in the future.

One limitation of our study is the suboptimal quality of included studies. Meta-analysis combines or integrates the results of several independent studies. The quality and reliability of a meta-analysis depends on the quality of included studies. We use the QUADAS tool for assessing methodological quality of individual studies. This tool was specifically developed for quality assessment of diagnostic accuracy studies included in systematic reviews and has been used to help identify severe methodological shortcomings [21]. Most included studies in this meta-analysis had a suboptimal design in regard to the reporting of selection criteria, the description of the execution of the reference standard, the interpretation of the reference standard results without knowledge of the index test results, the interpretation of the index test results without knowledge of the reference standard, reporting of uninterpretable and/or intermediate test results, or explanation of withdrawals from the study (Table 3).

In test accuracy studies, interpretation of the results of the index test may be influenced by knowledge of the results of the reference standard, and vice versa. This is known as review bias and may lead to inflated measures of diagnostic accuracy. As the index test, DWI was always performed first, and interpretation of the results of the DWI was usually done without knowledge of the results of the reference standard. However, if test result was interpreted at a later date, after both DWI and reference standard had been completed, then it was still important for a study to provide a description of whether the interpretation of each test was performed blind to the results of the other tests. Uninterpretable results produced in test accuracy studies are often not reported. Instead, they are simply removed from the analysis. This problem may lead to the biased assessment of the test characteristics. Whether bias will arise depends on the possible correlation between uninterpretable test results and the true disease status. If uninterpretable results occur randomly and are not related to the true disease status of the individual, these should not have any effect on test performance. However, whatever the cause of uninterpretable results, it is important that they are reported so that the impact of these results on test performance can be determined. Patients' withdrawal from the study can occur prior to the results of either or both of the index test and reference standard in diagnostic studies being available. If patients lost to follow-up differ systematically from those who remain in studies, then estimates of test performance may be biased. Therefore, it is important to report patients lost to follow-up. In brief, most studies in evaluating the diagnostic values of DWI for breast lesions had some clinical operational challenges. Therefore a systematic data reporting method such as the standards for reporting diagnostic accuracy (STARD) should be advocated to improve the quality of reporting test accuracy studies [48].

All of individuals showed that the mean ADC value of malignant lesions was lower than that of benign lesions. However, the reported mean ADC values of malignant and benign tumors ranged from 0.87 to 1.36 and 1.00 to 1.82 × 10^-3 mm²/s, respectively, resulting in recommended threshold values of ADC ranging from 0.90 to 1.76 × 10^-3 mm²/s. Even in the subgroup of studies using maximum b = 1000 s/mm², the minimum and maximum threshold values were 1.10 × 10^-3 mm²/s [14, 18] and 1.38 × 10^-3 mm²/s [43], respectively. The substantial variance of threshold values might be influenced by different b values, selection method, bias of patient selection, pathological characteristic of lesions and measurement of ADC values. This meta-analysis does not seem to predict or determine the unified threshold value to differentiate malignant and benign breast lesions because selection of the threshold value should be determined according to the purpose of examination. For example, a relatively higher threshold value may be recommended to minimize missing malignancy in breast cancer screening. If DWI is appended to the contrast-enhanced MRI, a relatively lower threshold value may be recommended to reduce false positive results.

ADC measurement of DWI is useful for differentiation between malignant and benign breast lesions with pooled sensitivity of 0.84 and specificity of 0.84 in one homogenous subgroup of studies using maximum b = 1000 s/mm², and the area under curve of sROC was 0.9085. DWI has a higher specificity but lower sensitivity compared to that of contrast-enhanced MRI. However, with all of methodological issues considered, results must be interpreted with caution. Large scale randomized control trials (RCTs) are necessary to assess and confirm its clinical value. A threshold value for malignant/benign lesions classification could not be made based on this study because it is influenced by different b values, bias of patient selection, lesions' pathological characteristics and ADC measurement. Selection of the threshold value should be determined according to the purpose of examination.