Method validation of circulating tumour cell enumeration at low cell counts
- Jeffrey Cummings†1Email author,
- Karen Morris†1,
- Cong Zhou†1,
- Robert Sloane1,
- Matt Lancashire1,
- Daniel Morris1,
- Stephen Bramley1,
- Matt Krebs1,
- Leila Khoja1 and
- Caroline Dive1
© Cummings et al.; licensee BioMed Central Ltd. 2013
Received: 31 May 2013
Accepted: 2 September 2013
Published: 11 September 2013
Circulating tumour cells (CTC) are receiving increasing attention as prognostic, predictive and pharmacodynamic biomarkers in cancer patients. However, their clinical significance can be dependent on an accurate determination of CTC around cut-off values at low cell counts (<10 cells/7.5 ml). Consequently, we have conducted method validation of the CellSearch™ system focusing on clinical samples containing CTC in the cut-off region.
Analytical accuracy was first assessed employing quality controls (QC) and spiked healthy volunteer blood specimens. Results were analysed by β-expectation tolerance intervals (BETI). Inter-operator error (6 different readers) was then characterised in 38 different patient samples, 68% of which had ≤5 CTC and data were analysed by β-content γ-confidence tolerance intervals (BCTI).
Results from QCs and spiked blood confirmed a 3-4-fold higher degree of imprecision at the low (48 cells, BETI = + 0.288/-0.345, β = 95%) compared to the high QC (987 cells, BETI = +0.065/-0.140, β = 95%). However, when data for individual analysts were interrogated characteristic systematic errors were detected. In the analysis of patient samples again individual analysts introduced a highly specific error into the interpretation of CTC images, which correlated to the level of training and experience. When readers were selected based on BETI and BCTI results, the high level of between-operator error (up to 170%) observed at CTC of ≤ 5 was reduced to < 30%.
Inter-operator variability in enumeration of CTC at low cell counts can be considerable, but is also potentially avoidable by following simple guidance steps.
KeywordsCirculating tumour cells Predictive biomarker Enumeration Low cell counts Method validation Statistical analysis
Detection, enumeration and characterisation of circulating tumour cells (CTC) as a potential biomarker currently represents one of the most actively pursued areas in translational cancer research . CTC are believed to act as the ‘seeds’ for the establishment of metastatic disease, and also a mechanism to re-populate the primary tumour, and their presence has been shown to correlate to both progression free survival and overall survival [2–5]. In these studies, a discrete cut-off point was discriminated at extremely low numbers of CTC above which poorer prognosis was observed: ≥ 5 in 7.5 ml blood for metastatic breast cancer, metastatic castration resistant prostate cancer and non-small cell lung cancer; ≥ 3 for metastatic colorectal cancer; ≥ 2 in melanoma and ≥ 1 in neuroendocrine tumours [2–8]. In addition, CTC have been shown to be predictive of response to both chemotherapy and targeted agents in post-treatment samples and to act as a pharmacodynamic biomarker [2, 9–12].
Isolation of rare cells (1 CTC in 108 leucocytes) presents considerable technological challenges requiring a robust analytical technique and a number of different approaches have been developed based on the physical and biological properties of CTC [1, 13]. To date, the CellSearch™ system (Veridex, Raritan, NJ, USA) remains the only platform that is cleared by a regulatory authority (the FDA in the USA), as an aid in the monitoring of patients with metastatic breast, colorectal and prostate cancer http://www.accessdata.fda.gov/cdrh_docs/pdf7/K073338.pdf. The system consists of two major instrumental components: the AutoPrep station for the fully-automated immunomagnetic isolation of cells from whole blood and an Analyser for the semi-automated identification of CTC based on 4-channel fluorescence microscopy . Although, the subject of method validation studies in the past, these have tended to focus on either precision of quality control samples or between laboratory concordance (reproducibility) [15, 16]. Only limited studies have been performed evaluating the analytical accuracy of the technique - determination of the true value for CTC in the patient sample . Nonetheless, CellSearch remains the benchmark against which all new technologies should be assessed .
Assays employed as a prognostic or predictive biomarker require a credible level of analytical validation [19, 20] and in the case of CTC that should include a demonstration that the technique is accurate and reproducible at the cut-off level [18, 21]. Therefore, in the present study method validation of the CellSearch system was conducted focusing on the analysis of patient samples containing low cell counts, in the region of the published clinically relevant cut-off points. To address the issue of analytical accuracy statistical approaches to the interpretation of data including β-expectation tolerance intervals (BETI) were employed . In addition, a major goal was to achieve a reduction in inter-operator variability and this aspect utilised a modification of incurred sample reproducibility (ISR) and β-content γ-confidence tolerance intervals (BCTI) .
Patients and blood sample collection
Blood samples (7.5 ml) for CTC enumeration were collected from a total of 38 different lung, prostate, melanoma and colorectal patients receiving standard of care chemotherapy at the Christie Hospital, Manchester and entered into a number of experimental medicine studies being conducted at the Paterson Institute for Cancer Research. Written informed consent was obtained from all subjects and the studies were ethically approved by the Tameside and Glossop Research Ethics Committee (Manchester, UK) and the Declaration of Helsinki Principles was followed. Samples were harvested into CellSave tubes (Veridex, Raritan, NJ, USA), containing EDTA and a cellular preservative and maintained at room temperature for no longer than 72 hours prior to analysis. Blood was also collected from healthy volunteers for recovery experiments according to a local ethics committee approved protocol.
CTC enumeration by CellSearch
CTC were essentially enumerated as previously described in detail [7, 24, 25]. In brief, blood was diluted, centrifuged and incubated with ferrofluid particles coated with anti-EpCAM antibodies utilising the CellTracks AutoPrep station (Veridex). After immunomagnetic enrichment, ferrofluid-captured cells were permeabilised and fluorescently labelled using phycoerythrin-conjugated anti-cytokeratin antibodies (pan-keratin antibody C-11) to identify epithelial cells and allophycocyanin conjugated anti-CD45 antibody to identify and discount leucocytes. 4-6-Diamidino-2-phenylindole (DAPI) was incorporated to identify cell nuclei. Upon repeated magnetic separation, the fluorescently labelled cells were oriented to the surface of the (MagNest™) cartridge for interrogation using the CellTracks Analyser II (Veridex). Image frames covering the entire surface of the cartridge were captured by the software and a gallery of objects meeting pre-determined criteria presented to the analyst to confirm or otherwise the presence of CTC. Image galleries were assessed by the operator without prior knowledge of patient data. A CTC is defined as a nucleated cell staining positively for cytokeratin and negatively for CD45 and results are reported as CTC number per 7.5 ml of blood.
To establish the level of analytical accuracy achievable by the CellSearch system, the quality control (QC) reagents provided by Veridex were utilised. These are certified to contain a specified range of human tumour cells (SK-BR3 cells), at a high and low cell count, and are integral to the quality control procedures of the system. The statistical evaluation of analytical accuracy employed β-expectation tolerance intervals (see below). QC data obtained over a 3 month period during the analysis of 27 different batches of patient samples were collated for statistical evaluation. Variables investigated included the influence of two different CellSearch systems and 3 different operators.
In a second statistical evaluation of analytical accuracy by BETI, healthy donor blood was spiked with a known number of human tumour cells according to the following protocol. Approximately 30 ml of normal donor blood was collected into a CellSave tube. Growing cultures of either SW620 or H1048 cells were trypsinised and re-suspended in 1 ml phosphate buffered saline (PBS) and counted. Cells were re-suspended to a final concentration of either 3 or 25 in 100 μl (i.e. 30 or 250/ml) in PBS. 100 μl of cell suspension was then added to 3 empty CTC isolation tubes followed by 7.5 ml of normal donor blood. A control sample of 7.5 ml of normal donor blood with no spiked cells was also included. The number of cells spiked to each tube was unknown to the 6 different operators who then enumerated the cell numbers by CellSearch.
The final experiment involved the interrogation by 6 different operators of the same image galleries obtained from the analysis of 38 different patient samples. Analysts were selected on the basis of varying levels of training and experience. In the statistical evaluation of the resultant data a two-sided β-content γ-confidence tolerance interval was employed (see below).
Mathematical calculations and statistical analysis
Calculation of BETI was preformed utilising MATLAB (Version R2009a, MathWorks, Natick, MA, United States) as described previously . Tolerance intervals were calculated at β = 67%, 80% and 95%. A plot of BETI (y-axis) against the nominal concentration of the QCs or spike (x-axis) is referred to as the ‘accuracy profile’ and is used extensively throughout this report.
Z (1 + β)/2 is the upper (1 + β)/2 quantile of the standard distribution and is the lower γ quantile of the chi-squared distribution (N-1 degrees of freedom). Calculation of BCTI was performed utilising MATLAB (as above) at β = 67% and 95% . A plot of BCTI (y-axis) against the operator pair (x-axis) represents a modified form of the ‘accuracy profile’. All code developed in MATLAB was validated against previously published data sets as reported previously .
Analytical accuracy of the CellSearch system characterised by β-expectation tolerance intervals
Nonetheless, when the equivalent data were plotted for each operator involved in the analysis (see Figure 1B-D), striking differences in the resultant accuracy profiles emerged (P < 0.05, ANOVA). Analyst 1 introduced a positive bias in the determination of the low QC which was significantly different from the performance of the other two analysts (Newman-Keuls multiple comparison test) while Analyst 2 introduced a negative bias in the low QC. A large degree of imprecision (random error) was evident in the low QC data attributed to Analyst 3 coupled to a small negative bias in the high QC. Identification of such discrete analytical errors allows for the possibility of their correction, demonstrating the potential power of the BETI approach to method validation. No significant differences in QC values were recorded when the results obtained from the two separate CellSearch systems were compared (Student’s t test).
Six different operators independently interrogated the image galleries produced by the analysis of volunteer blood samples spiked with low numbers of tumour cells (3 and 25, n = 3). Here, BETI for the samples spiked with 25 cells was +0.562 and −0.546 at β = 95%, total error was 24.5% and average recovery 101% ± 24% coefficient of variation (CV), consistent with a large degree of random error but absence of systematic error and in keeping with previous studies . Due to the relatively small number of specimens in this study, it was not possible to discriminate the individual contribution of each analyst towards the overall level of error. BETI for the samples spiked with 3 cells was +0.486 and −0.264 at β = 95%, total error was 28.3% and average recovery was 90% ± 9.6%.
Incurred sample reproducibility of the CellSearch system characterised by β-content γ-confidence tolerance intervals
When Analyst 2 was compared against the other operators, there was a considerable increase in the tolerance intervals recorded (approximately 2-fold) and the introduction of a strong positive bias (Figure 2B). Likewise, in the case of Analyst 3, who was intermediate in experience, there was also a large increase in the level of random error but without any notable bias (Figure 2C). Analyst 6, the most in-experienced operator, was consistently associated with a much greater level of error than any other analyst (see Figures 2A-D). Apart from Analyst 1, the benchmark operator, the two other experienced analysts - 4 and 5 - appeared to be able to function within or close to the 30% margin of error recommended as acceptable for biomarker assays  (Figure 2D).
Amelioration of the inter-operator error associated with the enumeration of low CTC utilising BETI and BCTI
The aim of the present study was to improve the accuracy of CTC enumeration by the CellSearch system at low cell counts (<5) utilising the statistical produces of BETI and BCTI. In a seminal paper published by scientists based at Veridex (then called Immunicon Corporation) a statistical model was developed to describe the main sources of error associated with the determination of CTC in human blood . Three major error components were identified: a) sample collection, b) recovery of CTC through immunomagnetic depletion and c) inter-reader variability in the assignment of objects. A stimulus to conduct this study was based on the apparent arbitrary nature of the cut-off point of 5 CTC observed in metastatic breast cancer patients [2, 24]. No clear biologic basis could be proffered for such a discrete value [3, 8, 24, 28].
To explain the uncertainties associated with sample collection in CTC enumeration Poisson statistics were incorporated into the error model . Poisson statistics are believed to describe most accurately the effect of counting randomly distributed objects, such as CTC, in a discrete volume . The effect of Poisson statistics on CTC enumeration is illustrated as follows. Where the true number of CTC is 5, the probability of detecting 5 cells in a single sample collected from a patient is relatively small (17.5%). A feature of the Poisson distribution is that the variance is equal to the population mean. Thus, the theoretical CV for a set of measurements carried out on the same samples, based entirely on statistical probabilities, is 44.7% at 5 CTC. A number of previous validation studies, including the present work, have confirmed good agreement between the level of experimental error measured at low CTC and the theoretical level of error dictated by Poisson statistics [16, 27].
While it is difficult to control for the uncertainties introduced in the analysis of CTC by Poisson statistics, other than collecting a larger volume or many replicates of the same sample [24, 28, 29], the other two components in the error model are more amenable to correction. In the case of sample recovery, this component has been demonstrated to exert only a modest effect [16, 27], which was confirmed in the present study using spiked healthy volunteer blood samples.
Where there is scope for improvement is in the area of inter-reader variability, which has been proposed as one of the main reasons to explain the arbitrary nature of the cut-off levels observed in clinical trials [3, 24, 28]. Indeed, it is possible that due to experimental error the true cut-off levels may be even lower than those previously reported . Inter-operator error was also identified as the major contributor to between-laboratory variations observed during an external quality assurance assessment programme .
To apply BETI to method validation requires that the true or a certified value of the analyte of interest is known . This limited our evaluation to data derived from the QC samples provided by the vendor (Veridex). BETI is normally associated with bioanalytical techniques, although there are limited examples of its application in cut-off interpretation of ELISA data . To the best of our knowledge, the present report represents the first to apply this approach to method validation of the CellSearch system for CTC analysis. The strength of this procedure is that it informs on analytical accuracy and the quality of result obtained in future measurements, at any operator defined level of probability [30, 32]. Results obtained with the QC samples made two significant observations. First, it confirmed that a high level of analytical accuracy was possible, with virtually no bias observed at the high QC coupled to a low level of imprecision. Here the total error was always less than 15%, well within the recommended level for a biomarker technique [33, 34]. Second, and most importantly, it identified that at lower CTC, characteristic systematic and random errors could be ascribed to individual analysts.
However, it is also well recognised that QC samples often reflect poorly the analytical behaviour of clinical specimens, especially in the biomarker field [35, 36]. In the case of the CellSearch system, the QCs comprised breast cancer cells reconstituted into a non-biologic matrix. Therefore, the main focus of the present paper revolved around the analysis of cancer patient specimens containing low numbers of CTC (<5). To identify inter-reader errors the ISR methodology described by Hoffman was adopted utilising BCTI . In our modification of this process we substituted a different operator to analyse the repeat sample. To control against sampling artefacts, a relatively large number of patient specimens were analysed in a number of different assays . Results obtained clearly showed that an individual analyst could introduce a highly specific error into the interpretation of CTC images, analogous to the QC data. However, it was also demonstrated that with training and experience these errors could be significantly reduced.
Increasingly, the regulators in the USA and Europe are placing more stringent requirements on the validation of biomarker assays [36–38]. Hence, validation data will be adjudged in the future, not merely on the basis of technical performance characteristics with QCs , but in terms of the quality and significance of the data generated during the analysis of clinical specimens. As the gold standard, the most reliable biomarker data will be derived from multi-centre trials, where the analysis is subject to external proficiency testing schemes and inter-laboratory comparison programmes . This is especially true when data generated by the assay is intended to be used in the stratification of patients into different treatment arms, thus defining the biomarker as integral to the progress of the trial [19, 20]. Here, the assay will have to be demonstrated to possess a proven ability to discriminate between different cohorts with a high degree of diagnostic sensitivity and specificity [37, 38]. The first clinical study to conduct patient stratification based on a CTC cut-off value is SWOG S0500 (NCT00382018) in women with metastatic breast cancer receiving chemotherapy [18, 40]. Other stratification trials employing CTC as a biomarker are likely to follow, such as the ‘CriTiCal Trial’ (Circulating Tumour Cell guided Chemotherapy Trial in Colorectal Cancer) planned in the UK.
It has been shown that while inter-operator variability in enumeration of CTC at low cell counts can be considerable; it can be ameliorated by following simple guidance steps. First, and perhaps logically, operators should be trained to a high degree and experienced in the field. Second, utilising statistical based techniques, potential analysts should confirm in a training set of images derived from 30–50 patient samples run in 5–10 different assays, that consistency can be achieved against benchmark analysts. Finally, CTC enumeration in patient samples should be conducted on 2 or preferably 3 separate collections of blood, with a different operator analysing each, for two reasons. First by collecting up to 3 samples, one may attenuate the unavoidable uncertainties imposed on CTC enumeration by Poisson statistics, increasing the probability of detecting ≥ 1 (when present) up to 95% . Second, by employing different analysts one can confirm or otherwise the lack of inter-operator variability.
Circulating tumour cells
β-expectation tolerance intervals
β-content γ-confidence tolerance intervals
Incurred sample reproducibility
Phosphate buffered saline
Coefficient of variation.
The present study was supported with funding from the following: Cancer Research UK (C147/A12328), the Experimental Cancer Medicine Centre Network (ECMC) and AstraZeneca. Jessica Booth is thanked for her technical assistance.
- Alix-Panabieres C, Pantel K: Circulating tumor cells: liquid biopsy of cancer. Clin Chem. 2013, 59 (1): 110-118. 10.1373/clinchem.2012.194258.View ArticlePubMedGoogle Scholar
- Cristofanilli M, Budd GT, Ellis MJ, Stopeck A, Matera J, Miller MC, Reuben JM, Doyle GV, Allard WJ, Terstappen LW, et al: Circulating tumor cells, disease progression, and survival in metastatic breast cancer. N Engl J Med. 2004, 351 (8): 781-791. 10.1056/NEJMoa040766.View ArticlePubMedGoogle Scholar
- Cohen SJ, Punt CJ, Iannotti N, Saidman BH, Sabbath KD, Gabrail NY, Picus J, Morse M, Mitchell E, Miller MC, et al: Relationship of circulating tumor cells to tumor response, progression-free survival, and overall survival in patients with metastatic colorectal cancer. J Clin Oncol. 2008, 26 (19): 3213-3221. 10.1200/JCO.2007.15.8923.View ArticlePubMedGoogle Scholar
- Danila DC, Heller G, Gignac GA, Gonzalez-Espinoza R, Anand A, Tanaka E, Lilja H, Schwartz L, Larson S, Fleisher M, et al: Circulating tumor cell number and prognosis in progressive castration-resistant prostate cancer. Clin Cancer Res. 2007, 13 (23): 7053-7058. 10.1158/1078-0432.CCR-07-1506.View ArticlePubMedGoogle Scholar
- Kim MY, Oskarsson T, Acharyya S, Nguyen DX, Zhang XH, Norton L, Massague J: Tumor self-seeding by circulating cancer cells. Cell. 2009, 139 (7): 1315-1326. 10.1016/j.cell.2009.11.025.View ArticlePubMedPubMed CentralGoogle Scholar
- Krebs MG, Sloane R, Priest L, Lancashire L, Hou JM, Greystoke A, Ward TH, Ferraldeschi R, Hughes A, Clack G, et al: Evaluation and prognostic significance of circulating tumor cells in patients with non-small-cell lung cancer. J Clin Oncol. 2011, 29 (12): 1556-1563. 10.1200/JCO.2010.28.7045.View ArticlePubMedGoogle Scholar
- Khoja L, Lorigan P, Zhou C, Lancashire M, Booth J, Cummings J, Califano R, Clack G, Hughes A, Dive C: Biomarker utility of circulating tumor cells in metastatic cutaneous melanoma. J Invest Dermatol. 2013, 133 (6): 1582-1590. 10.1038/jid.2012.468.View ArticlePubMedGoogle Scholar
- Khan MS, Kirkwood A, Tsigani T, Garcia-Hernandez J, Hartley JA, Caplin ME, Meyer T: Circulating tumor cells as prognostic markers in neuroendocrine tumors. J Clin Oncol. 2013, 31 (3): 365-372. 10.1200/JCO.2012.44.2905.View ArticlePubMedGoogle Scholar
- Tol J, Koopman M, Miller MC, Tibbe A, Cats A, Creemers GJ, Vos AH, Nagtegaal ID, Terstappen LW, Punt CJ: Circulating tumour cells early predict progression-free and overall survival in advanced colorectal cancer patients treated with chemotherapy and targeted agents. Ann Oncol. 2010, 21 (5): 1006-1012. 10.1093/annonc/mdp463.View ArticlePubMedGoogle Scholar
- De-Bono JS, Scher HI, Montgomery RB, Parker C, Miller MC, Tissing H, Doyle GV, Terstappen LW, Pienta KJ, Raghavan D: Circulating tumor cells predict survival benefit from treatment in metastatic castration-resistant prostate cancer. Clin Cancer Res. 2008, 14 (19): 6302-6309. 10.1158/1078-0432.CCR-08-0872.View ArticlePubMedGoogle Scholar
- Scher HI, Jia X, De-Bono JS, Fleisher M, Pienta KJ, Raghavan D, Heller G: Circulating tumour cells as prognostic markers in progressive, castration-resistant prostate cancer: a reanalysis of IMMC38 trial data. Lancet Oncol. 2009, 10 (3): 233-239. 10.1016/S1470-2045(08)70340-1.View ArticlePubMedPubMed CentralGoogle Scholar
- Devriese LA, Voest EE, Beijnen JH, Schellens JH: Circulating tumor cells as pharmacodynamic biomarker in early clinical oncological trials. Cancer Treat Rev. 2011, 37 (8): 579-589. 10.1016/j.ctrv.2011.04.006.View ArticlePubMedGoogle Scholar
- Mostert B, Sleijfer S, Foekens JA, Gratama JW: Circulating tumor cells (CTCs): detection methods and their clinical relevance in breast cancer. Cancer Treat Rev. 2009, 35 (5): 463-474. 10.1016/j.ctrv.2009.03.004.View ArticlePubMedGoogle Scholar
- Tibbe AG, De-Grooth BG, Greve J, Dolan GJ, Terstappen LW: Imaging technique implemented in Cell Tracks system. Cytometry. 2002, 47 (4): 248-255. 10.1002/cyto.10085.View ArticlePubMedGoogle Scholar
- Riethdorf S, Fritsche H, Muller V, Rau T, Schindlbeck C, Rack B, Janni W, Coith C, Beck K, Janicke F, et al: Detection of circulating tumor cells in peripheral blood of patients with metastatic breast cancer: a validation study of the Cell Search system. Clin Cancer Res. 2007, 13 (3): 920-928. 10.1158/1078-0432.CCR-06-1695.View ArticlePubMedGoogle Scholar
- Allard WJ, Matera J, Miller MC, Repollet M, Connelly MC, Rao C, Tibbe AG, Uhr JW, Terstappen LW: Tumor cells circulate in the peripheral blood of all major carcinomas but not in healthy subjects or patients with nonmalignant diseases. Clin Cancer Res. 2004, 10 (20): 6897-6904. 10.1158/1078-0432.CCR-04-0378.View ArticlePubMedGoogle Scholar
- Cummings J, Ward TH, Greystoke A, Ranson M, Dive C: Biomarker method validation in anticancer drug development. Br J Pharmacol. 2008, 153 (4): 646-656. 10.1038/sj.bjp.0707441.View ArticlePubMedGoogle Scholar
- Parkinson DR, Dracopoli N, Petty BG, Compton C, Cristofanilli M, Deisseroth A, Hayes DF, Kapke G, Kumar P, Lee J, et al: Considerations in the development of circulating tumor cell technology for clinical use. J Transl Med. 2012, 10: 138-10.1186/1479-5876-10-138.View ArticlePubMedPubMed CentralGoogle Scholar
- Khleif SN, Doroshow JH, Hait WN: AACR-FDA-NCI Cancer Biomarkers Collaborative consensus report: advancing the use of biomarkers in cancer drug development. Clin Cancer Res. 2010, 16 (13): 3299-3318. 10.1158/1078-0432.CCR-10-0880.View ArticlePubMedGoogle Scholar
- Dancey JE, Dobbin KK, Groshen S, Jessup JM, Hruszkewycz AH, Koehler M, Parchment R, Ratain MJ, Shankar LK, Stadler WM, et al: Guidelines for the development and incorporation of biomarker studies in early clinical trials of novel agents. Clin Cancer Res. 2010, 16 (6): 1745-1755. 10.1158/1078-0432.CCR-09-2167.View ArticlePubMedGoogle Scholar
- Kraan J, Sleijfer S, Strijbos MH, Ignatiadis M, Peeters D, Pierga JY, Farace F, Riethdorf S, Fehm T, Zorzino L, et al: External quality assurance of circulating tumor cell enumeration using the Cell Search((R)) system: a feasibility study. Cytometry B Clin Cytom. 2011, 80 (2): 112-118.View ArticlePubMedGoogle Scholar
- Rozet E, Ceccato A, Hubert C, Ziemons E, Oprean R, Rudaz S, Boulanger B, Hubert P: Analysis of recent pharmaceutical regulatory documents on analytical method validation. J Chromatogr A. 2007, 1158 (1–2): 111-125.View ArticlePubMedGoogle Scholar
- Hoffman D: Statistical considerations for assessment of bioanalytical incurred sample reproducibility. AAPS J. 2009, 11 (3): 570-580. 10.1208/s12248-009-9134-z.View ArticlePubMedPubMed CentralGoogle Scholar
- Tibbe AG, Miller MC, Terstappen LW: Statistical considerations for enumeration of circulating tumor cells. Cytometry A. 2007, 71 (3): 154-162.View ArticlePubMedGoogle Scholar
- Hou JM, Greystoke A, Lancashire L, Cummings J, Ward T, Board R, Amir E, Hughes S, Krebs M, Hughes A, et al: Evaluation of circulating tumor cells and serological cell death biomarkers in small cell lung cancer patients undergoing chemotherapy. Am J Pathol. 2009, 175 (2): 808-816. 10.2353/ajpath.2009.090078.View ArticlePubMedPubMed CentralGoogle Scholar
- Cummings J, Zhou C, Dive C: Application of the beta-expectation tolerance interval to method validation of the M30 and M65 ELISA cell death biomarker assays. J Chromatogr B Analyt Technol Biomed Life Sci. 2011, 879 (13–14): 887-893.View ArticlePubMedGoogle Scholar
- Rao C, Bui T, Connelly M, Doyle G, Karydis I, Middleton MR, Clack G, Malone M, Coumans FA, Terstappen LW: Circulating melanoma cells and survival in metastatic melanoma. Int J Oncol. 2011, 38 (3): 755-760.PubMedGoogle Scholar
- Allan AL, Keeney M: Circulating tumor cell analysis: technical and statistical considerations for application to the clinic. J Oncol. 2010, 2010: 426218-View ArticlePubMedGoogle Scholar
- Coumans FA, Ligthart ST, Uhr JW, Terstappen LW: Challenges in the enumeration and phenotyping of CTC. Clin Cancer Res. 2012, 18 (20): 5711-5718. 10.1158/1078-0432.CCR-12-1585.View ArticlePubMedGoogle Scholar
- Feinberg M: Validation of analytical methods based on accuracy profiles. J Chromatogr A. 2007, 1158 (1–2): 174-183.View ArticlePubMedGoogle Scholar
- Boemer F, Bours V, Schoos R, Hubert P, Rozet E: Analytical validation based on total error measurement and cut-off interpretation of a neonatal screening TSH-immunoassay. J Chromatogr B Analyt Technol Biomed Life Sci. 2009, 877 (23): 2412-2417. 10.1016/j.jchromb.2008.11.004.View ArticlePubMedGoogle Scholar
- Feinberg M, Boulanger B, Dewe W, Hubert P: New advances in method validation and measurement uncertainty aimed at improving the quality of chemical data. Anal Bioanal Chem. 2004, 380 (3): 502-514. 10.1007/s00216-004-2791-y.View ArticlePubMedGoogle Scholar
- Lee JW, Devanarayan V, Barrett YC, Weiner R, Allinson J, Fountain S, Keller S, Weinryb I, Green M, Duan L, et al: Fit-for-purpose method development and validation for successful biomarker measurement. Pharm Res. 2006, 23 (2): 312-328. 10.1007/s11095-005-9045-3.View ArticlePubMedGoogle Scholar
- Lee JW, Weiner RS, Sailstad JM, Bowsher RR, Knuth DW, O’Brien PJ, Fourcroy JL, Dixit R, Pandite L, Pietrusko RG, et al: Method validation and measurement of biomarkers in nonclinical and clinical samples in drug development: a conference report. Pharm Res. 2005, 22 (4): 499-511. 10.1007/s11095-005-2495-9.View ArticlePubMedGoogle Scholar
- Findlay JW: Some important considerations for validation of ligand-binding assays. J Chromatogr B Analyt Technol Biomed Life Sci. 2009, 877 (23): 2191-2197. 10.1016/j.jchromb.2008.10.045.View ArticlePubMedGoogle Scholar
- Cummings J, Raynaud F, Jones L, Sugar R, Dive C: Fit-for-purpose biomarker method validation for application in clinical trials of anticancer drugs. Br J Cancer. 2010, 103 (9): 1313-1317. 10.1038/sj.bjc.6605910.View ArticlePubMedPubMed CentralGoogle Scholar
- Jennings L, Van-Deerlin VM, Gulley ML: Recommended principles and practices for validating clinical molecular pathology tests. Arch Pathol Lab Med. 2009, 133 (5): 743-755.PubMedGoogle Scholar
- Scher HI, Morris MJ, Larson S, Heller G: Validation and clinical utility of prostate cancer biomarkers. Nat Rev Clin Oncol. 2013, 10 (4): 225-234. 10.1038/nrclinonc.2013.30.View ArticlePubMedPubMed CentralGoogle Scholar
- Hougton R, Gouty D, Allinson J, Green R, Losauro M, Lowes S, LeLacheur R, Garofolo F, Couerbe P, Bronner S, et al: Recommendations on biomarker bioanalytical method validation by GCC. Biogeosciences. 2012, 4 (20): 2439-2446.Google Scholar
- Hayashi N, Yamauchi H: Role of circulating tumor cells and disseminated tumor cells in primary breast cancer. Breast Cancer. 2012, 19 (2): 110-117. 10.1007/s12282-011-0282-5.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2407/13/415/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.