The androgen receptor (AR) axis is a major effector in the development and progression of prostate cancer and an important target in the rational drug design of new anticancer agents [1]. Prior to interaction with ligand (principally 5α-dihydrotestosterone, testosterone and androstenedione) the AR is localised in the cytoplasm bound to heat shock proteins and remains pre-dominantly inactive [2, 3]. Upon binding of an androgen the receptor dissociates from heat shock proteins and translocates to the nucleus where it binds to androgen response elements located in the promoter and enhancer regions of target genes, resulting eventually in the formation of an active transcription complex after recruitment of co-regulatory proteins [2, 3].

AZD3514 [6-(4-{4-[2-(4-acetylpiperazin-1-yl)ethoxy]phenyl}piperidin-1-yl)-3-(trifluoromethyl)-7,8-dihydro[1, 2, 4]triazolo[4,3-b]pyridazine] emerged as the preferred clinical candidate from an extensive programme of rational drug design aimed at identifying small molecule inhibitors of the AR [4, 5]. The drug has been shown to work by binding to the AR with high affinity, preventing nuclear translocation of the protein and inhibiting ligand-dependent and independent transcriptional activity [4, 6]. Unique to AZD3514 as a pharmacological modulator is its ability to down regulate the protein both in vitro and in vivo [6] and its anti-proliferative activity against Dunning R3327H prostate tumours in rats correlated to a reduction in AR levels in tumour tissue [6]. Also, activity of this class of drug is significantly enhanced in animal models by castration. Therefore, AZD3514 entered phase I trials as an oral agent in patients with castration-resistant prostate cancer (CRPC) [7].

During the phase I trial of AZD3514 (EudraCT 2010-020232-19, NCT01162395), detection and enumeration of circulating tumour cells (CTC) pre- and post-administration by the CellSearch System [8] was the major focus of the pharmacodynamic assessment of the drug [7]. Several previous studies have demonstrated that pre-dose CTC numbers around a cut-off value of 5 per 7.5 ml of blood are prognostic of outcome in CRPC (where ≥ 5 predicts for bad prognosis) [9–11]. Thus, CTC were incorporated primarily as a surrogate marker of the anti-tumour activity of AZD3514.

In the present study an immunohistochemistry (IHC) method has been developed for the determination of AR expression in CTC and subject to assay validation for potential application to the clinical evaluation of AZD3514. CTC were harvested from blood samples collected from the Astra Zeneca Sponsored Clinical Study (D1330N00013) by an approach termed Isolation by Size of Epithelial Tumour Cells (ISET), which lends itself more readily to IHC than the CellSearch System [12]. After an initial evaluation of specificity, validation studies focused on reproducibility [13, 14], where advanced statistical techniques, as recently applied to the validation of CTC enumeration by CellSearch, were employed in the analysis of data [15].

Specificity of the ISET/IHC technique for the AR in CTC was investigated employing, as controls, human prostate cancer cell lines of known AR status, together with treatment of cells with AZD3514 in order to modulate the levels of the protein. The positive control AR expressing cell line was LNCaP while the negative control was PC3. AR status was confirmed in these cell lines by Western blot analysis utilising the same antibody (AR441) as that employed in the IHC method through the presence of a band at 110 kDa in LNCaP and the absence of a band in PC3 (Inset to Figure 1), in keeping with previously published studies [19]. Figure 1 illustrates the results obtained from a typical ISET/IHC experiment, where blood from healthy volunteers was spiked with either untreated LNCaP cells, PC3 cells or LNCaP cells incubated with AZD3514 at a dose shown to reduce AR protein expression in this cell line [6]. Untreated LNCaP cells - the positive control - demonstrated strong nuclear brown staining for (translocated) AR, whereas the PC3 cells - the negative control - displayed a complete absence of brown staining. The almost complete absence of cytoplasmic staining in LNCaP cells may be due to constitutive autocrine stimulation of the AR signaling pathway [20]. In addition, in LNCaP cells pre-treated with AZD3514 prior to spiking into blood there was a marked reduction in the level of nuclear staining. These data indicate that the ISET/IHC method described herein can distinguish between AR positive and AR negative cancer cells in the blood of human subjects and is also sufficiently sensitive to detect a drug induced (pharmacodynamic) knockdown in protein levels.

The major focus of the present study was to characterise between-operator and within-operator variability, employing multiple blood samples collected from either different patients or at different time points in the same patient. The number of CTCs captured in the different ISET filter spots ranged from 0 to 30 per ml of blood, within the range of CTC previously reported in prostate cancer patients [21].

β-Content γ-confidence tolerance intervals (BCTI) reports on ISR both in the form of precision/imprecision and trueness/bias. Figure 2 illustrates the accuracy profiles at both 67% and 95% probability for the staining intensity assignments of two operators. These demonstrate a relatively large degree of imprecision, as might be expected with categorical data. However, they also highlight a significant bias, where operator 1 favoured a score of 2++ (moderate staining intensity) over 3+++ (strong), whereas operator 2 favoured 3+++ over 2++. Such a bias would not be evident by κ alone.

After the first validation experiment a programme of staff training was embarked upon. A gallery of 20 IHC images of CTC harvested by ISET from a number of different patients was constructed in order to facilitate a supervised training workshop. Here analysts were presented with 5 different sets of images where each set included an example of a patient CTC expressing AR at weak, moderate, strong and negative staining levels. One such set of 4 graded images taken from the gallery is illustrated in Figure 3. In the second validation experiment, the bias between operators observed in experiment 1 was completely eliminated (see Figure 4), and the value of κ for inter-operator agreement increased from 0.59 to 0.81 (Table 1), the latter being in the category of almost perfect agreement [22].

Since the samples proffered to the analysts in the second validation experiment were identical to those proffered in the first, this allowed for a conventional assessment of ISR [17] (see Figure 5). These data highlight the effect of the training programme, where analyst 1 showed a greater difference in scoring objects as 2++ between experiments , while analyst 2, as anticipated, showed a greater difference in scoring objects in the 3+++ category. It is also clear that the major effect of the training programme was more manifest on operator 2, while the degree of ISR achieved by operator 1 approached 30%.

Fit-for-purpose biomarker method validation defines 5 categories of assay based on readout ranging from absolute quantitation to a nominal positive/negative result [13, 14, 24, 25]. Along this spectrum, IHC in ocular microscopy mode is identified as an ordinal qualitative assay that yields categorical data presented as discrete scoring scales. Many of the performance characteristics normally associated with bioanalytical methods are not relevant to IHC, for example accuracy [26, 27]. However, the main parameter of relevance to a qualitative assay is reproducibility: defined by the ICH as the “precision of repeated measurements between (operators and) laboratories” [14]. Or put more simply, “the property of receiving consistent results from following a specific procedure” [26]. Although there are many technical variables that could impact on the reproducibility of an IHC method, such as processing and embedding tissue and selection of section thickness, [27] the major source of error is recognised as that which is introduced by the reader [28, 29].

In the present paper, method validation was performed on an IHC method for the determination of AR in CTC. While the focus was obviously reproducibility in addition to an assessment of specificity, the issue of inter-operator variability was addressed in a novel manner, through a modification of ISR [30, 31]. Thus, in addition to conducting ISR in the conventional format (where typically a single operator analyses the same set of samples twice), in our modification two different operators each analysed the same set of samples once and the results were then subjected to statistical analysis by BCTI. We have previously demonstrated that in the case of CTC enumeration by CellSearch this modified approach to ISR was exquisitely sensitive in differentiating between both systematic (bias) and random (imprecision) errors [15]. Our results confirm that even in the relatively less complex scenario of a single CTC sitting on a filter, there is still considerable between-operator variability in the assignment of a scoring intensity. However, since BCTI identifies the nature of the inter-operator error, it also allowed for its correction through additional staff training. In addition, by adoption of this approach it has been demonstrated that a highly trained analyst is capable of achieving scores in the repeat analysis of samples that varied by 30%, which is within the accepted benchmark for total error of a typical quantitative biomarker/pharmacodynamic assay such as an ELISA [14, 25, 32].

As a categorical assay, IHC has a dynamic range restricted to a limited number (normally 3 to 4) of band widths of staining intensity. Nonetheless, in preclinical studies with AZD3514, and utilising the same scoring structure as the present method (0+, 1+, 2+ and 3+), IHC was able to demonstrate a dose dependent reduction on AR in tumour tissue, at doses of drug that produced only a modest inhibition of tumour growth [6]. In the same report a comparative evaluation was conducted between ocular microscopy and image analysis using the Aperio system (ePathology Solutions, Oxford, UK). Reassuringly, both approaches reported a similar level of knockdown in AR. Among current IHC techniques approved by the FDA as diagnostic, prognostic or predicative biomarkers, no claims are made that image analysis is any more accurate than visual assessment by a trained pathologist [27].

The ISET technique is an example of a tumour-marker-independent technology based on filtration through 8 μm pores, unlike the FDA cleared CellSearch System which relies critically on the presence of epithelial markers (EpCam) and antibody directed capture [21, 33]. However, the ISET technique may have an advantage over CellSearch since it is now believed that a significant proportion of malignant CTCs lose their “epithelial markers” in preference to mesenchymal antigens, in a process known as epithelial to mesenchymal transition [34–36]. As a consequence the ISET technique invariably harvests larger populations of CTC from patient blood than the CellSearch system [37]. Nonetheless, while the ISET technique is claimed to effectively remove the majority of hematologic cells - red blood cells by lysis and peripheral blood leukocytes by filtration [12] - rare hematologic cells (megakaryocytes or large monocytes) or mesenchymal (endothelial) cells may be difficult to distinguish from epithelial tumour cells by cytopathologic or immunochemical analysis [38]. Indeed, it has been demonstrated in a number of different disease types, that ISET harvests cells of a non-malignant morphology from a small but significant group of subjects that could potentially result in a false-positive diagnosis [38]. These results advise caution when relying on a single technique to isolate CTC from patients.

A novel procedure is presented for the fit-for-purpose evaluation of the reproducibility of an IHC method for the determination of AR receptor expression levels in CTC isolated from patients as a biomarker assay utilising a modification of ISR and BCTI for statistical analysis of results. The procedure was employed to identify and correct inter-operator bias in the assignment of a scoring intensity and poor reproducibility, resulting potentially in a reduction of measurement error to a level normally associated with a quantitative biomarker assay, such as an ELISA. The methodological approach could be applied theoretically to any generic IHC method.