Analytical validation of the PAM50-based Prosigna Breast Cancer Prognostic Gene Signature Assay and nCounter Analysis System using formalin-fixed paraffin-embedded breast tumor specimens
© Nielsen et al.; licensee BioMed Central Ltd. 2014
Received: 24 October 2013
Accepted: 12 February 2014
Published: 13 March 2014
NanoString’s Prosigna™ Breast Cancer Prognostic Gene Signature Assay is based on the PAM50 gene expression signature. The test outputs a risk of recurrence (ROR) score, risk category, and intrinsic subtype (Luminal A/B, HER2-enriched, Basal-like). The studies described here were designed to validate the analytical performance of the test on the nCounter Analysis System across multiple laboratories.
Analytical precision was measured by testing five breast tumor RNA samples across 3 sites. Reproducibility was measured by testing replicate tissue sections from 43 FFPE breast tumor blocks across 3 sites following independent pathology review at each site. The RNA input range was validated by comparing assay results at the extremes of the specified range to the nominal RNA input level. Interference was evaluated by including non-tumor tissue into the test.
The measured standard deviation (SD) was less than 1 ROR unit within the analytical precision study and the measured total SD was 2.9 ROR units within the reproducibility study. The ROR scores for RNA inputs at the extremes of the range were the same as those at the nominal input level. Assay results were stable in the presence of moderate amounts of surrounding non-tumor tissue (<70% by area).
The analytical performance of NanoString’s Prosigna assay has been validated using FFPE breast tumor specimens across multiple clinical testing laboratories.
KeywordsPAM50 Analytical validation ROR Subtype Breast cancer Prosigna NanoString nCounter Reproducibility FFPE Gene expression
Molecular biomarkers have played an increasingly important role in identifying cancer patients with different prognostic outcomes and in predicting response to chemotherapy [1–3]. Molecular assays targeting these biomarkers are now routinely performed in local pathology labs to help guide treatment decisions in breast cancer [4, 5], lung cancer , and colorectal cancer . Gene expression analysis has helped identify distinct molecular signatures in breast cancer that have different prognostic outcomes [8–10]. Multigene assays targeting 21 – 70 genes are now routinely used in clinical practice to assess risk of recurrence in early stage breast cancer [11, 12], and prospective clinical trials are also underway to provide further supporting evidence for the clinical utility of these assays [13, 14]. To date, breast cancer multigene clinical assays have been largely limited to central reference laboratories due to the complexity of performing the test. Ultimately, development of assays with a simplified workflow is required to move these multigene expression tests into the local pathology lab setting, where efficiencies such as shorter turnaround time and direct interaction between laboratory physicians and the clinicians will benefit active patient care.
The PAM50 gene signature measures the expression levels of 50 genes in a surgically resected breast cancer sample to classify a tumor as one of four intrinsic subtypes (Luminal A, Luminal B, HER2-enriched, and Basal-like) , which have been shown to be prognostic in both untreated (i.e. no adjuvant systemic therapy) and tamoxifen treated patient populations [15, 16]. In addition to identifying a tumor’s intrinsic subtype, the PAM50 signature generates an individualized score estimating a patient’s probability of disease recurrence by weighting the molecular subtype correlations, a subset of proliferation genes, and pathologic tumor size [15, 16]. The PAM50 test was adapted to be performed using the nCounter Analysis System in order to develop a simplified workflow that could be performed in a local pathology lab (Prosigna™ Breast Cancer Gene Signature Assay, NanoString Technologies, Seattle). This technology uses multiplexed gene-specific fluorescently-labeled probe pairs  to measure gene expression in frozen or formalin-fixed paraffin-embedded (FFPE) tissues with equivalent ease and efficiency . A recent clinical validation performed on RNA extracted from over 1000 FFPE tumor specimens from the ATAC clinical trial demonstrated that the Prosigna risk of recurrence (ROR) score, based on the PAM50 gene expression signature, added significant prognostic information beyond the Oncotype DX® Recurrence Score® in estimating the likelihood of distant recurrence in hormone receptor positive, post-menopausal breast cancer patients  treated with endocrine therapy alone. A second clinical validation study performed on over 1400 FFPE patient samples from the ABCSG-8 trial has independently confirmed the clinical validity and demonstrated additional prognostic value in node-positive patients and for the risk of late recurrence [20, 21]. Based in part on the results from these clinical studies and the analytical studies described herein, NanoString obtained a CE Mark for its Prosigna assay in 2012, followed by US Food and Drug Administration (FDA) clearance in September of 2013.
Recently, requirements for demonstrating utility of a tumor biomarker were established that include not only clinical validity, but also analytical reproducibility and robustness [22, 23]. The results of ATAC and ABCSG-8, including a follow up combined analysis of the two studies  meet this high level of evidence (Level I) for clinical validity using archived specimens . The studies described herein were designed to test the analytical validity of decentralized use of the Prosigna assay across multiple clinical testing sites, following established guidelines . These studies were also designed to validate procedures for training laboratory personnel to perform the Prosigna assay on the nCounter system.
NanoString Prosigna assay
The tissue input for the Prosigna assay was FFPE tissue that had been previously diagnosed to contain viable invasive breast carcinoma. The breast tumor tissue must be classified by a pathologist as invasive carcinoma (ductal, lobular, mixed, or no special type). A pathologist reviews an H&E stain of a slide mounted tumor section to identify and circle the region of viable invasive breast carcinoma. The tumor surface area on the H&E stained section must be ≥ 4 mm2 per slide, with tumor cellularity ≥ 10%. Non-tumor tissue from outside the circled area is removed by macrodissection of the corresponding unstained slides. RNA was extracted from slide mounted breast tissue sections using a RNA extraction kit manufactured by Roche to NanoString’s specifications . For RNA isolation, a single 10-micron slide mounted tissue section was input for RNA extraction when the tumor surface area measured ≥ 100 mm2, whereas 3 slides were input when the tumor surface measured 4-99 mm2. Following extraction of total RNA and removal of genomic DNA, RNA was eluted (30 μL volume) and tested to ensure it met the specifications for concentration (≥ 12.5 ng/ μL) and purity (OD 260/280 nm 1.7-2.5).
The NanoString Prosigna assay  measures the expression levels of 50 target genes plus eight constitutively expressed normalization genes [15, 27, 28]. Assay controls are included to ensure that test samples and the test process meet pre-defined quality thresholds. Exogenous probes with no sequence homology to human RNA sequences are included as positive and negative assay controls. Positive controls are comprised of a six point linear titration of in vitro transcribed RNA covering an approximately 1000 fold RNA concentration range (0.125 – 128 fM) and corresponding probes [29, 30]. Negative controls consist of a set of probes without the corresponding targets. Each assay run includes two reference control samples comprising in vitro transcribed RNA of the 58 targets for qualification and normalization purposes.
Extracted RNA samples meeting quality and concentration specifications were hybridized (without reverse transcription or amplification) to capture and reporter probes for the measured genes and assay controls. The multiplexed hybridizations are carried out in a single-tube for 15 – 21 hrs at 65°C using 125 – 500 ng RNA (nominal input of 250 ng). After hybridization, the target-probe complexes were processed on the nCounter Analysis System. Test sample data must meet a minimum threshold for expression of normalizing genes to ensure that the assay signal is high enough for the algorithm to produce precise results. The linearity of the positive control target titration and the non-specific background from negative control probes included in each assay is used to determine whether each assay performed within specification. Since the test is designed to be run in local molecular pathology labs, all quality thresholds are applied automatically to the data by embedded software; any failing metric causes an assay failure notice which prevents output of a Prosigna assay result. For samples meeting all quality thresholds, a clinically validated algorithm is used to determine the intrinsic subtype and ROR score, which are prognostic indicators of risk of distant recurrence of breast cancer [19, 21]. The normalized gene expression profile of each breast tumor sample is correlated to prototypical gene expression profiles of the four breast cancer intrinsic subtypes (Luminal A, Luminal B, HER2-enriched, and Basal-like). The primary tumor size (categorical input of ≤ 2 cm or > 2 cm) and normalized gene expression profile of each breast tumor sample is used to calculate the numerical ROR score. Risk categories are assigned to allow interpretation of the ROR score by using pre-specified cutoffs (defined in a clinical validation study) related to risk of distant recurrence after 10 years .
Operators for these studies were required to undergo training procedures to demonstrate proficiency, equivalent to what will be used to train users in molecular pathology laboratories for the decentralized test. Each site was given an overview of the NanoString technology and Prosigna assay procedures followed by an in-lab exercise where users were trained and qualified on tissue processing and assay procedures (requiring 10-12 hours of total hands-on time). Briefly, each user extracted RNA from three FFPE breast tumor tissue samples to demonstrate proficiency in tissue processing, and each user processed four prototypical breast tumor RNA samples (one of each intrinsic subtype with known expected ROR score values) along with a negative control sample to demonstrate proficiency on the nCounter Analysis System.
The analytical studies described herein were performed using pre-specified SOPs, statistical analysis plans and acceptance criteria using clinical-grade reagents, instrumentation, and software formatted such that no comparison of results between test centers could even be possible until the study was completed.
RNA precision: study design
The RNA Precision study assessed the reproducibility of the Prosigna assay using a common template of purified RNA, thereby isolating the device-specific components of analytical validity from variables associated with tissue processing. The experimental design for analytically validating the precision of the assay from RNA was based on Clinical Laboratory and Standards Institute (CLSI) guidelines for the evaluation of precision of in vitro diagnostic devices outlined in EP05-A2 . This design measured the variability between and within a number of assay variables including testing site (n = 3), operator (n = 6), reagent lot (n = 3) and assay run (n = 18/site). Two of the three sites used were CLIA-certified, CAP-accredited laboratories at the British Columbia Cancer Agency (Vancouver), and Washington University (St. Louis); the third site was NanoString Technologies (Seattle).
RNA precision study sample summary
Estimated ROR score
Luminal B #1
Luminal B #2
Upon completion of the study the blinded data were collected from each site and merged with the expected test result and study variables (site, operator, reagent lot, etc.) associated with each unique sample ID. The prospectively defined analysis plan was then executed on the merged analysis dataset.
RNA precision: statistical analysis
The pre-specified primary aim of the RNA precision validation was to demonstrate that there was no significant differences for the continuous ROR score assay output across the three testing sites.
The following variance components model was used to characterize the sources of variability:
ROR Score = site + operator + lot + run + within-run
where all components were treated as random components, and the RNA assay component of variation was defined as the sum of all these components. Variance components were estimated using the R procedure “lmer”. To test whether sites were significantly different, the following versions of the above model were fitted:
ROR Score = site + operator + lot + run + within-run
& ROR Score = operator + lot + run + within-run
where site was now treated as fixed and all other components were treated as random. A likelihood ratio test with 2-degrees of freedom was performed using the fitted models to determine whether the effect of site was significant (α = 0.05). A similar analysis was performed for the assay reagent lots.
For each of the 5 pooled samples, the classifications into the 4 intrinsic subtype categories (Luminal A, Luminal B, Basal-Like, HER2-enriched) were summarized using frequency tables.
Reproducibility: study design
The reproducibility study assessed the analytical validity of the Prosigna assay, including all steps involving in clinical lab implementation (i.e. tissue handling and RNA isolation SOPs as well as the device-specific assay steps), using a common set of breast cancer tissue samples.
The experimental design for analytically validating the reproducibility from tissue was based on CLSI guidelines for the evaluation of precision of in vitro diagnostic devices outlined in CLSI EP05-A2. This design allows for the measurement of variability between and within a number of assay variables including testing site, FFPE sample block, operator, reagent lot, and assay run.
Every case should represent a unique breast cancer patient
All must be primary breast cancers
All are pathology confirmed invasive ductal or lobular carcinoma, a mixtures of these types, or classified as no special type
All are hormone receptor positive (ER + or PgR+) breast cancer
All must have a recorded tumor size
FFPE blocks should be < 10 years old
A minimum of 10 cases each of ≥ 100 mm2 tumor area (1 slide/test) and 4 - 100 mm2 tumor area (3 slides/test)
The criterion that at least 10 cases contain ≥ 100 mm2 and at least 10 cases contain 4 - 99 mm2 tumor area was implemented to validate the number of slides required for the assay. The blocks were not prescreened with the assay prior to inclusion, but it was anticipated that the 43 samples would cover a broad range of ROR scores representative of the intended use population, including both node-negative and node-positive patients, and each risk classification group. Seventeen tissue samples were from node-negative patients, 6 from node-positive patients and 20 were from patients whose regional lymph node status was provided by the biobank as NX.
For each tissue sample, a test run consisting of macrodissection, RNA extraction, and testing with the Prosigna assay was performed by a single operator at each site following the provided standard operating procedures. Each operator performed a minimum of four test runs consisting of up to 10 tissue samples per run. Each batch of tissue samples required a minimum run time of 3 days from tissue processing to result. Isolated RNA that met the quantity and quality specifications from each of the slide mounted sections was tested twice in separate assay runs. Different lots of RNA isolation kit reagents were used at each site, and a single lot of the Prosigna assay kit was used at all three sites.
The test results for all samples remained blinded to all personnel at all sites until the study was complete. Upon completion of the study the blinded Prosigna assay data were collected from each site and merged with the expected test result and study variables (site, operator, reagent lot, etc.) associated with each unique sample ID. The prospectively defined analysis plan was then executed on the merged analysis dataset.
Reproducibility: statistical analysis
The pre-specified primary aim of the tissue reproducibility validation was to demonstrate the Prosigna assay is highly reproducible, when combining all sources of variation. For this study, “highly reproducible” was defined as a total standard deviation (SD) of less than 4.3 ROR units. The value of <4.3 was chosen because if two samples have true ROR scores that differ by 10 units, a total SD of 4.3 means that 95% of the time the higher of the two will still have a higher individual observed ROR score. A change of 10 ROR units corresponds to an average change in 10-year distant recurrence free survival of 7% and 6% for node negative and node positive patients respectively .
The following variance components model was used to characterize the sources of variability:where FFPE Block was treated as a fixed component, and site and section were treated as random components. The “site” term measured the systematic site-specific variation that was constant across all tissue samples (pathologist, technician, extraction kit). The tissue section component measures random variation that differed as a function of review/processing or within FFPE block variation. The error term was derived from the duplicate RNA samples and estimated the combination of run-to-run and within-run variance. Variance components were estimated using the R procedure “lmer”. In the above model, the variance components were estimated from a combined analysis of all FFPE blocks after verifying that were no systematic changes in tissue-specific variation as a function of ROR score.
Measurement = FFPE Block + site + tissue section + error
where was estimated as the sum of the site-to-site and section component estimated in the tissue reproducibility study, and was estimated as the total variation from the RNA precision study.
Additional categorical analyses were performed using two classifications:
3 risk-categories (low, intermediate, and high) using both the node-negative and node-positive cutoffs,
4 intrinsic subtype categories (Luminal A, Luminal B, Basal-Like, HER2-enriched)
RNA from each tissue sample was tested twice at each site so there are 4 possible comparisons between sites for each tissue sample leading to a total number of possible comparisons of 4*number of tissue samples. For each of the two classification schemes (risk category or subtype), the pair-wise concordance between sites was estimated as the fraction of all possible comparisons that were concordant and an exact-type 95% confidence interval was calculated.
In addition, a post hoc analysis compared the normalized gene expression from the 50 classifier genes between the tissue replicates from all valid specimens tested at each site using a linear regression and correlation analysis
RNA input: study design
RNA input: statistical analysis
where the average is across the n different samples. In this equation, is the average of two replicates at the nominal level and s the average of two replicates for input levels within specification, or is the single result for input levels outside of specification. Equivalence was pre-defined as an observed absolute average ROR difference significantly less than 3. To test the non-equivalence hypothesis that the true absolute mean difference is greater than 3, a 90% confidence interval for the difference was calculated. This 90% confidence interval corresponds to the two one-sided test approach for bioequivalence . The input level was determined to be equivalent to the nominal level if the 90% confidence interval is completely contained within -3 and 3.
For each pooled sample a linear regression and correlation analysis was also performed between each replicate at each RNA input level and one of the two replicates run at 250 ng of RNA. The difference in the ROR score (ΔROR) from the nominal RNA input level (250 ng) for each replicate at each RNA input level was calculated by subtracting the ROR score calculated from one of the two replicates run at 250 ng from ROR scores calculated at the other input levels. Additionally, the ΔROR was calculated and linear regression and correlation analyses were also performed between the two replicates at 250 ng. The mean ΔROR, slope, intercept, and correlation values (with 95% confidence intervals) were calculated using the pairwise comparisons for all passing samples at each input level for both kit lots.
For the no-target (water) samples, the percentage of samples failing the minimum threshold for expression of normalizing genes was calculated. All no-target samples were required to give a failing test result.
Tissue interferents: study design and analysis
Twenty three FFPE breast tumor blocks were obtained containing pathologically-confirmed infiltrating ductal carcinoma microscopically-assessed to have 10 – 95% of the total tissue area containing normal/non-tumor tissue. Pathologists identified additional tumor interferents (DCIS, necrotic tissue, or blood/hemorrhagic tissue) within or near the margins of the tumor in ten of the 23 blocks.
For each FFPE breast tumor block, H&E stained slides were prepared and up to nine unstained sections were cut and mounted on slides. For the inclusion of the interferent, the sections were processed according to the assay procedure with the exception that identified normal/non-tumor tissue or any additional interferents were included in the isolation (“non-macrodissected slides”). For the macrodissection where the non-tumor and other interferents were removed, three or (in the case of small tumor surface areas) three and six slides were processed according to the Prosigna assay protocol.
RNA precision: variance components analysis
Variance components for the five pooled RNA samples across 108 replicates
Pooled RNA sample
Mean ROR score
Variance component (%)
Luminal B 1
Luminal B 2
Reproducibility: test sample quality control and characterization
The call rate for the 43 tissue samples evaluated was 95%, 93%, and 100% for sites 1, 2, and 3 respectively. Forty samples yielded results at all sites (RNA isolation of one sample at one site required repeating). One tissue sample yielded results at 2 sites, and 2 samples yielded results at a single site, while the other sites did not obtain sufficient RNA to perform the assay for these samples. The measured tumor surface area for 4/5 RNA isolation failures was very small (≤ 15 mm2). One hundred percent (100%) of samples passing tissue review and RNA isolation specifications yielded passing results from the Prosigna assay.
Reproducibility: variance components analysis (primary objective)
Total variability (from tissue and RNA processing) of the Prosigna assay
Tissue processing variability
RNA processing variability
The total SD including all source of variation (tissue and RNA processing variability) was 2.9 indicating that the Prosigna assay can measure a difference between two ROR scores of 6.75 with 95% confidence.
Reproducibility: subtype and risk category classifications concordance
The site-to-site concordances for the two categorical classifications are shown in Table 4, in each case with exact-type 95% confidence intervals. For each comparison (subtype and node negative and positive risk categories), the average concordance between sites was at least 90%. There were no samples where the risk category changed from low risk to high risk (or vice versa) between or within sites when the samples were assumed to be from node negative patients. There were only two intermediate/high risk samples that did not give identical subtypes across all 6 replicates:
One sample had duplicate Luminal A results at one site and duplicate Luminal B results at each of the other two sites.
One specimen had duplicate Luminal A results at one site, duplicate HER2-enriched results at another site and one each of Luminal A and HER2-enriched at the third site.
Concordance of subtype calls and risk categories between the three sites
Pairwise Concordance [95% CI]
Site 1 vs. Site 2
Site 1 vs. Site 3
Site 2 vs. Site 3
(n = 40)
(n = 41)
(n = 40)
Reproducibility: pairwise correlation coefficients of gene expression
Site to site gene expression comparisons from the tissue reproducibility study
Site 1 vs. Site 2
Site 1 vs. Site 3
Site 2 vs. Site 3
RNA input: test sample quality control
The average ROR score for the tested samples covered a broad range (20 – 82) and all intrinsic subtypes – including 5 Luminal A, 4 Luminal B, 3 HER2-enriched and 1 Basal-like sample (Additional file 4: FigureS2). One FFPE block was tested with a single kit lot due to insufficient RNA mass from the isolation for the second lot. Two runs (each with different samples) failed to provide passing results for one of the two lots tested due to a processing error detected by system controls with insufficient RNA to repeat the assay. All measured no-target samples (n = 46) were well below the threshold for signal and yielded a failing test result (0% call rate). All tumor RNA measurements within assay specification (n = 138) yielded a passing test result (100% call rate). One hundred percent (100%) of specimens with input above specification (625 ng) yielded a passing test result. Eighty-three percent (83%) of specimens (10/12) tested at input below specification (62.5 ng) yielded a test result in lot 1, as did 100% in lot 2.
RNA input: ROR score difference and pairwise correlation coefficients of gene expression
For each of the two reagent lots tested, the confidence interval around the mean ROR score difference between the nominal input and the RNA input limits (125 and 500 ng) were completely contained within -3 and 3 ROR units. The ROR scores at 125 and 500 ng RNA were therefore equivalent to those at the target input concentration of 250 ng for each of the two reagent kit lots tested meeting the primary objective of the study. Of note, when characterizing the RNA levels outside of the assay specification, the ROR scores at 62.5 ng RNA were not equivalent (with an upper confidence limit at 3.26) to those at the target input concentration of 250 ng for one of the two lots tested. This illustrates the importance of performing the assay according to the defined procedure.
Comparison of gene expression at different masses from the RNA input study
Tissue interferents: test sample quality control
Out of 23 samples six were Luminal A, seven were Luminal B, two were HER2-enriched, and eight were Basal-like. The average ROR score for the 23 samples covered a broad range (10 – 83), (Additional file 5: Figure S3).
Tissue interferents: impact on ROR score
Breast cancer gene expression testing has been the subject of many studies demonstrating its capacity to stratify breast cancers by prognostic risk [9, 15, 16, 32, 33]. Increasingly, studies are also showing the value of such signatures to predict response to therapy, for example by using these tests to evaluate archival specimens from randomized clinical trials [34–36]. The integration of molecular genomic testing into cancer care is an active area of development, with huge genomic datasets becoming available. Great improvements in experimental design and bioinformatic analysis have led to the development of robust signatures ripe for translation into clinical tests. Studies applying these signatures to different clinical series with observational, case-control, cohort and randomized trial designs have generated increasingly strong evidence for clinical validity, particularly in breast cancer [19, 34, 35, 37]. It is in this backdrop that the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) working group was formed to guide best practices in experimental design and the interpretation of evidence for utility in clinical practice . Fundamental to EGAPP criteria is the concept that clinical utility requires not only clinical validity (linking test results to clinical presentation, treatment and outcome), but just as importantly, analytical validity (the capacity of the test classifier to be sensitive, specific and reproducible in practice). However, EGAPP found that relatively few studies of breast cancer molecular classifiers have directly reported on analytical reproducibility .
Analytical reproducibility is a requirement for the implementation of all diagnostic tests, but it is especially critical for decentralized tests given the challenges of maintaining reproducibility across pathologists, technical operators, and instrumentation. However, decentralized tests also have many advantages over Laboratory Developed Tests that are performed at single central laboratories. By avoiding the need for shipping tissues, turnaround times and costs are reduced. The capacity for the laboratory physician to interact directly with the treating physician greatly aids medical care, for example in facilitating appropriate prioritization of critical specimens, explaining equivocal or unexpected results, and quickly recognizing inadequate specimens and what can be done to get a result helpful to the patient as soon as possible. Although first generation breast cancer prognostic tests were performed in central labs , second generation tests are being developed and validated to realize the advantages of decentralized testing .
The Prosigna assay was tested across a range of RNA mass inputs that is consistent with what will be expected in a clinical lab setting. The assay is robust across that range, similar to what has been reported with other multigene breast cancer tests [32, 39]. Additionally, the assay gave consistent results outside the specified assay RNA input limits; only 2 samples failed to produce passing results at half the lowest specified mass further illustrating the robustness of the assay.
The observation of biased subtype calls and ROR scores with the inclusion of non-tumor tissue is consistent with a prior study , however the bias reported herein is less severe. Similar to what is expected to be experienced in clinical practice, the interferent being measured here is reported as percent adjacent non-tumor tissue included, rather than percent non-tumor RNA from a separate paired normal tissue sample reported in the earlier study. Normal breast tissue yields less total RNA compared to tumor tissue  and adjacent non-tumor tissue at the margins of the tumor have certain cancer pathways activated where matched healthy breast does not [42, 43]. Nonetheless, this study illustrates the importance of performing the macrodissection according to the defined procedure to maximize the accuracy of the test.
The precision and reproducibility of the Prosigna assay, estimated from repeat measurements of pooled tumor RNA sample(s) and de-identified patient tissue samples across multiple testing sites is similar (relative to the overall test range) to what was previously reported for centralized lab tests [32, 38]. These results demonstrate that the Prosigna assay is analytically reproducible even when performed at multiple test sites and including all process variables. It will be important for local labs to verify the reproducibility reported herein when implementing this decentralized assay to ensure the quality of the test’s results, including ongoing process monitoring .
Our experience of implementing the nCounter platform in our CLIA-certified hospital laboratory environments proved to be straightforward, confirming the simplicity of the assay and its suitability as an in vitro diagnostic test. Training of the assay workflow (including tissue macrodissection, RNA isolation and setup of Prosigna assay) takes less than one week. The pre-specified SOPs are easy to follow and the procedure of RNA extraction and Prosigna assay are straightforward. All operators, most of whom were naïve users to the Prosigna assay, were able to pass the training procedures on the first attempt, before executing the pre-specified study protocols. Although overnight incubations are required during RNA extraction and RNA – probe hybridization, the incubation temperature is constant, and hands-on time requirements for the whole experiment are very limited. Furthermore, the analyses for subtype call and ROR score are simplified and controlled by integrating the algorithm into the software for raw data processing, reducing the potential for human error in data cleaning and analysis.
The FDA cleared and CE marked Prosigna assay based on the PAM50 gene expression signature has recently been shown to predict the risk of distant recurrence in women with hormone receptor positive early stage breast cancer treated with five years of endocrine therapy [19, 20]. This demonstration of analytical reproducibility generates a strong body of evidence supporting the decentralized use of this test as a tool for breast cancer risk stratification. Additional ongoing studies of the clinical validity of the PAM50 gene expression signature for chemosensitivity prediction [34–36] could, if confirmed, be considered clinically actionable given the demonstrated analytical validity of this test.
TN, JS, BW, CS, SF, SL contributed to the study design and protocols and drafted the manuscript. CS, JS, and BW performed statistical analysis and presentation of data. DG performed tissue review for the tissue reproducibility study and the tissue interference study. ND, MM, and GB performed the assay for the RNA precision study. ND and GB performed the assay for the tissue reproducibility study. ND performed the assay for the RNA input and tissue interferents studies. All authors read and approved the final manuscript.
Risk of recurrence
US Food and Drug Administration
Clinical Laboratory and Standards Institute
Evaluation of genomic applications in practice and prevention
We thank Sandra McDonald, MD from Washington University School of Medicine for her excellent work in procuring, maintaining the integrity and de-identification, and histological review of tissue specimens for the tissue reproducibility study.
We thank Shashikant Kulkarni PhD, FACMG, Vishwanathan Hucthagowder PhD, and Mike Evenson for providing laboratory space, technical expertise, and execution of tissue reproducibility and RNA precision protocols.
We thank Katherine Deschryver, MD from Washington University School of Medicine for her histological review of tissue specimens for the tissue reproducibility study.
We thank Nasrin Mawji from the Centre for Translational and Applied Genomics at BC Cancer Agency for her work in executing the RNA precision protocol.
- Henry NL, Hayes DF: Cancer biomarkers. Mol Oncol. 2012, 6 (2): 140-146. 10.1016/j.molonc.2012.01.010.View ArticlePubMedGoogle Scholar
- Dietel M, Johrens K, Laffert M, Hummel M, Blaker H, Muller BM, Lehmann A, Denkert C, Heppner FL, Koch A, Sers C, Anagnostopoulos I: Predictive molecular pathology and its role in targeted cancer therapy: a review focussing on clinical relevance. Cancer Gene Ther. 2013, 20 (4): 211-221. 10.1038/cgt.2013.13.View ArticlePubMedGoogle Scholar
- Duffy MJ, Crown J: Companion Biomarkers: Paving the Pathway to Personalized Treatment for Cancer. Clin Chem. 2013, 59 (10): 1447-1456. 10.1373/clinchem.2012.200477.View ArticlePubMedGoogle Scholar
- Gown AM: Current issues in ER and HER2 testing by IHC in breast cancer. Mod Pathol. 2008, 21 (Suppl 2): S8-S15.View ArticlePubMedGoogle Scholar
- Wolff AC, Hammond ME, Schwartz JN, Hagerty KL, Allred DC, Cote RJ, Dowsett M, Fitzgibbons PL, Hanna WM, Langer A, McShane LM, Paik S, Pegram MD, Perez EA, Press MF, Rhodes A, Sturgeon C, Taube SE, Tubbs R, Vance GH, van de Vijver M, Wheeler TM, Hayes DF, American Society of Clinical Oncology, College of American Pathologists: American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. J Clin Oncol. 2007, 25 (1): 118-145.View ArticlePubMedGoogle Scholar
- Sholl LM, Xiao Y, Joshi V, Yeap BY, Cioffredi LA, Jackman DM, Lee C, Janne PA, Lindeman NI: EGFR mutation is a better predictor of response to tyrosine kinase inhibitors in non-small cell lung carcinoma than FISH, CISH, and immunohistochemistry. Am J Clin Pathol. 2010, 133 (6): 922-934. 10.1309/AJCPST1CTHZS3PSZ.View ArticlePubMedPubMed CentralGoogle Scholar
- Weichert W, Schewe C, Lehmann A, Sers C, Denkert C, Budczies J, Stenzinger A, Joos H, Landt O, Heiser V, Rocken C, Dietel M: KRAS genotyping of paraffin-embedded colorectal cancer tissue in routine diagnostics: comparison of methods and impact of histology. J Mol Diagn. 2010, 12 (1): 35-42. 10.2353/jmoldx.2010.090079.View ArticlePubMedPubMed CentralGoogle Scholar
- Perou CM, Sorlie T, Eisen MB, van de Rijn M, Jeffrey SS, Rees CA, Pollack JR, Ross DT, Johnsen H, Akslen LA, Fluge O, Pergamenschikov A, Williams C, Zhu SX, Lonning PE, Borresen-Dale AL, Brown PO, Botstein D: Molecular portraits of human breast tumours. Nature. 2000, 406 (6797): 747-752. 10.1038/35021093.View ArticlePubMedGoogle Scholar
- Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, Baehner FL, Walker MG, Watson D, Park T, Hiller W, Fisher ER, Wickerham DL, Bryant J, Wolmark N: A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med. 2004, 351 (27): 2817-2826. 10.1056/NEJMoa041588.View ArticlePubMedGoogle Scholar
- van de Vijver MJ, He YD, van’t Veer LJ, Dai H, Hart AA, Voskuil DW, Schreiber GJ, Peterse JL, Roberts C, Marton MJ, Parrish M, Atsma D, Witteveen A, Glas A, Delahaye L, van der Velde T, Bartelink H, Rodenhuis S, Rutgers ET, Friend SH, Bernards R: A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med. 2002, 347 (25): 1999-2009. 10.1056/NEJMoa021967.View ArticlePubMedGoogle Scholar
- Arpino G, Generali D, Sapino A, Del Lucia M, Frassoldati A, De Laurentis M, Paolo P, Mustacchi G, Cazzaniga M, De Placido S, Conte P, Cappelletti M, Zanoni V, Antonelli A, Martinotti M, Puglisi F, Berruti A, Bottini A, Dogliotti L: Gene expression profiling in breast cancer: a clinical perspective. Breast. 2013, 22 (2): 109-120. 10.1016/j.breast.2013.01.016.View ArticlePubMedGoogle Scholar
- Harris L, Fritsche H, Mennel R, Norton L, Ravdin P, Taube S, Somerfield MR, Hayes DF, Bast RC, American Society of Clinical Oncology: American Society of Clinical Oncology 2007 update of recommendations for the use of tumor markers in breast cancer. J Clin Oncol. 2007, 25 (33): 5287-5312. 10.1200/JCO.2007.14.2364.View ArticlePubMedGoogle Scholar
- Zujewski JA, Kamin L: Trial assessing individualized options for treatment for breast cancer: the TAILORx trial. Future Oncol. 2008, 4 (5): 603-610. 10.2217/147966184.108.40.2063.View ArticlePubMedGoogle Scholar
- Cardoso F, Van’t Veer L, Rutgers E, Loi S, Mook S, Piccart-Gebhart MJ: Clinical application of the 70-gene profile: the MINDACT trial. J Clin Oncol. 2008, 26 (5): 729-735. 10.1200/JCO.2007.14.3222.View ArticlePubMedGoogle Scholar
- Parker JS, Mullins M, Cheang MC, Leung S, Voduc D, Vickery T, Davies S, Fauron C, He X, Hu Z, Quackenbush JF, Stijleman IJ, Palazzo J, Marron JS, Nobel AB, Mardis E, Nielsen TO, Ellis MJ, Perou CM, Bernard PS: Supervised risk predictor of breast cancer based on intrinsic subtypes. J Clin Oncol. 2009, 27 (8): 1160-1167. 10.1200/JCO.2008.18.1370.View ArticlePubMedPubMed CentralGoogle Scholar
- Nielsen TO, Parker JS, Leung S, Voduc D, Ebbert M, Vickery T, Davies SR, Snider J, Stijleman IJ, Reed J, Cheang MC, Mardis ER, Perou CM, Bernard PS, Ellis MJ: A comparison of PAM50 intrinsic subtyping with immunohistochemistry and clinical prognostic factors in tamoxifen-treated estrogen receptor-positive breast cancer. Clin Cancer Res. 2010, 16 (21): 5222-5232. 10.1158/1078-0432.CCR-10-1282.View ArticlePubMedPubMed CentralGoogle Scholar
- Geiss GK, Bumgarner RE, Birditt B, Dahl T, Dowidar N, Dunaway DL, Fell HP, Ferree S, George RD, Grogan T, James JJ, Maysuria M, Mitton JD, Oliveri P, Osborn JL, Peng T, Ratcliffe AL, Webster PJ, Davidson EH, Hood L, Dimitrov K: Direct multiplexed measurement of gene expression with color-coded probe pairs. Nat Biotechnol. 2008, 26 (3): 317-325. 10.1038/nbt1385.View ArticlePubMedGoogle Scholar
- Reis PP, Waldron L, Goswami RS, Xu W, Xuan Y, Perez-Ordonez B, Gullane P, Irish J, Jurisica I, Kamel-Reid S: mRNA transcript quantification in archival samples using multiplexed, color-coded probes. BMC Biotechnol. 2011, 11: 46-6750. 10.1186/1472-6750-11-46. 11-46View ArticlePubMedPubMed CentralGoogle Scholar
- Dowsett M, Sestak I, Lopez-Knowles E, Sidhu K, Dunbier AK, Cowens JW, Ferree S, Storhoff J, Schaper C, Cuzick J: Comparison of PAM50 Risk of Recurrence Score With Oncotype DX and IHC4 for Predicting Risk of Distant Recurrence After Endocrine Therapy. J Clin Oncol. 2013, 31 (22): 2783-2790. 10.1200/JCO.2012.46.1558.View ArticlePubMedGoogle Scholar
- Gnant M, Filipits M, Mlineritsch B, Dubsky P, Jakesz R, Kwasny W, Fitzal F, Rudas M, Knauer M, Singer C, Greil R, Ferree S, Storhoff J, Cowens J, Schaper C, Liu S, Nielsen T: Clinical validation of the PAM50 risk of recurrence (ROR) score for predicting residual risk of distant-recurrence (DR) after endocrine therapy in postmenopausal women with ER + early breast cancer (EBC): An ABCSG study [abstract]. Cancer Res. 2012, 72 (24): P2-10-02.Google Scholar
- Gnant M, Filipits M, Dubsky P, Rudas M, Balic M, Greil R, Ferree S, Cowens J, Schaper C, Nielsen T: 53O_PR Predicting risk for late metastasis: The PAM50 risk of recurrence (ROR) score after 5 years of endocrine therapy in postmenopausal women with HR + early breast cancer: A study on 1,478 patients from the ABCSG-8 trial [abstract]. Ann Oncol. 2013, 24 (3): 29-doi:10.1093/annonc/mdt084.1Google Scholar
- Simon RM, Paik S, Hayes DF: Use of archived specimens in evaluation of prognostic and predictive biomarkers. J Natl Cancer Inst. 2009, 101 (21): 1446-1452. 10.1093/jnci/djp335.View ArticlePubMedPubMed CentralGoogle Scholar
- Teutsch SM, Bradley LA, Palomaki GE, Haddow JE, Piper M, Calonge N, Dotson WD, Douglas MP, Berg AO, EGAPP Working Group: The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative: methods of the EGAPP Working Group. Genet Med. 2009, 11 (1): 3-14. 10.1097/GIM.0b013e318184137c.View ArticlePubMedPubMed CentralGoogle Scholar
- Gnant M, Dowsett M, Filipits M, Lopez-Knowles E, Greil R, Balic M, Cowens J, Nielsen T, Shaper C, Sestak I, Fesl C, Cuzick J: Identifying clinically relevant prognostic subgroups in node-positive postmenopausal HR + early breast cancer patients treated with endocrine therapy: A combined analysis of 2,485 patients from ABCSG-8 and ATAC using the PAM50 risk of recurrence (ROR) score and intrinsic subtype [abstract]. J Clin Oncol. 2013, 31: 506-Google Scholar
- Tholen D, Kallner A, Kennedy J, Krouwer J, Meier K: Clinical and Laboratory Standards Institute (CLSI). Evaluation of precision performance of quantitative measurement methods; approved guideline – second edition. CLSI document EP05-A2. 950 west valley road, suite 2500. 2004, Wayne, PA, USA: CLSIGoogle Scholar
- NanoString Technologies Inc: Prosigna™ Breast Cancer Prognostic Gene Signature Assay [Package Insert]. 2013, Seattle, WA: NanoString Technologies, IncGoogle Scholar
- Majidzadeh-A K, Esmaeili R, Abdoli N: TFRC and ACTB as the best reference genes to quantify Urokinase Plasminogen Activator in breast cancer. BMC Res Notes. 2011, 4: 215-0500. 10.1186/1756-0500-4-215. 4-215View ArticlePubMedPubMed CentralGoogle Scholar
- Szabo A, Perou CM, Karaca M, Perreard L, Palais R, Quackenbush JF, Bernard PS: Statistical modeling for selecting housekeeper genes. Genome Biol. 2004, 5 (8): R59-10.1186/gb-2004-5-8-r59.View ArticlePubMedPubMed CentralGoogle Scholar
- Baker SC, Bauer SR, Beyer RP, Brenton JD, Bromley B, Burrill J, Causton H, Conley MP, Elespuru R, Fero M, Foy C, Fuscoe J, Gao X, Gerhold DL, Gilles P, Goodsaid F, Guo X, Hackett J, Hockett RD, Ikonomi P, Irizarry RA, Kawasaki ES, Kaysser-Kranich T, Kerr K, Kiser G, Koch WH, Lee KY, Liu C, Liu ZL, Lucas A, Manohar CF, Miyada G, Modrusan Z, Parkes H, Puri RK, Reid L, Ryder TB, Salit M, Samaha RR, Scherf U, Sendera TJ, Setterquist RA, Shi L, Shippy R, Soriano JV, Wagar EA, Warrington JA, Williams M, Wilmer F, Wilson M, Wolber PK, Wu X, Zadro R, External RNA Controls Consortium: The External RNA Controls Consortium: a progress report. Nat Methods. 2005, 2 (10): 731-734. 10.1038/nmeth1005-731.View ArticlePubMedGoogle Scholar
- Warrington J, Corbisier P, Feilotter H, Hackett J, Reid L, Salit M, Wagar E, Williams P, Wolber P: Clinical and Laboratory Standards Institute (CLSI). Use of External RNA Controls in Gene Expression Assays; Approved Guideline. CLSI Document MM16-A. 940 west valley road, suite 1400. 2006, Wayne, PA, USA: CLSIGoogle Scholar
- Berger R, Hsu J: Bioequivalence trials, intersection-union tests and equivalence confidence sets. Stat Sci. 1996, 11: 283-319.View ArticleGoogle Scholar
- Cronin M, Sangli C, Liu ML, Pho M, Dutta D, Nguyen A, Jeong J, Wu J, Langone KC, Watson D: Analytical validation of the Oncotype DX genomic diagnostic test for recurrence prognosis and therapeutic response prediction in node-negative, estrogen receptor-positive breast cancer. Clin Chem. 2007, 53 (6): 1084-1091. 10.1373/clinchem.2006.076497.View ArticlePubMedGoogle Scholar
- Nuyten DS, Hastie T, Chi JT, Chang HY, van de Vijver MJ: Combining biological gene expression signatures in predicting outcome in breast cancer: An alternative to supervised classification. Eur J Cancer. 2008, 44 (15): 2319-2329. 10.1016/j.ejca.2008.07.015.View ArticlePubMedGoogle Scholar
- Chia SK, Bramwell VH, Tu D, Shepherd LE, Jiang S, Vickery T, Mardis E, Leung S, Ung K, Pritchard KI, Parker JS, Bernard PS, Perou CM, Ellis MJ, Nielsen TO: A 50-gene intrinsic subtype classifier for prognosis and prediction of benefit from adjuvant tamoxifen. Clin Cancer Res. 2012, 18 (16): 4465-4472. 10.1158/1078-0432.CCR-12-0286.View ArticlePubMedPubMed CentralGoogle Scholar
- Cheang MC, Voduc KD, Tu D, Jiang S, Leung S, Chia SK, Shepherd LE, Levine MN, Pritchard KI, Davies S, Stijleman IJ, Davis C, Ebbert MT, Parker JS, Ellis MJ, Bernard PS, Perou CM, Nielsen TO: Responsiveness of intrinsic subtypes to adjuvant anthracycline substitution in the NCIC.CTG MA.5 randomized trial. Clin Cancer Res. 2012, 18 (8): 2402-2412. 10.1158/1078-0432.CCR-11-2956.View ArticlePubMedPubMed CentralGoogle Scholar
- Jorgensen C, Nielsen T, Bjerre K, Liu S, Wallden B, Balslev E, Nielsen D, Ejlertsen B: PAM50 breast cancer intrinsic subtypes and effect of gemcitabine in advanced breast cancer patients. Acta Oncol. 2013Google Scholar
- Filipits M, Rudas M, Jakesz R, Dubsky P, Fitzal F, Singer CF, Dietze O, Greil R, Jelen A, Sevelda P, Freibauer C, Muller V, Janicke F, Schmidt M, Kolbl H, Rody A, Kaufmann M, Schroth W, Brauch H, Schwab M, Fritz P, Weber KE, Feder IS, Hennig G, Kronenwett R, Gehrmann M, Gnant M, EP Investigators: A new molecular predictor of distant recurrence in ER-positive, HER2-negative breast cancer adds independent information to conventional clinical risk factors. Clin Cancer Res. 2011, 17 (18): 6012-6020. 10.1158/1078-0432.CCR-11-0926.View ArticlePubMedGoogle Scholar
- Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group: Recommendations from the EGAPP Working Group: can tumor gene expression profiling improve outcomes in patients with breast cancer?. Genet Med. 2009, 11 (1): 66-73.View ArticleGoogle Scholar
- Kronenwett R, Bohmann K, Prinzler J, Sinn BV, Haufe F, Roth C, Averdick M, Ropers T, Windbergs C, Brase JC, Weber KE, Fisch K, Muller BM, Schmidt M, Filipits M, Dubsky P, Petry C, Dietel M, Denkert C: Decentral gene expression analysis: analytical validation of the Endopredict genomic multianalyte breast cancer prognosis test. BMC Cancer. 2012, 12: 456-2407. 10.1186/1471-2407-12-456. 12-456View ArticlePubMedPubMed CentralGoogle Scholar
- Elloumi F, Hu Z, Li Y, Parker JS, Gulley ML, Amos KD, Troester MA: Systematic bias in genomic classification due to contaminating non-neoplastic tissue in breast tumor samples. BMC Med Genomics. 2011, 4 (54): 8794-4-54.Google Scholar
- Mee BC, Carroll P, Donatello S, Connolly E, Griffin M, Dunne B, Burke L, Flavin R, Rizkalla H, Ryan C, Hayes B, D’Adhemar C, Banville N, Faheem N, Muldoon C, Gaffney EF: Maintaining Breast Cancer Specimen Integrity and Individual or Simultaneous Extraction of Quality DNA, RNA, and Proteins from Allprotect-Stabilized and Nonstabilized Tissue Samples. Biopreserv Biobank. 2011, 9 (4): 389-398. 10.1089/bio.2011.0034.View ArticlePubMedPubMed CentralGoogle Scholar
- Graham K, Ge X, De Las MA, Tripathi A, Rosenberg CL: Gene expression profiles of estrogen receptor-positive and estrogen receptor-negative breast cancers are detectable in histologically normal breast epithelium. Clin Cancer Res. 2011, 17 (2): 236-246. 10.1158/1078-0432.CCR-10-1369.View ArticlePubMedGoogle Scholar
- Clare S, Pardo I, Mathieson T, Lillemoe H, Blosser R, Choi M, Sauder C, Doxey D, Badve S, Storniolo A, Atale R, Radovich M: “Normal” tissue adjacent to breast cancer is not normal [abstract]. Cancer Res. 2012, 72 (24 Suppl): P1-03-02.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2407/14/177/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.