Analytic performance studies and clinical reproducibility of a real-time PCRassay for the detection of epidermal growth factor receptor gene mutations informalin-fixed paraffin-embedded tissue specimens of non-small cell lungcancer
BMC Cancer volume 13, Article number: 210 (2013)
Epidermal growth factor receptor (EGFR) gene mutations identify patients withnon-small cell lung cancer (NSCLC) who have a high likelihood of benefitingfrom treatment with anti-EGFR tyrosine kinase inhibitors. Sanger sequencingis widely used for mutation detection but can be technically challenging,resulting in longer turn-around-time, with limited sensitivity for lowlevels of mutations. This manuscript details the technical performanceverification studies and external clinical reproducibility studies of thecobas EGFR Mutation Test, a rapid multiplex real-time PCR assay designed todetect 41 mutations in exons 18, 19, 20 and 21.
The assay’s limit of detection was determined using 25 formalin-fixedparaffin-embedded tissue (FFPET)-derived and plasmid DNA blends. Assayperformance for a panel of 201 specimens was compared against Sangersequencing with resolution of discordant specimens by quantitative massivelyparallel pyrosequencing (MPP). Internal and external reproducibility wasassessed using specimens tested in duplicate by different operators, usingdifferent reagent lots, instruments and at different sites. The effects onthe performance of the cobas EGFR test of endogenous substances and ninetherapeutic drugs were evaluated in ten FFPET specimens. Other testsincluded an evaluation of the effects of necrosis, micro-organisms andhomologous DNA sequences on assay performance, and the inclusivity of theassay for less frequent mutations.
A >95% hit rate was obtained in blends with >5% mutant alleles, asdetermined by MPP analysis, at a total DNA input of 150 ng. The overallpercent agreement between Sanger sequencing and the cobas test was 96.7%(negative percent agreement 97.5%; positive percent agreement 95.8%). Assayrepeatability was 98% when tested with two operators, instruments, andreagent lots. In the external reproducibility study, the agreementwas > 99% across all sites, all operators and all reagentlots for 11/12 tumors tested. Test performance was not compromised byendogenous substances, therapeutic drugs, necrosis up to 85%, and commonmicro-organisms. All of the assessed less common mutations except one (exon19 deletion mutation 2236_2248 > AGAC) were detected at asimilar DNA input level as that for the corresponding predominantmutation.
The cobas EGFR Mutation Test is a sensitive, accurate, rapid, andreproducible assay.
Lung cancer has the highest incidence of all solid organ cancers and is the mostcommon cause of death from cancer worldwide, accounting for over 1.6 million newcases annually and 1.38 million deaths . Almost 85% of all lung cancers are non-small cell lung cancer (NSCLC).The observation that the epidermal growth factor receptor (EGFR) is over-expressedin most cases of NSCLC led to the development of the specific anti-EGFR tyrosinekinase inhibitors (TKIs) gefitinib and erlotinib as targeted therapeutic agents.However, clinical trials with these agents revealed that in most cases, respondersharbored specific activating mutations in exons 18–21 which collectivelyencode the kinase domain of the EGFR gene [2–5]. The majority of mutations that have been associated with sensitivity togefitinib and erlotinib are located in exon 19 (45%) and exon 21 (40–45%),although ~5% are located in exon 18 and <1% in exon 20 . In addition, certain mutations in exon 20, such as T790M, predictresistance to these TKIs .
The association between sensitizing mutations in the EGFR gene and responseto treatment has led to recommendations by major oncology organizations that NSCLCtumors should be tested for the presence of these mutations before treatment [8–10]. Thus, from a practical perspective, optimal care of patients will dependon interactions between a patient’s pulmonologist and oncologist, relying oninformation from the molecular pathology of the tumor tissue . In recognition of the need for accurate testing, these organizationshave started consolidating guidelines for molecular testing in lung cancer to followstandards for sensitivity, specificity, and time to results to ensure quality ofpatient treatment .
As with other tumor types, diagnostic assays should be optimized for use withformalin-fixed paraffin-embedded tissue (FFPET) specimens, which continue torepresent the vast majority of NSCLC samples in clinical practice today. Moleculartesting in NSCLC poses particular challenges for the pathologist and clinicianalike. In many cases the amount of tumor tissue available for testing (e.g.bronchial biopsy) is very limited and, given the growing number of molecular andimmunohistochemical studies that are performed as part of the diagnostic workup,there are competing diagnostic demands for the small amount of available material.Thus, an optimal diagnostic test should require a small amount of DNA. Furthermore,many patients with metastatic NSCLC are often quite ill and require promptinitiation of targeted therapy when indicated, making a rapid molecular assay highlydesirable.
The importance of using standardized techniques for both extraction and molecularanalysis was stressed by a recently convened expert working group who discussed thechallenges of NSCLC diagnosis in the current era . This group recommended against using laboratory developed tests, as suchmethods are subject to great inter- and intra-laboratory variability and do notalways pass adequate quality control schemes that ensure reproducibility of results.Instead, the group recommends, where possible, using certified diagnostic kits withprior laboratory validation.
We designed a highly sensitive, specific, reproducible test that detects mutations inexons 18, 19, 20, and 21 in tumor samples from patients with NSCLC to identifyindividuals who are most likely to respond to EGFR TKI therapy using one 5-microntissue section. Here, we present the technical performance verification studies ofthe cobas EGFR Mutation Test, including studies of the analytic sensitivity,internal and external reproducibility, minimal tumor content, interferingsubstances, effects of necrosis, and cross-reactivity with other mutations.
FFPET specimens of NSCLC tumors were obtained from US and European commercialvendors: Analytical Biological Services Inc. (Wilmington, DE, USA), Asterand,Inc. (Detroit, MI, USA), BioServe (Beltsville, MD, USA), Conversant (Huntsville,AL, USA), Cureline Inc. (South San Francisco, CA, USA), Cytomyx (Lexington, MA,USA), Discovery Life Sciences, Inc. (Los Osos, CA, USA), ILSBio, LLC(Chestertown, MD, USA), Indivumed (Hamburg, Germany), OriGene (Rockville, MD,USA), and ProteoGenex (Culver City, CA, USA). In addition, FFPET specimens ofNSCLC tumors were provided by Astellas Pharma US, Inc. (Deerfield, IL, USA). Allspecimens were aged between 3 and 10 years.
Human epidermal growth factor receptor (HER) plasmids: HER2, HER3, and HER4 werepurchased from Integrated DNA Technologies (San Diego, CA, USA).
K562, human genomic DNA, used as wild-type sequence, was obtained from the humanlymphoma cell line K562 (Promega, Madison WI; part number DD201X).
Methods described below were not used in the diagnosis or treatment of anypatients. Patient consent forms were obtained through the commercial vendor. RMSand the principal investigators from the external reproducibility study abidedby the International Conference on Harmonisation Good Clinical PracticeGuidelines and regulations of the US Food and Drug Administration (FDA) in theconduct of this study. Before the start of the study, the protocol and otherdocuments necessary for participating sites to perform the study was submittedto an independent Institutional Review Board (IRB) in accordance with FDA andlocal legal requirements. IRB approval was obtained at each site.
cobas EGFR mutation test
The cobas EGFR Mutation Test (“cobas EGFR test”, Roche MolecularSystems, Inc, Branchburg, NJ, USA) is a CE-IVD marked multiplex assay that usesallele-specific polymerase chain reaction (AS-PCR) to detect 41 mutations inexons 18, 19, 20, and 21 in FFPET specimens of human NSCLC. The test consists oftwo major steps: (1) a manual DNA isolation step, and (2) PCR amplification anddetection of target DNA using complementary primer pairs and oligonucleotideprobes labeled with fluorescent dyes (Figure 1). Thetest is designed to detect G719X (G719A, G719C, and G719S) in exon 18; deletionsand complex mutations in exon 19; S768I, T790M, and insertions in exon 20; andL858R in exon 21. The specific mutations detected by the assay are detailed inTable 1. A mutant control and a negative controlare included in each run to confirm the validity of the run. The test uses 150ng total input DNA, an amount which can typically be extracted from a single 5μm section of an FFPET specimen using the cobas DNA Sample Preparation Kit(Roche Molecular Systems, Inc, Branchburg, NJ, USA), following the standardpackage insert protocol . Macrodissection of the tissue section is only required if theestimated tumor content is < 10% by pathological assessment.Mutation analysis is performed through real-time PCR analysis using the cobas4800 System (version 2.0), and the analysis of raw data and reporting of resultsare fully automated. The DNA isolation, amplification and detection, and resultreporting can be performed in less than 8 hours. Testing for this study wasconducted with cobas 4800 System Software (version 18.104.22.1688). Analysis wasperformed with the EGFR Analysis Package Software V.1.0
DNA from FFPET specimens using the same extraction method as described for thecobas EGFR test were extracted and amplified at Roche Molecular Systems, andsent out for 2× bidirectional Sanger sequencing (Sanger) by a ClinicalLaboratory Improvement Amendments (CLIA)-certified laboratory (SeqWright,Houston, TX, USA) using a validated protocol.
Quantitative massively parallel pyrosequencing
NSCLC FFPET-derived DNA blends and specimens that gave discordant cobas EGFR testand Sanger test results as well as a randomly selected subset of concordantspecimens were tested using a quantitative massively parallel pyrosequencingmethod (“MPP”, 454 GS Titanium, 454 Life Sciences, Branford, CT,USA) . DNA was extracted and amplified at Roche Molecular Systems prior tobeing sent to a CLIA-certified laboratory (SeqWright, Houston, TX, USA) to besequenced using a validated protocol for EGFR mutation detection. The analyticalsensitivity of MPP for EGFR mutations was validated to a limit of detection of1.25%.
The analytical sensitivity of the cobas EGFR test was assessed usingFFPET-derived DNA blends and plasmid DNA blends. For the FFPET DNA blends,seventeen NSCLC FFPET specimens were selected for their mutation status (threepositive for exon 19 deletions; three positive for L858R mutations; one positivefor L858R and T790M mutations; one positive for S768I and G719C mutations; onepositive for G719A mutation; one positive for an exon 20 insertion mutation; andseven EGFR wild-type specimens) as determined by Sanger sequencing.Specimen blends were prepared targeting approximately 10%, 5%, 2.5%, and 1.25%mutant DNA as quantified by MPP pyrosequencing. Serial dilutions of eachspecimen were prepared and eight replicates were tested with three cobas EGFRtest reagent lots, yielding a total of 24 replicates per panel member.
Six plasmid constructs containing the most frequently observed mutation for eachmutation group detected by the test were blended with K562 wild-type DNA suchthat each sample contained a ~5% blend of mutant plasmid at the copy numberequivalent of 50 ng/PCR. Serial dilutions of each specimen were prepared to makepanels with five members. DNA samples were diluted while leaving the percentmutation constant. An additional panel member containing 100% wild-type DNA wasincluded to each panel. Each of the six levels of the six plasmid DNA blendspecimens was tested with each of three unique cobas EGFR test lots. Threedilutions were formulated for each plasmid in each mutation group for each ofthe three reagent kit lots. Eight replicates of each of the three dilutionseries was tested for each of the three kit lots, yielding a total ofseventy-two replicates per panel member.
Analytical performance of the cobas EGFR test was compared against 2×bidirectional Sanger sequencing using 201 FFPET human NSCLC specimens.Correlation between the two methods was assessed by agreement analysis,including positive percent agreement (PPA), negative percent agreement (NPA),and overall percent agreement (OPA). Specimens with invalid results on eithermethod were excluded from the correlation analysis. The cobas EGFR test resultswere considered invalid if any or all of the mutation calls were reported asinvalid. Sanger sequencing results were considered invalid if any or all of thefour exons failed to provide a valid result for a specimen. Sanger sequencingresults were considered invalid if any or all of the four exons failed toprovide a valid result for a specimen. Specimens with discordant cobas EGFR testand Sanger sequencing results and a randomly selected subset of specimens withconcordant results were subjected to MPP.
The internal repeatability of the cobas EGFR test was evaluated using six NSCLCFFPET specimens: two EGFR wild-type and four EGFR-mutationpositive specimens (one exon 19 deletion, one with G719X and S768I mutations,one with L858R and T790M mutations, and one with exon 20 insertion mutation).Testing was performed in duplicate by two operators, using two different reagentlots and two cobas z 480 analyzers over 4 days. Each operator performed one runper reagent lot per day for 4 days, giving a total of 16 runs and 32 replicateresults for each of the six specimens.
The external reproducibility of the cobas EGFR test across three clinicallaboratories using three reagent lots and two operators per site was evaluatedusing a 13-member panel of NSCLC specimens containing five different deletionsin exon 19 and exon 21 L858R in the EGFR gene (Table 2). Operators performed blinded runs (two replicates of each panelmember/run), on five nonconsecutive days, using a single instrument per site. Atotal of 180 DNA replicate specimens were prepared for each panel member, andeach of the three sites performed a total of 780 tests. All statistical analyseswere performed using SAS®/STAT® software. The sample identificationnumbers of panel members were randomized using PROC PLAN in SAS, and the orderof panel members within each run was randomized. Only valid tests from validruns were included in the statistical analyses.
The mutation status and percent mutant alleles of the specimens chosen for therepeatability and reproducibility studies was determined using Sanger sequencingand MPP, respectively, and tumor content was estimated for each specimen byassessment from an external pathologist. DNA was extracted and blended to createsamples containing levels of EGFR mutation above the limit of detection for thecobas platform. The percentage of mutant alleles in the blended samples wasverified by MPP.
Potential interfering substances
The effects on the performance of the cobas EGFR test from two endogenoussubstances (hemoglobin and triglycerides) and nine therapeutic drugs that may bepresent in human NSCLC specimens (albuterol, ipratropium, fluticasone,ceftazidime, imipenem, piperacillin-tazobactam, cilastatin sodium, povidoneiodide, and lidocaine) were investigated with 10 NSCLC FFPET specimens.Specimens were selected for mutation status based on Sanger and/or MPP. Fivespecimens were EGFR mutation-positive and five were wild type.Specimens were tested in the absence and presence of each potential interferent.Each potential interferent was spiked during the lysis step. Hemoglobin andtriglycerides were added to achieve 1× the upper limit of normalconcentration seen in common pathological conditions (as defined by the Clinicaland Laboratory Standards Institute [CLSI] EP7-A2 Guideline; 2 g/L hemoglobin and37 mM triglycerides) . The therapeutic drugs were added to achieve a final concentration of3× the maximal plasma concentration (as defined by the CLSI EP7-A2Guideline) , if known. Povidone iodide was tested as a 10% weight by volumesolution; lidocaine was tested at a concentration of 12 μg/mL, asrecommended by the CLSI EP7-A2 Guideline .
Effects of necrosis
The impact of tissue necrosis on the cobas EGFR test detection of mutations wasevaluated. Twenty NSCLC FFPET specimens were tested in duplicate: ten specimenscovering a range of percent mutation from the exon 19 deletion, S768I, L858R,G719X, and exon 20 insertion mutation groups, and ten wild-type specimens.Percent necrosis, as assessed by a pathologist, varied from 0% to 60% for mutantspecimens and from 5% to 85% for wild-type specimens.
To confirm that other gene sequences homologous to the targeted EGFRexons do not interfere with the performance of the cobas EGFR test, potentialcross-reactivity was assessed for three members of the ErbB family of receptortyrosine kinases (HER2, HER3, and HER4). The homologous sequences in HER2, HER3,and HER4 corresponding to the probe-targeted portions of exons 18, 19, 20, and21 in the EGFR gene were individually cloned into 12 plasmids (fourexon regions per HER gene) and evaluated with the cobas EGFR test. Wealso sought to determine if the assay, which is designed to detect 29 deletionsin exon 19 would also detect the rare exon 19 L747S point mutation, using aplasmid containing this mutation. Ten NSCLC FFPET specimens (four with EGFRmutations, six wild type) were evaluated in the presence (spiked to aconcentration of 15,850 copies/PCR well, the equivalent of 50 ng of genomic DNA)and absence of each of the HER plasmids as well as the plasmid containing theL747S mutation. The plasmids were spiked into individual replicates of each ofthe ten specimens after extraction; one replicate of each of the 10 specimenswas not spiked with plasmid and was used as the control.
To assess the inclusivity of the assay for mutations in all four key exons ofEGFR (exons 18–21), the detection of less common non-predominantEGFR mutations was studied for each of the four exons (G719X pointmutations in exon 18, deletions in exon 19 deletions, insertions in exon 20, anda two base pair mutation that yields variant in the L858R mutation in exon 21).Plasmid constructs containing these less common mutations were blended withwild-type DNA (K562). The initial plasmid DNA input level was determined by thefindings from the analytical sensitivity study for the predominant mutation (asdetailed above). If the hit rate at this level was too low, then the nexthighest DNA input level was tested, with levels subsequently increased up amaximum of 50 ng/PCR. Each plasmid DNA blend sample was tested with one test kitlot, and a total of 24 replicates were tested per sample.
Ten NSCLC FFPET specimens (five mutation positive, five wild-type) were testedwith two common respiratory microorganisms (Haemophilus influenzae andStreptococcus pneumoniae), Controls (normal substance level which didnot contain any added organism) were used for all specimens. Microorganisms werespiked at 1e6 CFU/mL. A total of 30 test conditions were run.
The analytical sensitivity of the cobas EGFR test for exon 19 deletion, L858R,S768I, T790M, G719X, and exon 20 insertion mutations was assessed using NSCLCFFPET-derived DNA blends and six plasmid DNA blends. For the FFPET-derived DNAblends the lowest percent mutation level that was associated with ≥95% hitrate with 50 ng/PCR reaction ranged from 1.3% to 5.6% (Table 3). For the plasmid blends, the amount of DNA in 5% copy equivalentto achieve ≥ 95% mutation detected rate ranged from 0.78 and3.13 ng/PCR reaction (Table 4). Together, the datashow that the cobas EGFR test can detect the predominant mutation for each ofthe six mutation groups when it is present as 5% mutant alleles.
Method correlation and test failure rate
Of the 201 specimens evaluated in the methods correlation between the cobas EGFRtest and Sanger sequencing, 49 specimens gave invalid test results for one orboth methods produced an invalid result. Forty-eight specimens were invalid bySanger sequencing (23.8%). Six specimens (3.0%) were invalid by cobas EGFR testusing reagent lot 1 (5/6 of these specimens were also invalid by Sanger), andfive specimens (2.5%) were invalid using reagent lot 2 (4/5 specimens were alsoinvalid for Sanger).
The comparison of the remaining 152 valid results is shown in Table 5. The OPA between both cobas EGFR test lots and Sangersequencing was 96.7%, with five discordant specimens for each lot. All specimensyielding discordant resultants with either reagent lot were further analyzed byMPP. Discordant analysis results are listed in Table 6. Sanger sequencing detected two mutation calls (one G719A, oneexon 19 deletion) that were not confirmed by the cobas EGFR test or MPP. Twospecimens designated “mutation not detected” by Sanger were detectedby MPP (exon 19 deletion, exon 20 insertion). Both lots of the cobas EGFR testcalled one specimen “mutation not detected” that was called as G719Sby MPP at 1.1% mutation, which is below the 5% limit of detection of the cobasEGFR test. One specimen was detected as an exon 19 deletion by cobas EGFR lot 2,but not detected for both Sanger and cobas EGFR lot 1. This specimen wasdetected as an exon 19 deletion at 3% mutation by MPP, which is below the limitof detection of the cobas EGFR test. Lastly, cobas EGFR test lot 1 detected onespecimen with an exon 20 insertion. This specimen was called “mutation notdetected” by Sanger sequencing, cobas EGFR test lot 2, and MPP.
Internal Repeatability/External Reproducibility
All runs from the internal repeatability analysis were valid across allspecimens, reagent lots, operators, and instruments combined. A single replicateof one specimen gave an invalid result. The specimen was repeated and the validresult replaced the invalid result, which was excluded from data analysis.Initially six (6) false calls out of 192 specimens were observed generating atotal percent accuracy of 96.9%. Two of the results were resolved to confirm theobserved result by the cobas EGFR test. Three of the false calls were confirmedby MPP; the L858R false call was not confirmed by MPP. With two of the six falsecalls resolved the assay delivered 188 correct calls out of 192 specimenstested, or an accuracy of 97.9%.
In the external reproducibility study, a total of 2,340 tests were performed onthe 13 panel members in 90 valid runs (see Table 2for list of panel member. No invalid results were obtained. No false positiveresults were observed, as all 180 replicates of wild-type specimens (95% CI[98–100%]) gave a Mutation Not Detected result. For the exon 19 and exon21 panel members with 5% mutation, one panel member (EX19_2240_2257del18) had ahit rate below 95% (62.8%, -95% CI [55.3–69.9%]),This may have been due topoor DNA quality in the tumor block used. Although this panel member appeared tohave a lower than 95% hit rate, the Ctr SD and CV(%) for this panel member werewithin the range of the remaining panel members. For all exon 19 and exon 21panel members with ≤10% mutation had 99.4% (95% CI [96.9–100])agreement. Overall the external reproducibility study showed little variation inthe cobas EGFR test performance at multiple clinical sites (Table 7).
Interference/Cross-Reactivity/Effects of necrosis
No interference was observed for hemoglobin and triglycerides at CLSI-recommendedtest concentrations of 2 g/L and 37 mM for any of the 10 FFPET specimens. Nointerference by therapeutic drugs was observed on the performance of the cobasEGFR test.
No interference from necrotic tissue was observed when evaluating the performanceof the cobas EGFR test. Results for all specimens were concordant with Sangersequencing and MPP results. Thus, levels of necrosis up to 85% did not affecttest performance.
Results for the ten FFPET specimens tested under the 13 conditions using thecobas EGFR test matched the expected results for HER2/3/4 cross-reactivity. Onespecimen that was spiked with the HER4 exon 21 analog plasmid initially produceda result of “Mutation Not Detected”, but yielded the correct callupon retesting. The plasmid with the exon 19 L747S mutation yielded an exon 19deletion call in all specimens that did not already contain an exon 19 deletion,confirming cross-reactivity between the L747S mutation and the cobas EGFR test.The BLAST (Basic Local Alignment Search Tool) results demonstrated that theprimers and probes in the cobas EGFR test are unlikely to cross-hybridize withsequences other than the target sequence. Analogous sequences to the targetedEGFR exons from the HER2, HER3, and HER4 genes did not interfere with theperformance of the cobas EGFR test.
Results are presented in Additional file 1: Table S1.All of the assessed less common mutations except one (exon 19 deletion mutation2236_2248 > AGAC) were detected at a similar DNA input level asthat for the corresponding predominant mutation. The exon 19 deletion mutation2236_2248 > AGAC was not consistently detected at any DNA inputlevel.
Neither Haemophilus influenzae nor Streptococcus pneumoniae hadany effect on the performance of the cobas EGFR test (data not shown).
There is a pressing clinical need for a well-validated EGFR testing method withoptimal analytical performance, turnaround time, using the least amount ofdifficult-to-obtain patient specimens. There is also a clear need for guidelinessurrounding method performance characteristics. Here, we present results on sevenout of 25 analytical validation studies performed on over 200 clinical FFPETspecimens as well as external reproducibility study of the test run at multipleclinical sites. It is important to note that validation studies were performed onplasmid specimens as well as FFPET specimens, allowing an accurate understanding theof test performance in typical clinical specimens. Performance of the test inalternative specimen types is currently being conducted.
One commonly used method for interrogating mutations in the EGFR gene is Sangersequencing. Sanger sequencing is highly variable based on lab-validated protocols.In some cases, Sanger sequencing takes up to 600 ng of DNA to interrogate all 4exons in the EGFR gene . Particularly in the field of NSCLC, where patient samples are difficultto obtain and testing (molecular and immunohistochemical) is being prioritized fortreatment decisions, the efficient use of limited specimen is of great importance.The cobas EGFR test detects 41 mutations in exons 18, 19, 20, and 21 and uses 150ngof total DNA input. The studies described in this manuscript indicate that the cobasEGFR test is able to detect mutations in EGFR exons 18, 19, 20, and 21 at ≥5%mutation level using only 50 ng of DNA per reaction well, an amount that typicallycan be extracted from a single 5 μm curl. The cobas EGFR test was able todetect mutations that were confirmed by MPP but not detected by Sanger sequencing.The increased sensitivity of the cobas EGFR test is consistent with previous studiesof other PCR-based mutation assays [17–19]. The sensitivity of Sanger sequencing may be increased to some extent bytaking measures to enrich for tumor tissue, such as macrodissection or lasermicrocapture. However, these measures require extra time and effort on the part ofthe pathologist, and in some cases require the use of specialized equipment. Bycontrast, the cobas EGFR test does not require macrodissection unless the estimatedtumor content in the specimen is below 10%.
To confirm the greater sensitivity of the cobas EGFR test compared to Sanger, a thirdcomparator method was used, MPP. To eliminate any sequencing bias, both Sangersequencing and MPP were performed by an external laboratory that was blinded to theresults of the cobas EGFR test. In the four of six cases, MPP confirmed the cobasEGFR test result. The Sanger sequencing provided two false positive mutation callsand two false negative mutation calls, which in the clinical setting would haveresulted in two patients who would be unlikely to respond to treatment, receivingtreatment, and patients who would benefit from treatment being denied theintervention. Occasional false positive results with Sanger sequencing have beenobserved in other studies [17, 20, 21], perhaps reflecting some inherent subjectivity in the interpretation ofSanger sequencing results. Such subjectivity is eliminated from the cobas EGFR test,as the analysis and reporting of results are fully automated.
Low invalid rates expedite time to result and avoiding the unnecessary use ofadditional specimens for retesting. Of interest, the low invalid rates were observeddespite the samples being between 3 and 10 years old. The studies also show that thecobas EGFR test is more robust than Sanger sequencing with a lower invalid test rate(3% for cobas vs 23.8% for Sanger). Very few reported method comparison studies havecompared invalid test rates between different assay methods. However, we havepreviously demonstrated very low invalid test rates for other mutation assays onthis platform [17, 20].
A further benefit of the cobas EGFR test is its rapid turnaround time (~1 day for 24samples; 1 kit), which is considerably shorter than for Sanger sequencing (~5 days).The slower turnaround time for Sanger sequencing and its higher invalid test rate,which potentially results in the need for reanalysis, could lead to important delaysin patients receiving appropriate treatment for NSCLC. This is an important concernas the majority of patients present with advanced, disseminated disease . This rapid and sensitive method enables efficient testing of limitedtissue specimens, where patient samples are difficult to obtain and moleculartesting must be prioritized for treatment algorithms.
As part of the validation of the cobas EGFR test we examined both internalrepeatability and external reproducibility. In the internal repeatability analysis,the cobas EGFR test had high accuracy (98%) across all specimens, reagent lots,operators, and instruments combined. High reproducibility was observed in theexternal reproducibility analysis although one sample was observed to contribute adisproportionate amount to the variability observed. This sample had 5% mutation;however, analysis at ≤10% improved reproducibility to >97%. An evaluationof EGFR testing in 15 French centers showed low concordance between sites,ranging from median kappa values of 0.47 (0.45-0.49) for Exon 19 and 21,underpinning the critical need to set standards for EGFR mutation testing [8, 23]. The external reproducibility study is targeted for submission alongsideresults from clinical trial entitled, “Phase III Study (Tarceva®) vsChemotherapy to Treat Advanced Non-Small Cell Lung Cancer (NSCLC) in Patients WithMutations in the TK Domain of EGFR” (clinical trial # NCT00446225). Theclinical utility of the cobas EGFR test was assessed through a retrospectiveanalysis of specimens from the EURTAC trial (clinical trial # NCT00446225). Thoughthere has been consideration of the use of next generation sequencing in routineclinical diagnostics, for the accurate selection of patient therapy, method oftesting for EGFR mutations should be well validated both clinically andanalytically.
Our study also demonstrated that a variety of potential interfering substances– including endogenous substances, common medications, and respiratorymicroorganisms – had no significant effect on the assay’s analyticperformance. A thorough understanding of the specimen attributes that could affect amolecular assay are a key component of test optimization and validation.
The analytic studies presented here show that the cobas EGFR test is a sensitive,accurate, rapid, and reproducible assay for EGFR mutations that allowsclinicians to identify those patients with advanced NSCLC who have a high likelihoodof benefiting from treatment with anti-EGFR TKI therapies.
Epidermal growth factor receptor
Non-small cell lung cancer
Formalin-fixed paraffin-embedded tissue
Massively parallel pyrosequencing
Overall percent agreement
Tyrosine kinase inhibitors
Allele-specific polymerase chainreaction.
Jemal A, Bray F, Center MM, Ferlay J, Ward E, Forman D: Global cancer statistics. Cancer J Clin. 2011, 61 (2): 69-90. 10.3322/caac.20107.
Hirsch FR, Varella-Garcia M, Bunn PA, Franklin WA, Dziadziuszko R, Thatcher N, Chang A, Parikh P, Pereira JR, Ciuleanu T: Molecular predictors of outcome with gefitinib in a phase IIIplacebo-controlled study in advanced non-small-cell lung cancer. J Clin Oncol. 2006, 24 (31): 5034-5042. 10.1200/JCO.2006.06.3958.
Fukuoka M, Wu YL, Thongprasert S, Sunpaweravong P, Leong SS, Sriuranpong V, Chao TY, Nakagawa K, Chu DT, Saijo N: Biomarker analyses and final overall survival results from a phase III,randomized, open-label, first-line study of gefitinib versuscarboplatin/paclitaxel in clinically selected patients with advancednon-small-cell lung cancer in Asia (IPASS). J Clin Oncol. 2011, 29 (21): 2866-2874. 10.1200/JCO.2010.33.4235.
Rosell R, Carcereny E, Gervais R, Vergnenegre A, Massuti B, Felip E, Palmero R, Garcia-Gomez R, Pallares C, Sanchez JM: Erlotinib versus standard chemotherapy as first-line treatment for Europeanpatients with advanced EGFR mutation-positive non-small-cell lung cancer(EURTAC): a multicentre, open-label, randomised phase 3 trial. Lancet Oncol. 2012, 13 (3): 239-246. 10.1016/S1470-2045(11)70393-X.
Zhou C, Wu YL, Chen G, Feng J, Liu XQ, Wang C, Zhang S, Wang J, Zhou S, Ren S: Erlotinib versus chemotherapy as first-line treatment for patients withadvanced EGFR mutation-positive non-small-cell lung cancer (OPTIMAL,CTONG-0802): a multicentre, open-label, randomised, phase 3 study. Lancet Oncol. 2011, 12 (8): 735-742. 10.1016/S1470-2045(11)70184-X.
Sharma SV, Bell DW, Settleman J, Haber DA: Epidermal growth factor receptor mutations in lung cancer. Nat Rev Cancer. 2007, 7: 169-181. 10.1038/nrc2088.
Pao W, Miller VA, Politi KA, Riely GJ, Somwar R, Zakowski MF, Kris MG, Varmus H: Acquired resistance of lung adenocarcinomas to gefitinib or erlotinib isassociated with a second mutation in the EGFR kinase domain. PLoS. 2005, 2: 225-235. 10.1371/journal.pmed.0020073.
Pirker R, Herth FJ, Kerr KM, Filipits M, Taron M, Gandara D, Hirsch FR, Grunenwald D, Popper H, Smit E: Consensus for EGFR mutation testing in non-small cell lung cancer: resultsfrom a European workshop. J Thorac Oncol. 2010, 5 (10): 1706-1713. 10.1097/JTO.0b013e3181f1c8de.
Febbo PG, Ladanyi M, Aldape KD, De Marzo AM, Hammond ME, Hayes DF, Iafrate AJ, Kelley RK, Marcucci G, Ogino S: NCCN Task Force report: Evaluating the clinical utility of tumor markers inoncology. J Natl Compr Canc Netw. 2011, 9 (Suppl 5): S1-S32. quiz S33,
Keedy VL, Temin S, Somerfield MR, Beasley MB, Johnson DH, McShane LM, Milton DT, Strawn JR, Wakelee HA, Giaccone G: American Society of Clinical Oncology provisional clinical opinion: epidermalgrowth factor receptor (EGFR) Mutation testing for patients with advancednon-small-cell lung cancer considering first-line EGFR tyrosine kinaseinhibitor therapy. J Clin Oncol. 2011, 29 (15): 2121-2127. 10.1200/JCO.2010.31.8923.
Thunnissen E, Kerr KM, Herth FJ, Lantuejoul S, Papotti M, Rintoul RC, Rossi G, Skov BG, Weynand B, Bubendorf L: The challenge of NSCLC diagnosis and predictive analysis on small samples.Practical approach of a working group. Lung Cancer. 2012, 76 (1): 1-18. 10.1016/j.lungcan.2011.10.017.
CAP/IASLC/AMP Molecular Testing Guidelines for Selection of Lung CancerPatients for EGFR and ALK Tyrosine Kinase Inhibitors. http://www.cap.org/apps/docs/membership/transformation/new/lung_public_comment_supporting_materials.pdf,
Roche Molecular Systems Inc: cobas EGFR Mutation Test CE-IVD Package Insert. 2011, USA: Roche Molecular Systems, Inc
Thomas RK, Nickerson E, Simons JF, Janne PA, Tengs T, Yuza Y, Garraway LA, LaFramboise T, Lee JC, Shah K: Sensitive mutation detection in heterogeneous cancer specimens by massivelyparallel picoliter reactor sequencing. Nat Med. 2006, 12 (7): 852-855. 10.1038/nm1437.
McEnroe RJ, Burritt MF, Powers DM, Rheinheimer DW, Wallace BH: Interference testing in clinical chemistry; Approved Guideline - SecondEdition. Clinical and Laboratory Standards Institute document, Volume 25. 2005, Wayne, Pennsylvania, USA: Clinical and Laboratory StandardsInstitute, 1-105.
Lopez-Rios F, Angulo B, Gomez B, Mair D, Martinez R, Conde E, Shieh F, Tsai J, Current R, Lawrence HJ: Comparison of molecular testing methods for the detection of EGFR mutationsin formalin-fixed paraffin-embedded tissue (FFPET) specimens of non-smallcell lung cancer (NSCLC). J Thorac Oncol. 2012, 7 (6): S7-S89.
Anderson S, Bloom KJ, Vallera DU, Rueschoff J, Meldrum C, Schilling R, Kovach B, Lee JR, Ochoa P, Langland R: Multisite Analytic Performance Studies of a Real-Time Polymerase ChainReaction Assay for the Detection of BRAF V600E Mutations in Formalin-FixedParaffin-Embedded Tissue Specimens of Malignant Melanoma. Arch Pathol Lab Med. 2012, 136 (11): 1385-1391. 10.5858/arpa.2011-0505-OA.
Angulo B, Garcia-Garcia E, Martinez R, Suarez-Gauthier A, Conde E, Hidalgo M, Lopez-Rios F: A commercial real-time PCR kit provides greater sensitivity than directsequencing to detect KRAS mutations: a morphology-based approach incolorectal carcinoma. J Mol Diagn. 2010, 12 (3): 292-299. 10.2353/jmoldx.2010.090139.
Molinari F, Felicioni L, Buscarino M, De Dosso S, Buttitta F, Malatesta S, Movilia A, Luoni M, Boldorini R, Alabiso O: Increased detection sensitivity for KRAS mutations enhances the prediction ofanti-EGFR monoclonal antibody resistance in metastatic colorectal cancer. Clin Cancer Res. 2011, 17 (14): 4901-4914. 10.1158/1078-0432.CCR-10-3137.
Lee S, Brophy VH, Cao J, Velez M, Hoeppner C, Soviero S, Lawrence HJ: Analytical performance of a PCR assay for the detection of KRAS mutations(codons 12/13 and 61) in formalin-fixed paraffin-embedded tissue samples ofcolorectal carcinoma. Virchows Arch. 2011, 460 (2): 141-149.
Tsiatis AC, Norris-Kirby A, Rich RG, Hafez MJ, Gocke CD, Eshleman JR, Murphy KM: Comparison of Sanger sequencing, pyrosequencing, and melting curve analysisfor the detection of KRAS mutations: diagnostic and clinicalimplications. J Mol Diagn. 2010, 12 (4): 425-432. 10.2353/jmoldx.2010.090188.
Decker RH, Tanoue LT, Colasanto JM, Detterbeck FC, Wilson LD: Evaluation and definitive management of medically inoperable early stagenon-small-cell lung cancer. Part 2: newer treatment modalities. Oncology. 2006, 20 (8): 899-905. discussion 905–898, 913,
Beau-Faller M, Degeorges A, Rolland E, Mounawar M, Antoine M, Poulot V, Mauguen A, Barbu V, Coulet F, Pretet JL: Cross-Validation Study for Epidermal Growth Factor Receptor and KRAS MutationDetection in 74 Blinded Non-small Cell Lung Carcinoma Samples: A Total of5550 Exons Sequenced by 15 Molecular French Laboratories (Evaluation of theEGFR Mutation Status for the Administration of EGFR-TKIs in Non-Small CellLung Carcinoma [ERMETIC] Project-Part 1). J Thorac Oncol. 2011, 6 (6): 1006-1015. 10.1097/JTO.0b013e318211dcee.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2407/13/210/prepub
We thank Lucy Kanan from Miller Medical for her contributions on the manuscript.We thank the groups from GE Healthcare, Labcorp, and Targeted MolecularDiagnostics to for their contributions to the clinical reproducibilitystudy.
All authors except KB, SA, and WM are employees of Roche Molecular Systems. HJL is aformer employee for RMS. Kits and specimens were provided by RMS for the clinicalreproducibility study.
PA, JF, JS, RC, TR, JT, HBT, SC, and MC contributed to study design and running allanalytical performance and verification testing. FS was involved in drafting themanuscript and interpretation of the data. WW, LU, SS were involved in study designand acquisition of the data. HJL oversaw the study design and conduct of theexternal reproducibility study and was involved in drafting of the manuscript. RSwas involved in the study design and conduct of the clinical reproducibility study.KB, WM, and SA performed all clinical reproducibility studies and data analysis. Allauthors have read and approved the final version of the manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
O’Donnell, P., Ferguson, J., Shyu, J. et al. Analytic performance studies and clinical reproducibility of a real-time PCRassay for the detection of epidermal growth factor receptor gene mutations informalin-fixed paraffin-embedded tissue specimens of non-small cell lungcancer. BMC Cancer 13, 210 (2013). https://doi.org/10.1186/1471-2407-13-210