Skip to main content

Analytical performance evaluation of a commercial next generation sequencing liquid biopsy platform using plasma ctDNA, reference standards, and synthetic serial dilution samples derived from normal plasma

Abstract

Background

Circulating tumor (ct) DNA assays performed in clinical laboratories provide tumor biomarker testing support for biopharmaceutical clinical trials. Yet it is neither practical nor economically feasible for many of these clinical laboratories to internally develop their own liquid biopsy assay. Commercially available ctDNA kits are a potential solution for laboratories seeking to incorporate liquid biopsy into their test menus. However, the scarcity of characterized patient samples and cost of purchasing validation reference standards creates a barrier to entry. In the current study, we evaluated the analytical performance of the AVENIO ctDNA liquid biopsy platform (Roche Sequencing Solutions) for use in our clinical laboratory.

Method

Intra-laboratory performance evaluation of AVENIO ctDNA Targeted, Expanded, and Surveillance kits (Research Use Only) was performed according to College of American Pathologists (CAP) guidelines for the validation of targeted next generation sequencing assays using purchased reference standards, de-identified human plasma cell-free (cf) DNA samples, and contrived samples derived from commercially purchased normal and cancer human plasma. All samples were sequenced at read depths relevant to clinical settings using the NextSeq High Output kit (Illumina).

Results

At the clinically relevant read depth, Avenio ctDNA kits demonstrated 100% sensitivity in detecting single nucleotide variants (SNVs) at ≥0.5% allele frequency (AF) and 50% sensitivity in detecting SNVs at 0.1% AF using 20–40 ng sample input amount. The assay integrated seamlessly into our laboratory’s NGS workflow with input DNA mass, target allele frequency (TAF), multiplexing, and number of reads optimized to support a high-throughput assay appropriate for biopharmaceutical trials.

Conclusions

Our study demonstrates that AVENIO ctDNA liquid biopsy platform provides a viable alternative for efficient incorporation of liquid biopsy assays into the clinical laboratory for detecting somatic alterations as low as 0.5%. Accurate detection of variants lower than 0.5% could potentially be achieved by deeper sequencing when clinically indicated and economically feasible.

Peer Review reports

Background

Genomic analysis of tumor DNA is integral to diagnosis and treatment planning for many human malignancies, particularly non-small cell lung cancer (NSCLC) [1]. Yet, inaccessibility of tumor tissue and the patient’s underlying medical condition can complicate the use of core needle biopsies for retrieval of tumor DNA. In situations where tissue biopsy is contraindicated, the ctDNA component of cell free (cf) DNA can be accessed through routine phlebotomy and evaluated by molecular methods for detection of somatic (tumor specific) genomic alterations [2]. Deep sequencing analysis of circulating tumor (ct) DNA obtained by liquid biopsy provides a minimally invasive method for comprehensive evaluation of actionable tumor biomarkers [3,4,5]. Additionally, access to tumor DNA through the patient’s blood allows incorporation of prospectively-planned longitudinal biomarker monitoring into biopharmaceutical clinical trials as opposed to relying on the diagnostic biopsy to provide a static guide to the tumor mutation landscape [6, 7]. As an example, formalin fixed, paraffin embedded (FFPE) tumor DNA preserved in tissue blocks at the time of diagnosis may not be representative of targetable genomic alterations (such as EGFR T790M) developing weeks to months post-biopsy through tumor genome evolution [8].

Incorporation of liquid biopsy into a clinical trial protocol requires support of a clinical laboratory capable of providing high-throughput deep sequencing for ctDNA detection. Although applications for liquid biopsy are increasing, widespread adoption by clinical laboratories has been impeded by multiple factors including the need for ultralow detection limits, interference from sequencing artifacts at low allele frequencies, and the requirement for coverage of diverse mutation types with broad clinical applications [9, 10]. Given these design considerations and bioinformatic challenges, it is neither practical nor economically feasible for most clinical laboratories (even those already running NGS testing) to internally develop an in-house liquid biopsy assay. Use of commercially available ctDNA kits is an alternative strategy for laboratories seeking to incorporate liquid biopsy sample analysis into their test menus. For laboratories launching ctDNA testing with a commercial kit, critical considerations include the number of genes and types of variants represented by the pre-selected gene panel, sensitivity and specificity of the kit for detection of somatic mutations at low (< 1%) allele frequencies, ease of assay workflow, hands-on time, sample input requirements, and a robust bioinformatics pipeline that provides a clinically actionable variant report.

Liquid biopsy assays face unique technical challenges. The cfDNA is fragmented (~ 160 bp) and present at a very small quantity in patient samples (typically < 10 ng/ml plasma). In addition, tumor DNA (ctDNA) is a minute fraction of cfDNA, making it highly challenging to accurately detect rare variants (< 1%) from low ctDNA inputs. A number of bioinformatic solutions are now available for addressing the unique analysis challenges created by extremely low ctDNA levels in cancer patient plasma [11, 12]. Our laboratory chose to evaluate the AVENIO ctDNA liquid biopsy platform (Roche Sequencing Solutions, Pleasanton, CA) as a commercially available option to provide the laboratory with an end-to-end solution (DNA extraction to clinically actionable patient report) for testing liquid biopsy samples. AVENIO ctDNA analysis kits are based on cancer personalized profiling by deep sequencing (CAPP-Seq) [13] with options for Targeted, Expanded and Surveillance panels designed to interrogate clinically relevant mutations in 17, 77 or 197 genes (Supplementary Figure S1, S2 and S3), respectively [14]. In addition, the Avenio platform includes specialized bioinformatics analysis workflow which has integrated digital error suppression (iDES) system. iDES augments CAPP-Seq through in silico removal of stereotypical sequencing artifacts combined with molecular barcoding [13]. In the current study we evaluated the analytical performance characteristics of all three AVENIO kits using purchased reference standards, human plasma cfDNA samples, and contrived samples derived from normal human plasma.

Methods

Analytical evaluation

Intra-laboratory analytical evaluation and validation of the AVENIO ctDNA Analysis Kit was performed at Pacific Diagnostics (PacificDx, Irvine, CA) according to College of American Pathologists guidelines for validation of targeted next generation sequencing assays [15, 16]. Accuracy, precision, and limit of detection were evaluated in the following sample types: reference standards purchased from Horizon Discovery (Waterbeach, UK; n = 8, Multiplex I cfDNA Reference Standard Set) and SeraCare (n = 7) (Milford, MA; n = 7, Seraseq ctDNA Mutation Mix v2 and Seraseq ctDNA complete mutation mix); cfDNA extracted from normal plasma (n = 8) purchased from Biological Specialty Corporation (Colmar, PA); human plasma samples collected from various cancer individuals (n = 12) purchased from ProteoGenex, Inc. (Ingelwood, CA), synthetic serial dilution samples of “mutant” and wild type (WT) cfDNA derived from normal plasma (n = 4) produced at Pacific Diagnostics; and NA12878 (n = 1) as a negative control (Coriell, Camden NJ.) Reference Standards were chosen to represent all variation classes (SNVs, indels, CNAs and SVs) tested by the assay. Standards were engineered by the manufacturer (Horizon and Seracare) to contain representative mutations within human cancer cell lines with DNA fragmented to an average fragment length of 160 bp, resembling cfDNA extracted from plasma. Allelic frequency and copy number data provided by the manufacturer via Droplet Digital PCR (ddPCR) was used to perform concordance with data obtained from Avenio ctDNA analysis kits. In addition, 67 de-identified cfDNA samples at concentrations from 10 to 50 ng were evaluated for observance of QC metrics across all three panels (Targeted, Expanded, Surveillance.)

Synthesis of serial dilution samples

Normal human plasma from de-identified donors was purchased from Biological specialty corporation (Allentown, PA). Multiple cfDNA replicates were extracted from each of the four normal human plasma samples. DNA libraries were created from each donor and sequenced in quadruplicate using the AVENIO ctDNA Expanded Kit (Roche Sequencing, Pleasanton, CA). An additional four normal human plasma specimens were extracted and sequenced in singlicate. Variants reported in the unfiltered data set of AVENIO software were used to identify 22 “mutations” present in one of the normal plasma samples at ~ 50% allele frequency, designated as the “mutant” sample. Selected mutations were identified in the following genes: NTRK1, FGFR2, BRCA1, BRCA2, ALK, FGFR3, PDGFRA, CSF1R, PMS2, PTCH1 (Table 1). Three normal plasma samples lacking these 22 mutations were designated as the normal wild type (WT) and mixed to create a background normal sample. Serial dilutions were prepared by spiking the mutant into the WT with resultant expected allele frequencies of: 5.0, 1.0, 0.5, and 0.1%. Dilutions were made such that the total amount of cfDNA for each reaction was 12 ng.

Table 1 Synthetic serial dilution variants

Isolation of cfDNA from plasma using the Avenio cfDNA extraction kit

Whole blood was collected in K2-EDTA tubes. Plasma was separated by double centrifugation at 1500 x g for 10 min with brakes off within 4 h. of collection. Plasma was stored frozen at − 70-80 °C until cfDNA isolation.

In preparation for DNA isolation, 2–5 ml of plasma was thawed, centrifuged at 1800 x g for 5 min at room temperature and transferred to a conical tube. After addition and incubation with Proteinase K, DNA Paraffin Binding Buffer, and isopropanol, samples were transferred to a High Pure Extender Assembly (HPEA) unit. The HPEA unit comes pre-assembled as a 50 ml tube containing a High Pure Extender to hold the plasma and a Filter Tube to bind the cfDNA. Following centrifugation for 5 min at 3270 x g, the Filter Tube was removed from the HPEA and placed into a collection tube. The cfDNA was then washed and eluted from the detached filter tube. Following isolation of the purified cfDNA, samples were either stored at 2–8 °C or used immediately for DNA library preparation. After extraction, cfDNA concentration was quantified using the Qubit dsDNA High Sensitivity Assay (ThermoFisher Scientific, Waltham, MA) and recorded along with the calculated isolated DNA mass. Additionally, sample quality was confirmed using the High Sensitivity DNA Kit on the Agilent 2100 Bioanalyzer.

Preparation of sequencing libraries using the Avenio kit

Sequencing libraries with unique sample indexes were prepared using the AVENIO cfDNA Library Prep sub-Kit (Roche) according to the manufacturer’s instructions. In brief, 10 μl of unique Sample Adapter was added to each sample followed by overnight incubation at 16 °C. Following post-ligation cleanup using AVENIO cleanup beads, the library molecules were amplified using the AVENIO Pre-Enrichment PCR Master Mix reagents and primers. Following post-PCR cleanup, library size and quantity were verified by High Sensitivity DNA Kit on the Agilent Bioanalyzer and Qubit. After library QC, samples were either stored at − 15 to − 20 °C or moved immediately to the Library Enrichment using the AVENIO ctDNA Enrichment Kit, one of the three AVENIO Panels (Targeted, Expanded, or Surveillance), and the AVENIO Post-Hybridization sub-Kits according to the manufacturer’s instructions. Post-hybridization washes were followed by a second PCR amplification step in preparation for sequencing. Following post-PCR cleanup, samples were stored at − 15 to − 20 °C or moved quantified for sequencing. Library concentration was assessed using the Qubit dsDNA High Sensitivity Assay followed by Bioanalyzer High Sensitivity DNA kit on Bioanalyzer to determine the size of each library. Region tables were set from 200 to 1000 bp and library sizes were recorded. Library quantification was confirmed using absolute qPCR-based quantification of libraries containing Illumina P5 and sequences using the KAPA Universal Library Quantification kit (KapaBiosystems, Wilmington, MA).

Preparation of sequencing libraries using the TruSight oncology 500 ctDNA kit

cfDNA was extracted from 6 ml of plasma samples utilizing the QIAamp circulating Nucleic Acid Kit (QIAGEN, Cat No./ID: 55114) according to kit instructions. Libraries were constructed using a minimum 30 ng of cfDNA measured by capillary electrophoresis method (75-300 bp range) as per the manufacturer’s instructions for the TruSight Oncology 500 ctDNA Kit (Illumina, 20,012,860). Indexed pre-capture libraries were enriched for specific targeted regions covered by the TruSight Oncology 500 ctDNA kit by two rounds of hybridization, streptavidin bead capture and clean up. The enriched libraries were amplified, purified with sample purification beads and normalized with normalization beads prior to sequencing. Samples were sequenced on a NovaSeq 6000 instrument using either a S2 300 cycle Reagent Kit (8 samples/run) or a S4 300 cycle Reagent Kit (6 samples/lane or 24 samples/flow cell) in conjunction with a NovaSeq Xp 4-Lane Kit ((Illumina, 20,012,866 and 20,021,665). Data analysis was performed utilizing the DRAGEN TruSight Oncology 500 ctDNA Analysis Software with Illumina DRAGEN server v3.

DNA sequencing and data analysis of the Avenio data

The addition of unique sample adapters during library prep enabled multiplexed sequencing of up to 11 samples per sequencing run for the AVENIO Expanded and Surveillance kits. Higher multiplexing (up to 16 samples per Nextseq High output run) was achieved for the AVENIO Targeted kit, which has a smaller panel. Library pools were prepared by combining an equal mass of each library and sequenced using the 300 cycle NextSeq 500/550 High Output kit v2.5 on the NextSeq 500/550 (Illumina, San Diego CA). Data was analyzed with the AVENIO ctDNA Analysis Software (version 1.0.0 & 2.0.0).

Results

Assay QC metrics

Assay performance QC was tested using 67 human plasma and reference standards across all three AVENIO ctDNA analysis kits [Expanded (28), Targeted (29), and Surveillance [10]]. Sample inputs across the entire recommended range (10–50 ng) were tested. The sequencing depth per sample was determined by the panel used to ensure adequate coverage: an average of 30 million paired end reads were obtained for the AVENIO Expanded and Surveillance kit samples and 20 million paired end reads were obtained for AVENIO Targeted kit samples (Table 2 and Supplementary Table S1). All three kits obtained an average of 70% on-target reads. Because the unique coverage is a function of DNA input, the unique median coverage obtained was dependent on sample input: 50 ng sample input resulted in 7000-8000X unique coverage and 10 ng sample input resulted 1000-1100X unique coverage. All three kits had low error rate (1–3 X10− 5) and uniform coverage (98–100% of the targeted regions were covered within 10-fold of the median coverage) (Fig. 1). All three kits across all 67 samples displayed >500x unique depth coverage even in lowest 5’th percentile of target region (Table 2).

Table 2 Sample QC Metrics. Data in Table was generated by AVENIO OAS
Fig. 1
figure1

Example coverage uniformity obtained on all three panels. a. Plasma sample tested using Expanded panel b. ctDNA reference standard tested using Expanded panel c. Plasma sample tested using Surveillance panel d. Plasma sample tested using Targeted panel

Accuracy, sensitivity, specificity and limit of detection

Assay sensitivity and specificity were tested for AVENIO Expanded, Targeted and Surveillance panels by comparing expected results against experimental results from well characterized reference standards and four blended human normal plasma samples (236 variants across 14 samples). All three panels displayed 100% sensitivity in detecting Single Nucleotide Variants (SNVs) and Indels at 0.5% allele frequency (AF) in samples with an input range from 20 ng–40 ng (68 variants). The AVENIO Surveillance panel displayed 100% sensitivity for detecting SNVs/Indels at 0.5% AF, 72% sensitivity at 0.25% AF, and 28% sensitivity at 0.1% AF (18 variants at each AF) at 20 ng sample input amount (Table 3 and Supplementary Table S3). The sensitivity for SNVs/Indels in the AVENIO Expanded kit were 100% at 1% AF, 100% at 0.5% AF, and 50% at 0.1% AF with 40 ng sample input (8 variants at each AF) As expected, the sensitivity was reduced to 100% at 1% AF, 82% at 0.5% AF, and 29% at 0.1% AF (28 variants at each AF) with 15 ng input (Table 3 and Supplementary Table S2). Sensitivity for detecting SNV and INDELs by Targeted panel was 100% at 0.5%AF at 20 ng input and it was reduced to 93% at 10 ng input (Table 3 and Supplementary Table 4). All three panels displayed 100% sensitivity in detecting all fusions and amplifications down to 2.5% AF except CD74-ROS1 fusion. CD74-ROS1 fusion was captured and sequenced by both Targeted and Expanded panel (Table 4 and data not shown). However, it was not called by the Avenio software at any allele frequency tested. This could possibly be due to synthetic fusion created in the reference standard that was limited to short low complexity region of the CD74 gene and was filtered out in Avenio software. In future, it will be of interest to test this fusion with other reference standards or plasma samples.

Table 3 Avenio ctDNA kit sensitivity for SNV and INDEL
Table 4 Avenio ctDNA kit sensitivity for Fusions and amplifications

Additionally, the Expanded panel was tested for specificity using one wild type reference standard from Horizon (HD776), one well characterized HAPMAP cell line (NA12878), and eight normal human plasma sample. Specificity for the Surveillance panel was tested using one wild type reference standard and NA12878. Specificity for the Targeted panel was tested using a Seracare reference standard (0.5 and 2.5% AF). Specificity for SNV and INDEL for each panel was limited to variants in loci of interest for each panel. All three panels displayed very high specificity for detecting SNV ≥0.5% AF, INDELs, fusions and amplifications (100%). Limited number of SNV were detected below 0.5%. Two plasma samples displayed germline variants (Table 5). A PMS2, p.Lys651Arg variant was detected in four out of eight plasma samples tested indicating high prevalence of this mutation in general population. Overall, all three panels displayed > 99.7% specificity in detecting SNV and INDEL within the loci of interest region and 100% specificity in detection of fusions and amplifications (Table 5).

Table 5 AVENIO ctDNA kit specificity

Accuracy of the Avenio assay was further determined by testing plasma samples collected from 12 cancer patients (breast cancer, colorectal cancer, prostate cancer and NSCLC; 3 each) at various stages of cancer (stage IIB-IV) using the Avenio Expanded kit and the TruSight500 (TSO500) ctDNA kit (Illumina, Inc.). The TruSight500 ctDNA kit is a liquid biopsy kit from Illumina that can detect alterations at low allele frequencies (0.5% AF) in cfDNA samples. The TruSight and Avenio Expanded kits have many genes that are covered in common, making TruSight a suitable kit to compare data obtained from the Avenio kit. Eight ml of plasma from each cancer specimen was purchased from a commercial vendor (ProteoGenex, Inc.) and was split between the Avenio Expanded kit and the TSO500ctDNA kit for testing (2 ml and 6 ml respectively). cfDNA destined for the Avenio assay was extracted using the cfDNA extraction kit provided with the Avenio assay. cfDNA destined for the TSO500 kit was extracted using the QIAamp circulating nucleic acid kit (Qiagen, Inc.) 10–50 ng cfDNA input was used for the Avenio assay except for two samples where 9.0 ng input was used. As per the manufacturer’s recommendation, a minimum 30 ng input was used for the TSO500 ctDNA assay. Two samples did not meet sample input requirements for the TSO 500 ctDNA kit and were not tested. Eight of the 12 cancer patients detected alterations (SNV and CNV) in the frequently mutated cancer genes (3/3 CRC, 2/3 breast cancer, 1/3 prostate cancer and 2/3 NSCLC) using the Avenio assay. Six of eight positive samples were also tested using the TSO500 ctDNA kit and the same alterations were detected at similar allele frequency range in both assays. Similarly, four of four negative samples also did not detect any alteration in the Avenio intersected region in the TruSight500 assay, indicating 100% concordance between the Avenio and the TSO500 ctDNA assays (Table 6).

Table 6 Accuracy for detecting alteration in human cancer samples

Precision

Assay precision was tested by with five reference standards with 0–5% AF and four normal plasma samples. Each precision sample was subjected to three library preparations, sequencing runs, and data analyses (Table 7). Library preparations for repeatability (Run 1 and Run 2) were prepared on the same day, used the same lot of library preparation reagents, and were run on a single NextSeq run. Library preparations for reproducibility (Run 3) were performed on a different day from the first two runs by a distinct operator and sequenced on a separate NextSeq run. Assay concordance was 100% for SNVs and indels at allele frequencies ≥0.5%. As expected, the variant frequency variability (%CV) was higher for variants with lower allele frequencies (≤ 1%).

Table 7 Avenio Expanded ctDNA analysis kit precision

Assay linearity

To assess linearity of the AVENIO Expanded panel for variants at allele frequencies below 5%, the reported allele frequency values for SNVs and indels in the reference standards and normal human plasma samples (with frequencies ranging from 0.5–6%; 47 total data points) were plotted against the expected allele frequency values and fitted by linear regression. The correlation was high (R2 = 0.979), suggesting quantitative accuracy at low variant frequencies (Fig. 2).

Fig. 2
figure2

Linearity of SNVs and Indels – Experimental versus Expected AF values for the AVENIO Expanded Panel data

Reportable range

The reportable range of the AVENIO ctDNA Expanded, Targeted, and Surveillance panels, is defined as the fraction of targeted genomic regions for which calls of acceptable quality can be generated [17]. The Expanded Panel contains 77 genes, the Targeted Panel contains 17 genes and the Surveillance Panel contains 197 genes, including those currently in the US National Comprehensive Cancer Network (NCCN) Guidelines. The total panel size for the Expanded kit is 192 kb, the Targeted kit is 81 Kb and the Surveillance kit is 198 kb. All three panels include all four mutation classes – SNVs, INDELs, fusions and CNA. The assay reports SNV in all regions interrogated by the assay (Supplementary Figure S1, S2 and S3). INDELs are limited to variants in a pre-specified list of positions, referred to as “Loci of Interest”, except for EGFR exon 19 long deletions, EGFR exon 20 long insertions, and MET long insertions, which are not restricted to a pre-defined set of Indels (Supplementary Table S5 and S6). CNV is limited to MET, ERBB2 and EGFR genes in all three panels. The fusions are tested for 6 gene targets (ALK, ROS1, RET, NTRK1, FGFR2 and FGFR3) in the Expanded panel and 3 gene targets in the Targeted and Surveillance panels (ALK, ROS1 and RET). All three panels displayed very high uniformity of coverage (≥99% bases within 10-fold of median; Fig. 1) and displayed >500x unique depth coverage even in the lowest 5’th percentile of target regions covered by the panel (Table 2). No low coverage or drop out regions were detected during this validation. Although not called by the Avenio reporting software, the CD74-ROS1 fusion was manually detected in BAM file. The CD74 region of the fusion is in a GA-rich, low complexity region (as defined by Repeat Masker), which suggests the fusion call is being filtered out with current version of software (v2.0.0). As such, the ROS1-CD74 fusion will not be included in reportable range for Avenio assay for this software version.

Discussion

In our study, we evaluated the performance and accuracy of a commercially available ctDNA liquid biopsy platform. The AVENIO assay integrated seamlessly into our laboratory’s existing NGS workflow with an average sample-to-report TAT of 5 working days (Fig. 3). Multiple stopping points are included within the protocol where samples can be frozen to accommodate a variety of laboratory schedules, including those without weekend staff coverage. As our laboratory was already performing NGS using hybrid capture for library preparation, there was no interruption to established protocols or need for new equipment to incorporate AVENIO assays into the daily testing routine. The iDES insert and index barcodes incorporated into DNA strands during library preparation create unique identifiers (UIDs) used for sample multiplexing [13]. Technicians performing the assay noted little or no difference in procedures between FFPE (data not shown) and ctDNA DNA kits.

Fig. 3
figure3

Avenio workflow

In our hands all three AVENIO kits displayed very high on-target rate (~ 70%; Supplementary Table 1) and median unique depth (~4000x unique coverage using 30 ng input) on human plasma samples. Unique depth is a critical measure of performance in liquid biopsy assays as it indicates how many initial cfDNA molecules were retained throughout the library prep and sequencing process and ultimately dictates the assay sensitivity. Similarly, the panel displayed very high precision (< 25%CV) in detection of variants ≤0.5% AF. The increased CV at lower allele frequency is expected due to Poisson variance when performing library prep on very few mutant ctDNA molecules. Higher variance at low allele frequency can result in false negative and/or false positive results for variants < 0.5% AF.

We chose to perform in-depth validation of AVENIO ctDNA Expanded panel for intra-laboratory accuracy studies since the clinically informative genetic alterations targeted by the panel are suitable for biopharma clinical trial support. The 77 genes targeted by the panel represent 567 known hotspot tumor variants, including specific alterations in genes such as EGFR, KRAS, and BRAF linked to eligibility for on-label targeted therapies [14, 18]. Additional genes and gene regions allow for discovery of off-label actionable biomarkers. The Expanded Panel also incorporates recurrent genetic alterations from the CAPP-Seq selector library allowing identification of a unique cancer personalized profile (CAPP) that can be monitored throughout the course of a patient’s disease [9]. Additionally, advantages to hybrid capture-based enrichment strategies, in contrast to amplification-based assays, are well documented and include minimal allele dropout, deeper uniform coverage, and higher sensitivity [19]. A recent retrospective matched tissue-plasma analysis from NSCLC subjects using the AVENIO Surveillance hybrid capture strategy reported tissue-plasma concordance to be positively associated with tumor size and cancer stage [20].

As an additional evaluation of the AVENIO assay we compared accuracy and performance with Illumina’s TSO500 liquid biopsy panel for somatic mutations in intersected regions. We found that both panels displayed similar performance characteristics. However, TSO500 panel required higher number of sequencing reads due to its larger panel size (500 genes) and required higher cfDNA input (30 ng) making it less economical and clinically feasible in comparison to Avenio assay. A similar study was recently reported by Lam, et al. in which the AVENIO platform was compared to Qiagen’s QIAseq Human Comprehensive Cancer panel for panel coverage of clinically relevant variants and overall sequencing performance [21]. Detailed results from both comparison studies reveal strengths and shortcomings unique to each of these different assays. Overall, any of these NGS assay provide much larger amount of clinically relevant information than traditional targeted ddPCR assays, however NGS assays can be more expensive than PCR assays. Based on the strength and weakness of each assay clinical laboratories should make informed decisions about which panel(s) and platforms are best suited to their specific needs.

As the intended use of the panel is for routine monitoring of tumor biomarkers, we chose economical sample multiplexing and a higher TAF of ≥0.5% in order to maximize the number of samples (n = 11) per NextSeq run. Optimal performance was observed using 10–50 ng of cfDNA input mass and 30 million paired-end reads with deduplicated coverage averaging above 4000 reads per base. Although some laboratories are seeking to identify variants at lower TAFs using alternative methodologies, we found the commercial kit performance for multiplexed samples to be most robust at TAFs of ≥0.5%. This finding is supported by other studies demonstrating higher correlation with tissue biopsy results at AFs in the range of 1.0% [20, 22]. In clinical scenarios where it is biologically indicated for small tumors or low levels of residual disease, the AVENIO Expanded panel can be run in “High Output” mode with as few as 1–3 samples to achieve near 90% sensitivity at TAF of 0.05% [17].

Conclusions

The AVENIO liquid biopsy platform provides a user friendly, accurate solution for incorporating ctDNA analysis into the workflow of an NGS laboratory. In the “moderate output” mode tested in this study, performance is most robust at TAFs ≥0.5%, suitable for an intended use in clinical trials of late stage cancers where a higher ctDNA content is expected in the plasma due to larger and/or metastatic tumors. When biologically indicated and economically feasible, future studies of “high output” mode for lower allele frequencies (≤ 0.1%), reflective of small tumors and minimal residual disease, will be performed by our laboratory. With increasing adoption of ctDNA analysis in the clinical laboratory, we envision incorporation of routine tumor biomarker monitoring for patients enrolled in oncology clinical trials.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available in the NCBI Sequence Read Archive (SRA), project ID PRJNA66392; study ID SRP282644; https://www.ncbi.nlm.nih.gov/sra?term=SRP282644

Abbreviations

cfDNA:

cell free DNA

ctDNA:

circulating tumor DNA

SNV:

Single nucleotide variant

AF:

Allele frequency

IDES:

Integrated digital error suppression

CAPP-Seq:

Cancer personalized profiling by deep sequencing

UIDs:

Unique identifiers

TAF:

Target allele frequency

References

  1. 1.

    Mayekar M, Bivona T. Current landscape of targeted therapy in lung cancer. Clin Pharmacol Ther. 2017;102:757–64.

    Article  Google Scholar 

  2. 2.

    Perez-Barrios C, Nieto-Alcolado I, Torrente M, Jimenez-Sanchez C, Calvo V, Gutierrez-Sanz L, Palka M, Donoso-Navarro E, Procencio M, Romero A. Comparison of methods for circulating cell-free DNA isolation using blood from cancer patients: impact on biomarker testing. Transl Lung Cancer Res. 2016;5:665–72.

    CAS  Article  Google Scholar 

  3. 3.

    Speicher M, Pantel K. Tumor signatures in the blood. Nat Biotechnol. 2014;32:441–3.

    CAS  Article  Google Scholar 

  4. 4.

    Wan J, Massie C, Garcia-Corbacho J, Mouliere F, Brenton J, Caldas C, Pacey S, Baird R, Rosenfield N. Liquid biopsies come of age: towards implementation of circulating tumor DNA. Nat Rev Cancer. 2017;17:223–38.

    CAS  Article  Google Scholar 

  5. 5.

    Merker J, Oxnard G, Compton C, Diehn M, Hurley P, Lazar A, Lindeman N, Lockwodd C, Rai A, Schilsky R, Tsimberidou A, Vasalos P, Billman B, Oliver T, Bruninooge S, Hayes D, Turner N. Circulating tumor DNA analysis in patients with cancer: American Society of Clinical Oncology and College of American Pathologists joint review. J Clin Oncol. 2018;36:1631–41.

    CAS  Article  Google Scholar 

  6. 6.

    Crowley E, Di Nicolantonio F, Loupakis F, Bardelli A. Liquid biopsy: monitoring cancer-genetics in the blood. Nat Rev Clin Oncol. 2013;10:472–84.

    CAS  Article  Google Scholar 

  7. 7.

    Diaz L, Bardelli A. Liquid biopsies: genotyping circulating tumor DNA. J Clin Oncol. 2014;32:579–86.

    Article  Google Scholar 

  8. 8.

    Beca F, Beck H. Precision cancer diagnostics: tracking genomic evolution in clinical trials. PLoS Med. 2016. https://doi.org/10.1371/journal.pmed.1002177.

  9. 9.

    Newman A, Bratman S, To J, Wynne J, Eclov N, Modlin L, Li C, Neal J, Wakelee H, Merritt R, Shrager J, Loo B, Alizadeh A, Diehn M. An ultrasensitive method for quantitating circulating tumor DNA with broad patient coverage. Nat Med. 2014;20:548–54.

    CAS  Article  Google Scholar 

  10. 10.

    Kindle I, Wu J, Papadopoulos N, Knzler K, Vogelstein B. Detection and quantification of rare mutations with massively parallel sequencing. Proc Natl Acad Sci U S A. 2011;108:9530–5.

    Article  Google Scholar 

  11. 11.

    Bai Y, Zhao H. Liquid biopsy in tumors: opportunities and challenges. Ann Transl Med. 2018;6(Suppl 1):S89. https://doi.org/10.21037/atm.2018.11.31.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Volik S, Alcaide M, Morin RD, Collins C. Cell-free DNA (cfDNA): clinical significance and utility in cancer shaped by emerging technologies. Mol Cancer Res. 2016;14(10):898–908. https://doi.org/10.1158/1541-7786.MCR-16-0044.

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Newman A, Lovejoy A, Klass D, Kurtz D, Chabon J, Scherer F, Stehr H, Liu C, Bratman S, Say C, Carter J, West R, Sledge G, Shrager J, Loo B, Neal J, Wakelee H, Diehn M, Alizadeh A. Integrated digital error suppression for improved detection of circulating tumor DNA. Nat Biotechnol. 2016;34:547–55.

    CAS  Article  Google Scholar 

  14. 14.

    Roche Sequencing Solutions, Inc. Avenio family of NGS oncology assays: ctDNA and Tumor Tissue Analysis Kits. Sequencing.roche.com/Aveniobrochure-avenio-family-ngs-oncology-assays-SEQ100331.pdf. 2018.

    Google Scholar 

  15. 15.

    Gunn S, Verma S, Sims C, Moore M, Cotter P. Early evaluation site experience with a liquid biopsy kit designed for next generation sequencing of circulating tumor DNA. J Mol Diagn. 2017;19:1021.

    Google Scholar 

  16. 16.

    Jennings L, Arcila M, Corless C, Kamel-Reid S, Lubin I, Pfeifer J, Temple-Smolkin R, Voelkerding K. Nikiforova: guidelines for validation of next-generation sequencing-based oncology panels: a joint consensus recommendation of the Association for Molecular Pathology and College of American pathologists. J Mol Diagn. 2017;19:341–65.

    Article  Google Scholar 

  17. 17.

    Tack V, Spans L, Schuuring E, Keppens C, Zwaenpoel K, Pauwels P, Houdt J, Dequeker EMC. Describing the reportable range is important for reliable treatment decision: a multiple laboratory study for molecular tumor profiling using next generation sequencing. J Mol Diagn. 2018;20:743–53.

    Article  Google Scholar 

  18. 18.

    Roche Sequencing Solutions, Inc. In-depth assessment reveals powerful performance and flexibility of the Avenio ctDNA Analysis Kits. sequencing.roche.com/AvenioSEQ100108_AVENIO_ctDNA_Performance_White_Paper.pdf 2018.

    Google Scholar 

  19. 19.

    Hung S, Meissner B, Chavez E, Ben-Neriah S, Ennishi D, Jones M, Shulha H, Chan F, Boyle M, Knidel R, Gascoyne R, Mungall A, Marra M, Scott D, Coonors J, Seidl C. Assessment of capture and amplicon-based approaches for the development of a targeted next generation sequencing pipeline to personalize lymphoma management. J Mol Diagn. 2018;20:2013–214.

    Article  Google Scholar 

  20. 20.

    Jiang J, Adams H, Yao L, Yaung S, Lal P, Balasubramanyam A, Fuhlbruck TN, Lovejoy A, Froehler S, Fang L, Achenbach H, Floegel R, Krugel R, Palma J. Concordance of genomic alterations by next generation sequencing in tumor tissue versus cell-free DNA in stage 1-IV non-small cell lung cancer. J Mol Diagn. 2020;22:228–35.

    CAS  Article  Google Scholar 

  21. 21.

    Lam S, Zhou Y, Chan Y, Foo C, Lee P, Mok W, Wong W, Fyng Y, Wong K, Huang J, Chow C. Comparison of target enrichment platforms for circulating tumor DNA detection. Sci Rep. 2020;10:4124. https://doi.org/10.1038/s41598-020-60375-x.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Kuderer N, Burton K, Blau S, Rose A, Parker S, Lyman G, Blau A. Comparison of 2 commercially available next-generation sequencing platforms in oncology. JAMA. 2017;3:996–8.

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was partially funded by Roche diagnostics and Foundation Medicine for independent post-launch evaluation of Avenio ctDNA kits for Cancer Personalized Profiling by Deep Sequencing and for TruSight500 ctDNA kits respectively. The funders did not influence design, analysis or interpretation of this study in anyway.

Author information

Affiliations

Authors

Contributions

SV, KH, SA, TN, RR, and AG analyzed and interpreted the data. PC and MM designed the assays. SV and SG were a major contributor in writing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Suman Verma.

Ethics declarations

Ethics approval and consent to participate

INTEG Review IRB Committee has determined this study to be exempt from ethics approval under exempt category#4 according to 45CFR 46.101(b). (Study protocol#: PLE for CAP-Seq/RD003387). According to the exemption, no informed consent from participants is required for this study.

Consent for publication

N/A

Competing interests

S.G. has received speaker and consultant advisory board fees from Roche. All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1:

Supplementary Figure S1. Targeted regions covered by Avenio Expanded Panel.

Additional file 2:

Supplementary Figure S2. Targeted regions covered by Avenio Surveillance Panel.

Additional file 3:

Supplementary Figure S3. Targeted regions covered by Avenio Targeted Panel

Additional file 4:

Supplementary Table S1. Sample QC Metrics across three panels [Targeted, Expanded, Surveillance]

Additional file 5:

Supplementary Table S2. AVENIO Expanded ctDNA analysis kit sensitivity (SNV/INDEL)

Additional file 6:

Supplementary Table S3. Avenio Surveillance ctDNA kit sensitivity (SNV/INDEL)

Additional file 7:

Supplementary Table S4. Avenio Targeted ctDNA kit sensitivity (SNV/INDEL)

Additional file 8:

Supplementary Table S5. INDEL Loci of interest for Avenio Expanded kit

Additional file 9:

Supplementary Table S6. INDEL Loci of interest for Avenio Targeted and Surveillance kit

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Verma, S., Moore, M.W., Ringler, R. et al. Analytical performance evaluation of a commercial next generation sequencing liquid biopsy platform using plasma ctDNA, reference standards, and synthetic serial dilution samples derived from normal plasma. BMC Cancer 20, 945 (2020). https://doi.org/10.1186/s12885-020-07445-5

Download citation

Keywords

  • ctDNA
  • Liquid biopsy
  • Tumor biomarkers
  • NGS