Comparison of analytical and clinical performance of CLART HPV2 genotyping assay to Linear Array and Hybrid Capture 2: a split-sample study

Abstract

Background

Human Papillomavirus (HPV) genotyping has an increasingly important role in cervical cancer screening and vaccination monitoring, however, without an internationally agreed standard reference assay. The test results from the most widely used genotyping assays are read manually and hence prone to inter-observer variability. The reading of test results on the CLART HPV2 genotyping assay is, on the other hand, automated. The aim of our study was to directly compare the detection of HPV genotypes and high-grade cervical intraepithelial neoplasia (CIN) by CLART, Linear Array (LA), and Hybrid Capture 2 (HC2) using samples stored in SurePath.

Methods

Residual material from 401 routine samples from women with abnormal cytology was tested by CLART, LA, and HC2 (ClinicalTrial.gov: NCT01671462, Ethical Committee approval: H-2012-070). Histological outcomes were ascertained by linkage to the Danish nation-wide Pathology Data Bank. For comparison of CLART and LA in terms of genotype detection, we calculated κ-coefficients, and proportions of overall and positive agreement. For comparison of CIN detection between CLART, LA, and HC2, we calculated the relative sensitivity and specificity for high-grade CIN.

Results

The κ-coefficient for agreement in detection of genotypes 16, 18, 31, 33, 35, and 51 was ≥0.90 (overall agreement: 98-99%, positive agreement: 84-95%). The values were slightly lower, but still in the “substantial” range for genotypes 39, 45, 52, 56, 58, 59, and several low-risk genotypes. The relative sensitivity of CLART for ≥ CIN2 and ≥ CIN3 was not significantly lower than that of LA and HC2, although CLART showed a higher specificity than HC2.

Conclusions

In Danish women with abnormal SurePath cytology, CLART and LA were highly comparable for detection of most high-risk and low-risk genotypes; and CLART’s sensitivity for high-grade CIN was comparable to that of both LA and HC2.

Background

Cervical cancer is caused by high-risk Human Papillomavirus (HPV) genotypes, whereas low-risk genotypes cause benign lesions [1-3]. Genotyping of HPV infections has an increasing role in cervical screening and vaccination monitoring [4,5], however, without an internationally agreed standard reference HPV genotyping assay [4]. With more than 100 genotyping assays on the market, the question remains: which genotyping assays have the requisite validation data to support their use. The two most widely used, Linear Array (LA; Roche Diagnostics, Pleasanton, CA), and INNO-LiPA (Fujirebio Europe, Ghent, Belgium), detect 37 and 28 genotypes, respectively, and are typically read manually and hence prone to inter-observer variability in reporting test results. Papillocheck (Greiner Bio-One, Frickenhausen, Gemany), on the other hand, detects 24 genotypes, and uses automated reading [6-8]. In contrast to these commercially available genotyping assays, the GP5+/6+ polymerase chain reaction (PCR) followed by enzyme immunoassay is an in-house assay and its performance may be laboratory-dependent.

CLART HPV2 (CLART; Genomica, Madrid, Spain) is a commercially available PCR-based genotyping HPV DNA assay, based on genotype amplicon-specific hybridization on a microarray. The assay has two internal controls, a DNA control (human CTFR gene) for sample sufficiency, and an amplification control (plasmid) for process control in each tube. It detects 35 genotypes: the 13 high-risk (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 68) [1] and 22 low-risk (6, 11, 26, 40, 42, 43, 44, 53, 54, 61, 62, 66, 70, 71, 72, 73, 81, 82, 83, 84, 85, 89). Detection of individual genotypes was calibrated against known copies of cloned plasmids. Essential for high-throughput screening settings, the reading of test results is automated. Furthermore, CLART can be applied to several sample types, including formalin-fixed paraffin-embedded specimens [9,10]. Several laboratories participated with CLART in the WHO HPV LabNet Proficiency Studies [4,11], emphasizing that while it is rarely described in scientific publications [12-15], it is frequently used in clinical, non-research laboratories.

Here, we compared the analytical and clinical characteristics of CLART to those of LA and Hybrid Capture 2 (HC2; Qiagen, Gaithersburg, MD) in Danish women with abnormal cytology.

Methods

The data presented in this study were partially collected within the Danish arm of a European CE-IVD trial evaluating a new molecular HPV assay (ClinicalTrials.Gov ID: NCT01671462). From this trial, test results on HC2 and LA were used here, whereas the CLART HPV2 testing was undertaken specifically for the purpose of the current study. Residual material from 411 consecutive, unselected SurePath samples with abnormal cytology (atypical squamous cells of undetermined significance or worse, ≥ASCUS) were collected from up to 10 routine racks per day processed in the laboratory between September and October 2012. After the samples had been collected, we excluded those with insufficient quantity, ≤1.0 ml, of the residual SurePath material available post the routine cytology. Of the collected samples, 10 were excluded due to this criterion.

Cytology evaluation was undertaken by Focal Point assisted screening (BD, Burlington, NC). Slides were read by cytoscreeners, with abnormal findings adjudicated by pathologists and reported using the Bethesda 2001 system. Women aged ≥30 years with ASCUS had routine reflex HC2 HPV triage. After a negative HC2 test result, any initial ASCUS diagnoses were routinely downgraded to normal cytology, with women being referred back to routine screening. Other cytology reading was undertaken blinded to HPV testing. Women with HC2-positive ASCUS were referred for colposcopy, as were women with high-grade squamous intraepithelial lesions (HSIL), atypical squamous cells – cannot exclude HSIL (ASC-H), atypical glandular cells (AGC), adenocarcinoma in situ (AIS), cytological squamous carcinoma, and women with persistent ASCUS at age <30 years or low-grade squamous intraepithelial lesions (LSIL). Women with ASCUS at age <30 years or LSIL had repeated cytological testing. Follow-up tests until end of February 2014 were retrieved from the Danish National Pathology Data Bank (Patobank; [16]). Cases of cervical cancer were adjudicated, based on the free text in the Patobank, by an expert pathologist from the same laboratory.

HPV testing

One-half ml of SurePath sample material was centrifuged for five minutes at 14,000 revolutions per minute. Cell pellets were re-suspended in a mix of 180 μl phosphate buffered saline (10× conc. pH 7.4, Pharmacy product) and 20 μl Proteinase K (Roche Diagnostics, Rotkreuz, Switzerland). Samples were vortexed and incubated for one hour at 56°C and one hour at 90°C. DNA was purified using MagNa Pure LC 96 instrument with MagNa Pure LC Total Nucleic Acid Isolation Kit (Roche Diagnostics). Aliquots of extracted DNA were used for both CLART and LA testing. On average, samples were DNA extracted 17 days (range 10–27) after having been received in the laboratory. Extracted DNA was stored frozen until LA and CLART testing.

PCR amplification was performed using CLART HPV2 Amplification kit (Genomica). Five μl of purified DNA were used as template per reaction. Prior to visualization, the PCR products were denatured at 95°C for 10 minutes. Hybridization was performed using 10 μl of the denatured PCR products on the CLART microarray, and subsequent visualization was done according to manufacturer’s specifications. The genotyping results were analyzed and reported automatically on the Clinical Array Reader (Genomica).

LA detects the 13 high-risk, and 24 low-risk genotypes. The assay has an internal human β-globin control for sample sufficiency and assay performance. With the final volume of 50 μl, 12.5 μl of purified sample DNA and 4 μl of purified control DNA were added for each sample and control PCR reaction. PCR was performed on GeneAmp PCR system 9700 (Applied Biosystems, Foster City, CA). Twenty-five μl of PCR reaction were used for LA testing according to the manufacturer’s protocol. The results on the LA strips were read independently by DME and a scientific assistant. In case of disagreement, consensus was sought. The difference in the DNA input per test between LA and CLART reflects the manufacturer specifications.

HC2 analysis was undertaken on the SurePath post-quot material, in concordance with the manufacturer’s specifications. Samples were denatured manually prior to the analysis on the manual HC2 Modular system (Qiagen, Gaithersburg, MD, USA). On average, samples were denatured 17 (range: 3–23) days after having been received in the laboratory and stored according to the manufacturer’s recommendations prior to HC2 testing.

Statistical analyses

A sample was considered high-risk positive for HPV if at least one high-risk genotype was detected, and low-risk positive when at least one of the remaining 18 genotypes detectable by both assays (6, 11, 26, 40, 42, 53, 54, 61, 62, 66, 70, 71, 72, 73, 81, 82, 83, 84) was detected without any high-risk genotype. CLART automatically reports genotypes separately when they are detected in an “uncertainty” range, i.e. with weak signals. Reflecting routine practice in our facility, these genotypes are considered positive only if part of multiple infections. The same definition was used for LA in case of bands with weak signal intensity. A positive HC2 test result was defined as RLU/CO value ≥1.0.

Differences in the distribution of women’s characteristics for the three assays were calculated using the Χ² distribution. For all 31 HPV genotypes detectable by both assays, we calculated the κ-coefficients and proportions of overall and positive agreement. κ-coefficients >0.60 were considered to indicate “substantial” agreement [17]. Overall agreement was calculated as the proportion of all samples that returned the same test result on both CLART and LA (no genotype, or same genotype). Positive agreement was calculated as conditional probability that both assays detected a particular genotype if at least one did. The proportion of high-grade CIN (≥CIN2 or ≥ CIN3) with a positive test result on a particular HPV assay was used as an indicator of the assay’s clinical sensitivity. As an indicator of clinical specificity, we calculated the proportion of women testing negative among those without high-grade CIN; we assumed that women with cytology but without histology in follow-up had no high-grade CIN, and excluded women who were lost to follow-up. The 95% confidence intervals (CI) for sensitivity and specificity were calculated using binomial distribution. We calculated the relative clinical sensitivity and specificity for CLART by comparing its sensitivity and specificity to LA and HC2. The 95% CI for relative sensitivity and specificity, and for the relative prevalence (RP) of genotypes (CLART vs. LA), were calculated assuming that their logarithms were approximately normally distributed.

Ethical approval

LA and HC2 data were collected with informed consent as part of the Danish arm of a multicenter European trial (ClinicalTrial.gov Identifier: NCT01671462), approved by the Danish Capital Region Ethical Committee (H-2012-070). Informed consent was obtained by the sample taking gynecologists, and maintained in the women’s patient records, as well as in copy at the Department of Pathology in concordance with Danish Ethical guidelines. Additional testing on CLART, not used for clinical management, was undertaken as a quality development study, for which ethical approval and informed consent are not required, in concordance with the current Danish law.

Results

The 401 women were aged 17–78 years (mean 32.8, median 29). Most (N = 357, 89%) were in the screening age (23–65 years). ASCUS was diagnosed in 103 (26%) women, 161 had LSIL (40%), 30 ASC-H (7%), 106 HSIL (26%) and one (<1%) had cytological signs of carcinoma (Table 1). On average, women were followed for 17 months (range: 506–542 days). Seventeen (4%) were lost to follow-up. On CLART and LA, the proportion of high-risk genotypes decreased by age and increased by the severity of the cytologic interpretation; on HC2, the trends were not statistically significant. The differences between CLART and LA were not statistically significant. Between CLART and HC2, some differences were seen, particularly by age where more women aged ≥30 years had high-risk HPV genotypes detected on HC2 than on CLART. The differences in the distribution of test results in women lost to follow-up were not statistically significant.

Table 1 Description of the 401 women included in the study

Detection of HPV genotypes

In total, 311 (78%) women had high-risk genotypes on CLART, and 326 on LA (81%; RP: 0.95, 95% CI: 0.89-1.02). CLART detected statistically significantly fewer HPV 39, 45, 54, 62, and 73 infections than LA, whereas LA detected fewer HPV 58 and 82 (Table 2). For HPV 16, 18, 31, 33, 35, and 51, the agreement between CLART and LA was excellent (κ ≥ 0.90, overall agreement 98-99%, positive agreement 84-95%). For HPV 39, 45, 52, 56, 58, and 59, the agreement was substantial (κ ≥ 0.60, overall agreement 94-96%, positive agreement 46-64%); however, for HPV 68, the agreement was poor (κ = 0.26, overall agreement 93%, positive agreement 17%). For the 18 low-risk HPV genotypes detectable by both genotyping assays the agreement was in general good, although for genotypes HPV 54, 62, 73, and 82, the agreement was poor (κ < 0.60, overall agreement 93-96%, positive agreement 35-42%). However, these genotypes and HPV 68 were not highly prevalent in this population. This was similar in 125 women with ≥ CIN2 (treatment threshold in Denmark), with CLART detecting statistically significantly fewer HPV 45 infections than LA, RP: 0.35 (95% CI: 0.14-0.87; Table 3). CLART found single HPV infections in 130 (32%), and multiple infections in 235 (59%) women (Table 4). For LA, this was the case in 121 (30%) and 259 (65%), respectively. These differences were not statistically significant, RP for single infections: 1.07 (95% CI: 0.87-1.32). The κ was 0.64, with an overall agreement of 81% (95% CI: 77–85). For detecting high- and low-risk infections (Table 5), the κ was 0.76, with an overall agreement of 92% (95% CI: 88–94), and positive agreement (for detecting at least one high-risk genotype) of 92% (95% CI: 89–95). The differences in detecting high-risk infections overall (for detecting at least one high-risk genotype) were not significant, RP: 0.95 (95% CI: 0.89-1.02).

Table 2 Detection of individual high-risk and low-risk HPV genotypes by CLART and LA in 401 women with abnormal cytology

Table 3 Detection of individual high-risk HPV genotypes by CLART and LA in 125 women with ≥ CIN2

Table 4 Agreement between CLART and LA with respect to single and multiple infections in 401 women with abnormal cytology

Table 5 Agreement between CLART and LA in 401 women with abnormal cytology

The agreement with HC2 in detecting high-risk HPV infections was lower for both genotyping assays (Table 6): for CLART, κ = 0.45, overall agreement = 84% (95% CI: 80–87), and for LA, κ = 0.51, overall agreement 87% (95% CI: 84–90). Of the 355 HC2-positive samples, CLART detected only low-risk genotypes on 43 (12%), and no genotypes on 11 (3%). For LA, this was 36 (10%) and 4 (1%), respectively. Not surprisingly, the agreement with HC2 was better in women with ≥ CIN2.

Table 6 CLART and LA: agreement with HC2

Detection of cervical lesions

CLART detected 116 of 125 ≥ CIN2 (sensitivity: 93%, 95% CI: 87–97), and 84 of 90 ≥ CIN3 (sensitivity: 93%, 95% CI: 86–98; Table 7). LA detected 120 ≥ CIN2 (sensitivity: 96%, 95% CI: 91–99) and 87 ≥ CIN3 (sensitivity: 97%, 95% CI: 91–99). HC2 detected 123 ≥ CIN2 and 89 ≥ CIN3, sensitivity 98% (95% CI: 94–100), and 99% (95% CI: 94–100), respectively. These differences, assessed through relative sensitivity (Table 7), were not statistically significant. Three women with cervical cancer tested positive for high-risk HPV on all three assays. The fourth woman tested negative on CLART, and positive on LA (genotype 39) and HC2. Given that all women had cytological abnormalities, the specificity of all three assays was low, but significantly higher (assessed through relative specificity) for CLART (30%, 95% CI: 25–36, for ≥ CIN2) and LA (26%, 95% CI: 21–32) than for HC2 (17%, 95% CI: 12–22).

Table 7 CLART, LA, and HC2: sensitivity and specificity for ≥ CIN2 and ≥ CIN3

Discussion

In Danish women with abnormal cytology, CLART and LA were highly comparable for detection of HPV genotypes 16, 18, 31, 33, 35, and 51. Furthermore, “substantial” agreement was observed for HPV 39, 45, 52, 56, 58, and 59, which translated into ~50-60% of cases mutually detected by the two assays. Finally, the agreement was poor for HPV 68, present in <1% of cervical cancers [5]. There were no statistically significant differences in detecting high-risk HPV infections overall, and the two assays detected similar numbers of high-grade CIN. For low-risk genotypes, the differences were somewhat more pronounced, but still generally acceptable, although for genotypes HPV 54, 62, 71, 73, and 82 the agreement was poor. The agreement in detecting HPV infections with HC2, a thoroughly validated clinical screening assay [18], was moderate for both CLART and LA, but with no statistically significant differences in detecting high-grade CIN.

CLART was previously compared to LA using ThinPrep samples. Using 538 samples from women in opportunistic examination, Chranioti and colleagues found high levels of agreement in detecting the 13 high-risk HPV genotypes [19]. HPV 68 was though detected in only two samples, in which it was detected by both assays. Analytical performance of CLART and LA was reported as part of the WHO HPV LabNet Proficiency Studies [4,11,20]. In the most recent published evaluation [4], both assays had high analytical sensitivity for HPV 16 and 18, even at low plasmid concentration levels (5–50 international units/genomic equivalent). CLART more often correctly detected HPV 6, 11, 31, 33, 35, 51, 52, 58, 59, and 66 than LA, although the number of compared datasets was small. The opposite was observed for HPV 45, 56 and 68b, which were compared at high concentration levels (500 international units/genomic equivalent). Moreover, WHO LabNet panel data from 2011 showed similar performance in genotype detection between CLART and PapilloCheck [4]. Data from the most recent 2013 WHO global proficiency panel are awaited.

Pista and colleagues compared CLART to HC2 in women attending primary and gynecologic outpatient clinics, and found the same sensitivity for ≥ CIN2, 96%, with similar specificities (74% vs. 71%) [14]. In a study of women referred for colposcopy reported by Szarewski and colleagues, the sensitivity of LA for ≥ CIN2 was 98%, and that of HC2 100%. CLART did not perform optimally owing to “technical problems during the evaluation”, with sensitivity for ≥ CIN2 of only 81%. However, it should be noted that the study used an earlier version of the CLART assay. Furthermore, the accuracy of HPV test results using genotyping assays may improve with a laboratory’s experience with a particular assay, and differences between laboratories can be substantial [4].

Our study is the first comparison of genotype detection and clinical performance of CLART and LA using SurePath samples. It is also one of the first reports on LA with SurePath in general. Previously, Chernesky and colleagues studied 133 routine samples and found a 94% overall agreement in detecting high-risk HPV infections between LA and HC2, with κ = 0.86 [21]. This was substantially higher than in our study; however, the samples in the study by Chernesky and colleagues were tested in two laboratories, and the distribution of cytological abnormalities, an important determinant of agreement between HPV assays [15] was not reported.

One of the strengths of this study was the use of fresh, routine SurePath cytology samples from a large Danish cervical screening laboratory with well-established cytology performance. The genotyping assays were compared on equal terms: all testing was undertaken in the same laboratory by the same staff; samples were processed manually, and the analysis was limited to the 31 HPV genotypes that are detectable by both assays. Each LA hybridization strip was read by two experienced staff members, and discrepancies were resolved by consensus. Histological diagnoses were available for 86% of the women, with only 4% lost to follow-up.

Interpretation of the detection of genotypes with weak signals on both CLART and LA might be considered a weakness of our study in the sense that other laboratories may have opted for different approaches. Our current clinical standard operating procedure calls for weak signals (below cut off, but visible) to be considered positive if the weak signal is detected along with other genotypes detected above the cut off. This approach was though not playing a major role in our data; after including genotypes with weak signals, the test results changed from low-risk to high-risk positive in 7 samples for CLART, and 5 for LA, with 1 and 0 ≥ CIN2, respectively. This low number of ≥ CIN2 was consistent with previous observations of low numbers of CIN lesions found close to manufacturer-determined cut-offs for other HPV assays [22]. The LA and CLART package inserts do not provide information on how to interpret genotype findings with weak signals, leaving it up to the individual laboratory to establish their own interpretation algorithms. Our approach is justified by the fact that individual genotypes appear more difficult to detect in co-infections, as compared to single infections [4], owing to the dynamics of multiplex PCR reactions. In particular, HPV genotypes with high viral loads may through primer or reagent competition lessen the PCR amplification of other truly present genotypes in a sample. This issue and its consequences for the clinical management have been little discussed in the literature.

For use of any HPV assay in cervical screening, quality control and quality assurance aspects should also be considered. In this respect, sample identification is not provided for the individual LA strips, whereas CLART’s software stores sample-specific identification information printed on the individual array by the manufacturer, which is reported alongside the testing results from the automated reader. Hence, CLART has a state-of-the-art chain of custody and is not prone to inter-observer variability given the automated read-out.

Conclusions

In our referral population, CLART was comparable to LA in terms of analytical and clinical performance, and CLART’s clinical sensitivity was comparable to that of HC2, whereas its specificity was higher. In the absence of an internationally recognized genotyping gold standard, CLART HPV2 appears to be a good candidate for genotyping HPV infections in clinical settings where high throughput and chain of custody is required.