Prediction of resistance to chemotherapy in ovarian cancer: a systematic review

Background Patient response to chemotherapy for ovarian cancer is extremely heterogeneous and there are currently no tools to aid the prediction of sensitivity or resistance to chemotherapy and allow treatment stratification. Such a tool could greatly improve patient survival by identifying the most appropriate treatment on a patient-specific basis. Methods PubMed was searched for studies predicting response or resistance to chemotherapy using gene expression measurements of human tissue in ovarian cancer. Results 42 studies were identified and both the data collection and modelling methods were compared. The majority of studies utilised fresh-frozen or formalin-fixed paraffin-embedded tissue. Modelling techniques varied, the most popular being Cox proportional hazards regression and hierarchical clustering which were used by 17 and 11 studies respectively. The gene signatures identified by the various studies were not consistent, with very few genes being identified by more than two studies. Patient cohorts were often noted to be heterogeneous with respect to chemotherapy treatment undergone by patients. Conclusions A clinically applicable gene signature capable of predicting patient response to chemotherapy has not yet been identified. Research into a predictive, as opposed to prognostic, model could be highly beneficial and aid the identification of the most suitable treatment for patients. Electronic supplementary material The online version of this article (doi:10.1186/s12885-015-1101-8) contains supplementary material, which is available to authorized users.


Background
Ovarian cancer is the fifth most common cancer in women in the UK and accounted for 4% of cancer diagnoses in women between 2008 and 2010 [1]. Worryingly, it was also responsible for 6% of cancer-related deaths in women over the same time period [1] and the fiveyear survival of women diagnosed with ovarian cancer between 2005 and 2009 was 42% [2]. It has been observed that although 40%-60% of patients achieve complete clinical response to first-line chemotherapy treatment [3], around 50% of these patients relapse within 5 years [4] and only 10%-15% of patients presenting with advanced stage disease achieve long-term remission [5]. It is thought that the high relapse rate is at least in part due to resistance to chemotherapy, which may be inherent or acquired by altered gene expression [6]. For ovarian cancer in the UK, the standard of care for first-line chemotherapy treatment recommended by the National Institute for Health and Care Excellence is 'paclitaxel in combination with a platinum-based compound or platinum-based therapy alone' [7]. This uniform approach ignores the complexity of ovarian cancer histologic types, particularly as there is evidence to suggest differences in response [8]. Winter et al. [9] investigated the survival of patients following paclitaxel and platinum chemotherapy and found histology to be a significant predictor of overall survival in multivariate Cox proportional hazards regression.
Improvement in survival has also been poor in ovarian cancer. Between 1971 and 2007 there was a 38% increase in relative 10-year survival in breast cancer, whereas the increase in ovarian cancer was 17% [10]. This difference in progress is likely to be due, at least in part, to the lack of tools with which to predict chemotherapy response in ovarian cancer.
Gene expression based tools for the prediction of patient prognosis after surgery or chemotherapy are currently available for some cancers. For example, MammaPrint ® uses the expression of 70 genes to predict the likelihood of metastasis in breast cancer [11]. Similarly, the Oncotype DX ® assay uses the expression of a panel of 21 genes to predict recurrence after treatment of breast cancer [12]. The Oncotype DX assay is also available for colon [13] and prostate cancers [14]. The development of a similar tool for ovarian cancer could greatly improve patient prognosis and quality of life by guiding chemotherapy choices. The prediction of cancer prognosis using gene signatures is a popular research field, within which a wide variety of approaches have been considered. Popular RNA or protein expression measurement techniques include cDNA hybridisation microarrays, endpoint and quantitative reverse transcription PCR, and immunohistochemistry approaches.
Another variable aspect of studies predicting chemotherapy response is the computational and statistical approaches utilised. One of most popular methods for survival analysis is Cox proportional hazards regression. This model assumes that the hazard of death is proportional to the exponential of a linear predictor formed of the explanatory variables. This model has the advantage that, unlike many other regression techniques, it can appropriately deal with right-censored data such as that found in medical studies where patients leave before the end of the study period [15].
Other popular modelling techniques include linear models, support vector machines, hierarchical clustering, principal components analysis and the formation of a scoring algorithm. When dealing with data sets of varying sizes it is important to consider the number of samples and the amount of data per patient when choosing a modelling method. If the number of patients is large it is clear that a model will be better informed about the population from which the patient sample was drawn, and hence is likely to generalise more effectively to independent data sets. As the number of measurements per patient increases, the dimensionality and hence the flexibility of the model may increase. However, it is also important that the number of patients is sufficiently large to supply enough information about the factors being considered. Of the models identified here, linear models are relatively restrictive as the relationship between any factor and the outcome is assumed to be linear and so are suitable for smaller data sets. Conversely, hierarchical clustering simply finds groups of similar samples and there are minimal assumptions concerning the relationship between factors and outcome.
Classification models are used to predict which of a number of groups an individual falls into and are used for categorical variables, such as tumour grade and having or not having a disease. For visualisation and the assessment of classification model predictive power, a Kaplan-Meier plot is often combined with the log-rank test to investigate significance. It is worth noting that this method does not compare predictions with measurements, it simply considers the difference in survival between groups.
Many of the studies identified by this review involved developing a model using one set of samples, a training set, followed by testing of the model carried out on an independent set of samples, the test or validation set. This partitioning of samples is important as it allows the generalisability of the model to be assessed, and hence guards against over-fitting. If this check is not carried out, the true predictive ability of the model will not be known.
The aim of this review is to investigate the literature surrounding the prediction of chemotherapy response in ovarian cancer using gene expression. It has been observed, for example by Gillet et al. [16], that gene signatures obtained from cancer cell lines are not always relevant to in vivo studies, and that cell lines are inaccurate models of chemosensitivity [17]. The search was therefore restricted to studies involving human tissue in order to ensure that the resulting gene signatures are applicable in a clinical setting. It was also specified that the study must involve patients who have undergone chemotherapy treatment, so that the effects of resistance may be investigated.

Search methodology
The aim of this review is to investigate the literature on the prediction of chemoresistance in patients with ovarian cancer. Therefore, the six most important requirements identified were: • Concerned with (specifically) ovarian cancer • Patients were treated with chemotherapy • Gene expression was measured for use in predictions • Predictions are related to a measure of chemoresistance (e.g. response rates, progression-free survival) • Measurements were taken on human tissue (not cell lines) • The research aim is to develop a diagnostic tool or predict response A PubMed search was carried out on 6th August 2014 to identify studies fulfilling the above requirements. The search terms may be found in Additional file 1. This search resulted in 78 papers.

Filtering
The search results were filtered twice, once based on abstracts and once based on full texts, by KL. An overview of the filtering process may be found in Figure 1. For the abstract-based filtering, papers were excluded if the six essential criteria were not all met, if the paper was a review article or if the paper was non-English language. This resulted in 48 papers remaining. For the full-text-based filtering, exclusion was due to not fulfilling the search criteria or papers that were not available. 42 papers were remaining after full-text-based filtering.

Data extraction
Data was extracted using a pre-defined table created for the purpose. Extraction was carried out in duplicate by a single author (KL) with a wash-out period of 3 months to avoid bias. Variables extracted were: author, year, journal, number of samples, number of genes measured, study end-point, tissue source, percentage cancerous tissue, gene or protein expression measurement technique, sample histological types and stages, patient prior chemotherapy, modelling techniques applied, whether the model accounts for heterogeneity in patient chemotherapy, whether the model was prognostic or predictive, whether the model was validated, model predictive ability including any metrics or statistics, and the genes found to be predictive.

Bias analysis
Bias in the studies selected for the systematic review was assessed according to QUADAS-2 [18], a tool for the quality assessment of diagnostic accuracy studies. Levels of evidence were also assessed according to the CEBM 2011 Levels of Evidence [19]. Results of these analyses may be found in Additional files 2 and 3. Briefly, the majority of studies were considered to be low risk, with six studies judged to have unclear risk for at least one domain and seven studies judged to be high risk for at least one domain. Thirty-six studies where judged to have evidence of level 2, with the remaining six having evidence of level 3. These levels of risk and evidence suggest that the majority of conclusions drawn from these studies are representative and applicable to the review question.

Gene set enrichment
Gene set enrichment analysis was applied to the gene sets reported by the studies selected for this review. Analysis was performed using the R package HTSanalyseR [20]. Where reported, gene sets were extracted and combined according to the chemotherapy treatments applied to patients in each study. The two groups assessed were those studies where all patients were treated with platinum and taxane in combination, and those studies where  If more than one value is given, the study used multiple different starting gene-sets or found multiple gene signatures. NS: Not Specified.

Study Tissue source % Cancerous tissue
Jeong et al. [22] Lisowska et al. [23] Fresh-frozen NS Roque et al. [24] FFPE, Fresh-frozen min. 70% Li et al. [3] FFPE NS Schwede et al. [25] Verhaak et al. [26] Obermayr et al. [27] Fresh-frozen, Blood NS Han et al. [28] Hsu et al. [29] Lui et al. [30] Kang et al. [31] Gillet et al. [32] Fresh-frozen min. 75% Ferriss et al. [  patients were given treatments other than platinum and taxane. The second group includes those given platinum as a single agent. Any studies reporting treatments from both groups were excluded, as were studies that did not report the chemotherapy treatments used. Kyoto Encyclopedia of Genes and Genomes (KEGG) terms were identified for each gene and gene set collection analysis was carried out, which applies hypergeometric tests and gene set enrichment analysis. A p-value cut-off of 0.0001 was used. Enrichment maps were then plotted, using the 30 most significant KEGG terms. P-values were adjusted using the 'BH' correction [21].

Ethics statement
Ethical approval was not required for this systematic review, which deals exclusively with previously published data.

Results
Tables 1, 2, 3, 4, 5 and 6 detail some key information regarding the studies included in the review. Table 1 contains the number of samples analysed, the number of genes considered for the model, and the resulting genes retained as the predictive gene signature. Table 2 provides information about the tissue used for gene expression measurements and whether the studies assessed the percent neoplastic tissue before measurement, and Table 3 details the gene expression measurement techniques used. Table 4 contains the reported histological types and stages of the samples processed by each study. Table 5 provides information on chemotherapy treatments undergone by patients, whether the model was prognostic or predictive, and whether the model was validated using either an independent set of samples or cross validation. Table 6 lists the outcome to be predicted, the modelling techniques applied, and the predictive ability of the resulting model.

Tissue source
For studies involving RNA extraction the tissue source is an important consideration, as RNA degradation and fragmentation could affect the results of techniques involving amplification. This is a notable issue in formalin fixed paraffin embedded (FFPE) tissue, due to the cross-linking of genetic material and proteins [63]. Of the 42 papers included in this review, the majority used fresh-frozen biopsy tissue. The numbers of each tissue source may be found in Table 7, and the tissue source used by individual papers may be found in Table 2. Nine papers did not use an RNA source directly as secondary data was used. Data sources were mostly other studies or data repositories, such as the TCGA dataset. Two studies did not specify the source tissue though extraction and expression measurement methods were detailed.
The majority of papers in this review used fresh-frozen tissue. This choice was likely made to minimise RNA degradation and hence improve measurement accuracy. Due to the risk of RNA degradation because of long storage times and the fixing process applied to FFPE tissue, it is often expected that FFPE tissue will be irreversibly cross-linked and fragmented. However, following investigation into RNA integrity when extracted from paired FFPE and fresh-frozen tissue, Rentoft et al. [64] found that for most samples up-and down-regulation of four genes was found to be the same whether measured in FFPE or fresh-frozen tissue. They concluded that, if samples were screened to ensure RNA quality, FFPE material can successfully provide RNA for gene expression measurement.
The use of fresh-frozen tissue in a research setting is not unusual, as can be seen from the fact that this tissue type was most popular in this review. However, for translational research expected to lead to a clinical test, this is not as reasonable. FFPE tissue is much more readily available, due to simpler acquisition and storage, and tissue is already taken for histological analysis. Therefore a model capable of using data obtained from FFPE tissue is much more likely to be applicable in a clinical setting.
Another important consideration is the proportion of neoplastic cells in the sample. For each paper the reported proportion may be seen in Table 2. Of the 42 papers, 14 reported that the proportion of cancerous cells was measured. This was usually done using hematoxylin and eosin stained histologic slides. It is important for the gene expression measurement that the tissue used contains a high proportion of neoplastic cells, and hence it is important that this pre-analytical variable is controlled. Of the studies in this review, those reporting the percentage cancerous cells were evenly distributed between FFPE and fresh-frozen tissues.

Gene or protein expression quantification
Of the studies highlighted by this review, there were four main techniques applied for gene or protein expression measurement: Probe-target hybridization microarrays, quantitative PCR, reverse transcription end-point-PCR, and immunohistochemical staining. Of these methods only immunohistochemistry measures protein expression, via classification of the level of staining, and the other methods quantify gene expression via measurement of mRNA copy number.
Methods involving probe-target hybridization are available commercially, and 19 of the 42 studies utilised these. For example the Affymetrix ® Human U133A 2.0 GeneChip and the Agilent ® Whole Human Genome Oligo Microarray were both used by multiple studies. Additionally, 7 studies used custom-made probe-target hybridization arrays. Probe-target hybridisation arrays generally measure thousands of genes and hence can provide a   wealth data per sample. TaqMan ® microfluidic arrays or quantitative-PCR were used by 16 studies. These techniques are typically used for smaller panels of genes. The TaqMan ® arrays for example may contain up to 384 genes per array. These methods are more targeted and hence the price per sample is usually lower. Immunohistochemistry is a more labour-intensive technique, requiring staining for each gene considered, and hence was mostly only used by studies using small numbers of genes. This technique, which is semi-quantitative due to the scoring systems employed, also suffers from a lack of standardisation of procedures. Of the 11 papers using this technique, the maximum number of genes analysed was seven, and the mean number of genes assessed was 2.8. Although these studies provide useful information regarding the correlation of particular genes with outcome, the small numbers of genes is likely to result in an incomplete gene signature and low predictive power.
Several of the papers utilising quantifiable techniques used an alternative method or replicates to obtain a measure of the assay variability. Five papers involving commercial or custom microarrays also used reverse transcription PCR (RT-PCR) to measure the expression of a small number of genes for comparison and one study used samples run in duplicate to calculate the coefficient of variation. Of the studies using TaqMan microfluidic arrays, two used samples run in duplicate to obtain the coefficient of variation. However, even fewer papers reported a metric representing the level of variability found. Two studies reported a coefficient of variation; Glaysher et al. [41] reported CoV = 2% = 0.02 for TaqMan arrays and Hartmann et al. [57] reported CoV = 0.2 for their custom microarray. Another two reported Spearman's or Pearson's r coefficients of correlation between microarray and RT-PCR results. Yoshihara et al. [43] gave Pearson r values ranging from 0.5 to 0.8, and Crijns et al. [47] gave Spearman's r values between -0.6 and -0.9. Table 4 details the histology (types and stages) of the patient samples used by each study. As may be seen, the majority of studies were heterogeneous with respect to the types of cancer included. However, 23 of the 42 studies used at least 80% serous samples, suggesting that the majority of information contributed to the gene signatures of these studies is related to the mechanisms and pathways in serous cancer. In the authors' opinion it is important to identify the histologies of patient samples: although treatment is currently the same across types, response to chemotherapy has been found to vary [9,65,66]. It therefore may be advisable for future studies to include histological information when developing models predicting chemotherapy response. Table 5 lists the chemotherapy treatments undergone by patients in each study. The 10 papers labelled NS did not specify the regimen applied, though the patients did have chemotherapy. These cohorts cannot therefore be assumed to be homogeneous with respect to patient chemotherapy treatment. All studies that specified the chemotherapy regimen undergone by patients noted at least one platinum-based treatment. Of these, 24 included    patients treated with a platinum-taxane combination and 10 with a cyclophosphamide-platinum combination. It is important to note that 19 of the 42 papers stated the population was heterogeneous with regards to chemotherapy treatments and, of those that did, only 8 included patient treatment history as a feature of the study. The aims of the majority of the studies were to identify genes of which the expression may be used to predict survival time, or prognosis. As already noted, the presence of resistance to the chemotherapy agent administered will dramatically affect the survival of a patient. It is therefore reasonable to expect the gene signatures identified to include genes responsible for chemoresistance, which will depend on the mechanism of action of the drug. Using a heterogeneous cohort in terms of chemotherapy treatment may then be causing problems with the identification of a minimal predictive gene set.

End-point to be predicted
As may be expected, there was variation between the end-point chosen by studies for prediction. Popular endpoints include overall survival, progression-free survival and response to chemotherapy. The endpoints considered by each study may be found in Table 6. Of these some are clinical endpoints, such as overall survival, others use non-clinical endpoints, such as response to chemotherapy, many of which are considered to be surrogates for overall survival. For cancer studies, overall survival is considered to be the most reliable and is the variable that is of most interest when considering the effect of an intervention.

Model development
Within this review, many different modelling techniques were used to identify an explanatory gene signature to predict patient outcome. The most popular was Cox proportional hazards regression, which was applied by 17 studies. This was closely followed by hierarchical clustering, which was used by 11 studies. All other methods were used by 8 or fewer studies. In total 24 different types of modelling techniques were applied, ranging from statistical tests such as Student's T test and Mann-Whitney U test, to logistic regression, to ridge regression. Table 8 lists the modelling techniques identified and the number of studies that employed them. It is of interest that most of the techniques applied are forms of classification. These methods result in samples being assigned to groups, such as 'good prognosis' and 'poor prognosis' . Whilst this may be useful in some settings, for a clinically-applicable tool a regression technique may be more appropriate as it will provide a value, such as a likelihood of relapse, rather than simply a class. Techniques in Table 8 capable of a numeric prediction include logistic and linear regression, Cox proportional hazards regression, and ridge regression. Jointly with the modelling methods identified above, 23 of the 42 studies implemented Kaplan-Meier curves to visualise the survival of the patient classes identified by the models. This enables the difference in survival between classes, for example 'good prognosis' and 'poor prognosis' , to be seen and assessed. The application of a log-rank test assesses the separation of the curves and identifies whether there is a statistically significant difference in survival distribution between the classes. It should be noted that, although this gives an idea of separation of classes achieved by the model, the model results must still be compared with known outcomes to check positive and negative predictive power. This step was missing in several papers, such as Gillet et al. [38], where the p value returned by the log-rank test is given as the measure of model success.
It is important to highlight the difference between prognostic and predictive models. A prognostic model is one capable of predicting prognosis, such as survival time, using patient information and biomarkers and does not vary between different treatment options. In contrast, a predictive model is one able to predict the effect of a treatment on patient prognosis [67,68]. It is therefore clear that, although prognostic models may be useful for research purposes and when one treatment option is available (such as the standard platinum-taxane combination), predictive models have a much greater part to play in stratified medicine where the aim is to identify the most appropriate treatment on a patient-by-patient basis. In order for a model to be predictive, the effects of multiple treatments must be considered and the response compared with the biomarker status. Classification of the studies as prognostic or predictive may be seen in Table 5. Of the papers identified by this review, only a minority considered the effects of chemotherapy treatment on the predicted outcome and hence could be considered predictive. Glaysher et al. [41] and Vogt et al. [62] produced separate models for various treatments, allowing the effects of different drugs and combinations to be compared. Both studies applied drugs in vitro to cultured tissue to measure response to chemotherapy. This was combined with gene expression measurements to form the model training data set. In this way the same patient samples may be used to create a set of models predicting response to a variety of drugs. These models are therefore predictive rather than prognostic. Alternatively, models may be trained on sets of patients split by treatments undergone, which would lead to treatment-specific models predicting response to the particular drug. This method was used by Jeong et al. [22], Ferriss et al. [33], Williams et al. [44] and Matsumura et al. [46]. Additionally, the use of a model variable specifying patient treatment history could allow these models to be combined onto one using a single training set of all patients. The model may then be passed a variable specifying the drug of interest for resistance prediction. A simple version of this method was implemented by Crijns et al. [47], who included a feature for whether a patient was treated with paclitaxel. It is clear that the integration of patient chemotherapy treatment into these models is underused, and it is likely to be beneficial for this to be incorporated into future research.

Genes identified
Of the 42 papers in this review, 32 provided full or partial lists of the genes identified by their models. Of the remainder, it was common that the gene sets were large or that the genes were not explicitly identified by the model, as is the case with modelling techniques such as principal components analysis.
In total across the papers, 1298 unique genes were selected by models and of these 93.53% were found by only one paper. The most commonly chosen gene was selected by only four papers. Table 9 shows the numbers and percentages of genes chosen by one to four papers.
A list of the genes identified by the papers in the review may be found in Table 10.
It is clear that the gene sets selected by the studies are very different and there is very little overlap. The genes chosen by two or more studies may be seen in Table 11. Many of these genes are known to have links to cancer, which may suggest that these genes are therefore implicated in ovarian cancer. It is possible that, although the genes selected varied, they in fact represent similar mechanisms. This could occur if there are large sets of highly covariate genes representing particular cellular processes and the genes in the signatures were simply random selections from these gene sets. The same gene being selected by multiple papers would then be unlikely, although the same information contribution would be made. It may then be more informative to assess and compare the mechanisms controlled by the genes chosen as part of the models.

Table 10 List of genes reported by studies included in this review
Gene names have been standardised. Genes in bold were selected by more than two studies.
Seidal [35], Schlumbrecht et al. [40], Yoshihara et al. [43], Denkert et al. [45], Hartmann et al. [57], Iba et al. [60], and Kamazawa et al. [61]. Studies falling into the other treatments group were Obermayr et al. [27], Sabatier et al. [27], Yan et al. [42], Netinatsunthorn et al. [51], and Helleman et al. [53]. The results of the gene set enrichment using the KEGG system may be seen in Figures 2 and 3. From the plots, it may be seen that both groups identify several cancer-related pathways relevant to the drug mechanisms of action.  It is informative to consider the KEGG terms in the context of the mechanisms of action of the chemotherapy drugs applied. Both groups contain patients treated with platinum single agents or platinum-containing combinations. It should therefore be expected that processes associated with the mechanism of action of Figure 2 Gene set enrichment networks for studies assessing ovarian cancer patients treated with platinum and taxane. Network maps of the 30 most enriched KEGG pathways. Node marker size signifies the number of genes in this category, and the thickness of edges indicate the Jaccard similarity coefficient between categories. Node markers are coloured according to adjusted p value as reported by the hypergeometric test, where darker red denotes more highly significant. platinum will be enriched. Once activated, the platinum binds to DNA and results in the formation of monoadducts, intra-strand crosslinking, inter-strand crosslinking and protein crosslinking. This DNA structure change affects the ability of the DNA to be unwound and replicated, resulting in the triggering of the G2-M DNA damage checkpoint and cell cycle arrest. The affected cell will attempt DNA repair and, if unsuccessful, undergo apoptosis [69]. Expected KEGG terms therefore include those relating to apoptosis and DNA damage.
From Figure 2, KEGG pathways highlighted for this group of studies include ten cancer-specific terms and six cancer-related terms. Here italics denote a KEGG term. The ErbB signalling pathway has been found to influence in proliferation, migration, differentiation and apoptosis in cancer [70] and overexpression of ERBB1 and ERBB2 have been implicated in head and neck and breast cancers. The neurotrophin signalling pathway is known to trigger MAPK and PI3K signalling, affecting differentiation, proliferation and development, and survival, growth, motility and angiogenesis respectively [71]. Altered expression of Figure 3 Gene set enrichment networks for studies assessing ovarian cancer patients treated with treatments other than platinum and taxane. Network maps of the 30 most enriched KEGG pathways. Node marker size signifies the number of genes in this category, and the thickness of edges indicate the Jaccard similarity coefficient between categories. Node markers are coloured according to adjusted p value as reported by the hypergeometric test, where darker red denotes more highly significant. genes in this pathway has been found to correlate with poorer survival in colon, breast, lung and prostate cancers. Changes in expression of genes relating to focal adhesion, which is responsible for attachment of cells to the extracellular matrix, have been implicated in cancer migration, invasion, survival and growth [72]. The TGF-beta signalling pathway also regulates many cellular processes, including proliferation, cellular adhesion and motility, coregulation of telomerase function, regulation of apoptosis, angiogenesis, immunosuppression and DNA repair [73]. The p53 signalling pathway has many varied links to cancer. This pathway many be triggered by various stress signals and can result in several responses, including cell cycle arrest, apoptosis, the inhibition of angiogenesis and metastasis, and DNA repair [74]. Finally, nucleotide excision repair is known to promote cancer development when both up and down regulated. Downregulation correlates is thought to increases susceptibility to mutation formation and hence the formation of cancer [75], whereas up-regulation has been found to correlate with resistance to platinum as the DNA damage caused by the chemotherapy agent is repaired [76].
The first group of studies considered patients treated with taxanes in addition to platinum. Taxanes act by stabilising tubulin, preventing the microtubule structure formation required for mitosis. This results in cell cycle arrest at the G2/M DNA damage checkpoint and apoptosis. Mechanisms for taxane resistance are, however, not well understood. Two suggested mechanisms include the increased expression of multidrug transporters, and changes in the expression of the β-tubulin isoforms [77]. Neither of these mechanisms seem to be enriched in the platinum and taxol group. In addition to the single-agent effects of platinum and taxanes, there is an additional synergistic effect [78]. However, this effect is also not well studied and hence the mechanisms by which this occurs are not clear.
The second group, as seen in Figure 3, was composed of studies applying chemotherapy treatments other than platinum and taxanes. This group is heterogeneous with respect to chemotherapy treatment, and mainly consists of studies reporting treatment as 'platinum-based' . The other drug explicitly mentioned by studies in this group is cyclophosphamide. This drug is an alkylating agent and acts to form adducts in DNA [79]. This DNA damage triggers the G2/M DNA damage checkpoint, resulting in DNA repair or apoptosis. This suggests that the same DNA repair mechanisms related to platinum treatment are also relevant to cyclophosphamide. For this group, the KEGG pathway analysis shows that the gene set is enriched with 14 pathways related to cancer, in addition to two general cancer-related terms. The mTOR signalling pathway is downstream to the PI3K/AKT pathway and regulates growth, proliferation and survival [80]. The MAPK signalling pathway controls the cell cycle, and has been found to contribute to the control of proliferation, differentiation, apoptosis, migration and inflammation in cancer [81]. The chemokine signalling pathway has been found to regulate growth, survival and migration in addition to its role in inflammation [82]. Angiogenesis and vasculogenesis are known to be regulated by the VEGF signalling pathway [83], which is already the target of treatments such as bevacizumab. Purine metabolism is required for the production and recycling of adenine and guanine, and hence is required for DNA replication. This process is the target of chemotherapies such as methotrexate. The term drug metabolism -other enzymes is partially cancer related; this term refers to five drugs: azathioprine, 6-mercaptopurine, irinotecan, fluorouracil and isoniazid. Of these, two are chemotherapy treatments; irinotecan is a topoisomerase-I inhibitor and fluorouracil acts as a purine analogue. Also featuring in Figure 3 are apoptosis, ErbB signalling pathway, focal adhesion, neurotrophin signalling pathway, B cell receptor signalling pathway and Jak-STAT signalling pathway, all of which are known to be related to cancer.
Overall, the gene sets appear to be enriched for cancerrelated resistance mechanisms [84]. However, when combined there is little evidence from this analysis to suggest that the signatures are capturing chemotherapy-specific mechanisms in addition to more general survival pathways. The DNA repair terms may suggest a response to platinum-based treatment, though the down-regulation of these mechanisms is also related to cancer development and resistance in general [85]. It is likely that, due to the varying reliability suggested by the bias analysis and the reported model development techniques, the signalto-noise ratio of informative genes is low when the gene signatures are combined, preventing the identification of processes of interest.

Model predictive ability Sensitivity and specificity
The comparison of the success of the various models is difficult, particularly due to the fact that many papers report different metrics as measures of model accuracy. Many of these are also incomplete, not providing enough information to fully describe the model. Ideally, models should be applied to an independent set of samples with known outcomes and performance measures on this data set reported. For classification models an informative set of measures would be positive predictive value, negative predictive value, specificity and sensitivity: Sensitivity = n true positive n true positive + n false negative Specificity = n true negative n true negative + n false positive PPV = n true positive n true positive + n false positive NPV = n true negative n true negative + n false negative where n true positive is the number of true positive predictions, n false positive is the number of false positive predictions, n true negative is the number of true negative predictions and n false negative is the number of false negative predictions.
Together these provide information on true positive and negative rates as well as false positive and false negative rates, all of which are important when assessing the performance of a model.
Using the sensitivity and specificity the positive and negative likelihood ratios may be calculated and, using the prevalence of the condition in the test population, the probability of a patient having the condition based on the test results may be found, as in the equations below.
1−P(Condition−) · LR -ve + 1 These post-test probabilities are much easier to interpret and incorporate the prevalence of the condition. It should be noted that in order for the test to be applied in a clinical situation the pre-test probabilities used, P(Condition+) and P(Condition−), should be correct for the population of patients to whom the test will be applied.
Here the sample prevalence from each study was used for convenience. However, it would be informative to recalculate P(Condition + |Test+) and P(Condition + |Test−) for the general population of ovarian cancer patients, as this would provide a better comparison between models. Table 12 details the post-test probabilities of patients having a condition based on a positive or negative test result from the models developed by studies in this review. The papers appearing here are those that supplied sensitivity and specificity and the numbers of patients with and with without the condition, or alternative information allowing these to be calculated such as numbers of true and false positives and negatives.
From the table it may be seen that there is a great variety between the success of the models. For example, Kamazawa et al. [61] and Hartmann et al. [57] both achieved P(Condition + |Test+) = 0.95 on their respective samples of the population. This means that if a patient tests positive, there is a 95% probability that they are positive for the condition in question, which in these cases are 'responding to chemotherapy' and 'poor prognosis' respectively. In contrast, Obermayr et al. [27], Helleman et al. [53] and Gevaert et al. [49] only achieved P(Condition + |Test+) of between 0.20 and 0.40. These results suggest that the tests are not able to predict the outcome of a patient any better than a random choice, and in the case of tests in the region of 0.20 it is likely that most patients are simply assigned to the same class.
The ability of tests to not commit type II errors and give false negatives is also important. Ferriss et al. [33] and Hartmann et al. [57] both achieved well in this regard, with P(Condition + |Test−) = 0.07 and P(Condition + |Test−) = 0.05 respectively. Several studies, by contrast, had very poor probabilities of false negatives; Obermayr et al. [27], Helleman et al. [53] and Gevaert et al. [49] all have P(Condition + |Test−) > 0.5, which suggests that these models give a false negative more often than a random assignment.
Kamazawa et al. [61] and Selvanayagam et al. [59] both achieved extremely impressive prediction abilities, as may be seen by the very large P(Condition + |Test+) and very small P(Condition + |Test−) values. However, these studies exemplify why care must be taken in assessing the predictive ability of models. Both studies calculated sensitivity and specificity based on only training set results and hence there is no way to judge the generalisability of the models. There is a tendency for models to perform better on the training set than any following independent data set to which it is subsequently applied. Secondly, the training set used by Selvanayagam et al. [59] is extremely small at eight patients and has a 50 : 50 ratio of chemoresistant to chemosensitive patients. This sample is not representative of the population and hence the values of P(Condition + |Test+) and P(Condition + |Test−) will be skewed by unrepresentative P(Condition+) and P(Condition−).
Overall, the most successful model of this group is that by Hartmann et al. [57] as it makes predictions with good reliability and has been validated on an independent data set. The least successful models were Obermayr et al. [27], Helleman et al. [53] and Gevaert et al. [49]. These studies suffered from low ability to identify true positives and high probability of false positives, resulting in poor predictive ability.

Hazard ratios
It is common for studies of survival to quote hazard ratios comparing the results of clusters identified by classification models or relative-risk models such as Cox proportional hazards regression. These ratios represent the ratio of the probability of an event occurring to a patient in each of the two groups. The event is often death, but could also be recurrence for example. The studies listed in Table 13 supplied hazard ratios as measures of predictive ability. The hazard ratios vary from 0.23 to 4.6 with the majority around 2 to 3. A hazard ratio that is not equal to 1 suggests that the variable has predictive ability, and a ratio of 4, for example, suggests that a member of the high-risk group is 4 times as likely to die within the study period than a member of the low-risk group. The study with the highest hazard ratio is Spentzos et al. [58], with HR = 4.6. This is closely followed by Raspollini [56] with HR = 0.23 and Skirnisdottir and Seidal [35] with HR = 4.12. The confidence intervals on the hazard ratios of all the studies are large and, with the exception of Spentzos et al. [58], at the lowest edge the hazard ratio is very close to 1. This suggests that, although all these hazard ratios were found to be significant, some were close to not reaching the arbitrary 5% level. Most notable are Roque et al. [24], Schlumbrecht and Seidal [40], and Denkert et al. [45]. These models would need further investigation to determine their predictive ability. Of the papers in this group, Spentzos et al. [58] appears to have the best predictive ability when classifying patients into two clusters with significantly different survival times.

Linear regression
Two papers reported the success of model assessed using linear regression: Glaysher et al. [41] and Kang et al. [31]. These studies plotted the predicted values or model score against the measured values and applied linear regression to obtain a line of best fit. The R 2 or R 2 adj of this line is then calculated to assess the discrimination of the model. Glaysher et al. [41] achieved R 2 = 0.901 (R 2 adj = 0.836) for a model predicting resistance to cisplatin via crossvalidation and Kang et al. [31] achieved R 2 = 0.84 for a model predicting recurrence-free survival in the data set on which it was derived. These values suggest a good level of predictive ability, both in terms of calibration and discrimination, with the model by Glaysher et al. [41] achieving the better predictions.

Cox proportional hazards models
When studies identified by this review applied the Cox proportional hazards model to predict patient outcome, it was common for the main analysis of the model to be assessing whether the gene signature was found to be significant and whether the signature was an independent predictor. However, the application of this model to an independent data set was much less common. As may be seen from Table 6, the success of many models was judged using the significance of covariates including the gene signature in the model. It is likely that this model was not applied to external data sets due to subtleties in what the model predicts when compared to methods such as linear regression. Whereas in linear regression the survival times are predicted directly, Cox proportional hazards regression predicts hazard ratios. Royston and Altman [86] developed techniques for the external validation of Cox proportional hazards models by application to an independent data set. These rely on having at least the weights of the variables included in the linear predictor, and ideally the baseline survival function. The first allows the assessment of the discriminatory power of a model, whereas the second is also required to allow the calibration of the model to be assessed. Royston and Altman [86] are of the opinion that the inclusion of a log-rank test p-value is not informative due to the irrelevance of the null hypothesis being tested, and hence this should not be considered when judging model performance. An alternative to the log-rank test to compare survival between groups would be time-dependent ROC curves [87].

Failure to predict
Of the studies identified by this review, some models failed to achieve significant predictive ability. These include Lisowska et al. [23], Vogt et al. [62] and Brun et al. [34]. Of these papers, Vogt et al. [62] and Brun et al. [34] both considered small numbers of genes when constructing their models. It is possible then that these models failed because no informative genes were considered. Conversely, Lisowska [23] applied their modelling technique to over 47000 genes using 127 patients. It is therefore a possibility that genes were selected by their model purely by chance rather than due to true explanatory ability. This model was tested using an independent data. When the model was applied to this data set it performed poorly, suggesting that the genes chosen did not generalise to the second cohort of patients. Neither Vogt et al. [62] nor Brun et al. [34] reported measuring the precision or accuracy of the gene expression measurements. Lisowska et al. [23] used RT-PCR to measure the expression of 18 genes from the microarray, but the RT-PCR measurements were carried out on a separate set of samples and hence are not useful when considering accuracy. It is therefore unknown whether the gene expression measurement techniques applied by these studies were sufficiently accurate.

Discussion
The papers identified as part of this review tackled the important issue of chemoresistance and survival prediction in ovarian cancer via gene or protein expression. The concept of identifying gene signatures is popular, but requires careful handling to extract the information required for this to be successful. It was observed that of the many different tissue preservation techniques applied, the most common were fresh-frozen and formalin fixed, paraffin embedded tissue. It is our opinion that, due to the high quality expression measurements that may now be achieved with FFPE tissue, this is the most appropriate choice for research intended to translate into a clinical setting.
It was found that the majority of the studies included in this review were heterogeneous with respect to the histological type of the patient cohort. This suggests that, due to the differing response of different types of ovarian cancer to chemotherapy, the gene signatures may be identifying different pathways and mechanisms. However, it should also be noted that although 27 of the 42 studies were heterogeneous, 12 of these consisted of greater than 80% serous samples. Therefore, for these studies the inclusion of multiple histological types is likely to have less effect on the gene signature and mechanisms highlighted could be expected to occur in serous ovarian cancer. It would be advisable for future studies to include histological type and grade as model features.
The majority of studies identified by this review attempt to classify patients into groups with different characteristics, for example 'poor prognosis' and 'good prognosis' or 'chemosensitive' and 'chemoresistant' . However, variables such as response to chemotherapy and prognosis are rarely so well separated into classes; they are by nature continuous variables. Altman and Royston [88] are clear that dichotomising continuous variables into categories (such as high-risk vs. low-risk) should be avoided, as it results in loss of information and may lead to underestimation of variation and the masking of non-linearity. Arbitrary choices of cutoff values may further obscure the situation, when the original continuous variable could serve the same purpose in many models. In terms of a clinical test it therefore may be more appropriate to apply alternative techniques, such as various types of regression, to obtain a real valued prediction of patient outcome.
It was noted that the metrics reported as measures of predictive ability vary between studies. These vary in the amount of information conveyed and hence care should be taken to use metrics that fully describe the model. Sensitivity and specificity are commonly reported for classification techniques and, together with the numbers of patients in each class in the data set, allows the probabilities of a patient having the condition of interest given that they have tested positive or negative. It is the ultimate aim of most classification studies to obtain these probabilities, as it allows the predictive ability of the test to be assessed and the applicability of the test to be evaluated. Of the studies reporting sensitivity, specificity and related information, the best predictive ability was achieved by Hartmann et al. [57] and the worst by Helleman et al. [53]. It is important to note that from the sensitivity and specificity the model by Helleman et al. [53] does not appear to be any worse than some of the others, but these probabilities incorporate the prevalence of the condition of interest in the test population. It would therefore be highly informative to recalculate these probabilities using the prevalence of the condition in the population of ovarian cancer patients. Since some of the test populations were not representative of the overall population (having so called 'spectrum bias'), this would give a much more reliable indication of the predictive ability of the models in a clinical setting.
One of the main aims of the studies identified was to obtain a 'gene signature' , the expression of which can explain and predict the response in the patient. To this end, the majority of the papers (32 of 42) provided full or partial list of the genes selected by the modelling process. An analysis of these gene signatures resulted in the conclusion that the signatures were very dissimilar, with the most commonly selected gene appearing in only four papers. 93.53% of genes were selected by only one paper. This seems to indicate that the gene signatures identified were not based on underlying cellular processes, or at least that the processes being highlighted were not the same across the papers. It should be noted that many of the studies used cohorts of patients who were heterogeneous in terms of chemotherapy treatment and, due to the development of resistance to chemotherapy via gene expression changes, this may affect the genes found to be explanatory. It may be that several gene signatures from sub-populations of patients treated with different drugs are combining and hence reducing the predictive ability of the models.
In order to assess the biological relevance of the genes selected for the gene signatures, gene set enrichment analysis was carried out. This technique is used to highlight processes and pathways that are over-represented in the gene signature compared to the set of all genes. For the purposes of this review, two groups of studies were considered: those where the patients were treated with platinum and taxane, and those where the patients were treated with other platinum based treatments. These groups were selected due to the low numbers of studies using a single treatment option. For example, there were no studies considering platinum, taxane or cyclophosphamide as single agents. Following the analysis, 30 KEGG terms were returned for each group. Of these, each list comprised of approximately half cancer related terms. Of these the majority were processes often up-or down-regulated in cancer cells, such as proliferation, apoptosis, and motility and metastasis [89]. It is unclear whether the change in regulation of these processes is further altered in response to specific chemotherapy treatments. However, one process worthy of additional consideration is DNA repair. DNA repair is known to be an important mechanism in cancer both though cancer development when downregulated or mutated [75] and resistance to DNA damaging chemotherapy when up-regulated [76]. Therefore, the strong presence of DNA repair terms may suggest the presence of platinum resistance pathways in the gene signatures. It is the authors' opinion that, although the combined gene signatures appear not to include predictive chemotherapy-specific information, they may be capable of providing prognostic information. It is also thought that some studies, such as Glaysher et al., may include genes relevant to additional chemotherapy-specific processes which are 'drowned out' when combined with other signatures.

Conclusion
It is clear that the prediction of response to chemotherapy in ovarian cancer is an ongoing research problem that has been attracting attention for many years. However, although many studies have been published, a clinical tool is still not available. It is our belief that, although not yet accomplished, progress within the field suggests that the development of a predictive model is possible. There is great variability between the approaches and success of existing studies in the literature, and there have been very high levels of variation in the genes identified as explanatory. It is the authors' opinion that, if more care is taken when selecting the patients for inclusion to control for treatment history, these gene signatures may be simplified and models able to predict response to treatment may be developed.