MC1R, nevi and personal history of non-melanoma skin cancer were identified as the strongest predictors of melanoma risk in our study of early-onset melanoma. The contribution of MC1R to prediction of melanoma was similar to that obtained from measuring self-reported nevi, which is considered a strong and discriminative risk factor [5, 33, 34]. When added separately to the base model, MC1R and self-reported nevi increased the AUC by 6% and 5% respectively, and both improved classification for about a quarter of the cases and controls through net movement of cases into higher quartiles and controls into lower quartiles of predicted risk. Total number of nevi measured by physicians (dermatology trainees) was the strongest predictor of risk overall, increasing the AUC by 11% and reclassifying 39% of participants. Although MC1R genotype is strongly associated with skin and hair phenotype [12, 14], it was a better predictor of early-onset melanoma than was pigmentation characteristics.
Our models demonstrated high discrimination: an AUC of 0.78 for the self-reported model and 0.83 for the physician-measured model in the Australian (development) dataset and 0.71 and 0.79 in the English (validation) dataset. The additional predictive value of MC1R, nevi and non-melanoma skin cancer variables when added to the demographic ‘base’ model was similar for both studies, suggesting good generalisability of our results to other genetically similar populations. The differences in the AUC for the base models of the two studies is less important, as it is strongly influenced by how age, sex, ethnic and regional differences have already been accounted for in the study design. It is expected that models will perform better on the development dataset than the validation dataset because of overfitting.
Three other, preliminary, melanoma risk prediction models containing MC1R genotype have been published. Whiteman and Green  published a prototype for a melanoma risk prediction tool but provided no details on predictive performance. In a published conference abstract, Smith et al. showed an AUC of 0.72 (95% CI 0.70-0.75) for a model containing conventional risk factors and 0.75 (95% CI 0.72-0.77) when they added MC1R genotype, outdoor UV and indoor UV exposure, based on data from a case–control study of people aged 25–59 years in Minnesota. In a Greek study with 284 cases and 284 controls, Stefanaki and colleagues  derived a melanoma risk prediction model containing phenotypic traits (except nevi was not available) and 8 single nucleotide polymorphisms (SNPs) from several genes that included the MC1R locus. They found no appreciable change to the AUC after the addition of the 8 SNPs (AUC changed from 0.833 to 0.839), which they suggested may have been partly due to lower risk allele frequencies in their Greek population compared to other European populations . Measurement of single SNPs rather than the causal variants might also underestimate the contribution of genetic variation to melanoma risk.
Other published risk prediction models for melanoma have reported AUCs in the range of 0.54 to 0.86 [34–41]. The highest reported AUC of 0.86 was for a model containing age, hair colour, personal history of melanoma and suspicious melanocytic lesion on dermoscopy, developed using a German cohort . We were not able to include personal history of melanoma in our models because the study eligibility criteria specifically excluded these cases; however previous primary melanoma would be rare in people younger than 40 years and thus would be less relevant to our analysis than in studies with older participants. Previous non-melanoma skin cancer was a very strong risk factor in our study (OR 9), however because it had a low prevalence, the contribution to the AUC and more particularly to NRI was modest. This can be a limitation of these prediction methods, as factors that increase risk substantially can have minimal overall improvement to the model if they are rare. Another risk factor affected in this way is family history of melanoma. This variable did not improve discrimination in our analysis. We used a relatively low threshold for defining positive family history, requiring only one confirmed melanoma in a first-degree relative. This definition is consistent with many other population-based studies  and is associated with approximately two-fold increased melanoma risk [7, 19]. However, a more extensive family history is associated with higher risk estimates  and thus, for some individuals, this factor will strongly influence their personal melanoma risk. This issue demonstrates the different priorities that are placed on the design of risk prediction tools for different settings: one type for stratifying the population into broad risk categories to aid primary prevention strategies, and the other type for more precisely estimating personal risk of melanoma.
Harbauer et al. developed a model containing number of nevi, skin phototype and skin UV damage, adjusted for age and sex, with an AUC of 0.73 (95% CI 0.68-0.77) when measured by self-report and 0.77 (95% CI 0.73-0.83) when measured by a dermatologist. Similarly, our physician-measured model also achieved better discrimination than our self-report model, which was due solely to improved measurement of nevi. Our results indicate that counting all nevi greater than 2 mm is a better predictor of melanoma than counting only large, dysplastic or raised nevi. There was no added benefit to having physicians measure hair colour, eye colour or skin colour, as the discriminative ability of the pigmentation score was similar whether measured by self-report or by a physician.
Including sun and sunbed exposure variables resulted in little improvement to discrimination; when added as a group to the base model, they increased the AUC by 1.5%, but this diminished when the other factors were in the model. Although these sun and sunbed exposures have been shown to be associated with melanoma in our study [21, 43], it has been demonstrated that very strong, independent associations with risk are required to meaningfully increase the AUC [29, 44, 45]. Sun exposure-related factors generally do not have very strong effect estimates, which may be partly due to inherent difficulties measuring past sun exposure  but also reflects the strong influence of genetically determined risk factors.
The Hosmer-Lemeshow test of calibration indicated poor agreement between observed and predicted outcomes for the physician-measured model, for both the development and external validation datasets. We examined the observed and expected values within each decile from the Hosmer-Lemeshow test but there no was consistent pattern describing the differences. Calibration has been rarely reported for other published melanoma risk prediction models. A prospective cohort study with follow-up of individuals for development of melanoma would be the ideal method to evaluate melanoma prediction probabilities.
Several strengths and limitations of the Australian Melanoma Family Study have been discussed previously [15, 19, 21]. We had low participation from cases and population controls, which is a common problem for population-based studies , especially when conducted with highly mobile, young adults . Although poor participation can sometimes lead to selection bias , we did not find strong evidence of this occurring in our study . This analysis was restricted to participants who had a complete set of data for self-reported and clinically-measured risk factors. Having a clinical skin examination was a preferred but not compulsory aspect of participation in the Australian Melanoma Family Study, and thus was only completed by a subset of case and control participants. Although this reduced the sample size for our analysis, it is unlikely to have introduced systematic bias, as a comparison of those with (n = 676) versus without (n = 270) clinical skin examinations showed no statistically significant differences on predictors including sex, self-reported nevi, hair colour, MC1R genotype, previous non-melanoma skin cancer, and childhood blistering sunburns.
Our study focused on prediction of early-onset melanoma. A younger age at diagnosis is more likely to reflect an underlying genetic susceptibility to the disease, thus the contribution of MC1R and traditional factors to risk prediction in our study may differ for those diagnosed at older ages or for other ethnicities. Nevertheless, our external validation results do suggest generalisability of our results to a broader-aged population. The generalisability of our results may also differ by country and the phenotypic and behavioural characteristics of the population. External validation of risk prediction models using an independent dataset has been rarely performed by other studies, and is an important strength of our study.