AUC: Area Under the Receiver Operating Characteristic Curve; PPV: Positive Predictive Value; NPV: Negative Predictive Value

Melanoma diagnosed at early stages is typically cured by surgical excision alone [1], while melanoma presenting at advanced stages is often deadly [2]. For example, the 10-year survival for stage IA melanoma (localized and <= 1 mm thick without ulceration or high mitotic rate) is 93%, which differs dramatically from the 39% 10-year survival for even stage IIC melanoma (still localized but> 4 mm thick with ulceration) [3]. However, it is unknown whether screening for early detection of melanoma improves survival [4]. The United States Preventive Services Task Force has determined that there is insufficient evidence for or against routine melanoma screening [4] and has recommended further research to identify individuals at high risk for developing melanoma [5]. It is generally felt that targeted screening of high risk individuals, compared to screening the entire population, would be more feasible, less costly and less prone to false positive screens, unnecessary procedures, and patient anxiety [5-13].

Self-assessment of melanoma risk status has several potential advantages over risk assessment by a healthcare provider. Having individuals perform their own risk assessment could decrease the burden on health care system in terms of cost and manpower [6-8]. It would also allow for population-based screening [14], potentially better capturing older menand individuals without a regular primary care physician, groups at increased risk of presenting with advanced melanoma [15-17]. The feasibility of such a risk score is enhanced by the availability of several validated self-assessed risk factors, such as number of nevi on the arms [18,19] and density of freckling [19-25], which are strong predictors of melanoma.

Our objective was to create a melanoma risk score based entirely on self-assessed risk factors and to determine the proportion of melanomas captured using various high-risk cutoffs of the score. We did this by developing and validating a melanoma prediction model using data from a population-based case-control study of invasive cutaneous melanoma in Washington State, USA.

Study population

We used data from a case-control study of melanoma from western Washington State, which is described in detail elsewhere [26,27]. In this population, we identified all cases of newly diagnosed primary invasive cutaneous melanoma (excluding lentigo maligna melanoma) during the calendar year 1997 in the 13-county Seattle-Puget Sound Surveillance, Epidemiology, and End Results (SEER) cancer registry area. Lentigo maligna melanoma was excluded because it is thought to have etiological differences from other subtypes of melanoma [28]. Of 482 eligible cases, interviews were completed on 386, for a response rate of 80%. Cases were white and between the ages of 35 to 74. Controls were identified by random-digit dialing. Of 2787 potentially eligible controls, interviews were completed in 1751, for a response rate of 63%. Of these, several individuals were excluded (918 who were outside of the age range and geographic regions of the cases, 90 who were non-white race, and 16 with a history of melanoma), leaving 727 controls for the final analysis. Melanoma risk factors, demographics, and other health factors were collected via telephone survey by trained interviewers. Examples of key questions are found in the Figure 1. The institutional review board of the Fred Hutchinson Cancer Research Center approved the study.

Figure 1: Example questions from the telephone survey.

Statistical analysis: We generated a multivariate model to predict invasive melanoma using unconditional logistic regression in Stata, Version 10. We used a 75% random sample (i.e., the training set) to generate the model and tested the predictive ability of the model in the remaining 25% (i.e., the validation set) by calculating the area under the receiver operating characteristic curve (AUC). The AUC is a measure of predictive accuracy ranging from 0.5 to 1, with higher scores indicating better prediction. We also calculated the AUC on the training set to compare to other melanoma prediction models that did not use a validation method. We used the likelihood ratio test at a significance level of 0.05 for variable selection, testing first the most reliably measured variables and then the strongest predictors from exploratory stepwise models. Variables considered for the model included sex, age, education, income, marital status, tendency to sunburn, ability to tan, number of severe sunburns ages 2-18, natural hair color at age 15, density of freckles on arms before age 20, number of raised moles on both arms, prior mole removal, number of moles removed, and prior non-melanoma skin cancer. All variables were categorical. Tested multiplicative interaction terms included sex*age, sex*hair color, sex*freckles, hair color*sunburns, freckles*hair color, freckles*sunburns, freckles*moles, and age*moles; all were nonsignificant and thus not included in the final model. An analysis using conditional logistic regression matching on sex was performed to explore whether response bias by sex between cases and controls affected the estimates.

After variable selection using the training sample and calculation of the AUC in the validation sample, we used the full sample to provide the most accurate estimates of the coefficients for the final prediction model and to generate the risk score. The risk score was the sum of the risk factor values multiplied by the parameter estimates (i.e., betas or log odds ratios). The betas were simplified into points (i.e., multiplied by 10 and rounded up or down as appropriate) for ease of use. We calculated the risk score for each individual in the study and ranked the participants in the control group by risk score to generate risk strata [7,9,29]. We calculated the sensitivity and specificity using various high-risk cutoffs. We also calculated the positive predictive value and negative predictive value of these cutoffs using various melanoma prevalence estimates.

There were 1113 participants, 386 cases and 727 controls. Mean participant age was 50.7 years (range 35-74, sd 10.5). Men comprised 52.1% of cases, whereas 35.9% of controls were men. Additional participant characteristics are found in Table 1. In age-and-sex-adjusted analyses, male sex, age, tendency to sunburn, ability to tan, number of severe sunburns ages 2-18, natural hair color at age 15, density of freckles on the arms before age 20, number of raised moles on both arms, prior mole removal, number of moles removed, and prior non-melanoma skin cancer were significantly predictive of melanoma (Table 1). In the final multivariate model developed on the 75% test set, only age, male sex, number of severe sunburns ages 2-18, natural hair color at age 15, density of freckles on the arms before age 20, number of raised moles on both arms, and prior non-melanoma skin cancer remained significant. The AUC of this model in the 25% validation set was 0.70 (95% CI: 0.64, 0.77), indicating that the model is moderately predictive of melanoma.The AUC on the test set (calculated for comparison to unvalidated models of melanoma risk) was 0.77 (95% CI: 0.73, 0.81). An exploratory analysis using conditional logistic regression matching on sex showed no appreciable changes in the estimates, suggesting that response bias by sex between cases and controls did not influence the results.

Table 1: Characteristics of cases and controls with age- and sex-adjusted odds ratios for invasive melanoma.

To calculate the risk score and estimate sensitivity and specificity of various risk score cutoffs, we used the entire dataset. The betas (i.e., the log odds ratios) of the variables in the final model (Table 2) were converted to simplified "points" (Figure 2) and used to calculate a risk score for each participant. The range of possible scores was 0-67. The actual range for controls was 0-50 (mean 18.5, median 18.0). The actual range for cases was 1-61 (mean 28.4, median 28.0). The sensitivity, specificity, relative risk, positive predictive value (PPV), and negative predictive value (NPV) for various high-risk cutoffs (e.g., top 5%, top 10%, top 15%) are shown in Table 3. As expected, sensitivity decreased while specificity increased with higher risk cutoffs. A majority of melanomas, 50%, were captured in a relatively small high-risk group, the top 15%. The PPV, or the proportion of persons considered high risk by the score who would develop melanoma in one year, was fairly low when applied to the entire population, but it was higher in men and women age 50 or older, a group at higher risk of melanoma. The PPV increased even further when considering the proportion of highrisk persons who would develop melanoma in a five-year time period. Of participants age 50 or older in the top 15% risk group, about 1% would be expected to develop a new melanoma over 5 years. The NPV, or the proportion of persons considered low risk who would develop melanoma, was 99.8% or higher for all groups.

Table 2: Odds ratios and betas for melanoma from the final multivariate prediction model.

Table 3: Relative risk (RR), sensitivity, and specificity for various risk score cutoffs and estimated positive predictive value (PPV) and negative predictive value (NPV) calculated for various populations and time periods.

Figure 2: Melanoma risk score.

We used data from a Washington State melanoma case-control study with self-assessed risk factors to develop a melanoma risk score. For white persons ages 35-74, we found that the most predictive risk factors were male sex, older age, higher number of severe sunburns between ages 2-18, lighter natural hair color at age 15, higher density of freckles on the arms before age 20, higher number of raised moles on both arms,andprior non-melanoma skin cancer. The validated AUC of 0.70 indicates that the model predicts melanoma moderately well. Screening for melanoma in the top 15% risk category(with full body skin examination by a dermatologist, primary care physician, or other trained health care professional, for example) could capture a relatively high proportion of melanomas (up to 50% if the screening examination was highly sensitive). The one-year PPV, the proportion of persons with a high-risk score who would be expected to develop melanoma in the next year, was low, since melanoma is a rare event despite being the sixth most common cancer in the U.S. in 2004-2008 [30]. However, the PPV was naturally higher when considering a longer follow-up period of 5 years. The PPV also increased when the risk score was applied to a higher risk population (individuals age 50 or older). The five-year PPV was nearly 1% in the top 15% risk among persons over age 50, so applying the risk score to individuals over 50 could increase the melanoma yield while minimizing the cost and clinical burden of unnecessary screening.

We are aware of only two other comparable self-assessed melanoma risk prediction models derived from case-control or cohort study data [7,11], and our model compares favorably to them. One of these studies [11] used prospective data from three large cohort studies (the Nurses' Health Studies I and II and the Health Professionals Follow-Up Study) with 535 incident cases of invasive melanoma. The final model included age, male sex, family history of melanoma, number of moles larger than 3 mm on arms or lower legs, and hair color. The AUC was 0.62, which was lower than our AUC of 0.70 and may be an overestimate, as it was not clearly validated using data separate from that used to build the model. The sensitivity was 23% in the top 10% risk and 38% in the top 20% risk participants, which is lower than our sensitivities at those cutoffs, 42% and 61% respectively. The other study [7] used data from a partly clinic-based case-control study in Austria and included 185 invasive melanomas and 17 in situ melanomas. It was age- andsex- matched, and risk factors in the final model included Fitzpatrick skin phototype (I-IV), skin damage related to solar radiation (absent, moderate, or severe), and total number of nevi (0-5, 6-10, 11-25, 26-50, or>50). The AUC was 0.73, but it may be an overestimate as it was not clearly validated and is lower than the AUC on our training set of 0.77. The sensitivity was 42% for the top 10% risk, which is identical to our findings. One other self-assessed model is difficult to compare to our study due to differences in methods and no reported AUC or sensitivity for high-risk cutoffs [20]. There are several self-assessed melanoma risk factor questionnaires [6,8,13,14,31], but they were not derived from statistical models, so we are unable to know how well these questionnaires predict melanoma [32].

There have been several melanoma risk prediction models that used clinician-assessed risk factors such as number of total nevi, number of atypical nevi, or sun damage on the back [9,33-41]. These types of risk assessments are conceptually different from ours, because a clinic visit with a dermatologist or primary care provider (which is costly and time-consuming) is necessary to determine the risk level, while our risk score can be calculated by a telephone interview or potentially by a selfassessed written questionnaire. Guther et al. created a model to predict invasive or in situ melanoma from a prospective cohort of patients who underwent free total-body skin examination by a dermatologist as part of Germany's mass skin cancer screening program [41]. Variables in the final melanoma prediction model included age, red or blond hair color, past history of melanoma, and suspicious melanocytic lesion on dermoscopy with an AUC of 0.86 (with validation by bootstrapping methodology) and unknown sensitivity of the model for melanoma in the high-risk group (since melanoma and squamous cell carcinoma models were combined for the risk stratification component). The model by Fortes et al. was age- and sex-matched andincluded in the final model total number of nevi as counted by a dermatologist [42], hair color, skin color, presence of freckles, and history of at least one sunburn in childhood [40]. The model was derived from an Italian case-control study and validated in a Brazilian population with an AUC of 0.79, and the sensitivity was 49.9% for the top 12.5% risk, which is similar to our findings. English et al. included number of raised moles on arms as assessed by a study nurse, age on arrival in Australia, history of non-melanoma skin cancer, mean hours per week spent outdoors in the summer between ages 10-24, and family history of melanoma [9]. No AUC was given, but the sensitivity was 54% in the top 16% risk, which is again similar to our findings. Other models with providerassessed risk factors are difficult to compare to our study, as they did not calculate AUC and the sensitivity of high-risk cutoffs [33-39].

The number of nevi on the arm is a strong predictor of melanoma, when counted by an examiner [20,21,24,43-47] and when selfassessed [18,19]. There is evidence that nevus counts on the arms are representative of total body nevi [18,47-49] and that self-counting of arm nevi is reliable when compared to examination by a dermatologist [14,50]. Focusing on this limited and easily accessible part of the body is ideal for population-based risk assessment and may improve compliance with and precision of self-counting of nevi [18,24,50]. Limiting the counts to raised moles can prevent the inclusion of freckles or solar lentigines [18], but there is the potential for misclassification due to the counting as nevi of seborrheic keratoses, which are common on the arms in older individuals [50]. Our study included individuals up to age 75, and the number of nevi on the arms was still a very strong risk factor for melanoma (OR 2.93 for 3 or more compared to none), despite this possibility of misclassification.

In our study, the association of melanoma with self-assessed freckle density was strong (OR 2.69 for "a lot" of freckles compared to none), with a positive dose-response relationship(increasing odds of melanoma with increasing density of freckles). These two factors support the validity of self-assessed freckle density as a melanoma risk factor. Further support is provided by other studies, whichhave also showna strong association between self-assessed freckle density and melanoma [19-25] with a dose-response relationship [20,21].

Our study has several strengths. It was a large population-based study. We used well-validated risk factors that were self-assessed, making our score easy to use in a broad array of settings. The predictive ability as measured by the AUC is strong compared to other melanoma models and other cancer prediction models. (For example, the AUC range is 0.58-0.64 for breast cancer [51] and 0.70-0.76 for prostate cancer [52]. Unlike many prior melanoma risk prediction studies, we calculated the AUC on data separate from that used to develop the model, thus validating it.

Our study may have been limited by response differences by sex, as only 36% of controls were men. Nevertheless, the odds ratio of 2.0 for male sex in our risk model is consistent with current estimates of the relatively higher incidence of melanoma in men compared to women, particularly for whites age 55 and older [30]. Also, a secondary analysis matching on sex gave very similar results, indicating that response bias is not likely affecting the estimates to a great extent. Another limitation is the potential for recall bias, as with all case control studies. We also did not assess family history of melanoma, but this risk factor has been variably predictive in other studies [7,41] and is prone to unreliable measurement [53]. We did not collect data on the number of dysplastic nevi, but this variableis difficult for patients to assess themselves [54]. Finally, our participants were from western Washington State only, where melanoma incidence is somewhat surprisingly higher than the overall U.S. incidence [30], thus generalizability to other areas is unknown.

In summary, our study suggests that melanoma risk assessment could be performed via self-assessed questionnaire and that screening the 15% highest risk persons could detect about 50% of melanomas, assuming moderate fit of the model. This self-assessed risk score could be used as part of a comprehensive program of populationbased melanoma screening and education. As opposed to clinicbased interventions, population-based risk assessmentand education couldpotentially bettercaptureolder men and persons without a primary care physician, groups at highrisk for presenting with advanced melanoma [15-17]. Further study is needed to validate this score in other populations, develop and validate a self-administered written questionnaire from the current telephone-based survey, and to determine the costs, feasibility, and risks of screening a high-risk group.

This research was supported by National Institutes of Health grant K05CA154337. We are indebted to Dr. Maryam Asgari for her assistance in the design of this analysis.The authors have no conflicts of interest to declare.