Vulnerability in elderly patients with gastrointestinal cancer – translation, cultural adaptation and validation of the European Portuguese version of the Vulnerable Elders Survey (VES-13)

Translation and face validity

The authors followed the European Organization for Research and Treatment of Cancer (EORTC) guidelines - Quality of Life Group Translation Procedure; and Guillermin et al. recommendations [18, 19]. Briefly, the original questionnaire was translated into European Portuguese and culturally adapted by two healthcare professionals with English fluency, knowledgeable of the translation purpose. This draft version was translated back into English, by two English translators, and compared to the original questionnaire by the investigators and the original VES-13 authors, to assess comprehension of the applied concepts and wording. No problems were identified at this stage.

Face validity of the translated questionnaire was assessed by six medical oncologists at our GI Cancer Clinic. They were asked to review the original and translated questionnaires and classify each question, according to comprehension and accuracy of the translation, using a numerical rating scale of 10 points (1 - poorly clear, to 10 - completely clear).

Patient recruitment

Cancer patients admitted at our Comprehensive Cancer Centre age ≥65 years with histologically confirmed GI Cancer, Portuguese fluency, and no history of previous systemic therapy for cancer were eligible for both the pilot and prospective validation cohort. Patients presenting cognitive impairment, confusional syndrome or who were illiterate or foreign individuals were excluded from the pilot study. The pilot study also excluded patients unable to read.

This work has been approved by the ethical committee of the “Instituto Português de Oncologia do Porto” in Portugal, institution where it was developed and all the subjects gave their informed consent.

Pilot study: cultural adaptation and test-retest reliability

The questionnaire was applied by one of the investigators to included consecutive patients (first pilot n = 20, second pilot n = 22) who were asked to rate each question for comprehension using the previously described numerical rating scale of 10 points. Each patient completed the VES-13 questionnaire twice within 1 to 30 days. At this point, a question was to be reviewed if it had a single rating ≤5 (corresponding to reasonably clear), or if any comprehension problem was noted by the interviewer. Concerns regarding question 3f made necessary a second pilot, after questionnaire adaptation.

Prospective cohort study: construct and criterion validity

After completion of the pilot study, the European Portuguese version of VES-13 was prospectively applied to a cohort of 200 patients to assess internal consistency and construct and criterion validity [20–22]. To assess construct validity we selected EQ-5D-5L as comparator [23]. EQ-5D-5L is a generic health related quality of life questionnaire which includes five dimensions and a visual analogue scale (VAS) assessing general health. Each dimension is recorded in five severity levels (no problems, slight, moderate, severe and extreme problems, graded from 1 to 5, respectively). The VAS records an individual’s rating for their current health-related quality of life (ranging from 0 - worst imaginable health, to 100 – best imaginable health). Predefined hypothesis about relationships among dimensions of EQ-5D-5L and VES-13 were tested to assess construct validity. To assess criterion validity we used the clinical impression of a trained medical oncologist, blinded to the responses on VES-13, regarding patient’s vulnerability and performance status (PS). Each medical oncologist was instructed to consider the Eastern Cooperative Oncology Group classification – ECOG [24] and Karnofsky scale – KPS [25], and to categorize each patient into the following groups: fit, vulnerable or frail. Performance status was estimated according to exact ECOG PS and KPS scales definition (ECOG PS ranging from 0 –able to carry on all pre-disease performance without restriction, to 5 – dead; KPS ranging from 100 – normal, no complaints, to 0 – dead). Correlations among these criteria and VES-13 were evaluated to assess criterion validity.

Statistical analysis

Patient’s demographics and clinical characteristics were studied using descriptive statistics as appropriate. Numerical variables were described with means and standard deviation or with medians and interquartile ranges, depending on the asymmetry of their distributions. Categorical variables were described as absolute and relative (percentages) frequencies. Performance status was categorized as follows: ECOG ≤1 and ≥2 and KPS 100–80 and ≤70. Charlson comorbidity index (CCI) was used to estimate the burden of co-morbid conditions. When testing hypothesis about continuous variables, Student’s t-tests were used to compare two groups when normality assumptions were confirmed and Mann–Whitney U tests were used if normality could not be assumed). When testing hypothesis about categorical variables, Chi-square test and Fisher’s exact test were used as appropriate.

The test-retest reliability of the Portuguese version of VES-13 was assessed in the pilot study by calculating the Kappa statistic for each item to assess agreement between test and retest scores [26]. This index takes values between −1 and 1; values near 1 show high test-retest reliability. The categorization by Landis and Koch was used for interpretation of κ values (<0.00 – no agreement; 0.01-0.20 – slight; 0.21-0.40 – fair; 0.41-0.60 – moderate; 0.61-0.80 – substantial and 0.81-1.00 – almost perfect agreement) [27]. Additionally, we calculated the test-retest reliability coefficient (correlation coefficient) for VES-13 total scale score.

Internal consistency of translated VES-13 items was explored by analyzing the inter-item and item-total correlation matrices and calculation of Cronbach’s alpha coefficients. This coefficient ranges from 0 to 1, and larger values indicate higher internal consistency. As recommended by Nunnally and Bernstein, alphas ≥0.70 were considered adequate [21]. An estimation of Cronbach’s alpha if an item were to be deleted from the scale was used to identify which items affected the questionnaire’s internal consistency the most.

Construct and criterion validity was assessed by calculating Spearman’s correlation coefficient between VES-13 and each EQ-5D-5L dimension, clinical judgment and performance status. Interpretation of correlation coefficients was based on the quantitative criteria and qualitative descriptors defined by Cohen [28] (low correlations for coefficients with absolute value between 0.10 and 0.29; moderate correlations for coefficients between 0.30 and 0.49 and high correlations for coefficients between 0.50 and 1.00).

Exploratory factor analysis for VES 13 European Portuguese version was performed using principal components analysis for factor extraction. The hypothesis of unidimensionality of VES-13 was assessed. Selection of the number of factors to retain took into account Kaiser’s criterion (eigenvalues larger than one); graphical analysis of the Scree-plot; and the total variance explained. If adequate, to improve interpretation of factors, orthogonal varimax rotations were to be applied. The Kaiser-Meyer-Olkin (KMO) measure and the Bartlett’s test of sphericity were assessed.

Finally, we performed a ROC curve analysis, to assess the best cutoff point for VES-13 total score for discrimination of Frail/Vulnerable elders, assuming the attending physician’s clinical judgment as the gold standard. Best cutoff selection criterion was based on the method of minimization of the distance to the left upper corner of the ROC plot, calculated as √(1-Sn)² + (1-Sp)².

A prospective cohort of 200 consecutively enrolled senior GI cancer patients (≥65 years), would allow an estimation of the prevalence of vulnerability/frailty with a 95 % confidence level margin of error of 0.07. This sample size would also allow an estimation of validity coefficients (correlation coefficients) larger than 0.20, with 95 % confidence level and 90 % power.

Statistical analysis was performed using the Statistical Package for the Social Sciences Version 20.0 for Windows (SPSS®). Whenever statistical hypothesis testing was used, a significance level of α = 5 % was considered.

Translation and cultural adaptation

After translation and cultural adaptation, all questions scored 6 or higher, corresponding to reasonable comprehension during face validation and no changes were deemed necessary.

Internal consistency and construct and criterion validity

The VES-13, EQ-5D-5L and medical oncologist’s clinical assessment were applied during 6 months (June to November, 2012). During this period, 296 elderly patients with GI Cancer were admitted to our GI Cancer Clinic and a total of 206 patients were included (Fig. 1 describes the selection process and reasons for exclusion). Demographic and clinical characteristics of the cohort are described in Table 2. The 90 individuals not included in the sample had epidemiological and clinical characteristics similar to those included in the study.

	VES-13 < 3		VES-13 ≥ 3		Total
	n = 104	(%)	n = 102	(%)	n = 206	(%)
Male	77	(74)	56	(55)	133	(65)
Age - years, median [range]	72	[65–84]	77	[65–89]	73	[65–89]
Charlson comorbidity index, median [range]	6	[5–15]	8	[5–15]	7	[5–15]
Diabetes mellitus	29	(28)	33	(32)	62	(30)
Cardiovascular disease	12	(12)	24	(24)	36	(17)
No. concomitant drugs, median [range]	3	[1–9]	4	[1–8]	3	[1–9]
Primary cancer topography
Colorectal	66	(64)	64	(63)	130	(63)
Gastro-esophageal	29	(28)	24	(24)	53	(26)
Pancreas	3	(3)	10	(10)	13	(6)
Cancer stage (AJCC 7th edition)
I	9	(9)	8	(8)	17	(8)
II	23	(22)	20	(20)	43	(21)
III	43	(42)	34	(33)	77	(37)
IV	22	(21)	34	(33)	56	(27)

VES-13 – Vulnerable Elders Survey: < 3 → fit; ≥ 3 → vulnerable/fragile; AJCC – American Joint Committee of Cancer

The cohort’s median age was 73 years (29 % ≥80 years) and 65 % of the subjects were male. Colorectal cancer was the most frequent tumor location (63 %) and 3 % of patients had a history of previous neoplasms. The most prevalent co-morbidities were diabetes (n = 62, 30 %) and cardiovascular disorders (n = 36, 17 %). The median CCI was 7 (interquartile range: 6–11). Fifty-eight patients (28 %) were taking five or more daily drugs, and the more frequent therapeutic groups, as defined by the World Health Organization Anatomical Therapeutic Chemical/Defined Daily Dose, were “cardiovascular system” and “alimentary tract and metabolism” drugs.

VES-13 European Portuguese version of the questionnaire showed high internal consistency, with Cronbach’s alpha if item deleted ranging from 0.826 to 0.880, and a Cronbach’s alpha for the scale score of 0.848, Table 4. When assessing the correlation of VES-13 and EQ-5D-5L dimensions we obtained, as expected, higher correlation scores for “mobility”, “self-care” and “usual activities” (r_s: 0.634, 0.625 and 0.652 respectively). Although not so strong, statistically significant correlations with “pain/discomfort” (r_s: 0.329) and “anxiety/depression” (r_s: 0.178) domains and with VAS scale (r_s: −0,527) were also observed. There were moderate to strong correlations between VES-13 and clinical judgment, ECOG and KPS scales (r_s: −0.499, 0.599, and −0.576, respectively). Table 5 presents the summary statistics for the correlation coefficients between VES-13 and EQ-5D-5L, performance status and clinical impression. In Fig. 2 we can also see and contrast the distribution of VES-13 total score for the fit and vulnerable/fragile elders, as classified by the clinical judgment of the attending physician. In Fig. 3 we present a ROC curve analysis of the VES-13 total score, assuming the clinical judgment of the attending physician as the gold standard; and showing the cutoff value of >3 as the most appropriate for maximizing both sensitivity and specificity. For this cutoff value the sensitivity was 71 % and the specificity was 84 %. The estimate of the area under the ROC curve was C = 0.818 (95 % CI [0.762 – 0.875]).

VES-13 question	Classification (points)	n	(%)	Cronbach’s alpha if item deleted
Item 1	0	119	(58)	0.880
	1	68	(33)
	3	19	(9)
Item 2	0	61	(30)	0.849
	1	145	(70)
Item 3a	0	155	(75)	0.829
	1	51	(25)
Item 3b	0	162	(79)	0.829
	1	44	(21)
Item 3c	0	184	(89)	0.835
	1	22	(11)
Item 3d	0	194	(94)	0.847
	1	12	(6)
Item 3e	0	164	(80)	0.829
	1	42	(20)
Item 3f	0	118	(57)	0.826
	1	88	(43)
Item 4a	0	156	(76)	0.827
	4	50	(24)
Item 4b	0	178	(86)	0.835
	4	28	(14)
Item 4c	0	189	(92)	0.836
	4	17	(8)
Item 4d	0	165	(80)	0.833
	4	41	(20)
Item 4e	0	155	(75)	0.829
	4	51	(25)
VES-13 TOTAL SCORE	-	-	-	0.848

	VES-13
	rs	(p value)
EQ-5D-5L
Mobility	0.688	(<0.001)
Self-care	0.690	(<0.001)
Usual activities	0.732	(<0.001)
Pain/discomfort	0.405	(<0.001)
Anxiety/depression	0.237	(0.001)
VAS (mean)	−0.592	(<0.001)
CLINICAL JUDGMENT	−0.570	(<0.001)
ECOG PS	0.614	(<0.001)
KPS	−0.622	(<0.001)

Exploratory factor analysis, with factor extraction using principal components, was performed for VES-13. The factor solution included a first component with eigenvalue 6.41 and 49.3 % of the variance explained, a second component with eigenvalue 1.08 and 8.3 % of the variance explained, and all other components with eigenvalues lower than 1.00 and smaller percentages of variance explained. Based on the analysis of the scree plot, the eigenvalues of the components and the percentage of the variance explained, the one factor solution was clearly the more appropriate, supporting the hypothesis of unidimensonality of VES-13. Although the strict application of the Kaiser rule would imply the selection of a two factor solution, the fact is that the second component had an eigenvalue marginally above 1.00 and a low percentage of variance explained, thus a one factor solution is clearly a more sensible solution in this case.

Prevalence of vulnerability in elderly patients with GI cancer

The proportion of vulnerable elderly GI cancer patients was 0.50 (CI95%: 0.43-0.56). Vulnerable patients were more likely to have higher EQ-5D-5L scores on every dimension, meaning higher prevalence and magnitude of problems, and lower EQ-VAS values, meaning a perception of worse quality of life. These patients also had worse performance status (higher mean ECOG-PS and lower KPS). Vulnerable patients had higher CCI scores (p < 0.001) and were also more likely to have higher polypharmacy levels (32 % versus 19 % of patients were receiving ≥5 daily drugs).