Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

The quality of working life questionnaire for cancer survivors (QWLQ-CS): factorial structure, internal consistency, construct validity and reproducibility

BMC Cancer volume 18, Article number: 66 (2018) Cite this article

Abstract

Background

To assess the factorial structure, internal consistency, construct validity and reproducibility of the Quality of Working Life Questionnaire for Cancer Survivors (QWLQ-CS).

Methods

An Exploratory Factor Analysis (EFA) was performed on QWLQ-CS data from a sample of employed cancer survivors to establish the final number of items and factorial structure of the QWLQ-CS. Internal consistency was assessed using Cronbach’s alpha. In a second sample of (self-)employed cancer survivors, construct validity was tested by convergent validity (correlations of QWLQ-CS with construct-related questionnaires), and discriminative validity (difference in QWLQ-CS scores between cancer survivors and employed people without cancer). In a subgroup of stable cancer survivors subtracted from the second sample, reproducibility was evaluated by Intraclass Correlation Coefficient (ICC) and Standard Error of Measurement (SEM).

Results

EFA on QWLQ-CS data of 302 cancer survivors resulted in 23 items and five factors. The internal consistency of the QWLQ-CS was Cronbach’s α = 0.91. Convergent validity on data of 130 cancer survivors resulted in r = 0.61–0.70. QWLQ-CS scores of these cancer survivors statistically differed (p = 0.04) from employed people without cancer (N = 45). Reproducibility of QWLQ-CS data from 87 cancer survivors demonstrated an ICC of 0.84 and a SEM of 9.59.

Conclusions

The five-factor QWLQ-CS with 23 items and adequate internal consistency, construct validity, and reproducibility at group level can be used in clinical and occupational healthcare, and research settings.

Peer Review reports

Background

By 2025, cancer incidence is expected to rise to 19.3 million cases worldwide [1]. As new treatments and screening instruments increase the chances of surviving cancer [2] and as more people work longer, an increasing number of cancer survivors are continuing to work or returning to employment [3]. Unfortunately, cancer survivors can encounter difficulties at work. Cancer survivors are 1.4 times more likely to be unemployed than ‘healthy’ employees [4] for example, and when cancer survivors are employed, they report facing psychological and physical difficulties at work [5, 6].

Although a cancer diagnosis can have a negative physical, cognitive and psychological impact on a person’s working life [7, 8], work also benefits cancer survivors. For instance, work allows them to maintain a sense of identity and self-esteem and provides financial security [9]. Getting adequate support from one’s general physician or the workplace is related to a successful return to work [10]. Yet, there are additional actors involved in the occupational rehabilitation of cancer survivors, such as occupational physicians, oncologists and other healthcare professionals [11, 12]. To provide adequate support to cancer survivors, these actors should be able to assess the overall work situation of the patient and not only work-related outcomes such as work productivity [13].

Cancer survivors perceive various difficulties in the workplace, such as coping with fatigue [14] or lack of understanding from their work environment [15]. These difficulties are likely to contribute to subjective work outcomes, such as Quality of Working Life (QWL). We define QWL as ‘the experiences and perceptions of cancer survivors in the work situation’ [16]. Previous research indicates that ‘healthy’ employees with a high QWL show lower levels of turnover intention [17, 18]. Research on QWL is often performed among ‘healthy’ employees. For instance, existing Quality of Working Life questionnaires [19,20,21,22] were developed for ‘healthy’ employees or particular occupations [23] and do not incorporate items on the effect of a cancer diagnosis and treatment, such as fatigue and anxiety.

To measure QWL among cancer survivors, and to take account of the impact of cancer diagnosis and treatment on a cancer survivors’ working life, we developed the self-administered Quality of Working Life Questionnaire for Cancer Survivors (QWLQ-CS) [24]. The development of the QWLQ-CS was based on the guidelines for developing Questionnaire Modules provided by the EORTC Quality of Life Group [25]. We generated items from the literature [26] and held focus groups with employed cancer survivors and interviews with oncological occupational physicians and employers [24]. An initial version was constructed and pre-tested among employed and self-employed cancer survivors which resulted in a preliminary version of the 104-item QWLQ-CS [16]. This article describes two field studies that were based on the last phase of questionnaire development [25]. The objective of field study I was to reduce the number of items in the preliminary QWLQ-CS and determine its factorial structure and internal consistency. The objective of field study II was to test the construct validity and reproducibility of the final version of the QWLQ-CS.

FIELD STUDY I: Item reduction, factorial structure and internal consistency of the QWLQ-CS

Methods

Design

Field study I was based on a cross-sectional design, with the aim of reducing the number of items in the preliminary QWLQ-CS and determining its underlying factorial structure. To guarantee a high level of methodological quality in evaluating the measurement properties of the instrument, the COSMIN checklist [27] was used. The Medical Ethics Committee of the Academic Medical Center (AMC) deemed ethical approval to be unnecessary (W14_323#14.17.0387).

Participants

Cancer survivors were recruited in Dutch hospitals (N = 3). The cooperating hospital departments were those of specialising in breast cancer, gastrointestinal cancer, dermatological oncology, gynaecological oncology, head and neck surgical oncology, oncological lung diseases, radiotherapy and urological oncology. After selection by patient administrations, cancer survivors were invited to participate by their oncological specialist during an appointment or by post. Cancer survivors were also recruited by issuing invitations through a Dutch online cancer platform and a patient organisation’s homepage. Furthermore, 12 cancer survivors who had been recruited for a previous study [16], but who had not participated because the required sample size had been achieved, received a new invitation.

Inclusion criteria were: (1) diagnosed with malignant cancer (2) diagnosed between three months and ten years ago, (3) currently between 18 and 65 years of age, (4) 18 years or older when diagnosed with cancer, (5) employed or self-employed, and participated in work in the last four weeks, and (6) fluent in Dutch. Exclusion criteria were: being diagnosed with a severe psychiatric disorder or receiving palliative treatment. The recruitment strategy via the Dutch hospitals and Dutch online cancer platform allowed only for a pre-selection on a few inclusion criteria (e.g., age, diagnosis) as no more demographics were available. Therefore, the other inclusion criteria were checked upon response by a participant prior to participation.

Informed consent

If cancer survivors wanted information about the study or wished to participate, they consented to being contacted by the research team. Next, all cancer survivors who agreed to participate by telephone received an informed consent form for study participation by post, which had to be signed.

Procedure

Data collection took place between May 2015 and December 2015. Cancer survivors were asked to complete the preliminary QWLQ-CS in paper or digital form. The digital version of the QWLQ-CS was designed using the online survey software Fluidsurveys (SurveyMonkey Europe, Ireland 2014).

Instruments

Preliminary version of the QWLQ-CS

The preliminary QWLQ-CS was developed in Dutch and consisted of 104 items. Positively and negatively phrased items could be answered on a 6-point Likert scale without numbers (Totally disagree - Totally agree). The items had a reference period of the past four weeks. The extra response option ‘Not applicable’ was provided for cases in which cancer survivors felt an item was not applicable to their work or health situation (e.g., if self-employed cancer survivors were asked to answer items about their immediate supervisor or colleagues).

Other variables

Demographic, health- and work-related variables were assessed (Table 1).

Table 1 Sample characteristics field study I and II
Full size table

Data analysis

The answers on the digital QWLQ-CS were directly exported from the online survey software Fluidsurveys to the software IBM SPSS Statistics 23. The researchers entered the paper versions of the QWLQ-CS into Fluidsurveys twice. The data entry for two of every ten (20%) paper versions of the QWLQ-CS was checked by exporting the data to SPSS and calculating the margin of error. If ≥2% of the data entry was wrong, all of the paper versions were checked by a different researcher.

Explorative factor analysis

An important first step in testing a new questionnaire is to assess its content by determining if the variables of the construct to be measures are related. Therefore it is necessary to assess the underlying factor structure of this new set of variables with an EFA. The EFA was performed on the 104-item QWLQ-CS in seven steps (Table 2). In step 1, each item was removed if it fulfilled one of two conditions. The first condition was aimed at preventing an uneven distribution of answers, which might lead to an inability to detect any improvement or to distinguish between patients [28]. The second condition was aimed at removing non-generic items. For instance, items were removed if ≥20% of cancer survivors had answered ‘Not applicable’. Step 2 assessed the inter-item correlation matrix. Items were removed if they had extremely low correlations (<0.2) with ≥80% of the other items, on the grounds that they were not related to any of the other items and were not measuring the same construct, or if they correlated too highly (>0.9) with other items, which implied that the content of these items was too similar [29].

Table 2 Steps in Exploratory Factor Analysis (EFA)
Full size table

In order to perform the Principal Component Analysis (PCA) in IBM SPSS Statistics 23, the test assumption had to be met in step 3. The Kaiser-Meyer-Olkin test was used to assess the sample adequacy, and if this value was >0.6, the sample size was sufficient. For items to be correlated, Bartlett’s test of sphericity had to be p < 0.05 [30]. In step 4, the number of underlying factors in the QWLQ-CS was explored by analysing the outcomes on Catell’s scree test [31] and Parallel Analysis (PA). In a scree test, the number of factors are based on the break in the plot [32]. PA was used to compare the outcomes of the PCA eigenvalue of our data set to the mean eigenvalue of 100 random data set with the same number of items and sample size [33]. To determine the best fit for the rotation structure in step 5, the PCA was performed on a fixed set of factors, resulting from the scree test and PA and with various rotation methods (Table 2). Based on the rotation plots, we decided which rotation best fitted the data. In step 6, the final decision on the number of factors was made by carefully examining the number of items, their content and the items’ factor loadings on the different number of factors that had been retrieved in step 4. Items with a factor loading of >0.5 were allocated to that factor [29]. Items with a factor loading of <0.5 were removed, and a new PCA was performed after the removal of each item in order to analyse the new factor loadings of the items. For items with factor loadings of >0.3 on more than one factor, removal was discussed, because the interpretation of this item might be ambiguous [29].

Finally, in step 7 items were removed by analysing the internal consistency per factor. The internal consistency indicates the interrelatedness of the scale of the extent to which items assess the same construct [29]. Multiple parameters of internal consistency were analysed (Additional file 1). An item was deleted if it had an inter-item correlation of ≥0.7 with another item, and if it had low inter-item correlations (0.2–0.4) with half of the items in that factor. Finally, a Cronbach’s alpha between 0.7 and 0.9 was acceptable [29], with >0.9 suggesting a high level of item redundancy [28]. Therefore, items were deleted when the Cronbach’s alpha of the factor was <0.7 and >0.9. One PCA was performed to examine the stability of the factor structure.

Results

Of the 1617 cancer survivors who were pre-selected on a selection of inclusion criteria (e.g., on age, diagnosis) and invited, a total of 490 cancer survivors responded. Of this group 308 cancer survivors met the other inclusion criteria as well and agreed to participate, and 182 cancer survivors did not met the other inclusion criteria (e.g., not employed) or responded to indicate they were not interested in participation. Ultimately, 302 cancer survivors completed the QWLQ-CS (Table 1).

Explorative factor analysis (EFA)

In step 1, there were no items for which ≥95% of the responses fell into one category. However, 14 of the 104 items were removed because ≥20% of cancer survivors indicated that this item was not applicable to them (Table 2). None of the items in step 2 correlated ≥0.9 with other items, but four items did correlate ≤0.2 with ≥80% of the other items and were removed. In step 3, the PCA was therefore performed with 86 items. Test assumptions were achieved; the Kaiser-Meyer-Olkin test was 0.86 and Bartlett’s test of sphericity was significant (p < .001). In step 4, Catell’s scree test yielded four factors and PA identified eight factors. Varimax rotation was the best fit for the data in step 5. After carefully examining the number of factors resulting from the scree test and PA, the number of items, their content and the items factor loadings, a five-factor structure was determined in step 6. We removed 21 items because they had a factor loading below 0.5, and two items that showed overly high (above 0.3) loadings on other than their main factor. In step 7, 40 of the 63 remaining items were deleted based on inter-item correlations of ≥0.7, inter-item correlation between 0.2–0.4 of multiple items, or the scale’s Cronbach’s alpha of <0.7. This resulted in a total of 23 items that were divided into the subscales: 1) Meaning of work, 2) Perception of the work situation, 3) Atmosphere in the work environment, 4) Understanding and recognition in the organisation, and 5) Problems due to the health situation. These five factors explained 51% of the variance and the QWLQ-CS had good internal consistency (Cronbach’s alpha = 0.91). The Cronbach’s alpha of the subscales varied between 0.83 and 0.86 (Table 3).

Table 3 Exploratory Factor Analysis (EFA): factor loadings on five-factor structure
Full size table

FIELD STUDY II: Construct validity and reproducibility of the QWLQ-CS

Methods

Design

In field study II, we evaluated the psychometric properties of the final version of the QWLQ-CS. The study used two measurements: at baseline and at follow-up after four weeks. The measurement at baseline was executed to test the construct validity (i.e. convergent validity and discriminative validity) of the QWLQ-CS. The measurements at baseline and the four-week follow-up were used to determine its reproducibility. Again, we used the COSMIN checklist, and ethical approval from the Medical Ethics Committee of the AMC was deemed unnecessary (W14_323#14.17.0387).

Participants

Cancer survivors

The recruitment process and inclusion criteria were similar to those used in field study I. Cancer survivors who had signed up for participation in field study I, but who had not participated because the sample size was sufficient, were included in field study II. Furthermore, the sample was completed by cancer survivors who were recruited from the patient administrations of the departments of breast cancer, gastrointestinal cancer and haematological cancer in three different hospitals.

Employed people without cancer

To assess the discriminative validity of the QWLQ-CS, a sample of employed people without cancer or other physical/mental limitations affecting their job performance was recruited. An item in the questionnaire verified whether the participant met these criteria. The participants were recruited by asking participating cancer survivors to voluntarily pass on information about the study and the participation form to an employed friend, relative, neighbour or colleague of the same sex and age. Furthermore, recruitment was undertaken within the hospital (e.g., via the website and research boards). It was not necessary to gain informed consent because the participants participated voluntary and the questionnaire was anonymous.

Procedure

Data were collected between March 2016 and April 2016. At baseline, cancer survivors were asked to complete the questionnaire that comprised the QWLQ-CS, the Copenhagen Psychosocial Questionnaire subscales (COPSOQ) [34], the return-to-work self-efficacy scale (RTW-SE) [35], the 36-Item Short Form Health Survey subscale (SF-36) [36, 37], three items measured on a Visual Analogue Scale (VAS), demographic items, and health- and work-related items. At the four-week follow-up, cancer survivors were asked again to fill in the QWLQ-CS and two anchor questions. Employed people without cancer or other physical/mental limitations affecting their job performance also completed a paper or digital questionnaire comprising the QWLQ-CS, demographic items and work-related items.

Instruments

Final version of the QWLQ-CS

The final version of the QWLQ-CS consisted of 23 items (Additional file 2). The overall QWLQ-CS score and sum scores of the subscales are calculated with a standardised score of 0–100, and at least 50% of the items need to be answered. Scores on negative items were reversed (N = 5). The scores are calculated by: ((sum of item scores – lowest possible score)/ range between lowest and highest possible score)*100. A higher score corresponds with a better QWL. Responses were given on a 6-point Likert scale (Totally disagree - Totally agree). The extra response category ‘Not applicable’ was available for items related to the work situation of self-employed cancer survivors, such as items about colleagues or supervisors. These items were analysed as missing.

Copenhagen Psychosocial Questionnaire (COPSOQ) subscales

Included were three COPSOQ subscales: ‘Meaning of work’, ‘Social community at work’, and ‘Social support from supervisors’ [34]. All subscales are scored 0–100 points, with a higher score indicating higher value on the subscale. The COPSOQ is a valid and reliable questionnaire for employees [34].

Return-to-work self-efficacy scale (RTW-SE)

The RTW-SE is an 11-item scale that consist of statements about the participant’s job. The scale score was calculated by computing a mean score. A higher score indicates better self-efficacy. The RTW-SE is validated among people with mental health problems [35].

36-item short form health survey (SF-36) subscale

Included was the subscale ‘Role limitations due to physical health problems’ (score range 0–100). A higher score corresponded with less role limitations. The SF-36 has been validated in a population with cancer [38, 39].

Visual analogue scale (VAS)

Three items with a VAS measured overall QWL, overall work satisfaction and satisfaction with fringe benefits. The scores on all items ranged from 0 to 100, with a higher score referring to a higher QWL or level of satisfaction. The VAS is a valid and reliable instrument for measuring quality of life [40] and is widely used in cancer research [41].

Other variables

The same demographic, health- and work-related variables as in field study I were assessed. Furthermore, to assess the reproducibility of the QWLQ-CS, we identified a stable subgroup of cancer survivors who responded to the following two anchor questions with ‘no’: ‘Did a major change take place in your health situation/work situation within the last four weeks?’

Data analysis

Construct validity

To measure the construct validity of the QWLQ-CS, convergent and discriminative validity were analysed at baseline (Additional file 1).

Convergent validity

Convergent validity of the QWLQ-CS was assessed by calculating the correlation between the QWLQ-CS, or one of its subscales, and existing reliable and valid scales or questionnaires that measure similar constructs. It was expected that the scales would correlate, and eight hypotheses about the magnitude and direction of the Spearman’s correlation coefficient were formulated (Table 4). Convergent validity was considered sufficient if ≥75% of the hypotheses were confirmed [29].

Table 4 Convergent validity: Spearman’s correlation coefficients
Full size table

Discriminative validity

Hypotheses about expected differences in QWL between two groups indicated discriminative validity of the QWLQ-CS [29]. The scores of cancer survivors on the QWLQ-CS were compared to those of employed people without cancer or other physical/mental limitations affecting their job performance. The response option ‘Not applicable’ was available for subscale 5 ‘Problems due to the health situation’. It was hypothesised that the outcomes on the QWLQ-CS would differ, with cancer survivors getting lower QWLQ-CS scores. The Mann-Whitney U-test assessed whether there were statistical significant differences between the two groups (p ≤ 0.05) [30].

Reproducibility

The reproducibility of the QWLQ-CS was assessed by test-retest reliability and the level of agreement (Additional file 1), measured at baseline and the four-week follow-up in the stable subgroup of cancer survivors. The Intraclass Correlation Coefficient with absolute agreement (ICCagreement) for the QWLQ-CS and subscales was calculated as a measure of test-retest reliability. We accepted an ICCagreement of ≥0.70 for use at group level [42]. Next, we measured the level of agreement by calculating the Standard Error of Measurement with absolute agreement (SEMagreement) of the QWLQ-CS and its subscales [43]. To detect any statistical errors between the measurements at baseline and the four-week follow-up, a t-test was performed to analyse whether the mean differences differed from zero in a statistically significant way [44]. Finally, we analysed the Limits of Agreement (LoA) by constructing a Bland and Altman plot [45]. In addition, a 95% Confidence Interval (CI) was calculated to assess the variability in the estimated limits [46].

Floor and ceiling effects

When >15% of the cancer survivors scored the lowest or highest possible score on the QWLQ-CS or its subscales, this was considered an indication of a floor or ceiling effect [29].

Results

Construct validity

The sample at baseline consisted of 130 cancer survivors (Table 1). The score average and standard deviations on the QWLQ-CS and its subscales are displayed in Table 5.

Table 5 Intraclass Correlation Coeffient (ICC) and Standard Error of Measurement (SEM) of the stable subgroup of cancer survivors
Full size table

Convergent validity

Spearman’s correlation coefficients between the overall QWLQ-CS score and VAS overall quality of working life was 0.70, and with VAS overall work satisfaction 0.61 (Table 4). The correlation between QWLQ-CS subscale 1 ‘Meaning of work’ and COPSOQ subscale ‘Meaning of work’ was 0.34, and between QWLQ-CS subscale 2 ‘Perceptions of the work situation’ and RTW-SE 0.53. A correlation of 0.58 was found for QWLQ-CS subscale 3 ‘Atmosphere in the work environment’ and COPSOQ subscale ‘Social community at work’. Correlations of 0.61 and 0.53 were found for QWLQ-CS subscale 4 ‘Understanding and recognition in the organisation’ and COPSOQ subscale ‘Support from supervisors’ and VAS satisfaction with fringe benefits respectively. The correlation between QWLQ-CS subscale 5 ‘Problems due to the health situation’ and SF-36 subscale ‘Role limitations’ was 0.63. Overall, ≥75% of the a priori hypotheses were confirmed.

Discriminative validity

Employed people without cancer or other physical/mental limitations affecting their job performance (N = 45) completed the QWLQ-CS (Table 1). Statistically significant differences were found between their overall QWLQ-CS mean score (M = 79) with a Standard deviation (SD = 11) and that of cancer survivors (M = 75, SD = 10) (p = 0.04). There were significant statistical differences between the mean scores on subscale 5 ‘Problems due to the health situation’ (p = 0.00) for employed people without cancer (M = 81, SD = 16) and cancer survivors (M = 57, SD = 24). There were no statistically significant differences (p = 0.13–0.95) on other subscales.

Reproducibility

Of the sample at baseline (N = 130), 100 cancer survivors completed the questionnaire at follow-up (23% lost to follow-up). Eighty-seven cancer survivors who indicated no change in response to the two anchor questions were allocated to the stable subgroup and their QWLQ-CS and subscales scores are displayed in Table 5.

The single measures ICCagreement for the overall QWLQ-CS and subscales ranged between 0.57 and 0.88 (Table 5). Subscales 2 ‘Perception of the work situation’ and 3 ‘Atmosphere in the work environment’ did not have an ICCagreement ≥ 0.70. The level of agreement, which we assessed by SEMagreement, ranged between 9.59 and 13.89, except for subscale 5 ‘Problems due to the health situation’, which scored a higher SEMagreement of 24.17. The mean differences of the overall QWLQ-CS score at baseline and follow-up did not statistically differ from zero (p = 0.694). The Bland and Altman plot (Fig. 1) displays the LoA with the means of baseline and four-week follow-up for the QWLQ-CS and the differences between these two measurements between the 95% confidence interval.

Fig. 1
figure 1

Bland and Altman plot for QWLQ-CS of the stable subgroup of cancer survivors (N = 87)

Full size image

Floor and ceiling effects

No cancer survivor had the lowest (0) or highest (100) possible overall QWLQ-CS score. The percentages of cancer survivors that scored the lowest or highest possible score on the subscales were <15%, so there were no floor or ceiling effects.

Discussion

The items in the final version of the QWLQ-CS were reduced to 23 within a five-factor structure: 1) Meaning of work, 2) Perception of the work situation, 3) Atmosphere in the work environment, 4) Understanding and recognition in the organisation, and 5) Problems due to the health situation. The QWLQ-CS had adequate internal consistency, construct validity, and reproducibility at group level. No floor or ceiling effects were detected.

QWLQ-Cs

In field study I we adequately performed EFA because the QWLQ-CS and its subscales had good internal consistency. In field study II we concluded that convergent validity was also good, although one hypothesis about correlations between a QWLQ-CS subscale and an existing subscale was not confirmed. The QWLQ-CS subscale 1 ‘Meaning of work’ had a low correlation with the COPSOQ subscale ‘Meaning of work’ (0.34). However, we followed the assumption that convergent validity is adequate when ≥75% of the hypotheses were confirmed [42], furthermore, the other correlations ranged between 0.53 and 0.70, which are moderate to strong correlations (r > 0.40) [29]. A possible explanation for the low correlation is that the COPSOQ subscale probably measured another construct than the QWLQ-CS subscale did. Although the items in both subscales looked similar, the latter subscale had included one different item: ‘Do you feel motivated and involved in your work?’. This might indicates a construct related to motivation and involvement in the organisation as well, whereas QWLQ-CS subscale 1 ‘Meaning of work’ does not. We recommend future research to study the convergent validity of this QWLQ-CS subscale with a different questionnaire that measures the same construct. For instance, the ‘Dedication’ subscale of the Utrecht Work Engagement Scale (UWES) [47].

Discriminative validity was assessed between cancer survivors and employed people without cancer or other physical/mental limitations affecting their job performance. There were statistically significant differences between the scores of the overall QWLQ-CS and subscale 5 ‘Problems due to the health situation’ for the two groups, with cancer survivors having a lower QWL score. This outcome is not surprising, as cancer survivors experience challenges in employment [8], which might influence QWL. However, cancer survivors do not always negatively differ from control groups. For instance, the quality of life of male cancer survivors (e.g. germ cell tumours) was similar to that of age-adjusted men [48]. Perhaps cancer survivors face more health issues at work (which might influence QWL) than in other areas of their lives that are influenced by quality of life. Furthermore, it seems that only health-related problems lower the QWL of cancer survivors. The scores of cancer survivors and employees without cancer did not differ on the subscales that contain generic items, only on the disease-specific items. Therefore, it might be interesting to study if the QWLQ-CS is also a valid QWL questionnaire for healthy employed people.

The reproducibility of the overall QWLQ-CS score was adequate with an ICCagreement of 0.84, when ≥0.70 is acceptable for use at a group level [42]. However, subscale 2 ‘Perception of the work situation’ yielded a lower ICCagreement of 0.57. Perhaps, this is caused by the homogeneity of the sample in regard to this subscale. By including cancer survivors with different backgrounds we assumed to have composed a heterogeneous sample. However, most cancer survivors had the same job for a long time, which might have influenced these items about self-efficacy, and made it difficult to distinguish between QWLQ-CS scores. Another parameter that determined the reproducibility of the QWLQ-CS was the SEMagreement, which ranged between 9.59 and13.89 (range 0–100) for the subscales. This is not uncommon, these SEM values are similar to quality of life outcomes on the SF-36 scale among COPD patients [49], and the SF-36 has been widely used because of its good psychometric properties. However, subscale 5 ‘Problems due to the health situation’ yielded a very high SEMagreement of 24.17, which suggests that the repeated measures on this subscale for cancer survivors are far apart, and it is more difficult to achieve accuracy on the ‘true’ score and measure clinically important changes. A possible explanation for this high SEM is that the sample of cancer survivors differed in the experience of health-related problems, which may be a consequence of including cancer survivors who were diagnosed between 0 and 10 years ago. To test this assumption we should analyse the reproducibility of the QWLQ-CS only among cancer survivors who are diagnosed <1 year ago.

In sum, the reproducibility of the QWLQ-CS at group level is adequate. However, to use the QWLQ-CS at individual level, the reproducibility should be improved. For instance, by enhancing the true variance, which can be done by improving the scale design [28]. In regards to the QWLQ-CS, this would imply that the descriptions on the scale could be modified [28] or the number of response categories could be enhanced [50].

Strengths and limitations

One strength of field study I is that we reduced the number of items in a systematic manner by performing an EFA. The steps taken in the EFA were statistically grounded, and decisions were made based on pre-set rules and extensive deliberation within the research group. By including cancer survivors with different demographic, health- and work-related backgrounds we developed a questionnaire with good psychometric properties that adequately measures QWL of different cancer survivors at group level.

Unfortunately, the results are less representative for cancer survivors with a different ethnic background, due to the lack of ethnic variety in the samples. Research in the USA revealed that racial or ethnic minorities of women with breast cancer were more likely to stop working compared to white women with breast cancer [51]. Perhaps this was also the reason for the lack of variety in ethnicity in our sample. However, the design of our study might also have led to non-participation: a systematic review on cancer survivorship research among minorities has shown that when working with minority populations, it is advisable to work inside the community and to draw on the help of respected leaders and involve minority members at every stage in the process [52]. This was not the case for our recruitment strategies, and therefore the results are not generalizable to cancer survivors with a different ethnic background. The implication for future research is that it is necessary to extend recruitment beyond hospitals and to recruit within the communities.

A methodological consideration concerns the number of factors in the QWLQ-CS. We used a combination of the scree plot and PA because previous literature suggested that there is no ‘golden standard’ [53]. The scree test provides a clear overview, but PA has proven to be more accurate in determining the number of underlying factors [53, 54]. Therefore, in case of any discrepancy, the final decision should be based on PA [54]. Ultimately, the two methods resulted in a unanimous number of factors: scree plot (N = 4) and PA (N = 8) with varying content and number of items within the factors. For instance, some factors in the PA contained a small number of items (N = 2), which is not desirable; a minimum of three items per factor is considered ideal [29]. Although no consistent evidence exists, it is suggested that PA might lead to overfactoring in case of smaller factor loadings and sample sizes [55]. Our sample size was not expected to be a problem as we had a ratio of 3.5 subjects per item, and a ratio of 4 subjects per item is recommended with a sample minimum of 100 [29]. However, QWL is a multi-dimensional construct and consists of items with very different contents which might lead to more low factor loadings. Eventually, we decided to determine the number of factors based on the outcomes of the scree test and PA, not only on the number of items, content and factor loadings. Therefore, we assume that the EFA did not result in an underfactoring or overfactoring of the number of factors in the QWLQ-CS. The adequate results of the psychometric properties of the QWLQ-CS support this conclusion.

Implications for practice and research

The QWLQ-CS is ready for use in clinical and occupational healthcare and research settings at a group level. As the QWLQ-CS was developed in Dutch we do advise cross-cultural testing if used in other countries. Because the QWLQ-CS is a self-administered questionnaire, it is easy to use in practice and research. The QWLQ-CS can be used in studying differences in QWL between groups of cancer survivors. Previous research found that return to work of head and neck cancer survivors was associated with oral dysfunctions, and problems with social eating and social contacts [56]. It might be possible that different diagnoses lead to difficult problems at work that subsequently influence QWL. These results could be helpful in the development of tailored interventions aimed at return to work or work continuation of cancer survivors. In organisations with many employed cancer survivors, for instance, but also in research settings where rehabilitation interventions are evaluated at a group level and QWL may function as an additional research outcome. Furthermore, as some items such as ‘Because of my health situation I have problems in my work with fatigue and/or lack of energy’ seem to be relevant to other health problems besides cancer, it would be interesting to study if the QWLQ-CS can be applied to employees with other chronic diseases as well. Finally, it would be interesting in future research to measure the responsiveness of the QWLQ-CS evaluating whether it is possible to measure clinical changes in QWL which increases the applicability of the QWLQ-CS in practice.

Conclusions

The five-factor QWLQ-CS with 23 items and adequate internal consistency, construct validity, and reproducibility at group level can be used in clinical and occupational healthcare and research settings.

Abbreviations

AMC:

Academic Medical Center

CI:

Confidence Interval

COPSOQ:

Copenhagen Psychosocial Questionnaire subscales

COSMIN:

hecklist COnsensus-based Standards for the selection of health Measurement INstruments

EFA:

Exploratory Factor Analysis

EORTC:

European Organisation for Research and Treatment of Cancer

ICC:

Intraclass Correlation Coefficient

LoA:

the Limits of Agreement

PA:

Parallel Analysis

PCA:

Principal Component Analysis

QWL:

Quality of Working Life

QWLQ-CS:

Quality of Working Life Questionnaire for Cancer Survivors

RTW-SE:

return-to-work self-efficacy scale

SEM:

Standard Error of Measurement

SF-36:

the 36-Item Short Form Health Survey subscale

UWES:

Utrecht Work Engagement Scale

VAS:

Visual Analogue Scale

References

  1. Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, et al. (2013). GLOBOCAN 2012 v1.0, cancer incidence and mortality worldwide: IARC CancerBase no. 11 [internet]. Lyon, France: International Agency for Research on Cancer; 2013. http://globocan.iarc.fr. Accessed 25 April 2016.

  2. Siesling S, Sonke G, de Raaf A, Jansen-Landheer M, Huijgens P. (2014). Presentation of Cancer Care Organisation [In Dutch: Integraal Kankercentrum Nederland (IKNL)]. https://www.kanker.nl/uploads/file_element/content/4604/kankerzorg-in-beeld-_kib_.pdf. Accessed 25 April 2016.

  3. Wells M, Amir Z, Cox T, Eva G, Greenfield D, Hubbard G, et al. Time to act: the challenges of working during and after cancer, initiatives in research and practice. Eur J Oncol Nurs. 2014;18:1–2.

    Article  PubMed  Google Scholar 

  4. de Boer AG, Taskila T, Ojajarvi A, van Dijk FJ, Verbeek JH. Cancer survivors and unemployment: a meta-analysis and meta-regression. JAMA. 2009;301:753–62.

    Article  CAS  PubMed  Google Scholar 

  5. Duijts SF, van Egmond MP, Spelten E, van Muijen P, Anema JR, van der Beek AJ. Physical and psychosocial problems in cancer survivors beyond return to work: a systematic review. Psychooncology. 2014;23:481–92.

    Article  PubMed  Google Scholar 

  6. Moskowitz MC, Todd BL, Chen R, Feuerstein M. Function and friction at work: a multidimensional analysis of work outcomes in cancer survivors. J Cancer Surviv. 2014;8:173–82.

    Article  PubMed  Google Scholar 

  7. Steiner JF, Cavender TA, Nowels CT, Beaty BL, Bradley CJ, Fairclough DL, et al. The impact of physical and psychosocial factors on work characteristics after cancer. Psychooncology. 2008;17:138–47.

    Article  PubMed  Google Scholar 

  8. Mehnert A, de Boer A, Feuerstein M. Employment challenges for cancer survivors. Cancer. 2013;119(Suppl 11):2151–9.

    Article  PubMed  Google Scholar 

  9. Wells M, Williams B, Firnigl D, Lang H, Coyle J, Kroll T, et al. Supporting 'work-related goals' rather than 'return to work' after cancer? A systematic review and meta-synthesis of 25 qualitative studies. Psychooncology. 2013;22:1208–19.

    Article  PubMed  Google Scholar 

  10. Pryce J, Munir F, Haslam C. Cancer survivorship and work: symptoms, supervisor response, co-worker disclosure and work adjustment. J Occup Rehabil. 2007;17:83–92.

    Article  PubMed  Google Scholar 

  11. Amir Z, Wynn P, Whitaker S, Luker K. Cancer survivorship and return to work: UK occupational physician experience. Occup Med. 2009;59:390–6.

    Article  Google Scholar 

  12. De Blasi G, Bouteyre E, Bretteville J, Boucher L, Rollin L. Multidisciplinary department of “return to work after a cancer”: a French experience of support groups for vocational rehabilitation. J Psychosoc Oncol. 2014;32:74–93.

    Article  CAS  PubMed  Google Scholar 

  13. Nugent BD, Weimer J, Choi CJ, Bradley CJ, Bender CM, Ryan CM, et al. Work productivity and neuropsychological function in persons with skull base tumors. Neurooncol Pract. 2014;1:106–13.

    PubMed  PubMed Central  Google Scholar 

  14. Kim YA, Yun YH, Chang YJ, Lee J, Kim MS, Lee HS, et al. Employment status and work-related difficulties in lung cancer survivors compared with the general population. Ann Surg. 2014;259:569–75.

    Article  PubMed  Google Scholar 

  15. Tamminga SJ, de Boer AG, Verbeek JH, Frings-Dresen MH. Breast cancer survivors' views of factors that influence the return-to-work process--a qualitative study. Scand J Work Environ Health. 2012;38:144–54.

    Article  PubMed  Google Scholar 

  16. de Jong M, Tamminga SJ, de Boer AG, Frings-Dresen MH. The quality of working life questionnaire for cancer survivors (QWLQ-CS): a pre-test study. BMC Health Serv Res. 2016;16:194.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chan KW, Wyatt TA. Quality of work life: a study of employees in shanghai, China. Asia Pacific. Bus Rev. 2007;13:501–17.

    Google Scholar 

  18. Mosadeghrad AM. Quality of working life: an antecedent to employee turnover intention. Int J Health Policy Manag. 2013;1:43–50.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Ventegodt S, Andersen NJ, Kandel I, Enevoldsen L, Merrick J. Scientific research in the quality of working-life (QWL): generic measuring of the global working life quality with the SEQWL questionnaire. Int J Disabil Hum Dev. 2008;7:201–17.

    Google Scholar 

  20. van Laar D, Edwards JA, Easton S. The work-related quality of life scale for healthcare workers. J Adv Nurs. 2007;60:325–33.

    Article  PubMed  Google Scholar 

  21. Murphy L. Job stress research at NIOSH: 1972-2002. In: Perrewé PL, Ganster DC, editors. Research in occupational stress and well-being: historical and current perspectives on stress and health, vol. 2. Kidlington, Oxford: Emerald Group Publishing Limited; 2002. p. 1–55.

    Chapter  Google Scholar 

  22. Martel JP, Dupuis G. Quality of work life: theoretical and methodological problems, and presentation of a new model and measuring instrument. Soc Indic Res. 2006;77:333–68.

    Article  Google Scholar 

  23. Brooks BA, Anderson MA. Defining quality of nursing work life. Nurs Econ. 2005;23:319–26.

    PubMed  Google Scholar 

  24. de Jong M, Tamminga SJ, de Boer AG, Frings-Dresen MH. Quality of working life of cancer survivors: development of a cancer-specific questionnaire. J Cancer Surviv. 2016;10:394–405.

    Article  PubMed  Google Scholar 

  25. Johnson C, Aaronson N, Blazeby JM, Bottomley A, Fayers P, Koller M, et al. EORTC quality of life group: guidelines for developing questionnaires modules. Brussels: EORTC; 2011. http://groups.eortc.be/qol/sites/default/files/archives/guidelines_for_developing_questionnaire-_final.pdf. Accessed 25 April 2016

    Google Scholar 

  26. de Jong M, de Boer AG, Tamminga SJ, Frings-Dresen MH. Quality of working life issues of employees with a chronic physical disease: a systematic review. J Occup Rehabil. 2015;25:182–96.

    Article  PubMed  Google Scholar 

  27. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539–49.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press; 2003.

    Google Scholar 

  29. de Vet HC, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine. A practical guide. Cambridge: Cambridge University Press; 2011.

    Book  Google Scholar 

  30. Field A. Discovering statistics using spss. 2nd ed. London, UK: Sage Publications Ltd.; 2005.

    Google Scholar 

  31. Fayers PM, Machin D. Quality of life: the assessment, analysis and interpretation of patient-reported outcomes. 2nd ed. UK: John Wiley & Sons Ltd.; 2007.

    Book  Google Scholar 

  32. Cattell RB. The scree test for the number of factors. Multivariate Behav Res. 1966;1:245–76.

    Article  CAS  PubMed  Google Scholar 

  33. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ Res Methods. 2004;7:191–205.

    Article  Google Scholar 

  34. Kristensen TS, Hannerz H, Hogh A, Borg V. The Copenhagen psychosocial questionnaire--a tool for the assessment and improvement of the psychosocial work environment. Scand J Work Environ Health. 2005;31:438–49.

    Article  PubMed  Google Scholar 

  35. Lagerveld SE, Blonk RWB, Brenninkmeijer V, Schaufeli WB. Return to work among employees with mental health problems: development and validation of a self-efficacy questionnaire. Work Stress. 2010;24:359–75.

    Article  Google Scholar 

  36. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care. 1992;30:473–83.

    Article  PubMed  Google Scholar 

  37. Aaronson NK, Acquadro C, Alonso J, Apolone G, Bucquet D, Bullinger M, et al. International quality of life assessment (IQOLA) project. Qual Life Res.1:349–51.

  38. Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, et al. Translation, validation, and norming of the Dutch language version of the SF-36 health survey in community and chronic disease populations. J Clin Epidemiol. 1998;51:1055–68.

    Article  CAS  PubMed  Google Scholar 

  39. Brazier JE, Harper R, Jones NM, O'Cathain A, Thomas KJ, Usherwood T, et al. Validating the SF-36 health survey questionnaire: new outcome measure for primary care. BMJ. 1992;305:160–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. de Boer AG, van Lanschot JJ, Stalmeier PF, van Sandick JW, Hulscher JB, de Haes JC, et al. Is a single-item visual analogue scale as valid, reliable and responsive as multi-item scales in measuring quality of life? Qual Life Res. 2004;13:311–20.

    Article  CAS  PubMed  Google Scholar 

  41. Pickard AS, Neary MP, Cella D. Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Outcomes. 2007;5:70.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.

    Article  PubMed  Google Scholar 

  43. de Vet HC, Terwee CB, Knol DL, Bouter LM. When to use agreement versus reliability measures. J Clin Epidemiol. 2006;59:1033–9.

    Article  PubMed  Google Scholar 

  44. Mantha S, Roizen MF, Fleisher LA, Thisted R, Foss J. Comparing methods of clinical measurement: reporting standards for bland and altman analysis. Anesth Analg. 2000;90:593–602.

    Article  CAS  PubMed  Google Scholar 

  45. Bland MJ, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327:307–10.

    Article  Google Scholar 

  46. Hamilton C, Stamey J. Using bland-Altman to assess agreement between two medical devices--don't forget the confidence intervals. J Clin Monit Comput. 2007;21:331–3.

    Article  PubMed  Google Scholar 

  47. Schaufeli WB, Bakker AB, Salanova M. The measurement of work engagement with a short questionnaire a cross-national study. Educ Psychol Meas. 2006;66:701–16.

    Article  Google Scholar 

  48. Hartung TJ, Mehnert A, Friedrich M, Hartmann M, Vehling S, Bokemeyer C, et al. Age-related variation and predictors of long-term quality of life in germ cell tumor survivors. Urol Oncol. 2016;34:60.e1–6.

  49. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clinical Epidemiol. 1999;52:861–73.

    Article  CAS  Google Scholar 

  50. Weng LJ. Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educ Psychol Meas. 2004;64:956–72.

    Article  Google Scholar 

  51. Mujahid MS, Janz NK, Hawley ST, Griggs JJ, Hamilton AS, Katz SJ. The impact of sociodemographic, treatment, and work support on missed work after breast cancer diagnosis. Breast Cancer Res Treat. 2010;119:213–20.

    Article  PubMed  Google Scholar 

  52. Aziz NM, Rowland JH. Cancer survivorship research among ethnic minority and medically underserved groups. Oncol Nurs Forum. 2002;29:789–801.

    Article  PubMed  Google Scholar 

  53. Reio TG, Shuck B. Exploratory factor analysis: implications for theory, research, and practice. Adv Devel. Hum Resour. 2015;17:12–25.

    Google Scholar 

  54. Gaskin CJ, Happell B. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud. 2014;51:511–21.

    Article  PubMed  Google Scholar 

  55. Crawford AV, Green SB, Levy R, Lo W-J, Scott L, Svetina D, et al. Evaluation of parallel analysis methods for determining the number of factors. Educ Psychol Meas. 2010; https://doi.org/10.1177/0013164410379332.

  56. Verdonck-de Leeuw IM, van Bleek W-J, Leemans CR, de Bree R. Employment and return to work in head and neck cancer survivors. Oral Oncol. 2010;46:56–60.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank our research assistant, S. van Hezel, for her help in data collection. We are grateful to the Dutch online cancer platform ‘Kanker.nl’ and the Dutch cancer patient organisation ‘Hoofd-Hals’ for their help in the recruitment of cancer survivors. This work was supported by COST Action IS1211 CANWON (A.G.E.M. de Boer, S.J. Tamminga & M. de Jong).

Funding

The study is part of the research project ‘Quality of working life of cancer survivors’ and was funded with a grant from the Dutch Cancer Society, registration number 2011–5228.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to ongoing research but are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

  1. Coronel Institute of Occupational Health, Amsterdam Public Health research institute, Academic Medical Center, P.O. box 22660, 1100, DD, Amsterdam, the Netherlands

    Merel de Jong, Sietske J. Tamminga, Monique H. W. Frings-Dresen & Angela G. E. M. de Boer

  2. Department of Head and Neck Surgical Oncology, UMC Utrecht Cancer Center, University Medical Center Utrecht, P.O. box 85500, 3508, GA, Utrecht, the Netherlands

    Robert J. J. van Es

Authors
  1. Merel de Jong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sietske J. Tamminga
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert J. J. van Es
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Monique H. W. Frings-Dresen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Angela G. E. M. de Boer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MF is the principal investigator of this study. All authors participated in designing the study. MdJ and RvE were involved in data collection. Data analysis was performed by MdJ, ST and AdB. MdJ generated the first draft of the manuscript, which was revised by ST, RvE, MF and AdB. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Angela G. E. M. de Boer.

Ethics declarations

Ethics approval and consent to participate

The Medical Ethics Committee of the Academic Medical Center (AMC) deemed ethical approval to be unnecessary (W14_323#14.17.0387). Written informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Psychometric properties. Psychometric properties of QWLQ-CS. (DOC 40 kb)

Additional file 2:

English version of the Quality of Working Life Questionnaire for Cancer Survivors (QWLQ-CS). Word file of English version of the Quality of Working Life Questionnaire for Cancer Survivors (QWLQ-CS). (DOC 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Jong, M., Tamminga, S.J., van Es, R.J.J. et al. The quality of working life questionnaire for cancer survivors (QWLQ-CS): factorial structure, internal consistency, construct validity and reproducibility. BMC Cancer 18, 66 (2018). https://doi.org/10.1186/s12885-017-3966-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12885-017-3966-1

Keywords

BMC Cancer

ISSN: 1471-2407

Contact us