Validating a generic cancer consumer quality index in eight European countries, patient reported experiences and the influence of cultural differences

Background Taking patient centeredness into account is important in healthcare. The European Cancer Consumer Quality Index (ECCQI) is a validated tool for international benchmarking of patient experiences and satisfaction. This study aimed to further validate the ECCQI in larger and more uniform groups of high volume tumours such as breast and prostate cancer. A second objective was the verification of the influence of cultural factors of the country to determine its possible use in international benchmarking. Methods Data from two survey studies in eight European countries were combined. Socio-demographic correlations were analysed with Kruskall-Wallis and Mann-Whitney tests. Cronbach’s alpha was calculated to validate internal consistency. Influences of masculinity (MAS), power distance (PD) and uncertainty avoidance (UA) were determined by linear regression analysis in a general model and subgroup models. Results A total of 1322 surveys were included in the analysis (1093 breast- and 348 prostate cancer patients). Cronbach’s alpha was good (α ≥ 0.7) or acceptable (0.5 ≤ α ≤ 0.7) in 8 out of 9 questionnaire categories, except in the category ‘Safety’ (α = 0.305). Overall ECCQI scores ranged from 22.1 to 25.1 between countries on a 1–35 scale (categories had a 1–4 scale). In certain subcategories such as ‘Organisation’ (range 2.2 vs 3.0) and ‘Supervision & Support’ (range 3.0 vs 3.8) a large difference was observed between countries. Differences in ‘Overall opinion’ were however small: mean scores of 3.7 vs 3.9, whereas median scores were all the maximum of 4.0. Power distance was positively associated with higher patient satisfaction scores whereas Uncertainty avoidance was negatively associated with these scores. Masculinity was only associated with patient satisfaction scores in lower educated patients. We found the highest impact of culture on overall scores in Hungary and Portugal and the lowest in Romania. Conclusions The ECCQI shows high internal consistency in all categories except ‘Safety’. Especially in separate categories and overall ECCQI scores the questionnaire showed discriminative value. This study showed a positive correlation of power distance and a negative correlation for uncertainty avoidance in some countries. When using the ECCQI for international benchmarking these two dimensions of culture should be taken into account. Supplementary Information The online version contains supplementary material available at 10.1186/s12885-021-07943-0.


Background
Oncological care is complex and multifaceted. Patients often see multiple healthcare providers that are engaged in prevention, diagnosis, treatment and follow-up. This requires a high degree of coordination and if inadequately organised, can result in fragmented and discontinued care and bad experiences for patients [1]. Patient centeredness is defined as: care that respects and responds to individual patient's preferences, needs, and values, and involves clinical decisions guided by patients. Improved health outcomes and better treatment adherence are associated with this approach [2,3]. Wessels et al. [4] reported that expertise and attitude of healthcare providers were more important to cancer patients than healthcare professionals expected. This underlines the importance of patient reported information on the perspective of the patient, which can be obtained using questionnaires. A generic consumer quality index was developed and piloted in six European countries in a previous study, the European Cancer Consumer Quality Index (ECCQI) [5]. An advantage of a generic questionnaire is, the possibility of usage for patients irrespective of tumour types. Moreover, this enables international comparison and benchmarking of patient experiences [3].
The main objective of this study is to further validate the ECCQI in two large volume tumour patient groups, breast and prostate cancer patients, as a generic instrument applicable in an international setting. Breast and prostate cancer were selected as both are one of the top five cancers worldwide [6]. The International Agency for Research on Cancer (IARC) estimates that worldwide from 2020 to 2040 the incidence of breast cancer will increase with 40% (from 2.179.457 to 3.059.829 patients) and for prostate cancer with 69% (from 1.356.176 to 2.293.818 patients) [7].
The results of the pilot of the ECCQI by Wind et al. [5] demonstrated significant differences in patient satisfaction between countries. The (sub) groups in this pilot where however small and the influence of differences in country culture was not taken into account [5]. According to Napier et al. [8] culture should be considered when looking at health and healthcare provision, as social determinants of health can vary from culture to culture and cultural attitudes of both patients and providers can vary over place. For example variations in health between European countries could partly be explained by cultural differences [9]. Hofstede's cultural six-dimension model [10] categorises aspects of cultural behaviour across countries so that they can be measured and compared [11]. An example of cultural attitudes influencing healthcare can be found by looking at power distance, one of the cultural dimensions of Hofstede [10]. In high power distance society's hierarchy is important, patients will treat doctors as superiors. In contrast, in low power distance society's patients treat doctors as equals [12]. Uncertainty avoidance, also part of Hofstede's model, can also be used to explain differences when looking at health [10]. Uncertainty-avoiding cultures look for structure in their organisations, institutions and relations in order to make events clearly interpretable and predictable [10]. Self-ratings of health across countries tend to correlate negatively with the uncertainty avoidance index [13], in other words the higher the uncertainty avoidance the lower the rating of one owns health. Hofstede's masculinity domain relates to ambition as a driving force and values being assertive and competitive compared to more feminine values like modesty and caring [12]. A feminine culture is focused more on quality of life and process versus a masculine culture with a stronger focus on task and more result orientation, e.g. is the patient cured [12]. Little is known about the link between patient perceived healthcare quality and cultural dimensions, hence the effect of cultural attitudes on patient reported experiences of care is unknown. Therefore, the second objective of this study is the verification of the influence of cultural factors in responses to this questionnaire in view of its possible use in international benchmarking.

Methods
The questionnaire used for this study, the European Cancer Consumer Quality Index (ECCQI), was previously develop and piloted by Wind et al. [5]. Data of breast and prostate cancer patients from two studies, the study in which the ECCQI was piloted [5] and a study that focused on patient involvement in which the ECCQI was further validated, were combined i.e. participating patients of both studies were included in our analysis if they met the inclusion criteria. We applied the following inclusion criteria: (1) Respondents had to be at least 18 years old, (2) Respondents had to be examined, treated or had aftercare for cancer within the last 2 years in the examined institute, (3) Age, gender and level of education had to be known and (4) at least 50% of the questions had to be answered.
The data of the first (pilot) study [5] was collected in six hospitals in Hungary, Portugal, Romania, Lithuania and Italy (n = 2) via a paper based survey and through an online survey in The Netherlands in 2015. In the first study all hospitals collected data for both prostate cancer patients and breast cancer patients. Respondents were selected by convenience sampling. The data of the second (involvement) study was collected in two other Dutch hospitals (one hospital collected only breast cancer patients and one hospital collected only prostate cancer patients) and one Czech (both prostate cancer patients and breast cancer patients collected) and Finnish hospital (only breast cancer patients collected) via an online survey in 2019. In total patients from 11 hospitals in eight countries in total (Hungary, Portugal, Romania, Lithuania, Italy, The Netherlands, Czech Republic and Finland) were included based on both studies. Both in the online as in the paper based survey the option Force answer was not used.

European Cancer consumer quality index
The ECCQI measures patient experiences and satisfaction with cancer care in hospitals in European countries. In the study by Wind et al. [5] patients indicated that in general the questionnaire was appropriate to measure patient satisfaction and experience. Confirmatory factor analysis revealed that the ECCQI measurement model had a moderate to good fit to the data in the first study [5] (RMSEA = 0.039, CFI = 0.943). The ECCQI consists of 63 questions divided into 11 categories. The core of the ECCQI measurement instrument are questions about the experiences of patients with care that are formulated as experience questions and questions about general appreciation. For the corresponding answer categories, the following applies: -Experience questions are formulated in terms of how often a particular quality aspect occurred (e.g. 'Did you understand the therapist's explanation?); the corresponding answer categories are 'never', 'sometimes', 'usually' and 'always'. If in questions of experience a frequency distribution is not relevant (a quality aspect is present or not) we worked with 'no', 'yes' as answer categories. For the questions pertaining to the attention of health professionals we had the following categories: 'none of them', 'only nurses', 'only doctors', only others', 'most' and 'all of them'. -For general appreciation questions, the respondent is asked to indicate on a scale of 0 (very bad) to 10 (excellent) what he/she thinks of the care provider or aspects of the care provision. The three categories with demographic or disease specific information had different answer categories and were used as background so were not part of the analysis. The analysis therefore includes 9 categories (40 questions). These categories are: Organisation (5); Safety (2); Attitude of Healthcare Professionals (6); Communication and information (4); Own inputs (2); Coordination (4); Supervision and support (10); Rounding of treatment (5); Overall opinion (2). Patients were given the opportunity to comment on the questionnaire. The full questionnaire can be found in Additional file 1.

Cultural influences
To adjust for cultural differences, we used Hofstede's cultural six-dimension model. For our study we use the three commonly used cultural domains [14] of masculinity (MAS), power distance (PD) and uncertainty avoidance (UA) domains [15,16]. Within Hofstede's model each domain is rated on a 1-100 scale in which, i.e., a score of 100 on masculinity describes a highly masculine society, which is associated with lower and more extreme scores on reviews [10,12]. MAS ranged between 14 (The Netherlands) and 88 (Hungary). PD ranged between 33 (Finland) and 90 (Romania). The lowest UA score was 53 (The Netherlands) and the highest UA was 99 (Portugal). A low masculine score indicates more tenderness and sympathy for others, possibly resulting in less willingness to provide criticism and therefore higher satisfaction scores. The used cultural model is a general description of a culture and does not have scores for socio-demographic groups. The effect of cultural differences was analysed using linear regression with MAS, PD and UA as independent variable for total scale score. The scores for MAS, PD and UA of all nationalities in the dataset were collected from 'Hofstede's insights country comparison tool' on 6-12-2019 and can be found in Additional file 2.

Recoding
Data were recoded in order to be analyzed. Almost all categories of the ECCQI consist of questions with four response options which were recoded into: never = 1, sometimes = 2, usually = 3 and always = 4. For the categories that did not consist of those four response options, the options were recoded into one of the four options above. For example, questions with sub-answers in the category "attitude of healthcare professional" had the option to state only nurses (score = 2), only doctors (score = 3) or only other healthcare professionals (score = 4) listen to the patient. These scores were recoded into "some of them did" (score = 2). The answer categories "most of them" (score = 5) and all of them (score = 6) were thereafter recoded into the scores 3 and 4. Questions related to the timing of events and the patients expectation, which have five answer categories, namely much sooner, sooner, when I'd expected it, later and much later were recoded into a four-scale by combining the options "sooner and much sooner" into one category, as done in the previous ECCQI study. Response codes of the questions about demographic characteristics were also recoded; (i) Age: 18-34, 35-64, and 65 or older; (ii) Years of education: low (1-8 years), moderate (9-13 years), and high (14 and higher). The answers 'I don't know/I no longer remember' and 'Not applicable' were scored as missing.

Analysis
Cases from the first and second study were selected from separate datasets and merged into one database if they met all four inclusion criteria. Data from this database was analysed in IBM SPSS statistics 24. Weighted means were calculated for each category of the ECCQI and country and depended on the number of items rated by the patient. Scale scores were summed and a weighted mean of overall patient experience was calculated [17]. The distribution of patient characteristics was determined by performing a chi-square test. Differences in total score were evaluated with a Kruskall-Wallis test and series of Mann-Withney tests, of which only the significant results are presented in this article. Internal consistency was evaluated using Chronbachs alpha for ordinal items [18]. The internal consistency was found good if α ≥ 0.7, acceptable if 0.5 ≤ α ≤ 0.7 and unacceptable if α ≤ 0.5 [19].
The influence of socio-demographic characteristics on the score of the ECCQI was analysed in order to verify whether age and education have an influence on reported quality. Gender was not analysed since no male breast cancer patients responded therefore analysing differences for tumour type is the same as analysing results for different gender. In addition, analysis of the dataset per participating county was performed, as previous results demonstrated significant differences between countries [5].

Response
Pooling the datasets resulted in a final dataset of 1441 patients, of which 1093 breast cancer patients and 348 prostate cancer patients. The selection process of the surveys is visualised in Fig. 1 and respondent characteristics can be found in Table 1. Significant differences were found in the chi-square test for age (χ2(14) = 121.614, p < 0.000), sex (χ2(7) = 602.647, p < 0.000), education (χ2(14) = 452.345, p < 0.000) and physical health((χ2(28) = 118.856, p < 0.000) and last time patients went to the hospital (χ2(35) = 360.286, p < 0.000). Table 2 shows the descriptive statistics of the ECCQI. The weighted mean of the scale scores ranged between 2.366 and 3.902. No significant differences were found in the initial comparison (χ2(7) = 31.226 p = 0.000) of the eight categories. Looking into the categories of the ECCQI we found that Italy scored high on the 'Organisation of care' (mean = 3.008, SD = 0.396). On the 'Support and supervision' category all countries scored above 3.0, except for Romania (mean = 2.959, SD = 0.661). Finland was found to be the highest scoring country in the category of 'Rounding of the treatment' (mean = 3.751, SD = 0.800). The highest scores of overall patient experiences were reported by patients from the Czech Republic (mean = 3.930, SD = 0.228) and Finland (mean = 3.907, SD = 0.314).

Results of the ECCQI per country
A series of 28 Mann-Whithney tests, which were used as post-hoc analysis, demonstrated significant differences in mean total score between multiple comparisons of countries. Portugal's score was significantly lower than the scores of The Netherlands (U = 5379.000, p = 0.011) and Romania (U = 545.000, p = 0.013). Portugal scored higher than Italy (U = 572.500, p = 0.010) and Czech Republic (U = 1483.500, p = 0.012). Finland scored lower than three countries, namely the Czech Republic (U = 33,824.000, p = 0.003), The Netherlands (U = 123,967.500, p = 0.000) and Romania (U = 12,821.000, p = 0.026). The last significant difference was a significantly higher score for Finland in the comparison with Italy (U = 13,179.000, p = 0.009).
The scale scores for each category are comparable to the scale scores between countries, which can be found in Table 2. We found that the category 'Organisation' was the category with the lowest average scale score. This category scored 2.3 on average, compared to average scale scores of > 3.0 in other categories of the ECCQ I.

Patient characteristics
Patients 65 and older reported a significantly higher score (mean = 3.469, SD = 0.389) compared to patients aged between 34 and 65 (mean = 3.520, SD = 0.381) (U = 218,217.000, p = 0.007). No significant differences in mean scale score were found between subgroups based on tumour type (equal to men and women), educational level and physical health.

Internal consistency
Five categories of the ECCQI had a good internal consistency level (α > 0.7), namely 'Attitude of the healthcare professional', 'Communication and information', 'Supervision and support', 'Rounding of the treatment' and 'Overall opinion'. The internal consistency of the categories 'Coordination', 'Organisation' and 'Own inputs' were acceptable (0.5 ≤ α ≤ 0.7). The internal consistency of 'Safety' was unacceptable (α = 0.305).
The category 'Organisation' with an overall acceptable internal consistency had unacceptable internal consistencies in the Czech Republic, Lithuania and Portugal and could not be calculated for Finland due to a large proportion of patients skipping questions of this category. The category 'Safety' had an unacceptable or barely acceptable internal consistency in most counties, and could not be calculated for Portugal due to zero variance in one of the two questions of this category. The only exception was seen in the Czech Republic, with α of 0.703. The overall internal consistency and internal consistencies per country can be found in Table 3.

Influence of culture
Eight regression models for different socio-demographic groups were made using MAS, PD and UA as determinants based on the linear regression analysis. As none of the determinants had a significant influence on the youngest age group, no model for this group is included in this article. The constants of the seven models included in this article (see Table 4) varied between 24.983 (constant of breast model) and 28.608 (constant moderate education model). The maximum value of the intercept is 35 (maximum possible score on the ECCQI). MAS had no significant influence on total scale score. The coefficients for MAS were only significant in the models of the three different education groups. A negative association with MAS was found in the lower educated patient model (β = − 0.610, p = 0.014). A positive association was found in the models of moderate (β = 0.037, p = 0.031) and higher educated patients (β = 0.042, p = 0.023).
PD was significant positive in all models, varying between 0.093 and 0.129, except in the prostate cancer model. A high score on PD is therefore associated with a higher score on the ECCQI. UA had in almost all regressions a significant negative association with perceived quality of care, varying between − 0.091 and − 0.192. So the higher the score on UA the lower the score on the ECCQI would be. However, no significant influence of UA was found on the total scale score in the prostate cancer, and lower educated regression models.

Discussion
In this study we further validated the ECCQI in eight countries among breast and prostate cancer patients. Comparability is an essential prerequisite of (international) benchmarking. It is therefore important to correct as much as possible for potential influencing factors [20]. We found differences between countries with both a small number of included as large number of included surveys on both the overall score and individual categories. Differences in score seem therefore to be independent from number of respondents, which enforces that applicability of the ECCQI for benchmarking. The significant differences ranged from 22.104 to 25.094 on a 1-35 scale. Patient experiences is on average scored the lowest in Italy and the highest in Romania. It must be considered that all Finnish respondents were breast cancer patients, in contrast to other countries, of which the respondent groups existed of both breast and prostate cancer patients. Although we found significant differences in mean scale scores, the total scale scores of the countries varied within a small range. It must be considered that all participating countries have a high level of quality of care since they are listed in the top 25 of the health access and quality index (HAQ). The HAQ ranks the access and quality of care of 195 countries [21].  The second objective, verifying the influence of cultural factors, helps to further establish the ECCQI as a tool for international benchmarking by looking into a possible influencing factor. We found that masculinity is negatively associated with reported patient experiences in lower educated patients. This could imply that the ECCQI score in masculine countries will be lower than in more feminine counties for lower educated patients. The opposite effect was found for patients with a moderate or higher education; these patients will probably score higher in a more masculine country. The effect of masculinity for lower educated patients on the patient reported experience is comparable to the association of lower patient satisfaction in masculine societies [15,16]. Uncertainty avoidance is also negatively associated with patient reported experiences in this study. This is consistent with the theory that, although medical statistics show no evidence of objective health differences, people in uncertainty tolerant countries still feel healthier [10]. A positive association was found between the power distance and patient reported experience of care. This means that patient reported experience score is higher in societies with a large power distance. Our general model demonstrated that in case of equal scores on power distance and uncertainty avoidance, i.e. for Romania, the effects of these cultural determinants is eliminated. All else the same, the patient satisfaction score of countries with a power distance score higher than their uncertainty avoidance score will be affected positively. When the ECCQI is used for international benchmarking it is important to look into differences in the cultural domains and if applicable correct for them. However, to further determine the influence of cultural differences on satisfaction with healthcare it would be suggested to assess people from different cultural backgrounds within one country. This eliminates the influence of the healthcare system setting.

Limitations
The internal consistency of the categories of the ECCQI was acceptable to good in most categories. Previous assumptions, that the low internal consistency of the category 'Safety' might have been caused by the low number of respondents, were falsified. A low internal For all categories the minimum score is 1 and the maximum score is 4, except for total where the maximum is 35 The α for each ECCQI category per country is displayed in bold. For each α, the valid N is shown between brackets. Means α could not be calculated consistency was namely also reported in countries with a high number of respondents. In addition, Cronbach's alpha for Portugal could not be calculated since one of the two questions had no variance. The category 'Safety' in the ECCQI should therefore be re-examined. The category 'Organisation' for Finland could not be calculated due to too little answers. We did not include the option "Force answer" in our survey tool, as forcing an answer is associated with higher drop-out rates [22].
The results of this dataset may be influenced by the differences in response of the various participating countries. A possible limitation of this study design is the sampling method. With convenience sampling the chance of selection bias is high which could have influenced the outcomes. For example, with regard to education level a majority of the Portuguese patients had a low education level, a majority of the Italian patient had a moderate education while in the other countries the majority had a high education level. Regarding physical health, patients in Portugal were more negative giving a moderate score, while in the other countries most patients rated their physical health as good or excellent. Analysis of the total study population however showed no influence of demographic characteristics.
The response target of the studies of which we pooled data was both set on 100 per participating hospital per tumour type. The second study however exceeded this target by 2.65. It is likely that the high response is caused by the differences in recruitment strategy of the involvement study, in which hospitals sent the survey per email to patients. This difference in response between recruitment strategies was also found in the first study, in which the institutes using a digitalized strategy had a higher response rate than the centres using a paper based survey. At the start of the recruitment, the Czech hospital stated that it is likely that a low number of elderly patients would respond due to digital illiteracy. The proportion of elderly patients participating in the Czech Republic did however deviate little from the proportions of participating elderly patients in The Netherlands and Finland. We did encounter a big difference in response between breast cancer patients (75.9%) and prostate cancer patients (24.1%). A possible explanation for this could be that breast cancer has a higher incidence in Europe (female breast cancer is the most commonly diagnosed cancer, prostate cancer holds a third place) [23]. There were also more hospitals in this study that provided data for breast cancer patients (10 out of 11) compared to prostate cancer (9 out of 11).
None of the hospitals that participated in the first study participated again in the second study and there were 4 years between the first and the second study.

Conclusion
This research confirmed conclusions about differences in patient satisfaction using the ECCQI as a measure. Significant differences between countries are likely not to be influenced by numbers of respondents. This study demonstrated good and acceptable internal consistency of the ECCQI. The items in the 'Safety' category of the ECCQI should be re-evaluated in order to hopefully increase the internal consistency of this category. Although the differences between the total scores are little, the ECCQI can discriminate between countries and used for benchmarking when looking into the category scores of this ECCQI. To our knowledge, this is the first study that reports associations of cultural aspects divided in masculinity, power distance and uncertainty avoidance with experiences and satisfaction of cancer patients measured by the ECCQI. The positive effect of power distance is however often outweighed by the negative effect of uncertainty avoidance. Power distance and uncertainty avoidance should therefore be included in international benchmarking in order to make valid comparisons.