Skip to main content

Effects of blended learning training for oncology physicians to advise their patients about complementary and integrative therapies: results from the multicenter cluster-randomized KOKON-KTO trial



Many oncology physicians are confronted with the topic of complementary and integrative medicine (CIM) by cancer patients. This study examined whether a blended learning (e-learning and a workshop) to train oncology physicians in providing advice on CIM therapies to their cancer patients, in addition to distributing an information leaflet about reputable CIM websites, had different effects on physician-reported outcomes in regard to consultations compared with only distributing the leaflet.


In a multicenter, cluster-randomized trial, 48 oncology physicians were randomly allocated to an intervention group (CIM consultation and an information leaflet) or a control group (information leaflet only). After the training, the oncology physicians conducted 297 consultations with their cancer patients. Measurements were assessed at oncology physician, physician–patient-interaction (measured by external reviewers), and patient levels. This analysis focused on the physician outcomes of stress reaction and perceived consultation skill competency. In addition, qualitative interviews were conducted with a subsample of oncology physicians who experienced both, the intervention and control condition.


The oncology physicians in the intervention group showed a lower stress reaction in all measured dimensions after CIM consultations than those in the control group. There was no significant difference between oncology physicians in the intervention and control groups regarding the perceived consultation skill competency (overburden: intervention 1.4 [95% CI: 0.7;2.1]; control 2.1 [95% CI: 1.4;2.7], tension: 1.3 [95% CI: 0.7;2.0] vs. 1.9 [95% CI: 1.3;2.5], and discomfort with consultation situations: 1.0 [95% CI: 0.4;1.7]; vs. 1.7 [95% CI: 1.2;2.3]). The qualitative data showed that only providing the leaflet seemed impersonal to oncology physicians, while the training made them feel well prepared to conduct a full conversation about CIM and provide the information leaflet.


In our exploratory study providing structured CIM consultations showed positive effects on the perceived stress of oncology physicians, and the training was subjectively experienced as an approach that improved physician preparation for advising cancer patients about CIM, however no effects regarding perceived consultation skill competency were found.

Trial registration

The trial registration number of the KOKON-KTO study is DRKS00012704 in the German Clinical Trials Register (Date of registration: 28.08.2017).

Peer Review reports


Many oncology physicians are confronted with the topic of complementary and integrative medicine (CIM) in their daily work. Many cancer patients already use CIM or ask their oncology physicians about CIM therapies [1,2,3]. Good communication about CIM between oncology physicians and their cancer patients could help to reduce potential pharmacological interactions between CIM and cancer treatment [4]. Furthermore, good communication can lead to higher levels of trust in cancer treatment and compliance, as well as increased patient satisfaction, less stress, and greater well-being [5]. To overcome knowledge- and communication-based challenges, structured training in the use of CIM in oncology is urgently required by oncology physicians [6, 7].

Generally, communication skills training for health care professionals who work with cancer patients has been widely implemented in recent years. Many of these trainings lead to improved communication skills, and oncology physicians who participate in such trainings are less likely to provide facts alone without individualizing their responses to the patient’s emotions [8]. However, evidence is limited as to whether training oncology physicians is not only beneficial for oncology physicians themselves but also well-tailored to the counseling needs of patients [8]. A paper from a consensus meeting among European experts emphasizes that trainings should support oncology physicians in feeling confident in mastering difficult communicative tasks. Moreover, it was agreed that training should enable physicians to respectfully take their patient’s clinical and personal context into account. In terms of educational methods, interactive and experiential approaches such as feedback and role-play are recommended so that physicians can reflect on themselves, their learning progress, and their communication [9].

An interprofessional consensus procedure focusing on competencies that are of specific interest for trainings in integrative oncology found that such consultations are accompanied by unique challenges [10]. For instance, there are many potential complementary and integrative medicine options with highly variable evidence bases [11], and a CIM consultation thus requires not only fundamental knowledge of CIM therapies but also the ability to provide evidence-based, balanced, resource-oriented, up-to-date CIM information that assists patients in making decisions. Moreover, oncology physicians should be open-minded and take patients’ beliefs into account [10].

In addition to the competencies taught, it is important that the information about CIM that is necessary to inform a patient is tailored to the needs of the physicians and to the consultation situation. Oncology physicians prefer lectures, face-to-face workshops, and flexible online education that is accessible all the time [12]. In regard to the consultation situation, the experience of a previously conducted trial shows that CIM consultations cannot be easily integrated into everyday practice due to time constraints and should therefore be brief and adjusted to the needs of both parties in the interaction [13].

The KOKON-KTO study was part of the Competence Network Complementary Medicine in Oncology (KOKON), a collaborative research project in Germany that was funded in a second funding period by German Cancer Aid [14]. The KOKON-KTO study (KTO = Consultation Training for Oncology Physicians) aimed to evaluate whether training oncology physicians to advise their cancer patients on CIM, in addition to distributing an information leaflet about reputable websites, had different effects on outcomes at the patient, oncology physician, and physician–patient levels compared to only distributing the information leaflet. The evidence-based blended learning training is tailored to oncology physicians with little experience in CIM who advise cancer patients about CIM in only 20 min [15]. This paper focuses mostly on oncology physician-reported outcomes.



The KOKON-KTO study was a prospective, multicenter, cluster-randomized trial in which oncology physicians trained in giving CIM advice (intervention) were compared to an untrained group of the same profession (control). Following the quantitative study, qualitative data were also assessed to learn more about the consultation from oncology physicians who could relate to both consultation settings. A detailed description of the study design can be found elsewhere [15].


A total of 48 oncology physicians (50% who specialize in gynecologic oncology) from hospital departments or private practices in Germany were included in the study. All potential participants needed to have a specialization in the field of oncology and were screened in advance in a telephone interview conducted by one of the authors. If they answered the following questions with yes, they were included: little knowledge of CIM, no previous structured trainings in the use of CIM in the field of oncology, little experience in advising cancer patients on CIM, sufficient resources to conduct consultations with cancer patients. Little knowledge of CIM was operationalized as exceeding basic knowledge that was taught during standard medical training. Little experience in advising cancer patients on CIM was operationalized as actively providing information on CIM to oncology patients. In addition, it was required that oncology physicians have sufficient resources to conduct consultations with cancer patients, were able to attend the workshop, and have good German language skills. After completing the training (intervention or delayed intervention for control oncology physicians), the participants received 34 Continuing Medical Education (CME) points from the German Doctors Association.

All the participating oncology physicians completed written informed consent forms and were told that they could withdraw from the intervention at any time during the study.

Hospital departments or private practices served as clusters. From each hospital department, two oncologists had to be eligible and served as one cluster. Randomization of clusters was stratified by type of center (hospital department or practice) and specialization (gynecology, other type of oncology) using an allocation ratio of 1:1. The randomization was performed by personal not otherwise involved in implementation of the study or analysis. After randomization, one oncology physician specializing in gynecologic oncology working in a hospital department was not able to participate in the study for personal reasons.


The training framework was previously published [16]. In brief, oncology physicians in the intervention group took part in a two-phase intervention. First, they participated in a blended learning (e-learning and workshops) that qualified oncology physicians to provide structured advice to their cancer patients on CIM using the KOKON-KTO consultation manual. The KOKON-KTO consultation manual was developed specifically for this occasion based on educational competencies [10] and evidence-based CIM knowledge [16]. Second, each intervention physician conducted consultations following the KOKON-KTO consultation manual with their own patients and provided an information leaflet about reputable websites on the use of CIM in oncology (KOKON-KTO information leaflet). Oncology physicians in the control group participated in a three-phase control intervention. First, the oncology physicians participated in a short e-learning that introduced them to the study and the KOKON-KTO information leaflet. Second, the oncology physicians in the control group provided the KOKON-KTO information leaflet to their own patients along with a short introduction to the leaflet and its use. Third, the oncology physicians in the control group also received KOKON-KTO training.

Data assessment

All oncology physicians were asked to answer questionnaires at multiple time points during the KOKON-KTO study. First, a web-based survey prior to e-learning was implemented (baseline MD). Second, paper–pencil questionnaires were distributed to the oncology physicians to provide information after each communication with their patients or provision of the information leaflet to their patients, respectively. Third, each oncology physician received a paper–pencil questionnaire after having conversations with as many patients as they were able to enroll in the study. For the physician–patient interaction level, a systematic external rating of a consultation of the participating oncology physicians was conducted by two experienced independent raters. For this, a consultation with a standardized patient was undertaken during the onsite skills-training workshop in the intervention and control groups.


Outcome measures for the evaluation of training in oncology have been defined based on an international consensus process [17]. Within this process, international experts discussed possible outcomes that have been identified as the most important outcomes within a systematic literature review. Workshop experts suggested that several outcomes should be assessed on three different levels: outcomes related to (1) the oncology physicians, (2) the specific interaction between oncology physicians and patients, and (3) patients who communicate with oncology physicians [17].

  • a) Physician level.

Two main outcomes at the physician level were selected: (1) the perceived consultation skill competency, indicated by three questions on subjective experiences of overburden, tension, and discomfort with the consultation situation; and (2) the perceived stress reaction, indicated by feeling secure at various moments in a CIM consultation (five questions on assessing previous CIM experiences, giving information about the effectiveness of CIM therapies, dealing with questions about unclear evidence, staying informed about interactions between antitumor therapy and CIM, and giving concrete advice). Both instruments were self-developed instruments as they needed to be tailored to the specific KOKON-KTO consultation setting and were measured on numerical rating scales (NRS) ranging from not at all (0) to fully competent/secure (10). The oncology physicians were asked to complete the perceived consultation skill competency instrument after each consultation, and the perceived stress instrument after all conducted consultations.

Further outcomes were the level of CIM knowledge in cancer care (multiple-choice questions); expectations regarding the effectiveness of CIM (5-point Likert scale from fully agree (5) to fully disagree (1)) and expectations regarding the side effects of CIM (NRS from very safe (0) to not safe at all (10)). Moreover, the oncology physicians were asked to indicate the duration of the CIM consultation in minutes. If the consultation took more than the anticipated 20 min, the oncology physicians were asked to provide reasons. All these questions were asked at baseline and after the training.

Furthermore, to obtain more information about the feasibility of the intervention, the physicians were asked questions after conducting all conversations. The ability of the manual-based consultation to be implemented in the treating oncology physicians’ daily work was measured on a 6-point Likert scale from very good (1) to not at all (6). Furthermore, the oncology physicians were asked to indicate their attitude toward CIM on four levels (the importance of being well informed about CIM, feeling confident during conversations, trying to avoid CIM conversations with patients, wishing that the patient would engage less in CIM) using an NRS from fully disagree (0) to fully agree (10)).

  • b) Patient level.

To learn more about the usefulness of the information leaflet about reputable websites on the use of CIM in oncology (KOKON-KTO information leaflet) that all the oncology physicians provided to their patients, the patients were asked whether they used the recommended websites. The response options were: No; Not yet, but I want to use it in the future; and Yes.

  • c) Physician–patient interaction level.

As a surrogate variable for indicating the physician–patient interaction, oncology physicians in the intervention group were asked to lead one KOKON-KTO consultation with a standardized cancer patient during the KOKON-KTO workshop. During these consultations, two independent external raters observed whether the oncology physicians followed the structure of the KOKON-KTO consultation [16] and rated the interactive and communicative competencies that are necessary for a good physician–patient interaction. The interactive and communicative competencies were rated using the “Munich Physician Patient Interaction Inventory” (MAPI) [18] (11 questions; 5-point Likert scale). All the raters were trained for reliability. For interactive and communicative competencies, the interrater reliability was substantial (average interclass correlation, R2 = 0.77 ± 0.19).

Statistical analyses

Mean values and standard deviations were calculated for metric variables, and frequencies and percentages were calculated for categorical variables. To investigate the effects of the intervention on those outcome variables assessed after each consultation, linear mixed models (LMMs) were employed in an intent-to-treat analysis. Fixed effects were the control group effect, the intervention group effect, and the variables with which the randomization was stratified (hospital/practice and oncology/gynecology). The clustering of patients treated by one physician was included as a random effect. Linear regression was performed as intent-to-treat for variables assessed once after conducting all consultations, and was adjusted for baseline, hospital/practice and oncology/gynecology. As suggested in the review process an additional analysis was performed reflecting the randomization by center; using the individual centers as clusters (random effect) instead of the single physician within each center (see Supplemental Materials). Since all statistical analyses are considered exploratory, adjusted group means and 95% confidence intervals are displayed, and no sample size was calculated.

Sensitivity analyses were performed for the per-protocol population (oncology physicians having KOKON-KTO consultations with at least five patients). Additionally, we controlled for the sex of the oncology physicians. Statistical analyses were conducted using RStudio [19].

Qualitative data

Qualitative data were collected in phase III of the study. Oncology physicians in the control group received KOKON-KTO training following the intervention phase as an incentive to participate in the study [15]. Furthermore, these oncology physicians were asked to voluntarily conduct the KOKON-KTO consultation with up to five of their cancer patients. Since this group of oncology physicians was experienced in both settings (conducting the KOKON-KTO consultation, and in short consultations only providing the KOKON-KTO information leaflet), semistructured telephone interviews were conducted 8–10 weeks after the KOKON-KTO training to asses both settings. The physicians were asked about: 1) their positive or negative experiences with both settings and possible reasons for these; 2) their own and their perception of the patients’ satisfaction with both settings; and 3) possible differences between the two settings (see Rogge et al. [20] for further information on the interview guidelines).

After nine interviews, an acceptable saturated spectrum of expressed topics and experiences [21] was reached. The interviews for the qualitative part were audiotaped and transcribed verbatim. A qualitative content analysis, following the methods described by Flick [22] and using the qualitative data analysis software MAXQDA [21], was performed. The transcripts were coded in sense units combining deductive and inductive coding strategies. The research team predefined deductive codes according to the KOKON-KTO consultation manual [16] and added further subcategories in a continuous process of inductively building codes from the data, and an intersubjective validation of the coding by two independent researchers (AAR and CC) to verify the reliability and robustness of the data analysis.


Baseline characteristics of oncology physicians and study flow

A total of 47 oncology physicians (n = 23 in the intervention group, n = 24 in the control group) completed the questionnaire (baseline MD) at baseline. Twenty-one oncology physicians in the intervention group completed the e-learning modules of the KOKON-KTO training, and 2 did not continue with the study due to changes of workplace or personal reasons. After the e-learning session, 1 oncology physician in the intervention group could no longer participate in the workshop for health reasons.

All the oncology physicians were asked to conduct up to 10 KOKON-KTO conversations each with their cancer patients and to provide the KOKON-KTO information leaflet (intervention group) or only provide the leaflet to patients (control group); then, the physicians were asked to complete a questionnaire after each consultation/provision of the KOKON-KTO information leaflet (t1 MD). The average number of consultations per oncology physician was 5.6 in the intervention group and 7.0 in the control group. Two oncology physicians in the intervention group were not able to enroll any patients in the study due to a lack of suitable cancer patients in their hospital. After completing all consultations with their cancer patients (t2 MD), the follow-up questionnaire was answered by 18 oncology physicians in the intervention group and 23 oncology physicians in the control group. A detailed description of the study flow can be found elsewhere [20].

The demographic characteristics of the participating oncology physicians are presented in Table 1. In total, 70.2% of oncology physicians were female (intervention group: 60.9%, control group: 79.1%). The oncology physicians in the intervention group were slightly younger and had fewer years of experience working with patients.

Table 1 Oncology physicians’ characteristics

Of the 297 included patients, 84.2% were female and 55.0 years old on average (intervention: 54.1 ± 10.6; control: 55.6 ± 11.8). The graduation level of 120 patients was A-Level or higher (intervention: 49 (30.4%) vs. control: 71 (42.0%)). Fifty-six percent of patients were diagnosed with breast cancer (intervention: 60.9% vs. control: 52.1%) and 9.8% with other gynecological cancers (intervention: 7.8% vs. control: 11.2%). Oncology physicians in the control group treated patients with more severe tumors, such as gastrointestinal tumors (intervention: 10.2% vs. control: 15.4%) and pulmonological tumors (intervention: 3.9% vs. control: 7.1%). The treatment objective of 62.3% of patients (intervention: 68.8% vs. control: 57.4%) was curative/adjuvant, for 34.3% it was palliative (intervention: 26.6% vs. control: 40.2%), and for 3.4% (intervention: 5.7% vs. control: 2.3%) it was not clear.

  • a) Physician level.

In regard to the anticipated effectiveness of specific CIM therapies, physical activity during anti-tumor therapy was concordantly rated as effective among most intervention oncology physicians, followed by acupuncture (95.2% agreement). More oncology physicians rated mistletoe, ginseng, and acupuncture as effective following e-learning, whereas a substantial decrease was found for the rated effectiveness of curcuma. Mindfulness training and acupuncture were considered safe CIM therapies by most oncology physicians after the training. The percentage of participants who considered hypnosis, mistletoe, and acupuncture safe increased, and the percentage of participants who considered physical activity safe decreased. The results of both time points are presented in Table 2.

Table 2 Oncology physicians’ ratings of their anticipated CIM effectiveness (in %) and CIM safety

The duration of consultations in the intervention group was 22.6 min on average (SD = 4.7). The oncology physicians in the control group communicated with their cancer patients for an average of 7.1 min (SD = 4.2).

Regarding the perceived stress reaction, oncology physicians in the intervention group scored lower than those in the control group on all five items. There was no significant difference between oncology physicians in the intervention and control group regarding perceived consultation skill competency. However, the mean scores were lower for all three levels of consultation skills (overburden: intervention 1.4 (95% CI: 0.7;2.1), control 2.1 (95% CI: 1.4;2.7); tension: intervention 1.3 (95% CI: 0.7;2.0), control 1.9 (1.3;2.5); and discomfort with the consultation situation: intervention 1.0 (95% CI: 0.4;1.7), control 1.7 (95% CI 1.2;2.3)) (Table 3).

Table 3 Primary outcomes at the physician level in the intention-to-treat sample. Linear Mixed Model analyses (adjusted means and differences between the intervention and control group)

The results of the sensitivity analyses were comparable to those utilizing the intention-to-treat sample.

There was no statistically significant difference in the rating of the ability of the consultation to be implemented between the two groups, with only small differences between the groups (intervention: 5.0 (95% CI: 4.8; 5.3), control 5.2 (95% CI: 5.0;5.5)).

Regarding the attitudes of oncology physicians toward CIM, the intervention group scored statistically significantly higher in being confident in conducting conversations, and statistically significantly lower in trying to avoid CIM conversations (Table 4).

Table 4 Results of linear regression analyses on attitudes toward CIM

Reasons stated by participating oncology physicians in the intervention group for conducting a KOKON-KTO consultation longer than 20 min included motivating the patient to change a behavior and having an overall more detailed consultation.

  • b) Patient level.

Of the 275 participating patients, the vast majority of both groups read the information leaflet about recommended CIM websites (intervention: 63.9%; control: 73.2%), and 68% of patients reported using at least one of the recommended websites. A smaller group (intervention: 20.5%; control: 15.7%) did not read the information leaflet before completing the questionnaire but were planning to do so in the future.

  • c) Physician–patient interaction.

The oncology physicians who participated in the workshop were able to demonstrate interactive and communicative competencies that are necessary for good physician–patient interactions (see table in the supplemental materials). In an earlier publication was reported that oncology physicians in the intervention group were able to apply the skills they had learned during the KOKON-KTO training after the workshop [16].

Qualitative results

Nine oncology physicians in the control group who received delayed KOKON-KTO training (88.9% were female and 42.0 ± 12.7 years old on average, 88.9% from hospital departments) reported that from the perspective of the physician leading the consultation, they felt competent and well prepared for both settings (KOKON-KTO consultation vs. only providing the KOKON-KTO information leaflet) [20]. Even though the oncology physicians still considered themselves to be lacking CIM knowledge, they found it helpful to be able to give advice on reputable online sources for CIM knowledge based on the leaflet. More than half of the oncology physicians (n = 6) stated that they felt as if they outsourced the topic of CIM and hence the patients’ concerns. For some oncology physicians, only providing the KOKON-KTO information leaflet felt impersonal. Moreover, some oncology physicians described doubting all patients’ abilities to use the internet. In the KOKON-KTO consultation setting, the oncology physicians felt competent and able to “meet their own quality requirements”. They liked the structure of the consultation and being an active part of it. However, some oncology physicians stated that such consultations might exceed the time frame and may “open Pandora’s box” regarding discussion of other CIM therapies that lack evidence (see the supplemental materials for the qualitative results).


Oncology physicians in the KOKON-KTO training group reported significantly less perceived stress after all consultations compared to the control group who only provided the KOKON-KTO information leaflet to their cancer patients. Regarding the perceived consultation skill competency measured after each consultation, no significant differences were found; however, oncology physicians receiving KOKON-KTO training scored lower on all items (less overburden, tension, and discomfort with the consultation situation) than control physicians. Oncology physicians in the intervention group were more confident in conversations and less likely to avoid CIM conversations. According to the qualitative data, the oncology physicians felt competent and well prepared for a conversation on CIM and for only providing the information leaflet, even though the latter felt somewhat impersonal.

The self-reported quantitative data of the patients in our study that have been published elsewhere support these findings. Patients who received the KOKON-KTO consultation exhibited higher levels of satisfaction in the group [20]. Both outcome levels (patient and physician) showed concordant trends between the groups, which result in a convincing basis for the KOKON-KTO training.

In a previous pilot study by Bloedt et al. [13], oncology physicians who were trained in a complementary communication blended learning program in providing advice during CIM-designated consultations achieved high patient satisfaction. However, the control group experienced comparable results, which might be explained by ceiling effects that are a common challenge in measuring the effects of communication trainings in cancer care [23, 24]. However, in this study, the view of oncology physicians who reported that additional consultations about CIM were difficult to integrate into everyday practice was also taken into account [13]. In our recent study, there was no difference in the subjective ability of the approach to be implemented between groups; therefore, 20 min appears not to interfere with everyday work.

In a study by Hayward et al., a complementary medicine practice guideline also led to positive results for health care providers, including oncologists (improvement in self-reported CIM knowledge and readiness to answer cancer patients’ questions about CIM) [25]. This education session was also created to accommodate the schedules of health care professionals and was available in multiple formats (in person as well as online). Shorter educational programs can serve as a sufficient alternative when oncology physicians aim to improve their CIM knowledge.

The oncology physicians in the control group in our study also received a short intervention and were introduced to providing an information leaflet about reputable websites on CIM. The majority of patients used the recommended websites, and the qualitative results showed that the oncology physicians found it helpful to use these resources. Sharing information leaflets might be a suitable alternative that is easy to implement in everyday care for oncology physicians who want to give their patients a helpful introduction to CIM, but also want to have more time for direct conversations on other topics [26]. The internet is a source that is frequently used to obtain information by cancer patients, but a study found that only approximately 1/3 of the obtained information influenced patients’ treatment decisions [27]. One reason for this discrepancy may be that online information is not evaluated in terms of quality and patients do not trust the information provided. Consequently, a pre-selection of high-quality, evidence-based, online resources about CIM and CIM-related adverse effects by health care providers might be helpful for cancer patients [28]. However, in addition to online information, oncology physicians should have knowledge and skills to inform themselves about specific information, such as the side effects of therapies, which could cause fear [29].

In our study, we used a blended learning method to train physicians. A review showed that it remains uncertain whether e‐learning alone improves or reduces health professionals’ skills [30]. A blended learning approach combines the advantages of online knowledge transfer, which can be adapted in its pace to physicians’ work lives with the possibility of direct interactions, exchanges, and relations with other learners, and the physical presence of the teacher during the onsite workshop.

In our study, we also measured how attitudes about the effectiveness and safety of CIM therapies changed after completing the e-learning. Oncology physicians displayed changes in the rated effectiveness and safety of most CIM therapies after participating e-learning. However, CIM therapies were rated as being effective and safe by most oncology physicians prior to the e-learning. Nutrition, physical activity, yoga, and acupuncture are the therapies that were mostly rated as effective and safe after e-learning. Patients’ expectations of CIM play a role in the treatment outcome [31]. However, to our knowledge, no studies have assessed how attitudes toward the effectiveness and safety of CIM therapies shape the recommendations of oncology physicians. Studying these associations between expectancies and attitudes toward CIM among physicians, and recommending those to patients, could help reduce bias and improve the quality of medical decision-making. For this, future research based on validated instruments as established for patient populations [32] is needed.

To our knowledge, this is the first study using a standardized framework [16], which was evaluated in a cluster-randomized trial of oncology physicians, interactions between oncology physicians and patients, and at the patient level [20]. The use of multiple assessment methods combining quantitative and qualitative measures provides a more complete evaluation of the training. However, our study has several limitations that need to be acknowledged.

Our study needs to be considered exploratory, and we did not measure effects on a single primary outcome. We involved a structured consensus approach for finding suitable outcomes for measuring effects [17], which led to the recommendation to measure several outcomes on three levels. The instruments for measuring our main outcomes were self-developed, as both need to be tailored to the specific study situation. Therefore, we do not have comparable data and no further information on the validity of the instruments. There might be over- or underestimations in specific groups due to self-reported data. We cannot rule out that oncology physicians rated their performance/the conversations more positively based on their perception in the situation. We were not able to completely blind the participants since they received a full intervention; however, the control physicians also received a short e-learning and material for their conversations (KOKON-KTO information leaflet). In addition, we used short-term (information about each consultation) and long-term (impressions after all conversations) information, which are both subject to bias. Long-term information can lead to recall bias, and oncology physicians might remember the most recent conversations more precisely. However, short-term information can be influenced by subjective emotions and might be of limited predictive value for long-term behavior. Furthermore, not all oncology physicians were able to conduct ten consultations with cancer patients, which results in limited comparability. In addition, we were not able to fully assess the implementation of the KOKON-KTO manual outside of the training situation. In our preplanned statistical analysis, we did not use the cluster as a random effect, as commonly done in cluster-randomized trials. As mentioned in our protocol publication [15]: the multilevel structure of the data is due to patients treated within a physician, and physicians working within a cluster. Because of the small number of physicians per cluster (either one or two) and the assumption that the communication varies more between single physicians than between centers, the center as cluster was not considered in the model. The results of the additional analysis, which adjusts for the specific center of the physician instead of just for the physician, did not differ in any clinically relevant way, indicating an overall robustness of the study results with respect to single physician versus center adjustment.

We did not video record the consultations based on experiences from a previous study [13]. Since the KOKON-KTO training follows a blended learning approach focusing not only on education on CIM-related knowledge but also on communication skills for oncology physicians, we cannot distinguish between components. However, we assume that both components are essential for effective communication with patients and cannot stand alone in training [5]. Moreover, even though randomization of oncology physicians took place, there were differences in the patient sample that oncology physicians selected for the conversations between the intervention and control groups. In the intervention group, there was a higher number of patients with more severe cancers, and the treatment was more frequently palliative in the control group than in the intervention group. This probably led to unequal preconditions for the conversations and may also influence stress during the conversation and one’s feeling of competence.


This exploratory study evaluated a novel education intervention for training oncology physicians in providing CIM advice in routine care. Providing structured CIM consultations following a blended learning training had positive effects on the perceived stress of oncology physicians but no significant effects on skill competency. Receiving blended learning was associated with higher confidence in CIM conversations and lower avoidance of CIM conversations. Oncology physicians felt well prepared for both measures—a full conversation about CIM and a simple provision of the information leaflet—even though only providing the information leaflet about recommended websites felt impersonal. The recommended websites were used by most of the patients. Therefore, providing information about recommended CIM websites could be a suitable and easy approach for oncology physicians. However, having information about the evidence base of the use of CIM therapies in cancer care, and how to communicate about CIM, can be an option to not only meet the needs of patients asking about CIM but also reduce stress for physicians.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author upon reasonable request.



Complementary and Integrative Medicine


Competence Network for Complementary Medicine—Consultation Training for Oncology Physicians


  1. Chang KH, Brodie R, Choong MA, Sweeney KJ, Kerin MJ. Complementary and alternative medicine use in oncology: a questionnaire survey of patients and health care professionals. BMC Cancer. 2011;11:196.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Wode K, Henriksson R, Sharp L, Stoltenberg A, Hök NJ. Cancer patients’ use of complementary and alternative medicine in Sweden: a cross-sectional study. BMC Complement Altern Med. 2019;19(1):62.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Grimm D, Voiss P, Paepke D, Dietmaier J, Cramer H, Kümmel S, et al. Gynecologists’ attitudes toward and use of complementary and integrative medicine approaches: results of a national survey in Germany. Arch Gynecol Obstet. 2021;303(4):967–80.

    Article  PubMed  Google Scholar 

  4. Jermini M, Dubois J, Rodondi PY, Zaman K, Buclin T, Csajka C, et al. Complementary medicine use during cancer treatment and potential herb-drug interactions from a cross-sectional study in an academic centre. Sci Rep. 2019;9(1):5078.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Frenkel M, Cohen L. Effective communication about the use of complementary and integrative medicine in cancer care. J Altern Complement Med. 2014;20(1):12–8.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Corbin Winslow L, Shapiro H. Physicians want education about complementary and alternative medicine to enhance communication with their patients. Arch Intern Med. 2002;162(10):1176–81.

    Article  PubMed  Google Scholar 

  7. Schofield P, Diggens J, Charleson C, Marigliani R, Jefford M. Effectively discussing complementary and alternative medicine in a conventional oncology setting: communication recommendations for clinicians. Patient Educ Couns. 2010;79(2):143–51.

    Article  PubMed  Google Scholar 

  8. Moore PM, Rivera S, Bravo-Soto GA, Lawrie TA. Communication skills training for healthcare professionals working with people who have cancer. Cochrane Database Syst Rev. 2018;7(7):Cd003751.

    PubMed  Google Scholar 

  9. Stiefel F, Kiss A, Salmon P, Peters S, Razavi D, Cervantes A, et al. Training in communication of oncology clinicians: a position paper based on the third consensus meeting among European experts in 2018. Ann Oncol. 2018;29(10):2033–6.

    Article  CAS  PubMed  Google Scholar 

  10. Witt CM, Balneaves LG, Carlson LE, Cohen M, Deng G, Fouladbakhsh JM, et al. Education competencies for integrative oncology—results of a systematic review and an international and interprofessional consensus procedure. J Cancer Educ. 2020;37:499–507.

  11. Deng GE, Frenkel M, Cohen L, Cassileth BR, Abrams DI, Capodice JL, et al. Evidence-based clinical practice guidelines for integrative oncology: Complementary therapies and botanicals. J Soc Integr Oncol. 2009;7(3):85–120.

    PubMed  Google Scholar 

  12. Klein GE, Guethlin C. Information and Training Needs Regarding Complementary and Alternative Medicine: A Cross-sectional Study of Cancer Care Providers in Germany. Integr Cancer Ther. 2016;17(2):380–7.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Blödt S, Mittring N, Schützler L, Fischer F, Holmberg C, Horneber M, et al. A consultation training program for physicians for communication about complementary medicine with breast cancer patients: a prospective, multi-center, cluster-randomized, mixed-method pilot study. BMC Cancer. 2016;16(1):843.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Güthlin C, Bartsch HH, Joos S, Längler A, Lampert C, Ritter C, et al. KOKON: A Germany-Wide Collaborative Research Project to Identify Needs, Provide Information, Foster Communication and Support Decision-Making about Complementary and Alternative Medicine in Oncology. Complement Med Res. 2020;27(2):105–11.

    Article  PubMed  Google Scholar 

  15. Helmer SM, Rogge AA, Fischer F, Pach D, Horneber M, Roll S, et al. Evaluation of a blended-learning training concept to train oncology physicians to advise their patients about complementary and integrative medicine (KOKON-KTO): study protocol for a prospective, multi-center, cluster-randomized trial. Trials. 2019;20(1):90.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Witt CM, Helmer SM, Schofield P, Wastell M, Canella C, Thomae AV, et al. Training oncology physicians to advise their patients on complementary and integrative medicine: An implementation study for a manual-guided consultation. Cancer. 2020;126(13):3031–41.

    Article  PubMed  Google Scholar 

  17. Fischer F, Helmer S, Rogge A, Arraras JI, Buchholz A, Hannawa A, et al. Outcomes and outcome measures used in evaluation of communication training in oncology - a systematic literature review, an expert workshop, and recommendations for future research. BMC Cancer. 2019;19(1):808.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Casar D. Evaluation eines Arzt-Patien-Interaktions-Trainings im Krankenhaus anhand von Selbst- und Fremdbeoachtungen. München: Ludwig-Maximilians-Universität München; 2009.

    Google Scholar 

  19. RStudio Team. RStudio: Integrated Development Environment for R. Boston, MA: RStudio, PBC;

  20. Rogge AA, Helmer SM, King R, Canella C, Icke K, Pach D, et al. Effects of training oncology physicians advising patients on complementary and integrative therapies on patient-reported outcomes: A multicenter, cluster-randomized trial. Cancer. 2021;127(15):2683–92.

    Article  PubMed  Google Scholar 

  21. VERBI Software. MAXQDA (Version 2022) [Computer software]. Berlin: VERBI Software; 2022.

  22. Flick U. Doing triangulation and mixed methods. SAGE Publications Ltd; 2018.

  23. Shilling V, Jenkins V, Fallowfield L. Factors affecting patient and clinician satisfaction with the clinical consultation: can communication skills training for clinicians improve satisfaction? Psychooncology. 2003;12(6):599–611.

    Article  PubMed  Google Scholar 

  24. Fujimori M, Shirai Y, Asai M, Kubota K, Katsumata N, Uchitomi Y. Effect of Communication Skills Training Program for Oncologists Based on Patient Preferences for Communication When Receiving Bad News: A Randomized Controlled Trial. J Clin Oncol. 2014;32(20):2166–72.

    Article  PubMed  Google Scholar 

  25. Hayward EN, Watling CZ, Balneaves LG. A pre-post evaluation of oncology healthcare providers’ knowledge, attitudes, and practices following the implementation of a complementary medicine practice guideline. Support Care Cancer. 2021;29(12):7487–95.

    Article  PubMed  Google Scholar 

  26. Jarernsiripornkul N, Nakboon S, Anarj K, Wongtaweepkij K. Survey of healthcare professionals’ practices, expectations, and attitudes towards provision of patient information leaflets in Thailand. Int J Clin Pharm. 2020;42(2):539–48.

    Article  PubMed  Google Scholar 

  27. Walsh MC, Trentham-Dietz A, Schroepfer TA, Reding DJ, Campbell B, Foote ML, et al. Cancer Information Sources Used by Patients to Inform and Influence Treatment Decisions. J Health Commun. 2010;15(4):445–63.

    Article  PubMed  Google Scholar 

  28. Ng JY, Munford V, Thakar H. Web-based online resources about adverse interactions or side effects associated with complementary and alternative medicine: a systematic review, summarization and quality assessment. BMC Med Inform Decis Mak. 2020;20(1):290.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Herber OR, Gies V, Schwappach D, Thürmann P, Wilm S. Patient information leaflets: informing or frightening? A focus group study exploring patients’ emotional reactions and subsequent behavior towards package leaflets of commonly prescribed medications in family practices. BMC Fam Pract. 2014;15:163.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E‐learning for health professionals. Cochrane Database Syst Rev. 2018(1):CD011736.

  31. Linde K, Witt CM, Streng A, Weidenhammer W, Wagenpfeil S, Brinkhaus B, et al. The impact of patient expectations on outcomes in four randomized controlled trials of acupuncture in patients with chronic pain. Pain. 2007;128(3):264–71.

    Article  PubMed  Google Scholar 

  32. Barth J, Kern A, Lüthi S, Witt CM. Assessment of patients’ expectations: development and validation of the Expectation for Treatment Scale (ETS). BMJ Open. 2019;9(6):e026712.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


We would like to thank Iris Bartsch, Beatrice Eden, Markus Horneber, Katja Icke, and the scientists at the Institute for Social Medicine, Epidemiology, and Health Economics, Charité – Universitätsmedizin Berlin for their helpful comments about the study. Furthermore, we would like to thank all participants of the KOKON-KTO study for their participation.


The “Complementary Medicine Training for Oncologists” (KOKON-KTO)-project was part of the collaborative research project KOKON, which was supported by German Cancer Aid [grant number 109863]. Additional funding for the e-learning of the KOKON-KTO training was provided by the Günter und Regime Kelm Foundation, Switzerland.

Author information

Authors and Affiliations



CMW, SMH and AAR wrote the manuscript and conducted and contributed to the design of the study. DP contributed to the study design and revised the manuscript. CC supervised the qualitative analysis, and RK performed the statistical analysis. All the authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Claudia M. Witt.

Ethics declarations

Ethics approval and consent to participate

The independent medical ethics committees approved the KOKON-KTO study (Ethics Committee of Charité–Universitätsmedizin Berlin (EA1/127/17), Medical Association Hamburg (MC-368/17), Medical Association Baden-Wuerttemberg (B-F-2017–10), Medical Association Nord Rhine (2417337), Ethics Committee of the Medical Association of Westphalia-Lippe (2017–624-b-S), Ethics Committee at the Medical Faculty of Würzburg (274/17_z-me), Ethics Committee of the Medical Faculty of Heidelberg (S-550/2017), Ethics Commission of the Albert-Ludwigs-University of Freiburg (531/17)). The study was conducted in accordance with the Declaration of Helsinki and ethical guidelines for clinical studies. Informed consent was obtained from all participants (oncology physicians and patients).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Physician–patient interaction. Qualitative results. Analysis with cluster adjustment.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Helmer, S.M., Rogge, A.A., King, R. et al. Effects of blended learning training for oncology physicians to advise their patients about complementary and integrative therapies: results from the multicenter cluster-randomized KOKON-KTO trial. BMC Cancer 23, 836 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: