Utilization of a breast cancer risk assessment tool by internal medicine residents in a primary care clinic: impact of an educational program
BMC Cancer volume 19, Article number: 228 (2019)
Despite strong evidence of benefit, breast cancer risk assessment and chemoprevention are underutilized by primary care physicians. This study evaluates the impact of an educational program on knowledge and utilization of the NCI Breast Cancer Risk Assessment Tool (BCRAT) by internal medicine residents.
Internal medicine residents at the primary care clinic at William Beaumont Hospital participated in an educational program on breast cancer risk assessment and chemoprevention. A questionnaire was used to assess knowledge and practice before and after participation. Electronic health records of women between the ages of 35 and 65 who were seen by participating residents for annual health exams between Dec 15, 2015 and Dec 14, 2016 were reviewed. Utilization of BCRAT by the residents was compared pre- and post-educational program.
A total of 43 residents participated in the study. 31 (72.1%) residents reported no prior knowledge about BCRAT. The remaining 12 (27.9%) reported limited knowledge of BCRAT, but the majority of these (n = 10, 83.3%) had not used it in the last six months. For each question on the pre-educational knowledge assessment, fewer than 10% of the residents responded correctly. After implementation of the educational program, there was a significant increase in the proportion of residents who answered correctly (Range: 67 to 100%, p < 0.001).
Electronic health records of 301 clinic patients were reviewed, 118 (39.2%) in the pre-educational program group and 183 (60.8%) in the post-educational program group. There was a higher use of BCRAT in the post-educational program group compared to the pre-intervention group (3.8% vs. 0%, p < 0.05). However, a majority (n = 294, 98.7%) of eligible patients from both groups did not undergo breast cancer risk assessment.
Our study demonstrates that an educational intervention improved residents’ knowledge of BCRAT. Despite this improvement, a significant proportion of patients did not undergo breast cancer risk assessment. Expanding the scope and duration of this intervention and combining it with innovative use of technology to improve utilization should be the subject of future investigation.
One in eight women in the United States will develop invasive breast cancer in her lifetime . In 2017, it was estimated that 252,710 new cases of breast cancer were diagnosed and 40,610 women died from it . Although it is impossible to predict with certainty who will develop breast cancer, clinicians can identify women who may be at an increased risk and provide them with options for early detection and risk reduction. A number of validated, quantitative risk-assessment models incorporate features of a patient’s medical and family history to accurately estimate their individual risks . Examples of such risk assessment models in use today include the Gail model , the Claus model  and the Tyrer-Cuzick model , each with its own unique strengths and limitations.
The Gail model was originally designed in 1989  and then modified in 1999 . The modified Gail model, also known as the National Cancer Institute (NCI)-Gail model or the Breast Cancer Risk Assessment Tool (BCRAT), is available on the NCI’s website (https://bcrisktool.cancer.gov/). The National Comprehensive Cancer Network guidelines endorse the BCRAT to identify women who may be candidates for chemoprevention with one of the currently approved agents for risk reduction. . Similar guidelines issued by American Society of Clinical Oncology  and United States Preventative Services Task Force  also advocate for use of BCRAT and other breast cancer risk assessment tools to identify women who will benefit from chemoprevention. BCRAT is one of the most widely used breast cancer risk assessment tools by primary care physicians (PCPs) due to its simplicity and easy accessibility on the internet .
Several important medical decisions may be impacted by knowing a woman’s underlying risk of developing breast cancer . For women 35 years old with a five-year projected risk of ≥1.67% by BCRAT, chemoprevention with tamoxifen, raloxifene, or exemestane should be considered . Use of tamoxifen in this high-risk population decreases the risk of breast cancer by 49% . For patients with a lifetime breast cancer risk of 20–25%, as defined by family history based risk assessment model, MRI is recommended as a supplement to mammography . The sensitivity of MRI in this high-risk population is significantly higher, ranging from 71 to 100%, compared to 16 to 40% for mammography . In addition, risk assessment and identification of women who are at a higher risk of developing breast cancer can lead to referral to a high risk breast clinic; this, in turn, has the potential to identify breast cancer at an early stage and improve outcomes .
Despite the availability of these breast cancer risk assessment tools and their clinical validation, they have received very little attention outside of the oncology clinic . The use of breast cancer risk assessment tools by PCPs is less than 25% [16, 17]. This is concerning as women without a personal history of breast cancer are primarily managed by PCPs. For these women to benefit from breast cancer risk reduction strategies, it is important that their PCPs be aware of and able to use risk assessment models in daily clinical practice.
Our study focuses on one specific group of PCPs – internal medicine residents – since a prior study has demonstrated that resident PCPs are even less likely to use breast cancer risk assessment tools compared to their attending physicians . Hence, in this study, we evaluated the utilization of BCRAT by internal medicine residents at William Beaumont Hospital, Royal Oak.
The primary objective of this study was to evaluate the impact of a comprehensive educational program on the knowledge and utilization of BCRAT by internal medicine residents. The secondary objective was to evaluate the feasibility of such an educational program.
The study was conducted over an eight-month period, from April 2016 to November 2016. Each month internal medicine residents (PGY-1 through PGY-3) assigned to an outpatient clinic rotation were offered participation. The residents took part in a thirty-minute interactive educational program at the beginning of their clinic month. The program consisted of a didactic lecture focused on breast cancer risk assessment, chemoprevention and importance of high-risk screening. In addition, the residents were provided with current literature including professional societies’ guidelines, on the topic of breast cancer risk assessment and pharmacologic prevention. A smart-phrase was created in the electronic health system (EPIC) to assist in performing BCRAT. Pre- and post-educational program questionnaire (Additional file 1) consisting of multiple choice questions was used to assess the residents’ knowledge and practice prior to the educational program, and to measure the change in knowledge (Fig. 1).
In order to evaluate the utilization of BCRAT in clinical practice, electronic health record (EHR) of women between the ages of 35 and 65 who were seen by the participating residents in the clinic for annual health exams between December 15, 2015 and December 14, 2016 were reviewed. Women with a prior history of breast cancer or a known genetic mutation were excluded. Utilization of BCRAT by the residents was assessed pre- and post-program implementation.
Data was collected in Microsoft Excel (Ver. 2007) and statistical analysis was performed using SPSS 21(IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY:IBM Corp.). Fischer’s exact test was used for categorical variables and Mann-Whitney U-Test was used for continuous variables. All tests were two sided. Statistical significance was considered at p < 0.05.
A total of 43 (54.4%) out of the eligible 79 residents participated in the study. A majority of these residents were graduates of a United States/Canadian medical school and were in the first year of their residency training (Table 1).
Pre- and post-educational program questionnaire
All participating residents (n = 43) completed the pre-educational program questionnaire. When asked about their comfort level in assessing a woman’s risk for developing breast cancer, using an incremental linear scale from 0 to 10, more than half (58.1%) of the residents responded with a score of 5 or less (Median score: 5; Mode: 5). 31 (72.1%) residents stated that they had never heard of the Gail model or the NCI BCRAT before. The remaining 12 residents (27.9%) had heard of the Gail model but 10 (83.3%) out of the 12 stated they had never used it in the last six months.
In the pre-program questionnaire on knowledge about BCRAT, more than two-thirds of the residents answered each question incorrectly or did not select an answer. There was a significant increase (p < 0.001) in the proportion of correct answers in the post-program questionnaire, with more than two-thirds answering each of the questions correctly (Table 2).
Utilization of BCRAT
EHR of 301 patients seen for annual physical exams by the participating residents were analyzed.118 (39.2%) patients had been seen by the residents prior to their participation in the educational program (pre-educational program group), while 183 (60.8%) were seen after the residents’ participation in the program (post-educational program group). Both groups of patients were similar in terms of their baseline demographic characteristics (Table 3).
In total, 7 (3.8%) patients in the post-educational group underwent breast cancer risk assessment. There was a modestly higher use of BCRAT in the post-educational group of patients compared to the pre-educational group (3.8% vs. 0%, p < 0.05). Although the sample of the residents in each sub-group was small, there was no difference in the likelihood of performing BCRAT by place of training (US/Canada vs. Other) or level of training (Post-graduate year) for the residents.
A total of 294 (97.7%) patients, 118 from the pre-educational program group and 176 from the post-educational program group, did not have a breast cancer risk assessment performed. Out of these 294 patients, 40 (13.3%) had sufficient information in the chart to calculate their Gail risk retrospectively, and 5 (12.5%) out of these 40 women had a 5-year risk of breast cancer ≥1.67%. For the remaining 254 patients, Gail risk could not be calculated from the existing information in EHR. Of these 254, 102 patients were from the pre-educational group while 152 patients were from the post-educational group, constituting 86.4 and 83.0% of their corresponding groups respectively. In more than two-thirds of the patients, information on age at menarche and age at first live birth was not available in the EHR, thus making the BCRAT calculation not possible. However, among these patients, 28 (11%) had at least one first degree relative with breast cancer.
Our study demonstrates that a comprehensive lecture-based educational program improved the knowledge of BCRAT in internal medicine resident PCPs. A prior study that evaluated the role of a multimodal curriculum to teach internal medicine residents evidence-based breast health demonstrated improvement in knowledge after implementation of the program, but did not evaluate the impact on clinical practice . To the best of our knowledge, our study is the first one to evaluate the impact of an educational program on both knowledge and practice of internal medicine resident PCPs regarding BCRAT.
In our study, more than two-third of the residents expressed that they had no knowledge of the Gail model/BCRAT prior to implementation of our program, and among those who knew about it, the self-reported utilization was minimal. Evaluation of EHR of 118 eligible patients who were seen by these residents prior to their participation in the program revealed that none of the patients had breast cancer risk documented. Although the utilization rate of BCRAT by PCPs has been shown to be low in prior studies [16, 17, 19], the rate observed in our study is much lower. However, in contrast to our study, which looked directly into the EHR for utilization data, most of the prior studies relied on self-reported utilization rates and also included attending physicians.
The low rate of utilization of breast cancer risk assessment by resident PCPs is concerning. Whether this observation holds true in other residency programs is not known, but warrants further investigation. Our study serves as a cautionary reminder for other training programs to evaluate their curriculum and to emphasize on utilization of BCRAT in the context of teaching early detection and prevention of breast cancer. Considering the fact that residency training impacts future practice behavior [20, 21], it is imperative that residents become familiar with breast risk assessment tools so that they will carry this knowledge into their future practice. Furthermore, the resident staffed clinics often serve a patient population with a disproportionately larger share of complex medical, psychosocial and financial or insurance issues . Since disparities in health care correlates with adverse outcomes in patients with breast cancer , it is even more important that resident PCPs are thoroughly engaged in primary prevention of breast cancer in this population.
Chemoprevention agents such as tamoxifen and raloxifene have demonstrated clear benefit in reducing risk of breast cancer in high-risk women [13, 24,25,26]. Despite this, they are significantly underutilized in the primary care clinic . One of the proposed reasons for underutilization of chemoprevention is lack of knowledge, training or time on part of the PCPs to assess a woman’s risk of developing breast cancer . In our study, 12.5% of patients for whom the Gail risk was calculated by the study investigators retrospectively were found to have an elevated five-year risk of breast cancer. These patients would potentially benefit from a balanced discussion about chemoprevention, and as such may represent missed opportunities for preventing breast cancer.
On a more positive note, our study demonstrates that internal medicine residents are willing and able to absorb knowledge on breast cancer risk assessment and prevention. However, despite this improvement in knowledge and understanding, the uptake of BCRAT increased only to a modest 3.8%. Our study did not assess for reasons behind why the residents did not assess breast cancer risk despite having adequate knowledge to do so. There are several potential reasons for the low uptake including time constraint in a busy clinic, not attributing enough clinical significance to breast cancer risk assessment over other preventative measures and not being required to perform BCRAT by attending physicians. Future studies should aim to explore some of these potential reasons. Our study also suggests that knowledge of BCRAT may not be the only barrier to implementing a successful program on breast cancer risk assessment.
Our study primarily focused on didactic lecture as the means of conveying information, but future studies should evaluate more innovative methods such as team-based or problem-based learning techniques, which may be more effective than the traditional lecture forum [29,30,31,32]. Also, encouraging use of mobile applications to calculate Gail risk may also improve utilization. It is notable that more than two-thirds of the patients did not have any information in EHR on date of menarche or first live birth, two important components of BCRAT. These clinical parameters are considered very useful in an office visit with a gynecologist but it is likely that not enough significance is attributed to these clinical parameters in an internal medicine clinic, and hence were likely not collected prior to or during the patient visit. Embedding BCRAT into EHR or allowing patients to input their information related to BCRAT in EHR might help residents more easily identify patients at an increased risk. While clinic attendings were by design not part of our intervention, they play an important role in resident education and supervision, and can further reinforce and encourage the residents to use BCRAT.
There are several limitations of our study including the small number of residents from a single institution and limited scope of our intervention. We did not evaluate the long-term impact of our educational program nor did we specifically look at reasons for low uptake. We did not evaluate or educate the residents on how to effectively communicate the results of BCRAT to patients, which is another limitation. Future studies should build upon our findings and include ongoing education to reinforce the important concepts of breast cancer risk assessment and prevention.
Our study demonstrates that an educational program geared towards resident PCPs improves knowledge and utilization of BCRAT. Despite this improvement, there is a need to expand the scope and duration of this intervention in order to optimize the use of BCRAT in a busy clinic. Combining educational programs with other teaching modalities and innovative use of technology may further improve the utilization of BCRAT and therefore, should be investigated in future studies.
Breast Cancer Risk Assessment Tool
Electronic Health Record
National Cancer Institute
Primary Care Physicians
Feuer EJ, Wun LM, Boring CC, Flanders WD, Timmel MJ, Tong T. The lifetime risk of developing breast cancer. J Natl Cancer Inst. 1993;85:892–7.
Siegel RL, Miller KD, Jemal A. Cancer statistics, 2017. CA Cancer J Clin. 2017;67:7–30.
Evans DG, Howell A. Breast cancer risk-assessment models. Breast Cancer Res. 2007;9(5):213.
Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, Mulvihill JJ. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989;81(24):1879–86.
Claus EB, Risch N, Thompson WD. Autosomal dominant inheritance of early-onset breast cancer. Implications for risk prediction. Cancer. 1994;73(3):643–51.
Tyrer J, Duffy SW, Cuzick J. A breast cancer prediction model incorporating familial and personal risk factors. Stat Med. 2004;23(7):1111–30.
Costantino JP, Gail MH, Pee D, et al. Validation studies for models predicting the risk of invasive and total breast cancer incidence. J Natl Cancer Inst. 1999;91(18):1541–8.
National Comprehensive Cancer Network. Breast cancer risk reduction (version 1.2017). Available at: https://www.nccn.org/professionals/physician_gls/pdf/breast_risk.pdf. Accessed 5 June 2017.
Visvanathan K, Chlebowski RT, Hurley P, Col NF, Ropka M, Collyar D, et al. American society of clinical oncology clinical practice guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction. J Clin Oncol. 2009;27(19):3235–58.
Moyer VA, Preventive Services US, Force T. Medications to decrease the risk for breast cancer in women: recommendations from the U.S. Preventive Services task Force recommendation statement. Ann Intern Med. 2013;159(10):698–708.
Korde LA, Gadalla SM. Cancer risk assessment for the primary care physician. Prim Care. 2009;36(3):471–88.
Armstrong K, Eisen A, Weber B. Assessing the risk of breast Cancer. N Engl J Med. 2000;342:564–71.
Fisher B, Costantino JP, Wickerham DL, Redmond CK, Kavanah M, Cronin WM, et al. Tamoxifen for prevention of breast cancer: report of the National Surgical Adjuvant Breast and bowel project P-1 study. J Natl Cancer Inst. 1998;90(18):1371–88.
Saslow D, Boetes C, Burke W, Harms S, Leach MO, Lehman CD, et al. American Cancer Society guidelines for breast screening with MRI as an adjunct to mammography. CA Cancer J Clin. 2007;57(2):75–89.
Cadiz F, Kuerer HM, Puga J, Camacho J, Cunill E, Arun B. Establishing a program for individuals at high risk of breast cancer. J Cancer. 2013;4(5):433–46.
Sabatino SA, McCarthy EP, Phillips RS, Burns RB. Breast cancer risk assessment and management in primary care: provider attitudes, practices, and barriers. Cancer Detect Prev. 2007;31(5):375–83.
Corbelli J, Borrero S, Bonnema R, McNamara M, Kraemer K, Rubio DJ, et al. Use of the Gail model and breast cancer preventive therapy among three primary care specialties. Womens Health (Larchmt). 2014;23(9):746–52.
Corbelli J, Bonnema R, Rubio D, Comer D, McNeil M. An effective multimodal curriculum to teach internal medicine residents evidence-based breast health. J Grad Med Educ. 2014;6(4):721–5.
Ravdin PM. The lack, need, and opportunities for decision-making and informational tools to educate primary-care physicians and women about breast cancer chemoprevention. Cancer Prev Res (Phila). 2010;3(6):686–8.
van der Leeuw RM, Lombarts KM, Arah OA, Heineman MJ. A systematic review of the effects of residency training on patient outcomes. BMC Med. 2012;10:65.
Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009;302(12):1277–83.
Holmboe ES, Bowen JL, Green M, Gregg J, DiFrancesco L, Reynolds E, et al. Reforming internal medicine residency training. A report from the Society of General Internal Medicine's task force for residency reform. J Gen Intern Med. 2005;20(12):1165–72.
Ayanian JZ, Kohler BA, Abe T, Epstein AM. The relation between health insurance coverage and clinical outcomes among women with breast cancer. N Engl J Med. 1993;329(5):326–31.
Cuzick J, Powles T, Veronesi U, Forbes J, Edwards R, Ashley S, Boyle P. Overview of the main outcomes in breast-cancer prevention trials. Lancet. 2003;361(9354):296–300.
Vogel VG, Costantino JP, Wickerham DL, et al. Effects of tamoxifenvsraloxifene on the risk of developing invasive breast cancer and other disease outcomes: the NSABP study of tamoxifen and Raloxifene (STAR) P-2 trial. JAMA. 2006;295(23):2727–41.
Cummings SR, Eckert S, Krueger KA, et al. The effect of raloxifene on risk of breast cancer in postmenopausal women: results from the MORE randomized trial. Multiple Outcomes of RaloxifeneEvaluationJAMA. 1999;281(23):2189–97.
Amir E, Goodwin PJ. Breast cancer chemoprevention gets personal. J ClinOncol. 2011;29(17):2296–8.
Peres J. Why is breast cancer chemoprevention such a hard sell? J Natl Cancer Inst. 2014;106(5):diju139.
McMullen I, Cartledge J, Finch E, Levine R, Iversen A. How we implemented team-based learning for postgraduate doctors. Med Teach. 2014;36(3):191–5.
Wamsley MA, Julian KA, O'Sullivan P, McCance-Katz EF, Batki SL, Satre DD, et al. Team-based learning exercise efficiently teaches brief intervention skills to medicine residents. SubstAbus. 2013;34(4):344–9.
Inuwa IM, Al-Rawahy M, Roychoudhry S, Taranikanti V. Implementing a modified team-based learning strategy in the first phase of an outcome-based curriculum—challenges and prospects. Med Teach. 2012;34(7):e492–9.
Anwar K, Shaikh AA, Dash NR, Khurshid S. Comparing the efficacy of team based learning strategies in a problem based learning curriculum. APMIS. 2012;120(9):718–23.
This study was funded by Beaumont Health Oncology Research Seed Grant – Concept Award (HIC-2015-465; R/C: 91428). The funding body did not play any role in study design, collection, analysis and interpretation of data, and writing the manuscript.
Availability of data and materials
All data used and/or analysed during the current study are available from the corresponding author on a reasonable request.
Ethics approval and consent to participate
The requirement for informed consent for chart review portion of this study was waived due to retrospective nature of the data. Prior to the participation in the study, resident physicians were provided with an Institutional Review Board-approved written print-out discussing the methodology of the study, the voluntary nature of participation including the option to withdraw from the study at any point, and the contact addresses for further questions and concerns. Since participation into the study and completion of questionnaire was optional for the resident physicians, consent was implied when residents participated in the program and completed the questionnaire. This study was approved by Institutional Review Board at Beaumont Health (HIC-2015-465).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Yadav, S., Hartkop, S., Cardenas, P.Y. et al. Utilization of a breast cancer risk assessment tool by internal medicine residents in a primary care clinic: impact of an educational program. BMC Cancer 19, 228 (2019). https://doi.org/10.1186/s12885-019-5418-6