Developing quality measures for breast cancer surgery is critical given the high frequency of surgical breast procedures and potential variation that exists across facilities and surgeons. Wasif et al. comment that quality measures must 1) be acceptable to various stakeholders; 2) include measureable elements; and 3) impact outcomes . While several national and international groups have developed quality cancer care measures, most focus on treatment other than breast surgery and participation in these programs tends to be voluntary. BRCASO aims to develop a more comprehensive, validated set of breast cancer surgical quality measures, and use these data to conduct research on surgical quality variation among a generalizable (i.e. non-voluntary) population.
Current quality measurement programs for cancer care delivery, while valuable, are limited in their ability to specifically address surgical quality and provide detailed data for research. Two of these programs, QOPI - ASCO's Quality Oncology Practice Initiative  and the American College of Surgeons' (ACoS) Commission on Cancer (CoC),  are primarily geared toward measuring and documenting overall quality of cancer care. Together, the CoC and American Cancer Society developed the National Cancer Data Base, one of the largest national registries of new cancer diagnoses. The data collected for each of these efforts primarily relate to documentation and treatment other than surgery.
One recent program dedicated primarily to breast care quality improvement is the American Society of Breast Surgeons' Mastery of Breast Surgery Program, another voluntary initiative that helps surgeons document their clinical performance; early results showed program participation rates among surgeons practicing in academic settings was low . Facilities may also receive certification from the National Accreditation Program for Breast Centers (NAPBC) . Accredited breast centers are required to meet certain surgical standards including > 50% of early stage breast cancer patients treated with breast conserving surgery, SNL for all patients with stage I or II disease, and diagnosis with palpation- or image-guided biopsy rather than open biopsy.
Perhaps the most detailed surgical quality measures come from the National Consortium of Breast Centers, Inc. and its National Quality Measures for Breast Centers (NQMBC) program . The NQMBC's quality measures include 37 measures for breast cancer diagnosis and treatment including surgical timeliness, pathology timeliness, pathology report completeness (tumor size, margins, lymph nodes, and specimen sampling adequacy), sentinel node biopsy, breast conservation surgery rate, and re-excision rate. The program includes over 200 voluntary participants and allows them to compare their performance with other centers across the US. Members can collect all measures or a select few depending on their certification level. The validity of some of these measures as reasonable quality measures has not been tested in a broad population of patients.
The data collected from the BRCASO study will enable us to evaluate a number of quality measures specific to breast cancer surgery. Our three initial proposed surgical quality measures are 1) initial mastectomy rate, 2) initial partial mastectomy positive margin rate, and 3) initial partial mastectomy re-excision rate. At this time, we are not recommending benchmarks for these measures, but instead, will evaluate variability in these measures by patient-, surgeon-, facility-, and regional-level factors, and by invasive and in situ diagnoses that may inform future benchmarks. We will be able to control these outcomes for clinical factors such as estimated tumor size, known multi-centric disease, breast imaging, anesthesia, localization technique, neo-adjuvant treatment, and detailed tumor characteristics. We also have detailed data on timeliness of procedures, extent of lymph node procedures, positive node counts, and reconstruction, which may serve as additional surgical quality measures. Most importantly, we have collected margin status for each procedure performed, which is important in assessing surgical quality and cannot be extracted from tumor registry data.
The BRCASO database has the potential to be one of the largest, multi-site breast cancer surgical quality research databases to-date. The research possibilities may extend even beyond surgical quality measures. One of the major differences between BRCASO and the national efforts described above is that BRCASO does not rely on voluntary reporting. BRCASO includes a diverse geographic sample of all surgeons who practice at our varied institutions (health maintenance organization, university hospital, contracted community hospitals). In addition, a proportion of the data from the CRN sites were collected from electronic administrative data, a process that has the potential to be extended to other organizations with similarly organized billing data.
However, the BRCASO database is not without limitations. Our electronic administrative data were less sufficient than we had hoped for in pre-filling the database. Thus our chart abstraction took longer than we anticipated, and we fell short of the originally anticipated 6,095 cases. We have no detailed information on surgeon training, surgeon specialty, patient breast size, family history or patient preferences - all of which can account for variability in surgical procedures and quality. The focus of BRCASO was to collect data on specific short-term surgical outcomes such as reexcisions rates; thus we do not have data on long-term outcomes of recurrence or mortality, but plan to expand our efforts to collect these data in the future.
We learned several important lessons that others may want to consider when collecting surgical quality data. First, while electronic administrative data were useful in prefilling some of our data elements, they have serious limitations. Most importantly, there is no way to get accurate detailed margin or lymph node data from electronic administrative data; re-excision data may also be inaccurate as repeat procedures do not appear to be well documented in electronic data. Tumor registry data generally represent final surgical procedure and final pathology, but do not offer a clear characterization of multiple procedures that a patient may receive. The CoC database does collect information on positive/negative margin status, but does not include detail on margin distance or direction. Until more detailed, standard definitions are developed and implemented, we can only obtain detailed surgical quality data through medical record abstraction; but these data are time consuming and expensive to collect. We noted modest variation in the information provided in pathology, surgical operative, and other clinical reports between organizations and facilities. While standardizing these reports across institutions would be ideal, it was not possible in this retrospective study. Therefore, having highly trained medical record abstractors, a detailed coding manual, and regular channels for abstraction-related queries were invaluable to reducing data discrepancies in our studies.
Collecting a large amount of multi-site surgical quality data is a complex task - but the potential benefits and knowledge gained may be substantial. While some of the national efforts mentioned here will provide larger databases with which to evaluate quality of breast cancer care, only one (the NQMBC program) appears to collect the level of detail needed to evaluate quality of surgical care. These databases also rely solely on self-reported information from providers who opt-in to each system, and may be more likely to involve high volume breast surgeons. These providers may not be representative of the national population of surgeons because many surgeons performing breast surgery in the United States are neither high volume breast surgeons nor work within a certified breast center . Our study collected data on every surgeon who performs breast procedures at one of four geographically disparate institutions, with variable volume in breast surgery. Standardizing clinical data collection and reporting efforts across programs and databases (including ours), with institutional participation rather than voluntary, and publicly reporting results will likely be necessary to develop timely benchmarks for surgical quality. Additional programming work, such as Natural Language Processing, may help improve data collection timeliness and efficiency by automatically interpreting text from electronic medical records, thus reducing the need for manual abstraction. Creating standard templates in electronic medical records for pathology and surgery reports may be another method that can improve both quality and consistency of data collected and reported. We hope the work from BRCASO provides a stepping stone for research on and development of standard surgical quality measures that can be generalized to and implemented by other providers and institutions.