Skip to main content
Download PDF

Developing a risk assessment tool for cancer-related venous thrombosis in China: a modified Delphi-analytic hierarchy process study

BMC Cancer volume 24, Article number: 120 (2024) Cite this article

Abstract

Objective

To develop a Risk Assessment Tool for Cancer-related Venous Thrombosis in China.

Methods

A modified two-round Delphi method was employed to establish consensus within a field to reach an agreement via a questionnaire or by interviewing a multidisciplinary panel of experts by collecting their feedback to inform the next round, exchanging their knowledge, experience, and opinions anonymously, and resolving uncertainties. Furthermore, The AHP (Analytic Hierarchy Process) was used to determine the final quality indicators’ relative importance.

Results

The expert’s positive coefficient was 85.19% in the first round and 82.61% in the second round, with authoritative coefficients of 0.89 and 0.92 in the respective surveys. The P-value of Kendall’s W test was all less than 0.001 for each round, and the W-value for concordance at the end of the two rounds was 0.115. The final Risk Assessment Tool for Cancer-related Venous Thrombosis consisted of three domains, ten subdomains, and 39 indicators, with patient factors weighing 0.1976, disease factors weighing 0.4905, and therapeutic factors weighing 0.3119.

Conclusion

The tool is significantly valid and reliable with a strong authority and coordination degree, and it can be used to assess the risk of cancer-related VTE and initiate appropriate thrombophylactic interventions in China.

Peer Review reports

Introduction

Venous thromboembolism (VTE), a medical condition that encompasses pulmonary embolism (P.E.) and deep vein thrombosis (DVT), is the second most common condition after acute myocardial infarction and stroke [1]. It has become a significant cause of unexpected death and a common complication for cancer patients. Previous studies have demonstrated that individuals with Cancer have a higher risk of developing thrombosis, which was 4–7 times greater than those without Cancer and accounted for approximately 20% of all VTE cases [2, 3]. Nevertheless, the occurrence and mortality rate of thrombosis in patients can be lowered with proper evaluation and early prevention [4]. It is important to note that excessive prevention may lead to a higher risk of bleeding and financial burden. As a result, it is crucial to utilize appropriate risk assessment tools to forecast venous thromboembolism associated with Cancer.

There have been several cancer-related VTE assessment tools developed from 2008 to 2021, including 5 in the United States, 2 in Italy, 2 in Spain, and 1 in China [5,6,7,8,9,10,11,12,13,14,15,16,17,18]. One of the initial risk assessment tools for VTE in outpatients undergoing chemotherapy was developed by Alok A. Khorana et al. in 2008 [5]. However, previous evaluations [19] have shown that these existing tools are highly biased, and their verification results vary significantly. The overall methodological quality of these tools needs improvement, and there is a need to study their risk stratification ability. Furthermore, these tools’ lack of reliability and validity is crucial for patient outcomes such as morbidity and mortality. As mentioned, China’s clinical practice still has limitations, particularly regarding systematic and practical VTE risk assessment tools for cancer patients.

The Delphi method effectively gathered expert opinions by providing anonymity, feedback, and statistical analysis, which led to a consensus through repeated information exchange and feedback. However, this approach cannot compare the importance of these indicators. The Analytic Hierarchy Process (AHP) is a method that can be used to compare the importance of indicators through weight analysis. This method complements indicators by systematically assigning weights to each indicator based on relative importance. Decision-makers can make more informed and rational decisions by considering the weights assigned to each indicator, enhancing the overall quality and reliability of the decision-making process. This study combined a modified Delphi method and AHP to devise a cancer-related VTE risk assessment tool in China.

Methods

Literature retrieval and construction of preliminary index pool

A literature retrieval was conducted to identify the risk factors for cancer-related VTE. The search encompassed several databases, including CNKI, WanFang, VIP, CBM, PubMed, Embase, and 21 related institutions’ and societies’ websites, and it continued until January 2023. Additionally, the references of relevant studies were manually searched to ensure that all critical information was included. English and Chinese search terms were utilized, including Venous Thromboembolism, Deep Vein Thrombosis, Pulmonary Embolism, cancer, tumor, neoplasms, risk assessment tool/score/model, risk evaluation tool/score/model. The 21 websites of related institutions and societies included 15 health-related websites in China, the United States, Japan, Canada, and other countries, as well as specific medical associations and societies such as the Chinese Medical Association, Chinese Society of Clinical Oncology(CSCO), National Comprehensive Cancer Network (NCCN), American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), and International Society of Thrombosis and Hemostasis (IGTH), for a total of 6 society websites. Based on the references and discussion in the group, the research referred to the existing and commonly used VTE risk assessment tools, combined with the actual situation in China, and developed a risk item pool. Finally, an initial preliminary index pool for cancer-related VTE was formed, including 4 first-level items, 8 second-level items, and 35 third-level items. The flow chart of searching the literature is shown in Fig. 1.

Fig. 1
figure 1

The flow chart of searching the literature

Full size image

Study design

In the early stage, we obtained a preliminary index pool through a literature review and formed a preliminary correspondence form after discussion by the research group. After two rounds of Delphi expert consultation, the risk predictors of cancer-related VTE were obtained, and AHP analyzed the index weights. The detailed design process is shown in Fig. 2.

Fig. 2
figure 2

A study design

Full size image

Expert selection

A distinguished group of 27 professionals, comprising clinical, pharmaceutical, and nursing experts from Level A general hospitals, were chosen as Delphi consultation experts. These experts met stringent criteria, including over a decade of work experience in their field, a bachelor’s degree or above, the title of an associate professor or higher, familiarity with the research topic, and a willingness to participate in multiple rounds of Delphi consultation communication.

Data collection

A group of researchers created indicators for cancer-related VTE risk using evidence-based research. This scale included four first-level items, eight second-level items, and 35 third-level items. Experts were contacted via e-mail or telephone and provided a questionnaire outlining the research background, objectives, and methodology. The questionnaire gathered information in four areas: i) demographic information of the experts, such as years of work, education, and research field. ii) Delphi expert consultation content, where each expert rated the necessity, importance, and operability of 47 potential indicators using a Likert scale from 1 to 5. iii)A familiarity scale ranging from 1 (unfamiliar) to 5 (very familiar), and iv) a basis for their judgments, which included theoretical analysis, experience, peer understanding, and personal intuition.

Throughout each round, experts were allowed to ask open questions and suggest changes to the index system by revising, deleting, or adding indicators. After a comprehensive analysis, We have made adjustments to the indicators accordingly. Additionally, the questionnaire was modified based on the previous round’s qualitative feedback and statistical analysis. After reaching a consensus, we established the final index system.

Data analysis

The data was analyzed using the SPSS 22.0 statistical software and the YAAHP 10.1 analytic hierarchy process software. To be included in the analysis, items had to receive a necessity, importance, and operability score of ≥ 4.0 points, achieve a percentage of full score over 50%, and have a variation coefficient<0.25. The Analytic Hierarchy Process (AHP) was used to establish the hierarchical structure and construct the judgment matrix. The values were determined based on the average ratings of each criterion using Satya’s scale. This process helped in determining the criteria. At last, the combined weight of third-level items was multiplied by 100 and rounded to an integer value, representing the risk score of the corresponding three-level items for practical purposes.

Results

Characteristics of the experts

This study surveyed 23 experts from various regions of China, including Sichuan (18, 78.26%), Chongqing (2, 8.7%), Guangdong (1, 4.35%), Guizhou (1, 4.35%), Yunnan (1, 4.35%), and et al., with diverse occupations. Of the participants, 11 (47.83%) were clinicians, 10 (43.48%) were pharmacy staff, and 2 (8.7%) were nursing staff. The educational background of the specialists varied, with 14 (60.87%) holding a master’s degree, 6 (26.09%) holding a Ph.D, and 3 (13.04%) holding a bachelor’s degree. All professionals held senior or associate senior titles, with 9 (39.13%) seniors and 14 (60.87%) associate seniors. More information about the demographics of the experts is available in Table 1.

Table 1 Characteristics of expert panelists(n = 23)
Full size table

Expert’s authority coefficient

The reliability of consulting outcomes is measured by the Expert’s Authority Coefficient (Cr), which is taken into account Expert Familiarity (Cs) and Judgment Basis (Ca) [20]. During the initial round, 87.0% (47 indicators) of the outcomes had a Cr of ≥ 0.80(ranging from 0.73 to 0.99, mean = 0.89). The second round had a slightly higher average Cr of 0.90 (ranging from 0.78 to 1.00) than the first. (Table 2)

Table 2 The values of Cr in two surveys
Full size table

Degree of coordination of experts’ opinions

In the first round, the coefficient of variation (CV) for the 44 indicators’ necessity was<0.25 (ranging from 0.00 to 0.24, mean = 0.16). Meanwhile, the CV for the six indicators in importance was ≥ 0.25, and the CV for the 44 indicators’ operability was<0.25 (ranging from 0.00 to 0.23, mean = 0.14). The Kendall’s W coefficient was 0.185, 0.190 and 0.159. Finally, we would filter indicators by combining CV, mean, and full score ratio. Table 3 for details.

Table 3 Coordination degree of experts’ opinions in the two-round survey
Full size table

Weight and score of risk factors for cancer-related VTE

We consulted with experts twice and reached a consensus on the final risk assessment for cancer-related VTE, comprised of 3 domains, ten subdomains, and 39 indicators. We used the average random consistency index R.I. to determine the matrix’s consistency at various levels. The results indicated that every matrix’s consistent ratio (C.R.) value < 0.1 meets the consistency test requirements. (Table 4)

Table 4 Weight Value and Score for the Indicators
Full size table

Discussion

In this study, we developed a cancer-related VTE risk assessment tool with Delphi and AHP in China. The extensive study involved consultation with experts to identify specific risk factors relevant to the Chinese population. AHP was used to determine a weighted score for each indicator to establish an effective risk threshold. Our study has successfully addressed a crucial gap in our population, where foreign tools are the only available options. In this study, we invited 23 experts in various fields, including clinical Pharmacy, Pharmaceutical Affairs, Oncology, Surgery, and Internal medicine. Among them, 20 held doctorates or master’s degrees. Our study’s tool demonstrated good reliability, with authority coefficients (Cr) of 0.89 and 0.90 in the two survey rounds. These coefficients indicate that the experts involved in our study had high authority. Generally, a coefficient exceeding 0.70 is considered reliable. Additionally, Kendall’s concordance coefficient W was 0.190 and 0.132 in the two-round survey, respectively, indicating that all experts had a consistent and high opinion [21].

A few strengths of our research are worth noting. First and foremost, we have created a comprehensive scale for evaluating the danger of cancer-associated VTE that factors in a broader range of risk elements, including those that can decrease the likelihood of VTE, not just those that increase it. Previous research has indicated that anticoagulants can reduce the risk of VTE [16]. Furthermore, evidence-based reviews have demonstrated that the risk of VTE in Western populations is more significant than in Asian populations. By combining the weight value of the Asian population and the use of anticoagulants, we calculated scores of 0.0046 and 0.0562 for the two factors, respectively. Consequently, our research assigned a negative score of -1 to the Asian population and − 6 to anticoagulant use, a distinct feature compared to other assessment tools. This is the only tool that uses a negative score after the “IMPEDE and SAVED score.” Our tool offers a more precise VTE risk assessment for cancer patients.

Secondly, we have modified the VTE risk score related to BMI. According to European standards, BMI > 25 kg/m2 is considered overweight, while that of China is > 24 kg/m2. Similarly, BMI > 30 kg/m2 is classified as obese in Europe, while in China, it is 28 kg/m2 or higher. Hence, BMI > 24 kg/m2 and ≥ 28 kg/m2 are used as risk factors for BMI in this study, with weights of 0.0059 and 0.0097, respectively, due to previous research indicating that higher BMIs increase VTE risk. Our study used a novel approach to assign 1 and 2 points to differentiate between the two risk factors, which has distinct advantages over other methods. The reasonable setting of the BMI threshold can more accurately evaluate patients’ VTE scores, increase the specificity of risk assessment tools, and be more suitable for the Chinese population.

Third, one of our unique contributions is the development of a new risk metric called “Plateau(Altitude ≥ 2500 m),” which is not found in other tools. It is a vital part of our thorough risk factor analysis, and we guarantee a sufficient sample size to determine VTE risk levels associated with tumors. This tool is specifically designed for clinical research purposes.

Fourth, open-ended questions were asked to gain deeper insight into the indicator during each round. This helped to define indicators and provide guidance for satisfactory practice, resulting in a more suitable index system for risk assessment tools.

However, the present study has also a few limitations. At first, Experts were only invited from 5 provinces and 12 hospitals in China, with potential experts from other regions not included. Furthermore, face-to-face discussions were not provided to address differing views. Further research is necessary before implementing this tool in practice. Clinical cases will be included to establish VTE risk thresholds and differentiate between high and low risk. Reliability and validity will be verified, and existing tools will be compared.

Conclusion

In summary, this study centers on the characteristics of the Chinese population. We utilized the Delphi-AHP methodology to develop a VTE risk assessment instrument specifically for cancer patients in China. The tool encompasses 39 different factors. After two consultative rounds with Delphi experts, we confirmed the tool’s precision and dependability, with a significant degree of credibility and consistency.

Data availability

All data generated or analysed during this study are included in this published article.

References

  1. Li XY, Fan J, Cheng YQ, et al. Incidence and prevention of venous thromboembolism in acutely ill hospitalized elderly Chinese. Chin Med J (Engl). 2011;124(3):335–40.

    PubMed  Google Scholar 

  2. Khorana AA, Francis CW, Culakova E, et al. Thromboembolism is a leading cause of death in cancer patients receiving outpatient chemotherapy. J Thromb Haemost. 2007;5(3):632–4.

    Article  CAS  PubMed  Google Scholar 

  3. Ay C, Pabinger I, Cohen AT. Cancer-associated venous thromboembolism: Burden, mechanisms, and management. Thromb Haemost. 2017;117(2):219–30.

    Article  PubMed  Google Scholar 

  4. Kearon C, Akl EA, Comerota AJ, et al. Antithrombotic therapy for VTE disease: Antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians evidence-based clinical practice Guidelines. Chest. 2012;141(2 Suppl):e419S–96.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Khorana AA, Kuderer NM, Culakova E, et al. Development and validation of a predictive model for chemotherapy-associated thrombosis. Blood. 2008;111(10):4902–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Ay C, Dunkler D, Marosi C, et al. Prediction of venous thromboembolism in cancer patients. Blood. 2010;116(24):5377–82.

    Article  CAS  PubMed  Google Scholar 

  7. Verso M, Agnelli G, Barni S, et al. A modified Khorana risk assessment score for venous thromboembolism in cancer patients receiving chemotherapy: the Protecht score. Intern Emerg Med. 2012;7(3):291–2.

    Article  PubMed  Google Scholar 

  8. Angelini DE, Greene MT, Wietzke JN et al. A novel risk assessment model to predict venous thromboembolism (VTE) in cancer inpatients: the canclot score. Blood. 2016;128(22).

  9. Antic D, Milic N, Nikolovski S, et al. Development and validation of multivariable predictive model for thromboembolic events in lymphoma patients. Am J Hematol. 2016;91(10):1014–9.

    Article  CAS  PubMed  Google Scholar 

  10. Cella CA, Di Minno G, Carlomagno C, et al. Preventing venous thromboembolism in ambulatory cancer patients: the ONKOTEV study. Oncologist. 2017;22(5):601–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Gerotziafas GT, Taher A, Abdel-Razeq H, et al. A predictive score for thrombosis associated with breast, colorectal, lung, or ovarian cancer: the prospective COMPASS-cancer-associated thrombosis study. Oncologist. 2017;22(10):1222–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Munoz MA, Ortega I, Font C, et al. Multivariable clinical-genetic risk model for predicting venous thromboembolic events in patients with cancer cancer. Br J Cancer. 2018;118(8):1056–61.

    Article  Google Scholar 

  13. Pabinger I, van Es N, Heinze G, et al. A clinical prediction model for cancer-associated venous thromboembolism: a development and validation study in two independent prospective cohorts. Lancet Haematol. 2018;5(7):e289–98.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Rojas-Hernandez CM, Tang VK, Sanchez-Petitto G, et al. Development of a clinical prediction tool for cancer-associated venous thromboembolism (CAT): the MD Anderson Cancer Center CAT model. Support Care Cancer. 2020;28(8):3755–61.

    Article  PubMed  Google Scholar 

  15. Li A, Wu Q, Luo S, et al. Derivation and validation of a risk assessment model for immunomodulatory drug-associated thrombosis among patients with multiple myeloma. J Natl Compr Canc Netw. 2019;17(7):840–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Sanfilippo KM, Luo S, Wang TF, et al. Predicting venous thromboembolism in multiple myeloma: development and validation of the IMPEDE VTE score. Am J Hematol. 2019;94(11):1176–84.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Li Y, Shen L, Ding J, et al. Derivation and validation of a nomogram model for pulmonary thromboembolism in patients undergoing lung cancer surgery. Transl Lung Cancer Res. 2021;10(4):1829–40.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Bastos-Oreiro M, Ortiz J, Pradillo V, et al. Incorporating genetic and clinical data into the prediction of thromboembolism risk in patients with lymphoma. Cancer Med. 2021;10(21):7585–92.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Zhang M, Liu M, Wang D, et al. Development of a risk assessment scale for perinatal venous thromboembolism in Chinese women using a Delphi-AHP approach. BMC Pregnancy Childbirth. 2022;22(1):426.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Jiang F, Liu T, Zhou H, et al. Developing medical record-based, healthcare quality indicators for psychiatric hospitals in China: a modified Delphi-analytic hierarchy process study. Int J Qual Health Care. 2019;31(10):733–40.

    PubMed  Google Scholar 

  21. Jiang Q, Zeng W, Yu J, et al. Development of the first value assessment index system for off-label use of antineoplastic agents in China: a Delphi study. Front Pharmacol. 2020;11:771.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all the experts who participated in our Delphi study sharing for their time and expertise.

Funding

This research was funded by the Youth of the Natural Science Foundation of China (grant number 72204039), the Natural Science Foundation of Sichuan Province (grant number 23NSFSC4722).

Author information

Author notes

  1. Xiaoli Qin and Xiurong Gao contributed equally to this work.

Authors and Affiliations

  1. Department of Pharmacy, The Third People’s Hospital of Chengdu, 610031, Chengdu, Sichuan, P.R. China

    Xiaoli Qin & Yujie Yang

  2. School of Pharmacy, Chengdu Medical College, 610500, Chengdu, Sichuan, P.R. China

    Xiaoli Qin & Xiurong Gao

  3. Department of Pharmacy, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, 610041, Chengdu, Sichuan, P.R. China

    Shunlong Ou & Qian Jiang

  4. Department of Pharmacy, The Second People’s Hospital of Yibin, 644000, Yibin, Sichuan, P.R. China

    Jing Luo

  5. Department of Pharmacy, Chengdu Second People’s Hospital, 610011, Chengdu, Sichuan, P.R. China

    Hua Wei

Authors
  1. Xiaoli Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiurong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shunlong Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hua Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qian Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XL.Q and XR.G: drafting and revising the manuscript critically for important intellectual content. YJ.Y, SL.O, J.L, and H.W: collecting data and evaluation Q.J: Study design, conceiving the study, revising the manuscript. Funding acquisition. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Qian Jiang.

Ethics declarations

Ethics approval and consent to participate

The Ethics Committee of the Third People’s Hospital of Chengdu has approved the study (No. [2023]S-186). And Informed consent was obtained from all subjects and/or their legal guardian(s).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, X., Gao, X., Yang, Y. et al. Developing a risk assessment tool for cancer-related venous thrombosis in China: a modified Delphi-analytic hierarchy process study. BMC Cancer 24, 120 (2024). https://doi.org/10.1186/s12885-024-11877-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12885-024-11877-8

Keywords

BMC Cancer

ISSN: 1471-2407

Contact us