Skip to main content

Feasibility, validity and reliability of objective smartphone measurements of physical activity and fitness in patients with cancer

Abstract

Background

A patient’s physical function plays a leading role in the treatment prescription for patients with cancer. Objective assessments of physical function may be more predictive for treatment tolerability and survival than frequently used subjective measures, such as the Eastern Cooperative Oncology Group/World Health Organization (ECOG/WHO) performance score. The use of smartphones to measure physical activity and fitness may provide an excellent opportunity to objectively estimate a patient’s physical function against low costs and little time. We investigated feasibility, validity and reliability of smartphone measurements of step count and physical fitness in patients with cancer.

Methods

In total, 72 patients participated. They wore a smartphone for 14 days to measure the mean number of steps per day, concomitant with an accelerometer during the first 7 days. Patients performed a six-minute walk test (6MWT) twice outdoors via a smartphone application and once in a test environment in the hospital. Feasibility was evaluated by the proportion of patients who completed the study as well as smartphone assessments of step count and physical fitness. Validity was assessed with the intraclass correlation coefficient (ICC) between the accelerometer and the first week of the smartphone for step count, and between the 6MWT in the hospital and via the application for physical fitness. Test-retest reliability was assessed with the ICC between step count levels of the first and second week of smartphone assessments, and between the first and second six-minute walk test in the home environment.

Results

The completeness of smartphone measurements was approximately 90% for step count and 64% for physical fitness assessments. Validity was excellent for step count (ICC = 0.97, p < 0.001) and fair for fitness (ICC = 0.47, p < 0.001). We found excellent test-retest reliability for step count (ICC = 0.91, p < 0.001) and physical fitness (ICC = 0.88, p < 0.001).

Conclusions

This study showed that objective smartphone measurements of step count in clinical practice are feasible, valid and reliable. These findings indicate that the use of smartphones to objectively assess physical activity in clinical cancer practice is promising and may be used to select patients for treatment and study participation, to monitor patients during treatment and to guide treatment decisions.

Peer Review reports

Background

A patient’s physical function plays a leading role in the prescription of treatment for patients with cancer [1, 2]. The physical function of a patient relates to the levels of physical activity and fitness and is in current clinical practice often estimated with the subjective Eastern Cooperative Oncology Group/World Health Organization (ECOG/WHO) performance score [3]. Physical activity is defined as any bodily movement caused by contraction of skeletal muscles resulting in expenditure of energy [4] and physical fitness is defined as a set of health and skill-related attributes that people have, of which cardiorespiratory fitness is one of the main components [4]. Step count, defined as the number of steps per day, is a measure of physical activity, which can easily and objectively be assessed and implemented on a large scale, with low-cost devices (e.g. pedometers) [5]. Maximum oxygen uptake assessed during a maximal exercise test is the gold standard for assessing cardiorespiratory fitness [6]. Unfortunately, the required specialized equipment, well-trained personnel and high costs hamper the implementation on a large scale in clinical practice. A 6 min walk-test (6MWT), which assesses the distance that a person can walk in 6 min, is considered a valid and reliable measurement of cardiorespiratory fitness [7]. However, the conduct of a 6MWT requires qualified personnel and an appropriate location inside the hospital, which are both too time consuming and costly in clinical practice [6].

Currently, many people own smartphones which are equipped with advanced technologies (e.g. gyroscope) [8], providing the opportunity for patients to objectively assess their levels of physical activity (step counts) and fitness in their home environment with little time investment and at low costs [9, 10]. In a previous systematic review, it was reported that smartphones have average-to-excellent measurement accuracy in healthy volunteers [11], but were less accurate at slower walking speeds [8]. Objective measurements of physical activity and fitness might improve the estimation of a patient’s physical function [3] and may be useful to optimize treatment selection or to monitor physical activity and fitness levels during and following treatment. Before implementation of objective smartphone measurements in clinical practice, knowledge of feasibility, validity and reliability in patients with cancer, who have lower physical activity levels and reduced physical fitness [6, 12,13,14,15], is required.

This study aimed to examine the feasibility, validity and reliability of smartphone measurements of step count and physical fitness in patients with cancer. We hypothesize that the smartphone measurements are feasible, valid and reliable in patients with cancer.

Methods

Study design

Patients were recruited from the outpatient Medical Oncology department of Amsterdam University Medical Centers (Amsterdam UMC), location VUmc, and approached consecutively by a researcher (JD). Patients were eligible if they (i) were 18 years or older, (ii) were diagnosed with advanced cancer or received adjuvant (chemo)therapy for localized cancer, and (iii) had a stable performance score. The performance score is a widely used method to assess the functional status of patients with cancer and ranges from 0 (fully active) to 5 (dead), in which 1 reflects a patient who is “restricted in physically strenuous activity but ambulatory and able to carry out work of a light or sedentary nature” and 2 meaning that the patient is “ambulatory and capable of all self-care but unable to carry out any work activities; up and about more than 50% of waking hours” [16]. To exceed the recommended number of 50 patients for validation studies [17], we aimed to include 70 patients. Because earlier studies suggest a low validity of smartphone measurements at slow walking speeds, we aimed to achieve an equal distribution of patients with a different performance score. Consequently, patient enrollment in a specific performance score subgroup stopped after inclusion of 25 patients.

Measurements

The performance score of the patient had to be scored by the treating medical oncologist before inclusion and needed to be stable, which was defined as the same performance score for two consecutive consultation visits.

Usability and user-friendliness was assessed with the system usability scale (SUS), a 10-item questionnaire designed and validated to assess usability of electronical systems [18]. The questionnaire yields a total score ranging from 0 to 100, for which a ≥ 70 is considered good usability [18].

Physical activity was defined as the mean number of steps per day and was assessed with both an accelerometer (Actigraph wGT3X) and a smartphone (IPhone SE, iOS 10.2). Patients were instructed to wear the smartphone for 14 consecutive days in the hip-waist region, either in a pocket or attached to a belt, during all waking hours, concomitant with a waist-worn accelerometer during the first 7 days.

To calculate the mean daily number of steps per week, at least 4 valid days per week of wearing-time were needed [19]. For the accelerometer, a valid day of wearing-time was defined as 8 h and non-wearing time was defined as 60 min of consecutive zero counts [19]. Raw accelerometer data were processed using ActiLife Software version 6.13.2 (ActiGraph, Pensacola, Florida, USA). Due to the inability to perform a comprehensive analysis of wearing time for the smartphone measurements, we considered every day the smartphone had recorded any steps as a valid wear day. Physical fitness was assessed with the six-minute walk test (6MWT), measuring the maximum distance walked in 6 min [20]. Patients were instructed to perform a 6WMT twice outdoors in their home environment using a smartphone application (Walkmeter), which used the Global Positioning System (GPS)-signal to assess distance. During the same week, a 6MWT was performed under standardized conditions in the hospital [20].

Age, gender, height, weight, zip code, performance score, tumor type, treatment type and treatment intention were retrieved from the medical records. BMI was calculated based on the objective measurements of height and weight (body weight/height2, kg/m2). Socio-economic status (SES) was determined using zip codes of the patients’ living area [21]. Zip codes were translated to SES according to The Netherlands Institute for Social Research. This system describes the social status of a district compared to other districts in The Netherlands using an algorithm based on mean income, percentage of people with low income, percentage of people with low education, and percentage of people with without a job. Therefore, the mean score of all districts in The Netherlands is zero.

Statistical analysis

Differences in sociodemographic and clinical characteristics between the participants and non-participants were investigated with univariable logistic regression analyses.

Feasibility was evaluated by the proportion of patients who (i) completed the study, (ii) had ≥4 valid wear days with the smartphone and (iii) completed the 6MWT via the application at least once. Criterion validity of smartphone physical activity assessments was determined by calculating the intraclass correlation coefficient for agreement (ICC) and 95% confidence intervals (CI) between the mean number of steps per day assessed with the smartphone and the accelerometer as reference measure. For physical fitness, we calculated the ICC between the 6MWT performed with the smartphone and the 6MWT performed in the hospital. Bland-Altman plots were used to visualize systematic differences and 95% limits of agreement. Potential proportional bias was quantified with a linear regression analysis between the difference and the mean of both measurements (accelerometer and smartphone). Test-retest reliability of the smartphone’s physical activity and fitness measurements was determined by calculating the ICC, between the mean number of steps during the first and second week and between the first and second 6MWT. An ICC ≥ 0.75 was considered excellent [17, 22]. Standard error of measurement (SEM) and smallest detectable difference at a 95% confidence interval (SDD95) were calculated [17]. Sensitivity analyses were conducted to examine validity and reliability separately per performance score.

Results

Mainly due to toxicity and/or complications of treatment and progressive disease, 17 of 89 eligible patients (19%) dropped out (Fig. 1). Baseline characteristics of participants and non-participants are presented in Table 1. The proportion of men was 63% amongst the participants and 45% amongst the non-participants (p = 0.04). No other statistically significant differences in sociodemographic and clinical factors between the participants and non-participants were found. The proportion of patients with ≥4 valid wear days with the smartphone was 90% in the first and 88% in the second week. At least one 6MWT was performed via the application by 64% of patients. Of the 25 patients who did not successfully complete a 6MWT, 15 did not report a specific reason for the missing data, 4 patients forgot to perform a 6MWT, and 6 patients performed a 6MWT but either encountered a technical problem of the application (n = 3), had a poor GPS-signal (n = 2), or accidently erased the results (n = 1). The mean (SD) score of the SUS was 69 (17).

Fig. 1
figure 1

Flowchart of patient inclusion and feasibility of assessments. Legend: ECOG/WHO PS = ECOG/WHO performance score, 6MWT = six-minute walk test

Table 1 Baseline characteristics of participants

The smartphone’s validity was excellent for step count (ICC = 0.97, 95% CI = 0.95–0.98, p < 0.001) and fair for fitness (ICC = 0.47, 95% CI = 0.21–0.67, p = 0.001) (Table 2). For step count, there were no signs of systematic differences between the measurement with smartphone and accelerometer (Fig. 2). A larger difference between the 6MWT in the hospital and the 6MWT assessed via the application was found in patients with longer walking distances, in favor of the latter (regression coefficient = 0.62, 95% CI = 0.30;0.94, p < 0.001) (Fig. 2).

Table 2 Construct validity and test-retest reliability for smartphone measurements of physical activity and fitness
Fig. 2
figure 2

Bland-Altman plots for physical activity and fitness. Legend: β = regression coefficient, LOA = 95% limits of agreement.

Test-retest reliability was excellent both for step count (ICC = 0.91, 95% CI = 0.85–0.94, p < 0.001) and for physical fitness (ICC = 0.88, 95% CI = 0.74–0.94, p < 0.001) (Table 2). SEM and SDD95 were 833 and 2309 respectively for step count and 70 and 193 for physical fitness (Table 2). Results of the sensitivity analyses across different performance scores were comparable.

Discussion

The results of this study showed excellent test-retest reliability of smartphone measurements of step count and physical fitness. Feasibility and validity for step count measurements were excellent, but in its present form, they were fair for physical fitness. To the best of our knowledge, this is the first study to investigate feasibility, validity and reliability of smartphone measurements in patients with cancer.

The high validity of smartphone measurements of step count is in line with studies that investigated activity trackers [11, 23] and smartphones [8] in healthy populations. In contrast to results for activity trackers that showed strong dependency on walking speed [23], our sensitivity analyses indicated that results were consistent across different performance scores. The excellent test-retest reliability of smartphone assessments of step count and physical fitness has not been described previously. However, our results indicated that a minimal difference of 2300 steps for step count and 190 m for physical fitness between two measurements is required to detect a real change over time with 95% certainty. It is yet unclear whether this yields sufficient responsiveness to change in clinical settings.

For physical fitness, fair validity and broad limits of agreement indicate that implementation of the 6MWT smartphone application in clinical practice in its present form may be limited. Possible reasons are discouraging user-friendliness of the application and problems with the GPS-signal. Despite relatively low completeness of measurements of physical fitness with the current application and the fair usability of the application, we believe that physical fitness assessments with a customized application might be promising for use in clinical practice.

Strengths of this study are the sufficient sample size and the inclusion of patients with various performance scores, which allows us to draw conclusions about feasibility, validity and reliability of smartphone measurements in patients with cancer with both good and poor physical function. However, men were more likely to participate in the study than women, which may be related to a higher interest in technical gadgets [24]. A limitation of this study is that step count was used as a measure of physical activity, which could lead to an underestimation of total physical activities as it is unable to measure activities such as cycling and swimming [25]. However, both the accelerometer and smartphone assessments of step counts are prone to this underestimation and it is therefore unlikely that it has affected the validity and reliability estimates of step counts. Another limitation is the relatively high number of patients who did not complete a 6MWT via the application. This may limit the implementation of smartphone measurements in clinical practice, but is less likely to affect the results on validity and reliability of smartphone measurements. Our results indicate that technical improvements of the application (e.g. reminders, GPS-processing) could further improve data collection.

Excellent validity and reliability of smartphone measurements of step count support implementation in clinical practice. Furthermore, the results of this study provide evidence that smartphone measurements are feasible and well tolerated in patients with cancer in clinical practice. However, it remains unclear whether objective assessments of physical activity and fitness with smartphones has added value in clinical practice. Therefore, we have initiated a prospective study, aiming to investigate whether smartphone measurements of step count and physical fitness are predictive for trial feasibility in patients with cancer participating in phase I-II clinical trials (NCT03493672). Additionally, it must be verified in future studies whether objective assessment of physical activity and fitness in clinical practice may support timely referral to exercise or rehabilitation interventions.

Conclusion

In conclusion, the results of this study show that smartphones provide feasible, valid and reliable objective assessments of step count in patients with cancer. For physical fitness, reliability of smartphone measurements was excellent, but in its present form its feasibility and validity was fair. The use of smartphones to objectively assess physical activity and fitness in clinical (cancer) practice is promising.

References

  1. Eichler M, Singer S, Janni W, Harbeck N, Rack B, Augustin D, Wischnik A, Kiechle M, Ettl J, Scholz C, et al. Pretreatment quality of life, performance status and their relation to treatment discontinuation and treatment changes in high-risk breast cancer patients receiving chemotherapy: results from the prospective randomized ADEBAR trial. Breast Cancer. 2017;24(2):319–25.

    Article  PubMed  Google Scholar 

  2. Yates JW, Chalmer B, Mckegney FP. Evaluation of patients with advanced Cancer using the Karnofsky performance status. Cancer. 1980;45(8):2220–4.

    Article  CAS  PubMed  Google Scholar 

  3. Sonpavde G, Vogelzang NJ, Galsky MD, Raghavan VA, Daniel S. Objective measures of physical functional capacity warrant exploration to complement or replace the subjective physician estimated performance status. Am J Clin Oncol. 2012;35(2):163–6.

    Article  PubMed  Google Scholar 

  4. Caspersen CJ, Powell KE, Christenson GM. Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep. 1985;100(2):126–31.

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Purswani JM, Ohri N, Champ C. Tracking steps in oncology: the time is now. Cancer Manag Res. 2018;10:2439–47.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Steins Bisschop CN, Velthuis MJ, Wittink H, Kuiper K, Takken T, van der Meulen WJ, Lindeman E, Peeters PH, May AM. Cardiopulmonary exercise testing in cancer rehabilitation: a systematic review. Sports Med. 2012;42(5):367–79.

    Article  PubMed  Google Scholar 

  7. A.T.S.Committee. ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med. 2002;166(1):111–7.

    Article  Google Scholar 

  8. Major MJ, Alford M. Validity of the iPhone M7 motion co-processor as a pedometer for able-bodied ambulation. J Sports Sci. 2016:1–5.

  9. Majmudar MD, Colucci LA, Landman AB. The quantified patient of the future: opportunities and challenges. Healthc (Amst). 2015;3(3):153–6.

    Article  Google Scholar 

  10. del Rosario MB, Redmond SJ, Lovell NH. Tracking the evolution of smartphone sensing for monitoring human movement. Sensors (Basel). 2015;15(8):18901–33.

    Article  Google Scholar 

  11. Bort-Roig J, Gilson ND, Puig-Ribera A, Contreras RS, Trost SG. Measuring and influencing physical activity with smartphone technology: a systematic review. Sports Med. 2014;44(5):671–86.

    Article  PubMed  Google Scholar 

  12. Forbes CC, Blanchard CM, Mummery WK, Courneya K. Prevalence and correlates of strength exercise among breast, prostate, and colorectal cancer survivors. Oncol Nurs Forum. 2015;42(2):118–27.

    Article  PubMed  Google Scholar 

  13. Je Y, Jeon JY, Giovannucci EL, Meyerhardt JA. Association between physical activity and mortality in colorectal cancer: a meta-analysis of prospective cohort studies. Int J Cancer. 2013;133(8):1905–13.

    Article  CAS  PubMed  Google Scholar 

  14. Broderick JM, Ryan J, O'Donnell DM, Hussey J. A guide to assessing physical activity using accelerometry in cancer patients. Support Care Cancer. 2014;22(4):1121–30.

    Article  CAS  PubMed  Google Scholar 

  15. Kampshoff CS, Chinapaw MJ, Brug J, Twisk JW, Schep G, Nijziel MR, van Mechelen W, Buffart LM. Randomized controlled trial of the effects of high intensity and low-to-moderate intensity exercise on physical fitness and fatigue in cancer survivors: results of the resistance and endurance exercise after ChemoTherapy (REACT) study. BMC Med. 2015;13:275.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, Carbone PP. Toxicity and response criteria of the eastern cooperative oncology group. Am J Clin Oncol. 1982;5(6):649–55.

    Article  CAS  PubMed  Google Scholar 

  17. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.

    Article  PubMed  Google Scholar 

  18. Brooke J. SUS-A quick and dirty usability scale. Usability evaluation in industry. 1996;189(194):4–7.

    Google Scholar 

  19. Oliver M, Badland HM, Schofield GM, Shepherd J. Identification of accelerometer nonwear time and sedentary behavior. Res Q Exerc Sport. 2011;82(4):779–83.

    Article  PubMed  Google Scholar 

  20. Strath SJ, Kaminsky LA, Ainsworth BE, Ekelund U, Freedson PS, Gary RA, Richardson CR, Smith DT, Swartz AM, American Heart Association physical activity Committee of the Council on L, et al. Guide to the assessment of physical activity: clinical and research applications: a scientific statement from the American Heart Association. Circulation. 2013;128(20):2259–79.

    Article  PubMed  Google Scholar 

  21. The Netherlands Institute for Social Research; Statusscores. https://www.scp.nl/Onderzoek/Lopend_onderzoek/A_Z_alle_lopende_onderzoeken/Statusscores. Accessed 4 Sept 2018.

  22. Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. Hoboken: John Wiley & Sons; 2013.

  23. Fokkema T, Kooiman TJ, Krijnen WP, CP VDS, DEG M. Reliability and validity of ten consumer activity trackers depend on walking speed. Med Sci Sports Exerc. 2017;49(4):793–800.

    Article  PubMed  Google Scholar 

  24. Ormel HL, van der Schoot GGF, Westerink NL, Sluiter WJ, Gietema JA, Walenkamp AME. Self-monitoring physical activity with a smartphone application in cancer patients: a randomized feasibility study (SMART-trial). Support Care Cancer. 2018;26(11):3915–23.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Blaauwbroek R, Bouma MJ, Tuinier W, Groenier KH, de Greef MH, Meyboom-de Jong B, Kamps WA, Postma A. The effect of exercise counselling with feedback from a pedometer on fatigue in adult survivors of childhood cancer: a pilot study. Support Care Cancer. 2009;17(8):1041–8.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

Not applicable

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

JD, LB and HV have made substantial contributions to conception and design. JD managed the acquisition of data, by inclusion and measurements of patients. JD and LB performed analysis of data. JD, LB and HV have made substantial contributions to the interpretation of data. JD, LB and HV have been involved in drafting the manuscript or revising it critically for important intellectual content. JD, LB and HV have given final approval of the version to be published. JD, LB and HV have agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Laurien M. Buffart.

Ethics declarations

Competing interest

The authors declare that they have no competing interests.

Ethics approval and consent to participate

The research protocol was reviewed and approved by the Medical Ethical Committee of the VU University Medical Center. All patients provided written informed consent prior to participation.

Consent for publication

Not applicable

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Douma, J.A.J., Verheul, H.M.W. & Buffart, L.M. Feasibility, validity and reliability of objective smartphone measurements of physical activity and fitness in patients with cancer. BMC Cancer 18, 1052 (2018). https://doi.org/10.1186/s12885-018-4983-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12885-018-4983-4

Keywords