Only brief descriptions of the Programme and the evaluation methods are provided here, further details can be accessed elsewhere [26].
Setting
The Programme was evaluated in four prostate cancer treatment centres in the NHS in England. Three of the centres were involved in an expression of interest process and were selected according to readiness for implementation and to represent providers of urology services in both rural and urban areas. The fourth site had been involved in development and piloting of the Programme and was included in the evaluation to boost recruitment.
Study design
The evaluation was historically controlled, comparing outcomes for men on the Programme with a comparator group of men recruited whilst attending a follow-up care appointment in the period prior to the introduction of the Programme. Outcome was assessed at four and eight months post-baseline. An embedded, within trial cost comparison analysis compared costs between the two groups from a health care provider perspective. A concurrent qualitative evaluation of factors that facilitated or inhibited implementation of the Programme will be reported separately.
Eligibility
Men were eligible to take part in the evaluation if: their mode of primary treatment was radical prostatectomy (RP), radiotherapy (RT) or primary androgen deprivation therapy (PADT); they were assessed as suitable for the Programme by their clinical team, using predefined clinical eligibility criteria (see Additional File 1); they were within three years of completion of RP or RT, or within three years of commencement of PADT. Men were assessed for the programme from six weeks post RP or RT, or three months post commencement of PADT, and were reassessed periodically if they did not meet the criteria at their first assessment. Men with metastatic disease who met the clinical eligibility criteria were included in the Programme.
Comparator
Men in the comparator group were managed according to the follow up care protocol in place at their treatment centre prior to the introduction of the Programme, and this varied by centre and by treatment group. National guidance informing this did not prescribe intervals of follow up, mode of follow up or professional involved. Consequently, each centre worked to a different frequency of follow up appointments (commonly for RP patients, 3 appointments during the first year, then 6 monthly in year 2 and then annually; RT patients commonly, 3 appointments during the first year, then annually; and for HT patients commonly between 3 and 6 monthly appointments), setting of the appointment (often face-to-face but some telephone appointments for some patients; all were followed up in secondary care), professional involved (most often a mix of nurse and consultant care), and overall duration of follow-up (varying by clinician and centre, between 5 and 15 years).
The Programme
The Programme is designed to deliver personalised survivorship care through remote monitoring and supported self-management. The Programme is managed on a day-to-day basis by a healthcare worker, known as a support worker, who acts as the co-ordinator of the patient’s follow-up care and as first point of contact for any problems or queries. A uro-oncology Clinical Nurse Specialist (CNS) oversees this work, with overall patient responsibility remaining with the supervising urologist/oncologist. Men’s suitability for the Programme is determined according to agreed clinical criteria (Additional file 1). Men on the Programme do not have scheduled urology/oncology follow-up appointments; their follow-up care is instead facilitated through a bespoke Patient Online System (called My Medical Record), with access for both men and the prostate cancer team. Recurrence of prostate cancer is tracked through periodic blood samples to detect Prostate Specific Antigen (PSA). For men in the Programme, blood samples are taken at the man’s General Practice or at their hospital phlebotomy service and are transferred to the Online System directly from the pathology laboratory. Results are accessible to the patient through the Online System as soon as they are available, and they can also view other personal information such as treatment summaries and care plans, complete a Holistic Needs Assessment known as a Health MOT, or securely message their clinical team. PSA results and completed Health MOTs are reviewed by the team during ‘virtual clinics’, and contact made with the man if there is any concern. There is a system of rapid re-access to clinic if needed. Systems are in place for men who do not wish to use Information Technology. Men attend a four-hour face-to-face group self-management workshop to prepare them for the Programme. The workshop is run by the CNS and support worker, and aims to educate men about their follow-up care, consequences of treatment, important signs and symptoms of possible recurrence, healthy lifestyles and setting of health and wellbeing goals.
Study integrity
Ethical approval was granted by the National Research Ethics Service, East of England – Cambridge South (reference number 11/EE/1021). Written consent was sought for participation in the study, with separate consent for access to medical records. As an evaluation of a service improvement initiative, trial registration was not required. Reporting of the study follows appropriate guidance [27, 28].
Data collection
Patient reported outcome data were collected by postal questionnaire. Outcomes were measured using validated instruments where available, details of these are given below. Demographic characteristics were also collected. Medical details, including cancer stage, grade, date of diagnosis and treatment received, were collected from hospital records.
Resources related to the provision of the Programme and care of the comparator group were collected from the cancer centre teams, databases and finance managers. For the Programme, the clinical teams provided patient-level data regarding utilisation of follow-up services (virtual clinic appointments, telephone and electronic correspondence and face-to face appointments), workshop attendance, registration with the Patient Online System and details of other prostate cancer related hospital service use. Time related to Programme provision (such as delivery of the workshop, registration of men on the online portal and conduct of the health needs assessment) was reported by staff for the estimation of costs. Information on the follow-up care of the comparator group was extracted from treatment centre databases. Data on wider prostate cancer related health service use (primary care, secondary care, allied health professionals and community-based care) were collected from both groups via the questionnaire at four and eight months.
Outcome measures
Cancer survivors’ unmet needs
The primary outcome was men’s unmet needs, as measured by a modified Cancer Survivors’ Unmet Needs Survey (CaSUN) [29]. The CaSUN [29] comprises 35 items within five domains: existential survivorship (that is, life perspective), comprehensive care, information, quality of life, and relationships, with an additional six items about positive life changes. This study has followed others in using a simplified four point response format [30] of no need, low need, moderate need and high need. The CaSUN can be scored as totals of strength of unmet need (possible scores of 0 to 140) and number of unmet needs (possible scores of 0–35), or for each of the five domains separately. The range of attainable values for each domain subscale are provided within the table of results. Higher scores indicate more need.
Treatment side effects
The Expanded Prostate Cancer Index Composite Short Form [31] (EPIC-26) was used to measure prostate cancer treatment related symptom and bother. The EPIC-26 comprises five subscales of urinary incontinence, urinary irritative/obstructive, bowel, sexual, and hormonal symptoms and bother. Each subscale is scored from 0 to 100, with higher scores indicating better function.
Cancer specific quality of life
Cancer specific quality of life was measured using the Functional Assessment of Cancer Therapy Scale (FACT-G) [32], which comprises physical, emotional, social and functional subscales, to calculate subscale scores and a total score (0 to 108). Higher scores indicate better functioning. Possible subscale scores are provided within the table of results.
Psychological wellbeing
The General Health Questionnaire (GHQ-12) [33] measures current mental health. The measure has two scoring methods: the 0111 giving a maximum total score of 12 and the 0123 method a maximum total score of 36. A higher score indicates worse mental health.
Fear of cancer recurrence
Fear of cancer recurrence was measured using the 2 item Worry of Cancer Scale [34], which is scored from 0 to 20 with a higher score indicating greater worry.
Activation for self-management
Activation to self-manage was measured with the short form Patient Activation Measure (PAM)® [35, 36]. This measure comprises 13 items and results in a total score with a maximum of 100, a higher score indicating greater activation.
General health behaviours
The Fruit and Vegetable Screening Tool [37] was used to measure healthy eating behaviours. This measure asks about fruit and vegetable consumption within a typical day and was scored from 0 to 10, with higher scores being better.
The Godin Leisure Time questionnaire [38] asks about leisure time physical activity as mild, moderate and strenuous activity, plus activity to work up a sweat, as number of units of 15 plus minutes per week. A total score is calculated (0 to 400 with higher being better), which can be used to classify respondents as active, moderately active and insufficiently active.
Service use
Fifteen questions about contact with health and community services for prostate cancer related issues and patient costs related to prostate cancer were developed for a previous evaluation in which two of the current authors were involved [39].
Satisfaction with care
Eleven questions regarding experience and acceptability of follow-up care were also developed for the previous evaluation [39].
Analysis
Programme effectiveness
Descriptive statistics were used to compare baseline clinical and demographic characteristics of the programme and comparator groups, as well as to compare those completing both 4 and eight-month follow-up questionnaires with those lost to attrition. Outcome measure total scales and subscales were computed according to the guidelines for each instrument. When guidelines were not available, a prorated score was calculated when at least 75% of the items for a scale/subscale were present, otherwise the score was set as missing. Means and standard deviations were calculated for all available cases for each outcome measure at each time point, along with the change from baseline. Patient satisfaction data were analysed using Mann-Whitney U tests.
Regression analyses were conducted for outcome measures at both follow-up times separately, controlling for pre-specified variables: cancer centre, each outcome at baseline, age, type of treatment, educational attainment, time since diagnosis, living status, co-morbidity, employment status, and ethnicity.
Outcome comparisons were also obtained from a mixed model including an interaction between group and time so that separate programme comparisons were estimated at four and eight months. Results supported the initial regression analyses, with additional significant differences in seven of the outcomes at the eight month time point (not shown).
Three pre-specified sets of subgroup analysis were also conducted: age with pre-specified dichotomisation at 70 years; none versus one or more comorbidity; and being in the 20% most deprived areas according to the English Index of Multiple Deprivation [40]. Separate regression analysis for the difference in outcome between the programme and comparator groups at each time point, with the same controlling variables as used in the previous analyses, was repeated for each subgroup, and the interaction term between each specified subgroup and the programme versus comparator group factor was examined. Estimates of the difference between programme and comparator for each subgroup were examined when the interaction term was found to be significant (at the 5% level). Only one test of interaction was statistically significant (not shown).
For all analyses, terms were deemed statistically significant at the 5% level.
Economic evaluation
Costs of all items of service use for the programme and the comparator groups were calculated for each patient individually, based on the frequency of service use (Additional file 2) multiplied by the corresponding unit costs sourced from national tariffs [41] or finance managers (Additional file 3). Software and system costs were averaged and applied top down. Mean costs for each item and overall costs were compared between groups. Costs are reported in British pounds, 2015. The analyses were conducted on complete case data, that is respondents who returned service use data at all time points. In cases where some items of service use data were completed but others were missing, simple mean imputation stratified by group and cancer centre was used for the missing items. Health economics analyses were conducted using STATA14 [42] and Microsoft Excel 2013.