- Research article
- Open Access
- Open Peer Review
Accuracy of the Delirium Observational Screening Scale (DOS) as a screening tool for delirium in patients with advanced cancer
BMC Cancer volume 19, Article number: 160 (2019)
The Delirium Observation Screening Scale (DOS) was developed to facilitate early recognition of delirium by nurses during routine clinical care. It has shown good validity in a variety of patient populations, but has not yet been validated in hospitalized patients with advanced cancer, although the DOS is commonly used in this setting in daily practice. The aim of this study was to evaluate the accuracy of the DOS in hospitalized patients with advanced cancer using the revised version of the Delirium Rating Scale (DRS-R− 98) as the gold standard.
Patients with advanced cancer admitted to the medical oncology ward were screened for delirium with the DOS and DRS-R−98. Sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV) of the DOS were calculated, using a DOS score ≥ 3 as a cut-off for delirium.
Ninety-five DOS negative and 98 DOS positive patients were identified. Sensitivity of the DOS, was > 99.9% (95%-CI, 95.8–100.0%), specificity was 99.5% (95%-CI 95.5–99.96%), PPV was 94.6% (95% CI 88.0–97.7), and NPV was > 99.9% (95% CI 96.1–100.0).
The DOS is an accurate screening tool for delirium in patients with advanced cancer. Since it has the benefit of being easily implicated in daily practice, we recommend to educate caregivers to screen patients with advanced cancer by DOS analysis. By early recognition and adequate treatment of this distressing delirium syndrome the quality of life of patients with advanced cancer can be improved.
Delirium is the most common neuropsychiatric complication in patients with advanced cancer especially during hospitalization, with incidence rates ranging from 16 to 85%, depending on the stage of disease [1,2,3,4,5,6,7]. In a previous study from our group we found a higher number of patients with skin cancer and brain cancer in the group of patients with delirium compared to the patients without delirium . Other studies showed conflicting results [9, 10]. Therefore, there is no compelling evidence that delirium is more prevalent in certain cancer types. Because attention and awareness deficits impede the ability to communicate and participate in treatment decisions and symptom assessment, delirium has a negative influence on quality of life in a crucial phase at the end of life . The presentation of delirium is quite variable among patients, and even within a given patient because of its waxing and waning course . This hampers recognition and adequate treatment of delirium . Therefore it is recommended to screen for delirium in patients with (advanced) cancer admitted to the hospital [1, 11]. The diagnosis delirium should be made according to the DSM-criteria for delirium. Currently, version 5 is the most recent, but most screening and diagnostic instruments are based on the DSM-IV . Efforts are being made to validate these instruments with the DSM 5 criteria [13, 14].
Available screening instruments which have been designed to be used by health care professionals for evaluating patients for possible delirium symptoms include: CAM , NEECHAM Confusion Scale , DOSS/DOS [17, 18], Nu-DESC , ICDSC , and PAED scale . Some of these instruments have been designed to be used in a specific treatment setting like the ICU, whereas others focus on specific age groups, like children and adolescents. There is no specific screening instrument for delirium in patients with advanced cancer. A comparison between various features of the available screening and diagnostic instruments for delirium in adults was made by Grover and Kate in 2012 .
Delirium screening should be preferably performed by nurses because they have frequent contact with the patient throughout the day, and could therefore easily observe changes in the patient’s attention and awareness over time, which is one of the main criteria for delirium according to the DSM 5 criteria . The Delirium Observation Scale (DOS) appears to be the most suitable nurse-rated screening instrument for patients in general medical and surgical wards with a strong foundation in the DSM-IV criteria and good psychometric properties [3, 22]. It can be assessed by nurses without specific training, and is experienced as user-friendly. A previous, small study by Detroyer et al suggests good sensitivity and specificity of the DOS in a palliative care population . The aim of our study is to evaluate the diagnostic accuracy of the DOS as screening instrument for hospitalized patients diagnosed with advanced cancer.
Hospitalized patients with advanced cancer admitted to the medical oncology ward of six sites (1 university cancer center, 3 teaching hospitals, 2 high-care hospices) in the Netherlands were recruited between January 2011 through December 2015. Patients and/or their legal representatives were asked for informed consent to participate in this diagnostic study, as part of a randomized controlled trial (RCT), which compared the efficacy of haloperidol to olanzapine in case the patient was diagnosed with delirium (ClinicalTrials.gov identifier NCT01539733). The study was conducted according to Good Clinical Practice guidelines, the Declaration of Helsinki and local laws, and was approved by the institutional review boards of each participating study site.
Patients with any type of cancer in an advanced stage, of 18 years and older who were fluent in the Dutch language were considered eligible. Patients with pre-existing cognitive impairment (such as Alzheimer’s disease), or psychiatric comorbidity that might hamper delirium diagnosis (e.g. schizophrenia) were excluded from this study. Also, patients using antipsychotic or neuroleptic medication for other reasons than neuropathic pain management were excluded. Additional exclusion criteria were based on contra-indications for the use of haloperidol or olanzapine, like high risk at alcohol withdrawal delirium, glaucoma, Parkinson’s disease, QTc-interval prolongation > 500 msec at baseline ECG, a history of malignant neuroleptic syndrome and concomitant treatment with anti-convulsive drugs.
Patients who were included in the trial were screened for delirium by their nurse using the DOS at the moment of admittance, and subsequently three times a day, biweekly during their stay in the hospital. Each DOS positive patient (DOS ≥3) was randomly matched with a DOS negative patient (DOS < 3) to evaluate the accuracy of the DOS. DOS positive patients and the randomly matched DOS negative patients were assessed with the Delirium Rating Scale-R− 98 (DRS-R-98) by a trained independent assessor, who was blinded with regard to the DOS score of a patient.
To prevent duplication bias patients were excluded from the DOS negative group if they were included in the randomised treatment part of the study during a later admission.
The original version, the Delirium Observational Screening Scale (DOSS), consisted of a 25-item scale based on the DSM-IV criteria for delirium . The scale was designed to capture early symptoms of delirium that nurses could observe during regular care. The scale was subsequently reduced to 13 observations, and is known as the Delirium Observation Scale (DOS) . Each item can be rated as normal (score, 0) or abnormal (score, 1). A total score of 3 or more points indicates delirium. Completion of the instrument requires less than 5 min. A small descriptive study evaluated the DOS in patients admitted to a palliative care unit . The DOS was compared to the CAM and the Delirium Index and showed good psychometric properties. Moreover, it was experienced as user-friendly by the bedside nurses. Internal consistency, predictive validity, and concurrent and construct validity of the DOS were tested in two prospective studies in high risk groups: geriatric medicine patients and elderly hip fracture patients . The DOS has high internal consistency (0.96) and high content validity (α = 0.93). In these study groups the DOS scale had a sensitivity of 89–100% and a specificity of 68–88%. The positive predictive value was 47%, the negative predictive value was almost 100%. The DOS was also able to measure severity of delirium in geriatric patients . Moreover, the DOS proved to be a good instrument to facilitate early recognition of delirium in patients who undergo cardiac surgery: the sensitivity and specificity of the DOS was 100 and 96.6% respectively .
The DRS-R-98 is a revised version of the Delirium Rating Scale [26, 27]. The DRS-R-98 consists of 13 severity items that are scored from 0 (not present) to 3 points (severely present), and three diagnostic items, all of which are rated over the past 24 h. Severity scores range from 0 to 39, and total scores range from 0 to 46. The DRS-R-98 is designed to be completed by a trained professional and takes about 10 to 15 min to complete . The DRS-R-98 has a high internal consistency (0.90) and when using a cut-off of 17.75 points on the total scale a sensitivity of 92% and specificity of 95% . Inter-rater-consistency was high in the validation study by Trzepacz et al (0.99) . The DRS-R-98 severity scale has the great benefit that it can be used for repeated measurements to assess the response to delirium treatment . Recently, the DRS-R-98 has been validated for the new DSM-5 criteria . The DRS-R-98 was chosen as a gold standard to evaluate the accuracy of the DOS because of its good psychometric qualities, because it has been validated in a palliative care setting, and is available in the Dutch language (the first language of the included patients and the researchers in this study group) .
A sample size calculation was conducted to determine an 80% power in demonstrating that the DOS has a sensitivity of at least 90%. A sample of 100 patients per group (DOS positive vs. DOS negative) was needed when the sensitivity of the DOS was assumed to be 95% [18, 25].
Primary endpoints to assess the accuracy of the DOS are the sensitivity and specificity of the DOS score compared with the DRS-R-98 as the gold standard. Sensitivity, specificity, negative and positive predictive value are reported with 95% confidence intervals, calculated with the Wilson method . Analyses will be corrected for partial verification, because only a proportion of DOS negative patients admitted to the study sites were included in this study. For the baseline characteristics standard descriptive statistics were used. Statistical analyses were performed with IBM SPSS version 22.0 (IBM, Armonk, NY, USA).
Between January 2010 and January 2016, 100 consecutive DOS negative and 95 DOS positive patients were included in the study. One DOS positive patient withdrew consent for participating in the study while recovering from delirium. Four DOS negative patients were included in the DOS positive group during a later hospital admission. These patients were excluded from the DOS negative group during the analyses. For one patient in each group all data were missing, leading to a total of 94 DOS negative and 93 DOS positive patients (Fig. 1). The demographic and clinical characteristics are described in Table 1. The mean (SD) age was 68 (11) in the DOS positive group and 60 (13) in the DOS negative group (p < 0.001). Other characteristics obtained at baseline were not significantly different between the groups. None of the patients in the DOS negative group were diagnosed with delirium. Eighty-eight of the patients in the DOS positive group were diagnosed with delirium, see Table 2. The sensitivity and specificity of the DOS were > 99.9% (95%CI: 95.8–100.0%) and 99.6.% (95%CI: 95.5–100.0%) respectively, the negative predictive value (NPV) was 94.6 (95%CI, 88.0–97.7%), and the positive predictive value (PPV) was > 99.9% (95%CI, 96.1–100.0%), see Table 3.
The median DOS score of the DOS positive patients was 4.0 (IQR 4.0–6.0). Higher DOS scores correlated with higher DRS-R-98 total scores, see Table 4.
Four of the five patients with a DOS of 3 or higher who were not confirmed to be delirious by the DRS-R-98 score (score < 17.75), had DRS-R-98 total scores > 12 points. The latter cut-off has been used as a more inclusive cut-off for delirium in some studies [30, 31]. One patient scored being delirious on the DOS but only scored three points on the DRS-R-98. We speculate that he/she may have displayed only temporarily signs of delirium which had resolved by the time the DRS-R-98 was conducted.
To our knowledge this is the first study to evaluate the accuracy of the DOS in a large group of patients with advanced cancer. Our data showed that the DOS is a very sensitive and specific instrument to screen for delirium in hospitalized patients with advanced cancer. It might also give an impression of the delirium severity.
Compared to other screening instruments, such as the CAM , NEECHAM Confusion Scale , and the Nu-DESC , the DOS has shown better sensitivity and specificity. It has the benefits that it is quick to administer, and does not require training .
Strengths of the study are that we were able to include a large group of patients, and that the DRS-R-98 assessment was performed by an independent assessor who was blinded to the DOS score of the patient. Also, the nursing staff was already used to complete the DOS scale during routine care, so we did not need a run-in or training period for the study. In addition, during the conduct of the study the nurses gave very positive spontaneous feedback on the effort it took to complete this questionnaire.
There are several limitations to this study. First, the observed incidence of delirium at the study sites was low . Therefore, it was necessary to adjust the results for non-verification. Even after this adjustment, we have convincing results that the DOS is an accurate screening instrument for delirium screening in patients with advanced cancer. Second, while other baseline characteristics were evenly distributed over the DOS positive and DOS negative patients, patients who were DOS negative were younger than the DOS positive patients. This is concordant with the fact that advanced age is one of the known risk factors for the development of delirium . Using age-matched comparators would however hamper the applicability of this study to the whole population of hospitalized patients with advanced cancer.
For this study we used a DRS-R-98 score of > 17.75 as a cut-off for delirium. The patients who screened positive for delirium with the DOS scale but not with the DRS-R-98, had (except for one) DRS-R-98 scores > 12, which has been classified as subsyndromal delirium in previous studies [30, 31]. One might argue that it might be beneficial for these patients as well to be treated for their symptoms.
The DOS is an accurate instrument for detection of delirium in a population of hospitalized patients with advanced cancer. The DOS can be completed by nurses based on the observations made during regular nursing care, which makes it easily applicable as a screening tool for delirium in patients with advanced cancer. By early recognition and adequate treatment of this distressing delirium syndrome the quality of life of patients with advanced cancer can be improved.
95% confidence interval
Confusion assessment method
- DOS :
Delirium Observation Scale
- DOSS :
Delirium observation screening scale
Diagnostic statistical manual
Electro Cardio Graphy
Intensive care delirium screening checklist
Intensive care unit
- IQR :
Inter quartile range
Neelon Champagne Confusion Scale
- NPV :
Negative predictive value
Nursing Delirium Screening Scale
- PAED :
Pediatric anesthesia emergence delirium scale
Positive predictive value
Randomized controlled trial
Standard deviation from the mean
Lawlor PG, Gagnon B, Mancini IL, Pereira JL, Hanson J, Suarez-Almazor ME, et al. Occurrence, causes, and outcome of delirium in patients with advanced cancer: a prospective study. Arch Intern Med. 2000;160:786–94.
Neufeld KJ, Thomas C. Delirium. Definition, epidemiology, and diagnosis. J Clin Neurophysiol. 2013;30:438–42.
Inouye SK, Charpentier PA. Precipitating factors for delirium in hospitalized elderly persons: predictive model and interrelationship with baseline vulnerability. JAMA. 1996;275:852–7.
Francis J, Martin D, Kapoor W. A prospective study of delirium in hospitalized elderly. JAMA. 1990;263:1097–101.
Gaudreau JD, Gagnon P, Harel F, Tremblay A, Roy MA. Fast, systematic and continuous delirium assessment in hospitalized patients: the nursing delirium screening scale. J Pain Symptom Manag. 2005;29:368–75.
Inouye SK, Viscoli CM, Horwitz RI, Hurst LD, Tinetti ME. A predictive model for delirium in hospitalized elderly medical patients based on admission characteristics. Ann Intern Med. 1993;119:474–81.
Lawlor PG, Gagnon B, Mancini IL, Pereira JL, Hanson J, Suarez-Almazor ME, et al. Occurrence, causes, and outcome of delirium in patients with advanced Cancer. Arch Intern Med. 2000;160:786–94.
Neefjes EC, van der Vorst MJ, Verdegaal BA, Beekman AT, Berkhof J, Verheul HM. Identification of patients with cancer with a high risk to develop delirium. Cancer Med. 2017;6:1861–70.
Mercadante S, Masedu F, Balzani I, de Giovanni D, Montanari L, Pittureri C, et al. Prevalence of delirium in advanced cancer patients in home care and hospice and outcomes after 1 week of palliative care. Support Care Cancer. 2018;26:913–9.
Grandahl MG, Nielsen SE, Koerner EA, Schultz HH, Arnfred SM. Prevalence of delirium among patients at a cancer ward: clinical risk factors and prediction by bedside cognitive tests. Nord J Psychiatry. 2016;70:413–7.
Practice guideline for the treatment of patients with delirium: American Psychiatric Association. Am J Psychiatry. 1999;156(5 Suppl):1-20.
Grover S, Kate N. Assessment scales for delirium: a review. World J Psychiatr. 2012;2:58–70.
Meagher DJ, Morandi A, Inouye SK, Ely W, Adamis D, Maclullich AJ, et al. Concordance between DSM-IV and DSM-5 criteria for delirium diagnosis in a pooled database of 768 prospectively evaluated patients using the delirium rating scale-revised-98. BMC Med. 2014;12:164–73.
Adamis D, Rooney S, Meagher D, Mulligan O, McCarthy G. A comparison of delirium diagnosis in elderly medical inpatients using the CAM, DRS-R98, DSM-IV and DSM-5 criteria. Int Psychogeriatr. 2015;27:883–9.
Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113:941–8.
Neelon VJ, Champagne MT, Carlson JR, Funk SG. The NEECHAM confusion scale: construction, validation, and clinical testing. Nurs Res. 1996;45:324–30.
Schuurmans MJ, Donders RT, Shortridge-Baggett LM, Duursma SA. Delirium case finding: pilot testing of a new screening scale for nurses. J Am Geriatr Soc. 2002;50(suppl A8):S3.
Schuurmans MJ, Shortridge-Baggett LM, Duursma SA. The delirium observation screening scale: a screening instrument for delirium. Res Theory Nurs Pract. 2003;17:31–50.
Bergeron N, Dubois JM, Dumont M, Dial S, Skrobic Y. Intensive care delirium screening checklist: evaluation of a new screening tool. Intensive Care Med. 2001;27:859–64.
Sickich N, Lerman J. Development and psychometric evaluation of the pediatric anesthesia emergence delirium scale. Anesthesiology. 2004;100:1138–45.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders, Fifth Edition. Arlington: American Psychiatric Association; 2013.
Braiteh F, el Osta B, Palmer JL, Reddy SK, Bruera E. Characteristics, findings, and outcomes of palliative care inpatient consultations at a comprehensive cancer center. J Palliat Med. 2007;10:948–55.
Detroyer E, Clement PM, Baeten N, Pennemans M, Decruyenaere M, Vandenberghe J, et al. Detection of delirium in palliative care unit patients: a prospective descriptive study of the delirium observation screening scale administered by bedside nurses. Palliat Med. 2014;28:79–86.
Scheffer AC, van Munster BC, Schuurmans MJ, de Rooij SE. Assessing severity of delirium by the delirium observation screening scale. Int J Geriatr Psychiatry. 2011;26:284–91.
Koster S, Hensens AG, Oosterveld FG, Wijma A, van der Palen J. The delirium observation screening scale recognizes delirium early after cardiac surgery. Eur J Cardiovasc Nurs. 2009;8:309–14.
Trzepacz PT, Baker RW, Greenhouse J. A symptom rating scale for delirium. Psychiatry Res. 1988;23:89–97.
Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Validation of the delirium rating scale-Revised-98: comparison with the delirium rating scale and the cognitive test for delirium. J Neuropsychiatry Clin Neurosci. 2001;13:229–42.
Sepulveda E, Franco JG, Trzepacz PT, Gaviria AM, Viñuelas E, Palma J, et al. Performance of the delirium rating scale-Revised-98 against different delirium diagnostic criteria in a population with a high prevalence of dementia. Psychosomatics. 2015;56:530–41.
Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22:209–12.
Meagher DJ, Adamis D, Trzepacz PT, Leonard M. Features of subsyndromal and persistent delirium. Br J Psychiatry. 2012;200:37–44.
Kato M, Kishi Y, Okuyama T, Trzepacz PT, Hosaka T. Japanese version of the delirium rating scale, Revised-98 (DRS-R98-J): reliability and validity. Psychosomatics. 2010;51:425–31.
We would like to thank and acknowledge all participants for their willingness to contribute to this study.
This study was supported by the Netherlands Organization for Health Research and Development (ZonMw) Palliative Care Program (No. 11510011). The funding source had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This study was approved by the Medical research Ethics Committee of the VU University medical center for all participating sites (Additional file 1) and performed in accordance with the Helsinki Declaration of the World Medical Association and the Dutch law on medical-scientific research on human beings ‘wet medisch-wetenschappelijk onderzoek met mensen’.
All patients provided written informed consent to participate in this study. If patients were not able to provide informed consent because of their suspected delirium, their legal representative provided informed consent, which was verified with the patient after recovery.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.