Table 4 Inter-rater reliability between pathologists using the S-score for two experiments on the same set conducted with a one month gap
From: Inference of core needle biopsy whole slide images requiring definitive therapy for prostate cancer
| Â | Predicted label | Number of pathologists | Interpretation | ||
|---|---|---|---|---|---|
| Â | 10 (all) | 5 (< 10 yrs) | 5 (\(\ge\) 10 yrs) | ||
1st Consensus | |||||
    Benign (25/25) | benign | 0.93 | 0.95 | 0.90 | almost perfect agreement |
    Indolent (25/25) | indolent | 0.58 | 0.56 | 0.58 | moderate agreement |
    Indolent (13/25), aggressive (12/25) | indolent & aggressive | 0.18 | 0.10 | 0.17 | slight agreement |
    Indolent (13/25) | indolent & aggressive | 0.18 | 0.10 | 0.19 | slight agreement |
    Aggressive (12/25) | indolent & aggressive | 0.18 | 0.10 | 0.15 | slight agreement |
    Aggressive (25/25) | aggressive | 0.61 | 0.48 | 0.70 | moderate to substantial agreement |
2nd Consensus | |||||
    Benign (25/25) | benign | 0.95 | 0.95 | 0.95 | almost perfect agreement |
    Indolent (25/25) | indolent | 0.70 | 0.65 | 0.72 | substantial agreement |
    Indolent (13/25), aggressive (12/25) | indolent & aggressive | 0.26 | 0.18 | 0.22 | slight to fair agreement |
    Indolent (13/25) | indolent & aggressive | 0.28 | 0.19 | 0.26 | slight to fair agreement |
    Aggressive (12/25) | indolent & aggressive | 0.23 | 0.18 | 0.18 | slight to fair agreement |
    Aggressive (25/25) | aggressive | 0.75 | 0.68 | 0.81 | substantial to almost perfect agreement |