A novel computer-assisted tool for 3D imaging of programmed death-ligand 1 expression in immunofluorescence-stained and optically cleared breast cancer specimens

Background Programmed death-1 (PD-1) and programmed death-ligand 1 (PD-L1) are the two most common immune checkpoints targeted in triple-negative breast cancer (BC). Refining patient selection for immunotherapy is non-trivial and finding an appropriate digital pathology framework for spatial analysis of theranostic biomarkers for PD-1/PD-L1 inhibitors remains an unmet clinical need. Methods We describe a novel computer-assisted tool for three-dimensional (3D) imaging of PD-L1 expression in immunofluorescence-stained and optically cleared BC specimens (n = 20). The proposed 3D framework appeared to be feasible and showed a high overall agreement with traditional, clinical-grade two-dimensional (2D) staining techniques. Additionally, the results obtained for automated immune cell detection and analysis of PD-L1 expression were satisfactory. Results The spatial distribution of PD-L1 expression was heterogeneous across various BC tissue layers in the 3D space. Notably, there were six cases (30%) wherein PD-L1 expression levels along different layers crossed the 1% threshold for admitting patients to PD-1/PD-L1 inhibitors. The average PD-L1 expression in 3D space was different from that of traditional immunohistochemistry (IHC) in eight cases (40%). Pending further standardization and optimization, we expect that our technology will become a valuable addition for assessing PD-L1 expression in patients with BC. Conclusion Via a single round of immunofluorescence imaging, our approach may provide a considerable improvement in patient stratification for cancer immunotherapy as compared with standard techniques. Supplementary Information The online version contains supplementary material available at 10.1186/s12885-023-11748-8.

head of HRNetV2 (38) -resulting in over 4.7 million learnable parameters for both tumor cell and immune cell segmentation models.

Model training and evaluation
The learning targets of the tumor cell and immune cell segmentation models were prepared separately.Ground-truth tumor cell regions were manually annotated by experienced pathologists, whereas ground-truth annotations of immune cell regions were processed from CD45 immunofluorescence staining.Specifically, the binary masks for the CD45 and SYTO16 signals were generated by applying the proposed computer-vision-based approach.The SYTO16 signal attaching to CD45 was used as the learning target for the immune cell segmentation model.Conversely, the training sample images for both segmentation models only relied on the SYTO16 and DiD fluorescent channels -without information concerning CD45 staining.
Prior to model training, training images and learning targets were divided into small patches.Each patch consisted of 128 × 128 pixels, with neighboring patches overlapping by 20% with respect to width/height (equivalent to 25 pixels).All of the background patches with no fluorescent staining and 90% of patches that included no learning target pixels were discarded.The remaining patches were randomly divided (8:1:1 ratio) into the training, validation, and testing datasets (Supplement 5A).
Additionally, training images were augmented on their saturation, brightness, and contrast; to this aim, each parameter was adjusted by a uniformly sampled factor between 0.7 and 1.3.
The tumor cell and immune cell segmentation models were trained over 100 epochs, and each training step consisted of a 64-patch batch.We used the binary cross entropy loss function as the learning objective for both models; positive and negative pixels inversely weighted according to their counts in the training dataset.During training, the learning rate was adjusted through a cosine decay schedule with restarts (37).The initial leaning rate was set to 0.002 and decayed to zero within 1300 steps; subsequently, each restart halved the initial learning rate and doubled the decaying period.After training, the model checkpoint with the least validation loss was retained as the segmentation model for the computer-assisted IC algorithm.The hyperparameters of both models were adjusted jointly over their validation patch dataset and 16 fluorescent images from breast cancer specimens, which led to a pixelwise classification threshold of 0.7 and 0.5 for the tumor cell and immune cell segmentation models, respectively.The two trained models were subsequently evaluated through the testing datasets.The model accuracy, specificity, and sensitivity are presented in Supplements 5B and 5C.

Tumor-infiltrating immune cell detection algorithm
The detailed parameters used for devising the IC detection algorithm are shown in Supplement 5D.In brief, the SYTO16 and DiD channels in each fluorescent image were separated from the SP142 channel.The image was subsequently sectioned into patches (size: 128 × 128 pixels) from which the predicted masks of tumor cell and immune cell regions were generated via the segmentation models; to this aim, the tuned classification threshold was applied.Both predicted masks were processed with edge smoothing -which was based on the following operations performed sequentially: upscaling, median blurring, and downscaling.Smoothed masks were filtered to exclude small connected components.
Next, we approximated the tumor area and the area covered by PD-L1-expressing immune cells using computer-vision-based methods.Specifically, the predicted mask of immune cell regions was dilated, and its connected components attached to the predicted tumor regions were determined.The tumor area was defined as the union between the predicted tumor cell regions and attached immune cell regions, whereas the area of PD-L1 expression was calculated by extracting SP142 and SYTO16 signals from the fluorescent image.To this aim, the union of the SP142 binary mask and its attached SYTO16 signal located outside of the tumor cell regions predicted by the segmentation model was taken into account.Finally, the predicted IC score was calculated by dividing the area of PD-L1 expression by the tumor area.