ABSTRACT
Purpose To present results from a literature survey on practices in deep learning segmentation algorithm evaluation and perform a study on expert quality perception of brain tumor segmentation. Materials and Methods A total of 180 articles reporting on brain tumor segmentation algorithms were surveyed for the reported quality evaluation. Additionally, ratings of segmentation quality on a four-point scale were collected from medical professionals for 60 brain tumor segmentation cases. Results Of the surveyed articles, Dice score, sensitivity, and Hausdorff distance were the most popular metrics to report segmentation performance. Notably, only 2.8% of the articles included clinical experts' evaluation of segmentation quality. The experimental results revealed a low interrater agreement (Krippendorff α, 0.34) in experts' segmentation quality perception. Furthermore, the correlations between the ratings and commonly used quantitative quality metrics were low (Kendall tau between Dice score and mean rating, 0.23; Kendall tau between Hausdorff distance and mean rating, 0.51), with large variability among the experts. Conclusion The results demonstrate that quality ratings are prone to variability due to the ambiguity of tumor boundaries and individual perceptual differences, and existing metrics do not capture the clinical perception of segmentation quality. Keywords: Brain Tumor Segmentation, Deep Learning Algorithms, Glioblastoma, Cancer, Machine Learning Clinical trial registration nos. NCT00756106 and NCT00662506 Supplemental material is available for this article. © RSNA, 2023.
Subject(s)
Brain Neoplasms , Deep Learning , Glioblastoma , Humans , Algorithms , Benchmarking , Brain Neoplasms/diagnostic imaging , Glioblastoma/diagnostic imagingABSTRACT
The glioblastoma tumor microenvironment is highly immunosuppressed. This immunosuppressive state is engineered by inhibitory molecules secreted by tumor cells that limit activation of immune effector cells, drive T-cell exhaustion, and enhance the immunosuppressive action of tumor-associated myeloid cells. Immunotherapeutic approaches have sought to combat glioblastoma microenvironment immunosuppression with agents such as immune checkpoint inhibitors. Although immune checkpoint blockade in glioblastoma has yielded disappointing results thus far, there is significant interest in the combination of immune checkpoint blockade with other approaches to enhance response.