Revisiting the Trustworthiness of Saliency Methods in Radiology AI.

Purpose To determine whether saliency maps in radiology artificial intelligence (AI) are vulnerable to subtle perturbations of the input, which could lead to misleading interpretations, using prediction-saliency correlation (PSC) for evaluating the sensitivity and robustness of saliency methods. Materials and Methods In this retrospective study, locally trained deep learning models and a research prototype provided by a commercial vendor were systematically evaluated on 191 229 chest radiographs from the CheXpert dataset and 7022 MR images from a human brain tumor classification dataset. Two radiologists performed a reader study on 270 chest radiograph pairs. A model-agnostic approach for computing the PSC coefficient was used to evaluate the sensitivity and robustness of seven commonly used saliency methods. Results The saliency methods had low sensitivity (maximum PSC, 0.25; 95% CI: 0.12, 0.38) and weak robustness (maximum PSC, 0.12; 95% CI: 0.0, 0.25) on the CheXpert dataset, as demonstrated by leveraging locally trained model parameters. Further evaluation showed that the saliency maps generated from a commercial prototype could be irrelevant to the model output, without knowledge of the model specifics (area under the receiver operating characteristic curve decreased by 8.6% without affecting the saliency map). The human observer studies confirmed that it is difficult for experts to identify the perturbed images; the experts had less than 44.8% correctness. Conclusion Popular saliency methods scored low PSC values on the two datasets of perturbed chest radiographs, indicating weak sensitivity and robustness. The proposed PSC metric provides a valuable quantification tool for validating the trustworthiness of medical AI explainability. Keywords: Saliency Maps, AI Trustworthiness, Dynamic Consistency, Sensitivity, Robustness Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Yanagawa and Sato in this issue.

Dynamically consistent objective and subjective rationality.

A group of experts, for instance climate scientists, is to advise a decision maker about the choice between two policies f and g. Consider the following decision rule. If all experts agree that the expected utility of f is higher than the expected utility of g, the unanimity rule applies, and f is chosen. Otherwise, the precautionary principle is implemented and the policy yielding the highest minimal expected utility is chosen. This decision rule may lead to time inconsistencies when adding an intermediate period of partial resolution of uncertainty. We show how to coherently reassess the initial set of experts' beliefs so that precautionary choices become dynamically consistent: new beliefs should be added until one obtains the smallest "rectangular set" that contains the original one. Our analysis offers a novel behavioral characterization of rectangularity and a prescriptive way to aggregate opinions in order to avoid sure regret.

Automated optimization of residual reduction algorithm parameters in OpenSim.

The residual reduction algorithm (RRA) in OpenSim is designed to improve dynamic consistency of kinematics and ground reaction forces in movement simulations of musculoskeletal models. RRA requires the user to select numerous tracking weights for the joint kinematics to reduce residual errors. Selection is often performed manually, which can be time-consuming and is unlikely to yield optimal tracking weights. A multi-heuristic optimization algorithm was used to expedite tracking weight decision making to reduce residual errors. This method produced more rigorous results than manual iterations and although the total computation time was not significantly reduced, this method does not require the user to monitor the algorithm's progress to find a solution, thereby reducing manual tuning. Supporting documentation and code to implement this optimization is freely provided to assist the community with developing movement simulations.

Semi-supervised learning with progressive unlabeled data excavation for label-efficient surgical workflow recognition.

Surgical workflow recognition is a fundamental task in computer-assisted surgery and a key component of various applications in operating rooms. Existing deep learning models have achieved promising results for surgical workflow recognition, heavily relying on a large amount of annotated videos. However, obtaining annotation is time-consuming and requires the domain knowledge of surgeons. In this paper, we propose a novel two-stage Semi-Supervised Learning method for label-efficient Surgical workflow recognition, named as SurgSSL. Our proposed SurgSSL progressively leverages the inherent knowledge held in the unlabeled data to a larger extent: from implicit unlabeled data excavation via motion knowledge excavation, to explicit unlabeled data excavation via pre-knowledge pseudo labeling. Specifically, we first propose a novel intra-sequence Visual and Temporal Dynamic Consistency (VTDC) scheme for implicit excavation. It enforces prediction consistency of the same data under perturbations in both spatial and temporal spaces, encouraging model to capture rich motion knowledge. We further perform explicit excavation by optimizing the model towards our pre-knowledge pseudo label. It is naturally generated by the VTDC regularized model with prior knowledge of unlabeled data encoded, and demonstrates superior reliability for model supervision compared with the label generated by existing methods. We extensively evaluate our method on two public surgical datasets of Cholec80 and M2CAI challenge dataset. Our method surpasses the state-of-the-art semi-supervised methods by a large margin, e.g., improving 10.5% Accuracy under the severest annotation regime of M2CAI dataset. Using only 50% labeled videos on Cholec80, our approach achieves competitive performance compared with full-data training method.

Prediction of ground reaction forces and moments during various activities of daily living.

Inverse dynamics based simulations on musculoskeletal models is a commonly used method for the analysis of human movement. Due to inaccuracies in the kinematic and force plate data, and a mismatch between the model and the subject, the equations of motion are violated when solving the inverse dynamics problem. As a result, dynamic inconsistency will exist and lead to residual forces and moments. In this study, we present and evaluate a computational method to perform inverse dynamics-based simulations without force plates, which both improves the dynamic consistency as well as removes the model׳s dependency on measured external forces. Using the equations of motion and a scaled musculoskeletal model, the ground reaction forces and moments (GRF&Ms) are derived from three-dimensional full-body motion. The method entails a dynamic contact model and optimization techniques to solve the indeterminacy problem during a double contact phase and, in contrast to previously proposed techniques, does not require training or empirical data. The method was applied to nine healthy subjects performing several Activities of Daily Living (ADLs) and evaluated with simultaneously measured force plate data. Except for the transverse ground reaction moment, no significant differences (P>0.05) were found between the mean predicted and measured GRF&Ms for almost all ADLs. The mean residual forces and moments, however, were significantly reduced (P>0.05) in almost all ADLs using our method compared to conventional inverse dynamic simulations. Hence, the proposed method may be used instead of raw force plate data in human movement analysis using inverse dynamics.