Biomechanical Signals of Varied Modality and Location Contribute Differently to Recognition of Transient Locomotion.

Intent recognition in lower-limb assistive devices typically relies on neuromechanical sensing of an affected limb acquired through embedded device sensors. It remains unknown whether signals from more widespread sources such as the contralateral leg and torso positively influence intent recognition, and how specific locomotor tasks that place high demands on the neuromuscular system, such as changes of direction, contribute to intent recognition. In this study, we evaluated the performances of signals from varying mechanical modalities (accelerographic, gyroscopic, and joint angles) and locations (the trailing leg, leading leg and torso) during straight walking, changes of direction (cuts), and cuts to stair ascent with varying task anticipation. Biomechanical information from the torso demonstrated poor performance across all conditions. Unilateral (the trailing or leading leg) joint angle data provided the highest accuracy. Surprisingly, neither the fusion of unilateral and torso data nor the combination of multiple signal modalities improved recognition. For these fused modality data, similar trends but with diminished accuracy rates were reported during unanticipated conditions. Finally, for datasets that achieved a relatively accurate (≥90%) recognition of unanticipated tasks, these levels of recognition were achieved after the mid-swing of the trailing/transitioning leg, prior to a subsequent heel strike. These findings suggest that mechanical sensing of the legs and torso for the recognition of straight-line and transient locomotion can be implemented in a relatively flexible manner (i.e., signal modality, and from the leading or trailing legs) and, importantly, suggest that more widespread sensing is not always optimal.

Leveraging global binary masks for structure segmentation in medical images.

Deep learning (DL) models for medical image segmentation are highly influenced by intensity variations of input images and lack generalization due to primarily utilizing pixels' intensity information for inference. Acquiring sufficient training data is another challenge limiting models' applications. Here, we proposed to leverage the consistency of organs' anatomical position and shape information in medical images. We introduced a framework leveraging recurring anatomical patterns through global binary masks for organ segmentation. Two scenarios were studied: (1) global binary masks were the only input for the U-Net based model, forcing exclusively encoding organs' position and shape information for rough segmentation or localization. (2) Global binary masks were incorporated as an additional channel providing position/shape clues to mitigate training data scarcity. Two datasets of the brain and heart computed tomography (CT) images with their ground-truth were split into (26:10:10) and (12:3:5) for training, validation, and test respectively. The two scenarios were evaluated using full training split as well as reduced subsets of training data. In scenario (1), training exclusively on global binary masks led to Dice scores of 0.77 ± 0.06 and 0.85 ± 0.04 for the brain and heart structures respectively. Average Euclidian distance of 3.12 ± 1.43 mm and 2.5 ± 0.93 mm were obtained relative to the center of mass of the ground truth for the brain and heart structures respectively. The outcomes indicated encoding a surprising degree of position and shape information through global binary masks. In scenario (2), incorporating global binary masks led to significantly higher accuracy relative to the model trained on only CT images in small subsets of training data; the performance improved by 4.3%-125.3% and 1.3%-48.1% for 1-8 training cases of the brain and heart datasets respectively. The findings imply the advantages of utilizing global binary masks for building models that are robust to image intensity variations as well as an effective approach to boost performance when access to labeled training data is highly limited.

A deep learning approach for automatic delineation of clinical target volume in stereotactic partial breast irradiation (S-PBI).

Accurate and efficient delineation of the clinical target volume (CTV) is of utmost significance in post-operative breast cancer radiotherapy. However, CTV delineation is challenging as the exact extent of microscopic disease encompassed by CTV is not visualizable in radiological images and remains uncertain. We proposed to mimic physicians' contouring practice for CTV segmentation in stereotactic partial breast irradiation (S-PBI) where CTV is derived from tumor bed volume (TBV) via a margin expansion followed by correcting the extensions for anatomical barriers of tumor invasion (e.g. skin, chest wall). We proposed a deep-learning model, where CT images and the corresponding TBV masks formed a multi-channel input for a 3D U-Net based architecture. The design guided the model to encode the location-related image features and directed the network to focus on TBV to initiate CTV segmentation. Gradient weighted class activation map (Grad-CAM) visualizations of the model predictions revealed that the extension rules and geometric/anatomical boundaries were learnt during model training to assist the network to limit the expansion to a certain distance from the chest wall and the skin. We retrospectively collected 175 prone CT images from 35 post-operative breast cancer patients who received 5-fraction partial breast irradiation regimen on GammaPod. The 35 patients were randomly split into training (25), validation (5) and test (5) sets. Our model achieved mean (standard deviation) of 0.94 (±0.02), 2.46 (±0.5) mm, and 0.53 (±0.14) mm for Dice similarity coefficient, 95th percentile Hausdorff distance, and average symmetric surface distance respectively on the test set. The results are promising for improving the efficiency and accuracy of CTV delineation during on-line treatment planning procedure.

Deep-learning and radiomics ensemble classifier for false positive reduction in brain metastases segmentation.

Stereotactic radiosurgery (SRS) is now the standard of care for brain metastases (BMs) patients. The SRS treatment planning process requires precise target delineation, which in clinical workflow for patients with multiple (>4) BMs (mBMs) could become a pronounced time bottleneck. Our group has developed an automated BMs segmentation platform to assist in this process. The accuracy of the auto-segmentation, however, is influenced by the presence of false-positive segmentations, mainly caused by the injected contrast during MRI acquisition. To address this problem and further improve the segmentation performance, a deep-learning and radiomics ensemble classifier was developed to reduce the false-positive rate in segmentations. The proposed model consists of a Siamese network and a radiomic-based support vector machine (SVM) classifier. The 2D-based Siamese network contains a pair of parallel feature extractors with shared weights followed by a single classifier. This architecture is designed to identify the inter-class difference. On the other hand, the SVM model takes the radiomic features extracted from 3D segmentation volumes as the input for twofold classification, either a false-positive segmentation or a true BM. Lastly, the outputs from both models create an ensemble to generate the final label. The performance of the proposed model in the segmented mBMs testing dataset reached the accuracy (ACC), sensitivity (SEN), specificity (SPE) and area under the curve of 0.91, 0.96, 0.90 and 0.93, respectively. After integrating the proposed model into the original segmentation platform, the average segmentation false negative rate (FNR) and the false positive over the union (FPoU) were 0.13 and 0.09, respectively, which preserved the initial FNR (0.07) and significantly improved the FPoU (0.55). The proposed method effectively reduced the false-positive rate in the BMs raw segmentations indicating that the integration of the proposed ensemble classifier into the BMs segmentation platform provides a beneficial tool for mBMs SRS management.

Ensemble learning for glioma patients overall survival prediction using pre-operative MRIs.

Objective: Gliomas are the most common primary brain tumors. Approximately 70% of the glioma patients diagnosed with glioblastoma have an averaged overall survival (OS) of only ∼16 months. Early survival prediction is essential for treatment decision-making in glioma patients. Here we proposed an ensemble learning approach to predict the post-operative OS of glioma patients using only pre-operative MRIs.Approach: Our dataset was from the Medical Image Computing and Computer Assisted Intervention Brain Tumor Segmentation challenge 2020, which consists of multimodal pre-operative MRI scans of 235 glioma patients with survival days recorded. The backbone of our approach was a Siamese network consisting of twinned ResNet-based feature extractors followed by a 3-layer classifier. During training, the feature extractors explored traits of intra and inter-class by minimizing contrastive loss of randomly paired 2D pre-operative MRIs, and the classifier utilized the extracted features to generate labels with cost defined by cross-entropy loss. During testing, the extracted features were also utilized to define distance between the test sample and the reference composed of training data, to generate an additional predictor via K-NN classification. The final label was the ensemble classification from both the Siamese model and the K-NN model.Main results: Our approach classifies the glioma patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (<10 months). The performance is assessed by the accuracy (ACC) and the area under the curve (AUC) of 3-class classification. The final result achieved an ACC of 65.22% and AUC of 0.81.Significance: Our Siamese network based ensemble learning approach demonstrated promising ability in mining discriminative features with minimal manual processing and generalization requirement. This prediction strategy can be potentially applied to assist timely clinical decision-making.

Continuous Classification of Locomotion in Response to Task Complexity and Anticipatory State.

OBJECTIVE: Intent recognition in lower-extremity assistive devices (e.g., prostheses and exoskeletons) is typically limited to either recognition of steady-state locomotion or changes of terrain (e.g., level ground to stair) occurring in a straight-line path and under anticipated condition. Stability is highly affected during non-steady changes of direction such as cuts especially when they are unanticipated, posing high risk of fall-related injuries. Here, we studied the influence of changes of direction and user anticipation on task recognition, and accordingly introduced classification schemes accommodating such effects. METHODS: A linear discriminant analysis (LDA) classifier continuously classified straight-line walking, sidestep/crossover cuts (single transitions), and cuts-to-stair locomotion (mixed transitions) performed under varied task anticipatory conditions. Training paradigms with varying levels of anticipated/unanticipated exposures and analysis windows of size 100-600 ms were examined. RESULTS: More accurate classification of anticipated relative to unanticipated tasks was observed. Including bouts of target task in the training data was necessary to improve generalization to unanticipated locomotion. Only up to two bouts of target task were sufficient to reduce errors to <20% in unanticipated mixed transitions, whereas, in single transitions and straight walking, substantial unanticipated information (i.e., five bouts) was necessary to achieve similar outcomes. Window size modifications did not have a significant influence on classification performance. CONCLUSION: Adjusting the training paradigm helps to achieve classification schemes capable of adapting to changes of direction and task anticipatory state. SIGNIFICANCE: The findings could provide insight into developing classification schemes that can adapt to changes of direction and user anticipation. They could inform intent recognition strategies for controlling lower-limb assistive to robustly handle "unknown" circumstances, and thus deliver increased level of reliability and safety.

Saliency-guided deep learning network for automatic tumor bed volume delineation in post-operative breast irradiation.

Efficient, reliable and reproducible target volume delineation is a key step in the effective planning of breast radiotherapy. However, post-operative breast target delineation is challenging as the contrast between the tumor bed volume (TBV) and normal breast tissue is relatively low in CT images. In this study, we propose to mimic the marker-guidance procedure in manual target delineation. We developed a saliency-based deep learning segmentation (SDL-Seg) algorithm for accurate TBV segmentation in post-operative breast irradiation. The SDL-Seg algorithm incorporates saliency information in the form of markers' location cues into a U-Net model. The design forces the model to encode the location-related features, which underscores regions with high saliency levels and suppresses low saliency regions. The saliency maps were generated by identifying markers on CT images. Markers' location were then converted to probability maps using a distance transformation coupled with a Gaussian filter. Subsequently, the CT images and the corresponding saliency maps formed a multi-channel input for the SDL-Seg network. Our in-house dataset was comprised of 145 prone CT images from 29 post-operative breast cancer patients, who received 5-fraction partial breast irradiation (PBI) regimen on GammaPod. The 29 patients were randomly split into training (19), validation (5) and test (5) sets. The performance of the proposed method was compared against basic U-Net. Our model achieved mean (standard deviation) of 76.4(±2.7) %, 6.76(±1.83) mm, and 1.9(±0.66) mm for Dice similarity coefficient, 95 percentile Hausdorff distance, and average symmetric surface distance respectively on the test set with computation time of below 11 seconds per one CT volume. SDL-Seg showed superior performance relative to basic U-Net for all the evaluation metrics while preserving low computation cost. The findings demonstrate that SDL-Seg is a promising approach for improving the efficiency and accuracy of the on-line treatment planning procedure of PBI, such as GammaPod based PBI.

Body Segment Mechanical Signal Contributions to Continuous Prediction of Locomotor Transitions Performed under Varying Anticipation.

A reliable, flexible and simple source of information would benefit robust handling of predicting locomotion modes for assistive device control (e.g., prostheses). However, to date, the sources of mechanical signals have been mainly limited to the information acquired through embedded sensors in the device. It remains unclear whether biomechanical signals from unaffected or less affected locations (e.g., contralateral side or upper body) would be reliable sources of information. Furthermore, the possible influence of the anticipatory state of the task on recognition accuracy, emphasizes the need to identify reliable data sources for both anticipated and unanticipated tasks. Here, accelerographic and gyroscopic signals from the leading leg, trailing leg, trunk-pelvis, and their fusion were compared with respect to their ability to predict changes of direction (cuts), cut-to-stair transitions, and level-ground walking performed under varied task anticipation. We hypothesized that fusion of lower- and upper-body signals would provide better accuracy than unilateral information (i.e., trailing/leading leg), and recognition accuracy would diminish when tasks were unanticipated. Surprisingly, signal fusion appeared not to be advantageous to unilateral signals. Leading and trailing leg data demonstrated statistically identical performances, and trunk-pelvis signals showed significantly (α=0.05) inferior performance relative to unilateral data. While anticipated tasks were accurately predicted (≥90%) even as early as 500 ms prior to entering each locomotor transition, in unanticipated tasks, similar accuracy rates were achieved only after the mid-swing of the transitioning leg. The findings could provide insight into flexible, yet, dependable sensor sets for intent recognition frameworks during varying user cognitive states.

Prediction of neural differentiation fate of rat mesenchymal stem cells by quantitative morphological analyses using image processing techniques.

Differentiation of bone marrow mesenchymal stem cells (BMSCs) into neural cells has received significant attention in recent years. However, there is still no practical method to evaluate differentiation process non-invasively and practically. The cellular quality evaluation method is still limited to conventional techniques, which are based on extracting genes or proteins from the cells. These techniques are invasive, costly, time consuming, and should be performed by relevant experts in equipped laboratories. Moreover, they cannot anticipate the future status of cells. Recently, cell morphology has been introduced as a feasible way of monitoring cell behavior because of its relationship with cell proliferation, functions and differentiation. In this study, rat BMSCs were induced to differentiate into neurons. Subsequently, phase contrast images of cells taken at certain intervals were subjected to a series of image processing steps and cell morphology features were calculated. In order to validate the viability of applying image-based approaches for estimating the quality of differentiation process, neural-specific markers were measured experimentally throughout the induction. The strong correlation between quantitative imaging metrics and experimental outcomes revealed the capability of the proposed approach as an auxiliary method of assessing cell behavior during differentiation.