COVID-Net Biochem: an explainability-driven framework to building machine learning models for predicting survival and kidney injury of COVID-19 patients from clinical and biochemistry data.

Since the World Health Organization declared COVID-19 a pandemic in 2020, the global community has faced ongoing challenges in controlling and mitigating the transmission of the SARS-CoV-2 virus, as well as its evolving subvariants and recombinants. A significant challenge during the pandemic has not only been the accurate detection of positive cases but also the efficient prediction of risks associated with complications and patient survival probabilities. These tasks entail considerable clinical resource allocation and attention. In this study, we introduce COVID-Net Biochem, a versatile and explainable framework for constructing machine learning models. We apply this framework to predict COVID-19 patient survival and the likelihood of developing Acute Kidney Injury during hospitalization, utilizing clinical and biochemical data in a transparent, systematic approach. The proposed approach advances machine learning model design by seamlessly integrating domain expertise with explainability tools, enabling model decisions to be based on key biomarkers. This fosters a more transparent and interpretable decision-making process made by machines specifically for medical applications. More specifically, the framework comprises two phases: In the first phase, referred to as the "clinician-guided design" phase, the dataset is preprocessed using explainable AI and domain expert input. To better demonstrate this phase, we prepared a benchmark dataset of carefully curated clinical and biochemical markers based on clinician assessments for survival and kidney injury prediction in COVID-19 patients. This dataset was selected from a patient cohort of 1366 individuals at Stony Brook University. Moreover, we designed and trained a diverse collection of machine learning models, encompassing gradient-based boosting tree architectures and deep transformer architectures, specifically for survival and kidney injury prediction based on the selected markers. In the second phase, called the "explainability-driven design refinement" phase, the proposed framework employs explainability methods to not only gain a deeper understanding of each model's decision-making process but also to identify the overall impact of individual clinical and biochemical markers for bias identification. In this context, we used the models constructed in the previous phase for the prediction task and analyzed the explainability outcomes alongside a clinician with over 8 years of experience to gain a deeper understanding of the clinical validity of the decisions made. The explainability-driven insights obtained, in conjunction with the associated clinical feedback, are then utilized to guide and refine the training policies and architectural design iteratively. This process aims to enhance not only the prediction performance but also the clinical validity and trustworthiness of the final machine learning models. Employing the proposed explainability-driven framework, we attained 93.55% accuracy in survival prediction and 88.05% accuracy in predicting kidney injury complications. The models have been made available through an open-source platform. Although not a production-ready solution, this study aims to serve as a catalyst for clinical scientists, machine learning researchers, and citizen scientists to develop innovative and trustworthy clinical decision support solutions, ultimately assisting clinicians worldwide in managing pandemic outcomes.

OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection.

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model's latency show that the OutlierNet architectures can achieve as much as 30× lower latency than published networks.

MEDUSA: Multi-Scale Encoder-Decoder Self-Attention Deep Neural Network Architecture for Medical Image Analysis.

Medical image analysis continues to hold interesting challenges given the subtle characteristics of certain diseases and the significant overlap in appearance between diseases. In this study, we explore the concept of self-attention for tackling such subtleties in and between diseases. To this end, we introduce, a multi-scale encoder-decoder self-attention (MEDUSA) mechanism tailored for medical image analysis. While self-attention deep convolutional neural network architectures in existing literature center around the notion of multiple isolated lightweight attention mechanisms with limited individual capacities being incorporated at different points in the network architecture, MEDUSA takes a significant departure from this notion by possessing a single, unified self-attention mechanism with significantly higher capacity with multiple attention heads feeding into different scales in the network architecture. To the best of the authors' knowledge, this is the first "single body, multi-scale heads" realization of self-attention and enables explicit global context among selective attention at different levels of representational abstractions while still enabling differing local attention context at individual levels of abstractions. With MEDUSA, we obtain state-of-the-art performance on multiple challenging medical image analysis benchmarks including COVIDx, Radiological Society of North America (RSNA) RICORD, and RSNA Pneumonia Challenge when compared to previous work. Our MEDUSA model is publicly available.

COVID-Net CXR-S: Deep Convolutional Neural Network for Severity Assessment of COVID-19 Cases from Chest X-ray Images.

The world is still struggling in controlling and containing the spread of the COVID-19 pandemic caused by the SARS-CoV-2 virus. The medical conditions associated with SARS-CoV-2 infections have resulted in a surge in the number of patients at clinics and hospitals, leading to a significantly increased strain on healthcare resources. As such, an important part of managing and handling patients with SARS-CoV-2 infections within the clinical workflow is severity assessment, which is often conducted with the use of chest X-ray (CXR) images. In this work, we introduce COVID-Net CXR-S, a convolutional neural network for predicting the airspace severity of a SARS-CoV-2 positive patient based on a CXR image of the patient's chest. More specifically, we leveraged transfer learning to transfer representational knowledge gained from over 16,000 CXR images from a multinational cohort of over 15,000 SARS-CoV-2 positive and negative patient cases into a custom network architecture for severity assessment. Experimental results using the RSNA RICORD dataset showed that the proposed COVID-Net CXR-S has potential to be a powerful tool for computer-aided severity assessment of CXR images of COVID-19 positive patients. Furthermore, radiologist validation on select cases by two board-certified radiologists with over 10 and 19 years of experience, respectively, showed consistency between radiologist interpretation and critical factors leveraged by COVID-Net CXR-S for severity assessment. While not a production-ready solution, the ultimate goal for the open source release of COVID-Net CXR-S is to act as a catalyst for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative new clinical decision support solutions for helping clinicians around the world manage the continuing pandemic.

MPCaD: a multi-scale radiomics-driven framework for automated prostate cancer localization and detection.

BACKGROUND: Quantitative radiomic features provide a plethora of minable data extracted from multi-parametric magnetic resonance imaging (MP-MRI) which can be used for accurate detection and localization of prostate cancer. While most cancer detection algorithms utilize either voxel-based or region-based feature models, the complexity of prostate tumour phenotype in MP-MRI requires a more sophisticated framework to better leverage available data and exploit a priori knowledge in the field. METHODS: In this paper, we present MPCaD, a novel Multi-scale radiomics-driven framework for Prostate Cancer Detection and localization which leverages radiomic feature models at different scales as well as incorporates a priori knowledge of the field. Tumour candidate localization is first performed using a statistical texture distinctiveness strategy that leverages a voxel-resolution feature model to localize tumour candidate regions. Tumour region classification via a region-resolution feature model is then performed to identify tumour regions. Both voxel-resolution and region-resolution feature models are built upon and extracted from six different MP-MRI modalities. Finally, a conditional random field framework that is driven by voxel-resolution relative ADC features is used to further refine the localization of the tumour regions in the peripheral zone to improve the accuracy of the results. RESULTS: The proposed framework is evaluated using clinical prostate MP-MRI data from 30 patients, and results demonstrate that the proposed framework exhibits enhanced separability of cancerous and healthy tissue, as well as outperforms individual quantitative radiomics models for prostate cancer detection. CONCLUSION: Quantitative radiomic features extracted from MP-MRI of prostate can be utilized to detect and localize prostate cancer.

DeepPredict: A deep predictive intelligence platform for patient monitoring.

A novel platform, DeepPredict, for predicting hospital bed exit events from video camera systems is proposed. DeepPredict processes video data with a deep convolutional neural network consisting of five main layers: a 1 × 1 3D convolutional layer used for generating feature maps from raw video data, a context-aware pooling layer used for rectifying data from different camera angles, two fully connected layers used for applying pre-trained deep features, and an output layer used to provide a likelihood of a bed exit event. Results for a model trained on 180 hours of data demonstrate accuracy, sensitivity, and specificity of 86.47%, 78.87%, and 94.07%, respectively, when predicting a bed exit event up to seven seconds in advance.

Discovery radiomics via evolutionary deep radiomic sequencer discovery for pathologically proven lung cancer detection.

While lung cancer is the second most diagnosed form of cancer in men and women, a sufficiently early diagnosis can be pivotal in patient survival rates. Imaging-based, or radiomics-driven, detection methods have been developed to aid diagnosticians, but largely rely on hand-crafted features that may not fully encapsulate the differences between cancerous and healthy tissue. Recently, the concept of discovery radiomics was introduced, where custom abstract features are discovered from readily available imaging data. We propose an evolutionary deep radiomic sequencer discovery approach based on evolutionary deep intelligence. Motivated by patient privacy concerns and the idea of operational artificial intelligence, the evolutionary deep radiomic sequencer discovery approach organically evolves increasingly more efficient deep radiomic sequencers that produce significantly more compact yet similarly descriptive radiomic sequences over multiple generations. As a result, this framework improves operational efficiency and enables diagnosis to be run locally at the radiologist's computer while maintaining detection accuracy. We evaluated the evolved deep radiomic sequencer (EDRS) discovered via the proposed evolutionary deep radiomic sequencer discovery framework against state-of-the-art radiomics-driven and discovery radiomics methods using clinical lung CT data with pathologically proven diagnostic data from the LIDC-IDRI dataset. The EDRS shows improved sensitivity (93.42%), specificity (82.39%), and diagnostic accuracy (88.78%) relative to previous radiomics approaches.

Compensated Row-Column Ultrasound Imaging System Using Multilayered Edge Guided Stochastically Fully Connected Random Fields.

The row-column method received a lot of attention for 3-D ultrasound imaging. By reducing the number of connections required to address the 2-D array and therefore reducing the amount of data to handle, this addressing method allows for real time 3-D imaging. Row-column still has its limitations: the issues of sparsity, speckle noise inherent to ultrasound, the spatially varying point spread function, and the ghosting artifacts inherent to the row-column method must all be taken into account when building a reconstruction framework. In this research, we build on a previously published system and propose an edge-guided, compensated row-column ultrasound imaging system that incorporates multilayered edge-guided stochastically fully connected conditional random fields to address the limitations of the row-column method. Tests carried out on simulated and real row-column ultrasound images show the effectiveness of our proposed system over other published systems. Visual assessment show our proposed system's potential at preserving edges and reducing speckle. Quantitative analysis shows that our proposed system outperforms previously published systems when evaluated with metrics such as Peak Signal-to-Noise Ratio, Coefficient of Correlation, and Effective Number of Looks. These results show the potential of our proposed system as an effective tool for enhancing 3-D row-column imaging.

Sparse reconstruction of compressive sensing MRI using cross-domain stochastically fully connected conditional random fields.

BACKGROUND: Magnetic Resonance Imaging (MRI) is a crucial medical imaging technology for the screening and diagnosis of frequently occurring cancers. However, image quality may suffer from long acquisition times for MRIs due to patient motion, which also leads to patient discomfort. Reducing MRI acquisition times can reduce patient discomfort leading to reduced motion artifacts from the acquisition process. Compressive sensing strategies applied to MRI have been demonstrated to be effective in decreasing acquisition times significantly by sparsely sampling the k-space during the acquisition process. However, such a strategy requires advanced reconstruction algorithms to produce high quality and reliable images from compressive sensing MRI. METHODS: This paper proposes a new reconstruction approach based on cross-domain stochastically fully connected conditional random fields (CD-SFCRF) for compressive sensing MRI. The CD-SFCRF introduces constraints in both k-space and spatial domains within a stochastically fully connected graphical model to produce improved MRI reconstruction. RESULTS: Experimental results using T2-weighted (T2w) imaging and diffusion-weighted imaging (DWI) of the prostate show strong performance in preserving fine details and tissue structures in the reconstructed images when compared to other tested methods even at low sampling rates. CONCLUSIONS: The ability to better utilize a limited amount of information to reconstruct T2w and DWI images in a short amount of time while preserving the important details in the images demonstrates the potential of the proposed CD-SFCRF framework as a viable reconstruction algorithm for compressive sensing MRI.

Noise-Compensated, Bias-Corrected Diffusion Weighted Endorectal Magnetic Resonance Imaging via a Stochastically Fully-Connected Joint Conditional Random Field Model.

Diffusion weighted magnetic resonance imaging (DW-MR) is a powerful tool in imaging-based prostate cancer screening and detection. Endorectal coils are commonly used in DW-MR imaging to improve the signal-to-noise ratio (SNR) of the acquisition, at the expense of significant intensity inhomogeneities (bias field) that worsens as we move away from the endorectal coil. The presence of bias field can have a significant negative impact on the accuracy of different image analysis tasks, as well as prostate tumor localization, thus leading to increased inter- and intra-observer variability. Retrospective bias correction approaches are introduced as a more efficient way of bias correction compared to the prospective methods such that they correct for both of the scanner and anatomy-related bias fields in MR imaging. Previously proposed retrospective bias field correction methods suffer from undesired noise amplification that can reduce the quality of bias-corrected DW-MR image. Here, we propose a unified data reconstruction approach that enables joint compensation of bias field as well as data noise in DW-MR imaging. The proposed noise-compensated, bias-corrected (NCBC) data reconstruction method takes advantage of a novel stochastically fully connected joint conditional random field (SFC-JCRF) model to mitigate the effects of data noise and bias field in the reconstructed MR data. The proposed NCBC reconstruction method was tested on synthetic DW-MR data, physical DW-phantom as well as real DW-MR data all acquired using endorectal MR coil. Both qualitative and quantitative analysis illustrated that the proposed NCBC method can achieve improved image quality when compared to other tested bias correction methods. As such, the proposed NCBC method may have potential as a useful retrospective approach for improving the consistency of image interpretations.

A unified Bayesian-based compensated magnetic resonance imaging.

Magnetic resonance (MR) images of higher quality is demanded for helping with more accurate and earlier diagnosis of different diseases. The overall quality of MR images is limited due to the existence of different degradation factors such as (1) MR aberrations due to intrinsic properties of the MR scanner, (2) magnetic field inhomogeneity, and (3) inherent MRI noise. Correcting each MRI degradation factor could be solely useful for the quality enhancement of MR imaging with a limited impact. Here, we propose a unified Bayesian based compensated MR imaging (CMRI) system which jointly corrects for the different aforementioned MR aberrations as well as MR noise and hence generates compensated MR (CMR) images with a higher quality. Testing the proposed CMRI system on both MR physical phantom as well as diffusion weighted and T2 weighted MR imaging data resulted in producing MR images with an overall higher quality that better represents different structures of tissue. The quantitative performance analysis shows a higher Signal to Noise (SNR) and Contrast to Noise (CNR) ratios as well as less Coefficient of Variation (CV) for reconstructed images using the proposed CMRI system compared to the Blind Deconvolution Compensation (BDC) method as state-of-the-art. As such, the proposed CMRI system has potential in improving MR image quality, which is important for accurate and consistent clinical interpretation.

Compensated Row-Column Ultrasound Imaging System Using Fisher Tippett Multilayered Conditional Random Field Model.

3-D ultrasound imaging offers unique opportunities in the field of non destructive testing that cannot be easily found in A-mode and B-mode images. To acquire a 3-D ultrasound image without a mechanically moving transducer, a 2-D array can be used. The row column technique is preferred over a fully addressed 2-D array as it requires a significantly lower number of interconnections. Recent advances in 3-D row-column ultrasound imaging systems were largely focused on sensor design. However, these imaging systems face three intrinsic challenges that cannot be addressed by improving sensor design alone: speckle noise, sparsity of data in the imaged volume, and the spatially dependent point spread function of the imaging system. In this paper, we propose a compensated row-column ultrasound image reconstruction system using Fisher-Tippett multilayered conditional random field model. Tests carried out on both simulated and real row-column ultrasound images show the effectiveness of our proposed system as opposed to other published systems. Visual assessment of the results show our proposed system's potential at preserving detail and reducing speckle. Quantitative analysis shows that our proposed system outperforms previously published systems when evaluated with metrics such as Peak Signal to Noise Ratio, Coefficient of Correlation, and Effective Number of Looks. These results show the potential of our proposed system as an effective tool for enhancing 3-D row-column imaging.

A Deep-Structured Conditional Random Field Model for Object Silhouette Tracking.

In this work, we introduce a deep-structured conditional random field (DS-CRF) model for the purpose of state-based object silhouette tracking. The proposed DS-CRF model consists of a series of state layers, where each state layer spatially characterizes the object silhouette at a particular point in time. The interactions between adjacent state layers are established by inter-layer connectivity dynamically determined based on inter-frame optical flow. By incorporate both spatial and temporal context in a dynamic fashion within such a deep-structured probabilistic graphical model, the proposed DS-CRF model allows us to develop a framework that can accurately and efficiently track object silhouettes that can change greatly over time, as well as under different situations such as occlusion and multiple targets within the scene. Experiment results using video surveillance datasets containing different scenarios such as occlusion and multiple targets showed that the proposed DS-CRF approach provides strong object silhouette tracking performance when compared to baseline methods such as mean-shift tracking, as well as state-of-the-art methods such as context tracking and boosted particle filtering.

Apparent Ultra-High b-Value Diffusion-Weighted Image Reconstruction via Hidden Conditional Random Fields.

A promising, recently explored, alternative to ultra-high b-value diffusion weighted imaging (UHB-DWI) is apparent ultra-high b-value diffusion-weighted image reconstruction (AUHB-DWR), where a computational model is used to assist in the reconstruction of apparent DW images at ultra-high b -values. Firstly, we present a novel approach to AUHB-DWR that aims to improve image quality. We formulate the reconstruction of an apparent DW image as a hidden conditional random field (HCRF) in which tissue model diffusion parameters act as hidden states in this random field. The second contribution of this paper is a new generation of fully connected conditional random fields, called the hidden stochastically fully connected conditional random fields (HSFCRF) that allows for efficient inference with significantly reduced computational complexity via stochastic clique structures. The proposed AUHB-DWR algorithms, HCRF and HSFCRF, are evaluated quantitatively in nine different patient cases using Fisher's criteria, probability of error, and coefficient of variation metrics to validate its effectiveness for the purpose of improving intensity delineation between expert identified suspected cancerous and healthy tissue within the prostate gland. The proposed methods are also examined using a prostate phantom, where the apparent ultra-high b-value DW images reconstructed using the tested AUHB-DWR methods are compared with real captured UHB-DWI. The results illustrate that the proposed AUHB-DWR methods has improved reconstruction quality and improved intensity delineation compared with existing AUHB-DWR approaches.

Statistical conditional sampling for variable-resolution video compression.

In this study, we investigate a variable-resolution approach to video compression based on Conditional Random Field and statistical conditional sampling in order to further improve compression rate while maintaining high-quality video. In the proposed approach, representative key-frames within a video shot are identified and stored at full resolution. The remaining frames within the video shot are stored and compressed at a reduced resolution. At the decompression stage, a region-based dictionary is constructed from the key-frames and used to restore the reduced resolution frames to the original resolution via statistical conditional sampling. The sampling approach is based on the conditional probability of the CRF modeling by use of the constructed dictionary. Experimental results show that the proposed variable-resolution approach via statistical conditional sampling has potential for improving compression rates when compared to compressing the video at full resolution, while achieving higher video quality when compared to compressing the video at reduced resolution.