Deep Learning - Methods to Amplify Epidemiological Data Collection and Analyses.

Deep learning is a subfield of artificial intelligence and machine learning based mostly on neural networks and often combined with attention algorithms that has been used to detect and identify objects in text, audio, images, and video. Serghiou and Rough (Am J Epidemiol. 0000;000(00):0000-0000) present a primer for epidemiologists on deep learning models. These models provide substantial opportunities for epidemiologists to expand and amplify their research in both data collection and analyses by increasing the geographic reach of studies, including more research subjects, and working with large or high dimensional data. The tools for implementing deep learning methods are not quite yet as straightforward or ubiquitous for epidemiologists as traditional regression methods found in standard statistical software, but there are exciting opportunities for interdisciplinary collaboration with deep learning experts, just as epidemiologists have with statisticians, healthcare providers, urban planners, and other professionals. Despite the novelty of these methods, epidemiological principles of assessing bias, study design, interpretation and others still apply when implementing deep learning methods or assessing the findings of studies that have used them.

Road Traffic Injuries and the Built Environment in Bogotá, Colombia, 2015-2019: A Cross-Sectional Analysis.

Nine in 10 road traffic deaths occur in low- and middle-income countries (LMICs). Despite this disproportionate burden, few studies have examined built environment correlates of road traffic injury in these settings, including in Latin America. We examined road traffic collisions in Bogotá, Colombia, occurring between 2015 and 2019, and assessed the association between neighborhood-level built environment features and pedestrian injury and death. We used descriptive statistics to characterize all police-reported road traffic collisions that occurred in Bogotá between 2015 and 2019. Cluster detection was used to identify spatial clustering of pedestrian collisions. Adjusted multivariate Poisson regression models were fit to examine associations between several neighborhood-built environment features and rate of pedestrian road traffic injury and death. A total of 173,443 police-reported traffic collisions occurred in Bogotá between 2015 and 2019. Pedestrians made up about 25% of road traffic injuries and 50% of road traffic deaths in Bogotá between 2015 and 2019. Pedestrian collisions were spatially clustered in the southwestern region of Bogotá. Neighborhoods with more street trees (RR, 0.90; 95% CI, 0.82-0.98), traffic signals (0.89, 0.81-0.99), and bus stops (0.89, 0.82-0.97) were associated with lower pedestrian road traffic deaths. Neighborhoods with greater density of large roads were associated with higher pedestrian injury. Our findings highlight the potential for pedestrian-friendly infrastructure to promote safer interactions between pedestrians and motorists in Bogotá and in similar urban contexts globally.

3D Visualization, Skeletonization and Branching Analysis of Blood Vessels in Angiogenesis.

Angiogenesis is the process of new blood vessels growing from existing vasculature. Visualizing them as a three-dimensional (3D) model is a challenging, yet relevant, task as it would be of great help to researchers, pathologists, and medical doctors. A branching analysis on the 3D model would further facilitate research and diagnostic purposes. In this paper, a pipeline of vision algorithms is elaborated to visualize and analyze blood vessels in 3D from formalin-fixed paraffin-embedded (FFPE) granulation tissue sections with two different staining methods. First, a U-net neural network is used to segment blood vessels from the tissues. Second, image registration is used to align the consecutive images. Coarse registration using an image-intensity optimization technique, followed by finetuning using a neural network based on Spatial Transformers, results in an excellent alignment of images. Lastly, the corresponding segmented masks depicting the blood vessels are aligned and interpolated using the results of the image registration, resulting in a visualized 3D model. Additionally, a skeletonization algorithm is used to analyze the branching characteristics of the 3D vascular model. In summary, computer vision and deep learning is used to reconstruct, visualize and analyze a 3D vascular model from a set of parallel tissue samples. Our technique opens innovative perspectives in the pathophysiological understanding of vascular morphogenesis under different pathophysiological conditions and its potential diagnostic role.

Extra-articular shoulder resections: outcomes of 54 patients.

BACKGROUND: The survival of patients with tumors around the shoulder treated with extra-articular resection, the rates of reconstructions-related complications, and the function of the shoulder cannot be estimated because of limited available data from mainly small published related series and case reports. METHODS: We studied 54 patients with tumors around the shoulder treated with extra-articular shoulder resections and proximal humeral megaprosthetic reconstructions from 1985 to 2012. Mean tumor volume was 549 cm3, and the mean length of the proximal humeral resection was 110 mm. Mean follow-up was 7.8 years (range, 3-21 years). We evaluated the outcomes (survival, metastases, recurrences, and function) and the survival and complications of the reconstruction. RESULTS: Survival of patients with malignant tumors was 47%, 38%, and 35%, at 5, 10, and 20 years, respectively. Rates for metastasis and local recurrence were 60% and 18.5%, respectively. Survival was significantly higher for patients without metastases at diagnosis, tumor volume <549 cm3, and type IV resections. Survival of reconstructions was 56% at 10 years and 48% 20 years. Overall, 19 patients (35.2%) experienced 30 complications (55.5%), the most common being soft tissue failures that required subsequent surgery without, however, implant removal. The mean Musculoskeletal Tumour Society score was 25 points, without any significant difference between the types of extra-articular resections. CONCLUSION: Tumor stage and volume as well as type of resection are important predictors of survival of patients with malignant tumors around the shoulder. Survival of the reconstructions is satisfactory; nevertheless, the complication rate is high. The Musculoskeletal Tumour Society score is similar with respect to the type of resection.

Automated particle correspondence and accurate tilt-axis detection in tilted-image pairs.

Tilted electron microscope images are routinely collected for an ab initio structure reconstruction as a part of the Random Conical Tilt (RCT) or Orthogonal Tilt Reconstruction (OTR) methods, as well as for various applications using the "free-hand" procedure. These procedures all require identification of particle pairs in two corresponding images as well as accurate estimation of the tilt-axis used to rotate the electron microscope (EM) grid. Here we present a computational approach, PCT (particle correspondence from tilted pairs), based on tilt-invariant context and projection matching that addresses both problems. The method benefits from treating the two problems as a single optimization task. It automatically finds corresponding particle pairs and accurately computes tilt-axis direction even in the cases when EM grid is not perfectly planar.

Grade III chondrosarcoma of the left iliopubic branch: A case report.

INTRODUCTION AND IMPORTANCE: Chondrosarcomas are the third most frequent malignant bone tumors. With pelvic bones being their most common primary location, diagnosis and treatment of these tumors is especially challenging due to the diverse clinical manifestations and involvement of critical anatomic structures. We present the case of a grade III pelvic chondrosarcoma of the left iliopubic branch managed through a multidisciplinary approach. CASE PRESENTATION: A 26-year-old male patient presented with a 1-year history of a mass in the left iliopubic branch. The imaging findings suggested chondrosarcoma and showed extrinsic compression of pelvic structures causing right hydronephrosis, marked elongation and tortuosity of the sigmoid colon, and anterior and superior displacement of the bladder. Following multidisciplinary meeting it was decided to perform a left hemicolectomy, colostomy, and internal hemipelvectomy in the 1-2-3 left zones, with resection of the intrapelvic and intra-abdominal tumor, and preservation of the left lower extremity. The patient presented two episodes of intestinal obstruction, which resolved with medical management. Was discharged without presenting further complications. CLINICAL DISCUSSION: Chondrosarcomas management demands a methodical approach. Appropriate surgical strategy requires individualization according to the characteristics of the lesion and the degree of involvement of surrounding structures. Complete resection of the tumor and preservation of the lower extremity function are critical achievements. CONCLUSION: This case underscores the effective management of a challenging tumor such as pelvic chondrosarcoma. The multidisciplinary approach and collaboration of several specialties was crucial to reach an appropriate surgical strategy.

Nuclei Detection and Segmentation of Histopathological Images Using a Feature Pyramidal Network Variant of a Mask R-CNN.

Cell nuclei interpretation is crucial in pathological diagnostics, especially in tumor specimens. A critical step in computational pathology is to detect and analyze individual nuclear properties using segmentation algorithms. Conventionally, a semantic segmentation network is used, where individual nuclear properties are derived after post-processing a segmentation mask. In this study, we focus on showing that an object-detection-based instance segmentation network, the Mask R-CNN, after integrating it with a Feature Pyramidal Network (FPN), gives mature and reliable results for nuclei detection without the need for additional post-processing. The results were analyzed using the Kumar dataset, a public dataset with over 20,000 nuclei annotations from various organs. The dice score of the baseline Mask R-CNN improved from 76% to 83% after integration with an FPN. This was comparable with the 82.6% dice score achieved by modern semantic-segmentation-based networks. Thus, evidence is provided that an end-to-end trainable detection-based instance segmentation algorithm with minimal post-processing steps can reliably be used for the detection and analysis of individual nuclear properties. This represents a relevant task for research and diagnosis in digital pathology, which can improve the automated analysis of histopathological images.

Multi-Center Fetal Brain Tissue Annotation (FeTA) Challenge 2022 Results.

Segmentation is a critical step in analyzing the developing human fetal brain. There have been vast improvements in automatic segmentation methods in the past several years, and the Fetal Brain Tissue Annotation (FeTA) Challenge 2021 helped to establish an excellent standard of fetal brain segmentation. However, FeTA 2021 was a single center study, limiting real-world clinical applicability and acceptance. The multi-center FeTA Challenge 2022 focused on advancing the generalizability of fetal brain segmentation algorithms for magnetic resonance imaging (MRI). In FeTA 2022, the training dataset contained images and corresponding manually annotated multi-class labels from two imaging centers, and the testing data contained images from these two centers as well as two additional unseen centers. The multi-center data included different MR scanners, imaging parameters, and fetal brain super-resolution algorithms applied. 16 teams participated and 17 algorithms were evaluated. Here, the challenge results are presented, focusing on the generalizability of the submissions. Both in- and out-of-domain, the white matter and ventricles were segmented with the highest accuracy (Top Dice scores: 0.89, 0.87 respectively), while the most challenging structure remains the grey matter (Top Dice score: 0.75) due to anatomical complexity. The top 5 average Dices scores ranged from 0.81-0.82, the top 5 average 95th percentile Hausdorff distance values ranged from 2.3-2.5mm, and the top 5 volumetric similarity scores ranged from 0.90-0.92. The FeTA Challenge 2022 was able to successfully evaluate and advance generalizability of multi-class fetal brain tissue segmentation algorithms for MRI and it continues to benchmark new algorithms.

Optimal and fast rotational alignment of volumes with missing data in Fourier space.

Electron tomography of intact cells has the potential to reveal the entire cellular content at a resolution corresponding to individual macromolecular complexes. Characterization of macromolecular complexes in tomograms is nevertheless an extremely challenging task due to the high level of noise, and due to the limited tilt angle that results in missing data in Fourier space. By identifying particles of the same type and averaging their 3D volumes, it is possible to obtain a structure at a more useful resolution for biological interpretation. Currently, classification and averaging of sub-tomograms is limited by the speed of computational methods that optimize alignment between two sub-tomographic volumes. The alignment optimization is hampered by the fact that the missing data in Fourier space has to be taken into account during the rotational search. A similar problem appears in single particle electron microscopy where the random conical tilt procedure may require averaging of volumes with a missing cone in Fourier space. We present a fast implementation of a method guaranteed to find an optimal rotational alignment that maximizes the constrained cross-correlation function (cCCF) computed over the actual overlap of data in Fourier space.

PiGLET: Pixel-Level Grounding of Language Expressions With Transformers.

This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework for the study of this new task, including new ground truth and metrics. We propose PiGLET, a novel multi-modal Transformer architecture to tackle the Panoptic Narrative Grounding task, and to serve as a stepping stone for future work. We exploit the intrinsic semantic richness in an image by including panoptic categories, and we approach visual grounding at a fine-grained level using segmentations. In terms of ground truth, we propose an algorithm to automatically transfer Localized Narratives annotations to specific regions in the panoptic segmentations of the MS COCO dataset. PiGLET achieves a performance of 63.2 absolute Average Recall points. By leveraging the rich language information on the Panoptic Narrative Grounding benchmark on MS COCO, PiGLET obtains an improvement of 0.4 Panoptic Quality points over its base method on the panoptic segmentation task. Finally, we demonstrate the generalizability of our method to other natural language visual grounding problems such as Referring Expression Segmentation. PiGLET is competitive with previous state-of-the-art in RefCOCO, RefCOCO+ and RefCOCOg.

STRIDE: Street View-based Environmental Feature Detection and Pedestrian Collision Prediction.

This paper introduces a novel benchmark to study the impact and relationship of built environment elements on pedestrian collision prediction, intending to enhance environmental awareness in autonomous driving systems to prevent pedestrian injuries actively. We introduce a built environment detection task in large-scale panoramic images and a detection-based pedestrian collision frequency prediction task. We propose a baseline method that incorporates a collision prediction module into a state-of-the-art detection model to tackle both tasks simultaneously. Our experiments demonstrate a significant correlation between object detection of built environment elements and pedestrian collision frequency prediction. Our results are a stepping stone towards understanding the interdependencies between built environment conditions and pedestrian safety.

Electron microscopy of biotinylated protein complexes bound to streptavidin monolayer crystals.

Chemical biotinylation of protein complexes followed by binding to two-dimensional (monolayer) crystals of streptavidin is shown to be an effective way to prepare cryo-EM specimens from samples at low protein concentration. Three different multiprotein complexes are used to demonstrate the generality of this method. In addition, native thermosomes, purified from Sulfolobus solfataricus P2, are used to demonstrate that a uniform distribution of Euler angles is produced, even though this particle is known to adopt a preferred orientation when other methods of cryo-EM specimen preparation are used.

Rational Discovery of Antimicrobial Peptides by Means of Artificial Intelligence.

Antibiotic resistance is a worldwide public health problem due to the costs and mortality rates it generates. However, the large pharmaceutical industries have stopped searching for new antibiotics because of their low profitability, given the rapid replacement rates imposed by the increasingly observed resistance acquired by microorganisms. Alternatively, antimicrobial peptides (AMPs) have emerged as potent molecules with a much lower rate of resistance generation. The discovery of these peptides is carried out through extensive in vitro screenings of either rational or non-rational libraries. These processes are tedious and expensive and generate only a few AMP candidates, most of which fail to show the required activity and physicochemical properties for practical applications. This work proposes implementing an artificial intelligence algorithm to reduce the required experimentation and increase the efficiency of high-activity AMP discovery. Our deep learning (DL) model, called AMPs-Net, outperforms the state-of-the-art method by 8.8% in average precision. Furthermore, it is highly accurate to predict the antibacterial and antiviral capacity of a large number of AMPs. Our search led to identifying two unreported antimicrobial motifs and two novel antimicrobial peptides related to them. Moreover, by coupling DL with molecular dynamics (MD) simulations, we were able to find a multifunctional peptide with promising therapeutic effects. Our work validates our previously proposed pipeline for a more efficient rational discovery of novel AMPs.

Predicting target-ligand interactions with graph convolutional networks for interpretable pharmaceutical discovery.

Drug Discovery is an active research area that demands great investments and generates low returns due to its inherent complexity and great costs. To identify potential therapeutic candidates more effectively, we propose protein-ligand with adversarial augmentations network (PLA-Net), a deep learning-based approach to predict target-ligand interactions. PLA-Net consists of a two-module deep graph convolutional network that considers ligands' and targets' most relevant chemical information, successfully combining them to find their binding capability. Moreover, we generate adversarial data augmentations that preserve relevant biological backgrounds and improve the interpretability of our model, highlighting the relevant substructures of the ligands reported to interact with the protein targets. Our experiments demonstrate that the joint ligand-target information and the adversarial augmentations significantly increase the interaction prediction performance. PLA-Net achieves 86.52% in mean average precision for 102 target proteins with perfect performance for 30 of them, in a curated version of actives as decoys dataset. Lastly, we accurately predict pharmacologically-relevant molecules when screening the ligands of ChEMBL and drug repurposing Hub datasets with the perfect-scoring targets.

Smart pooling: AI-powered COVID-19 informative group testing.

Massive molecular testing for COVID-19 has been pointed out as fundamental to moderate the spread of the pandemic. Pooling methods can enhance testing efficiency, but they are viable only at low incidences of the disease. We propose Smart Pooling, a machine learning method that uses clinical and sociodemographic data from patients to increase the efficiency of informed Dorfman testing for COVID-19 by arranging samples into all-negative pools. To do this, we ran an automated method to train numerous machine learning models on a retrospective dataset from more than 8000 patients tested for SARS-CoV-2 from April to July 2020 in Bogotá, Colombia. We estimated the efficiency gains of using the predictor to support Dorfman testing by simulating the outcome of tests. We also computed the attainable efficiency gains of non-adaptive pooling schemes mathematically. Moreover, we measured the false-negative error rates in detecting the ORF1ab and N genes of the virus in RT-qPCR dilutions. Finally, we presented the efficiency gains of using our proposed pooling scheme on proof-of-concept pooled tests. We believe Smart Pooling will be efficient for optimizing massive testing of SARS-CoV-2.

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results.

Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at https://github.com/RagMeh11/QU-BraTS.

The Medical Segmentation Decathlon.

International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.

Experimental evaluation of support vector machine-based and correlation-based approaches to automatic particle selection.

The goal of this study is to evaluate the performance of software for automated particle-boxing, and in particular the performance of a new tool (TextonSVM) that recognizes the characteristic texture of particles of interest. As part of a high-throughput protocol, we use human editing that is based solely on class-average images to create final data sets that are enriched in what the investigator considers to be true-positive particles. The Fourier shell correlation (FSC) function is then used to characterize the homogeneity of different single-particle data sets that are derived from the same micrographs by two or more alternative methods. We find that the homogeneity is generally quite similar for class-edited data sets obtained by the texture-based method and by SIGNATURE, a cross-correlation-based method. The precision-recall characteristics of the texture-based method are, on the other hand, significantly better than those of the cross-correlation based method; that is to say, the texture-based approach produces a smaller fraction of false positives in the initial set of candidate particles. The computational efficiency of the two approaches is generally within a factor of two of one another. In situations when it is helpful to use a larger number of templates (exemplars), however, TextonSVM scales in a much more efficient way than do boxing programs that are based on localized cross-correlation.

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks.

The discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules' target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 97.7% in the Receiver Operating Characteristic curve (ROC-AUC) and 65.5% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained three (3) potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

MIcro-surgical anastomose workflow recognition challenge report.

BACKGROUND AND OBJECTIVE: Automatic surgical workflow recognition is an essential step in developing context-aware computer-assisted surgical systems. Video recordings of surgeries are becoming widely accessible, as the operational field view is captured during laparoscopic surgeries. Head and ceiling mounted cameras are also increasingly being used to record videos in open surgeries. This makes videos a common choice in surgical workflow recognition. Additional modalities, such as kinematic data captured during robot-assisted surgeries, could also improve workflow recognition. This paper presents the design and results of the MIcro-Surgical Anastomose Workflow recognition on training sessions (MISAW) challenge whose objective was to develop workflow recognition models based on kinematic data and/or videos. METHODS: The MISAW challenge provided a data set of 27 sequences of micro-surgical anastomosis on artificial blood vessels. This data set was composed of videos, kinematics, and workflow annotations. The latter described the sequences at three different granularity levels: phase, step, and activity. Four tasks were proposed to the participants: three of them were related to the recognition of surgical workflow at three different granularity levels, while the last one addressed the recognition of all granularity levels in the same model. We used the average application-dependent balanced accuracy (AD-Accuracy) as the evaluation metric. This takes unbalanced classes into account and it is more clinically relevant than a frame-by-frame score. RESULTS: Six teams participated in at least one task. All models employed deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), or a combination of both. The best models achieved accuracy above 95%, 80%, 60%, and 75% respectively for recognition of phases, steps, activities, and multi-granularity. The RNN-based models outperformed the CNN-based ones as well as the dedicated modality models compared to the multi-granularity except for activity recognition. CONCLUSION: For high levels of granularity, the best models had a recognition rate that may be sufficient for applications such as prediction of remaining surgical time. However, for activities, the recognition rate was still low for applications that can be employed clinically. The MISAW data set is publicly available at http://www.synapse.org/MISAW to encourage further research in surgical workflow recognition.