Tuberculosis Chest X-Ray Image Retrieval System Using Deep Learning Based Biomarker Predictions.

The world health organization's global tuberculosis (TB) report for 2022 identifies TB, with an estimated 1.6 million, as a leading cause of death. The number of new cases has risen since 2020, particularly the number of new drug-resistant cases, estimated at 450,000 in 2021. This is concerning, as treatment of patients with drug resistant TB is complex and may not always be successful. The NIAID TB Portals program is an international consortium with a primary focus on patient centric data collection and analysis for drug resistant TB. The data includes images, their associated radiological findings, clinical records, and socioeconomic information. This work describes a TB Portals' Chest X-ray based image retrieval system which enables precision medicine. An input image is used to retrieve similar images and the associated patient specific information, thus facilitating inspection of outcomes and treatment regimens from comparable patients. Image similarity is defined using clinically relevant biomarkers: gender, age, body mass index (BMI), and the percentage of lung affected per sextant. The biomarkers are predicted using variations of the DenseNet169 convolutional neural network. A multi-task approach is used to predict gender, age and BMI incorporating transfer learning from an initial training on the NIH Clinical Center CXR dataset to the TB portals dataset. The resulting gender AUC, age and BMI mean absolute errors were 0.9854, 4.03years and 1.67kgm2. For the percentage of sextant affected by lesions the mean absolute errors ranged between 7% to 12% with higher error values in the middle and upper sextants which exhibit more variability than the lower sextants. The retrieval system is currently available from https://rap.tbportals.niaid.nih.gov/find_similar_cxr.

Multi-omic profiling of follicular lymphoma reveals changes in tissue architecture and enhanced stromal remodeling in high-risk patients.

Follicular lymphoma (FL) is a generally incurable malignancy that evolves from developmentally blocked germinal center (GC) B cells. To promote survival and immune escape, tumor B cells undergo significant genetic changes and extensively remodel the lymphoid microenvironment. Dynamic interactions between tumor B cells and the tumor microenvironment (TME) are hypothesized to contribute to the broad spectrum of clinical behaviors observed among FL patients. Despite the urgent need, existing clinical tools do not reliably predict disease behavior. Using a multi-modal strategy, we examined cell-intrinsic and -extrinsic factors governing progression and therapeutic outcomes in FL patients enrolled onto a prospective clinical trial. By leveraging the strengths of each platform, we identify several tumor-specific features and microenvironmental patterns enriched in individuals who experience early relapse, the most high-risk FL patients. These features include stromal desmoplasia and changes to the follicular growth pattern present 20 months before first progression and first relapse.

Correction to: SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research.

This paper had published originally without open access, but has since been republished with open access.

SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research.

Modern scientific endeavors increasingly require team collaborations to construct and interpret complex computational workflows. This work describes an image-analysis environment that supports the use of computational tools that facilitate reproducible research and support scientists with varying levels of software development skills. The Jupyter notebook web application is the basis of an environment that enables flexible, well-documented, and reproducible workflows via literate programming. Image-analysis software development is made accessible to scientists with varying levels of programming experience via the use of the SimpleITK toolkit, a simplified interface to the Insight Segmentation and Registration Toolkit. Additional features of the development environment include user friendly data sharing using online data repositories and a testing framework that facilitates code maintenance. SimpleITK provides a large number of examples illustrating educational and research-oriented image analysis workflows for free download from GitHub under an Apache 2.0 license: github.com/InsightSoftwareConsortium/SimpleITK-Notebooks .

Multi-resolution correlative focused ion beam scanning electron microscopy: applications to cell biology.

Efficient correlative imaging of small targets within large fields is a central problem in cell biology. Here, we demonstrate a series of technical advances in focused ion beam scanning electron microscopy (FIB-SEM) to address this issue. We report increases in the speed, robustness and automation of the process, and achieve consistent z slice thickness of ∼3 nm. We introduce "keyframe imaging" as a new approach to simultaneously image large fields of view and obtain high-resolution 3D images of targeted sub-volumes. We demonstrate application of these advances to image post-fusion cytoplasmic intermediates of the HIV core. Using fluorescently labeled cell membranes, proteins and HIV cores, we first produce a "target map" of an HIV infected cell by fluorescence microscopy. We then generate a correlated 3D EM volume of the entire cell as well as high-resolution 3D images of individual HIV cores, achieving correlative imaging across a volume scale of 10(9) in a single automated experimental run.

The Design of SimpleITK.

SimpleITK is a new interface to the Insight Segmentation and Registration Toolkit (ITK) designed to facilitate rapid prototyping, education and scientific activities via high level programming languages. ITK is a templated C++ library of image processing algorithms and frameworks for biomedical and other applications, and it was designed to be generic, flexible and extensible. Initially, ITK provided a direct wrapping interface to languages such as Python and Tcl through the WrapITK system. Unlike WrapITK, which exposed ITK's complex templated interface, SimpleITK was designed to provide an easy to use and simplified interface to ITK's algorithms. It includes procedural methods, hides ITK's demand driven pipeline, and provides a template-less layer. Also SimpleITK provides practical conveniences such as binary distribution packages and overloaded operators. Our user-friendly design goals dictated a departure from the direct interface wrapping approach of WrapITK, toward a new facade class structure that only exposes the required functionality, hiding ITK's extensive template use. Internally SimpleITK utilizes a manual description of each filter with code-generation and advanced C++ meta-programming to provide the higher-level interface, bringing the capabilities of ITK to a wider audience. SimpleITK is licensed as open source software library under the Apache License Version 2.0 and more information about downloading it can be found at http://www.simpleitk.org.

Visualizing cells and humans in 3D: biomedical image analysis at nanometer and meter scales.

Researchers analyzed and presented volume data from the Visible Human Project (VHP) and data from high-resolution 3D ion-abrasion scanning electron microscopy (IA-SEM). They acquired the VHP data using cryosectioning, a destructive approach to 3D human anatomical imaging resulting in whole-body images with a field of view approaching 2 meters and a minimum resolvable feature size of 300 microns. IA-SEM is a type of block-face imaging microscopy, a destructive approach to microscopic 3D imaging of cells. The field of view of IA-SEM data is on the order of 10 microns (whole cell) with a minimum resolvable feature size of 15 nanometers (single-slice thickness). Despite the difference in subject and scale, the analysis and modeling methods were remarkably similar. They are derived from image processing, computer vision, and computer graphics techniques. Moreover, together we are employing medical illustration, visualization, and rapid prototyping to inform and inspire biomedical science. By combining graphics and biology, we are imaging across nine orders of magnitude of space to better promote public health through research.

Correlative 3D imaging of whole mammalian cells with light and electron microscopy.

We report methodological advances that extend the current capabilities of ion-abrasion scanning electron microscopy (IA-SEM), also known as focused ion beam scanning electron microscopy, a newly emerging technology for high resolution imaging of large biological specimens in 3D. We establish protocols that enable the routine generation of 3D image stacks of entire plastic-embedded mammalian cells by IA-SEM at resolutions of ∼10-20nm at high contrast and with minimal artifacts from the focused ion beam. We build on these advances by describing a detailed approach for carrying out correlative live confocal microscopy and IA-SEM on the same cells. Finally, we demonstrate that by combining correlative imaging with newly developed tools for automated image processing, small 100nm-sized entities such as HIV-1 or gold beads can be localized in SEM image stacks of whole mammalian cells. We anticipate that these methods will add to the arsenal of tools available for investigating mechanisms underlying host-pathogen interactions, and more generally, the 3D subcellular architecture of mammalian cells and tissues.

3D visualization of HIV transfer at the virological synapse between dendritic cells and T cells.

The efficiency of HIV infection is greatly enhanced when the virus is delivered at conjugates between CD4+ T cells and virus-bearing antigen-presenting cells such as macrophages or dendritic cells via specialized structures known as virological synapses. Using ion abrasion SEM, electron tomography, and superresolution light microscopy, we have analyzed the spatial architecture of cell-cell contacts and distribution of HIV virions at virological synapses formed between mature dendritic cells and T cells. We demonstrate the striking envelopment of T cells by sheet-like membrane extensions derived from mature dendritic cells, resulting in a shielded region for formation of virological synapses. Within the synapse, filopodial extensions emanating from CD4+ T cells make contact with HIV virions sequestered deep within a 3D network of surface-accessible compartments in the dendritic cell. Viruses are detected at the membrane surfaces of both dendritic cells and T cells, but virions are not released passively at the synapse; instead, virus transfer requires the engagement of T-cell CD4 receptors. The relative seclusion of T cells from the extracellular milieu, the burial of the site of HIV transfer, and the receptor-dependent initiation of virion transfer by T cells highlight unique aspects of cell-cell HIV transmission.

Ion-abrasion scanning electron microscopy reveals distorted liver mitochondrial morphology in murine methylmalonic acidemia.

Methylmalonic acidemia is a lethal inborn error of metabolism that causes mitochondrial impairment, multi-organ dysfunction and a shortened lifespan. Previous transmission electron microscope studies of thin sections from normal (Mut(+/+)) and diseased (Mut(-/-)) tissue found that the mitochondria appear to occupy a progressively larger volume of mutant cells with age, becoming megamitochondria. To assess changes in shape and volume of mitochondria resulting from the mutation, we carried out ion-abrasion scanning electron microscopy (IA-SEM), a method for 3D imaging that involves the iterative use of a focused gallium ion beam to abrade the surface of the specimen, and a scanning electron beam to image the newly exposed surface. Using IA-SEM, we show that mitochondria are more convoluted and have a broader distribution of sizes in the mutant tissue. Compared to normal cells, mitochondria from mutant cells have a larger surface-area-to-volume ratio, which can be attributed to their convoluted shape and not to their elongation or reduced volume. The 3D imaging approach and image analysis described here could therefore be useful as a diagnostic tool for the evaluation of disease progression in aberrant cells at resolutions higher than that currently achieved using confocal light microscopy.

Ion-abrasion scanning electron microscopy reveals surface-connected tubular conduits in HIV-infected macrophages.

HIV-1-containing internal compartments are readily detected in images of thin sections from infected cells using conventional transmission electron microscopy, but the origin, connectivity, and 3D distribution of these compartments has remained controversial. Here, we report the 3D distribution of viruses in HIV-1-infected primary human macrophages using cryo-electron tomography and ion-abrasion scanning electron microscopy (IA-SEM), a recently developed approach for nanoscale 3D imaging of whole cells. Using IA-SEM, we show the presence of an extensive network of HIV-1-containing tubular compartments in infected macrophages, with diameters of approximately 150-200 nm, and lengths of up to approximately 5 microm that extend to the cell surface from vesicular compartments that contain assembling HIV-1 virions. These types of surface-connected tubular compartments are not observed in T cells infected with the 29/31 KE Gag-matrix mutant where the virus is targeted to multi-vesicular bodies and released into the extracellular medium. IA-SEM imaging also allows visualization of large sheet-like structures that extend outward from the surfaces of macrophages, which may bend and fold back to allow continual creation of viral compartments and virion-lined channels. This potential mechanism for efficient virus trafficking between the cell surface and interior may represent a subversion of pre-existing vesicular machinery for antigen capture, processing, sequestration, and presentation.