Ion-abrasion scanning electron microscopy reveals surface-connected tubular conduits in HIV-infected macrophages.

HIV-1-containing internal compartments are readily detected in images of thin sections from infected cells using conventional transmission electron microscopy, but the origin, connectivity, and 3D distribution of these compartments has remained controversial. Here, we report the 3D distribution of viruses in HIV-1-infected primary human macrophages using cryo-electron tomography and ion-abrasion scanning electron microscopy (IA-SEM), a recently developed approach for nanoscale 3D imaging of whole cells. Using IA-SEM, we show the presence of an extensive network of HIV-1-containing tubular compartments in infected macrophages, with diameters of approximately 150-200 nm, and lengths of up to approximately 5 microm that extend to the cell surface from vesicular compartments that contain assembling HIV-1 virions. These types of surface-connected tubular compartments are not observed in T cells infected with the 29/31 KE Gag-matrix mutant where the virus is targeted to multi-vesicular bodies and released into the extracellular medium. IA-SEM imaging also allows visualization of large sheet-like structures that extend outward from the surfaces of macrophages, which may bend and fold back to allow continual creation of viral compartments and virion-lined channels. This potential mechanism for efficient virus trafficking between the cell surface and interior may represent a subversion of pre-existing vesicular machinery for antigen capture, processing, sequestration, and presentation.

The National Library of Medicine Pill Image Recognition Challenge: An Initial Report.

In January 2016 the U.S. National Library of Medicine announced a challenge competition calling for the development and discovery of high-quality algorithms and software that rank how well consumer images of prescription pills match reference images of pills in its authoritative RxIMAGE collection. This challenge was motivated by the need to easily identify unknown prescription pills both by healthcare personnel and the general public. Potential benefits of this capability include confirmation of the pill in settings where the documentation and medication have been separated, such as in a disaster or emergency; and confirmation of a pill when the prescribed medication changes from brand to generic, or for any other reason the shape and color of the pill change. The data for the competition consisted of two types of images, high quality macro photographs, reference images, and consumer quality photographs of the quality we expect users of a proposed application to acquire. A training dataset consisting of 2000 reference images and 5000 corresponding consumer quality images acquired from 1000 pills was provided to challenge participants. A second dataset acquired from 1000 pills with similar distributions of shape and color was reserved as a segregated testing set. Challenge submissions were required to produce a ranking of the reference images, given a consumer quality image as input. Determination of the winning teams was done using the mean average precision quality metric, with the three winners obtaining mean average precision scores of 0.27, 0.09, and 0.08. In the retrieval results, the correct image was amongst the top five ranked images 43%, 12%, and 11% of the time, out of 5000 query/consumer images. This is an initial promising step towards development of an NLM software system and application-programming interface facilitating pill identification. The training dataset will continue to be freely available online at: http://pir.nlm.nih.gov/challenge/submission.html.

Engineering and algorithm design for an image processing Api: a technical report on ITK--the Insight Toolkit.

We present the detailed planning and execution of the Insight Toolkit (ITK), an application programmers interface (API) for the segmentation and registration of medical image data. This public resource has been developed through the NLM Visible Human Project, and is in beta test as an open-source software offering under cost-free licensing. The toolkit concentrates on 3D medical data segmentation and registration algorithms, multimodal and multiresolution capabilities, and portable platform independent support for Windows, Linux/Unix systems. This toolkit was built using current practices in software engineering. Specifically, we embraced the concept of generic programming during the development of these tools, working extensively with C++ templates and the freedom and flexibility they allow. Software development tools for distributed consortium-based code development have been created and are also publicly available. We discuss our assumptions, design decisions, and some lessons learned.

The NIH 3D Print Exchange: A Public Resource for Bioscientific and Biomedical 3D Prints.

The National Institutes of Health (NIH) has launched the NIH 3D Print Exchange, an online portal for discovering and creating bioscientifically relevant 3D models suitable for 3D printing, to provide both researchers and educators with a trusted source to discover accurate and informative models. There are a number of online resources for 3D prints, but there is a paucity of scientific models, and the expertise required to generate and validate such models remains a barrier. The NIH 3D Print Exchange fills this gap by providing novel, web-based tools that empower users with the ability to create ready-to-print 3D files from molecular structure data, microscopy image stacks, and computed tomography scan data. The NIH 3D Print Exchange facilitates open data sharing in a community-driven environment, and also includes various interactive features, as well as information and tutorials on 3D modeling software. As the first government-sponsored website dedicated to 3D printing, the NIH 3D Print Exchange is an important step forward to bringing 3D printing to the mainstream for scientific research and education.

A robust method to track colonoscopy videos with non-informative images.

PURPOSE: Continuously, optical and virtual image alignment can significantly supplement the clinical value of colonoscopy. However, the co-alignment process is frequently interrupted by non-informative images. A video tracking framework to continuously track optical colonoscopy images was developed and tested. METHODS: A video tracking framework with immunity to non-informative images was developed with three essential components: temporal volume flow, region flow, and incremental egomotion estimation. Temporal volume flow selects two similar images interrupted by non-informative images; region flow measures large visual motion between selected images; and incremental egomotion processing estimates significant camera motion by decomposing each large visual motion vector into a sequence of small optical flow vectors. The framework was extensively evaluated via phantom and colonoscopy image sequences. We constructed two colon-like phantoms, a straight phantom and a curved phantom, to measure actual colonoscopy motion. RESULTS: In the straight phantom, after 48 frames were excluded, the tracking error was [Formula: see text]3 mm of 16 mm traveled. In the curved phantom, the error was [Formula: see text]4 mm of 23.88 mm traveled after 72 frames were excluded. Through evaluations with clinical sequences, the robustness of the tracking framework was demonstrated on 30 colonoscopy image sequences from 22 different patients. Four specific sequences among these were chosen to illustrate the algorithm's decreased sensitivity to (1) fluid immersion, (2) wall contact, (3) surgery-induced colon deformation, and (4) multiple non-informative image sequences. CONCLUSION: A robust tracking framework for real-time colonoscopy was developed that facilitates continuous alignment of optical and virtual images, immune to non-informative images that enter the video stream. The system was validated in phantom testing and achieved success with clinical image sequences.

An optical flow approach to tracking colonoscopy video.

We can supplement the clinical value of an optical colonoscopy procedure if we can continuously co-align corresponding virtual colonoscopy (from preoperative X-ray CT exam) and optical colonoscopy images. In this work, we demonstrate a computer vision algorithm based on optical flow to compute egomotion from live colonoscopy video, which is then used to navigate and visualize the corresponding patient anatomy from X-ray CT data. The key feature of the algorithm lies in the effective combination of sparse and dense optical flow fields to compute the focus of expansion (FOE); FOE permits independent computation of camera translational and rotational parameters, directly contributing to the algorithm's accuracy and robustness. We performed extensive evaluation via a colon phantom and clinical colonoscopy data. We constructed two colon like phantoms, a straight phantom and a curved phantom to measure actual colonoscopy motion; tracking accuracy was quantitatively evaluated by comparing estimated motion parameters (velocity and displacement) to ground truth. Thirty straight and curved phantom sequences were collected at 10, 15 and 20 mm/s (5 trials at each speed), to simulate typical velocities during colonoscopy procedures. The average error in velocity estimation was within 3 mm/s in both straight and curved phantoms. Displacement error was under 7 mm over a total distance of 287-288 mm in the straight and curved phantoms. Algorithm robustness was successfully demonstrated on 27 optical colonoscopy image sequences from 20 different patients, and spanning 5 different colon segments. Specific sequences among these were chosen to illustrate the algorithm's decreased sensitivity to (1) recording interruptions, (2) errors in colon segmentation, (3) illumination artifacts, (4) presence of fluid, and (5) changes in colon structure, such as deformation, polyp removal, and surgical tool movement during a procedure.

Visualizing cells and humans in 3D: biomedical image analysis at nanometer and meter scales.

Researchers analyzed and presented volume data from the Visible Human Project (VHP) and data from high-resolution 3D ion-abrasion scanning electron microscopy (IA-SEM). They acquired the VHP data using cryosectioning, a destructive approach to 3D human anatomical imaging resulting in whole-body images with a field of view approaching 2 meters and a minimum resolvable feature size of 300 microns. IA-SEM is a type of block-face imaging microscopy, a destructive approach to microscopic 3D imaging of cells. The field of view of IA-SEM data is on the order of 10 microns (whole cell) with a minimum resolvable feature size of 15 nanometers (single-slice thickness). Despite the difference in subject and scale, the analysis and modeling methods were remarkably similar. They are derived from image processing, computer vision, and computer graphics techniques. Moreover, together we are employing medical illustration, visualization, and rapid prototyping to inform and inspire biomedical science. By combining graphics and biology, we are imaging across nine orders of magnitude of space to better promote public health through research.

Region flow: a multi-stage method for colonoscopy tracking.

Co-located optical and virtual colonoscopy images provide important clinical information during routine colonoscopy procedures. Tracking algorithms that rely on image features to align virtual and optical images can fail when they encounter blurry image sequences. This is a common occurrence in colonoscopy images, when the endoscope touches a wall or is immersed in fluid. We propose a region-flow based matching algorithm to determine the large changes between images that bridge such interruptions in the visual field. The region flow field is used as the means to limit the search space for computing corresponding feature points; a sequence of refining steps is performed to identify the most reliable and accurate feature point pairs. The feature point pairs are then used in a deformation based scheme to compute the final camera parameters. We have successfully tested this algorithm on four clinical colonoscopy image sequences containing anywhere from 9-57 consecutive blurry images. Two additional tabletop experiments were performed to quantitatively validate the algorithm: the endoscope was moved along a slightly curved path by 24 mm and along a straight path by 40 mm. Our method reported errors within 1-5% in these experiments.

Visualization of semantic indexing similarity over MeSH.

We present an interactive visualization system for the evaluation of indexing results of the MEDLINE data-base over the Medical Subject Headings (MeSH) structure in a graphical radial-tree layout. It displays indexing similarity measurements with 2D color coding and a 3D height field permitting the evaluation of the automatic Medical Text Indexer (MTI), compared with human indexers.

The Visible Human data sets (VHD) and Insight Toolkit (ITk): experiments in open source software.

From its inception in 1989, the Visible Human Project was designed as an experiment in open source software. In 1994 and 1995 the male and female Visible Human data sets were released by the National Library of Medicine (NLM) as open source data sets. In 2002 the NLM released the first version of the Insight Toolkit (ITk) as open source software.