Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography (MSOT) imaging.

Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through-plane resolution volumetric MSOT is time-consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross-sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG-conjugated nanoworms particles (NWs-ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.

Super-resolution and segmentation deep learning for breast cancer histopathology image analysis.

Traditionally, a high-performance microscope with a large numerical aperture is required to acquire high-resolution images. However, the images' size is typically tremendous. Therefore, they are not conveniently managed and transferred across a computer network or stored in a limited computer storage system. As a result, image compression is commonly used to reduce image size resulting in poor image resolution. Here, we demonstrate custom convolution neural networks (CNNs) for both super-resolution image enhancement from low-resolution images and characterization of both cells and nuclei from hematoxylin and eosin (H&E) stained breast cancer histopathological images by using a combination of generator and discriminator networks so-called super-resolution generative adversarial network-based on aggregated residual transformation (SRGAN-ResNeXt) to facilitate cancer diagnosis in low resource settings. The results provide high enhancement in image quality where the peak signal-to-noise ratio and structural similarity of our network results are over 30 dB and 0.93, respectively. The derived performance is superior to the results obtained from both the bicubic interpolation and the well-known SRGAN deep-learning methods. In addition, another custom CNN is used to perform image segmentation from the generated high-resolution breast cancer images derived with our model with an average Intersection over Union of 0.869 and an average dice similarity coefficient of 0.893 for the H&E image segmentation results. Finally, we propose the jointly trained SRGAN-ResNeXt and Inception U-net Models, which applied the weights from the individually trained SRGAN-ResNeXt and inception U-net models as the pre-trained weights for transfer learning. The jointly trained model's results are progressively improved and promising. We anticipate these custom CNNs can help resolve the inaccessibility of advanced microscopes or whole slide imaging (WSI) systems to acquire high-resolution images from low-performance microscopes located in remote-constraint settings.

Bioengineering horizon scan 2020.

Horizon scanning is intended to identify the opportunities and threats associated with technological, regulatory and social change. In 2017 some of the present authors conducted a horizon scan for bioengineering (Wintle et al., 2017). Here we report the results of a new horizon scan that is based on inputs from a larger and more international group of 38 participants. The final list of 20 issues includes topics spanning from the political (the regulation of genomic data, increased philanthropic funding and malicious uses of neurochemicals) to the environmental (crops for changing climates and agricultural gene drives). The early identification of such issues is relevant to researchers, policy-makers and the wider public.

Metalens-Based Miniaturized Optical Systems.

Metasurfaces have been studied and widely applied to optical systems. A metasurface-based flat lens (metalens) holds promise in wave-front engineering for multiple applications. The metalens has become a breakthrough technology for miniaturized optical system development, due to its outstanding characteristics, such as ultrathinness and cost-effectiveness. Compared to conventional macro- or meso-scale optics manufacturing methods, the micro-machining process for metalenses is relatively straightforward and more suitable for mass production. Due to their remarkable abilities and superior optical performance, metalenses in refractive or diffractive mode could potentially replace traditional optics. In this review, we give a brief overview of the most recent studies on metalenses and their applications with a specific focus on miniaturized optical imaging and sensing systems. We discuss approaches for overcoming technical challenges in the bio-optics field, including a large field of view (FOV), chromatic aberration, and high-resolution imaging.

MEMS Actuators for Optical Microendoscopy.

Growing demands for affordable, portable, and reliable optical microendoscopic imaging devices are attracting research institutes and industries to find new manufacturing methods. However, the integration of microscopic components into these subsystems is one of today's challenges in manufacturing and packaging. Together with this kind of miniaturization more and more functional parts have to be accommodated in ever smaller spaces. Therefore, solving this challenge with the use of microelectromechanical systems (MEMS) fabrication technology has opened the promising opportunities in enabling a wide variety of novel optical microendoscopy to be miniaturized. MEMS fabrication technology enables abilities to apply batch fabrication methods with high-precision and to include a wide variety of optical functionalities to the optical components. As a result, MEMS technology has enabled greater accessibility to advance optical microendoscopy technology to provide high-resolution and high-performance imaging matching with traditional table-top microscopy. In this review the latest advancements of MEMS actuators for optical microendoscopy will be discussed in detail.

Novel p16 binding peptide development for p16-overexpressing cancer cell detection using phage display.

Protein p(16INK4a) (p16) is a well-known biomarker for diagnosis of human papillomavirus (HPV) related cancers. In this work, we identify novel p16 binding peptides by using phage display selection method. A random heptamer phage display library was screened on purified recombinant p16 protein-coated plates to elute only the bound phages from p16 surfaces. Binding affinity of the bound phages was compared with each other by enzyme-linked immunosorbent assay (ELISA), fluorescence imaging technique, and bioinformatic computations. Binding specificity and binding selectivity of the best candidate phage-displayed p16 binding peptide were evaluated by peptide blocking experiment in competition with p16 monoclonal antibody and fluorescence imaging technique, respectively. Five candidate phage-displayed peptides were isolated from the phage display selection method. All candidate p16 binding phages show better binding affinity than wild-type phage in ELISA test, but only three of them can discriminate p16-overexpressing cancer cell, CaSki, from normal uterine fibroblast cell, HUF, with relative fluorescence intensities from 2.6 to 4.2-fold greater than those of wild-type phage. Bioinformatic results indicate that peptide 'Ser-His-Ser-Leu-Leu-Ser-Ser' binds to p16 molecule with the best binding score and does not interfere with the common protein functions of p16. Peptide blocking experiment shows that the phage-displayed peptide 'Ser-His-Ser-Leu-Leu-Ser-Ser' can conceal p16 from monoclonal antibody interaction. This phage clone also selectively interacts with the p16 positive cell lines, and thus, it can be applied for p16-overexpressing cell detection.

Advances in imaging probes and optical microendoscopic imaging techniques for early in vivo cancer assessment.

A new chapter in the history of medical diagnosis happened when the first X-ray technology was invented in the late 1800s. Since then, many non-invasive and minimally invasive imaging techniques have been invented for clinical diagnosis to research in cellular biology, drug discovery, and disease monitoring. These imaging modalities have leveraged the benefits of significant advances in computer, electronics, and information technology and, more recently, targeted molecular imaging. The development of targeted contrast agents such as fluorescent and nanoparticle probes coupled with optical imaging techniques has made it possible to selectively view specific biological events and processes in both in vivo and ex vivo systems with great sensitivity and selectivity. Thus, the combination of targeted molecular imaging probes and optical imaging techniques have become a mainstay in modern medicinal and biological research. Many promising results have demonstrated great potentials to translate to clinical applications. In this review, we describe a discussion of employing imaging probes and optical microendoscopic imaging techniques for cancer diagnosis.

In vivo near-infrared dual-axis confocal microendoscopy in the human lower gastrointestinal tract.

Near-infrared confocal microendoscopy is a promising technique for deep in vivo imaging of tissues and can generate high-resolution cross-sectional images at the micron-scale. We demonstrate the use of a dual-axis confocal (DAC) near-infrared fluorescence microendoscope with a 5.5-mm outer diameter for obtaining clinical images of human colorectal mucosa. High-speed two-dimensional en face scanning was achieved through a microelectromechanical systems (MEMS) scanner while a micromotor was used for adjusting the axial focus. In vivo images of human patients are collected at 5 frames/sec with a field of view of 362×212 µm(2) and a maximum imaging depth of 140 µm. During routine endoscopy, indocyanine green (ICG) was topically applied a nonspecific optical contrasting agent to regions of the human colon. The DAC microendoscope was then used to obtain microanatomic images of the mucosa by detecting near-infrared fluorescence from ICG. These results suggest that DAC microendoscopy may have utility for visualizing the anatomical and, perhaps, functional changes associated with colorectal pathology for the early detection of colorectal cancer.

Optical design and imaging performance testing of a 9.6-mm diameter femtosecond laser microsurgery probe.

We present the optical design of a 9.6-mm diameter fiber-coupled probe for combined femtosecond laser microsurgery and nonlinear optical imaging. Towards enabling clinical use, we successfully reduced the dimensions of our earlier 18-mm microsurgery probe by half, while improving optical performance. We use analytical and computational models to optimize the miniaturized lens system for off-axis scanning aberrations. The optimization reveals that the optical system can be aberration-corrected using simple aspheric relay lenses to achieve diffraction-limited imaging resolution over a large field of view. Before moving forward with custom lenses, we have constructed the 9.6-mm probe using off-the-shelf spherical relay lenses and a 0.55 NA aspheric objective lens. In addition to reducing the diameter by nearly 50% and the total volume by 5 times, we also demonstrate improved lateral and axial resolutions of 1.27 µm and 13.5 µm, respectively, compared to 1.64 µm and 16.4 µm in our previous work. Using this probe, we can successfully image various tissue samples, such as rat tail tendon that required 2-3 × lower laser power than the current state-of-the-art. With further development, image-guided, femtosecond laser microsurgical probes such as this one can enable physicians to achieve the highest level of surgical precision anywhere inside the body.

In vivo micro-image mosaicing.

Recent advances in optical imaging have led to the development of miniature microscopes that can be brought to the patient for visualizing tissue structures in vivo. These devices have the potential to revolutionize health care by replacing tissue biopsy with in vivo pathology. One of the primary limitations of these microscopes, however, is that the constrained field of view can make image interpretation and navigation difficult. In this paper, we show that image mosaicing can be a powerful tool for widening the field of view and creating image maps of microanatomical structures. First, we present an efficient algorithm for pairwise image mosaicing that can be implemented in real time. Then, we address two of the main challenges associated with image mosaicing in medical applications: cumulative image registration errors and scene deformation. To deal with cumulative errors, we present a global alignment algorithm that draws upon techniques commonly used in probabilistic robotics. To accommodate scene deformation, we present a local alignment algorithm that incorporates deformable surface models into the mosaicing framework. These algorithms are demonstrated on image sequences acquired in vivo with various imaging devices including a hand-held dual-axes confocal microscope, a miniature two-photon microscope, and a commercially available confocal microendoscope.

In vivo imaging of human and mouse skin with a handheld dual-axis confocal fluorescence microscope.

Advancing molecular therapies for the treatment of skin diseases will require the development of new tools that can reveal spatiotemporal changes in the microanatomy of the skin and associate these changes with the presence of the therapeutic agent. For this purpose, we evaluated a handheld dual-axis confocal (DAC) microscope that is capable of in vivo fluorescence imaging of skin, using both mouse models and human skin. Individual keratinocytes in the epidermis were observed in three-dimensional image stacks after topical administration of near-infrared (NIR) dyes as contrast agents. This suggested that the DAC microscope may have utility in assessing the clinical effects of a small interfering RNA (siRNA)-based therapeutic (TD101) that targets the causative mutation in pachyonychia congenita (PC) patients. The data indicated that (1) formulated indocyanine green (ICG) readily penetrated hyperkeratotic PC skin and normal callused regions compared with nonaffected areas, and (2) TD101-treated PC skin revealed changes in tissue morphology, consistent with reversion to nonaffected skin compared with vehicle-treated skin. In addition, siRNA was conjugated to NIR dye and shown to penetrate through the stratum corneum barrier when topically applied to mouse skin. These results suggest that in vivo confocal microscopy may provide an informative clinical end point to evaluate the efficacy of experimental molecular therapeutics.

Micromirror-scanned dual-axis confocal microscope utilizing a gradient-index relay lens for image guidance during brain surgery.

A fluorescence confocal microscope incorporating a 1.8-mm-diam gradient-index relay lens is developed for in vivo histological guidance during resection of brain tumors. The microscope utilizes a dual-axis confocal architecture to efficiently reject out-of-focus light for high-contrast optical sectioning. A biaxial microelectromechanical system (MEMS) scanning mirror is actuated at resonance along each axis to achieve a large field of view with low-voltage waveforms. The unstable Lissajous scan, which results from actuating the orthogonal axes of the MEMS mirror at highly disparate resonance frequencies, is optimized to fully sample 500x500 pixels at two frames per second. Optically sectioned fluorescence images of brain tissues are obtained in living mice to demonstrate the utility of this microscope for image-guided resections.

MEMS-Based Dual Axes Confocal Microendoscopy.

We demonstrate a miniature, near-infrared microscope (λ = 785 nm) that uses a novel dual axes confocal architecture. Scalability is achieved with post-objective scanning, and a MEMS mirror provides real time (>4 Hz) in vivo imaging. This instrument can achieve sub-cellular resolution with deep tissue penetration and large field of view. An endoscope-compatible version can image digestive tract epithelium to guide tissue biopsy and monitor therapy.

In vivo brain imaging using a portable 2.9 g two-photon microscope based on a microelectromechanical systems scanning mirror.

We present a two-photon microscope that is approximately 2.9 g in mass and 2.0 x 1.9 x 1.1 cm(3) in size and based on a microelectromechanical systems (MEMS) laser-scanning mirror. The microscope has a focusing motor and a micro-optical assembly composed of four gradient refractive index lenses and a dichroic microprism. Fluorescence is captured without the detected emissions reflecting off the MEMS mirror, by use of separate optical fibers for fluorescence collection and delivery of ultrashort excitation pulses. Using this microscope we imaged neocortical microvasculature and tracked the flow of erythrocytes in live mice.

Three-dimensional in vivo imaging by a handheld dual-axes confocal microscope.

We present a handheld dual-axes confocal microscope that is based on a two-dimensional microelectromechanical systems (MEMS) scanner. It performs reflectance and fluorescence imaging at 488 nm wavelength, with three-dimensional imaging capability. The fully packaged microscope has a diameter of 10 mm and acquires images at 4 Hz frame rate with a maximum field of view of 400 microm x 260 microm. The transverse and axial resolutions of the handheld probe are 1.7 microm and 5.8 microm, respectively. Capability to perform real time small animal imaging is demonstrated in vivo in transgenic mice.

Miniaturized probe for femtosecond laser microsurgery and two-photon imaging.

Combined two-photon fluorescence microscopy and femtosecond laser microsurgery has many potential biomedical applications as a powerful "seek-and-treat" tool. Towards developing such a tool, we demonstrate a miniaturized probe which combines these techniques in a compact housing. The device is 10 x 15 x 40 mm(3) in size and uses an aircore photonic crystal fiber to deliver femtosecond laser pulses at 80 MHz repetition rate for imaging and 1 kHz for microsurgery. A fast two-axis microelectromechanical system scanning mirror is driven at resonance to produce Lissajous beam scanning at 10 frames per second. Field of view is 310 microm in diameter and the lateral and axial resolutions are 1.64 microm and 16.4 microm, respectively. Combined imaging and microsurgery is demonstrated using live cancer cells.

Miniature near-infrared dual-axes confocal microscope utilizing a two-dimensional microelectromechanical systems scanner.

The first, to our knowledge, miniature dual-axes confocal microscope has been developed, with an outer diameter of 10 mm, for subsurface imaging of biological tissues with 5-7 microm resolution. Depth-resolved en face images are obtained at 30 frames per second, with a field of view of 800 x 100 microm, by employing a two-dimensional scanning microelectromechanical systems mirror. Reflectance and fluorescence images are obtained with a laser source at 785 nm, demonstrating the ability to perform real-time optical biopsy.

Two-axis MEMS Scanning Catheter for Ultrahigh Resolution Three-dimensional and En Face Imaging.

Ultrahigh resolution two and three-dimensional optical coherence tomography (OCT) imaging was performed using a miniaturized, two-axis scanning catheter based upon microelectromechanical systems (MEMS) mirror technology. The catheter incorporated a custom-designed and fabricated, 1-mm diameter MEMS mirror driven by angular vertical comb (AVC) actuators on both an inner mirror axis and an outer, orthogonal gimbal axis. Using a differential drive scheme, a linearized position response over +/- 6 degrees mechanical angle was achieved. The flexible, fiber-optic catheter device measured < 5 mm in outer diameter with a rigid length of ~ 2.5 cm at the distal end. In vivo and ex vivo images are presented with < 4 microm axial and ~ 12 microm transverse resolution in tissue.

Fast-scanning two-photon fluorescence imaging based on a microelectromechanical systems two- dimensional scanning mirror.

Towards overcoming the size limitations of conventional two-photon fluorescence microscopy, we introduce two-photon imaging based on microelectromechanical systems (MEMS) scanners. Single crystalline silicon scanning mirrors that are 0.75 mm x 0.75 mm in size and driven in two dimensions by microfabricated vertical comb electrostatic actuators can provide optical deflection angles through a range of approximately16 degrees . Using such scanners we demonstrated two-photon microscopy and microendoscopy with fast-axis acquisition rates up to 3.52 kHz.

Fiber-optic fluorescence imaging.

Optical fibers guide light between separate locations and enable new types of fluorescence imaging. Fiber-optic fluorescence imaging systems include portable handheld microscopes, flexible endoscopes well suited for imaging within hollow tissue cavities and microendoscopes that allow minimally invasive high-resolution imaging deep within tissue. A challenge in the creation of such devices is the design and integration of miniaturized optical and mechanical components. Until recently, fiber-based fluorescence imaging was mainly limited to epifluorescence and scanning confocal modalities. Two new classes of photonic crystal fiber facilitate ultrashort pulse delivery for fiber-optic two-photon fluorescence imaging. An upcoming generation of fluorescence imaging devices will be based on microfabricated device components.