Kilohertz volumetric imaging of in-vivo dynamics using squeezed light field microscopy.

Volumetric functional imaging of transient cellular signaling and motion dynamics poses a significant challenge to current microscopy techniques, primarily due to limitations in hardware bandwidth and the restricted photon budget within short exposure times. In response to this challenge, we present squeezed light field microscopy (SLIM), a computational imaging method that enables rapid detection of high-resolution three-dimensional (3D) light signals using only a single, low-format camera sensor area. SLIM pushes the boundaries of 3D optical microscopy, achieving over one thousand volumes per second across a large field of view of 550 µm in diameter and 300 µm in depth. Using SLIM, we demonstrated blood cell velocimetry across the embryonic zebrafish brain and in a free-moving tail exhibiting high-frequency swinging motion. The millisecond temporal resolution also enables accurate voltage imaging of neural membrane potentials in the leech ganglion. These results collectively establish SLIM as a versatile and robust imaging tool for high-speed microscopy applications.

Multifunctional Ablative Gastrointestinal Imaging Capsule (MAGIC) for Esophagus Surveillance and Interventions.

Objective and Impact Statement: A clinically viable technology for comprehensive esophagus surveillance and potential treatment is lacking. Here, we report a novel multifunctional ablative gastrointestinal imaging capsule (MAGIC) technology platform to address this clinical need. The MAGIC technology could also facilitate the clinical translation and adoption of the tethered capsule endomicroscopy (TCE) technology. Introduction: Recently developed optical coherence tomography (OCT) TCE technologies have shown a promising potential for surveillance of Barrett's esophagus and esophageal cancer in awake patients without the need for sedation. However, it remains challenging with the current TCE technology for detecting early lesions and clinical adoption due to its suboptimal resolution, imaging contrast, and lack of visual guidance during imaging. Methods: Our technology reported here integrates dual-wavelength OCT imaging (operating at 800 and 1300 nm), an ultracompact endoscope camera, and an ablation laser, aiming to enable comprehensive surveillance, guidance, and potential ablative treatment of the esophagus. Results: The MAGIC has been successfully developed with its multimodality imaging and ablation capabilities demonstrated by imaging swine esophagus ex vivo and in vivo. The 800-nm OCT imaging offers exceptional resolution and contrast for the superficial layers, well suited for detecting subtle changes associated with early neoplasia. The 1300-nm OCT imaging provides deeper penetration, essential for assessing lesion invasion. The built-in miniature camera affords a conventional endoscopic view for assisting capsule deployment and laser ablation. Conclusion: By offering complementary and clinically viable functions in a single device, the reported technology represents an effective solution for endoscopic screening, diagnosis, and potential ablation treatment of the esophagus of a patient in an office setting.

Classification of Mobile-Based Oral Cancer Images Using the Vision Transformer and the Swin Transformer.

Oral cancer, a pervasive and rapidly growing malignant disease, poses a significant global health concern. Early and accurate diagnosis is pivotal for improving patient outcomes. Automatic diagnosis methods based on artificial intelligence have shown promising results in the oral cancer field, but the accuracy still needs to be improved for realistic diagnostic scenarios. Vision Transformers (ViT) have outperformed learning CNN models recently in many computer vision benchmark tasks. This study explores the effectiveness of the Vision Transformer and the Swin Transformer, two cutting-edge variants of the transformer architecture, for the mobile-based oral cancer image classification application. The pre-trained Swin transformer model achieved 88.7% accuracy in the binary classification task, outperforming the ViT model by 2.3%, while the conventional convolutional network model VGG19 and ResNet50 achieved 85.2% and 84.5% accuracy. Our experiments demonstrate that these transformer-based architectures outperform traditional convolutional neural networks in terms of oral cancer image classification, and underscore the potential of the ViT and the Swin Transformer in advancing the state of the art in oral cancer image analysis.

Characterizing close-focus lenses for microendoscopy.

Microendoscopes are commonly used in small lumens in the body, for which a focus near to the distal tip and ability to operate in an aqueous environment are paramount for navigation and disease detection. Commercially available distal optic systems below 1mm in diameter are severely limited, and custom micro lenses are generally very expensive. Gradient index of refraction (GRIN) singlets are available in small diameters but have limited optical performance adjustability. Three-dimensional (3D) printed monolithic optical systems are an emerging option that may be suitable for enabling high performance, close-focus imaging. In this manuscript, we compared the optical performance of three custom distal optic systems; a custom-pitch GRIN singlet, 3D-printed monolithic doublet, and 3D-printed monolithic triplet, with a nominal working distance (WD) of 1.5mm, 0.5mm and 0.4mm in 0.9% saline. These short WDs are ideal for microendoscopy in collapsed or flushed lumens such as pancreatic duct or fallopian tube. The GRIN singlet had performance limited only by the fiber bundle relay over 0.9mm to 1.6 mm depth of field (DOF). The 3D printed doublet was able to achieve a comparable DOF of 0.71mm, while the 3D printed triplet suffered the most limited DOF of 0.55mm. 3D printing enables flexible design of monolithic multi-element systems with aspheric surfaces of very short WDs and relative ease of integration.

UV-curable thiol-ene system for broadband infrared transparent objects.

Conventional infrared transparent materials, including inorganic ceramic, glass, and sulfur-rich organic materials, are usually processed through thermal or mechanical progress. Here, we report a photo-curable liquid material based on a specially designed thiol-ene strategy, where the multithiols and divinyl oligomers were designed to contain only C, H, and S atoms. This approach ensures transparency in a wide range spectrum from visible light to mid-wave infrared (MWIR), and to long-wave infrared (LWIR). The refractive index, thermal properties, and mechanical properties of samples prepared by this thiol-ene resin were characterized. Objects transparent to LWIR and MWIR were fabricated by molding and two-photon 3D printing techniques. We demonstrated the potential of our material in a range of applications, including the fabrication of IR optics with high imaging resolution and the construction of micro-reactors for temperature monitoring. This UV-curable thiol-ene system provides a fast and convenient alternative for the fabrication of thin IR transparent objects.

Bio-inspired Compact, High-resolution Snapshot Hyperspectral Imaging System with 3D Printed Glass Lightguide Array.

To address the major challenges to obtain high spatial resolution in snapshot hyperspectral imaging, 3D printed glass lightguide array has been developed to sample the intermediate image in high spatial resolution and redistribute the pixels in the output end to achieve high spectral resolution. Curved 3D printed lightguide array can significantly simplify the snapshot hyperspectral imaging system, achieve better imaging performance, and reduce the system complexity and cost. We have developed two-photon polymerization process to print glass lightguide array, and demonstrated the system performance with biological samples. This new snapshot technology will catalyze new hyperspectral imaging system development and open doors for new applications from UV to IR.

Inter-observer agreement among specialists in the diagnosis of oral potentially malignant disorders and oral cancer using store-and-forward technology.

OBJECTIVES: Oral cancer is a leading cause of morbidity and mortality. Screening and mobile Health (mHealth)-based approach facilitates early detection remotely in a resource-limited settings. Recent advances in eHealth technology have enabled remote monitoring and triage to detect oral cancer in its early stages. Although studies have been conducted to evaluate the diagnostic efficacy of remote specialists, to our knowledge, no studies have been conducted to evaluate the consistency of remote specialists. The aim of this study was to evaluate interobserver agreement between specialists through telemedicine systems in real-world settings using store-and-forward technology. MATERIALS AND METHODS: The two remote specialists independently diagnosed clinical images (n=822) from image archives. The onsite specialist diagnosed the same participants using conventional visual examination, which was tabulated. The diagnostic accuracy of two remote specialists was compared with that of the onsite specialist. Images that were confirmed histopathologically were compared with the onsite diagnoses and the two remote specialists. RESULTS: There was moderate agreement (k= 0.682) between two remote specialists and (k= 0.629) between the onsite specialist and two remote specialists in the diagnosis of oral lesions. The sensitivity and specificity of remote specialist 1 were 92.7% and 83.3%, respectively, and those of remote specialist 2 were 95.8% and 60%, respectively, each compared with histopathology. CONCLUSION: The diagnostic accuracy of the two remote specialists was optimal, suggesting that "store and forward" technology and telehealth can be an effective tool for triage and monitoring of patients. CLINICAL RELEVANCE: Telemedicine is a good tool for triage and enables faster patient care in real-world settings.

Clinical assessment of a low-cost, hand-held, smartphone-attached intraoral imaging probe for 5-aminolevulinic acid photodynamic therapy monitoring and guidance.

Significance: India has one of the highest rates of oral squamous cell carcinoma (OSCC) in the world, with an incidence of 15 per 100,000 and more than 70,000 deaths per year. The problem is exacerbated by a lack of medical infrastructure and routine screening, especially in rural areas. New technologies for oral cancer detection and timely treatment at the point of care are urgently needed. Aim: Our study aimed to use a hand-held smartphone-coupled intraoral imaging device, previously investigated for autofluorescence (auto-FL) diagnostics adapted here for treatment guidance and monitoring photodynamic therapy (PDT) using 5-aminolevulinic acid (ALA)-induced protoporphyrin IX (PpIX) fluorescence (FL). Approach: A total of 12 patients with 14 buccal mucosal lesions having moderately/well-differentiated micro-invasive OSCC lesions (<2 cm diameter and <5 mm depth) were systemically (in oral solution) administered three doses of 20 mg/kg ALA (total 60 mg/kg). Lesion site PpIX and auto-FL were imaged using the multichannel FL and polarized white-light oral cancer imaging probe before/after ALA administration and after light delivery (fractionated, total 100 J/cm2 of 635 nm red LED light). Results: The handheld device was conducive for access to lesion site images in the oral cavity. Segmentation of ratiometric images in which PpIX FL is mapped relative to auto-FL enabled improved demarcation of lesion boundaries relative to PpIX alone. A relative FL (R-value) threshold of 1.4 was found to segment lesion site PpIX production among the patients with mild to severe dysplasia malignancy. The segmented lesion size is well correlated with ultrasound findings. Lesions for which R-value was >1.65 at the time of treatment were associated with successful outcomes. Conclusion: These results indicate the utility of a low-cost, handheld intraoral imaging probe for image-guided PDT and treatment monitoring while also laying the groundwork for an integrated approach, combining cancer screening and treatment with the same hardware.

Compliance with Specialist Referral for Increased Cancer Risk in Low-Resource Settings: In-Person vs. Telehealth Options.

Efforts are underway to improve the accuracy of non-specialist screening for oral cancer (OC) risk, yet better screening will only translate into improved outcomes if at-risk individuals comply with specialist referral. Most individuals from low-resource, minority, and underserved (LRMU) populations fail to complete a specialist referral for OC risk. The goal was to evaluate the impact of a novel approach on specialist referral compliance in individuals with a positive OC risk screening outcome. A total of 60 LRMU subjects who had screened positive for increased OC risk were recruited and given the choice of referral for an in-person (20 subjects) or a telehealth (40 subjects) specialist visit. Referral compliance was tracked weekly over 6 months. Compliance was 30% in the in-person group, and 83% in the telehealth group. Approximately 83-85% of subjects from both groups who had complied with the first specialist referral complied with a second follow-up in-person specialist visit. Overall, 72.5% of subjects who had chosen a remote first specialist visit had entered into the continuum of care by the study end, vs. 25% of individuals in the in-person specialist group. A two-step approach that uses telehealth to overcome barriers may improve specialist referral compliance in LRMU individuals with increased OC risk.

Motionless volumetric structured light sheet microscopy.

To meet the increasing need for low-cost, compact imaging technology with cellular resolution, we have developed a microLED-based structured light sheet microscope for three-dimensional ex vivo and in vivo imaging of biological tissue in multiple modalities. All the illumination structure is generated directly at the microLED panel-which serves as the source-so light sheet scanning and modulation is completely digital, yielding a system that is simpler and less prone to error than previously reported methods. Volumetric images with optical sectioning are thus achieved in an inexpensive, compact form factor without any moving parts. We demonstrate the unique properties and general applicability of our technique by ex vivo imaging of porcine and murine tissue from the gastrointestinal tract, kidney, and brain.

Inter-observer agreement among specialists in the diagnosis of Oral Potentially Malignant Disorders and Oral Cancer using Store-and-Forward technology.

Oral Cancer is one of the most common causes of morbidity and mortality. Screening and mobile Health (mHealth) based approach facilitates remote early detection of Oral cancer in a resource-constrained settings. The emerging eHealth technology has aided specialist reach to rural areas enabling remote monitoring and triaging to downstage Oral cancer. Though the diagnostic accuracy of the remote specialist has been evaluated, there are no studies evaluating the consistency among the remote specialists, to the best of our knowledge. The purpose of the study was to evaluate the interobserver agreement between the specialists through telemedicine systems in real-world settings using store and forward technology. Two remote specialists independently diagnosed the clinical images from image repositories, and the diagnostic accuracy was compared with onsite specialist and histopathological diagnosis when available. Moderate agreement (k = 0.682) between two remote specialists and (k = 0.629) between the onsite specialist and two remote specialists in diagnosing oral lesions. The sensitivity and specificity of remote specialist 1 were 92.7% and 83.3%, whereas remote specialist 2 was 95.8% and 60%, respectively, compared to histopathology. The store and forward technology and telecare can be effective tools in triaging and surveillance of patients.

On-axis deflectometric system for freeform surface measurement.

We propose an on-axis deflectometric system for the accurate measurement of freeform surfaces with large slope ranges. A miniature plane mirror is attached on the illumination screen to fold the optical path and achieve the on-axis deflectometric testing. Due to the existence of the miniature folding mirror, the deep-learning method is applied to recover the missing surface data in a single measurement. Low sensitivity to the calibration error of system geometry and high testing accuracy can be achieved with the proposed system. The feasibility and accuracy of the proposed system have been validated. The system is low in cost and simple in configuration, and it provides a feasible way for the flexible and general testing of freeform surfaces, with a significant potential of the application in on-machine testing.

Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map.

Convolutional neural networks have demonstrated excellent performance in oral cancer detection and classification. However, the end-to-end learning strategy makes CNNs hard to interpret, and it can be challenging to fully understand the decision-making procedure. Additionally, reliability is also a significant challenge for CNN based approaches. In this study, we proposed a neural network called the attention branch network (ABN), which combines the visual explanation and attention mechanisms to improve the recognition performance and interpret the decision-making simultaneously. We also embedded expert knowledge into the network by having human experts manually edit the attention maps for the attention mechanism. Our experiments have shown that ABN performs better than the original baseline network. By introducing the Squeeze-and-Excitation (SE) blocks to the network, the cross-validation accuracy increased further. Furthermore, we observed that some previously misclassified cases were correctly recognized after updating by manually editing the attention maps. The cross-validation accuracy increased from 0.846 to 0.875 with the ABN (Resnet18 as baseline), 0.877 with SE-ABN, and 0.903 after embedding expert knowledge. The proposed method provides an accurate, interpretable, and reliable oral cancer computer-aided diagnosis system through visual explanation, attention mechanisms, and expert knowledge embedding.

PEGylated and Non-PEGylated TCP-1 Probes for Imaging of Colorectal Cancer.

PURPOSE: Previous studies indicate that 99mTc- and fluorescent-labeled c[Cys-Thr-Pro-Ser-Pro-Phe-Ser-His-Cys]OH (TCP-1) peptides were able to detect colorectal cancer (CRC) and tumor-associated vasculature. This study was designed to characterize the targeting properties of PEGylated and non-PEGylated TCP-1 peptides for CRC imaging. PROCEDURES: Cell uptake of cyanine 7 (Cy7)-labeled TCP-1 probes (Cy7-PEG4-TCP-1 and Cy7-TCP-1) was investigated in three CRC cell lines (human, HCT116 and HT29; mouse, CT26). Xenograft and orthotopic CRC tumor models with HCT116 and CT26 cells were used to characterize biodistribution and CRC tumor-targeting properties of TCP-1 fluorescence and radioligand with and without PEGylation, [99mTc]Tc-HYNIC-PEG4-TCP-1 vs. [99mTc]Tc-HYNIC-TCP-1. RESULTS: Fluorescence images showed that TCP-1 probes were distributed in the cytoplasm and nucleus of CRC cells. When CT26 cells were treated with unlabeled TCP-1 peptide prior to the cell incubation with Cy7-PEG4-TCP-1, cell fluorescent signals were significantly reduced relative to the cells without blockade. Relative to Cy7-TCP-1, superior brilliance and visibility of fluorescence was observed in the tumor with Cy7-PEG4-TCP-1 and maintained up to 18 h post-injection. [99mTc]Tc-HYNIC-PEG4-TCP-1 images in xenograft and orthotopic CRC models demonstrated that TCP-1 PEGylation preserved tumor-targeting capability of TCP-1, but its distribution (%ID/g) in the liver and intestine was higher than that of [99mTc]Tc-HYNIC-TCP-1 (1.51 ± 0.29 vs 0.53 ± 0.12, P < 0.01). Better tumor visualization by [99mTc]Tc-HYNIC-TCP-1 was observed in the orthotopic CRC model due to lower intestinal radioactivity. CONCLUSIONS: TCP-1-based probes undergo endocytosis and localize in the cytoplasm and nucleus of human and mouse CRC cells. Tumor detectability of fluorescent TCP-1 peptide with a PEG4 spacer is promising due to its enhanced tumor binding affinity and rapid clearance kinetics from nontumor tissues. Non-PEGylated [99mTc]Tc-HYNIC-TCP-1 exhibits lower nonspecific accumulation in the liver and gastrointestinal tract and might have better capability for detecting CRC lesions in clinical sites. TCP-1 may represent an innovative targeting molecule for detecting CRC noninvasively.

Exploring uncertainty measures in convolutional neural network for semantic segmentation of oral cancer images.

Significance: Oral cancer is one of the most prevalent cancers, especially in middle- and low-income countries such as India. Automatic segmentation of oral cancer images can improve the diagnostic workflow, which is a significant task in oral cancer image analysis. Despite the remarkable success of deep-learning networks in medical segmentation, they rarely provide uncertainty quantification for their output. Aim: We aim to estimate uncertainty in a deep-learning approach to semantic segmentation of oral cancer images and to improve the accuracy and reliability of predictions. Approach: This work introduced a UNet-based Bayesian deep-learning (BDL) model to segment potentially malignant and malignant lesion areas in the oral cavity. The model can quantify uncertainty in predictions. We also developed an efficient model that increased the inference speed, which is almost six times smaller and two times faster (inference speed) than the original UNet. The dataset in this study was collected using our customized screening platform and was annotated by oral oncology specialists. Results: The proposed approach achieved good segmentation performance as well as good uncertainty estimation performance. In the experiments, we observed an improvement in pixel accuracy and mean intersection over union by removing uncertain pixels. This result reflects that the model provided less accurate predictions in uncertain areas that may need more attention and further inspection. The experiments also showed that with some performance compromises, the efficient model reduced computation time and model size, which expands the potential for implementation on portable devices used in resource-limited settings. Conclusions: Our study demonstrates the UNet-based BDL model not only can perform potentially malignant and malignant oral lesion segmentation, but also can provide informative pixel-level uncertainty estimation. With this extra uncertainty information, the accuracy and reliability of the model's prediction can be improved.

pHSCNN: CNN-based hyperspectral recovery from a pair of RGB images.

To increase the fidelity of hyperspectral recovery from RGB images, we propose a pairwise-image-based hyperspectral convolutional neural network (pHSCNN) to recover hyperspectral images from a pair of RGB images, obtained by the same color sensor with and without an optical filter in front of the imaging lens. The proposed method avoids the pitfall of requiring multiple color sensors to obtain different RGB images and achieves higher accuracy than recovery from single RGB image. Besides, pHSCNN can also optimize the optical filter to further improve the performance. To experiment on real data, we built a dual-camera hyperspectral imaging system and created a real-captured hyperspectral-RGB dataset. Experimental results demonstrate the superiority of pHSCNN with the highest accuracy of the recovered hyperspectral signature perceptually and numerically.

Theoretical model and digital extraction of subsurface damage in ground fused silica.

Based on the fracture mechanics and grinding kinematics, a theoretical model is developed to determine various subsurface damage (SSD) parameters and roughness Rz of the ground brittle material with consideration of the material removal mode and spring back. Based on the image processing, a digital method is proposed to extract various SSD parameters from the cross-section micrograph of the ground sample. To verify the model and method, many fused silica samples are ground under different processing parameters, and their SSD depth and roughness Rz are measured. The research results show the average SSD depth (SSDa) can be expressed as SSDa = χ1Rz4/3 + χ2Rz (χ1 and χ2 are coefficients). The SSDa is closer to half of the maximum SSD depth (SSDm) as the wheel speed decreases or the grinding depth, feed speed, or abrasive diameter increases. The SSD length or density basically increases linearly with the increase of the SSDm. The digital method is reliable with a largest relative error of 6.65% in SSD depth, extraction speed of about 1.63s per micrograph, and good robustness to the micrograph size and small-scale residue interference. The research will contribute to the evaluation of SSDs and the optimization of the grinding process of fused silica.

Experimental Studies on Fabricating Lenslet Array with Slow Tool Servo.

On the demand of low-cost, lightweight, miniaturized, and integrated optical systems, precision lenslet arrays are widely used. Diamond turning is often used to fabricate lenslet arrays directly or molds that are used to mold lenslet arrays. In this paper, mainly by real-time monitoring position following error for slow tool servo, different fabrication parameters are quantitatively studied and optimized for actual fabrication, then by actual fabrication validation, uniform and high-fidelity surface topography across the actual whole lenslet array is achieved. The evaluated fabrication parameters include sampling strategy, inverse time feed, arc-length, etc. The study provides a quick, effective, and detailed reference for both convex and concave lenslet array cutting parameter selection. At the end, a smooth zonal machining strategy toolpath is demonstrated for fabricating concave lenslet arrays.

Short-wave infrared polarimetric image reconstruction using a deep convolutional neural network based on a high-frequency correlation.

Imaging in visible and short-wave infrared (SWIR) wavebands is essential in most remote sensing applications. However, compared to visible imaging cameras, SWIR cameras typically have lower spatial resolution, which limits the detailed information shown in SWIR images. We propose a method to reconstruct high-resolution polarization SWIR images with the help of color images using the deep learning method. The training dataset is constructed from color images, and the trained model is well suited for SWIR image reconstruction. The experimental results show the effectiveness of the proposed method in enhancing the quality of the polarized SWIR images with much better spatial resolution. Some buried spatial and polarized information may be recovered in the reconstructed SWIR images.

Diagnosis of paediatric tuberculosis by optically detecting two virulence factors on extracellular vesicles in blood samples.

Sensitive and specific blood-based assays for the detection of pulmonary and extrapulmonary tuberculosis would reduce mortality associated with missed diagnoses, particularly in children. Here we report a nanoparticle-enhanced immunoassay read by dark-field microscopy that detects two Mycobacterium tuberculosis virulence factors (the glycolipid lipoarabinomannan and its carrier protein) on the surface of circulating extracellular vesicles. In a cohort study of 147 hospitalized and severely immunosuppressed children living with HIV, the assay detected 58 of the 78 (74%) cases of paediatric tuberculosis, 48 of the 66 (73%) cases that were missed by microbiological assays, and 8 out of 10 (80%) cases undiagnosed during the study. It also distinguished tuberculosis from latent-tuberculosis infections in non-human primates. We adapted the assay to make it portable and operable by a smartphone. With further development, the assay may facilitate the detection of tuberculosis at the point of care, particularly in resource-limited settings.