All-reflective tethered capsule endoscope for multimodal optical coherence tomography in the esophagus.

Significance: Esophageal cancer is becoming increasingly prevalent in Western countries. Early detection is crucial for effective treatment. Multimodal imaging combining optical coherence tomography (OCT) with complementary optical imaging techniques may provide enhanced diagnostic capabilities by simultaneously assessing tissue morphology and biochemical content. Aim: We aim to develop a tethered capsule endoscope (TCE) that can accommodate a variety of point-scanning techniques in addition to OCT without requiring design iterations on the optical or mechanical design. Approach: We propose a TCE utilizing exclusively reflective optics to focus and steer light from and to a double-clad fiber. Specifically, we use an ellipsoidal mirror to achieve finite conjugation between the fiber tip and the imaging plane. Results: We demonstrate a functional all-reflective TCE. We first detail the design, fabrication, and assembly steps required to obtain such a device. We then characterize its performance and demonstrate combined OCT at 1300 nm and visible spectroscopic imaging in the 500- to 700-nm range. Finally, we discuss the advantages and limitations of the proposed design. Conclusions: An all-reflective TCE is feasible and allows for achromatic high-quality imaging. Such a device could be utilized as a platform for testing various combinations of modalities to identify the optimal candidates without requiring design iterations.

PETcoil: first results from a second-generation RF-penetrable TOF-PET brain insert for simultaneous PET/MRI.

Simultaneous positron emission tomography (PET)/magnetic resonance imaging provides concurrent information about anatomic, functional, and molecular changes in disease. We are developing a second generation MR-compatible RF-penetrable TOF-PET insert. The insert has a smaller scintillation crystal size and ring diameter compared to clinical whole-body PET scanners, resulting in higher spatial resolution and sensitivity. This paper reports the initial system performance of this full-ring PET insert. The global photopeak energy resolution and global coincidence time resolution, 11.74 ± 0.03 % FWHM and 238.1 ± 0.5 ps FWHM, respectively, are preserved as we scaled up the system to a full ring comprising 12, 288 LYSO-SiPM channels (crystal size: 3.2 × 3.2 × 20 mm3). Throughout a ten-hour experiment, the system performance remained stable, exhibiting a less than 1% change in all measured parameters. In a resolution phantom study, the system successfully resolved all 2.8 mm diameter rods, achieving an average VPR of 0.28 ± 0.08 without TOF and 0.24 ± 0.07 with TOF applied. Moreover, the implementation of TOF in the Hoffman phantom study also enhanced image quality. Initial MR compatibility studies of the full PET ring were performed with it unpowered as a milestone to focus on looking for material and geometry-related artifacts. During all MR studies, the MR body coil functioned as both the transmit and receive coil, and no observable artifacts were detected. As expected, using the body coil also as the RF receiver, MR image signal-to-noise ratio exhibited degradation (∼30%), so we are developing a high quality receive-only coil that resides inside the PET ring.

Multimodal Imaging Using Raman Spectroscopy and FTIR in a Single Analytical Instrument with a Microscope (Infrared Raman Microscopy AIRsight, Shimadzu): Opportunities and Applications.

Raman spectroscopy and Fourier transform infrared (FTIR) spectroscopy are powerful analytical techniques widely used separately in different fields of study. Integrating these two powerful spectroscopic techniques into one device represents a groundbreaking advance in multimodal imaging. This new combination which merges the molecular vibrational information from Raman spectroscopy with the ability of FTIR to study polar bonds, creates a unique and complete analytical tool. Through a detailed examination of the microscope's operation and case studies, this article illustrates how this integrated analytical instrument can provide more thorough and accurate analysis than traditional methods, potentially revolutionising analytical sample characterisation. This article aims to present the features and possible uses of a unified instrument merging FTIR and Raman spectroscopy for multimodal imaging. It particularly focuses on the technological progress and collaborative benefits of these two spectroscopic techniques within the microscope system. By emphasising this approach's unique benefits and improved analytical capabilities, the authors aim to illustrate its applicability in diverse scientific and industrial sectors.

Construction and Application of a PD-L1-Targeted Multimodal Diagnostic and Dual-Functional Theranostics Nanoprobe.

Background: In recent years, PD-L1 has been primarily utilized as an immune checkpoint marker in cancer immunotherapy. However, due to tumor heterogeneity, the response rate to such therapies often falls short of expectations. In addition to its role in immunotherapy, PD-L1 serves as a specific target on the surface of tumor cells for targeted diagnostic and therapeutic interventions. There is an absence of a fully developed PD-L1-targeted diagnostic and therapeutic probe for clinical use, which constrains the exploration and clinical exploitation of this target. Methods and Results: In this study, we engineered a PD-L1-targeted probe with multimodal imaging and dual therapeutic functionalities utilizing organic melanin nanoparticles. Functionalization with the WL12-SH peptide endowed the nanoprobe with specific targeting capabilities. Subsequent radiolabeling with 89Zr (half-life: 100.8 hours) and chelation of Mn2+ ions afforded the probe the capacity for simultaneous PET and MRI imaging modalities. Cellular uptake assays revealed pronounced specificity, with -positive cells exhibiting significantly higher uptake than -negative counterparts (p < 0.05). Dual-modal PET/MRI imaging delineated rapid and sustained accumulation at the neoplastic site, yielding tumor-to-non-tumor (T/NT) signal ratios at 24 hours post-injection of 16.67±3.45 for PET and 6.63±0.64 for MRI, respectively. We conjugated the therapeutic radionuclide 131I (half-life: 8.02 days) to the construct and combined low-dose radiotherapy and photothermal treatment (PTT), culminating in superior antitumor efficacy while preserving a high safety profile. The tumors in the cohort receiving the dual-modality therapy exhibited significantly reduced volume and weight compared to those in the control and monotherapy groups. Conclusion: We developed and applied a novel -targeted multimodal theranostic nanoprobe, characterized by its high specificity and superior imaging capabilities as demonstrated in PET/MRI modalities. Furthermore, this nanoprobe facilitates potent therapeutic efficacy at lower radionuclide doses when used in conjunction with PTT.

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data.

Depression is a major psychological disorder with a growing impact worldwide. Traditional methods for detecting the risk of depression, predominantly reliant on psychiatric evaluations and self-assessment questionnaires, are often criticized for their inefficiency and lack of objectivity. Advancements in deep learning have paved the way for innovations in depression risk detection methods that fuse multimodal data. This paper introduces a novel framework, the Audio, Video, and Text Fusion-Three Branch Network (AVTF-TBN), designed to amalgamate auditory, visual, and textual cues for a comprehensive analysis of depression risk. Our approach encompasses three dedicated branches-Audio Branch, Video Branch, and Text Branch-each responsible for extracting salient features from the corresponding modality. These features are subsequently fused through a multimodal fusion (MMF) module, yielding a robust feature vector that feeds into a predictive modeling layer. To further our research, we devised an emotion elicitation paradigm based on two distinct tasks-reading and interviewing-implemented to gather a rich, sensor-based depression risk detection dataset. The sensory equipment, such as cameras, captures subtle facial expressions and vocal characteristics essential for our analysis. The research thoroughly investigates the data generated by varying emotional stimuli and evaluates the contribution of different tasks to emotion evocation. During the experiment, the AVTF-TBN model has the best performance when the data from the two tasks are simultaneously used for detection, where the F1 Score is 0.78, Precision is 0.76, and Recall is 0.81. Our experimental results confirm the validity of the paradigm and demonstrate the efficacy of the AVTF-TBN model in detecting depression risk, showcasing the crucial role of sensor-based data in mental health detection.

HyperMRI: hyperspectral and magnetic resonance fusion methodology for neurosurgery applications.

PURPOSE: Magnetic resonance imaging (MRI) is a common technique in image-guided neurosurgery (IGN). Recent research explores the integration of methods like ultrasound and tomography, among others, with hyperspectral (HS) imaging gaining attention due to its non-invasive real-time tissue classification capabilities. The main challenge is the registration process, often requiring manual intervention. This work introduces an automatic, markerless method for aligning HS images with MRI. METHODS: This work presents a multimodal system that combines RGB-Depth (RGBD) and HS cameras. The RGBD camera captures the patient's facial geometry, which is used for registration with the preoperative MR through ICP. Once MR-depth registration is complete, the integration of HS data is achieved using a calibrated homography transformation. The incorporation of external tracking with a novel calibration method allows camera mobility from the registration position to the craniotomy area. This methodology streamlines the fusion of RGBD, HS and MR images within the craniotomy area. RESULTS: Using the described system and an anthropomorphic phantom head, the system has been characterised by registering the patient's face in 25 positions and 5 positions resulted in a fiducial registration error of 1.88 ± 0.19 mm and a target registration error of 4.07 ± 1.28 mm, respectively. CONCLUSIONS: This work proposes a new methodology to automatically register MR and HS information with a sufficient accuracy. It can support the neurosurgeons to guide the diagnosis using multimodal data over an augmented reality representation. However, in its preliminary prototype stage, this system exhibits significant promise, driven by its cost-effectiveness and user-friendly design.

An open-source MRI compatible frame for multimodal presurgical mapping in macaque and capuchin monkeys.

BACKGROUND: High-precision neurosurgical targeting in nonhuman primates (NHPs) often requires presurgical anatomical mapping with noninvasive neuroimaging techniques (MRI, CT, PET), allowing for translation of individual anatomical coordinates to surgical stereotaxic apparatus. Given the varied tissue contrasts that these imaging techniques produce, precise alignment of imaging-based coordinates to surgical apparatus can be cumbersome. MRI-compatible stereotaxis with radiopaque fiducial markers offer a straight-forward and reliable solution, but existing commercial options do not fit in conformal head coils that maximize imaging quality. NEW METHOD: We developed a compact MRI-compatible stereotaxis suitable for a variety of NHP species (Macaca mulatta, Macaca fascicularis, and Cebus apella) that allows multimodal alignment through technique-specific fiducial markers. COMPARISON WITH EXISTING METHODS: With the express purpose of compatibility with clinically available MRI, CT, and PET systems, the frame is no larger than a human head, while allowing for imaging NHPs in the supinated position. This design requires no marker implantation, special software, or additional knowledge other than the operation of a common large animal stereotaxis. RESULTS: We demonstrated the applicability of this 3D-printable apparatus across a diverse set of experiments requiring presurgical planning: 1) We demonstrate the accuracy of the fiducial system through a within-MRI cannula insertion and subcortical injection of a viral vector. 2) We also demonstrated accuracy of multimodal (MRI and CT) alignment and coordinate transfer to guide a surgical robot electrode implantation for deep-brain electrophysiology. CONCLUSIONS: The computer-aided design files and engineering drawings are publicly available, with the modular design allowing for low cost and manageable manufacturing.

SLE diagnosis research based on SERS combined with a multi-modal fusion method.

As artificial intelligence technology gains widespread adoption in biomedicine, the exploration of integrating biofluidic Raman spectroscopy for enhanced disease diagnosis opens up new prospects for the practical application of Raman spectroscopy in clinical settings. However, for systemic lupus erythematosus (SLE), origin Raman spectral data (ORS) have relatively weak signals, making it challenging to obtain ideal classification results. Although the surface enhancement technique can enhance the scattering signal of Raman spectroscopic data, the sensitivity of the SERS substrate to airborne impurities and the inhomogeneous distribution of hotspots degrade part of the signal. To fully utilize both kinds of data, this paper proposes a two-branch residual-attention network (DBRAN) fusion technique, which allows the ORS to complement the degraded portion and thus improve the model's classification accuracy. The features are extracted using the residual module, which retains the original features while extracting the deep features. At the same time, the study incorporates the attention module in both the upper and lower branches to handle the weight allocation of the two modal features more efficiently. The experimental results demonstrate that both the low-level fusion method and the intermediate-level fusion method can significantly improve the diagnostic accuracy of SLE disease classification compared with a single modality, in which the intermediate-level fusion of DBRAN achieves 100% classification accuracy, sensitivity, and specificity. The accuracy is improved by 10% and 7% compared with the ORS unimodal and the SERS unimodal modalities, respectively. The experiment, by fusing the multimodal spectral, realized rapid diagnosis of SLE disease by fusing multimodal spectral data, which provides a reference idea in the field of Raman spectroscopy and can be further promoted to clinical practical applications in the future.

Multimodality imaging in patients with implantable loop recorders: Tips and tricks.

An implantable loop recorder (ILR) is a leadless rectangular device used for prolonged electrocardiographic monitoring for up to 3 years. This miniaturized device, inserted subcutaneously, allows clinicians to investigate possible cardiac rhythm disturbances in patients suffering from recurrent unexplained syncope. As the age of the population increases rapidly and the number of ILR patients amplifies, the clinical significance of ILRs is undeniable. Although radioopaque and easily seen on plain chest radiographs and other imaging modalities, ILRs may represent a challenge for clinicians and radiologists to recognize their classic appearance and differentiate them from numerous other cardiac devices. This article aims to summarize current literature on ILRs, their basic function, types, and indications for implantation, but most of all, it aims to familiarize clinicians and radiologists with common imaging features of these devices, safety issues, and artifact-reducing methods. Specifically, this review discusses the typical appearance of ILRs on major diagnostic imaging modalities, including chest X-ray, mammography, ultrasonography, computed tomography, and magnetic resonance imaging (MRI). Furthermore, optimization strategies to mitigate image artifacts and safety issues regarding MRI are discussed.

Developing diagnostic assessment of breast lumpectomy tissues using radiomic and optical signatures.

High positive margin rates in oncologic breast-conserving surgery are a pressing clinical problem. Volumetric X-ray scanning is emerging as a powerful ex vivo specimen imaging technique for analyzing resection margins, but X-rays lack contrast between non-malignant and malignant fibrous tissues. In this study, combined micro-CT and wide-field optical image radiomics were developed to classify malignancy of breast cancer tissues, demonstrating that X-ray/optical radiomics improve malignancy classification. Ninety-two standardized features were extracted from co-registered micro-CT and optical spatial frequency domain imaging samples extracted from 54 breast tumors exhibiting seven tissue subtypes confirmed by microscopic histological analysis. Multimodal feature sets improved classification performance versus micro-CT alone when adipose samples were included (AUC = 0.88 vs. 0.90; p-value = 3.65e-11) and excluded, focusing the classification task on exclusively non-malignant fibrous versus malignant tissues (AUC = 0.78 vs. 0.85; p-value = 9.33e-14). Extending the radiomics approach to high-dimensional optical data-termed "optomics" in this study-offers a promising optical image analysis technique for cancer detection. Radiomic feature data and classification source code are publicly available.

Multimodal imaging as optical biopsy system for gastritis diagnosis in humans, and input of the mouse model.

BACKGROUND: Gastric inflammation is a major risk factor for gastric cancer. Current endoscopic methods are not able to efficiently detect and characterize gastric inflammation, leading to a sub-optimal patients' care. New non-invasive methods are needed. Reflectance mucosal light analysis is of particular interest in this context. The aim of our study was to analyze reflectance light and specific autofluorescence signals, both in humans and in a mouse model of gastritis. METHODS: We recruited patients undergoing gastroendoscopic procedure during which reflectance was analysed with a multispectral camera. In parallel, the gastritis mouse model of Helicobacter pylori infection was used to investigate reflectance from ex vivo gastric samples using a spectrometer. In both cases, autofluorescence signals were measured using a confocal microscope. FINDINGS: In gastritis patients, reflectance modifications were significant in near-infrared spectrum, with a decrease between 610 and 725 nm and an increase between 750 and 840 nm. Autofluorescence was also modified, showing variations around 550 nm of emission. In H. pylori infected mice developing gastric inflammatory lesions, we observed significant reflectance modifications 18 months after infection, with increased intensity between 617 and 672 nm. Autofluorescence was significantly modified after 1, 3 and 6 months around 550 and 630 nm. Both in human and in mouse, these reflectance data can be considered as biomarkers and accurately predicted inflammatory state. INTERPRETATION: In this pilot study, using a practical measuring device, we identified in humans, modification of reflectance spectra in the visible spectrum and for the first time in near-infrared, associated with inflammatory gastric states. Furthermore, both in the mouse model and humans, we also observed modifications of autofluorescence associated with gastric inflammation. These innovative data pave the way to deeper validation studies on larger cohorts, for further development of an optical biopsy system to detect gastritis and finally to better surveil this important gastric cancer risk factor. FUNDING: The project was funded by the ANR EMMIE (ANR-15-CE17-0015) and the French Gastroenterology Society (SNFGE).

First proof-of-concept evaluation of the FUSION-X-US-II prototype for the performance of automated breast ultrasound in healthy volunteers.

PURPOSE: The FUSION-X-US-II prototype was developed to combine 3D-automated breast ultrasound (ABUS) and digital breast tomosynthesis in a single device without decompressing the breast. We evaluated the technical function, feasibility of the examination workflow, image quality, breast tissue coverage and patient comfort of the ABUS device of the new prototype. METHODS: In this prospective feasibility study, the FUSION-X-US-II prototype was used to perform ABUS in 30 healthy volunteers without history of breast cancer. The ABUS images of the prototype were interpreted by a physician with specialization in breast diagnostics. Any detected lesions were measured and classified using BI-RADS® scores. Image quality was rated subjectively by the physician and coverage of the breast was measured. Patient comfort was evaluated by a questionnaire after the examination. RESULTS: One hundred and six scans were performed (61 × CC, 23 × ML, 22 × MLO) in 60 breasts. Image acquisition and processing by the prototype was fast and accurate. Breast coverage by ABUS was approximately 90.8%. Sixteen breast lesions (all benign, classified as BIRADS® 2) were identified. The examination was tolerated by all patients. CONCLUSION: The FUSION-X-US-II prototype allows a rapid ABUS scan with mostly high patient comfort. Technical developments resulted in an improvement of quality and coverage compared to previous prototype versions. The results are encouraging for a test of the prototype in a clinical setting in combination with tomosynthesis.

Background-free dual-mode optical and 13C magnetic resonance imaging in diamond particles.

Multimodal imaging-the ability to acquire images of an object through more than one imaging mode simultaneously-has opened additional perspectives in areas ranging from astronomy to medicine. In this paper, we report progress toward combining optical and magnetic resonance (MR) imaging in such a "dual" imaging mode. They are attractive in combination because they offer complementary advantages of resolution and speed, especially in the context of imaging in scattering environments. Our approach relies on a specific material platform, microdiamond particles hosting nitrogen vacancy (NV) defect centers that fluoresce brightly under optical excitation and simultaneously "hyperpolarize" lattice [Formula: see text] nuclei, making them bright under MR imaging. We highlight advantages of dual-mode optical and MR imaging in allowing background-free particle imaging and describe regimes in which either mode can enhance the other. Leveraging the fact that the two imaging modes proceed in Fourier-reciprocal domains (real and k-space), we propose a sampling protocol that accelerates image reconstruction in sparse-imaging scenarios. Our work suggests interesting possibilities for the simultaneous optical and low-field MR imaging of targeted diamond nanoparticles.

Multimodal neural recordings with Neuro-FITM uncover diverse patterns of cortical-hippocampal interactions.

Many cognitive processes require communication between the neocortex and the hippocampus. However, coordination between large-scale cortical dynamics and hippocampal activity is not well understood, partially due to the difficulty in simultaneously recording from those regions. In the present study, we developed a flexible, insertable and transparent microelectrode array (Neuro-FITM) that enables investigation of cortical-hippocampal coordinations during hippocampal sharp-wave ripples (SWRs). Flexibility and transparency of Neuro-FITM allow simultaneous recordings of local field potentials and neural spiking from the hippocampus during wide-field calcium imaging. These experiments revealed that diverse cortical activity patterns accompanied SWRs and, in most cases, cortical activation preceded hippocampal SWRs. We demonstrated that, during SWRs, different hippocampal neural population activity was associated with distinct cortical activity patterns. These results suggest that hippocampus and large-scale cortical activity interact in a selective and diverse manner during SWRs underlying various cognitive functions. Our technology can be broadly applied to comprehensive investigations of interactions between the cortex and other subcortical structures.

Punctate inner choroidopathy with atypical presentation / Coroidopatia interna ponteada de apresentação atípica

ABSTRACT We report a case of a young Caucasian female presenting with sudden decrease of vision in the left eye, metamorphopsia, and nasal scotoma. Past medical history revealed a diagnosis of myasthenia gravis, which was currently treated with azathioprine, pyridostigmine, and prednisone. Ophthalmological examination showed fundus with clear vitreous and yellow-white lesions that were isolated and perimacular in the right eye, multiple and confluent in the macula, and punctate in periphery in the left eye. Laboratory workup ruled out the presence of infectious and inflammatory diseases. Fundus autofluorescence disclosed hypoautoflurescence with hyperfluorescent margins corresponding to the lesions observed in both eyes and the angiogram revealed hyperfluorescence since early phases without late leakage. Spectral-domain optical coherence tomography showed areas of intermittent retinal pigment epithelium elevations and disruption of the ellipsoid zone. She was diagnosed with punctate inner choroidopathy and then treated with an increased dose of daily prednisone, which resulted in progressive improvement of her visual acuity and anatomical status.(AU)

RESUMO Relato de caso de mulher jovem, caucasiana, com súbita diminuição de acuidade visual de olho esquerdo, metamorfopsia e escotoma nasal. Apresentava diagnóstico de Miastenia gravis, em tratamento com Azatioprina, Piridostigmina e Prednisona. Fundo de olho demonstrava vítreo límpido e lesões amarelo-esbranquiçadas, perimaculares e isoladas em olho direito, múltiplas e confluentes em mácula e pontilhadas em periferia no olho esquerdo. Exames laboratoriais descartaram doenças infecciosas e inflamatórias. Auto-fluorescência revelou lesões hipoautofluorescentes com margens hiperfluorescentes correspondentes às observadas em ambos os olhos, enquanto angiofluoresceinografia mostrou hiperfluorescência desde as fases iniciais sem vazamento tardio. Tomografia de coerência óptica de domínio espectral revelou áreas de elevações intermitentes do epitélio pigmentar da retina e interrupção da zona elipsóide correspondente. Definiu-se como diagnóstico a coroidopatia interna ponteada, sendo instituído aumento na dose diária de Prednisona, com melhoria progressiva da acuidade visual e do aspecto de fundo de olho da paciente.(AU)

Can ultrasound guidance reduce radiation exposure significantly in percutaneous nephrolithotomy in pediatric patients?

In this study, we aimed to compare clinical and technical outcomes between pediatric patients who underwent percutaneous nephrolithotomy (PCNL) under fluoroscopy (FL) and those that underwent this procedure under FL with ultrasound assistance (FLUSA). The data of 66 PCNL patients were analyzed retrospectively. Renal puncture was successful in 22 patients in the FLUSA group and 44 patients in the FL group. In all cases, FL was used for tract dilation and confirmation of ureteral catheter positioning at the beginning of the procedure. The sample consisted of 46 males and 20 females with a mean age of 7.2 ± 2.1 years (range 1-17 years). Stone size varied from 8.0 to 75.4 mm, and 89% of patients achieved a completely stone-free state. The median puncture time was 130.5 ± 25.3 s for FLUSA and 295 ± 82.8 s for FL, the median fluoroscopic screening time was 95 ± 33 and 230 ± 116 s, respectively, and the median radiation dose was 19.04 ± 9.9 dGy/cm2 and 54 ± 21.4 dGy/cm2, respectively. The median puncture time, fluoroscopic screening time, and radiation dose were statistically lower in the FLUSA group (p = 0.001, Mann-Whitney U test). The greatest problem in PCNL is the use of fluoroscopy. Due to some anatomical differences from adults, applying PCNL in pediatric patients using only ultrasound may decrease the success rate. Puncture with ultrasound significantly reduces the radiation dose in children. Puncture with ultrasound and dilation under fluoroscopy is a successful and safe treatment method with low morbidity and high success rates and shorter hospital stay in pediatric patients.

Multispectral Imaging for Automated Tissue Identification of Normal Human Surgical Specimens.

OBJECTIVE: Safe surgery requires the accurate discrimination of tissue intraoperatively. We assess the feasibility of using multispectral imaging and deep learning to enhance surgical vision by automated identification of normal human head and neck tissues. STUDY DESIGN: Construction and feasibility testing of novel multispectral imaging system for surgery. SETTING: Academic university hospital. SUBJECTS AND METHODS: Multispectral images of fresh-preserved human cadaveric tissues were captured with our adapted digital operating microscope. Eleven tissue types were sampled, each sequentially exposed to 6 lighting conditions. Two convolutional neural network machine learning models were developed to classify tissues based on multispectral and white-light color images (ARRInet-M and ARRInet-W, respectively). Blinded otolaryngology residents were asked to identify tissue specimens from white-light color images, and their performance was compared with that of the ARRInet models. RESULTS: A novel multispectral imaging system was developed with minimal adaptation to an existing digital operating microscope. With 81.8% accuracy in tissue identification of full-size images, the multispectral ARRInet-M classifier outperformed the white-light-only ARRInet-W model (45.5%) and surgical residents (69.7%). Challenges with discrimination occurred with parotid vs fat and blood vessels vs nerve. CONCLUSIONS: A deep learning model using multispectral imaging outperformed a similar model and surgical residents using traditional white-light imaging at the task of classifying normal human head and neck tissue ex vivo. These results suggest that multispectral imaging can enhance surgical vision and augment surgeons' ability to identify tissues during a procedure.

Bio-inspired multimodal 3D endoscope for image-guided and robotic surgery.

Image-guided and robotic surgery based on endoscopic imaging technologies can enhance cancer treatment by ideally removing all cancerous tissue and avoiding iatrogenic damage to healthy tissue. Surgeons evaluate the tumor margins at the cost of impeding surgical workflow or working with dimmed surgical illumination, since current endoscopic imaging systems cannot simultaneous and real-time color and near-infrared (NIR) fluorescence imaging under normal surgical illumination. To overcome this problem, a bio-inspired multimodal 3D endoscope combining the excellent characteristics of human eyes and compound eyes of mantis shrimp is proposed. This 3D endoscope, which achieves simultaneous and real-time imaging of three-dimensional stereoscopic, color, and NIR fluorescence, consists of three parts: a broad-band binocular optical system like as human eye, an optical relay system, and a multiband sensor inspired by the mantis shrimp's compound eye. By introducing an optical relay system, the two sub-images after the broad-band binocular optical system can be projected onto one and the same multiband sensor. A series of experiments demonstrate that this bio-inspired multimodal 3D endoscope not only provides surgeons with real-time feedback on the location of tumor tissue and lymph nodes but also creates an immersive experience for surgeons without impeding surgical workflow. Its excellent characteristics and good scalability can promote the further development and application of image-guided and robotic surgery.

Laser-driven x-ray and proton micro-source and application to simultaneous single-shot bi-modal radiographic imaging.

Radiographic imaging with x-rays and protons is an omnipresent tool in basic research and applications in industry, material science and medical diagnostics. The information contained in both modalities can often be valuable in principle, but difficult to access simultaneously. Laser-driven solid-density plasma-sources deliver both kinds of radiation, but mostly single modalities have been explored for applications. Their potential for bi-modal radiographic imaging has never been fully realized, due to problems in generating appropriate sources and separating image modalities. Here, we report on the generation of proton and x-ray micro-sources in laser-plasma interactions of the focused Texas Petawatt laser with solid-density, micrometer-sized tungsten needles. We apply them for bi-modal radiographic imaging of biological and technological objects in a single laser shot. Thereby, advantages of laser-driven sources could be enriched beyond their small footprint by embracing their additional unique properties, including the spectral bandwidth, small source size and multi-mode emission.

High Resolution Multimodal Photoacoustic Microscopy and Optical Coherence Tomography Visualization of Choroidal Vascular Occlusion.

Photoacoustic microscopy is a novel, non-ionizing, non-invasive imaging technology that evaluates tissue absorption of short-pulsed light through the sound waves emitted by the tissue and has numerous biomedical applications. In this study, a custom-built multimodal imaging system, including photoacoustic microscopy (PAM) and optical coherence tomography (OCT), has been developed to evaluate choroidal vascular occlusion (CVO). CVO was performed on three living rabbits using laser photocoagulation. Longitudinal imaging of CVO was obtained using multiple imaging tools such as color fundus photography, fluorescein angiography, indocyanine green angiography (ICGA), OCT, and PAM. PAM images were acquired at different wavelengths, ranging from 532 to 700 nm. The results demonstrate that the CVO was clearly observed on PAM in both two dimensions (2D) and 3D with high resolution longitudinally over 28 days. In addition, the location and margin of the CVO were distinguished from the surrounding choroidal vasculature after the injection of ICG contrast agent. PAM imaging was achieved using a laser energy of approximately 80 nJ, which is about half of the American National Standards Institute safety limit. The proposed imaging technique may provide a potential tool for the evaluation of different chorioretinal vascular disease pathogeneses and other biological studies.