In vivo laser speckle contrast imaging of microvascular blood perfusion using a chip-on-tip camera.

Laser speckle contrast imaging (LSCI) is an important non-invasive capability for real-time imaging for tissue-perfusion assessment. Yet, the size and weight of current clinical standard LSCI instrumentation restricts usage to mainly peripheral skin perfusion. Miniaturization of LSCI could enable hand-held instrumentation to image internal organ/tissue to produce accurate speckle-perfusion maps. We characterized a 1mm2 chip-on-tip camera for LSCI of blood perfusion in vivo and with a flow model. A dedicated optical setup was built to compare chip-on-tip camera to a high specification reference camera (GS3) for LSCI. We compared LSCI performance using a calibration standard and a flow phantom. Subsequently the camera assessed placenta perfusion in a small animal model. Lastly, a human study was conducted on the perfusion in fingertips of 13-volunteers. We demonstrate that the chip-on-tip camera can perform wide-field, in vivo, LSCI of tissue perfusion with the ability to measure physiological blood flow changes comparable with a standard reference camera.

Fiber Bundle Image Reconstruction Using Convolutional Neural Networks and Bundle Rotation in Endomicroscopy.

Fiber-bundle endomicroscopy has several recognized drawbacks, the most prominent being the honeycomb effect. We developed a multi-frame super-resolution algorithm exploiting bundle rotation to extract features and reconstruct underlying tissue. Simulated data was used with rotated fiber-bundle masks to create multi-frame stacks to train the model. Super-resolved images are numerically analyzed, which demonstrates that the algorithm can restore images with high quality. The mean structural similarity index measurement (SSIM) improved by a factor of 1.97 compared with linear interpolation. The model was trained using images taken from a single prostate slide, 1343 images were used for training, 336 for validation, and 420 for testing. The model had no prior information about the test images, adding to the robustness of the system. Image reconstruction was completed in 0.03 s for 256 × 256 images indicating future real-time performance is within reach. The combination of fiber bundle rotation and multi-frame image enhancement through machine learning has not been utilized before in an experimental setting but could provide a much-needed improvement to image resolution in practice.

Molecular detection of Gram-positive bacteria in the human lung through an optical fiber-based endoscope.

PURPOSE: The relentless rise in antimicrobial resistance is a major societal challenge and requires, as part of its solution, a better understanding of bacterial colonization and infection. To facilitate this, we developed a highly efficient no-wash red optical molecular imaging agent that enables the rapid, selective, and specific visualization of Gram-positive bacteria through a bespoke optical fiber-based delivery/imaging endoscopic device. METHODS: We rationally designed a no-wash, red, Gram-positive-specific molecular imaging agent (Merocy-Van) based on vancomycin and an environmental merocyanine dye. We demonstrated the specificity and utility of the imaging agent in escalating in vitro and ex vivo whole human lung models (n = 3), utilizing a bespoke fiber-based delivery and imaging device, coupled to a wide-field, two-color endomicroscopy system. RESULTS: The imaging agent (Merocy-Van) was specific to Gram-positive bacteria and enabled no-wash imaging of S. aureus within the alveolar space of whole ex vivo human lungs within 60 s of delivery into the field-of-view, using the novel imaging/delivery endomicroscopy device. CONCLUSION: This platform enables the rapid and specific detection of Gram-positive bacteria in the human lung.

Quantitative real-time imaging of intracellular FRET biosensor dynamics using rapid multi-beam confocal FLIM.

Fluorescence lifetime imaging (FLIM) is a quantitative, intensity-independent microscopical method for measurement of diverse biochemical and physical properties in cell biology. It is a highly effective method for measurements of Förster resonance energy transfer (FRET), and for quantification of protein-protein interactions in cells. Time-domain FLIM-FRET measurements of these dynamic interactions are particularly challenging, since the technique requires excellent photon statistics to derive experimental parameters from the complex decay kinetics often observed from fluorophores in living cells. Here we present a new time-domain multi-confocal FLIM instrument with an array of 64 visible beamlets to achieve parallelised excitation and detection with average excitation powers of ~ 1-2 µW per beamlet. We exemplify this instrument with up to 0.5 frames per second time-lapse FLIM measurements of cAMP levels using an Epac-based fluorescent biosensor in live HeLa cells with nanometer spatial and picosecond temporal resolution. We demonstrate the use of time-dependent phasor plots to determine parameterisation for multi-exponential decay fitting to monitor the fractional contribution of the activated conformation of the biosensor. Our parallelised confocal approach avoids having to compromise on speed, noise, accuracy in lifetime measurements and provides powerful means to quantify biochemical dynamics in living cells.

High-speed dual color fluorescence lifetime endomicroscopy for highly-multiplexed pulmonary diagnostic applications and detection of labeled bacteria.

We present a dual-color laser scanning endomicroscope capable of fluorescence lifetime endomicroscopy at one frame per second (FPS). The scanning system uses a coherent imaging fiber with 30,000 cores. High-speed lifetime imaging is achieved by distributing the signal over an array of 1024 parallel single-photon avalanche diode detectors (SPADs), minimizing detection dead-time maximizing the number of photons detected per excitation pulse without photon pile-up to achieve the high frame rate. This also enables dual color fluorescence imaging by temporally shifting the dual excitation lasers, with respect to each other, to separate the two spectrally distinct fluorescent decays in time. Combining the temporal encoding, to provide spectral separation, with lifetime measurements we show a one FPS, multi-channel endomicroscopy platform for clinical applications and diagnosis. We demonstrate the potential of the system by imaging SmartProbe labeled bacteria in ex vivo samples of human lung using lifetime to differentiate bacterial fluorescence from the strong background lung autofluorescence which was used to provide structural information.

Multifocal multiphoton volumetric imaging approach for high-speed time-resolved Förster resonance energy transfer imaging in vivo.

In this Letter, we will discuss the development of a multifocal multiphoton fluorescent lifetime imaging system where four individual fluorescent intensity and lifetime planes are acquired simultaneously, allowing us to obtain volumetric data without the need for sequential scanning at different axial depths. Using a phase-only spatial light modulator (SLM) with an appropriate algorithm to generate a holographic pattern, we project a beamlet array within a sample volume of a size, which can be preprogrammed by the user. We demonstrate the capabilities of the system to image live-cell interactions. While only four planes are shown, this technique can be rescaled to a large number of focal planes, enabling full 3D acquisition and reconstruction.

Bimodal fluorogenic sensing of matrix proteolytic signatures in lung cancer.

Optical biosensing based on the activation of fluorescent reporters offers a powerful methodology for the real-time molecular interrogation of pathology. Here we report a first-in-class, bimodal fluorescent reporter strategy for the simultaneous and highly specific detection of two independent proteases (thrombin and matrix metalloproteases (MMPs)) pivotal in the fibroproliferative process surrounding lung cancer, based on a dual, multiplexing, peptide FRET system. This sophisticated synthetic smartprobe, with a molecular weight of 6 kDa, contains two independent fluorophores and quenchers that generate photonic signatures at two specific wavelengths upon activation by target enzymes within human lung cancer tissue.

Low-cost high sensitivity pulsed endomicroscopy to visualize tricolor optical signatures.

A highly sensitive, modular three-color fluorescence endomicroscopy imaging platform spanning the visible to near-infrared (NIR) range is demonstrated. Light-emitting diodes (LEDs) were sequentially pulsed along with the camera acquisition to provide up to 20 frames per second (fps) three-color imaging performance or 60 fps single color imaging. The system was characterized for bacterial and cellular molecular imaging in ex vivo human lung tissue and for bacterial and indocyanine green imaging in ex vivo perfused sheep lungs. A practical method to reduce background tissue autofluorescence is also proposed. The platform was clinically translated into six patients with pulmonary disease to delineate healthy, cancerous, and fibrotic tissue autofluorescent structures. The instrument is the most broadband clinical endomicroscopy system developed to date (covering visible to the NIR, 500 to 900 nm) and demonstrates significant potential for future clinical utility due to its low cost and modular capability to suit a wide variety of molecular imaging applications.

Multiplexed single-mode wavelength-to-time mapping of multimode light.

When an optical pulse propagates along an optical fibre, different wavelengths travel at different group velocities. As a result, wavelength information is converted into arrival-time information, a process known as wavelength-to-time mapping. This phenomenon is most cleanly observed using a single-mode fibre transmission line, where spatial mode dispersion is not present, but the use of such fibres restricts possible applications. Here we demonstrate that photonic lanterns based on tapered single-mode multicore fibres provide an efficient way to couple multimode light to an array of single-photon avalanche detectors, each of which has its own time-to-digital converter for time-correlated single-photon counting. Exploiting this capability, we demonstrate the multiplexed single-mode wavelength-to-time mapping of multimode light using a multicore fibre photonic lantern with 121 single-mode cores, coupled to 121 detectors on a 32 × 32 detector array. This work paves the way to efficient multimode wavelength-to-time mapping systems with the spectral performance of single-mode systems.

New high-speed centre of mass method incorporating background subtraction for accurate determination of fluorescence lifetime.

We demonstrate an implementation of a centre-of-mass method (CMM) incorporating background subtraction for use in multifocal fluorescence lifetime imaging microscopy to accurately determine fluorescence lifetime in live cell imaging using the Megaframe camera. The inclusion of background subtraction solves one of the major issues associated with centre-of-mass approaches, namely the sensitivity of the algorithm to background signal. The algorithm, which is predominantly implemented in hardware, provides real-time lifetime output and allows the user to effectively condense large amounts of photon data. Instead of requiring the transfer of thousands of photon arrival times, the lifetime is simply represented by one value which allows the system to collect data up to limit of pulse pile-up without any limitations on data transfer rates. In order to evaluate the performance of this new CMM algorithm with existing techniques (i.e. rapid lifetime determination and Levenburg-Marquardt), we imaged live MCF-7 human breast carcinoma cells transiently transfected with FRET standards. We show that, it offers significant advantages in terms of lifetime accuracy and insensitivity to variability in dark count rate (DCR) between Megaframe camera pixels. Unlike other algorithms no prior knowledge of the expected lifetime is required to perform lifetime determination. The ability of this technique to provide real-time lifetime readout makes it extremely useful for a number of applications.

Electrophysiological and Anatomical Correlates of Spinal Cord Optical Coherence Tomography.

Despite the continuous improvement in medical imaging technology, visualizing the spinal cord poses severe problems due to structural or incidental causes, such as small access space and motion artifacts. In addition, positional guidance on the spinal cord is not commonly available during surgery, with the exception of neuronavigation techniques based on static pre-surgical data and of radiation-based methods, such as fluoroscopy. A fast, bedside, intraoperative real-time imaging, particularly necessary during the positioning of endoscopic probes or tools, is an unsolved issue. The objective of our work, performed on experimental rats, is to demonstrate potential intraoperative spinal cord imaging and probe guidance by optical coherence tomography (OCT). Concurrently, we aimed to demonstrate that the electromagnetic OCT irradiation exerted no particular effect at the neuronal and synaptic levels. OCT is a user-friendly, low-cost and endoscopy-compatible photonics-based imaging technique. In particular, by using a Fourier-domain OCT imager, operating at 850 nm wavelength and scanning transversally with respect to the spinal cord, we have been able to: 1) accurately image tissue structures in an animal model (muscle, spine bone, cerebro-spinal fluid, dura mater and spinal cord), and 2) identify the position of a recording microelectrode approaching and inserting into the cord tissue 3) check that the infrared radiation has no actual effect on the electrophysiological activity of spinal neurons. The technique, potentially extendable to full three-dimensional image reconstruction, shows prospective further application not only in endoscopic intraoperative analyses and for probe insertion guidance, but also in emergency and adverse situations (e.g. after trauma) for damage recognition, diagnosis and fast image-guided intervention.

Two-color widefield fluorescence microendoscopy enables multiplexed molecular imaging in the alveolar space of human lung tissue.

We demonstrate a fast two-color widefield fluorescence microendoscopy system capable of simultaneously detecting several disease targets in intact human ex vivo lung tissue. We characterize the system for light throughput from the excitation light emitting diodes, fluorescence collection efficiency, and chromatic focal shifts. We demonstrate the effectiveness of the instrument by imaging bacteria (Pseudomonas aeruginosa) in ex vivo human lung tissue. We describe a mechanism of bacterial detection through the fiber bundle that uses blinking effects of bacteria as they move in front of the fiber core providing detection of objects smaller than the fiber core and cladding (∼3 µm ∼3 µm ). This effectively increases the measured spatial resolution of 4 µm 4 µm . We show simultaneous imaging of neutrophils, monocytes, and fungus (Aspergillus fumigatus) in ex vivo human lung tissue. The instrument has 10 nM and 50 nM sensitivity for fluorescein and Cy5 solutions, respectively. Lung tissue autofluorescence remains visible at up to 200 fps camera acquisition rate. The optical system lends itself to clinical translation due to high-fluorescence sensitivity, simplicity, and the ability to multiplex several pathological molecular imaging targets simultaneously.

Real-time fluorescence lifetime actuation for cell sorting using a CMOS SPAD silicon photomultiplier.

Time-correlated single photon counting (TCSPC) is a fundamental fluorescence lifetime measurement technique offering high signal to noise ratio (SNR). However, its requirement for complex software algorithms for histogram processing restricts throughput in flow cytometers and prevents on-the-fly sorting of cells. We present a single-point digital silicon photomultiplier (SiPM) detector accomplishing real-time fluorescence lifetime-activated actuation targeting cell sorting applications in flow cytometry. The sensor also achieves burst-integrated fluorescence lifetime (BIFL) detection by TCSPC. The SiPM is a single-chip complementary metal-oxide-semiconductor (CMOS) sensor employing a 32×32 single-photon avalanche diode (SPAD) array and eight pairs of time-interleaved time to digital converters (TI-TDCs) with a 50 ps minimum timing resolution. The sensor's pile-up resistant embedded center of mass method (CMM) processor accomplishes low-latency measurement and thresholding of fluorescence lifetime. A digital control signal is generated with a 16.6 µs latency for cell sorter actuation allowing a maximum cell throughput of 60,000 cells per second and an error rate of 0.6%.

0.5 billion events per second time correlated single photon counting using CMOS SPAD arrays.

We present a digital architecture for fast acquisition of time correlated single photon counting (TCSPC) events from a 32×32 complementary metal oxide semiconductor (CMOS) single photon avalanche detector (SPAD) array (Megaframe) to the computer memory. Custom firmware was written to transmit event codes from 1024-TCSPC-enabled pixels for fast transfer of TCSPC events. Our 1024-channel TCSPC system is capable of acquiring up to 0.5×10(9) TCSPC events per second with 16 histogram bins spanning a 14 ns width. Other options include 320×10(6) TCSPC events per second with 256 histogram bins spanning either a 14 or 56 ns time window. We present a wide-field fluorescence microscopy setup demonstrating fast fluorescence lifetime data acquisition. To the best of our knowledge, this is the fastest direct TCSPC transfer from a single photon counting device to the computer to date.

256 × 2 SPAD line sensor for time resolved fluorescence spectroscopy.

We present a CMOS chip 256 × 2 single photon avalanche diode (SPAD) line sensor, 23.78 µm pitch, 43.7% fill factor, custom designed for time resolved emission spectroscopy (TRES). Integrating time-to-digital converters (TDCs) implement on-chip mono-exponential fluorescence lifetime pre-calculation allowing timing of 65k photons/pixel at 200 Hz line rate at 40 ps resolution using centre-of-mass method (CMM). Per pixel time-correlated single-photon counting (TCSPC) histograms can also be generated with 320 ps bin resolution. We characterize performance in terms of dark count rate, instrument response function and lifetime uniformity for a set of fluorophores with lifetimes ranging from 4 ns to 6 ns. Lastly, we present fluorescence lifetime spectra of multicolor microspheres and skin autofluorescence acquired using a custom built spectrometer. In TCSPC mode, time-resolved spectra are acquired within 5 minutes whilst in CMM mode spectral lifetime signatures are acquired within 2 ms for fluorophore in cuvette and 200 ms for skin autofluorescence. We demonstrate CMOS line sensors to be a versatile tool for time-resolved fluorescence spectroscopy by providing parallelized and flexible spectral detection of fluorescence decay.

A high speed multifocal multiphoton fluorescence lifetime imaging microscope for live-cell FRET imaging.

We demonstrate diffraction limited multiphoton imaging in a massively parallel, fully addressable time-resolved multi-beam multiphoton microscope capable of producing fluorescence lifetime images with sub-50ps temporal resolution. This imaging platform offers a significant improvement in acquisition speed over single-beam laser scanning FLIM by a factor of 64 without compromising in either the temporal or spatial resolutions of the system. We demonstrate FLIM acquisition at 500 ms with live cells expressing green fluorescent protein. The applicability of the technique to imaging protein-protein interactions in live cells is exemplified by observation of time-dependent FRET between the epidermal growth factor receptor (EGFR) and the adapter protein Grb2 following stimulation with the receptor ligand. Furthermore, ligand-dependent association of HER2-HER3 receptor tyrosine kinases was observed on a similar timescale and involved the internalisation and accumulation or receptor heterodimers within endosomes. These data demonstrate the broad applicability of this novel FLIM technique to the spatio-temporal dynamics of protein-protein interaction.

Single-photon sensitive light-in-fight imaging.

The ability to record images with extreme temporal resolution enables a diverse range of applications, such as fluorescence lifetime imaging, time-of-flight depth imaging and characterization of ultrafast processes. Recently, ultrafast imaging schemes have emerged, which require either long acquisition times or raster scanning and have a requirement for sufficient signal that can only be achieved when light is reflected off an object or diffused by a strongly scattering medium. Here we present a demonstration of the potential of single-photon detector arrays for visualization and rapid characterization of events evolving on picosecond time scales. The single-photon sensitivity, temporal resolution and full-field imaging capability enables the observation of light-in-flight in air, as well as the measurement of laser-induced plasma formation and dynamics in its natural environment. The extreme sensitivity and short acquisition times pave the way for real-time imaging of ultrafast processes or visualization and tracking of objects hidden from view.

Time-resolved multifocal multiphoton microscope for high speed FRET imaging in vivo.

Imaging the spatiotemporal interaction of proteins in vivo is essential to understanding the complexities of biological systems. The highest accuracy monitoring of protein-protein interactions is achieved using Förster resonance energy transfer (FRET) measured by fluorescence lifetime imaging, with measurements taking minutes to acquire a single frame, limiting their use in dynamic live cell systems. We present a diffraction limited, massively parallel, time-resolved multifocal multiphoton microscope capable of producing fluorescence lifetime images with 55 ps time-resolution, giving improvements in acquisition speed of a factor of 64. We present demonstrations with FRET imaging in a model cell system and demonstrate in vivo FLIM using a GTPase biosensor in the zebrafish embryo.

Development of a doubly weighted Gerchberg-Saxton algorithm for use in multibeam imaging applications.

We report on the development of a doubly weighted Gerchberg-Saxton algorithm (DWGS) to enable generation of uniform beamlet arrays with a spatial light modulator (SLM) for use in multiphoton multifocal imaging applications. The algorithm incorporates the WGS algorithm as well as feedback of fluorescence signals from the sample measured with a single-photon avalanche diode (SPAD) detector array. This technique compensates for issues associated with nonuniform illumination onto the SLM, the effects due to aberrations and the variability in gain between detectors within the SPAD array to generate a uniformly illuminated multiphoton fluorescence image. We demonstrate the use of the DWGS with a number of beamlet array patterns to image muscle fibers of a 5-day-old fixed zebrafish larvae.