Smartphone-based optical sectioning (SOS) microscopy with a telecentric design for fluorescence imaging.

We propose a smartphone-based optical sectioning (SOS) microscope based on the HiLo technique, with a single smartphone replacing a high-cost illumination source and a camera sensor. We built our SOS with off-the-shelf optical, mechanical cage systems with 3D-printed adapters to seamlessly integrate the smartphone with the SOS main body. The liquid light guide can be integrated with the adapter, guiding the smartphone's LED light to the digital mirror device (DMD) with neglectable loss. We used an electrically tuneable lens (ETL) instead of a mechanical translation stage to realise low-cost axial scanning. The ETL was conjugated to the objective lens's back pupil plane (BPP) to construct a telecentric design by a 4f configuration to maintain the lateral magnification for different axial positions. SOS has a 571.5 µm telecentric scanning range and an 11.7 µm axial resolution. The broadband smartphone LED torch can effectively excite fluorescent polystyrene (PS) beads. We successfully used SOS for high-contrast fluorescent PS beads imaging with different wavelengths and optical sectioning imaging of multilayer fluorescent PS beads. To our knowledge, the proposed SOS is the first smartphone-based HiLo optical sectioning microscopy (£1965), which can save around £7035 compared with a traditional HiLo system (£9000). It is a powerful tool for biomedical research in resource-limited areas.

Compact and robust deep learning architecture for fluorescence lifetime imaging and FPGA implementation.

This paper reports a bespoke adder-based deep learning network for time-domain fluorescence lifetime imaging (FLIM). By leveraging thel1-norm extraction method, we propose a 1D Fluorescence Lifetime AdderNet (FLAN) without multiplication-based convolutions to reduce the computational complexity. Further, we compressed fluorescence decays in temporal dimension using a log-scale merging technique to discard redundant temporal information derived as log-scaling FLAN (FLAN+LS). FLAN+LS achieves 0.11 and 0.23 compression ratios compared with FLAN and a conventional 1D convolutional neural network (1D CNN) while maintaining high accuracy in retrieving lifetimes. We extensively evaluated FLAN and FLAN+LS using synthetic and real data. A traditional fitting method and other non-fitting, high-accuracy algorithms were compared with our networks for synthetic data. Our networks attained a minor reconstruction error in different photon-count scenarios. For real data, we used fluorescent beads' data acquired by a confocal microscope to validate the effectiveness of real fluorophores, and our networks can differentiate beads with different lifetimes. Additionally, we implemented the network architecture on a field-programmable gate array (FPGA) with a post-quantization technique to shorten the bit-width, thereby improving computing efficiency. FLAN+LS on hardware achieves the highest computing efficiency compared to 1D CNN and FLAN. We also discussed the applicability of our network and hardware architecture for other time-resolved biomedical applications using photon-efficient, time-resolved sensors.

Simple and Robust Deep Learning Approach for Fast Fluorescence Lifetime Imaging.

Fluorescence lifetime imaging (FLIM) is a powerful tool that provides unique quantitative information for biomedical research. In this study, we propose a multi-layer-perceptron-based mixer (MLP-Mixer) deep learning (DL) algorithm named FLIM-MLP-Mixer for fast and robust FLIM analysis. The FLIM-MLP-Mixer has a simple network architecture yet a powerful learning ability from data. Compared with the traditional fitting and previously reported DL methods, the FLIM-MLP-Mixer shows superior performance in terms of accuracy and calculation speed, which has been validated using both synthetic and experimental data. All results indicate that our proposed method is well suited for accurately estimating lifetime parameters from measured fluorescence histograms, and it has great potential in various real-time FLIM applications.

Hardware Inspired Neural Network for Efficient Time-Resolved Biomedical Imaging.

Convolutional neural networks (CNN) have revealed exceptional performance for fluorescence lifetime imaging (FLIM). However, redundant parameters and complicated topologies make it challenging to implement such networks on embedded hardware to achieve real-time processing. We report a lightweight, quantized neural architecture that can offer fast FLIM imaging. The forward-propagation is significantly simplified by replacing matrix multiplications in each convolution layer with additions and data quantization using a low bit-width. We first used synthetic 3-D lifetime data with given lifetime ranges and photon counts to assure correct average lifetimes can be obtained. Afterwards, human prostatic cancer cells incubated with gold nanoprobes were utilized to validate the feasibility of the network for real-world data. The quantized network yielded a 37.8% compression ratio without performance degradation. Clinical relevance - This neural network can be applied to diagnose cancer early based on fluorescence lifetime in a non-invasive way. This approach brings high accuracy and accelerates diagnostic processes for clinicians who are not experts in biomedical signal processing.

Smart Wide-field Fluorescence Lifetime Imaging System with CMOS Single-photon Avalanche Diode Arrays.

Wide-field fluorescence lifetime imaging (FLIM) is a promising technique for biomedical and clinic applications. Integrating with CMOS single-photon avalanche diode (SPAD) sensor arrays can lead to cheaper and portable real-time FLIM systems. However, the FLIM data obtained by such sensor systems often have sophisticated noise features. There is still a lack of fast tools to recover lifetime parameters from highly noise-corrupted fluorescence signals efficiently. This paper proposes a smart wide-field FLIM system containing a 192×128 COMS SPAD sensor and a field-programmable gate array (FPGA) embedded deep learning (DL) FLIM processor. The processor adopts a hardware-friendly and light-weighted neural network for fluorescence lifetime analysis, showing the advantages of high accuracy against noise, fast speed, and low power consumption. Experimental results demonstrate the proposed system's superior and robust performances, promising for many FLIM applications such as FLIM-guided clinical surgeries, cancer diagnosis, and biomedical imaging.

Biomimetic apposition compound eye fabricated using microfluidic-assisted 3D printing.

After half a billion years of evolution, arthropods have developed sophisticated compound eyes with extraordinary visual capabilities that have inspired the development of artificial compound eyes. However, the limited 2D nature of most traditional fabrication techniques makes it challenging to directly replicate these natural systems. Here, we present a biomimetic apposition compound eye fabricated using a microfluidic-assisted 3D-printing technique. Each microlens is connected to the bottom planar surface of the eye via intracorporal, zero-crosstalk refractive-index-matched waveguides to mimic the rhabdoms of a natural eye. Full-colour wide-angle panoramic views and position tracking of a point source are realized by placing the fabricated eye directly on top of a commercial imaging sensor. As a biomimetic analogue to naturally occurring compound eyes, the eye's full-colour 3D to 2D mapping capability has the potential to enable a wide variety of applications from improving endoscopic imaging to enhancing machine vision for facilitating human-robot interactions.

Ultrasound and Near-Infrared Light Dual-Triggered Upconversion Zeolite-Based Nanocomposite for Hyperthermia-Enhanced Multimodal Melanoma Therapy via a Precise Apoptotic Mechanism.

High reactive oxygen species (ROS) generation efficiency and induction of targeted cell apoptosis are recognized as key objectives to achieve a highly efficient strategy for cancer therapy with minimum side effects of inflammatory reactions. However, it is still a challenge to realize higher therapeutic efficiency with a cell apoptosis model. Herein, we present strong upconversion luminescent biosafe cores derived from Linde Type A (LTA) zeolites and modification with targeted/therapeutic drugs for multimodal therapy, in which sonodynamic therapy (SDT) combined with photodynamic therapy (PDT) increases therapeutic efficiency especially in deep sites of tumor via producing cytoplasmic ROS and mitochondrial superoxide and photothermal therapy (PTT) enhances PDT effects via higher fluorescence resonance energy transfer (FRET) efficacy attributed to an increased temperature. Furthermore, the transcriptomic analysis reveals that cellular internalization of the nanosystem can lead to tumor ablation via cell apoptosis. We expect that the multimodal therapy based on LTA zeolite drug nanocarriers could be applied in the cancer therapeutics in the near future.

Colour compound lenses for a portable fluorescence microscope.

In this article, we demonstrated a handheld smartphone fluorescence microscope (HSFM) that integrates dual-functional polymer lenses with a smartphone. The HSFM consists of a smartphone, a field-portable illumination source, and a dual-functional polymer lens that performs both optical imaging and filtering. Therefore, compared with the existing smartphone fluorescence microscope, the HSFM does not need any additional optical filters. Although fluorescence imaging has traditionally played an indispensable role in biomedical and clinical applications due to its high specificity and sensitivity for detecting cells, proteins, DNAs/RNAs, etc., the bulky elements of conventional fluorescence microscopes make them inconvenient for use in point-of-care diagnosis. The HSFM demonstrated in this article solves this problem by providing a multifunctional, miniature, small-form-factor fluorescence module. This multifunctional fluorescence module can be seamlessly attached to any smartphone camera for both bright-field and fluorescence imaging at cellular-scale resolutions without the use of additional bulky lenses/filters; in fact, the HSFM achieves magnification and light filtration using a single lens. Cell and tissue observation, cell counting, plasmid transfection evaluation, and superoxide production analysis were performed using this device. Notably, this lens system has the unique capability of functioning with numerous smartphones, irrespective of the smartphone model and the camera technology housed within each device. As such, this HSFM has the potential to pave the way for real-time point-of-care diagnosis and opens up countless possibilities for personalized medicine.

Fabrication of large micro-structured high-numerical-aperture optofluidic compound eyes with tunable angle of view.

A large optofluidic compound eye is developed by using a straightforward, rapid, and low-cost technique. The compound eye's angle of view can be adjusted by injecting deionized water/calcium chloride solution of different volume into the optofluidic chip. Optofluidic compound eyes containing about 78,000 microlenses of 50 µm diameter are fabricated for analysis. The angle of view can be tuned up to 104°. With the compound eye's deformation, the microlenses' focal length increases, due to the variation in profile. Owing to the non-uniform strain over the compound eye, the central lenses experience more variation. Furthermore, optical imaging of the compound eye is demonstrated and sharp images can be obtained from the omnidirectional microlenses.