Temporal focusing multiphoton microscopy with cross-modality multi-stage 3D U-Net for fast and clear bioimaging.

Temporal focusing multiphoton excitation microscopy (TFMPEM) enables fast widefield biotissue imaging with optical sectioning. However, under widefield illumination, the imaging performance is severely degraded by scattering effects, which induce signal crosstalk and a low signal-to-noise ratio in the detection process, particularly when imaging deep layers. Accordingly, the present study proposes a cross-modality learning-based neural network method for performing image registration and restoration. In the proposed method, the point-scanning multiphoton excitation microscopy images are registered to the TFMPEM images by an unsupervised U-Net model based on a global linear affine transformation process and local VoxelMorph registration network. A multi-stage 3D U-Net model with a cross-stage feature fusion mechanism and self-supervised attention module is then used to infer in-vitro fixed TFMPEM volumetric images. The experimental results obtained for in-vitro drosophila mushroom body (MB) images show that the proposed method improves the structure similarity index measures (SSIMs) of the TFMPEM images acquired with a 10-ms exposure time from 0.38 to 0.93 and 0.80 for shallow- and deep-layer images, respectively. A 3D U-Net model, pretrained on in-vitro images, is further trained using a small in-vivo MB image dataset. The transfer learning network improves the SSIMs of in-vivo drosophila MB images captured with a 1-ms exposure time to 0.97 and 0.94 for shallow and deep layers, respectively.

Dual-resonant scanning multiphoton microscope with ultrasound lens and resonant mirror for rapid volumetric imaging.

A dual-resonant scanning multiphoton (DRSM) microscope incorporating a tunable acoustic gradient index of refraction lens with a resonant mirror is developed for high-speed volumetric imaging. In the proposed microscope, the pulse train signal of a femtosecond laser is used to trigger an embedded field programmable gate array to sample the multiphoton excited fluorescence signal at the rate of one pixel per laser pulse. It is shown that a frame rate of around 8000 Hz can be obtained in the x-z plane for an image region with a size of 256 × 80 pixels. Moreover, a volumetric imaging rate of over 30 Hz can be obtained for a large image volume of 343 × 343 × 120 µm3 with an image size of 256 × 256 × 80 voxels. Moreover, a volumetric imaging rate of over 30 Hz can be obtained for a large image volume of 256 × 256 × 80 voxels, which represents 343 × 343 × 120 µm3 in field-of-view. The rapid volumetric imaging rate eliminates the aliasing effect for observed temporal frequencies lower than 15 Hz. The practical feasibility of the DRSM microscope is demonstrated by observing the mushroom bodies of a drosophila brain and performing 3D dynamic observations of moving 10-µm fluorescent beads.

Image improvement of temporal focusing multiphoton microscopy via superior spatial modulation excitation and Hilbert-Huang transform decomposition.

Temporal focusing-based multiphoton excitation microscopy (TFMPEM) just provides the advantage of widefield optical sectioning ability with axial resolution of several micrometers. However, under the plane excitation, the photons emitted from the molecules in turbid tissues undergo scattering, resulting in complicated background noise and an impaired widefield image quality. Accordingly, this study constructs a general and comprehensive numerical model of TFMPEM utilizing Fourier optics and performs simulations to determine the superior spatial frequency and orientation of the structured pattern which maximize the axial excitation confinement. It is shown experimentally that the optimized pattern minimizes the intensity of the out-of-focus signal, and hence improves the quality of the image reconstructed using the Hilbert transform (HT). However, the square-like reflection components on digital micromirror device leads to pattern residuals in the demodulated image when applying high spatial frequency of structured pattern. Accordingly, the HT is replaced with Hilbert-Huang transform (HHT) in order to sift out the low-frequency background noise and pattern residuals in the demodulation process. The experimental results obtained using a kidney tissue sample show that the HHT yields a significant improvement in the TFMPEM image quality.

Three-dimensional-generator U-net for dual-resonant scanning multiphoton microscopy image inpainting and denoising.

A dual-resonant scanning multiphoton (DRSM) microscope incorporating a tunable acoustic gradient index of refraction lens and a resonant mirror is developed for rapid volumetric bioimaging. It is shown that the microscope achieves a volumetric imaging rate up to 31.25 volumes per second (vps) for a scanning volume of up to 200 × 200 × 100 µm3 with 256 × 256 × 128 voxels. However, the volumetric images have a severe negative signal-to-noise ratio (SNR) as a result of a large number of missing voxels for a large scanning volume and the presence of Lissajous patterning residuals. Thus, a modified three-dimensional (3D)-generator U-Net model trained using simulated microbead images is proposed and used to inpaint and denoise the images. The performance of the 3D U-Net model for bioimaging applications is enhanced by training the model with high-SNR in-vitro drosophila brain images captured using a conventional point scanning multiphoton microscope. The trained model shows the ability to produce clear in-vitro drosophila brain images at a rate of 31.25 vps with a SNR improvement of approximately 20 dB over the original images obtained by the DRSM microscope. The training convergence time of the modified U-Net model is just half that of a general 3D U-Net model. The model thus has significant potential for 3D in-vivo bioimaging transfer learning. Through the assistance of transfer learning, the model can be extended to the restoration of in-vivo drosophila brain images with a high image quality and a rapid training time.

Light-field microscopy with temporal focusing multiphoton illumination for scanless volumetric bioimaging.

A temporal focusing multiphoton illumination (TFMI) method is proposed for achieving selective volume illumination (SVI) (i.e., illuminating only the volume of interest) in light-field microscopy (LFM). The proposed method minimizes the background noise of the LFM images and enhances the contrast, and thus improves the imaging quality. Three-dimensional (3D) volumetric imaging is achieved by reconstructing the LFM images using a phase-space deconvolution algorithm. The experimental results obtained using 100-nm fluorescent beads show that the proposed TFMI-LFM system achieves lateral and axial resolutions of 1.2 µm and 1.1 µm, respectively, at the focal plane. Furthermore, the TFMI-LFM system enables 3D images of the single lobe of the drosophila mushroom body with GFP biomarker (OK-107) to be reconstructed in a one-snapshot record.

Temporal focusing multiphoton microscopy with optimized parallel multiline scanning for fast biotissue imaging.

SIGNIFICANCE: Line scanning-based temporal focusing multiphoton microscopy (TFMPM) has superior axial excitation confinement (AEC) compared to conventional widefield TFMPM, but the frame rate is limited due to the limitation of the single line-to-line scanning mechanism. The development of the multiline scanning-based TFMPM requires only eight multiline patterns for full-field uniform multiphoton excitation and it still maintains superior AEC. AIM: The optimized parallel multiline scanning TFMPM is developed, and the performance is verified with theoretical simulation. The system provides a sharp AEC equivalent to the line scanning-based TFMPM, but fewer scans are required. APPROACH: A digital micromirror device is integrated in the TFMPM system and generates the multiline pattern for excitation. Based on the result of single-line pattern with sharp AEC, we can further model the multiline pattern to find the best structure that has the highest duty cycle together with the best AEC performance. RESULTS: The AEC is experimentally improved to 1.7 µm from the 3.5 µm of conventional TFMPM. The adopted multiline pattern is akin to a pulse-width-modulation pattern with a spatial period of four times the diffraction-limited line width. In other words, ideally only four π / 2 spatial phase-shift scans are required to form a full two-dimensional image with superior AEC instead of image-size-dependent line-to-line scanning. CONCLUSIONS: We have demonstrated the developed parallel multiline scanning-based TFMPM has the multiline pattern for sharp AEC and the least scans required for full-field uniform excitation. In the experimental results, the temporal focusing-based multiphoton images of disordered biotissue of mouse skin with improved axial resolution due to the near-theoretical limit AEC are shown to clearly reduce background scattering.

Individualised selection of left-sided breast cancer patients for proton therapy based on cost-effectiveness.

INTRODUCTION: The significantly greater cost of proton therapy compared with X-ray therapy is frequently justified by the expected reduction in normal tissue toxicity. This is often true for indications such as paediatric and skull base cancers. However, the benefit is less clear for other more common indications such as breast cancer, and it is possible that the degree of benefit may vary widely between these patients. The aim of this work was to demonstrate a method of individualised selection of left-sided breast cancer patients for proton therapy based on cost-effectiveness of treatment. METHODS: 16 left-sided breast cancer patients had a treatment plan generated for the delivery of intensity-modulated proton therapy (IMPT) and of intensity-modulated photon therapy (IMRT) with the deep inspiration breath-hold (DIBH) technique. The resulting dosimetric data was used to predict probabilities of tumour control and toxicities for each patient. These probabilities were used in a Markov model to predict costs and the number of quality-adjusted life years expected as a result of each of the two treatments. RESULTS: IMPT was not cost-effective for the majority of patients but was cost-effective where there was a greater risk reduction of second malignancies with IMPT. CONCLUSION: The Markov model predicted that IMPT with DIBH was only cost-effective for selected left-sided breast cancer patients where IMRT resulted in a significantly greater dose to normal tissue. The presented model may serve as a means of evaluating the cost-effectiveness of IMPT on an individual patient basis.

Comparative proton versus photon treatment planning for the Medicare Medical Treatment Overseas Program: The Royal Adelaide Hospital experience.

INTRODUCTION: Australia's first proton beam therapy (PBT) service, The Australian Bragg Centre for Proton Therapy and Research, is scheduled to open in the near future providing PBT for patients closer to home. Patients currently access Commonwealth funding for PBT via the Medicare Medical Treatment Overseas Program (MTOP). Proton versus photon treatment planning is a pre-requisite for the MTOP application. The Royal Adelaide Hospital (RAH) Department of Radiation Oncology has been providing this since 2016. We aim to provide a descriptive overview of our proton versus photon treatment planning process, presenting a summary of the comparative planning results and the treatment pathways selected for the patients referred. METHODS: All patients referred to the RAH for comparative planning between January 2016 and December 2018 were included in the analysis. Comparative plans were generated for each case using Pinnacle or Eclipse treatment planning systems. The planning techniques used and plan quality metrics were reported. RESULTS: Forty three patients were referred for comparative planning. The age range was 1-63 years, with the majority (72%) being paediatric patients (age ≤18 years). Of the 19 cases that have been submitted to MTOP, 16 have been accepted and 3 denied. Two of the accepted cases chose not to travel abroad for PBT. The other 14 cases have received PBT overseas. CONCLUSIONS: The RAH has provided an important service to demonstrate the dosimetric difference between PBT and photon therapy for Australian patients, an important step in supporting the funding of patients for treatment overseas.

Disengagement of motor cortex from movement control during long-term learning.

Motor learning involves reorganization of the primary motor cortex (M1). However, it remains unclear how the involvement of M1 in movement control changes during long-term learning. To address this, we trained mice in a forelimb-based motor task over months and performed optogenetic inactivation and two-photon calcium imaging in M1 during the long-term training. We found that M1 inactivation impaired the forelimb movements in the early and middle stages, but not in the late stage, indicating that the movements that initially required M1 became independent of M1. As previously shown, M1 population activity became more consistent across trials from the early to middle stage while task performance rapidly improved. However, from the middle to late stage, M1 population activity became again variable despite consistent expert behaviors. This later decline in activity consistency suggests dissociation between M1 and movements. These findings suggest that long-term motor learning can disengage M1 from movement control.

Corticostriatal Flow of Action Selection Bias.

The posterior parietal cortex (PPC) performs many functions, including decision making and movement control. It remains unknown which input and output pathways of PPC support different functions. We addressed this issue in mice, focusing on PPC neurons projecting to the dorsal striatum (PPC-STR) and the posterior secondary motor cortex (PPC-pM2). Projection-specific, retrograde labeling showed that PPC-STR and PPC-pM2 represent largely distinct subpopulations, with PPC-STR receiving stronger inputs from association areas and PPC-pM2 receiving stronger sensorimotor inputs. Two-photon calcium imaging during decision making revealed that the PPC-STR population encodes history-dependent choice bias more strongly than PPC-pM2 or general PPC populations. Furthermore, optogenetic inactivation of PPC-STR neurons or their terminals in STR decreased history-dependent bias, while inactivation of PPC-pM2 neurons altered movement kinematics. Therefore, PPC biases action selection through its STR projection while controlling movements through PPC-pM2 neurons. PPC may support multiple functions through parallel subpopulations, each with distinct input-output connectivity.

Temporal focusing-based widefield multiphoton microscopy with spatially modulated illumination for biotissue imaging.

A developed temporal focusing-based multiphoton excitation microscope (TFMPEM) has a digital micromirror device (DMD) which is adopted not only as a blazed grating for light spatial dispersion but also for patterned illumination simultaneously. Herein, the TFMPEM has been extended to implement spatially modulated illumination at structured frequency and orientation to increase the beam coverage at the back-focal aperture of the objective lens. The axial excitation confinement (AEC) of TFMPEM can be condensed from 3.0 µm to 1.5 µm for a 50 % improvement. By using the TFMPEM with HiLo technique as two structured illuminations at the same spatial frequency but different orientation, reconstructed biotissue images according to the condensed AEC structured illumination are shown obviously superior in contrast and better scattering suppression. Picture: TPEF images of the eosin-stained mouse cerebellar cortex by conventional TFMPEM (left), and the TFMPEM with HiLo technique as 1.09 µm-1 spatially modulated illumination at 90° (center) and 0° (right) orientations.

Fast volumetric imaging with patterned illumination via digital micro-mirror device-based temporal focusing multiphoton microscopy.

Temporal focusing multiphoton microscopy (TFMPM) has the advantage of area excitation in an axial confinement of only a few microns; hence, it can offer fast three-dimensional (3D) multiphoton imaging. Herein, fast volumetric imaging via a developed digital micromirror device (DMD)-based TFMPM has been realized through the synchronization of an electron multiplying charge-coupled device (EMCCD) with a dynamic piezoelectric stage for axial scanning. The volumetric imaging rate can achieve 30 volumes per second according to the EMCCD frame rate of more than 400 frames per second, which allows for the 3D Brownian motion of one-micron fluorescent beads to be spatially observed. Furthermore, it is demonstrated that the dynamic HiLo structural multiphoton microscope can reject background noise by way of the fast volumetric imaging with high-speed DMD patterned illumination.

Nonlinear structured-illumination enhanced temporal focusing multiphoton excitation microscopy with a digital micromirror device.

In this study, the light diffraction of temporal focusing multiphoton excitation microscopy (TFMPEM) and the excitation patterning of nonlinear structured-illumination microscopy (NSIM) can be simultaneously and accurately implemented via a single high-resolution digital micromirror device. The lateral and axial spatial resolutions of the TFMPEM are remarkably improved through the second-order NSIM and projected structured light, respectively. The experimental results demonstrate that the lateral and axial resolutions are enhanced from 397 nm to 168 nm (2.4-fold) and from 2.33 µm to 1.22 µm (1.9-fold), respectively, in full width at the half maximum. Furthermore, a three-dimensionally rendered image of a cytoskeleton cell featuring ~25 nm microtubules is improved, with other microtubules at a distance near the lateral resolution of 168 nm also able to be distinguished.

Wavefront sensorless adaptive optics temporal focusing-based multiphoton microscopy.

Temporal profile distortions reduce excitation efficiency and image quality in temporal focusing-based multiphoton microscopy. In order to compensate the distortions, a wavefront sensorless adaptive optics system (AOS) was integrated into the microscope. The feedback control signal of the AOS was acquired from local image intensity maximization via a hill-climbing algorithm. The control signal was then utilized to drive a deformable mirror in such a way as to eliminate the distortions. With the AOS correction, not only is the axial excitation symmetrically refocused, but the axial resolution with full two-photon excited fluorescence (TPEF) intensity is also maintained. Hence, the contrast of the TPEF image of a R6G-doped PMMA thin film is enhanced along with a 3.7-fold increase in intensity. Furthermore, the TPEF image quality of 1µm fluorescent beads sealed in agarose gel at different depths is improved.

Temporal focusing-based multiphoton excitation microscopy via digital micromirror device.

This Letter presents an enhanced temporal focusing-based multiphoton excitation (MPE) microscope in which the conventional diffraction grating is replaced by a digital micromirror device (DMD). Experimental results from imaging a thin fluorescence film show that the 4.0 µm axial resolution of the microscope is comparable with that of a setup incorporating a 600 lines/mm grating; hence, the optical sectioning ability of the proposed setup is demonstrated. Similar to a grating, the DMD diffracts illuminating light frequencies for temporal focusing; additionally, it generates arbitrary patterns. Since the DMD is placed on the image-conjugate plane of the objective lens' focal plane, the MPE pattern can be projected on the focal plane precisely.