Rapid on-site histology of lung and pleural biopsies using higher harmonic generation microscopy and artificial intelligence analysis.

Lung cancer is both one of the most prevalent and lethal cancers. To improve health outcomes while reducing the healthcare burden, it becomes crucial to move towards early detection and cost-effective workflows. Currently there is no method for on-site rapid histological feedback on biopsies taken in diagnostic endoscopic or surgical procedures. Higher harmonic generation (HHG) microscopy is a laser-based technique that provides images of unprocessed tissue. Here, we report the feasibility of a HHG portable microscope in the clinical workflow in terms of acquisition time, image quality and diagnostic accuracy in suspected pulmonary and pleural malignancy. 109 biopsies of 47 patients were imaged and a biopsy overview image was provided within a median of 6 minutes after excision. The assessment by pathologists and an artificial intelligence (AI) algorithm showed that image quality was sufficient for a malignancy or non-malignancy diagnosis in 97% of the biopsies, and 87% of the HHG images were correctly scored by the pathologists. HHG is therefore an excellent candidate to provide rapid pathology outcome on biopsy samples enabling immediate diagnosis and (local) treatment.

Self-Supervised Image Denoising of Third Harmonic Generation Microscopic Images of Human Glioma Tissue by Transformer-Based Blind Spot (TBS) Network.

Third harmonic generation (THG) microscopy shows great potential for instant pathology of brain tumor tissue during surgery. However, due to the maximal permitted exposure of laser intensity and inherent noise of the imaging system, the noise level of THG images is relatively high, which affects subsequent feature extraction analysis. Denoising THG images is challenging for modern deep-learning based methods because of the rich morphologies contained and the difficulty in obtaining the noise-free counterparts. To address this, in this work, we propose an unsupervised deep-learning network for denoising of THG images which combines a self-supervised blind spot method and a U-shape Transformer using a dynamic sparse attention mechanism. The experimental results on THG images of human glioma tissue show that our approach exhibits superior denoising performance qualitatively and quantitatively compared with previous methods. Our model achieves an improvement of 2.47-9.50 dB in SNR and 0.37-7.40 dB in CNR, compared to six recent state-of-the-art unsupervised learning models including Neighbor2Neighbor, Blind2Unblind, Self2Self+, ZS-N2N, Noise2Info and SDAP. To achieve an objective evaluation of our model, we also validate our model on public datasets including natural and microscopic images, and our model shows a better denoising performance than several recent unsupervised models such as Neighbor2Neighbor, Blind2Unblind and ZS-N2N. In addition, our model is nearly instant in denoising a THG image, which has the potential for real-time applications of THG microscopy.

Uniaxial mechanical stretch properties correlated with three-dimensional microstructure of human dermal skin.

The intact and healthy skin forms a barrier to the outside world and protects the body from mechanical impact. The skin is a complex structure with unique mechano-elastic properties. To better direct the design of biomimetic materials and induce skin regeneration in wounds with optimal outcome, more insight is required in how the mechano-elastic properties emerge from the skin's main constituents, collagen and elastin fibers. Here, we employed two-photon excited autofluorescence and second harmonic generation microscopy to characterize collagen and elastin fibers in 3D in 24 human dermis skin samples. Through uniaxial stretching experiments, we derive uni-directional mechanical properties from resultant stress-strain curves, including the initial Young's modulus, elastic Young's modulus, maximal stress, and maximal and mid-strain values. The stress-strain curves show a large variation, with an average Young's modules in the toe and linear regions of 0.1 MPa and 21 MPa. We performed a comprehensive analysis of the correlation between the key mechanical properties with age and with microstructural parameters, e.g., fiber density, thickness, and orientation. Age was found to correlate negatively with Young's modulus and collagen density. Moreover, real-time monitoring during uniaxial stretching allowed us to observe changes in collagen and elastin alignment. Elastin fibers aligned significantly in both the heel and linear regions, and the collagen bundles engaged and oriented mainly in the linear region. This research advances our understanding of skin biomechanics and yields input for future first principles full modeling of skin tissue.

Leukocyte differentiation in bronchoalveolar lavage fluids using higher harmonic generation microscopy and deep learning.

BACKGROUND: In diseases such as interstitial lung diseases (ILDs), patient diagnosis relies on diagnostic analysis of bronchoalveolar lavage fluid (BALF) and biopsies. Immunological BALF analysis includes differentiation of leukocytes by standard cytological techniques that are labor-intensive and time-consuming. Studies have shown promising leukocyte identification performance on blood fractions, using third harmonic generation (THG) and multiphoton excited autofluorescence (MPEF) microscopy. OBJECTIVE: To extend leukocyte differentiation to BALF samples using THG/MPEF microscopy, and to show the potential of a trained deep learning algorithm for automated leukocyte identification and quantification. METHODS: Leukocytes from blood obtained from three healthy individuals and one asthma patient, and BALF samples from six ILD patients were isolated and imaged using label-free microscopy. The cytological characteristics of leukocytes, including neutrophils, eosinophils, lymphocytes, and macrophages, in terms of cellular and nuclear morphology, and THG and MPEF signal intensity, were determined. A deep learning model was trained on 2D images and used to estimate the leukocyte ratios at the image-level using the differential cell counts obtained using standard cytological techniques as reference. RESULTS: Different leukocyte populations were identified in BALF samples using label-free microscopy, showing distinctive cytological characteristics. Based on the THG/MPEF images, the deep learning network has learned to identify individual cells and was able to provide a reasonable estimate of the leukocyte percentage, reaching >90% accuracy on BALF samples in the hold-out testing set. CONCLUSIONS: Label-free THG/MPEF microscopy in combination with deep learning is a promising technique for instant differentiation and quantification of leukocytes. Immediate feedback on leukocyte ratios has potential to speed-up the diagnostic process and to reduce costs, workload and inter-observer variations.

Fast intraoperative histology-based diagnosis of gliomas with third harmonic generation microscopy and deep learning.

Management of gliomas requires an invasive treatment strategy, including extensive surgical resection. The objective of the neurosurgeon is to maximize tumor removal while preserving healthy brain tissue. However, the lack of a clear tumor boundary hampers the neurosurgeon's ability to accurately detect and resect infiltrating tumor tissue. Nonlinear multiphoton microscopy, in particular higher harmonic generation, enables label-free imaging of excised brain tissue, revealing histological hallmarks within seconds. Here, we demonstrate a real-time deep learning-based pipeline for automated glioma image analysis, matching video-rate image acquisition. We used a custom noise detection scheme, and a fully-convolutional classification network, to achieve on average 79% binary accuracy, 0.77 AUC and 0.83 mean average precision compared to the consensus of three pathologists, on a preliminary dataset. We conclude that the combination of real-time imaging and image analysis shows great potential for intraoperative assessment of brain tissue during tumor surgery.

Confinement in crystal lattice alters entire photocycle pathway of the Photoactive Yellow Protein.

Femtosecond time-resolved crystallography (TRC) on proteins enables resolving the spatial structure of short-lived photocycle intermediates. An open question is whether confinement and lower hydration of the proteins in the crystalline state affect the light-induced structural transformations. Here, we measured the full photocycle dynamics of a signal transduction protein often used as model system in TRC, Photoactive Yellow Protein (PYP), in the crystalline state and compared those to the dynamics in solution, utilizing electronic and vibrational transient absorption measurements from 100 fs over 12 decades in time. We find that the photocycle kinetics and structural dynamics of PYP in the crystalline form deviate from those in solution from the very first steps following photon absorption. This illustrates that ultrafast TRC results cannot be uncritically extrapolated to in vivo function, and that comparative spectroscopic experiments on proteins in crystalline and solution states can help identify structural intermediates under native conditions.

Effective enzymatic debridement of burn wounds depends on the denaturation status of collagen.

The treatment of burn wounds by enzymatic debridement using bromelain has shown promising results in our burn center. However, inadequate debridement occurred in a few cases in which the etiology of the burn was attributed to relatively low temperature burns. We hypothesized that bromelain is ineffective in burns in which collagen denaturation, which occurs approximately at 65°C, has not taken place. Our objective was to assess whether there is a relationship between the denaturation of collagen and the ability of bromelain to debride acute scald burn wounds of different temperatures. Ex vivo human skin from four different donors was cut into 1x1 cm samples, and scald burns were produced by immersion in water at temperatures of 40°C, 50°C, 60°C, 70°C, and 100°C for 20 minutes. Denaturation of collagen was assessed with histology, using hematoxylin and eosin (H&E) staining and a fluorescently labeled collagen hybridizing peptide (CHP), and with second harmonic generation (SHG) microscopy. Burned samples and one control sample (room temperature) were weighed before and after application of enzymatic debridement to assess the efficacy of enzymatic debridement. After enzymatic debridement, a weight reduction of 80% was seen in the samples heated to 70°C and 100°C, whereas the other samples showed a reduction of 20%. Unfolding of collagen, loss of basket-weave arrangement, and necrosis was seen in samples heated to 60°C or higher. Evident CHP fluorescence, indicative of collagen denaturation, was seen in samples of 60°C, 70°C and 100°C. SHG intensity, signifying intact collagen, was significantly lower in the 70°C and 100°C group (P <.05) compared to the lower temperatures. In conclusion, denaturation of collagen in skin samples occurred between 60°C and 70°C and strongly correlated with the efficacy of enzymatic debridement. Therefore, enzymatic debridement with the use of bromelain is ineffective in scald burns lower than 60°C.

Label-free stimulated Raman scattering imaging reveals silicone breast implant material in tissue.

Millions of women worldwide have silicone breast implants. It has been reported that implant failure occurs in approximately a tenth of patients within 10 years, and the consequences of dissemination of silicone debris are poorly understood. Currently, silicone detection in histopathological slides is based on morphological features as no specific immunohistochemical technique is available. Here, we show the feasibility and sensitivity of stimulated Raman scattering (SRS) imaging to specifically detect silicone material in stained histopathological slides, without additional sample treatment. Histology slides of four periprosthetic capsules from different implant types were obtained after explantation, as well as an enlarged axillary lymph node from a patient with a ruptured implant. SRS images coregistered with bright-field images revealed the distribution and quantity of silicone material in the tissue. Fast and high-resolution imaging of histology slides with molecular specificity using SRS provides an opportunity to investigate the role of silicone debris in the pathophysiology of implant-linked diseases.

Compact portable multiphoton microscopy reveals histopathological hallmarks of unprocessed lung tumor tissue in real time.

During lung cancer operations a rapid and reliable assessment of tumor tissue can reduce operation time and potentially improve patient outcomes. We show that third harmonic generation (THG), second harmonic generation (SHG) and two-photon excited autofluorescence (2PEF) microscopy reveals relevant, histopathological information within seconds in fresh unprocessed human lung samples. We used a compact, portable microscope and recorded images within 1 to 3 seconds using a power of 5 mW. The generated THG/SHG/2PEF images of tumorous and nontumorous tissues are compared with the corresponding standard histology images, to identify alveolar structures and histopathological hallmarks. Cellular structures (tumor cells, macrophages and lymphocytes) (THG), collagen (SHG) and elastin (2PEF) are differentiated and allowed for rapid identification of carcinoid with solid growth pattern, minimally enlarged monomorphic cell nuclei with salt-and-pepper chromatin pattern, and adenocarcinoma with lipidic and micropapillary growth patterns. THG/SHG/2PEF imaging is thus a promising tool for clinical intraoperative assessment of lung tumor tissue.

Quantitative Third Harmonic Generation Microscopy for Assessment of Glioma in Human Brain Tissue.

Distinguishing tumors from normal brain cells is important but challenging in glioma surgery due to the lack of clear interfaces between the two. The ability of label-free third harmonic generation (THG) microscopy in combination with automated image analysis to quantitatively detect glioma infiltration in fresh, unprocessed tissue in real time is assessed. The THG images reveal increased cellularity in grades II-IV glioma samples from 23 patients, as confirmed by subsequent hematoxylin and eosin histology. An automated image quantification workflow is presented for quantitative assessment of the imaged cellularity as a reflection of the degree of glioma invasion. The cellularity is validated in three ways: 1) Quantitative comparison of THG imaging with fluorescence microscopy of nucleus-stained samples demonstrates that THG reflects the true tissue cellularity. 2) Thresholding of THG cellularity differentiates normal brain from glioma infiltration, with 96.6% sensitivity and 95.5% specificity, in nearly perfect (93%) agreement with pathologists. 3) In one patient, a good correlation between THG cellularity and preoperative magnetic resonance and positron emission tomography imaging is demonstrated. In conclusion, quantitative real-time THG microscopy accurately assesses glioma infiltration in ex vivo human brain samples, and therefore holds strong potential for improving the accuracy of surgical resection.

Second and third harmonic generation microscopy visualizes key structural components in fresh unprocessed healthy human breast tissue.

Real-time assessment of excised tissue may help to improve surgical results in breast tumor surgeries. Here, as a step towards this purpose, the potential of second and third harmonic generation (SHG, THG) microscopy is explored. SHG and THG are nonlinear optical microscopic techniques that do not require labeling of tissue to generate 3D images with intrinsic depth-sectioning at sub-cellular resolution. Until now, this technique had been applied on fixated breast tissue or to visualize the stroma only, whereas most tumors start in the lobules and ducts. Here, SHG/THG images of freshly excised unprocessed healthy human tissue are shown to reveal key breast components-lobules, ducts, fat tissue, connective tissue and blood vessels, in good agreement with hematoxylin and eosin histology. DNA staining of fresh unprocessed mouse breast tissue was performed to aid in the identification of cell nuclei in label-free THG images. Furthermore, 2- and 3-photon excited auto-fluorescence images of mouse and human tissue are collected for comparison. The SHG/THG imaging modalities generate high quality images of freshly excised tissue in less than a minute with an information content comparable to that of the gold standard, histopathology. Therefore, SHG/THG microscopy is a promising tool for real-time assessment of excised tissue during surgery.

Photoactivation Mechanism, Timing of Protein Secondary Structure Dynamics and Carotenoid Translocation in the Orange Carotenoid Protein.

The orange carotenoid protein (OCP) is a two-domain photoactive protein that noncovalently binds an echinenone (ECN) carotenoid and mediates photoprotection in cyanobacteria. In the dark, OCP assumes an orange, inactive state known as OCPO; blue light illumination results in the red active state, known as OCPR. The OCPR state is characterized by large-scale structural changes that involve dissociation and separation of C-terminal and N-terminal domains accompanied by carotenoid translocation into the N-terminal domain. The mechanistic and dynamic-structural relations between photon absorption and formation of the OCPR state have remained largely unknown. Here, we employ a combination of time-resolved UV-visible and (polarized) mid-infrared spectroscopy to assess the electronic and structural dynamics of the carotenoid and the protein secondary structure, from femtoseconds to 0.5 ms. We identify a hereto unidentified carotenoid excited state in OCP, the so-called S* state, which we propose to play a key role in breaking conserved hydrogen-bond interactions between carotenoid and aromatic amino acids in the binding pocket. We arrive at a comprehensive reaction model where the hydrogen-bond rupture with conserved aromatic side chains at the carotenoid ß1-ring in picoseconds occurs at a low yield of <1%, whereby the ß1-ring retains a trans configuration with respect to the conjugated π-electron chain. This event initiates structural changes at the N-terminal domain in 1 µs, which allow the carotenoid to translocate into the N-terminal domain in 10 µs. We identified infrared signatures of helical elements that dock on the C-terminal domain ß-sheet in the dark and unfold in the light to allow domain separation. These helical elements do not move within the experimental range of 0.5 ms, indicating that domain separation occurs on longer time scales, lagging carotenoid translocation by at least 2 decades of time.

Structural and Mechanical Comparison of Human Ear, Alar, and Septal Cartilage.

BACKGROUND: In the human ear and nose, cartilage plays a key role in establishing its form and function. Interestingly, there is a noticeable paucity on biochemical, structural, and mechanical studies focused on facial cartilage. Such studies are needed to provide elementary knowledge that is fundamental to tissue engineering of cartilage. Therefore, in this study, a comparison is made of the biochemical, structural, and mechanical differences between ear, ala nasi, and septum on the extracellular matrix (ECM) level. METHODS: Cartilage samples were harvested from 10 cadaveric donors. Each sample was indented 10 times with a nanoindenter to determine the effective Young's modulus. Structural information of the cartilage was obtained by multiple-photon laser scanning microscopy capable of revealing matrix components at subcellular resolution. Biochemistry was performed to measure glycosaminoglycan (GAG), DNA, elastin, and collagen content. RESULTS: Significant differences were seen in stiffness between ear and septal cartilage (P = 0.011) and between ala nasi and septal cartilage (P = 0.005). Elastin content was significantly higher in ear cartilage. Per cartilage subtype, effective Young's modulus was not significantly correlated with cell density, GAG, or collagen content. However, in septal cartilage, low elastin content was associated with higher stiffness. Laser microscopy showed a distinct difference between ear cartilage and cartilage of nasal origin. CONCLUSION: Proposed methods to investigate cartilage on the ECM level provided good results. Significant differences were seen not only between ear and nasal cartilage but also between the ala nasi and septal cartilage. Albeit its structural similarity to septal cartilage, the ala nasi has a matrix stiffness comparable to ear cartilage.

Quantitative comparison of 3D third harmonic generation and fluorescence microscopy images.

Third harmonic generation (THG) microscopy is a label-free imaging technique that shows great potential for rapid pathology of brain tissue during brain tumor surgery. However, the interpretation of THG brain images should be quantitatively linked to images of more standard imaging techniques, which so far has been done qualitatively only. We establish here such a quantitative link between THG images of mouse brain tissue and all-nuclei-highlighted fluorescence images, acquired simultaneously from the same tissue area. For quantitative comparison of a substantial pair of images, we present here a segmentation workflow that is applicable for both THG and fluorescence images, with a precision of 91.3 % and 95.8 % achieved respectively. We find that the correspondence between the main features of the two imaging modalities amounts to 88.9 %, providing quantitative evidence of the interpretation of dark holes as brain cells. Moreover, 80 % bright objects in THG images overlap with nuclei highlighted in the fluorescence images, and they are 2 times smaller than the dark holes, showing that cells of different morphologies can be recognized in THG images. We expect that the described quantitative comparison is applicable to other types of brain tissue and with more specific staining experiments for cell type identification.

Extracting morphologies from third harmonic generation images of structurally normal human brain tissue.

MOTIVATION: The morphologies contained in 3D third harmonic generation (THG) images of human brain tissue can report on the pathological state of the tissue. However, the complexity of THG brain images makes the usage of modern image processing tools, especially those of image filtering, segmentation and validation, to extract this information challenging. RESULTS: We developed a salient edge-enhancing model of anisotropic diffusion for image filtering, based on higher order statistics. We split the intrinsic 3-phase segmentation problem into two 2-phase segmentation problems, each of which we solved with a dedicated model, active contour weighted by prior extreme. We applied the novel proposed algorithms to THG images of structurally normal ex-vivo human brain tissue, revealing key tissue components-brain cells, microvessels and neuropil, enabling statistical characterization of these components. Comprehensive comparison to manually delineated ground truth validated the proposed algorithms. Quantitative comparison to second harmonic generation/auto-fluorescence images, acquired simultaneously from the same tissue area, confirmed the correctness of the main THG features detected. AVAILABILITY AND IMPLEMENTATION: The software and test datasets are available from the authors. CONTACT: z.zhang@vu.nl. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Unfolding of the C-Terminal Jα Helix in the LOV2 Photoreceptor Domain Observed by Time-Resolved Vibrational Spectroscopy.

Light-triggered reactions of biological photoreceptors have gained immense attention for their role as molecular switches in their native organisms and for optogenetic application. The light, oxygen, and voltage 2 (LOV2) sensing domain of plant phototropin binds a C-terminal Jα helix that is docked on a ß-sheet and unfolds upon light absorption by the flavin mononucleotide (FMN) chromophore. In this work, the signal transduction pathway of LOV2 from Avena sativa was investigated using time-resolved infrared spectroscopy from picoseconds to microseconds. In D2O buffer, FMN singlet-to-triplet conversion occurs in 2 ns and formation of the covalent cysteinyl-FMN adduct in 10 µs. We observe a two-step unfolding of the Jα helix: The first phase occurs concomitantly with Cys-FMN covalent adduct formation in 10 µs, along with hydrogen-bond rupture of the FMN C4═O with Gln-513, motion of the ß-sheet, and an additional helical element. The second phase occurs in approximately 240 µs. The final spectrum at 500 µs is essentially identical to the steady-state light-minus-dark Fourier transform infrared spectrum, indicating that Jα helix unfolding is complete on that time scale.

Cartilage Tissue Engineering: Preventing Tissue Scaffold Contraction Using a 3D-Printed Polymeric Cage.

Scaffold contraction is a common but underestimated problem in the field of tissue engineering. It becomes particularly problematic when creating anatomically complex shapes such as the ear. The aim of this study was to develop a contraction-free biocompatible scaffold construct for ear cartilage tissue engineering. To address this aim, we used three constructs: (i) a fibrin/hyaluronic acid (FB/HA) hydrogel, (ii) a FB/HA hydrogel combined with a collagen I/III scaffold, and (iii) a cage construct containing (ii) surrounded by a 3D-printed poly-ɛ-caprolactone mold. A wide range of different cell types were tested within these constructs, including chondrocytes, perichondrocytes, adipose-derived mesenchymal stem cells, and their combinations. After in vitro culturing for 1, 14, and 28 days, all constructs were analyzed. Macroscopic observation showed severe contraction of the cell-seeded hydrogel (i). This could be prevented, in part, by combining the hydrogel with the collagen scaffold (ii) and prevented in total using the 3D-printed cage construct (iii). (Immuno)histological analysis, multiphoton laser scanning microscopy, and biomechanical analysis showed extracellular matrix deposition and increased Young's modulus and thereby the feasibility of ear cartilage engineering. These results demonstrated that the 3D-printed cage construct is an adequate model for contraction-free ear cartilage engineering using a range of cell combinations.

Ultrafast infrared spectroscopy in photosynthesis.

In recent years visible pump/mid-infrared (IR) probe spectroscopy has established itself as a key technology to unravel structure-function relationships underlying the photo-dynamics of complex molecular systems. In this contribution we review the most important applications of mid-infrared absorption difference spectroscopy with sub-picosecond time-resolution to photosynthetic complexes. Considering several examples, such as energy transfer in photosynthetic antennas and electron transfer in reaction centers and even more intact structures, we show that the acquisition of ultrafast time resolved mid-IR spectra has led to new insights into the photo-dynamics of the considered systems and allows establishing a direct link between dynamics and structure, further strengthened by the possibility of investigating the protein response signal to the energy or electron transfer processes. This article is part of a Special Issue entitled: Vibrational spectroscopies and bioenergetic systems.