Self-supervised anomaly detection in computer vision and beyond: A survey and outlook.

Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or events that deviate from normal behavior. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning.

Feature-level ensemble approach for COVID-19 detection using chest X-ray images.

Severe acute respiratory syndrome coronavirus 2 (SARS CoV-2), also known as the coronavirus disease 2019 (COVID-19), has threatened many human beings around the world and capsized economies at unprecedented magnitudes. Therefore, the detection of this disease using chest X-ray modalities has played a pivotal role in producing fast and accurate medical diagnoses, especially in countries that are unable to afford laboratory testing kits. However, identifying and distinguishing COVID-19 from virtually similar thoracic abnormalities utilizing medical images is challenging because it is time-consuming, demanding, and susceptible to human-based errors. Therefore, artificial-intelligence-driven automated diagnoses, which excludes direct human intervention, may potentially be used to achieve consistently accurate performances. In this study, we aimed to (i) obtain a customized dataset composed of a relatively small number of images collected from publicly available datasets; (ii) present the efficient integration of the shallow handcrafted features obtained from local descriptors, radiomics features specialized for medical images, and deep features aggregated from pre-trained deep learning architectures; and (iii) distinguish COVID-19 patients from healthy controls and pneumonia patients using a collection of conventional machine learning classifiers. By conducting extensive experiments, we demonstrated that the feature-based ensemble approach provided the best classification metrics, and this approach explicitly outperformed schemes that used only either local, radiomic, or deep features. In addition, our proposed method achieved state-of-the-art multi-class classification results compared to the baseline reference for the currently available COVID-19 datasets.

Deep Learning-Based Multilevel Classification of Alzheimer's Disease Using Non-invasive Functional Near-Infrared Spectroscopy.

The timely diagnosis of Alzheimer's disease (AD) and its prodromal stages is critically important for the patients, who manifest different neurodegenerative severity and progression risks, to take intervention and early symptomatic treatments before the brain damage is shaped. As one of the promising techniques, functional near-infrared spectroscopy (fNIRS) has been widely employed to support early-stage AD diagnosis. This study aims to validate the capability of fNIRS coupled with Deep Learning (DL) models for AD multi-class classification. First, a comprehensive experimental design, including the resting, cognitive, memory, and verbal tasks was conducted. Second, to precisely evaluate the AD progression, we thoroughly examined the change of hemodynamic responses measured in the prefrontal cortex among four subject groups and among genders. Then, we adopted a set of DL architectures on an extremely imbalanced fNIRS dataset. The results indicated that the statistical difference between subject groups did exist during memory and verbal tasks. This presented the correlation of the level of hemoglobin activation and the degree of AD severity. There was also a gender effect on the hemoglobin changes due to the functional stimulation in our study. Moreover, we demonstrated the potential of distinguished DL models, which boosted the multi-class classification performance. The highest accuracy was achieved by Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) using the original dataset of three hemoglobin types (0.909 ± 0.012 on average). Compared to conventional machine learning algorithms, DL models produced a better classification performance. These findings demonstrated the capability of DL frameworks on the imbalanced class distribution analysis and validated the great potential of fNIRS-based approaches to be further contributed to the development of AD diagnosis systems.

Spontaneous evolution of equilibrium morphology in phospholipid-cholesterol monolayers.

Competition between intradomain electrostatic repulsions and interdomain line tension leads to domain shape transitions in phase-separating lipid monolayers. The question remains if these morphologies are energy minima or are kinetically trapped metastable states. We show the reversible evolution of uniform width stripe domains from polydisperse semicircular domains in monolayers of dipalmitoylphosphatidylcholine (DPPC), hexadecanol (HD) or palmitic acid (PA), and dihydrocholesterol (DChol). The initial semicircular domains grow at a fixed 2:1 DPPC:HD (or PA) stoichiometry, depleting the liquid phase of HD, leaving behind a liquid enriched in DPPC and DChol. At higher surface pressures, the remaining DPPC precipitates onto existing domains, decreasing the ratio of line tension to the square of the dipole density difference, λ/µ2. Theory predicts that, as λ/µ2 decreases, circular domains reversibly transform to uniform width stripes as the minimum energy structure. Measuring the stripe width provides the first estimates of λ/µ2 at liquid condensed-liquid expanded phase coexistence.

All-fiber single-cavity dual-comb for coherent anti-Stokes Raman scattering spectroscopy based on spectral focusing.

We report an all-fiber free-running bidirectional dual-comb laser system for coherent anti-Stokes Raman scattering spectroscopy based on spectral focusing. The mode-locked oscillator is a bidirectional ring-cavity erbium fiber laser running at a repetition rate of ∼114MHz. One output of the bidirectional laser is wavelength-shifted from 1560 to 1060 nm via supercontinuum generation for use as the pump source. We have been able to record the Raman spectra of various samples such as polystyrene, olive oil, polymethyl methacrylate (PMMA), and polyethylene in the C-H stretching window. We believe that this all-fiber laser design has promising potential for coherent Raman spectroscopy and also label-free imaging for a variety of practical applications.

Feasibility of multimodal multiphoton microscopy to facilitate surgical margin assessment in pancreatic cancer.

Pancreatic cancer is a common cancer with poor odds of survival for the patient, with surgical resection offering the only hope of cure. Current surgical practice is time-consuming and, due to time constraints, does not sample the whole cut surface sufficiently to check for remaining cancer. Although microscopy with hematoxylin and eosin (H&E) stain is the gold standard for microscopic evaluation, multiphoton microscopy (MPM) has emerged as an alternative tool for imaging tissue architecture and cellular morphology without labels. We explored the use of multimodal MPM for the label-free identification of normal and cancerous tissue of the pancreas in a mouse model by comparing the images to H&E microscopy. Our early studies indicate that MPM using second-harmonic generation, third-harmonic generation, and multiphoton excitation of endogenous fluorescent proteins can each contribute to the label-free analysis of the pancreatic surgical margin.

Characterization of multiphoton microscopes by the nonlinear knife-edge technique.

Imaging submicron fluorescent microspheres are the standard method for measuring resolution in multiphoton microscopy. However, when using high-energy pulsed lasers, photobleaching and heating of the solution medium may deteriorate the images, resulting in an inaccurate resolution measurement. Moreover, due to the weak higher-order response of fluorescent microspheres, measuring three-photon resolution using three-photon fluorescence (3PEF) and third-harmonic generation (THG) signals is more difficult. In this report, we demonstrate a methodology for complete characterization of multiphoton microscopes based on second- and third-harmonic generation signals from the sharp edge of GaAs wafers. This simple methodology, which we call the nonlinear knife-edge technique, provides fast and consistent lateral and axial resolution measurement with negligible photobleaching effect on semiconductor wafers. In addition, this technique provides information on the field curvature of the imaging system, and perhaps other distortions of the imaging system, adding greater capability compared to existing techniques.

All-fiber high-power 1700 nm femtosecond laser based on optical parametric chirped-pulse amplification.

We present the design and construction of an all-fiber high-power optical parametric chirped-pulse amplifier working at 1700 nm, an important wavelength for bio-photonics and medical treatments. The laser delivers 1.42 W of output average power at 1700 nm, which corresponds to ∼40 nJ pulse energy. The pulse can be de-chirped with a conventional grating pair compressor to ∼450 fs. Furthermore, the laser has a stable performance with relative intensity noise typically below the -130 dBc/Hz level for the idler pulses at 1700 nm from 10kHz to 16.95 MHz, half of the laser repetition rate f/2.

Watt-level all-fiber optical parametric chirped-pulse amplifier working at 1300 nm.

We report watt-level average output power near 1300 nm from an all-fiber ultrafast optical parametric chirped-pulse amplifier. A compressed output pulse duration of ∼300 fs is achieved. Multiphoton imaging of a variety of samples carried out with this light source shows a good signal-to-noise ratio. With the demonstrated imaging capability, we believe that this high-power ultrafast laser source addresses a key need in deep tissue multiphoton microscopy.

Self-triggered Asynchronous Optical Sampling Terahertz Spectroscopy using a Bidirectional Mode-locked Fiber Laser.

We report a self-triggered asynchronous optical sampling terahertz spectroscopy system based on a single bidirectional mode-locked fiber laser and plasmonics-enhanced photoconductive nanoantennas. The fiber laser generates two optical mutually coherent pulse trains with a stable repetition rate difference, enabling time-domain terahertz spectroscopy without using any mechanical delay line, stabilization electronics, or external trigger. The resolved terahertz spectra over a 0.1-2 THz frequency range and a 30-second measurement time show more than a 70-dB dynamic range, revealing water absorption lines matching the HITRAN database, through a light-weight and compact spectroscopy setup.

Multiphoton Microscopy of π-Conjugated Copolymers and Copolymer/Fullerene Blends for Organic Photovoltaic Applications.

Organic photovoltaic (OPV) cells based on π-conjugated copolymer/fullerene blends are devices with the highest power conversion efficiencies within the class of organic semiconductors. Although a number of image microscopies have been applied to films of π-conjugated copolymers and their fullerene blends, seldom have they been able to detect microscopic defects in the blend films. We have applied multiphoton microscopy (MPM) using a 65 fs laser at 1.56 µm for spectroscopy and mapping of films of various π-conjugated copolymers and their fullerene blends. All pristine copolymer films have shown third harmonic generation (THG) and two-photon or three-photon photoluminescence that could be used for mapping the films with micrometer spatial resolution. Since the fullerenes have much weaker THG efficiency than those of the copolymers, we could readily map the copolymer/fullerene blend films that showed interpenetrating micron-sized grains of the two constituents. In addition, we also found second harmonic generation from various micron-sized defects in the films that are formed during film deposition or light illumination at ambient conditions, which do not possess inversion symmetry. The MPM method is therefore beneficial for organic films and devices for investigating the properties and growth of copolymer/fullerene blends for OPV applications.

Compact fiber-based multi-photon endoscope working at 1700 nm.

We present the design, implementation and performance analysis of a compact multi-photon endoscope based on a piezo electric scanning tube. A miniature objective lens with a long working distance and a high numerical aperture (≈ 0.5) is designed to provide a diffraction limited spot size. Furthermore, a 1700 nm wavelength femtosecond fiber laser is used as an excitation source to overcome the scattering of biological tissues and reduce water absorption. Therefore, the novel optical system along with the unique wavelength allows us to increase the imaging depth. We demonstrate that the endoscope is capable of performing third and second harmonic generation (THG/SHG) and three-photon excitation fluorescence (3PEF) imaging over a large field of view (> 400 µm) with high lateral resolution (2.2 µm). The compact and lightweight probe design makes it suitable for minimally-invasive in-vivo imaging as a potential alternative to surgical biopsies.

Pancreatic cancer cell detection by targeted lipid microbubbles and multiphoton imaging.

Surgical resection of pancreatic cancer represents the only chance of cure and long-term survival in this common disease. Unfortunately, determination of a cancer-free margin at surgery is based on one or two tiny frozen section biopsies, which is far from ideal. Not surprisingly, cancer is usually left behind and is responsible for metastatic disease. We demonstrate a method of receptor-targeted imaging using peptide ligands, lipid microbubbles, and multiphoton microscopy that could lead to a fast and accurate way of examining the entire cut surface during surgery. Using a plectin-targeted microbubble, we performed a blinded in-vitro study to demonstrate avid binding of targeted microbubbles to pancreatic cancer cells but not noncancerous cell lines. Further work should lead to a much-needed point-of-care diagnostic test for determining clean margins in oncologic surgery.

All-fiber dissipative soliton Raman laser based on phosphosilicate fiber.

We propose and demonstrate an all-fiber, synchronously pumped Raman laser based on phosphosilicate fiber (P-doped fiber) for deep tissue multiphoton imaging. The laser operates in a dissipative soliton regime and produces 2.2 ps chirped pulses (compressible to 317 fs) with energy up to 9.2 nJ, 0.3 W average power and at 1240 nm center wavelength. We have also found a new cross-polarization Raman lasing operation that offers access to an important wavelength near 930 nm for calcium imaging.

All-reflective multiphoton microscope.

We present the design, construction, and characterization of a multiphoton microscope that uses reflective elements for beam shaping and steering. This compact all reflective design removes the adverse effects of dispersion on laser pulse broadening as well as chromatic aberration in the focusing of broadband and multicolored laser sources. The design of this system is discussed in detail, including aberrations analysis via ray-tracing simulation and opto-mechanical design. The resolution of this mirror based all-reflective microscope is characterized using fluorescent microbeads. The performance of the system at multiple wavelengths is investigated along with some potential multiphoton imaging and writing applications.

Ultra-strong nonlinear optical processes and trigonal warping in MoS2 layers.

Nonlinear optical processes, such as harmonic generation, are of great interest for various applications, e.g., microscopy, therapy, and frequency conversion. However, high-order harmonic conversion is typically much less efficient than low-order, due to the weak intrinsic response of the higher-order nonlinear processes. Here we report ultra-strong optical nonlinearities in monolayer MoS2 (1L-MoS2): the third harmonic is 30 times stronger than the second, and the fourth is comparable to the second. The third harmonic generation efficiency for 1L-MoS2 is approximately three times higher than that for graphene, which was reported to have a large χ (3). We explain this by calculating the nonlinear response functions of 1L-MoS2 with a continuum-model Hamiltonian and quantum mechanical diagrammatic perturbation theory, highlighting the role of trigonal warping. A similar effect is expected in all other transition-metal dichalcogenides. Our results pave the way for efficient harmonic generation based on layered materials for applications such as microscopy and imaging.Harmonic generation is a nonlinear optical process occurring in a variety of materials; the higher orders generation is generally less efficient than lower orders. Here, the authors report that the third-harmonic is thirty times stronger than the second-harmonic in monolayer MoS2.

Rapid visualization of grain boundaries in monolayer MoS2 by multiphoton microscopy.

Grain boundaries have a major effect on the physical properties of two-dimensional layered materials. Therefore, it is important to develop simple, fast and sensitive characterization methods to visualize grain boundaries. Conventional Raman and photoluminescence methods have been used for detecting grain boundaries; however, these techniques are better suited for detection of grain boundaries with a large crystal axis rotation between neighbouring grains. Here we show rapid visualization of grain boundaries in chemical vapour deposited monolayer MoS2 samples with multiphoton microscopy. In contrast to Raman and photoluminescence imaging, third-harmonic generation microscopy provides excellent sensitivity and high speed for grain boundary visualization regardless of the degree of crystal axis rotation. We find that the contrast associated with grain boundaries in the third-harmonic imaging is considerably enhanced by the solvents commonly used in the transfer process of two-dimensional materials. Our results demonstrate that multiphoton imaging can be used for fast and sensitive characterization of two-dimensional materials.

Rapid and Large-Area Characterization of Exfoliated Black Phosphorus Using Third-Harmonic Generation Microscopy.

Black phosphorus (BP) is a layered semiconductor that recently has been the subject of intense research due to its novel electrical and optical properties, which compare favorably to those of graphene and the transition metal dichalcogenides. In particular, BP has a direct bandgap that is thickness-dependent and highly anisotropic, making BP an interesting material for nanoscale optical and optoelectronic applications. Here, we present a study of the anisotropic third-harmonic generation (THG) in exfoliated BP using a fast scanning multiphoton characterization method. We find that the anisotropic THG arises directly from the crystal structure of BP. We calculate the effective third-order susceptibility of BP to be ∼1.64 × 10-19 m2 V-2. Further, we demonstrate that multiphoton microscopy can be used for rapid, large-area characterization indexing of the crystallographic orientations of many exfoliated BP flakes from one set of multiphoton images. This method is therefore beneficial for samples of areas ∼1 cm2 in future investigations of the properties and growth of BP.

Design and characterization of a combined OCT and wide field imaging falloposcope for ovarian cancer detection.

Early detection of ovarian cancer is only achieved in around 20% of women due to lack of effective screening. We propose a method for surveillance of high risk women based on a microendoscope introduced transvaginally to image the fallopian tubes and ovaries. This requires extreme miniaturization of the optics and catheter sheath. We describe the design of a falloposcope that combines optical coherence tomography (OCT) and wide field imaging into a sub-1 mm diameter package. We characterize the systems and show that they provide contrast on ex-vivo samples of ovary and fallopian tube. In addition, we show the mechanical performance of the endoscope in an anatomically correct model of the female reproductive tract.

Imaging of targeted lipid microbubbles to detect cancer cells using third harmonic generation microscopy.

The use of receptor-targeted lipid microbubbles imaged by ultrasound is an innovative method of detecting and localizing disease. However, since ultrasound requires a medium between the transducer and the object being imaged, it is impractical to apply to an exposed surface in a surgical setting where sterile fields need be maintained and ultrasound gel may cause the bubbles to collapse. Multiphoton microscopy (MPM) is an emerging tool for accurate, label-free imaging of tissues and cells with high resolution and contrast. We have recently determined a novel application of MPM to be used for detecting targeted microbubble adherence to the upregulated plectin-receptor on pancreatic tumor cells. Specifically, the third-harmonic generation response can be used to detect bound microbubbles to various cell types presenting MPM as an alternative and useful imaging method. This is an interesting technique that can potentially be translated as a diagnostic tool for the early detection of cancer and inflammatory disorders.