Semisupervised Training of Deep Generative Models for High-Dimensional Anomaly Detection.

Abnormal behaviors in industrial systems may be early warnings on critical events that may cause severe damages to facilities and security. Thus, it is important to detect abnormal behaviors accurately and timely. However, the anomaly detection problem is hard to solve in practice, mainly due to the rareness and the expensive cost to get the labels of the anomalies. Deep generative models parameterized by neural networks have achieved state-of-the-art performance in practice for many unsupervised and semisupervised learning tasks. We present a new deep generative model, Latent Enhanced regression/classification Deep Generative Model (LEDGM), for the anomaly detection problem with multidimensional data. Instead of using two-stage decoupled models, we adopt an end-to-end learning paradigm. Instead of conditioning the latent on the class label, LEDGM conditions the label prediction on the learned latent so that the optimization goal is more in favor of better anomaly detection than better reconstruction that the previously proposed deep generative models have been trained for. Experimental results on several synthetic and real-world small- and large-scale datasets demonstrate that LEDGM can achieve improved anomaly detection performance on multidimensional data with very sparse labels. The results also suggest that both labeled anomalies and labeled normal are valuable for semisupervised learning. Generally, our results show that better performance can be achieved with more labeled data. The ablation experiments show that both the original input and the learned latent provide meaningful information for LEDGM to achieve high performance.

Predicting potentially hazardous chemical reactions using an explainable neural network.

Predicting potentially dangerous chemical reactions is a critical task for laboratory safety. However, a traditional experimental investigation of reaction conditions for possible hazardous or explosive byproducts entails substantial time and cost, for which machine learning prediction could accelerate the process and help detailed experimental investigations. Several machine learning models have been developed which allow the prediction of major chemical reaction products with reasonable accuracy. However, these methods may not present sufficiently high accuracy for the prediction of hazardous products which particularly requires a low false negative result for laboratory safety in order not to miss any dangerous reactions. In this work, we propose an explainable artificial intelligence model that can predict the formation of hazardous reaction products in a binary classification fashion. The reactant molecules are transformed into substructure-encoded fingerprints and then fed into a convolutional neural network to make the binary decision of the chemical reaction. The proposed model shows a false negative rate of 0.09, which can be compared with 0.47-0.66 using the existing main product prediction models. To provide explanations for what substructures of the given reactant molecules are important to make a decision for target hazardous product formation, we apply an input attribution method, layer-wise relevance propagation, which computes the contributions of individual inputs per input data. The computed attributions indeed match some of the existing chemical intuitions and mechanisms, and also offer a way to analyze possible data-imbalance issues of the current predictions based on relatively small positive datasets. We expect that the proposed hazardous product prediction model will be complementary to existing main product prediction models and experimental investigations.

Spatiotemporal Characterization of GPCR Activity and Function during Endosomal Trafficking Pathway.

G protein-coupled receptor (GPCR) is activated by extracellular signals. After their function at plasma membrane, GPCRs are internalized to be desensitized, while emerging evidence suggests that some GPCRs maintain their activity even after internalization. The endosomal trafficking pathway of a prototypic GPCR, ß adrenergic receptor 2 (B2AR), is in the range of several hours, however, spatiotemporal B2AR activity during this long-term endosomal trafficking pathway has not been characterized yet. Here, we analyze an agonist-induced real-time B2AR activity and its downstream function at the level of individual vesicles, utilizing a fluorescence resonance energy transfer (FRET)-based B2AR biosensor and cAMP reporters tethered at different trafficking stages of endosomes. Our results report that the internalized B2ARs sustain the activity and maintain the production of cAMP for several hours during the endosomal trafficking pathway. Temporal kinetics of B2AR activity is mathematically well explained by our active-vesicle population model modified from the Ricker model. Therefore, our GPCR monitoring system and a new kinetics model can be applied to understand the spatiotemporal GPCR activity and its downstream function during the endosomal trafficking pathway.

Scheduling PID Attitude and Position Control Frequencies for Time-Optimal Quadrotor Waypoint Tracking under Unknown External Disturbances.

Recently, the use of quadrotors has increased in numerous applications, such as agriculture, rescue, transportation, inspection, and localization. Time-optimal quadrotor waypoint tracking is defined as controlling quadrotors to follow the given waypoints as quickly as possible. Although PID control is widely used for quadrotor control, it is not adaptable to environmental changes, such as various trajectories and dynamic external disturbances. In this work, we discover that adjusting PID control frequencies is necessary for adapting to environmental changes by showing that the optimal control frequencies can be different for different environments. Therefore, we suggest a method to schedule the PID position and attitude control frequencies for time-optimal quadrotor waypoint tracking. The method includes (1) a Control Frequency Agent (CFA) that finds the best control frequencies in various environments, (2) a Quadrotor Future Predictor (QFP) that predicts the next state of a quadrotor, and (3) combining the CFA and QFP for time-optimal quadrotor waypoint tracking under unknown external disturbances. The experimental results prove the effectiveness of the proposed method by showing that it reduces the travel time of a quadrotor for waypoint tracking.

Visualization of Autophagy Progression by a Red-Green-Blue Autophagy Sensor.

Autophagy is a major degradation process of cytosolic components and misfolded proteins that is crucial for cellular homeostasis and for the pathogenesis of diverse diseases. Autophagy is initiated by the formation of phagophores, which mature to autophagosomes. The autophagosomes then fuse to lysosomes to form autolysosomes. Different stages of autophagy can be deregulated to cause autophagy-related diseases, and thus, an accurate detection of each stage of autophagy progression is critical for efficient therapeutic strategies for these diseases. To identify the different stages of autophagy progression, here, we developed a new autophagy flux sensor, named red-green-blue-LC3 (RGB-LC3). RGB-LC3 is composed of LC3 and red-green-blue (RGB) fluorescent proteins, which were carefully chosen by considering their separate spectral profiles, stability, brightness, and most importantly different pH sensitivities. Utilizing this RGB-LC3 and the predicted pH, we could clearly identify phagophores, autophagosomes, fusion stage, early autolysosomes, and mature autolysosomes in live cells. Furthermore, the RGB-LC3 sensor was successfully applied to distinguish different effects of Aß monomers and oligomers on autophagy flux. Therefore, we developed a new autophagy flux sensor, RGB-LC3, which may be a valuable tool to further investigate the molecular mechanisms of autophagy and to develop efficient therapeutic strategies for autophagy-related diseases.

XAI-Explainable artificial intelligence.

Explainability is essential for users to effectively understand, trust, and manage powerful artificial intelligence applications.

Localization of Rod Bipolar Cells in the Mammalian Retina Using an Antibody Against the α1c L-type Ca(2+) Channel.

Bipolar cells transmit stimuli via graded changes in membrane potential and neurotransmitter release is modulated by Ca(2+) influx through L-type Ca(2+) channels. The purpose of this study was to determine whether the α1c subunit of L-type voltage-gated Ca(2+) channel (α1c L-type Ca(2+) channel) colocalizes with protein kinase C alpha (PKC-α), which labels rod bipolar cells. Retinal whole mounts and vertical sections from mouse, hamster, rabbit, and dog were immunolabeled with antibodies against PKC-α and α1c L-type Ca(2+) channel, using fluorescein isothiocyanate (FITC) and Cy5 as visualizing agents. PKC-α-immunoreactive cells were morphologically identical to rod bipolar cells as previously reported. Their cell bodies were located within the inner nuclear layer, dendritic processes branched into the outer plexiform layer, and axons extended into the inner plexiform layer. Immunostaining showed that α1c L-type Ca(2+) channel colocalized with PKC-α in rod bipolar cells. The identical expression of PKC-α and α1c L-type Ca(2+) channel indicates that the α1c L-type Ca(2+) channel has a specific role in rod bipolar cells, and the antibody against the α1c L-type Ca(2+) channel may be a useful marker for studying the distribution of rod bipolar cells in mouse, hamster, rabbit, and dog retinas.

Use of machine learning methods to reduce predictive error of groundwater models.

Quantitative analyses of groundwater flow and transport typically rely on a physically-based model, which is inherently subject to error. Errors in model structure, parameter and data lead to both random and systematic error even in the output of a calibrated model. We develop complementary data-driven models (DDMs) to reduce the predictive error of physically-based groundwater models. Two machine learning techniques, the instance-based weighting and support vector regression, are used to build the DDMs. This approach is illustrated using two real-world case studies of the Republican River Compact Administration model and the Spokane Valley-Rathdrum Prairie model. The two groundwater models have different hydrogeologic settings, parameterization, and calibration methods. In the first case study, cluster analysis is introduced for data preprocessing to make the DDMs more robust and computationally efficient. The DDMs reduce the root-mean-square error (RMSE) of the temporal, spatial, and spatiotemporal prediction of piezometric head of the groundwater model by 82%, 60%, and 48%, respectively. In the second case study, the DDMs reduce the RMSE of the temporal prediction of piezometric head of the groundwater model by 77%. It is further demonstrated that the effectiveness of the DDMs depends on the existence and extent of the structure in the error of the physically-based model.

Glutamate receptors GluR1 and GluR4 in the hamster superior colliculus: distribution and co-localization with calcium-binding proteins and GABA.

We investigated the distributions of AMPA glutamate receptor subtypes GluR1 and GluR4 in the hamster superior colliculus (SC) with antibody immunocytochemistry and the effect of enucleation on these distributions. We compared these labelings to those of GluR2/3 in our previous report (Park et al., 2004, Neurosci Res., 49:139-155) and calcium-binding proteins calbindin D28K, calretinin, parvalbumin, and GABA. Anti-GluR1-immunoreactive (IR) cells were scattered throughout the SC. By contrast, anti-GluR4-IR cells formed distinct clusters within the lower lateral stratum griseum intermediale (SGI) and lateral stratum album intermediale (SAI). The GluR1- and GluR4-IR neurons varied in size and morphology. The average diameter of the GluR1-IR cells was 13.00 microm, while the GluR4-IR cells was 20.00 microm. The large majority of IR neurons were round or oval cells, but they also included stellate, vertical fusiform and horizontal cells. Monocular enucleation appeared to have no effect on the GluR1 and GluR4 immunoreactivity. Some GluR1-IR cells expressed calbindin D28K (9.50%), calretinin (6.59%), parvalbumin (2.53%), and GABA (20.54%). By contrast, no GluR4-IR cells expressed calcium-binding proteins or GABA. Although the function of the AMPA receptor subunits in SC is not yet clear, the distinct segregation of the GluR subunits, its differential colocalization with calcium-binding proteins and GABA, and differential responses to enucleation suggest the functional diversity of the receptor subunits in visuo-motor integration in the SC.

AII amacrine cells in the inner nuclear layer of bat retina: identification by parvalbumin immunoreactivity.

The purpose of this investigation is to characterize parvalbumin-immunoreactive (PV-IR) amacrine cells in bat retina through immunocytochemistry, quantitative analysis, and confocal microscopy. PV immunoreactivity was present in ganglion cell and inner nuclear layers. The regular distribution of PV-IR neurons, the inner marginal locations of their cell bodies in the inner nuclear layers, and the distinctive bilaminar morphologies of their dendritic arbors in the inner plexiform layers suggested that these PV-IR cells were AII amacrine cells. PV-IR neurons were double labeled forcalretinin, a marker for AII cells. These results indicate that PV antibodies can be used to label AII cells selectively in bats. The existence of AII cells suggests that bats have retinas involved in both rod-driven and cone-driven signals.

Photoreactive immobilization of 11-(2,4-dinitro-5-fluorobenzene)undecenamide on a hydrogenated silicon (100) surface for protein immobilizations.

Several nucleophiles such as proteins or poly(ethyleneimine) could be easily conjugated with a 11-(2,4-dinitro-5-fluorobenzene)undecenamide (DFUA) monolayer photochemically prepared on a silicon (100) surface.

Organization of calbindin D28K-immunoreactive neurons in the dog superior colliculus.

We localized calbindin D28K-immunoreactive (IR) neurons in the superior colliculus (SC) of the dog and studied the distribution and effect of enucleation on the distribution of this protein. We also compared this labeling to that of GABA. Calbindin D28K was localized with antibody immunocytochemistry. Calbindin D28K-IR neurons formed three laminar tiers in the SC, one within the lower superficial gray layer (SGL), the second within the upper intermediate gray layers (IGL), and the third within the deep gray layer (DGL). The third tier was not very distinctive when compared with the other two tiers. Calbindin D28K-IR neurons in the SC varied dramatically in morphology and size, and included round/oval, vertical fusiform, stellate, pyriform, and horizontal neurons. Neurons with varicose dendrite were also labeled in the IGL. Enucleation appeared to have no effect on the distribution of calbindin D28K-IR neurons in the contralateral SC. Two-color immunofluorescence revealed that a small percentage (11.20%) of calbindin D28K-IR neurons co-localized with GABA. The current results demonstrate that the patterned distribution of calbindin D28K-IR neurons in the intermediate and deep SC is comparable with other animals, but that the distribution of this protein in the superficial SC is strikingly different from that in previously studied animals. The results also suggest that retinal projection may not control the activity of the expression of calbindin D28K in the dog SC. These results will not only provide valuable knowledge of the basic neurochemical architecture of the dog visual system, but also provide clues for the understanding of the similarities and differences among species.

Ionotropic glutamate receptor GluR1 in the visual cortex of hamster: distribution and co-localization with calcium-binding proteins and GABA.

The subunit composition of the AMPA receptor is critical to its function. AMPA receptors that display very low calcium permeability include the GluR2 subunit, while AMPA receptors that contain other subunits, such as GluR1, display high calcium permeability. We have studied the distribution and morphology of neurons containing GluR1 in the hamster visual cortex with antibody immunocytochemistry. We compared this labeling to that for calbindin D28K, parvalbumin, and GABA. Anti-GluR1-immunoreactive (IR) neurons were located in all layers. The highest density of GluR1-IR neurons was found in layers II/III. The labeled neurons were non-pyramidal neurons, but were varied in morphology. The majority of the labeled neurons were round or oval cells. However, stellate, vertical fusiform, pyriform, and horizontal neurons were also labeled with the anti-GluR1 antibody. Two-color immunofluorescence revealed that many of the GluR1-IR neurons in the hamster visual cortex were double-labeled with either calbindin D28K (31.50%), or parvalbumin (22.91%), or GABA (63.89%). These results indicate that neurons in the hamster visual cortex express GluR1 differently according to different layers and selective cell types, and that many of the GluR1-IR neurons are limited to neurons that express calbindin D28K, parvalbumin, or GABA. The present study elucidates the neurochemical structure of GluR1, a useful clue in understanding the differential vulnerability of GluR1-containing neurons with regard to calcium-dependent excitotoxic mechanisms.

Calcium-binding protein calretinin immunoreactivity in the dog superior colliculus.

We studied calretinin-immunoreactive (IR) fibers and cells in the canine superior colliculus (SC) and studied the distribution and effect of enucleation on the distribution of this protein. Localization of calretinin was immunocytochemically observed. A dense plexus of anti--calretinin-IR fibers was found within the upper part of the superficial gray layer (SGL). Almost all of the labeled fibers were small in diameter with few varicosities. The intermediate and deep layers contained many calretinin-IR neurons. Labeled neurons within the intermediate gray layer (IGL) formed clusters in many sections. By contrast, labeled neurons in the deep gray layer (DGL) did not form clusters. Calretinin-IR neurons in the IGL and DGL varied in morphology and included round/oval, vertical fusiform, stellate, and horizontal neurons. Neurons with varicose dendrites were also labeled in the IGL. Most of the labeled neurons were small to medium in size. Monocular enucleation produced an almost complete reduction of calretinin-IR fibers in the SC contralateral to the enucleation. However, many calretinin-IR cells appeared in the contralateral superficial SC. Enucleation appeared to have no effect on the distribution of calretinin-IR neurons in the contralateral intermediate and deep layers of the SC. The calretinin-IR neurons in the superficial dog SC were heterogeneous small- to medium-sized neurons including round/oval, vertical fusiform, stellate, pyriform, and -horizontal in shape. Two-color immunofluorescence revealed that no cells in the dog SC -expressed both calretinin and GABA. Many horseradish peroxidase (HRP)-labeled retinal ganglion cells were seen after injections into the superficial layers. The vast majority of the double-labeled cells (HRP and calretinin) were small cells. The present results indicate that antibody to calretinin labels subpopulations of neurons in the dog SC, which do not express GABA. The results also suggest that the calretinin-IR afferents in the superficial layers of the dog SC originate from small class retinal ganglion cells. The expression of calretinin might be changed by the cellular activity of selective superficial collicular neurons. These results are valuable in delineating the basic neurochemical architecture of the dog visual system.