Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain.

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z- and w-latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w-latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.

Brain2Pix: Fully convolutional naturalistic video frame reconstruction from brain activity.

Reconstructing complex and dynamic visual perception from brain activity remains a major challenge in machine learning applications to neuroscience. Here, we present a new method for reconstructing naturalistic images and videos from very large single-participant functional magnetic resonance imaging data that leverages the recent success of image-to-image transformation networks. This is achieved by exploiting spatial information obtained from retinotopic mappings across the visual system. More specifically, we first determine what position each voxel in a particular region of interest would represent in the visual field based on its corresponding receptive field location. Then, the 2D image representation of the brain activity on the visual field is passed to a fully convolutional image-to-image network trained to recover the original stimuli using VGG feature loss with an adversarial regularizer. In our experiments, we show that our method offers a significant improvement over existing video reconstruction techniques.

Multifunctional colloidal particles for in situ remediation of chlorinated hydrocarbons.

Effective in situ injection technology for the remediation of dense nonaqueous phase liquids (DNAPLs) such as trichloroethylene (TCE) requires the use of decontamination agents that effectively migrate through the soil media and react efficiently with dissolved TCE and bulk TCE. We describe the use of a novel decontamination system containing highly uniform carbon microspheres in the optimal size range for transport through the soil. The microspheres are enveloped in a polyelectrolyte (carboxymethyl cellulose, CMC) to which a bimetallic nanoparticle system of zero-valent iron and Pd is attached. The carbon serves as a strong adsorbent to TCE, while the bimetallic nanoparticle system provides the reactive component. The polyelectrolyte serves to stabilize the carbon microspheres in aqueous solution. The overall system resembles a colloidal micelle with a hydrophilic shell (polyelectrolyte coating) and hard hydrophobic core (carbon). In contact with bulk TCE, there is a sharp partitioning of the system to the TCE side of the interface due to the hydrophobicity of the core. These multifunctional systems appear to satisfy criteria related to remediation and are made with potentially environmentally benign materials.