Efficient Masked Autoencoders With Self-Consistency.

Inspired by the masked language modeling (MLM) in natural language processing tasks, the masked image modeling (MIM) has been recognized as a strong self-supervised pre-training method in computer vision. However, the high random mask ratio of MIM results in two serious problems: 1) the inadequate data utilization of images within each iteration brings prolonged pre-training, and 2) the high inconsistency of predictions results in unreliable generations, i.e., the prediction of the identical patch may be inconsistent in different mask rounds, leading to divergent semantics in the ultimately generated outcomes. To tackle these problems, we propose the efficient masked autoencoders with self-consistency (EMAE) to improve the pre-training efficiency and increase the consistency of MIM. In particular, we present a parallel mask strategy that divides the image into K non-overlapping parts, each of which is generated by a random mask with the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and the model minimizes the loss between the predictions and the masked patches. Besides, we design the self-consistency learning to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, our method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves the best performance on ViT-Large with only 13% of MAE pre-training time using NVIDIA A100 GPUs. After pre-training on diverse datasets, EMAE consistently obtains state-of-the-art transfer ability on a variety of downstream tasks, such as image classification, object detection, and semantic segmentation.

Advancements in the study of synaptic plasticity and mitochondrial autophagy relationship.

Synapses serve as the points of communication between neurons, consisting primarily of three components: the presynaptic membrane, synaptic cleft, and postsynaptic membrane. They transmit signals through the release and reception of neurotransmitters. Synaptic plasticity, the ability of synapses to undergo structural and functional changes, is influenced by proteins such as growth-associated proteins, synaptic vesicle proteins, postsynaptic density proteins, and neurotrophic growth factors. Furthermore, maintaining synaptic plasticity consumes more than half of the brain's energy, with a significant portion of this energy originating from ATP generated through mitochondrial energy metabolism. Consequently, the quantity, distribution, transport, and function of mitochondria impact the stability of brain energy metabolism, thereby participating in the regulation of fundamental processes in synaptic plasticity, including neuronal differentiation, neurite outgrowth, synapse formation, and neurotransmitter release. This article provides a comprehensive overview of the proteins associated with presynaptic plasticity, postsynaptic plasticity, and common factors between the two, as well as the relationship between mitochondrial energy metabolism and synaptic plasticity.