Less confidence, less forgetting: Learning with a humbler teacher in exemplar-free Class-Incremental learning.

Class-Incremental learning (CIL) is challenging due to catastrophic forgetting (CF), which escalates in exemplar-free scenarios. To mitigate CF, Knowledge Distillation (KD), which leverages old models as teacher models, has been widely employed in CIL. However, based on a case study, our investigation reveals that the teacher model exhibits over-confidence in unseen new samples. In this article, we conduct empirical experiments and provide theoretical analysis to investigate the over-confident phenomenon and the impact of KD in exemplar-free CIL, where access to old samples is unavailable. Building on our analysis, we propose a novel approach, Learning with Humbler Teacher, by systematically selecting an appropriate checkpoint model as a humbler teacher to mitigate CF. Furthermore, we explore utilizing the nuclear norm to obtain an appropriate temporal ensemble to enhance model stability. Notably, LwHT outperforms the state-of-the-art approach by a significant margin of 10.41%, 6.56%, and 4.31% in various settings while demonstrating superior model plasticity.

Nursing model of midwifery and postural and psychological interventions: Impact on maternal and fetal outcomes and negative emotions of primiparas.

BACKGROUND: Primiparas are usually at high risk of experiencing perinatal depression, which may cause prolonged labor, increased blood loss, and intensified pain, affecting maternal and fetal outcomes. Therefore, interventions are necessary to improve maternal and fetal outcomes and alleviate primiparas' negative emotions (NEs). AIM: To discusses the impact of nursing responsibility in midwifery and postural and psychological interventions on maternal and fetal outcomes as well as primiparas' NEs. METHODS: As participants, 115 primiparas admitted to Quanzhou Maternity and Child Healthcare Hospital between May 2020 and May 2022 were selected. Among them, 56 primiparas (control group, Con) were subjected to conventional midwifery and routine nursing. The remaining 59 (research group, Res) were subjected to the nursing model of midwifery and postural and psychological interventions. Both groups were comparatively analyzed from the perspectives of delivery mode (cesarean, natural, or forceps-assisted), maternal and fetal outcomes (uterine inertia, postpartum hemorrhage, placental abruption, neonatal pulmonary injury, and neonatal asphyxia), NEs (Hamilton Anxiety/Depression-rating Scale, HAMA/HAMD), labor duration, and nursing satisfaction. RESULTS: The Res exhibited a markedly higher natural delivery rate and nursing satisfaction than the Con. Additionally, the Res indicated a lower incidence of adverse events (e.g., uterine inertia, postpartum hemorrhage, placental abruption, neonatal lung injury, and neonatal asphyxia) and shortened duration of various stages of labor. It also showed statistically lower post-interventional HAMA and HAMD scores than the Con and pre-interventional values. CONCLUSION: The nursing model of midwifery and postural and psychological interventions increase the natural delivery rate and reduce the duration of each labor stage. These are also conducive to improving maternal and fetal outcomes and mitigating primiparas' NEs and thus deserve popularity in clinical practice.

Decoupled neural network training with re-computation and weight prediction.

To break the three lockings during backpropagation (BP) process for neural network training, multiple decoupled learning methods have been investigated recently. These methods either lead to significant drop in accuracy performance or suffer from dramatic increase in memory usage. In this paper, a new form of decoupled learning, named decoupled neural network training scheme with re-computation and weight prediction (DTRP) is proposed. In DTRP, a re-computation scheme is adopted to solve the memory explosion problem, and a weight prediction scheme is proposed to deal with the weight delay caused by re-computation. Additionally, a batch compensation scheme is developed, allowing the proposed DTRP to run faster. Theoretical analysis shows that DTRP is guaranteed to converge to crical points under certain conditions. Experiments are conducted by training various convolutional neural networks on several classification datasets, showing comparable or better results than the state-of-the-art methods and BP. These experiments also reveal that adopting the proposed method, the memory explosion problem is effectively solved, and a significant acceleration is achieved.

Fully Decoupled Neural Network Learning Using Delayed Gradients.

Training neural networks with backpropagation (BP) requires a sequential passing of activations and gradients. This has been recognized as the lockings (i.e., the forward, backward, and update lockings) among modules (each module contains a stack of layers) inherited from the BP. In this brief, we propose a fully decoupled training scheme using delayed gradients (FDG) to break all these lockings. The FDG splits a neural network into multiple modules and trains them independently and asynchronously using different workers (e.g., GPUs). We also introduce a gradient shrinking process to reduce the stale gradient effect caused by the delayed gradients. Our theoretical proofs show that the FDG can converge to critical points under certain conditions. Experiments are conducted by training deep convolutional neural networks to perform classification tasks on several benchmark data sets. These experiments show comparable or better results of our approach compared with the state-of-the-art methods in terms of generalization and acceleration. We also show that the FDG is able to train various networks, including extremely deep ones (e.g., ResNet-1202), in a decoupled fashion.