Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting.

Deep learning is often criticized by two serious issues that rarely exist in natural nervous systems: overfitting and catastrophic forgetting. It can even memorize randomly labeled data, which has little knowledge behind the instance-label pairs. When a deep network continually learns over time by accommodating new tasks, it usually quickly overwrites the knowledge learned from previous tasks. Referred to as the neural variability, it is well known in neuroscience that human brain reactions exhibit substantial variability even in response to the same stimulus. This mechanism balances accuracy and plasticity/flexibility in the motor learning of natural nervous systems. Thus, it motivates us to design a similar mechanism, named artificial neural variability (ANV), that helps artificial neural networks learn some advantages from "natural" neural networks. We rigorously prove that ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. This result theoretically guarantees ANV a strictly improved generalizability, robustness to label noise, and robustness to catastrophic forgetting. We then devise a neural variable risk minimization (NVRM) framework and neural variable optimizers to achieve ANV for conventional network architectures in practice. The empirical studies demonstrate that NVRM can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.

Pterostilbene attenuates the inflammatory reaction induced by ischemia/reperfusion in rat heart.

The role of pterostilbene (Pte) in inflammation induced by ischemia/reperfusion is not well understood. The aim of this study was to investigate whether Pte modulates neutrophil accumulation and the induction of tumor necrosis factor-α (TNF-α) in an ischemia/reperfusion (I/R)-injured rat heart model. Rats were randomly exposed to a sham operation, myocardial ischemia/reperfusion (MI/R) alone, MI/R+Pte, MI/R+Pte+L-NAME and MI/R+Pte+ (methylene blue) MB. The results demonstrated that compared with MI/R, Pte reduced the area of myocardial infarction, the levels of myocardial myeloperoxidase, serum creatinine kinase and lactate dehydrogenase, and the production of serum and myocardial TNF-α. These Pte-induced effects were eliminated by the administration of L-NAME, a nitric oxide (NO) synthase inhibitor, and MB, a cyclic guanosine monophosphate (cGMP) inhibitor. In conclusion, Pte produces cardioprotective and anti-inflammatory effects. These effects may be associated with an increase in NO production, the inhibition of neutrophil accumulation, and induction of TNF-α and cGMP signaling pathways in myocardium subjected to MI/R.

Impact of off-pump techniques on sex differences in early and late outcomes after isolated coronary artery bypass grafts.

BACKGROUND: Off-pump coronary artery bypass graft surgery (OPCAB) is associated with lower early mortality and benefits women disproportionately. The objective of this study was to assess the impact of off-pump techniques on sex differences in late outcomes. METHODS: We reviewed a clinical database of consecutive patients who underwent isolated coronary artery bypass graft surgery (CABG) at FuWai Hospital from 1999 to 2005. Logistic regression analysis and proportional hazards modeling were used to investigate whether sex or surgery type were associated with early mortality and late outcomes (mortality, major cardiac and cerebral event). RESULTS: Female sex was associated with higher rates of early death (adjusted odds ratio, 4.726; p < 0.0001), and OPCAB benefited women disproportionately for early mortality. Odds ratio of death for women versus men was 4.726 (p < 0.0001) in the conventional CABG on cardiopulmonary bypass group; odds ratio of death for women versus men was 1.344 (p = 0.5617) in the OPCAB group. Analysis of late outcomes indicated that OPCAB and cardiopulmonary bypass resulted in similar survival, regardless of sex. The women versus men hazard ratio of late mortality after CABG on cardiopulmonary bypass and OPCAB for women was 0.851 (p = 0.4984) and 0.650 (p = 0.2005), respectively. Women treated with OPCAB were less likely to be free from major cardiac and cerebral events than men treated with OPCAB. The women versus men hazard ratio of major cardiac and cerebral events after CABG on cardiopulmonary bypass and OPCAB for women was 1.079 (p = 0.4992) and 1.299 (p = 0.0387), respectively. CONCLUSIONS: Compared with men, women are a high-risk group and benefit from off-pump operation in terms of early mortality after CABG. Conversely, during follow-up, women have high adjusted risks of major cardiac and cerebral events after OPCAB.