Calibrating the Adaptive Learning Rate to Improve Convergence of ADAM.

Adaptive gradient methods (AGMs) have become popular in optimizing the nonconvex problems in deep learning area. We revisit AGMs and identify that the adaptive learning rate (A-LR) used by AGMs varies significantly across the dimensions of the problem over epochs (i.e., anisotropic scale), which may lead to issues in convergence and generalization. All existing modified AGMs actually represent efforts in revising the A-LR. Theoretically, we provide a new way to analyze the convergence of AGMs and prove that the convergence rate of Adam also depends on its hyper-parameter є, which has been overlooked previously. Based on these two facts, we propose a new AGM by calibrating the A-LR with an activation (softplus) function, resulting in the Sadam and SAMSGrad methods. We further prove that these algorithms enjoy better convergence speed under nonconvex, non-strongly convex, and Polyak-Lojasiewicz conditions compared with Adam. Empirical studies support our observation of the anisotropic A-LR and show that the proposed methods outperform existing AGMs and generalize even better than S-Momentum in multiple deep learning tasks.

Asynchronous Parallel Stochastic Quasi-Newton Methods.

Although first-order stochastic algorithms, such as stochastic gradient descent, have been the main force to scale up machine learning models, such as deep neural nets, the second-order quasi-Newton methods start to draw attention due to their effectiveness in dealing with ill-conditioned optimization problems. The L-BFGS method is one of the most widely used quasi-Newton methods. We propose an asynchronous parallel algorithm for stochastic quasi-Newton (AsySQN) method. Unlike prior attempts, which parallelize only the calculation for gradient or the two-loop recursion of L-BFGS, our algorithm is the first one that truly parallelizes L-BFGS with a convergence guarantee. Adopting the variance reduction technique, a prior stochastic L-BFGS, which has not been designed for parallel computing, reaches a linear convergence rate. We prove that our asynchronous parallel scheme maintains the same linear convergence rate but achieves significant speedup. Empirical evaluations in both simulations and benchmark datasets demonstrate the speedup in comparison with the non-parallel stochastic L-BFGS, as well as the better performance than first-order methods in solving ill-conditioned problems.

ROS-induced NLRP3 inflammasome priming and activation mediate PCB 118- induced pyroptosis in endothelial cells.

Growing epidemiological evidence has shown that exposure to polychlorinated biphenyls (PCBs) is harmful to the cardiovascular system. However, how PCB 118-induced oxidative stress mediates endothelial dysfunction is not fully understood. Here, we explored whether and how PCB 118 exposure-induced oxidative stress leads to NLRP3 inflammasome-dependent pyroptosis in endothelial cells. As expected, PCB 118 was cytotoxic to HUVECs and induced caspase-1 activation and cell membrane disruption, which are characteristics of pyroptosis. Moreover, PCB 118-induced pyroptosis may have been due to the activation of the NLRP3 infammasomes. PCB 118 also induced excessive reactive oxygen species (ROS) in HUVECs. The ROS scavenger (±)-α-tocopherol and the NFκB inhibitor BAY11-7082 reversed the upregulation of NLRP3 expression and the increase in NLRP3 inflammasome activation induced by PCB 118 exposure in HUVECs. Additionally, PCB 118-induced oxidative stress and pyroptosis were dependent on Aryl hydrocarbon receptor (AhR) activation and subsequent cytochrome P450 1A1 upregulation, which we confirmed by using the AhR selective antagonist CH 223191. These data suggest that PCB 118 exposure induces NLRP3 inflammasome activation and subsequently leads to pyroptosis in endothelial cells in vitro and in vivo. AhR-mediated ROS production play a central role in PCB 118-induced pyroptosis by priming NFκB-dependent NLRP3 expression and promoting inflammasome activation.

2,3',4,4',5-Pentachlorobiphenyl impairs insulin-induced NO production partly through excessive ROS production in endothelial cells.

Polychlorinated biphenyls (PCBs) have been reported to be associated with increased risk to hypertension, atherosclerosis, cardiovascular disease, etc. 2,3',4,4',5-Pentachlorobiphenyl, known as PCB-118, is a member of coplanar PCBs which renders their structure similar to polychlorinated dibenzo-p-dioxins (PCDDs) and has dioxin-like activity. In our current study, we investigated the effect of PCB-118 exposure on nitric oxide (NO) production and the underlying mechanisms in vitro. Exposure of PCB-118 impaired insulin-induced NO production and endothelial nitric oxide synthase (eNOS) activity in human umbilical vein endothelial cells (HUVECs) with no significant effect on cell viability. Furthermore, PCB-118 treatment induced oxidative stress. In addition, scavenging of reactive oxygen species (ROS) by 10 µM N-acetyl-l-cysteine (NAC) partly rescued impaired insulin-induced eNOS activities and NO productions induced by PCB-118 in HUVECs. Taken together, these results indicate that PCB-118 mediates lower eNOS activity and impairs insulin-induced NO production partly through excessive ROS production in endothelial cells.

PCB 118-induced endothelial cell apoptosis is partially mediated by excessive ROS production.

Endothelial cell apoptosis, which may alter the integrity of the endothelium and lead to plaque instability, plays a critical role in the development and pathogenesis of atherosclerosis. Exposure of polychlorinated biphenyls (PCBs) is associated with increased risk of atherosclerosis and cardiovascular disease. In our present study, we explored whether exposure to PCB 118 influences endothelial cell apoptosis in vitro and the underlying mechanisms involved. As expected, exposure to PCB 118 increased the intracellular reactive oxygen species (ROS) levels in HUVECs. Increases in apoptosis and Bax/Bcl-2 ratios were observed in PCB 118-treated HUVECs. N-acetyl-l-cysteine (NAC), a ROS scavenger, partially reduced PCB 118-induced apoptosis and Bax/Bcl-2 ratios in HUVECs. Taken together, PCB 118-induced endothelial cell apoptosis was partially initiated by excessive ROS production.

[ERK signaling pathway mediated epithelial-mesenchymal transition induced by SiO2 in human bronchial epithelial cells].

OBJECTIVE: To determine the role of extracellular signal regulated kinase (ERK) signaling pathway in SiO2 induced epithelial-mesenchymal transition (EMT) in human bronchial epithelial cells (HBEC) in vitro. METHODS: HBEC were treated with SiO2 (0-300 µg/mL) for 72 h or pretreated with U0126 (0-30 µmol/L) for 1 h and then treated with 200 µg/mL SiO2 for 72 h. Western blot was used to detect the protein expression of E-cadherin and α-smooth muscle actin (α-SMA). The activity of ERK was examined by mitogen-activated protein kinase (MAPK) activity assay kit in HBEC exposing to SiO2 (200 µg/mL) for 0-8 h. RESULTS: The expression of E-cadherin decreased gradually in SiO2 -stimulated HBEC, and the effect was most significant at 300 µg/mL (P<0.01). The expression of α-SMA increased and the effect was most evident at 200 µg/mL (P<0.01). With SiO2 treatment, the activity of ERK was upregulated significantly. The phosphorylation of ERK increased at 30 min and decreased after 1 h. U0126 significantly inhibited SiO2 -induced expression changes in E-cadherin and α-SMA. At 30 µmol/L, the effect was most evident(P<0.01). CONCLUSION: ERK signaling pathway mediated EMT induced by SiO2 in HBEC.

[The influence of SiO2 on epithelial-mesenchymal transition (EMT) in human bronchial epithelial cells].

OBJECTIVE: To investigate SiO2-induced EMT in human bronchial epithelial cells HBE in vitro. METHODS: HBE cells were cultured and then stimulated with indicated doses of SiO2 (0, 50, 100, 200, 300 µg/ml). The morphological changes were observed by microscope. In addition, Western blot was per-formed to detect the expression of E-cad, α-SMA and Vim. The changes of migration ability were examined by wound-healing assay in vitro. RESULTS: (1) After exposure to SiO2, HBE cells lost contact with their neighbor and displayed a spindle-shape, fibroblast-like morphology. (2) Compared with the control, the E-cad (300 µg/ml group) expression downregulated 2.98 fold (P < 0.05), and the Vim (300 µg/ml group) and α-SMA (200 µg/ml group) expression upregulated 4.46 fold and 3.55 fold (P < 0.05). There were significant differences between 100, 200, 300 µg/ml groups and the control group (P < 0.05). (3) In the test group, the percentage of wound-healing areas/wound areas were larger than those in control group (P < 0.05). CONCLUSIONS: SiO2 could induce EMT in human bronchial epithelial cells.

An Efficient Algorithm for Deep Stochastic Contextual Bandits.

In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to examine whether and where these methods converge. In this work, we formulate the SCB that uses a DNN reward function as a non-convex stochastic optimization problem, and design a stage-wise stochastic gradient descent algorithm to optimize the problem and determine the action policy. We prove that with high probability, the action sequence chosen by this algorithm converges to a greedy action policy respecting a local optimal reward function. Extensive experiments have been performed to demonstrate the effectiveness and efficiency of the proposed algorithm on multiple real-world datasets.

Secretory carcinoma of the breast with multiple distant metastases in the brain and unfavorable prognosis: a case report and literature review.

BACKGROUND: Secretory carcinoma of the breast is one of the rarest entities, accounting for less than 0.15 % of all infiltrating breast carcinomas. It has characteristic histopathological and molecular features and, in general, a more favorable prognosis. In this case report, we describe a local, advanced secretory carcinoma of the breast with aggressive course and an unfavorable outcome. CASE PRESENTATION: A hard, painless, and palpably bossed mass approximately 12.0 cm in diameter occupied most of the left breast of a 39-year-old woman with fixation to the overlying skin. Breast ultrasonography and magnetic resonance imaging (MRI) scans gave the same grading as BI-RADS IV. A needle biopsy was performed, and the pathological diagnosis was secretory carcinoma. Neoadjuvant chemotherapy (NAC) was then performed, after which ultrasonography and MRI scans revealed chemo-resistance of the tumor to NAC. Left breast mastectomy and axillary lymphadenectomy were subsequently performed. Tumor cells were triple-negative and positive for S-100 and periodic acid-Schiff (PAS) staining. Fluorescence in-situ hybridization (FISH) analysis indicated a fusion arrangement of the ETV6-NTRK3 gene. The patient developed multiple distant metastases in the brain and died of these metastases 19 months after initial diagnosis. CONCLUSIONS: Secretory carcinomas of the breast have been described as a low-grade histologic subtype with a favorable prognosis. This case showed chemo-resistance to neoadjuvant chemotherapy, multiple distant metastases, and a final unfavorable outcome. Further research is needed to better understand the behavior and treatment of this rare tumor.

Effect of Rho signaling pathway on SiO2 induced α-SMA expression in human bronchial epithelial cells.

OBJECTIVE: To investigate the role of Rho in SiO2 induced α-SMA expression in human bronchial epithelial cells (HBECs). METHODS: HBECs were cultured and stimulated with SiO2. Immunocytochemistry and Western blot were used to detect the expression of α-SMA. The activity of Rho was determined by GST pull down assay. In the prevention experiment, SiO2-stimulated HBECs were incubated with Rho inhibitor Y27632, and the expression of α-SMA was examined by Western blot. RESULTS: With SiO2 (0-300 µg/mL) treatment, the expression of α-SMA increased gradually, and 200 µg/mL of SiO2 led to the highest expression of α-SMA which was (5.09±1.98) times of the expression of α-SMA in the control group(P<0.01). HBECs treated with SiO2 (200 µg/mL) for indicated time (1, 2, 6, 12, and 24 h) showed an obvious increase of Rho activity(P<0.01). Y27632 inhibited SiO2-induced α-SMA expression significantly, and the inhibition rate of 20 and 30 µmol/L Y27632 was 68% and 75%, respectively (P<0.01). CONCLUSION: Rho signaling pathway may mediate SiO2 induced α-SMA expression in HBECs.

The effect of retroviral vector on uptake of human lactoferrin DNA by Yak (Bos Grunniens) spermatozoa and their fertilizability in vitro.

The objectives of this study were to evaluate the transfection effectiveness of retroviral vector PLNCX2 in yak sperm-mediated gene transfer (SMGT) and the effect of SMGT on sperm motility and fertilizability. Human lactoferrin (hLF) DNA was ligated into PLNCX2 to construct recombinant plasmid PLNCX2-hLF, then, using PLNCX2-hLF+FuGene 6 to generate SMGT yak spermatozoa for fertilizing bovine oocytes. The result showed that DNA-binding rate increased with the extension of incubation period and DNA treatment did not decrease sperm motility. Oocytes inseminated with SMGT-spermatozoa had a lower (P < 0.05) cleavage rate (57.7%) than the control (73.4%), but development up to blastocyst stage was not different (26.8 vs. 31.7%). It appears that PLNCX2 is useful for generating transgenic yaks by SMGT.

Precise control of iron activating persulfate by current generation in an electrochemical membrane reactor.

Activated persulfate (PS) oxidation is promising for contaminant removal but a lack of controllable activation can lead to a loss of reagents and thus low contamination degradation. Herein, we have proposed and investigated an innovative method to control PS activation by introducing ion exchange membrane into electrochemically activated PS. This electrochemical membrane reactor (EMR) could precisely control PS activation by adjusting electrical current for slow release of Fe2+, and also avoid direct contact between PS and a sacrificial anode electrode (iron electrode)/an alkaline cathode solution. It was found that the PS decomposition rate constant was linearly increased by increasing the applied current (R2 = 0.988). The rate of the released Fe2+ also exhibited a linear relationship with the applied current (R2 = 0.995). Compared to one-time dosage of Fe2+, the EMR-based slow-release process had higher contamination degradation and better PS utilization (molar ratio of the decomposed PS to the migrated Fe, 1.04 ±â€¯0.01:1), thereby minimizing the waste of both reaction reagents and generated radicals. The EMR was also employed to degrade a representative dye contaminant in a controllable manner and achieved 95.7 ±â€¯0.7% removal percentage with PS dosage of 3.0 g L-1 within 20 min. This study is among the earliest to explore effective approaches for precisely controlling PS activation and subsequent oxidation of contaminants.

Enhancing recovery of magnesium as struvite from landfill leachate by pretreatment of calcium with simultaneous reduction of liquid volume via forward osmosis.

Landfill leachate contains substances that can be potentially recovered as valuable resources. In this study, magnesium in a landfill leachate was recovered as struvite with calcium pretreatment; meanwhile, the leachate volume was reduced by using a submerged forward osmosis (FO) process, thereby enabling significant reduction of further treatment footprint and cost. Without pretreatment, calcium exhibited strong competition for phosphate with magnesium. The pretreatment with a Ca2+: CO32- molar ratio of 1:1.4 achieved a relatively low loss rate of Mg2+ (24.1±2.0%) and high Ca2+ removal efficiency (89.5±1.7%). During struvite recovery, 98.6±0.1% of magnesium could be recovered with a significantly lower residual PO43--P concentration (<25mgL-1) under the condition of (Mg+Caresidual): P molar ratio of 1:1.5 and pH9.5. The obtained struvite had a similar crystal structure and composition (19.3% Mg and 29.8% P) to that of standard struvite. The FO process successfully recovered water from the leachate and reduced its volume by 37%. The configuration of calcium pretreatment - FO - struvite recovery was found to be the optimal arrangement in terms of FO performance. These results have demonstrated the feasibility of magnesium recovery from landfill leachate and the importance of the calcium pretreatment, and will encourage further efforts to assess the value and purity of struvite for commercial use and to develop new methods for resource recovery from leachate.

miR-503 suppresses the proliferation and metastasis of esophageal squamous cell carcinoma by triggering autophagy via PKA/mTOR signaling.

MicroRNA (miR)-503 is involved in the regulation of the malignant phenotype in multiple tumor types, and has been proven to be a novel diagnostic and therapeutic target; however, its function and mechanisms of action have not yet been fully elucidated in esophageal squamous cell carcinoma (ESCC). In the current study, we detected miR­503 expression by RT­qPCR and found that miR­503 expression was increased in ESCC, but negatively correlated with lymph node metastasis, TNM stage and tumor differentiation. Functionally, we confirmed that miR­503 inhibited the proliferation and metastasis of ESCC cells by triggering cellular autophagy. Mechanistically, we confirmed that miR­503 exerted its biological effects by targeting protein kinase CAMP­activated catalytic subunit alpha (PRKACA) in ESCC by dual luciferase reporter assay. Moreover, miR­503 was found to trigger autophagy in ESCC cells through the protein kinase A (PKA)/mammalian target of rapamycin (mTOR) pathway. Taken together, our results demonstrate that miR­503 suppresses the proliferation and metastasis of ESCC via the activation of autophagy, mediated by the PKA/mTOR signaling pathway.

ATP2B1 Gene Silencing Increases NO Production Under Basal Conditions Through the Ca2+/calmodulin/eNOS Signaling Pathway in Endothelial Cells.

Emerging epidemiological and experimental evidence has shown that the ATP2B1 gene is associated with blood pressure control. Impaired eNOS activity and NO production may be among the mechanisms involved. However, little is known about how PMCA1, which is encoded by the ATP2B1 gene, regulates the activity of eNOS and NO production. In the present study, we investigated the role of the ATP2B1 gene in regulating eNOS activity and NO production under basal conditions in HUVECs and explored the mechanisms involved. Silencing ATP2B1 gene expression resulted in higher NO production and eNOS activity under basal conditions in HUVECs. Additionally, ATP2B1 gene silencing resulted in enhanced intracellular calcium concentrations compared to that in the negative siRNA-transfected HUVECs. The enhanced eNOS activity mediated by ATP2B1 gene silencing was Ca2+/calmodulin dependent, as verified by the administration of the calcium chelator BAPTA-AM or the calmodulin-specific antagonist W7. Taken together, silencing ATP2B1 gene expression results in higher NO production and eNOS activity under basal conditions in HUVECs. Furthermore, the enhanced eNOS activity induced by ATP2B1 gene silencing may be mediated via higher levels of intracellular Ca2+, and the effect was confirmed to be dependent on the eNOS-calmodulin interaction.

The generation of biogenic manganese oxides and its application in the removal of As(III) in groundwater.

The generation of biogenic manganese oxides (BMnOx) by Microbacterium sp. CSA40, and As(III) removal efficiency and mechanism by BMnOx were investigated in this study. The propagation and growth of Microbacterium sp. CSA40 was conducted in half-strength Luria Broth with 10 mg/L Mn(II), then high concentration of Microbacterium sp. CSA40 was added to PYG medium making its OD600 = 0.9 ± 0.05 for BMnOx generation. The initial Mn(II) concentrations, excessively oligotrophic condition, and pH had great influence on generation of BMnOx by Microbacterium sp. CSA40. An appropriate Mn(II) concentration (50 mg/L) was obtained for generation of BMnOx, and higher or lower Mn(II) concentration would interfere Mn(II) oxidization performance. Mn(II) oxidation ability performed best in weak alkaline conditions and would be restricted in an excessively oligotrophic condition. As(III) oxidization and As(V) adsorption proceed simultaneously by BMnOx, what is more, more than 90% of total As was removed by 0.5 g/L BMnOx. During the application process, no Mn(II) was released in the solution, that is, BMnOx retained its ability for Mn(II) oxidization caused by activity of Microbacterium sp. CSA40. Therefore, BMnOx would be a pollution-free, cost-effective, and high-efficiency material for As(III) treatment in groundwater.

Evaluation of heavy metal mobilization in creek sediment: Influence of RAC values and ambient environmental factors.

The risk assessment code (RAC) is a common method for assessing heavy metal (HM) mobility and their potential health risks, based on HM total concentration and chemical speciation. In this study, both the RAC and the influence of ambient environmental factors were investigated in a river sediment system. Sixty-eight sediment samples were collected from the main river system in Shanghai, China. The total concentration and chemical speciation of Cu, Zn, Ni, Pb, Cd, Cr, As, and Hg were determined in the samples. The influence of sediment environmental factors, such as acid-volatile sulfide (AVS), Fe & Mn, and total organic carbon (TOC), on total metal concentrations and speciation was also investigated. The relationship between the main environmental media and distribution of HMs was discussed using PCA and NMDS. The transfer-transformation behaviors of Pb, Ni, and Cr were mainly controlled by AVS and TOC while Zn, Cu, and Cd were influenced by Fe & Mn and TOC. The relationship between the RAC value of HM and environmental factors revealed that 7% of Cr, 23% of Ni, and 15% of Pb had a high risk of mobility at TOC values below 3.5% and sulfite contents below 10mmol/kg. In comparison, 29%, 10%, and 10% of Zn, Cu, and Cd, respectively, had a high risk of mobility at TOC<3.5% and Fe & Mn content >4×105mg/kg. Evidently, the chemical fractions of HM had a strong dependence on the S, Fe, Mn, and organic compounds in the sediment. This study provides a promising pathway for the rapid evaluation of potential risks from HMs in river sediments.

MACC1 induces autophagy to regulate proliferation, apoptosis, migration and invasion of squamous cell carcinoma.

Metastasis-associated colon cancer-1 (MACC1) plays an important role in cancer development, but the role and mechansim of MACC1 in squamous cell carcinoma (ESCC) remain unclear. In this study, we found that MACC1 expression was increased in ESCC, and correlated with lymph node metastasis. MACC1 knockdown suppresed ESCC cell proliferation, metastasis and enchanced cell apoptosis. Moreover, MACC1 knockdown inhibited ESCC cell autophagy, and 3-methyladenine was able to rescue MACC1-induced malignant phenotype of ESCC cells. Furthermore, MACC1 knockdown inactivated AMPK-ULK1 signaling pathway, and metformin could rescue MACC1-induced autophagy in ESCC cells. Collectively, this study found that upregulation of MACC1 in ESCC was associated with lymph node metastasis of patients, and MACC1 regulated ESCC cell proliferation, apoptosis, migration and invasion mainly through AMPK-ULK1 induced autophagy.

ATP2B1 gene Silencing Increases Insulin Sensitivity through Facilitating Akt Activation via the Ca2+/calmodulin Signaling Pathway and Ca2+-associated eNOS Activation in Endothelial Cells.

Endothelial cell insulin resistance may be partially responsible for the higher risk of atherosclerosis and cardiovascular disease in populations with insulin resistance and type 2 diabetes mellitus (T2DM). A genome-wide association study revealed a significant association between the ATPase plasma membrane Ca2+ transporting 1 (ATP2B1) gene and T2DM in two community-based cohorts from the Korea Association Resource Project. However, little is known about the implication of the ATP2B1 gene on T2DM. In the present study, we investigated the role of the ATP2B1 gene in endothelial cell insulin sensitivity. ATP2B1 gene silencing resulted in enhanced intracellular calcium concentrations and increased insulin-induced Akt activation compared to that in the negative siRNA-transfected HUVECs (Human Umbilical Vein Endothelial Cells). The elevated insulin sensitivity mediated by ATP2B1 gene silencing was Ca2+/calmodulin-dependent, as verified by administration of the calcium chelator BAPTA-AM or the calmodulin-specific antagonist W7. Moreover, higher levels of phosphorylation of eNOS (Ser1177) were observed in ATP2B1-silenced HUVECs. In addition to BAPTA-AM and W7, L-NAME, an eNOS antagonist, abolished insulin-induced Akt phosphorylation at Ser473 in both si-Neg and si-ATP2B1-transfected endothelial cells. These results indicate that the enhanced insulin sensitivity in ATP2B1-silenced endothelial cells is alternatively dependent on an increase in intracellular Ca2+ and the subsequent activation of the Ca2+/calmodulin/eNOS/Akt signaling pathway. In summary, ATP2B1 gene silencing increased insulin sensitivity in endothelial cells by directly modulating the Ca2+/calmodulin signaling pathway and via the Ca2+/calmodulin/eNOS/Akt signaling pathway alternatively.

A Sparse Interactive Model for Matrix Completion with Side Information.

Matrix completion methods can benefit from side information besides the partially observed matrix. The use of side features that describe the row and column entities of a matrix has been shown to reduce the sample complexity for completing the matrix. We propose a novel sparse formulation that explicitly models the interaction between the row and column side features to approximate the matrix entries. Unlike early methods, this model does not require the low rank condition on the model parameter matrix. We prove that when the side features span the latent feature space of the matrix to be recovered, the number of observed entries needed for an exact recovery is O(log N) where N is the size of the matrix. If the side features are corrupted latent features of the matrix with a small perturbation, our method can achieve an ε-recovery with O(log N) sample complexity. If side information is useless, our method maintains a O(N3/2) sampling rate similar to classic methods. An efficient linearized Lagrangian algorithm is developed with a convergence guarantee. Empirical results show that our approach outperforms three state-of-the-art methods both in simulations and on real world datasets.