Magnetic control of tokamak plasmas through deep reinforcement learning.

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and 'snowflake' configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained 'droplets' on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

Muscle-selective RUNX3 dependence of sensorimotor circuit development.

The control of all our motor outputs requires constant monitoring by proprioceptive sensory neurons (PSNs) that convey continuous muscle sensory inputs to the spinal motor network. Yet the molecular programs that control the establishment of this sensorimotor circuit remain largely unknown. The transcription factor RUNX3 is essential for the early steps of PSNs differentiation, making it difficult to study its role during later aspects of PSNs specification. Here, we conditionally inactivate Runx3 in PSNs after peripheral innervation and identify that RUNX3 is necessary for maintenance of cell identity of only a subgroup of PSNs, without discernable cell death. RUNX3 also controls the sensorimotor connection between PSNs and motor neurons at limb level, with muscle-by-muscle variable sensitivities to the loss of Runx3 that correlate with levels of RUNX3 in PSNs. Finally, we find that muscles and neurotrophin 3 signaling are necessary for maintenance of RUNX3 expression in PSNs. Hence, a transcriptional regulator that is crucial for specifying a generic PSN type identity after neurogenesis is later regulated by target muscle-derived signals to contribute to the specialized aspects of the sensorimotor connection selectivity.

The role of Sema3-Npn-1 signaling during diaphragm innervation and muscle development.

Correct innervation of the main respiratory muscle in mammals, namely the thoracic diaphragm, is a crucial pre-requisite for the functionality of this muscle and the viability of the entire organism. Systemic impairment of Sema3A-Npn-1 (Npn-1 is also known as NRP1) signaling causes excessive branching of phrenic nerves in the diaphragm and into the central tendon region, where the majority of misguided axons innervate ectopic musculature. To elucidate whether these ectopic muscles are a result of misguidance of myoblast precursors due to the loss of Sema3A-Npn-1 signaling, we conditionally ablated Npn-1 in somatic motor neurons, which led to a similar phenotype of phrenic nerve defasciculation and, intriguingly, also formation of innervated ectopic muscles. We therefore hypothesize that ectopic myocyte fusion is caused by additional factors released by misprojecting growth cones. Slit2 and its Robo receptors are expressed by phrenic motor axons and migrating myoblasts, respectively, during innervation of the diaphragm. In vitro analyses revealed a chemoattractant effect of Slit2 on primary diaphragm myoblasts. Thus, we postulate that factors released by motor neuron growth cones have an influence on the migration properties of myoblasts during establishment of the diaphragm.

Functional dissection of the Pax6 paired domain: Roles in neural tube patterning and peripheral nervous system development.

During development of the CNS, stem and progenitor cell proliferation, cell fate designation, and patterning decisions are tightly regulated by interdependent networks of key transcriptional regulators. In a genetic approach we analyzed divergent functionality of the PAI and RED sub-domains of the Pax6 Paired domain (PD) during progenitor zone formation, motor and interneuron development, and peripheral connectivity at distinct levels within the neural tube: within the hindbrain, mutation of the PAI sub-domain severely affected patterning of the p3 and pMN domains and establishment of the corresponding motor neurons. Exit point designation of hypoglossal axons was disturbed in embryos harboring either mutations in the PD sub-domains or containing a functional Pax6 Null allele. At brachial spinal levels, we propose a selective involvement of the PAI sub-domain during patterning of ventral p2 and pMN domains, critically disturbing generation of specific motor neuron subtypes and increasing V2 interneuron numbers. Our findings present a novel aspect of how Pax6 not only utilizes its modular structure to perform distinct functions via its paired and homeodomain. Individual sub-domains can exert distinct functions, generating a new level of complexity for transcriptional regulation by one single transcription factor not only in dorso-ventral, but also rostro-caudal neural tube patterning.

Loss of Npn1 from motor neurons causes postnatal deficits independent from Sema3A signaling.

The correct wiring of neuronal circuits is of crucial importance for the function of the vertebrate nervous system. Guidance cues like the neuropilin receptors (Npn) and their ligands, the semaphorins (Sema) provide a tight spatiotemporal control of sensory and motor axon growth and guidance. Among this family of guidance partners the Sema3A-Npn1 interaction has been shown to be of great importance, since defective signaling leads to wiring deficits and defasciculation. For the embryonic stage these defects have been well described, however, also after birth the organism can adapt to new challenges by compensational mechanisms. Therefore, we used the mouse lines Olig2-Cre;Npn1(cond) and Npn1(Sema-) to investigate how postnatal organisms cope with the loss of Npn1 selectively from motor neurons or a systemic dysfunctional Sema3A-Npn1 signaling in the entire organism, respectively. While in Olig2-Cre(+);Npn1(cond-/-) mice clear anatomical deficits in paw posturing, bone structure, as well as muscle and nerve composition became evident, Npn1(Sema-) mutants appeared anatomically normal. Furthermore, Olig2-Cre(+);Npn1(cond) mutants revealed a dysfunctional extensor muscle innervation after single-train stimulation of the N.radial. Interestingly, these mice did not show obvious deficits in voluntary locomotion, however, skilled motor function was affected. In contrast, Npn1(Sema-) mutants were less affected in all behavioral tests and able to improve their performance over time. Our data suggest that loss of Sema3A-Npn1 signaling is not the only cause for the observed deficits in Olig2-Cre(+);Npn1(cond-/-) mice and that additional, yet unknown binding partners for Npn1 may be involved that allow Npn1(Sema-) mutants to compensate for their developmental deficits.

MicroRNA-9 promotes the switch from early-born to late-born motor neuron populations by regulating Onecut transcription factor expression.

Motor neurons in the vertebrate spinal cord are stereotypically organized along the rostro-caudal axis in discrete columns that specifically innervate peripheral muscle domains. Originating from the same progenitor domain, the generation of spinal motor neurons is orchestrated by a spatially and temporally tightly regulated set of secreted molecules and transcription factors such as retinoic acid and the Lim homeodomain transcription factors Isl1 and Lhx1. However, the molecular interactions between these factors remained unclear. In this study we examined the role of the microRNA 9 (miR-9) in the specification of spinal motor neurons and identified Onecut1 (OC1) as one of its targets. miR-9 and OC1 are expressed in mutually exclusive patterns in the developing chick spinal cord, with high OC1 levels in early-born motor neurons and high miR-9 levels in late-born motor neurons. miR-9 efficiently represses OC1 expression in vitro and in vivo. Overexpression of miR-9 leads to an increase in late-born neurons, while miR-9 loss-of-function induces additional OC1(+) motor neurons that display a transcriptional profile typical of early-born neurons. These results demonstrate that regulation of OC1 by miR-9 is a crucial step in the specification of spinal motor neurons and support a model in which miR-9 expression in late-born LMCl neurons downregulates Isl1 expression through inhibition of OC1. In conclusion, our study contributes essential factors to the molecular network specifying spinal motor neurons and emphasizes the importance of microRNAs as key players in the generation of neuronal diversity.

Onecut transcription factors act upstream of Isl1 to regulate spinal motoneuron diversification.

During development, spinal motoneurons (MNs) diversify into a variety of subtypes that are specifically dedicated to the motor control of particular sets of skeletal muscles or visceral organs. MN diversification depends on the coordinated action of several transcriptional regulators including the LIM-HD factor Isl1, which is crucial for MN survival and fate determination. However, how these regulators cooperate to establish each MN subtype remains poorly understood. Here, using phenotypic analyses of single or compound mutant mouse embryos combined with gain-of-function experiments in chick embryonic spinal cord, we demonstrate that the transcriptional activators of the Onecut family critically regulate MN subtype diversification during spinal cord development. We provide evidence that Onecut factors directly stimulate Isl1 expression in specific MN subtypes and are therefore required to maintain Isl1 production at the time of MN diversification. In the absence of Onecut factors, we observed major alterations in MN fate decision characterized by the conversion of somatic to visceral MNs at the thoracic levels of the spinal cord and of medial to lateral MNs in the motor columns that innervate the limbs. Furthermore, we identify Sip1 (Zeb2) as a novel developmental regulator of visceral MN differentiation. Taken together, these data elucidate a comprehensive model wherein Onecut factors control multiple aspects of MN subtype diversification. They also shed light on the late roles of Isl1 in MN fate decision.

Npn-1 contributes to axon-axon interactions that differentially control sensory and motor innervation of the limb.

The initiation, execution, and completion of complex locomotor behaviors are depending on precisely integrated neural circuitries consisting of motor pathways that activate muscles in the extremities and sensory afferents that deliver feedback to motoneurons. These projections form in tight temporal and spatial vicinities during development, yet the molecular mechanisms and cues coordinating these processes are not well understood. Using cell-type specific ablation of the axon guidance receptor Neuropilin-1 (Npn-1) in spinal motoneurons or in sensory neurons in the dorsal root ganglia (DRG), we have explored the contribution of this signaling pathway to correct innervation of the limb. We show that Npn-1 controls the fasciculation of both projections and mediates inter-axonal communication. Removal of Npn-1 from sensory neurons results in defasciculation of sensory axons and, surprisingly, also of motor axons. In addition, the tight coupling between these two heterotypic axonal populations is lifted with sensory fibers now leading the spinal nerve projection. These findings are corroborated by partial genetic elimination of sensory neurons, which causes defasciculation of motor projections to the limb. Deletion of Npn-1 from motoneurons leads to severe defasciculation of motor axons in the distal limb and dorsal-ventral pathfinding errors, while outgrowth and fasciculation of sensory trajectories into the limb remain unaffected. Genetic elimination of motoneurons, however, revealed that sensory axons need only minimal scaffolding by motor axons to establish their projections in the distal limb. Thus, motor and sensory axons are mutually dependent on each other for the generation of their trajectories and interact in part through Npn-1-mediated fasciculation before and within the plexus region of the limbs.

Learning agile soccer skills for a bipedal robot with deep reinforcement learning.

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent's tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

Cranial nerve fasciculation and Schwann cell migration are impaired after loss of Npn-1.

Interaction of the axon guidance receptor Neuropilin-1 (Npn-1) with its repulsive ligand Semaphorin 3A (Sema3A) is crucial for guidance decisions, fasciculation, timing of growth and axon-axon interactions of sensory and motor projections in the embryonic limb. At cranial levels, Npn-1 is expressed in motor neurons and sensory ganglia and loss of Sema3A-Npn-1 signaling leads to defasciculation of the superficial projections to the head and neck. The molecular mechanisms that govern the initial fasciculation and growth of the purely motor projections of the hypoglossal and abducens nerves in general, and the role of Npn-1 during these events in particular are, however, not well understood. We show here that selective removal of Npn-1 from somatic motor neurons impairs initial fasciculation and assembly of hypoglossal rootlets and leads to reduced numbers of abducens and hypoglossal fibers. Ablation of Npn-1 specifically from cranial neural crest and placodally derived sensory tissues recapitulates the distal defasciculation of mixed sensory-motor nerves of trigeminal, facial, glossopharyngeal and vagal projections, which was observed in Npn-1(-/-) and Npn-1(Sema-) mutants. Surprisingly, the assembly and fasciculation of the purely motor hypoglossal nerve are also impaired and the number of Schwann cells migrating along the defasciculated axonal projections is reduced. These findings are corroborated by partial genetic elimination of cranial neural crest and embryonic placodes, where loss of Schwann cell precursors leads to aberrant growth patterns of the hypoglossal nerve. Interestingly, rostral turning of hypoglossal axons is not perturbed in any of the investigated genotypes. Thus, initial hypoglossal nerve assembly and fasciculation, but not later guidance decisions depend on Npn-1 expression and axon-Schwann cell interactions.

Class 3 semaphorins influence oligodendrocyte precursor recruitment and remyelination in adult central nervous system.

Oligodendrocyte precursor cells, which persist in the adult central nervous system, are the main source of central nervous system remyelinating cells. In multiple sclerosis, some demyelinated plaques exhibit an oligodendroglial depopulation, raising the hypothesis of impaired oligodendrocyte precursor cell recruitment. Developmental studies identified semaphorins 3A and 3F as repulsive and attractive guidance cues for oligodendrocyte precursor cells, respectively. We previously reported their increased expression in experimental demyelination and in multiple sclerosis. Here, we show that adult oligodendrocyte precursor cells, like their embryonic counterparts, express class 3 semaphorin receptors, neuropilins and plexins and that neuropilin expression increases after demyelination. Using gain and loss of function experiments in an adult murine demyelination model, we demonstrate that semaphorin 3A impairs oligodendrocyte precursor cell recruitment to the demyelinated area. In contrast, semaphorin 3F overexpression accelerates not only oligodendrocyte precursor cell recruitment, but also remyelination rate. These data open new avenues to understand remyelination failure and promote repair in multiple sclerosis.

Adenylate Cyclase 1 modulates peripheral nerve branching patterns.

The Ca(2+)-stimulated adenylate cyclase 1 (AC1) is a key mediator of retinotopic map refinement and is required for the retraction response of retinal growth cones to the guidance cue ephrin-A5. We show here that AC1 is dynamically expressed in subpopulations of motor neurons in the spinal cord and sensory neurons of the dorsal root ganglia during development. AC1 was first detected around E12.5 in motoneurons of the medial aspect of the lateral motor column (LMCm) and the lateral region of the medial motor column (MMCl), which project to the ventral limb and body wall musculature, respectively. Expression levels gradually increased until they reached a maximum at a time when peripheral sensory and motor axons branch and establish connections with their targets. In barrelless mice, where a mutation inactivates the AC1 gene, sensory projections to the skin in the limbs and trunk region as well as innervations of the intercostal musculature provided by MMCl axons show increased branching. These results suggest a function of AC1 in the formation of peripheral nerve trajectories such as branching and pruning, after the initial projections have been laid down.

Semaphorin 3A-Neuropilin-1 signaling regulates peripheral axon fasciculation and pathfinding but not developmental cell death patterns.

In early development, an excess of neurons is generated, of which later about half will be lost by cell death due to a limited supply of trophic support by their respective target areas. However, some of the neurons die when their axons have not yet reached their target, thus suggesting that additional causes of developmental cell death exist. Semaphorin 3A (Sema3A), in addition to its function as a guidance cue and mediator of timing and fasciculation of motor and sensory axon outgrowth, can also induce death of sensory neurons in vitro. However, it is unknown whether Neuropilin-1 (Npn-1), its binding receptor in axon guidance, also mediates the death-inducing activity. We show here that abolished Sema3A-Npn-1 signaling does not influence the cell death patterns of motor or sensory neurons in mouse during the developmental wave of programmed cell death. The number of motor and sensory neurons was unchanged at embryonic day 15.5 when this wave is concluded. Interestingly, the defasciculation of early motor and sensory projections that is observed in the absence of Sema3A or Npn-1 persists to postnatal stages. Thus, Sema3A-Npn-1 signaling plays an important role in the guidance and fasciculation of motor and sensory axons but does not contribute to the developmental elimination of these neurons.

Distinct roles for secreted semaphorin signaling in spinal motor axon guidance.

Neuropilins, secreted semaphorin coreceptors, are expressed in discrete populations of spinal motor neurons, suggesting they provide critical guidance information for the establishment of functional motor circuitry. We show here that motor axon growth and guidance are impaired in the absence of Sema3A-Npn-1 signaling. Motor axons enter the limb precociously, showing that Sema3A controls the timing of motor axon in-growth to the limb. Lateral motor column (LMC) motor axons within spinal nerves are defasciculated as they grow toward the limb and converge in the plexus region. Medial and lateral LMC motor axons show dorso-ventral guidance defects in the forelimb. In contrast, Sema3F-Npn-2 signaling guides the axons of a medial subset of LMC neurons to the ventral limb, but plays no major role in regulating their fasciculation. Thus, Sema3A-Npn-1 and Sema3F-Npn-2 signaling control distinct steps of motor axon growth and guidance during the formation of spinal motor connections.

Cysteine-rich protein 2, a novel downstream effector of cGMP/cGMP-dependent protein kinase I-mediated persistent inflammatory pain.

The cGMP/cGMP-dependent protein kinase I (cGKI) signaling pathway plays an important role in spinal nociceptive processing. However, downstream targets of cGKI in this context have not been identified to date. Using a yeast two-hybrid screen, we isolated cysteine-rich protein 2 (CRP2) as a novel cGKI interactor in the spinal cord. CRP2 is expressed in laminas I and II of the mouse spinal cord and is colocalized with cGKI, calcitonin gene-related peptide, and isolectin B4. Moreover, the majority of CRP2 mRNA-positive dorsal root ganglion (DRG) neurons express cGKI and peripherin. CRP2 is phosphorylated in a cGMP-dependent manner, and its expression increases in the spinal cord and in DRGs after noxious stimulation of a hindpaw. To elucidate the functional role of CRP2 in nociception, we analyzed mice with a targeted deletion of CRP2. CRP2-deficient (CRP2-/-) mice demonstrate normal behavioral responses to acute nociception and after axonal injury of the sciatic nerve, but increased nociceptive behavior in models of inflammatory hyperalgesia compared with wild-type mice. Intrathecal administration of cGMP analogs increases the nociceptive behavior in wild-type but not in CRP2-/- mice, indicating that the presence of CRP2 is important for cGMP-mediated nociception. These data suggest that CRP2 is a new downstream effector of cGKI-mediated spinal nociceptive processing and point to an inhibitory role of CRP2 in the generation of inflammatory pain.

Pharmacologic properties of AG-012986, a pan-cyclin-dependent kinase inhibitor with antitumor efficacy.

AG-012986 is a multitargeted cyclin-dependent kinase (CDK) inhibitor active against CDK1, CDK2, CDK4/6, CDK5, and CDK9, with selectivity over a diverse panel of non-CDK kinases. Here, we report the potent antitumor efficacies of AG-012986 against multiple tumor lines in vitro and in vivo. AG-012986 showed antiproliferative activities in vitro with IC(50)s of <100 nmol/L in 14 of 18 tumor cell lines. In vivo, significant antitumor efficacy induced by AG-012986 was seen (tumor growth inhibition, >83.1%) in 10 of 11 human xenograft tumor models when administered at or near the maximum tolerated dose for 8 or 12 days. AG-012986 caused dose-dependent hypophosphorylation at Ser(795) of the retinoblastoma protein, cell cycle arrest, and apoptosis in vitro. Colony-forming assays indicated that the potency of AG-012986 substantially decreased with treatment time of <24 h. In vivo, AG-012986 also showed dose-dependent retinoblastoma Ser(795) hypophosphorylation, cell cycle arrest, decreased Ki-67 tumor staining, and apoptosis in conjunction with antitumor activity. Studies comparing i.p. bolus with s.c. implanted minipump dosing regimens revealed that in vivo efficacy correlated with the duration of minimally effective plasma levels rather than maximal drug plasma levels. Dosing optimization of AG-012986 provided guidance for selecting a treatment schedule to achieve the best antitumor efficacy while minimizing the risk of adverse side effects.

How axons see their way--axonal guidance in the visual system.

In humans up to 80% of the information received from the outside world is processed by the visual pathway. Therefore, understanding the molecular and cellular bases of the formation of the retinofugal projection has been in the focus of research during the last decades. Besides our interest in the development of the visual pathway per se this circuit is also an excellent model system to study axon guidance, midline crossing, and formation of topographic neuronal maps in general. The generation of genetic animal models as well as the design of in vitro loss- and gain-of-function paradigms have provided insight into transcriptional networks, identified signalling molecules, extracellular matrix components, morphogens, and activity patterns which are involved in the establishment of the visual pathway. To provide a picture as complete as possible, we will summarize molecular mechanisms involved in axon guidance and retinotopic mapping as well as neuronal activity shaping retinal and thalamocortical projections focusing on the mouse as a model system and highlight discoveries made in other organisms that contribute to our understanding.

TDP-43 induces p53-mediated cell death of cortical progenitors and immature neurons.

TAR DNA-binding protein 43 (TDP-43) is a key player in neurodegenerative diseases including frontotemporal lobar degeneration (FTLD) and amyotrophic lateral sclerosis (ALS). Accumulation of TDP-43 is associated with neuronal death in the brain. How increased and disease-causing mutant forms of TDP-43 induce cell death remains unclear. Here we addressed the role of TDP-43 during neural development and show that reduced TDP-43 causes defects in neural stem/progenitor cell proliferation but not cell death. However, overexpression of wild type and TDP-43A315T proteins induce p53-dependent apoptosis of neural stem/progenitors and human induced pluripotent cell (iPS)-derived immature cortical neurons. We show that TDP-43 induces expression of the proapoptotic BH3-only genes Bbc3 and Bax, and that p53 inhibition rescues TDP-43 induced cell death of embryonic mouse, and human cortical neurons, including those derived from TDP-43G298S ALS patient iPS cells. Hence, an increase in wild type and mutant TDP-43 induces p53-dependent cell death in neural progenitors developing neurons and this can be rescued. These findings may have important implications for accumulated or mutant TDP-43 induced neurodegenerative diseases.

Characterizing Semaphorin-Mediated Effects on Sensory and Motor Axon Pathfinding and Connectivity During Embryonic Development.

How are precise connectivity to peripheral targets and corresponding sensory-motor networks established during developmental innervation of the vertebrate extremities? The formation of functional sensory-motor circuits requires highly appropriate temporal and spatial regulation of axon growth which is achieved through the combination of different molecular mechanisms such as communication between heterotypic fiber systems, axon-environment, or axon-glia interactions that ensure proper fasciculation and accurate pathfinding to distal targets. Family members of the class 3 semaphorins and their cognate receptors, the neuropilins, were shown to govern various events during wiring of central and peripheral circuits, with mice lacking Sema3-Npn signaling showing deficits in timing of growth, selective fasciculation, guidance fidelity, and coupling of sensory axon growth to motor axons at developmental time points. Given the accuracy with which these processes have to interact in a stepwise manner, deficiency of the smallest cog in the wheel may impact severely on the faithful establishment and functionality of peripheral circuitries, ultimately leading to behavioral impairments or even cause the death of the animal. Reliable quantitative analyses of sensory-motor fasciculation, extension, and guidance of axons to their cognate target muscles and the skin during development, but also assessment of physiological and behavioral consequences at adult age, are therefore a necessity to extend our understanding of the molecular mechanisms of peripheral circuit formation. In this chapter we provide a detailed methodology to characterize class 3 semaphorin-mediated effects on peripheral sensory and motor axon pathfinding and connectivity during embryonic development.