When to Switch: Planning and Learning for Partially Observable Multi-Agent Pathfinding.

Multi-agent pathfinding (MAPF) is a problem that involves finding a set of non-conflicting paths for a set of agents confined to a graph. In this work, we study a MAPF setting, where the environment is only partially observable for each agent, i.e., an agent observes the obstacles and other agents only within a limited field-of-view. Moreover, we assume that the agents do not communicate and do not share knowledge on their goals, intended actions, etc. The task is to construct a policy that maps the agent's observations to actions. Our contribution is multifold. First, we propose two novel policies for solving partially observable MAPF (PO-MAPF): one based on heuristic search and another one based on reinforcement learning (RL). Next, we introduce a mixed policy that is based on switching between the two. We suggest three different switch scenarios: the heuristic, the deterministic, and the learnable one. A thorough empirical evaluation of all the proposed policies in a variety of setups shows that the mixing policy demonstrates the best performance is able to generalize well to the unseen maps and problem instances, and, additionally, outperforms the state-of-the-art counterparts (Primal2 and PICO). The source-code is available at https://github.com/AIRI-Institute/when-to-switch.

Possible role of translation in unfertilized sea urchin eggs.

Unfertilized eggs of animals contain maternal messenger RNAs (mRNAs) and proteins, which are required for the maintenance of metabolism and regulation of development during the initial stages of embryogenesis. Unfertilized eggs are transcriptionally and translationally quiescent. After fertilization, activated translation of maternal mRNAs is one of the major forces that direct the early stages of embryogenesis before activation of the zygotic genome. However, a low rate and level of protein synthesis have been detected in unfertilized sea urchin eggs indicating that translation is not completely inhibited. Analysis of translatomes of unfertilized eggs and early embryos detected three sets of maternal mRNAs translated either before or after fertilization, or both before and after fertilization. Proteins encoded by maternal mRNAs, which are translated in unfertilized eggs, perform many different functions required for homeostasis, fertilization, egg activation, and early development. This suggests that translation in unfertilized sea urchin eggs may be required to renew the pool of proteins involved in these processes. Thus, translation may be necessary to maintain the fertility and developmental potential of sea urchin eggs during the long-term storage of eggs in ovaries until spawning begins.

Pathfinding in stochastic environments: learning vs planning.

Among the main challenges associated with navigating a mobile robot in complex environments are partial observability and stochasticity. This work proposes a stochastic formulation of the pathfinding problem, assuming that obstacles of arbitrary shapes may appear and disappear at random moments of time. Moreover, we consider the case when the environment is only partially observable for an agent. We study and evaluate two orthogonal approaches to tackle the problem of reaching the goal under such conditions: planning and learning. Within planning, an agent constantly re-plans and updates the path based on the history of the observations using a search-based planner. Within learning, an agent asynchronously learns to optimize a policy function using recurrent neural networks (we propose an original efficient, scalable approach). We carry on an extensive empirical evaluation of both approaches that show that the learning-based approach scales better to the increasing number of the unpredictably appearing/disappearing obstacles. At the same time, the planning-based one is preferable when the environment is close-to-the-deterministic (i.e., external disturbances are rare). Code available at https://github.com/Tviskaron/pathfinding-in-stochastic-envs.

An approach to quantitate maternal transcripts localized in sea urchin egg cortex using RT-qPCR with accurate normalization.

The sea urchin egg cortex is a peripheral region of eggs comprising a cell membrane and adjacent cytoplasm, which contains actin and tubulin cytoskeleton, cortical granules and some proteins required for early development. Method for isolation of cortices from sea urchin eggs and early embryos was developed in 1970s. Since then, this method has been reliable tool to study protein localization and cytoskeletal organization in cortex of unfertilized eggs and embryos during first cleavages. This study was aimed to estimate the reliability of RT-qPCR to analyze levels of maternal transcripts that are localized in egg cortex. Firstly, we selected seven potential reference genes, 28S, Cycb, Ebr1, GAPDH, Hmg1, Smtnl1 and Ubb, the transcripts of which are maternally deposited in sea urchin eggs. The candidate reference genes were ranked by five different algorithms (BestKeeper, CV, ΔCt, geNorm and NormFinder) based on calculated level of stability in both eggs as well as isolated cortices. Our results showed that gene ranking differs in total RNA and mRNA samples, though Ubb is most suitable reference gene in both cases. To validate feasibility of comparative analysis of eggs and isolated egg cortices, we selected Daglb-2 as a gene of interest, which transcripts are potentially localized in cortex according to transcriptome analysis, and observed increased level of Daglb-2 in egg cortices by RT-qPCR. This suggests that proposed RNA isolation method with subsequent quantitative RT-qPCR analysis can be used to determine cortical association of transcripts in sea urchin eggs.

DNA Conformational Changes Induced by Its Interaction with Binuclear Platinum Complexes in Solution Indicate the Molecular Mechanism of Platinum Binding.

Platinum anticancer drugs inhibit the division of cancer cells through a DNA binding mechanism. The bimetallic platinum compounds have a possibility for blocking DNA replication via the cross-linking of DNA functional groups at different distances. Many compounds with metals of the platinum group have been tested for possible antitumor activity. The main target of their biological action is a DNA molecule. A combined approach to the study of the interaction of DNA with biologically active compounds of this type is proposed. The capabilities of various methods (hydrodynamic, spectral, microscopy) in obtaining information on the type of binding of coordination compounds to DNA are compared. The analysis of DNA binding with platinum binuclear compounds containing pyrazine, tetrazole, 5- methyltetrazole, 3-propanediamine as bridging ligands in a solution was carried out with the methods of circular dichroism (CD), luminescent spectroscopy (LS), low gradient viscometry (LGV), flow birefringence (FB) and atomic force microscopy (AFM). The competitive binding of different platinum compounds to DNA and the analysis of platinum attachment to DNA after protonation of its nitrogen bases simply indicates the involvement of N7 guanine in binding. Fluorescent dye DAPI was also used to recognize the location of platinum compounds in DNA grooves. DNA conformational changes recorded by variations in persistent length, polyelectrolyte swelling, DNA secondary structure, and its stability clarify the molecular mechanism of the biological activity of platinum compounds.

Tetrodotoxins Secretion and Voltage-Gated Sodium Channel Adaptation in the Ribbon Worm Kulikovia alborostrata (Takakura, 1898) (Nemertea).

Nemertea is a phylum of marine worms whose members bear various toxins, including tetrodotoxin (TTX) and its analogues. Despite the more than 30 years of studying TTXs in nemerteans, many questions regarding their functions and the mechanisms ensuring their accumulation and usage remain unclear. In the nemertean Kulikovia alborostrata, we studied TTX and 5,6,11-trideoxyTTX concentrations in body extracts and in released mucus, as well as various aspects of the TTX-positive-cell excretion system and voltage-gated sodium (Nav1) channel subtype 1 mutations contributing to the toxins' accumulation. For TTX detection, an immunohistological study with an anti-TTX antibody and HPLC-MS/MS were conducted. For Nav1 mutation searching, PCR amplification with specific primers, followed by Sanger sequencing, was used. The investigation revealed that, in response to an external stimulus, subepidermal TTX-positive cells released secretions actively to the body surface. The post-release toxin recovery in these cells was low for TTX and high for 5,6,11-trideoxyTTX in captivity. According to the data obtained, there is low probability of the targeted usage of TTX as a repellent, and targeted 5,6,11-trideoxyTTX secretion by TTX-bearing nemerteans was suggested as a possibility. The Sanger sequencing revealed identical sequences of the P-loop regions of Nav1 domains I-IV in all 17 studied individuals. Mutations comprising amino acid substitutions, probably contributing to nemertean channel resistance to TTX, were shown.

The 3'UTR of the Drosophila CPEB translation factor gene orb2 plays a crucial role in spermatogenesis.

CPEB proteins are conserved translation regulators involved in multiple biological processes. One of these proteins in Drosophila, Orb2, is a principal player in spermatogenesis. It is required for meiosis and spermatid differentiation. During the later process, orb2 mRNA and protein are localized within the developing spermatid. To evaluate the role of the orb2 mRNA 3'UTR in spermatogenesis, we used the CRISPR/Cas9 system to generate a deletion of the orb2 3'UTR, orb2R. This deletion disrupts the process of spermatid differentiation but has no apparent effect on meiosis. Differentiation abnormalities include defects in the initial polarization of the 64-cell spermatid cysts, mislocalization of mRNAs and proteins in the elongating spermatid tails, altered morphology of the elongating spermatid tails, and defects in the assembly of the individualization complex. These disruptions in differentiation appear to arise because orb2 mRNA and protein are not properly localized within the 64-cell spermatid cyst.

Maternal control of early patterning in sea urchin embryos.

Sea urchin development has been studied extensively for more than a century and considered regulative since the first experimental evidence. Further investigations have repeatedly supported this standpoint by revealing the presence of inductive mechanisms that alter cell fate decisions at early cleavage stages and flexibility of development in response to environmental conditions. Some features indicate that sea urchin development is not completely regulative, but actually includes determinative events. In 16-cell embryos, mesomeres and macromeres represent multipotency, while the cell fate of most vegetal micromeres is restricted. It is known that the mature sea urchin eggs are polarized by the asymmetrical distribution of some maternal mRNAs and proteins. Spatially-distributed maternal factors are necessary for the orientation of the primary animal-vegetal axis, which is established by both maternal and zygotic mechanisms later in development. The secondary dorsal-ventral axis is conditionally specified later in development. Dorsal-ventral polarity is very liable during the early cleavages, though more recent data argue that its direction may be oriented by maternal asymmetry. In this review, we focus on the role of maternal factors in initial embryonic patterning during the first cleavages of sea urchin embryos before activation of the embryonic genome.

Germ plasm-related structures in marine medaka gametogenesis; novel sites of Vasa localization and the unique mechanism of germ plasm granule arising.

Germ plasm, a cytoplasmic factor of germline cell differentiation, is suggested to be a perspective tool for in vitro meiotic differentiation. To discriminate between the: (1) germ plasm-related structures (GPRS) involved in meiosis triggering; and (2) GPRS involved in the germ plasm storage phase, we investigated gametogenesis in the marine medaka Oryzias melastigma. The GPRS of the mitosis-to-meiosis period are similar in males and females. In both sexes, five events typically occur: (1) turning of the primary Vasa-positive germ plasm granules into the Vasa-positive intermitochondrial cement (IMC); (2) aggregation of some mitochondria by IMC followed by arising of mitochondrial clusters; (3) intramitochondrial localization of IMC-originated Vasa; followed by (4) mitochondrial cluster degradation; and (5) intranuclear localization of Vasa followed by this protein entering the nuclei (gonial cells) and synaptonemal complexes (zygotene-pachytene meiotic cells). In post-zygotene/pachytene gametogenesis, the GPRS are sex specific; the Vasa-positive chromatoid bodies are found during spermatogenesis, but oogenesis is characterized by secondary arising of Vasa-positive germ plasm granules followed by secondary formation and degradation of mitochondrial clusters. A complex type of germ plasm generation, 'the follicle cell assigned germ plasm formation', was found in late oogenesis. The mechanisms discovered are recommended to be taken into account for possible reconstruction of those under in vitro conditions.

Close association between vasa-positive germ plasm granules and mitochondria correlates with cytoplasmic localization of 12S and 16S mtrRNAs during zebrafish spermatogenesis.

The phenomenon of the cytoplasmic localisation of mitochondrial ribosomal subunits (12 S mitochondrial rRNA and 16 S mitochondrial rRNA) has been discovered by scientific teams working with spermatogenic cells of mice. Previous reports showed that the release of mitochondrial substance occurs during interaction of mitochondria with the germ plasm granules (GG). To determine if the interplay between the vasa-positive GG and the mitochondria is associated with cytoplasmic localisation of mtrRNAs, we studied the spermatogenic cells of zebrafish, Danio rerio. It was revealed that in type A undifferentiated spermatogonia the GG did not contact mitochondria, and the extra-mitochondrial localisation of the mtrRNAs was not found. In type A differentiated spermatogonia, the amount of GG in contact with mitochondria increased, but the extra-mitochondrial localisation of the mtrRNAs was not found either. In type B late spermatogonia, which are pre-meiotic cells, the GG/mitochondrion complexes were typically found in contact with the nucleus. This stage was associated with the intra-mitochondrial localisation of GG-originated vasa and extra-mitochondrial localisation of 12 S mtrRNA and 16 S mtrRNA. Until the onset of meiosis, which was determined by the observation of synaptonemal complexes in zygotene-pachytene spermatocytes I, the GG/mitochondrion complexes disappeared, but both types of mtrRNAs persisted in the cytoplasm of spermatids and spermatozoa.

The CPEB translational regulator, Orb, functions together with Par proteins to polarize the Drosophila oocyte.

orb is a founding member of the CPEB family of translational regulators and is required at multiple steps during Drosophila oogenesis. Previous studies showed that orb is required during mid-oogenesis for the translation of the posterior/germline determinant oskar mRNA and the dorsal-ventral determinant gurken mRNA. Here, we report that orb also functions upstream of these axes determinants in the polarization of the microtubule network (MT). Prior to oskar and gurken translational activation, the oocyte MT network is repolarized. The MT organizing center at the oocyte posterior is disassembled, and a new MT network is established at the oocyte anterior. Repolarization depends upon cross-regulatory interactions between anterior (apical) and posterior (basal) Par proteins. We show that repolarization of the oocyte also requires orb and that orb is needed for the proper functioning of the Par proteins. orb interacts genetically with aPKC and cdc42 and in egg chambers compromised for orb activity, Par-1 and aPKC protein and aPKC mRNA are mislocalized. Moreover, like cdc42-, the defects in Par protein localization appear to be connected to abnormalities in the cortical actin cytoskeleton. These abnormalities also disrupt the localization of the spectraplakin Shot and the microtubule minus-end binding protein Patronin. These two proteins play a critical role in the repolarization of the MT network.

Transcriptional quiescence in primordial germ cells.

In most animal species, newly formed primordial germ cells (PGCs) acquire the special characteristics that distinguish them from the surrounding somatic cells. Proper fate specification of the PGCs is coupled with transcriptional quiescence, whether they are segregated by determinative or inductive mechanisms. Inappropriate differentiation of PGCs into somatic cells is thought to be prevented due to repression of RNA polymerase (Pol) II-dependent transcription. In the case of a determinative mode of PGC formation (Drosophila, Caenorhabditis elegans, etc.), there is a broad downregulation of Pol II activity. By contrast, PGCs display only gene-specific repression in organisms that rely on inductive signaling-based mechanism (e.g., mice). In addition to the global block of Pol II activity in PGCs, gene expression can be suppressed in other ways, such as chromatin remodeling and Piwi-mediated RNAi. Here, we discuss the mechanisms responsible for the transcriptionally silent state of PGCs in common experimental animals, such as Drosophila, C. elegans, Danio rerio, Xenopus, and mouse. While a PGC-specific downregulation of transcription is a common feature among these organisms, the diverse nature of underlying mechanisms suggests that this functional trait likely evolved independently on several instances. We discuss the possible biological relevance of these silencing mechanisms vis-a-vis fate determination of PGCs.

The death pathways in mussel larval cells after a freeze-thaw cycle.

We analyzed cell viability, caspase activity, plasma membrane alterations and cell ultrastructure morphology to estimate the morphological and biochemical alterations that occur in bivalve molluscan cell cultures during cryopreservation. The use of 5% dymethyl sulfoxide as a cryoprotectant resulted in greater cell survival and a scarcity of destroyed cells lacking cytosol among dead cells. In this case, almost all cells died through necrosis or apoptosis, which appeared to increase in mussel cell cultures after a freeze-thaw cycle. Apoptosis was not a main death pathway in mussel cells, but it was induced in a significant part of these cells (up to 24%) immediately after thawing and depended mostly on the cryoprotectant used. Regardless of the type of the used cryoprotectant, we observed some nuclear aberrations in cells after freezing-thawing, such as few multipolar mitoses or the absence of a division spindle in mitotic cells. After analyzing different methods for assessing cell damage, the best results were obtained from optimal approaches that could provide information regarding the cell disruption level after freezing-thawing and could be considered for future studies.

The contribution of apoptosis and necrosis in freezing injury of sea urchin embryonic cells.

Sea urchins have recently been reported to be a promising tool for investigations of oxidative stress, UV light perturbations and senescence. However, few available data describe the pathway of cell death that occurs in sea urchin embryonic cells after cryopreservation. Our study is focused on the morphological and functional alterations that occur in cells of these animals during the induction of different cell death pathways in response to cold injury. To estimate the effect of cryopreservation on sea urchin cell cultures and identify the involved cell death pathways, we analyzed cell viability (via trypan blue exclusion test, MTT assay and DAPI staining), caspase activity (via flow cytometry and spectrophotometry), the level of apoptosis (via annexin V-FITC staining), and cell ultrastructure alterations (via transmission electron microscopy). Using general caspase detection, we found that the level of caspase activity was low in unfrozen control cells, whereas the number of apoptotic cells with activated caspases rose after freezing-thawing depending on cryoprotectants used, also as the number of dead cells and cells in a late apoptosis. The data using annexin V-binding assay revealed a very high apoptosis level in all tested samples, even in unfrozen cells (about 66%). Thus, annexin V assay appears to be unsuitable for sea urchin embryonic cells. Typical necrotic cells with damaged mitochondria were not detected after freezing in sea urchin cell cultures. Our results assume that physical cell disruption but not freezing-induced apoptosis or necrosis is the predominant reason of cell death in sea urchin cultures after freezing-thawing with any cryoprotectant combination.

Establishing and maintaining cell polarity with mRNA localization in Drosophila.

How cell polarity is established and maintained is an important question in diverse biological contexts. Molecular mechanisms used to localize polarity proteins to distinct domains are likely context-dependent and provide a feedback loop in order to maintain polarity. One such mechanism is the localized translation of mRNAs encoding polarity proteins, which will be the focus of this review and may play a more important role in the establishment and maintenance of polarity than is currently known. Localized translation of mRNAs encoding polarity proteins can be used to establish polarity in response to an external signal, and to maintain polarity by local production of polarity determinants. The importance of this mechanism is illustrated by recent findings, including orb2-dependent localized translation of aPKC mRNA at the apical end of elongating spermatid tails in the Drosophila testis, and the apical localization of stardust A mRNA in Drosophila follicle and embryonic epithelia.

Localization of germ plasm-related structures during sea urchin oogenesis.

BACKGROUND: Animal germ cells have specific organelles that are similar to ribonucleoprotein complex, called germ plasm, which is accumulated in eggs. Germ plasm is essential for inherited mechanism of germ line segregation in early embryogenesis. Sea urchins have early germ line segregation in early embryogenesis. Nevertheless, organization of germ plasm-related organelles and their molecular composition are still unclear. Another issue is whether maternally accumulated germ plasm exists in the sea urchin eggs. RESULTS: I analyzed intracellular localization of germ plasm during oogenesis in sea urchin Strongylocentrotus intermedius by using morphological approach and immunocytochemical detection of Vasa, a germ plasm marker. All ovarian germ cells have germ plasm-related organelles in the form of germ granules, Balbiani bodies, and perinuclear nuage found previously in germ cells in other animals. Maternal germ plasm is accumulated in late oogenesis at the cell periphery. Cytoskeletal drug treatment showed an association of Vasa-positive granules with actin filaments in the egg cortex. CONCLUSIONS: All female germ cells of sea urchins have germ plasm-related organelles. Eggs have a maternally accumulated germ plasm associated with cortical cytoskeleton. These findings correlate with early segregation of germ line in sea urchins.

Vascular endothelial growth factors: A comparison between invertebrates and vertebrates.

This review aims to summarize recent data concerning the structure and role of the members of the vascular endothelial growth factor (VEGF) and vascular endothelial growth factor receptor (VEGFR) families in the context of early development, organogenesis and regeneration, with a particular emphasis on the role of these factors in the development of invertebrates. Homologs of VEGF and/or VEGFR have been found in all Eumetazoa, in both Radiata and Bilateria, where they are expressed in the descendants of different germ layers and play a pivotal role in the development of animals with and without a vascular system. VEGF is a well-known angiogenesis regulator, but this factor also control cell migration during neurogenesis and the development of branching organs (the trachea) in invertebrate and vertebrate species. A possible explanation for the origin of Vegf/Vegfr in the animal kingdom and a pathway of Vegf/Vegfr evolution are discussed.

Expression pattern of vascular endothelial growth factor 2 during sea urchin development.

The VEGF family in the sea urchin is comprised of three members designated Vegf1 through Vegf3. In this study, we found a high level of similarity between the PDGF/VEGF domain of the predicted gene Sp-Vegf2 in the sea urchin Strongylocentrotus purpuratus and the same domain of a gene that we found in a closely related sea urchin, Strongylocentrotus intermedius. The sequence of the Si-Vegf2 cDNA was determined, and the expression of the Si-Vegf2 mRNA throughout early sea urchin development was studied by RT-PCR and in situ hybridization. Also we analyzed phylogenetic relationships of Si-Vegf2 and other members of the PDGF and VEGF families. We have found that the Si-Vegf2 present during the time span from the egg to the 4-arm pluteus stage. This mRNA is uniformly distributed in eggs, cleaving embryos and early blastulae. At the gastrula stage, the Si-Vegf2 transcripts are localized in the ventrolateral clusters of primary mesenchyme cells, and later, at the prism stage, they are detected in the forming apex. At the early pluteus stage, Si-Vegf2 mRNAs are found in two groups of mesenchyme cells in the scheitel region on the apical pole. We have determined that Si-Vegf2 is a mesenchyme-expressed factor but its developmental function is unknown.

Isolation of oogonia from ovaries of the sea urchin Strongylocentrotus nudus.

The presence of oogonia in the ovaries of adult females is typical in species with a broadcast spawning reproductive strategy, including invertebrates and lower vertebrates. In sea urchins, difficulties in the study of oogonia arise from the small number of these cells and the lack of specific markers for their identification. Therefore, more reliable methods are needed for identifying and manipulating oogonial cells in quantities sufficient for experimentation. Homologs of the DEAD-box RNA helicase vasa expressed in germline cells have been proposed for use as markers to detect germline cells in diverse species. We have developed a method for the isolation of sea urchin oogonia by using immunocytochemistry with vasa antibodies, together with reverse transcription and the polymerase chain reaction to detect the expression of Sp-vasa and Sp-nanos2 homologs and a morphological approach to identify germline cells in sea urchin ovaries and cell fractions isolated from the ovarian germinal epithelium. This method has allowed us to obtain 15%-18% of small oogonia with 70%-75% purity from the total amount of isolated germ cells. Our findings represent the first methodological basis for obtaining cell populations containing sea urchin oogonia; this method might be useful as a tool for further investigations of the early stages of sea urchin oogenesis.

Agrobacterium-mediated transformation of sea urchin embryos.

Agrobacterium-mediated transformation of higher plants is a well-known and powerful tool for transgene delivery to plant cells. In the present work, we studied whether Agrobacterium can transfer genetic information to animal (sea urchin) embryos. Sea urchin embryos were co-cultivated with A. tumefaciens strains carrying binary vectors containing the nptII marker gene and agrobacterial rolC and rolB oncogenes. Bacterial plasmid T-DNA-sea urchin DNA junction sites were identified in the genome of these embryos, thus indicating successful transformation. The nptII and both rol genes were expressed in the transformed embryos. The processes of transgene integration and transgene expression were suppressed when Agrobacteria contained mutated virA, virB or virG genes, suggesting that Agrobacterium transforms sea urchin cells by a mechanism similar to that which mediates T-DNA transfer to plants. Some of the embryos co-cultivated with Agrobacterium developed teratoma-like structures. The ability of Agrobacterium strains to trigger formation of teratoma-like structures was diminished when they contained the mutated vir genes. In summary, our results demonstrate that Agrobacterium is able to transform animal (sea urchin) embryonic cells, thus indicating a potential of this natural system for gene delivery to animal hosts. We also discuss the possibility of horizontal gene transfer from Agrobacterium to marine invertebrates.