DyCR: A Dynamic Clustering and Recovering Network for Few-Shot Class-Incremental Learning.

Few-shot class-incremental learning (FSCIL) aims to continually learn novel data with limited samples. One of the major challenges is the catastrophic forgetting problem of old knowledge while training the model on new data. To alleviate this problem, recent state-of-the-art methods adopt a well-trained static network with fixed parameters at incremental learning stages to maintain old knowledge. These methods suffer from the poor adaptation of the old model with new knowledge. In this work, a dynamic clustering and recovering network (DyCR) is proposed to tackle the adaptation problem and effectively mitigate the forgetting phenomena on FSCIL tasks. Unlike static FSCIL methods, the proposed DyCR network is dynamic and trainable during the incremental learning stages, which makes the network capable of learning new features and better adapting to novel data. To address the forgetting problem and improve the model performance, a novel orthogonal decomposition mechanism is developed to split the feature embeddings into context and category information. The context part is preserved and utilized to recover old class features in future incremental learning stages, which can mitigate the forgetting problem with a much smaller size of data than saving the raw exemplars. The category part is used to optimize the feature embedding space by moving different classes of samples far apart and squeezing the sample distances within the same classes during the training stage. Experiments show that the DyCR network outperforms existing methods on four benchmark datasets. The code is available at: https://github.com/zichengpan/DyCR.

Development of spirulina for the manufacture and oral delivery of protein therapeutics.

The use of the edible photosynthetic cyanobacterium Arthrospira platensis (spirulina) as a biomanufacturing platform has been limited by a lack of genetic tools. Here we report genetic engineering methods for stable, high-level expression of bioactive proteins in spirulina, including large-scale, indoor cultivation and downstream processing methods. Following targeted integration of exogenous genes into the spirulina chromosome (chr), encoded protein biopharmaceuticals can represent as much as 15% of total biomass, require no purification before oral delivery and are stable without refrigeration and protected during gastric transit when encapsulated within dry spirulina. Oral delivery of a spirulina-expressed antibody targeting campylobacter-a major cause of infant mortality in the developing world-prevents disease in mice, and a phase 1 clinical trial demonstrated safety for human administration. Spirulina provides an advantageous system for the manufacture of orally delivered therapeutic proteins by combining the safety of a food-based production host with the accessible genetic manipulation and high productivity of microbial platforms.

Robust Tensor Decomposition for Image Representation Based on Generalized Correntropy.

Traditional tensor decomposition methods, e.g., two dimensional principal component analysis and two dimensional singular value decomposition, that minimize mean square errors, are sensitive to outliers. To overcome this problem, in this paper we propose a new robust tensor decomposition method using generalized correntropy criterion (Corr-Tensor). A Lagrange multiplier method is used to effectively optimize the generalized correntropy objective function in an iterative manner. The Corr-Tensor can effectively improve the robustness of tensor decomposition with the existence of outliers without introducing any extra computational cost. Experimental results demonstrated that the proposed method significantly reduces the reconstruction error on face reconstruction and improves the accuracies on handwritten digit recognition and facial image clustering.

Conservation of trans-acting circuitry during mammalian regulatory evolution.

The basic body plan and major physiological axes have been highly conserved during mammalian evolution, yet only a small fraction of the human genome sequence appears to be subject to evolutionary constraint. To quantify cis- versus trans-acting contributions to mammalian regulatory evolution, we performed genomic DNase I footprinting of the mouse genome across 25 cell and tissue types, collectively defining ∼8.6 million transcription factor (TF) occupancy sites at nucleotide resolution. Here we show that mouse TF footprints conjointly encode a regulatory lexicon that is ∼95% similar with that derived from human TF footprints. However, only ∼20% of mouse TF footprints have human orthologues. Despite substantial turnover of the cis-regulatory landscape, nearly half of all pairwise regulatory interactions connecting mouse TF genes have been maintained in orthologous human cell types through evolutionary innovation of TF recognition sequences. Furthermore, the higher-level organization of mouse TF-to-TF connections into cellular network architectures is nearly identical with human. Our results indicate that evolutionary selection on mammalian gene regulation is targeted chiefly at the level of trans-regulatory circuitry, enabling and potentiating cis-regulatory plasticity.

A comparative encyclopedia of DNA elements in the mouse genome.

The laboratory mouse shares the majority of its protein-coding genes with humans, making it the premier model organism in biomedical research, yet the two mammals differ in significant ways. To gain greater insights into both shared and species-specific transcriptional and cellular regulatory programs in the mouse, the Mouse ENCODE Consortium has mapped transcription, DNase I hypersensitivity, transcription factor binding, chromatin modifications and replication domains throughout the mouse genome in diverse cell and tissue types. By comparing with the human genome, we not only confirm substantial conservation in the newly annotated potential functional sequences, but also find a large degree of divergence of sequences involved in transcriptional regulation, chromatin state and higher order chromatin organization. Our results illuminate the wide range of evolutionary forces acting on genes and their regulatory regions, and provide a general resource for research into mammalian biology and mechanisms of human diseases.

Mouse regulatory DNA landscapes reveal global principles of cis-regulatory evolution.

To study the evolutionary dynamics of regulatory DNA, we mapped >1.3 million deoxyribonuclease I-hypersensitive sites (DHSs) in 45 mouse cell and tissue types, and systematically compared these with human DHS maps from orthologous compartments. We found that the mouse and human genomes have undergone extensive cis-regulatory rewiring that combines branch-specific evolutionary innovation and loss with widespread repurposing of conserved DHSs to alternative cell fates, and that this process is mediated by turnover of transcription factor (TF) recognition elements. Despite pervasive evolutionary remodeling of the location and content of individual cis-regulatory regions, within orthologous mouse and human cell types the global fraction of regulatory DNA bases encoding recognition sites for each TF has been strictly conserved. Our findings provide new insights into the evolutionary forces shaping mammalian regulatory DNA landscapes.

Distributed dictionary learning for sparse representation in sensor networks.

This paper develops a distributed dictionary learning algorithm for sparse representation of the data distributed across nodes of sensor networks, where the sensitive or private data are stored or there is no fusion center or there exists a big data application. The main contributions of this paper are: 1) we decouple the combined dictionary atom update and nonzero coefficient revision procedure into two-stage operations to facilitate distributed computations, first updating the dictionary atom in terms of the eigenvalue decomposition of the sum of the residual (correlation) matrices across the nodes then implementing a local projection operation to obtain the related representation coefficients for each node; 2) we cast the aforementioned atom update problem as a set of decentralized optimization subproblems with consensus constraints. Then, we simplify the multiplier update for the symmetry undirected graphs in sensor networks and minimize the separable subproblems to attain the consistent estimates iteratively; and 3) dictionary atoms are typically constrained to be of unit norm in order to avoid the scaling ambiguity. We efficiently solve the resultant hidden convex subproblems by determining the optimal Lagrange multiplier. Some experiments are given to show that the proposed algorithm is an alternative distributed dictionary learning approach, and is suitable for the sensor network environment.

Robust ellipse fitting based on sparse combination of data points.

Ellipse fitting is widely applied in the fields of computer vision and automatic industry control, in which the procedure of ellipse fitting often follows the preprocessing step of edge detection in the original image. Therefore, the ellipse fitting method also depends on the accuracy of edge detection besides their own performance, especially due to the introduced outliers and edge point errors from edge detection which will cause severe performance degradation. In this paper, we develop a robust ellipse fitting method to alleviate the influence of outliers. The proposed algorithm solves ellipse parameters by linearly combining a subset of ("more accurate") data points (formed from edge points) rather than all data points (which contain possible outliers). In addition, considering that squaring the fitting residuals can magnify the contributions of these extreme data points, our algorithm replaces it with the absolute residuals to reduce this influence. Moreover, the norm of data point errors is bounded, and the worst case performance optimization is formed to be robust against data point errors. The resulting mixed l1-l2 optimization problem is further derived as a second-order cone programming one and solved by the computationally efficient interior-point methods. Note that the fitting approach developed in this paper specifically deals with the overdetermined system, whereas the current sparse representation theory is only applied to underdetermined systems. Therefore, the proposed algorithm can be looked upon as an extended application and development of the sparse representation theory. Some simulated and experimental examples are presented to illustrate the effectiveness of the proposed ellipse fitting approach.

An expansive human regulatory lexicon encoded in transcription factor footprints.

Regulatory factor binding to genomic DNA protects the underlying sequence from cleavage by DNase I, leaving nucleotide-resolution footprints. Using genomic DNase I footprinting across 41 diverse cell and tissue types, we detected 45 million transcription factor occupancy events within regulatory regions, representing differential binding to 8.4 million distinct short sequence elements. Here we show that this small genomic sequence compartment, roughly twice the size of the exome, encodes an expansive repertoire of conserved recognition sequences for DNA-binding proteins that nearly doubles the size of the human cis-regulatory lexicon. We find that genetic variants affecting allelic chromatin states are concentrated in footprints, and that these elements are preferentially sheltered from DNA methylation. High-resolution DNase I cleavage patterns mirror nucleotide-level evolutionary conservation and track the crystallographic topography of protein-DNA interfaces, indicating that transcription factor structure has been evolutionarily imprinted on the human genome sequence. We identify a stereotyped 50-base-pair footprint that precisely defines the site of transcript origination within thousands of human promoters. Finally, we describe a large collection of novel regulatory factor recognition motifs that are highly conserved in both sequence and function, and exhibit cell-selective occupancy patterns that closely parallel major regulators of development, differentiation and pluripotency.

Multiple nucleotide preferences determine cleavage-site recognition by the HIV-1 and M-MuLV RNases H.

The RNase H activity of reverse transcriptase is required during retroviral replication and represents a potential target in antiviral drug therapies. Sequence features flanking a cleavage site influence the three types of retroviral RNase H activity: internal, DNA 3'-end-directed, and RNA 5'-end-directed. Using the reverse transcriptases of HIV-1 (human immunodeficiency virus type 1) and Moloney murine leukemia virus (M-MuLV), we evaluated how individual base preferences at a cleavage site direct retroviral RNase H specificity. Strong test cleavage sites (designated as between nucleotide positions -1 and +1) for the HIV-1 and M-MuLV enzymes were introduced into model hybrid substrates designed to assay internal or DNA 3'-end-directed cleavage, and base substitutions were tested at specific nucleotide positions. For internal cleavage, positions +1, -2, -4, -5, -10, and -14 for HIV-1 and positions +1, -2, -6, and -7 for M-MuLV significantly affected RNase H cleavage efficiency, while positions -7 and -12 for HIV-1 and positions -4, -9, and -11 for M-MuLV had more modest effects. DNA 3'-end-directed cleavage was influenced substantially by positions +1, -2, -4, and -5 for HIV-1 and positions +1, -2, -6, and -7 for M-MuLV. Cleavage-site distance from the recessed end did not affect sequence preferences for M-MuLV reverse transcriptase. Based on the identified sequence preferences, a cleavage site recognized by both HIV-1 and M-MuLV enzymes was introduced into a sequence that was otherwise resistant to RNase H. The isolated RNase H domain of M-MuLV reverse transcriptase retained sequence preferences at positions +1 and -2 despite prolific cleavage in the absence of the polymerase domain. The sequence preferences of retroviral RNase H likely reflect structural features in the substrate that favor cleavage and represent a novel specificity determinant to consider in drug design.

Preferred sequences within a defined cleavage window specify DNA 3' end-directed cleavages by retroviral RNases H.

The RNase H activity of reverse transcriptase carries out three types of cleavage termed internal, RNA 5' end-directed, and DNA 3' end-directed. Given the strong association between the polymerase domain of reverse transcriptase and a DNA 3' primer terminus, we asked whether the distance from the primer terminus is paramount for positioning DNA 3' end-directed cleavages or whether preferred sequences and/or a cleavage window are important as they are for RNA 5' end-directed cleavages. Using the reverse transcriptases of human immunodeficiency virus, type 1 (HIV-1) and Moloney murine leukemia virus (M-MuLV), we determined the effects of sequence, distance, and substrate end structure on DNA 3' end-directed cleavages. Utilizing sequence-matched substrates, our analyses showed that DNA 3' end-directed cleavages share the same sequence preferences as RNA 5' end-directed cleavages, but the sites must fall in a narrow window between the 15th and 20th nucleotides from the recessed end for HIV-1 reverse transcriptase and between the 17th and 20th nucleotides for M-MuLV. Substrates with an RNA 5' end recessed by 1 (HIV-1) or 2-3 (M-MuLV) bases on a longer DNA could accommodate both types of end-directed cleavage, but further recession of the RNA 5' end excluded DNA 3' end-directed cleavages. For HIV-1 RNase H, the inclusion of the cognate dNTP enhanced DNA 3' end-directed cleavages at the 17th and 18th nucleotides. These data demonstrate that all three modes of retroviral RNase H cleavage share sequence determinants that may be useful in designing assays to identify inhibitors of retroviral RNases H.

Substitution of alanine for tyrosine-64 in the fingers subdomain of M-MuLV reverse transcriptase impairs strand displacement synthesis and blocks viral replication in vivo.

A distinctive property of reverse transcriptase is the ability to carry out strand displacement synthesis in the absence of accessory proteins such as helicases or single-strand DNA binding proteins. Structure-function studies indicate that the fingers subdomain in HIV-1 reverse transcriptase contacts the template strand downstream of the primer terminus and is involved in strand displacement synthesis. Based on structural comparisons to the HIV-1 enzyme, we made single amino acid substitutions at the Tyr-64 and Leu-99 positions in the fingers subdomain of the M-MuLV reverse transcriptase to ask whether this subdomain has a similar role in displacement synthesis. In vitro assays comparing non-displacement versus displacement synthesis revealed that substitution of alanine at Tyr-64 generated a reverse transcriptase that was impaired in its capacity to carry out DNA and RNA displacement synthesis without affecting polymerase processivity or RNase H activity. However, substitution of Tyr-64 with phenylalanine and a variety of substitutions at position Leu-99 had no specific effect on displacement synthesis. The Y64A substitution prevented viral replication in vivo, and Y64A virus generated reduced levels of reverse transcription intermediates at all steps beyond the synthesis of minus strong stop DNA. The role of the fingers subdomain and in particular the possible contributions of the Tyr-64 residue in displacement synthesis are discussed.

Sequence, distance, and accessibility are determinants of 5'-end-directed cleavages by retroviral RNases H.

The RNase H activity of reverse transcriptase is essential for retroviral replication. RNA 5'-end-directed cleavages represent a form of RNase H activity that is carried out on RNA/DNA hybrids that contain a recessed RNA 5'-end. Previously, the distance from the RNA 5'-end has been considered the primary determinant for the location of these cleavages. Employing model hybrid substrates and the HIV-1 and Moloney murine leukemia virus reverse transcriptases, we demonstrate that cleavage sites correlate with specific sequences and that the distance from the RNA 5'-end determines the extent of cleavage. An alignment of sequences flanking multiple RNA 5'-end-directed cleavage sites reveals that both enzymes strongly prefer A or U at the +1 position and C or G at the -2 position, and additionally for HIV-1, A is disfavored at the -4 position. For both enzymes, 5'-end-directed cleavages occurred when sites were positioned between the 13th and 20th nucleotides from the RNA 5'-end, a distance termed the cleavage window. In examining the importance of accessibility to the RNA 5'-end, it was found that the extent of 5'-end-directed cleavages observed in substrates containing a free recessed RNA 5'-end was most comparable to substrates with a gap of two or three bases between the upstream and downstream RNAs. Together these finding demonstrate that the selection of 5'-end-directed cleavage sites by retroviral RNases H results from a combination of nucleotide sequence, permissible distance, and accessibility to the RNA 5'-end.

Recognition of internal cleavage sites by retroviral RNases H.

The RNase H activity of reverse transcriptase is essential to complete retroviral replication. Many studies have characterized how reverse transcriptase associates with recessed and exposed DNA 3' ends or RNA 5' ends to position the RNase H domain for cleavage, but little is known about how a nick might affect RNase H cleavages, or how RNase H carries out internal cleavages, which do not require positioning by a nucleic acid end. We have addressed these issues using model hybrid substrates and the reverse transcriptases of Moloney murine leukemia virus (M-MuLV) and human immunodeficiency virus type 1 (HIV-1). Our results show that a nick separating an upstream RNA and a downstream RNA annealed to DNA is essentially ignored by RNase H, indicating that the RNA 5' end at a nick is not sufficient to position 5' end-directed cleavages. Cleavage sites that are located close to the 5' end of the downstream RNA are not recognized in the absence of the upstream RNA, and the 5' ends of the shorter upstream RNAs enhance cleavage at these sites. The recognition of an internal cleavage site depends on local sequence features found both upstream and downstream of the cleavage site, designated as the -1/+1 position. By analyzing the nucleotide frequencies in the sequence surrounding strong internal cleavage sites, preferred nucleotides have been identified in the flanking sequences spanning positions -14 to +1 for HIV-1 and -11 to +1 for M-MuLV. These data reveal that general degradation of the retroviral genome after minus-strand synthesis can occur through sequence-specific cleavages.

Specific cleavages by RNase H facilitate initiation of plus-strand RNA synthesis by Moloney murine leukemia virus.

Successful generation, extension, and removal of the plus-strand primer is integral to reverse transcription. For Moloney murine leukemia virus, primer removal at the RNA/DNA junction leaves the 3' terminus of the plus-strand primer abutting the downstream plus-strand DNA, but this 3' terminus is not efficiently reutilized for another round of extension. The RNase H cleavage to create the plus-strand primer might similarly result in the 3' terminus of this primer abutting downstream RNA, yet efficient initiation must occur to synthesize the plus-strand DNA. We hypothesized that displacement synthesis, RNase H activity, or both must participate to initiate plus-strand DNA synthesis. Using model hybrid substrates and RNase H-deficient reverse transcriptases, we found that displacement synthesis alone did not efficiently extend the plus-strand primer at a nick with downstream RNA. However, specific cleavage sites for RNase H were identified in the sequence immediately following the 3' end of the plus-strand primer. During generation of the plus-strand primer, cleavage at these sites generated a gap. When representative gaps separated the 3' terminus of the plus-strand primer from downstream RNA, primer extension significantly improved. The contribution of RNase H to the initiation of plus-strand DNA synthesis was confirmed by comparing the effects of downstream RNA versus DNA on plus-strand primer extension by wild-type reverse transcriptase. These data suggest a model in which efficient initiation of plus-strand synthesis requires the generation of a gap immediately following the plus-strand primer 3' terminus.

Evidence that DNase I hypersensitive site 5 of the human beta-globin locus control region functions as a chromosomal insulator in transgenic mice.

We have previously reported that DNase I hypersensitive site 5 (5'HS5) of the human beta-globin locus control region functions as a chromatin insulator in stable transfection assays. In this report we show that a 3.2 kb DNA fragment containing the entire 5'HS5 region can protect a position-sensitive (A)gamma-globin gene against position effects in transgenic mice. Bracketing is required for function of 5'HS5 as an insulator. The 5'HS5 insulator operates in adult as well as in embryonic murine erythroid cells. The insulator has no significant stimulatory effects of its own. These results indicate that 5'HS5 can function as a chromatin insulator in vivo.