A domain-agnostic approach for characterization of lifelong learning systems.

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

Intelligent Agents for the Optimization of Atomic Layer Deposition.

Atomic layer deposition (ALD) is a highly controllable thin film synthesis approach with applications in computing, energy, and separations. The flexibility of ALD means that it can access a massive chemical catalogue; however, this chemical and process diversity results in significant challenges in determining processing parameters that result in stable and uniform film growth with minimal precursor consumption. In situ measurements of the ALD growth per cycle (GPC) can accelerate process development but it still requires expert intuition and time-consuming trial and error to identify acceptable processing parameters. This procedure is made more difficult by the presence of experimental noise in the GPC values and the complexity of ALD surface chemistries. A need exists for efficient optimization approaches capable of autonomously determining processing conditions resulting in optimal ALD film growth. In this work, we present the development of three optimization strategies and compare their performance in optimizing four simulated ALD processes. Furthermore, the effect of noise in the GPC measurements on optimization convergence is studied.

Exploring Neuromodulation for Dynamic Learning.

A continual learning system requires the ability to dynamically adapt and generalize to new tasks with access to only a few samples. In the central nervous system, across species, it is observed that continual and dynamic behavior in learning is an active result of a mechanism known as neuromodulation. Therefore, in this work, neuromodulatory plasticity is embedded with dynamic learning architectures as a first step toward realizing power and area efficient few shot learning systems. An inbuilt modulatory unit regulates learning based on the context and internal state of the system. This renders the system an ability to self modify its weights. In one of the proposed architectures, ModNet, a modulatory layer is introduced in a random projection framework. ModNet's learning capabilities are enhanced by integrating attention along with compartmentalized plasticity mechanisms. Moreover, to explore modulatory mechanisms in conjunction with backpropagation in deeper networks, a modulatory trace learning rule is introduced. The proposed learning rule, uses a time dependent trace to modify the synaptic connections as a function of ongoing states and activations. The trace itself is updated via simple plasticity rules thus reducing the demand on resources. The proposed ModNet and learning rules demonstrate the ability to learn from few samples, train quickly, and perform few-shot image classification in a computationally efficient manner. The simple ModNet and the compartmentalized ModNet architecture learn benchmark image classification tasks in just 2 epochs. The network with modulatory trace achieves an average accuracy of 98.8%±1.16 on the omniglot dataset for five-way one-shot image classification task while requiring 20x fewer trainable parameters in comparison to other state of the art models.

Probing the Atomic-Scale Structure of Amorphous Aluminum Oxide Grown by Atomic Layer Deposition.

Atomic layer deposition (ALD) is a well-established technique for depositing nanoscale coatings with pristine control of film thickness and composition. The trimethylaluminum (TMA) and water (H2O) ALD chemistry is inarguably the most widely used and yet to date, we have little information about the atomic-scale structure of the amorphous aluminum oxide (AlOx) formed by this chemistry. This lack of understanding hinders our ability to establish process-structure-property relationships and ultimately limits technological advancements employing AlOx made via ALD. In this work, we employ synchrotron high-energy X-ray diffraction (HE-XRD) coupled with pair distribution function (PDF) analysis to characterize the atomic structure of amorphous AlOx ALD coatings. We combine ex situ and in operando HE-XRD measurements on ALD AlOx and fit these experimental data using stochastic structural modeling to reveal variations in the Al-O bond length, Al and O coordination environment, and extent of Al vacancies as a function of growth conditions. In particular, the local atomic structure of ALD AlOx is found to change with the substrate and number of ALD cycles. The observed trends are consistent with the formation of bulk Al2O3 surrounded by an O-rich surface layer. We deconvolute these data to reveal atomic-scale structural information for both the bulk and surface phases. Overall, this work demonstrates the usefulness of HE-XRD and PDF analysis in improving our understanding of the structure of amorphous ALD thin films and provides a pathway to evaluate how process changes impact the structure and properties of ALD films.

An Experimental and Numerical Analysis of the Compression of Bimetallic Cylinders.

This paper investigates the upsetting of bimetallic cylinders with an aluminum alloy center and a brass ring. The influence of the center-ring shape factor and type of assembly fit (interference and clearance), and the effect of friction on the compression force and ductile damage are comprehensively analyzed by means of a combined numerical-experimental approach. Results showed that the higher the shape factor, the lower the forces required, whereas the effect of friction is especially important for cylinders with the lowest shape factors. The type of assembly fit does not influence the compression force. The accumulated ductile damage in the compression of bimetallic cylinders is higher than in single-material cylinders, and the higher the shape factor, the lower the damage for the same amount of stroke. The highest values of damaged were found to occur at the middle plane, and typically in the ring. Results also showed that an interference fit was more favorable for preventing fracture of the ring than a clearance fit. Microstructural analysis by scanning electron microscopy revealed a good agreement with the finite element predicted distribution of ductile damage.

The chemical physics of sequential infiltration synthesis-A thermodynamic and kinetic perspective.

Sequential infiltration synthesis (SIS) is an emerging materials growth method by which inorganic metal oxides are nucleated and grown within the free volume of polymers in association with chemical functional groups in the polymer. SIS enables the growth of novel polymer-inorganic hybrid materials, porous inorganic materials, and spatially templated nanoscale devices of relevance to a host of technological applications. Although SIS borrows from the precursors and equipment of atomic layer deposition (ALD), the chemistry and physics of SIS differ in important ways. These differences arise from the permeable three-dimensional distribution of functional groups in polymers in SIS, which contrast to the typically impermeable two-dimensional distribution of active sites on solid surfaces in ALD. In SIS, metal-organic vapor-phase precursors dissolve and diffuse into polymers and interact with these functional groups through reversible complex formation and/or irreversible chemical reactions. In this perspective, we describe the thermodynamics and kinetics of SIS and attempt to disentangle the tightly coupled physical and chemical processes that underlie this method. We discuss the various experimental, computational, and theoretical efforts that provide insight into SIS mechanisms and identify approaches that may fill out current gaps in knowledge and expand the utilization of SIS.

Analysis of Favorable Process Conditions for the Manufacturing of Thin-Wall Pieces of Mild Steel Obtained by Wire and Arc Additive Manufacturing (WAAM).

One of the challenges in additive manufacturing (AM) of metallic materials is to obtain workpieces free of defects with excellent physical, mechanical, and metallurgical properties. In wire and arc additive manufacturing (WAAM) the influences of process conditions on thermal history, microstructure and resultant mechanical and surface properties of parts must be analyzed. In this work, 3D metallic parts of mild steel wire (American Welding Society-AWS ER70S-6) are built with a WAAM process by depositing layers of material on a substrate of a S235 JR steel sheet of 3 mm thickness under different process conditions, using as welding process the gas metal arc welding (GMAW) with cold metal transfer (CMT) technology, combined with a positioning system such as a computer numerical controlled (CNC) milling machine. Considering the hardness profiles, the estimated ultimate tensile strengths (UTS) derived from the hardness measurements and the microstructure findings, it can be concluded that the most favorable process conditions are the ones provided by CMT, with homogeneous hardness profiles, good mechanical strengths in accordance to conditions defined by standard, and without formation of a decohesionated external layer; CMT Continuous is the optimal option as the mechanical properties are better than single CMT.

Characterizing the field of Atomic Layer Deposition: Authors, topics, and collaborations.

This paper describes how Atomic Layer Deposition (ALD) has evolved over time using a combination of bibliometric, social network, and text analysis. We examined the rate of knowledge production as well as changes in authors, journals, and collaborators, showing a steady growth of ALD research. The study of the collaboration network of ALD scientists over time points out that the ALD research community is becoming larger and more interconnected, with a largest connected component that spans 90% of the authors in 2015. In addition, the evolution of network centrality measures (degree and betweenness centrality) and author productivity revealed the central figures in ALD over time, including new "stars" appearing in the last decade. Finally, the study of the title words in our dataset is consistent with a shift in focus on research topics towards energy applications and nanotechnology.

An instrument for in situ coherent x-ray studies of metal-organic vapor phase epitaxy of III-nitrides.

We describe an instrument that exploits the ongoing revolution in synchrotron sources, optics, and detectors to enable in situ studies of metal-organic vapor phase epitaxy (MOVPE) growth of III-nitride materials using coherent x-ray methods. The system includes high-resolution positioning of the sample and detector including full rotations, an x-ray transparent chamber wall for incident and diffracted beam access over a wide angular range, and minimal thermal sample motion, giving the sub-micron positional stability and reproducibility needed for coherent x-ray studies. The instrument enables surface x-ray photon correlation spectroscopy, microbeam diffraction, and coherent diffraction imaging of atomic-scale surface and film structure and dynamics during growth, to provide fundamental understanding of MOVPE processes.

A modular reactor design for in situ synchrotron x-ray investigation of atomic layer deposition processes.

Synchrotron characterization techniques provide some of the most powerful tools for the study of film structure and chemistry. The brilliance and tunability of the Advanced Photon Source allow access to scattering and spectroscopic techniques unavailable with in-house laboratory setups and provide the opportunity to probe various atomic layer deposition (ALD) processes in situ starting at the very first deposition cycle. Here, we present the design and implementation of a portable ALD instrument which possesses a modular reactor scheme that enables simple experimental switchover between various beamlines and characterization techniques. As first examples, we present in situ results for (1) X-ray surface scattering and reflectivity measurements of epitaxial ZnO ALD on sapphire, (2) grazing-incidence small angle scattering of MnO nucleation on silicon, and (3) grazing-incidence X-ray absorption spectroscopy of nucleation-regime Er2O3 ALD on amorphous ALD alumina and single crystalline sapphire.

New Insights into Sequential Infiltration Synthesis.

Sequential infiltration synthesis (SIS) is a process derived from ALD in which a polymer is infused with inorganic material using sequential, self-limiting exposures to gaseous precursors. SIS can be used in lithography to harden polymer resists rendering them more robust towards subsequent etching, and this permits deeper and higher-resolution patterning of substrates such as silicon. Herein we describe recent investigations of a model system: Al2O3 SIS using trimethyl aluminum (TMA) and H2O within the diblock copolymer, poly(styrene-block-methyl methacrylate) (PS-b-PMMA). Combining in-situ Fourier transform infrared absorption spectroscopy, quartz-crystal microbalance, and synchrotron grazing incidence small angle X-ray scattering with high resolution scanning transmission electron microscope tomography, we elucidate important details of the SIS process: 1) TMA adsorption in PMMA occurs through a weakly-bound intermediate; 2) the SIS kinetics are diffusion-limited, with desorption 10× slower than adsorption; 3) dynamic structural changes occur during the individual precursor exposures. These findings have important implications for applications such as SIS lithography.

Growth inhibition to enhance conformal coverage in thin film chemical vapor deposition.

We introduce the use of a growth inhibitor to enhance thin film conformality in low temperature chemical vapor deposition. Films of TiB(2) grown from the single source precursor Ti(BH(4))(3)(dme) are much more highly conformal when grown in the presence of one of the film growth byproducts, 1,2-dimethoxyethane (dme). This effect can be explained in terms of two alternative inhibitory mechanisms: one involving blocking of surface reactive sites, which is equivalent to reducing the rate of the forward reaction leading to film growth, the other analogous to Le Chatelier's principle, in which the addition of a reaction product increases the rate of the back reaction. The reduction in growth rate corresponds to a reduction in the sticking probability of the precursor, which enhances conformality by enabling the precursor to diffuse deeper into a recessed feature before it reacts.