Explaining neural scaling laws.

The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in the network. We propose a theory that explains the origins of and connects these scaling laws. We identify variance-limited and resolution-limited scaling behavior for both dataset and model size, for a total of four scaling regimes. The variance-limited scaling follows simply from the existence of a well-behaved infinite data or infinite width limit, while the resolution-limited regime can be explained by positing that models are effectively resolving a smooth data manifold. In the large width limit, this can be equivalently obtained from the spectrum of certain kernels, and we present evidence that large width and large dataset resolution-limited scaling exponents are related by a duality. We exhibit all four scaling regimes in the controlled setting of large random feature and pretrained models and test the predictions empirically on a range of standard architectures and datasets. We also observe several empirical relationships between datasets and scaling exponents under modifications of task and architecture aspect ratio. Our work provides a taxonomy for classifying different scaling regimes, underscores that there can be different mechanisms driving improvements in loss, and lends insight into the microscopic origin and relationships between scaling exponents.

Assessment of a Novel Position Verification Sensor to Identify and Isolate Robot Workcell Health Degradation.

Manufacturing processes have become increasingly sophisticated leading to greater usage of robotics. Sustaining successful manufacturing robotic operations requires a strategic maintenance program. Without careful planning, maintenance can be very costly. To reduce maintenance costs, manufacturers are exploring how they can assess the health of their robot workcell operations to enhance their maintenance strategies. Effective health assessment relies upon capturing appropriate data and generating intelligence from the workcell. Multiple data streams relevant to a robot workcell may be available including robot controller data, a supervisory programmable logic controller data, maintenance logs, process and part quality data, and equipment and process fault and failure data. These data streams can be extremely informative, yet the massive volume and complexity of this data can be overwhelming, confusing, and sometimes paralyzing. Researchers at the National Institute of Standards and Technology have developed a test method and companion sensor to assess the health of robot workcells which will yield an additional and unique data stream. The intent is that this data stream can either serve as a surrogate for larger data volumes to reduce the data collection and analysis burden on the manufacturer, or add more intelligence to assessing robot workcell health. This article presents the most recent effort focused on verifying the companion sensor. Results of the verification test process are discussed along with preliminary results of the sensor's performance during verification testing. Lessons learned indicate that the test process can be an effective means of quantifying the sensor's measurement capability particularly after test process anomalies are addressed in future efforts.

Altered Expression of Several Molecular Mediators of Cerebrospinal Fluid Production in Hyp Mice.

Context: X-linked hypophosphatemia (XLH) is a genetic disease, causing life-long hypophosphatemia due to overproduction of fibroblast growth factor 23 (FGF23). XLH is associated with Chiari malformations, cranial synostosis, and syringomyelia. FGF23 signals through FGFR1c and requires a coreceptor, α-Klotho, which is expressed in the renal distal convoluted tubules and the choroid plexus (ChP). In the ChP, α-Klotho participates in regulating cerebrospinal fluid (CSF) production by shuttling the sodium/potassium adenosine triphosphatase (Na+/K+-ATPase) to the luminal membrane. The sodium/potassium/chloride cotransporter 1 (NKCC1) also makes a substantial contribution to CSF production. Objective: Since CSF production has not been studied in XLH, we sought to determine if there are changes in the expression of these molecules in the ChP of Hyp mice, the murine model of XLH, as a first step toward testing the hypothesis that altered CSF production contributes to the cranial and spinal malformations seen this disease. Methods: Semi-quantitative real-time PCR was used to analyze the level of expression of transcripts for Fgfr1c, and thee key regulators of CSF production, Klotho, Atp1a1 and Slc12a2. In situ hybridization was used to provide anatomical localization for the encoded proteins. Results: Real-time polymerase chain reaction (RT-PCR) demonstrated significant upregulation of Klotho transcripts in the fourth ventricle of Hyp mice compared to controls. Transcript levels for Fgfr1c were unchanged in Hyp mice. Atp1a1 transcripts encoding the alpha-1 subunit of Na+/K+-ATPase were significantly downregulated in the third and lateral ventricles (LV). Expression levels of the Slc12a2 transcript (which encodes NKCC1) were unchanged in Hyp mice compared to controls. In situ hybridization (ISH) confirmed the presence of all 4 transcripts in the LV ChP both of WT and Hyp mice. Conclusion: This is the first study to document a significant change in the level of expression of the molecular machinery required for CSF production in Hyp mice. Whether similar changes occur in patients with XLH, potentially contributing to the cranial and spinal cord abnormalities frequently seen in XLH, remains to be determined.

Selective deletion of the receptor for CSF1, c-fms, in osteoclasts results in a high bone mass phenotype, smaller osteoclasts in vivo and an impaired response to an anabolic PTH regimen.

The receptor for Colony Stimulating Factor 1 (CSF1), c-fms, is highly expressed on mature osteoclasts suggesting a role for this cytokine in regulating the function of these cells. Consistent with this idea, in vitro studies have documented a variety of effects of CSF1 in mature osteoclasts. To better define the role of CSF1 in these cells, we conditionally deleted c-fms in osteoclasts (c-fms-OC-/-) by crossing c-fmsflox/flox mice with mice expressing Cre under the control of the cathepsin K promoter. The c-fms-OC-/- mice were of normal weight and had normal tooth eruption. However, when quantified by DXA, bone mass was significantly higher in the spine and femur of female knock out mice and in the femurs of male knock out mice. MicroCT analyses of femurs showed that female c-fms-OC-/- mice had significantly increased trabecular bone mass with a similar trend in males and both sexes demonstrated significantly increased trabecular number and reduced trabecular spacing. Histomorphometric analysis of the femoral trabecular bone compartment demonstrated a trend towards increased numbers of osteoclasts, +26% in Noc/BPm and +22% in OcS/BS in the k/o animals but this change was not significant. However, when the cellular volume of osteoclasts was quantified, the c-fms-OC-/- cells were found to be significantly smaller than controls. Mature osteoclasts show a marked spreading response when exposed to CSF1 in a non-gradient fashion. However, osteoclasts freshly isolated from c-fms-OC-/- mice had a near complete abrogation of this response. C-fms-OC-/- mice treated with (1-34)hPTH 80 ng/kg/d in single daily subcutaneous doses for 29 days showed an attenuated anabolic response in trabecular bone compared to wild-type animals. Taken together, these data indicate an important non-redundant role for c-fms in regulating mature osteoclast function in vivo.

Intratumoral temozolomide in spontaneous canine gliomas: feasibility of a novel therapy using implanted microcylinders.

Entotherapy an image-guided drug-eluting microcylinder platform, has the potential to bypass the limitations of systemic chemotherapy use in the treatment of canine brain tumours. Gliomas, which are common in dogs and also represent the majority of fatal brain tumours in humans, can be amenable to chemotherapy with temozolomide. Biopolymer microcylinders conjugated with temozolomide and gadolinium were implanted into partially resected tumours of four client-owned dogs with gliomas. All four dogs presented with generalized seizures and had mild to no neurologic deficits at the time of craniotomy. All dogs underwent craniotomy for implantation of the microcylinders into partially resected gliomas (glioblastoma multiforme {n = 1} or oligodendroglioma {n = 3}). All dogs recovered well from the craniotomy and implantation procedure. This novel procedure appears to be feasible and tolerated in tumour-bearing dogs. A future controlled clinical study can now aim to evaluate the microcylinder implantation for long-term efficacy.

Clinical and genetic analysis in a large Chinese cohort of patients with X-linked hypophosphatemia.

X-linked Hypophosphatemia (XLH) is caused by loss of function mutations in the PHEX gene. Given the recent availability of a new therapy for XLH, a retrospective analysis of the most recent 261 Chinese patients with XLH evaluated at Peking Union Medical College Hospital was conducted. Clinical, biochemical, radiographic studies, as well as genetic analyses, including Sanger sequencing for point mutations and Multiplex Ligation-dependent Probe Amplification (MLPA) to detect large deletions/duplications were employed. Based on the structure of Neprilysin (NEP), a member of M13 family that includes PHEX, a three-dimensional (3D) model of PHEX was constructed, missense and nonsense mutations were positioned on the predicted structure to visualize relative positions of these two types of variants. Sex differences and genotype-phenotype correlations were also undertaken. Genetic analyses identified 166 PHEX mutations in 261 XLH patients. One hundred and eleven of the 166 mutations were unreported. Four mutational 'hot-spots' were identified in this cohort (P534L, G579R, R747X, c.1645+1 G>A). Missense mutations, but not nonsense mutations, clustered in the two putative lobes of the PHEX protein, suggesting these are functionally important regions of the molecule. Circulating levels of intact FGF23 were significantly elevated (median level 101.9 pg/mL; reference range 16.1-42.2 pg/mL). No significant sex differences, as well as no phenotypic differences were identified between patients with putative truncating and non-truncating PHEX mutations. However, patients with N-terminal PHEX mutations had an earlier age of onset of disease (P = 0.015) and higher iFGF23 levels (P = 0.045) as compared to those with C-terminal mutations. These data provide a comprehensive characterization of the largest cohort of patients with XLH reported to date from China, which will help in evaluating the applicability of emerging therapies for this disease in this ethnic group.