Translational symmetry in convolutions with localized kernels causes an implicit bias toward high frequency adversarial examples.

Adversarial attacks are still a significant challenge for neural networks. Recent efforts have shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unknown. Inspired by theoretical work on linear convolutional models, we hypothesize that translational symmetry in convolutional operations together with localized kernels implicitly bias the learning of high-frequency features, and that this is one of the main causes of high frequency adversarial examples. To test this hypothesis, we analyzed the impact of different choices of linear and non-linear architectures on the implicit bias of the learned features and adversarial perturbations, in spatial and frequency domains. We find that, independently of the training dataset, convolutional operations have higher frequency adversarial attacks compared to other architectural parameterizations, and that this phenomenon is exacerbated with stronger locality of the kernel (kernel size) end depth of the model. The explanation for the kernel size dependence involves the Fourier Uncertainty Principle: a spatially-limited filter (local kernel in the space domain) cannot also be frequency-limited (local in the frequency domain). Using larger convolution kernel sizes or avoiding convolutions (e.g., by using Vision Transformers or MLP-style architectures) significantly reduces this high-frequency bias. Looking forward, our work strongly suggests that understanding and controlling the implicit bias of architectures will be essential for achieving adversarial robustness.

Estimating kidney function in patients with cancer: A narrative review.

AIM: Accurate evaluation of glomerular filtration rate (GFR) is crucial in Oncology as drug eligibility and dosing depend on estimates of GFR. However, there are no clear guidelines on the optimal method of determining kidney function in patients with cancer. We aimed to summarize the evidence on estimation of kidney function in patients with cancer. METHODS: We searched PubMed for literature discussing the performance of GFR estimating equations in patients with malignancy to create a table of the evidence for creatinine- and cystatin c-based equations. We further reviewed novel estimation techniques such as panel eGFR, real-time measured GFR, and functional magnetic resonance imaging. RESULTS: The commonly used GFR estimating equations were derived from populations of patients without cancer. These equations may be less applicable in Oncology due to severe sarcopenia, inflammation, and other physiologic changes in patients with cancer. The Cockcroft-Gault equation currently dominates in clinical Oncology despite significant limitations and accumulating evidence for use of the CKD-EPICr formula. Additional considerations in the practice of Oncology include a recently developed equation (CamGFRv2, also called the Janowitz formula) and the use of cystatin c-based equations to overcome some of the barriers to accurate GFR estimation based on creatinine alone. CONCLUSION: Overall, we suggest using the CKD-EPI equations (either cystatin c or creatinine-based) among patients with cancer in routine clinical practice and measured GFR for patients at a critical threshold for treatment decisions.

Robust deep learning object recognition models rely on low frequency information in natural images.

Machine learning models have difficulty generalizing to data outside of the distribution they were trained on. In particular, vision models are usually vulnerable to adversarial attacks or common corruptions, to which the human visual system is robust. Recent studies have found that regularizing machine learning models to favor brain-like representations can improve model robustness, but it is unclear why. We hypothesize that the increased model robustness is partly due to the low spatial frequency preference inherited from the neural representation. We tested this simple hypothesis with several frequency-oriented analyses, including the design and use of hybrid images to probe model frequency sensitivity directly. We also examined many other publicly available robust models that were trained on adversarial images or with data augmentation, and found that all these robust models showed a greater preference to low spatial frequency information. We show that preprocessing by blurring can serve as a defense mechanism against both adversarial attacks and common corruptions, further confirming our hypothesis and demonstrating the utility of low spatial frequency information in robust object recognition.

Decoding Depression Severity From Intracranial Neural Activity.

BACKGROUND: Disorders of mood and cognition are prevalent, disabling, and notoriously difficult to treat. Fueling this challenge in treatment is a significant gap in our understanding of their neurophysiological basis. METHODS: We recorded high-density neural activity from intracranial electrodes implanted in depression-relevant prefrontal cortical regions in 3 human subjects with severe depression. Neural recordings were labeled with depression severity scores across a wide dynamic range using an adaptive assessment that allowed sampling with a temporal frequency greater than that possible with typical rating scales. We modeled these data using regularized regression techniques with region selection to decode depression severity from the prefrontal recordings. RESULTS: Across prefrontal regions, we found that reduced depression severity is associated with decreased low-frequency neural activity and increased high-frequency activity. When constraining our model to decode using a single region, spectral changes in the anterior cingulate cortex best predicted depression severity in all 3 subjects. Relaxing this constraint revealed unique, individual-specific sets of spatiospectral features predictive of symptom severity, reflecting the heterogeneous nature of depression. CONCLUSIONS: The ability to decode depression severity from neural activity increases our fundamental understanding of how depression manifests in the human brain and provides a target neural signature for personalized neuromodulation therapies.

Human ureteric bud organoids recapitulate branching morphogenesis and differentiate into functional collecting duct cell types.

Directed differentiation of human pluripotent stem cells (hPSCs) into functional ureteric and collecting duct (CD) epithelia is essential to kidney regenerative medicine. Here we describe highly efficient, serum-free differentiation of hPSCs into ureteric bud (UB) organoids and functional CD cells. The hPSCs are first induced into pronephric progenitor cells at 90% efficiency and then aggregated into spheres with a molecular signature similar to the nephric duct. In a three-dimensional matrix, the spheres form UB organoids that exhibit branching morphogenesis similar to the fetal UB and correct distal tip localization of RET expression. Organoid-derived cells incorporate into the UB tips of the progenitor niche in chimeric fetal kidney explant culture. At later stages, the UB organoids differentiate into CD organoids, which contain >95% CD cell types as estimated by single-cell RNA sequencing. The CD epithelia demonstrate renal electrophysiologic functions, with ENaC-mediated vectorial sodium transport by principal cells and V-type ATPase proton pump activity by FOXI1-induced intercalated cells.

Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks.

Despite the enormous success of artificial neural networks (ANNs) in many disciplines, the characterization of their computations and the origin of key properties such as generalization and robustness remain open questions. Recent literature suggests that robust networks with good generalization properties tend to be biased toward processing low frequencies in images. To explore the frequency bias hypothesis further, we develop an algorithm that allows us to learn modulatory masks highlighting the essential input frequencies needed for preserving a trained network's performance. We achieve this by imposing invariance in the loss with respect to such modulations in the input frequencies. We first use our method to test the low-frequency preference hypothesis of adversarially trained or data-augmented networks. Our results suggest that adversarially robust networks indeed exhibit a low-frequency bias but we find this bias is also dependent on directions in frequency space. However, this is not necessarily true for other types of data augmentation. Our results also indicate that the essential frequencies in question are effectively the ones used to achieve generalization in the first place. Surprisingly, images seen through these modulatory masks are not recognizable and resemble texture-like patterns.

Shallow Univariate ReLU Networks as Splines: Initialization, Loss Surface, Hessian, and Gradient Flow Dynamics.

Understanding the learning dynamics and inductive bias of neural networks (NNs) is hindered by the opacity of the relationship between NN parameters and the function represented. Partially, this is due to symmetries inherent within the NN parameterization, allowing multiple different parameter settings to result in an identical output function, resulting in both an unclear relationship and redundant degrees of freedom. The NN parameterization is invariant under two symmetries: permutation of the neurons and a continuous family of transformations of the scale of weight and bias parameters. We propose taking a quotient with respect to the second symmetry group and reparametrizing ReLU NNs as continuous piecewise linear splines. Using this spline lens, we study learning dynamics in shallow univariate ReLU NNs, finding unexpected insights and explanations for several perplexing phenomena. We develop a surprisingly simple and transparent view of the structure of the loss surface, including its critical and fixed points, Hessian, and Hessian spectrum. We also show that standard weight initializations yield very flat initial functions, and that this flatness, together with overparametrization and the initial weight scale, is responsible for the strength and type of implicit regularization, consistent with previous work. Our implicit regularization results are complementary to recent work, showing that initialization scale critically controls implicit regularization via a kernel-based argument. Overall, removing the weight scale symmetry enables us to prove these results more simply and enables us to prove new results and gain new insights while offering a far more transparent and intuitive picture. Looking forward, our quotiented spline-based approach will extend naturally to the multivariate and deep settings, and alongside the kernel-based view, we believe it will play a foundational role in efforts to understand neural networks. Videos of learning dynamics using a spline-based visualization are available at http://shorturl.at/tFWZ2.

A novel automated junctional ectopic tachycardia detection tool for children with congenital heart disease.

Background: Junctional ectopic tachycardia (JET) is a prevalent life-threatening arrhythmia in children with congenital heart disease (CHD), with marked resemblance to normal sinus rhythm (NSR) often leading to delay in diagnosis. Objective: To develop a novel automated arrhythmia detection tool to identify JET. Methods: A single-center retrospective cohort study of children with CHD was performed. Electrocardiographic (ECG) data produced by bedside monitors is captured automatically by the Sickbay platform. Based on the detection of R and P wave peaks, 2 interpretable ECG features are calculated: P prominence median and PR interval interquartile range (IQR). These features are used as input to a simple logistic regression classification model built to distinguish JET from NSR. Results: This study analyzed a total of 64.5 physician-labeled hours consisting of 509,833 cardiac cycles (R-R intervals), from 40 patients with CHD. The extracted P prominence median feature is much smaller in JET compared to NSR, whereas the PR interval IQR feature is larger in JET compared to NSR. The area under the receiver operating characteristic curve for the unseen patient test cohort was 93%. Selecting a threshold of 0.73 results in a true-positive rate of 90% and a false-positive rate of 17%. Conclusion: This novel arrhythmia detection tool identifies JET, using 2 distinctive features of JET in ECG-the loss of a normal P wave and PR relationship-allowing for early detection and timely intervention.

Strategies to minimize heterogeneity and optimize clinical trials in Acute Respiratory Distress Syndrome (ARDS): Insights from mathematical modelling.

BACKGROUND: Mathematical modelling may aid in understanding the complex interactions between injury and immune response in critical illness. METHODS: We utilize a system biology model of COVID-19 to analyze the effect of altering baseline patient characteristics on the outcome of immunomodulatory therapies. We create example parameter sets meant to mimic diverse patient types. For each patient type, we define the optimal treatment, identify biologic programs responsible for clinical responses, and predict biomarkers of those programs. FINDINGS: Model states representing older and hyperinflamed patients respond better to immunomodulation than those representing obese and diabetic patients. The disparate clinical responses are driven by distinct biologic programs. Optimal treatment initiation time is determined by neutrophil recruitment, systemic cytokine expression, systemic microthrombosis and the renin-angiotensin system (RAS) in older patients, and by RAS, systemic microthrombosis and trans IL6 signalling for hyperinflamed patients. For older and hyperinflamed patients, IL6 modulating therapy is predicted to be optimal when initiated very early (<4th day of infection) and broad immunosuppression therapy (corticosteroids) is predicted to be optimally initiated later in the disease (7th - 9th day of infection). We show that markers of biologic programs identified by the model correspond to clinically identified markers of disease severity. INTERPRETATION: We demonstrate that modelling of COVID-19 pathobiology can suggest biomarkers that predict optimal response to a given immunomodulatory treatment. Mathematical modelling thus constitutes a novel adjunct to predictive enrichment and may aid in the reduction of heterogeneity in critical care trials. FUNDING: C.V. received a Marie Sklodowska Curie Actions Individual Fellowship (MSCA-IF-GF-2020-101028945). R.K.J.'s research is supported by R01-CA208205, and U01-CA 224348, R35-CA197743 and grants from the National Foundation for Cancer Research, Jane's Trust Foundation, Advanced Medical Research Foundation and Harvard Ludwig Cancer Center. No funder had a role in production or approval of this manuscript.

Quantification of Myxococcus xanthus Aggregation and Rippling Behaviors: Deep-Learning Transformation of Phase-Contrast into Fluorescence Microscopy Images.

Myxococcus xanthus bacteria are a model system for understanding pattern formation and collective cell behaviors. When starving, cells aggregate into fruiting bodies to form metabolically inert spores. During predation, cells self-organize into traveling cell-density waves termed ripples. Both phase-contrast and fluorescence microscopy are used to observe these patterns but each has its limitations. Phase-contrast images have higher contrast, but the resulting image intensities lose their correlation with cell density. The intensities of fluorescence microscopy images, on the other hand, are well-correlated with cell density, enabling better segmentation of aggregates and better visualization of streaming patterns in between aggregates; however, fluorescence microscopy requires the engineering of cells to express fluorescent proteins and can be phototoxic to cells. To combine the advantages of both imaging methodologies, we develop a generative adversarial network that converts phase-contrast into synthesized fluorescent images. By including an additional histogram-equalized output to the state-of-the-art pix2pixHD algorithm, our model generates accurate images of aggregates and streams, enabling the estimation of aggregate positions and sizes, but with small shifts of their boundaries. Further training on ripple patterns enables accurate estimation of the rippling wavelength. Our methods are thus applicable for many other phenotypic behaviors and pattern formation studies.

Privacy-Preserving Social Ambiance Measure From Free-Living Speech Associates With Chronic Depressive and Psychotic Disorders.

A social interaction consists of contributions by the individual, the environment and the interaction between the two. Ideally, to enable effective assessment and interventions for social isolation, an issue inherent to depressive and psychotic illnesses, the isolation must be identified in real-time and at an individual level. However, research addressing sociability deficits is largely focused on determining loneliness, rather than isolation, and lacks focus on the richness of the social environment the individual revolves in. In this paper, We describe the development of an automated, objective and privacy-preserving Social Ambiance Measure (SAM) that converts unconstrained audio recordings collected from wrist-worn audio-bands into four levels, ranging from none to active. The ambiance levels are based on the number of simultaneous speakers, which is a proxy for overall social activity in the environment. Results show that social ambiance patterns and time spent at each ambiance level differed between participants with depressive or psychotic disorders and healthy controls. Individuals with depression/psychosis spent less time in diverse environments and less time in moderate/active ambiance levels. Moreover, social ambiance patterns are found associated with the severity of self-reported depression, anxiety symptoms and personality traits. The results in this paper suggest that objectively measured social ambiance can be used as a marker of sociability, and holds potential to be leveraged to better understand social isolation and develop effective interventions for sociability challenges, thus improving mental health outcomes.

Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19.

As predicting the trajectory of COVID-19 is challenging, machine learning models could assist physicians in identifying high-risk individuals. This study compares the performance of 18 machine learning algorithms for predicting ICU admission and mortality among COVID-19 patients. Using COVID-19 patient data from the Mass General Brigham (MGB) Healthcare database, we developed and internally validated models using patients presenting to the Emergency Department (ED) between March-April 2020 (n = 3597) and further validated them using temporally distinct individuals who presented to the ED between May-August 2020 (n = 1711). We show that ensemble-based models perform better than other model types at predicting both 5-day ICU admission and 28-day mortality from COVID-19. CRP, LDH, and O2 saturation were important for ICU admission models whereas eGFR <60 ml/min/1.73 m2, and neutrophil and lymphocyte percentages were the most important variables for predicting mortality. Implementing such models could help in clinical decision-making for future infectious disease outbreaks including COVID-19.

DAPA-CKD: Significant Victory for CKD with or without Diabetes.

Sodium glucose cotransporter 2 inhibitors (SGLT2i) have revolutionized the treatment of heart failure and diabetic kidney disease. The DAPA-CKD trial by Heerspink et al. evaluated primary kidney outcomes with dapagliflozin in both diabetic and non-diabetic patients and identified kidney protection by SGLT2i independent of diabetes mellitus.

COVID-19: Antiviral agents and enzyme inhibitors/receptor blockers in development.

Novel 2019 coronavirus (severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) and coronavirus disease 2019 (COVID-19), the respiratory syndrome it causes, have shaken the world to its core by infecting and claiming the lives of many people since originating in December 2019 in Wuhan, China. World Health Organization and several states have declared a pandemic situation and state of emergency, respectively. As there is no treatment for COVID-19, several research institutes and pharmaceutical companies are racing to find a cure. Advances in computational approaches have allowed the screening of massive antiviral compound libraries to identify those that may potentially work against SARS-CoV-2. Antiviral agents developed in the past to combat other viruses are being repurposed. At the same time, new vaccine candidates are being developed and tested in preclinical/clinical settings. This review provides a detailed overview of select repurposed drugs, their mechanism of action, associated toxicities, and major clinical trials involving these agents.

Finerenone: A Non-steroidal Mineralocorticoid Receptor Blocker for Diabetic Kidney Disease.

Hemodynamic alterations, metabolic dysfunction, inflammation, and fibrosis are the main drivers of diabetic kidney disease (DKD) progression. Targeting inflammation and fibrosis as a driver, the FIDELIO-DKD (Effect of Finerenone on Chronic Kidney Disease Outcomes in Type 2 Diabetes) trial reported by Bakris et al. provides evidence of a new therapeutic class for treating DKD.

Domain-driven models yield better predictions at lower cost than reservoir computers in Lorenz systems.

Recent advances in computing algorithms and hardware have rekindled interest in developing high-accuracy, low-cost surrogate models for simulating physical systems. The idea is to replace expensive numerical integration of complex coupled partial differential equations at fine time scales performed on supercomputers, with machine-learned surrogates that efficiently and accurately forecast future system states using data sampled from the underlying system. One particularly popular technique being explored within the weather and climate modelling community is the echo state network (ESN), an attractive alternative to other well-known deep learning architectures. Using the classical Lorenz 63 system, and the three tier multi-scale Lorenz 96 system (Thornes T, Duben P, Palmer T. 2017 Q. J. R. Meteorol. Soc. 143, 897-908. (doi:10.1002/qj.2974)) as benchmarks, we realize that previously studied state-of-the-art ESNs operate in two distinct regimes, corresponding to low and high spectral radius (LSR/HSR) for the sparse, randomly generated, reservoir recurrence matrix. Using knowledge of the mathematical structure of the Lorenz systems along with systematic ablation and hyperparameter sensitivity analyses, we show that state-of-the-art LSR-ESNs reduce to a polynomial regression model which we call Domain-Driven Regularized Regression (D2R2). Interestingly, D2R2 is a generalization of the well-known SINDy algorithm (Brunton SL, Proctor JL, Kutz JN. 2016 Proc. Natl Acad. Sci. USA 113, 3932-3937. (doi:10.1073/pnas.1517384113)). We also show experimentally that LSR-ESNs (Chattopadhyay A, Hassanzadeh P, Subramanian D. 2019 (http://arxiv.org/abs/1906.08829)) outperform HSR ESNs (Pathak J, Hunt B, Girvan M, Lu Z, Ott E. 2018 Phys. Rev. Lett. 120, 024102. (doi:10.1103/PhysRevLett.120.024102)) while D2R2 dominates both approaches. A significant goal in constructing surrogates is to cope with barriers to scaling in weather prediction and simulation of dynamical systems that are imposed by time and energy consumption in supercomputers. Inexact computing has emerged as a novel approach to helping with scaling. In this paper, we evaluate the performance of three models (LSR-ESN, HSR-ESN and D2R2) by varying the precision or word size of the computation as our inexactness-controlling parameter. For precisions of 64, 32 and 16 bits, we show that, surprisingly, the least expensive D2R2 method yields the most robust results and the greatest savings compared to ESNs. Specifically, D2R2 achieves 68 × in computational savings, with an additional 2 × if precision reductions are also employed, outperforming ESN variants by a large margin. This article is part of the theme issue 'Machine learning for weather and climate modelling'.

In silico dynamics of COVID-19 phenotypes for optimizing clinical management.

Understanding the underlying mechanisms of COVID-19 progression and the impact of various pharmaceutical interventions is crucial for the clinical management of the disease. We developed a comprehensive mathematical framework based on the known mechanisms of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, incorporating the renin-angiotensin system and ACE2, which the virus exploits for cellular entry, key elements of the innate and adaptive immune responses, the role of inflammatory cytokines, and the coagulation cascade for thrombus formation. The model predicts the evolution of viral load, immune cells, cytokines, thrombosis, and oxygen saturation based on patient baseline condition and the presence of comorbidities. Model predictions were validated with clinical data from healthy people and COVID-19 patients, and the results were used to gain insight into identified risk factors of disease progression including older age; comorbidities such as obesity, diabetes, and hypertension; and dysregulated immune response. We then simulated treatment with various drug classes to identify optimal therapeutic protocols. We found that the outcome of any treatment depends on the sustained response rate of activated CD8+ T cells and sufficient control of the innate immune response. Furthermore, the best treatment-or combination of treatments-depends on the preinfection health status of the patient. Our mathematical framework provides important insight into SARS-CoV-2 pathogenesis and could be used as the basis for personalized, optimal management of COVID-19.