Effect of image resolution on automated classification of chest X-rays.

Purpose: Deep learning (DL) models have received much attention lately for their ability to achieve expert-level performance on the accurate automated analysis of chest X-rays (CXRs). Recently available public CXR datasets include high resolution images, but state-of-the-art models are trained on reduced size images due to limitations on graphics processing unit memory and training time. As computing hardware continues to advance, it has become feasible to train deep convolutional neural networks on high-resolution images without sacrificing detail by downscaling. This study examines the effect of increased resolution on CXR classification performance. Approach: We used the publicly available MIMIC-CXR-JPG dataset, comprising 377,110 high resolution CXR images for this study. We applied image downscaling from native resolution to 2048×2048 pixels, 1024×1024 pixels, 512×512 pixels, and 256×256 pixels and then we used the DenseNet121 and EfficientNet-B4 DL models to evaluate clinical task performance using these four downscaled image resolutions. Results: We find that while some clinical findings are more reliably labeled using high resolutions, many other findings are actually labeled better using downscaled inputs. We qualitatively verify that tasks requiring a large receptive field are better suited to downscaled low resolution input images, by inspecting effective receptive fields and class activation maps of trained models. Finally, we show that stacking an ensemble across resolutions outperforms each individual learner at all input resolutions while providing interpretable scale weights, indicating that diverse information is extracted across resolutions. Conclusions: This study suggests that instead of focusing solely on the finest image resolutions, multi-scale features should be emphasized for information extraction from high-resolution CXRs.

The Role of SnO2 Processing on Ionic Distribution in Double-Cation-Double Halide Perovskites.

Moving toward a future of efficient, accessible, and less carbon-reliant energy devices has been at the forefront of energy research innovations for the past 30 years. Metal-halide perovskite (MHP) thin films have gained significant attention due to their flexibility of device applications and tunable capabilities for improving power conversion efficiency. Serving as a gateway to optimize device performance, consideration must be given to chemical synthesis processing techniques. Therefore, how does common substrate processing techniques influence the behavior of MHP phenomena such as ion migration and strain? Here, we demonstrate how a hybrid approach of chemical bath deposition (CBD) and nanoparticle SnO2 substrate processing significantly improves the performance of (FAPbI3)0.97(MAPbBr3)0.03 by reducing micro-strain in the SnO2 lattice, allowing distribution of K+ from K-Cl treatment of substrates to passivate defects formed at the interface and produce higher current in light and dark environments. X-ray diffraction reveals differences in lattice strain behavior with respect to SnO2 substrate processing methods. Through use of conductive atomic force microscopy (c-AFM), conductivity is measured spatially with MHP morphology, showing higher generation of current in both light and dark conditions for films with hybrid processing. Additionally, time-of-flight secondary ionization mass spectrometry (ToF-SIMS) observed the distribution of K+ at the perovskite/SnO2 interface, indicating K+ passivation of defects to improve the power conversion efficiency (PCE) and device stability. We show how understanding the role of ion distribution at the SnO2 and perovskite interface can help reduce the creating of defects and promote a more efficient MHP device.

Deep Learning on Multimodal Chemical and Whole Slide Imaging Data for Predicting Prostate Cancer Directly from Tissue Images.

Prostate cancer is one of the most common cancers globally and is the second most common cancer in the male population in the US. Here we develop a study based on correlating the hematoxylin and eosin (H&E)-stained biopsy data with MALDI mass-spectrometric imaging data of the corresponding tissue to determine the cancerous regions and their unique chemical signatures and variations of the predicted regions with original pathological annotations. We obtain features from high-resolution optical micrographs of whole slide H&E stained data through deep learning and spatially register them with mass spectrometry imaging (MSI) data to correlate the chemical signature with the tissue anatomy of the data. We then use the learned correlation to predict prostate cancer from observed H&E images using trained coregistered MSI data. This multimodal approach can predict cancerous regions with ∼80% accuracy, which indicates a correlation between optical H&E features and chemical information found in MSI. We show that such paired multimodal data can be used for training feature extraction networks on H&E data which bypasses the need to acquire expensive MSI data and eliminates the need for manual annotation saving valuable time. Two chemical biomarkers were also found to be predicting the ground truth cancerous regions. This study shows promise in generating improved patient treatment trajectories by predicting prostate cancer directly from readily available H&E-stained biopsy images aided by coregistered MSI data.

Dual-analgesic loaded UHMWPE exhibits synergistic antibacterial effects against Staphylococci.

Total joint arthroplasty is one of the most common surgeries in the United States, with almost a million procedures performed annually. Periprosthetic joint infections (PJI) remain the most devastating complications associated with total joint replacement. Effective antibacterial prophylaxis after primary arthroplasty could substantially reduce incidence rate of PJI. In the present study we propose to provide post-arthroplasty prophylaxis via dual-analgesic loaded ultra-high molecular weight polyethylene (UHMWPE). Our approach is based on previous studies that showed pronounced antibacterial activity of analgesic- and NSAID-loaded UHMWPE against Staphylococci. Here, we prepared bupivacaine/tolfenamic acid-loaded UHMWPE and assessed its antibacterial activity against Staphylococcus aureus and Staphylococcus epidermidis. Dual-drug loaded UHMWPE yielded an additional 1-2 log reduction of bacteria, when compared with single-drug loaded UHMWPE. Analysis of the drug elution kinetics suggested that the observed increase in antibacterial activity is due to the increased tolfenamic acid elution from dual-drug loaded UHMWPE. We showed that the increased fractal dimension of the drug domains in UHMWPE could be associated with increased drug elution, leading to higher antibacterial activity. Dual-analgesic loaded UHMWPE proposed here can be used as part of multi-modal antibacterial prophylaxis and promises substantial reduction in post-arthroplasty mortality and morbidity.

In with the old, in with the new: machine learning for time to event biomedical research.

The predictive modeling literature for biomedical applications is dominated by biostatistical methods for survival analysis, and more recently some out of the box machine learning approaches. In this article, we show a presentation of a machine learning method appropriate for time-to-event modeling in the area of prostate cancer long-term disease progression. Using XGBoost adapted to long-term disease progression, we developed a predictive model for 118 788 patients with localized prostate cancer at diagnosis from the Department of Veterans Affairs (VA). Our model accounted for patient censoring. Harrell's c-index for our model using only features available at the time of diagnosis was 0.757 95% confidence interval [0.756, 0.757]. Our results show that machine learning methods like XGBoost can be adapted to use accelerated failure time (AFT) with censoring to model long-term risk of disease progression. The long median survival justifies and requires censoring. Overall, we show that an existing machine learning approach can be used for AFT outcome modeling in prostate cancer, and more generally for other chronic diseases with long observation times.

Direct Multimodal Nanoscale Visualization of Early Phosphorus-Based Antiwear Tribofilm Formation.

Understanding the mechanism of antiwear (AW) tribofilm formation and how to tune surface chemistry to control functionality is essential for the development of the next generation of oil lubricants. In particular, understanding and optimizing early AW tribofilm formation can increase the energy efficiency of mechanical systems. However, the mechanism for how these films form is not well understood. The majority of prior work has focused on analyzing only end-of-test surfaces long after the film has formed. In this work, we develop an in situ multimodal chemical imaging methodology to directly visualize the early formation of AW films on steel surfaces. We investigate an oil formulation containing a phosphorus-based additive commonly used to protect surfaces from wear and fatigue processes in machine elements, such as gears, bearings, and sliding contacts. Using nanoscale multimodal chemical imaging on combined platforms of atomic force microscopy (AFM) coupled directly with in situ nano-infrared (nano-IR) spectroscopy, and further combined ex situ with time-of-flight secondary ion mass spectrometry (ToF-SIMS), we demonstrate a direct correlation between changes in friction and local surface chemistry. In these experiments, the AFM probe acts as a single asperity contact to generate the tribofilm as well as a tool to analyze it in situ as it is forming. To verify our in situ measurements, we compare these results to the ex situ ToF-SIMS of macroscale block-on-ring tribometer-formed samples. The understanding gained here on how AW films form and how film properties can be modified by tuning the chemistry of the additives will facilitate developing transmission fluids to meet increasing demands for vehicle performance and efficiency.

Highly enhanced ferroelectricity in HfO2-based ferroelectric thin film by light ion bombardment.

Continuous advancement in nonvolatile and morphotropic beyond-Moore electronic devices requires integration of ferroelectric and semiconductor materials. The emergence of hafnium oxide (HfO2)-based ferroelectrics that are compatible with atomic-layer deposition has opened interesting and promising avenues of research. However, the origins of ferroelectricity and pathways to controlling it in HfO2 are still mysterious. We demonstrate that local helium (He) implantation can activate ferroelectricity in these materials. The possible competing mechanisms, including He ion-induced molar volume changes, vacancy redistribution, vacancy generation, and activation of vacancy mobility, are analyzed. These findings both reveal the origins of ferroelectricity in this system and open pathways for nanoengineered binary ferroelectrics.

Correlated mechanochemical maps of Arabidopsis thaliana primary cell walls using atomic force microscope infrared spectroscopy.

Spatial heterogeneity in composition and organisation of the primary cell wall affects the mechanics of cellular morphogenesis. However, directly correlating cell wall composition, organisation and mechanics has been challenging. To overcome this barrier, we applied atomic force microscopy coupled with infrared (AFM-IR) spectroscopy to generate spatially correlated maps of chemical and mechanical properties for paraformaldehyde-fixed, intact Arabidopsis thaliana epidermal cell walls. AFM-IR spectra were deconvoluted by non-negative matrix factorisation (NMF) into a linear combination of IR spectral factors representing sets of chemical groups comprising different cell wall components. This approach enables quantification of chemical composition from IR spectral signatures and visualisation of chemical heterogeneity at nanometer resolution. Cross-correlation analysis of the spatial distribution of NMFs and mechanical properties suggests that the carbohydrate composition of cell wall junctions correlates with increased local stiffness. Together, our work establishes new methodology to use AFM-IR for the mechanochemical analysis of intact plant primary cell walls.

Microstructural Evaluation of Phase Instability in Large Bandgap Metal Halide Perovskites.

The optoelectronic performance of organic-inorganic halide perovskite (OIHP)-based devices has been improved in recent years. Particularly, solar cells fabricated using mixed-cations and mixed-halides have outperformed their single-cation and single-halide counterparts. Yet, a systematic evaluation of the microstructural behavior of mixed perovskites is missing despite their known composition-dependent photoinstability. Here, we explore microstructural inhomogeneity in (FAPbI3)x(MAPbBr3)1-x using advanced scanning probe microscopy techniques. Contact potential difference (CPD) maps measured by Kelvin probe force microscopy show an increased fraction of grains exhibiting a low CPD with flat topography as MAPbBr3 concentration is increased. The higher portion of low CPD contributes to asymmetric CPD distribution curves. Chemical analysis reveals these grains being rich in MA, Pb, and I. The composition-dependent phase segregation upon illumination, reflected on the emergence of a low-energy peak emission in the original photoluminescence spectra, arises from the formation of such grains with flat topology. Bias-dependent piezo-response force microscopy measurements, in these grains, further confirm vigorous ion migration and cause a hysteretic piezo-response. Our results, therefore, provide insights into the microstructural evaluation of phase segregation and ion migration in OIHPs pointing toward process optimization as a mean to further enhance their optoelectronic performance.

Role of Decomposition Product Ions in Hysteretic Behavior of Metal Halide Perovskite.

Ion migration is one of the most debated mechanisms and credited with multiple observed phenomena and performance in metal halide perovskites (MHPs) semiconductor devices. However, to date, the migration of ions and their effects on MHPs are not still fully understood, largely due to a lack of direct observations of temporal ion migration. In this work, using direct observation of ion migration in-operando, we observe the hysteretic migration behavior of intrinsic ions (i.e., CH3NH3+ and I-) as well as reveal the migration behavior of CH3NH3+ decomposition ions. We find that CH3NH3+ decomposition products can be affected by light and accumulate at the interfaces under bias. These MHP decomposition products are tightly related to the device performance and stability. Complementary results of time-resolved Kelvin probe force microscopy (tr-KPFM) demonstrate a correlation between dynamics of these interfacial ions and charge carriers. Overall, we find that there are a number of mobile ions including CH3NH3+ decomposition products in MHPs that need to be taken into account when measuring MHP device responses (e.g., charge dynamics) and should be considered in future optimization studies of MHP semiconductor devices.

Correlating Crystallographic Orientation and Ferroic Properties of Twin Domains in Metal Halide Perovskites.

Metal halide perovskite (MHP) solar cells have attracted worldwide research interest. Although it has been well established that grain, grain boundary, and grain facet affect MHPs optoelectronic properties, less is known about subgrain structures. Recently, MHP twin stripes, a subgrain feature, have stimulated extensive discussion due to the potential for both beneficial and detrimental effects of ferroelectricity on optoelectronic properties. Connecting the ferroic behavior of twin stripes in MHPs with crystal orientation will be a vital step to understand the ferroic nature and the effects of twin stripes. In this work, we studied the crystallographic orientation and ferroic properties of CH3NH3PbI3 twin stripes, using electron backscatter diffraction (EBSD) and advanced piezoresponse force microscopy (PFM), respectively. Using EBSD, we discovered that the orientation relationship across the twin walls in CH3NH3PbI3 is a 90° rotation about ⟨1̅1̅0⟩, with the ⟨030⟩ and ⟨111⟩ directions parallel to the direction normal to the surface. By careful inspection of CH3NH3PbI3 PFM results including in-plane and out-of-plane PFM measurements, we demonstrate some nonferroelectric contributions to the PFM responses of this CH3NH3PbI3 sample, suggesting that the PFM signal in this CH3NH3PbI3 sample is affected by nonferroelectric and nonpiezoelectric forces. If there is piezoelectric response, it is below the detection sensitivity of our interferometric displacement sensor PFM (<0.615 pm/V). Overall, this work offers an integrated picture describing the crystallographic orientations and the origin of PFM signal of MHPs twin stripes, which is critical to understanding the ferroicity in MHPs.

Enhancing hyperspectral EELS analysis of complex plasmonic nanostructures with pan-sharpening.

Nanoscale hyperspectral techniques-such as electron energy loss spectroscopy (EELS)-are critical to understand the optical response in plasmonic nanostructures, but as systems become increasingly complex, the required sampling density and acquisition times become prohibitive for instrumental and specimen stability. As a result, there has been a recent push for new experimental methodologies that can provide comprehensive information about a complex system, while significantly reducing the duration of the experiment. Here, we present a pan-sharpening approach to hyperspectral EELS analysis, where we acquire two datasets from the same region (one with high spatial resolution and one with high spectral fidelity) and combine them to achieve a single dataset with the beneficial properties of both. This work outlines a straightforward, reproducible pathway to reduced experiment times and higher signal-to-noise ratios, while retaining the relevant physical parameters of the plasmonic response, and is generally applicable to a wide range of spectroscopy modalities.

Nanomechanical sampling of material for nanoscale mass spectrometry chemical analysis.

The ability to spatially resolve the chemical distribution of compounds on a surface is important in many applications ranging from biological to material science. To this extent, we have recently introduced a hybrid atomic force microscopy (AFM)-mass spectrometry (MS) system for direct thermal desorption and pyrolysis of material with nanoscale chemical resolution. However, spatially resolved direct surface heating using local thermal desorption becomes challenging on material surfaces with low melting points, because the material will undergo a melting phase transition due to heat dissipation prior to onset of thermal desorption. Therefore, we developed an approach using mechanical sampling and collection of surface materials on an AFM cantilever probe tip for real-time analysis directly from the AFM tip. This approach allows for material to be concentrated directly onto the probe for subsequent MS analysis. We evaluate the performance metrics of the technique and demonstrate localized MS sampling from a candelilla wax matrix containing UV stabilizers avobenzone and oxinoxate from areas down to 250 nm × 250 nm. Overall, this approach removes heat dissipation into the bulk material allowing for a faster desorption and concentration of the gas phase analyte from a single heating pulse enabling higher signal levels from a given amount of material in a single sampling spot.Graphical abstract.

Nanoscale Mass Spectrometry Multimodal Imaging via Tip-Enhanced Photothermal Desorption.

Materials ranging from adhesives, pharmaceuticals, lubricants, and personal care products are traditionally studied using macroscopic characterization techniques. However, their functionality is in reality defined by details of chemical organization on often noncrystalline matter with characteristic length scales on the order of microns to nanometers. Additionally, these materials are traditionally difficult to analyze using standard vacuum-based approaches that provide nanoscale chemical characterization due to their volatile and beam-sensitive nature. Therefore, approaches that operate under ambient conditions need to be developed that allow probing of nanoscale chemical phenomena and correlated functionality. Here, we demonstrate a tool for probing and visualizing local chemical environments and correlating them to material structure and functionality using advanced multimodal chemical imaging on a combined atomic force microscopy (AFM) and mass spectrometry (MS) system using tip-enhanced photothermal desorption with atmospheric pressure chemical ionization (APCI). We demonstrate enhanced performance metrics of the technique for correlated imaging and point sampling and illustrate the applicability for the analysis of trace chemicals on a human hair, additives in adhesives on paper, and pharmaceuticals samples notoriously difficult to analyze in a vacuum environment. Overall, this approach of correlating local chemical environments to structure and functionality is key to advancing research in many fields ranging from biology, to medicine, to material science.

Hysteretic Ion Migration and Remanent Field in Metal Halide Perovskites.

The gap in understanding how underlying chemical dynamics impact the functionality of metal halide perovskites (MHPs) leads to the controversy about the origin of many phenomena associated with ion migration in MHPs. In particular, the debate regarding the impact of ion migration on current-voltage (I-V) hysteresis of MHPs devices has lasted for many years, where the difficulty lies in directly uncovering the chemical dynamics, as well as identifying and separating the impact of specific ions. In this work, using a newly developed time-resolved time-of-flight secondary ion mass spectrometry CH3NH3 + and I- migrations in CH3NH3PbI3 are directly observed, revealing hysteretic CH3NH3 + and I- migrations. Additionally, hysteretic CH3NH3 + migration is illumination-dependent. Correlating these results with the I-V characterization, this work uncovers that CH3NH3 + redistribution can induce a remanent field leading to a spontaneous current in the device. It unveils that the CH3NH3 + migration is responsible for the illumination-associated I-V hysteresis in MHPs. Hysteretic ion migration has not been uncovered and the contribution of any ions (e.g., CH3NH3 +) has not been specified before. Such insightful and detailed information has up to now been missing, which is critical to improving MHPs photovoltaic performance and developing MHPs-based memristors and synaptic devices.

Direct Write of 3D Nanoscale Mesh Objects with Platinum Precursor via Focused Helium Ion Beam Induced Deposition.

The next generation optical, electronic, biological, and sensing devices as well as platforms will inevitably extend their architecture into the 3rd dimension to enhance functionality. In focused ion beam induced deposition (FIBID), a helium gas field ion source can be used with an organometallic precursor gas to fabricate nanoscale structures in 3D with high-precision and smaller critical dimensions than focused electron beam induced deposition (FEBID), traditional liquid metal source FIBID, or other additive manufacturing technology. In this work, we report the effect of beam current, dwell time, and pixel pitch on the resultant segment and angle growth for nanoscale 3D mesh objects. We note subtle beam heating effects, which impact the segment angle and the feature size. Additionally, we investigate the competition of material deposition and sputtering during the 3D FIBID process, with helium ion microscopy experiments and Monte Carlo simulations. Our results show complex 3D mesh structures measuring ~300 nm in the largest dimension, with individual features as small as 16 nm at full width half maximum (FWHM). These assemblies can be completed in minutes, with the underlying fabrication technology compatible with existing lithographic techniques, suggesting a higher-throughput pathway to integrating FIBID with established nanofabrication techniques.

Two-Photon Up-Conversion Photoluminescence Realized through Spatially Extended Gap States in Quasi-2D Perovskite Films.

A new approach to generate a two-photon up-conversion photoluminescence (PL) by directly exciting the gap states with continuous-wave (CW) infrared photoexcitation in solution-processing quasi-2D perovskite films [(PEA)2 (MA)4 Pb5 Br16 with n = 5] is reported. Specifically, a visible PL peaked at 520 nm is observed with the quadratic power dependence by exciting the gap states with CW 980 nm laser excitation, indicating a two-photon up-conversion PL occurring in quasi-2D perovskite films. Decreasing the gap states by reducing the n value leads to a dramatic decrease in the two-photon up-conversion PL signal. This confirms that the gap states are indeed responsible for generating the two-photon up-conversion PL in quasi-2D perovskites. Furthermore, mechanical scratching indicates that the different-n-value nanoplates are essentially uniformly formed in the quasi-2D perovskite films toward generating multi-photon up-conversion light emission. More importantly, the two-photon up-conversion PL is found to be sensitive to an external magnetic field, indicating that the gap states are essentially formed as spatially extended states ready for multi-photon excitation. Polarization-dependent up-conversion PL studies reveal that the gap states experience the orbit-orbit interaction through Coulomb polarization to form spatially extended states toward developing multi-photon up-conversion light emission in quasi-2D perovskites.

Publisher Correction: Reply to: On the ferroelectricity of CH3NH3PbI3 perovskites.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.