Sample-efficient multi-agent reinforcement learning with masked reconstruction.

Deep reinforcement learning (DRL) is a powerful approach that combines reinforcement learning (RL) and deep learning to address complex decision-making problems in high-dimensional environments. Although DRL has been remarkably successful, its low sample efficiency necessitates extensive training times and large amounts of data to learn optimal policies. These limitations are more pronounced in the context of multi-agent reinforcement learning (MARL). To address these limitations, various studies have been conducted to improve DRL. In this study, we propose an approach that combines a masked reconstruction task with QMIX (M-QMIX). By introducing a masked reconstruction task as an auxiliary task, we aim to achieve enhanced sample efficiency-a fundamental limitation of RL in multi-agent systems. Experiments were conducted using the StarCraft II micromanagement benchmark to validate the effectiveness of the proposed method. We used 11 scenarios comprising five easy, three hard, and three very hard scenarios. We particularly focused on using a limited number of time steps for each scenario to demonstrate the improved sample efficiency. Compared to QMIX, the proposed method is superior in eight of the 11 scenarios. These results provide strong evidence that the proposed method is more sample-efficient than QMIX, demonstrating that it effectively addresses the limitations of DRL in multi-agent systems.

Facile Fabrication of Defect-Controlled Graphene Oxide Membrane through Shear-Induced Alignment for Regulating Ion Transport.

Two-dimensional (2D) membranes enable ion-sieving through well-defined subnanoscale channels. In particular, graphene oxide (GO), a representative 2D material with a flexible structure, can be manufactured into various types of membranes, while defects such as pores and wrinkles are readily formed through self-aggregation and self-folding during membrane fabrication. Such defects provide a path for small ionic or molecular species to be easily penetrated between the layers, which deteriorates membrane performance. Here, we demonstrate the effect of shear-induced alignment with continuous agitation on GO membrane structure during pressure-assisted filtration. The shear stress exerted on the GO layers during deposition is controlled by varying the agitation rate and solution viscosity. The well-stacked 2D membrane is obtained via the facile shear-controlled process, leading to an improved salt rejection performance without additional physicochemical modifications. This simple approach can be extensively utilized to prepare the well-ordered structure of other 2D materials in various fields where the defect control is required.

A Machine Learning-Based Study of the Effects of Air Pollution and Weather in Respiratory Disease Patients Visiting Emergency Departments.

BACKGROUND: To date, investigating respiratory disease patients visiting the emergency departments related with fined dust is limited. This study aimed to analyze the effects of two variable-weather and air pollution on respiratory disease patients who visited emergency departments. METHODS: This study utilized the National Emergency Department Information System (NEDIS) database. The meteorological data were obtained from the National Climate Data Service. Each weather factor reflected the accumulated data of 4 days: a patient's visit day and 3 days before the visit day. We utilized the RandomForestRegressor of scikit-learn for data analysis. RESULT: The study included 525,579 participants. This study found that multiple variables of weather and air pollution influenced the respiratory diseases of patients who visited emergency departments. Most of the respiratory disease patients had acute upper respiratory infections [J00-J06], influenza [J09-J11], and pneumonia [J12-J18], on which PM10 following temperature and steam pressure was the most influential. As the top three leading causes of admission to the emergency department, pneumonia [J12-J18], acute upper respiratory infections [J00-J06], and chronic lower respiratory diseases [J40-J47] were highly influenced by PM10. CONCLUSION: Most of the respiratory patients visiting EDs were diagnosed with acute upper respiratory infections, influenza, and pneumonia. Following temperature, steam pressure and PM10 had influential relations with these diseases. It is expected that the number of respiratory disease patients visiting the emergency departments will increase by day 3 when the steam pressure and temperature values are low, and the variables of air pollution are high. The number of respiratory disease patients visiting the emergency departments will increase by day 3 when the steam pressure and temperature values are low, and the variables of air pollution are high.

ANNO: A General Annotation Tool for Bilingual Clinical Note Information Extraction.

OBJECTIVE: This study was conducted to develop a generalizable annotation tool for bilingual complex clinical text annotation, which led to the design and development of a clinical text annotation tool, ANNO. METHODS: We designed ANNO to enable human annotators to support the annotation of information in clinical documents efficiently and accurately. First, annotations for different classes (word or phrase types) can be tagged according to the type of word using the dictionary function. In addition, it is possible to evaluate and reconcile differences by comparing annotation results between human annotators. Moreover, if the regular expression set for each class is updated during annotation, it is automatically reflected in the new document. The regular expression set created by human annotators is designed such that a word tagged once is automatically labeled in new documents. RESULTS: Because ANNO is a Docker-based web application, users can use it freely without being subjected to dependency issues. Human annotators can share their annotation markups as regular expression sets with a dictionary structure, and they can cross-check their annotated corpora with each other. The dictionary-based regular expression sharing function, cross-check function for each annotator, and standardized input (Microsoft Excel) and output (extensible markup language [XML]) formats are the main features of ANNO. CONCLUSIONS: With the growing need for massively annotated clinical data to support the development of machine learning models, we expect ANNO to be helpful to many researchers.

Transparent and Flexible SiOC Films on Colorless Polyimide Substrate for Flexible Cover Window.

We fabricated transparent and flexible silicon oxycarbide (SiOC) hard coating (HC) films on a colorless polyimide substrate to use as cover window films for flexible and foldable displays using a reactive roll-to-roll (R2R) sputtering system at room temperature. At a SiOC thickness of 100 nm, the R2R-sputtered SiOC film showed a high optical transmittance of 87.43% at a visible range of 400 to 800 nm. The R2R-sputtered SiOC films also demonstrated outstanding flexibility, which is a key requirement of foldable and flexible displays. There were no cracks or surface defects on the SiOC films, even after bending (static folding), folding (dynamic folding), twisting, and rolling tests. Furthermore, the R2R-sputtered SiOC film showed good scratch resistance in a pencil hardness test (550 g) and steel wool test under a load of 250 g. To test the impact protection ability, we compared the performance of thin-film heaters (TFHs) and oxide-semiconductor-based thin-film transistors (TFTs) with and without SiOC cover films. The similar performance of the TFHs and TFTs with the SiOC cover window films demonstrate that the R2R-sputtered SiOC films offer promising cover window films for the next generation of flexible or foldable displays.

Transparent Molecular Adhesive Enabling Mechanically Stable ITO Thin Films.

With rapid advances in flexible electronics, transparent conductive electrodes (TCEs) have also been significantly developed as alternatives to the conventional indium tin oxide (ITO)-based material systems that exhibit low mechanical flexibility. Nanomaterial-based alternating materials, such as graphene, nanowire, and nanomesh, exhibit remarkable properties for TCE-based applications, such as high electrical conductivity, high optical transparency, and high mechanical stability. However, these nanomaterial-based systems lack scalability, which is a key requirement for practical applications, and exhibit a size-dependent property variation and inhomogeneous surface uniformity that limit reliable properties over a large area. Here, we exploited a conventional ITO-based material platform; however, we incorporated a transparent molecular adhesive, 4-aminopyridine (4-AP), to improve mechanical flexibility. While the presence of 4-AP barely affected optical transmittance and sheet resistance, it improved interfacial adhesion between the substrate and ITO as well as formed a wavy surface, which could improve the mechanical flexibility. Under various mechanical tests, ITO/4-AP/poly(ethylene terephthalate) (PET) exhibited remarkably improved mechanical flexibility as compared with that of ITO/PET. Furthermore, ITO/4-AP/PET was utilized for a flexible Joule heater application having spatial uniformity of heat generation, voltage-dependent temperature control, and mechanical flexibility under repeated bending tests. This molecular adhesive could overcome the current limitations of material systems for flexible electronics.

Deep-Asymmetry: Asymmetry Matrix Image for Deep Learning Method in Pre-Screening Depression.

To have an objective depression diagnosis, numerous studies based on machine learning and deep learning using electroencephalogram (EEG) have been conducted. Most studies depend on one-dimensional raw data and required fine feature extraction. To solve this problem, in the EEG visualization research field, short-time Fourier transform (STFT), wavelet, and coherence commonly used as method s for transferring EEG data to 2D images. However, we devised a new way from the concept that EEG's asymmetry was considered one of the major biomarkers of depression. This study proposes a deep-asymmetry methodology that converts the EEG's asymmetry feature into a matrix image and uses it as input to a convolutional neural network. The asymmetry matrix image in the alpha band achieved 98.85% accuracy and outperformed most of the methods presented in previous studies. This study indicates that the proposed method can be an effective tool for pre-screening major depressive disorder patients.

A data-driven approach to a chemotherapy recommendation model based on deep learning for patients with colorectal cancer in Korea.

BACKGROUND: Clinical Decision Support Systems (CDSSs) have recently attracted attention as a method for minimizing medical errors. Existing CDSSs are limited in that they do not reflect actual data. To overcome this limitation, we propose a CDSS based on deep learning. METHODS: We propose the Colorectal Cancer Chemotherapy Recommender (C3R), which is a deep learning-based chemotherapy recommendation model. Our model improves on existing CDSSs in which data-based decision making is not well supported. C3R is configured to study the clinical data collected at the Gachon Gil Medical Center and to recommend appropriate chemotherapy based on the data. To validate the model, we compared the treatment concordance rate with the National Comprehensive Cancer Network (NCCN) Guidelines, a representative set of cancer treatment guidelines, and with the results of the Gachon Gil Medical Center's Colorectal Cancer Treatment Protocol (GCCTP). RESULTS: For the C3R model, the treatment concordance rates with the NCCN guidelines were 70.5% for Top-1 Accuracy and 84% for Top-2 Accuracy. The treatment concordance rates with the GCCTP were 57.9% for Top-1 Accuracy and 77.8% for Top-2 Accuracy. CONCLUSIONS: This model is significant, i.e., it is the first colon cancer treatment clinical decision support system in Korea that reflects actual data. In the future, if sufficient data can be secured through cooperation among multiple organizations, more reliable results can be obtained.

Highly transparent and flexible Ag nanowire-embedded silk fibroin electrodes for biocompatible flexible and transparent heater.

We investigated the electrical, optical and mechanical properties of silver (Ag) nanowire (NW) embedded into a silk fibroin (SF) substrate to create high performance, flexible, transparent, biocompatible, and biodegradable heaters for use in wearable electronics. The Ag NW-embedded SF showed a low sheet resistance of 15 Ω sq-1, high optical transmittance of 85.1%, and a small inner/outer critical bending radius of 1 mm. In addition, the Ag NW-embedded SF showed a constant resistance change during repeated bending, folding, and rolling because the connectivity of the Ag NW embedded into the SF substrate was well maintained. Furthermore, the biocompatible and biodegradable Ag NW-embedded SF substrate served as a flexible interconnector for wearable electronics. The high performance of the transparent and flexible heater demonstrated that an Ag NW-embedded SF-based heater can act as a biocompatible and biodegradable substrate for wearable heaters for the human body.

Stacking Ensemble Technique for Classifying Breast Cancer.

OBJECTIVES: Breast cancer is the second most common cancer among Korean women. Because breast cancer is strongly associated with negative emotional and physical changes, early detection and treatment of breast cancer are very important. As a supporting tool for classifying breast cancer, we tried to identify the best meta-learner model in a stacking ensemble when the same machine learning models for the base learner and meta-learner are used. METHODS: We used machine learning models, such as the gradient boosted model, distributed random forest, generalized linear model, and deep neural network in a stacking ensemble. These models were used to construct a base learner, and each of them was used as a meta-learner again. Then, we compared the performance of machine learning models in the meta-learner to determine the best meta-learner model in the stacking ensemble. RESULTS: Experimental results showed that using the GBM as a meta-learner led to higher accuracy than that achieved with any other model for breast cancer data and using the GLM as a meta learner led to low root-mean-squared error for both sets of breast cancer data. CONCLUSIONS: We compared the performance of every meta-learner model in a stacking ensemble as a supporting tool for classifying breast cancer. The study showed that using specific models as a metalearner resulted in better performance than single classifiers, and using GBM and GLM as a meta-learner is appropriate as a supporting tool for classifying breast cancer data.