OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning.

Federated learning (FL) has emerged as a promising approach to collaboratively train machine learning models across multiple edge devices while preserving privacy. The success of FL hinges on the efficiency of participating models and their ability to handle the unique challenges of distributed learning. While several variants of Vision Transformer (ViT) have shown great potential as alternatives to modern convolutional neural networks (CNNs) for centralized training, the unprecedented size and higher computational demands hinder their deployment on resource-constrained edge devices, challenging their widespread application in FL. Since client devices in FL typically have limited computing resources and communication bandwidth, models intended for such devices must strike a balance between model size, computational efficiency, and the ability to adapt to the diverse and non-IID data distributions encountered in FL. To address these challenges, we propose OnDev-LCT: Lightweight Convolutional Transformers for On-Device vision tasks with limited training data and resources. Our models incorporate image-specific inductive biases through the LCT tokenizer by leveraging efficient depthwise separable convolutions in residual linear bottleneck blocks to extract local features, while the multi-head self-attention (MHSA) mechanism in the LCT encoder implicitly facilitates capturing global representations of images. Extensive experiments on benchmark image datasets indicate that our models outperform existing lightweight vision models while having fewer parameters and lower computational demands, making them suitable for FL scenarios with data heterogeneity and communication bottlenecks.

Contrastive encoder pre-training-based clustered federated learning for heterogeneous data.

Federated learning (FL) is a promising approach that enables distributed clients to collaboratively train a global model while preserving their data privacy. However, FL often suffers from data heterogeneity problems, which can significantly affect its performance. To address this, clustered federated learning (CFL) has been proposed to construct personalized models for different client clusters. One effective client clustering strategy is to allow clients to choose their own local models from a model pool based on their performance. However, without pre-trained model parameters, such a strategy is prone to clustering failure, in which all clients choose the same model. Unfortunately, collecting a large amount of labeled data for pre-training can be costly and impractical in distributed environments. To overcome this challenge, we leverage self-supervised contrastive learning to exploit unlabeled data for the pre-training of FL systems. Together, self-supervised pre-training and client clustering can be crucial components for tackling the data heterogeneity issues of FL. Leveraging these two crucial strategies, we propose contrastive pre-training-based clustered federated learning (CP-CFL) to improve the model convergence and overall performance of FL systems. In this work, we demonstrate the effectiveness of CP-CFL through extensive experiments in heterogeneous FL settings, and present various interesting observations.

Self-Organizing Democratized Learning: Toward Large-Scale Distributed Learning Systems.

Emerging cross-device artificial intelligence (AI) applications require a transition from conventional centralized learning systems toward large-scale distributed AI systems that can collaboratively perform complex learning tasks. In this regard, democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. The outlined principles are meant to study a generalization in distributed learning systems that go beyond existing mechanisms such as federated learning (FL). Moreover, such learning systems rely on hierarchical self-organization of well-connected distributed learning agents who have limited and highly personalized data and can evolve and regulate themselves based on the underlying duality of specialized and generalized processes. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this article. The approach consists of a self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and corresponding learning mechanism. Subsequently, hierarchical generalized learning problems in recursive forms are formulated and shown to be approximately solved using the solutions of distributed personalized learning problems and hierarchical update mechanisms. To that end, a distributed learning algorithm, namely DemLearn, is proposed. Extensive experiments on benchmark MNIST, Fashion-MNIST, FE-MNIST, and CIFAR-10 datasets show that the proposed algorithm demonstrates better results in the generalization performance of learning models in agents compared to the conventional FL algorithms. The detailed analysis provides useful observations to further handle both the generalization and specialization performance of the learning models in Dem-AI systems.

Results from the Survey of Antibiotic Resistance (SOAR) 2009-11 in Vietnam.

OBJECTIVES: To investigate the susceptibility of respiratory tract infection pathogens collected between 2009 and 2011 from the SOAR study in Vietnam. METHODS: MICs were determined using Etest(®) and susceptibility was assessed using CLSI, EUCAST and pharmacokinetic/pharmacodynamic (PK/PD) breakpoints. RESULTS: Two hundred and eighty-nine Streptococcus pneumoniae and 195 Haemophilus influenzae were collected from 11 centres. Overall, 4.8% of S. pneumoniae were penicillin susceptible (CLSI oral and EUCAST breakpoints). Using CLSI intravenous breakpoints, 86.9% were penicillin susceptible. Susceptibility to high-dose amoxicillin/clavulanic acid (or amoxicillin) using PK/PD breakpoints, cefuroxime (using CLSI or PK/PD breakpoints), cefaclor (CLSI breakpoint) and azithromycin (CLSI breakpoint) was 96.9%, 18.7%, 8% and 4.2%, respectively. Ofloxacin susceptibility was 93.4% by CLSI but 0% by EUCAST. All S. pneumoniae were fully vancomycin susceptible. S. pneumoniae from children were significantly less susceptible to most antimicrobials than those from the elderly. For ofloxacin, however, the reverse was true.Among H. influenzae isolates, 40.5% produced ß-lactamase and 13.8% were ß-lactamase negative but ampicillin resistant (BLNAR) by CLSI. H. influenzae were highly susceptible (97.4%) in vitro to amoxicillin/clavulanic acid and also to ceftriaxone by CLSI and PK/PD breakpoints but not EUCAST breakpoints. However, BLNAR isolates should be considered clinically resistant, with susceptibility reduced to 84.1%. With EUCAST breakpoints, amoxicillin/clavulanic acid susceptibility was lower, at 63.1%. Azithromycin susceptibility was 79.5% (CLSI). CONCLUSIONS: Resistance to antibacterials in Vietnam was high, with amoxicillin/clavulanic acid being the most active agent. Ceftriaxone was highly active against H. influenzae while ofloxacin appeared highly active against S. pneumoniae using CLSI but not by EUCAST breakpoints. Ongoing surveillance through SOAR will further assist in understanding susceptibility trends over time.

Determination of PCDD/Fs in breast milk of women living in the vicinities of Da Nang Agent Orange hot spot (Vietnam) and estimation of the infant's daily intake.

Seventeen toxic congeners of polychlorinated dibenzo-p-dioxins (PCDDs) and polychlorinated dibenzofurans (PCDFs) were determined in breast milks using the high resolution gas chromatography/high resolution mass spectrometry (HRGC/HRMS) method. Twenty seven breast milk samples were collected from primiparae who have lived over 5 years in wards namely Chinh Gian, An Khe, Khue Trung, and Hoa Thuan Tay which are located near the Da Nang Agent Orange hot spot (the AO/Dioxin hot spot). The samples were then analyzed for PCDD/F residues in order to assess the human exposure to dioxins from the AO/Dioxin hot spot, especially health risk to the breast-fed infants. The average TEQ levels in the four studied cohorts ranged from 8.1 to 26 pg/g lipid, with the highest level up to 51 pg TEQ/g lipid found in the An Khe ward. The TEQ level was correlated with geographical position and ranking in the order of Khue Trung, Hoa Thuan Tay, Chinh Gian and An Khe. The mean estimated PCDD/Fs infant's daily intake in the cohort of Khue Trung, Hoa Thuan Tay, Chinh Gian and An Khe was about 41, 122, 124, and 134 pg TEQ/kg bw/day, respectively, which are much higher than the tolerable daily intake proposed by the World Health Organization (4 pg TEQ/kg bw/day).

Timing of initiation of antiretroviral therapy in human immunodeficiency virus (HIV)--associated tuberculous meningitis.

BACKGROUND: The optimal time to initiate antiretroviral therapy (ART) in human immunodeficiency virus (HIV)-associated tuberculous meningitis is unknown. METHODS: We conducted a randomized, double-blind, placebo-controlled trial of immediate versus deferred ART in patients with HIV-associated tuberculous meningitis to determine whether immediate ART reduced the risk of death. Antiretroviral drugs (zidovudine, lamivudine, and efavirenz) were started either at study entry or 2 months after randomization. All patients were treated with standard antituberculosis treatment, adjunctive dexamethasone, and prophylactic co-trimoxazole and were followed up for 12 months. We conducted intention-to-treat, per-protocol, and prespecified subgroup analyses. RESULTS: A total of 253 patients were randomized, 127 in the immediate ART group and 126 in the deferred ART group; 76 and 70 patients died within 9 months in the immediate and deferred ART groups, respectively. Immediate ART was not significantly associated with 9-month mortality (hazard ratio [HR], 1.12; 95% confidence interval [CI], .81-1.55; P = .50) or the time to new AIDS events or death (HR, 1.16; 95% CI, .87-1.55; P = .31). The percentage of patients with severe (grade 3 or 4) adverse events was high in both arms (90% in the immediate ART group and 89% in the deferred ART group; P = .84), but there were significantly more grade 4 adverse events in the immediate ART arm (102 in the immediate ART group vs 87 in the deferred ART group; P = .04). CONCLUSIONS: Immediate ART initiation does not improve outcome in patients presenting with HIV-associated tuberculous meningitis. There were significantly more grade 4 adverse events in the immediate ART arm, supporting delayed initiation of ART in HIV-associated tuberculous meningitis. Clinical Trials Registration. ISRCTN63659091.

Contamination by persistent organic pollutants in dumping sites of Asian developing countries: implication of emerging pollution sources.

In Asian developing countries, large amounts of municipal wastes are dumped daily in open dumping sites without proper management. This practice may cause several adverse environmental consequences and increased health risk to local communities. To elucidate contamination by persistent organic pollutants (POPs)--including dichloro-diphenyl-trichloroethane and its metabolites (DDTs), hexachlorocyclohexanes (HCHs), chlordanes, hexachlorobenzene (HCB), and polychlorinated biphenyls (PCBs)--in such dumping sites, soil samples were collected from open dumping sites and respective control sites in Cambodia, India, and Vietnam from 1999 through 2001. Our results demonstrated that DDTs, PCBs, and HCHs were dominant contaminants in the dumping sites. However, the contamination pattern was not consistent, showing higher HCHs in India than in Cambodia and Vietnam. Interestingly, in all of the countries, extremely higher levels of POPs were observed in the dumping sites compared with those in the respective control sites, suggesting significant amplification of POP contamination in the dumping sites of Asian developing countries. Mean concentrations of DDTs and PCBs were 350 and 140 ng/g dry weight, respectively, in the dumping sites of Cambodia and 26 and 210 ng/g, respectively, in India. These residue levels were hundreds to thousands times higher than those in general soils, implying possible risk to human health of the local communities, especially to the rag pickers, including children who work in these sites to collect recyclable materials. Composition of DDT compounds suggested their recent use in populated areas, which in turn might have caused increased levels of DDTs in the open dumping sites. In addition, composition of HCH isomers revealed their different use pattern in different countries.