Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization.

Treatment of wastewater using activated sludge relies on several complex, nonlinear processes. While activated sludge systems can provide high levels of treatment, including nutrient removal, operating these systems is often challenging and energy intensive. Significant research investment has been made in recent years into improving control optimization of such systems, through both domain knowledge and, more recently, machine learning. This study leverages a novel interface between a common process modeling software and a Python reinforcement learning environment to evaluate four common reinforcement learning algorithms for their ability to minimize treatment energy use while maintaining effluent compliance within the Benchmark Simulation Model No. 1 (BSM1) simulation. Three of the algorithms tested, deep Q-learning, proximal policy optimization, and synchronous advantage actor critic, generally performed poorly over the scenarios tested in this study. In contrast, the twin delayed deep deterministic policy gradient (TD3) algorithm consistently produced a high level of control optimization while maintaining the treatment requirements. Under the best selection of state observation features, TD3 control optimization reduced aeration and pumping energy requirements by 14.3% compared to the BSM1 benchmark control, outperforming the advanced domain-based control strategy of ammonia-based aeration control, although future work is necessary to improve robustness of RL implementation.

Comparative investigation of PFAS adsorption onto activated carbon and anion exchange resins during long-term operation of a pilot treatment plant.

Widespread contamination of groundwater with per- and polyfluoroalkyl substances (PFAS) has required drinking water producers to quickly adopt practical and efficacious treatments to limit human exposure and deleterious health outcomes. This pilot-scale study comparatively investigated PFAS adsorption behaviors in granular activated carbon (GAC) and two strong-base gel anion exchange resin (AER) columns operated in parallel over a 441-day period to treat contaminated groundwater dominated by short-chain perfluorocarboxylic acids (PFCA). Highly-resolved breakthrough profiles of homologous series of 2-8 CF2 PFCA and perfluorosulfonic acids (PFSA), including ultrashort-chain compounds and branched isomers, were measured to elucidate adsorption trends. Sample ports at intermediate bed depths could predict 50% breakthrough of compounds on an accelerated basis, but lower empty bed contact times led to conservative estimates of initial breakthrough. Homologous PFAS series displayed linear (GAC) and log-linear (AER) relationships between chain-length and breakthrough, independent of initial concentration. AERs generally outperformed GAC on a normalized bed volume basis, and this advantage widened with increasing PFAS chain-length. As designed, all treatments would have short full-scale service times (≤142 days for GAC; ≤61 days for AERs) before initial breakthrough of short-chain (2-4 CF2) PFCA. However, AER displayed far longer breakthrough times for PFSA compared to GAC (>3× treatment time), and breakthrough was not observed for PFSA with >4 CF2 in AERs. GAC had a finite molar adsorption capacity for total PFAS, leading to a stoichiometric replacement of short-chain PFCA by PFSA and longer-chain PFCA over time. AERs quickly reached a finite adsorption capacity for PFCA, but they showed substantially greater selectivity for PFSA whose capacity was not reached within the duration of the pilot. Breakthrough characteristics of keto- and unsaturated-PFSA, identified in the groundwater by suspect screening, were also evaluated in absence of reference standards. Modified PFAS structures (branched, keto-, unsaturated-) broke through faster than linear and unmodified perfluorinated structures with equal degrees of fluorination, and the effects were more pronounced in GAC compared to AERs. The results highlight that the design of robust PFAS treatment systems should consider facets beyond current PFAS targets including operational complexities and impacts of unregulated and unmonitored co-contaminants.

Adaptation of selected models for describing competitive per- and polyfluoroalkyl substances breakthrough curves in groundwater treated by granular activated carbon.

Granular activated carbon (GAC) has proven to be a successful technology for per- and polyfluoroalkyl substances (PFAS) removal from contaminated drinking water supplies. Proper design of GAC treatment relies upon characterization of media service-life, which can change significantly depending on the PFAS contamination, treatment media, and water quality, and is often determined by fitting descriptive models to breakthrough curves. However, while common descriptive breakthrough models are favored for their ease-of-use, they have a significant shortcoming in that they are not able to properly fit PFAS desorption in competitive sorption scenarios. The present work adapts three common descriptive models to fit competitive PFAS breakthrough curves from a GAC pilot study. The adapted and original models were fit to the experimental breakthrough curves for 12 common PFAS and evaluated using adjusted R2 and reduced χ2 values. This study found that the novel adaptation of the common descriptive models successfully accounted for desorption of PFAS compounds from the GAC, accurately describing increased exposure risks due to elevated effluent levels during desorption without significantly increasing the complexity of implementing the models.