ABSTRACT
A deep reinforcement learning application is investigated to control the emissions of a compression ignition diesel engine. The main purpose of this study is to reduce the engine-out nitrogen oxide (NOx) emissions and to minimize fuel consumption while tracking a reference engine load. First, a physics-based engine simulation model is developed in GT-Power and calibrated using experimental data. Using this model and a GT-Power/Simulink co-simulation, a deep deterministic policy gradient is developed. To reduce the risk of an unwanted output, a safety filter is added to the deep reinforcement learning. Based on the simulation results, this filter has no effect on the final trained deep reinforcement learning; however, during the training process, it is crucial to enforce constraints on the controller output. The developed safe reinforcement learning is then compared with an iterative learning controller and a deep neural network-based nonlinear model predictive controller. This comparison shows that the safe reinforcement learning is capable of accurately tracking an arbitrary reference input while the iterative learning controller is limited to a repetitive reference. The comparison between the nonlinear model predictive control and reinforcement learning indicates that for this case reinforcement learning is able to learn the optimal control output directly from the experiment without the need for a model. However, to enforce output constraint for safe learning reinforcement learning, a simple model of system is required. In this work, reinforcement learning was able to reduce NOx emissions more than the nonlinear model predictive control; however, it suffered from slightly higher error in load tracking and a higher fuel consumption.
ABSTRACT
In cold temperatures, vehicles idle more, have high cold-start emissions including greenhouse gases, and have less effective exhaust filtration systems, which can cause up to ten-fold more harmful vehicular emissions. Only a few vehicle technologies have been tested for emissions below -7 °C (20 °F). Four-hundred-million people living in cities with sub-zero temperatures may be impacted. We conducted a scoping review to identify the existing knowledge about air-pollution-related health outcomes in a cold climate, and pinpoint any research gaps. Of 1019 papers identified, 76 were selected for review. The papers described short-term health impacts associated with air pollutants. However, most papers removed the possible direct effect of temperature on pollution and health by adjusting for temperature. Only eight papers formally explored the modifying effect of temperatures. Five studies identified how extreme cold and warm temperatures aggravated mortality/morbidity associated with ozone, particles, and carbon-monoxide. The other three found no health associations with tested pollutants and temperature. Additionally, in most papers, emissions could not be attributed solely to traffic. In conclusion, evidence on the relationship between cold temperatures, traffic-related pollution, and related health outcomes is lacking. Therefore, targeted research is required to guide vehicle regulations, assess extreme weather-related risks in the context of climate change, and inform public health interventions.
