A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment.

In the past two decades, unmanned aerial vehicles (UAVs) have demonstrated their efficacy in supporting both military and civilian applications, where tasks can be dull, dirty, dangerous, or simply too costly with conventional methods. Many of the applications contain tasks that can be executed in parallel, hence the natural progression is to deploy multiple UAVs working together as a force multiplier. However, to do so requires autonomous coordination among the UAVs, similar to swarming behaviors seen in animals and insects. This paper looks at flocking with small fixed-wing UAVs in the context of a model-free reinforcement learning problem. In particular, Peng's Q(λ) with a variable learning rate is employed by the followers to learn a control policy that facilitates flocking in a leader-follower topology. The problem is structured as a Markov decision process, where the agents are modeled as small fixed-wing UAVs that experience stochasticity due to disturbances such as winds and control noises, as well as weight and balance issues. Learned policies are compared to ones solved using stochastic optimal control (i.e., dynamic programming) by evaluating the average cost incurred during flight according to a cost function. Simulation results demonstrate the feasibility of the proposed learning approach at enabling agents to learn how to flock in a leader-follower topology, while operating in a nonstationary stochastic environment.

Evaluation of a novel personal nanoparticle sampler.

This work investigated the performance in terms of collection efficiency and aspiration efficiency of a personal sampler capable of collecting ultrafine particles (nanoparticles) in the occupational environment. This sampler consists of a cyclone for respirable particle classification, micro-orifice impactor stages with an acceleration nozzle to achieve nanoparticle classification and a backup filter to collect nanoparticles. Collection efficiencies of the cyclone and impactor stages were determined using monodisperse polystyrene latex and silver particles, respectively. Calibration of the cyclone and impactor stages showed 50% cut-off diameters of 3.95 µm and 94.7 nm meeting the design requirements. Aspiration efficiencies of the sampler were tested in a wind tunnel with wind speeds of 0.5, 1.0, and 1.5 m s(-1). The test samplers were mounted on a full size mannequin with three orientations toward the wind direction (0°, 90°, and 180°). Monodisperse oleic acid aerosols tagged with sodium fluorescein in the size range of 2 to 10 µm were used in the test. For particles smaller than 2 µm, the fluorescent polystyrene latex particles were generated by using nebulizers. For comparison of the aspiration efficiency, a NIOSH two-stage personal bioaerosol sampler was also tested. Results showed that the orientation-averaged aspiration efficiency for both samplers was close to the inhalable fraction curve. However, the direction of wind strongly affected the aspiration efficiency. The results also showed that the aspiration efficiency was not affected by the ratio of free-stream velocity to the velocity through the sampler orifice. Our evaluation showed that the current design of the personal sampler met the designed criteria for collecting nanoparticles ≤100 nm in occupational environments.

Novel active personal nanoparticle sampler for the exposure assessment of nanoparticles in workplaces.

A novel active personal nanoparticle sampler (PENS), which enables the collection of both respirable particulate mass (RPM) and nanoparticles (NPs) simultaneously, was developed to meet the critical demand for personal sampling of engineered nanomaterials (ENMs) in workplaces. The PENS consists of a respirable cyclone and a micro-orifice impactor with the cutoff aerodynamic diameter (d(pa50)) of 4 µm and 100 nm, respectively. The micro-orifice impactor has a fixed micro-orifice plate (137 nozzles of 55 µm in the inner diameter) and a rotating, silicone oil-coated Teflon filter substrate at 1 rpm to achieve a uniform particle deposition and avoid solid particle bounce. A final filter is used after the impactor to collect the NPs. Calibration results show that the d(pa50) of the respirable cyclone and the micro-orifice impactor are 3.92 ± 0.22 µm and 101.4 ± 0.1 nm, respectively. The d(pa50) at the loaded micro-Al(2)O(3) mass of 0.36-3.18 mg is shifted to 102.9-101.2 nm, respectively, while it is shifted to 98.9-97.8 nm at the loaded nano-TiO(2) mass of 0.92-1.78 mg, respectively. That is, the shift of d(pa50) due to solid particle loading is small if the PENS is not overloaded. Both NPs and RPM concentrations were found to agree well with those of the IOSH respirable cyclone and MOUDI. By using the present PENS, the collected samples can be further analyzed for chemical species concentrations besides gravimetric analysis to determine the actual exposure concentrations of ENMs in both RPM and NPs fractions in workplaces, which are often influenced by the background or incident pollution sources.