Hierarchical Training of Deep Neural Networks Using Early Exiting.

Deep neural networks (DNNs) provide state-of-the-art accuracy for vision tasks, but they require significant resources for training. Thus, they are trained on cloud servers far from the edge devices that acquire the data. This issue increases communication cost, runtime, and privacy concerns. In this study, a novel hierarchical training method for DNNs is proposed that uses early exits in a divided architecture between edge and cloud workers to reduce the communication cost, training runtime, and privacy concerns. The method proposes a brand-new use case for early exits to separate the backward pass of neural networks between the edge and the cloud during the training phase. We address the issues of most available methods that, due to the sequential nature of the training phase, cannot train the levels of hierarchy simultaneously or they do it with the cost of compromising privacy. In contrast, our method can use both edge and cloud workers simultaneously, does not share the raw input data with the cloud, and does not require communication during the backward pass. Several simulations and on-device experiments for different neural network architectures demonstrate the effectiveness of this method. It is shown that the proposed method reduces the training runtime for VGG-16 and ResNet-18 architectures by 29% and 61% in CIFAR-10 classification and by 25% and 81% in Tiny ImageNet classification, respectively, when the communication with the cloud is done over a low bit rate channel. This gain in the runtime is achieved, while the accuracy drop is negligible. This method is advantageous for online learning of high-accuracy DNNs on sensor-holding low-resource devices such as mobile phones or robots as a part of an edge-cloud system, making them more flexible in facing new tasks and classes of data.

Learning Generative Models Using Denoising Density Estimators.

Learning probabilistic models that can estimate the density of a given set of samples, and generate samples from that density, is one of the fundamental challenges in unsupervised machine learning. We introduce a new generative model based on denoising density estimators (DDEs), which are scalar functions parametrized by neural networks, that are efficiently trained to represent kernel density estimators of the data. Leveraging DDEs, our main contribution is a novel technique to obtain generative models by minimizing the Kullback-Leibler (KL)-divergence directly. We prove that our algorithm for obtaining generative models is guaranteed to converge consistently to the correct solution. Our approach does not require specific network architecture as in normalizing flows (NFs), nor use ordinary differential equation (ODE) solvers as in continuous NFs. Experimental results demonstrate substantial improvement in density estimation and competitive performance in generative model training.

Description of the modes governing the optical transmission through metal gratings.

An analytical model based on a modal expansion method is developed to investigate the optical transmission through metal gratings. This model gives analytical expressions for the transmission as well as for the dispersion relations of the modes responsible for high transmission. These expressions are accurate even for real metals used in the visible - near-infrared wavelength range, where surface plasmon polaritons (SPP's) are excited. The dispersion relations allow the nature of the modes to be assessed. We find that the transmission modes are hybrid between Fabry-Pérot like modes and SPP's. It is also shown that it is important to consider different refractive indices above and below the gratings in order to determine the nature of the hybrid modes. These findings are important as they clarify the nature of the modes responsible for high transmission. It can also be useful as a design tool for metal gratings for various applications.

Terahertz quantum cascade lasers based on two-dimensional photonic crystal resonators.

We demonstrate high spectral control from surface emitting THz Quantum Cascade Lasers based on a two-dimensional photonic crystal cavity. The perforated top metallic contact acts as an in plane resonator in a tight double-metal plasmonic waveguide providing a strong optical feedback without needing three-dimensional cavity features. The optical far-field patterns do not exhibit the expected symmetry and the shape of the cavity mode. The difference is attributed to a metal surface plasmon mediated light outcoupling mechanism also responsible for the relatively low extraction efficiency.

Terahertz photonic crystal quantum cascade lasers.

We combine photonic crystal and quantum cascade band engineering to create an in-plane laser at terahertz frequency. We demonstrate that such photonic crystal lasers strongly improve the performances of terahertz quantum cascade material in terms of threshold current, waveguide losses, emission mode selection, tunability and maximum operation temperature. The laser operates in a slow-light regime between the M saddle point and K band-edge in reciprocal lattice. Coarse frequency control of half of a terahertz is achieved by lithographically tuning the photonic crystal period. Thanks to field assisted gain shift and cavity pulling, the single mode emission is continuously tuned over 30 GHz.

Confocal microscopy using variable-focal-length microlenses and an optical fiber bundle.

The use of variable-focal-length (VFL) microlenses can provide a way to axially scan the foci across a sample by electronic control. We demonstrate an approach to coupling VFL microlenses individually to a fiber bundle as a way to create a high-throughput aperture array with a controllable aperture pattern. It would potentially be applied in real-time confocal imaging in vivo for biological specimens. The VFL microlenses that we used consist of a liquid-crystal film sandwiched between a pair of conductive substrates for which one has a hole-patterned electrode. One obtains the variation of the focal length by changing the applied voltage. The fiber bundle has been characterized by coupling with both coherent and incoherent light sources. We further demonstrate the use of a VFL microlens array in combination with the fiber bundle to build up a confocal system. The axial response of the confocal system has been measured without mechanical movement of the sample or the objective, and the FWHM is estimated to be approximately 16 microm, with asymmetric sidelobes.

Design, fabrication and optical characterization of quantum cascade lasers at terahertz frequencies using photonic crystal reflectors.

We designed, fabricated and characterised electrically injected quantum cascade lasers with photonic crystal reflectors emitting at terahertz frequencies (3.75 THz). These in-plane emitting structures display typical threshold current densities of 420 A/cm2 and output powers of up to 2 mW under pulsed excitation. The emission characteristics are shown to be robust, as with increasing current the emission remains singlemode with no drift in wavelength, this results from the narrow reflectivity band of the photonic crystal reflectors.