Alternative linear microgrid polarimeters: design, analysis, and demosaicing considerations.

Linear division of focal plane (DoFP), or integrated microgrid polarimeters, provide a measurement strategy for obtaining time-synchronized polarized intensity measurements across a scene. This is accomplished by masking pixels in the focal plane array sensor with a repeating pattern of different linear polarizers. The convention in industry has been to use a repeating 2×2 pattern of four linear polarizers with chosen polarizer orientation angles of {0,45,90,135}∘. Alternative designs based upon other P×Q modulation patterns have been proposed that demonstrate improved performance over conventional microgrid arrays due to better utilization of bandwidth in the frequency domain. Here, we develop a model for linear DoFP snapshot polarimeters that provides an in-depth understanding of these devices in both the spatial and frequency domains and relate this model to previously reported generalized DoFP channeled polarimeter models. We then use the model to identify practical modulation patterns and study their performance through empirical simulations based upon data collected from real polarimeters. We demonstrate the validity of the developed model and compare the performance of the identified modulation schemes against a common set of ground truth images. We find that choosing alternative sets of polarizer angles, in conjunction with modulators that improve bandwidth usage, result in the best overall designs that can improve performance over conventional microgrid polarimeters.

Conditional generative adversarial network demosaicing strategy for division of focal plane polarimeters.

Division of focal plane (DoFP), or integrated microgrid polarimeters, typically consist of a 2 × 2 mosaic of linear polarization filters overlaid upon a focal plane array sensor and obtain temporally synchronized polarized intensity measurements across a scene, similar in concept to a Bayer color filter array camera. However, the resulting estimated polarimetric images suffer a loss in resolution and can be plagued by aliasing due to the spatially-modulated microgrid measurement strategy. Demosaicing strategies have been proposed that attempt to minimize these effects, but result in some level of residual artifacts. In this work we propose a conditional generative adversarial network (cGAN) approach to the microgrid demosaicing problem. We evaluate the performance of our approach against full-resolution division-of-time polarimeter data as well as compare against both traditional and recent microgrid demosaicing methods. We apply these demosaicing strategies to data from both real and simulated visible microgrid imagery and provide an objective criteria for evaluating their performance. We demonstrate that the proposed cGAN approach results in estimated Stokes imagery that is comparable to full-resolution ground truth imagery from both a quantitative and qualitative perspective.

Adapting the HSV polarization-color mapping for regions with low irradiance and high polarization.

Many mappings from polarization into color have been developed so that polarization information can be displayed. One of the most common of these maps the angle of linear polarization into color hue and degree of linear polarization into color saturation, while preserving the irradiance information from the polarization data. While this strategy enjoys wide popularity, there is a large class of polarization images for which it is not ideal. It is common to have images where the strongest polarization signatures (in terms of degree of polarization) occur in regions of relatively low irradiance: either in shadow in reflective bands or in cold regions in emissive bands. Since the irradiance is low, the chromatic properties of the resulting images are generally not apparent. Here we present an alternate mapping that uses the statistics of the angle of polarization as a measure of confidence in the polarization signature, then amplifies the irradiance in regions of high confidence, and leaves it unchanged in regions of low confidence. Results are shown from an LWIR and a visible spectrum imager.

Super-resolution for imagery from integrated microgrid polarimeters.

Imagery from microgrid polarimeters is obtained by using a mosaic of pixel-wise micropolarizers on a focal plane array (FPA). Each distinct polarization image is obtained by subsampling the full FPA image. Thus, the effective pixel pitch for each polarization channel is increased and the sampling frequency is decreased. As a result, aliasing artifacts from such undersampling can corrupt the true polarization content of the scene. Here we present the first multi-channel multi-frame super-resolution (SR) algorithms designed specifically for the problem of image restoration in microgrid polarization imagers. These SR algorithms can be used to address aliasing and other degradations, without sacrificing field of view or compromising optical resolution with an anti-aliasing filter. The new SR methods are designed to exploit correlation between the polarimetric channels. One of the new SR algorithms uses a form of regularized least squares and has an iterative solution. The other is based on the faster adaptive Wiener filter SR method. We demonstrate that the new multi-channel SR algorithms are capable of providing significant enhancement of polarimetric imagery and that they outperform their independent channel counterparts.

Interpolation strategies for reducing IFOV artifacts in microgrid polarimeter imagery.

Microgrid polarimeters are composed of an array of micro-polarizing elements overlaid upon an FPA sensor. In the past decade systems have been designed and built in all regions of the optical spectrum. These systems have rugged, compact designs and the ability to obtain a complete set of polarimetric measurements during a single image capture. However, these systems acquire the polarization measurements through spatial modulation and each measurement has a varying instantaneous field-of-view (IFOV). When these measurements are combined to estimate the polarization images, strong edge artifacts are present that severely degrade the estimated polarization imagery. These artifacts can be reduced when interpolation strategies are first applied to the intensity data prior to Stokes vector estimation. Here we formally study IFOV error and the performance of several bilinear interpolation strategies used for reducing it.

Total elimination of sampling errors in polarization imagery obtained with integrated microgrid polarimeters.

Microgrid polarimeters operate by integrating a focal plane array with an array of micropolarizers. The Stokes parameters are estimated by comparing polarization measurements from pixels in a neighborhood around the point of interest. The main drawback is that the measurements used to estimate the Stokes vector are made at different locations, leading to a false polarization signature owing to instantaneous field-of-view (IFOV) errors. We demonstrate for the first time, to our knowledge, that spatially band limited polarization images can be ideally reconstructed with no IFOV error by using a linear system framework.

Dead pixel replacement in LWIR microgrid polarimeters.

LWIR imaging arrays are often affected by nonresponsive pixels, or "dead pixels." These dead pixels can severely degrade the quality of imagery and often have to be replaced before subsequent image processing and display of the imagery data. For LWIR arrays that are integrated with arrays of micropolarizers, the problem of dead pixels is amplified. Conventional dead pixel replacement (DPR) strategies cannot be employed since neighboring pixels are of different polarizations. In this paper we present two DPR schemes. The first is a modified nearest-neighbor replacement method. The second is a method based on redundancy in the polarization measurements.We find that the redundancy-based DPR scheme provides an order-of-magnitude better performance for typical LWIR polarimetric data.

The effects of thermal equilibrium and contrast in LWIR polarimetric images.

Long-wave infrared (LWIR) polarimetric signatures provide the potential for day-night detection and identification of objects in remotely sensed imagery. The source of optical energy in the LWIR is usually due to thermal emission from the object in question, which makes the signature dependent primarily on the target and not on the external environment. In this paper we explore the impact of thermal equilibrium and the temperature of (unseen) background objects on LWIR polarimetric signatures. We demonstrate that an object can completely lose its polarization signature when it is in thermal equilibrium with its optical background, even if it has thermal contrast with the objects that appear behind it in the image.

Generalized algebraic scene-based nonuniformity correction algorithm.

A generalization of a recently developed algebraic scene-based nonuniformity correction algorithm for focal plane array (FPA) sensors is presented. The new technique uses pairs of image frames exhibiting arbitrary one- or two-dimensional translational motion to compute compensator quantities that are then used to remove nonuniformity in the bias of the FPA response. Unlike its predecessor, the generalization does not require the use of either a blackbody calibration target or a shutter. The algorithm has a low computational overhead, lending itself to real-time hardware implementation. The high-quality correction ability of this technique is demonstrated through application to real IR data from both cooled and uncooled infrared FPAs. A theoretical and experimental error analysis is performed to study the accuracy of the bias compensator estimates in the presence of two main sources of error.

Radiometrically accurate scene-based nonuniformity correction for array sensors.

A novel radiometrically accurate scene-based nonuniformity correction (NUC) algorithm is described. The technique combines absolute calibration with a recently reported algebraic scene-based NUC algorithm. The technique is based on the following principle: First, detectors that are along the perimeter of the focal-plane array are absolutely calibrated; then the calibration is transported to the remaining uncalibrated interior detectors through the application of the algebraic scene-based algorithm, which utilizes pairs of image frames exhibiting arbitrary global motion. The key advantage of this technique is that it can obtain radiometric accuracy during NUC without disrupting camera operation. Accurate estimates of the bias nonuniformity can be achieved with relatively few frames, which can be fewer than ten frame pairs. Advantages of this technique are discussed, and a thorough performance analysis is presented with use of simulated and real infrared imagery.

An algebraic algorithm for nonuniformity correction in focal-plane arrays.

A scene-based algorithm is developed to compensate for bias nonuniformity in focal-plane arrays. Nonuniformity can be extremely problematic, especially for mid- to far-infrared imaging systems. The technique is based on use of estimates of interframe subpixel shifts in an image sequence, in conjunction with a linear-interpolation model for the motion, to extract information on the bias nonuniformity algebraically. The performance of the proposed algorithm is analyzed by using real infrared and simulated data. One advantage of this technique is its simplicity; it requires relatively few frames to generate an effective correction matrix, thereby permitting the execution of frequent on-the-fly nonuniformity correction as drift occurs. Additionally, the performance is shown to exhibit considerable robustness with respect to lack of the common types of temporal and spatial irradiance diversity that are typically required by statistical scene-based nonuniformity correction techniques.