Increasing optical pose estimation accuracy via freeform design and its application to hand-eye calibration.

For robot-assisted assembly of complex optical systems, the alignment is facilitated by an accurate pose estimation of its components. However, wavefront-based pose estimation is typically ill-conditioned due to the inherent geometry of conventional industrially manufactured optical components. Therefore, we propose a novel approach in this paper to increase wavefront-based pose estimation accuracy via the design of freeform optics. For this purpose, an optimization problem is derived that parameterizes the component's surfaces by a predetermined freeform surface model. To show the efficacy of our approach, we provide simulation results to compare the pose estimation accuracy for a variety of optical designs. As an application example for the resulting improved pose estimation, a hand-eye calibration of a wavefront sensor is performed. This calibration originates from the field of robotics and represents the identification of a sensor coordinate system with respect to a global reference frame. For quantitative evaluation, the calibrating results are first presented with the aid of simulation data. Finally, the practical feasibility is demonstrated using a conventional industrial robot and additively manufactured freeform lenses.

Pose Estimation and Damage Characterization of Turbine Blades during Inspection Cycles and Component-Protective Disassembly Processes.

Inspection in confined spaces and difficult-to-access machines is a challenging quality assurance task and particularly difficult to quantify and automate. Using the example of aero engine inspection, an approach for the high-precision inspection of movable turbine blades in confined spaces will be demonstrated. To assess the condition and damages of turbine blades, a borescopic inspection approach in which the pose of the turbine blades is estimated on the basis of measured point clouds is presented. By means of a feature extraction approach, film-cooling holes are identified and used to pre-align the measured point clouds to a reference geometry. Based on the segmented features of the measurement and reference geometry a RANSAC-based feature matching is applied, and a multi-stage registration process is performed. Subsequently, an initial damage assessment of the turbine blades is derived, and engine disassembly decisions can be assisted by metric geometry deviations. During engine disassembly, the blade root is exposed to high disassembly forces, which can damage the blade root and is crucial for possible repair. To check for dismantling damage, a fast inspection of the blade root is executed using the borescopic sensor.

Adapted Fringe Projection Sequences for Changing Illumination Conditions on the Example of Measuring a Wrought-Hot Object Influenced by Forced Cooling.

Optical 3D geometry reconstruction, or more specific, fringe projection profilometry, is a state-of-the-art technique for the measurement of the shape of objects in confined spaces or under rough environmental conditions, e.g., while inspecting a wrought-hot specimen after a forging operation. While the contact-less method enables the measurement of such an object, the results are influenced by the light deflection effect occurring due to the inhomogeneous refractive index field induced by the hot air around the measurand. However, the developed active compensation methods to fight this issue exhibits a major drawback, namely an additional cooling of the object and a subsequent transient illumination component. In this paper, we investigate the cooling and its effect on temporal phase reconstruction algorithms and take a theoretical approach to its compensation. The simulated compensation measures are transferred to a fringe projection profilometry setup and are evaluated using established and newly developed methods. The results show a significant improvement when measuring specimens under a transient illumination and are easily transferable to any kind of multi-frequency phase-shift measurement.

Fringe Projection Profilometry in Production Metrology: A Multi-Scale Comparison in Sheet-Bulk Metal Forming.

Fringe projection profilometry in combination with other optical measuring technologies has established itself over the last decades as an essential complement to conventional, tactile measuring devices. The non-contact, holistic reconstruction of complex geometries within fractions of a second in conjunction with the lightweight and transportable sensor design open up many fields of application in production metrology. Furthermore, triangulation-based measuring principles feature good scalability, which has led to 3D scanners for various scale ranges. Innovative and modern production processes, such as sheet-bulk metal forming, thus, utilize fringe projection profilometry in many respects to monitor the process, quantify possible wear and improve production technology. Therefore, it is essential to identify the appropriate 3D scanner for each application and to properly evaluate the acquired data. Through precise knowledge of the measurement volume and the relative uncertainty with respect to the specimen and scanner position, adapted measurement strategies and integrated production concepts can be realized. Although there are extensive industrial standards and guidelines for the quantification of sensor performance, evaluation and tolerancing is mainly global and can, therefore, neither provide assistance in the correct, application-specific positioning and alignment of the sensor nor reflect the local characteristics within the measuring volume. Therefore, this article compares fringe projection systems across various scale ranges by positioning and scanning a calibrated sphere in a high resolution grid.

Quantitative 3D Reconstruction from Scanning Electron Microscope Images Based on Affine Camera Models.

Scanning electron microscopes (SEMs) are versatile imaging devices for the micro- and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen's two-dimensional (2D) properties are provided by the acquired images, detailed morphological characterizations require knowledge about the three-dimensional (3D) surface structure. To overcome this limitation, a reconstruction routine is presented that allows the quantitative depth reconstruction from SEM image sequences. Based on the SEM's imaging properties that can be well described by an affine camera, the proposed algorithms rely on the use of affine epipolar geometry, self-calibration via factorization and triangulation from dense correspondences. To yield the highest robustness and accuracy, different sub-models of the affine camera are applied to the SEM images and the obtained results are directly compared to confocal laser scanning microscope (CLSM) measurements to identify the ideal parametrization and underlying algorithms. To solve the rectification problem for stereo-pair images of an affine camera so that dense matching algorithms can be applied, existing approaches are adapted and extended to further enhance the yielded results. The evaluations of this study allow to specify the applicability of the affine camera models to SEM images and what accuracies can be expected for reconstruction routines based on self-calibration and dense matching algorithms.

Scale-Aware Multi-View Reconstruction Using an Active Triple-Camera System.

This paper presents an active wide-baseline triple-camera measurement system designed especially for 3D modeling in general outdoor environments, as well as a novel parallel surface refinement algorithm within the multi-view stereo (MVS) framework. Firstly, the pre-processing module converts the synchronized raw triple images from one single-shot acquisition of our setup to aligned RGB-Depth frames, which are then used for camera pose estimation using iterative closest point (ICP) and RANSAC perspective-n-point (PnP) approaches. Afterwards, an efficient dense reconstruction method, mostly implemented on the GPU in a grid manner, takes the raw depth data as input and optimizes the per-pixel depth values based on the multi-view photographic evidence, surface curvature and depth priors. Through a basic fusion scheme, an accurate and complete 3D model can be obtained from these enhanced depth maps. For a comprehensive test, the proposed MVS implementation is evaluated on benchmark and synthetic datasets, and a real-world reconstruction experiment is also conducted using our measurement system in an outdoor scenario. The results demonstrate that (1) our MVS method achieves very competitive performance in terms of modeling accuracy, surface completeness and noise reduction, given an input coarse geometry; and (2) despite some limitations, our triple-camera setup in combination with the proposed reconstruction routine, can be applied to some practical 3D modeling tasks operated in outdoor environments where conventional stereo or depth senors would normally suffer.

Alignment strategy for an optomechanical image derotator using a laser Doppler vibrometer.

When monitoring the condition of technical components, it is of particular interest to obtain information of the current condition of the component during operation. The main objectives are safety and machine efficiency. A special extension in this area is the derotator, since the derotator can be used to extend measuring methods designed for static components to rotating systems. In this paper, we present a new, to the best of our knowledge, approach to align a derotator using a laser Doppler vibrometer (LDV). When aligning the derotator, the optical axis must be coaxial with the rotating object to be measured. Current conventional approaches only ensure that the optical axis and the rotational axis intersect at the center of the measuring object. In our approach, a LDV is used to determine the angular deviation between the two axes, which cannot be determined by common methods. Our approach is easy to implement because the LDV is often already used in combination with the derotator as a measuring device. Experiments are carried out to show that this approach is feasible. In addition, several investigations are being performed to analyze how different parameters affect the result, creating reproducible conditions and deriving an alignment approach. By combining the approach developed in this work for determining a rotational deviation with common image-based approaches for determining a translational deviation of the axes, the derotator can be calibrated entirely. In this way, the quality of the measurement with the derotator can be further increased, as calibration errors can be reduced.

Detection of material zones on the surface of a steel-aluminum hybrid component using reflection models and a monochromatic fringe projection profilometry system.

The limits of traditional lightweight engineering are being extended by the development of hybrid components. Lightweight potential is especially high when using dissimilar materials, e.g., a friction-welded steel-aluminum combination. An important factor for the mechanical properties of such a combination is the geometry and location of the joining zone between the materials. The geometry of these objects can be reconstructed by optical triangulation techniques such as fringe projection profilometry. In this paper, we present a method to robustly detect the different material zones on the surface of a hybrid steel-aluminum component. We use reflection models and data from a fringe projection profilometry system. The intensity values and 3D geometry data from the fringe projection system are used to estimate material-specific reflection parameters for each 3D point and detect different material areas based on a global threshold.

Metal Forming Tool Monitoring Based on a 3D Measuring Endoscope Using CAD Assisted Registration.

In order to provide timely, reliable, and comprehensive data for the maintenance of highly stressed geometries in sheet-bulk metal forming tools, this article features a possible setup by combining a 3D measuring endoscope with a two-stage kinematic. The measurement principle is based on the projection of structured light, allowing time-effective measurements of larger areas. To obtain data of proper quality, several hundred measurements are performed which then have to be registered and finally merged into one single point cloud. Factors such as heavy, unwieldy specimens affecting precise alignment. The rotational axes are therefore possibly misaligned and the kinematics and the hand-eye transformation remain uncalibrated. By the use of computer-aided design (CAD) data, registration can be improved, allowing a detailed examination of local features like gear geometries while reducing the sensitivity to detect shape deviations.

Predictor-corrector framework for the sequential assembly of optical systems based on wavefront sensing.

Alignment of optical components is crucial for the assembly of optical systems to ensure their full functionality. In this paper we present a novel predictor-corrector framework for the sequential assembly of serial optical systems. Therein, we use a hybrid optical simulation model that comprises virtual and identified component positions. The hybrid model is constantly adapted throughout the assembly process with the help of nonlinear identification techniques and wavefront measurements. This enables prediction of the future wavefront at the detector plane and therefore allows for taking corrective measures accordingly during the assembly process if a user-defined tolerance on the wavefront error is violated. We present a novel notation for the so-called hybrid model and outline the work flow of the presented predictor-corrector framework. A beam expander is assembled as demonstrator for experimental verification of the framework. The optical setup consists of a laser, two bi-convex spherical lenses each mounted to a five degree-of-freedom stage to misalign and correct components, and a Shack-Hartmann sensor for wavefront measurements.

Fringe projection system for high-temperature workpieces-design, calibration, and measurement.

In the Collaborative Research Centre 1153, Tailored Forming, the geometry of hot measurement objects needs to be captured quickly, areally, and with high precision. The documentation of the hybrid components' shrinkage behavior directly after the forming process can yield insight into the development of residual stresses. In this paper, we present a fringe projection measurement setup designed for the topography measurement of high-temperature steel shafts, comprising two cameras with different lenses and a projector. In order to separate the measurement signal from light by self-radiation, a green bandpass filter is installed in front of the measurement camera's sensor. The optical sensors are protected from the measurement object's temperature and possible scale by a glass panel and a working distance of at least 250 mm. High-resolution measurements are guaranteed due to a telecentric measurement camera and a triangulation angle of about 30°. The triangulation angle requires an additional entocentric calibration camera to provide a highly accurate projector model estimation. Special attention is therefore devoted to the developed calibration routine, the glass panel effect, and the applied distortion models. The quality of the calibration routine is validated by a reference sphere measurement. Furthermore, the geometry data of a red-glowing heating rod (approximately 1020°C) is acquired to demonstrate the performance of the presented system. In future applications, the presented setup will be used with a force-controlled clamping unit to enable secure and position stable topography acquisition of hot measurement objects.

Confocal signal evaluation algorithms for surface metrology: uncertainty and numerical efficiency.

Confocal microscopy is one of the dominating measurement techniques in surface metrology, with an enhanced lateral resolution compared to alternative optical methods. However, the axial resolution in confocal microscopy is strongly dependent on the accuracy of signal evaluation algorithms, which are limited by random noise. Here, we discuss the influence of various noise sources on confocal intensity signal evaluating algorithms, including center-of-mass, parabolic least-square fit, and cross-correlation-based methods. We derive results in closed form for the uncertainty in height evaluation on surface microstructures, also accounting for the number of axially measured intensity values and a threshold that is commonly applied before signal evaluation. The validity of our results is verified by numerical Monte Carlo simulations. In addition, we implemented all three algorithms and analyzed their numerical efficiency. Our results can serve as guidance for a suitable choice of measurement parameters in confocal surface topography measurement, and thus lead to a shorter measurement time in practical applications.

Background oriented schlieren measurement of the refractive index field of air induced by a hot, cylindrical measurement object.

To optically capture the topography of a hot measurement object with high precision, the light deflection by the inhomogeneous refractive index field-induced by the heat transfer from the measurement object to the ambient medium-has to be considered. We used the 2D background oriented schlieren method with illuminated wavelet background, an optical flow algorithm, and Ciddor's equation to quantify the refractive index field located directly above a red-glowing, hot measurement object. A heat transfer simulation has been implemented to verify the magnitude and the shape of the measured refractive index field. Provided that no forced external flow is disturbing the shape of the convective flow originating from the hot object, a laminar flow can be observed directly above the object, resulting in a sharply bounded, inhomogeneous refractive index field.

Using refraction in thick glass plates for optical path length modulation in low coherence interferometry.

In Michelson interferometer setups the standard way to generate different optical path lengths between a measurement arm and a reference arm relies on expensive high precision linear stages such as piezo actuators. We present an alternative approach based on the refraction of light at optical interfaces using a cheap stepper motor with high gearing ratio to control the rotation of a glass plate. The beam path is examined and a relation between angle of rotation and change in optical path length is devised. As verification, an experimental setup is presented, and reconstruction results from a measurement standard are shown. The reconstructed step height from this setup lies within 1.25% of the expected value.

PDMAA Hydrogel Coated U-Bend Humidity Sensor Suited for Mass-Production.

We present a full-polymer respiratory monitoring device suited for application in environments with strong magnetic fields (e.g., during an MRI measurement). The sensor is based on the well-known evanescent field method and consists of a 1 mm plastic optical fiber with a bent region where the cladding is removed and the fiber is coated with poly-dimethylacrylamide (PDMAA). The combination of materials allows for a mass-production of the device by spray-coating and enables integration in disposable medical devices like oxygen masks, which we demonstrate here. We also present results of the application of an autocorrelation-based algorithm for respiratory frequency determination that is relevant for real applications of the device.

Systematic errors on curved microstructures caused by aberrations in confocal surface metrology.

Optical aberrations of microscope lenses are known as a source of systematic errors in confocal surface metrology, which has become one of the most popular methods to measure the surface topography of microstructures. We demonstrate that these errors are not constant over the entire field of view but also depend on the local slope angle of the microstructure and lead to significant deviations between the measured and the actual surface. It is shown by means of a full vectorial high NA numerical model that a change in the slope angle alters the shape of the intensity depth response of the microscope and leads to a shift of the intensity peak of up to several hundred nanometers. Comparative experimental data are presented which support the theoretical results. Our studies allow for correction of optical aberrations and, thus, increase the accuracy in profilometric measurements.

Flexible, fast, and low-cost production process for polymer based diffractive optics.

The generation of diffractive optical elements often requires time and cost consuming production techniques such as photolithography. Especially in research and development, small series of diffractive microstructures are needed and flexible and cost effective fabrication techniques are desirable to enable the fabrication of versatile optical elements on a short time scale. In this work, we introduce a novel process chain for fabrication of diffractive optical elements in various polymers. It is based on a maskless lithography process step, where a computer generated image of the optical element is projected via a digital mirror device and a microscope setup onto a silicon wafer coated with photosensitive resist. In addition, a stitching process allows us to microstructure a large area on the wafer. After development, a soft stamp of the microstructure is made from Polydimethylsiloxane, which is used as a mold for the subsequent hot embossing process, where the final diffractive optical element is replicated into thermoplastic polymer. Experimental results are presented, which demonstrate the applicability of the process.

Cladded self-written multimode step-index waveguides using a one-polymer approach.

Low-loss optical-coupling structures are highly relevant for applications in fields as diverse as information and communication technologies, integrated circuits, or flexible and highly-functional polymer sensor networks. For this suitable and reliable production methods are crucial. Self-written waveguides are an interesting solution. In this work, we present a simple and efficient one-polymer approach for self-written optical connections between light-guiding structures such as single-mode and multi-mode optical fibers or waveguides that relies on self focusing of the light inside a photopolymerizing mixture. The optical connections are produced in a two-step process by writing into monomer resin using cw laser light in the blue wavelength range and subsequent UV curing. Since only one photopolymerizing resin is required, we reduced the fabrication complexity compared to previous approaches to obtain a waveguide embedded in a rigid cladding material. We discuss the production method, the results obtained as function of relevant process parameters such as writing speed or curing time, and evaluate optical properties and coupling efficiencies.

High-precision surface measurement with an automated multiangle low coherence interferometer.

We propose a novel measurement system based on a low coherence Michelson interferometer and six-axis hexapod platform to accurately measure structures with high aspect ratio using different tilt angles of the measured surface. In order to realize automatic measurement, the system is designed to automatically perform autofocusing, adjust the tilt angle of the test surface, make surface measurements, and merge the measurement data sets. Due to certain topography, e.g., structures with high aspect ratio, the interferometer cannot obtain enough reflected light to evaluate the height information in some areas of the test surface. For this reason, we developed a measurement system that uses measurements from different tilt angles of the test surface and stitching algorithms to realize a complete surface measurement data set. The performance of the proposed measurement system is evaluated experimentally and compared to the results of measurements using a perthometer.

Measurement Uncertainty of Microscopic Laser Triangulation on Technical Surfaces.

Laser triangulation is widely used to measure three-dimensional structure of surfaces. The technique is suitable for macroscopic and microscopic surface measurements. In this paper, the measurement uncertainty of laser triangulation is investigated on technical surfaces for microscopic measurement applications. Properties of technical surfaces are, for example, reflectivity, surface roughness, and the presence of scratches and pores. These properties are more influential in the microscopic laser triangulation than in the macroscopic one. In the Introduction section of this paper, the measurement uncertainty of laser triangulation is experimentally investigated for 13 different specimens. The measurements were carried out with and without a laser speckle reducer. In the Materials and Methods section of this paper, the surfaces of the 13 specimens are characterized in order to be able to find correlations between the surface properties and the measurement uncertainty. The last section of this paper describes simulations of the measurement uncertainty, which allow for the calculation of the measurement uncertainty with only one source of uncertainty present. The considerations in this paper allow for the assessment of the measurement uncertainty of laser triangulation on any technical surface when some surface properties, such as roughness, are known.