A Reconfigurable Bipolar Image Sensor for High-Efficiency Dynamic Vision Recognition.

Dynamic vision perception and processing (DVPP) is in high demand by booming edge artificial intelligence. However, existing imaging systems suffer from low efficiency or low compatibility with advanced machine vision techniques. Here, we propose a reconfigurable bipolar image sensor (RBIS) for in-sensor DVPP based on a two-dimensional WSe2/GeSe heterostructure device. Owing to the gate-tunable and reversible built-in electric field, its photoresponse shows bipolarity as being positive or negative. High-efficiency DVPP incorporating front-end RBIS and back-end CNN is then demonstrated. It shows a high recognition accuracy of over 94.9% on the derived DVS128 data set and requires much fewer neural network parameters than that without RBIS. Moreover, we demonstrate an optimized device with a vertically stacked structure and a stable nonvolatile bipolarity, which enables more efficient DVPP hardware. Our work demonstrates the potential of fabricating DVPP devices with a simple structure, high efficiency, and outputs compatible with advanced algorithms.

Rolling up 2D WSe2 Nanosheets to 1D Anisotropic Nanoscrolls for Polarization-Sensitive Photodetectors.

The intrinsic low-symmetry crystal structures or external geometries of low-dimensional materials are crucial for polarization-sensitive photodetection. However, these inherently anisotropic materials are limited in variety, and their anisotropy is confined to specific crystal directions. Transforming 2D semiconductors, such as WSe2, from isotropic 2D nanosheets into anisotropic 1D nanoscrolls expands their application in polarization photodetection. Despite this considerable potential, research on polarization photodetection based on nanoscrolls remains scarce. Here, the uniform crystalline orientation of WSe2 nanoscrolls is achieved conveniently and efficiently by applying ethanol droplets to vapor deposition-grown bilayer WSe2 nanosheets. Angle-resolved polarized Raman spectroscopy of WSe2 nanoscrolls demonstrates vibrational anisotropy. Photodetectors based on these nanoscrolls show competitive overall performance with a broadband detection range from 405 to 808 nm, a competitive on/off ratio of ≈900, a high detectivity of 3.4 × 108 Jones, and a fast response speed of ≈30 ms. Additionally, WSe2 nanoscroll-based photodetectors exhibit strong polarization-sensitive detection with a maximum dichroic ratio of 1.5. More interestingly, due to high photosensitivity, the WSe2 nanoscroll detectors can easily record sequential puppy images. This work reveals the potential of WSe2 nanoscrolls as excellent polarization-sensitive photodetectors and provides new insights into the development of high-performance optoelectronic devices.

Comparison of needle and conventional arthroscopy for visualisation of predefined anatomical structures of the knee joint: a feasibility study in human cadavers and patients.

BACKGROUND: In terms of the optics used for Knee arthroscopy, a large number of different endoscopes are currently available. However, the use of the 30° optics in knee arthroscopy has been established as the standard procedure for many years. As early as the 1990s, needle arthroscopy was occasionally used as a diagnostic tool. In addition to the development of conventional optics technology in terms of camera and resolution, needle arthroscopes are now available with chip-on-tip image sensor technology. To date, no study has compared the performance of this kind of needle arthroscopy versus standard arthroscopy in the clinical setting in terms of the visibility of anatomical landmarks. In this monocentric prospective feasibility study, our aim was to evaluate predefined anatomical landmarks of the knee joint using needle arthroscopy (0° optics) and conventional knee arthroscopy (30° optics) and compare their performance during knee surgery. METHODS: Examinations were performed on eight cadavers and seven patients who required elective knee arthroscopy. Two surgeons independently performed the examinations on these 15 knee joints, so that we were able to compare a total of 30 examinations. The focus was on the anatomical landmarks that could be visualized during a conventional diagnostic knee arthroscopy procedure. The quality of visibility was evaluated using a questionnaire. RESULTS: In summary, the average visibility for all the anatomic landmarks was rated 4.98/ 5 for the arthroscopy using 30° optics. For needle arthroscopy, an average score of 4.89/ 5 was obtained. Comparatively, the needle arthroscope showed slightly limited visibility of the retropatellar gliding surface in eight (4.5/ 5 vs. 5/ 5), medial rim of the patella in four (4.85/ 5 vs. 5/ 5), and suprapatellar recess in four (4.83/ 5 vs. 5/ 5) cases. Needle arthroscopy was slightly better at visualizing the posterior horn of the medial meniscus in four knee joints (4.9/ 5 vs. 4.85/ 5). CONCLUSION: Needle arthroscopy is a promising technology with advantages in terms of minimally invasive access and good visibility of anatomical landmarks. However, it also highlights some limitations, particularly in cases with challenging anatomy or the need for a wide field of view.

Synchronous Driving Method for Stitching Pixel Arrays Based on an Adaptive Correction Technique.

With the application of stitching technology in large-pixel-array CMOS image sensors, the problem of non-synchronized output signals from pixel array bilateral driver circuits has become progressively more serious and has led to the DC perforation of bilateral driver circuits, while conventional clock tree synchronization design methodology does not apply to stitching technology. Therefore, this paper analyses reasons for the inconsistency in the output signals of bilateral driving circuits and proposes a synchronous driving method applicable to stitching pixel arrays based on the idea of on-chip output signal delay detection and calibration. This method detects and corrects the non-synchrony of the row driver output signals on both sides according to changes in the operating environment of the chip. This method is characterized by a simple structure and high reliability. Finally, based on the 55 nm stitching process, simulations are carried out in a CMOS image sensor with a chip area of 77 mm × 84 mm to verify that this method is feasible. This large image sensor with a 150 M pixel array has a frame rate of over 10 FPS.

An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms.

Aiming at the problems of the poor robustness and universality of traditional contour matching algorithms in engineering applications, a method for improving the surface defect detection of industrial products based on contour matching algorithms is detailed in this paper. Based on the image pyramid optimization method, a three-level matching method is designed, which can quickly obtain the candidate pose of the target contour at the top of the image pyramid, combining the integral graph and the integration graph acceleration strategy based on weak classification. It can quickly obtain the rough positioning and rough angle of the target contour, which greatly improves the performance of the algorithm. In addition, to solve the problem that a large number of duplicate candidate points will be generated when the target candidate points are expanded, a method to obtain the optimal candidate points in the neighborhood of the target candidate points is designed, which can guarantee the matching accuracy and greatly reduce the calculation amount. In order to verify the effectiveness of the algorithm, functional test experiments were designed for template building function and contour matching function, including uniform illumination condition, nonlinear condition and contour matching detection under different conditions. The results show that: (1) Under uniform illumination conditions, the detection accuracy can be maintained at about 93%. (2) Under nonlinear illumination conditions, the detection accuracy can be maintained at about 91.84%. (3) When there is an external interference source, there will be a false detection or no detection, and the overall defect detection rate remains above 94%. It is verified that the proposed method can meet the application requirements of common defect detection, and has good robustness and meets the expected functional requirements of the algorithm, providing a strong technical guarantee and data support for the design of embedded image sensors in the later stage.

Millimeter-Wave Band Electro-Optical Imaging System Using Polarization CMOS Image Sensor and Amplified Optical Local Oscillator Source.

In this study, we developed and demonstrated a millimeter-wave electric field imaging system using an electro-optic crystal and a highly sensitive polarization measurement technique using a polarization image sensor, which was fabricated using a 0.35-µm standard CMOS process. The polarization image sensor was equipped with differential amplifiers that amplified the difference between the 0° and 90° pixels. With the amplifier, the signal-to-noise ratio at low incident light levels was improved. Also, an optical modulator and a semiconductor optical amplifier were used to generate an optical local oscillator (LO) signal with a high modulation accuracy and sufficient optical intensity. By combining the amplified LO signal and a highly sensitive polarization imaging system, we successfully performed millimeter-wave electric field imaging with a spatial resolution of 30×60 µm at a rate of 1 FPS, corresponding to 2400 pixels/s.

Exposure Time Control Method for Higher Intermediate Frequency in Optical Heterodyne Imaging and Its Application to Electric-Field Imaging Based on Electro-Optic Effect.

We propose and demonstrate a method for equivalent time sampling using image sensors to selectively detect only the target frequency. Shortening the exposure time of the image sensor and using equivalent time sampling allows for the detection of frequency components that are higher than the frame rate. However, the imaging system in our previous work was also sensitive to the frequency component at 1/4 of the frame rate. In this study, we control the phase relationship between the exposure time and observed signal by inserting an additional interval once every four frames to detect the target frequency selectively. With this technique, we conducted electric field imaging based on the electro-optic effect under high noise conditions in the low-frequency band to which the conventional method is sensitive. The results demonstrated that the proposed method improved the signal-to-noise ratio.

Using a Slit to Suppress Optical Aberrations in Laser Triangulation Sensors.

In this paper, we present a laser triangulation sensor to measure the distance between the sensor and an object without contact using a diffraction slit rather than a traditional lens. We show that by replacing the lens with a slit, we can exploit the resulting diffraction pattern to have finer and yet simpler image analysis, yielding better estimation of the distance to the object. To test our hypothesis, we build a precision position table and a laser triangulation sensor, generate large data sets to test different estimation algorithms on various materials, and compare data acquisition using a traditional lens versus using a slit. We show that position estimation when using a slit is both more precise and more accurate than comparable methods using a lens.

The Effect of Pixel Design and Operation Conditions on Linear Output Range of 4T CMOS Image Sensors.

We analyze several factors that affect the linear output range of CMOS image sensors, including charge transfer time, reset transistor supply voltage, the capacitance of integration capacitor, the n-well doping of the pinned photodiode (PPD) and the output buffer. The test chips are fabricated with 0.18 µm CMOS image sensor (CIS) process and comprise six channels. Channels B1 and B2 are 10 µm pixels and channels B3-B6 are 20 µm pixels, with corresponding pixel arrays of 1 × 2560 and 1 × 1280 respectively. The floating diffusion (FD) capacitance varies from 10 fF to 23.3 fF, and two different designs were employed for the n-well doping in PPD. The experimental results indicate that optimizing the FD capacitance and PPD design can enhance the linear output range by 37% and 32%, respectively. For larger pixel sizes, extending the transfer gate (TG) sampling time leads to an increase of over 60% in the linear output range. Furthermore, optimizing the design of the output buffer can alleviate restrictions on the linear output range. The lower reset voltage for noise reduction does not exhibit a significant impact on the linear output range. Furthermore, these methods can enhance the linear output range without significantly amplifying the readout noise. These findings indicate that the linear output range of pixels is not only influenced by pixel design but also by operational conditions. Finally, we conducted a detailed analysis of the impact of PPD n-well doping concentration and TG sampling time on the linear output range. This provides designers with a clear understanding of how nonlinearity is introduced into pixels, offering valuable insight in the design of highly linear pixels.

Three-Stage Interpolation Method for Demosaicking Monochrome Polarization DoFP Images.

The emergence of polarization image sensors presents both opportunities and challenges for real-time full-polarization reconstruction in scene imaging. This paper presents an innovative three-stage interpolation method specifically tailored for monochrome polarization image demosaicking, emphasizing both precision and processing speed. The method introduces a novel linear interpolation model based on polarization channel difference priors in the initial two stages. To enhance results through bidirectional interpolation, a continuous adaptive edge detection method based on variance differences is employed for weighted averaging. In the third stage, a total intensity map, derived from the previous two stages, is integrated into a residual interpolation process, thereby further elevating estimation precision. The proposed method undergoes validation using publicly available advanced datasets, showcasing superior performance in both global parameter evaluations and local visual details when compared with existing state-of-the-art techniques.

Investigating a Machine Learning Approach to Predicting White Pixel Defects in Wafers-A Case Study of Wafer Fabrication Plant F.

CMOS image sensor (CIS) semiconductor products are integral to mobile phones and photographic devices, necessitating ongoing enhancements in efficiency and quality for superior photographic outcomes. The presence of white pixels serves as a crucial metric for assessing CIS product performance, primarily arising from metal impurity contamination during the wafer production process or from defects introduced by the grinding blade process. While immediately addressing metal impurity contamination during production presents challenges, refining the handling of defects attributed to grinding blade processing can notably mitigate white pixel issues in CIS products. This study zeroes in on silicon wafer manufacturers in Taiwan, analyzing white pixel defects reported by customers and leveraging machine learning to pinpoint and predict key factors leading to white pixel defects from grinding blade operations. Such pioneering practical studies are rare. The findings reveal that the classification and regression tree (CART) and random forest (RF) models deliver the most accurate predictions (95.18%) of white pixel defects caused by grinding blade operations in a default parameter setting. The analysis further elucidates critical factors like grinding load and torque, vital for the genesis of white pixel defects. The insights garnered from this study aim to arm operators with proactive measures to diminish the potential for customer complaints.

Characterization of Partial Discharges in Dielectric Oils Using High-Resolution CMOS Image Sensor and Convolutional Neural Networks.

In this work, an exhaustive analysis of the partial discharges that originate in the bubbles present in dielectric mineral oils is carried out. To achieve this, a low-cost, high-resolution CMOS image sensor is used. Partial discharge measurements using that image sensor are validated by a standard electrical detection system that uses a discharge capacitor. In order to accurately identify the images corresponding to partial discharges, a convolutional neural network is trained using a large set of images captured by the image sensor. An image classification model is also developed using deep learning with a convolutional network based on a TensorFlow and Keras model. The classification results of the experiments show that the accuracy achieved by our model is around 95% on the validation set and 82% on the test set. As a result of this work, a non-destructive diagnosis method has been developed that is based on the use of an image sensor and the design of a convolutional neural network. This approach allows us to obtain information about the state of mineral oils before breakdown occurs, providing a valuable tool for the evaluation and maintenance of these dielectric oils.

Presenting a Multispectral Image Sensor for Quantification of Total Polyphenols in Low-Temperature Stressed Tomato Seedlings Using Hyperspectral Imaging.

Hyperspectral imaging was used to predict the total polyphenol content in low-temperature stressed tomato seedlings for the development of a multispectral image sensor. The spectral data with a full width at half maximum (FWHM) of 5 nm were merged to obtain FWHMs of 10 nm, 25 nm, and 50 nm using a commercialized bandpass filter. Using the permutation importance method and regression coefficients, we developed the least absolute shrinkage and selection operator (Lasso) regression models by setting the band number to ≥11, ≤10, and ≤5 for each FWHM. The regression model using 56 bands with an FWHM of 5 nm resulted in an R2 of 0.71, an RMSE of 3.99 mg/g, and an RE of 9.04%, whereas the model developed using the spectral data of only 5 bands with a FWHM of 25 nm (at 519.5 nm, 620.1 nm, 660.3 nm, 719.8 nm, and 980.3 nm) provided an R2 of 0.62, an RMSE of 4.54 mg/g, and an RE of 10.3%. These results show that a multispectral image sensor can be developed to predict the total polyphenol content of tomato seedlings subjected to low-temperature stress, paving the way for energy saving and low-temperature stress damage prevention in vegetable seedling production.

Synergistic Radiation Effects in PPD CMOS Image Sensors Induced by Neutron Displacement Damage and Gamma Ionization Damage.

The synergistic effects on the 0.18 µm PPD CISs induced by neutron displacement damage and gamma ionization damage are investigated. The typical characterizations of the CISs induced by the neutron displacement damage and gamma ionization damage are presented separately. The CISs are irradiated by reactor neutron beams up to 1 × 1011 n/cm2 (1 MeV neutron equivalent fluence) and 60Co γ-rays up to the total ionizing dose level of 200 krad(Si) with different sequential order. The experimental results show that the mean dark signal increase in the CISs induced by reactor neutron radiation has not been influenced by previous 60Co γ-ray radiation. However, the mean dark signal increase in the CISs induced by 60Co γ-ray radiation has been remarkably influenced by previous reactor neutron radiation. The synergistic effects on the PPD CISs are discussed by combining the experimental results and the TCAD simulation results of radiation damage.

Enhancement of Optical and Chemical Resistance Properties with a Novel Yellow Quinophthalone Derivative for Image Sensor Colorants.

A novel quinophthalone derivative, 4,5,6,7-tetrachloro-2-(2-(3-hydroxy-1-oxo-1H-cyclopenta[b]naphthalen-2-yl)quinolin-4-yl)isoindoline-1,3-dione (TCHCQ), was designed and synthesized as a yellow colorant additive for green color filters in image sensors. The characteristics of the new material were evaluated in terms of optical, thermal, and chemical properties under solution and color filter film conditions. TCHCQ exhibited a significantly enhanced molar extinction coefficient in solution, being 1.21 times higher than that of the commercially used yellow colorant Y138. It also demonstrated excellent thermal stability, with a decomposition temperature (Td) exceeding 450 °C. Utilizing the nano-pigmentation process, TCHCQ was used to prepare nano-sized particles with an excellent average size of 35 nm. This enabled the fabrication of a color filter film with outstanding properties. The optical properties of the produced film revealed outstanding yellow colorant transmittance of 0.97% at 435 nm and 91.2% at 530 nm. The color filter film exhibited similar optical and thermal stability to Y138, with an improved chemical stability, as evidenced by a ΔEab value of 0.52. The newly synthesized TCHCQ is considered a promising candidate for use as a yellow colorant additive in image sensor color filters, demonstrating superior optical, thermal, and chemical stability.

High-Speed Fully Differential Two-Step ADC Design Method for CMOS Image Sensor.

The application requirements of high frame rate CMOS image sensors (CIS) in the industry have not been satisfied due to the speed limitations in traditional single-slope and serial two-step analog-to-digital converters (ADCs). In this paper, a high-speed fully differential two-step ADC design method for CIS was proposed. The proposed method was based on differential ramp and time-to-digital conversion (TDC) technology. A parallel conversion mode was formed that is different from serial conversion, and the robustness of the system was ensured due to the existence of differential ramps. Aiming at the inconsistency between traditional TDC technology and single-slope ADC, a TDC technology based on level coding was proposed. The proposed technology achieves the TDC in the last clock cycle of analog-to-digital conversion, and realized a two-step conversion process at another level. This paper presents a complete circuit design, layout design, and test verification of the proposed design method based on the 55 nm 1P4M CMOS experimental platform. Under the design environment of the analog voltage of 3.3 V, the digital voltage of 1.2 V, the clock frequency of 100 MHz, and a dynamic input range of 1.6 V, this design was a 12-bit ADC with a conversion time of 480 ns, column-level power consumption of 62 µW, differential nonlinearity (DNL) of +0.6/-0.6 LSB, and integral nonlinearity (INL) of +1.2/-1.4 LSB. Furthermore, it achieved a signal-to-noise distortion ratio (SNDR) of 70.08 dB. The proposed design provided a large area array with a high frame rate, and compared with the existing advanced single-slope ADC, its conversion speed increased by more than 52%. It provides an effective solution for the implementation of high frame frequency CIS.

A Numerical Method of Aligning the Optical Stacks for All Pixels.

Reducing performance verification time is significant in product launch and production costs. This is especially true because aligning the optical stacks of off-axis pixels is a time-consuming task, but it is important to maintain sensitivity. In this paper, a numerical method to align the optical stacks of off-axis pixels is suggested in order to reduce performance test time. The components of the numerical method are the optical stack height, refractive index, and chief ray angle in order to calculate the optical stacks' optimal shift distance. The proposed method was investigated to confirm effectiveness by using optical simulation. The sub-micron backside illumination (BSI) pixels were simulated, having 2 × 2 microlens, quad-color filter array, and in-pixel deep trench isolation (DTI). Moreover, the proposed method was evaluated for various pixel pitches, microlens shapes, and CRAs. As a result, the optical stacks were optimized by using the numerical method and validated via optical simulation. Therefore, the proposed numerical method is expected to help reduce the time and cost.

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction.

Images produced by CMOS sensors may contain defective pixels due to noise, manufacturing errors, or device malfunction, which must be detected and corrected at early processing stages in order to produce images that are useful to human users and image-processing or machine-vision algorithms. This paper proposes a defective pixel detection and correction algorithm and its implementation using CMOS analog circuits, which are integrated with the image sensor at the pixel and column levels. During photocurrent integration, the circuit detects defective values in parallel at each pixel using simple arithmetic operations within a neighborhood. At the image-column level, the circuit replaces the defective pixels with the median value of their neighborhood. To validate our approach, we designed a 128×128-pixel imager in a 0.35µm CMOS process, which integrates our defective-pixel detection/correction circuits and processes images at 694 frames per second, according to post-layout simulations. Operating at that frame rate, our proposed algorithm and its CMOS implementation produce better results than current state-of-the-art algorithms: it achieves a Peak Signal to Noise Ratio (PSNR) and Image Enhancement Factor (IEF) of 45 dB and 198.4, respectively, in images with 0.5% random defective pixels, and a PSNR of 44.4 dB and IEF of 194.2, respectively, in images with 1.0% random defective pixels.

Backdoor Attack on Deep Neural Networks Triggered by Fault Injection Attack on Image Sensor Interface.

A backdoor attack is a type of attack method that induces deep neural network (DNN) misclassification. The adversary who aims to trigger the backdoor attack inputs the image with a specific pattern (the adversarial mark) into the DNN model (backdoor model). In general, the adversary mark is created on the physical object input to an image by capturing a photo. With this conventional method, the success of the backdoor attack is not stable because the size and position change depending on the shooting environment. So far, we have proposed a method of creating an adversarial mark for triggering backdoor attacks by means of a fault injection attack on the mobile industry processor interface (MIPI), which is the image sensor interface. We propose the image tampering model, with which the adversarial mark can be generated in the actual fault injection to create the adversarial mark pattern. Then, the backdoor model was trained with poison data images, which the proposed simulation model created. We conducted a backdoor attack experiment using a backdoor model trained on a dataset containing 5% poison data. The clean data accuracy in normal operation was 91%; nevertheless, the attack success rate with fault injection was 83%.

Modeling Signal-to-Noise Ratio of CMOS Image Sensors with a Stochastic Approach under Non-Stationary Conditions.

A stochastic model for characterizing the conversion gain of Active Pixel Complementary metal-oxide-semiconductor (CMOS) image sensors (APS), assuming stationary conditions was recently presented in this journal. In this study, we extend the stochastic approach to non-stationary conditions. Non-stationary conditions occur in gated imaging applications. This new stochastic model, which is based on fundamental physical considerations, enlightens us with new insights into gated CMOS imaging, regardless of the sensor. The Signal-to-Noise Ratio (SNR) is simulated, allowing optimized performance. The conversion gain should be determined under stationary conditions.