A Pixel Design of a Branching Ultra-Highspeed Image Sensor.

A burst image sensor named Hanabi, meaning fireworks in Japanese, includes a branching CCD and multiple CMOS readout circuits. The sensor is backside-illuminated with a light/charge guide pipe to minimize the temporal resolution by suppressing the horizontal motion of signal carriers. On the front side, the pixel has a guide gate at the center, branching to six first-branching gates, each bifurcating to second-branching gates, and finally connected to 12 (=6×2) floating diffusions. The signals are either read out after an image capture operation to replay 12 to 48 consecutive images, or continuously transferred to a memory chip stacked on the front side of the sensor chip and converted to digital signals. A CCD burst image sensor enables a noiseless signal transfer from a photodiode to the in-situ storage even at very high frame rates. However, the pixel count conflicts with the frame count due to the large pixel size for the relatively large in-pixel CCD memory elements. A CMOS burst image sensor can use small trench-type capacitors for memory elements, instead of CCD channels. However, the transfer noise from a floating diffusion to the memory element increases in proportion to the square root of the frame rate. The Hanabi chip overcomes the compromise between these pros and cons.

Toward the Super Temporal Resolution Image Sensor with a Germanium Photodiode for Visible Light.

The theoretical temporal resolution limit tT of a silicon photodiode (Si PD) is 11.1 ps. We call "super temporal resolution" the temporal resolution that is shorter than that limit. To achieve this resolution, Germanium is selected as a candidate material for the photodiode (Ge PD) for visible light since the absorption coefficient of Ge for the wavelength is several tens of times higher than that of Si, allowing a very thin PD. On the other hand, the saturation drift velocity of electrons in Ge is about 2/3 of that in Si. The ratio suggests an ultra-short propagation time of electrons in the Ge PD. However, the diffusion coefficient of electrons in Ge is four times higher than that of Si. Therefore, Monte Carlo simulations were applied to analyze the temporal resolution of the Ge PD. The estimated theoretical temporal resolution limit is 0.26 ps, while the practical limit is 1.41 ps. To achieve a super temporal resolution better than 11.1 ps, the driver circuit must operate at least 100 GHz. It is thus proposed to develop, at first, a short-wavelength infrared (SWIR) ultra-high-speed image sensor with a thicker and wider Ge PD, and then gradually decrease the size along with the progress of the driver circuits.

Light-In-Flight Imaging by a Silicon Image Sensor: Toward the Theoretical Highest Frame Rate.

Light in flight was captured by a single shot of a newly developed backside-illuminated multi-collection-gate image sensor at a frame interval of 10 ns without high-speed gating devices such as a streak camera or post data processes. This paper reports the achievement and further evolution of the image sensor toward the theoretical temporal resolution limit of 11.1 ps derived by the authors. The theoretical analysis revealed the conditions to minimize the temporal resolution. Simulations show that the image sensor designed following the specified conditions and fabricated by existing technology will achieve a frame interval of 50 ps. The sensor, 200 times faster than our latest sensor will innovate advanced analytical apparatuses using time-of-flight or lifetime measurements, such as imaging TOF-MS, FLIM, pulse neutron tomography, PET, LIDAR, and more, beyond these known applications.

An Image Signal Accumulation Multi-Collection-Gate Image Sensor Operating at 25 Mfps with 32 × 32 Pixels and 1220 In-Pixel Frame Memory.

The paper presents an ultra-high-speed image sensor for motion pictures of reproducible events emitting very weak light. The sensor is backside-illuminated. Each pixel is equipped with multiple collection gates (MCG) at the center of the front side. Each collection gate is connected to an in-pixel large memory unit, which can accumulate image signals captured by repetitive imaging. The combination of the backside illumination, image signal accumulation, and slow readout from the in-pixel signal storage after an image capturing operation offers a very high sensitivity. Pipeline signal transfer from the the multiple collection gates (MCG) to the in-pixel memory units enables the sensor to achieve a large frame count and a very high frame rate at the same time. A test sensor was fabricated with a pixel count of 32 × 32 pixels. Each pixel is equipped with four collection gates, each connected to a memory unit with 305 elements; thus, with a total frame count of 1220 (305 × 4) frames. The test camera achieved 25 Mfps, while the sensor was designed to operate at 50 Mfps.

Toward the Ultimate-High-Speed Image Sensor: From 10 ns to 50 ps.

The paper summarizes the evolution of the Backside-Illuminated Multi-Collection-Gate (BSI MCG) image sensors from the proposed fundamental structure to the development of a practical ultimate-high-speed silicon image sensor. A test chip of the BSI MCG image sensor achieves the temporal resolution of 10 ns. The authors have derived the expression of the temporal resolution limit of photoelectron conversion layers. For silicon image sensors, the limit is 11.1 ps. By considering the theoretical derivation, a high-speed image sensor designed can achieve the frame rate close to the theoretical limit. However, some of the conditions conflict with performance indices other than the frame rate, such as sensitivity and crosstalk. After adjusting these trade-offs, a simple pixel model of the image sensor is designed and evaluated by simulations. The results reveal that the sensor can achieve a temporal resolution of 50 ps with the existing technology.

The Theoretical Highest Frame Rate of Silicon Image Sensors.

The frame rate of the digital high-speed video camera was 2000 frames per second (fps) in 1989, and has been exponentially increasing. A simulation study showed that a silicon image sensor made with a 130 nm process technology can achieve about 1010 fps. The frame rate seems to approach the upper bound. Rayleigh proposed an expression on the theoretical spatial resolution limit when the resolution of lenses approached the limit. In this paper, the temporal resolution limit of silicon image sensors was theoretically analyzed. It is revealed that the limit is mainly governed by mixing of charges with different travel times caused by the distribution of penetration depth of light. The derived expression of the limit is extremely simple, yet accurate. For example, the limit for green light of 550 nm incident to silicon image sensors at 300 K is 11.1 picoseconds. Therefore, the theoretical highest frame rate is 90.1 Gfps (about 1011 fps).

von Kármán vortex street within an impacting drop.

The splashing of a drop impacting onto a liquid pool produces a range of different sized microdroplets. At high impact velocities, the most significant source of these droplets is a thin liquid jet emerging at the start of the impact from the neck that connects the drop to the pool. We use ultrahigh-speed video imaging in combination with high-resolution numerical simulations to show how this ejecta gives way to irregular splashing. At higher Reynolds numbers, its base becomes unstable, shedding vortex rings into the liquid from the free surface in an axisymmetric von Kármán vortex street, thus breaking the ejecta sheet as it forms.

Toward 100 Mega-frames per second: design of an ultimate ultra-high-speed image sensor.

Our experience in the design of an ultra-high speed image sensor targeting the theoretical maximum frame rate is summarized. The imager is the backside illuminated in situ storage image sensor (BSI ISIS). It is confirmed that the critical factor limiting the highest frame rate is the signal electron transit time from the generation layer at the back side of each pixel to the input gate to the in situ storage area on the front side. The theoretical maximum frame rate is estimated at 100 Mega-frames per second (Mfps) by transient simulation study. The sensor has a spatial resolution of 140,800 pixels with 126 linear storage elements installed in each pixel. The very high sensitivity is ensured by application of backside illumination technology and cooling. The ultra-high frame rate is achieved by the in situ storage image sensor (ISIS) structure on the front side. In this paper, we summarize technologies developed to achieve the theoretical maximum frame rate, including: (1) a special p-well design by triple injections to generate a smooth electric field backside towards the collection gate on the front side, resulting in much shorter electron transit time; (2) design technique to reduce RC delay by employing an extra metal layer exclusively to electrodes responsible for ultra-high speed image capturing; (3) a CCD specific complementary on-chip inductance minimization technique with a couple of stacked differential bus lines.

Ultra-high-speed image signal accumulation sensor.

Averaging of accumulated data is a standard technique applied to processing data with low signal-to-noise ratios (SNR), such as image signals captured in ultra-high-speed imaging. The authors propose an architecture layout of an ultra-high-speed image sensor capable of on-chip signal accumulation. The very high frame rate is enabled by employing an image sensor structure with a multi-folded CCD in each pixel, which serves as an in situ image signal storage. The signal accumulation function is achieved by direct connection of the first and the last storage elements of the in situ storage CCD. It has been thought that the multi-folding is achievable only by driving electrodes with complicated and impractical layouts. Simple configurations of the driving electrodes to overcome the difficulty are presented for two-phase and four-phase transfer CCD systems. The in situ storage image sensor with the signal accumulation function is named Image Signal Accumulation Sensor (ISAS).

Phototriggering system for an ultrahigh-speed video microscopy.

A phototrigger system is developed as a part of a video microscope mounting an ultrahigh-speed video camera capable of image capturing at frame rates as high as 1x10(6) framess. The extremely high frame rate is achieved by implementing in situ image storage. A distinguished feature of the camera is the on-chip overwriting mechanism that allows to keep in storage the latest image sequence of 103 frames; the old signals are continuously drained out of the storage. The trigger system is designed to synchronize recording operations with an occurrence of a target event within the limited image capturing duration. The target event is detected through a sudden change in the output of a sensor mounted to an optical port of the microscope. To reduce noise contribution, a two-sensor architecture is implemented. One sensor detects the target event while the one produces a reference signal used for noise reduction. Both sensors are connected to the same optical port by using a specially designed beam splitting unit. To provide high sensitivity, avalanche photodiodes are used as photoelements. System evaluation shows that its sensitivity is high and response time is less than 3 mus. This is sufficiently fast for high-speed video-microscopy observations at 1x10(6) frames/s when using a video camera with a storage of 103 frames. As an example, the system was used in a microscopic observation of a soap film collapse.