A Hybrid Approach to Industrial Augmented Reality Using Deep Learning-Based Facility Segmentation and Depth Prediction.

Typical AR methods have generic problems such as visual mismatching, incorrect occlusions, and limited augmentation due to the inability to estimate depth from AR images and attaching the AR markers onto physical objects, which prevents the industrial worker from conducting manufacturing tasks effectively. This paper proposes a hybrid approach to industrial AR for complementing existing AR methods using deep learning-based facility segmentation and depth prediction without AR markers and a depth camera. First, the outlines of physical objects are extracted by applying a deep learning-based instance segmentation method to the RGB image acquired from the AR camera. Simultaneously, a depth prediction method is applied to the AR image to estimate the depth map as a 3D point cloud for the detected object. Based on the segmented 3D point cloud data, 3D spatial relationships among the physical objects are calculated, which can assist in solving the visual mismatch and occlusion problems properly. In addition, it can deal with a dynamically operating or a moving facility, such as a robot-the conventional AR cannot do so. For these reasons, the proposed approach can be utilized as a hybrid or complementing function to existing AR methods, since it can be activated whenever the industrial worker requires handing of visual mismatches or occlusions. Quantitative and qualitative analyses verify the advantage of the proposed approach compared with existing AR methods. Some case studies also prove that the proposed method can be applied not only to manufacturing but also to other fields. These studies confirm the scalability, effectiveness, and originality of this proposed approach.

Translocation of DNA and protein through a sequentially polymerized polyurea nanopore.

Here, we investigated the translocation of biomolecules, such as DNA and protein, through a sequentially polymerized polyurea nanopore, with a thin (<10 nm) polymer membrane of uniform thickness. The polyurea membrane was synthesized by molecular layer deposition using p-phenylenediisocyanate (PDI) and p-phenylenediamine (PDA) as sequential precursors. The membrane exhibited a hydrophobic surface with a highly negative surface charge density (-51 mC m-2 at pH 8). It was particularly noted that the high surface charge density of the membrane resulted in a highly developed electro-osmotic flow which, in turn, strongly influenced the capture probability of biomolecules, depending on the balance between the electro-osmotic and electrophoretic forces. For instance, the capture frequency of negatively charged DNA was demonstrated to be quite low, since these two forces more or less cancelled each other, whereas that of positively charged MDM2 was much higher, since these two forces were additive. We also identified that the mean translocation time of MDM2 through the polyurea nanopore was 26.1 ± 3.7 µs while that of the SiN nanopore was 14.2 ± 2.0 µs, hence suggesting that the enhanced electrostatic interaction between positively charged MDM2 and the negatively charged pore surface affects the translocation speed.

The dynamics of electron beam scattering on metal membranes: nanopore formation in metal membranes using transmission electron microscopy.

The dynamics of nanopore formation in metal membranes using the highly focused and high energy electron beams (e-beams) of transmission electron microscopy instruments was investigated. Various metals such as Al, Ti, Cr, Cu, and Au were selected to investigate the effect of the atomic mass of the metal on nanopore drilling, namely, elastic versus inelastic scattering. We demonstrated that the effect of elastic scattering (pore formation by sputtering) decreased as the atomic mass of the metal increased. Furthermore, experimental cross-sections obtained from normalized drilled volume vs. electron dose curves (characteristic contrast curves) matched well the calculated atomic displacement cross-sections determined from elastic scattering data. The sputtering energies of Ti, Cr, and Cu were determined to be approximately 10, 9, and 7 eV, respectively, which were in good agreement with the reported range of sputtering energy values.

Recent Progress in Solid-State Nanopores.

The solid-state nanopore has attracted much attention as a next-generation DNA sequencing tool or a single-molecule biosensor platform with its high sensitivity of biomolecule detection. The platform has advantages of processability, robustness of the device, and flexibility in the nanopore dimensions as compared with the protein nanopore, but with the limitation of insufficient spatial and temporal resolution to be utilized in DNA sequencing. Here, the fundamental principles of the solid-state nanopore are summarized to illustrate the novelty of the device, and improvements in the performance of the platform in terms of device fabrication are explained. The efforts to reduce the electrical noise of solid-state nanopore devices, and thus to enhance the sensitivity of detection, are presented along with detailed descriptions of the noise properties of the solid-state nanopore. Applications of 2D materials including graphene, h-BN, and MoS2 as a nanopore membrane to enhance the spatial resolution of nanopore detection, and organic coatings on the nanopore membranes for the addition of chemical functionality to the nanopore are summarized. Finally, the recently reported applications of the solid-state nanopore are categorized and described according to the target biomolecules: DNA-bound proteins, modified DNA structures, proteins, and protein oligomers.

A low-noise silicon nitride nanopore device on a polymer substrate.

We report a novel low-noise nanopore device employing a polymer substrate. The Si substrate of a fabricated Si-substrate-based silicon nitride (Si3N4) membrane was replaced with a polymer substrate. As such, laser machining was used to make a micro-size hole through the polyimide (PI) substrate, and a thin Si3N4 membrane was then transferred onto the PI substrate. Finally, a nanopore was formed in the membrane using a transmission electron microscope for detection of biomolecules. Compared to the Si-substrate-based device, the dielectric noise was greatly reduced and the root-mean-square noise level was decreased from 146.7 to 5.4 pA. Using this device, the translocation of double-strand deoxyribonucleic acid (DNA) was detected with a high signal/noise (S/N) ratio. This type of device is anticipated to be available for future versatile sequencing technologies.

Highly reliable and low-noise solid-state nanopores with an atomic layer deposited ZnO membrane on a quartz substrate.

We present a fabrication scheme for a solid-state ZnO nanopore membrane directly deposited on top of a quartz substrate by atomic layer deposition (ALD) and investigate the characteristics of DNA translocation through the nanopores. We chose a ZnO membrane owing to its high isoelectric point (∼9.5) as well as its chemical and mechanical stability. Aside from the extremely low noise level exhibited by this device on a highly insulating and low dielectric quartz substrate, it also slows down the translocation speed of DNA by more than one order of magnitude as compared to that of a SiNx nanopore device. We propose that the electrostatic interaction between the positively charged ZnO nanopore wall, resulting from the high isoelectric point of ZnO, and the negatively charged phosphate backbone of DNA provides an additional frictional force that slows down the DNA translocation.

Noise and sensitivity characteristics of solid-state nanopores with a boron nitride 2-D membrane on a pyrex substrate.

We have fabricated highly sensitive and low noise solid-state nanopores with multiple layers of boron nitride (BN) membranes transferred onto a pyrex substrate. Both the dielectric and flicker noise of the device, which have been described as one of the bottlenecks to making highly sensitive 2-D membrane nanopore devices, have been reduced as follows. Firstly, a pyrex substrate with a low dielectric constant (εr = 4.7-5.1) and low dielectric loss (D < 0.001) is used instead of a Si substrate to reduce the dielectric noise of the device. Secondly, flicker noise is minimized by employing a 100 nm thick SiNx supporting layer with a small opening (less than 100 nm) for BN membrane transfer to enhance the mechanical stability. The flicker noise is further reduced by transferring multiple layers of BN instead of a single layer of BN. The resulting multi-layered BN device shows significant reduction of dielectric and 1/f noise as compared to the devices with a single layer of the BN and Si substrate. Furthermore, we demonstrate dsDNA translocations with a high signal to noise ratio around 50 at 100 and 10 kHz bandwidths.

A low-noise solid-state nanopore platform based on a highly insulating substrate.

A solid-state nanopore platform with a low noise level and sufficient sensitivity to discriminate single-strand DNA (ssDNA) homopolymers of poly-A40 and poly-T40 using ionic current blockade sensing is proposed and demonstrated. The key features of this platform are (a) highly insulating dielectric substrates that are used to mitigate the effect of parasitic capacitance elements, which decrease the ionic current RMS noise level to sub-10 pA and (b) ultra-thin silicon nitride membranes with a physical thickness of 5 nm (an effective thickness of 2.4 nm estimated from the ionic current) are used to maximize the signal-to-noise ratio and the spatial depth resolution. The utilization of an ultra-thin membrane and a nanopore diameter as small as 1.5 nm allow the successful discrimination of 40 nucleotide ssDNA poly-A40 and poly-T40. Overall, we demonstrate that this platform overcomes several critical limitations of solid-state nanopores and opens the door to a wide range of applications in single-molecule-based detection and analysis.

Noise and its reduction in graphene based nanopore devices.

Ionic current fluctuations in graphene nanopore devices are a ubiquitous phenomenon and are responsible for degraded spatial and temporal resolution. Here, we descriptively investigate the impact of different substrate materials (Si and quartz) and membrane thicknesses on noise characteristics of graphene nanopore devices. To mitigate the membrane fluctuations and pin-hole defects, a SiNx membrane is transferred onto the substrate and a pore of approximately 70 nm in diameter is perforated prior to the graphene transfer. Comprehensive noise study reveals that the few layer graphene transferred onto the quartz substrate possesses low noise level and higher signal to noise ratio as compared to single layer graphene, without deteriorating the spatial resolution. The findings here point to improvement of graphene based nanopore devices for exciting opportunities in future single-molecule genomic screening devices.