Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking.

Bin-picking of small parcels and other textureless planar-faced objects is a common task at warehouses. A general color image-based vision-guided robot picking system requires feature extraction and goal image preparation of various objects. However, feature extraction for goal image matching is difficult for textureless objects. Further, prior preparation of huge numbers of goal images is impractical at a warehouse. In this paper, we propose a novel depth image-based vision-guided robot bin-picking system for textureless planar-faced objects. Our method uses a deep convolutional neural network (DCNN) model that is trained on 15,000 annotated depth images synthetically generated in a physics simulator to directly predict grasp points without object segmentation. Unlike previous studies that predicted grasp points for a robot suction hand with only one vacuum cup, our DCNN also predicts optimal grasp patterns for a hand with two vacuum cups (left cup on, right cup on, or both cups on). Further, we propose a surface feature descriptor to extract surface features (center position and normal) and refine the predicted grasp point position, removing the need for texture features for vision-guided robot control and sim-to-real modification for DCNN model training. Experimental results demonstrate the efficiency of our system, namely that a robot with 7 degrees of freedom can pick randomly posed textureless boxes in a cluttered environment with a 97.5% success rate at speeds exceeding 1000 pieces per hour.

Left atrial pathological degeneration assessed by integrated backscatter transesophageal echocardiography as a predictor of progression to persistent atrial fibrillation: results from a prospective study of three-years follow-up.

BACKGROUND: It is recognized that one of the causes of atrial fibrillation (AF) is pathological degeneration of the left atrium (LA). However, prospective study that elucidated the relationship between the incidence of persistent AF and pathological degeneration has not been performed. The purpose of this study was to elucidate the usefulness of integrated backscatter (IBS) values for the prediction of progression from paroxysmal AF (PAF) to persistent AF. METHODS: We measured IBS values of the entire LA wall at 5 mm intervals (except the posterior wall) in 27 patients with paroxysmal AF and evaluated progression to persistent AF for three years. IBS values were acquired with transesophageal echocardiography (TEE) using a 4-7 MHz transducer. IBS values were calculated as the average power of the backscattered signal from regions of interest (ROI). Each IBS value was color-coded to construct three dimensional maps. RESULTS: Average IBS values of total voxels in color-coded maps in the persistent AF group were significantly greater than those in the non-persistent AF group (25.8 ± 5.0 dB vs. 17.4 ± 10.2 dB, p = 0.047), whereas there was no significant difference in LA diameter between the persistent AF and the non-persistent AF group. There was significant difference in persistent AF-free survival after the baseline measurements in the subjects stratified by IBS value (< 20 dB versus ≥ 20 dB) (univariate Cox regression analysis: hazard ratio: 8.74, p =0.046). CONCLUSION: Using IBS values measured by TEE, we can identify an increase in atrial degeneration that may predict the occurrence of persistent AF before LA dilation.

Relationship among coronary plaque compliance, coronary risk factors and tissue characteristics evaluated by integrated backscatter intravascular ultrasound.

BACKGROUND: The purpose of the present study was to evaluate the mechanical properties of coronary plaques and plaque behavior, and to elucidate the relationship among tissue characteristics of coronary plaques, mechanical properties and coronary risk factors using integrated backscatter intravascular ultrasound (IB-IVUS). METHODS: Non-targeted plaques with moderate stenosis (plaque burden at the minimal lumen site: 50-70%) located proximal to the site of the percutaneous coronary intervention target lesions were evaluated by IB-IVUS. Thirty-six plaques (less calcified group: an arc of calcification ≤10°) in 36 patients and 22 plaques (moderately calcified group: 10° < an arc of calcification ≤60°) in 22 patients were evaluated. External elastic membrane volume (EEMV) compliance, lumen volume (LV) compliance, plaque volume (PV) response (difference between PV in systole and diastole), EEM area stiffness index were measured at the minimal lumen site. Relative lipid volume (lipid volume/internal elastic membrane volume) was calculated by IB-IVUS. RESULTS: In the less calcified group, there was a significant correlation between EEMV compliance and the relative lipid volume (r = 0.456, p = 0.005). There was a significant inverse correlation between EEM area stiffness index and the relative lipid volume (p = 0.032, r = -0.358). The LV compliance and EEM area stiffness index were significantly different in the diabetes mellitus (DM) group than in the non-DM group (1.32 ± 1.49 vs. 2.47 ± 1.79%/10 mmHg, p =0.014 and 28.3 ± 26.0 vs. 15.7 ± 17.2, p =0.020). The EEMV compliance and EEM area stiffness index were significantly different in the hypertension (HTN) group than in the non-HTN group (0.77 ± 0.68 vs. 1.57 ± 0.95%/10 mmHg, p =0.012 and 26.5 ± 24.3 vs. 13.0 ± 16.7, p =0.020). These relationships were not seen in the moderately calcified group. CONCLUSION: The present study provided new findings that there was a significant correlation between mechanical properties and tissue characteristics of coronary arteries. In addition, our results suggested that the EEMV compliance and the LV compliance were independent and the compliance was significantly impaired in the patients with DM and/or HTN. Assessment of coronary mechanical properties during PCI may provide us with useful information regarding the risk stratification of patients with coronary heart disease.

Comparison between integrated backscatter intravascular ultrasound and 64-slice multi-detector row computed tomography for tissue characterization and volumetric assessment of coronary plaques.

BACKGROUND: The purpose of this study was to determine the cut-off values of Hounsfield units (HU) for the discrimination of plaque components and to evaluate the feasibility of measurement of the volume of plaque components using multi-detector row computed tomography (MDCT). METHODS: Coronary lesions (125 lesions in 125 patients) were visualized by both integrated backscatter intravascular ultrasound (IB-IVUS) and 64-slice MDCT at the same site. The IB values were used as a gold standard to determine the cut off values of HU for the discrimination of plaque components. RESULTS: Plaques were classified as lipid pool (n =50), fibrosis (n =65) or calcification (n =35) by IB-IVUS. The HU of lipid pool, fibrosis and calcification were 18 ± 18 HU (-19 to 58 HU), 95 ± 24 HU (46 to 154 HU) and 378 ± 99 HU (188 to 605 HU), respectively. Using receiver operating characteristic curve analysis, a threshold of 50 HU was the optimal cutoff values to discriminate lipid pool from fibrosis. Lipid volume measured by MDCT was correlated with that measured by IB-IVUS (r =0.66, p <0.001), whereas fibrous volume was not (r =0.21, p =0.059). CONCLUSION: Lipid volume measured by MDCT was moderately correlated with that measured by IB-IVUS. MDCT may be useful for volumetric assessment of the lipid volume of coronary plaques, whereas the assessment of fibrosis volume was unstable.

Learning Suction Graspability Considering Grasp Quality and Robot Reachability for Bin-Picking.

Deep learning has been widely used for inferring robust grasps. Although human-labeled RGB-D datasets were initially used to learn grasp configurations, preparation of this kind of large dataset is expensive. To address this problem, images were generated by a physical simulator, and a physically inspired model (e.g., a contact model between a suction vacuum cup and object) was used as a grasp quality evaluation metric to annotate the synthesized images. However, this kind of contact model is complicated and requires parameter identification by experiments to ensure real world performance. In addition, previous studies have not considered manipulator reachability such as when a grasp configuration with high grasp quality is unable to reach the target due to collisions or the physical limitations of the robot. In this study, we propose an intuitive geometric analytic-based grasp quality evaluation metric. We further incorporate a reachability evaluation metric. We annotate the pixel-wise grasp quality and reachability by the proposed evaluation metric on synthesized images in a simulator to train an auto-encoder-decoder called suction graspability U-Net++ (SG-U-Net++). Experiment results show that our intuitive grasp quality evaluation metric is competitive with a physically-inspired metric. Learning the reachability helps to reduce motion planning computation time by removing obviously unreachable candidates. The system achieves an overall picking speed of 560 PPH (pieces per hour).

Tissue characterization of coronary plaques and assessment of thickness of fibrous cap using integrated backscatter intravascular ultrasound. Comparison with histology and optical coherence tomography.

BACKGROUND: The purpose of this study was to develop a new online integrated backscatter intravascular ultrasound (IB-IVUS) system and to validate its ability to measure fibrous cap thickness by comparing IB-IVUS images with those from optical coherence tomography (OCT). METHODS AND RESULTS: Images were acquired from 125 segments of 26 coronary arteries obtained at autopsy from 11 cadavers. In the training study (n=30), 242 regions-of-interest on color-coded maps were compared with histology. In the validation study, 95 cross-sections were diagnosed by IB-IVUS and histology. In 24 patients with stable angina, 28 arterial cross-sections were imaged by IB-IVUS and OCT in vivo. In the training study, cutoff values of 39 decibels (dB) and 17dB were the optimal predictors of lipid pool/fibrosis and fibrosis/calcification, respectively, with 38-MHz mode; 42dB and 20dB, respectively, with 43-MHz mode. In the validation study, IB classified the fibrous, lipid-rich and fibrocalcific components with an accuracy of 92%, 91% and 95%, respectively. Agreement between the histological and IB-IVUS diagnoses was excellent (Cohen's κ=0.83). There was a correlation between the fibrous cap thickness measured by IB-IVUS and OCT (r=0.74, P<0.001). CONCLUSIONS: The IB-IVUS system with improved resolution provides high diagnostic accuracy for the analysis of the tissue characteristics of coronary plaques, and enables estimation of the thickness of the fibrous cap in the clinical setting.

Development of integrated backscatter intravascular ultrasound for tissue characterization of coronary plaques.

Tissue characterization of plaques of coronary arteries is important to clarify the process of acute coronary syndrome and prevent it. The purpose of this study is to develop an online integrated backscatter intravascular ultrasound (IB-IVUS) system and validate the diagnostic accuracy for the characterization of coronary plaques. A personal computer equipped with custom software was connected to an IVUS imaging system. Images were acquired from 242 segments of 46 coronary arteries from 25 cadavers obtained at autopsy. In the training study, a total of 724 regions-of-interests on color-coded maps were compared with histologic images. In the validation study, a total of 192 cross-sections of coronary arteries were evaluated. Receiver operating characteristic curve analysis showed that the cut-off points of -49 dB (area under curve = 0.98) and -29 dB (area under curve = 0.99) were the most reliable predictors of lipid pools, fibrosis and calcification. In the validation study, the analysis using IB values classified fibrous, lipid-rich and fibrocalcific plaque components with a high accuracy of 93%, 90% and 96%, respectively. The overall agreement between histologic and IB-IVUS diagnoses (n = 175) was high (Cohen's kappa = 0.81). The IB-IVUS system provides high diagnostic accuracy for analysis of tissue characteristics of coronary plaques.

Anti-hyperuricemic effect of taxifolin in cultured hepatocytes and model mice.

Hyperuricemia is recognized as an important risk factor for gout. High dietary intake of purine-rich foods such as meats and sea foods increases uric acid (UA) levels in the blood. Taxifolin present in Siberian larch and strawberries has been reported to possess health promoting activities including anti-oxidant effect. In this study, we examined anti-hyperuricemic effect of taxifolin in both cultured hepatocytes and hyperuricemic model mice. In cultured AML12 hepatocytes, taxifolin significantly suppressed UA production dose- and time-dependently. In mice with hyperuricemia induced by concurrent administration of guanosine-5'-monophosphate and inosine-5'-monophosphate, oral administration of taxifolin suppressed the increases in plasma and liver UA levels. In addition, it also suppressed hepatic xanthine oxidase (XO) activity. Thus, anti-hyperuricemic effect of taxifolin could be explained, at least partly, by suppressing UA production via inhibition of XO activity in the liver. These results suggest that taxifolin possesses a potent hypouricemic effect and it could be a potential candidate for an anti-hyperuricemic phytochemical.

Evaluation of left atrial degeneration for the prediction of atrial fibrillation: usefulness of integrated backscatter transesophageal echocardiography.

OBJECTIVES: The purpose of this study was to elucidate the usefulness of integrated backscatter (IBS) transesophageal echocardiography (TEE) for the evaluation of atrial degeneration and clarify whether atrial degeneration predicts the occurrence of atrial fibrillation (AF). BACKGROUND: One of the causes of AF is pathological degeneration of the left atrium (LA). However, there is no appropriate method to evaluate degeneration of the LA in the clinical setting. METHODS: The IBS images were acquired with TEE with a 4- to 7-MHz transducer. The IBS values were calculated as the average power of the backscattered signal from regions of interest (ROI). In the pathological study, we measured IBS values of 21 left atrial specimens obtained from 10 autopsied hearts. Relative interstitial area in the ROI was automatically calculated by a personal computer. In the clinical study, we measured IBS values of the entire LA wall at 5-mm intervals (except the posterior wall) in 42 patients (18 non-AF patients, 14 paroxysmal AF patients, and 10 chronic AF patients). Each IBS value was color-coded to construct 3-dimensional maps. RESULTS: There was a weak correlation between the relative interstitial area and IBS values (r = 0.45, p = 0.038). Average corrected IBS values of total voxels in color-coded maps in the AF group (24.4 +/- 6.4 dB) and the paroxysmal AF group (23.9 +/- 9.6 dB) were significantly greater than those in the non-AF group (15.6 +/- 7.4 dB, p = 0.007), whereas there was no significant difference in LA diameter between the paroxysmal AF group (39.4 +/- 6.5 mm) and the non-AF group (36.7 +/- 5.5 mm). CONCLUSIONS: With IBS-TEE, we can identify an increase in atrial degeneration that might predict the occurrence of AF before LA dilation.

Transcatheter embolization by autologous blood clot is useful management for small side branch perforation due to percutaneous coronary intervention guide wire.

A 75-year-old man underwent PCI for a bifurcation lesion with 90% stenosis in segment 6 and 75% proximal stenosis in segment 9 of the left coronary artery. We implanted a Duraflex coronary stent into segment 6 and kissing balloon inflation for segments 6 and 9. Although these 2 lesions were adequately dilated, we noticed coronary perforation caused by the guide wire in a small branch of segment 9. We tried to repair the perforation using a small balloon and long inflation, but unfortunately the perforation was not improved. We attempted to occlude the small branch including the perforation site with an autologous blood clot via a wire microcatheter inserted into the small branch. The autologous blood clot was suspended in contrast media and saline. Using this procedure, the small branch of segment 9 was occluded completely and the perforated site was repaired. After the procedure, no significant CPK elevation was detected, and 6 months later, we confirmed that small branch embolization was improved and coronary flow was good. Autologous blood clot is useful to occlude and repair perforations in small side branches of the coronary artery without myocardial damage.

Tissue characterization of coronary plaques: comparison of integrated backscatter intravascular ultrasound with virtual histology intravascular ultrasound.

BACKGROUND: Integrated backscatter (IB) intravascular ultrasound (IVUS) and IVUS Virtual Histology (VH) have been developed for tissue characterization, but have never been compared directly. The purpose of this study was to compare the overall agreement between IB-IVUS and IVUS-VH in the tissue characterization of plaques from the same coronary arterial cross-section. METHODS AND RESULTS: Images were acquired from 46 coronary arteries from 25 cadavers. Of a total of 392 histology/IVUS image pairs, 152 pairs were diagnosed as Stary's type III, IV, Va, Vb and Vc, and compared for IB-IVUS, IVUS-VH and histology. In the qualitative comparison, the overall agreement between histological and IB-IVUS diagnoses was higher (kappa = 0.81, 95% confidence interval (CI): 0.74-0.89) than that of the IVUS-VH diagnoses (kappa = 0.66, 95%CI: 0.56-0.75). The % fibrosis area determined by IB-IVUS was significantly correlated with the relative area of fibrosis based on histology (r = 0.67, p < 0.001). In the quantitative comparison, the overall agreement between the histological and IB-IVUS diagnoses was higher (kappa = 0.83, 95% CI: 0.75-0.91) than that of the IVUS-VH diagnoses (kappa = 0.73, 95% CI: 0.63-0.83). CONCLUSION: Based on histology as the gold standard, IB-IVUS provided higher diagnostic accuracy than IVUS-VH for tissue characterization of coronary plaques.

Can negative cardiac effect of proton pump inhibitor and high-dose H2-blocker have clinical influence on patients with stable angina?

BACKGROUND: Aspirin and anti-platelet drugs are used commonly for patients with coronary heart disease. Proton pump inhibitor (PPI) and high-dose H2-blocker were recommended for preventing NSAIDs-related ulcer. Previously H2-blocker reported to have some negative cardiovascular effects. Additionally, a recent in vitro study showed that PPI reduced cardiac contractility. In this study, we evaluated whether chronic administration of PPI and high-dose H2-blocker affects left ventricular function. METHOD: Fifty-two stable angina patients were enrolled and classified into PPI group ([P]; lansoprazole: 15 mg/day, n=28), H2-blocker group ([H]; famotidine: 40 mg/day, n=8), and control ([C]; none or mucosal-defense drug, n=16). Eligible patients showed normal cardiac function in initial catheterization without administrated PPI or H2-blocker. They received percutaneous coronary intervention and follow-up catheterization. We compared changes in ejection fraction (EF: %), end diastolic/systolic volume index (EDVI/ESVI: ml/m(2)), and peak positive/negative dp/dt (+/-dp/dt: mmHg/s) in left ventricular angiography series. RESULT: There were no significant differences among three groups regarding patient characteristics, backgrounds of angiographic and intervention, except for fewer smokers in [C]. Other drugs such as beta- and Ca-blocker did not have effects on cardiac function except for aspirin during 255+/-115 days follow-up. Rate of EF changes significantly decreased in [P], and tended to decrease in [H] (C: 3.8+/-9.8%, H: -1.6+/-7.6%, P: -2.1+/-5.9%; p<0.05 for [C] vs. [P]). Those of ESVI changes were significantly greater in [P], and tended to be greater in [H] (C: -4.5+/-16.2%, H: 4.9+/-15.5%, P: 7.3+/-16.2%; p<0.05 for [C] vs. [P]), though, EDVI changes' were similar (C: 2.5+/-8.9%, H: 2.6+/-3.6%, P: 1.6+/-6.1%; p=ns). Rate of +/-dp/dt-changes tended to decrease in [H] (+dp/dt: C: 3.9+/-15.5%, H: -10.0+/-25.2%, P: 0.3+/-19.6%; p=ns, -dp/dt: C: -0.1+/-19.5%, H: -8.5+/-20.4%, P: 5.7+/-27.7%; p=ns). CONCLUSION: In this study, PPI and high-dose H2-blocker have EF-reducing tendency. However, these changes were small and these drugs seemed to exhibit little influence clinically.

Assessment of vulnerable plaques causing acute coronary syndrome using integrated backscatter intravascular ultrasound.

OBJECTIVES: This study aims to define tissue characteristics of vulnerable plaques before acute coronary syndrome (ACS) by use of integrated backscatter intravascular ultrasound (IB-IVUS). BACKGROUND: Tissue characterization of coronary plaques is possible with the use of IB-IVUS. METHODS: The subjects were 140 patients with angina pectoris, and we selected 160 coronary lesions without significant stenosis for evaluation. Ultrasound signals were obtained by an IVUS system using a 40-MHz catheter. RESULTS: At the follow-up (30 +/- 7 months), 12 plaques caused ACS after the initial IVUS examination. Ten of the 12 plaques had IVUS parameters recorded at baseline. These 10 plaques were classified as vulnerable plaques (VP), and the other plaques were classified as stable plaques (SP; n = 143). There was no significant difference of vessel area, lumen area, and plaque area between VP and SP. However, plaque burden (60 +/- 9% vs. 52 +/- 9%; p = 0.014), eccentricity (0.70 +/- 0.10 vs. 0.55 +/- 0.17; p = 0.013), remodeling index (1.30 +/- 0.08 vs. 1.16 +/- 0.16; p = 0.006) and percentage lipid area (72 +/- 10% vs. 50 +/- 16%; p < 0.0001) were greater in VP than in SP. Percentage fibrous area (23 +/- 6% vs. 47 +/- 14%; p < 0.0001) was smaller in VP than in SP. The sensitivities, specificities, and positive predictive values of percentage fibrous area (90%, 96%, and 69%, respectively) and percentage lipid area (80%, 90%, and 42%, respectively) for classifying VP were evaluated. CONCLUSIONS: Tissue characteristics of VP before ACS were different from those of SP. This suggests that VP and SP as classified by IB-IVUS are useful in predicting ACS.

Compression of right ventricular out-flow due to localized hematoma after coronary perforation during PCI.

Although coronary perforation can cause tamponade during percutaneous coronary intervention (PCI), this is unusual for patients previously undergoing coronary artery bypass graft surgery (CABG) due to pericardial adhesions. We report here on a rare case of right ventricular out-flow obstruction complicating PCI in a patient with a previous CABG.

Novel technique using intravascular ultrasound-guided guidewire cross in coronary intervention for uncrossable chronic total occlusions.

The experience of using a novel application of intravascular ultrasound (IVUS)-guided percutaneous coronary interventions for chronic total occlusions is reported in 2 cases. In the first case, an IVUS catheter was advanced into a side branch to identify the entry point of the major branch. In the second case, IVUS-guided penetration of the guidewire from the false lumen to the true lumen after causing a dissection was successful.