Adaptive Discount Factor for Deep Reinforcement Learning in Continuing Tasks with Uncertainty.

Reinforcement learning (RL) trains an agent by maximizing the sum of a discounted reward. Since the discount factor has a critical effect on the learning performance of the RL agent, it is important to choose the discount factor properly. When uncertainties are involved in the training, the learning performance with a constant discount factor can be limited. For the purpose of obtaining acceptable learning performance consistently, this paper proposes an adaptive rule for the discount factor based on the advantage function. Additionally, how to use the advantage function in both on-policy and off-policy algorithms is presented. To demonstrate the performance of the proposed adaptive rule, it is applied to PPO (Proximal Policy Optimization) for Tetris in order to validate the on-policy case, and to SAC (Soft Actor-Critic) for the motion planning of a robot manipulator to validate the off-policy case. In both cases, the proposed method results in a better or similar performance compared with cases using the best constant discount factors found by exhaustive search. Hence, the proposed adaptive discount factor automatically finds a discount factor that leads to comparable training performance, and that can be applied to representative deep reinforcement learning problems.

Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor-Critic with Hindsight Experience Replay.

Since path planning for multi-arm manipulators is a complicated high-dimensional problem, effective and fast path generation is not easy for the arbitrarily given start and goal locations of the end effector. Especially, when it comes to deep reinforcement learning-based path planning, high-dimensionality makes it difficult for existing reinforcement learning-based methods to have efficient exploration which is crucial for successful training. The recently proposed soft actor-critic (SAC) is well known to have good exploration ability due to the use of the entropy term in the objective function. Motivated by this, in this paper, a SAC-based path planning algorithm is proposed. The hindsight experience replay (HER) is also employed for sample efficiency and configuration space augmentation is used in order to deal with complicated configuration space of the multi-arms. To show the effectiveness of the proposed algorithm, both simulation and experiment results are given. By comparing with existing results, it is demonstrated that the proposed method outperforms the existing results.

New organ-based tube current modulation method to reduce the radiation dose during computed tomography of the head: evaluation of image quality and radiation dose to the eyes in the phantom study.

A new organ-based tube current modulation (NOB-TCM) method was designed with the intent to decrease tube current by 30% over a prescribed 90° radial arc across the anterior aspect of the radiosensitive organ, without increasing tube current in the remaining radial arc. We compared a reference scan and five other dose-reducing methods with regard to effects on dose, practicality, and image quality to determine the most effective method for the reduction of the radiation dose to the eyes during CT examinations of the head. We compared the radiation doses to the eyes and physical image quality in different regions of interest for TCM and shielding scans. Three types of TCM scans were performed: longitudinal TCM, angular TCM, and NOB-TCM. A bismuth sheet and lead goggles were each applied for the shielding scan. Relative to the reference scan, the dose to the eye was reduced to 25.88% with NOB-TCM, 44.53% with lead goggles, and 36.91% with a bismuth shield. Relative to the reference scan, the mean signal-to-noise ratio (SNR) was decreased to 8.02% with NOB-TCM, 28.36% with lead goggles, and 32.95% with the bismuth shield. The SNR of the anterior region of interest was decreased to 11.89% with NOB-TCM and 87.89% with the bismuth shield. The average figure of merit was increased by 11.7% with longitudinal TCM and 13.39% with NOB-TCM, compared with the reference scan. NOB-TCM is a superior solution for head CT, including the orbital area, due to the reduction in radiation exposure without significant loss in image quality.

Assessment of the viability of integrating virtual reality programs in practical tests for the Korean Radiological Technologists Licensing Examination: a survey study

PURPOSE: The objective of this study was to assess the feasibility of incorporating virtual reality/augmented reality (VR/AR) programs into practical tests administered as part of the Korean Radiological Technologists Licensing Examination (KRTLE). This evaluation is grounded in a comprehensive survey that targeted enrolled students in departments of radiology across the nation. METHODS: In total, 682 students from radiology departments across the nation were participants in the survey. An online survey platform was used, and the questionnaire was structured into 5 distinct sections and 27 questions. A frequency analysis for each section of the survey was conducted using IBM SPSS ver. 27.0. RESULTS: Direct or indirect exposure to VR/AR content was reported by 67.7% of all respondents. Furthermore, 55.4% of the respondents expressed that VR/AR could be integrated into their classes, which signified a widespread acknowledgment of VR among the students. With regards to the integration of a VR/AR or mixed reality program into the practical tests for purposes of the KRTLE, a substantial amount of the respondents (57.3%) exhibited a positive inclination and recommended its introduction. CONCLUSION: The application of VR/AR programs within practical tests of the KRTLE will be used as an alternative for evaluating clinical examination procedures and validating job skills.

Pressure-wire based assessment of microvascular resistance using calibrated upstream balloon obstruction: a predictor of myocardial viability.

OBJECTIVES: We assess microvascular integrity as a marker of myocardial viability after coronary stenting, using only a pressure guidewire. BACKGROUND: Microvascular integrity generally is not assessed using pressure-only guidewires because the transducer lies upstream of microvasculature. We partially inflate a balloon inside a coronary stent to achieve a specific normalized pressure drop at rest (distal coronary/aortic pressure = 0.8) and then infuse a vasodilator, to render the wire sensitive to microvascular function. We hypothesize that the further decline in pressure (ΔFFR(0.8) ) predicts MRI myocardial viability. METHODS: We studied 29 subjects with acute coronary syndrome including myocardial infarction. After successful culprit stenting, the resting coronary/aortic pressure was set to 0.8 using temporary balloon obstruction. ΔFFR(0.8) was defined as 0.8-(distal coronary/aortic pressures) during adenosine-induced hyperemia. The average transmural extent of infarction was defined as the average area of MRI late gadolinium enhancement (after 2.8 ± 1.5 days) divided by the corresponding full thickness of the gadolinium enhanced sector in short axis slices, and was compared with ΔFFR(0.8) . RESULTS: ΔFFR(0.8) corresponded inversely and linearly with the average transmural extent of infarction (r(2) = 0.65, P < 0.001). We found that a transmural extent of infarction of 0.50 corresponded to a ΔFFR(0.8) threshold of 0.1, and had high sensitivity and specificity (100% and 94.4%, respectively). CONCLUSIONS: Using only an upstream pressure-sensitive guidewire and a partially obstructing balloon during pharmacologic hyperemia, we were able to predict MRI myocardial viability with high accuracy after relief of epicardial stenosis. With further validation, this may prove a useful clinical prognostic tool after percutaneous intervention.

Development of an Auxiliary Device for Patellar and Femoral Joint Tangential Axial Radiographic Imaging and a Method for Obtaining an Optimal Radiographic Image Using the Development.

This study evaluated the accuracy of tangential axial radiography of the patellar and femoral joint using an auxiliary device based on three image evaluation criteria, which we named the patellofemoral joint radiography auxiliary device (PJR). To compare the PJR method with conventional radiographic methods, such as Laurin, Merchant, and Settegast, a whole-body phantom (PBU-31) was used and three image evaluation items were set. The radiographic method, the smallest inclination of the patellar and showed the best half lateral image of the patella, is Settegast, and the measurement is 9.40. The second-best PJR measurement is 9.97, and the difference between the two measures is 5.76% (p = 0.001). The radiographic method showing the image with the largest distance between the patellar and femoral joint space is PJR which a measurement is 12.35. The second best Merchant measure is 10.55, and the difference between the two measures is 14.54% (p = 0.001). The method in which the two bones were well overlapped (i.e., evaluate the distortion of the image by measured as the distance between the femoral trochlear groove and the tibial tuberosity) is the PJR and the measurement is -0.37. The second-best Merchant measure is 3.93, and the difference between the two measures is 91.4% (p = 0.001). The Settegast has the image with the smallest inclination of the patella, but the PJR has the image that best describes the patellar-femoral joint and the least distortion of the image. As a result of the comprehensive evaluation, when using PJR, bending the knee by 40° and setting a 140° angle between the long axis of the femur and the long axis of the lower leg were considered to be the most beneficial conditions. Therefore, we propose the use of PJR for tangential axial radiography of the patellar-femoral joint.

Development of CT Effective Dose Conversion Factors from Clinical CT Examinations in the Republic of Korea.

The aim of this study was to determine the conversion factors for the effective dose (ED) per dose length product (DLP) for various computed tomography (CT) protocols based on the 2007 recommendations of the International Commission on Radiological Protection (ICRP). CT dose data from 369 CT scanners and 13,625 patients were collected through a nationwide survey. Data from 3793 patients with a difference in height within 5% of computational human phantoms were selected to calculate ED and DLP. The anatomical CT scan ranges for 11 scan protocols (adult-10, pediatric-1) were determined by experts, and scan lengths were obtained by matching scan ranges to computational phantoms. ED and DLP were calculated using the NCICT program. For each CT protocol, ED/DLP conversion factors were calculated from ED and DLP. Estimated ED conversion factors were 0.00172, 0.00751, 0.00858, 0.01843, 0.01103, 0.02532, 0.01794, 0.02811, 0.02815, 0.02175, 0.00626, 0.00458, 0.00308, and 0.00233 mSv∙mGy-1∙cm-1 for the adult brain, intra-cranial angiography, C-spine, L-spine, neck, chest, abdomen and pelvis, coronary angiography, calcium scoring, aortography, and CT examinations of pediatric brain of <2 years, 4-6 years, 9-11 years, and 13-15 years, respectively. We determined ED conversion factors for 11 CT protocols using CT data obtained from a nationwide survey in Korea and Monte Carlo-based dose calculations.

Radiation exposure of interventional cardiologists during coronary angiography: evaluation by phantom measurement and computer simulation.

During interventional cardiological procedures, operators are exposed to patients' scatter radiation. Therefore, we measured the radiation exposure of the operator's eyeball, thyroid, and chest wall during angiography. We used the optically stimulated luminescence dosimeter in the anthropomorphic phantom and developed Monte Carlo simulations using the Korean human voxel phantom. At 15 frames/s, the radiation dose of the operator's right eyeball (RE), left eyeball (LE), thyroid (T), and chest wall (CW) at the femoral artery puncture position (FAPP) with protective equipment (PE) was 0.015, 0.16, 0.012, and 0.014 mGy, respectively. At 7.5 frames/sec, the radiation dose of the operator's RE, LE, T, and CW at FAPP with PE was 33.33%, 18.75%, 52.94%, and 45.00% lower than that of those at the radial artery puncture position (RAPP), respectively. At 15 frames/s, the radiation dose of the operator's RE, LE, T, and CW at RAPP without PE was 1.76 times, 2.23 times, 2.76 times, and 2.05 times higher than that of those with PE. Per the simulation results, the absorbed radiation dose of the eye ball, thyroid gland, and myocardium of the heart at FAPP with and without PE under 15 frame/s was 9.68%, 13.04%, 8.33% and 9.98%, 6.00%, 8.82% lower than at RAPP under similar conditions. Effective measures for occupational radiological protection are lower frame rate exposure, increased distance from the X-ray source, and PE use. Radiologist protection in interventional cardiology cannot be handled independently of patient protection, owing to several correlations; thus, reducing the patient dose will reduce the operator dose.

Occupational radiation exposure in femoral artery approach is higher than radial artery approach during coronary angiography or percutaneous coronary intervention.

Medical radiation exposure is a significant concern for interventional cardiologists (IC). This study was aimed at estimating the radiation exposure of IC operators and assistants in real clinical practice. The radiation exposure of the operator and assistant was evaluated by conducting two types of procedures via coronary angiography (CAG) and percutaneous coronary intervention (PCI) on 1090 patients in 11-cardiovascular centers in Korea. Radiation exposure was measured using an electronic personal dosimeter (EPD). EPD were attached at 3 points on each participant: on the apron on the left anterior chest (A1), under the apron on the sternum (A2), and on the thyroid shield (T). Average radiation exposure (ARE) of operators at A1, A2, and T was 19.219 uSv, 4.398 uSv, and 16.949 uSv during CAG and 68.618 uSv, 15.213 uSv, and 51.197 uSv during PCI, respectively. ARE of assistants at A1, A2, and T was 4.941 uSv, 0.860 uSv, and 5.232 uSv during CAG and 20.517 uSv, 4.455 uSv, and 16.109 uSv during PCI, respectively. AED of operator was 3.4 times greater during PCI than during CAG.

NATIONAL DIAGNOSTIC REFERENCE LEVELS AND ACHIEVABLE DOSES FOR 13 ADULT CT PROTOCOLS AND A PAEDIATRIC HEAD CT PROTOCOL: NATIONAL SURVEY OF KOREAN HOSPITALS.

To develop a second set of diagnostic reference levels (DRLs) and achievable doses (ADs) for 13 adult computed tomography (CT) protocols and a paediatric head CT protocol in Korea. A survey of 13,625 CT examinations was performed based on 13 adult CT protocols and a paediatric non-contrast brain CT protocol using 369 CT systems, with patients grouped according to age. Most CT protocols in this survey had DRLs similar to those reported in other countries. However, chest and abdomen-pelvic CT had lower DRLs than those reported in the first Korean national survey and those from other countries. Paediatric non-contrast brain CT in each age group, with the exception of the 11-15-year age group, had lower DRLs than those reported in other countries. The DRLs presented here are similar to (or lower than for some protocols) those reported in the first Korean national survey and those from other countries.

A MULTICENTRE SURVEY OF LOCAL DIAGNOSTIC REFERENCE LEVELS AND ACHIEVABLE DOSE FOR CORONARY ANGIOGRAPHY AND PERCUTANEOUS TRANSLUMINAL CORONARY INTERVENTION PROCEDURES IN KOREA.

Interventional cardiology procedures can involve relatively high radiation doses compared to general radiography. During coronary angiography (CAG) and percutaneous transluminal coronary intervention (PCI), the same area is exposed to radiation for a long period. In this study, radiation exposure data of 1071 examinations in Korean hospitals were collected, and the achievable dose (AD) and diagnostic reference levels (DRLs) in actual medical practice for two types of interventional cardiology procedures in Korea were established. In CAG, 75th percentile DRLs and AD of the total kerma-area product were 47.0 and 33.1 Gy·cm 2, respectively. In PCI, those values were 171.3 and 102.6 Gy·cm2, respectively. This is the first study to introduce the DRLs for cardiovascular interventional procedures in Korea. These results will help optimise the interventional cardiology procedures for Korean cardiac centres.

Tensile property improvement of TWIP-cored three-layer steel sheets fabricated by hot-roll-bonding with low-carbon steel or interstitial-free steel.

TWIP-cored three-layer steel sheets were newly fabricated by hot rolling of TWIP steel sheet surrounded by low-carbon (LC) or interstitial-free (IF) steel sheets. TWIP/LC or TWIP/IF interfaces were well bonded without pores or voids, while a few pearlites were thinly formed along the interfaces. The strengths and elongation of the TWIP-cored sheets increased as the volume fraction of TWIP-cored region increased, and were also well matched with the ones calculated by a rule of mixtures based on volume fraction or force fraction. According to digital image correlation and electron back-scatter diffraction analyses, very high strain hardening effect in the initial deformation stage and active twin formation in the interfacial region beneficially affected the overall homogeneous deformation in the TWIP-cored sheets without any yield point phenomenon occurring in the LC sheet and serrations occurring in the TWIP sheet, respectively. These TWIP-cored sheets can cover a wide range of yield strength, tensile strength, and ductility levels, e.g., 320~498 MPa, 545~878 MPa, and 48~54%, respectively, by controlling the volume fraction of TWIP-cored region, and thus present new applications to multi-functional automotive steel sheets requiring excellent properties.

Novel twin-roll-cast Ti/Al clad sheets with excellent tensile properties.

Pure Ti or Ti alloys are recently spot-lighted in construction industries because they have excellent resistance to corrosions, chemicals, and climates as well as various coloring characteristics, but their wide applications are postponed by their expensiveness and poor formability. We present a new fabrication process of Ti/Al clad sheets by bonding a thin Ti sheet on to a 5052 Al alloy melt during vertical-twin-roll casting. This process has merits of reduced production costs as well as improved tensile properties. In the as-twin-roll-cast clad sheet, the homogeneously cast microstructure existed in the Al alloy substrate side, while the Ti/Al interface did not contain any reaction products, pores, cracks, or lateral delamination, which indicated the successful twin-roll casting. When this sheet was annealed at 350 °C~600 °C, the metallurgical bonding was expanded by interfacial diffusion, thereby leading to improvement in tensile properties over those calculated by a rule of mixtures. The ductility was also improved over that of 5052-O Al alloy (25%) or pure Ti (25%) by synergic effect of homogeneous deformation due to excellent Ti/Al bonding. This work provides new applications of Ti/Al clad sheets to lightweight-alloy clad sheets requiring excellent formability and corrosion resistance as well as alloy cost saving.

Evaluation of the Patient Effective Dose in Whole Spine Scanography Based on the Automatic Image Pasting Method for Digital Radiography.

BACKGROUND: Whole spine scanography (WSS) is a radiologic examination that requires whole body X-ray exposure. Consequently, the amount of patient radiation exposure is higher than the radiation dose following routine X-ray examination. OBJECTIVES: Several studies have evaluated the patient effective dose (ED) following single exposure film-screen WSS. The objective of this study was to evaluate patient ED during WSS, based on the automatic image pasting method for multiple exposure digital radiography (APMDR). Further, the calculated EDs were compared with the results of previous studies involving single exposure film-screen WSS. PATIENTS AND METHODS: We evaluated the ED of 50 consecutive patients (M:F = 28:22) who underwent WSS using APMDR. The anterior-posterior (AP) and lateral (LAT) projection EDs were evaluated based on the Monte Carlo simulation. RESULTS: Using APMDR, the mean number of exposures was 6.1 for AP and 6.5 for LAT projections. LAT projections required more exposures (6.55%) than AP projections. The mean ED was 0.6276 mSv (AP) and 0.6716 mSv (LAT). The mean ED for LAT projections was 0.6061 mSv in automatic exposure control (AEC) and 0.7694 mSv in manual mode. The relationship between dose-area-product (DAP) and ED revealed a proportional correlation (AP, R(2) = 0.943; LAT, R(2) = 0.773). Compared to prior research involving single exposure screen-film WSS, the patient ED following WSS using APMDR was lower on AP than on LAT projections. CONCLUSION: Despite multiple exposures, ED control is more effective if WSS is performed using APMDR in the AEC mode.

Evaluation of radiation doses in patient and medical staff during endoscopic retrograde cholangiopancreatography procedures.

The radiation exposure dose must be optimised because the hazard resulting from an interventional radiology procedure is long term depending on the patient. The aim of this study was to measure the radiation doses received by the patients and medical staff during endoscopic retrograde cholangiopancreatography (ERCP) procedures. Data were collected during 126 ERCP procedures, including the dose-area product (DAP), entrance dose (ED), effective dose (E), fluoroscopy time (T) and number of digital radiographs (F). The medical staff members each wore a personal thermoluminescence dosemeter to monitor exposure during ERCP procedures. The mean DAP, ED, E and T were 47.06 Gy cm(2), 196.06 mGy, 8.93 mSv, 7.65 min and 9.21 images, respectively. The mean dose to the staff was 0.175 mSv and that to the assistant was 0.069 mSv. The dose to the medical staff was minimal when appropriate protective measures were used. The large variation in the patient doses must be further investigated.

Novel strip-cast Mg/Al clad sheets with excellent tensile and interfacial bonding properties.

In order to broaden industrial applications of Mg alloys, as lightest-weight metal alloys in practical uses, many efforts have been dedicated to manufacture various clad sheets which can complement inherent shortcomings of Mg alloys. Here, we present a new fabrication method of Mg/Al clad sheets by bonding thin Al alloy sheet on to Mg alloy melt during strip casting. In the as-strip-cast Mg/Al clad sheet, homogeneously distributed equi-axed dendrites existed in the Mg alloy side, and two types of thin reaction layers, i.e., γ (Mg17Al12) and ß (Mg2Al3) phases, were formed along the Mg/Al interface. After post-treatments (homogenization, warm rolling, and annealing), the interfacial layers were deformed in a sawtooth shape by forming deformation bands in the Mg alloy and interfacial layers, which favorably led to dramatic improvement in tensile and interfacial bonding properties. This work presents new applications to multi-functional lightweight alloy sheets requiring excellent formability, surface quality, and corrosion resistance as well as tensile and interfacial bonding properties.

Suppression of ceramide-induced cell death by hepatitis C virus core protein.

The hepatitis C virus (HCV) core protein is believed to be one of viral proteins that are capable of preventing virus-infected cell death upon various stimuli. But, the effect of the HCV core protein on apoptosis that is induced by various stimuli is contradictory. We examined the possibility that the HCV core protein affects the ceramide-induced cell death in cells expressing the HCV core protein through the sphingomyelin pathway. Cell death that is induced by C(2)-ceramide and bacterial sphingomyelinase was analyzed in 293 cells that constitutively expressed the HCV core protein and compared with 293 cells that were stably transfected only with the expression vector. The HCV core protein inhibited the cell death that was induced by these reagents. The protective effects of the HCV core protein on ceramide-induced cell death were reflected by the reduced expression of p21(WAF1/Cip1/Sid1) and the sustained expression of the Bcl-2 protein in the HCV core-expressing cells with respect to the vector-transfected cells. These results suggest that the HCV core protein in 293 cells plays a role in the modulation of the apoptotic response that is induced by ceramide. Also, the ability of the HCV core protein to suppress apoptosis might have important implications in understanding the pathogenesis of the HCV infection.

Analysis of the Relationship between Students' Seating Preferences and Academic Achievement in Medical School.

PURPOSE: This study aimed to explore the relationship between students' seating preferences and academic achievement in medical school. METHODS: The subjects of this study were 109 second-year students in C medical school. The pattern of seat selection of 109 students was surveyed by participant observation for 48 days, and a questionnaire was administered to determine the factors that were considered by students. Using SPSS version 12.0, we analyzed the factors that students considered with regard to seat selection and seat preference and the frequency of seat movements between areas. We performed one-way ANOVA to analyze the differences in academic achievement between students who moved seats versus those who did not. RESULTS: The most common reasons for seat selection were to focus better on the lecture (60 students), to focus better on lecture, and to feel familiar with the same seat (60 students). Students' preferred seats were in rows A4, A7, A5, and A3 (in descending order), which are primarily the central sections, and columns B15, B1, B14, B19 (in descending order), which are primarily both ends of the division. The difference in academic achievement between students who moved seats and those who did not was not significant (p>0.05). Among students who did not move seats, the difference in academic achievement between 9 seating areas was not significant in 6 subjects (p>0.05). CONCLUSION: The results of this study suggest that we should reconsider a professor's general perception regarding academic achievement according to seat location.

Silver front electrode grids for ITO-free all printed polymer solar cells with embedded and raised topographies, prepared by thermal imprint, flexographic and inkjet roll-to-roll processes.

Semitransparent front electrodes for polymer solar cells, that are printable and roll-to-roll processable under ambient conditions using different approaches, are explored in this report. The excellent smoothness of indium-tin-oxide (ITO) electrodes has traditionally been believed to be difficult to achieve using printed front grids, as surface topographies accumulate when processing subsequent layers, leading to shunts between the top and bottom printed metallic electrodes. Here we demonstrate how aqueous nanoparticle based silver inks can be employed as printed front electrodes using several different roll-to-roll techniques. We thus compare hexagonal silver grids prepared using either roll-to-roll inkjet or roll-to-roll flexographic printing. Both inkjet and flexo grids present a raised topography and were found to perform differently due to only the conductivity of the obtained silver grid. The raised topographies were compared with a roll-to-roll thermally imprinted grid that was filled with silver in a roll-to-roll process, thus presenting an embedded topography. The embedded grid and the flexo grid were found to perform equally well, with the flexographic technique currently presenting the fastest processing and the lowest silver use, whereas the embedded grid presents the maximally achievable optical transparency and conductivity. Polymer solar cells were prepared in the same step, using roll-to-roll slot-die coating of zinc oxide as the electron transport layer, poly-3-hexylthiophene:phenyl-C(61)-butyric acid methyl ester (P3HT:PCBM) as the active layer and poly(3,4-ethylenedioxythiophene):poly(styrenesulfonate) (PEDOT:PSS) as the top electrode, along with a flat bed screen printed silver grid. The power conversion efficiency (PCE) obtained for large area devices (6 cm(2)) was 1.84%, 0.79% and 1.72%, respectively, for thermally imprinted, inkjet and flexographic silver grids, tested outside under the real sun. Central to all three approaches was that they employed environmentally friendly solvents, i.e. water based nanoparticle silver inks.