LMGAN: Linguistically Informed Semi-Supervised GAN with Multiple Generators.

Semi-supervised learning is one of the active research topics these days. There is a trial that solves semi-supervised text classification with a generative adversarial network (GAN). However, its generator has a limitation in producing fake data distributions that are similar to real data distributions. Since the real data distribution is frequently changing, the generator could not create adequate fake data. To overcome this problem, we present a novel approach for semi-supervised learning for text classification based on generative adversarial networks, Linguistically Informed SeMi-Supervised GAN with Multiple Generators, LMGAN. LMGAN uses trained bidirectional encoder representations from transformers (BERT) and the discriminator from GAN-BERT. In addition, LMGAN has multiple generators and utilizes the hidden layers of BERT. To reduce the discrepancy between the distribution of fake data and real data distribution, LMGAN uses fine-tuned BERT and the discriminator from GAN-BERT. However, since injecting fine-tuned BERT could induce incorrect fake data distribution, we utilize linguistically meaningful intermediate hidden layer outputs of BERT to enrich fake data distribution. Our model shows well-distributed fake data compared to the earlier GAN-based approach that failed to generate adequate high-quality fake data. Moreover, we can get better performances with extremely limited amounts of labeled data, up to 20.0%, compared to the baseline GAN-based model.

UAT: Universal Attention Transformer for Video Captioning.

Video captioning via encoder-decoder structures is a successful sentence generation method. In addition, using various feature extraction networks for extracting multiple features to obtain multiple kinds of visual features in the encoding process is a standard method for improving model performance. Such feature extraction networks are weight-freezing states and are based on convolution neural networks (CNNs). However, these traditional feature extraction methods have some problems. First, when the feature extraction model is used in conjunction with freezing, additional learning of the feature extraction model is not possible by exploiting the backpropagation of the loss obtained from the video captioning training. Specifically, this blocks feature extraction models from learning more about spatial information. Second, the complexity of the model is further increased when multiple CNNs are used. Additionally, the author of Vision Transformers (ViTs) pointed out the inductive bias of CNN called the local receptive field. Therefore, we propose the full transformer structure that uses an end-to-end learning method for video captioning to overcome this problem. As a feature extraction model, we use a vision transformer (ViT) and propose feature extraction gates (FEGs) to enrich the input of the captioning model through that extraction model. Additionally, we design a universal encoder attraction (UEA) that uses all encoder layer outputs and performs self-attention on the outputs. The UEA is used to address the lack of information about the video's temporal relationship because our method uses only the appearance feature. We will evaluate our model against several recent models on two benchmark datasets and show its competitive performance on MSRVTT/MSVD datasets. We show that the proposed model performed captioning using only a single feature, but in some cases, it was better than the others, which used several features.

Effective Pre-Training Method and Its Compositional Intelligence for Image Captioning.

With the increase in the performance of deep learning models, the model parameter has increased exponentially. An increase in model parameters leads to an increase in computation and training time, i.e., an increase in training cost. To reduce the training cost, we propose Compositional Intelligence (CI). This is a reuse method that combines pre-trained models for different tasks. Since the CI uses a well-trained model, good performance and small training cost can be expected in the target task. We applied the CI to the Image Captioning task. Compared to using a trained feature extractor, the caption generator is usually trained from scratch. On the other hand, we pre-trained the Transformer model as a caption generator and applied CI, i.e., we used a pre-trained feature extractor and a pre-trained caption generator. To compare the training cost of the From Scratch model and the CI model, early stopping was applied during fine-tuning of the image captioning task. On the MS-COCO dataset, the vanilla image captioning model reduced training cost by 13.8% and improved performance by up to 3.2%, and the Object Relation Transformer model reduced training cost by 21.3%.

Enhancing Targeted Minority Class Prediction in Sentence-Level Relation Extraction.

Sentence-level relation extraction (RE) has a highly imbalanced data distribution that about 80% of data are labeled as negative, i.e., no relation; and there exist minority classes (MC) among positive labels; furthermore, some of MC instances have an incorrect label. Due to those challenges, i.e., label noise and low source availability, most of the models fail to learn MC and get zero or very low F1 scores on MCs. Previous studies, however, have rather focused on micro F1 scores and MCs have not been addressed adequately. To tackle high mis-classification errors for MCs, we introduce (1) a minority class attention module (MCAM), and (2) effective augmentation methods specialized in RE. MCAM calculates the confidence scores on MC instances to select reliable ones for augmentation, and aggregates MCs information in the process of training a model. Our experiments show that our methods achieve a state-of-the-art F1 scores on TACRED as well as enhancing minority class F1 score dramatically.

HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks.

As the performance of devices that conduct large-scale computations has been rapidly improved, various deep learning models have been successfully utilized in various applications. Particularly, convolution neural networks (CNN) have shown remarkable performance in image processing tasks such as image classification and segmentation. Accordingly, more stable and robust optimization methods are required to effectively train them. However, the traditional optimizers used in deep learning still have unsatisfactory training performance for the models with many layers and weights. Accordingly, in this paper, we propose a new Adam-based hybrid optimization method called HyAdamC for training CNNs effectively. HyAdamC uses three new velocity control functions to adjust its search strength carefully in term of initial, short, and long-term velocities. Moreover, HyAdamC utilizes an adaptive coefficient computation method to prevent that a search direction determined by the first momentum is distorted by any outlier gradients. Then, these are combined into one hybrid method. In our experiments, HyAdamC showed not only notable test accuracies but also significantly stable and robust optimization abilities when training various CNN models. Furthermore, we also found that HyAdamC could be applied into not only image classification and image segmentation tasks.

Probing Out-of-Plane Charge Transport in Black Phosphorus with Graphene-Contacted Vertical Field-Effect Transistors.

Black phosphorus (BP) has recently emerged as a promising narrow band gap layered semiconductor with optoelectronic properties that bridge the gap between semimetallic graphene and wide band gap transition metal dichalcogenides such as MoS2. To date, BP field-effect transistors have utilized a lateral geometry with in-plane transport dominating device characteristics. In contrast, we present here a vertical field-effect transistor geometry based on a graphene/BP van der Waals heterostructure. The resulting device characteristics include high on-state current densities (>1600 A/cm(2)) and current on/off ratios exceeding 800 at low temperature. Two distinct charge transport mechanisms are identified, which are dominant for different regimes of temperature and gate voltage. In particular, the Schottky barrier between graphene and BP determines charge transport at high temperatures and positive gate voltages, whereas tunneling dominates at low temperatures and negative gate voltages. These results elucidate out-of-plane electronic transport in BP and thus have implications for the design and operation of BP-based van der Waals heterostructures.

A New Designed Expander Supported by Spike Miniscrews With Enhanced Stability.

INTRODUCTION: This article represents a new design of expander-spike expander-to correct transverse maxillary deficiency. This new type appliance is easy to fabricate, more hygienic, and less soft-tissue irritation because of elimination of acrylic plate. In addition, because of the position of mini-implant, it can provide a relatively parallel alveolar expansion, compared with other types of expander. MATERIALS AND METHODS: The spike expander consists of 2 parts: 4 spike screws and an expansion screw. Four spike screws were placed on the palatal slope. After installation of the spike screws, based on the position of spike screw, the extension arms of expansion screw were bended on the cast model, and connected to the spike screws by adding resin. The expansion screw was turned once a day (0.25 mm/d), and the process was terminated at 6 weeks. RESULTS: The stability and design of the spike screw enable widening of maxilla easier. Compared with other expander, spike expander reduces the forces being placed directly on teeth, which can maximize skeletal expansion, minimize dental tipping, produce a relatively parallel expansion, and reduce soft-tissue irritation caused by acrylic pads. Furthermore, the teeth can be aligned and leveled simultaneously using the fixed appliance because teeth are free of contact by the expander. And the best point for spike expander is that it is simply fabricated and removed. CONCLUSIONS: The design of spike expander is combining spike screws and an expansion screw, which can provide a good and easy treatment method for transverse maxillary deficiency.

Numerical reconstruction of a full parallax holographic stereogram with radial distortion.

Successful commercialization of holographic printers based on holographic stereograms requires a tool for their numerical replaying and quality assessment before the time-consuming and expensive process of holographic recording. A holographic stereogram encodes 2D images of a 3D scene that are incoherently captured from multiple perspectives and rearranged before recording. This study presents a simulator which builds a full parallax and full color white light viewable holographic stereogram from the perspective images captured by a virtual recentering camera with its further numerical reconstruction for any viewer location. By tracking all steps from acquisition to recording, the simulator allows for analysis of radial distortions caused by the optical elements used at the recording stage. Numerical experiments conducted at increasing degree of pincushion distortion proved its insignificant influence on the reconstructed images in all practical cases by using a peak signal-to-noise ratio and the structural similarity as an image quality metrics.

Accurate registration of cone-beam computed tomography scans to 3-dimensional facial photographs.

INTRODUCTION: Registering a 3-dimensional (3D) facial surface scan to a cone-beam computed tomography (CBCT) scan has various advantages. One major advantage is to compensate for the inaccuracy of the CBCT surface data. However, when registering CBCT and 3D facial scans, changes in facial expression, spatial soft-tissue changes, and differences in the patient's positioning can decrease the accuracy of the registration. In this study, we introduce a new 3D facial scanner that is combined with a CBCT apparatus. Our goal was to evaluate the registration accuracy of CBCT and 3D facial scans, which were taken with the shortest possible time between them. METHODS: The experiment was performed with 4 subjects. Each patient was instructed to hold as still as possible while the CBCT scan was taken, followed immediately by the 3D facial surface scan. The images were automatically registered with software. The accuracy was measured by determining the degree of agreement between the soft-tissue surfaces of the CBCT and the 3D facial images. RESULTS: The average surface discrepancy between the CBCT facial surface and 3D facial surface was 0.60 mm (SD, 0.12 mm). Registration accuracy was also visually verified by toggling between the images of the CBCT and 3D facial surface scans while rotating the registered images. CONCLUSIONS: Registration of consecutively taken CBCT and 3D facial images resulted in reliable accuracy.

Physically defined long-term and short-term synapses for the development of reconfigurable analog-type operators capable of performing health care tasks.

Extracting valuable information from the overflowing data is a critical yet challenging task. Dealing with high volumes of biometric data, which are often unstructured, nonstatic, and ambiguous, requires extensive computer resources and data specialists. Emerging neuromorphic computing technologies that mimic the data processing properties of biological neural networks offer a promising solution for handling overflowing data. Here, the development of an electrolyte-gated organic transistor featuring a selective transition from short-term to long-term plasticity of the biological synapse is presented. The memory behaviors of the synaptic device were precisely modulated by restricting ion penetration through an organic channel via photochemical reactions of the cross-linking molecules. Furthermore, the applicability of the memory-controlled synaptic device was verified by constructing a reconfigurable synaptic logic gate for implementing a medical algorithm without further weight-update process. Last, the presented neuromorphic device demonstrated feasibility to handle biometric information with various update periods and perform health care tasks.

Effect of heparin and alendronate coating on titanium surfaces on inhibition of osteoclast and enhancement of osteoblast function.

The failure of orthopedic and dental implants has been attributed mainly to loosening of the implant from host bone, which may be due to weak bonding of the implant material to bone tissue. Titanium (Ti) is used in the field of orthopedic and dental implants because of its excellent biocompatibility and outstanding mechanical properties. Therefore, in the field of materials science and tissue engineering, there has been extensive research to immobilize bioactive molecules on the surface of implant materials in order to provide the implants with improved adhesion to the host bone tissue. In this study, chemically active functional groups were introduced on the surface of Ti by a grafting reaction with heparin and then the Ti was functionalized by immobilizing alendronate onto the heparin-grafted surface. In the MC3T3-E1 cell osteogenic differentiation study, the alendronate-immobilized Ti substrates significantly enhanced alkaline phosphatase activity (ALP) and calcium content. Additionally, nuclear factor kappa B ligand (RANKL)-induced osteoclast differentiation of RAW264.7 cells was inhibited with the alendronate-immobilized Ti as confirmed by TRAP analysis. Real time PCR analysis showed that mRNA expressions of osteocalcin and osteopontin, which are markers for osteogenesis, were upregulated in MC3T3-E1 cells cultured on alendronate-immobilized Ti. The mRNA expressions of TRAP and Cathepsin K, markers for osteoclastogenesis, in RAW264.7 cells cultured on alendronate-immobilized Ti were down-regulated. Our study suggests that alendronate-immobilized Ti may be a bioactive implant with dual functions to enhance osteoblast differentiation and to inhibit osteoclast differentiation simultaneously.

Rectifying optoelectronic memory based on WSe2/graphene heterostructures.

van der Waals heterostructures composed of two-dimensional materials vertically stacked have been extensively studied to develop various multifunctional devices. Here, we report WSe2/graphene heterostructure devices with a top floating gate that can serve as multifunctional devices. They exhibit gate-controlled rectification inversion, rectified nonvolatile memory effects, and multilevel optoelectronic memory effects. Depending on the polarity of the gate voltage pulses (V Gp), electrons or holes can be trapped in the floating gate, resulting in rectified nonvolatile memory properties. Furthermore, upon repeated illumination with laser pulses, positive or negative staircase photoconductivity is observed depending on the history of V Gp, which is ascribed to the tunneling of electrons or holes between the WSe2 channel and the floating gate. These multifunctional devices can be used to emulate excitatory and inhibitory synapses that have different neurotransmitters. Various synaptic functions, such as potentiation/depression curves and spike-timing-dependent plasticity, have been also implemented using these devices. In particular, 128 optoelectronic memory states with nonlinearity less than 1 can be achieved by controlling applied laser pulses and V Gp, suggesting that the WSe2/graphene heterostructure devices with a top floating gate can be applied to optoelectronic synapse devices.

Ultrasound-guided sialo-irrigation for the treatment of chronic sialodochitis with sialolithiasis.

Sialolithiasis is one of the most common causes of salivary duct obstruction. In the last 20 years, minimally invasive procedures like sialendoscopy, extracorporeal lithotripsy, and basket snaring are increasingly being used for the treatment of salivary gland duct stones. Sialo-irrigation of the salivary gland is an effective procedure for treating inflammation and providing symptomatic relief. This procedure can be employed for the treatment of sialolithiasis using the back pressure of instilled saline. Sialo-irrigation under ultrasound (US) guidance allows for dynamic studies showing real-time images during diagnostic or surgical procedure and can be used for the removal of sialoliths. In addition, it can also be used to remove primitive sialoliths and microliths by washing out the ductal system, which prevents the recurrence of sialoliths. The aim of this study was to propose a minimally invasive technique for sialolithiasis using US-guided sialo-irrigation.

Radiation dosimetry analyses of radiographic imaging systems used for orthodontic treatment: comparison among child, adolescent, and adult patients.

OBJECTIVES: The aim of this study was to compare the effective doses of orthodontic radiographs in children, adolescents, and adults. METHODS: We exposed a child, an adolescent (simulated by an adult female phantom), and adult male phantoms using common scanning protocols for panoramic radiography, cephalography, and cone-beam computed tomography (CBCT). Glass dosimeters were placed in the organs of the phantom to measure the absorbed doses. The effective doses were deduced using tissue weighting factors as defined in the ICRP Publication 103. RESULTS: For panoramic imaging, the parotid gland had the highest absorbed dose in the child and the submandibular glands had the highest absorbed dose in both the adolescent and adult phantoms. For cephalography, the organs and tissues located closest to the X-ray tube had the highest absorbed dose values. For CBCT, the lenses of the eyes received the highest absorbed dose. Effective doses with CBCT were the greatest in the adolescent phantom, followed by in the adult and child phantoms. CONCLUSIONS: Dental practitioners should be aware of patient age, as younger patients will incur greater risks from radiation.

Root proximity of the anchoring miniscrews of orthodontic miniplates in the mandibular incisal area: Cone-beam computed tomographic analysis.

OBJECTIVE: This outcome analysis study evaluated the actual positions of the orthodontic miniplate and miniplate anchoring screws (MPASs) and the risk factors affecting adjacent anatomic structures after miniplate placement in the mandibular incisal area. METHODS: Cone-beam computed tomographic images of 97 orthodontic miniplates and their 194 MPASs (diameter, 1.5 mm; length, 4 mm) in patients whose miniplates provided sufficient clinical stability for orthodontic treatment were retrospectively reviewed. For evaluating the actual positions of the miniplates and analyzing the risk factors, including the effects on adjacent roots, MPAS placement height (PH), placement depth (PD), plate angle (PA), mental fossa angle (MA), and root proximity were assessed using the paired t-test, analysis of variance, and generalized linear model and regression analyses. RESULTS: The mean PDs of MPASs at positions 1 (P1) and 2 (P2) were 2.01 mm and 2.23 mm, respectively. PA was significantly higher in the Class III malocclusion group than in the other groups. PH was positively correlated with MA and PD at P1. Of the 97 MPASs at P1, 49 were in the no-root area and 48 in the dentulous area; moreover, 19 showed a degree of root contact (19.6%) without root perforation. All MPASs at P2 were in the no-root area. CONCLUSIONS: Positioning the miniplate head approximately 1 mm lower than the mucogingival junction is highly likely to provide sufficient PH for the P1- MPASs to be placed in the no-root area.

Artificial stimulus-response system capable of conscious response.

A stimulus-response system and conscious response enable humans to respond effectively to environmental changes and external stimuli. This paper presents an artificial stimulus-response system that is inspired by human conscious response and is capable of emulating it. The system is composed of an artificial visual receptor, artificial synapse, artificial neuron circuits, and actuator. By incorporating these artificial nervous components, a series of conscious response processes that markedly reduces response time as a result of learning from repeated stimuli are demonstrated. The proposed artificial stimulus-response system offers the promise of a new research field that would aid the development of artificial intelligence-based organs for patients with neurological disorders.

Effectiveness of 2D radiographs in detecting CBCT-based incidental findings in orthodontic patients.

Some craniofacial diseases or anatomical variations are found in radiographic images taken for other purposes. These incidental findings (IFs) can be detected in orthodontic patients, as various radiographs are required for orthodontic diagnosis. The radiographic data of 1020-orthodontic patients were interpreted to evaluate the rates of IFs in three-dimensional (3D) cone-beam-computed tomography (CBCT) with a large field of view (FOV) and investigate the effectiveness and accuracy of two-dimensional (2D) radiographs for detecting IFs compared to CBCT. Prevalence and accuracy in five areas was measured for sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). The accuracies of various 2D-radiograph were compared through a proportion test. A total of 709-cases (69.5%) of 1020-subjects showed one or more IFs in CBCT images. Nasal cavity was the most affected area. Based on the CBCT images as a gold standard, different accuracies of various 2D-radiographs were observed in each area of the findings. The highest accuracy was confirmed in soft tissue calcifications with comprehensive radiographs. For detecting nasal septum deviations, postero-anterior cephalograms were the most accurate 2D radiograph. In cases the IFs were not determined because of its ambiguity in 2D radiographs, considering them as an absence of findings increased the accuracy.

Cone-beam computed tomography evaluation of mini-implants after placement: Is root proximity a major risk factor for failure?

INTRODUCTION: The purposes of this study were to determine factors favoring successful mini-implant placement and to evaluate root proximity as a possible risk factor for failure of osseointegration-based mini-implants during orthodontic treatment. METHODS: Three-dimensional cone-beam computed tomography images were used to examine 50 sandblasted, large-grit, and acid-etched surface-treated mini-implants (C-implant, Seoul, Korea) placed in 25 patients. The images were analyzed for 3-dimensional position of the mini-implant (placement angle and depth) and any contact with root surfaces or maxillary sinuses. RESULTS: There were no remarkable differences in horizontal placement angles in the axial plane and placement depths of the mini-implants, but the vertical placement angle was significantly higher on the left side (24.5 degrees +/- 11.0 degrees ) compared with the right side (11.8 degrees +/- 11.6 degrees ). The horizontal mini-implant placement angle had a greater inclination tendency toward the maxillary first molar, and 11 mini-implants with root proximity showed mesiobuccal contact with the maxillary first molar root. Only 1 failure in 15 mini-implants with root proximity and 1 failure in 35 without root proximity were observed on the images. CONCLUSIONS: Root proximity alone was not considered a major risk factor for osseointegration-based mini-implant failure.

Vertical organic synapse expandable to 3D crossbar array.

Recently, three-terminal synaptic devices have attracted considerable attention owing to their nondestructive weight-update behavior, which is attributed to the completely separated terminals for reading and writing. However, the structural limitations of these devices, such as a low array density and complex line design, are predicted to result in low processing speeds and high energy consumption of the entire system. Here, we propose a vertical three-terminal synapse featuring a remote weight update via ion gel, which is also extendable to a crossbar array structure. This synaptic device exhibits excellent synaptic characteristics, which are achieved via precise control of ion penetration onto the vertical channel through the weight-control terminal. Especially, the applicability of the developed vertical organic synapse array to neuromorphic computing is demonstrated using a simple crossbar synapse array. The proposed synaptic device technology is expected to be an important steppingstone to the development of high-performance and high-density neural networks.

Leiomyosarcoma of the jaw: case series.

OBJECTIVES: Leiomyosarcoma is a malignant neoplasm that affects smooth muscle tissue and it is very rare in the field of oral and maxillofcial surgery. The purpose of this study was to obtain information on diagnosis of and treatment methods for leiomyosarcoma by retrospectively reviewing of the cases. PATIENTS AND METHODS: The study included nine patients who were diagnosed with leiomyosarcoma in the Department of Oral and Maxillofacial Surgery at Seoul National University Dental Hospital. The subjects were analyzed with respect to sex, age, clinical features, primary site of disease, treatment method, recurrence, and metastasis. RESULTS: Particular clinical features included pain, edema, mouth-opening limitations, dysesthesia, and enlarged lymph nodes. All cases except one were surgically treated, and recurrence was found in two cases. Four of nine patients were followed up without recurrence and one patient underwent additional surgery due to recurrence. CONCLUSION: In our case series, notable symptoms included pain, edema, mouth-opening limitations, and dysesthesia; however, it was difficult to label these as specific symptoms of leiomyosarcoma. Considering the aggressive characteristics of the disease and poor prognosis, surgical treatment is necessary with careful consideration of postoperative radiotherapy and chemotherapy.