Oversampling effect in pretraining for bidirectional encoder representations from transformers (BERT) to localize medical BERT and enhance biomedical BERT.

BACKGROUND: Pretraining large-scale neural language models on raw texts has made a significant contribution to improving transfer learning in natural language processing. With the introduction of transformer-based language models, such as bidirectional encoder representations from transformers (BERT), the performance of information extraction from free text has improved significantly in both the general and medical domains. However, it is difficult to train specific BERT models to perform well in domains for which few databases of a high quality and large size are publicly available. OBJECTIVE: We hypothesized that this problem could be addressed by oversampling a domain-specific corpus and using it for pretraining with a larger corpus in a balanced manner. In the present study, we verified our hypothesis by developing pretraining models using our method and evaluating their performance. METHODS: Our proposed method was based on the simultaneous pretraining of models with knowledge from distinct domains after oversampling. We conducted three experiments in which we generated (1) English biomedical BERT from a small biomedical corpus, (2) Japanese medical BERT from a small medical corpus, and (3) enhanced biomedical BERT pretrained with complete PubMed abstracts in a balanced manner. We then compared their performance with those of conventional models. RESULTS: Our English BERT pretrained using both general and small medical domain corpora performed sufficiently well for practical use on the biomedical language understanding evaluation (BLUE) benchmark. Moreover, our proposed method was more effective than the conventional methods for each biomedical corpus of the same corpus size in the general domain. Our Japanese medical BERT outperformed the other BERT models built using a conventional method for almost all the medical tasks. The model demonstrated the same trend as that of the first experiment in English. Further, our enhanced biomedical BERT model, which was not pretrained on clinical notes, achieved superior clinical and biomedical scores on the BLUE benchmark with an increase of 0.3 points in the clinical score and 0.5 points in the biomedical score. These scores were above those of the models trained without our proposed method. CONCLUSIONS: Well-balanced pretraining using oversampling instances derived from a corpus appropriate for the target task allowed us to construct a high-performance BERT model.

Diverse distribution patterns of segmental longitudinal strain are associated with different clinical features and outcomes in dilated cardiomyopathy.

BACKGROUND: Dilated cardiomyopathy (DCM) presents with diverse clinical courses, hardly predictable solely by the left ventricular (LV) ejection fraction (EF). Longitudinal strain (LS) offers distinct information from LVEF and exhibits various distribution patterns. This study aimed to evaluate the clinical significance of LS distribution patterns in DCM. METHODS: We studied 139 patients with DCM (LVEF ≤ 35%) who were admitted for heart failure (HF). LS distribution was assessed using a bull's eye map and the relative apical LS index (RapLSI), calculated by dividing apical LS by the sum of basal and mid-LS values. We evaluated the associations of LS distribution with cardiac events (cardiac death, LV assist device implantation, or HF hospitalization) and LV reverse remodeling (LVRR), as indicated by subsequent LVEF changes. RESULTS: Twenty six (19%) and 29 (21%) patients exhibited a pattern of relatively apical impaired or preserved LS (defined by RapLSI < 0.25 or > 0.75, signifying a 50% decrease or increase in apical LS compared to other segments), and the remaining patients exhibited a scattered/homogeneously impaired LS pattern. The proportion of new-onset heart failure and LVEF differed between the three groups. During the median 595-day follow-up, patients with relatively-impaired apical LS had a higher rate of cardiac events (both log-rank p < 0.05) and a lower incidence of LVRR (both p < 0.01) compared to patients with other patterns. RapLSI was significantly associated with cardiac event rates after adjusting for age, sex, and new-onset HF or global LS. CONCLUSION: DCM patients with reduced EF and distinct distribution patterns of impaired LS experienced different outcomes.

Development of the Clinical Specimen Information Management System for Multicenter Clinical Studies.

Some multicenter clinical studies require the acquisition of clinical specimens from patients, and the centralized management and analysis of clinical specimens at a research institution. In such cases, it is necessary to manage clinical specimens with anonymized patient information. In addition, clinical specimens need to be managed in connection with clinical information in clinical studies. In this study, we have developed a clinical specimen information management system that works with electronic data capture system for efficient specimen information management and the system workflow has verified at Osaka University Hospital. In addition, by combining this system with medical image collection system that we have developed previously, the integrated management of clinical information, medical image, and clinical specimen information will become possible. This specimen information management system may be expected to provide the platform for integrated analysis utilizing clinical information, medical image, and data from clinical specimens in multicenter clinical studies.

Classification of Diagnostic Certainty in Radiology Reports with Deep Learning.

A radiology report is prepared for communicating clinical information about observed abnormal structures and clinically important findings with referring clinicians. However, such observations and findings are often accompanied by ambiguous expressions, which can prevent clinicians from accurately interpreting the content of reports. To systematically assess the degree of diagnostic certainty for each observation and finding in a report, we defined an ordinal scale comprising five classes: definite, likely, may represent, unlikely, and denial. Furthermore, we applied a deep learning classification model to determine its applicability to in-house radiology reports. We trained and evaluated the model using 540 in-house chest computed tomography reports. The deep learning model achieved a micro F1-score of 97.61%, which indicated that our ordinal scale was suitable for measuring the diagnostic certainty of observations and findings in a report.

Implementation of a Multilingual Electronic Medical Questionnaire: For Use Inside or Outside the Hospital.

We implemented a multilingual medical questionnaire system, which allows patients to answer questionnaires both in and out of the hospital. The response data are sent to and stored as structured data on the server in hospital information system, and could be converted to Japanese and quoted as part of progress notes in the electronic medical record.

Extracting Clinical Information From Japanese Radiology Reports Using a 2-Stage Deep Learning Approach: Algorithm Development and Validation.

Background: Radiology reports are usually written in a free-text format, which makes it challenging to reuse the reports. Objective: For secondary use, we developed a 2-stage deep learning system for extracting clinical information and converting it into a structured format. Methods: Our system mainly consists of 2 deep learning modules: entity extraction and relation extraction. For each module, state-of-the-art deep learning models were applied. We trained and evaluated the models using 1040 in-house Japanese computed tomography (CT) reports annotated by medical experts. We also evaluated the performance of the entire pipeline of our system. In addition, the ratio of annotated entities in the reports was measured to validate the coverage of the clinical information with our information model. Results: The microaveraged F1-scores of our best-performing model for entity extraction and relation extraction were 96.1% and 97.4%, respectively. The microaveraged F1-score of the 2-stage system, which is a measure of the performance of the entire pipeline of our system, was 91.9%. Our system showed encouraging results for the conversion of free-text radiology reports into a structured format. The coverage of clinical information in the reports was 96.2% (6595/6853). Conclusions: Our 2-stage deep system can extract clinical information from chest and abdomen CT reports accurately and comprehensively.

Impact of an Electronic Medical Record-Connected Questionnaire on Efficient Nursing Documentation: Usability and Efficacy Study.

BACKGROUND: Documentation tasks comprise a large percentage of nurses' workloads. Nursing records were partially based on a report from the patient. However, it is not a verbatim transcription of the patient's complaints but a type of medical record. Therefore, to reduce the time spent on nursing documentation, it is necessary to assist in the appropriate conversion or citation of patient reports to professional records. However, few studies have been conducted on systems for capturing patient reports in electronic medical records. In addition, there have been no reports on whether such a system reduces the time spent on nursing documentation. OBJECTIVE: This study aims to develop a patient self-reporting system that appropriately converts data to nursing records and evaluate its effect on reducing the documenting burden for nurses. METHODS: An electronic medical record-connected questionnaire and a preadmission nursing questionnaire were administered. The questionnaire responses entered by the patients were quoted in the patient profile for inpatient assessment in the nursing system. To clarify its efficacy, this study examined whether the use of the electronic questionnaire system saved the nurses' time entering the patient profile admitted between August and December 2022. It also surveyed the usability of the electronic questionnaire between April and December 2022. RESULTS: A total of 3111 (78%) patients reported that they answered the electronic medical questionnaire by themselves. Of them, 2715 (88%) felt it was easy to use and 2604 (85%) were willing to use it again. The electronic questionnaire was used in 1326 of 2425 admission cases (use group). The input time for the patient profile was significantly shorter in the use group than in the no-use group (P<.001). Stratified analyses showed that in the internal medicine wards and in patients with dependent activities of daily living, nurses took 13%-18% (1.3 to 2 minutes) less time to enter patient profiles within the use group (both P<.001), even though there was no difference in the amount of information. By contrast, in the surgical wards and in the patients with independent activities of daily living, there was no difference in the time to entry (P=.50 and P=.20, respectively), but there was a greater amount of information in the use group. CONCLUSIONS: The study developed and implemented a system in which self-reported patient data were captured in the hospital information network and quoted in the nursing system. This system contributes to improving the efficiency of nurses' task recordings.

Practical use of a multicenter clinical research support system connected to electronic medical records.

BACKGROUND: Electronic medical records (EMRs) are widely used, but in many cases, they are used within a network physically separated from the Internet. Multicenter clinical studies use Internet-connected electronic data capture (EDC) systems to collect data, where data entered into the EMR are manually transcribed into the EDC system. In addition, medical images for clinical research are also collected manually. Variations in EMRs and differing data structures among vendors hamper the use of data for clinical research. METHODS: We solved this problem by developing a network infrastructure for clinical research between Osaka University Hospital and affiliated hospitals in the Osaka area and introducing a clinical data collection system (CDCS). In each hospital's EMR network, we implemented a CRF reporter that accumulated data for clinical research using a template and then sent the data to a management server in the Osaka University Hospital Data Center. To organize the patient profile data and clinical laboratory data stored in each EMR for use in clinical research, the data are retrieved from the template by an interface module developed by each vendor, according to our common data output interface specification. The data entered into the CRF reporter template for clinical research are also recorded in the EMR progress notes and sent to the data management server. This network infrastructure can also be used as a medical image collection system that automatically collects images for research from PACS at each hospital. These systems are managed under common subject numbers issued by the CDCS. RESULTS: A network infrastructure was established among 19 hospitals, and a CRF reporter was incorporated into the EMR. A medical image transfer system was introduced in 13 hospitals. Since 2013, 28 clinical studies have been conducted using this system, and data for 9,987 cases have been collected as of December 31, 2020. CONCLUSION: Incorporating a CRF reporter with medical image transfer system into the EMR has proven useful for collecting research data.

Automatic creation of annotations for chest radiographs based on the positional information extracted from radiographic image reports.

BACKGROUND AND OBJECTIVE: In this study, we tried to create a machine-learning method that detects disease lesions from chest X-ray (CXR) images using a data set annotated with extracted CXR reports information. We set the nodule as the target disease lesion. Manually annotating nodules is costly in terms of time. Therefore, we used the report information to automatically produce training data for the object detection task. METHODS: First, we use semantic segmentation model PSP-Net to recognize lung fields described in the CXR reports. Next, a classification model ResNeSt-50 is used to discriminate the nodule in segmented right and left field. It also can provide attention map by Grad-Cam. If the attention region corresponds to the location of the nodule in the CXR reports, an attention bounding box is generated. Finally, object detection model Faster-RCNN was performed using generated attention bounding box. The bounding boxes predicted by Faster-RCNN were filtered to satisfy the location extracted from CXR reports. RESULTS: For lung field segmentation, a mean intersection of union of 0.889 was achieved in our best model. 15,156 chest radiographs are used for classification. The area under the receiver operating characteristics curve was 0.843 and 0.852 for the left and right lung, respectively. The detection precision of the generated attention bounding box was 0.341 to 0.531 depending on the binary setting for attention map. Through object detection process, the detection precisions of the bounding boxes were improved to 0.567 to 0.800. CONCLUSION: We successfully generated bounding boxes with nodule on CXR images based on the positional information of the diseases extracted from the CXR reports. Our method has the potential to provide bounding boxes for various lung lesions which can reduce the annotation burden for specialists. SHORT ABSTRACT: Machine learning for computer aided image diagnosis requires annotation of images, but manual annotation is time-consuming for medical doctor. In this study, we tried to create a machine-learning method that creates bounding boxes with disease lesions on chest X-ray (CXR) images using the positional information extracted from CXR reports. We set the nodule as the target lesion. First, we use PSP-Net to segment the lung field according to the CXR reports. Next, a classification model ResNeSt-50 was used to discriminate the nodule in segmented lung field. We also created an attention map using the Grad-Cam algorithm. If the area of attention matched the area annotated by the CXR report, the coordinate of the bounding box was considered as a possible nodule area. Finally, we used the attention information obtained from the nodule classification model and let the object detection model trained by all of the generated bounding boxes. Through object detection model, the precision of the bounding boxes to detect nodule is improved.

Focal severe decrease in myocardial technetium-99 m sestamibi uptake indicates ventricular irreversibility in patients with dilated cardiomyopathy.

OBJECTIVE: Technetium-99 m sestamibi (99mTc-MIBI) scintigraphy can identify non-viable left ventricular (LV) myocardium. However, the optimal cut-off value and the details of decreased 99mTc-MIBI uptake of the non-viable LV myocardium in patients with dilated cardiomyopathy (DCM) have not been well established. This study aimed to evaluate the decrease in 99mTc-MIBI uptake in each segment and in the whole LV myocardium, and to determine cut-off values for identifying non-viable LV myocardium in DCM patients. METHODS: Overall, 53 DCM patients with reduced LV ejection fraction (LVEF ≤ 40%) who underwent 99mTc-MIBI scintigraphy and any optimization of heart failure treatments were evaluated. LV myocardium was classified as viable or non-viable based on the absolute increase in LVEF of ≥ 10% unit leading to an LVEF of > 40% at follow-up, respectively. The decrease in myocardial 99mTc-MIBI uptake in each of the 17 segments was evaluated using three indices determined by different thresholds or standard references: segmental %uptake, rest score, and defect extent. Changes in the whole LV myocardium were evaluated by the minimum %uptake, and the summed rest score (SRS) and extent of LV defect were obtained using summed data of 17 segments. RESULTS: Segmental evaluation indicated a mild decrease in 99mTc-MIBI uptake in 18 patients with viable LV myocardium, whereas focal severe decrease in uptake was observed in patients with non-viable LV myocardium. In the receiver-operating characteristic curve analysis, the cut-off values of minimum %uptake, SRS, and LV defect extent for predicting non-viable LV were 39% (p < 0.01, area under the curve [AUC]: 0.87), 10 (p < 0.01, AUC: 0.91), and 23% (p < 0.01, AUC: 0.92), respectively. CONCLUSIONS: In DCM patients, myocardial 99mTc-MIBI %uptake of < 40% indicated non-viable myocardium. The focal and severe decrease in uptake in approximately more than a quarter of the LV myocardium may indicate non-viable LV.

Extracting clinical terms from radiology reports with deep learning.

Extracting clinical terms from free-text format radiology reports is a first important step toward their secondary use. However, there is no general consensus on the kind of terms to be extracted. In this paper, we propose an information model comprising three types of clinical entities: observations, clinical findings, and modifiers. Furthermore, to determine its applicability for in-house radiology reports, we extracted clinical terms with state-of-the-art deep learning models and compared the results. We trained and evaluated models using 540 in-house chest computed tomography (CT) reports annotated by multiple medical experts. Two deep learning models were compared, and the effect of pre-training was explored. To investigate the generalizability of the model, we evaluated the use of other institutional chest CT reports. The micro F1-score of our best performance model using in-house and external datasets were 95.36% and 94.62%, respectively. Our results indicated that entities defined in our information model were suitable for extracting clinical terms from radiology reports, and the model was sufficiently generalizable to be used with dataset from other institutions.

An Automatic Image Collection System for Multicenter Clinical Studies.

The acquisition of medical images from multiple medial institutions has become important for high-quality clinical studies. In recent years, electronic data submission has enabled the transmission of image data to independent institutions more quickly and easily than before. However, the selection, anonymization, and transmission of medical images still require human resources in the form of clinical research collaborators. In this study, we developed an image collection system that works with the electronic data capture (EDC) system. In this image collection system, medical images are selected based on EDC input information, patient ID is anonymized to a subject ID issued by the EDC, and the selected anonymized images are transferred to the research institute without human intervention. In the research institute, clinical information registered by the EDC and clinical images collected by the image collection system are managed by the same subject ID and can be used for clinical studies. In October 2019, our image collection system was introduced to 13 medical institutions and has now begun collecting medical images from the in-hospital picture archiving and communication system (PACS) of those institutions.

End-to-End Approach for Structuring Radiology Reports.

Radiology reports include various types of clinical information that are used for patient care. Reports are also expected to have secondary uses (e.g., clinical research and the development of decision support systems). For secondary use, it is necessary to extract information from the report and organize it in a structured format. Our goal is to build an application to transform radiology reports written in a free-text form into a structured format. To this end, we propose an end-to-end method that consists of three elements. First, we built a neural network model to extract clinical information from the reports. We experimented on a dataset of chest X-ray reports. Second, we transformed the extracted information into a structured format. Finally, we built a tool that enabled the transformation of terms in reports to standard forms. Through our end-to-end method, we could obtain a structured radiology dataset that was easy to access for secondary use.

Usefulness of an IKr blocker for ablation of non-pulmonary vein ectopies that are unmappable due to easily initiated atrial fibrillation.

PURPOSE: When atrial fibrillation (AF) is initiated by a single or several non-pulmonary vein (PV) trigger ectopic beats, mapping the ectopy is often difficult, requiring a number of electrical cardioversion applications. Nifekalant is a rapidly activating delayed rectifier potassium channel (IKr) blocker which may suppress AF initiation without inhibiting ectopy development, thereby allowing the target ectopy to be mapped. To assess the efficacy of nifekalant in the ablation of non-PV ectopies that are unmappable due to easily initiated AF. METHODS: Eleven consecutive patients were administered nifekalant to map a non-PV ectopy that was unmappable using a conventional method due to easily initiated AF. Nifekalant was intravenously administered as a bolus dose of 0.2 mg/kg, and electrical cardioversion was delivered. Additional boluses of 0.2 mg/kg were repeatedly administered until AF initiation was prevented or until the appearance of significant prolongation of QT interval. RESULTS: AF suppression without inhibition of ectopy development was achieved in 7 patients. These patients had a higher rate of acute elimination of the ectopy than the remaining 4 patients without AF suppression (7 [100%] vs. 1 [25%] patients, p = 0.024). In addition, patients with suppression of AF initiation had a higher AF recurrence-free rate than those without (7 [100%] vs. 1 [25%] patients, p = 0.024). CONCLUSION: Nifekalant administration appears useful in the ablation of non-PV ectopies that easily initiate AF.

Simple Electrocardiographic Score Can Predict Left Ventricular Reverse Remodeling in Patients With Non-Ischemic Cardiomyopathy.

Background: Left ventricular reverse remodeling (LVRR) is a favorable response in non-ischemic, non-valvular cardiomyopathy (NICM) patients. Recently, 18-lead body surface electrocardiography (ECG), the standard 12-lead ECG with synthesized right-sided/posterior chest leads, has been developed, but its predictive value for LVRR has not been evaluated. Methods and Results: Of 216 consecutive hospitalized NICM patients with LV ejection fraction (LVEF) ≤35%, we studied 125 who received optimization of their heart failure treatment and had 18-lead ECG and echocardiography data available for evaluating LVRR, defined as an absolute increase in LVEF ≥10% concomitant with LVEF ≥35% after 1-year optimized treatment. Most 18-lead ECG parameters in the NICM patients differed from those in 312 age- and body mass index-matched subjects with normal echocardiography. LVRR occurred in 59 NICM patients and they had a larger QRS amplitude in the limb leads (I, II, aVR, and aVF), precordial leads (V3-V6), and synthesized leads (syn-V4R-5R), decreased QRS axis and duration, and lower prevalence of fragmented QRS than those without LVRR. The ECG score using 3 selected parameters (QRS amplitude in aVR ≥675 µV; QRS duration <106 ms without fragmentation; and QRS axis <67°) was associated with the incidence of LVRR even after adjusting for optimized treatment. Conclusions: The standard 12-lead ECG parameters are sufficiently predictive of LVRR in NICM patients.

Ratio of pulmonary artery diameter to ascending aortic diameter and severity of heart failure.

BACKGROUND: Treatment decisions in dilated cardiomyopathy (DCM) patients with severe heart failure (HF) and short clinical history are challenging because of the difficulty of determining HF stage or prognosis in the acute HF phase. We hypothesized that persistent decreased systemic or increased pulmonary arterial pressure, including in the sub-clinical phase, might affect the main pulmonary artery diameter (PAD), ascending aortic diameter (AoD), and their ratio (PAD/AoD). This study assessed AoD, PAD, and PAD/AoD by non-contrast computed tomography scans in DCM patients in the acute phase of HF and examined the association of these parameters with their clinical course. METHODS: Of 261 screened individuals, we studied 110 consecutive hospitalized patients with DCM suspected of being in advanced stage of HF and 45 age-matched controls, assessing clinical data and later events (cardiac death or left ventricular assist device implantation). RESULTS: Compared with controls, DCM patients had smaller AoD (26.6 ± 4.4 vs 30.6 ± 2.7 mm) and larger PAD (27.7 ± 3.5 vs 25.4 ± 2.8 mm) and PAD/AoD (1.05 ± 0.14 vs 0.83 ± 0.08; all p < 0.01). DCM patients with high PAD/AoD (median, > 1.05) had more frequent past HF hospitalizations, lower blood pressure, stroke volume, and ejection fraction, higher brain natriuretic peptide levels, smaller AoD, and similar PAD compared with patients with a low PAD/AoD. A higher PAD/AoD was associated with poorer outcomes even after adjusting for age, blood pressure, ejection fraction, or number of hospitalizations. CONCLUSION: Assessment of AoD and PAD may have important clinical implications in determining whether DCM patients are in an advanced stage of HF with a poorer prognosis.

Non-Ischemic Heart Failure With Reduced Ejection Fraction Is Associated With Altered Intestinal Microbiota.

BACKGROUND: Research suggests that heart failure with reduced ejection fraction (HFrEF) is a state of systemic inflammation that may be triggered by microbial products passing into the bloodstream through a compromised intestinal barrier. However, whether the intestinal microbiota exhibits dysbiosis in HFrEF patients is largely unknown.Methods and Results:Twenty eight non-ischemic HFrEF patients and 19 healthy controls were assessed by 16S rRNA analysis of bacterial DNA extracted from stool samples. After processing of sequencing data, bacteria were taxonomically classified, diversity indices were used to examine microbial ecology, and relative abundances of common core genera were compared between groups. Furthermore, we predicted gene carriage for bacterial metabolic pathways and inferred microbial interaction networks on multiple taxonomic levels.Bacterial communities of both groups were dominated by the Firmicutes and Bacteroidetes phyla. The most abundant genus in both groups wasBacteroides. Although α diversity did not differ between groups, ordination by ß diversity metrics revealed a separation of the groups across components of variation.StreptococcusandVeillonellawere enriched in the common core microbiota of patients, whileSMB53was depleted. Gene families in amino acid, carbohydrate, vitamin, and xenobiotic metabolism showed significant differences between groups. Interaction networks revealed a higher degree of correlations between bacteria in patients. CONCLUSIONS: Non-ischemic HFrEF patients exhibited multidimensional differences in intestinal microbial communities compared with healthy subjects.

Routine exercise testing could not predict T-wave oversensing in a patient after a subcutaneous implantable cardioverter-defibrillator implant.

Subcutaneous implantable cardioverter-defibrillators (S-ICDs) are susceptible to T-wave oversensing (TWOS) caused by high rate-dependent QRS-T morphology changes. We experienced an inappropriate S-ICD shock due to TWOS, which could not be predicted by routine exercise testing. A newly available high-pass filter might be effective for avoiding this.

Sodium channel blockade unmasked Brugada electrocardiographic pattern in a patient with complete right bundle branch block and early repolarization in the lateral leads.

Early repolarization syndrome (ERS) and Brugada syndrome (BrS) share many electrocardiographic and clinical features, and recently have been collectively grouped as J wave syndrome. However, the effects of sodium channel blockers on the J waves differ greatly between ERS and BrS.