Construction of a knowledge graph for breast cancer diagnosis based on Chinese electronic medical records: development and usability study.

BACKGROUND: Electronic medical records (EMRs) contain a wealth of information related to breast cancer diagnosis and treatment. Extracting relevant features from these medical records and constructing a knowledge graph can significantly contribute to an efficient data analysis and decision support system for breast cancer diagnosis. METHODS: An approach was proposed to develop a workflow for effectively extracting breast cancer-related features from Chinese breast cancer mammography reports and constructing a knowledge graph for breast cancer diagnosis. Firstly, the concept layer of the knowledge graph for breast cancer diagnosis was constructed based on breast cancer diagnosis and treatment guidelines, along with insights from clinical experts. .Next, a BiLSTM-Highway-CRF model was designed to extract the mammography features, which formed the data layer of the knowledge graph. Finally, the knowledge graph was constructed by combining the concept layer and the data layer in a Neo4j graph data platform, and then applied in visualization analysis, semantic query and computer assisted diagnosis. RESULTS: Mammographic features were extracted from a total of 1171 mammography examination reports. The overall extraction performance of the model achieved an accuracy rate of 97.16%, a recall rate of 98.06%, and a F1 score of 97.61%. Additionally, 47,660 relationships between entities were identified based on the four different types of relationships defined in the concept layer. The knowledge graph for breast cancer diagnosis was constructed after inputting mammographic features and relationships into the Neo4j graph data platform. The model was assessed from the concept layer, data layer, and application layer perspectives, and showed promising results. CONCLUSIONS: The proposed workflow is applicable for constructing knowledge graphs for breast cancer diagnosis based on Chinese EMRs. This study serves as a reference for the rapid design, construction, and application of knowledge graphs for diagnosis and treatment of other diseases. Furthermore, it offers a potential solution to address the issues of limited data sharing and format inconsistencies present in Chinese EMR data.

Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System.

Multi-object tracking (MOT) is a topic of great interest in the field of computer vision, which is essential in smart behavior-analysis systems for healthcare, such as human-flow monitoring, crime analysis, and behavior warnings. Most MOT methods achieve stability by combining object-detection and re-identification networks. However, MOT requires high efficiency and accuracy in complex environments with occlusions and interference. This often increases the algorithm's complexity, affects the speed of tracking calculations, and reduces real-time performance. In this paper, we present an improved MOT method combining an attention mechanism and occlusion sensing as a solution. A convolutional block attention module (CBAM) calculates the weights of space and channel attention from the feature map. The attention weights are used to fuse the feature maps to extract adaptively robust object representations. An occlusion-sensing module detects an object's occlusion, and the appearance characteristics of an occluded object are not updated. This can enhance the model's ability to extract object features and improve appearance feature pollution caused by the short-term occlusion of an object. Experiments on public datasets demonstrate the competitive performance of the proposed method compared with the state-of-the-art MOT methods. The experimental results show that our method has powerful data association capability, e.g., 73.2% MOTA and 73.9% IDF1 on the MOT17 dataset.

A label information fused medical image report generation framework.

Medical imaging is an important tool for clinical diagnosis. Nevertheless, it is very time-consuming and error-prone for physicians to prepare imaging diagnosis reports. Therefore, it is necessary to develop some methods to generate medical imaging reports automatically. Currently, the task of medical imaging report generation is challenging in at least two aspects: (1) medical images are very similar to each other. The differences between normal and abnormal images and between different abnormal images are usually trivial; (2) unrelated or incorrect keywords describing abnormal findings in the generated reports lead to mis-communications. In this paper, we propose a medical image report generation framework composed of four modules, including a Transformer encoder, a MIX-MLP multi-label classification network, a co-attention mechanism (CAM) based semantic and visual feature fusion, and a hierarchical LSTM decoder. The Transformer encoder can be used to learn long-range dependencies between images and labels, effectively extract visual and semantic features of images, and establish long-term dependent relationships between visual and semantic information to accurately extract abnormal features from images. The MIX-MLP multi-label classification network, the co-attention mechanism and the hierarchical LSTM network can better identify abnormalities, achieving visual and text alignment fusion and multi-label diagnostic classification to better facilitate report generation. The results of the experiments performed on two widely used radiology report datasets, IU X-RAY and MIMIC-CXR, show that our proposed framework outperforms current report generation models in terms of both natural linguistic generation metrics and clinical efficacy assessment metrics. The code of this work is available online at https://github.com/watersunhznu/LIFMRG.

Synthetic lumbar MRI can aid in diagnosis and treatment strategies based on self-pix networks.

CT and MR tools are commonly used to diagnose lumbar fractures (LF). However, numerous limitations have been found in practice. The aims of this study were to innovate and develop a spinal disease-specific neural network and to evaluate whether synthetic MRI of the LF affected clinical diagnosis and treatment strategies. A total of 675 LF patients who met the inclusion and exclusion criteria were included in the study. For each participant, two mid-sagittal CT and T2-weighted MR images were selected; 1350 pairs of LF images were also included. A new Self-pix based on Pix2pix and Self-Attention was constructed. A total of 1350 pairs of CT and MR images, which were randomly divided into a training group (1147 pairs) and a test group (203 pairs), were fed into Pix2pix and Self-pix. The quantitative evaluation included PSNR and SSIM (PSNR1 and SSIM1: real MR images and Pix2pix-generated MR images; PSNR2 and SSIM2: real MR images and Self-pix-generated MR images). The qualitative evaluation, including accurate diagnosis of acute fractures and accurate selection of treatment strategies based on Self-pix-generated MRI, was performed by three spine surgeons. In the LF group, PSNR1 and PSNR2 were 10.884 and 11.021 (p < 0.001), and SSIM1 and SSIM2 were 0.766 and 0.771 (p < 0.001), respectively. In the ROI group, PSNR1 and PSNR2 were 12.350 and 12.670 (p = 0.004), and SSIM1 and SSIM2 were 0.816 and 0.832 (p = 0.005), respectively. According to the qualitative evaluation, Self-pix-generated MRI showed no significant difference from real MRI in identifying acute fractures (p = 0.689), with a good sensitivity of 84.36% and specificity of 96.65%. No difference in treatment strategy was found between the Self-pix-generated MRI group and the real MRI group (p = 0.135). In this study, a disease-specific GAN named Self-pix was developed, which demonstrated better image generation performance compared to traditional GAN. The spine surgeon could accurately diagnose LF and select treatment strategies based on Self-pix-generated T2 MR images.