[Research on automatic generation of multimodal medical image reports based on memory driven].

The task of automatic generation of medical image reports faces various challenges, such as diverse types of diseases and a lack of professionalism and fluency in report descriptions. To address these issues, this paper proposes a multimodal medical imaging report based on memory drive method (mMIRmd). Firstly, a hierarchical vision transformer using shifted windows (Swin-Transformer) is utilized to extract multi-perspective visual features of patient medical images, and semantic features of textual medical history information are extracted using bidirectional encoder representations from transformers (BERT). Subsequently, the visual and semantic features are integrated to enhance the model's ability to recognize different disease types. Furthermore, a medical text pre-trained word vector dictionary is employed to encode labels of visual features, thereby enhancing the professionalism of the generated reports. Finally, a memory driven module is introduced in the decoder, addressing long-distance dependencies in medical image data. This study is validated on the chest X-ray dataset collected at Indiana University (IU X-Ray) and the medical information mart for intensive care chest x-ray (MIMIC-CXR) released by the Massachusetts Institute of Technology and Massachusetts General Hospital. Experimental results indicate that the proposed method can better focus on the affected areas, improve the accuracy and fluency of report generation, and assist radiologists in quickly completing medical image report writing.

Deep Learning for the Automatic Diagnosis and Analysis of Bone Metastasis on Bone Scintigrams.

OBJECTIVE: To develop an approach for automatically analyzing bone metastases (BMs) on bone scintigrams based on deep learning technology. METHODS: This research included a bone scan classification model, a regional segmentation model, an assessment model for tumor burden and a diagnostic report generation model. Two hundred eighty patients with BMs and 341 patients with non-BMs were involved. Eighty percent of cases were randomly extracted from two groups as training set. Remaining cases were as testing set. A deep residual convolutional neural network with different structures was used to determine whether metastatic bone lesions existed, regions of lesions were automatically segmented. Bone scan tumor burden index (BSTBI) was calculated; finally, diagnostic report could be automatically generated. The sensitivity, specificity and accuracy of classification model were compared with three physicians with different clinical experience. The Dice coefficient evaluated the effect of segmentation model and compared to the result of nnU-Net model. The correlation between BSTBI and blood alkaline phosphatase (ALP) level was analyzed to verify the efficiency of BSTBI. The performance of report generation model was evaluated by the accuracy of interpretation of report. RESULTS: In testing set, the sensitivity, specificity and accuracy of classification model were 92.59%, 85.51% and 88.62%, respectively. The accuracy showed no statistical difference with moderately and experienced physicians and obviously outperformed the inexperienced. The Dice coefficient of BMs area was 0.7387 in segmentation stage. Based on the whole model frame, our segmentation model outperformed the nnU-Net. BSTBI value changed as the BMs changed. There was a positive correlation between BSTBI and ALP level. The accuracy of report generation model was 78.05%. CONCLUSION: Deep learning based on automatic analysis frameworks for BMs can accurately identify BMs, preliminarily realize a fully automatic analysis process from raw data to report generation. BSTBI can be used as a quantitative evaluation indicator to assess the effect of therapy on BMs in different patients or in the same patient before and after treatment.

An image selection framework for automatic report generation.

The development of IoT technologies and social network services (SNS) are contributing to the growth of big data. However, the vast amount of data makes it difficult for users to find the information they need, and as a result, the demand for a system that provides the desired information in a well-organized form is increasing. Many studies are being conducted to extract desired information from data, and application studies such as automatic report generation are also being conducted. To generate a report for a given topic, a report generation system is required to extract essential information from big data and re-organize it in a compact form. Image selection system also plays an important role in automatic report generation as insertion of appropriate images can increase the completeness and readability of the report. In this study, we propose an image selection framework for recommending an appropriate image for a part of a report by combining textual information used in text-based image retrieval and visual features used in content-based image retrieval. In addition, the proposed image selection framework adopts an image filtering module that is specially designed for filtering out some images that are not suitable for use in reports. Through experiments on two datasets and comparative experiment with state-of-the-art work, we confirmed that our proposed method recommends images that fit the user's intention, and its practical applicability.

SiCoDEA: A Simple, Fast and Complete App for Analyzing the Effect of Individual Drugs and Their Combinations.

The administration of combinations of drugs is a method widely used in the treatment of different pathologies as it can lead to an increase in the therapeutic effect and a reduction in the dose compared to the administration of single drugs. For these reasons, it is of interest to study combinations of drugs and to determine whether a specific combination has a synergistic, antagonistic or additive effect. Various mathematical models have been developed, which use different methods to evaluate the synergy of a combination of drugs. We have developed an open access and easy to use app that allows different models to be explored and the most fitting to be chosen for the specific experimental data: SiCoDEA (Single and Combined Drug Effect Analysis). Despite the existence of other tools for drug combination analysis, SiCoDEA remains the most complete and flexible since it offers options such as outlier removal or the ability to choose between different models for analysis. SiCoDEA is an easy to use tool for analyzing drug combination data and to have a view of the various steps and offer different results based on the model chosen.

AutoProstate: Towards Automated Reporting of Prostate MRI for Prostate Cancer Assessment Using Deep Learning.

Multiparametric magnetic resonance imaging (mpMRI) of the prostate is used by radiologists to identify, score, and stage abnormalities that may correspond to clinically significant prostate cancer (CSPCa). Automatic assessment of prostate mpMRI using artificial intelligence algorithms may facilitate a reduction in missed cancers and unnecessary biopsies, an increase in inter-observer agreement between radiologists, and an improvement in reporting quality. In this work, we introduce AutoProstate, a deep learning-powered framework for automatic MRI-based prostate cancer assessment. AutoProstate comprises of three modules: Zone-Segmenter, CSPCa-Segmenter, and Report-Generator. Zone-Segmenter segments the prostatic zones on T2-weighted imaging, CSPCa-Segmenter detects and segments CSPCa lesions using biparametric MRI, and Report-Generator generates an automatic web-based report containing four sections: Patient Details, Prostate Size and PSA Density, Clinically Significant Lesion Candidates, and Findings Summary. In our experiment, AutoProstate was trained using the publicly available PROSTATEx dataset, and externally validated using the PICTURE dataset. Moreover, the performance of AutoProstate was compared to the performance of an experienced radiologist who prospectively read PICTURE dataset cases. In comparison to the radiologist, AutoProstate showed statistically significant improvements in prostate volume and prostate-specific antigen density estimation. Furthermore, AutoProstate matched the CSPCa lesion detection sensitivity of the radiologist, which is paramount, but produced more false positive detections.

Fuzzy logic based risk assessment system giving individualized advice for metabolic syndrome and fatal cardiovascular diseases.

In 2005, global cardiovascular diseases caused 30% of deaths in Europe, which is 46% of total deaths for all death groups. Today, according to the International Adult Diabetes Federation, 20% to 25% of the adult population in the world has Metabolic Syndrome. Turkish Statistical Institute claims that in Turkey 408782 people died of circulatory system diseases in 2016 and it is expected that numbers will dramatically increase. In 2003, total worldwide healthcare budget of Diabetes Mellitus was up to 64.9 billion International Dollars with the continuing rise in prevalence, it is expected that total costs will increase to 396 billion International Dollars by 2025. The main purpose of this study was to present a clinical decision support system that calculates Metabolic Syndrome existence and evaluate HeartScore risk level for Turkish population. The second objective was to create a detailed personal report about individual's risk level of Metabolic Syndrome and HeartScore and give advice to him/her to reduce it. The fuzzy logic risk assessment system (FLRAS) was formed in LabVIEW graphical development platform according to International Diabetes Federation and European Heart Journal's criteria. Mamdani type fuzzy logic sets were identified for each input variable and membership functions were assigned depending on the magnitude of the input limits. System's performance was tested on 96 (72 females, 24 males) patient data. Results show that the proposed system was able to evaluate the Metabolic Syndrome risk with 0.9285 specificity, 0.92708 accuracy and 0.925 sensitivity.

Low rates of automatic reporting of estimated glomerular filtration rate in Southern Brazilian laboratories.

OBJECTIVES: The aim of this study was to evaluate the rate of eGFR reporting in Southern Brazilian laboratories. DESIGN AND METHODS: The eGFR automatic reporting, as assessed by Modification of Diet in Renal Disease (MDRD) and/or Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine-based equations, was evaluated in a representative cross-sectional sample. A standardized questionnaire to obtain this information was given out by mail or email. RESULTS: Five-hundred fifty laboratories, evenly distributed in the different state regions, completed the questionnaire. The eGFR was automatically reported by 54 (9.8%) laboratories, and the MDRD was the most commonly used equation (94.5%). The Jaffe methods were the most employed technique (94%) to measure serum creatinine. CONCLUSION: The automatic eGFR reporting rate was unacceptably low, emphasizing the crucial role of educating medical teams and laboratories on the importance of having these tools available to optimize detection of renal disease and proper treatment.