Medical object detector jointly driven by knowledge and data.

Most of the existing object detection algorithms are trained on medical datasets and then used for prediction. When the features of an object are not obvious in an image, these models are prone to mislocalize and misclassify it. In this paper, we propose a medical Object Detection algorithm jointly driven by Knowledge and Data (ODKD). It enables medical semantic knowledge provided by specialized physicians to be effective and helpful when traditional models have difficulty in correctly detecting objects relying on features alone. Our model consists of a base object detector together with a fusion module: the base object detector is trained based on medical datasets to obtain data-driven results; then we use a graph to represent external semantic knowledge and map the data-driven results to the nodes embedding of this graph structure. In the fusion module, a graph convolution network is used to fuse the data-driven results with the external semantic knowledge to output category adjustment coefficients. Finally, the adjustment coefficients are used to adjust the data-driven results to obtain results jointly driven by knowledge and data. Experiments show that professional medical semantic knowledge can effectively correct the erroneous results of the base detector, and the effect of our model outperforms Faster Rcnn, YOLOv5, YOLOv7, etc. on three medical datasets, Camus, Synapse, and AMOS.

AERMNet: Attention-enhanced relational memory network for medical image report generation.

BACKGROUND AND OBJECTIVES: The automatic generation of medical image diagnostic reports can assist doctors in reducing their workload and improving the efficiency and accuracy of diagnosis. However, among the most existing report generation models, there are problems that the weak correlation between generated words and the lack of contextual information in the report generation process. METHODS: To address the above problems, we propose an Attention-Enhanced Relational Memory Network (AERMNet) model, where the relational memory module is continuously updated by the words generated in the previous time step to strengthen the correlation between words in generated medical image report. And the double LSTM with interaction module reduces the loss of context information and makes full use of feature information. Thus, more accurate disease information can be generated by AERMNet for medical image reports. RESULTS: Experimental results on four medical datasets Fetal heart (FH), Ultrasound, IU X-Ray and MIMIC-CXR, show that our proposed method outperforms some of the previous models with respect to language generation metrics (Cider improving by 2.4% on FH, Bleu1 improving by 2.4% on Ultrasound, Cider improving by 16.4% on IU X-Ray, Bleu2 improving by 9.7% on MIMIC-CXR). CONCLUSIONS: This work promotes the development of medical image report generation and expands the prospects of computer-aided diagnosis applications. Our code is released at https://github.com/llttxx/AERMNET.

Deep consistency-preserving hash auto-encoders for neuroimage cross-modal retrieval.

Cross-modal hashing is an efficient method to embed high-dimensional heterogeneous modal feature descriptors into a consistency-preserving Hamming space with low-dimensional. Most existing cross-modal hashing methods have been able to bridge the heterogeneous modality gap, but there are still two challenges resulting in limited retrieval accuracy: (1) ignoring the continuous similarity of samples on manifold; (2) lack of discriminability of hash codes with the same semantics. To cope with these problems, we propose a Deep Consistency-Preserving Hash Auto-encoders model, called DCPHA, based on the multi-manifold property of the feature distribution. Specifically, DCPHA consists of a pair of asymmetric auto-encoders and two semantics-preserving attention branches working in the encoding and decoding stages, respectively. When the number of input medical image modalities is greater than 2, the encoder is a multiple pseudo-Siamese network designed to extract specific modality features of different medical image modalities. In addition, we define the continuous similarity of heterogeneous and homogeneous samples on Riemann manifold from the perspective of multiple sub-manifolds, respectively, and the two constraints, i.e., multi-semantic consistency and multi-manifold similarity-preserving, are embedded in the learning of hash codes to obtain high-quality hash codes with consistency-preserving. The extensive experiments show that the proposed DCPHA has the most stable and state-of-the-art performance. We make code and models publicly available: https://github.com/Socrates023/DCPHA .

One-Step Construction of the Positively/Negatively Charged Ultrathin Janus Nanofiltration Membrane for the Separation of Li+ and Mg2.

To coordinate the trade-off between the separation and permeation of the nanofiltration membrane for the separation of Mg2+/Li+, we regulated poly(ethyleneimine)/piperazine interface polymerization parameters to construct a positively/negatively charged ultrathin Janus nanofiltration membrane at a free aqueous-organic interface. At the optimized interfacial polymerization parameters, 0.03 wt % of piperazine reacted with trimethylbenzene chloride prior to poly(ethyleneimine), forming a primary polyamide layer with fewer defects or limiting large-scale defects of the polyamide layer. The controlled subsequent reaction of poly(ethyleneimine) and trimethylbenzene chloride results in a Janus nanofiltration membrane, with one side enriched with the carboxyl groups, the other side enriched with the amine groups, and a dense polyamide structure in the middle. Under the optimum conditions, the positive potential of the rear surface of the prepared membrane was 14.57 mV, and the water contact angle reached 71.31°, while the negative potential of the front surface was -25.48 mV, and the water contact angle was 12.93°, confirming a Janus membrane with opposite charges and large hydrophilicity differences in the front and rear surfaces. With a high cross-linking degree, a 40 nm thick polyamide layer is 29.09% more thinner than the traditional polyamide membrane. The ultrathin Janus nanofiltration membrane showed an excellent separation factor (SLi,Mg of 18.26), stability, and water permeability flux (10.6 L·m-2·h-1·bar-1). The rejections to MgCl2, CaCl2, MgSO4, and Na2SO4 are measured above 90% at a nearly constant permeability of 10.6 L·m-2·h-1·bar-1, particularly stable rejections to MgCl2 and Na2SO4.

Preparation of aminated porous polyacrylonitrile nanofibers as adsorbent for methyl orange removal.

In this study, porous electrospinning polyacrylonitrile nanofiber (PPAN) surface functionalization with amine groups is studied for methyl orange (MO) dye removal from aqueous solution. A series of adsorption experiments were carried out to investigate the influence of initial solution pH value, contact time, initial solution concentration, and adsorption temperature on the adsorption performance. The experimental results showed that the removal of MO on these PPAN-PEI and PPAN-TEPA nanofibrous mats was a pH-dependent process with the maximum adsorption capacity at the initial solution pH of 3, and that the PPAN-PEI and PPAN-TEPA nanofibrous mats could be regenerated successfully after 4 recycling processes. The adsorption equilibrium data were all fitted well to the Langmuir isotherm equation, with maximum adsorption capacity of 1414.52 mg g-1 and 1221.09 mg g-1 for PPAN-PEI and PPAN-TEPA, respectively. The kinetic study indicated that the adsorption of MO could be well fitted by the pseudo-second-order equation and Weber-Morris model. Thermodynamic parameters such as free energy, enthalpy, and entropy of adsorption of the MO were also evaluated, and the results showed that the adsorption was a spontaneous exothermic adsorption process.

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval.

Benefitting from the low storage cost and high retrieval efficiency, hash learning has become a widely used retrieval technology to approximate nearest neighbors. Within it, the cross-modal medical hashing has attracted an increasing attention in facilitating efficiently clinical decision. However, there are still two main challenges in weak multi-manifold structure perseveration across multiple modalities and weak discriminability of hash code. Specifically, existing cross-modal hashing methods focus on pairwise relations within two modalities, and ignore underlying multi-manifold structures across over 2 modalities. Then, there is little consideration about discriminability, i.e., any pair of hash codes should be different. In this paper, we propose a novel hashing method named multi-manifold deep discriminative cross-modal hashing (MDDCH) for large-scale medical image retrieval. The key point is multi-modal manifold similarity which integrates multiple sub-manifolds defined on heterogeneous data to preserve correlation among instances, and it can be measured by three-step connection on corresponding hetero-manifold. Then, we propose discriminative item to make each hash code encoded by hash functions be different, which improves discriminative performance of hash code. Besides, we introduce Gaussian-binary Restricted Boltzmann Machine to directly output hash codes without using any continuous relaxation. Experiments on three benchmark datasets (AIBL, Brain and SPLP) show that our proposed MDDCH achieves comparative performance to recent state-of-the-art hashing methods. Additionally, diagnostic evaluation from professional physicians shows that all the retrieved medical images describe the same object and illness as the queried image.

Development and validation of a patient-reported outcome measure for patients with chronic respiratory failure: The CRF-PROM scale.

BACKGROUND: Various health-related quality-of-life (HRQOL) tools are used to evaluate patients with chronic respiratory failure (CRF), but there is a relative lack of tools available for the evaluation of social support and treatment in these patients. The present study focused on the development of a systematic patient-reported outcome measure (PROM) tool for use in patients with CRF. METHODS: The CRF-PROM scale conceptual framework and item bank were generated after reviewing the corresponding literature and HRQOL scales, interviewing CRF patients and focus groups. After creation of the initial scale, the items in the scale were selected through two item selection theories, and the final scale was created. The reliability, validity and feasibility of the final scale were assessed. RESULTS: The CRF-PROM scale includes four domains (i.e., physiological domain, psychological domain, social domain and therapeutic domain) and 10 dimensions. After the item selection process, the final scale included 50 items. Cronbach's α coefficients, which were all above 0.7, indicated the reliability of the scale. The results of structural validity met the relevant standards of confirmatory factor analysis. The response rates of the preinvestigation and the formal investigation were 93.3% and 97.6%, respectively. CONCLUSIONS: The CRF-PROM scale developed in the present study is effective and reliable. It could be used widely in the posthospital management of patients, in CRF studies and in clinical trials of new medical products and interventions. PATIENT OR PUBLIC CONTRIBUTION: Participants from eight different hospitals and communities participated in the development or validation phase of the CRF-PROM scale.

Generating diagnostic report for medical image by high-middle-level visual information incorporation on double deep learning models.

BACKGROUND AND OBJECTIVES: Writing diagnostic reports for medical images is a heavy and tedious work. The automatic generation of medical image diagnostic reports can assist doctors to reduce their workload and improve diagnosis efficiency. It is of great significance to introduce image caption algorithm into medical image processing. Existing approaches attempt to generate medical image diagnostic reports using image caption algorithms but without taking the accuracy of pathological information in generated diagnostic reports into account. METHODS: To solve the mentioned problem, we propose a Semantic Fusion Network (SFNet) including a lesion area detection model and a diagnostic generation model. The lesion area detection model can extract visual and pathological information from medical image, and the diagnostic report generation model can learn to fuse the two kinds of information to generate reports. Thus, the pathological information in the generated diagnostic reports can be more accurate. RESULTS: Experimental results have verified the performance of our model (Accuracy increases 1.2% on the Ultrasound Image Dataset and 2.4% on the Open-i X-ray Image Dataset), compared with the model only using visual feature to generate diagnostic reports. CONCLUSIONS: This work utilizes computer algorithms to generate the more accurate diagnostic reports for medical images automatically, which expands the application of computer-aided diagnosis and promotes the implementation of deep learning in the medical image analysis field.

BPGAN: Bidirectional CT-to-MRI prediction using multi-generative multi-adversarial nets with spectral normalization and localization.

Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are widely used detection technology in screening, diagnosis, and image-guided therapy for both clinical and research. However, CT imposes ionizing radiation to patients during acquisition. Compared to CT, MRI is much safer and does not involve any radiations, but it is more expensive and has prolonged acquisition time. Therefore, it is necessary to estimate one modal image from another given modal image of the same subject for the case of radiotherapy planning. Considering that there is currently no bidirectional prediction model between MRI and CT images, we propose a bidirectional prediction by using multi-generative multi-adversarial nets (BPGAN) for the prediction of any modal from another modal image in paired and unpaired fashion. In BPGAN, two nonlinear maps are learned by projecting same pathological features from one domain to another with cycle consistency strategy. Technologically, pathological prior information is introduced to constrain the feature generation to attack the potential risk of pathological variance, and edge retention metric is adopted to preserve geometrically distortion and anatomical structure. Algorithmically, spectral normalization is designed to control the performance of discriminator and to make predictor learn better and faster, and the localization is proposed to impose regularizer on predictor to reduce generalization error. Experimental results show that BPGAN generates better predictions than recently state-of-the-art methods. Specifically, BPGAN achieves average increment of MAE 33.2% and 37.4%, and SSIM 24.5% and 44.6% on two baseline datasets than comparisons.

Dictionary Pair Learning on Grassmann Manifolds for Image Denoising.

Image denoising is a fundamental problem in computer vision and image processing that holds considerable practical importance for real-world applications. The traditional patch-based and sparse coding-driven image denoising methods convert 2D image patches into 1D vectors for further processing. Thus, these methods inevitably break down the inherent 2D geometric structure of natural images. To overcome this limitation pertaining to the previous image denoising methods, we propose a 2D image denoising model, namely, the dictionary pair learning (DPL) model, and we design a corresponding algorithm called the DPL on the Grassmann-manifold (DPLG) algorithm. The DPLG algorithm first learns an initial dictionary pair (i.e., the left and right dictionaries) by employing a subspace partition technique on the Grassmann manifold, wherein the refined dictionary pair is obtained through a sub-dictionary pair merging. The DPLG obtains a sparse representation by encoding each image patch only with the selected sub-dictionary pair. The non-zero elements of the sparse representation are further smoothed by the graph Laplacian operator to remove the noise. Consequently, the DPLG algorithm not only preserves the inherent 2D geometric structure of natural images but also performs manifold smoothing in the 2D sparse coding space. We demonstrate that the DPLG algorithm also improves the structural SIMilarity values of the perceptual visual quality for denoised images using the experimental evaluations on the benchmark images and Berkeley segmentation data sets. Moreover, the DPLG also produces the competitive peak signal-to-noise ratio values from popular image denoising algorithms.

Preparation of activated ordered mesoporous carbons with a channel structure.

Activated ordered mesoporous carbons with a channel structure (AOMCs-CS) were successfully prepared by imposing CO(2) activation on ordered mesopore carbon C-FDU-15. It is found that the continuous carbon framework of the precursor C-FDU-15 plays an important role in keeping the order structure of the resulting AOMCs-CS. The mild activation (e.g., 31 wt % burnoff) does not impair the order degree. After that, the order degree gradually decreases with further increasing burnoff. However, the basic hexagonal mesostructure of C-FDU-15 can still be found in the AOMCs-CS when the burnoff is up to 73 wt %, although many carbon walls are punched and thus many larger mesopores and marcropores are generated. With increasing burnoff, the surface area and volume of micropores increase first and then decrease, and the surface area and volume of mesopores continuously increase. The highest measured Brunaruer-Emmett-Teller (BET) surface area, micropore volume, and total pore volume of the AOMCs-CS reach 2004 m(2)/g, 0.50 cm(3)/g, and 1.22 cm(3)/g, respectively.