Your browser doesn't support javascript.
loading
Mostrar: 20 | 50 | 100
Resultados 1 - 20 de 752
Filtrar
1.
Sensors (Basel) ; 24(17)2024 Sep 05.
Artículo en Inglés | MEDLINE | ID: mdl-39275687

RESUMEN

Underwater image enhancement technology is crucial for the human exploration and exploitation of marine resources. The visibility of underwater images is affected by visible light attenuation. This paper proposes an image reconstruction method based on the decomposition-fusion of multi-channel luminance data to enhance the visibility of underwater images. The proposed method is a single-image approach to cope with the condition that underwater paired images are difficult to obtain. The original image is first divided into its three RGB channels. To reduce artifacts and inconsistencies in the fused images, a multi-resolution fusion process based on the Laplace-Gaussian pyramid guided by a weight map is employed. Image saliency analysis and mask sharpening methods are also introduced to color-correct the fused images. The results indicate that the method presented in this paper effectively enhances the visibility of dark regions in the original image and globally improves its color, contrast, and sharpness compared to current state-of-the-art methods. Our method can enhance underwater images in engineering practice, laying the foundation for in-depth research on underwater images.

2.
Sensors (Basel) ; 24(17)2024 Sep 09.
Artículo en Inglés | MEDLINE | ID: mdl-39275771

RESUMEN

Infrared and visible image fusion can integrate rich edge details and salient infrared targets, resulting in high-quality images suitable for advanced tasks. However, most available algorithms struggle to fully extract detailed features and overlook the interaction of complementary features across different modal images during the feature fusion process. To address this gap, this study presents a novel fusion method based on multi-scale edge enhancement and a joint attention mechanism (MEEAFusion). Initially, convolution kernels of varying scales were utilized to obtain shallow features with multiple receptive fields unique to the source image. Subsequently, a multi-scale gradient residual block (MGRB) was developed to capture the high-level semantic information and low-level edge texture information of the image, enhancing the representation of fine-grained features. Then, the complementary feature between infrared and visible images was defined, and a cross-transfer attention fusion block (CAFB) was devised with joint spatial attention and channel attention to refine the critical supplemental information. This allowed the network to obtain fused features that were rich in both common and complementary information, thus realizing feature interaction and pre-fusion. Lastly, the features were reconstructed to obtain the fused image. Extensive experiments on three benchmark datasets demonstrated that the MEEAFusion proposed in this research has considerable strengths in terms of rich texture details, significant infrared targets, and distinct edge contours, and it achieves superior fusion performance.

4.
Sensors (Basel) ; 24(18)2024 Sep 18.
Artículo en Inglés | MEDLINE | ID: mdl-39338770

RESUMEN

In response to the issue that the fusion process of infrared and visible images is easily affected by lighting factors, in this paper, we propose an adaptive illumination perception fusion mechanism, which was integrated into an infrared and visible image fusion network. Spatial attention mechanisms were applied to both infrared images and visible images for feature extraction. Deep convolutional neural networks were utilized for further feature information extraction. The adaptive illumination perception fusion mechanism is then integrated into the image reconstruction process to reduce the impact of lighting variations in the fused images. A Median Strengthening Channel and Spatial Attention Module (MSCS) was designed to be integrated into the backbone of YOLOv8. In this paper, we used the fusion network to create a dataset named ivifdata for training the target recognition network. The experimental results indicated that the improved YOLOv8 network saw further enhancements of 2.3%, 1.4%, and 8.2% in the Recall, mAP50, and mAP50-95 metrics, respectively. The experiments revealed that the improved YOLOv8 network has advantages in terms of recognition rate and completeness, while also reducing the rates of false negatives and false positives.

5.
J Imaging Inform Med ; 2024 Sep 26.
Artículo en Inglés | MEDLINE | ID: mdl-39327379

RESUMEN

The differentiation of benign and malignant parotid gland tumors is of major significance as it directly affects the treatment process. In addition, it is also a vital task in terms of early and accurate diagnosis of parotid gland tumors and the determination of treatment planning accordingly. As in other diseases, the differentiation of tumor types involves several challenging, time-consuming, and laborious processes. In the study, Magnetic Resonance (MR) images of 114 patients with parotid gland tumors are used for training and testing purposes by Image Fusion (IF). After the Apparent Diffusion Coefficient (ADC), Contrast-enhanced T1-w (T1C-w), and T2-w sequences are cropped, IF (ADC, T1C-w), IF (ADC, T2-w), IF (T1C-w, T2-w), and IF (ADC, T1C-w, T2-w) datasets are obtained for different combinations of these sequences using a two-dimensional Discrete Wavelet Transform (DWT)-based fusion technique. For each of these four datasets, ResNet18, GoogLeNet, and DenseNet-201 architectures are trained separately, and thus, 12 models are obtained in the study. A Graphical User Interface (GUI) application that contains the most successful of these trained architectures for each data is also designed to support the users. The designed GUI application not only allows the fusing of different sequence images but also predicts whether the label of the fused image is benign or malignant. The results show that the DenseNet-201 models for IF (ADC, T1C-w), IF (ADC, T2-w), and IF (ADC, T1C-w, T2-w) are better than the others, with accuracies of 95.45%, 95.96%, and 92.93%, respectively. It is also noted in the study that the most successful model for IF (T1C-w, T2-w) is ResNet18, and its accuracy is equal to 94.95%.

7.
Ultrasonics ; 144: 107396, 2024 Dec.
Artículo en Inglés | MEDLINE | ID: mdl-39173277

RESUMEN

Ultrasound shear wave elastography is an imaging modality that noninvasively assesses mechanical properties of tissues. The results of elastic imaging are obtained by accurately estimating the propagation velocity of shear wave fronts. However, the acquisition rate of the shear wave acquisition device is limited by the hardware of the system. Therefore, increasing the collection rate of shear waves can directly improve the quality of shear wave velocity images. In addition, the problem of velocity reconstruction with relatively small elastic inclusions has always been a challenge in elastic imaging and a very important and urgent issue in early disease diagnosis. For the problem of elastography detection of the shape and boundary of inclusions in tissues, Time-sharing latency excitation frame composite imaging (TS-FCI) method is proposed for tissue elasticity measurement. The method fuses the shear wave motion data generated by time sharing and latency excitation to obtain a set of composite shear wave motion data. Based on the shear wave motion data, the local shear wave velocity image is reconstructed in the frequency domain to obtain the elastic information of the tissue. The experimental results show that the TS-FCI method has a velocity estimation error of 11 % and a contrast to noise ratio (CNR) of 3.81 when estimating inclusions with smaller dimensions (2.53 mm). Furthermore, when dealing with inclusions with small elastic changes (10 kPa), the velocity estimation error is 3 % and the CNR is 3.21. Compared to conventional time-domain and frequency-domain analysis methods, the proposed method has advantages. Results and analysis have shown that this method has potential promotional value in the quantitative evaluation of organizational elasticity.


Asunto(s)
Diagnóstico por Imagen de Elasticidad , Fantasmas de Imagen , Diagnóstico por Imagen de Elasticidad/métodos , Procesamiento de Imagen Asistido por Computador/métodos , Humanos , Algoritmos
8.
Entropy (Basel) ; 26(8)2024 Aug 16.
Artículo en Inglés | MEDLINE | ID: mdl-39202166

RESUMEN

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions.

9.
Diagnostics (Basel) ; 14(16)2024 Aug 16.
Artículo en Inglés | MEDLINE | ID: mdl-39202275

RESUMEN

Hybrid positron emission tomography/magnetic resonance imaging (PET/MR) opens new possibilities in multimodal multiparametric (m2p) image analyses. But even the simultaneous acquisition of positron emission tomography (PET) and magnetic resonance imaging (MRI) does not guarantee perfect voxel-by-voxel co-registration due to organs and distortions, especially in diffusion-weighted imaging (DWI), which would be, however, crucial to derive biologically meaningful information. Thus, our aim was to optimize fusion and voxel-wise analyses of DWI and standardized uptake values (SUVs) using a novel software for m2p analyses. Using research software, we evaluated the precision of image co-registration and voxel-wise analyses including the rigid and elastic 3D registration of DWI and [18F]-Fluorodeoxyglucose (FDG)-PET from an integrated PET/MR system. We analyzed DWI distortions with a volume-preserving constraint in three different 3D-printed phantom models. A total of 12 PET/MR-DWI clinical datasets (bronchial carcinoma patients) were referenced to the T1 weighted-DIXON sequence. Back mapping of scatterplots and voxel-wise registration was performed and compared to the non-optimized datasets. Fusion was rated using a 5-point Likert scale. Using the 3D-elastic co-registration algorithm, geometric shapes were restored in phantom measurements; the measured ADC values did not change significantly (F = 1.12, p = 0.34). Reader assessment showed a significant improvement in fusion precision for DWI and morphological landmarks in the 3D-registered datasets (4.3 ± 0.2 vs. 4.6 ± 0.2, p = 0.009). Most pronounced differences were noted for the chest wall (p = 0.006), tumor (p = 0.007), and skin contour (p = 0.014). Co-registration increased the number of plausible ADC and SUV combinations by 25%. The volume-preserving elastic 3D registration of DWI significantly improved the precision of fusion with anatomical sequences in phantom and clinical datasets. The research software allowed for a voxel-wise analysis and visualization of [18F]FDG-PET/MR data as a "combined diffusivity-metabolic index" (cDMI). The clinical value of the optimized PET/MR biomarker can thus be tested in future PET/MR studies.

10.
J Imaging Inform Med ; 2024 Aug 15.
Artículo en Inglés | MEDLINE | ID: mdl-39147889

RESUMEN

Multi-modal medical image (MI) fusion assists in generating collaboration images collecting complement features through the distinct images of several conditions. The images help physicians to diagnose disease accurately. Hence, this research proposes a novel multi-modal MI fusion modal named guided filter-based interactive multi-scale and multi-modal transformer (Trans-IMSM) fusion approach to develop high-quality computed tomography-magnetic resonance imaging (CT-MRI) fused images for brain tumor detection. This research utilizes the CT and MRI brain scan dataset to gather the input CT and MRI images. At first, the data preprocessing is carried out to preprocess these input images to improve the image quality and generalization ability for further analysis. Then, these preprocessed CT and MRI are decomposed into detail and base components utilizing the guided filter-based MI decomposition approach. This approach involves two phases: such as acquiring the image guidance and decomposing the images utilizing the guided filter. A canny operator is employed to acquire the image guidance comprising robust edge for CT and MRI images, and the guided filter is applied to decompose the guidance and preprocessed images. Then, by applying the Trans-IMSM model, fuse the detail components, while a weighting approach is used for the base components. The fused detail and base components are subsequently processed through a gated fusion and reconstruction network, and the final fused images for brain tumor detection are generated. Extensive tests are carried out to compute the Trans-IMSM method's efficacy. The evaluation results demonstrated the robustness and effectiveness, achieving an accuracy of 98.64% and an SSIM of 0.94.

11.
Sci Rep ; 14(1): 19261, 2024 08 20.
Artículo en Inglés | MEDLINE | ID: mdl-39164350

RESUMEN

Medical image fusion (MIF) techniques are proficient in combining medical images in distinct morphologies to obtain a reliable medical analysis. A single modality image could not offer adequate data for an accurate analysis. Therefore, a novel multimodal MIF-based artificial intelligence (AI) method has been presented. MIF approaches fuse multimodal medical images for exact and reliable medical recognition. Multimodal MIF improves diagnostic accuracy and clinical decision-making by combining complementary data in different imaging modalities. This article presents a new multimodal medical image fusion model utilizing Modified DWT with an Arithmetic Optimization Algorithm (MMIF-MDWTAOA) approach. The MMIF-MDWTAOA approach aims to generate a fused image with the significant details and features from each modality, leading to an elaborated depiction for precise interpretation by medical experts. The bilateral filtering (BF) approach is primarily employed for noise elimination. Next, the image decomposition process uses a modified discrete wavelet transform (MDWT) approach. However, the approximation coefficient of modality_1 and the detailed coefficient of modality_2 can be fused interchangeably. Furthermore, a fusion rule is derived from combining the multimodality data, and the AOA model is enforced to ensure the optimum selection of the fusion rule parameters. A sequence of simulations is accomplished to validate the enhanced output of the MMIF-MDWTAOA technique. The investigational validation of the MMIF-MDWTAOA technique showed the highest entropy values of 7.568 and 7.741 bits/pixel over other approaches.


Asunto(s)
Algoritmos , Imagen Multimodal , Análisis de Ondículas , Humanos , Imagen Multimodal/métodos , Procesamiento de Imagen Asistido por Computador/métodos , Interpretación de Imagen Asistida por Computador/métodos , Inteligencia Artificial , Tomografía Computarizada por Rayos X/métodos
12.
BMC Gastroenterol ; 24(1): 280, 2024 Aug 21.
Artículo en Inglés | MEDLINE | ID: mdl-39169297

RESUMEN

Radiofrequency ablation (RFA) offers a minimally invasive treatment for small hepatocellular carcinoma (HCC), but it faces challenges such as high local recurrence rates. This prospective study, conducted from January 2020 to July 2022, evaluated a novel approach using a three-channel, dual radiofrequency (RF) generator with separable clustered electrodes to improve RFA's efficacy and safety. The study employed a high-power, gradual, stepwise RFA method on HCCs (≤ 4 cm), utilizing real-time ultrasound-computed tomography (CT)/magnetic resonance imaging (MRI) fusion imaging. Involving 110 participants with 116 HCCs, the study reported no major complications. Local tumor progression (LTP) and intrahepatic remote recurrence (IRR) rates were low, with promising cumulative incidences at 1, 2, and 3 years for LTP (0.9%, 3.6%, 7.0%) and IRR (13.9%, 20.5%, 31.4%). Recurrence-free survival (RFS) rates were similarly encouraging: LTP (99.1%, 96.4%, 93.0%) and IRR (86.1%, 79.5%, 68.6%). This innovative gradual, incremental high-power RFA technique, featuring a dual switching monopolar mode and three electrodes, represents an effective and safer management option for small HCCs. TRIAL REGISTRATION: clinicaltrial.gov identifier: NCT05397860, first registered on 26/05/2022.


Asunto(s)
Carcinoma Hepatocelular , Neoplasias Hepáticas , Recurrencia Local de Neoplasia , Ablación por Radiofrecuencia , Humanos , Carcinoma Hepatocelular/cirugía , Carcinoma Hepatocelular/diagnóstico por imagen , Carcinoma Hepatocelular/patología , Neoplasias Hepáticas/cirugía , Neoplasias Hepáticas/patología , Neoplasias Hepáticas/diagnóstico por imagen , Estudios Prospectivos , Masculino , Femenino , Persona de Mediana Edad , Anciano , Ablación por Radiofrecuencia/métodos , Electrodos , Imagen por Resonancia Magnética , Adulto , Tomografía Computarizada por Rayos X , Resultado del Tratamiento , Progresión de la Enfermedad , Anciano de 80 o más Años , Ablación por Catéter/métodos
13.
Heliyon ; 10(15): e34402, 2024 Aug 15.
Artículo en Inglés | MEDLINE | ID: mdl-39145034

RESUMEN

The threat posed by Alzheimer's disease (AD) to human health has grown significantly. However, the precise diagnosis and classification of AD stages remain a challenge. Neuroimaging methods such as structural magnetic resonance imaging (sMRI) and fluorodeoxyglucose positron emission tomography (FDG-PET) have been used to diagnose and categorize AD. However, feature selection approaches that are frequently used to extract additional data from multimodal imaging are prone to errors. This paper suggests using a static pulse-coupled neural network and a Laplacian pyramid to combine sMRI and FDG-PET data. After that, the fused images are used to train the Mobile Vision Transformer (MViT), optimized with Pareto-Optimal Quantum Dynamic Optimization for Neural Architecture Search, while the fused images are augmented to avoid overfitting and then classify unfused MRI and FDG-PET images obtained from the AD Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies (OASIS) datasets into various stages of AD. The architectural hyperparameters of MViT are optimized using Quantum Dynamic Optimization, which ensures a Pareto-optimal solution. The Peak Signal-to-Noise Ratio (PSNR), the Mean Squared Error (MSE), and the Structured Similarity Indexing Method (SSIM) are used to measure the quality of the fused image. We found that the fused image was consistent in all metrics, having 0.64 SIMM, 35.60 PSNR, and 0.21 MSE for the FDG-PET image. In the classification of AD vs. cognitive normal (CN), AD vs. mild cognitive impairment (MCI), and CN vs. MCI, the precision of the proposed method is 94.73%, 92.98% and 89.36%, respectively. The sensitivity is 90. 70%, 90. 70%, and 90. 91% while the specificity is 100%, 100%, and 85. 71%, respectively, in the ADNI MRI test data.

14.
Phys Med Biol ; 69(18)2024 Sep 05.
Artículo en Inglés | MEDLINE | ID: mdl-39142339

RESUMEN

Objective.Respiratory motion, cardiac motion and inherently low signal-to-noise ratio (SNR) are major limitations ofin vivocardiac diffusion tensor imaging (DTI). We propose a novel enhancement method that uses unsupervised learning based invertible wavelet scattering (IWS) to improve the quality ofin vivocardiac DTI.Approach.Our method starts by extracting nearly transformation-invariant features from multiple cardiac diffusion-weighted (DW) image acquisitions using multi-scale wavelet scattering (WS). Then, the relationship between the WS coefficients and DW images is learned through a multi-scale encoder and a decoder network. Using the trained encoder, the deep features of WS coefficients of multiple DW image acquisitions are further extracted and then fused using an average rule. Finally, using the fused WS features and trained decoder, the enhanced DW images are derived.Main result.We evaluate the performance of the proposed method by comparing it with several methods on threein vivocardiac DTI datasets in terms of SNR, contrast to noise ratio (CNR), fractional anisotropy (FA), mean diffusivity (MD) and helix angle (HA). Comparing against the best comparison method, SNR/CNR of diastolic, gastric peristalsis influenced, and end-systolic DW images were improved by 1%/16%, 5%/6%, and 56%/30%, respectively. The approach also yielded consistent FA and MD values and more coherent helical fiber structures than the comparison methods used in this work.Significance.The ablation results verify that using the transformation-invariant and noise-robust wavelet scattering features enables us to effectively explore the useful information from the limited data, providing a potential mean to alleviate the dependence of the fusion results on the number of repeated acquisitions, which is beneficial for dealing with the issues of noise and residual motion simultaneously and therefore improving the quality ofinvivocardiac DTI. Code can be found inhttps://github.com/strawberry1996/WS-MCNN.


Asunto(s)
Aprendizaje Profundo , Imagen de Difusión Tensora , Procesamiento de Imagen Asistido por Computador , Imagen de Difusión Tensora/métodos , Procesamiento de Imagen Asistido por Computador/métodos , Relación Señal-Ruido , Humanos , Análisis de Ondículas , Corazón/diagnóstico por imagen , Corazón/fisiología , Diástole
15.
Neural Netw ; 179: 106603, 2024 Nov.
Artículo en Inglés | MEDLINE | ID: mdl-39146717

RESUMEN

Multi-focus image fusion (MFIF) is an important technique that aims to combine the focused regions of multiple source images into a fully clear image. Decision-map methods are widely used in MFIF to maximize the preservation of information from the source images. While many decision-map methods have been proposed, they often struggle with difficulties in determining focus and non-focus boundaries, further affecting the quality of the fused images. Dynamic threshold neural P (DTNP) systems are computational models inspired by biological spiking neurons, featuring dynamic threshold and spiking mechanisms to better distinguish focused and unfocused regions for decision map generation. However, original DTNP systems require manual parameter configuration and have only one stimulus. Therefore, they are not suitable to be used directly for generating high-precision decision maps. To overcome these limitations, we propose a variant called parameter adaptive dual channel DTNP (PADCDTNP) systems. Inspired by the spiking mechanisms of PADCDTNP systems, we further develop a new MFIF method. As a new neural model, PADCDTNP systems adaptively estimate parameters according to multiple external inputs to produce decision maps with robust boundaries, resulting in high-quality fusion results. Comprehensive experiments on the Lytro/MFFW/MFI-WHU dataset show that our method achieves advanced performance and yields comparable results to the fourteen representative MFIF methods. In addition, compared to the standard DTNP systems, PADCDTNP systems improve the fusion performance and fusion efficiency on the three datasets by 5.69% and 86.03%, respectively. The codes for both the proposed method and the comparison methods are released at https://github.com/MorvanLi/MFIF-PADCDTNP.


Asunto(s)
Redes Neurales de la Computación , Procesamiento de Imagen Asistido por Computador/métodos , Neuronas/fisiología , Humanos , Algoritmos , Potenciales de Acción/fisiología , Modelos Neurológicos , Animales
16.
Math Biosci Eng ; 21(7): 6710-6730, 2024 Jul 30.
Artículo en Inglés | MEDLINE | ID: mdl-39176416

RESUMEN

Infrared and visible image fusion (IVIF) is devoted to extracting and integrating useful complementary information from muti-modal source images. Current fusion methods usually require a large number of paired images to train the models in supervised or unsupervised way. In this paper, we propose CTFusion, a convolutional neural network (CNN)-Transformer-based IVIF framework that uses self-supervised learning. The whole framework is based on an encoder-decoder network, where encoders are endowed with strong local and global dependency modeling ability via the CNN-Transformer-based feature extraction (CTFE) module design. Thanks to the development of self-supervised learning, the model training does not require ground truth fusion images with simple pretext task. We designed a mask reconstruction task according to the characteristics of IVIF, through which the network can learn the characteristics of both infrared and visible images and extract more generalized features. We evaluated our method and compared it to five competitive traditional and deep learning-based methods on three IVIF benchmark datasets. Extensive experimental results demonstrate that our CTFusion can achieve the best performance compared to the state-of-the-art methods in both subjective and objective evaluations.

17.
Artif Intell Med ; 156: 102945, 2024 10.
Artículo en Inglés | MEDLINE | ID: mdl-39178622

RESUMEN

In the formulation of strategies for walking rehabilitation, achieving precise identification of the current state and making rational predictions about the future state are crucial but often unrealized. To tackle this challenge, our study introduces a unified framework that integrates a novel 3D walking motion capture method using multi-source image fusion and a walking rehabilitation simulation approach based on multi-agent reinforcement learning. We found that, (i) the proposal achieved an accurate 3D walking motion capture and outperforms other advanced methods. Experimental evidence indicates that, compared to similar visual skeleton tracking methods, the proposed approach yields results with higher Pearson correlation (r=0.93), intra-class correlation coefficient (ICC(2,1)=0.91), and narrower confidence intervals ([0.90,0.95] for r, [0.88,0.94] for ICC(2,1)) when compared to standard results. The outcomes of the proposed approach also exhibit commendable correlation and concurrence with those obtained through the IMU-based skeleton tracking method in the assessment of gait parameters ([0.85,0.89] for r, [0.75,0.81] for ICC(2,1)); (ii) multi-agent reinforcement learning has the potential to be used to solve the simulation task of gait rehabilitation. In mimicry experiment, our proposed simulation method for gait rehabilitation not only enables the intelligent agent to converge from the initial state to the target state, but also observes evolutionary patterns similar to those observed in clinical practice through motor state resolution. This study offers valuable contributions to walking rehabilitation, enabling precise assessment and simulation-based interventions, with potential implications for clinical practice and patient outcomes.


Asunto(s)
Marcha , Caminata , Humanos , Caminata/fisiología , Marcha/fisiología , Simulación por Computador , Refuerzo en Psicología , Imagenología Tridimensional/métodos , Aprendizaje Automático
18.
Comput Biol Med ; 179: 108771, 2024 Sep.
Artículo en Inglés | MEDLINE | ID: mdl-38970832

RESUMEN

Multimodal medical image fusion fuses images with different modalities and provides more comprehensive and integrated diagnostic information. However, current multimodal image fusion methods cannot effectively model non-local contextual feature relationships, and due to direct aggregation of the extracted features, they introduce unnecessary implicit noise into the fused images. To solve the above problems, this paper proposes a novel dual-branch hybrid fusion network called EMOST for medical image fusion that combines a convolutional neural network (CNN) and a transformer. First, to extract more comprehensive feature information, an effective feature extraction module is proposed, which consists of an efficient dense block (EDB), an attention module (AM), a multiscale convolution block (MCB), and three sparse transformer blocks (STB). Meanwhile, a lightweight efficient model (EMO) is used in the feature extraction module to exploit the efficiency of the CNN with the dynamic modeling capability of the transformer. Additionally, the STB is incorporated to adaptively maintain the most useful self-attention values and remove as much redundant noise as possible by developing the top-k selection operator. Moreover, a novel feature fusion rule is designed to efficiently integrate the features. Experiments are conducted on four types of multimodal medical images. The proposed method shows higher performance than the art-of-the-state methods in terms of quantitative and qualitative evaluations. The code of the proposed method is available at https://github.com/XUTauto/EMOST.


Asunto(s)
Redes Neurales de la Computación , Humanos , Imagen Multimodal/métodos , Procesamiento de Imagen Asistido por Computador/métodos , Algoritmos
19.
Sci Rep ; 14(1): 17609, 2024 07 30.
Artículo en Inglés | MEDLINE | ID: mdl-39080442

RESUMEN

Medical imaging is indispensable for accurate diagnosis and effective treatment, with modalities like MRI and CT providing diverse yet complementary information. Traditional image fusion methods, while essential in consolidating information from multiple modalities, often suffer from poor image quality and loss of crucial details due to inadequate handling of semantic information and limited feature extraction capabilities. This paper introduces a novel medical image fusion technique leveraging unsupervised image segmentation to enhance the semantic understanding of the fusion process. The proposed method, named DUSMIF, employs a multi-branch, multi-scale deep learning architecture that integrates advanced attention mechanisms to refine the feature extraction and fusion processes. An innovative approach that utilizes unsupervised image segmentation to extract semantic information is introduced, which is then integrated into the fusion process. This not only enhances the semantic relevance of the fused images but also improves the overall fusion quality. The paper proposes a sophisticated network structure that extracts and fuses features at multiple scales and across multiple branches. This structure is designed to capture a comprehensive range of image details and contextual information, significantly improving the fusion outcomes. Multiple attention mechanisms are incorporated to selectively emphasize important features and integrate them effectively across different modalities and scales. This approach ensures that the fused images maintain high quality and detail fidelity. A joint loss function combining content loss, structural similarity loss, and semantic loss is formulated. This function not only guides the network in preserving image brightness and texture but also ensures that the fused image closely resembles the source images in both content and structure. The proposed method demonstrates superior performance over existing fusion techniques in objective assessments and subjective evaluations, confirming its effectiveness in enhancing the diagnostic utility of fused medical images.


Asunto(s)
Imagen por Resonancia Magnética , Imagen Multimodal , Redes Neurales de la Computación , Semántica , Humanos , Imagen Multimodal/métodos , Imagen por Resonancia Magnética/métodos , Procesamiento de Imagen Asistido por Computador/métodos , Tomografía Computarizada por Rayos X/métodos , Aprendizaje Profundo , Algoritmos
20.
J Am Soc Mass Spectrom ; 35(8): 1797-1805, 2024 Aug 07.
Artículo en Inglés | MEDLINE | ID: mdl-38954826

RESUMEN

We have recently developed a charge inversion ion/ion reaction to selectively derivatize phosphatidylserine lipids via gas-phase Schiff base formation. This tandem mass spectrometry (MS/MS) workflow enables the separation and detection of isobaric lipids in imaging mass spectrometry, but the images acquired using this workflow are limited to relatively poor spatial resolutions due to the current time and limit of detection requirements for these ion/ion reaction imaging mass spectrometry experiments. This trade-off between chemical specificity and spatial resolution can be overcome by using computational image fusion, which combines complementary information from multiple images. Herein, we demonstrate a proof-of-concept workflow that fuses a low spatial resolution (i.e., 125 µm) ion/ion reaction product ion image with higher spatial resolution (i.e., 25 µm) ion images from a full scan experiment performed using the same tissue section, which results in a predicted ion/ion reaction product ion image with a 5-fold improvement in spatial resolution. Linear regression, random forest regression, and two-dimensional convolutional neural network (2-D CNN) predictive models were tested for this workflow. Linear regression and 2D CNN models proved optimal for predicted ion/ion images of PS 40:6 and SHexCer d38:1, respectively.


Asunto(s)
Iones , Espectrometría de Masas en Tándem , Espectrometría de Masas en Tándem/métodos , Iones/química , Iones/análisis , Animales , Fosfatidilserinas/química , Fosfatidilserinas/análisis , Redes Neurales de la Computación , Procesamiento de Imagen Asistido por Computador/métodos , Bases de Schiff/química , Ratones , Modelos Lineales
SELECCIÓN DE REFERENCIAS
DETALLE DE LA BÚSQUEDA