Automatic segmentation of 2D echocardiography ultrasound images by means of generative adversarial network.

Automated cardiac segmentation from two-dimensional (2D) echocardiographic images is a crucial step toward improving clinical diagnosis. Anatomical heterogeneity and inherent noise, however, present technical challenges and lower segmentation accuracy. The objective of this study is to propose a method for the automatic segmentation of the ventricular endocardium, the myocardium, and the left atrium, in order to accurately determine clinical indices. Specifically, we suggest using the recently introduced pixel-to-pixel Generative Adversarial Network (Pix2Pix GAN) model for accurate segmentation. To accomplish this, we integrate the backbone PatchGAN model for the discriminator and the UNET for the generator, for building the Pix2Pix GAN. The resulting model produces precisely segmented images, thanks to UNET's capability for precise segmentation and PatchGAN's capability for fine-grained discrimination. For the experimental validation, we use the Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) dataset, which consists of echocardiographic images from 500 patients in 2-chamber (2CH) and 4-chamber (4CH) views at the end-diastolic (ED) and end-systolic (ES) phases. Similarly to state-of-the-art studies on the same dataset, we followed the same train-test splits. Our results demonstrate that the proposed GAN-based technique improves segmentation performance for clinical and geometrical parameters compared to the state-of-the-art methods. More precisely, throughout the ED and ES phases, the mean Dice values for the left ventricular endocardium reached 0.961 and 0.930 for 2CH, and 0.959 and 0.950 for 4CH, respectively. Furthermore, the average ejection fraction correlation and Mean Absolute Error obtained were 0.95 and 3.2ml for 2CH, and 0.98 and 2.1ml for 4CH, outperforming the state-of-the-art results.

Unsuspected Limitations of 3D Printed Model in Planning of Complex Aortic Aneurysm Endovascular Treatment.

OBJECTIVE: Static 3-dimensional (3D) printing became attractive for operative planning in cases that involve difficult anatomy. An interactive (low cost, fast) 3D print allowing deliberate surgical practice can be used to improve interventional simulation and planning. BACKGROUND: Endovascular treatment of complex aortic aneurysms is technically challenging, especially in case of narrow aortic lumen or significant aortic angulation (hostile anatomy). The risk of complications such as graft kinking and target vessel occlusion is difficult to assess based solely on traditional software measuring methods and remain highly dependent on surgeon skills and expertise. METHODS: A patient with juxtarenal AAA with hostile anatomy had a 3-dimensional printed model constructed preoperatively according to computed tomography images. Endovascular graft implantation in the 3D printed aorta with a standard T-Branch Cook (Cook® Medical, Bloomington, IN, USA) was performed preoperatively in the simulation laboratory enabling optimized feasibility, surgical planning and intraoperative decision making. RESULTS: The 3D printed aortic model proved to be radio-opaque and allowed simulation of branched endovascular aortic repair (BREVAR). The assessment of intervention feasibility, as well as optimal branch position and orientation was found to be useful for surgeon confidence and the actual intervention in the patient. There was a remarkable agreement between the 3D printed model and both CT and X-ray angiographic images. Although the technical success was achieved as planned, a previously deployed renal stent caused unexpected difficulty in advancing the renal stent, which was not observed in the 3D model simulation. CONCLUSION: The 3D printed aortic models can be useful for determining feasibility, optimizing planning and intraoperative decision making in hostile anatomy improving the outcome. Despite already offering satisfying accuracy at present, further advancements could enhance the 3D model capability to replicate minor anatomical deformities and variations in tissue density.

Curated and Annotated Dataset of Lung US Images in Zambian Children with Clinical Pneumonia.

See also the commentary by Sitek in this issue. Supplemental material is available for this article.

Low-complexity lung ultrasound video scoring by means of intensity projection-based video compression.

Since the outbreak of COVID-19, efforts have been made towards semi-quantitative analysis of lung ultrasound (LUS) data to assess the patient's condition. Several methods have been proposed in this regard, with a focus on frame-level analysis, which was then used to assess the condition at the video and prognostic levels. However, no extensive work has been done to analyze lung conditions directly at the video level. This study proposes a novel method for video-level scoring based on compression of LUS video data into a single image and automatic classification to assess patient's condition. The method utilizes maximum, mean, and minimum intensity projection-based compression of LUS video data over time. This enables to preserve hyper- and hypo-echoic data regions, while compressing the video down to a maximum of three images. The resulting images are then classified using a convolutional neural network (CNN). Finally, the worst predicted score given among the images is assigned to the corresponding video. The results show that this compression technique can achieve a promising agreement at the prognostic level (81.62%), while the video-level agreement remains comparable with the state-of-the-art (46.19%). Conclusively, the suggested method lays down the foundation for LUS video compression, shifting from frame-level to direct video-level analysis of LUS data.

Time Efficient Ultrasound Localization Microscopy Based on A Novel Radial Basis Function 2D Interpolation.

Ultrasound localization microscopy (ULM) allows for the generation of super-resolved (SR) images of the vasculature by precisely localizing intravenously injected microbubbles. Although SR images may be useful for diagnosing and treating patients, their use in the clinical context is limited by the need for prolonged acquisition times and high frame rates. The primary goal of our study is to relax the requirement of high frame rates to obtain SR images. To this end, we propose a new time-efficient ULM (TEULM) pipeline built on a cutting-edge interpolation method. More specifically, we suggest employing Radial Basis Functions (RBFs) as interpolators to estimate the missing values in the 2-dimensional (2D) spatio-temporal structures. To evaluate this strategy, we first mimic the data acquisition at a reduced frame rate by applying a down-sampling (DS = 2, 4, 8, and 10) factor to high frame rate ULM data. Then, we up-sample the data to the original frame rate using the suggested interpolation to reconstruct the missing frames. Finally, using both the original high frame rate data and the interpolated one, we reconstruct SR images using the ULM framework steps. We evaluate the proposed TEULM using four in vivo datasets, a Rat brain (dataset A), a Rat kidney (dataset B), a Rat tumor (dataset C) and a Rat brain bolus (dataset D), interpolating at the in-phase and quadrature (IQ) level. Results demonstrate the effectiveness of TEULM in recovering vascular structures, even at a DS rate of 10 (corresponding to a frame rate of sub-100Hz). In conclusion, the proposed technique is successful in reconstructing accurate SR images while requiring frame rates of one order of magnitude lower than standard ULM.

A two-dimensional angular interpolation based on radial basis functions for high frame rate ultrafast imaging.

To solve the problem of reduced image quality in plane wave imaging (PWI), coherent plane wave compounding (CPWC) has been introduced, based on a combination of plane wave images from several directions (i.e., with different angles). However, the number of angles needed to reach a reasonable image quality affects the maximum achievable frame rate in CPWC. In this study, we suggest reducing the tradeoff between the image quality and the frame rate in CPWC by employing two-dimensional (2D) interpolation based on radial basis functions. More specifically, we propose constructing a three-dimensional spatio-angular structure to integrate both spatial and angular information into the reconstruction prior to 2D interpolation. The rationale behind our proposal is to reduce the number of transmissions and then apply the 2D interpolation along the angle dimension to reconstruct the missing information corresponding to the angles not selected for CPWC imaging. To evaluate the proposed technique, we applied it to the PWI challenges in the medical ultrasound database. Results show that we can achieve 3× to 4× improvement in frame rate while maintaining acceptable image quality compared to the case of using all the angles.

Ultrasound multifrequency strategy to estimate the lung surface roughness, in silico and in vitro results.

Lung ultrasound (LUS) is an important imaging modality to assess the state of the lung surface. Nevertheless, LUS is limited to the visual evaluation of imaging artifacts, especially the vertical ones. These artifacts are observed in pathologies characterized by a reduction of dimensions of air-spaces (alveoli). In contrast, there exist pathologies, such as chronic obstructive pulmonary disease (COPD), in which an enlargement of air-spaces can occur, which causes the lung surface to behave essentially as a perfect reflector, thus not allowing ultrasound penetration. This characteristic high reflectivity could be exploited to characterize the lung surface. Specifically, air-spaces of different sizes could cause the lung surface to have a different roughness, whose estimation could provide a way to assess the state of the lung surface. In this study, we present a quantitative multifrequency approach aiming at estimating the lung surface's roughness by measuring image intensity variations along the lung surface as a function of frequency. This approach was tested both in silico and in vitro, and it showed promising results. For the in vitro experiments, radiofrequency (RF) data were acquired from a novel experimental model. The results showed consistency between in silico and in vitro experiments.

Benchmark methodological approach for the application of artificial intelligence to lung ultrasound data from COVID-19 patients: From frame to prognostic-level.

Automated ultrasound imaging assessment of the effect of CoronaVirus disease 2019 (COVID-19) on lungs has been investigated in various studies using artificial intelligence-based (AI) methods. However, an extensive analysis of state-of-the-art Convolutional Neural Network-based (CNN) models for frame-level scoring, a comparative analysis of aggregation techniques for video-level scoring, together with a thorough evaluation of the capability of these methodologies to provide a clinically valuable prognostic-level score is yet missing within the literature. In addition to that, the impact on the analysis of the posterior probability assigned by the network to the predicted frames as well as the impact of temporal downsampling of LUS data are topics not yet extensively investigated. This paper takes on these challenges by providing a benchmark analysis of methods from frame to prognostic level. For frame-level scoring, state-of-the-art deep learning models are evaluated with additional analysis of best performing model in transfer-learning settings. A novel cross-correlation based aggregation technique is proposed for video and exam-level scoring. Results showed that ResNet-18, when trained from scratch, outperformed the existing methods with an F1-Score of 0.659. The proposed aggregation method resulted in 59.51%, 63.29%, and 84.90% agreement with clinicians at the video, exam, and prognostic levels, respectively; thus, demonstrating improved performances over the state of the art. It was also found that filtering frames based on the posterior probability shows higher impact on the LUS analysis in comparison to temporal downsampling. All of these analysis were conducted over the largest standardized and clinically validated LUS dataset from COVID-19 patients.

Increasing frame rate of echocardiography based on a novel 2D spatio-temporal meshless interpolation.

BACKGROUND: Increasing temporal resolution through numerical methods aids clinicians to evaluate fast moving structures of the heart with more confidence. METHODOLOGY: In this study, a spatio-temporal numerical method is proposed to increase the frame rate based on two-dimensional (2D) interpolation. More specifically, we propose a novel intensity variation time surface (IVTS) strategy to incorporate both temporal and spatial information in the reconstruction. In this regard, we exploit radial basis functions (RBFs) for 2D interpolation. The reason for choosing RBFs for this task is manifold. First, RBFs are able to interpolate on large-scale datasets. Moreover, their mathematical implementation is simple. Another important property of this interpolation technique, which is addressed in this study, is its meshless nature. The meshless property enables higher up-sampling (UpS) rates for echocardiography to improve temporal resolution without noticeably degrading image quality. To evaluate the proposed approach, we tested the RBF interpolation on 2D/3D echocardiography datasets. The reconstructed frames were analyzed using different image quality metrics, and the results were compared with two popular techniques from the literature. RESULTS: The findings demonstrated that, with a down-sampling rate of 3, the proposed technique outperformed the best existing method by 42%, 87%, 8%, and 11%, respectively, in terms of mean square error (MSE), contrast to noise ratio (CNR), peak signal-to-noise ratio (PSNR), and figure of merit (FOM). It should be noted that the proposed method is comparable to the best available method in terms of structural similarity (SSIM) index. Furthermore, when compared to the original images, the results of employing our technique on radio-frequency (RF) level analysis demonstrated that the reconstruction accuracy is satisfactory in terms of image quality criterion. CONCLUSION: Finally, it is worthwhile noting that the proposed method is better than (or comparable to) the other methods in terms of reconstruction performance and processing time. Therefore, the RBF interpolation can be a promising alternative to the existing methods.

Human-to-AI Interrater Agreement for Lung Ultrasound Scoring in COVID-19 Patients.

OBJECTIVES: Lung ultrasound (LUS) has sparked significant interest during COVID-19. LUS is based on the detection and analysis of imaging patterns. Vertical artifacts and consolidations are some of the recognized patterns in COVID-19. However, the interrater reliability (IRR) of these findings has not been yet thoroughly investigated. The goal of this study is to assess IRR in LUS COVID-19 data and determine how many LUS videos and operators are required to obtain a reliable result. METHODS: A total of 1035 LUS videos from 59 COVID-19 patients were included. Videos were randomly selected from a dataset of 1807 videos and scored by six human operators (HOs). The videos were also analyzed by artificial intelligence (AI) algorithms. Fleiss' kappa coefficient results are presented, evaluated at both the video and prognostic levels. RESULTS: Findings show a stable agreement when evaluating a minimum of 500 videos. The statistical analysis illustrates that, at a video level, a Fleiss' kappa coefficient of 0.464 (95% confidence interval [CI] = 0.455-0.473) and 0.404 (95% CI = 0.396-0.412) is obtained for pairs of HOs and for AI versus HOs, respectively. At prognostic level, a Fleiss' kappa coefficient of 0.505 (95% CI = 0.448-0.562) and 0.506 (95% CI = 0.458-0.555) is obtained for pairs of HOs and for AI versus HOs, respectively. CONCLUSIONS: To examine IRR and obtain a reliable evaluation, a minimum of 500 videos are recommended. Moreover, the employed AI algorithms achieve results that are comparable with HOs. This research further provides a methodology that can be useful to benchmark future LUS studies.

Multi-objective automatic analysis of lung ultrasound data from COVID-19 patients by means of deep learning and decision trees.

COVID-19 raised the need for automatic medical diagnosis, to increase the physicians' efficiency in managing the pandemic. Among all the techniques for evaluating the status of the lungs of a patient with COVID-19, lung ultrasound (LUS) offers several advantages: portability, cost-effectiveness, safety. Several works approached the automatic detection of LUS imaging patterns related COVID-19 by using deep neural networks (DNNs). However, the decision processes based on DNNs are not fully explainable, which generally results in a lack of trust from physicians. This, in turn, slows down the adoption of such systems. In this work, we use two previously built DNNs as feature extractors at the frame level, and automatically synthesize, by means of an evolutionary algorithm, a decision tree (DT) that aggregates in an interpretable way the predictions made by the DNNs, returning the severity of the patients' conditions according to a LUS score of prognostic value. Our results show that our approach performs comparably or better than previously reported aggregation techniques based on an empiric combination of frame-level predictions made by DNNs. Furthermore, when we analyze the evolved DTs, we discover properties about the DNNs used as feature extractors. We make our data publicly available for further development and reproducibility.

High Frame Rate Ultrasound Imaging by Means of Tensor Completion: Application to Echocardiography.

High frame rate ultrasound (US) imaging enables the monitoring of fast-moving organs. In echocardiography, this is especially needed due to the existence of rapidly moving structures, such as the heart valves. In the last two decades, various methods have been proposed to improve the frame rate. Here, we propose a novel method, based on binary coding patterns (BCPs) and tensor completion (TC), to increase the temporal resolution (i.e., frame rate) in the preprocessing stage of conventional focused ultrasound imaging (CFUI). The rationale behind our proposal is to perform, at first, the beamforming of a fraction of the scan lines, randomly selected in each frame based on BCP. Then, we reconstruct the missing scan lines through TC. The latter is an effective technique for recovering missing information from a low-rank tensor, based on a small number of observations using rank minimization. Following our approach, reducing the transmissions events needed to generate an image, the frame rate is increased by the same proportion. We have applied the proposed technique to a pre-beamformed radio frequency (RF) echocardiographic dataset. Our results show that we can improve the frame rate by a factor from 3 to 4, while keeping the structural similarity (SSIM) of the reconstructed tensor and the original one at values higher than 0.98.

Best Practice Recommendations for the Safe use of Lung Ultrasound.

The safety of ultrasound is of particular importance when examining the lungs, due to specific bioeffects occurring at the alveolar air-tissue interface. Lung is significantly more sensitive than solid tissue to mechanical stress. The causal biological effects due to the total reflection of sound waves have also not been investigated comprehensively.On the other hand, the clinical benefit of lung ultrasound is outstanding. It has gained considerable importance during the pandemic, showing comparable diagnostic value with other radiological imaging modalities.Therefore, based on currently available literature, this work aims to determine possible effects caused by ultrasound on the lung parenchyma and evaluate existing recommendations for acoustic output power limits when performing lung sonography.This work recommends a stepwise approach to obtain clinically relevant images while ensuring lung ultrasound safety. A special focus was set on the safety of new ultrasound modalities, which had not yet been introduced at the time of previous recommendations.Finally, necessary research and training steps are recommended in order to close knowledge gaps in the field of lung ultrasound safety in the future.These recommendations for practice were prepared by ECMUS, the safety committee of the EFSUMB, with participation of international experts in the field of lung sonography and ultrasound bioeffects.

New International Guidelines and Consensus on the Use of Lung Ultrasound.

Following the innovations and new discoveries of the last 10 years in the field of lung ultrasound (LUS), a multidisciplinary panel of international LUS experts from six countries and from different fields (clinical and technical) reviewed and updated the original international consensus for point-of-care LUS, dated 2012. As a result, a total of 20 statements have been produced. Each statement is complemented by guidelines and future developments proposals. The statements are furthermore classified based on their nature as technical (5), clinical (11), educational (3), and safety (1) statements.

State of the Art in Lung Ultrasound, Shifting from Qualitative to Quantitative Analyses.

Lung ultrasound (LUS) has been increasingly expanding since the 1990s, when the clinical relevance of vertical artifacts was first reported. However, the massive spread of LUS is only recent and is associated with the coronavirus disease 2019 (COVID-19) pandemic, during which semi-quantitative computer-aided techniques were proposed to automatically classify LUS data. In this review, we discuss the state of the art in LUS, from semi-quantitative image analysis approaches to quantitative techniques involving the analysis of radiofrequency data. We also discuss recent in vitro and in silico studies, as well as research on LUS safety. Finally, conclusions are drawn highlighting the potential future of LUS.

Deep Learning-Based Classification of Reduced Lung Ultrasound Data From COVID-19 Patients.

The application of lung ultrasound (LUS) imaging for the diagnosis of lung diseases has recently captured significant interest within the research community. With the ongoing COVID-19 pandemic, many efforts have been made to evaluate LUS data. A four-level scoring system has been introduced to semiquantitatively assess the state of the lung, classifying the patients. Various deep learning (DL) algorithms supported with clinical validations have been proposed to automate the stratification process. However, no work has been done to evaluate the impact on the automated decision by varying pixel resolution and bit depth, leading to the reduction in size of overall data. This article evaluates the performance of DL algorithm over LUS data with varying pixel and gray-level resolution. The algorithm is evaluated over a dataset of 448 LUS videos captured from 34 examinations of 20 patients. All videos are resampled by a factor of 2, 3, and 4 of original resolution, and quantized to 128, 64, and 32 levels, followed by score prediction. The results indicate that the automated scoring shows negligible variation in accuracy when it comes to the quantization of intensity levels only. Combined effect of intensity quantization with spatial down-sampling resulted in a prognostic agreement ranging from 73.5% to 82.3%.These results also suggest that such level of prognostic agreement can be achieved over evaluation of data reduced to 32 times of its original size. Thus, laying foundation to efficient processing of data in resource constrained environments.

Lung Ultrasound in COVID-19 and Post-COVID-19 Patients, an Evidence-Based Approach.

OBJECTIVES: Worldwide, lung ultrasound (LUS) was utilized to assess coronavirus disease 2019 (COVID-19) patients. Often, imaging protocols were however defined arbitrarily and not following an evidence-based approach. Moreover, extensive studies on LUS in post-COVID-19 patients are currently lacking. This study analyses the impact of different LUS imaging protocols on the evaluation of COVID-19 and post-COVID-19 LUS data. METHODS: LUS data from 220 patients were collected, 100 COVID-19 positive and 120 post-COVID-19. A validated and standardized imaging protocol based on 14 scanning areas and a 4-level scoring system was implemented. We utilized this dataset to compare the capability of 5 imaging protocols, respectively based on 4, 8, 10, 12, and 14 scanning areas, to intercept the most important LUS findings. This to evaluate the optimal trade-off between a time-efficient imaging protocol and an accurate LUS examination. We also performed a longitudinal study, aimed at investigating how to eventually simplify the protocol during follow-up. Additionally, we present results on the agreement between AI models and LUS experts with respect to LUS data evaluation. RESULTS: A 12-areas protocol emerges as the optimal trade-off, for both COVID-19 and post-COVID-19 patients. For what concerns follow-up studies, it appears not to be possible to reduce the number of scanning areas. Finally, COVID-19 and post-COVID-19 LUS data seem to show differences capable to confuse AI models that were not trained on post-COVID-19 data, supporting the hypothesis of the existence of LUS patterns specific to post-COVID-19 patients. CONCLUSIONS: A 12-areas acquisition protocol is recommended for both COVID-19 and post-COVID-19 patients, also during follow-up.

Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19.

Lung ultrasound (LUS) is a cheap, safe and non-invasive imaging modality that can be performed at patient bed-side. However, to date LUS is not widely adopted due to lack of trained personnel required for interpreting the acquired LUS frames. In this work we propose a framework for training deep artificial neural networks for interpreting LUS, which may promote broader use of LUS. When using LUS to evaluate a patient's condition, both anatomical phenomena (e.g., the pleural line, presence of consolidations), as well as sonographic artifacts (such as A- and B-lines) are of importance. In our framework, we integrate domain knowledge into deep neural networks by inputting anatomical features and LUS artifacts in the form of additional channels containing pleural and vertical artifacts masks along with the raw LUS frames. By explicitly supplying this domain knowledge, standard off-the-shelf neural networks can be rapidly and efficiently finetuned to accomplish various tasks on LUS data, such as frame classification or semantic segmentation. Our framework allows for a unified treatment of LUS frames captured by either convex or linear probes. We evaluated our proposed framework on the task of COVID-19 severity assessment using the ICLUS dataset. In particular, we finetuned simple image classification models to predict per-frame COVID-19 severity score. We also trained a semantic segmentation model to predict per-pixel COVID-19 severity annotations. Using the combined raw LUS frames and the detected lines for both tasks, our off-the-shelf models performed better than complicated models specifically designed for these tasks, exemplifying the efficacy of our framework.

Deep learning applied to lung ultrasound videos for scoring COVID-19 patients: A multicenter study.

In the current pandemic, lung ultrasound (LUS) played a useful role in evaluating patients affected by COVID-19. However, LUS remains limited to the visual inspection of ultrasound data, thus negatively affecting the reliability and reproducibility of the findings. Moreover, many different imaging protocols have been proposed, most of which lacked proper clinical validation. To address these problems, we were the first to propose a standardized imaging protocol and scoring system. Next, we developed the first deep learning (DL) algorithms capable of evaluating LUS videos providing, for each video-frame, the score as well as semantic segmentation. Moreover, we have analyzed the impact of different imaging protocols and demonstrated the prognostic value of our approach. In this work, we report on the level of agreement between the DL and LUS experts, when evaluating LUS data. The results show a percentage of agreement between DL and LUS experts of 85.96% in the stratification between patients at high risk of clinical worsening and patients at low risk. These encouraging results demonstrate the potential of DL models for the automatic scoring of LUS data, when applied to high quality data acquired accordingly to a standardized imaging protocol.