Towards a survival risk prediction model for metastatic NSCLC patients on durvalumab using whole-lung CT radiomics.

Background: Existing criteria for predicting patient survival from immunotherapy are primarily centered on the PD-L1 status of patients. We tested the hypothesis that noninvasively captured baseline whole-lung radiomics features from CT images, baseline clinical parameters, combined with advanced machine learning approaches, can help to build models of patient survival that compare favorably with PD-L1 status for predicting 'less-than-median-survival risk' in the metastatic NSCLC setting for patients on durvalumab. With a total of 1062 patients, inclusive of model training and validation, this is the largest such study yet. Methods: To ensure a sufficient sample size, we combined data from treatment arms of three metastatic NSCLC studies. About 80% of this data was used for model training, and the remainder was held-out for validation. We first trained two independent models; Model-C trained to predict survival using clinical data; and Model-R trained to predict survival using whole-lung radiomics features. Finally, we created Model-C+R which leveraged both clinical and radiomics features. Results: The classification accuracy (for median survival) of Model-C, Model-R, and Model-C+R was 63%, 55%, and 68% respectively. Sensitivity analysis of survival prediction across different training and validation cohorts showed concordance indices ([95 percentile]) of 0.64 ([0.63, 0.65]), 0.60 ([0.59, 0.60]), and 0.66 ([0.65,0.67]), respectively. We additionally evaluated generalization of these models on a comparable cohort of 144 patients from an independent study, demonstrating classification accuracies of 65%, 62%, and 72% respectively. Conclusion: Machine Learning models combining baseline whole-lung CT radiomic and clinical features may be a useful tool for patient selection in immunotherapy. Further validation through prospective studies is needed.

Robust foreground detection in video using pixel layers.

A framework for robust foreground detection that works under difficult conditions such as dynamic background and moderately moving camera is presented in this paper. The proposed method includes two main components: coarse scene representation as the union of pixel layers, and foreground detection in video by propagating these layers using a maximum-likelihood assignment. We first cluster into "layers" those pixels that share similar statistics. The entire scene is then modeled as the union of such non-parametric layer-models. An in-coming pixel is detected as foreground if it does not adhere to these adaptive models of the background. A principled way of computing thresholds is used to achieve robust detection performance with a pre-specified number of false alarms. Correlation between pixels in the spatial vicinity is exploited to deal with camera motion without precise registration or optical flow. The proposed technique adapts to changes in the scene, and allows to automatically convert persistent foreground objects to background and re-convert them to foreground when they become interesting. This simple framework addresses the important problem of robust foreground and unusual region detection, at about 10 frames per second on a standard laptop computer. The presentation of the proposed approach is complemented by results on challenging real data and comparisons with other standard techniques.

Video inpainting under constrained camera motion.

A framework for inpainting missing parts of a video sequence recorded with a moving or stationary camera is presented in this work. The region to be inpainted is general: it may be still or moving, in the background or in the foreground, it may occlude one object and be occluded by some other object. The algorithm consists of a simple preprocessing stage and two steps of video inpainting. In the preprocessing stage, we roughly segment each frame into foreground and background. We use this segmentation to build three image mosaics that help to produce time consistent results and also improve the performance of the algorithm by reducing the search space. In the first video inpainting step, we reconstruct moving objects in the foreground that are "occluded" by the region to be inpainted. To this end, we fill the gap as much as possible by copying information from the moving foreground in other frames, using a priority-based scheme. In the second step, we inpaint the remaining hole with the background. To accomplish this, we first align the frames and directly copy when possible. The remaining pixels are filled in by extending spatial texture synthesis techniques to the spatiotemporal domain. The proposed framework has several advantages over state-of-the-art algorithms that deal with similar types of data and constraints. It permits some camera motion, is simple to implement, fast, does not require statistical models of background nor foreground, works well in the presence of rich and cluttered backgrounds, and the results show that there is no visible blurring or motion artifacts. A number of real examples taken with a consumer hand-held camera are shown supporting these findings.

Mitral valve annulus localization in 3D echocardiography.

The Mitral Valve is a structure on the left side of the human heart that regulates the flow of oxygenated blood into the Left Ventricle and also helps maintain the pressure within the Left Ventricle when the blood gets pumped to the rest of the body from the Left Ventricle. Pathology of the Mitral Valve often manifests through structural changes in the anatomy. Assessment of Mitral Valve pathology as well as determination of specific interventions require quantification of various structures of the Mitral Valve and one of these structures of interest is the Mitral Valve annulus. Three dimensional echocardiography is a popular imaging technique that clinicians use for volumetric quantification of cardiac structures - but manual quantification is cumbersome and subjective. In this paper we describe a semi-automated approach for maximum-a-posteriori estimation of the Mitral Valve annulus from three-dimensional echocardiography images using a simple analytic shape model coupled with a Näive Bayes classifier. We validate our approach against manual annotations on over 15 real patient cases with an average localization error ≤ 2.59 mm.

Toward Quantitative Assessment of Rheumatoid Arthritis Using Volumetric Ultrasound.

GOAL: Rheumatoid arthritis (RA) is characterized by inflammation within the joint space as well as erosion or destruction of the bone surface. We believe that volumetric (3-D) ultrasound imaging of the joints in conjunction with automated image-analysis tools for segmenting and quantifying the regions of interest can lead to improved RA assessment. METHODS: In this paper, we describe our proposed algorithms for segmenting 1) the 3 -D bone surface and 2) the 3-D joint capsule region. We improve and extend previous 2-D bone extraction methods to 3-D and make our algorithm more robust to the intensity loss due to surface normals facing away from incident acoustic beams. The extracted bone surfaces coupled with a joint-specific anatomical model are used to initialize a coarse localization of the joint capsule region. The joint capsule segmentation is refined iteratively utilizing a probabilistic speckle model. RESULTS: We apply our methods on 51 volumes from 8 subjects, and validate segmentation results with expert annotations. We also provide the quantitative comparison of our bone detection with magnetic resonance imaging. These automated methods have achieved average sensitivity/precision rates of 94%/93% for bone surface detection, and 87%/83% for joint capsule segmentation. Segmentations of normal and inflamed joints are compared to demonstrate the potential of using proposed tools to assess RA pathology at the joint level. CONCLUSION: The proposed image-analysis methods showed encouraging results as compared to expert annotations. SIGNIFICANCE: These computer-assisted tools can be used to help visualize 3-D anatomy in joints and help develop quantitative measurements toward RA assessment.

Symmetry and appearance based automated detection of salient anatomical regions in ultrasound.

In this paper we present a method for automated detection of enclosed anatomical regions in ultrasound images by utilizing the coarse shape symmetry as well as relative homogeneity of their sonographic appearance. The proposed method comprises of two steps: First, local phase based filtering [2] is used to detect points in the image which are roughly positioned along the axes of spatial symmetry with respect to structures around them. Secondly, the sonographic 'appearance' and location of these points is used to define a distance-map on the image, which is supplied to a simple fast-marching algorithm in order to provide the final feature detections. The method is robust to ultrasound speckle and works well with or without specialized pre-processing (e.g. speckle-reduction filtering). We illustrate the proposed method with qualitative results on in-vivo Ultrasound images.

Volumetric ultrasound and computer-assisted analysis at the point-of-care: a musculoskeletal exemplar.

In this paper we motivate the hypothesis that the use of volumetric ultrasound imaging and automated image analysis tools would improve clinical workflows as well as outcomes at the point-of-care. To make our case, this paper presents results from a rheumatoid arthritis (RA) study where several image analysis techniques have been applied to volumetric ultrasound, highlighting anatomy of interest to better understand disease progression. Pathologies related to RA in joints, manifest themselves commonly as changes in the bone (e.g. erosions) and the region enclosed by the joint-capsule (e.g. synovitis). Automated tools for detecting and segmenting such structures would help significantly towards objective and quantitative assessment of RA in joints. Extracted bone coupled with a simple anatomical model of the joint provides a coarse localization of the joint-capsule region. A probabilistic speckle model is then used to iteratively refine the capsule segmentation. We illustrate the performance of proposed algorithms through quantitative comparisons with expert annotations as well as qualitative results on over 30 scans obtained from 11 subjects.