HAtt-Flow: Hierarchical Attention-Flow Mechanism for Group-Activity Scene Graph Generation in Videos.

Group-activity scene graph (GASG) generation is a challenging task in computer vision, aiming to anticipate and describe relationships between subjects and objects in video sequences. Traditional video scene graph generation (VidSGG) methods focus on retrospective analysis, limiting their predictive capabilities. To enrich the scene-understanding capabilities, we introduced a GASG dataset extending the JRDB dataset with nuanced annotations involving appearance, interaction, position, relationship, and situation attributes. This work also introduces an innovative approach, a Hierarchical Attention-Flow (HAtt-Flow) mechanism, rooted in flow network theory to enhance GASG performance. Flow-attention incorporates flow conservation principles, fostering competition for sources and allocation for sinks, effectively preventing the generation of trivial attention. Our proposed approach offers a unique perspective on attention mechanisms, where conventional "values" and "keys" are transformed into sources and sinks, respectively, creating a novel framework for attention-based models. Through extensive experiments, we demonstrate the effectiveness of our Hatt-Flow model and the superiority of our proposed flow-attention mechanism. This work represents a significant advancement in predictive video scene understanding, providing valuable insights and techniques for applications that require real-time relationship prediction in video data.

Impact of stereopure chimeric backbone chemistries on the potency and durability of gene silencing by RNA interference.

Herein, we report the systematic investigation of stereopure phosphorothioate (PS) and phosphoryl guanidine (PN) linkages on siRNA-mediated silencing. The incorporation of appropriately positioned and configured stereopure PS and PN linkages to N-acetylgalactosamine (GalNAc)-conjugated siRNAs based on multiple targets (Ttr and HSD17B13) increased potency and durability of mRNA silencing in mouse hepatocytes in vivo compared with reference molecules based on clinically proven formats. The observation that the same modification pattern had beneficial effects on unrelated transcripts suggests that it may be generalizable. The effect of stereopure PN modification on silencing is modulated by 2'-ribose modifications in the vicinity, particularly on the nucleoside 3' to the linkage. These benefits corresponded with both an increase in thermal instability at the 5'-end of the antisense strand and improved Argonaute 2 (Ago2) loading. Application of one of our most effective designs to generate a GalNAc-siRNA targeting human HSD17B13 led to ∼80% silencing that persisted for at least 14 weeks after administration of a single 3 mg/kg subcutaneous dose in transgenic mice. The judicious use of stereopure PN linkages improved the silencing profile of GalNAc-siRNAs without disrupting endogenous RNA interference pathways and without elevating serum biomarkers for liver dysfunction, suggesting they may be suitable for therapeutic application.

Deep reinforcement learning in medical imaging: A literature review.

Deep reinforcement learning (DRL) augments the reinforcement learning framework, which learns a sequence of actions that maximizes the expected reward, with the representative power of deep neural networks. Recent works have demonstrated the great potential of DRL in medicine and healthcare. This paper presents a literature review of DRL in medical imaging. We start with a comprehensive tutorial of DRL, including the latest model-free and model-based algorithms. We then cover existing DRL applications for medical imaging, which are roughly divided into three main categories: (i) parametric medical image analysis tasks including landmark detection, object/lesion detection, registration, and view plane localization; (ii) solving optimization tasks including hyperparameter tuning, selecting augmentation strategies, and neural architecture search; and (iii) miscellaneous applications including surgical gesture segmentation, personalized mobile health intervention, and computational model personalization. The paper concludes with discussions of future perspectives.

Narrow Band Active Contour Attention Model for Medical Segmentation.

Medical image segmentation is one of the most challenging tasks in medical image analysis and widely developed for many clinical applications. While deep learning-based approaches have achieved impressive performance in semantic segmentation, they are limited to pixel-wise settings with imbalanced-class data problems and weak boundary object segmentation in medical images. In this paper, we tackle those limitations by developing a new two-branch deep network architecture which takes both higher level features and lower level features into account. The first branch extracts higher level feature as region information by a common encoder-decoder network structure such as Unet and FCN, whereas the second branch focuses on lower level features as support information around the boundary and processes in parallel to the first branch. Our key contribution is the second branch named Narrow Band Active Contour (NB-AC) attention model which treats the object contour as a hyperplane and all data inside a narrow band as support information that influences the position and orientation of the hyperplane. Our proposed NB-AC attention model incorporates the contour length with the region energy involving a fixed-width band around the curve or surface. The proposed network loss contains two fitting terms: (i) a high level feature (i.e., region) fitting term from the first branch; (ii) a lower level feature (i.e., contour) fitting term from the second branch including the (ii1) length of the object contour and (ii2) regional energy functional formed by the homogeneity criterion of both the inner band and outer band neighboring the evolving curve or surface. The proposed NB-AC loss can be incorporated into both 2D and 3D deep network architectures. The proposed network has been evaluated on different challenging medical image datasets, including DRIVE, iSeg17, MRBrainS18 and Brats18. The experimental results have shown that the proposed NB-AC loss outperforms other mainstream loss functions: Cross Entropy, Dice, Focal on two common segmentation frameworks Unet and FCN. Our 3D network which is built upon the proposed NB-AC loss and 3DUnet framework achieved state-of-the-art results on multiple volumetric datasets.

Enhance Portable Radiograph for Fast and High Accurate COVID-19 Monitoring.

This work aimed to assist physicians by improving their speed and diagnostic accuracy when interpreting portable CXRs as well as monitoring the treatment process to see whether a patient is improving or deteriorating with treatment. These objectives are in especially high demand in the setting of the ongoing COVID-19 pandemic. With the recent progress in the development of artificial intelligence (AI), we introduce new deep learning frameworks to align and enhance the quality of portable CXRs to be more consistent, and to more closely match higher quality conventional CXRs. These enhanced portable CXRs can then help the doctors provide faster and more accurate diagnosis and treatment planning. The contributions of this work are four-fold. Firstly, a new database collection of subject-pair radiographs is introduced. For each subject, we collected a pair of samples from both portable and conventional machines. Secondly, a new deep learning approach is presented to align the subject-pairs dataset to obtain a pixel-pairs dataset. Thirdly, a new PairFlow approach is presented, an end-to-end invertible transfer deep learning method, to enhance the degraded quality of portable CXRs. Finally, the performance of the proposed system is evaluated by UAMS doctors in terms of both image quality and topological properties. This work was undertaken in collaboration with the Department of Radiology at the University of Arkansas for Medical Sciences (UAMS) to enhance portable/mobile COVID-19 CXRs, to improve the speed and accuracy of portable CXR images and aid in urgent COVID-19 diagnosis, monitoring and treatment.

Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation.

Variational Level Set (LS) has been a widely used method in medical segmentation. However, it is limited when dealing with multi-instance objects in the real world. In addition, its segmentation results are quite sensitive to initial settings and highly depend on the number of iterations. To address these issues and boost the classic variational LS methods to a new level of the learnable deep learning approaches, we propose a novel definition of contour evolution named Recurrent Level Set (RLS) 1 to employ Gated Recurrent Unit under the energy minimization of a variational LS functional. The curve deformation process in RLS is formed as a hidden state evolution procedure and updated by minimizing an energy functional composed of fitting forces and contour length. By sharing the convolutional features in a fully end-to-end trainable framework, we extend RLS to Contextual RLS (CRLS) to address semantic segmentation in the wild. The experimental results have shown that our proposed RLS improves both computational time and segmentation accuracy against the classic variational LS-based method whereas the fully end-to-end system CRLS achieves competitive performance compared to the state-of-the-art semantic segmentation approaches.

Compressed Submanifold Multifactor Analysis.

Although widely used, Multilinear PCA (MPCA), one of the leading multilinear analysis methods, still suffers from four major drawbacks. First, it is very sensitive to outliers and noise. Second, it is unable to cope with missing values. Third, it is computationally expensive since MPCA deals with large multi-dimensional datasets. Finally, it is unable to maintain the local geometrical structures due to the averaging process. This paper proposes a novel approach named Compressed Submanifold Multifactor Analysis (CSMA) to solve the four problems mentioned above. Our approach can deal with the problem of missing values and outliers via SVD-L1. The Random Projection method is used to obtain the fast low-rank approximation of a given multifactor dataset. In addition, it is able to preserve the geometry of the original data. Our CSMA method can be used efficiently for multiple purposes, e.g. noise and outlier removal, estimation of missing values, biometric applications. We show that CSMA method can achieve good results and is very efficient in the inpainting problem as compared to [1], [2]. Our method also achieves higher face recognition rates compared to LRTC, SPMA, MPCA and some other methods, i.e. PCA, LDA and LPP, on three challenging face databases, i.e. CMU-MPIE, CMU-PIE and Extended YALE-B.

Spartans: Single-Sample Periocular-Based Alignment-Robust Recognition Technique Applied to Non-Frontal Scenarios.

In this paper, we investigate a single-sample periocular-based alignment-robust face recognition technique that is pose-tolerant under unconstrained face matching scenarios. Our Spartans framework starts by utilizing one single sample per subject class, and generate new face images under a wide range of 3D rotations using the 3D generic elastic model which is both accurate and computationally economic. Then, we focus on the periocular region where the most stable and discriminant features on human faces are retained, and marginalize out the regions beyond the periocular region since they are more susceptible to expression variations and occlusions. A novel facial descriptor, high-dimensional Walsh local binary patterns, is uniformly sampled on facial images with robustness toward alignment. During the learning stage, subject-dependent advanced correlation filters are learned for pose-tolerant non-linear subspace modeling in kernel feature space followed by a coupled max-pooling mechanism which further improve the performance. Given any unconstrained unseen face image, the Spartans can produce a highly discriminative matching score, thus achieving high verification rate. We have evaluated our method on the challenging Labeled Faces in the Wild database and solidly outperformed the state-of-the-art algorithms under four evaluation protocols with a high accuracy of 89.69%, a top score among image-restricted and unsupervised protocols. The advancement of Spartans is also proven in the Face Recognition Grand Challenge and Multi-PIE databases. In addition, our learning method based on advanced correlation filters is much more effective, in terms of learning subject-dependent pose-tolerant subspaces, compared with many well-established subspace methods in both linear and non-linear cases.

SparCLeS: dynamic l1 sparse classifiers with level sets for robust beard/moustache detection and segmentation.

Robust facial hair detection and segmentation is a highly valued soft biometric attribute for carrying out forensic facial analysis. In this paper, we propose a novel and fully automatic system, called SparCLeS, for beard/moustache detection and segmentation in challenging facial images. SparCLeS uses the multiscale self-quotient (MSQ) algorithm to preprocess facial images and deal with illumination variation. Histogram of oriented gradients (HOG) features are extracted from the preprocessed images and a dynamic sparse classifier is built using these features to classify a facial region as either containing skin or facial hair. A level set based approach, which makes use of the advantages of both global and local information, is then used to segment the regions of a face containing facial hair. Experimental results demonstrate the effectiveness of our proposed system in detecting and segmenting facial hair regions in images drawn from three databases, i.e., the NIST Multiple Biometric Grand Challenge (MBGC) still face database, the NIST Color Facial Recognition Technology FERET database, and the Labeled Faces in the Wild (LFW) database.