Unsupervised Modality-Transferable Video Highlight Detection With Representation Activation Sequence Learning.

Identifying highlight moments of raw video materials is crucial for improving the efficiency of editing videos that are pervasive on internet platforms. However, the extensive work of manually labeling footage has created obstacles to applying supervised methods to videos of unseen categories. The absence of an audio modality that contains valuable cues for highlight detection in many videos also makes it difficult to use multimodal strategies. In this paper, we propose a novel model with cross-modal perception for unsupervised highlight detection. The proposed model learns representations with visual-audio level semantics from image-audio pair data via a self-reconstruction task. To achieve unsupervised highlight detection, we investigate the latent representations of the network and propose the representation activation sequence learning (RASL) module with k-point contrastive learning to learn significant representation activations. To connect the visual modality with the audio modality, we use the symmetric contrastive learning (SCL) module to learn the paired visual and audio representations. Furthermore, an auxiliary task of masked feature vector sequence (FVS) reconstruction is simultaneously conducted during pretraining for representation enhancement. During inference, the cross-modal pretrained model can generate representations with paired visual-audio semantics given only the visual modality. The RASL module is used to output the highlight scores. The experimental results show that the proposed framework achieves superior performance compared to other state-of-the-art approaches.

A novel inflammatory nutrient index for predicting survival outcomes in patients with non-small cell lung cancer.

Background: Lung cancer is one of the most common contributors to cancer-related deaths worldwide. This study aimed to develop a new blood index on the basis of the patient's systemic inflammation and nutritional status, which can be used to predict the prognosis of patients with non-small cell lung cancer (NSCLC). Methods: Pre-treatment blood markers were analyzed in 556 NSCLC patients from 2010 to 2019. A least absolute shrinkage and selection operator (LASSO) method was used to select indicators to establish a new integrated biomarker (PNAGR). Kaplan-Meier survival curves were used to assess the prognostic impact of platelet-to-lymphocyte ratio (PLR), albumin (ALB), and the PNAGR. The prognostic value was verified using univariate and multivariate Cox analyses. Results: We used four biomarkers including PLR, ALB, 1/albumin-to-globulin ratio (1/AGR), and neutrophil/albumin-to-globulin ratio (N/AGR) were used to screen for the PNAGR using LASSO. Patients with high PNAGR demonstrated lower overall survival (OS) compared to those with low PNAGR. In both univariate and multivariate analyses, PNAGR was revealed as an independent prognostic factor for OS. The predictive power of PNAGR [area under the curve (AUC): 0.753] was higher than that of the metrics alone. Conclusions: PNAGR is a novel and effective clinical prognostic tool with good clinical predictive value for NSCLC patients.