ABSTRACT
Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learning new knowledge from limited training examples without forgetting previous knowledge. However, we observe that existing methods face a challenge known as supervision collapse, where the model disproportionately emphasizes class-specific features of base classes at the detriment of novel class representations, leading to restricted cognitive capabilities. To alleviate this issue, we propose a new framework, Model aTtention Expansion for Few-Shot Class-Incremental Learning (MTE-FSCIL), aimed at expanding the model attention fields to improve transferability without compromising the discriminative capability for base classes. Specifically, the framework adopts a dual-stage training strategy, comprising pre-training and meta-training stages. In the pre-training stage, we present a new regularization technique, named the Reserver (RS) loss, to expand the global perception and reduce over-reliance on class-specific features by amplifying feature map activations. During the meta-training stage, we introduce the Repeller (RP) loss, a novel pair-based loss that promotes variation in representations and improves the model's recognition of sample uniqueness by scattering intra-class samples within the embedding space. Furthermore, we propose a Transformational Adaptation (TA) strategy to enable continuous incorporation of new knowledge from downstream tasks, thus facilitating cross-task knowledge transfer. Extensive experimental results on mini-ImageNet, CIFAR100, and CUB200 datasets demonstrate that our proposed framework consistently outperforms the state-of-the-art methods.
ABSTRACT
The brain's function of clearance and transport is closely related to the prognosis of acute ischemic stroke (AIS). In this study, we proposed a novel method, clearance rate of contrast extravasation (CROCE), to measure brain clearance and transport function in AIS patients undergoing endovascular therapy (EVT), and examined its association with cerebral edema and functional outcome. We conducted a pooled analysis of AIS patients of anterior circulation large vessel occlusion who underwent EVT in two academic hospitals. Patients who experienced contrast extravasation but not intracerebral hemorrhage following EVT were included. CROCE was defined as the mass of contrast agent cleared per hour on non-contrast CT (NCCT). Among the 215 patients finally included, we found that high CROCE was significantly associated with 90-day favorable functional outcome, and the association retained after adjustment for potential confounders. Different correlation analysis demonstrated a significant correlation between CROCE, cerebral edema, and functional outcome. Further mediation analysis revealed that cerebral edema mediated the effect of CROCE on functional outcome. These results revealed that CROCE may be a promising indicator of brain clearance function for patients who received EVT and had contrast extravasation.
ABSTRACT
Impedance cardiography (ICG) plays a crucial role in clinically evaluating cardiac systolic and diastolic functions, along with various other cardiac parameters. However, its accuracy heavily depends on precisely identifying feature points reflecting cardiac function. Moreover, traditional signal processing techniques used to mitigate random noise and breathing artifacts may inadvertently distort the amplitude and temporal characteristics of ICG signals. To address this issue, this study investigates a noise and artifact elimination method based on Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) and Particle Swarm Optimization-based Variational Mode Decomposition Algorithm (PSO-VMD). The goal is to preserve the amplitude and temporal features of ICG signals to ensure accurate feature point extraction and computation of associated cardiac parameters. Comparative analysis with signal processing methods employing various wavelet families and Ensemble Empirical Mode Decomposition (EEMD) in ICG signal processing applications reveals that the proposed method achieves superior signal-to-noise ratio (SNR) and lower root-mean-square error (RMSE), while demonstrating enhanced correlation and waveform consistency with the original signal.
ABSTRACT
The relationship between aboveground biomass and plant diversity has been extensively examined to understand the role of biodiversity in ecosystem functions and services. Degraded grassland restoration projects can enhance carbon sequestration. However, the relationship between biomass and diversity remains one of the most actively debated topics regarding grassland ecosystems in degraded grassland restoration projects. We speculated that establishing the linear relationships between aboveground biomass and plant species diversity could contribute to enhancing the efficacy of degraded grassland restoration projects. This study sought to determine whether these relationships were linear during the initial stages of the restoration projects of degraded grasslands in Xing'an League, China. The investigations were based on an examination of seventy-six 1 × 1 m2 plots distributed among 15 areas in which the degraded grassland was at the initial stages of restoration. To quantify the species diversity of the degraded grassland communities, we used the species richness, Shannon-Wiener, inverse Simpson's reciprocal, and Pielou's evenness indices. Our analyses revealed that aboveground biomass had clear positive linear relationships with species richness during the initial stages of degraded grassland restoration. However, there were less pronounced associations with species diversity as assessed using the Shannon and inverse Simpson indices, based on regression models. Furthermore, weed biomass was found to have significant negative effects on species richness and Pielou's evenness. The weak linear relationship between aboveground biomass and species richness could be ascribed to an increase in weed biomass. We concluded that aboveground biomass and plant species diversity could be enhanced during the initial stages of degraded grassland restoration projects and suggest that the extent of weed biomass could serve as a key indicator of the efficacy of restoration from the perspective of plant species diversity and aboveground biomass in carbon sequestration projects.
ABSTRACT
The demand for ambulatory blood pressure monitoring (ABPM) is increasing due to the global rise in cardiovascular disease patients. However, conventional ABPM methods are discontinuous and can disrupt daily activities and sleep patterns. Photoplethysmography (PPG) is gaining attention from researchers due to its simplicity, portability, affordability, and ease of signal acquisition. This paper critically examines the advancements achieved in the technology of PPG-guided noninvasive blood pressure (BP) monitoring and explores future opportunities. We have performed a literature search using the Web of Science and PubMed search engines, from January 2018 to October 2023, for PPG signal quality assessment (SQA), cuffless BP estimation using single PPG, and associations between circadian rhythm and BP. Based on this foundation, we first examine the impact of PPG signal quality on blood pressure estimation results while focusing on methods for assessing PPG signal quality. Subsequently, the methods documented for estimating cuff-free BP from PPG signals are summarized. Furthermore, the study examines how individual differences affect the accuracy of BP estimation, incorporating the factors that influence arterial blood pressure (ABP) and elucidating the impact of circadian rhythm on blood pressure. Finally, there will be a summary of the study's findings and suggestions for future research directions.
ABSTRACT
INTRODUCTION: The safety and efficacy of CRS + HIPEC combined with urinary tract resection and reconstruction are controversial. This study aims to summarize the clinicopathological features and to evaluate the safety and survival prognosis of CRS + HIPEC combined with urinary tract resection and reconstruction. METHODS: The patients who underwent urinary tract resection and reconstruction as part of CRS surgery were retrospectively selected from our disease-specific database for analysis. The clinicopathological characteristics, treatment-related variables, perioperative adverse events (AEs), and survival outcomes were studied using a descriptive approach and the K-M analysis with log-rank comparison. RESULTS: Forty-nine patients were enrolled. Perioperative serious AEs (SAEs) were observed in 11 patients (22.4%), with urinary SAEs occurring in 3 patients (6.1%). Additionally, there were 23 cases (46.8%) involving urinary adverse events (UAEs). The median overall survival (OS) in the entire cohort was 59.2 (95%CI: 42.1-76.4) months. The median OS of the UAE group and No-UAE group were 59.2 months (95%CI not reached), and 50.5 (95%CI: 11.5 to 89.6) months, respectively, with no significant difference (P = 0.475). Furthermore, there were no significant differences in OS based on the grade of UAEs or the number of UAEs (P = 0.562 and P = 0.622, respectively). CONCLUSION: The combination of CRS + HIPEC with urinary tract resection and reconstruction is associated with a high incidence of Grade I-II UAEs, which do not have an impact on OS. The safety profile of this combined technique is acceptable. However, this is a retrospective single-center single-arm analysis, with limitations of generalizability and potential selection bias. The findings need high-level validation.
Subject(s)
Cytoreduction Surgical Procedures , Hyperthermia, Induced , Hyperthermic Intraperitoneal Chemotherapy , Humans , Retrospective Studies , Female , Male , Middle Aged , Survival Rate , Prognosis , Aged , Hyperthermia, Induced/methods , Hyperthermia, Induced/adverse effects , Hyperthermia, Induced/mortality , Hyperthermic Intraperitoneal Chemotherapy/methods , Follow-Up Studies , Adult , Cytoreduction Surgical Procedures/methods , Cytoreduction Surgical Procedures/adverse effects , Cytoreduction Surgical Procedures/mortality , Combined Modality Therapy , Peritoneal Neoplasms/therapy , Peritoneal Neoplasms/mortality , Peritoneal Neoplasms/pathology , Urinary Tract/surgery , Urinary Tract/pathology , Urologic Surgical Procedures/methods , Plastic Surgery Procedures/methods , Postoperative Complications/etiologyABSTRACT
BACKGROUND: White matter hyperintensity (WMH) burden may lead to poor clinical outcomes after endovascular thrombectomy (EVT). But the relationship between WMH burden and cerebral edema (CED) is unclear. PURPOSE: To examine the association between WMH burden and CED and functional outcome in patients treated with EVT. STUDY TYPE: Retrospective. SUBJECT: 344 patients with acute anterior circulation large-vessel occlusion stroke who received EVT at two comprehensive stroke centers. Mean age was 62.6 ± 11.6 years and 100 patients (29.1%) were female. FIELD STRENGTH/SEQUENCE: 3T, including diffusion-weighted imaging and fluid-attenuated inversion recovery (FLAIR) images. ASSESSMENT: The severity of WMH was evaluated using the Fazekas scale on a FLAIR sequence before EVT. The severity of CED was assessed using CED score (three for malignant cerebral edema [MCE]) and net water uptake (NWU)/time on post-EVT cranial CT. The impact of WMH burden on MCE, NWU/time, and 3-month poor outcome (modified Rankin scale >2) after EVT were assessed. STATISTICAL TESTS: Pearson's chi-squared test, Fisher exact test, 2-tailed t test, Mann-Whitney U test, multivariable logistic regression, multivariate regression analysis, Sobel test. A P value <0.05 was considered statistically significant. RESULTS: WMH burden was not significantly associated with MCE and parenchymal hemorrhage (PH) in the whole population (P = 0.072; P = 0.714). WMH burden was significantly associated with an increased risk of MCE (OR, 1.550; 95% CI, 1.128-2.129), higher NWU/time (Coefficient, 0.132; 95% CI, 0.012-0.240), and increased risk of 3-month poor outcome (OR, 1.434; 95% CI, 1.110-1.853) in the subset of patients without PH. Moreover, the connection between WMH burden and poor outcome was partly mediated by CED in patients without PH (regression coefficient changed by 29.8%). DATA CONCLUSION: WMH burden is associated with CED, especially MCE, and poor outcome in acute ischemic stroke patients treated with EVT. The association between WMH burden and poor outcome may partly be attributed to postoperative CED. TECHNICAL EFFICACY: Stage 5.
ABSTRACT
Milk fat is an important indicator for evaluating the quality of cow's milk. In this study, we used bovine mammary epithelial cells (BMECs) to investigate the role and molecular mechanism of KLF4 in the regulation of milk fat synthesis. The results showed that KLF4 was more highly expressed in mammary tissues of high-fat cows compared with low-fat cows. KLF4 positively regulated the expression of genes related to milk fat synthesis in BMECs, increasing intracellular triglycerides content, and KLF4 promoted milk fat synthesis by activating the PI3K-AKT-mTOR signaling pathway. Furthermore, the results of animal experiments also confirmed that knockdown of KLF4 inhibited milk fat synthesis. In addition, yeast one-hybrid assays and dual-luciferase reporter gene assays confirmed that KLF4 directly targets and binds to the fatty acid synthase (FASN) promoter region to promote FASN transcription. These results demonstrate that KLF4 is a key transcription factor for milk fat synthesis in BMECs.
ABSTRACT
Inflammatory bowel disease (IBD) is difficult to cure, and formulating a dietary plan is an effective means to prevent and treat this disease. Wheat peptide contains a variety of bioactive peptides with anti-inflammatory and antioxidant functions. The results of this study showed that preventive supplementation with wheat peptide (WP) can significantly alleviate the symptoms of dextran sulfate sodium (DSS)-induced colitis in mice. WP can increase body weight, alleviate colon shortening, and reduce disease activity index (DAI) scores. In addition, WP improved intestinal microbial disorders in mice with colitis. Based on LC-MS, a total of 313 peptides were identified in WP, 4 of which were predicted to be bioactive peptides. The regulatory effects of WP and four bioactive peptides on the Keap1-Nrf2 signaling pathway were verified in Caco-2 cells. In conclusion, this study demonstrated that WP alleviates DSS-induced colitis by helping maintain gut barrier integrity and targeting the Keap1-Nrf2 axis; these results provided a rationale for adding WP to dietary strategies to prevent IBD.
Subject(s)
Colitis , Dextran Sulfate , Kelch-Like ECH-Associated Protein 1 , Mice, Inbred C57BL , NF-E2-Related Factor 2 , Peptides , Signal Transduction , Triticum , Animals , Humans , Male , Mice , Caco-2 Cells , Colitis/chemically induced , Colitis/drug therapy , Colitis/metabolism , Dextran Sulfate/adverse effects , Disease Models, Animal , Intestinal Mucosa/metabolism , Intestinal Mucosa/drug effects , Kelch-Like ECH-Associated Protein 1/drug effects , Kelch-Like ECH-Associated Protein 1/metabolism , NF-E2-Related Factor 2/drug effects , NF-E2-Related Factor 2/metabolism , Peptides/pharmacology , Signal Transduction/drug effects , Triticum/chemistryABSTRACT
BACKGROUND: Gonadotropin precisely controls mammalian reproductive activities. Systematic analysis of the mechanisms by which epigenetic modifications regulate the synthesis and secretion of gonadotropin can be useful for more precise regulation of the animal reproductive process. Previous studies have identified many differential m6A modifications in the GnRH-treated adenohypophysis. However, the molecular mechanism by which m6A modification regulates gonadotropin synthesis and secretion remains unclear. RESULTS: Herein, it was found that GnRH can promote gonadotropin synthesis and secretion by promoting the expression of FTO. Highly expressed FTO binds to Foxp2 mRNA in the nucleus, exerting a demethylation function and reducing m6A modification. After Foxp2 mRNA exits the nucleus, the lack of m6A modification prevents YTHDF3 from binding to it, resulting in increased stability and upregulation of Foxp2 mRNA expression, which activates the cAMP/PKA signaling pathway to promote gonadotropin synthesis and secretion. CONCLUSIONS: Overall, the study reveals the molecular mechanism of GnRH regulating the gonadotropin synthesis and secretion through FTO-mediated m6A modification. The results of this study allow systematic interpretation of the regulatory mechanism of gonadotropin synthesis and secretion in the pituitary at the epigenetic level and provide a theoretical basis for the application of reproductive hormones in the regulation of animal artificial reproduction.
Subject(s)
Alpha-Ketoglutarate-Dependent Dioxygenase FTO , Gonadotropin-Releasing Hormone , Animals , Alpha-Ketoglutarate-Dependent Dioxygenase FTO/metabolism , Alpha-Ketoglutarate-Dependent Dioxygenase FTO/genetics , Gonadotropin-Releasing Hormone/metabolism , Gonadotropin-Releasing Hormone/genetics , Gonadotropins/metabolism , RNA Methylation , RNA, Messenger/metabolism , RNA, Messenger/genetics , RatsABSTRACT
Nearly half of the patients undergoing endovascular treatment (EVT) do not have favorable outcomes despite successful recanalization of the occluded artery, which is also known as clinically ineffective reperfusion. We proposed a novel index-the systemic inflammatory protein index (SIPI), based on albumin, globulin, and C-reaction protein (CRP). We aimed to evaluate the relationship between inflammatory biomarkers at varying time points and the 90-day functional outcomes and investigate inflammatory biomarkers' dynamic changes during hospitalization in acute ischemic stroke (AIS) patients of anterior circulation undergoing EVT. We retrospectively recruited consecutive patients diagnosed with AIS of anterior circulation and treated with EVT from January 2018 to June 2022 in Nanfang Hospital. Albumin, globulin, and CRP were recorded on admission, 1 day, 3 days, and 7 days after EVT. An unfavorable functional outcome was defined as 90-day modified Rankin Scale (mRS) of 3-6. Albumin-to-globulin ratio (AGR), C-reactive protein-to-albumin ratio (CAR), and SIPI were calculated as follows: AGR = albumin/globulin; CAR = CRP/albumin; SIPI = CRP × globulin/albumin. A total of 238 consecutive anterior circulation AIS patients with EVT were included, among which 145 (60.9%) patients had unfavorable outcomes. After adjusting for confounding factors, admission globulin, admission AGR, 1-day AGR, 3-day albumin, 3-day CRP, 3-day CAR, 3-day SIPI, 7-day albumin, 7-day CRP, 7-day CAR, and 7-day SIPI showed an independent association with 90-day functional outcome. Of them, 3-day SIPI had the most robust discriminative ability with an area under the curve of 0.719 (CI 0.630-0.808, p < 0.001). There were differences in the dynamic change of inflammatory biomarkers between the subjects with favorable and unfavorable functional outcomes. Inflammatory biomarkers, including albumin, globulin, CRP, AGR, CAR, and SIPI, are independent predictors of 90-day unfavorable outcomes in anterior circulation AIS patients with EVT. SIPI of day 3 has the highest predictive value.
ABSTRACT
We establish a theoretical model to analyze the photoassociative spectroscopy of 85Rb 133Cs molecules in the (3)3Σ+ state. The vibrational energy, spin-spin coupling constant, and hyperfine interaction constant of the (3)3Σ+ state are determined based on nine observed vibrational levels. Consequently, the Rydberg-Klein-Rees potential energy curve of the (3)3Σ+ state is obtained and compared with the ab initial potential energy curve. Our model can be adopted to analyze the photoassociative spectroscopy of other heteronuclear alkali-metal diatomic molecules in the (3)3Σ+ state.
ABSTRACT
Deep neural networks tend to suffer from the overfitting issue when the training data are not enough. In this paper, we introduce two metrics from the intra-class distribution of correct-predicted and incorrect-predicted samples to provide a new perspective on the overfitting issue. Based on it, we propose a knowledge distillation approach without pretraining a teacher model in advance named Tolerant Self-Distillation (TSD) for alleviating the overfitting issue. It introduces an online updating memory and selectively stores the class predictions of the samples from the past iterations, making it possible to distill knowledge across the iterations. Specifically, the class predictions stored in the memory bank serve as the soft labels for supervising the samples from the same class for the current iteration in a reverse way, i.e. the correct-predicted samples are supervised with the incorrect predictions while the incorrect-predicted samples are supervised with the correct predictions. Consequently, the premature convergence issue caused by the over-confident samples would be mitigated, which helps the model to converge to a better local optimum. Extensive experimental results on several image classification benchmarks, including small-scale, large-scale, and fine-grained datasets, demonstrate the superiority of the proposed TSD.
Subject(s)
Benchmarking , Knowledge , Neural Networks, ComputerABSTRACT
Generating circularly polarized luminescence (CPL) with simultaneous high photoluminescence quantum yield (PLQY) and dissymmetry factor (glum) is difficult due to usually unmatched electric transition dipole moment (µ) and magnetic transition dipole moment (m) of materials. Herein we tackle this issue by playing a "cascade cationic insertion" trick to achieve strong CPL (with PLQY of ~100 %) in lead-free metal halides with high glum values reaching -2.3×10-2 without using any chiral inducers. Achiral solvents of hydrochloric acid (HCl) and N, N-dimethylformamide (DMF) infiltrate the crystal lattice via asymmetric hydrogen bonding, distorting the perovskite structure to induce the "intrinsic" chirality. Surprisingly, additional insertion of Cs+ cation to substitute partial (CH3)2NH2 + transforms the chiral space group to achiral but the crystal maintains chiroptical activity. Further doping of Sb3+ stimulates strong photoluminescence as a result of self-trapped excitons (STEs) formation without disturbing the crystal framework. The chiral perovskites of indium-antimony chlorides embedded on LEDs chips demonstrate promising potential as CPL emitters. Our work presents rare cases of chiroptical activity of highly luminescent perovskites from only achiral building blocks via spontaneous resolution as a result of symmetry breaking.
ABSTRACT
The precise mechanism of ferroptosis as a regulatory cell death in intestinal ischemia injury induced by vascular intestinal obstruction (Vio) remains to be elucidated. Here, we evaluated iron levels, glutathione peroxidase 4 (GPX4) and Acyl-CoA synthetase long-chain family member 4 (ACSL4) changes after intestinal ischemia injury to validate ferroptosis. As an enzyme for Fe3+ reduction to Fe2+, Ferric Chelate Reductase 1 (FRRS1) is involved in the electron transport chain and the tricarboxylic acid (TCA) cycle in mitochondria. However, whether it is involved in ferroptosis and its role in intestinal ischemia injury need to be clarified. In the present study, FRRS1 was overexpressed in vivo and in vitro. The results showed that overexpression of FRRS1 prevented ischemia-induced iron levels, reactive oxygen species (ROS) production, lipid peroxidation, inflammatory responses, and cell death. Meanwhile, FRRS1 overexpression promoted GPX4 expression and suppressed ACSL4 levels. Further studies revealed that FRRS1 overexpression inhibited the activity of large tumor suppressor 1 (LATS1) / Yes-associated protein (YAP) / transcriptional co-activator with PDZ-binding motif (TAZ), a key component of Hippo signaling. In conclusion, this study demonstrates that FRRS1 is intimately involved in the inhibition of ferroptosis and thus protection of the intestine from intestinal ischemia injury, its downstream mechanism was related to Hippo signaling. These data provide new sight for the prevention and treatment of intestinal ischemia injury.
Subject(s)
Coenzyme A Ligases , Ferroptosis , Hippo Signaling Pathway , Intestines , Mice, Inbred C57BL , Protein Serine-Threonine Kinases , Signal Transduction , Animals , Mice , Male , Coenzyme A Ligases/metabolism , Coenzyme A Ligases/genetics , Protein Serine-Threonine Kinases/metabolism , Protein Serine-Threonine Kinases/genetics , Intestines/pathology , Phospholipid Hydroperoxide Glutathione Peroxidase/metabolism , Phospholipid Hydroperoxide Glutathione Peroxidase/genetics , Ischemia/metabolism , YAP-Signaling Proteins/metabolism , Reactive Oxygen Species/metabolism , HumansABSTRACT
As a promising field, Multi-Query Image Retrieval (MQIR) aims at searching for the semantically relevant image given multiple region-specific text queries. Existing works mainly focus on a single-level similarity between image regions and text queries, which neglect the hierarchical guidance of multi-level similarities and result in incomplete alignments. Besides, the high-level semantic correlations that intrinsically connect different region-query pairs are rarely considered. To address above limitations, we propose a novel Hierarchical Matching and Reasoning Network (HMRN) for MQIR. It disentangles MQIR into three hierarchical semantic representations, which is responsible to capture fine-grained local details, contextual global scopes, and high-level inherent correlations. HMRN consists of two modules: Scalar-based Matching (SM) module and Vector-based Reasoning (VR) module. Specifically, the SM module characterizes the multi-level alignment similarity, which consists of a fine-grained local-level similarity and a context-aware global-level similarity. Afterwards, the VR module is developed to excavate the potential semantic correlations among multiple region-query pairs, which further explores the high-level reasoning similarity. Finally, these three-level similarities are aggregated into a joint similarity space to form the ultimate similarity. Extensive experiments on the benchmark dataset demonstrate that our HMRN substantially surpasses the current state-of-the-art methods. For instance, compared with the existing best method Drill-down, the metric R@1 in the last round is improved by 23.4%. Our source codes will be released at https://github.com/LZH-053/HMRN.
Subject(s)
Benchmarking , Problem Solving , Semantics , SoftwareABSTRACT
In this study, the ecological risk assessment of PAHs pollution, the existing S-T model was improved and applied to this PAHs pollution assessment in surface sediment in Lake Chaohu. The potential sources and contributions of PAHs in the surface sediment were estimated by molecular diagnostic ratio (MDR) and positive matrix factorization (PMF). The results showed that the average concentration of 16 priority PAHs in the surface sediment was 718.16 ng/g in 2009 and 334.67 ng/g in 2020. In 2020, PAHs concentration has decreased compared to 2009 and the dominant composition has changed from high- to low-molecular-weight PAHs. The estimated PAHs mass inventory of the top 2 cm surface sediment was 2712 tons in 2009 and 1263 tons in 2020. Ecosystem risk assessment by improved S-T models suggested that the overall ecosystem risk of the studied regions was acceptable.
Subject(s)
Polycyclic Aromatic Hydrocarbons , Water Pollutants, Chemical , Polycyclic Aromatic Hydrocarbons/analysis , Ecosystem , Environmental Monitoring , Lakes/analysis , Geologic Sediments , Water Pollutants, Chemical/analysis , Risk Assessment , ChinaABSTRACT
As a fundamental and challenging task in bridging language and vision domains, Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality, and its key challenge is to measure the semantic similarity across different modalities. Although significant progress has been achieved, existing approaches typically suffer from two major limitations: (1) It hurts the accuracy of the representation by directly exploiting the bottom-up attention based region-level features where each region is equally treated. (2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism. To address these limitations, we propose a Unified Semantic Enhancement Momentum Contrastive Learning (USER) method for ITR. Specifically, we delicately design two simple but effective Global representation based Semantic Enhancement (GSE) modules. One learns the global representation via the self-attention algorithm, noted as Self-Guided Enhancement (SGE) module. The other module benefits from the pre-trained CLIP module, which provides a novel scheme to exploit and transfer the knowledge from an off-the-shelf model, noted as CLIP-Guided Enhancement (CGE) module. Moreover, we incorporate the training mechanism of MoCo into ITR, in which two dynamic queues are employed to enrich and enlarge the scale of negative sample pairs. Meanwhile, a Unified Training Objective (UTO) is developed to learn from mini-batch based and dynamic queue based samples. Extensive experiments on the benchmark MSCOCO and Flickr30K datasets demonstrate the superiority of both retrieval accuracy and inference efficiency. For instance, compared with the existing best method NAAF, the metric R@1 of our USER on the MSCOCO 5K Testing set is improved by 5% and 2.4% on caption retrieval and image retrieval without any external knowledge or pre-trained model while enjoying over 60 times faster inference speed. Our source code will be released at https://github.com/zhangy0822/USER.
ABSTRACT
Conformal prediction (CP) is a learning framework controlling prediction coverage of prediction sets, which can be built on any learning algorithm for point prediction. This work proposes a learning framework named conformal loss-controlling prediction, which extends CP to the situation where the value of a loss function needs to be controlled. Different from existing works about risk-controlling prediction sets and conformal risk control with the purpose of controlling the expected values of loss functions, the proposed approach in this article focuses on the loss for any test object, which is an extension of CP from miscoverage loss to some general loss. The controlling guarantee is proved under the assumption of exchangeability of data in finite-sample cases and the framework is tested empirically for classification with a class-varying loss and statistical postprocessing of numerical weather forecasting applications, which are introduced as point-wise classification and point-wise regression problems. All theoretical analysis and experimental results confirm the effectiveness of our loss-controlling approach.
ABSTRACT
Video inpainting gains an increasing amount of attention ascribed to its wide applications in intelligent video editing. However, despite tremendous progress made in RGB video inpainting, the existing RGB-D video inpainting models are still incompetent to inpaint real-world RGB-D videos, as they simply fuse color and depth via explicit feature concatenation, neglecting the natural modality gap. Moreover, current RGB-D video inpainting datasets are synthesized with homogeneous and delusive RGB-D data, which is far from real-world application and cannot provide comprehensive evaluation. To alleviate these problems and achieve real-world RGB-D video inpainting, on one hand, we propose a Mutually-guided Color and Depth Inpainting Network (MCD-Net), where color and depth are reciprocally leveraged to inpaint each other implicitly, mitigating the modality gap and fully exploiting cross-modal association for inpainting. On the other hand, we build a Video Inpainting with Depth (VID) dataset to supply diverse and authentic RGB-D video data with various object annotation masks to enable comprehensive evaluation for RGB-D video inpainting under real-world scenes. Experimental results on the DynaFill benchmark and our collected VID dataset demonstrate our MCD-Net not only yields the state-of-the-art quantitative performance but successfully achieves high-quality RGB-D video inpainting under real-world scenes. All resources are available at https://github.com/JCATCV/MCD-Net.