Search Results for author:
Found 69 papers, 21 papers with code
Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document
To alleviate such limitations, we propose a GlObal Structure knowledgeguided relation Extraction (GOSE) framework, which captures dependencies between entity pairs in an iterative manner.
Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images.
InstructVid2Vid: Controllable Video Editing with Natural Language Instructions
To improve the consistency between adjacent frames of generated videos, we propose the Frame Difference Loss, which is incorporated during the training process.
Continual Vision-Language Representation Learning with Off-Diagonal Information
Moreover, we empirically and theoretically demonstrate how SD leads to a performance decline for CLIP on cross-modal retrieval tasks.
SkillQG: Learning to Generate Question for Reading Comprehension Assessment
We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models.
Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels
Conventional multi-label classification (MLC) methods assume that all samples are fully labeled and identically distributed.
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Based on that, we further introduce a novel Entangled cross-modal prompt approach for open-world predicate scene graph generation (Epic), where models can generalize to unseen predicates in a zero-shot manner.
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization
Though effective, prompt tuning under few-shot settings on the one hand heavily relies on a good initialization of soft prompts.
SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference
In this paper, we propose a novel dynamic early exiting combined with layer skipping for BERT inference named SmartBERT, which adds a skipping gate and an exiting operator into each layer of BERT.
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by learning the ``soft prompts'' to condition frozen pre-training models.
Structure-Aware Group Discrimination with Adaptive-View Graph Encoder: A Fast Graph Contrastive Learning Framework
Albeit having gained significant progress lately, large-scale graph representation learning remains expensive to train and deploy for two main reasons: (i) the repetitive computation of multi-hop message passing and non-linearity in graph neural networks (GNNs); (ii) the computational cost of complex pairwise contrastive learning loss.
Lformer: Text-to-Image Generation with L-shape Block Parallel Decoding
To better accelerate the generative transformers while keeping good generation quality, we propose Lformer, a semi-autoregressive text-to-image generation model.
SGL-PT: A Strong Graph Learner with Graph Prompt Tuning
And aiming for graph classification task, we unify pre-training and fine-tuning by designing a novel verbalizer-free prompting function, which reformulates the downstream task in a similar format as pretext task.
A Study on ReLU and Softmax in Transformer
This paper sheds light on the following points: 1) Softmax and ReLU use different normalization methods over elements which lead to different variances of results, and ReLU is good at dealing with a large number of key-value slots; 2) FFN and key-value memory are equivalent, and thus the Transformer can be viewed as a memory network where FFNs and self-attention networks are both key-value memories.
Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
To systematically benchmark the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention
Furthermore, we show that the hidden state dimension can be approximated by extending the Johnson-Lindenstrauss lemma, optimizing the attention in bilinear form.
Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction
Since the error rate of the incorrect sentence is usually low (e. g., 10\%), the correction model can only learn to correct on limited error tokens but trivially copy on most tokens (correct tokens), which harms the effective training of error correction.
Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning
Novel category discovery aims at adapting models trained on known categories to novel categories.
Distilling Task-specific Logical Rules from Large Pre-trained Models
Specifically, we borrow recent prompt-based language models as the knowledge expert to yield initial seed rules, and based on the formed high-quality instance pool that acts as an intermediary role, we keep teaching the expert to fit our task and learning task-specific logical rules.
Citation Trajectory Prediction via Publication Influence Representation Using Temporal Knowledge Graph
In this paper, we propose CTPIR, a new citation trajectory prediction framework that is able to represent the influence (the momentum of citation) of either new or existing publications using the history information of all their attributes.
Fine-Grained Semantically Aligned Vision-Language Pre-Training
Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks.
Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos
In this paper, we introduce a new task, named Temporal Emotion Localization in videos~(TEL), which aims to detect human emotions and localize their corresponding temporal boundaries in untrimmed videos with aligned subtitles.
BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval
In this scenario, the input image serves as an intuitive context and background for the search, while the corresponding language expressly requests new traits on how specific characteristics of the query image should be modified in order to get the intended target image.
Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning
Motivated by the industry practice of labeling data, we propose an innovative Inconsistency-based virtual aDvErsarial Active Learning (IDEAL) algorithm to further investigate SSL-AL's potential superiority and achieve mutual enhancement of AL and SSL, i. e., SSL propagates label information to unlabeled samples and provides smoothed embeddings for AL, while AL excludes samples with inconsistent predictions and considerable uncertainty for SSL.
Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile
Moreover, when handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise on a corrupted dataset.
QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance
Existing metrics for assessing question generation not only require costly human reference but also fail to take into account the input context of generation, rendering the lack of deep understanding of the relevance between the generated questions and input contexts.
RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning
To the best of our awareness, RoSA is the first work focuses on the non-aligned node-node graph contrastive learning problem.
Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning
To systematically measure the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
Feeding What You Need by Understanding What You Learned
In this paper, we argue that a deep understanding of model capabilities and data properties can help us feed a model with appropriate training data based on its learning status.
Learning To Learn by Jointly Optimizing Neural Architecture and Weights
Existing NAS-based meta-learning methods apply a two-stage strategy, i. e., first searching architectures and then re-training meta-weights on the searched architecture.
Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
The depth estimation branch is trained with RGB-D images and then used to estimate the pseudo depth maps for all unlabeled RGB images to form the paired data.
Relational Graph Learning for Grounded Video Description Generation
Such a setting can help explain the decisions of captioning models and prevents the model from hallucinating object words in its description.
Consensus Graph Representation Learning for Better Grounded Image Captioning
The contemporary visual captioning models frequently hallucinate objects that are not actually in a scene, due to the visual misclassification or over-reliance on priors that resulting in the semantic inconsistency between the visual information and the target lexical words.
Learning to Generate Visual Questions with Noisy Supervision
The task of visual question generation (VQG) aims to generate human-like neural questions from an image and potentially other side information (e. g., answer type or the answer itself).
Self-Supervised Class Incremental Learning
Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels.
Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning
This paper investigates the feasibility of federated representation learning under the constraints of communication cost and privacy protection.
On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals
It is a consensus that small models perform quite poorly under the paradigm of self-supervised contrastive learning.
Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference
Secondly, we introduce semantic coherence learning to explicitly encourage the semantic coherence of the adaptive hierarchical graph network from three hierarchies.
Revisiting Catastrophic Forgetting in Class Incremental Learning
After investigating existing strategies, we observe that there is a lack of study on how to prevent the inter-phase confusion.
Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training
In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels.
CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction
The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task.
Improving Weakly-supervised Object Localization via Causal Intervention
The recent emerged weakly supervised object localization (WSOL) methods can learn to localize an object in the image only using image-level labels.
Disentangled Motif-aware Graph Learning for Phrase Grounding
In this paper, we propose a novel graph learning framework for phrase grounding in the image.
Ranked #6 on
Phrase Grounding
on Flickr30k Entities Test
Robust Meta-learning with Noise via Eigen-Reptile
In particular, we first cast the meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit on existing examples (clean or corrupted) of an individual task at every gradient step.
Semi-Supervised Active Learning for Semi-Supervised Models: Exploit Adversarial Examples With Graph-Based Virtual Labels
Although current mainstream methods begin to combine SSL and AL (SSL-AL) to excavate the diverse expressions of unlabeled samples, these methods' fully supervised task models are still trained only with labeled data.
Differentiable Graph Optimization for Neural Architecture Search
To this end, we learn a differentiable graph neural network as a surrogate model to rank candidate architectures, which enable us to obtain gradient w. r. t the input architectures.
Ask Question with Double Hints: Visual Question Generation with Answer-awareness and Region-reference
The task of visual question generation~(VQG) aims to generate human-like questions from an image and potentially other side information (e. g. answer type or the answer itself).
Run Away From your Teacher: a New Self-Supervised Approach Solving the Puzzle of BYOL
Recently, a newly proposed self-supervised framework Bootstrap Your Own Latent (BYOL) seriously challenges the necessity of negative samples in contrastive-based learning frameworks.
Connection-Adaptive Meta-Learning
In this paper, we aim to obtain better meta-learners by co-optimizing the architecture and meta-weights simultaneously.
Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
Recently, a newly proposed self-supervised framework Bootstrap Your Own Latent (BYOL) seriously challenges the necessity of negative samples in contrastive learning frameworks.
MGD-GAN: Text-to-Pedestrian generation through Multi-Grained Discrimination
In this paper, we investigate the problem of text-to-pedestrian synthesis, which has many potential applications in art, design, and video surveillance.
Two Step Joint Model for Drug Drug Interaction Extraction
When patients need to take medicine, particularly taking more than one kind of drug simultaneously, they should be alarmed that there possibly exists drug-drug interaction.
Drug–drug Interaction Extraction
named-entity-recognition
+4
Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling
In this paper, we focus on enhancing the generalization ability of the VIST model by considering the few-shot setting.
Neural-DINF: A Neural Network based Framework for Measuring Document Influence
Measuring the scholarly impact of a document without citations is an important and challenging problem.
Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis
Infectious keratitis is the most common entities of corneal diseases, in which pathogen grows in the cornea leading to inflammation and destruction of the corneal tissues.
NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results
This challenge is based on a newly collected validation and testing image datasets, and hence, named SIDD+.
Quda: Natural Language Queries for Visual Data Analytics
To address this challenge, we present a new dataset, called Quda, that aims to help V-NLIs recognize analytic tasks from free-form natural language by training and evaluating cutting-edge multi-label classification models.
Generating Natural Language Adversarial Examples on a Large Scale with Generative Models
It can attack text classification models with a higher success rate than existing methods, and provide acceptable quality for humans in the meantime.
Grounded and Controllable Image Completion by Incorporating Lexical Semantics
Existing image completion procedure is highly subjective by considering only visual context, which may trigger unpredictable results which are plausible but not faithful to a grounded knowledge.
Deep Neural Network for Fast and Accurate Single Image Super-Resolution via Channel-Attention-based Fusion of Orientation-aware Features
Firstly, we proposed a novel Orientation-Aware feature extraction and fusion Module (OAM), which contains a mixture of 1D and 2D convolutional kernels (i. e., 5 x 1, 1 x 5, and 3 x 3) for extracting orientation-aware features.
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e. g., television) using only visual observations.
Learning Dynamic Context Augmentation for Global Entity Linking
Despite of the recent success of collective entity linking (EL) methods, these "global" inference methods may yield sub-optimal results when the "all-mention coherence" assumption breaks, and often suffer from high computational cost at the inference stage, due to the complex search space.
Ranked #5 on
Entity Disambiguation
on AIDA-CoNLL
Walking with MIND: Mental Imagery eNhanceD Embodied QA
The experimental results and further analysis prove that the agent with the MIND module is superior to its counterparts not only in EQA performance but in many other aspects such as route planning, behavioral interpretation, and the ability to generalize from a few examples.
Informative Visual Storytelling with Cross-modal Rules
To solve this problem, we propose a method to mine the cross-modal rules to help the model infer these informative concepts given certain visual input.
KCAT: A Knowledge-Constraint Typing Annotation Tool
Fine-grained Entity Typing is a tough task which suffers from noise samples extracted from distant supervision.
Posterior-regularized REINFORCE for Instance Selection in Distant Supervision
This paper provides a new way to improve the efficiency of the REINFORCE training process.
Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering
Recently, distant supervision has gained great success on Fine-grained Entity Typing (FET).
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations.
NITE: A Neural Inductive Teaching Framework for Domain Specific NER
In domain-specific NER, due to insufficient labeled training data, deep models usually fail to behave normally.
