Search Results for author:
Found 155 papers, 59 papers with code
De-Biased Court's View Generation with Causality
Court{'}s view generation is a novel but essential task for legal AI, aiming at improving the interpretability of judgment prediction results and enabling automatic legal document generation.
EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Model
In this paper, we propose the Eyecare Kit, which systematically tackles the aforementioned three key challenges with the tailored dataset, benchmark and model: First, we construct a multi-agent data engine with real-life ophthalmology data to produce Eyecare-100K, a high-quality ophthalmic visual instruction dataset.
Generalized Visual Relation Detection with Diffusion Models
To this end, we propose to model visual relations as continuous embeddings, and design diffusion models to achieve generalized VRD in a conditional generative manner, termed Diff-VRD.
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks
Recent advances in deep thinking models have demonstrated remarkable reasoning capabilities on mathematical and coding tasks.
SOYO: A Tuning-Free Approach for Video Style Morphing via Style-Adaptive Interpolation in Diffusion Models
Diffusion models have achieved remarkable progress in image and video stylization.
InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models
Advanced reasoning in large language models has achieved remarkable performance on challenging tasks, but the prevailing long-context reasoning paradigm faces critical limitations: quadratic computational scaling with sequence length, reasoning constrained by maximum context boundaries, and performance degradation beyond pre-training context windows.
Think Twice, Click Once: Enhancing GUI Grounding via Fast and Slow Systems
This structured decomposition enables systematic understanding of both interface layouts and visual relationships.
MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task
We then apply these models to enhance existing mathematical reasoning datasets by inserting detailed intermediate steps into their solution chains, creating MathFimer-expanded versions.
HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
To effectively learn the HealthGPT, we devise a comprehensive medical domain-specific comprehension and generation dataset called VL-Health.
STRIDE: Automating Reward Design, Deep Reinforcement Learning Training and Feedback Optimization in Humanoid Robotics Locomotion
Humanoid robotics presents significant challenges in artificial intelligence, requiring precise coordination and control of high-degree-of-freedom systems.
2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining
Compared to its counterparts, our video-centric textbook offers more coherent context, richer knowledge, and better image-text alignment.
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM
Finally, we meticulously create a VideoRefer-Bench to comprehensively assess the spatial-temporal understanding capability of a Video LLM, evaluating it across various aspects.
MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulation
Diffusion-based text-to-image (T2I) models have demonstrated remarkable results in global video editing tasks.
Boosting Private Domain Understanding of Efficient MLLMs: A Tuning-free, Adaptive, Universal Prompt Optimization Framework
Efficient multimodal large language models (EMLLMs), in contrast to multimodal large language models (MLLMs), reduce model size and computational costs and are often deployed on resource-constrained devices.
Meta-Reflection: A Feedback-Free Reflection Learning Framework
Despite the remarkable capabilities of large language models (LLMs) in natural language understanding and reasoning, they often display undesirable behaviors, such as generating hallucinations and unfaithful reasoning.
Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
Digital agents are increasingly employed to automate tasks in interactive digital environments such as web pages, software applications, and operating systems.
Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativeness
Instruction tuning fine-tunes pre-trained Multi-modal Large Language Models (MLLMs) to handle real-world tasks.
STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training
Video Large Language Models (Video-LLMs) have recently shown strong performance in basic video understanding tasks, such as captioning and coarse-grained question answering, but struggle with compositional reasoning that requires multi-step spatio-temporal inference across object relations, interactions, and events.
Spatially Visual Perception for End-to-End Robotic Learning
Our results demonstrate that our approach significantly boosts the success rate across diverse camera exposures, where previous models experience performance collapse.
AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea
Instruction-based image editing aims to modify specific image elements with natural language instructions.
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
We use GPT4V to bridge the gap between the reference image and the text input for the T2I model, allowing T2I models to understand image content.
GaVaMoE: Gaussian-Variational Gated Mixture of Experts for Explainable Recommendation
GaVaMoE introduces two key components: (1) a rating reconstruction module that employs Variational Autoencoder (VAE) with a Gaussian Mixture Model (GMM) to capture complex user-item collaborative preferences, serving as a pre-trained multi-gating mechanism; and (2) a set of fine-grained expert models coupled with the multi-gating mechanism for generating highly personalized explanations.
RADAR: Robust Two-stage Modality-incomplete Industrial Anomaly Detection
To address this practical challenge, we introduce a first-of-its-kind study that comprehensively investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD), to consider the imperfect learning environment in which the multimodal information may be incomplete.
Align$^2$LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation
Recent advances in Multi-modal Large Language Models (MLLMs), such as LLaVA-series models, are driven by massive machine-generated instruction-following data tuning.
TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition
Considering this, we introduce an innovative PEFT method, TeamLoRA, consisting of a collaboration and competition module for experts, and thus achieving the right balance of effectiveness and efficiency: (i) For collaboration, a novel knowledge-sharing and -organizing mechanism is devised to appropriately reduce the scale of matrix operations, thereby boosting the training and inference speed.
Ranked #196 on
Visual Question Answering
on MM-Vet
Logic Distillation: Learning from Code Function by Function for Planning and Decision-making
Subsequently, based on the function base, LD fine-tunes S-LLMs to learn the logic employed by L-LLMs in planning and decision-making.
Distilling Vision-Language Foundation Models: A Data-Free Approach via Prompt Diversification
In this paper, we discuss the extension of DFKD to Vision-Language Foundation Models without access to the billion-level image-text datasets.
IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarization
Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization.
From Easy to Hard: Learning Curricular Shape-aware Features for Robust Panoptic Scene Graph Generation
Despite remarkable progress in PSG, almost all existing methods neglect the importance of shape-aware features, which inherently focus on the contours and boundaries of objects.
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model
In light of this, we design a multi-modal self-instruct, utilizing large language models and their code capabilities to synthesize massive abstract images and visual reasoning instructions across daily scenarios.
Ask Questions with Double Hints: Visual Question Generation with Answer-awareness and Region-reference
The visual question generation (VQG) task aims to generate human-like questions from an image and potentially other side information (e. g. answer type).
Bridging Local Details and Global Context in Text-Attributed Graphs
Representation learning on text-attributed graphs (TAGs) is vital for real-world applications, as they combine semantic textual and contextual structural information.
Improving Large Models with Small models: Lower Costs and Better Performance
On the other hand, certain tasks can be broken down into multiple subtasks, some of which can be completed without powerful capabilities.
T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text
In this work, we propose a two-stage sign language production (SLP) paradigm that first encodes sign language sequences into discrete codes and then autoregressively generates sign language from text based on the learned codebook.
DuetRAG: Collaborative Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) methods augment the input of Large Language Models (LLMs) with relevant retrieved passages, reducing factual errors in knowledge-intensive tasks.
Auto-Encoding Morph-Tokens for Multimodal LLM
For multimodal LLMs, the synergy of visual comprehension (textual output) and generation (visual output) presents an ongoing challenge.
WorldGPT: Empowering LLM as Multimodal World Model
As for evaluation, we build WorldNet, a multimodal state transition prediction benchmark encompassing varied real-life scenarios.
LASER: Tuning-Free LLM-Driven Attention Control for Efficient Text-conditioned Image-to-Animation
Revolutionary advancements in text-to-image models have unlocked new dimensions for sophisticated content creation, e. g., text-conditioned image editing, allowing us to edit the diverse images that convey highly complex visual concepts according to the textual guidance.
Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales
Their ability to execute intricate compositional reasoning tasks is also constrained, culminating in a stagnation of learning progression for these models.
HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models
Recent advancements indicate that scaling up Multimodal Large Language Models (MLLMs) effectively enhances performance on downstream multimodal tasks.
Ranked #198 on
Visual Question Answering
on MM-Vet
ProSwitch: Knowledge-Guided Instruction Tuning to Switch Between Professional and Non-Professional Responses
Large Language Models (LLMs) have demonstrated efficacy in various linguistic applications, including question answering and controlled text generation.
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization
Large Language Models (LLMs) exhibit robust problem-solving capabilities for diverse tasks.
Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering
We evaluated the performance of our framework using three benchmark datasets, and the results show that our framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks, yielding F1 scores of 11. 8% and 20. 7%, respectively.
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks.
Explore Synergistic Interaction Across Frames for Interactive Video Object Segmentation
Interactive Video Object Segmentation (iVOS) is a challenging task that requires real-time human-computer interaction.
Interactive Video Object Segmentation
Semantic Segmentation
+1
Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives
Experiments conducted on a series of reasoning and translation tasks with different LLMs serve to underscore the effectiveness and generality of our strategy.
TaskBench: Benchmarking Large Language Models for Task Automation
To address this, we introduce TaskBench, a comprehensive framework to evaluate the capability of LLMs in task automation.
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and generation tasks.
Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer
Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate. This setting neglects the more practical scenario where training data are collected from multiple sources.
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
Visual programming, a modular and generalizable paradigm, integrates different modules and Python operators to solve various vision-language tasks.
Adapt Anything: Tailor Any Image Classifiers across Domains And Categories Using Text-to-Image Diffusion Models
Such a one-for-all adaptation paradigm allows us to adapt anything in the world using only one text-to-image generator as well as the corresponding unlabeled target data.
Improving Vision Anomaly Detection with the Guidance of Language Modality
To learn a more compact latent space for the vision anomaly detector, CMLE learns a correlation structure matrix from the language modality, and then the latent space of vision modality will be learned with the guidance of the matrix.
Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model
Our approach employs a pretrained T2I diffusion model to generate each video frame in an autoregressive fashion.
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
This shortcoming results in MLLMs' underperformance in comprehending demonstrative instructions consisting of multiple, interleaved, and multimodal instructions that demonstrate the required context to complete a task.
Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion
In this work, we propose a novel strategy named \textbf{Degeneration-Tuning (DT)} to shield contents of unwanted concepts from SD weights.
ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single Object Tracking
MSDeAOT efficiently propagates object masks from previous frames to the current frame using two feature scales of 16 and 8.
Improving Reference-based Distinctive Image Captioning with Contrastive Rewards
A recent DIC method proposes to generate distinctive captions by comparing the target image with a set of semantic-similar reference images, i. e., reference-based DIC (Ref-DIC).
Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow
The advancements are twofold: First, it is a code-centric agent that receives human requests and generates code as an intermediary to handle massive data, which is quite flexible for large-scale data processing tasks.
PromptNER: Prompt Locating and Typing for Named Entity Recognition
Prompt learning is a new paradigm for utilizing pre-trained language models and has achieved great success in many tasks.
Ranked #1 on
Nested Named Entity Recognition
on ACE 2004
Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images.
DiffusionNER: Boundary Diffusion for Named Entity Recognition
In this paper, we propose DiffusionNER, which formulates the named entity recognition task as a boundary-denoising diffusion process and thus generates named entities from noisy spans.
Ranked #2 on
Nested Named Entity Recognition
on GENIA
Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models
To dynamically fuse different cues, we further introduce a chain-of-thought method that prompts LLMs to generate reasonable weights for different visual cues.
InstructVid2Vid: Controllable Video Editing with Natural Language Instructions
We introduce InstructVid2Vid, an end-to-end diffusion-based methodology for video editing guided by human language instructions.
Continual Vision-Language Representation Learning with Off-Diagonal Information
Moreover, we empirically and theoretically demonstrate how SD leads to a performance decline for CLIP on cross-modal retrieval tasks.
DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition
Also, we discover that the limited context length causes the retrieval knowledge to be invisible to the model.
Multilingual Named Entity Recognition
named-entity-recognition
+4
Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels
Conventional multi-label classification (MLC) methods assume that all samples are fully labeled and identically distributed.
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence.
Automatic Machine Learning Model Selection
Model Selection
+2
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Based on that, we further introduce a novel Entangled cross-modal prompt approach for open-world predicate scene graph generation (Epic), where models can generalize to unseen predicates in a zero-shot manner.
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by learning the ``soft prompts'' to condition frozen pre-training models.
Learning Combinatorial Prompts for Universal Controllable Image Captioning
To this end, we propose a novel prompt-based framework for CIC by learning Combinatorial Prompts, dubbed as ComPro.
Lformer: Text-to-Image Generation with L-shape Block Parallel Decoding
To better accelerate the generative transformers while keeping good generation quality, we propose Lformer, a semi-autoregressive text-to-image generation model.
Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
To systematically benchmark the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
1st Place Solution for ECCV 2022 OOD-CV Challenge Image Classification Track
In the test-time training stage, we use the pre-trained model to assign noisy label for the unlabeled target data, and propose a Label-Periodically-Updated DivideMix method for noisy label learning.
1st Place Solution for ECCV 2022 OOD-CV Challenge Object Detection Track
The domain adaptation part is implemented as a Source-Free Domain Adaptation paradigm, which only uses the pre-trained model and the unlabeled target data to further optimize in a self-supervised training manner.
Unsupervised Prompt Tuning for Text-Driven Object Detection
In this paper, we delve into this problem and propose an Unsupervised Prompt Tuning framework for text-driven object detection, which is composed of two novel mean teaching mechanisms.
DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention
Furthermore, we show that the hidden state dimension can be approximated by extending the Johnson-Lindenstrauss lemma, optimizing the attention in bilinear form.
Query-based Instance Discrimination Network for Relational Triple Extraction
Joint entity and relation extraction has been a core task in the field of information extraction.
Contrastive Learning
Joint Entity and Relation Extraction
+1
Citation Trajectory Prediction via Publication Influence Representation Using Temporal Knowledge Graph
In this paper, we propose CTPIR, a new citation trajectory prediction framework that is able to represent the influence (the momentum of citation) of either new or existing publications using the history information of all their attributes.
Fine-Grained Semantically Aligned Vision-Language Pre-Training
Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks.
Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos
In this paper, we introduce a new task, named Temporal Emotion Localization in videos~(TEL), which aims to detect human emotions and localize their corresponding temporal boundaries in untrimmed videos with aligned subtitles.
BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval
In this scenario, the input image serves as an intuitive context and background for the search, while the corresponding language expressly requests new traits on how specific characteristics of the query image should be modified in order to get the intended target image.
ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022
In this report, we present the ReLER@ZJU-Alibaba submission to the Ego4D Natural Language Queries (NLQ) Challenge in CVPR 2022.
Ranked #5 on
Natural Language Queries
on Ego4D
Label Matching Semi-Supervised Object Detection
To remedy this issue, we present a novel label assignment mechanism for self-training framework, namely proposal self-assignment, which injects the proposals from student into teacher and generates accurate pseudo labels to match each proposal in the student model accordingly.
Slimmable Domain Adaptation
In this paper, we introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank, from which models of different capacities can be sampled to accommodate different accuracy-efficiency trade-offs.
Transductive CLIP with Class-Conditional Contrastive Learning
This framework can reduce the impact of noisy labels from CLIP model effectively by combining both techniques.
Learning Domain Adaptive Object Detection with Probabilistic Teacher
In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter.
Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile
Moreover, when handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise on a corrupted dataset.
Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning
To systematically measure the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
Parallel Instance Query Network for Named Entity Recognition
Each instance query predicts one entity, and by feeding all instance queries simultaneously, we can query all entities in parallel.
Ranked #1 on
Nested Named Entity Recognition
on GENIA
Chinese Named Entity Recognition
named-entity-recognition
+5
DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition
Our system wins 10 out of 13 tracks in the MultiCoNER shared task.
Multilingual Named Entity Recognition
Named Entity Recognition
+1
Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
The depth estimation branch is trained with RGB-D images and then used to estimate the pseudo depth maps for all unlabeled RGB images to form the paired data.
Learning To Learn by Jointly Optimizing Neural Architecture and Weights
Existing NAS-based meta-learning methods apply a two-stage strategy, i. e., first searching architectures and then re-training meta-weights on the searched architecture.
MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning
We propose the Multimodal relAtional Graph adversarIal inferenCe (MAGIC) framework for diverse and unpaired TextCap.
Relational Graph Learning for Grounded Video Description Generation
Such a setting can help explain the decisions of captioning models and prevents the model from hallucinating object words in its description.
Consensus Graph Representation Learning for Better Grounded Image Captioning
The contemporary visual captioning models frequently hallucinate objects that are not actually in a scene, due to the visual misclassification or over-reliance on priors that resulting in the semantic inconsistency between the visual information and the target lexical words.
Learning to Generate Visual Questions with Noisy Supervision
The task of visual question generation (VQG) aims to generate human-like neural questions from an image and potentially other side information (e. g., answer type or the answer itself).
Self-Supervised Class Incremental Learning
Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels.
Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning
This paper investigates the feasibility of federated representation learning under the constraints of communication cost and privacy protection.
Natural Language Video Localization with Learnable Moment Proposals
Given an untrimmed video and a natural language query, Natural Language Video Localization (NLVL) aims to identify the video moment described by the query.
On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals
It is a consensus that small models perform quite poorly under the paradigm of self-supervised contrastive learning.
Revisiting Catastrophic Forgetting in Class Incremental Learning
After investigating existing strategies, we observe that there is a lack of study on how to prevent the inter-phase confusion.
Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference
Secondly, we introduce semantic coherence learning to explicitly encourage the semantic coherence of the adaptive hierarchical graph network from three hierarchies.
CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction
The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task.
Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training
In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels.
Deep Learning for Weakly-Supervised Object Detection and Object Localization: A Survey
With the success of deep neural networks in object detection, both WSOD and WSOL have received unprecedented attention.
A Sequence-to-Set Network for Nested Named Entity Recognition
We utilize a non-autoregressive decoder to predict the final set of entities in one pass, in which we are able to capture dependencies between entities.
Ranked #6 on
Nested Named Entity Recognition
on ACE 2005
VL-NMS: Breaking Proposal Bottlenecks in Two-Stage Visual-Language Matching
In this paper, we argue that these methods overlook an obvious \emph{mismatch} between the roles of proposals in the two stages: they generate proposals solely based on the detection confidence (i. e., query-agnostic), hoping that the proposals contain all instances mentioned in the text query (i. e., query-aware).
Disentangled Motif-aware Graph Learning for Phrase Grounding
In this paper, we propose a novel graph learning framework for phrase grounding in the image.
Ranked #6 on
Phrase Grounding
on Flickr30k Entities Test
Self-Supervised Noisy Label Learning for Source-Free Unsupervised Domain Adaptation
Usually, the given source domain pre-trained model is expected to optimize with only unlabeled target data, which is termed as source-free unsupervised domain adaptation.
Box Re-Ranking: Unsupervised False Positive Suppression for Domain Adaptive Pedestrian Detection
Therefore, it yields our attention to suppress false positive in each target domain in an unsupervised way.
Differentiable Graph Optimization for Neural Architecture Search
To this end, we learn a differentiable graph neural network as a surrogate model to rank candidate architectures, which enable us to obtain gradient w. r. t the input architectures.
Run Away From your Teacher: a New Self-Supervised Approach Solving the Puzzle of BYOL
Recently, a newly proposed self-supervised framework Bootstrap Your Own Latent (BYOL) seriously challenges the necessity of negative samples in contrastive-based learning frameworks.
Connection-Adaptive Meta-Learning
In this paper, we aim to obtain better meta-learners by co-optimizing the architecture and meta-weights simultaneously.
Robust Meta-learning with Noise via Eigen-Reptile
In particular, we first cast the meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit on existing examples (clean or corrupted) of an individual task at every gradient step.
Semi-Supervised Active Learning for Semi-Supervised Models: Exploit Adversarial Examples With Graph-Based Virtual Labels
Although current mainstream methods begin to combine SSL and AL (SSL-AL) to excavate the diverse expressions of unlabeled samples, these methods' fully supervised task models are still trained only with labeled data.
Ask Question with Double Hints: Visual Question Generation with Answer-awareness and Region-reference
The task of visual question generation~(VQG) aims to generate human-like questions from an image and potentially other side information (e. g. answer type or the answer itself).
A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data
However, it is difficult to evaluate the quality of pseudo labels since no labels are available in target domain.
Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
Recently, a newly proposed self-supervised framework Bootstrap Your Own Latent (BYOL) seriously challenges the necessity of negative samples in contrastive learning frameworks.
Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
The conventional solution to this task is to minimize the discrepancy between source and target to enable effective knowledge transfer.
Ranked #27 on
Synthetic-to-Real Translation
on SYNTHIA-to-Cityscapes
Federated Unsupervised Representation Learning
FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces.
Two Step Joint Model for Drug Drug Interaction Extraction
When patients need to take medicine, particularly taking more than one kind of drug simultaneously, they should be alarmed that there possibly exists drug-drug interaction.
Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling
In this paper, we focus on enhancing the generalization ability of the VIST model by considering the few-shot setting.
Neural-DINF: A Neural Network based Framework for Measuring Document Influence
Measuring the scholarly impact of a document without citations is an important and challenging problem.
Learning Decomposed Representation for Counterfactual Inference
The fundamental problem in treatment effect estimation from observational data is confounder identification and balancing.
Stable Prediction via Leveraging Seed Variable
By assuming the relationships between causal variables and response variable are invariant across data, to address this problem, we propose a conditional independence test based algorithm to separate those causal variables with a seed variable as priori, and adopt them for stable prediction.
Balance-Subsampled Stable Prediction
However, this assumption is often violated in practice because the sample selection bias may induce the distribution shift from training data to test data.
Counterfactual Samples Synthesizing for Robust Visual Question Answering
To reduce the language biases, several recent works introduce an auxiliary question-only model to regularize the training of targeted VQA model, and achieve dominating performance on VQA-CP.
Ranked #1 on
Visual Question Answering (VQA)
on VQA-CP
(using extra training data)
Bi-Decoder Augmented Network for Neural Machine Translation
Neural Machine Translation (NMT) has become a popular technology in recent years, and the encoder-decoder framework is the mainstream among all the methods.
Deep Neural Network for Fast and Accurate Single Image Super-Resolution via Channel-Attention-based Fusion of Orientation-aware Features
Firstly, we proposed a novel Orientation-Aware feature extraction and fusion Module (OAM), which contains a mixture of 1D and 2D convolutional kernels (i. e., 5 x 1, 1 x 5, and 3 x 3) for extracting orientation-aware features.
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e. g., television) using only visual observations.
Time2Graph: Revisiting Time Series Modeling with Dynamic Shapelets
Time series modeling has attracted extensive research efforts; however, achieving both reliable efficiency and interpretability from a unified model still remains a challenging problem.
Video Dialog via Progressive Inference and Cross-Transformer
Video dialog is a new and challenging task, which requires the agent to answer questions combining video information with dialog history.
Learning Dynamic Context Augmentation for Global Entity Linking
Despite of the recent success of collective entity linking (EL) methods, these "global" inference methods may yield sub-optimal results when the "all-mention coherence" assumption breaks, and often suffer from high computational cost at the inference stage, due to the complex search space.
Ranked #5 on
Entity Disambiguation
on AIDA-CoNLL
Walking with MIND: Mental Imagery eNhanceD Embodied QA
The experimental results and further analysis prove that the agent with the MIND module is superior to its counterparts not only in EQA performance but in many other aspects such as route planning, behavioral interpretation, and the ability to generalize from a few examples.
Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference
We observe that people usually use some discourse markers such as "so" or "but" to represent the logical relationship between two sentences.
Ranked #16 on
Natural Language Inference
on SNLI
MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models
In this paper, we transfer knowledge learned from machine comprehension to the sequence-to-sequence tasks to deepen the understanding of the text.
Informative Visual Storytelling with Cross-modal Rules
To solve this problem, we propose a method to mine the cross-modal rules to help the model infer these informative concepts given certain visual input.
Ranked #11 on
Visual Storytelling
on VIST
Weak Supervision Enhanced Generative Network for Question Generation
More specifically, we devise a discriminator, Relation Guider, to capture the relations between the whole passage and the associated answer and then the Multi-Interaction mechanism is deployed to transfer the knowledge dynamically for our question generation system.
KCAT: A Knowledge-Constraint Typing Annotation Tool
Fine-grained Entity Typing is a tough task which suffers from noise samples extracted from distant supervision.
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
It is both crucial and natural to extend this research direction to the video domain for video question answering (VideoQA).
Ranked #31 on
Video Question Answering
on ActivityNet-QA
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
Posterior-regularized REINFORCE for Instance Selection in Distant Supervision
This paper provides a new way to improve the efficiency of the REINFORCE training process.
Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering
Recently, distant supervision has gained great success on Fine-grained Entity Typing (FET).
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations.
Representation Learning for Scale-free Networks
Existing network embedding works primarily focus on preserving the microscopic structure, such as the first- and second-order proximity of vertexes, while the macroscopic scale-free property is largely ignored.
NITE: A Neural Inductive Teaching Framework for Domain Specific NER
In domain-specific NER, due to insufficient labeled training data, deep models usually fail to behave normally.
Deeply-Learned Part-Aligned Representations for Person Re-Identification
In this paper, we address the problem of person re-identification, which refers to associating the persons captured from different cameras.
Ranked #112 on
Person Re-Identification
on Market-1501
Video Question Answering via Attribute-Augmented Attention Network Learning
Video Question Answering is a challenging problem in visual information retrieval, which provides the answer to the referenced video content according to the question.
Zero-Shot Recognition using Dual Visual-Semantic Mapping Paths
This mapping is learned on training data of seen classes and is expected to have transfer ability to unseen classes.
Task-driven Visual Saliency and Attention-based Visual Question Answering
Visual question answering (VQA) has witnessed great progress since May, 2015 as a classic problem unifying visual and textual data into a system.
Deep Learning Driven Visual Path Prediction from a Single Image
The highly effective visual representation and deep context models ensure that our framework makes a deep semantic understanding of the scene and motion pattern, consequently improving the performance of the visual path prediction task.
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
In this paper, we propose a new approach, namely Hierarchical Recurrent Neural Encoder (HRNE), to exploit temporal information of videos.
DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
A key problem in salient object detection is how to effectively model the semantic properties of salient objects in a data-driven manner.
Online Metric-Weighted Linear Representations for Robust Visual Tracking
Object identification results for an entire video sequence are achieved by systematically combining the tracking information and visual recognition at each frame.
Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking
As an important and challenging problem in computer vision and graphics, keypoint-based object tracking is typically formulated in a spatio-temporal statistical learning framework.
