Search Results for author:
Found 86 papers, 49 papers with code
MIRTT: Learning Multimodal Interaction Representations from Trilinear Transformers for Visual Question Answering
On the other hand, trilinear models such as the CTI model efficiently utilize the inter-modality information between answers, questions, and images, while ignoring intra-modality information.
Sparse Adversarial Attack via Perturbation Factorization
Based on this factorization, we formulate the sparse attack problem as a mixed integer programming (MIP) to jointly optimize the binary selection factors and continuous perturbation magnitudes of all pixels, with a cardinality constraint on selection factors to explicitly control the degree of sparsity.
Prior Bilinear Based Models for Knowledge Graph Completion
Bilinear based models are powerful and widely used approaches for Knowledge Graphs Completion (KGC).
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Large Language Models (LLMs) excel in various tasks, but they rely on carefully crafted prompts that often demand substantial human effort.
TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild
We release our dataset, model, and demo to foster future research in the area of multimodal instruction following.
ToonTalker: Cross-Domain Face Reenactment
Moreover, since no paired data is provided, we propose a novel cross-domain training scheme using data from two domains with the designed analogy constraint.
AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years.
NewsDialogues: Towards Proactive News Grounded Conversation
In this paper, we propose a novel task, Proactive News Grounded Conversation, in which a dialogue system can proactively lead the conversation based on some key topics of the news.
Global and Local Semantic Completion Learning for Vision-Language Pre-training
Therefore, in this paper, we propose a novel Global and Local Semantic Completion Learning (GLSCL) task to facilitate global-local alignment and local-local alignment simultaneously.
D2Match: Leveraging Deep Learning and Degeneracy for Subgraph Matching
Subgraph matching is a fundamental building block for graph-based applications and is challenging due to its high-order combinatorial nature.
Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution
These elaborated designs enable our model to generate portraits with robust multi-view semantic consistency, eliminating the need for optimization-based methods.
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
To address the DoT problem, we propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution.
TaleCrafter: Interactive Story Visualization with Multiple Characters
Accurate Story visualization requires several necessary elements, such as identity consistency across frames, the alignment between plain text and visual content, and a reasonable layout of objects in images.
SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation
To address this issue, we propose Semantic-assisted Object Cluster (SOC), which aggregates video content and textual guidance for unified temporal modeling and cross-modal alignment.
Ranked #1 on
Referring Video Object Segmentation
on Refer-YouTube-VOS
(using extra training data)
Referring Expression Segmentation
Referring Video Object Segmentation
+2
Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling
Recently, diffusion models have demonstrated a remarkable ability to solve inverse problems in an unsupervised manner.
Question Answering as Programming for Solving Time-Sensitive Questions
In this work we try to apply Large Language Models (LLMs) to reframe the Question Answering task as Programming (QAaP).
MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Prediction
Generative methods greatly promote aspect-based sentiment analysis via generating a sequence of sentiment elements in a specified format.
Ranked #1 on
Aspect-Based Sentiment Analysis (ABSA)
on ACOS
Aspect Category Detection
Aspect-Category-Opinion-Sentiment Quadruple Extraction
+9
Recouple Event Field via Probabilistic Bias for Event Extraction
Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs).
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Unlike these models, humans typically utilize external tools to cross-check and refine their initial content, like using a search engine for fact-checking, or a code interpreter for debugging.
Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment
Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) without relying on pristine-quality image information.
Meta-Auxiliary Network for 3D GAN Inversion
In the first stage, we invert the input image to an editable latent code using off-the-shelf inversion techniques.
Weight-Inherited Distillation for Task-Agnostic BERT Compression
Knowledge Distillation (KD) is a predominant approach for BERT compression.
Feature Expansion for Graph Neural Networks
Graph neural networks aim to learn representations for graph-structured data and show impressive performance, particularly in node classification.
Exploring Human-Like Translation Strategy with Large Language Models
Large language models (LLMs) have demonstrated impressive capabilities in general scenarios, exhibiting a level of aptitude that approaches, in some aspects even surpasses, human-level intelligence.
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Distilling high-accuracy Graph Neural Networks~(GNNs) to low-latency multilayer perceptrons~(MLPs) on graph tasks has become a hot research topic.
Global Knowledge Calibration for Fast Open-Vocabulary Segmentation
Recent advancements in pre-trained vision-language models, such as CLIP, have enabled the segmentation of arbitrary concepts solely from textual inputs, a process commonly referred to as open-vocabulary semantic segmentation (OVS).
Knowledge Distillation
Open Vocabulary Semantic Segmentation
+3
SynGen: A Syntactic Plug-and-play Module for Generative Aspect-based Sentiment Analysis
To the best of our knowledge, we are the first one to introduce syntactic information to generative ABSA frameworks.
Generalizable Black-Box Adversarial Attack with Meta Learning
In the scenario of black-box adversarial attack, the target model's parameters are unknown, and the attacker aims to find a successful adversarial perturbation based on query feedback under a query budget.
RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition
A frozen CLIP text encoder is introduced in the text flow, and a semantic-enhanced module is used to enhance text features.
GLeaD: Improving GANs with A Generator-Leading Task
Generative adversarial network (GAN) is formulated as a two-player game between a generator (G) and a discriminator (D), where D is asked to differentiate whether an image comes from real data or is produced by G. Under such a formulation, D plays as the rule maker and hence tends to dominate the competition.
3D GAN Inversion with Facial Symmetry Prior
It is natural to associate 3D GANs with GAN inversion methods to project a real image into the generator's latent space, allowing free-view consistent synthesis and editing, referred as 3D GAN inversion.
Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning
Cross-modal alignment is essential for vision-language pre-training (VLP) models to learn the correct corresponding information across different modalities.
Modeling Fine-grained Information via Knowledge-aware Hierarchical Graph for Zero-shot Entity Retrieval
We extract the knowledge units from the corresponding context and then construct a mention/entity centralized graph.
Solving Math Word Problems via Cooperative Reasoning induced Language Models
This inspires us to develop a cooperative reasoning-induced PLM for solving MWPs, called Cooperative Reasoning (CoRe), resulting in a human-like reasoning architecture with system 1 as the generator and system 2 as the verifier.
Ranked #32 on
Arithmetic Reasoning
on GSM8K
MCSCSet: A Specialist-annotated Dataset for Medical-domain Chinese Spelling Correction
In this work, we define the task of Medical-domain Chinese Spelling Correction and propose MCSCSet, a large scale specialist-annotated dataset that contains about 200k samples.
Optical Character Recognition
Optical Character Recognition (OCR)
+1
Improving Your Graph Neural Networks: A High-Frequency Booster
Graph neural networks (GNNs) hold the promise of learning efficient representations of graph-structured data, and one of its most important applications is semi-supervised node classification.
Global Spectral Filter Memory Network for Video Object Segmentation
Besides, we empirically find low frequency feature should be enhanced in encoder (backbone) while high frequency for decoder (segmentation head).
Semantic Segmentation
Semi-Supervised Video Object Segmentation
+1
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model
Multimodal semantic understanding often has to deal with uncertainty, which means the obtained messages tend to refer to multiple targets.
Towards Real-World Video Deblurring by Exploring Blur Formation Process
We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.
Modelling Latent Dynamics of StyleGAN using Neural ODEs
The entire sequence is seen as discrete-time observations of a continuous trajectory of the initial latent code, by considering each latent code as a moving particle and the latent space as a high-dimensional dynamic system.
Rethinking Alignment in Video Super-Resolution Transformers
In this paper, we rethink the role of alignment in VSR Transformers and make several counter-intuitive observations.
Ranked #1 on
Video Super-Resolution
on Vid4 - 4x upscaling
Learning Quality-aware Dynamic Memory for Video Object Segmentation
However, they mainly focus on better matching between the current frame and the memory frames without explicitly paying attention to the quality of the memory.
Ranked #9 on
Semi-Supervised Video Object Segmentation
on DAVIS 2017 (test-dev)
(using extra training data)
Semantic Segmentation
Semi-Supervised Video Object Segmentation
+1
MORE: A Metric Learning Based Framework for Open-domain Relation Extraction
To address these problems, in this work, we propose a novel learning framework named MORE (Metric learning-based Open Relation Extraction).
Learning Adaptive Warping for Real-World Rolling Shutter Correction
This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video.
EmpHi: Generating Empathetic Responses with Human-like Intents
To address the bias of the empathetic intents distribution between empathetic dialogue models and humans, we propose a novel model to generate empathetic responses with human-consistent empathetic intents, EmpHi for short.
Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network
Image quality assessment (IQA) algorithm aims to quantify the human perception of image quality.
Ranked #1 on
Image Quality Assessment
on MSU FR VQA Database
NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results
This challenge includes three tracks.
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment
No-Reference Image Quality Assessment (NR-IQA) aims to assess the perceptual quality of images in accordance with human subjective perception.
VDTR: Video Deblurring with Transformer
For multi-frame temporal modeling, we adapt Transformer to fuse multiple spatial features efficiently.
High-fidelity GAN Inversion with Padding Space
In this work, we propose to involve the padding space of the generator to complement the latent space with spatial information.
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
Our framework elevates the resolution of the synthesized talking face to 1024*1024 for the first time, even though the training dataset has a lower resolution.
Context Enhanced Short Text Matching using Clickthrough Data
The short text matching task employs a model to determine whether two short texts have the same semantic meaning or intent.
STaR: Knowledge Graph Embedding by Scaling, Translation and Rotation
Therefore, we propose a corresponding bilinear model Scaling Translation and Rotation (STaR) consisting of the above two parts.
Accelerating Neural Network Optimization Through an Automated Control Theory Lens
We conduct experiments both from a control theory lens through a phase locus verification and from a network training lens on several models, including CNNs, Transformers, MLPs, and on benchmark datasets.
Adder Attention for Vision Transformer
With the new operation, vision transformers constructed using additions can also provide powerful feature representations.
Identity-guided Face Generation with Multi-modal Contour Conditions
Concretely, we propose a novel dual-encoder architecture, in which an identity encoder extracts the identity-related feature, accompanied by a main encoder to obtain the rough contour information and further fuse all the information together.
Guiding Topic Flows in the Generative Chatbot by Enhancing the ConceptNet with the Conversation Corpora
Further analysis individually proves the effectiveness of the enhanced concept graph and the Edge-Transformer architecture.
Real-time Human-Centric Segmentation for Complex Video Scenes
To alleviate this problem, we propose a mechanism named Inner Center Sampling to improve the accuracy of instance segmentation.
PoseDet: Fast Multi-Person Pose Estimation Using Pose Embedding
This simple framework achieves an unprecedented speed and a competitive accuracy on the COCO benchmark compared with state-of-the-art methods.
Augmenting Anchors by the Detector Itself
Furthermore, AADI is a learning-based anchor augmentation method, but it does not add any parameters or hyper-parameters, which is beneficial for research and downstream tasks.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
Region-Adaptive Deformable Network for Image Quality Assessment
Image quality assessment (IQA) aims to assess the perceptual quality of images.
Coarse-to-Fine Searching for Efficient Generative Adversarial Networks
This paper studies the neural architecture search (NAS) problem for developing efficient generator networks.
Towards Open-World Text-Guided Face Image Generation and Manipulation
To be specific, we propose a brand new paradigm of text-guided image generation and manipulation based on the superior characteristics of a pretrained GAN model.
Ranked #5 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation
The ambiguity naturally leads to the issue of \emph{implicit multi-label}, motivating the need for diverse predictions.
AACP: Model Compression by Accurate and Automatic Channel Pruning
Firstly, AACP represents the structure of a model as a structure vector and introduces a pruning step vector to control the compressing granularity of each layer.
Augmenting Proposals by the Detector Itself
Lacking enough high quality proposals for RoI box head has impeded two-stage and multi-stage object detectors for a long time, and many previous works try to solve it via improving RPN's performance or manually generating proposals from ground truth.
GAN Inversion: A Survey
GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator.
TediGAN: Text-Guided Diverse Face Image Generation and Manipulation
In this work, we propose TediGAN, a novel framework for multi-modal image generation and manipulation with textual descriptions.
Ranked #6 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
DT-QDC: A Dataset for Question Comprehension in Online Test
With the transformation of education from the traditional classroom environment to online education and assessment, it is more and more important to accurately assess the difficulty of questions than ever.
Controllable Continuous Gaze Redirection
The encoder maps images into a well-disentangled and hierarchically-organized latent space.
Cognitive Representation Learning of Self-Media Online Article Quality
The automatic quality assessment of self-media online articles is an urgent and new issue, which is of great value to the online recommendation and search.
HGCN4MeSH: Hybrid Graph Convolution Network for MeSH Indexing
Recently deep learning has been used in Medical subject headings (MeSH) indexing to reduce the time and monetary cost by manual annotation, including DeepMeSH, TextCNN, etc.
Towards Multimodal Response Generation with Exemplar Augmentation and Curriculum Optimization
Furthermore, to facilitate the convergence of Gaussian mixture prior and posterior distributions, we devise a curriculum optimization strategy to progressively train the model under multiple training criteria from easy to hard.
HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation
The sequence-to-sequence (Seq2Seq) model generates target words iteratively given the previously observed words during decoding process, which results in the loss of the holistic semantics in the target response and the complete semantic relationship between responses and dialogue histories.
Cooperative Semantic Segmentation and Image Restoration in Adverse Environmental Conditions
Most state-of-the-art semantic segmentation approaches only achieve high accuracy in good conditions.
Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
Image-to-image translation has drawn great attention during the past few years.
Domain Fingerprints for No-reference Image Quality Assessment
Human fingerprints are detailed and nearly unique markers of human identity.
Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
Image generation has received increasing attention because of its wide application in security and entertainment.
Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cube
Witnessed the development of deep learning, increasing number of studies try to build computer aided diagnosis systems for 3D volumetric medical data.
A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification
Deep learning approaches for sentiment classification do not fully exploit sentiment linguistic knowledge.
Ranked #4 on
Sentiment Analysis
on MR
Multi-glance Reading Model for Text Understanding
Based on the multi-glance mechanism, we design two types of recurrent neural network models for repeated reading: Glance Cell Model (GCM) and Glance Gate Model (GGM).
Faster Spatially Regularized Correlation Filters for Visual Tracking
Our approach achieves equivalent performance to the baseline tracker SRDCF on all three datasets.
