Search Results for author:
Found 72 papers, 25 papers with code
ForkGAN: Seeing into the Rainy Night
We present a ForkGAN for task-agnostic image translation that can boost multiple vision tasks in adverse weather conditions.
Mitigating the Inconsistency Between Word Saliency and Model Confidence with Pathological Contrastive Training
We show that the pathological inconsistency is caused by the representation collapse issue, which means that the representation of the sentences with tokens in different saliency reduced is somehow collapsed, and thus the important words cannot be distinguished from unimportant words in terms of model confidence changing.
PARSE: An Efficient Search Method for Black-box Adversarial Text Attacks
Previous works on word-level attacks widely use word importance ranking (WIR) methods and complex search methods, including greedy search and heuristic algorithms, to find optimal substitutions.
Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer
The complex traffic environment and various weather conditions make the collection of LiDAR data expensive and challenging.
Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey
Compared to traditional sentiment analysis, which only considers text, multimodal sentiment analysis needs to consider emotional signals from multimodal sources simultaneously and is therefore more consistent with the way how humans process sentiment in real-world scenarios.
Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning
The integration of generative Large Language Models (LLMs) into various applications, including the legal domain, has been accelerated by their expansive and versatile nature.
PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM
Layout generation is the keystone in achieving automated graphic design, requiring arranging the position and size of various multi-modal design elements in a visually pleasing and constraint-following manner.
Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction
Inferring 3D human motion is fundamental in many applications, including understanding human activity and analyzing one's intention.
Large Language Models Meet NLP: A Survey
While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored.
Align-Free Multi-Plane Phase Retrieval
The multi-plane phase retrieval method provides a budget-friendly and effective way to perform phase imaging, yet it often encounters alignment challenges due to shifts along the optical axis in experiments.
Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study
In particular, we first explore the ways of transforming structured tabular data into sequential text prompts, as to feed them into LLMs and analyze which table content contributes most to the NL2Vis.
Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation
We argue that these factual reasoning-based explanations cannot answer critical what-if questions: What would happen to the GNN's decision if we were to alter the code graph into alternative structures?
DO3D: Self-supervised Learning of Decomposed Object-aware 3D Motion and Depth from Monocular Videos
Our system contains a depth estimation module to predict depth, and a new decomposed object-wise 3D motion (DO3D) estimation module to predict ego-motion and 3D object motion.
Machine learning classification of power converter control mode
To ensure the proper functioning of the current and future electrical grid, it is necessary for Transmission System Operators (TSOs) to verify that energy providers comply with the grid code and specifications provided by TSOs.
GUESS:GradUally Enriching SyntheSis for Text-Driven Human Motion Generation
The whole text-driven human motion synthesis problem is then divided into multiple abstraction levels and solved with a multi-stage generation framework with a cascaded latent diffusion model: an initial generator first generates the coarsest human motion guess from a given text description; then, a series of successive generators gradually enrich the motion details based on the textual description and the previous synthesized results.
Ranked #7 on
Motion Synthesis
on KIT Motion-Language
Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model
Besides, we introduce a multi-denoiser framework for the advanced diffusion model to ease the learning of high-dimensional model and fully explore the generative potential of the diffusion model.
A Novel 3D Non-stationary Localization-assisted ISAC Channel Model
Integrated sensing and communication (ISAC) has attracted wide attention as an emerging application scenario for the sixth generation (6G) wireless communication system.
An Early Evaluation of GPT-4V(ision)
In this paper, we evaluate different abilities of GPT-4V including visual understanding, language understanding, visual puzzle solving, and understanding of other modalities such as depth, thermal, video, and audio.
Towards Open-World Co-Salient Object Detection with Generative Uncertainty-aware Group Selective Exchange-Masking
To simultaneously consider the uncertainty introduced by irrelevant images and the consensus features of the remaining relevant images in the group, we designed a latent variable generator branch and CoSOD transformer branch.
Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video
Therefore, directly learning a mapping function from speech to the entire head image is prone to ambiguity, particularly when using a short video for training.
Community-Based Hierarchical Positive-Unlabeled (PU) Model Fusion for Chronic Disease Prediction
Positive-Unlabeled (PU) Learning is a challenge presented by binary classification problems where there is an abundance of unlabeled data along with a small number of positive data instances, which can be used to address chronic disease screening problem.
Integrating Offline Reinforcement Learning with Transformers for Sequential Recommendation
To this end, we train a farsighted recommender by using an offline RL algorithm with the policy network in our model architecture that has been initialized from a pre-trained transformer model.
NOFA: NeRF-based One-shot Facial Avatar Reconstruction
In this work, we propose a one-shot 3D facial avatar reconstruction framework that only requires a single source image to reconstruct a high-fidelity 3D facial avatar.
Improving Cross-Task Generalization with Step-by-Step Instructions
Instruction tuning has been shown to be able to improve cross-task generalization of language models.
Multi-object Video Generation from Single Frame Layouts
In this paper, we study video synthesis with emphasis on simplifying the generation conditions.
Decompose More and Aggregate Better: Two Closer Looks at Frequency Representation Learning for Human Motion Prediction
Encouraged by the effectiveness of encoding temporal dynamics within the frequency domain, recent human motion prediction systems prefer to first convert the motion representation from the original pose space into the frequency space.
Face Clustering via Graph Convolutional Networks with Confidence Edges
Experiments show that our method outperforms existing methods on the face and person datasets to achieve state-of-the-art.
Co-Salient Object Detection With Uncertainty-Aware Group Exchange-Masking
To address this issue, this paper presents a group exchange-masking (GEM) strategy for robust CoSOD model learning.
Instance-level Heterogeneous Domain Adaptation for Limited-labeled Sketch-to-Photo Retrieval
Therefore, previous works pre-train their models on rich-labeled photo retrieval data (i. e., source domain) and then fine-tune them on the limited-labeled sketch-to-photo retrieval data (i. e., target domain).
Ranked #1 on
Image Retrieval
on PKU-Reid
Exploring Graph-aware Multi-View Fusion for Rumor Detection on Social Media
Automatic detecting rumors on social media has become a challenging task.
Specialized Re-Ranking: A Novel Retrieval-Verification Framework for Cloth Changing Person Re-Identification
Cloth changing person re-identification(Re-ID) can work under more complicated scenarios with higher security than normal Re-ID and biometric techniques and is therefore extremely valuable in applications.
An Efficient End-to-End Transformer with Progressive Tri-modal Attention for Multi-modal Emotion Recognition
At the low-level, we propose the progressive tri-modal attention, which can model the tri-modal feature interactions by adopting a two-pass strategy and can further leverage such interactions to significantly reduce the computation and memory complexity through reducing the input token length.
A Constrained Deformable Convolutional Network for Efficient Single Image Dynamic Scene Blind Deblurring with Spatially-Variant Motion Blur Kernels Estimation
Most existing deep-learning-based single image dynamic scene blind deblurring (SIDSBD) methods usually design deep networks to directly remove the spatially-variant motion blurs from one inputted motion blurred image, without blur kernels estimation.
A Weighted Random Forest Based PositioningAlgorithm for 6G Indoor Communications
Due to the indoor none-line-of-sight (NLoS) propagation and multi-access interference (MAI), it is a great challenge to achieve centimeter-level positioning accuracy in indoor scenarios.
MACSA: A Multimodal Aspect-Category Sentiment Analysis Dataset with Multimodal Fine-grained Aligned Annotations
In this paper, we propose a new dataset, the Multimodal Aspect-Category Sentiment Analysis (MACSA) dataset, which contains more than 21K text-image pairs.
UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection
Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
Ranked #4 on
Highlight Detection
on YouTube Highlights
Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors
Through further analysis of the ASR outputs, we find that in some cases the sentiment words, the key sentiment elements in the textual modality, are recognized as other words, which makes the sentiment of the text change and hurts the performance of multimodal sentiment models directly.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
a novel attention-based network for fast salient object detection
In the current salient object detection network, the most popular method is using U-shape structure.
ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval
The huge domain gap between sketches and photos and the highly abstract sketch representations pose challenges for sketch-based image retrieval (\underline{SBIR}).
Explore before Moving: A Feasible Path Estimation and Memory Recalling Framework for Embodied Navigation
PEMR includes a "looking ahead" process, \textit{i. e.} a visual feature extractor module that estimates feasible paths for gathering 3D navigational information, which is mimicking the human sense of direction.
Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
Visual recognition is currently one of the most important and active research areas in computer vision, pattern recognition, and even the general field of artificial intelligence.
Uniformity in Heterogeneity:Diving Deep into Count Interval Partition for Crowd Counting
Therefore, we propose a novel count interval partition criterion called Uniform Error Partition (UEP), which always keeps the expected counting error contributions equal for all intervals to minimize the prediction risk.
Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework
In this paper, we propose a purely point-based framework for joint crowd counting and individual localization.
Ranked #6 on
Crowd Counting
on ShanghaiTech A
SiamRCR: Reciprocal Classification and Regression for Visual Object Tracking
In addition, we add a localization branch to predict the localization accuracy, so that it can work as the replacement of the regression assistance link during inference.
Uniformity in Heterogeneity: Diving Deep Into Count Interval Partition for Crowd Counting
Therefore, we propose a novel count interval partition criterion called Uniform Error Partition (UEP), which always keeps the expected counting error contributions equal for all intervals to minimize the prediction risk.
Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework
In this paper, we propose a purely point-based framework for joint crowd counting and individual localization.
Detection of magnetic gap in the topological surface states of MnBi2Te4
An essential ingredient to realize these quantum states is the magnetic gap in the topological surface states induced by the out-of-plane ferromagnetism on the surface of MnBi2Te4.
Materials Science
An Iterative Emotion Interaction Network for Emotion Recognition in Conversations
We noticed that the gold emotion labels of the context utterances can provide explicit and accurate emotion interaction, but it is impossible to input gold labels at inference time.
Ranked #44 on
Emotion Recognition in Conversation
on IEMOCAP
Video Region Annotation with Sparse Bounding Boxes
Video analysis has been moving towards more detailed interpretation (e. g. segmentation) with encouraging progresses.
Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking
Existing Multiple-Object Tracking (MOT) methods either follow the tracking-by-detection paradigm to conduct object detection, feature extraction and data association separately, or have two of the three subtasks integrated to form a partially end-to-end solution.
ReMOTS: Self-Supervised Refining Multi-Object Tracking and Segmentation
We aim to improve the performance of Multiple Object Tracking and Segmentation (MOTS) by refinement.
Ranked #1 on
Multi-Object Tracking
on MOTS20
Multi-Object Tracking
Multi-Object Tracking and Segmentation
+2
When Person Re-identification Meets Changing Clothes
We find that changing clothes makes ReID a much harder problem in the sense of bringing difficulties to learning effective representations and also challenges the generalization ability of previous ReID models to identify persons with unseen (new) clothes.
Multiple Object Tracking by Flowing and Fusing
Most of Multiple Object Tracking (MOT) approaches compute individual target features for two subtasks: estimating target-wise motions and conducting pair-wise Re-Identification (Re-ID).
Compressing 3DCNNs Based on Tensor Train Decomposition
However, due to the higher dimension of convolutional kernels, the space complexity of 3DCNNs is generally larger than that of traditional two dimensional convolutional neural networks (2DCNNs).
Ranked #1 on
Quantization
on Knowledge-based:
Using Panoramic Videos for Multi-person Localization and Tracking in a 3D Panoramic Coordinate
3D panoramic multi-person localization and tracking are prominent in many applications, however, conventional methods using LiDAR equipment could be economically expensive and also computationally inefficient due to the processing of point cloud data.
Ranked #1 on
Multi-Object Tracking
on MOT15_3D
(using extra training data)
A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models
To solve this problem, we derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation.
Make Skeleton-based Action Recognition Model Smaller, Faster and Better
Although skeleton-based action recognition has achieved great success in recent years, most of the existing methods may suffer from a large model size and slow execution speed.
Dynamic Face Video Segmentation via Reinforcement Learning
To the best of our knowledge, this is the first work to use reinforcement learning for online key-frame decision in dynamic video segmentation, and also the first work on its application on face videos.
Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification
An efficient and effective person re-identification (ReID) system relieves the users from painful and boring video watching and accelerates the process of video analysis.
ReshapeGAN: Object Reshaping by Providing A Single Reference Image
We present the tailored models of the proposed ReshapeGAN for all the problem settings, and have them tested on 8 kinds of reshaping tasks with 13 different datasets, demonstrating the ability of ReshapeGAN on generating convincing and superior results for object reshaping.
Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation
Current approaches have made great progress on image-to-image translation tasks benefiting from the success of image synthesis methods especially generative adversarial networks (GANs).
Generative Adversarial Network
Image-to-Image Translation
+1
FRAME Revisited: An Interpretation View Based on Particle Evolution
FRAME (Filters, Random fields, And Maximum Entropy) is an energy-based descriptive model that synthesizes visual realism by capturing mutual patterns from structural input signals.
Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition
Humans can naturally understand an image in depth with the aid of rich knowledge accumulated from daily lives or professions.
Fine-Grained Image Classification
Fine-Grained Image Recognition
+2
Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning
In contrast to this body of research, we propose to treat active learning algorithm design as a meta-learning problem and learn the best criterion from data.
Metric Learning via Maximizing the Lipschitz Margin Ratio
In this paper, we propose the Lipschitz margin ratio and a new metric learning framework for classification through maximizing the ratio.
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Official Torch7 implementation of "V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map", CVPR 2018
Ranked #5 on
Hand Pose Estimation
on HANDS 2017
Pose-Normalized Image Generation for Person Re-identification
Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations.
Joint Hierarchical Category Structure Learning and Large-Scale Image Classification
We propose a novel image classification method based on learning hierarchical inter-class structures.
Saturation-Preserving Specular Reflection Separation
Specular reflection generally decreases the saturation of surface colors, which will be possibly confused with other colors that have the same hue but lower saturation.
Collaborative Representation for Classification, Sparse or Non-sparse?
Sparse representation based classification (SRC) has been proved to be a simple, effective and robust solution to face recognition.
