Search Results for author:
Found 224 papers, 106 papers with code
DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models
In this paper, we present DesignDiffusion, a simple yet effective framework for the novel task of synthesizing design images from textual descriptions.
Towards Practical Real-Time Neural Video Compression
In practice, the coding speed of NVCs depends on 1) computational costs, and 2) non-computational operational costs, such as memory I/O and the number of function calls.
Model Evolution Framework with Genetic Algorithm for Multi-Task Reinforcement Learning
To this end, we propose a Model Evolution framework with Genetic Algorithm (MEGA), which enables the model to evolve during training according to the difficulty of the tasks.
Uni-Sign: Toward Unified Sign Language Understanding at Scale
Sign language pre-training has gained increasing attention for its ability to enhance performance across various sign language understanding (SLU) tasks.
Ranked #1 on
Gloss-free Sign Language Translation
on OpenASL
(using extra training data)
Computational Efficiency
Gloss-free Sign Language Translation
+3
SmartEraser: Remove Anything from Images using Masked-Region Guidance
This paradigm retains the masked region in the input, using it as guidance for the removal process.
SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation
To solve this issue, this paper approaches a novel framework named SceneBooth for subject-preserved text-to-image generation, which consumes inputs of a subject image, object phrases and text prompts.
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
Knowledge distillation (KD) has become a prevalent technique for compressing large language models (LLMs).
RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion
For the latter, we initially incorporate a radar-guided depth head to refine the transformation from image view to BEV.
RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement
After iterative refinements, our curling AI based on the decision tree ranks first on the Jidi platform among 34 curling AIs in total, which demonstrates that LLMs can significantly enhance the robustness and adaptability of decision trees, representing a substantial advancement in the field of Game AI.
Make-It-Animatable: An Efficient Framework for Authoring Animation-Ready 3D Characters
3D characters are essential to modern creative industries, but making them animatable often demands extensive manual work in tasks like rigging and skinning.
ROOT: VLM based System for Indoor Scene Understanding and Beyond
To train our SceneVLM, we collect over 610, 000 images from various public indoor datasets and implement a scene data generation pipeline with a semi-automated technique to establish relationships and estimate distances among indoor objects.
BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language?
Dense retrieval, which aims to encode the semantic information of arbitrary text into dense vector representations or embeddings, has emerged as an effective and efficient paradigm for text retrieval, consequently becoming an essential component in various natural language processing systems.
Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
Trustworthiness is an essential prerequisite for the real-world application of large language models.
MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning
We introduce MotionRL, the first approach to utilize Multi-Reward Reinforcement Learning (RL) for optimizing text-to-motion generation tasks and aligning them with human preferences.
StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splatting
Reconstructing urban street scenes is crucial due to its vital role in applications such as autonomous driving and urban planning.
P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task
Embodied Everyday Task is a popular task in the embodied AI community, requiring agents to make a sequence of actions based on natural language instructions and visual observations.
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning
Recent works propose to employ supervised fine-tuning (SFT) to mitigate the sycophancy issue, while it typically leads to the degeneration of LLMs' general capability.
AdaptVision: Dynamic Input Scaling in MLLMs for Versatile Scene Understanding
We hypothesize that the requisite number of visual tokens for the model is contingent upon both the resolution and content of the input image.
LaneTCA: Enhancing Video Lane Detection with Temporal Context Aggregation
In this work, we propose LaneTCA to bridge the individual video frames and explore how to effectively aggregate the temporal context.
Scaling up Multimodal Pre-training for Sign Language Understanding
To facilitate communication between the deaf-mute and hearing people, a series of sign language understanding (SLU) tasks have been studied in recent years, including isolated/continuous sign language recognition (ISLR/CSLR), gloss-free sign language translation (GF-SLT) and sign language retrieval (SL-RT).
SwinShadow: Shifted Window for Ambiguous Adjacent Shadow Detection
In this case, when the color of the object is similar to that of the shadow, existing methods struggle to achieve accurate detection.
SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval
Furthermore, existing RGB-based sign retrieval works suffer from the huge memory cost of dense visual data embedding in end-to-end training, and adopt offline RGB encoder instead, leading to suboptimal feature representation.
Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesis
Synthesizing realistic 3D indoor scenes is a challenging task that traditionally relies on manual arrangement and annotation by expert designers.
RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation
Existing rectification methods are limited to central fisheye images, while this paper proposes a novel method that extends to deviated fisheye image rectification.
Text-Animator: Controllable Visual Text Video Generation
Video generation is a challenging yet pivotal task in various industries, such as gaming, e-commerce, and advertising.
Prediction and Reference Quality Adaptation for Learned Video Compression
With these filters, our codec can adapt to different reference qualities, making it easier to achieve the target reconstruction quality and reduce the reconstruction error propagation.
Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition
On one hand, since the semantics of sign language are expressed by the cooperation of fine-grained hands and coarse-grained trunks, we utilize both granularity information and encode them into latent spaces.
Semi-Supervised Spoken Language Glossification
Spoken language glossification (SLG) aims to translate the spoken language text into the sign language gloss, i. e., a written record of sign language.
Mini Honor of Kings: A Lightweight Environment for Multi-Agent Reinforcement Learning
To address these issues, we introduce the first publicly available map editor for the popular mobile game Honor of Kings and design a lightweight environment, Mini Honor of Kings (Mini HoK), for researchers to conduct experiments.
TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy
In this mechanism, all the involved diverse visual table understanding (VTU) tasks and multi-source visual embeddings are abstracted as concepts.
MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition
To this end, we propose a Motion-Aware masked autoencoder with Semantic Alignment (MASA) that integrates rich motion cues and global semantic information in a self-supervised learning paradigm for SLR.
EG4D: Explicit Generation of 4D Object without Score Distillation
In recent years, the increasing demand for dynamic 3D assets in design and gaming applications has given rise to powerful generative pipelines capable of synthesizing high-quality 4D objects.
Learning Generalizable Human Motion Generator with Reinforcement Learning
Text-driven human motion generation, as one of the vital tasks in computer-aided content creation, has recently attracted increasing attention.
Progressive Multi-modal Conditional Prompt Tuning
Initialization is responsible for encoding images and text using a VLM, followed by a feature filter that selects text features similar to image.
TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding
The image overview stage provides a comprehensive understanding of the global scene information, and the coarse localization stage approximates the image area containing the answer based on the question asked.
Cross-Lingual Transfer for Natural Language Inference via Multilingual Prompt Translator
To efficiently transfer soft prompt, we propose a novel framework, Multilingual Prompt Translator (MPT), where a multilingual prompt translator is introduced to properly process crucial knowledge embedded in prompt by changing language knowledge while retaining task knowledge.
Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstruction
To address the above problem, we propose a novel motion-aware enhancement framework for dynamic scene reconstruction, which mines useful motion cues from optical flow to improve different paradigms of dynamic 3DGS.
GaussNav: Gaussian Splatting for Visual Navigation
In embodied vision, Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment.
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised man- ner.
Image2Sentence based Asymmetrical Zero-shot Composed Image Retrieval
To tackle the above problems, we propose Image2Sentence based Asymmetric zero-shot composed image retrieval (ISA), which takes advantage of the VL model and only relies on unlabeled images for composition learning.
Asymmetric Feature Fusion for Image Retrieval
Then, a dynamic mixer is introduced to aggregate these features into compact embedding for efficient search.
Structure Similarity Preservation Learning for Asymmetric Image Retrieval
The centroid vectors in the quantizer serve as anchor points in the embedding space of the gallery model to characterize its structure.
DeepEraser: Deep Iterative Context Mining for Generic Text Eraser
In this work, we present DeepEraser, an effective deep network for generic text removal.
Sinkhorn Distance Minimization for Knowledge Distillation
We propose the Sinkhorn Knowledge Distillation (SinKD) that exploits the Sinkhorn distance to ensure a nuanced and precise assessment of the disparity between teacher and student distributions.
Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigation
As a new embodied vision task, Instance ImageGoal Navigation (IIN) aims to navigate to a specified object depicted by a goal image in an unexplored environment.
Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compression
With the SDD-based motion model and long short-term temporal contexts fusion, our proposed learned video codec can obtain more accurate inter prediction.
Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
In this study, we tackle the challenge of classifying the object category in point clouds, which previous works like PointCLIP struggle to address due to the inherent limitations of the CLIP architecture.
Generative Latent Coding for Ultra-Low Bitrate Image Compression
To address this issue we introduce a Generative Latent Coding (GLC) architecture which performs transform coding in the latent space of a generative vector-quantized variational auto-encoder (VQ-VAE) instead of in the pixel space.
Passive Non-Line-of-Sight Imaging with Light Transport Modulation
In this work, we propose NLOS-LTM, a novel passive NLOS imaging method that effectively handles multiple light transport conditions with a single network.
TinySAM: Pushing the Envelope for Efficient Segment Anything Model
Massive following works have developed various applications based on the pre-trained SAM and achieved impressive performance on downstream vision tasks.
DanZero+: Dominating the GuanDan Game through Reinforcement Learning
The utilization of artificial intelligence (AI) in card games has been a well-explored subject within AI research for an extensive period.
Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs
Moreover, we curate a collection of text-rich images and prompt the text-only GPT-4 to generate 12K high-quality conversations, featuring textual locations within text-rich scenarios.
DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding
This work presents DocPedia, a novel large multimodal model (LMM) for versatile OCR-free document understanding, capable of parsing images up to 2, 560$\times$2, 560 resolution.
PersonMAE: Person Re-Identification Pre-Training with Masked AutoEncoders
Pre-training is playing an increasingly important role in learning generic feature representation for Person Re-identification (ReID).
Progressive Recurrent Network for Shadow Removal
To handle this issue, we consider removing the shadow in a coarse-to-fine fashion and propose a simple but effective Progressive Recurrent Network (PRNet).
I$^2$MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
Different from existing distillation solutions that transfer the knowledge of a pre-trained and fixed teacher to the student, in CMD, the knowledge is continuously updated and bidirectionally distilled between modalities during pre-training.
Accelerate Presolve in Large-Scale Linear Programming via Reinforcement Learning
Specifically, we formulate the routine design task as a Markov decision process and propose an RL framework with adaptive action sequences to generate high-quality presolve routines efficiently.
MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles
As vehicular communication and networking technologies continue to advance, infrastructure-based roadside perception emerges as a pivotal tool for connected automated vehicle (CAV) applications.
Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning
In this study, we introduce an uncertainty-aware in-context learning framework to empower the model to enhance or reject its output in response to uncertainty.
Sign Language Translation with Iterative Prototype
Technically, IP-SLT consists of feature extraction, prototype initialization, and iterative prototype refinement.
Ranked #6 on
Sign Language Translation
on CSL-Daily
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding
However, existing advanced algorithms are limited to effectively utilizing the immense representation capabilities and rich world knowledge inherent to these large pre-trained models, and the beneficial connections among tasks within the context of text-rich scenarios have not been sufficiently explored.
SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning
To make the best of such rectification cues, we introduce SimFIR, a simple framework for fisheye image rectification based on self-supervised representation learning.
Text-Only Training for Visual Storytelling
Visual storytelling aims to generate a narrative based on a sequence of images, necessitating both vision-language alignment and coherent story generation.
DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory
Our experiments validate the effectiveness of DragNUWA, demonstrating its superior performance in fine-grained control in video generation.
Masked Motion Predictors are Strong 3D Action Representation Learners
To be specific, the proposed MAMP takes as input the masked spatio-temporal skeleton sequence and predicts the corresponding temporal motion of the masked human joints.
Few-Shot Skeleton-Based Action Recognition
motion prediction
+2
Cyclic-Bootstrap Labeling for Weakly Supervised Object Detection
These inaccurate high-scoring region proposals will mislead the training of subsequent refinement modules and thus hamper the detection performance.
Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video
Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e. g. self- and mutual occlusion and similar textures.
AltFreezing for More General Video Face Forgery Detection
In this paper, we propose to capture both spatial and temporal artifacts in one model for face forgery detection.
VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision
Such compact representations need to be decoded back to pixels before being displayed to humans and - as usual - before being enhanced/analyzed by machine vision algorithms.
Exploring Effective Mask Sampling Modeling for Neural Image Compression
Specifically, Cube Mask Sampling Module (CMSM) is proposed to apply both spatial and channel mask sampling modeling to image compression in the pre-training stage.
MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learning
Recent approaches have utilized self-supervised auxiliary tasks as representation learning to improve the performance and sample efficiency of vision-based reinforcement learning algorithms in single-agent settings.
Detect Any Shadow: Segment Anything for Video Shadow Detection
Segment anything model (SAM) has achieved great success in the field of natural image segmentation.
Hybrid and Collaborative Passage Reranking
In passage retrieval system, the initial passage retrieval results may be unsatisfactory, which can be refined by a reranking scheme.
SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding
In our framework, the hand pose is regarded as a visual token, which is derived from an off-the-shelf detector.
Ranked #2 on
Sign Language Recognition
on MSASL-1000
(using extra training data)
O-GNN: Incorporating Ring Priors into Molecular Modeling
Despite the recent success of molecular modeling with graph neural networks (GNNs), few models explicitly take rings in compounds into consideration, consequently limiting the expressiveness of the models.
Ranked #1 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
DocMAE: Document Image Rectification via Self-supervised Representation Learning
Tremendous efforts have been made on document image rectification, but how to learn effective representation of such distorted images is still under-explored.
Deep Unrestricted Document Image Rectification
To our best knowledge, this is the first learning-based method for the rectification of unrestricted document images.
Ranked #1 on
Local Distortion
on DocUNet
Learning Transferable Pedestrian Representation from Multimodal Information Supervision
Recent researches on unsupervised person re-identification~(reID) have demonstrated that pre-training on unlabeled person images achieves superior performance on downstream reID tasks than pre-training on ImageNet.
HandNeRF: Neural Radiance Fields for Animatable Interacting Hands
We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands, enabling the rendering of photo-realistic images and videos for gesture animation from arbitrary views.
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation.
Human Pose as Compositional Tokens
Human pose is typically represented by a coordinate vector of body joints or their heatmap embeddings.
Ranked #1 on
Pose Estimation
on MPII Human Pose
DIRE for Diffusion-Generated Image Detection
We find that existing detectors struggle to detect images generated by diffusion models, even if we include generated images from a specific diffusion model in their training data.
Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation
Currently, a popular UDA framework lies in self-training which endows the model with two-fold abilities: (i) learning reliable semantics from the labeled images in the source domain, and (ii) adapting to the target domain via generating pseudo labels on the unlabeled images.
ROCO: A Roundabout Traffic Conflict Dataset
With the traffic conflict data collected, we discover that failure to yield to circulating vehicles when entering the roundabout is the largest contributing reason for traffic conflicts.
BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization
In this work, we are dedicated to leveraging the BERT pre-training success and modeling the domain-specific statistics to fertilize the sign language recognition~(SLR) model.
Recurrent Generic Contour-based Instance Segmentation with Progressive Learning
It maintains a single estimate of the contour that is progressively deformed toward the object boundary.
Ranked #1 on
Semantic Contour Prediction
on Sbd val
OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection
To this end, we propose OA-BEV, a network that can be plugged into the BEV-based 3D object detection framework to bring out the objects by incorporating object-aware pseudo-3D features and depth features.
Motion Information Propagation for Neural Video Compression
Meanwhile, besides assisting frame coding at the current time step, the feature from context generation will be propagated as motion condition when coding the subsequent motion latent.
Multi-Agent Reinforcement Learning with Shared Resources for Inventory Management
In this paper, we consider the inventory management (IM) problem where we need to make replenishment decisions for a large number of stock keeping units (SKUs) to balance their supply and demand.
Hand-Object Interaction Image Generation
In this work, we are dedicated to a new task, i. e., hand-object interaction image generation, which aims to conditionally generate the hand-object image under the given hand, object and their interaction status.
CLIP2GAN: Towards Bridging Text with the Latent Space of GANs
The key idea of our CLIP2GAN is to bridge the output feature embedding space of CLIP and the input latent space of StyleGAN, which is realized by introducing a mapping network.
SinDiffusion: Learning a Diffusion Model from a Single Natural Image
We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image.
Ranked #1 on
Image Generation
on Places50
Stare at What You See: Masked Image Modeling without Reconstruction
However, unlike the low-level features such as pixel values, we argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image. This raises one question: is reconstruction necessary in Masked Image Modeling (MIM) with a teacher model?
DanZero: Mastering GuanDan Game with Reinforcement Learning
We compare it with 8 baseline AI programs which are based on heuristic rules and the results reveal the outstanding performance of DanZero.
Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding
In this work, we propose a novel candidate-free framework: Fine-grained Semantic Alignment Network (FSAN), for weakly supervised TLG.
Geometric Representation Learning for Document Image Rectification
In document image rectification, there exist rich geometric constraints between the distorted image and the ground truth one.
UDoc-GAN: Unpaired Document Illumination Correction with Background Light Prior
To this end, we propose UDoc-GAN, the first framework to address the problem of document illumination correction under the unpaired setting.
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment
and 2) how to mitigate the impact of these factors?
Ranked #2 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
In this work, we formulate the cross-modal interaction as a bidirectional knowledge distillation problem.
3D Action Recognition
Few-Shot Skeleton-Based Action Recognition
+3
Low-Light Video Enhancement with Synthetic Event Guidance
Low-light video enhancement (LLVE) is an important yet challenging task with many applications such as photographing and autonomous driving.
Neighbor Correspondence Matching for Flow-based Video Frame Synthesis
Since the current frame is not available in video frame synthesis, NCM is performed in a current-frame-agnostic fashion to establish multi-scale correspondences in the spatial-temporal neighborhoods of each pixel.
Ranked #4 on
Video Frame Interpolation
on X4K1000FPS
Unified 2D and 3D Pre-Training of Molecular Representations
The model is pre-trained on three tasks: reconstruction of masked atoms and coordinates, 3D conformation generation conditioned on 2D graph, and 2D graph generation conditioned on 3D conformation.
Self-Adaptive Label Augmentation for Semi-supervised Few-shot Classification
A major novelty of SALA is the task-adaptive metric, which can learn the metric adaptively for different tasks in an end-to-end fashion.
TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
For another, we devise Language Conditioned Vision Transformer that removes external fusion modules and reuses the uni-modal ViT for vision-language fusion at the intermediate layers.
Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer
Utilizing MARL algorithms to coordinate multiple control units in the grid, which is able to handle rapid changes of power systems, has been widely studied in active voltage control task recently.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation
To further illustrate the commercial value of our approach, we conduct experiments on three generation tasks in real-world advertisements applications.
Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods
Actor-critic Reinforcement Learning (RL) algorithms have achieved impressive performance in continuous control tasks.
Multi-Target Active Object Tracking with Monte Carlo Tree Search and Target Motion Modeling
In this work, we are dedicated to multi-target active object tracking (AOT), where there are multiple targets as well as multiple cameras in the environment.
LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning
In this way, agents dealing with the same subtask share their learning of specific abilities and different subtasks correspond to different specific abilities.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Estimation of Reliable Proposal Quality for Temporal Action Detection
For the region perspective, we introduce Region Evaluate Module (REM) which uses a new and efficient sampling method for proposal feature representation containing more contextual information compared with point feature to refine category score and proposal boundary.
Domain-Agnostic Prior for Transfer Semantic Segmentation
Unsupervised domain adaptation (UDA) is an important topic in the computer vision community.
DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learning
Recent years have witnessed the great breakthrough of deep reinforcement learning (DRL) in various perfect and imperfect information games.
Large-Scale Pre-training for Person Re-identification with Noisy Labels
Since theses ID labels automatically derived from tracklets inevitably contain noises, we develop a large-scale Pre-training framework utilizing Noisy Labels (PNL), which consists of three learning modules: supervised Re-ID learning, prototype-based contrastive learning, and label-guided contrastive learning.
Ranked #8 on
Person Re-Identification
on CUHK03
Learning Enriched Illuminants for Cross and Single Sensor Color Constancy
Color constancy aims to restore the constant colors of a scene under different illuminants.
Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents
To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
Multi-agent Reinforcement Learning
reinforcement-learning
+4
CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learning
In this way, CTDS balances the full utilization of global observation during training and the feasibility of decentralized execution for online inference.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
TAPE: Task-Agnostic Prior Embedding for Image Restoration
In this paper, we propose a novel approach that embeds a task-agnostic prior into a transformer.
MVP: Multimodality-guided Visual Pre-training
Recently, masked image modeling (MIM) has become a promising direction for visual pre-training.
Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking
In our approach, we regard each camera as an agent and address AMOT with a multi-agent reinforcement learning solution.
MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning
Specifically, we model both individual Q-values and global Q-value with categorical distribution.
Revisiting QMIX: Discriminative Credit Assignment by Gradient Entropy Regularization
In cooperative multi-agent systems, agents jointly take actions and receive a team reward instead of individual rewards.
Direct Molecular Conformation Generation
Molecular conformation generation aims to generate three-dimensional coordinates of all the atoms in a molecule and is an important task in bioinformatics and pharmacology.
Representing Videos as Discriminative Sub-graphs for Action Recognition
For each action category, we execute online clustering to decompose the graph into sub-graphs on each scale through learning Gaussian Mixture Layer and select the discriminative sub-graphs as action prototypes for recognition.
Contextual Similarity Distillation for Asymmetric Image Retrieval
To this end, we propose a flexible contextual similarity distillation framework to enhance the small query model and keep its output feature compatible with that of large gallery model, which is crucial with asymmetric retrieval.
Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization
Many existing algorithms learn robust policies by modeling the disturbance and applying it to source environments during training, which usually requires prior knowledge about the disturbance and control of simulators.
Radio-Assisted Human Detection
In this paper, we propose a radio-assisted human detection framework by incorporating radio information into the state-of-the-art detection methods, including anchor-based onestage detectors and two-stage detectors.
Learning Token-based Representation for Image Retrieval
Next, a refinement block is introduced to enhance the visual tokens with self-attention and cross-attention.
Ranked #3 on
Image Retrieval
on RParis (Medium)
VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion
However, this approach often suffers from the mismatch between the resolution of point clouds and RGB images, leading to sub-optimal performance.
Dual Progressive Prototype Network for Generalized Zero-Shot Learning
This enables DPPN to produce visual representations with accurate attribute localization ability, which benefits the semantic-visual alignment and representation transferability.
Unsupervised Person Re-Identification with Wireless Positioning under Weak Scene Labeling
To this end, we propose to explore unsupervised person re-identification with both visual data and wireless positioning trajectories under weak scene labeling, in which we only need to know the locations of the cameras.
DocScanner: Robust Document Image Rectification with Progressive Learning
The iterative refinements make DocScanner converge to a robust and superior rectification performance, while the lightweight recurrent architecture ensures the running efficiency.
Contextual Similarity Aggregation with Self-attention for Visual Re-ranking
Since our re-ranking model is not directly involved with the visual feature used in the initial retrieval, it is ready to be applied to retrieval result lists obtained from various retrieval algorithms.
DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction
Specifically, DocTr consists of a geometric unwarping transformer and an illumination correction transformer.
Equivalence Analysis between Counterfactual Regret Minimization and Online Mirror Descent
Follow-the-Regularized-Lead (FTRL) and Online Mirror Descent (OMD) are regret minimization algorithms for Online Convex Optimization (OCO), they are mathematically elegant but less practical in solving Extensive-Form Games (EFGs).
SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
To validate the effectiveness of our method on SLR, we perform extensive experiments on four public benchmark datasets, i. e., NMFs-CSL, SLR500, MSASL and WLASL.
Ranked #3 on
Sign Language Recognition
on WLASL100
(using extra training data)
Multi-Agent Reinforcement Learning with Shared Resource in Inventory Management
We consider inventory management (IM) problem for a single store with a large number of SKUs (stock keeping units) in this paper, where we need to make replenishment decisions for each SKU to balance its supply and demand.
One-shot Key Information Extraction from Document with Deep Partial Graph Matching
However, collecting and labeling a large dataset is time-consuming and is not a user-friendly requirement for many cloud platforms.
Learning Fine-Grained Motion Embedding for Landscape Animation
To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation.
Discovering Representation Sprachbund For Multilingual Pre-Training
Then we cluster all the target languages into multiple groups and name each group as a representation sprachbund.
Heredity-aware Child Face Image Generation with Latent Space Disentanglement
The main idea is to disentangle the latent space of a pre-trained generation model and precisely control the face attributes of child images with clear semantics.
Conditional DETR for Fast Training Convergence
Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention.
Ranked #109 on
Object Detection
on COCO minival
Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation
In this paper, we uncover that the negative examples play a critical role in the performance of contrastive learning for image translation.
Joint Inductive and Transductive Learning for Video Object Segmentation
In this work, we propose to integrate transductive and inductive learning into a unified framework to exploit the complementarity between them for accurate and robust video object segmentation.
From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection
To this end, in this work, we regard point clouds as hollow-3D data and propose a new architecture, namely Hallucinated Hollow-3D R-CNN ($\text{H}^2$3D R-CNN), to address the problem of 3D object detection.
Supervised Off-Policy Ranking
Inspired by the two observations, in this work, we study a new problem, supervised off-policy ranking (SOPR), which aims to rank a set of target policies based on supervised learning by leveraging off-policy data and policies with known performance.
Revisiting Knowledge Distillation: An Inheritance and Exploration Framework
The inheritance part is learned with a similarity loss to transfer the existing learned knowledge from the teacher model to the student model, while the exploration part is encouraged to learn representations different from the inherited ones with a dis-similarity loss.
Weakly Supervised Temporal Adjacent Network for Language Grounding
To this end, we introduce a novel weakly supervised temporal adjacent network (WSTAN) for temporal language grounding.
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
Multi-Modal 3D Object Detection in Autonomous Driving: a Survey
In this survey, we first introduce the background of popular sensors used for self-driving, their data properties, and the corresponding object detection algorithms.
ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Image Segmentation
Semi-supervised learning is a useful tool for image segmentation, mainly due to its ability in extracting knowledge from unlabeled data to assist learning from labeled data.
Model-Aware Gesture-to-Gesture Translation
Then, a transformation flow is calculated based on the correspondence of the source and target topology map.
Dual-view Molecule Pre-training
After pre-training, we can use either the Transformer branch (this one is recommended according to empirical results), the GNN branch, or both for downstream tasks.
Ranked #1 on
Molecular Property Prediction
on HIV dataset
Uformer: A General U-Shaped Transformer for Image Restoration
Powered by these two designs, Uformer enjoys a high capability for capturing both local and global dependencies for image restoration.
Ranked #3 on
Deblurring
on RealBlur-R (trained on GoPro)
Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task
By simply pulling the different augmented views of each image together or other novel mechanisms, they can learn much unsupervised knowledge and significantly improve the transfer performance of pre-training models.
Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
Finally, the synthetic parallel data serves as a strong supplement for the end-to-end training of the encoder-decoder SLT framework.
Ranked #5 on
Sign Language Translation
on CSL-Daily
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training
The pre-trained neural models have recently achieved impressive performances in understanding multimodal content.
TransVG: End-to-End Visual Grounding with Transformers
In this paper, we present a neat yet effective transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to the corresponding region onto an image.
Ranked #18 on
Referring Expression Comprehension
on RefCOCO
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
ProphetNet is a pre-training based natural language generation method which shows powerful performance on English text summarization and question generation tasks.
Task-Independent Knowledge Makes for Transferable Representations for Generalized Zero-Shot Learning
First, DCEN leverages task labels to cluster representations of the same semantic category by cross-modal contrastive learning and exploring semantic-visual complementarity.
Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE
We propose a two-stage model for diverse inpainting, where the first stage generates multiple coarse results each of which has a different structure, and the second stage refines each coarse result separately by augmenting texture.
IOT: Instance-wise Layer Reordering for Transformer Structures
Based on this observation, in this work, we break the assumption of the fixed layer order in the Transformer and introduce instance-wise layer reordering into the model structure.
3D Local Convolutional Neural Networks for Gait Recognition
Second, different body parts possess different scales, and even the same part in different frames can appear at different locations and scales.
Ranked #4 on
Gait Recognition
on OUMVLP
Learning Deep Local Features With Multiple Dynamic Attentions for Large-Scale Image Retrieval
To this end, we propose a novel deep local feature learning architecture to simultaneously focus on multiple discriminative local patterns in an image.
Consistent Instance Classification for Unsupervised Representation Learning
The proposed consistent instance classification (ConIC) approach simultaneously optimizes the classification loss and an additional consistency loss explicitly penalizing the feature dissimilarity between the augmented views from the same instance.
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
In this paper, we take a slightly different viewpoint -- we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy.
Ranked #4 on
3D Object Detection
on KITTI Cars Moderate val
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation.
Contrastive Transformation for Self-supervised Correspondence Learning
It is worth mentioning that our method also surpasses the fully-supervised affinity representation (e. g., ResNet) and performs competitively against the recent fully-supervised algorithms designed for the specific tasks (e. g., VOT and VOS).
Unsupervised Pre-training for Person Re-identification
In this paper, we present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" and make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation.
Ranked #1 on
Person Re-Identification
on Market-1501
(mAP metric)
Promoting Stochasticity for Expressive Policies via a Simple and Efficient Regularization Method
However, in continuous action spaces, integrating entropy regularization with expressive policies is challenging and usually requires complex inference procedures.
Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition
The topology of the adjacency graph is a key factor for modeling the correlations of the input skeletons.
Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations
Contrastive learning has achieved great success in self-supervised visual representation learning, but existing approaches mostly ignored spatial information which is often crucial for visual representation.
Can Semantic Labels Assist Self-Supervised Visual Representation Learning?
Recently, contrastive learning has largely advanced the progress of unsupervised visual representation learning.
ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
We build a dataset from a real-word sponsored search engine and carry out experiments to analyze different generative retrieval models.
Masked Contrastive Representation Learning for Reinforcement Learning
During inference, the CNN encoder and the policy network are used to take actions, and the Transformer module is discarded.
Boosting Continuous Sign Language Recognition via Cross Modality Augmentation
Continuous sign language recognition (SLR) deals with unaligned video-text pair and uses the word error rate (WER), i. e., edit distance, as the main evaluation metric.
Improving Person Re-identification with Iterative Impression Aggregation
Not only does such a simple method improve the performance of the baseline models, it also achieves comparable performance with latest advanced re-ranking methods.
Global-local Enhancement Network for NMFs-aware Sign Language Recognition
Sign language recognition (SLR) is a challenging problem, involving complex manual features, i. e., hand gestures, and fine-grained non-manual features (NMFs), i. e., facial expression, mouth shapes, etc.
Vision Meets Wireless Positioning: Effective Person Re-identification with Recurrent Context Propagation
Existing person re-identification methods rely on the visual sensor to capture the pedestrians.
Unsupervised Deep Representation Learning for Real-Time Tracking
The advancement of visual tracking has continuously been brought by deep learning models.
Single Shot Video Object Detector
Single shot detectors that are potentially faster and simpler than two-stage detectors tend to be more applicable to object detection in videos.
Efficient Integer-Arithmetic-Only Convolutional Neural Networks
Considering the tradeoff between activation quantization error and network learning ability, we set an empirical rule to tune the bound of each Bounded ReLU.
Cascaded Regression Tracking: Towards Online Hard Distractor Discrimination
In the second stage, a discrete sampling based ridge regression is designed to double-check the remaining ambiguous hard samples, which serves as an alternative of fully-connected layers and benefits from the closed-form solver for efficient learning.
M-LVC: Multiple Frames Prediction for Learned Video Compression
To compensate for the compression error of the auto-encoders, we further design a MV refinement network and a residual refinement network, taking use of the multiple reference frames as well.
Long Short-Term Relation Networks for Video Action Detection
It has been well recognized that modeling human-object or object-object relations would be helpful for detection task.
Incorporating BERT into Neural Machine Translation
While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning.
Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition
Our STMC network consists of a spatial multi-cue (SMC) module and a temporal multi-cue (TMC) module.
Soft Hindsight Experience Replay
However, due to the brittleness of deterministic methods, HER and its variants typically suffer from a major challenge for stability and convergence, which significantly affects the final performance.
Deep Model-Based Reinforcement Learning via Estimated Uncertainty and Conservative Policy Optimization
In this paper, We propose a Policy Optimization method with Model-Based Uncertainty (POMBU)---a novel model-based approach---that can effectively improve the asymptotic performance using the uncertainty in Q-values.
Model-based Reinforcement Learning
reinforcement-learning
+2
A Generalization Theory based on Independent and Task-Identically Distributed Assumption
The derived generalization bound based on the ITID assumption identifies the significance of hypothesis invariance in guaranteeing generalization performance.
Quantization Networks
The proposed quantization function can be learned in a lossless and end-to-end manner and works for any weights and activations of neural networks in a simple and uniform way.
AETv2: AutoEncoding Transformations for Self-Supervised Representation Learning by Minimizing Geodesic Distances in Lie Groups
For this reason, we should use the geodesic to characterize how an image transform along the manifold of a transformation group, and adopt its length to measure the deviation between transformations.
An End-to-End Foreground-Aware Network for Person Re-Identification
By presenting a target attention loss, the pedestrian features extracted from the foreground branch become more insensitive to the backgrounds, which greatly reduces the negative impacts of changing backgrounds on matching an identical across different camera views.
Progressive Unsupervised Person Re-identification by Tracklet Association with Spatio-Temporal Regularization
In our approach, we first collect tracklet data within each camera by automatic person detection and tracking.
Learn Interpretable Word Embeddings Efficiently with von Mises-Fisher Distribution
Results show that our model can outperform the dominant models consistently in these tasks.
Relation Distillation Networks for Video Object Detection
In this paper, we introduce a new design to capture the interactions across the objects in spatio-temporal context.
Real-Time Correlation Tracking via Joint Model Compression and Transfer
In the distillation process, we propose a fidelity loss to enable the student network to maintain the representation capability of the teacher network.
Online Filter Clustering and Pruning for Efficient Convnets
Pruning filters is an effective method for accelerating deep neural networks (DNNs), but most existing approaches prune filters on a pre-trained network directly which limits in acceleration.
Progressive Learning of Low-Precision Networks
Recent years have witnessed the great advance of deep learning in a variety of vision tasks.
