Search Results for author:
Found 243 papers, 144 papers with code
Region Proposal by Guided Anchoring
State-of-the-art detectors mostly rely on a dense anchoring scheme, where anchors are sampled uniformly over the spatial domain with a predefined set of scales and aspect ratios.
Ranked #1 on
Region Proposal
on COCO test-dev
Hybrid Task Cascade for Instance Segmentation
In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.
Ranked #32 on
Object Detection
on COCO-O
Libra R-CNN: Towards Balanced Learning for Object Detection
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
Ranked #149 on
Object Detection
on COCO test-dev
Prime Sample Attention in Object Detection
Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector.
CARAFE: Content-Aware ReAssembly of FEatures
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Ranked #3 on
Feature Upsampling
on ImageNet
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
Side-Aware Boundary Localization for More Precise Object Detection
To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket.
Seesaw Loss for Long-Tailed Instance Segmentation
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
Feature Pyramid Grids
Feature pyramid networks have been widely adopted in the object detection literature to improve feature representations for better handling of variations in scale.
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
Once trained, the motion module can be inserted into a personalized T2I model to form a personalized animation generator.
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
The development of text-to-video (T2V), i. e., generating videos with a given text prompt, has been significantly advanced in recent years.
PSANet: Point-wise Spatial Attention Network for Scene Parsing
We notice information flow in convolutional neural networks is restricted inside local neighborhood regions due to the physical design of convolutional filters, which limits the overall understanding of complex scenes.
Ranked #51 on
Semantic Segmentation
on Cityscapes test
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
In this paper, we study this problem with a practice built on a fully convolutional single-stage detector and propose a general framework FCOS3D.
Ranked #323 on
3D Object Detection
on nuScenes
Probabilistic and Geometric Depth: Detecting Objects in Perspective
As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty.
Ranked #10 on
3D Object Detection
on KITTI Cars Hard val
MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding
We present MMOCR-an open-source toolbox which provides a comprehensive pipeline for text detection and recognition, as well as their downstream tasks such as named entity recognition and key information extraction.
Temporal Action Detection with Structured Segment Networks
Detecting actions in untrimmed videos is an important yet challenging task.
Ranked #6 on
Action Recognition
on THUMOS’14
Temporal Segment Networks for Action Recognition in Videos
Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.
Ranked #5 on
Video Classification
on COIN
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
The other contribution is our study on a series of good practices in learning ConvNets on video data with the help of temporal segment network.
Ranked #3 on
Multimodal Activity Recognition
on EV-Action
Omni-sourced Webly-supervised Learning for Video Recognition
Then a joint-training strategy is proposed to deal with the domain gaps between multiple data sources and formats in webly-supervised learning.
Ranked #5 on
Action Recognition
on UCF101
(using extra training data)
Revisiting Skeleton-based Action Recognition
In this work, we propose PoseC3D, a new approach to skeleton-based action recognition, which relies on a 3D heatmap stack instead of a graph sequence as the base representation of human skeletons.
Ranked #1 on
Action Recognition
on NTU RGB+D
Temporal RoI Align for Video Object Recognition
In this work, considering the features of the same object instance are highly similar among frames in a video, a novel Temporal RoI Align operator is proposed to extract features from other frames feature maps for current frame proposals by utilizing feature similarity.
Ranked #1 on
Video Instance Segmentation
on YouTube-VIS
Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so.
Ranked #13 on
Contrastive Learning
on imagenet-1k
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
Improving Pixel-based MIM by Reducing Wasted Modeling Capability
There has been significant progress in Masked Image Modeling (MIM).
Unsupervised Feature Learning via Non-Parametric Instance Discrimination
Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so.
Ranked #40 on
Semi-Supervised Image Classification
on ImageNet - 1% labeled data
(Top 5 Accuracy metric)
PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling
Masked Image Modeling (MIM) has achieved promising progress with the advent of Masked Autoencoders (MAE) and BEiT.
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition.
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Dynamics of human body skeletons convey significant information for human action recognition.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition.
Ranked #9 on
Visual Question Answering (VQA)
on InfiMM-Eval
ShareGPT4V: Improving Large Multi-Modal Models with Better Captions
In the realm of large multi-modal models (LMMs), efficient modality alignment is crucial yet often constrained by the scarcity of high-quality image-text data.
Ranked #1 on
visual instruction following
on LLaVA-Bench
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model excelling in free-form text-image composition and comprehension.
Ranked #16 on
Visual Question Answering
on MM-Vet
DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models
We present DualFocus, a novel framework for integrating macro and micro perspectives within multi-modal large language models (MLLMs) to enhance vision-language task performance.
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution.
Ranked #11 on
Visual Question Answering
on MM-Vet
PYSKL: Towards Good Practices for Skeleton Action Recognition
The toolbox supports a wide variety of skeleton action recognition algorithms, including approaches based on GCN and CNN.
Ranked #19 on
Skeleton Based Action Recognition
on NTU RGB+D 120
DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition
Graph convolution networks (GCN) have been widely used in skeleton-based action recognition.
Ranked #7 on
Skeleton Based Action Recognition
on NTU RGB+D
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space.
Ranked #11 on
LIDAR Semantic Segmentation
on nuScenes
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited.
Ranked #2 on
3D Semantic Segmentation
on ScribbleKITTI
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
Self-Supervised Scene De-occlusion
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
Ranked #4 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
Learning to Cluster Faces on an Affinity Graph
Face recognition sees remarkable progress in recent years, and its performance has reached a very high level.
Learning to Cluster Faces via Confidence and Connectivity Estimation
With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.
A Pursuit of Temporal Accuracy in General Activity Detection
Detecting activities in untrimmed videos is an important but challenging task.
Ranked #29 on
Temporal Action Localization
on ActivityNet-1.3
3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping.
Alpha-CLIP: A CLIP Model Focusing on Wherever You Want
Alpha-CLIP not only preserves the visual recognition ability of CLIP but also enables precise control over the emphasis of image contents.
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
Learning a good image prior is a long-term goal for image restoration and manipulation.
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected.
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
Neural rendering methods have significantly advanced photo-realistic 3D scene rendering in various academic and industrial applications.
Optimizing Video Object Detection via a Scale-Time Lattice
High-performance object detection relies on expensive convolutional networks to compute features, often leading to significant challenges in applications, e. g. those that require detecting objects from video streams in real time.
OneLLM: One Framework to Align All Modalities with Language
In detail, we first train an image projection module to connect a vision encoder with LLM.
Ranked #73 on
Visual Question Answering
on MM-Vet
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases.
Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction
Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.
WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models
The rise in popularity of ChatGPT and GPT-4 has significantly accelerated the development of large models, leading to the creation of numerous impressive large language models(LLMs) and multimodal large language models (MLLMs).
PointLLM: Empowering Large Language Models to Understand Point Clouds
The unprecedented advancements in Large Language Models (LLMs) have shown a profound impact on natural language processing but are yet to fully embrace the realm of 3D understanding.
Ranked #3 on
3D Question Answering (3D-QA)
on 3D MM-Vet
Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future
With the continuous maturation and application of autonomous driving technology, a systematic examination of open-source autonomous driving datasets becomes instrumental in fostering the robust evolution of the industry ecosystem.
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets
We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions.
Ranked #7 on
Long-tail Learning
on VOC-MLT
Monocular 3D Object Detection with Depth from Motion
Perceiving 3D objects from monocular inputs is crucial for robotic systems, given its economy compared to multi-sensor settings.
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions.
Real or Not Real, that is the Question
While generative adversarial networks (GAN) have been widely adopted in various topics, in this paper we generalize the standard GAN to a new perspective by treating realness as a random variable that can be estimated from multiple angles.
VBench: Comprehensive Benchmark Suite for Video Generative Models
We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.
SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition
End-to-end scene text spotting has attracted great attention in recent years due to the success of excavating the intrinsic synergy of the scene text detection and recognition.
Ranked #3 on
Text Spotting
on Inverse-Text
GPT4Point: A Unified Framework for Point-Language Understanding and Generation
Multimodal Large Language Models (MLLMs) have excelled in 2D image-text comprehension and image generation, but their understanding of the 3D world is notably deficient, limiting progress in 3D language understanding and generation.
CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
This paper presents the method that underlies our submission to the untrimmed video classification task of ActivityNet Challenge 2016.
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
It is a large-scale digital library for head avatars with three key attributes: 1) High Fidelity: all subjects are captured by 60 synchronized, high-resolution 2K cameras in 360 degrees.
A Local-to-Global Approach to Multi-modal Movie Scene Segmentation
Scene, as the crucial unit of storytelling in movies, contains complex activities of actors and their interactions in a physical environment.
Detecting Visual Relationships with Deep Relational Networks
Relationships among objects play a crucial role in image understanding.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math.
Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents.
Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases
We conducted a series of structured experiments to evaluate their performance in various industrial application scenarios, offering a comprehensive perspective on their practical utility.
GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences.
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets.
UntrimmedNets for Weakly Supervised Action Recognition and Detection
We exploit the learned models for action recognition (WSR) and detection (WSD) on the untrimmed video datasets of THUMOS14 and ActivityNet.
Ranked #3 on
Action Classification
on ActivityNet-1.2
Weakly Supervised Action Localization
Weakly-Supervised Action Recognition
Person Search in Videos with One Portrait Through Visual and Temporal Links
In real-world applications, e. g. law enforcement and video retrieval, one often needs to search a certain person in long videos with just one portrait.
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
Based on the observation, OPERA introduces a penalty term on the model logits during the beam-search decoding to mitigate the over-trust issue, along with a rollback strategy that retrospects the presence of summary tokens in the previously generated tokens, and re-allocate the token selection if necessary.
Vision Transformer with Progressive Sampling
As a typical example, the Vision Transformer (ViT) directly applies a pure transformer architecture on image classification, by simply splitting images into tokens with a fixed length, and employing transformers to learn relations between these tokens.
T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step
Based on that, we further introduce T-Eval to evaluate the tool utilization capability step by step.
DSNAS: Direct Neural Architecture Search without Parameter Retraining
We argue that given a computer vision task for which a NAS method is expected, this definition can reduce the vaguely-defined NAS evaluation to i) accuracy of this task and ii) the total computation consumed to finally obtain a model with satisfying accuracy.
Ranked #16 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Understanding the wiring evolution in differentiable neural architecture search
To this end, we pose questions that future differentiable methods for neural wiring discovery need to confront, hoping to evoke a discussion and rethinking on how much bias has been enforced implicitly in existing NAS methods.
Self-Supervised Learning via Conditional Motion Propagation
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
SPTS: Single-Point Text Spotting
For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance.
Ranked #3 on
Text Spotting
on SCUT-CTW1500
SPTS v2: Single-Point Scene Text Spotting
Within the context of our SPTS v2 framework, our experiments suggest a potential preference for single-point representation in scene text spotting when compared to other representations.
Ranked #15 on
Text Spotting
on ICDAR 2015
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
Recent advances in adversarial attacks uncover the intrinsic vulnerability of modern deep neural networks.
A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud.
Unified Human-Scene Interaction via Prompted Chain-of-Contacts
Based on the definition, UniHSI constitutes a Large Language Model (LLM) Planner to translate language prompts into task plans in the form of CoC, and a Unified Controller that turns CoC into uniform task execution.
Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
In this paper, we propose Generative Occupancy Fields (GOF), a novel model based on generative radiance fields that can learn compact object surfaces without impeding its training convergence.
Adversarial Robustness under Long-Tailed Distribution
We then perform a systematic study on existing long-tailed recognition methods in conjunction with the adversarial training framework.
BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues
In contrast, other LLMs struggle to generate multi-turn dialogues of satisfactory quality due to poor instruction-following capability, tendency to generate lengthy utterances, or limited general capability.
Are We on the Right Way for Evaluating Large Vision-Language Models?
We evaluate 16 leading LVLMs on MMStar to assess their multi-modal capabilities, and on 7 benchmarks with the proposed metrics to investigate their data leakage and actual multi-modal gain.
SongComposer: A Large Language Model for Lyric and Melody Composition in Song Generation
We present SongComposer, an innovative LLM designed for song composition.
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Despite the substantial progress in recent years, the image captioning techniques are still far from being perfect. Sentences produced by existing methods, e. g. those based on RNNs, are often overly rigid and lacking in variability.
Deep Markov Random Field for Image Modeling
Markov Random Fields (MRFs), a formulation widely used in generative image modeling, have long been plagued by the lack of expressive power.
Position-Guided Point Cloud Panoptic Segmentation Transformer
DEtection TRansformer (DETR) started a trend that uses a group of learnable queries for unified visual perception.
Ranked #1 on
Panoptic Segmentation
on SemanticKITTI
Scaling Laws of RoPE-based Extrapolation
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking
Recent multi-camera 3D object detectors usually leverage temporal information to construct multi-view stereo that alleviates the ill-posed depth estimation.
Few-Shot Object Detection via Association and DIscrimination
1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space.
DevBench: A Comprehensive Benchmark for Software Development
Recent advancements in large language models (LLMs) have significantly enhanced their coding capabilities.
InterControl: Generate Human Motion Interactions by Controlling Every Joint
Furthermore, we demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model (LLM).
From Trailers to Storylines: An Efficient Way to Learn from Movies
Experiments on this dataset showed that the proposed method can substantially reduce the training time while obtaining highly effective features and coherent temporal structures.
Unifying Identification and Context Learning for Person Recognition
In this work, we aim to move beyond such limitations and propose a new framework to leverage context for person recognition.
CLEVA: Chinese Language Models EVAluation Platform
With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue.
SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds
Multi-class 3D object detection aims to localize and classify objects of multiple categories from point clouds.
Online Hyper-parameter Learning for Auto-Augmentation Strategy
Data augmentation is critical to the success of modern deep learning techniques.
MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training
This paper introduces the Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training and a carefully designed data-efficient 3D object detection benchmark on the Waymo dataset.
SpotServe: Serving Generative Large Language Models on Preemptible Instances
This paper aims to reduce the monetary cost for serving LLMs by leveraging preemptible GPU instances on modern clouds, which offer accesses to spare GPUs at a much cheaper price than regular instances but may be preempted by the cloud at any time.
POPQORN: Quantifying Robustness of Recurrent Neural Networks
The vulnerability to adversarial attacks has been a critical issue for deep neural networks.
SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models
In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount.
Guided Diffusion Model for Adversarial Purification
In this paper, we propose a novel purification approach, referred to as guided diffusion model for purification (GDMP), to help protect classifiers from adversarial attacks.
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
Recently, the large language model (LLM) community has shown increasing interest in enhancing LLMs' capability to handle extremely long documents.
RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition
Notably, our approach demonstrates a significant improvement in performance on 5 fine-grained visual recognition benchmarks, 11 few-shot image recognition datasets, and the 2 object detection datasets under the zero-shot recognition setting.
Accelerating Diffusion Models via Early Stop of the Diffusion Process
By modeling the reverse process of gradually diffusing the data distribution into a Gaussian distribution, generating a sample in DDPMs can be regarded as iteratively denoising a randomly sampled Gaussian noise.
Multi-Level Logit Distillation
Through this framework, the prediction alignment is not only conducted at the instance level, but also at the batch and class level, through which the student model learns instance prediction, input correlation, and category correlation simultaneously.
Characterization of Large Language Model Development in the Datacenter
Large Language Models (LLMs) have presented impressive performance across several transformative tasks.
Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant
Semi-Supervised Semantic Segmentation aims at training the segmentation model with limited labeled data and a large amount of unlabeled data.
Motion Guided 3D Pose Estimation from Videos
We propose a new loss function, called motion loss, for the problem of monocular 3D Human pose estimation from 2D pose.
Ranked #19 on
3D Human Pose Estimation
on Human3.6M
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K
In contrast, the average context lengths of mainstream benchmarks are insufficient (5k-21k), and they suffer from potential knowledge leakage and inaccurate metrics, resulting in biased evaluation.
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation
In this paper, we propose a simple yet effective approach for self-supervised video object segmentation (VOS).
CriticBench: Evaluating Large Language Models as Critic
Critique ability are crucial in the scalable oversight and self-improvement of Large Language Models (LLMs).
TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition
To mitigate this problem, we developed TransRank, a unified framework for recognizing Transformations in a Ranking formulation.
Towards Evaluating and Training Verifiably Robust Neural Networks
Recent works have shown that interval bound propagation (IBP) can be used to train verifiably robust neural networks.
OCSampler: Compressing Videos to One Clip with Single-step Sampling
Recent works prefer to formulate frame sampling as a sequential decision task by selecting frames one by one according to their importance, while we present a new paradigm of learning instance-specific video condensation policies to select informative frames for representing the entire video only in a single step.
Policy Continuation with Hindsight Inverse Dynamics
This approach learns from Hindsight Inverse Dynamics based on Hindsight Experience Replay, enabling the learning process in a self-imitated manner and thus can be trained with supervised learning.
Flames: Benchmarking Value Alignment of LLMs in Chinese
The widespread adoption of large language models (LLMs) across various regions underscores the urgent need to evaluate their alignment with human values.
A Neural Compositional Paradigm for Image Captioning
Mainstream captioning models often follow a sequential structure to generate captions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance.
Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks
Deep learning models have achieved excellent recognition results on large-scale video benchmarks.
Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback
This reliance limits the applicability of RLHF and hinders the development of professional assistants tailored to diverse human preferences.
Fastened CROWN: Tightened Neural Network Robustness Certificates
We draw inspiration from such work and further demonstrate the optimality of deterministic CROWN (Zhang et al. 2018) solutions in a given linear programming problem under mild constraints.
Static and Dynamic Concepts for Self-supervised Video Representation Learning
In this paper, we propose a novel learning scheme for self-supervised video representation learning.
Recursive Visual Sound Separation Using Minus-Plus Net
Sounds provide rich semantics, complementary to visual data, for many tasks.
IRLAS: Inverse Reinforcement Learning for Architecture Search
Motivated by the fact that human-designed networks are elegant in topology with a fast inference speed, we propose a mirror stimuli function inspired by biological cognition theory to extract the abstract topological knowledge of an expert human-design network (ResNeXt).
F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
Discover and Learn New Objects from Documentaries
Despite the remarkable progress in recent years, detecting objects in a new context remains a challenging task.
Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos
In the second stage, for each semantics, we randomly sample slots from the corresponding Gaussian distribution and perform masked feature aggregation within the semantic area to exploit temporal correspondence patterns for instance identification.
An Embarrassingly Simple Approach for Knowledge Distillation
Knowledge Distillation (KD) aims at improving the performance of a low-capacity student model by inheriting knowledge from a high-capacity teacher model.
Evolutionary Stochastic Policy Distillation
Solving the Goal-Conditioned Reward Sparse (GCRS) task is a challenging reinforcement learning problem due to the sparsity of reward signals.
Novel Policy Seeking with Constrained Optimization
In problem-solving, we humans can come up with multiple novel solutions to the same problem.
Peephole: Predicting Network Performance Before Training
The quest for performant networks has been a significant force that drives the advancements of deep learning in recent years.
Low-Latency Video Semantic Segmentation
Recent years have seen remarkable progress in semantic segmentation.
Ranked #6 on
Video Semantic Segmentation
on Cityscapes val
Accelerated Training for Massive Classification via Dynamic Class Selection
Massive classification, a classification task defined over a vast number of classes (hundreds of thousands or even millions), has become an essential part of many real-world systems, such as face recognition.
Learning Sparse Visual Representations with Leaky Capped Norm Regularizers
To the best of our knowledge, this is the first convergence analysis of the 3D recovery problem.
Be Your Own Prada: Fashion Synthesis with Structural Coherence
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
Contrastive Learning for Image Captioning
Specifically, via two constraints formulated on top of a reference model, the proposed method can encourage distinctiveness, while maintaining the overall quality of the generated captions.
Scalable Estimation of Dirichlet Process Mixture Models on Distributed Data
We consider the estimation of Dirichlet Process Mixture Models (DPMMs) in distributed environments, where data are distributed across multiple computing nodes.
Integrating Specialized Classifiers Based on Continuous Time Markov Chain
Specialized classifiers, namely those dedicated to a subset of classes, are often adopted in real-world recognition systems.
Adjustable Bounded Rectifiers: Towards Deep Binary Representations
This suggests that the semantic structure of a neural network may be manifested through a guided binarization process.
Generating Multi-Sentence Lingual Descriptions of Indoor Scenes
This paper proposes a novel framework for generating lingual descriptions of indoor scenes.
Move Forward and Tell: A Progressive Generator of Video Descriptions
We present an efficient framework that can generate a coherent paragraph to describe a given video.
Rethinking the Form of Latent States in Image Captioning
Taking advantage of this, we visually reveal the internal dynamics in the process of caption generation, as well as the connections between input visual domain and output linguistic domain.
Pose Guided Human Video Generation
Human pose, on the other hand, can represent motion patterns intrinsically and interpretably, and impose the geometric constraints regardless of appearance.
Generative Adversarial Frontal View to Bird View Synthesis
Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging.
Probabilistic Ensemble of Collaborative Filters
Collaborative filtering is an important technique for recommendation.
Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
Due to the expensive and time-consuming annotations (e. g., segmentation) for real-world images, recent works in computer vision resort to synthetic data.
Improving On-policy Learning with Statistical Reward Accumulation
Most methods in deep-RL achieve good results via the maximization of the reward signal provided by the environment, typically in the form of discounted cumulative returns.
Trajectory Convolution for Action Recognition
How to leverage the temporal dimension is a key question in video analysis.
Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation
To tackle this problem, we propose a Bayesian learning algorithm for DP mixture models.
Coupling Nonparametric Mixtures via Latent Dirichlet Processes
Mixture distributions are often used to model complex data.
Learning Globally Optimized Object Detector via Policy Gradient
In this paper, we propose a simple yet effective method to learn globally optimized detector for object detection, which is a simple modification to the standard cross-entropy gradient inspired by the REINFORCE algorithm.
Recognize Actions by Disentangling Components of Dynamics
Despite the remarkable progress in action recognition over the past several years, existing methods remain limited in efficiency and effectiveness.
Find and Focus: Retrieve and Localize Video Events with Natural Language Queries
The thriving of video sharing services brings new challenges to video retrieval, e. g. the rapid growth in video duration and content diversity.
Lifelong Learning via Progressive Distillation and Retrospection
Lifelong learning aims at adapting a learned model to new tasks while retaining the knowledge gained earlier.
Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis
For recovering 3D object poses from 2D images, a prevalent method is to pre-train an over-complete dictionary $\mathcal D=\{B_i\}_i^D$ of 3D basis poses.
Visual Semantic Search: Retrieving Videos via Complex Textual Queries
In this paper, we tackle the problem of retrieving videos using complex natural language queries.
What are You Talking About? Text-to-Image Coreference
In this paper we exploit natural sentential descriptions of RGB-D scenes in order to improve 3D semantic parsing.
Recognize Complex Events From Static Images by Fusing Deep Channels
A considerable portion of web images capture events that occur in our personal lives or social activities.
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
Open Compound Domain Adaptation
A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).
Biased Estimates of Advantages over Path Ensembles
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths.
A Graph-Based Framework to Bridge Movies and Synopses
On top of this dataset, we develop a framework to perform matching between movie segments and synopsis paragraphs.
Learning to Synthesize Fashion Textures
Another important observation is that fashion textures are multi-modal.
Learning a Decision Module by Imitating Driver's Control Behaviors
In this work, we propose a hybrid framework to learn neural decisions in the classical modular pipeline through end-to-end imitation learning.
Regularizing Reasons for Outfit Evaluation with Gradient Penalty
In this paper, we build an outfit evaluation system which provides feedbacks consisting of a judgment with a convincing explanation.
Learning Diverse Fashion Collocation by Neural Graph Filtering
To facilitate a comprehensive study on diverse fashion collocation, we reorganize Amazon Fashion dataset with carefully designed evaluation protocols.
Reconfigurable Voxels: A New Representation for LiDAR-Based Point Clouds
LiDAR is an important method for autonomous driving systems to sense the environment.
FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnastic videos.
Intra- and Inter-Action Understanding via Temporal Action Parsing
Current methods for action recognition primarily rely on deep convolutional networks to derive feature embeddings of visual and motion features.
Placepedia: Comprehensive Place Understanding with Multi-Faceted Annotations
Place is an important element in visual understanding.
Learn to Propagate Reliably on Noisy Affinity Graphs
Recent works have shown that exploiting unlabeled data through label propagation can substantially reduce the labeling cost, which has been a critical issue in developing visual recognition models.
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Stereophonic audio is an indispensable ingredient to enhance human auditory experience.
MovieNet: A Holistic Dataset for Movie Understanding
We believe that such a holistic dataset would promote the researches on story-based long video understanding and beyond.
A Unified Framework for Shot Type Classification Based on Subject Centric Lens
Shots are key narrative elements of various videos, e. g. movies, TV series, and user-generated videos that are thriving over the Internet.
Online Multi-modal Person Search in Videos
The task of searching certain people in videos has seen increasing potential in real-world applications, such as video organization and editing.
Caption-Supervised Face Recognition: Training a State-of-the-Art Face Model without Manual Annotation
Captioned images are widely available on the web, while the captions often contain the names of the subjects in the images.
FLAVA: Find, Localize, Adjust and Verify to Annotate LiDAR-Based Point Clouds
Recent years have witnessed the rapid progress of perception algorithms on top of LiDAR, a widely adopted sensor for autonomous driving systems.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
Visually Informed Binaural Audio Generation without Binaural Audios
Moreover, combined with binaural recordings, our method is able to further boost the performance of binaural audio generation under supervised settings.
WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection
The performance of object detection, to a great extent, depends on the availability of large annotated datasets.
Scene-aware Generative Network for Human Motion Synthesis
We revisit human motion synthesis, a task useful in various real world applications, in this paper.
Transcript to Video: Efficient Clip Sequencing from Texts
To meet the demands for non-experts, we present Transcript-to-Video -- a weakly-supervised framework that uses texts as input to automatically create video sequences from an extensive collection of shots.
Towards Balanced Learning for Instance Recognition
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
BlockPlanner: City Block Generation With Vectorized Graph Representation
City modeling is the foundation for computational urban planning, navigation, and entertainment.
3D Building Reconstruction From Monocular Remote Sensing Images
3D building reconstruction from monocular remote sensing imagery is an important research problem and an economic solution to large-scale city modeling, compared with reconstruction from LiDAR data and multi-view imagery.
INTERN: A New Learning Paradigm Towards General Vision
Enormous waves of technological innovations over the past several years, marked by the advances in AI technologies, are profoundly reshaping the industry and the society.
Learning with Social Influence through Interior Policy Differentiation
In this work we model the social influence into the scheme of reinforcement learning, enabling the agents to learn both from the environment and from their peers.
