Search Results for author:
Found 300 papers, 140 papers with code
GigaGS: Scaling up Planar-Based 3D Gaussians for Large Scene Surface Reconstruction
To this end, we propose GigaGS, the first work for high-quality surface reconstruction for large-scale scenes using 3DGS.
GenAgent: Build Collaborative AI Systems with Automated Workflow Generation -- Case Studies on ComfyUI
Much previous AI research has focused on developing monolithic models to maximize their intelligence and capability, with the primary goal of enhancing performance on specific tasks.
MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling
In an era of frequent extreme weather and global warming, obtaining precise, fine-grained near-surface weather forecasts is increasingly essential for human activities.
NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction
Signed Distance Function (SDF)-based volume rendering has demonstrated significant capabilities in surface reconstruction.
ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area
We benchmark ChemVLM against a range of open-source and proprietary multimodal large language models on various tasks.
Fast Information Streaming Handler (FisH): A Unified Seismic Neural Network for Single Station Real-Time Earthquake Early Warning
FisH is designed to process real-time streaming seismic data and generate simultaneous results for phase picking, location estimation, and magnitude estimation in an end-to-end fashion.
Point Transformer V3 Extreme: 1st Place Solution for 2024 Waymo Open Dataset Challenge in Semantic Segmentation
In this technical report, we detail our first-place solution for the 2024 Waymo Open Dataset Challenge's semantic segmentation track.
VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecasting
By capturing the uncertainties in vegetation changes and modeling the complex influence of relevant variables, VegeDiff outperforms existing deterministic methods, providing clear and accurate forecasting results of future vegetation states.
TCFormer: Visual Recognition via Token Clustering Transformer
Our dynamic tokens possess two crucial characteristics: (1) Representing image regions with similar semantic meanings using the same vision token, even if those regions are not adjacent, and (2) concentrating on regions with valuable details and represent them using fine tokens.
TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation
To fully exploit rich information hidden in long-term temporal point clouds and images, we present the Temporal Aggregation Network, termed TASeg.
Semi-supervised 3D Object Detection with PatchTeacher and PillarMix
In this paper, we propose PatchTeacher, which focuses on partial scene 3D object detection to provide high-quality pseudo labels for the student.
PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines
In this paper, we introduce PredBench, a benchmark tailored for the holistic evaluation of spatio-temporal prediction networks.
VEnhancer: Generative Space-Time Enhancement for Video Generation
Given a generated low-quality video, our approach can increase its spatial and temporal resolution simultaneously with arbitrary up-sampling space and time scales through a unified video diffusion model.
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models
Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks.
Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space
In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space.
Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
In this work, we identify a pitfall of vanilla iterative DPO - improved response quality can lead to increased verbosity.
BEACON: Benchmark for Comprehensive RNA Tasks and Language Models
RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms.
Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B
This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative integration of Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS), designed to enhance performance in complex mathematical reasoning tasks.
Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT
Lumina-T2X is a nascent family of Flow-based Large Diffusion Transformers that establishes a unified framework for transforming noise into various modalities, such as images and videos, conditioned on text instructions.
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision
Parameter-efficient fine-tuning (PEFT) has become increasingly important as foundation models continue to grow in both popularity and size.
FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilation
Data assimilation is a vital component in modern global medium-range weather forecasting systems to obtain the best estimation of the atmospheric state by combining the short-term forecast and observations.
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series
To improve the transparency of LLMs, the research community has formed to open-source truly open LLMs (e. g., Pythia, Amber, OLMo), where more details (e. g., pre-training corpus and training code) are being provided.
Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification
In the task-specific evaluation setting, gallery sets are categorized according to specific ReID tasks.
ORCA: A Global Ocean Emulator for Multi-year to Decadal Predictions
Hindcasts of key oceanic variables demonstrate ORCA's remarkable prediction skills in predicting ocean variations compared with state-of-the-art numerical OGCMs and abilities in capturing occurrences of extreme events at the subsurface ocean and ENSO vertical patterns.
EMR-Merging: Tuning-Free High-Performance Model Merging
In this paper, we rethink and analyze the existing model merging paradigm.
Dense Connector for MLLMs
We witness the rise of larger and higher-quality instruction datasets, as well as the involvement of larger-sized LLMs.
Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling
Data-driven artificial intelligence (AI) models have made significant advancements in weather forecasting, particularly in medium-range and nowcasting.
DocReLM: Mastering Document Retrieval with Language Model
With over 200 million published academic documents and millions of new documents being written each year, academic researchers face the challenge of searching for information within this vast corpus.
An Embarrassingly Simple Approach to Enhance Transformer Performance in Genomic Selection for Crop Breeding
Genomic selection (GS), as a critical crop breeding strategy, plays a key role in enhancing food production and addressing the global hunger crisis.
Physical formula enhanced multi-task learning for pharmacokinetics prediction
Overall, our work illustrates the benefits and potential of using PEMAL in AIDD and other scenarios with data scarcity and noise.
Taming Stable Diffusion for Text to 360° Panorama Image Generation
Generative models, e. g., Stable Diffusion, have enabled the creation of photorealistic images from text prompts.
How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks
This paper focuses on an early stage phase of deep learning research, prior to model development, and proposes a strategic framework for estimating the amount of annotated data required to train patch-based segmentation networks.
RS-Mamba for Large Remote Sensing Image Dense Prediction
RSM is specifically designed to capture the global context of remote sensing images with linear complexity, facilitating the effective processing of large VHR images.
Ranked #1 on
Road Segmentation
on Massachusetts Roads Dataset
(F1 metric)
Building change detection for remote sensing images
Change Detection
+1
RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
The ultimate goals of robotic learning is to acquire a comprehensive and generalizable robotic system capable of performing both seen skills within the training distribution and unseen skills in novel environments.
GVGEN: Text-to-3D Generation with Volumetric Representation
To better optimize the representation of these details, we propose a unique pruning and densifying method named the Candidate Pool Strategy, enhancing detail fidelity through selective optimization.
DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM
We present DetToolChain, a novel prompting paradigm, to unleash the zero-shot object detection ability of multimodal large language models (MLLMs), such as GPT-4V and Gemini.
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised man- ner.
Agent3D-Zero: An Agent for Zero-shot 3D Understanding
The ability to understand and reason the 3D real world is a crucial milestone towards artificial general intelligence.
PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest
In particular, our PoIFusion follows the paradigm of query-based object detection, formulating object queries as dynamic 3D boxes and generating a set of PoIs based on each query box.
LOCR: Location-Guided Transformer for Optical Character Recognition
Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR).
Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from these saliency maps and the significant inter-task correlation between saliency detection and semantic segmentation.
NeRF-Det++: Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection
We project the freely available 3D segmentation annotations onto the 2D plane and leverage the corresponding 2D semantic maps as the supervision signal, significantly enhancing the semantic awareness of multi-view detectors.
ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models
This paper introduces ConceptMath, a bilingual (English and Chinese), fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs).
MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues
By conducting a detailed analysis of real multi-turn dialogue data, we construct a three-tier hierarchical ability taxonomy comprising 4208 turns across 1388 multi-turn dialogues in 13 distinct tasks.
FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Model
We have released all the codes and models at https://github. com/whlzy/FiT to promote the exploration of diffusion transformer models for arbitrary-resolution image generation.
Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
Large language models (LLMs) undergo safety alignment to ensure safe conversations with humans.
Self-consistent Validation for Machine Learning Electronic Structure
Machine learning has emerged as a significant approach to efficiently tackle electronic structure problems.
Revealing Decurve Flows for Generalized Graph Propagation
\textbf{Secondly}, we propose the {\em Continuous Unified Ricci Curvature} (\textbf{CURC}), an extension of celebrated {\em Ollivier-Ricci Curvature} for directed and weighted graphs.
ChemLLM: A Chemical Large Language Model
However, the community lacks an LLM specifically designed for chemistry.
CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling
Precipitation nowcasting based on radar data plays a crucial role in extreme weather prediction and has broad implications for disaster management.
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning
These outcomes suggest that the 3D point cloud is a valuable observation modality for intricate robotic tasks.
Integration of cognitive tasks into artificial general intelligence test for large models
During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application.
ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast
Data-driven weather forecast based on machine learning (ML) has experienced rapid development and demonstrated superior performance in the global medium-range forecast compared to traditional physics-based dynamical models.
A Comprehensive Survey on 3D Content Generation
Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e. g., text, image, video, audio and 3D.
FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting
Therefore, building a kilometer-scale global forecast model is a persistent pursuit in the meteorology domain.
From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities
Multi-modal Large Language Models (MLLMs) have shown impressive abilities in generating reasonable responses with respect to multi-modal contents.
Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method
In this paper, we extend meteorological downscaling to arbitrary scattered station scales, establish a brand new benchmark and dataset, and retrieve meteorological states at any given station location from a coarse-resolution meteorological field.
Bilateral Reference for High-Resolution Dichotomous Image Segmentation
It comprises two essential components: the localization module (LM) and the reconstruction module (RM) with our proposed bilateral reference (BiRef).
Ranked #1 on
Camouflaged Object Segmentation
on COD
Camouflaged Object Segmentation
Dichotomous Image Segmentation
+3
Point Transformer V3: Simpler Faster Stronger
This paper is not motivated to seek innovation within the attention mechanism.
Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers
We first validate eight manually-defined partial fine-tuning strategies across kinds of datasets and vision transformer architectures, and find that some partial fine-tuning strategies (e. g., ffn only or attention only) can achieve better performance with fewer tuned parameters than full fine-tuning, and selecting appropriate layers is critical to partial fine-tuning.
Merging Vision Transformers from Different Tasks and Domains
This work targets to merge various Vision Transformers (ViTs) trained on different tasks (i. e., datasets with different object categories) or domains (i. e., datasets with the same categories but different environments) into one unified model, yielding still good performance on each task or domain.
Efficient Architecture Search via Bi-level Data Pruning
Improving the efficiency of Neural Architecture Search (NAS) is a challenging but significant task that has received much attention.
Towards an end-to-end artificial intelligence driven global weather forecasting system
Initial states are typically generated by traditional data assimilation components, which are computational expensive and time-consuming.
ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing
De novo peptide sequencing from mass spectrometry (MS) data is a critical task in proteomics research.
ResoNet: Robust and Explainable ENSO Forecasts with Hybrid Convolution and Transformer Networks
Recent studies have shown that deep learning (DL) models can skillfully predict the El Ni\~no-Southern Oscillation (ENSO) forecasts over 1. 5 years ahead.
FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation
Weather forecasting is a crucial yet highly challenging task.
Point Transformer V3: Simpler, Faster, Stronger
This paper is not motivated to seek innovation within the attention mechanism.
Ranked #1 on
3D Semantic Segmentation
on ScanNet++
(using extra training data)
UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation
Recent advancements in text-to-3D generation technology have significantly advanced the conversion of textual descriptions into imaginative well-geometrical and finely textured 3D objects.
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
In this paper, from a novel perspective, we systematically study the challenges that remain in O2O RL and identify that the reason behind the slow improvement of the performance and the instability of online finetuning lies in the inaccurate Q-value estimation inherited from offline pretraining.
Hulk: A Universal Knowledge Translator for Human-Centric Tasks
Human-centric perception tasks, e. g., pedestrian detection, skeleton-based action recognition, and pose estimation, have wide industrial applications, such as metaverse and sports analysis.
Ranked #1 on
Pedestrian Image Caption
on CUHK-PEDES
GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition?
Our study centers on the evaluation of GPT-4's linguistic and visual capabilities in zero-shot visual recognition tasks: Firstly, we explore the potential of its generated rich textual descriptions across various categories to enhance recognition performance without any training.
Point Cloud Pre-training with Diffusion Models
This generator aggregates the features extracted by the backbone and employs them as the condition to guide the point-to-point recovery from the noisy point cloud, thereby assisting the backbone in capturing both local and global geometric priors as well as the global point density distribution of the object.
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
While this phenomenon has been overlooked in previous work, we propose a novel and extensible framework, called Octavius, for comprehensive studies and experimentation on multimodal learning with Multimodal Large Language Models (MLLMs).
GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection
It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning.
I$^2$MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
Different from existing distillation solutions that transfer the knowledge of a pre-trained and fixed teacher to the student, in CMD, the knowledge is continuously updated and bidirectionally distilled between modalities during pre-training.
MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements.
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm
In this paper, we introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation, thereby establishing a pathway to 3D foundational models.
Ranked #2 on
Semantic Segmentation
on S3DIS
(using extra training data)
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
In the context of autonomous driving, the significance of effective feature learning is widely acknowledged.
Toward Understanding BERT-Like Pre-Training for DNA Foundation Models
However, existing pre-training methods for DNA sequences largely rely on direct adoptions of BERT pre-training from NLP, lacking a comprehensive understanding and a specifically tailored approach.
Towards Fair and Comprehensive Comparisons for Image-Based 3D Object Detection
In this work, we build a modular-designed codebase, formulate strong training recipes, design an error diagnosis toolbox, and discuss current methods for image-based 3D object detection.
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
A single language model, even when aligned with labelers through reinforcement learning from human feedback (RLHF), may not suit all human preferences.
Understanding Masked Autoencoders From a Local Contrastive Perspective
Masked AutoEncoder (MAE) has revolutionized the field of self-supervised learning with its simple yet effective masking and reconstruction strategies.
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters.
NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space
Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs.
3D Semantic Scene Completion from a single 2D image
3D Semantic Scene Completion from a single RGB image
+1
Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training
Our approach enables collaboration among multiple clinical sites without compromising data privacy under a federated learning paradigm that incorporates a noise-robust training strategy based on label correction.
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior
We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks in a unified framework.
Ranked #1 on
Blind Face Restoration
on LFW
Boosting Residual Networks with Group Knowledge
Specifically, we implicitly divide all subnets into hierarchical groups by subnet-in-subnet sampling, aggregate the knowledge of different subnets in each group during training, and exploit upper-level group knowledge to supervise lower-level subnet groups.
STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning
Scale variation is a deep-rooted problem in object counting, which has not been effectively addressed by existing scale-aware algorithms.
Masked Motion Predictors are Strong 3D Action Representation Learners
To be specific, the proposed MAMP takes as input the masked spatio-temporal skeleton sequence and predicts the corresponding temporal motion of the masked human joints.
Ranked #8 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Experts Weights Averaging: A New General Training Scheme for Vision Transformers
As Vision Transformers (ViTs) are gradually surpassing CNNs in various visual tasks, one may question: if a training scheme specifically for ViTs exists that can also achieve performance improvement without increasing inference cost?
MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation
Building upon the observation that the attended regions of the one-class token in the standard vision transformer can contribute to a class-agnostic localization map, we explore the potential of the transformer model to capture class-specific attention for class-discriminative object localization by learning multiple class tokens.
Object Localization
Weakly supervised Semantic Segmentation
+1
Pro-PRIME: A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins
Designing protein mutants of both high stability and activity is a critical yet challenging task in protein engineering.
Theoretically Guaranteed Policy Improvement Distilled from Model-Based Planning
Model-based reinforcement learning (RL) has demonstrated remarkable successes on a range of continuous control tasks due to its high sample efficiency.
Meta-Transformer: A Unified Framework for Multimodal Learning
Multimodal learning aims to build models that can process and relate information from multiple modalities.
What Can Simple Arithmetic Operations Do for Temporal Modeling?
We conduct comprehensive ablation studies on the instantiation of ATMs and demonstrate that this module provides powerful temporal modeling capability at a low computational cost.
Ranked #4 on
Action Recognition
on Something-Something V1
UniG3D: A Unified 3D Object Generation Dataset
However, the quality and diversity of existing 3D object generation methods are constrained by the inadequacies of existing 3D object datasets, including issues related to text quality, the incompleteness of multi-modal data representation encompassing 2D rendered images and 3D assets, as well as the size of the dataset.
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans.
Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting
In this work, we propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting by exploiting the spatial hierarchy and modeling multi-scale spatial correlations.
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions
This paper strives to resolve this problem by proposing a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions.
LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
To the best of our knowledge, we present one of the very first open-source endeavors in the field, LAMM, encompassing a Language-Assisted Multi-Modal instruction tuning dataset, framework, and benchmark.
Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection
LiDAR and Radar are two complementary sensing approaches in that LiDAR specializes in capturing an object's 3D shape while Radar provides longer detection ranges as well as velocity hints.
PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer
The key to associating the two different representations is our introduced input-dependent Query Initialization module, which could efficiently generate reference points and content queries.
Clothes-Invariant Feature Learning by Causal Intervention for Clothes-Changing Person Re-identification
Clothes-invariant feature extraction is critical to the clothes-changing person re-identification (CC-ReID).
Stimulative Training++: Go Beyond The Performance Limits of Residual Networks
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training scheme as well as three improved strategies for boosting residual networks beyond their performance limits.
FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI).
Automatically Predict Material Properties with Microscopic Image Example Polymer Compatibility
Computer image recognition with machine learning method can make up the defects of artificial judging, giving accurate and quantitative judgement.
HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining
Specifically, we propose a \textbf{HumanBench} based on existing datasets to comprehensively evaluate on the common ground the generalization abilities of different pretraining methods on 19 datasets from 6 diverse downstream tasks, including person ReID, pose estimation, human parsing, pedestrian attribute recognition, pedestrian detection, and crowd counting.
Ranked #1 on
Pedestrian Attribute Recognition
on PA-100K
(using extra training data)
UniHCP: A Unified Model for Human-Centric Perceptions
When adapted to a specific task, UniHCP achieves new SOTAs on a wide range of human-centric tasks, e. g., 69. 8 mIoU on CIHP for human parsing, 86. 18 mA on PA-100K for attribute prediction, 90. 3 mAP on Market1501 for ReID, and 85. 8 JI on CrowdHuman for pedestrian detection, performing better than specialized models tailored for each task.
Ranked #1 on
Pose Estimation
on MS-COCO
Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phase
As past poses often contain useful auxiliary hints, in this paper, we propose a task-agnostic deep learning method, namely Multi-scale Control Signal-aware Transformer (MCS-T), with an attention based encoder-decoder architecture to discover the auxiliary information implicitly for synthesizing controllable motion without explicitly requiring auxiliary information such as phase.
Saliency Guided Contrastive Learning on Scene Images
Despite being feasible, recent works largely overlooked discovering the most discriminative regions for contrastive learning to object representations in scene images.
Learning from pseudo-labels: deep networks improve consistency in longitudinal brain volume estimation
An accurate and robust quantitative measurement of brain volume change is paramount for translational research and clinical applications.
Fast-BEV: A Fast and Strong Bird's-Eye View Perception Baseline
Our Fast-BEV consists of five parts, We novelly propose (1) a lightweight deployment-friendly view transformation which fast transfers 2D image feature to 3D voxel space, (2) an multi-scale image encoder which leverages multi-scale information for better performance, (3) an efficient BEV encoder which is particularly designed to speed up on-vehicle inference.
$β$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search
To address the robustness problem, we first benchmark different NAS methods under a wide range of proxy data, proxy channels, proxy layers and proxy epochs, since the robustness of NAS under different kinds of proxies has not been explored before.
Semi-Supervised Semantic Segmentation under Label Noise via Diverse Learning Groups
Each learning group consists of a teacher network, a student network and a novel filter module.
Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization
Weakly supervised dense object localization (WSDOL) relies generally on Class Activation Mapping (CAM), which exploits the correlation between the class weights of the image classifier and the pixel-level features.
Crossing the Gap: Domain Generalization for Image Captioning
In this paper, we introduce a new setting called Domain Generalization for Image Captioning (DGIC), where the data from the target domain is unseen in the learning process.
Open-Set Fine-Grained Retrieval via Prompting Vision-Language Evaluator
PLEor could leverage pre-trained CLIP model to infer the discrepancies encompassing both pre-defined and unknown subcategories, called category-specific discrepancies, and transfer them to the backbone network trained in the close-set scenarios.
Ponder: Point Cloud Pre-training via Neural Rendering
We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering.
Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models
In this paper, we propose a novel framework called BIKE, which utilizes the cross-modal bridge to explore bidirectional knowledge: i) We introduce the Video Attribute Association mechanism, which leverages the Video-to-Text knowledge to generate textual auxiliary attributes for complementing video recognition.
Ranked #1 on
Zero-Shot Action Recognition
on ActivityNet
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?
Most existing text-video retrieval methods focus on cross-modal matching between the visual content of videos and textual query sentences.
Ranked #7 on
Video Retrieval
on VATEX
MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency
Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas.
3D Point Cloud Pre-training with Knowledge Distillation from 2D Images
The recent success of pre-trained 2D vision models is mostly attributable to learning from large-scale datasets.
EPCL: Frozen CLIP Transformer is An Efficient Point Cloud Encoder
These token embeddings are concatenated with a task token and fed into the frozen CLIP transformer to learn point cloud representation.
GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds
In contrast to previous 3D MAE frameworks, which either design a complex decoder to infer masked information from maintained regions or adopt sophisticated masking strategies, we instead propose a much simpler paradigm.
Reconstructing Hand-Held Objects from Monocular Video
The key idea is that the hand motion naturally provides multiple views of the object and the motion can be reliably estimated by a hand pose tracker.
ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency
In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action.
Ranked #1 on
SMAC
on SMAC 3s5z_vs_3s6z
3D-QueryIS: A Query-based Framework for 3D Instance Segmentation
Previous top-performing methods for 3D instance segmentation often maintain inter-task dependencies and the tendency towards a lack of robustness.
Boosting Semi-Supervised 3D Object Detection with Semi-Sampling
When training a teacher-student semi-supervised framework, we randomly select gt samples and pseudo samples to both labeled frames and unlabeled frames, making a strong data augmentation for them.
The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition
Long-tail distribution is widely spread in real-world applications.
Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training strategy to strengthen the performance of residual networks.
CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training
To address this issue, we propose CLIP2Point, an image-depth pre-training method by contrastive learning to transfer CLIP to the 3D domain, and adapt it to point cloud classification.
Ranked #3 on
Training-free 3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Towards Frame Rate Agnostic Multi-Object Tracking
In this paper, we propose a Frame Rate Agnostic MOT framework with a Periodic training Scheme (FAPS) to tackle the FraMOT problem for the first time.
ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
We propose a single-network approach, termed ZoomNet, to take into account the hierarchical structure of the full human body and solve the scale variation of different body parts.
Ranked #5 on
2D Human Pose Estimation
on COCO-WholeBody
An Empirical Study of Pseudo-Labeling for Image-based 3D Object Detection
We hope this work can provide insights for the image-based 3D detection community under a semi-supervised setting.
Fine-grained Retrieval Prompt Tuning
In this paper, we develop Fine-grained Retrieval Prompt Tuning (FRPT), which steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompting and feature adaptation.
3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal
Unlike most previous works that directly predict the 3D poses of two interacting hands simultaneously, we propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately.