Search Results for author:
Found 218 papers, 135 papers with code
Hallucinating Visual Instances in Total Absentia
This seemingly minor difference in fact makes the HVITA a much challenging task, as the restoration algorithm would have to not only infer the category of the object in total absentia, but also hallucinate an object of which the appearance is consistent with the background.
Collaboration by Competition: Self-coordinated Knowledge Amalgamation for Multi-talent Student Learning
To this end, we propose a self-coordinate knowledge amalgamation network (SOKA-Net) for learning the multi-talent student model.
Discrete Diffusion in Large Language and Multimodal Models: A Survey
In this work, we provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs).
Test3R: Learning to Reconstruct 3D at Test Time
Extensive experiments demonstrate that our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multi-view depth estimation tasks.
Diversity-Guided MLP Reduction for Efficient Large Vision Transformers
To this end, we focus on the recoverability of the compressed models and propose a Diversity-Guided MLP Reduction (DGMR) method to significantly reduce the parameters of large vision transformers with only negligible performance degradation.
PixelThink: Towards Efficient Chain-of-Pixel Reasoning
Existing reasoning segmentation approaches typically fine-tune multimodal large language models (MLLMs) using image-text pairs and corresponding mask labels.
Minute-Long Videos with Dual Parallelisms
Diffusion Transformer (DiT)-based video diffusion models generate high-quality videos at scale but incur prohibitive processing latency and memory costs for long videos.
Can MLLMs Guide Me Home? A Benchmark Study on Fine-Grained Visual Reasoning from Transit Maps
Multimodal large language models (MLLMs) have recently achieved significant progress in visual tasks, including semantic scene understanding and text-image alignment, with reasoning variants enhancing performance on complex tasks involving mathematics and logic.
VeriThinker: Learning to Verify Makes Reasoning Model Efficient
Extensive experiments validate that VeriThinker substantially reduces reasoning chain lengths while maintaining or even slightly improving accuracy.
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
We observe that training with a purely discrete diffusion approach leads to significant training instability, suboptimal performance, and severe length bias issues.
dKV-Cache: The Cache for Diffusion Language Models
Our approach is motivated by the observation that different tokens have distinct representation dynamics throughout the diffusion process.
Thinkless: LLM Learns When to Think
Reasoning Language Models, capable of extended chain-of-thought reasoning, have demonstrated remarkable performance on tasks requiring complex logical inference.
Top-Down Compression: Revisit Efficient Vision Token Projection for Visual Instruction Tuning
Visual instruction tuning aims to enable large language models to comprehend the visual world, with a pivotal challenge lying in establishing an effective vision-to-language projection.
PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning
Self-supervised representation learning for point cloud has demonstrated effectiveness in improving pre-trained model performance across diverse tasks.
Efficient Reasoning Models: A Survey
Reasoning models have demonstrated remarkable progress in solving complex and logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to arriving at a final answer.
1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering
(Q1) \textbf{Short-Lifespan Gaussians}: 4DGS uses a large portion of Gaussians with short temporal span to represent scene dynamics, leading to an excessive number of Gaussians.
POSTA: A Go-to Framework for Customized Artistic Poster Generation
To train our models, we develop the PosterArt dataset, comprising high-quality artistic posters annotated with layout, typography, and pixel-level stylized text segmentation.
OminiControl2: Efficient Conditioning for Diffusion Transformers
Fine-grained control of text-to-image diffusion transformer models (DiT) remains a critical challenge for practical deployment.
PE3R: Perception-Efficient 3D Reconstruction
PE3R employs a feed-forward architecture to enable rapid 3D semantic field reconstruction.
Understanding Dataset Distillation via Spectral Filtering
To address this limitation, we further propose Curriculum Frequency Matching (CFM), which gradually adjusts the filter parameter to cover both low- and high-frequency information of the FFC and FLC matrices.
Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling
In response, we introduce Efficient Dynamic Gaussian Splatting (EDGS), which represents dynamic scenes via sparse time-variant attribute modeling.
GraphBridge: Towards Arbitrary Transfer Learning in GNNs
Empirical validation, conducted over 16 datasets representative of these scenarios, confirms the framework's capacity for task- and domain-agnostic transfer learning within graph-like data, marking a significant advancement in the field of GNNs.
Introducing Visual Perception Token into Multimodal Large Language Model
The Region Selection Token explicitly identifies specific regions in an image that require further perception, while the Vision Re-Encoding Token uses its hidden states as control signals to guide additional visual perception processes.
CoT-Valve: Length-Compressible Chain-of-Thought Tuning
Moreover, we show that this property is valuable for compressing the reasoning chain.
Seeing World Dynamics in a Nutshell
We consider the problem of efficiently representing casually captured monocular videos in a spatially- and temporally-coherent manner.
Few-shot Implicit Function Generation via Equivariance
To tackle this, we propose EquiGen, a framework that can generate new INRs from limited data.
Event-based Video Super-Resolution via State Space Models
In this paper, we introduce MamEVSR, a Mamba-based network for event-based VSR that leverages the selective state space model, Mamba.
Diffusion Model is Effectively Its Own Teacher
We further validate the effectiveness of our method on text-to-image diffusion models, such as Stable Diffusion, and also observe notable improvement in image quality.
Generative Sparse-View Gaussian Splatting
Novel view synthesis from limited observations remains a significant challenge due to the lack of information in under-sampled regions, often resulting in noticeable artifacts.
CoSER: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation
To further enhance cross-view consistency and alleviate content drift, CoSER rapidly scan all views in spiral bidirectional manner to aware holistic information and then scores each point based on semantic material.
CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up
Diffusion Transformers (DiT) have become a leading architecture in image generation.
Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising
The goal of Remix-DiT is to craft N diffusion experts for different denoising timesteps, yet without the need for expensive training of N independent models.
One-shot Federated Learning via Synthetic Distiller-Distillate Communication
Additionally, they may encounter scalability issues with complex datasets due to inherent two-step information loss: first, during local training (from data to model), and second, when transferring knowledge to the server model (from model to inversed data).
TinyFusion: Diffusion Transformers Learned Shallow
In this work, we present TinyFusion, a depth pruning method designed to remove redundant layers from diffusion transformers via end-to-end learning.
Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective
Self-supervised heterogeneous graph learning (SHGL) has shown promising potential in diverse scenarios.
Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient
The large model serves as the 'drafter', specializing in generating low-frequency content at smaller scales, while the smaller model serves as the 'refiner', solely focusing on predicting high-frequency details at larger scales.
OminiControl: Minimal and Universal Control for Diffusion Transformer
In this paper, we introduce OminiControl, a highly versatile and parameter-efficient framework that integrates image conditions into pre-trained Diffusion Transformer (DiT) models.
HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters
We further design a label-propagated contrastive loss and two self-supervised losses to optimize dual adapters and incorporate unlabeled nodes as potential labeled data.
Adversarial Training: A Survey
Adversarial training (AT) refers to integrating adversarial examples -- inputs altered with imperceptible perturbations that can significantly impact model predictions -- into the training process.
Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matching
Addressing these concerns, this paper introduces Teddy, a Taylor-approximated dataset distillation framework designed to handle large-scale dataset and enhance efficiency.
Poison-splat: Computation Cost Attack on 3D Gaussian Splatting
However, in this work, we reveal a significant security vulnerability that has been largely overlooked in 3DGS: the computation cost of training 3DGS could be maliciously tampered by poisoning the input data.
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
This approach facilitates end-to-end training on large-scale datasets and offers two notable advantages: 1) High-quality Masks - our method effectively scales to large datasets and learns accurate masks; 2) Transferability - the probabilistic modeling of mask distribution enables the transfer learning of sparsity across domains or tasks.
Attention Prompting on Image for Large Vision-Language Models
To fill this gap, in this work, we propose a new prompting technique named Attention Prompting on Image, which just simply overlays a text-query-guided attention heatmap on the original input image and effectively enhances LVLM on various tasks.
Vista3D: Unravel the 3D Darkside of a Single Image
We embark on the age-old quest: unveiling the hidden dimensions of objects from mere glimpses of their visible parts.
Kolmogorov-Arnold Transformer
In this paper, we introduce the Kolmogorov-Arnold Transformer (KAT), a novel architecture that replaces MLP layers with Kolmogorov-Arnold Network (KAN) layers to enhance the expressiveness and performance of the model.
Ranked #492 on
Image Classification
on ImageNet
FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally
Extensive experiments demonstrate the efficiency and robustness of our method in segmenting various scenes, and its superior performance in downstream tasks such as object removal and inpainting.
IFAdapter: Instance Feature Control for Grounded Text-to-Image Generation
The Layout-to-Image (L2I) task was introduced to address the positioning challenges by incorporating bounding boxes as spatial control signals, but it still falls short in generating precise instance features.
LinFusion: 1 GPU, 1 Minute, 16K Image
Modern diffusion models, particularly those utilizing a Transformer-based UNet for denoising, rely heavily on self-attention operations to manage complex spatial relationships, thus achieving impressive generation performance.
Dual-Path Adversarial Lifting for Domain Shift Correction in Online Test-time Adaptation
Specifically, we introduce an extra token, referred to as \textit{domain shift token}, at each layer of the transformer network.
Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation
To further enhance cross-view consistency and alleviate content drift, CoSER rapidly scan all views in spiral bidirectional manner to aware holistic information and then scores each point based on semantic material.
Heavy Labels Out! Dataset Distillation with Label Space Lightening
Extensive experiments demonstrate that with only about 0. 003% of the original storage required for a complete set of soft labels, we achieve comparable performance to current state-of-the-art dataset distillation methods on large-scale datasets.
MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities
Using MM-Vet v2 to benchmark large multimodal models, we found that Claude 3. 5 Sonnet is the best model with a score of 71. 8, slightly outperforming GPT-4o which scored 71. 0.
Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-Light Conditions
Our primary novelty lies in leveraging two complementary-teacher networks to generate more reliable pseudo labels, enabling our model achieves competitive performance on extremely low-light images without the need for training with low-light ground truths.
Ranked #1 on
2D Human Pose Estimation
on ExLPose-LL-E
Encapsulating Knowledge in One Prompt
This paradigm encapsulates knowledge from various models into a solitary prompt without altering the original models or requiring access to the training data, which enables us to achieve efficient and convenient knowledge transfer in more realistic scenarios.
LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesis
Latent diffusion models have shown promising results in audio generation, making notable advancements over traditional methods.
Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach
The synthesized query equipped with task-specific knowledge serves to extract the useful features for downstream tasks from the intermediate representations of the pre-trained model in a query-only manner.
Isomorphic Pruning for Vision Models
For instance, we improve the accuracy of DeiT-Tiny from 74. 52% to 77. 50% by pruning an off-the-shelf DeiT-Base model.
Video-Infinity: Distributed Long Video Generation
Specifically, we propose two coherent mechanisms: Clip parallelism and Dual-scope attention.
Neural Lineage
Given a well-behaved neural network, is possible to identify its parent, based on which it was tuned?
Few-Shot Anomaly Detection via Category-Agnostic Registration Learning
At test time, an image and its corresponding support set, consisting of a few normal images from the same category, are supplied, and anomalies are identified by comparing the registered features of the test image to its corresponding support image features.
AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
To address this, we introduce AsyncDiff, a universal and plug-and-play acceleration scheme that enables model parallelism across multiple devices.
Compositional Video Generation as Flow Equalization
Despite the promising results, a significant challenge remains: these models struggle to fully grasp complex compositional interactions between multiple concepts and actions.
MVGamba: Unify 3D Content Generation as State Space Sequence Modeling
Recent 3D large reconstruction models (LRMs) can generate high-quality 3D content in sub-seconds by integrating multi-view diffusion models with scalable multi-view reconstructors.
Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
In this study, we make an interesting and somehow surprising observation: the computation of a large proportion of layers in the diffusion transformer, through introducing a caching mechanism, can be readily removed even without updating the model parameters.
GFlow: Recovering 4D World from Monocular Video
To solve this, we introduce GFlow, a new framework that utilizes only 2D priors (depth and optical flow) to lift a video to a 4D scene, as a flow of 3D Gaussians through space and time.
MambaOut: Do We Really Need Mamba for Vision?
For vision tasks, as image classification does not align with either characteristic, we hypothesize that Mamba is not necessary for this task; Detection and segmentation tasks are also not autoregressive, yet they adhere to the long-sequence characteristic, so we believe it is still worthwhile to explore Mamba's potential for these tasks.
DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation
User-friendly 3D object editing is a challenging task that has attracted significant attention recently.
Distilled Datamodel with Reverse Gradient Matching
To investigate the impact of changes in training data on a pre-trained model, a common approach is leave-one-out retraining.
Ungeneralizable Examples
The training of contemporary deep learning models heavily relies on publicly available data, posing a risk of unauthorized access to online data and raising concerns about data privacy.
MindBridge: A Cross-Subject Brain Decoding Framework
Currently, brain decoding is confined to a per-subject-per-model paradigm, limiting its applicability to the same individual for whom the decoding model is trained.
Hash3D: Training-free Acceleration for 3D Generation
The evolution of 3D generative modeling has been notably propelled by the adoption of 2D diffusion models.
Unsegment Anything by Simulating Deformation
Foundation segmentation models, while powerful, pose a significant risk: they enable users to effortlessly extract any objects from any digital content with a single click, potentially leading to copyright infringement or malicious misuse.
Relation Rectification in Diffusion Model
Despite their exceptional generative abilities, large text-to-image diffusion models, much like skilled but careless artists, often struggle with accurately depicting visual relationships between objects.
Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction
We tackle the challenge of efficiently reconstructing a 3D asset from a single image at millisecond speed.
Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images
Recent advancements in large-scale visual-language pre-trained models have led to significant progress in zero-/few-shot anomaly detection within natural image domains.
Through the Dual-Prism: A Spectral Perspective on Graph Data Augmentation for Graph Classification
Graph Neural Networks have become the preferred tool to process graph data, with their efficacy being boosted through graph data augmentation techniques.
Mutual-modality Adversarial Attack with Semantic Perturbation
Our approach outperforms state-of-the-art attack methods and can be readily deployed as a plug-and-play solution.
DreamDrone: Text-to-Image Diffusion Models are Zero-shot Perpetual View Generators
We introduce DreamDrone, a novel zero-shot and training-free pipeline for generating unbounded flythrough scenes from textual prompts.
SlimSAM: 0.1% Data Makes Segment Anything Slim
To address this challenging trade-off, we introduce SlimSAM, a novel data-efficient SAM compression method that achieves superior performance with extremely less training data.
Generator Born from Classifier
In this paper, we make a bold attempt toward an ambitious task: given a pre-trained classifier, we aim to reconstruct an image generator, without relying on any data samples.
DeepCache: Accelerating Diffusion Models for Free
Diffusion models have recently gained unprecedented attention in the field of image synthesis due to their remarkable generative capabilities.
Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models
However, performance advancements are limited when relying solely on intricate algorithmic designs for a single model, even one exhibiting strong performance, e. g., CLIP-ViT-B/16.
Ranked #2 on
Prompt Engineering
on ImageNet
C-Procgen: Empowering Procgen with Controllable Contexts
We present C-Procgen, an enhanced suite of environments on top of the Procgen benchmark.
GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph
To mitigate that, we propose an effective adapter-style tuning strategy, dubbed GraphAdapter, which performs the textual adapter by explicitly modeling the dual-modality structure knowledge (i. e., the correlation of different semantics/classes in textual and visual modalities) with a dual knowledge graph.
Priority-Centric Human Motion Generation in Discrete Latent Space
We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token within the entire motion sequence.
SG-Former: Self-guided Transformer with Evolving Token Reallocation
At the heart of our approach is to utilize a significance map, which is estimated through hybrid-scale self-attention and evolves itself during training, to reallocate tokens based on the significance of each region.
Diffusion Model as Representation Learner
In this paper, we conduct an in-depth investigation of the representation power of DPMs, and propose a novel knowledge transfer method that leverages the knowledge acquired by generative DPMs for recognition tasks.
Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey
Image restoration (IR) has been an indispensable and challenging task in the low-level vision field, which strives to improve the subjective quality of images distorted by various forms of degradation.
Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction
To work with auxiliary tasks, we propose a novel auxiliary-adapted transformer, which can handle incomplete, corrupted motion data and achieve coordinate recovery via capturing spatial-temporal dependencies.
Ranked #4 on
Human Pose Forecasting
on Human3.6M
PARTNER: Level up the Polar Representation for LiDAR 3D Object Detection
Recently, polar-based representation has shown promising properties in perceptual tasks.
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
Problems include: (1) How to systematically structure and evaluate the complicated multimodal tasks; (2) How to design evaluation metrics that work well across question and answer types; and (3) How to give model insights beyond a simple performance ranking.
PseudoCal: A Source-Free Approach to Unsupervised Uncertainty Calibration in Domain Adaptation
The conventional in-domain calibration method, \textit{temperature scaling} (TempScal), encounters challenges due to domain distribution shifts and the absence of labeled target domain data.
Distribution Shift Inversion for Out-of-Distribution Prediction
Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing distribution by searching for a unified predictor or invariant feature representation.
Evolving Knowledge Mining for Class Incremental Segmentation
In this paper, we for the first time investigate the efficient multi-grained knowledge reuse for CISS, and propose a novel method, Evolving kNowleDge minING (ENDING), employing a frozen backbone.
Class-Incremental Semantic Segmentation
Knowledge Distillation
Lens-to-lens bokeh effect transformation. NTIRE 2023 challenge report
We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge.
LLM-Pruner: On the Structural Pruning of Large Language Models
With LLM being a general-purpose task solver, we explore its compression in a task-agnostic manner, which aims to preserve the multi-task solving and language generation ability of the original LLM.
Structural Pruning for Diffusion Models
Generative modeling has recently undergone remarkable advancements, primarily propelled by the transformative implications of Diffusion Probabilistic Models (DPMs).
Can SAM Boost Video Super-Resolution?
To use the SAM-based prior, we propose a simple yet effective module -- SAM-guidEd refinEment Module (SEEM), which can enhance both alignment and fusion procedures by the utilization of semantic information.
Deep Graph Reprogramming
In this paper, we explore a novel model reusing task tailored for graph neural networks (GNNs), termed as "deep graph reprogramming".
Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfer
On the other hand, different from the vanilla version, we adopt a learnable scaling operation on content features before content-style feature interaction, which better preserves the original similarity between a pair of content features while ensuring the stylization quality.
Segment Anything in Non-Euclidean Domains: Challenges and Opportunities
The recent work known as Segment Anything (SA) has made significant strides in pushing the boundaries of semantic segmentation into the era of foundation models.
Anything-3D: Towards Single-view Anything Reconstruction in the Wild
3D reconstruction from a single-RGB image in unconstrained real-world scenarios presents numerous challenges due to the inherent diversity and complexity of objects and environments.
Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate
Existing approaches either apply the holistic style of the style image in a global manner, or migrate local colors and textures of the style image to the content counterparts in a pre-defined way.
Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation
The whole temporal neighborhood of nodes reveals the varying preferences of nodes.
TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration
We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities.
Ranked #3 on
Motion Synthesis
on AIST++
InceptionNeXt: When Inception Meets ConvNeXt
Inspired by the long-range modeling ability of ViTs, large-kernel convolutions are widely studied and adopted recently to enlarge the receptive field and improve model performance, like the remarkable work ConvNeXt which employs 7x7 depthwise convolution.
EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning
In motion prediction tasks, maintaining motion equivariance under Euclidean geometric transformations and invariance of agent interaction is a critical and fundamental principle.
Ranked #1 on
Human Pose Forecasting
on HARPER
Partial Network Cloning
Unlike prior methods that update all or at least part of the parameters in the target network throughout the knowledge transfer process, PNC conducts partial parametric "cloning" from a source network and then injects the cloned module to the target, without modifying its parameters.
DepGraph: Towards Any Structural Pruning
Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks.
Dataset Distillation: A Comprehensive Review
Recent success of deep learning is largely attributed to the sheer amount of data used for training deep neural networks. Despite the unprecedented success, the massive data, unfortunately, significantly increases the burden on storage and transmission and further gives rise to a cumbersome model training process.
Slimmable Dataset Condensation
In this paper, we explore the problem of slimmable dataset condensation, to extract a smaller synthetic dataset given only previous condensation results.
Few-Shot Dataset Distillation via Translative Pre-Training
We pre-train the translator on some large datasets like ImageNet so that it requires only a limited number of adaptation steps on the target dataset.
Diffusion Probabilistic Model Made Slim
Despite the recent visually-pleasing results achieved, the massive computational cost has been a long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, greatly limits their applications on resource-limited platforms.
AvatarGen: A 3D Generative Model for Animatable Human Avatars
Specifically, we decompose the generative 3D human synthesis into pose-guided mapping and canonical representation with predefined human pose and shape, such that the canonical representation can be explicitly driven to different poses and shapes with the guidance of a 3D parametric human model SMPL.
Task Residual for Tuning Vision-Language Models
Large-scale vision-language models (VLMs) pre-trained on billion-level data have learned general visual representations and broad visual concepts.
Dataset Factorization for Condensation
In this paper, we study dataset distillation (DD), from a novel perspective and introduce a \emph{dataset factorization} approach, termed \emph{HaBa}, which is a plug-and-play strategy portable to any existing DD baseline.
Dataset Distillation via Factorization
In this paper, we study \xw{dataset distillation (DD)}, from a novel perspective and introduce a \emph{dataset factorization} approach, termed \emph{HaBa}, which is a plug-and-play strategy portable to any existing DD baseline.
MetaFormer Baselines for Vision
By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85. 5% at 224x224 resolution, under normal supervised training without external data or distillation.
Ranked #2 on
Domain Generalization
on ImageNet-C
(using extra training data)
Deep Model Reassembly
Given a collection of heterogeneous models pre-trained from distinct sources and with diverse architectures, the goal of DeRy, as its name implies, is to first dissect each model into distinctive building blocks, and then selectively reassemble the derived blocks to produce customized networks under both the hardware resource and performance constraints.
Reachability-Aware Laplacian Representation in Reinforcement Learning
In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a task-agnostic state representation that encodes the geometry of the environment.
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
With the proposed SSF, our model obtains 2. 46% (90. 72% vs. 88. 54%) and 11. 48% (73. 10% vs. 65. 57%) performance improvement on FGVC and VTAB-1k in terms of Top-1 accuracy compared to the full fine-tuning but only fine-tuning about 0. 3M parameters.
Training Spiking Neural Networks with Local Tandem Learning
The LTL rule follows the teacher-student learning approach by mimicking the intermediate feature representations of a pre-trained ANN.
Attention Diversification for Domain Generalization
Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features.
Bottom-Up 2D Pose Estimation via Dual Anatomical Centers for Small-Scale Persons
In multi-person 2D pose estimation, the bottom-up methods simultaneously predict poses for all persons, and unlike the top-down methods, do not rely on human detection.
AvatarGen: a 3D Generative Model for Animatable Human Avatars
Unsupervised generation of clothed virtual humans with various appearance and animatable poses is important for creating 3D human avatars and other AR/VR applications.
Federated Selective Aggregation for Knowledge Amalgamation
The goal of FedSA is to train a student model for a new task with the help of several decentralized teachers, whose pre-training tasks and data are different and agnostic.
Learning Graph Neural Networks for Image Style Transfer
To this end, we develop an elaborated GNN model with content and style local patches as the graph vertices.
Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection
Anomaly detection aims at identifying deviant samples from the normal data distribution.
Learning with Recoverable Forgetting
Life-long learning aims at learning a sequence of tasks without forgetting the previously acquired knowledge.
DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation
Prior approaches, despite the promising results, have relied on either estimating dense attention to compute per-point matching, which is limited to only coarse scales due to the quadratic memory cost, or fixing the number of correspondences to achieve linear complexity, which lacks flexibility.
Factorizing Knowledge in Neural Networks
The core idea of KF lies in the modularization and assemblability of knowledge: given a pretrained network model as input, KF aims to decompose it into several factor networks, each of which handles only a dedicated task and maintains task-specific knowledge factorized from the source network.
Slimmable Domain Adaptation
In this paper, we introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank, from which models of different capacities can be sampled to accommodate different accuracy-efficiency trade-offs.
Learning Domain Adaptive Object Detection with Probabilistic Teacher
In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter.
Inception Transformer
Recent studies show that Transformer has strong capability of building long-range dependencies, yet is incompetent in capturing high frequencies that predominantly convey local information.
FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders
Firstly, randomly masked face images are used to train the reconstruction module in FaceMAE.
Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction
However, existing best deep AL methods are mostly developed for a single type of learning task (e. g., single-label classification), and hence may not perform well in molecular property prediction that involves various task types.
Prompting to Distill: Boosting Data-Free Knowledge Distillation via Reinforced Prompt
Data-free knowledge distillation (DFKD) conducts knowledge distillation via eliminating the dependence of original training data, and has recently achieved impressive results in accelerating pre-trained language models.
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database
In this work, we propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset, M3ED, which contains 990 dyadic emotional dialogues from 56 different TV series, a total of 9, 082 turns and 24, 449 utterances.
Reliable Label Correction is a Good Booster When Learning with Extremely Noisy Labels
This paradigm, however, is prone to significant degeneration under heavy label noise, as the number of clean samples is too small for conventional methods to behave well.
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation
Text-based video segmentation aims to segment the target object in a video based on a describing sentence.
Ranked #10 on
Referring Expression Segmentation
on A2D Sentences
Optical Flow Estimation
Referring Expression Segmentation
+4
PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision
Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions like consistency loss to guide the learning, which, inevitably, leads to inferior results in real-world scenarios with unseen poses.
Ranked #41 on
3D Human Pose Estimation
on MPI-INF-3DHP
Point2Seq: Detecting 3D Objects as Sequences
We further propose a lightweight scene-to-sequence decoder that can auto-regressively generate words conditioned on features from a 3D scene as well as cues from the preceding words.
CAFE: Learning to Condense Dataset by Aligning Features
Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one.
Geometric Structure Preserving Warp for Natural Image Stitching
Further, an optimized GES energy term is presented to reasonably determine the weights of the sampling points on the geometric structure, and the term is added into the Global Similarity Prior (GSP) stitching model called GES-GSP to achieve a smooth transition between local alignment and geometric structure preservation.
PONet: Robust 3D Human Pose Estimation via Learning Orientations Only
Conventional 3D human pose estimation relies on first detecting 2D body keypoints and then solving the 2D to 3D correspondence problem. Despite the promising results, this learning paradigm is highly dependent on the quality of the 2D keypoint detector, which is inevitably fragile to occlusions and out-of-image absences. In this paper, we propose a novel Pose Orientation Net (PONet) that is able to robustly estimate 3D pose by learning orientations only, hence bypassing the error-prone keypoint detector in the absence of image evidence.
Ranked #85 on
3D Human Pose Estimation
on MPI-INF-3DHP
Up to 100$\times$ Faster Data-free Knowledge Distillation
At the heart of our approach is a novel strategy to reuse the shared common features in training data so as to synthesize different data instances.
Safe Distillation Box
In other words, all users may employ a model in SDB for inference, but only authorized users get access to KD from the model.
Shunted Self-Attention via Multi-Scale Token Aggregation
This novel merging scheme enables the self-attention to learn relationships between objects with different sizes and simultaneously reduces the token numbers and the computational cost.
MetaFormer Is Actually What You Need for Vision
Based on this observation, we hypothesize that the general architecture of the Transformers, instead of the specific token mixer module, is more essential to the model's performance.
Ranked #9 on
Semantic Segmentation
on DensePASS
Meta Clustering Learning for Large-scale Unsupervised Person Re-identification
Unsupervised Person Re-identification (U-ReID) with pseudo labeling recently reaches a competitive performance compared to fully-supervised ReID methods based on modern clustering algorithms.
Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain.
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity.
Unleash the Potential of Adaptation Models via Dynamic Domain Labels
In this paper, we propose an embarrassing simple yet highly effective adversarial domain adaptation (ADA) method for effectively training models for alignment.
How Well Does Self-Supervised Pre-Training Perform with Streaming ImageNet?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks
In this paper, we study a novel meta aggregation scheme towards binarizing graph neural networks (GNNs).
Structure-Aware Feature Generation for Zero-Shot Learning
In this paper, we introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to explicitly account for the topological structure in learning both the latent space and the generative networks.
Boundary Knowledge Translation based Reference Semantic Segmentation
In this paper, we introduce a novel Reference semantic segmentation Network (Ref-Net) to conduct visual boundary knowledge translation.
Edge-competing Pathological Liver Vessel Segmentation with Limited Labels
The diagnosis of MVI needs discovering the vessels that contain hepatocellular carcinoma cells and counting their number in each vessel, which depends heavily on experiences of the doctor, is largely subjective and time-consuming.
Visual Boundary Knowledge Translation for Foreground Segmentation
To this end, we propose a Translation Segmentation Network (Trans-Net), which comprises a segmentation network and two boundary discriminators.
Amalgamating Knowledge From Heterogeneous Graph Neural Networks
In this paper, we study a novel knowledge transfer task in the domain of graph neural networks (GNNs).
Tree-Like Decision Distillation
Knowledge distillation pursues a diminutive yet well-behaved student network by harnessing the knowledge learned by a cumbersome teacher model.
Turning Frequency to Resolution: Video Super-Resolution via Event Cameras
To this end, we propose an Event-based VSR framework (E-VSR), of which the key component is an asynchronous interpolation (EAI) module that reconstructs a high-frequency (HF) video stream with uniform and tiny pixel displacements between neighboring frames from an event stream.
Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking
In this paper, we propose a novel online approach to learning the pose dynamics, which are independent of pose detections in current fame, and hence may serve as a robust estimation even in challenging scenarios including occlusion.
Contrastive Model Inversion for Data-Free Knowledge Distillation
In this paper, we propose Contrastive Model Inversion~(CMI), where the data diversity is explicitly modeled as an optimizable objective, to alleviate the mode collapse issue.
KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation
Knowledge distillation (KD) has recently emerged as an efficacious scheme for learning compact deep neural networks (DNNs).
How Well Does Self-Supervised Pre-Training Perform with Streaming Data?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
Online Multiple Object Tracking with Cross-Task Synergy
On the other hand, such reliable embeddings can boost identity-awareness through memory aggregation, hence strengthen attention modules and suppress drifts.
Training Generative Adversarial Networks in One Stage
Based on the adversarial losses of the generator and discriminator, we categorize GANs into two classes, Symmetric GANs and Asymmetric GANs, and introduce a novel gradient decomposition method to unify the two, allowing us to train both classes in one stage and hence alleviate the training effort.
SPAGAN: Shortest Path Graph Attention Network
SPAGAN therefore allows for a more informative and intact exploration of the graph structure and further {a} more effective aggregation of information from distant neighbors into the center node, as compared to node-based GCN methods.
Self-Born Wiring for Neural Trees
Neural trees aim at integrating deep neural networks and decision trees so as to bring the best of the two worlds, including representation learning from the former and faster inference from the latter.
Mutual-Complementing Framework for Nuclei Detection and Segmentation in Pathology Image
The existing two unsupervised methods are prone to failure on degenerated samples.
Stochastic Partial Swap: Enhanced Model Generalization and Interpretability for Fine-Grained Recognition
Learning mid-level representation for fine-grained recognition is easily dominated by a limited number of highly discriminative patterns, degrading its robustness and generalization capability.
Overcoming Catastrophic Forgetting in Graph Neural Networks
Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks.
Progressive Network Grafting for Few-Shot Knowledge Distillation
In this paper, we investigate the practical few-shot knowledge distillation scenario, where we assume only a few samples without human annotations are available for each category.
SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data
As the main discriminative information of a fine-grained image usually resides in subtle regions, methods along this line are prone to heavy label noise in fine-grained recognition.
One-sample Guided Object Representation Disassembling
In this paper, we introduce the One-sample Guided Object Representation Disassembling (One-GORD) method, which only requires one annotated sample for each object category to learn disassembled object representation from unannotated images.
Learning Propagation Rules for Attribution Map Generation
Prior gradient-based attribution-map methods rely on handcrafted propagation rules for the non-linear/activation layers during the backward pass, so as to produce gradients of the input and then the attribution map.
Factorizable Graph Convolutional Networks
In this paper, we introduce a novel graph convolutional network (GCN), termed as factorizable graph convolutional network(FactorGCN), that explicitly disentangles such intertwined relations encoded in a graph.
Ranked #3 on
Node Classification
on PATTERN 100k
Tracking-by-Counting: Using Network Flows on Crowd Density Maps for Tracking Multiple Targets
State-of-the-art multi-object tracking~(MOT) methods follow the tracking-by-detection paradigm, where object trajectories are obtained by associating per-frame outputs of object detectors.
Impression Space from Deep Template Network
This network is referred to as the {\emph{Template Network}} because its filters will be used as templates to reconstruct images from the impression.
Disassembling Object Representations without Labels
In this paper, we study a new representation-learning task, which we termed as disassembling object representations.
Learning Oracle Attention for High-fidelity Face Completion
While recent works adopted the attention mechanism to learn the contextual relations among elements of the face, they have largely overlooked the disastrous impacts of inaccurate attention scores; in addition, they fail to pay sufficient attention to key facial components, the completion results of which largely determine the authenticity of a face image.
Distilling Knowledge from Graph Convolutional Networks
To enable the knowledge transfer from the teacher GCN to the student, we propose a local structure preserving module that explicitly accounts for the topological semantics of the teacher.
Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN
Then a dual generator is trained by taking the output from the former generator as input.
DEPARA: Deep Attribution Graph for Deep Knowledge Transferability
In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs.
Data-Free Adversarial Distillation
Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer.
Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers
In this paper, we propose a new method, termed as Lip by Speech (LIBS), of which the goal is to strengthen lip reading by learning from speech recognizers.
Ranked #2 on
Lipreading
on CMLR
Dynamic Instance Normalization for Arbitrary Style Transfer
Prior normalization methods rely on affine transformations to produce arbitrary image style transfers, of which the parameters are computed in a pre-defined way.
Deep Model Transferability from Attribution Maps
Exploring the transferability between heterogeneous tasks sheds light on their intrinsic interconnections, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter.
Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation
To this end, we introduce a dual-step strategy that first extracts the task-specific knowledge from the heterogeneous teachers sharing the same sub-task, and then amalgamates the extracted knowledge to build the student network.
Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning
An increasing number of well-trained deep networks have been released online by researchers and developers, enabling the community to reuse them in a plug-and-play way without accessing the training annotations.
One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation
The model cascade (MC) strategy significantly alleviates the class imbalance issue via running a set of individual deep models for coarse-to-fine segmentation.
Ranked #1 on
Brain Tumor Segmentation
on BRATS-2015
(using extra training data)
Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers
Many well-trained Convolutional Neural Network(CNN) models have now been released online by developers for the sake of effortless reproducing.
Not All Parts Are Created Equal: 3D Pose Estimation by Modelling Bi-directional Dependencies of Body Parts
We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated.
Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More
In this paper, we investigate a novel deep-model reusing task.
A Light Dual-Task Neural Network for Haze Removal
In LDTNet, the haze-free image and the transmission map are produced simultaneously.
Amalgamating Knowledge towards Comprehensive Classification
We propose in this paper to study a new model-reusing task, which we term as \emph{knowledge amalgamation}.
