Search Results for author:
Found 42 papers, 27 papers with code
Local Correlation Consistency for Knowledge Distillation
Sufficient knowledge extraction from the teacher network plays a critical role in the knowledge distillation task to improve the performance of the student network.
AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding
Multimodal Large Language Models (MLLMs) have revolutionized video understanding, yet are still limited by context length when processing long videos.
MegaSR: Mining Customized Semantics and Expressive Guidance for Image Super-Resolution
Pioneering text-to-image (T2I) diffusion models have ushered in a new era of real-world image super-resolution (Real-ISR), significantly enhancing the visual perception of reconstructed images.
HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models
Recent Multi-modal Large Language Models (MLLMs) have made great progress in video understanding.
Continuous Knowledge-Preserving Decomposition for Few-Shot Continual Learning
To this end, we propose Continuous Knowledge-Preserving Decomposition for FSCIL (CKPD-FSCIL), a framework that decomposes a model's weights into two parts: one that compacts existing knowledge (knowledge-sensitive components) and another that carries redundant capacity to accommodate new abilities (redundant-capacity components).
class-incremental learning
Few-Shot Class-Incremental Learning
+1
LipGen: Viseme-Guided Lip Video Generation for Enhancing Visual Speech Recognition
The advent of deep learning techniques and advancements in hardware capabilities have significantly enhanced the performance of lip reading models.
ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understanding
Video Large Language Models (VideoLLMs) have made significant strides in video understanding but struggle with long videos due to the limitations of their backbone LLMs.
Efficient Dataset Distillation via Diffusion-Driven Patch Selection for Improved Generalization
Dataset distillation offers an efficient way to reduce memory and computational costs by optimizing a smaller dataset with performance comparable to the full-scale original.
RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training
Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks.
Preview-based Category Contrastive Learning for Knowledge Distillation
It first distills the structural knowledge of both instance-level feature correspondence and the relation between instance features and category centers in a contrastive learning fashion, which can explicitly optimize the category representation and explore the distinct correlation between representations of instances and categories, contributing to discriminative category centers and better classification results.
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
For iterative refinement, we first leverage a video-language model to generate synthetic annotations, resulting in a refined dataset.
DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture
Furthermore, considering that the source data is either unaccessible or too enormous to store for current generative models, we introduce a new paradigm for their distillation without source data, termed Data-Free Knowledge Distillation for Diffusion Models (DKDM).
Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning
The dual design enables the model to maintain the robust features of base classes, while adaptively learning distinctive feature shifts for novel classes.
class-incremental learning
Few-Shot Class-Incremental Learning
+3
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning
For the latter, we use the instruction data from the fine-tuning task, such as math or coding, to orientate the decomposition and train the largest $r$ components that most correspond to the task to learn.
GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning
To tackle these challenges, we present GenView, a controllable framework that augments the diversity of positive views leveraging the power of pretrained generative models while preserving semantics.
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
Large Language Models (LLMs) face significant deployment challenges due to their substantial memory requirements and the computational demands of auto-regressive text generation process.
SNP-S3: Shared Network Pre-training and Significant Semantic Strengthening for Various Video-Text Tasks
By employing one shared BERT-type network to refine textual and cross-modal features simultaneously, SNP is lightweight and could support various downstream applications.
Detecting and Grounding Multi-Modal Media Manipulation and Beyond
HAMMER performs 1) manipulation-aware contrastive learning between two uni-modal encoders as shallow manipulation reasoning, and 2) modality-aware cross-attention by multi-modal aggregator as deep manipulation reasoning.
Temporal Sentence Grounding in Streaming Videos
The goal of TSGSV is to evaluate the relevance between a video stream and a given sentence query.
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.
class-incremental learning
Few-Shot Class-Incremental Learning
+1
Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation
Existing methods mostly focus on analyzing video content, neglecting users' social influence and tag relation.
CHMATCH: Contrastive Hierarchical Matching and Robust Adaptive Threshold Boosted Semi-Supervised Learning
In the meantime, we make full use of the structured information in the hierarchical labels to learn an accurate affinity graph for contrastive learning.
Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem
Several studies have recently pointed that existing Visual Question Answering (VQA) models heavily suffer from the language prior problem, which refers to capturing superficial statistical correlations between the question type and the answer whereas ignoring the image contents.
Semantic-aware Modular Capsule Routing for Visual Question Answering
Visual Question Answering (VQA) is fundamentally compositional in nature, and many questions are simply answered by decomposing them into modular sub-problems.
HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors
We observe that the core difficulty for heterogeneous KD (hetero-KD) is the significant semantic gap between the backbone features of heterogeneous detectors due to the different optimization manners.
Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation
Scene Graph Generation, which generally follows a regular encoder-decoder pipeline, aims to first encode the visual contents within the given image and then parse them into a compact summary graph.
Ranked #1 on
Unbiased Scene Graph Generation
on Visual Genome
(mR@20 metric)
DualGNN: Dual Graph Neural Network for Multimedia Recommendation
Specifically, we first introduce a single-modal representation learning module, which performs graph operations on the user-microvideo graph in each modality to capture single-modal user preferences on different modalities.
Ranked #4 on
Multi-modal Recommendation
on Amazon Clothing
High Quality Segmentation for Ultra High-resolution Images
To segment 4K or 6K ultra high-resolution images needs extra computation consideration in image segmentation.
Dynamic Modality Interaction Modeling for Image-Text Retrieval
To address these issues, we develop a novel modality interaction modeling network based upon the routing mechanism, which is the first unified and dynamic multimodal interaction framework towards image-text retrieval.
Graph Contrastive Clustering
On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments.
Fast and Differentiable Matrix Inverse and Its Extension to SVD
First of all, to perform matrix inverse, we provide a differentiable yet efficient way, named LD-Minv, which is a learnable deep neural network (DNN) with each layer being an $L$-th order matrix polynomial.
Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space
In this paper, we examine the diversity of teacher models in the gradient space and regard the ensemble knowledge distillation as a multi-objective optimization problem so that we can determine a better optimization direction for the training of student network.
Multi-modal Cooking Workflow Construction for Food Recipes
Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe.
Maximum-and-Concatenation Networks
While successful in many fields, deep neural networks (DNNs) still suffer from some open problems such as bad local minima and unsatisfactory generalization performance.
Dynamical System Inspired Adaptive Time Stepping Controller for Residual Network Families
We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance.
SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances.
Ranked #8 on
Panoptic Segmentation
on Cityscapes test
Expectation-Maximization Attention Networks for Semantic Segmentation
It is designed to compute the representation of each position by a weighted sum of the features at all positions.
Ranked #7 on
Semantic Segmentation
on BDD100K val
Differentiable Linearized ADMM
Recently, a number of learning-based optimization methods that combine data-driven architectures with the classical optimization algorithms have been proposed and explored, showing superior empirical performance in solving various ill-posed inverse problems, but there is still a scarcity of rigorous analysis about the convergence behaviors of learning-based optimization.
Deep Comprehensive Correlation Mining for Image Clustering
Recent developed deep unsupervised methods allow us to jointly learn representation and cluster unlabelled data.
Ranked #9 on
Image Clustering
on Tiny-ImageNet
Matrix Recovery with Implicitly Low-Rank Data
To tackle this issue, we propose a novel method for matrix recovery in this paper, which could well handle the case where the target matrix is low-rank in an implicit feature space but high-rank or even full-rank in its original form.
Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining
In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers.
Ranked #7 on
Single Image Deraining
on Test2800
Essential Tensor Learning for Multi-view Spectral Clustering
In this paper, we focus on the Markov chain based spectral clustering method and propose a novel essential tensor learning method to explore the high order correlations for multi-view representation.
