Search Results for author:
Found 29 papers, 16 papers with code
Modality-Inconsistent Continual Learning of Multimodal Large Language Models
In this paper, we introduce Modality-Inconsistent Continual Learning (MICL), a new continual learning scenario for Multimodal Large Language Models (MLLMs) that involves tasks with inconsistent modalities (image, audio, or video) and varying task types (captioning or question-answering).
Enhancing Robustness of CLIP to Common Corruptions through Bimodal Test-Time Adaptation
Although open-vocabulary classification models like Contrastive Language Image Pretraining (CLIP) have demonstrated strong zero-shot learning capabilities, their robustness to common image corruptions remains poorly understood.
GradAlign for Training-free Model Performance Inference
Architecture plays an important role in deciding the performance of deep neural networks.
Continual Audio-Visual Sound Separation
The task is inherently challenging as our models must not only effectively utilize information from both modalities in current tasks but also preserve their cross-modal association in old tasks to mitigate catastrophic forgetting during audio-visual continual learning.
STONE: A Submodular Optimization Framework for Active 3D Object Detection
A key requirement for training an accurate 3D object detector is the availability of a large amount of LiDAR-based point cloud data.
Continual Distillation Learning: An Empirical Study of Knowledge Distillation in Prompt-based Continual Learning
To this end, we conducted experiments to study the CDL problem with three prompt-based CL models, i. e., L2P, DualPrompt and CODA-Prompt, where we utilized logit distillation, feature distillation and prompt distillation for knowledge distillation from a teacher model to a student model.
Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation
In instance segmentation tasks on seven core datasets of the BOP challenge, our method is around 4. 5 times faster than the leading published RGB method and surpasses it by 3. 6 AP.
KITE: A Kernel-based Improved Transferability Estimation Method
Existing methods tackle this problem by analyzing the output of the pre-trained model or by comparing the pre-trained model with a probe model trained on the target dataset.
PALM: Pushing Adaptive Learning Rate Mechanisms for Continual Test-Time Adaptation
Real-world vision models in dynamic environments face rapid shifts in domain distributions, leading to decreased recognition performance.
Not Just Change the Labels, Learn the Features: Watermarking Deep Neural Networks with Multi-View Data
Unfortunately, existing methodologies based on trigger sets are still susceptible to functionality-stealing attacks, potentially enabling adversaries to steal the functionality of the source model without a reliable means of verifying ownership.
Inconsistency-Based Data-Centric Active Open-Set Annotation
The presence of unknown classes in the data can significantly impact the performance of existing active learning methods due to the uncertainty they introduce.
Segment Every Out-of-Distribution Object
Semantic segmentation models, while effective for in-distribution categories, face challenges in real-world deployment due to encountering out-of-distribution (OoD) objects.
Towards Effective Semantic OOD Detection in Unseen Domains: A Domain Generalization Perspective
In this study, we introduce a novel problem, semantic OOD detection across domains, which simultaneously addresses both distributional shifts.
VEATIC: Video-based Emotion and Affect Tracking in Context Dataset
Along with the dataset, we propose a new computer vision task to infer the affect of the selected character via both context and character information in each video frame.
Audio-Visual Class-Incremental Learning
We demonstrate that joint audio-visual modeling can improve class-incremental learning, but current methods fail to preserve semantic similarity between audio and visual features as incremental step grows.
Unsupervised Feature Learning with Emergent Data-Driven Prototypicality
With our feature mapper simply trained to spread out training instances in hyperbolic space, we observe that images move closer to the origin with congealing, validating our idea of unsupervised prototypicality discovery.
Self-Supervised Unseen Object Instance Segmentation via Long-Term Robot Interaction
By applying multi-object tracking and video object segmentation on the images collected via robot pushing, our system can generate segmentation masks of all the objects in these images in a self-supervised way.
SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge
To address the challenges, we propose Self-Supervised ContrAstive Lifelong LEarning without Prior Knowledge (SCALE) which can extract and memorize representations on the fly purely from the data continuum.
Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers
We enforce spatial consistency of grouping and bootstrap feature learning with co-segmentation among multiple views of the same image, and enforce semantic consistency across the grouping hierarchy with clustering transformers between coarse- and fine-grained features.
CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data
We propose CO-SNE, which extends the Euclidean space visualization tool, t-SNE, to hyperbolic space.
Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers
Hyperbolic space can naturally embed hierarchies, unlike Euclidean space.
Attacking Lifelong Learning Models with Gradient Reversion
We evaluate the effectiveness of traditional attack methods such as FGSM and PGD. The results show that A-GEM still possesses strong continual learning ability in the presence of adversarial examples in the memory and simple defense techniques such as label smoothing can further alleviate the adversarial effects.
A Broader Study of Cross-Domain Few-Shot Learning
Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning.
Ranked #3 on
Cross-Domain Few-Shot
on Plantae
cross-domain few-shot learning
Few-Shot Image Classification
+1
AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
Fine-tuning is a popular transfer learning technique for deep neural networks where a few rounds of training are applied to the parameters of a pre-trained model to adapt them to a new task.
Improved Schemes for Episodic Memory-based Lifelong Learning
This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II.
Learning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient
In this paper, we introduce a novel and effective lifelong learning algorithm, called MixEd stochastic GrAdient (MEGA), which allows deep neural networks to acquire the ability of retaining performance on old tasks while learning new tasks.
Depthwise Convolution is All You Need for Learning Multiple Visual Domains
A model aware of the relationships between different domains can also be trained to work on new domains with less resources.
SpotTune: Transfer Learning through Adaptive Fine-tuning
Transfer learning, which allows a source task to affect the inductive bias of the target task, is widely used in computer vision.
A Survey on Methods and Theories of Quantized Neural Networks
For all its popularity, deep neural networks are also criticized for consuming a lot of memory and draining battery life of devices during training and inference.
