Search Results for author:
Found 51 papers, 19 papers with code
I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
The remarkable achievements of large language models (LLMs) have led many to conclude that they exhibit a form of intelligence.
Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss
We then design a motion consistency loss to maintain similar feature correlation patterns in the generated video, using the gradient of this loss in the latent space to guide the generation process for precise motion control.
Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA
We investigate whether the pre-trained knowledge of vision-language models (VLMs), such as CLIP, can be retained or even enhanced during continual learning (CL) while absorbing knowledge from a data stream.
DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling
Text-to-image diffusion model alignment is critical for improving the alignment between the generated images and human preferences.
Learning Mamba as a Continual Learner
By formulating CL as a sequence prediction task, meta-continual learning (MCL) enables to meta-learn an efficient continual learner based on the recent advanced sequence models, e. g., Transformers.
Is Less More? Exploring Token Condensation as Training-free Adaptation for CLIP
To address these questions, we propose Token Condensation as Adaptation (TCA), a training-free adaptation method for CLIP by pruning class-irrelevant visual tokens while merging class-ambiguous tokens.
Mining Your Own Secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models
Personalized text-to-image diffusion models have grown popular for their ability to efficiently acquire a new concept from user-defined text descriptions and a few images.
UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation
Inspired by the success of deep generative models and fine-tuning techniques, we proposed a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning.
Highway Graph to Accelerate Reinforcement Learning
Building on this observation, we introduce the highway graph to model state transitions.
CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models
Cooperating with the diverse range of existing prompting methods, CLAP can surpass the predominant deterministic finetuning approaches for CL with CLIP.
Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning
Continual learning (CL) aims to continually accumulate knowledge from a non-stationary data stream without catastrophic forgetting of learned knowledge, requiring a balance between stability and adaptability.
Identifiable Latent Neural Causal Models
This work establishes a {sufficient} and {necessary} condition characterizing the types of distribution shifts for identifiability in the context of latent additive noise models.
Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure
It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated in the unified network structure.
Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning
We show here that the combination of a large language model and an image generation model can similarly provide useful premonitions as to how a continual learning challenge might develop over time.
SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets
We suggest building virtual pinhole cameras to resolve the distortion problem of fisheye cameras and unify the processing for the two types of 360$^\circ$ cameras.
Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models
Multimodal contrastive representation learning methods have proven successful across a range of domains, partly due to their ability to generate meaningful shared representations of complex phenomena.
Learning with Mixture of Prototypes for Out-of-Distribution Detection
To tackle these issues, we propose PrototypicAl Learning with a Mixture of prototypes (PALM) which models each class with multiple prototypes to capture the sample diversities, and learns more faithful and compact samples embeddings to enhance OOD detection.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Identifiable Latent Polynomial Causal Models Through the Lens of Change
However, this progress rests on the assumption that the causal relationships among latent causal variables adhere strictly to linear Gaussian models.
FocDepthFormer: Transformer with latent LSTM for Depth Estimation from Focal Stack
Most existing methods for depth estimation from a focal stack of images employ convolutional neural networks (CNNs) using 2D or 3D convolutions over a fixed set of images.
Learning Informative Latent Representation for Quantum State Tomography
QST aims to recover the density matrix or the properties of the quantum state from the measured frequencies.
RanPAC: Random Projections and Pre-trained Models for Continual Learning
In this paper, we propose a concise and effective approach for CL with pre-trained models.
Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes
To let the geometric perception learned from multi-view cues in static areas propagate to the monocular representation in dynamic areas and let monocular cues enhance the representation of multi-view cost volume, we propose a cross-cue fusion (CCF) module, which includes the cross-cue attention (CCA) to encode the spatially non-local relative intra-relations from each source to enhance the representation of the other.
Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognition
In this work, we propose to automatically design efficient 3D CNN architectures via a novel training-free neural architecture search approach tailored for 3D CNNs considering the model complexity.
Ranked #84 on
Action Recognition
on Something-Something V2
The Neural Process Family: Survey, Applications and Perspectives
We shed light on their potential to bring several recent advances in other deep learning domains under one umbrella.
Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift
Within this new paradigm, we present an intricate causal generative model by introducing latent noises across domains, along with a latent content variable and a latent style variable to achieve more nuanced rendering of observational data.
Truncated Matrix Power Iteration for Differentiable DAG Learning
Recovering underlying Directed Acyclic Graph (DAG) structures from observational data is highly challenging due to the combinatorial nature of the DAG-constrained optimization problem.
Identifying Weight-Variant Latent Causal Models
The task of causal representation learning aims to uncover latent higher-level causal representations that affect lower-level observations.
NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results
The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i. e. solutions can not exceed a given number of operations).
Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning
This minimizes the interference between parameters for different tasks.
Ranked #5 on
Continual Learning
on Tiny-ImageNet (10tasks)
Memory-Augmented Dynamic Neural Relational Inference
This motivates us to propose a memory-augmented dynamic neural relational inference method, which maintains two associative memory pools: one for the interactive relations and the other for the individual entities.
COVID-19 Chest CT Image Segmentation -- A Deep Convolutional Neural Network Solution
Inspired by the observation that the boundary of the infected lung can be enhanced by adjusting the global intensity, in the proposed deep CNN, we introduce a feature variation block which adaptively adjusts the global properties of the features for segmenting COVID-19 infection.
Memorizing Comprehensively to Learn Adaptively: Unsupervised Cross-Domain Person Re-ID with Multi-level Memory
Due to the lack of supervision in the target domain, it is crucial to identify the underlying similarity-and-dissimilarity relationships among the unlabelled samples in the target domain.
Semi-supervised Learning via Conditional Rotation Angle Estimation
Our insight is that the prediction target in SemSL can be modeled as the latent factor in the predictor for the SlfSL target.
Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation
Most learning-based super-resolution (SR) methods aim to recover high-resolution (HR) image from a given low-resolution (LR) image via learning on LR-HR image pairs.
Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation
3D point cloud semantic and instance segmentation is crucial and fundamental for 3D scene understanding.
Ranked #29 on
3D Instance Segmentation
on ScanNet(v2)
Part-Guided Attention Learning for Vehicle Instance Retrieval
Finally, we aggregate the global appearance and part features to improve the feature performance further.
Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
To tackle the unsupervised domain adaptation problem, we explore the possibilities to generate high-quality labels as proxy labels to supervise the training on target data.
An Effective Two-Branch Model-Based Deep Network for Single Image Deraining
Removing rain effects from an image is of importance for various applications such as autonomous driving, drone piloting, and photo editing.
Attention-guided Network for Ghost-free High Dynamic Range Imaging
Ghosting artifacts caused by moving objects or misalignments is a key challenge in high dynamic range (HDR) imaging for dynamic scenes.
Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection
At the test stage, the learned memory will be fixed, and the reconstruction is obtained from a few selected memory records of the normal data.
Knowledge Adaptation for Efficient Semantic Segmentation
To tackle this dilemma, we propose a knowledge distillation method tailored for semantic segmentation to improve the performance of the compact FCNs with large overall stride.
RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion
RGB images differentiate from depth images as they carry more details about the color and texture information, which can be utilized as a vital complementary to depth for boosting the performance of 3D semantic scene completion (SSC).
Ranked #20 on
3D Semantic Scene Completion
on NYUv2
Variational Bayesian Dropout with a Hierarchical Prior
Then, we incorporate such a prior into inferring the joint posterior over network weights and the variance in the hierarchical prior, with which both the network training and the dropout rate estimation can be cast into a joint optimization problem.
End-to-End Diagnosis and Segmentation Learning from Cardiac Magnetic Resonance Imaging
In this paper, we propose a learning method to train diagnosis models, where our approach is designed to work with relatively small datasets.
MPTV: Matching Pursuit Based Total Variation Minimization for Image Deconvolution
Compared to existing methods, MPTV is less sensitive to the choice of the trade-off parameter between data fitting and regularization.
Deblurring Natural Image Using Super-Gaussian Fields
Existing sparsity-based priors are usually rooted in modeling the response of images to some specific filters (e. g., image gradients), which are insufficient to capture the complicated image structures.
Seeing Deeply and Bidirectionally: A Deep Learning Approach for Single Image Reflection Removal
Reflections often obstruct the desired scene when taking photos through glass panels.
Learning Deep Gradient Descent Optimization for Image Deconvolution
Extensive experiments on synthetic benchmarks and challenging real-world images demonstrate that the proposed deep optimization method is effective and robust to produce favorable results as well as practical for real-world image deblurring applications.
Self-Paced Kernel Estimation for Robust Blind Image Deblurring
Rather than attempt to identify outliers to the model a priori, we instead propose to sequentially identify inliers, and gradually incorporate them into the estimation process.
From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur
The critical observation underpinning our approach is thus that learning the motion flow instead allows the model to focus on the cause of the blur, irrespective of the image content.
Blind Image Deconvolution by Automatic Gradient Activation
We show here that a subset of the image gradients are adequate to estimate the blur kernel robustly, no matter the gradient image is sparse or not.
