Search Results for author:
Found 158 papers, 80 papers with code
Learning a Tree of Metrics with Disjoint Visual Features
Given a hierarchical taxonomy that captures semantic similarity between the objects, we learn a corresponding tree of metrics (ToM).
Semantic Kernel Forests from Multiple Taxonomies
When learning features for complex visual recognition problems, labeled image exemplars alone can be insufficient.
A Unified Semantic Embedding: Relating Taxonomies and Attributes
We propose a method that learns a discriminative yet semantic space for object categorization, where we also embed auxiliary semantic entities such as supercategories and attributes.
Hierarchical Maximum-Margin Clustering
We present a hierarchical maximum-margin clustering method for unsupervised data analysis.
Expanding Object Detector's Horizon: Incremental Learning Framework for Object Detection in Videos
By incrementally detecting object instances in video and adding confident detections into the model, we are able to dynamically adjust the complexity of the detector over time by instantiating new prototypes to span all domains the model has seen.
Combined Group and Exclusive Sparsity for Deep Neural Networks
The number of parameters in a deep neural network is usually very large, which helps with its learning capacity but also hinders its scalability and practicality due to memory/time inefficiency and overfitting.
Deep Asymmetric Multi-task Feature Learning
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process.
SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization
We propose a novel deep neural network that is both lightweight and effectively structured for model parallelization.
Lifelong Learning with Dynamically Expandable Networks
We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks.
Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification
We consider the problem of active feature acquisition, where we sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way.
DropMax: Adaptive Variational Softmax
Moreover, the learning of dropout rates for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
DropMax: Adaptive Stochastic Softmax
Moreover, the learning of dropout probabilities for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them.
Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class.
Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare
We present a personalized and reliable prediction model for healthcare, which can provide individually tailored medical services such as diagnosis, disease treatment, and prevention.
Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss
We demonstrate the effectiveness of our trainable quantizer on ImageNet dataset with various network architectures such as ResNet-18, -34 and AlexNet, on which it outperforms existing methods to achieve the state-of-the-art accuracy.
ADAPTIVE NETWORK SPARSIFICATION VIA DEPENDENT VARIATIONAL BETA-BERNOULLI DROPOUT
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Learning the Compositional Spaces for Generalized Zero-shot Learning
This paper studies the problem of Generalized Zero-shot Learning (G-ZSL), whose goal is to classify instances belonging to both seen and unseen classes at the test time.
Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding
We consider the problem of active feature acquisition where the goal is to sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way at test time.
Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number of tasks.
Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability.
Learning What and Where to Transfer
To address the issue, we propose a novel transfer learning approach based on meta-learning that can automatically learn what knowledge to transfer from the source network to where in the target network.
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks
While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed.
Meta Dropout: Learning to Perturb Features for Generalization
Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.
Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks
Among many approaches, the simplest and most intuitive way is zero imputation, which treats the value of a missing entry simply as zero.
Reliable Estimation of Individual Treatment Effect with Causal Information Bottleneck
We also introduce an additional form of a regularizer from the perspective of understanding ITE in the semi-supervised learning framework to ensure more reliable representations.
Ranked #1 on
Causal Inference
on IDHP
Learning to Generalize to Unseen Tasks with Bilevel Optimization
To tackle this issue, we propose a simple yet effective meta-learning framework for metricbased approaches, which we refer to as learning to generalize (L2G), that explicitly constrains the learning on a sampled classification task to reduce the classification error on a randomly sampled unseen classification task with a bilevel optimization scheme.
Adversarial Neural Pruning with Latent Vulnerability Suppression
Despite the remarkable performance of deep neural networks on various computer vision tasks, they are known to be susceptible to adversarial perturbations, which makes it challenging to deploy them in real-world safety-critical applications.
Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
Yet, most of them cannot effectively defend against whitebox attacks where an adversary has a knowledge of the model and defense.
Self-supervised Label Augmentation via Input Transformations
Our main idea is to learn a single unified task with respect to the joint distribution of the original and self-supervised labels, i. e., we augment original labels via self-supervision of input transformation.
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
Meta-Learned Confidence for Few-shot Learning
To tackle this issue, we propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries such that they improve the model's transductive inference performance on unseen tasks.
Federated Continual Learning with Weighted Inter-client Transfer
There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios.
Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs
By combining these two learning schemes, our model outperforms existing state-of-the-art speaker verification models learned with a standard supervised learning framework on short utterance (1-2 seconds) on the VoxCeleb datasets.
Meta Dropout: Learning to Perturb Latent Features for Generalization
Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.
Ranked #1 on
Meta-Learning
on OMNIGLOT - 1-Shot, 20-way
Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs
We validate our Information Maximizing Hierarchical Conditional Variational AutoEncoder (Info-HCVAE) on several benchmark datasets by evaluating the performance of the QA model (BERT-base) using only the generated QA pairs (QA-based evaluation) or by using both the generated and human-labeled pairs (semi-supervised learning) for training, against state-of-the-art baseline models.
Ranked #1 on
Question Generation
on Natural Questions
Cost-effective Interactive Attention Learning with Neural Attention Processes
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction
For transductive link prediction, we further propose a stochastic embedding layer to model uncertainty in the link prediction between unseen entities.
Adversarial Self-Supervised Contrastive Learning
In this paper, we propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures
As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize to heterogeneous tasks and architectures.
Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Learning to Generate Noise for Multi-Attack Robustness
Adversarial learning has emerged as one of the successful techniques to circumvent the susceptibility of existing methods against adversarial perturbations.
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning
Through extensive experimental validation of our method in the two different scenarios, we show that our method outperforms both local semi-supervised learning and baselines which naively combine federated learning with semi-supervised learning.
Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning
Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss.
Set Based Stochastic Subsampling
Deep models are designed to operate on huge volumes of high dimensional data such as images.
Localization Uncertainty Estimation for Anchor-Free Object Detection
To tackle these limitations, we propose a new localization uncertainty estimation method called UAD for anchor-free object detection.
Ranked #116 on
Object Detection
on COCO test-dev
Learning to Sample with Local and Global Contexts in Experience Replay Buffer
Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL).
Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced.
Time-Reversal Symmetric ODE Network
Time-reversal symmetry, which requires that the dynamics of a system should not change with the reversal of time axis, is a fundamental property that frequently holds in classical and quantum mechanics.
Few-shot Visual Reasoning with Meta-analogical Contrastive Learning
While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task.
Bootstrapping Neural Processes
While this "data-driven" way of learning stochastic processes has proven to handle various types of data, NPs still rely on an assumption that uncertainty in stochastic processes is modeled by a single latent variable, which potentially limits the flexibility.
Neural Complexity Measures
While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging.
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation
We propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pre-training, such that we can effectively adapt the language model to a particular target task (e. g. question answering).
Attribution Preservation in Network Compression for Reliable Network Interpretation
Neural networks embedded in safety-sensitive applications such as self-driving cars and wearable health monitors rely on two important techniques: input attribution for hindsight analysis and network compression to reduce its size for edge-computing.
Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection
Although deep neural networks have shown promising performances on various tasks, they are susceptible to incorrect predictions induced by imperceptibly small perturbations in inputs.
Contrastive Learning with Adversarial Perturbations for Conditional Text Generation
In this work, we propose to mitigate the conditional text generation problem by contrasting positive pairs with negative pairs, such that the model is exposed to various valid or incorrect perturbations of the inputs, for improved generalization.
Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning
Then, the learned model can be used for downstream few-shot classification tasks, where we obtain task-specific parameters by performing semi-supervised EM on the latent representations of the support and query set, and predict labels of the query set by computing aggregated posteriors.
Meta-Learned Confidence for Transductive Few-shot Learning
A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples, or confidence-weighted average of all the query samples.
Improving Neural Network Accuracy and Calibration Under Distributional Shift with Prior Augmented Data
We demonstrate the effectiveness of our method and validate its performance on both classification and regression problems by applying it to the training of recent state-of-the-art neural network models.
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bitwise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
Rapid Neural Pruning for Novel Datasets with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Model-Augmented Q-learning
We show that the proposed scheme, called Model-augmented $Q$-learning (MQL), obtains a policy-invariant solution which is identical to the solution obtained by learning with true reward.
Large-Scale Meta-Learning with Continual Trajectory Shifting
Meta-learning of shared initialization parameters has shown to be highly effective in solving few-shot learning tasks.
Improving Uncertainty Calibration via Prior Augmented Data
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators.
Accurate Learning of Graph Representations with Graph Multiset Pooling
Graph neural networks have been widely used on modeling graph data, achieving impressive results on node classification and link prediction tasks.
Ranked #1 on
Graph Classification
on ToxCast
Task-Adaptive Neural Network Search with Meta-Contrastive Learning
To address such limitations, we introduce a novel problem of \emph{Neural Network Search} (NNS), whose goal is to search for the optimal pretrained network for a novel dataset and constraints (e. g. number of parameters), from a model zoo.
Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding
The proposed method adheres to the required symmetries of invariance and equivariance as well as maintaining MBC for any partition of the input set.
Consistency Regularization for Adversarial Robustness
Adversarial training (AT) is currently one of the most successful methods to obtain the adversarial robustness of deep neural networks.
Learning to Perturb Word Embeddings for Out-of-distribution QA
QA models based on pretrained language mod-els have achieved remarkable performance on various benchmark datasets. However, QA models do not generalize well to unseen data that falls outside the training distribution, due to distributional shifts. Data augmentation (DA) techniques which drop/replace words have shown to be effective in regularizing the model from overfitting to the training data. Yet, they may adversely affect the QA tasks since they incur semantic changes that may lead to wrong answers for the QA task.
Online Coreset Selection for Rehearsal-based Continual Learning
We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.
Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation
In this work, we propose StyleSpeech, a new TTS model which not only synthesizes high-quality speech but also effectively adapts to new speakers.
Adversarial purification with Score-based generative models
Recently, an Energy-Based Model (EBM) trained with Markov-Chain Monte-Carlo (MCMC) has been highlighted as a purification model, where an attacked image is purified by running a long Markov-chain using the gradients of the EBM.
HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning
To overcome such limitations, we propose Hardware-adaptive Efficient Latency Predictor (HELP), which formulates the device-specific latency estimation problem as a meta-learning problem, such that we can estimate the latency of a model's performance for a given task on an unseen device with a few samples.
Entropy Weighted Adversarial Training
Adversarial training methods, which minimizes the loss of adversarially-perturbed training examples, have been extensively studied as a solution to improve the robustness of the deep neural networks.
Edge Representation Learning with Hypergraphs
This dual hypergraph construction allows us to apply message-passing techniques for node representations to edges.
FedMix: Approximation of Mixup under Mean Augmented Federated Learning
Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device, thus preserving privacy and eliminating the need to store data globally.
Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets
Despite the success of recent Neural Architecture Search (NAS) methods on various tasks which have shown to output networks that largely outperform human-designed networks, conventional NAS methods have mostly tackled the optimization of searching for the network architecture for a single task (dataset), which does not generalize well across multiple tasks (datasets).
Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss
Network quantization, which aims to reduce the bit-lengths of the network weights and activations, has emerged for their deployments to resource-limited devices.
Model-augmented Prioritized Experience Replay
Experience replay is an essential component in off-policy model-free reinforcement learning (MfRL).
Meta Learning Low Rank Covariance Factors for Energy Based Deterministic Uncertainty
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer.
Agnostic Personalized Federated Learning with Kernel Factorization
We then study two essential challenges of the agnostic personalized federated learning, which are (1) Label Heterogeneity where local clients learn from the same single domain but labeling schemes are not synchronized with each other and (2) Domain Heterogeneity where the clients learn from the different datasets which can be semantically similar or dissimilar for each other.
Task Conditioned Stochastic Subsampling
Deep Learning algorithms are designed to operate on huge volumes of high dimensional data such as images.
MOG: Molecular Out-of-distribution Generation with Energy-based Models
To validate the ability to explore the chemical space beyond the known molecular distribution, we experiment with MOG to generate molecules of high absolute values of docking score, which is the affinity score based on a physical binding simulation between a target protein and a given molecule.
Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation
We further show with ablation studies that our method, predictive error-PER (FREED(PE)), significantly improves the model performance.
Online Hyperparameter Meta-Learning with Hypergradient Distillation
Many gradient-based meta-learning methods assume a set of parameters that do not participate in inner-optimization, which can be considered as hyperparameters.
Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning
Thanks to the gradients aligned between tasks by our method, the model becomes less vulnerable to negative transfer and catastrophic forgetting.
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer.
Representational Continuity for Unsupervised Continual Learning
Continual learning (CL) aims to learn a sequence of tasks without forgetting the previously acquired knowledge.
DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing
However, various factors such as different users, devices, and environments impact the performance of such applications, thus making the domain shift (i. e., distributional shift between the training domain and the target domain) a critical issue in mobile sensing.
Hardware-adaptive Efficient Latency Prediction for NAS via Meta-Learning
To overcome such limitations, we propose Hardware-adaptive Efficient Latency Predictor (HELP), which formulates the device-specific latency estimation problem as a meta-learning problem, such that we can estimate the latency of a model's performance for a given task on an unseen device with a few samples.
Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing
However, they now suffer from lack of sample diversification as they always deterministically select regions with maximum saliency, injecting bias into the augmented data.
MPViT: Multi-Path Vision Transformer for Dense Prediction
While Convolutional Neural Networks (CNNs) have been the dominant architectures for such tasks, recently introduced Vision Transformers (ViTs) aim to replace them as a backbone.
Ranked #38 on
Instance Segmentation
on COCO minival
Factorized-FL: Agnostic Personalized Federated Learning with Kernel Factorization & Similarity Matching
In real-world federated learning scenarios, participants could have their own personalized labels which are incompatible with those from other clients, due to using different label permutations or tackling completely different tasks or domains.
Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations
Specifically, we propose a new graph diffusion process that models the joint distribution of the nodes and edges through a system of stochastic differential equations (SDEs).
Graph Self-supervised Learning with Accurate Discrepancy Learning
Contrastive learning, while it can learn global graph-level similarities, its objective to maximize the similarity between two differently perturbed graphs may result in representations that cannot discriminate two similar graphs with different properties.
Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization
We introduce a pragmatic FL scenario with bitwidth heterogeneity across the participating devices, dubbed as Bitwidth Heterogeneous Federated Learning (BHFL).
Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation
Dense retrieval models, which aim at retrieving the most relevant document for an input query on a dense representation space, have gained considerable attention for their remarkable success.
Ranked #1000000000 on
Passage Retrieval
on Natural Questions
KALA: Knowledge-Augmented Language Model Adaptation
Pre-trained language models (PLMs) have achieved remarkable success on various natural language understanding tasks.
Skill-based Meta-Reinforcement Learning
While deep reinforcement learning methods have shown impressive results in robot learning, their sample inefficiency makes the learning of complex, long-horizon behaviors with real robot systems infeasible.
Set-based Meta-Interpolation for Few-Task Meta-Learning
Recently, several task augmentation methods have been proposed to tackle this issue using domain-specific knowledge to design augmentation techniques to densify the meta-training task distribution.
Exploring Chemical Space with Score-based Out-of-distribution Generation
A well-known limitation of existing molecular generative models is that the generated molecules highly resemble those in the training set.
Distortion-Aware Network Pruning and Feature Reuse for Real-time Video Segmentation
Real-time video segmentation is a crucial task for many real-world applications such as autonomous driving and robot control.
Personalized Subgraph Federated Learning
To this end, we introduce a new subgraph FL problem, personalized subgraph FL, which focuses on the joint improvement of the interrelated local GNNs rather than learning a single global model, and propose a novel framework, FEDerated Personalized sUBgraph learning (FED-PUB), to tackle it.
Forget-free Continual Learning with Winning Subnetworks
Inspired by Lottery Ticket Hypothesis that competitive subnetworks exist within a dense network, we propose a continual learning method referred to as Winning SubNetworks (WSN), which sequentially learns and selects an optimal subnetwork for each task.
BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation
Neural network quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation, while preserving the performance of the original model.
Dataset Condensation with Latent Space Knowledge Factorization and Sharing
In this paper, we introduce a novel approach for systematically solving dataset condensation problem in an efficient manner by exploiting the regularity in a given dataset.
StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation
We propose StyleTalker, a novel audio-driven talking head generation model that can synthesize a video of a talking person from a single reference image with accurately audio-synced lip shapes, realistic head poses, and eye blinks.
Object Detection in Aerial Images with Uncertainty-Aware Graph Network
To achieve this, we first detect objects and then measure their semantic and spatial distances to construct an object graph, which is then represented by a graph neural network (GNN) for refining visual CNN features for objects.
Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaranteeing the same output for all partitions.
On the Soft-Subnetwork for Few-shot Class Incremental Learning
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as \emph{Soft-SubNetworks (SoftNet)}.
Self-Distillation for Further Pre-training of Transformers
Pre-training a large transformer model on a massive amount of unlabeled data and fine-tuning it on labeled datasets for diverse downstream tasks has proven to be a successful strategy, for a variety of vision and natural language processing tasks.
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers.
Learning Transferable Adversarial Robust Representations via Multi-view Consistency
Despite the success on few-shot learning problems, most meta-learned models only focus on achieving good performance on clean examples and thus easily break down when given adversarially perturbed samples.
Language Detoxification with Attribute-Discriminative Latent Space
Transformer-based Language Models (LMs) have achieved impressive results on natural language understanding tasks, but they can also generate toxic text such as insults, threats, and profanity, limiting their real-world applications.
Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Models
There has been a significant progress in Text-To-Speech (TTS) synthesis technology in recent years, thanks to the advancement in neural generative modeling.
EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens
Masked Video Autoencoder (MVA) approaches have demonstrated their potential by significantly outperforming previous video representation learning methods.
Ranked #1 on
Object State Change Classification
on Ego4D
Object State Change Classification
Object State Change Classification on Ego4D
+4
Graph Generation with Diffusion Mixture
Generation of graphs is a major challenge for real-world tasks that require understanding the complex nature of their non-Euclidean structures.
Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement
Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multiple times.
The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation
In this paper, we introduce a novel learning scheme named weakly semi-supervised instance segmentation (WSSIS) with point labels for budget-efficient and high-performance instance segmentation.
Forget-free Continual Learning with Soft-Winning SubNetworks
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which states that competitive smooth (non-binary) subnetworks exist within a dense network in continual learning tasks, we investigate two proposed architecture-based continual learning methods which sequentially learn and select adaptive binary- (WSN) and non-binary Soft-Subnetworks (SoftNet) for each task.
Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
Token-based masked generative models are gaining popularity for their fast inference time with parallel decoding.
Direct Fact Retrieval from Knowledge Graphs without Entity Linking
There has been a surge of interest in utilizing Knowledge Graphs (KGs) for various natural language processing/understanding tasks.
ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models
Emotional Text-To-Speech (TTS) is an important task in the development of systems (e. g., human-like dialogue agents) that require natural and emotional speech.
DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models
To tackle such limitations of existing NAS methods, we propose a paradigm shift from NAS to a novel conditional Neural Architecture Generation (NAG) framework based on diffusion models, dubbed DiffusionNAG.
Set-based Neural Network Encoding
We propose an approach to neural network weight encoding for generalization performance prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network parameters.
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models
Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance.
Meta-prediction Model for Distillation-Aware NAS on Unseen Datasets
Previous DaNAS methods have mostly tackled the search for the neural architecture for fixed datasets and the teacher, which are not generalized well on a new task consisting of an unseen dataset and an unseen teacher, thus need to perform a costly search for any new combination of the datasets and the teachers.
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks
Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks that require a compound understanding of knowledge.
Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation
To overcome this limitation, we propose SUbgraph Retrieval-augmented GEneration (SURGE), a framework for generating context-relevant and knowledge-grounded dialogues with the KG.
Context-Preserving Two-Stage Video Domain Translation for Portrait Stylization
Portrait stylization, which translates a real human face image into an artistically stylized image, has attracted considerable interest and many prior works have shown impressive quality in recent years.
Phrase Retrieval for Open-Domain Conversational Question Answering with Conversational Dependency Modeling via Contrastive Learning
To address this problem, we further introduce a novel contrastive learning strategy, making sure to reflect previous turns when retrieving the phrase for the current context, by maximizing representational similarities of consecutive turns in a conversation while minimizing irrelevant conversational contexts.
Progressive Fourier Neural Representation for Sequential Video Compilation
Motivated by continual learning, this work investigates how to accumulate and transfer neural implicit representations for multiple complex video data over sequential encoding sessions.
Continual Learners are Incremental Model Generalizers
We believe this paper breaks the barriers between pre-training and fine-tuning steps and leads to a sustainable learning framework in which the continual learner incrementally improves model generalization, yielding better transfer to unseen tasks.
Drug Discovery with Dynamic Goal-aware Fragments
Additionally, the existing fragment-based generative models cannot update the fragment vocabulary with goal-aware fragments newly discovered during the generation.
SEA: Sparse Linear Attention with Estimated Attention Mask
SEA estimates the attention matrix with linear complexity via kernel-based linear attention, then subsequently creates a sparse attention matrix with a top-k selection to perform a sparse attention operation.
Self-Supervised Dataset Distillation for Transfer Learning
To achieve this, we also introduce the MSE between representations of the inner model and the self-supervised target model on the original full dataset for outer optimization.
Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes
Learning the distribution of data on Riemannian manifolds is crucial for modeling data from non-Euclidean space, which is required by many applications from diverse scientific fields.
STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment
Continuously learning a variety of audio-video semantics over time is crucial for audio-related reasoning tasks in our ever-evolving world.
Knowledge-Augmented Language Model Verification
Recent Language Models (LMs) have shown impressive capabilities in generating texts with the knowledge internalized in parameters.
Test-Time Self-Adaptive Small Language Models for Question Answering
Moreover, further finetuning LMs with labeled datasets is often infeasible due to their absence, but it is also questionable if we can transfer smaller LMs having limited knowledge only with unlabeled test data.
Co-training and Co-distillation for Quality Improvement and Compression of Language Models
2) The enhanced performance of the larger model further boosts the performance of the smaller model.
Context-dependent Instruction Tuning for Dialogue Response Generation
Recent language models have achieved impressive performance in natural language tasks by incorporating instructions with task input during fine-tuning.
Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Models
The dynamic nature of knowledge in an ever-changing world presents challenges for language models trained on static data; the model in the real world often requires not only acquiring new knowledge but also overwriting outdated information into updated ones.
KOALA: Self-Attention Matters in Knowledge Distillation of Latent Diffusion Models for Memory-Efficient and Fast Image Synthesis
We hope that due to its balanced speed-performance tradeoff, our KOALA models can serve as a cost-effective alternative to SDXL in resource-constrained environments.
LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
We propose a framework that leverages foundation models as teachers, guiding a reinforcement learning agent to acquire semantically meaningful behavior without human feedback.
Continual Learning: Forget-free Winning Subnetworks for Video Representations
Inspired by the Lottery Ticket Hypothesis (LTH), which highlights the existence of efficient subnetworks within larger, dense networks, a high-performing Winning Subnetwork (WSN) in terms of task performance under appropriate sparsity conditions is considered for various continual learning tasks.
BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation
We validate our method outperforms multiple CTTA scenarios including disjoint and gradual domain shits, while only requiring ~98% fewer trainable parameters.
Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks
Many existing works tackle this problem by generating synthetic data from the training data and then training models on them, recently using Large Language Models (LLMs).
Diffusion-based Neural Network Weights Generation
By learning the distribution of a neural network on a variety pretrained models, our approach enables adaptive sampling weights for unseen datasets achieving faster convergence and reaching competitive performance.
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
Retrieval-Augmented Large Language Models (LLMs), which incorporate the non-parametric knowledge from external knowledge bases into LLMs, have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA).
ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning
Panoptic segmentation, combining semantic and instance segmentation, stands as a cutting-edge computer vision task.
Towards Label-Efficient Human Matting: A Simple Baseline for Weakly Semi-Supervised Trimap-Free Human Matting
To address this challenge, we introduce a new learning paradigm, weakly semi-supervised human matting (WSSHM), which leverages a small amount of expensive matte labels and a large amount of budget-friendly segmentation labels, to save the annotation cost and resolve the domain generalization problem.
Rethinking Saliency-Guided Weakly-Supervised Semantic Segmentation
This paper presents a fresh perspective on the role of saliency maps in weakly-supervised semantic segmentation (WSSS) and offers new insights and research directions based on our empirical findings.
Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models
Our experiments demonstrate that MuDI can produce high-quality personalized images without identity mixing, even for highly similar subjects as shown in Figure 1.
ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
Scientific Research, vital for improving human life, is hindered by its inherent complexity, slow pace, and the need for specialized experts.
Cost-effective Interactive Attention Learning with Neural Attention Process
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
Meta Variance Transfer: Learning to Augment from the Others
Humans have the ability to robustly recognize objects with various factors of variations such as nonrigid transformation, background noise, and change in lighting conditions.
