Search Results for author:
Found 93 papers, 35 papers with code
Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class.
Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation
In this work, we propose StyleSpeech, a new TTS model which not only synthesizes high-quality speech but also effectively adapts to new speakers.
Lifelong Learning with Dynamically Expandable Networks
We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks.
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks
While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed.
Federated Continual Learning with Weighted Inter-client Transfer
There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios.
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning
Through extensive experimental validation of our method in the two different scenarios, we show that our method outperforms both local semi-supervised learning and baselines which naively combine federated learning with semi-supervised learning.
Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced.
TAM: Topology-Aware Margin Loss for Class-Imbalanced Node Classification
Learning unbiased node representations under class-imbalanced graph data is challenging due to interactions between adjacent nodes.
Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them.
DropMax: Adaptive Variational Softmax
Moreover, the learning of dropout rates for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare
We present a personalized and reliable prediction model for healthcare, which can provide individually tailored medical services such as diagnosis, disease treatment, and prevention.
Unbiased Classification through Bias-Contrastive and Bias-Balanced Learning
In such a biased dataset, models are susceptible to making predictions based on the biased features of the data.
Distilling Linguistic Context for Language Model Compression
A computationally expensive and memory intensive neural network lies behind the recent success of language representation learning.
Learning Input-agnostic Manipulation Directions in StyleGAN with Text Guidance
With the advantages of fast inference and human-friendly flexible manipulation, image-agnostic style manipulation via text guidance enables new applications that were not previously available.
SGEM: Test-Time Adaptation for Automatic Speech Recognition via Sequential-Level Generalized Entropy Minimization
Automatic speech recognition (ASR) models are frequently exposed to data distribution shifts in many real-world scenarios, leading to erroneous predictions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Meta Dropout: Learning to Perturb Features for Generalization
Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.
Meta Dropout: Learning to Perturb Latent Features for Generalization
Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.
Ranked #1 on
Meta-Learning
on OMNIGLOT - 1-Shot, 20-way
Bootstrapping Neural Processes
While this "data-driven" way of learning stochastic processes has proven to handle various types of data, NPs still rely on an assumption that uncertainty in stochastic processes is modeled by a single latent variable, which potentially limits the flexibility.
Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Deep Asymmetric Multi-task Feature Learning
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process.
Cost-effective Interactive Attention Learning with Neural Attention Processes
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning
Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss.
Attribution Preservation in Network Compression for Reliable Network Interpretation
Neural networks embedded in safety-sensitive applications such as self-driving cars and wearable health monitors rely on two important techniques: input attribution for hindsight analysis and network compression to reduce its size for edge-computing.
Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number of tasks.
Neural Complexity Measures
While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging.
FedMix: Approximation of Mixup under Mean Augmented Federated Learning
Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device, thus preserving privacy and eliminating the need to store data globally.
Time-Reversal Symmetric ODE Network
Time-reversal symmetry, which requires that the dynamics of a system should not change with the reversal of time axis, is a fundamental property that frequently holds in classical and quantum mechanics.
Towards the Practical Utility of Federated Learning in the Medical Domain
We evaluate six FL algorithms designed for addressing data heterogeneity among clients, and a hybrid algorithm combining the strengths of two representative FL algorithms.
Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks
Among many approaches, the simplest and most intuitive way is zero imputation, which treats the value of a missing entry simply as zero.
Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images
4D medical images, which represent 3D images with temporal information, are crucial in clinical practice for capturing dynamic changes and monitoring long-term disease progression.
TTD: Text-Tag Self-Distillation Enhancing Image-Text Alignment in CLIP to Alleviate Single Tag Bias
We identify a critical bias in contemporary CLIP-based models, which we denote as \textit{single tag bias}.
Ranked #1 on
Open Vocabulary Semantic Segmentation
on COCO-Stuff-171
(mIoU metric)
Multi-Label Text Classification
Open Vocabulary Semantic Segmentation
+3
Cost-effective Interactive Attention Learning with Neural Attention Process
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
TEDDY: Trimming Edges with Degree-based Discrimination strategY
Since the pioneering work on the lottery ticket hypothesis for graph neural networks (GNNs) was proposed in Chen et al. (2021), the study on finding graph lottery tickets (GLT) has become one of the pivotal focus in the GNN community, inspiring researchers to discover sparser GLT while achieving comparable performance to original dense networks.
Multitask Learning using Task Clustering with Applications to Predictive Modeling and GWAS of Plant Varieties
Inferring predictive maps between multiple input and multiple output variables or tasks has innumerable applications in data science.
Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification
We consider the problem of active feature acquisition, where we sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way.
Learning task structure via sparsity grouped multitask learning
We further demonstrate that our proposed method recovers groups and the sparsity patterns in the task parameters accurately by extensive experiments.
A General Family of Trimmed Estimators for Robust High-dimensional Data Analysis
We consider the problem of robustifying high-dimensional structured estimation.
Sequential Local Learning for Latent Graphical Models
Learning parameters of latent graphical models (GM) is inherently much harder than that of no-latent ones since the latent variables make the corresponding log-likelihood non-concave.
A Review of Multivariate Distributions for Count Data Derived from the Poisson Distribution
The Poisson distribution has been widely studied and used for modeling univariate count-valued data.
Robust Gaussian Graphical Modeling with the Trimmed Graphical Lasso
In this paper, we propose the Trimmed Graphical Lasso for robust estimation of sparse GGMs.
On Graphical Models via Univariate Exponential Family Distributions
Undirected graphical models, or Markov networks, are a popular class of statistical models, used in a wide variety of applications.
A General Framework for Mixed Graphical Models
"Mixed Data" comprising a large number of heterogeneous variables (e. g. count, binary, continuous, skewed continuous, among other data types) are prevalent in varied areas such as genomics and proteomics, imaging genetics, national security, social networking, and Internet advertising.
Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding
We consider the problem of active feature acquisition where the goal is to sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way at test time.
Closed-form Estimators for High-dimensional Generalized Linear Models
We propose a class of closed-form estimators for GLMs under high-dimensional sampling regimes.
Elementary Estimators for Graphical Models
We propose a class of closed-form estimators for sparsity-structured graphical models, expressed as exponential family distributions, under high-dimensional settings.
Dirty Statistical Models
We provide a unified framework for the high-dimensional analysis of “superposition-structured” or “dirty” statistical models: where the model parameters are a “superposition” of structurally constrained parameters.
Conditional Random Fields via Univariate Exponential Families
We thus introduce a “novel subclass of CRFs”, derived by imposing node-wise conditional distributions of response variables conditioned on the rest of the responses and the covariates as arising from univariate exponential families.
On Poisson Graphical Models
Undirected graphical models, such as Gaussian graphical models, Ising, and multinomial/categorical graphical models, are widely used in a variety of applications for modeling distributions over a large number of variables.
Graphical Models via Generalized Linear Models
Our models allow one to estimate networks for a wide class of exponential distributions, such as the Poisson, negative binomial, and exponential, by fitting penalized GLMs to select the neighborhood for each node.
Ordinal Graphical Models: A Tale of Two Approaches
While there have been some work on tractable approximations, these do not come with strong statistical guarantees, and moreover are relatively computationally expensive.
Sparse + Group-Sparse Dirty Models: Statistical Guarantees without Unreasonable Conditions and a Case for Non-Convexity
Imposing sparse + group-sparse superposition structures in high-dimensional parameter estimation is known to provide flexible regularization that is more realistic for many real-world problems.
DropMax: Adaptive Stochastic Softmax
Moreover, the learning of dropout probabilities for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
Spectral Approximate Inference
Our contribution is two-fold: (a) we first propose a fully polynomial-time approximation scheme (FPTAS) for approximating the partition function of GM associating with a low-rank coupling matrix; (b) for general high-rank GMs, we design a spectral mean-field scheme utilizing (a) as a subroutine, where it approximates a high-rank GM into a product of rank-1 GMs for an efficient approximation of the partition function.
Stochastic Gradient Methods with Block Diagonal Matrix Adaptation
Extensive experiments reveal that block-diagonal approaches achieve state-of-the-art results on several deep learning tasks, and can outperform adaptive diagonal methods, vanilla Sgd, as well as a modified version of full-matrix adaptation proposed very recently.
Reliable Estimation of Individual Treatment Effect with Causal Information Bottleneck
We also introduce an additional form of a regularizer from the perspective of understanding ITE in the semi-supervised learning framework to ensure more reliable representations.
Ranked #1 on
Causal Inference
on IDHP
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Set Based Stochastic Subsampling
Deep models are designed to operate on huge volumes of high dimensional data such as images.
A Revision of Neural Tangent Kernel-based Approaches for Neural Networks
(2) A generalization error bound invariant of network size was derived by using a data-dependent complexity measure (CMD).
Compressed Sensing via Measurement-Conditional Generative Models
A pre-trained generator has been frequently adopted in compressed sensing (CS) due to its ability to effectively estimate signals with the prior of NNs.
A General Family of Stochastic Proximal Gradient Methods for Deep Learning
We propose a unified framework for stochastic proximal gradient descent, which we term ProxGen, that allows for arbitrary positive preconditioners and lower semi-continuous regularizers.
Learning to Sample with Local and Global Contexts in Experience Replay Buffer
Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL).
Few-shot Visual Reasoning with Meta-analogical Contrastive Learning
While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task.
Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bitwise Regularization
As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.
Rapid Neural Pruning for Novel Datasets with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Contextual Knowledge Distillation for Transformer Compression
A computationally expensive and memory intensive neural network lies behind the recent success of language representation learning.
Model-Augmented Q-learning
We show that the proposed scheme, called Model-augmented $Q$-learning (MQL), obtains a policy-invariant solution which is identical to the solution obtained by learning with true reward.
Mutually-Constrained Monotonic Multihead Attention for Online ASR
Despite the feature of real-time decoding, Monotonic Multihead Attention (MMA) shows comparable performance to the state-of-the-art offline methods in machine translation and automatic speech recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning
Retrosynthesis, of which the goal is to find a set of reactants for synthesizing a target product, is an emerging research area of deep learning.
Online Coreset Selection for Rehearsal-based Continual Learning
We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.
Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss
Network quantization, which aims to reduce the bit-lengths of the network weights and activations, has emerged for their deployments to resource-limited devices.
Online Hyperparameter Meta-Learning with Hypergradient Distillation
Many gradient-based meta-learning methods assume a set of parameters that do not participate in inner-optimization, which can be considered as hyperparameters.
Stop just recalling memorized relations: Extracting Unseen Relational Triples from the context
In this paper, we show that although existing extraction models are able to memorize and recall already seen triples, they cannot generalize effectively for unseen triples.
Bias Decay Matters : Improving Large Batch Optimization with Connectivity Sharpness
To mitigate this issue, we propose simple bias decay methods including a novel adaptive one and found that this simple remedy can fill a large portion of the performance gaps that occur in large batch optimization.
Model-augmented Prioritized Experience Replay
Experience replay is an essential component in off-policy model-free reinforcement learning (MfRL).
Task Conditioned Stochastic Subsampling
Deep Learning algorithms are designed to operate on huge volumes of high dimensional data such as images.
GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification
In many real-world node classification scenarios, nodes are highly class-imbalanced, where graph neural networks (GNNs) can be readily biased to major class instances.
Graph Transplant: Node Saliency-Guided Graph Mixup with Local Structure Preservation
Graph-structured datasets usually have irregular graph sizes and connectivities, rendering the use of recent data augmentation techniques, such as Mixup, difficult.
Adaptive Proximal Gradient Methods for Structured Neural Networks
We consider the training of structured neural networks where the regularizer can be non-smooth and possibly non-convex.
ADAPTIVE NETWORK SPARSIFICATION VIA DEPENDENT VARIATIONAL BETA-BERNOULLI DROPOUT
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Temporal Probabilistic Asymmetric Multi-task Learning
To capture such dynamically changing asymmetric relationships between tasks and long-range temporal dependencies in time-series data, we propose a novel temporal asymmetric multi-task learning model, which learns to combine features from other tasks at diverse timesteps for the prediction of each task.
Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation
Deep neural networks (DNNs), despite their impressive ability to generalize over-capacity networks, often rely heavily on malignant bias as shortcuts instead of task-related information for discriminative tasks.
Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing
However, they now suffer from lack of sample diversification as they always deterministically select regions with maximum saliency, injecting bias into the augmented data.
Does it Really Generalize Well on Unseen Data? Systematic Evaluation of Relational Triple Extraction Methods
To keep a knowledge graph up-to-date, an extractor needs not only the ability to recall the triples it encountered during training, but also the ability to extract the new triples from the context that it has never seen before.
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel
Explaining generalizations and preventing over-confident predictions are central goals of studies on the loss landscape of neural networks.
Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding
Inspired by the impressive performance of recent face image editing methods, several studies have been naturally proposed to extend these methods to the face video editing task.
BiasAdv: Bias-Adversarial Augmentation for Model Debiasing
Our key idea is that an adversarial attack on a biased model that makes decisions based on spurious correlations may generate synthetic bias-conflicting samples, which can then be used as augmented training data for learning a debiased model.
ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models
Emotional Text-To-Speech (TTS) is an important task in the development of systems (e. g., human-like dialogue agents) that require natural and emotional speech.
BackTrack: Robust template update via Backward Tracking of candidate template
An effective method to tackle these challenges is template update, which updates the template to reflect the change of appearance in the target object during tracking.
PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label
Understanding point clouds captured from the real-world is challenging due to shifts in data distribution caused by varying object scales, sensor angles, and self-occlusion.
Face-StyleSpeech: Improved Face-to-Voice latent mapping for Natural Zero-shot Speech Synthesis from a Face Image
The prosody encoder is specifically designed to model prosodic features that are not captured only with a face image, allowing the face encoder to focus solely on capturing the speaker identity from the face image.
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
Recent advancements in large language models (LLMs) have raised concerns about inference costs, increasing the need for research into model compression.
No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs).
