Search Results for author:
Found 16 papers, 6 papers with code
On the Learning Dynamics of Attention Networks
Attention models are typically learned by optimizing one of three standard loss functions that are variously called -- soft attention, hard attention, and latent variable marginal likelihood (LVML) attention.
On the Interpretability of Attention Networks
In trained models with an attention mechanism, the outputs of an intermediate module that encodes the segment of input responsible for the output is often used as a way to peek into the `reasoning` of the network.
Consistent Multiclass Algorithms for Complex Metrics and Constraints
We present consistent algorithms for multiclass learning with complex performance metrics and constraints, where the objective and constraints are defined by arbitrary functions of the confusion matrix.
Predicting the success of Gradient Descent for a particular Dataset-Architecture-Initialization (DAI)
Despite their massive success, training successful deep neural networks still largely relies on experimentally choosing an architecture, hyper-parameters, initialization, and training mechanism.
Using noise resilience for ranking generalization of deep neural networks
Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels.
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets
We analyse both standard weight normalization (SWN) and exponential weight normalization (EWN), and show that the gradient flow path with EWN is equivalent to gradient flow on standard networks with an adaptive learning rate.
Convex Calibrated Surrogates for the Multi-Label F-Measure
In particular, the F-measure explicitly balances recall (fraction of active labels predicted to be active) and precision (fraction of labels predicted to be active that are actually so), both of which are important in evaluating the overall performance of a multi-label classifier.
Ablation-CAM: Visual Explanations for Deep Convolutional Network via Gradient-free Localization
In response to recent criticism of gradient-based visualization techniques, we propose a new methodology to generate visual explanations for deep Convolutional Neural Networks (CNN) - based models.
On Knowledge distillation from complex networks for response prediction
In order to overcome this, we use standard simple models which do not capture all pairwise interactions, but learn to emulate certain characteristics of a complex teacher network.
On Controllable Sparse Alternatives to Softmax
Converting an n-dimensional vector to a probability distribution over n objects is a commonly used component in many machine learning tasks like multiclass classification, multilabel classification, attention mechanisms etc.
Mixture Proportion Estimation via Kernel Embedding of Distributions
Mixture proportion estimation (MPE) is the problem of estimating the weight of a component distribution in a mixture, given samples from the mixture and component.
Consistent Algorithms for Multiclass Classification with a Reject Option
We consider the problem of $n$-class classification ($n\geq 2$), where the classifier can choose to abstain from making predictions at a given cost, say, a factor $\alpha$ of the cost of misclassification.
Consistent Classification Algorithms for Multi-class Non-Decomposable Performance Metrics
In this paper, we provide a unified framework for analysing a multi-class non-decomposable performance metric, where the problem of finding the optimal classifier for the performance metric is viewed as an optimization problem over the space of all confusion matrices achievable under the given distribution.
Convex Calibration Dimension for Multiclass Loss Matrices
We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning problems), to the general multiclass setting, and derive necessary and sufficient conditions for a surrogate loss to be calibrated with respect to a loss matrix in this setting.
Convex Calibrated Surrogates for Low-Rank Loss Matrices with Applications to Subset Ranking Losses
The design of convex, calibrated surrogate losses, whose minimization entails consistency with respect to a desired target loss, is an important concept to have emerged in the theory of machine learning in recent years.
Classification Calibration Dimension for General Multiclass Losses
We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning problems), to the general multiclass setting, and derive necessary and sufficient conditions for a surrogate loss to be classification calibrated with respect to a loss matrix in this setting.
