Search Results for author:
Found 44 papers, 6 papers with code
Time-Consistent Self-Supervision for Semi-Supervised Learning
In this paper, we study the dynamics of neural net outputs in SSL and show that selecting and using first the unlabeled samples with more consistent outputs over the course of training (i. e., "time-consistency") can improve the final test accuracy and save computation.
Deep Submodular Peripteral Networks
In this paper, we introduce deep submodular peripteral networks (DSPNs), a novel parametric family of submodular functions, and methods for their training using a contrastive-learning inspired GPC-ready strategy to connect and then tackle both of the above challenges.
Many-Objective Multi-Solution Transport
Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning.
Effective Backdoor Mitigation Depends on the Pre-training Objective
In this work, we demonstrate that the efficacy of CleanCLIP in mitigating backdoors is highly dependent on the particular objective used during model pre-training.
Accelerating Batch Active Learning Using Continual Learning Techniques
A major problem with Active Learning (AL) is high training costs since models are typically retrained from scratch after every query round.
Online SuBmodular + SuPermodular (BP) Maximization with Bandit Feedback
At round $i$, a user with unknown utility $h_q$ arrives; the optimizer selects a new item to add to $S_q$, and receives a noisy marginal gain.
Submodularity In Machine Learning and Artificial Intelligence
In this manuscript, we offer a gentle review of submodularity and supermodularity and their properties.
Abstractive Text Summarization
BIG-bench Machine Learning
+1
Diverse Client Selection for Federated Learning via Submodular Maximization
In every communication round of federated learning, a random subset of clients communicate their model updates back to the server which then aggregates them all.
An Effective Baseline for Robustness to Distributional Shift
In this work we present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention: when encountering a sample from an unseen class, the desired behavior is to abstain from predicting.
Ranked #1 on
Out-of-Distribution Detection
on CIFAR-100
(using extra training data)
Submodular Mutual Information for Targeted Data Subset Selection
With the rapid growth of data, it is becoming increasingly difficult to train or improve deep learning models with the right subset of data.
PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection
Examples of such problems include: i)targeted learning, where the goal is to find subsets with rare classes or rare attributes on which the model is underperforming, and ii)guided summarization, where data (e. g., image collection, text, document or video) is summarized for quicker human consumption with specific additional user intent.
Robust Curriculum Learning: from clean label detection to noisy label self-correction
Neural nets training can easily overfit to noisy labels and end with poor generalization performance.
A Simple and Effective Baseline for Out-of-Distribution Detection using Abstention
In this work we present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention: when encountering a sample from an unseen class, the desired behavior is to abstain from predicting.
A Unified Framework for Generic, Query-Focused, Privacy Preserving and Update Summarization using Submodular Information Measures
We study submodular information measures as a rich framework for generic, query-focused, privacy sensitive, and update summarization tasks.
Concave Aspects of Submodular Functions
In this paper, we try to provide a more complete picture of the relationship between submodularity with concavity.
Submodular Combinatorial Information Measures with Applications in Machine Learning
In this paper, we study combinatorial information measures that generalize independence, (conditional) entropy, (conditional) mutual information, and total correlation defined over sets of (not necessarily random) variables.
Coresets for Accelerating Incremental Gradient Methods
But because at each epoch the gradients are computed only on the subset S, we obtain a speedup that is inversely proportional to the size of S. Our subset selection algorithm is fully general and can be applied to most IG methods.
Coresets for Data-efficient Training of Machine Learning Models
Here we develop CRAIG, a method to select a weighted subset (or coreset) of training data that closely estimates the full gradient by maximizing a submodular function.
On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks
In this work, we discuss a hitherto untouched aspect of mixup training -- the calibration and predictive uncertainty of models trained with mixup.
Ranked #1 on
Out-of-Distribution Detection
on STL-10
Combating Label Noise in Deep Learning Using Abstention
In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise.
Bias Also Matters: Bias Attribution for Deep Neural Network Explanation
In particular, we study how to attribute a DNN's bias to its input features.
Jumpout: Improved Dropout for Deep Neural Networks with Rectified Linear Units
In this paper, we discuss three novel observations about dropout to better understand the generalization of DNNs with rectified linear unit (ReLU) activations: 1) dropout is a smoothing technique that encourages each local linear model of a DNN to be trained on data points from nearby regions; 2) a constant dropout rate can result in effective neural-deactivation rates that are significantly different for layers with different fractions of activated neurons; and 3) the rescaling factor of dropout causes an inconsistency to occur between the normalization during training and testing conditions when batch normalization is also used.
A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems
We are motivated by large scale submodular optimization problems, where standard algorithms that treat the submodular functions in the \emph{value oracle model} do not scale.
Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs
In this paper, we investigate a class of submodular problems which in general are very hard.
Constrained Interacting Submodular Groupings
We introduce the problem of grouping a finite ground set into blocks where each block is a subset of the ground set and where: (i) the blocks are individually highly valued by a submodular function (both robustly and in the average case) while satisfying block-specific matroid constraints; and (ii) block scores interact where blocks are jointly scored highly, thus making the blocks mutually non-redundant.
Greed is Still Good: Maximizing Monotone Submodular+Supermodular (BP) Functions
We analyze the performance of the greedy algorithm, and also a discrete semi-gradient based algorithm, for maximizing the sum of a suBmodular and suPermodular (BP) function (both of which are non-negative monotone non-decreasing) under two types of constraints, either a cardinality constraint or $p\geq 1$ matroid independence constraints.
Minimax Curriculum Learning: Machine Teaching with Desirable Difficulties and Scheduled Diversity
We introduce and study minimax curriculum learning (MCL), a new method for adaptively selecting a sequence of training subsets for a succession of stages in machine learning.
Stream Clipper: Scalable Submodular Maximization on Stream
We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice.
Scaling Submodular Maximization via Pruned Submodularity Graphs
We propose a new random pruning method (called "submodular sparsification (SS)") to reduce the cost of submodular maximization.
On Deep Multi-View Representation Learning: Objectives and Optimization
We consider learning representations (features) in the setting in which we have access to multiple unlabeled views of the data for learning while only one view is available for downstream tasks.
Submodular Hamming Metrics
We show that there is a largely unexplored class of functions (positive polymatroids) that can define proper discrete metrics over pairs of binary vectors and that are fairly tractable to optimize over.
Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications to Parallel Machine Learning and Multi-Label Image Segmentation
While the robust versions have been studied in the theory community, existing work has focused on tight approximation guarantees, and the resultant algorithms are not, in general, scalable to very large real-world applications.
Polyhedral aspects of Submodularity, Convexity and Concavity
This manuscript provides a more complete picture on the relationship between submodularity with convexity and concavity, by extending many of the results connecting submodularity with convexity to the concave aspects of submodularity.
Discrete Mathematics Data Structures and Algorithms
Divide-and-Conquer Learning by Anchoring a Conical Hull
We reduce a broad class of machine learning problems, usually addressed by EM or sampling, to the problem of finding the $k$ extremal rays spanning the conical hull of a data point set.
Graph Cuts with Interacting Edge Costs - Examples, Approximations, and Algorithms
We study an extension of the classical graph cut problem, wherein we replace the modular (sum of edge weights) cost function by a submodular set function defined over graph edges.
Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions
We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization).
Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints
We are motivated by a number of real-world applications in machine learning including sensor placement and data subset selection, which require maximizing a certain submodular function (like coverage or diversity) while simultaneously minimizing another (like cooperative cost).
The Lovasz-Bregman Divergence and connections to rank aggregation, clustering, and web ranking
We show how a number of recently used web ranking models are forms of Lovasz-Bregman rank aggregation and also observe that a natural form of Mallow's model using the LB divergence has been used as conditional ranking models for the 'Learning to Rank' problem.
Fast Semidifferential-based Submodular Function Optimization
We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials).
Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications
We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a difference between submodular functions.
Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (2009)
This is the Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, which was held in Montreal, QC, Canada, June 18 - 21 2009.
