Search Results for author: Jeff Bilmes

Found 39 papers, 6 papers with code

Time-Consistent Self-Supervision for Semi-Supervised Learning

no code implementations ICML 2020 Tianyi Zhou, Shengjie Wang, Jeff Bilmes

In this paper, we study the dynamics of neural net outputs in SSL and show that selecting and using first the unlabeled samples with more consistent outputs over the course of training (i. e., "time-consistency") can improve the final test accuracy and save computation.

Submodularity In Machine Learning and Artificial Intelligence

no code implementations31 Jan 2022 Jeff Bilmes

In this manuscript, we offer a gentle review of submodularity and supermodularity and their properties.

Abstractive Text Summarization feature selection

Diverse Client Selection for Federated Learning via Submodular Maximization

no code implementations ICLR 2022 Ravikumar Balakrishnan, Tian Li, Tianyi Zhou, Nageen Himayat, Virginia Smith, Jeff Bilmes

In every communication round of federated learning, a random subset of clients communicate their model updates back to the server which then aggregates them all.

Fairness Federated Learning

An Effective Baseline for Robustness to Distributional Shift

1 code implementation15 May 2021 Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, Jeff Bilmes

In this work we present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention: when encountering a sample from an unseen class, the desired behavior is to abstain from predicting.

 Ranked #1 on Out-of-Distribution Detection on 20 Newsgroups (using extra training data)

Out-of-Distribution Detection Robust classification

Submodular Mutual Information for Targeted Data Subset Selection

no code implementations30 Apr 2021 Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan, Jeff Bilmes, Rishabh Iyer

With the rapid growth of data, it is becoming increasingly difficult to train or improve deep learning models with the right subset of data.

Active Learning Image Classification

PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection

1 code implementation27 Feb 2021 Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan, Jeff Bilmes, Rishabh Iyer

Examples of such problems include: i)targeted learning, where the goal is to find subsets with rare classes or rare attributes on which the model is underperforming, and ii)guided summarization, where data (e. g., image collection, text, document or video) is summarized for quicker human consumption with specific additional user intent.

Image Classification

Robust Curriculum Learning: from clean label detection to noisy label self-correction

no code implementations ICLR 2021 Tianyi Zhou, Shengjie Wang, Jeff Bilmes

Neural nets training can easily overfit to noisy labels and end with poor generalization performance.

A Simple and Effective Baseline for Out-of-Distribution Detection using Abstention

no code implementations1 Jan 2021 Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, Jeff Bilmes

In this work we present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention: when encountering a sample from an unseen class, the desired behavior is to abstain from predicting.

Out-of-Distribution Detection Text Classification

Submodular Combinatorial Information Measures with Applications in Machine Learning

no code implementations27 Jun 2020 Rishabh Iyer, Ninad Khargonkar, Jeff Bilmes, Himanshu Asnani

In this paper, we study combinatorial information measures that generalize independence, (conditional) entropy, (conditional) mutual information, and total correlation defined over sets of (not necessarily random) variables.

Privacy Preserving

Concave Aspects of Submodular Functions

no code implementations27 Jun 2020 Rishabh Iyer, Jeff Bilmes

In this paper, we try to provide a more complete picture of the relationship between submodularity with concavity.

Coresets for Accelerating Incremental Gradient Methods

no code implementations25 Sep 2019 Baharan Mirzasoleiman, Jeff Bilmes, Jure Leskovec

But because at each epoch the gradients are computed only on the subset S, we obtain a speedup that is inversely proportional to the size of S. Our subset selection algorithm is fully general and can be applied to most IG methods.

Coresets for Data-efficient Training of Machine Learning Models

2 code implementations ICML 2020 Baharan Mirzasoleiman, Jeff Bilmes, Jure Leskovec

Here we develop CRAIG, a method to select a weighted subset (or coreset) of training data that closely estimates the full gradient by maximizing a submodular function.

Combating Label Noise in Deep Learning Using Abstention

2 code implementations27 May 2019 Sunil Thulasidasan, Tanmoy Bhattacharya, Jeff Bilmes, Gopinath Chennupati, Jamal Mohd-Yusof

In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise.

General Classification Image Classification +1

Jumpout: Improved Dropout for Deep Neural Networks with Rectified Linear Units

no code implementations ICLR 2019 Shengjie Wang, Tianyi Zhou, Jeff Bilmes

In this paper, we discuss three novel observations about dropout to better understand the generalization of DNNs with rectified linear unit (ReLU) activations: 1) dropout is a smoothing technique that encourages each local linear model of a DNN to be trained on data points from nearby regions; 2) a constant dropout rate can result in effective neural-deactivation rates that are significantly different for layers with different fractions of activated neurons; and 3) the rescaling factor of dropout causes an inconsistency to occur between the normalization during training and testing conditions when batch normalization is also used.

A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems

no code implementations26 Feb 2019 Rishabh Iyer, Jeff Bilmes

We are motivated by large scale submodular optimization problems, where standard algorithms that treat the submodular functions in the \emph{value oracle model} do not scale.

Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs

no code implementations26 Feb 2019 Rishabh Iyer, Jeff Bilmes

In this paper, we investigate a class of submodular problems which in general are very hard.

Greed is Still Good: Maximizing Monotone Submodular+Supermodular (BP) Functions

no code implementations ICML 2018 Wenruo Bai, Jeff Bilmes

We analyze the performance of the greedy algorithm, and also a discrete semi-gradient based algorithm, for maximizing the sum of a suBmodular and suPermodular (BP) function (both of which are non-negative monotone non-decreasing) under two types of constraints, either a cardinality constraint or $p\geq 1$ matroid independence constraints.

Constrained Interacting Submodular Groupings

no code implementations ICML 2018 Andrew Cotter, Mahdi Milani Fard, Seungil You, Maya Gupta, Jeff Bilmes

We introduce the problem of grouping a finite ground set into blocks where each block is a subset of the ground set and where: (i) the blocks are individually highly valued by a submodular function (both robustly and in the average case) while satisfying block-specific matroid constraints; and (ii) block scores interact where blocks are jointly scored highly, thus making the blocks mutually non-redundant.

Minimax Curriculum Learning: Machine Teaching with Desirable Difficulties and Scheduled Diversity

no code implementations ICLR 2018 Tianyi Zhou, Jeff Bilmes

We introduce and study minimax curriculum learning (MCL), a new method for adaptively selecting a sequence of training subsets for a succession of stages in machine learning.

Stream Clipper: Scalable Submodular Maximization on Stream

no code implementations1 Jun 2016 Tianyi Zhou, Jeff Bilmes

We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice.

Video Summarization

Scaling Submodular Maximization via Pruned Submodularity Graphs

no code implementations1 Jun 2016 Tianyi Zhou, Hua Ouyang, Yi Chang, Jeff Bilmes, Carlos Guestrin

We propose a new random pruning method (called "submodular sparsification (SS)") to reduce the cost of submodular maximization.

Video Summarization

On Deep Multi-View Representation Learning: Objectives and Optimization

1 code implementation2 Feb 2016 Weiran Wang, Raman Arora, Karen Livescu, Jeff Bilmes

We consider learning representations (features) in the setting in which we have access to multiple unlabeled views of the data for learning while only one view is available for downstream tasks.

Representation Learning Stochastic Optimization

Submodular Hamming Metrics

no code implementations NeurIPS 2015 Jennifer Gillenwater, Rishabh Iyer, Bethany Lusch, Rahul Kidambi, Jeff Bilmes

We show that there is a largely unexplored class of functions (positive polymatroids) that can define proper discrete metrics over pairs of binary vectors and that are fairly tractable to optimize over.

Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications to Parallel Machine Learning and Multi-Label Image Segmentation

no code implementations NeurIPS 2015 Kai Wei, Rishabh Iyer, Shengjie Wang, Wenruo Bai, Jeff Bilmes

While the robust versions have been studied in the theory community, existing work has focused on tight approximation guarantees, and the resultant algorithms are not, in general, scalable to very large real-world applications.

Distributed Optimization Interactive Segmentation +1

Polyhedral aspects of Submodularity, Convexity and Concavity

no code implementations24 Jun 2015 Rishabh Iyer, Jeff Bilmes

This manuscript provides a more complete picture on the relationship between submodularity with convexity and concavity, by extending many of the results connecting submodularity with convexity to the concave aspects of submodularity.

Discrete Mathematics Data Structures and Algorithms

Divide-and-Conquer Learning by Anchoring a Conical Hull

no code implementations NeurIPS 2014 Tianyi Zhou, Jeff Bilmes, Carlos Guestrin

We reduce a broad class of machine learning problems, usually addressed by EM or sampling, to the problem of finding the $k$ extremal rays spanning the conical hull of a data point set.

Graph Cuts with Interacting Edge Costs - Examples, Approximations, and Algorithms

no code implementations2 Feb 2014 Stefanie Jegelka, Jeff Bilmes

We study an extension of the classical graph cut problem, wherein we replace the modular (sum of edge weights) cost function by a submodular set function defined over graph edges.

Computer Vision

Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions

no code implementations NeurIPS 2013 Rishabh Iyer, Stefanie Jegelka, Jeff Bilmes

We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization).

Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints

no code implementations NeurIPS 2013 Rishabh Iyer, Jeff Bilmes

We are motivated by a number of real-world applications in machine learning including sensor placement and data subset selection, which require maximizing a certain submodular function (like coverage or diversity) while simultaneously minimizing another (like cooperative cost).

The Lovasz-Bregman Divergence and connections to rank aggregation, clustering, and web ranking

no code implementations24 Aug 2013 Rishabh Iyer, Jeff Bilmes

We show how a number of recently used web ranking models are forms of Lovasz-Bregman rank aggregation and also observe that a natural form of Mallow's model using the LB divergence has been used as conditional ranking models for the 'Learning to Rank' problem.

Information Retrieval Learning-To-Rank

Fast Semidifferential-based Submodular Function Optimization

no code implementations5 Aug 2013 Rishabh Iyer, Stefanie Jegelka, Jeff Bilmes

We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials).

Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications

no code implementations3 Jul 2012 Rishabh Iyer, Jeff Bilmes

We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a difference between submodular functions.

feature selection

Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (2009)

no code implementations13 Jun 2012 Jeff Bilmes, Andrew Ng

This is the Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, which was held in Montreal, QC, Canada, June 18 - 21 2009.

Cannot find the paper you are looking for? You can Submit a new open access paper.