Search Results for author: Stefanie Jegelka

Found 111 papers, 43 papers with code

Complexity of Finding Stationary Points of Nonconvex Nonsmooth Functions

no code implementations ICML 2020 Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka, Suvrit Sra, Ali Jadbabaie

Therefore, we introduce the notion of (delta, epsilon)-stationarity, a generalization that allows for a point to be within distance delta of an epsilon-stationary point and reduces to epsilon-stationarity for smooth functions.

Understanding and Estimating the Adaptability of Domain-Invariant Representations

no code implementations ICML 2020 Ching-Yao Chuang, Antonio Torralba, Stefanie Jegelka

We also propose a method for estimating how well a model based on domain-invariant representations will perform on the target domain, without having seen any target labels.

Model Selection Unsupervised Domain Adaptation

The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges

no code implementations12 Jul 2024 Sitao Luan, Chenqing Hua, Qincheng Lu, Liheng Ma, Lirong Wu, Xinyu Wang, Minkai Xu, Xiao-Wen Chang, Doina Precup, Rex Ying, Stan Z. Li, Jian Tang, Guy Wolf, Stefanie Jegelka

In this survey, we provide a comprehensive review of the latest progress on heterophilic graph learning, including an extensive summary of benchmark datasets and evaluation of homophily metrics on synthetic graphs, meticulous classification of the most updated supervised and unsupervised learning methods, thorough digestion of the theoretical analysis on homophily/heterophily, and broad exploration of the heterophily-related applications.

Graph Learning Graph Representation Learning

A Universal Class of Sharpness-Aware Minimization Algorithms

1 code implementation6 Jun 2024 Behrooz Tahmasebi, Ashkan Soleymani, Dara Bahri, Stefanie Jegelka, Patrick Jaillet

Finally, as instances of our proposed general framework, we present \textit{Frob-SAM} and \textit{Det-SAM}, which are specifically designed to minimize the Frobenius norm and the determinant of the Hessian of the training loss, respectively.

The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof

no code implementations30 May 2024 Derek Lim, Moe Putterman, Robin Walters, Haggai Maron, Stefanie Jegelka

Many algorithms and observed phenomena in deep learning appear to be affected by parameter symmetries -- transformations of neural network parameters that do not change the underlying neural network function.

Linear Mode Connectivity

On the Role of Attention Masks and LayerNorm in Transformers

no code implementations29 May 2024 Xinyi Wu, Amir Ajorlou, Yifei Wang, Stefanie Jegelka, Ali Jadbabaie

In the case of self-attention with LayerNorm, we first show that for certain classes of value matrices, collapse to a rank one subspace still happens exponentially.

A Theoretical Understanding of Self-Correction through In-context Alignment

no code implementations28 May 2024 Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, Yisen Wang

Going beyond mimicking limited human experiences, recent studies show initial evidence that, like humans, large language models (LLMs) are capable of improving their abilities purely by self-correction, i. e., correcting previous responses through self-examination, in certain circumstances.

In-Context Learning

A Canonization Perspective on Invariant and Equivariant Learning

no code implementations28 May 2024 George Ma, Yifei Wang, Derek Lim, Stefanie Jegelka, Yisen Wang

In this work, we introduce a canonization perspective that provides an essential and complete view of the design of frames.

In-Context Symmetries: Self-Supervised Learning through Contextual World Models

no code implementations28 May 2024 Sharut Gupta, Chenyu Wang, Yifei Wang, Tommi Jaakkola, Stefanie Jegelka

In this work, drawing insights from world models, we propose to instead learn a general representation that can adapt to be invariant or equivariant to different transformations by paying attention to context -- a memory module that tracks task-specific states, actions, and future states.

Self-Supervised Learning

How to Craft Backdoors with Unlabeled Data Alone?

1 code implementation10 Apr 2024 Yifei Wang, Wenhan Ma, Stefanie Jegelka, Yisen Wang

Relying only on unlabeled data, Self-supervised learning (SSL) can learn rich features in an economical and scalable way.

Backdoor Attack Self-Supervised Learning

Future Directions in the Theory of Graph Machine Learning

no code implementations3 Feb 2024 Christopher Morris, Fabrizio Frasca, Nadav Dym, Haggai Maron, İsmail İlkan Ceylan, Ron Levie, Derek Lim, Michael Bronstein, Martin Grohe, Stefanie Jegelka

Machine learning on graphs, especially using graph neural networks (GNNs), has seen a surge in interest due to the wide availability of graph data across a broad spectrum of disciplines, from life to social and engineering sciences.


On the hardness of learning under symmetries

no code implementations3 Jan 2024 Bobak T. Kiani, Thien Le, Hannah Lawrence, Stefanie Jegelka, Melanie Weber

We study the problem of learning equivariant neural networks via gradient descent.

Inductive Bias

A Poincaré Inequality and Consistency Results for Signal Sampling on Large Graphs

no code implementations17 Nov 2023 Thien Le, Luana Ruiz, Stefanie Jegelka

We prove a Poincar\'e inequality for graphon signals and show that complements of node subsets satisfying this inequality are unique sampling sets for Paley-Wiener spaces of graphon signals.

Graph Sampling

Sample Complexity Bounds for Estimating Probability Divergences under Invariances

no code implementations6 Nov 2023 Behrooz Tahmasebi, Stefanie Jegelka

Our results indicate a two-fold gain: (1) reducing the sample complexity by a multiplicative factor corresponding to the group size (for finite groups) or the normalized volume of the quotient space (for groups of positive dimension); (2) improving the exponent in the convergence rate (for groups of positive dimension).

Density Estimation

On the Stability of Expressive Positional Encodings for Graphs

2 code implementations4 Oct 2023 Yinan Huang, William Lu, Joshua Robinson, Yu Yang, Muhan Zhang, Stefanie Jegelka, Pan Li

Despite many attempts to address non-uniqueness, most methods overlook stability, leading to poor generalization on unseen graph structures.

Molecular Property Prediction Out-of-Distribution Generalization +1

Are Graph Neural Networks Optimal Approximation Algorithms?

1 code implementation1 Oct 2023 Morris Yau, Eric Lu, Nikolaos Karalias, Jessica Xu, Stefanie Jegelka

In this work we design graph neural network architectures that capture optimal approximation algorithms for a large class of combinatorial optimization problems, using powerful algorithmic tools from semidefinite programming (SDP).

Combinatorial Optimization Graph Neural Network

Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning

1 code implementation24 Jun 2023 Sharut Gupta, Joshua Robinson, Derek Lim, Soledad Villar, Stefanie Jegelka

Specifically, in the contrastive learning setting, we introduce an equivariance objective and theoretically prove that its minima forces augmentations on input space to correspond to rotations on the spherical embedding space.

Contrastive Learning Self-Supervised Learning

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization

no code implementations22 Jun 2023 Khashayar Gatmiry, Zhiyuan Li, Ching-Yao Chuang, Sashank Reddi, Tengyu Ma, Stefanie Jegelka

Recent works on over-parameterized neural networks have shown that the stochasticity in optimizers has the implicit regularization effect of minimizing the sharpness of the loss function (in particular, the trace of its Hessian) over the family zero-loss solutions.

Inductive Bias

Tetris-inspired detector with neural network for radiation mapping

1 code implementation7 Feb 2023 Ryotaro Okabe, Shangjie Xue, Jiankai Yu, Tongtong Liu, Benoit Forget, Stefanie Jegelka, Gordon Kohse, Lin-wen Hu, Mingda Li

Here we present a computational framework using Tetris-inspired detector pixels and machine learning for radiation mapping.

Debiasing Vision-Language Models via Biased Prompts

1 code implementation31 Jan 2023 Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, Stefanie Jegelka

Machine learning models have been shown to inherit biases from their training datasets.

Optimal algorithms for group distributionally robust optimization and beyond

no code implementations28 Dec 2022 Tasuku Soma, Khashayar Gatmiry, Stefanie Jegelka

Distributionally robust optimization (DRO) can improve the robustness and fairness of learning methods.


InfoOT: Information Maximizing Optimal Transport

1 code implementation6 Oct 2022 Ching-Yao Chuang, Stefanie Jegelka, David Alvarez-Melis

Optimal transport aligns samples across distributions by minimizing the transportation cost between them, e. g., the geometric distances.

Domain Adaptation Retrieval

Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networks

1 code implementation4 Oct 2022 Ching-Yao Chuang, Stefanie Jegelka

Understanding generalization and robustness of machine learning models fundamentally relies on assuming an appropriate metric on the data space.

Graph Classification

On the generalization of learning algorithms that do not converge

no code implementations16 Aug 2022 Nisha Chandramoorthy, Andreas Loukas, Khashayar Gatmiry, Stefanie Jegelka

To reduce this discrepancy between theory and practice, this paper focuses on the generalization of neural networks whose training dynamics do not necessarily converge to fixed points.

Learning Theory

Theory of Graph Neural Networks: Representation and Learning

no code implementations16 Apr 2022 Stefanie Jegelka

Graph Neural Networks (GNNs), neural network architectures targeted to learning representations of graphs, have become a popular learning model for prediction tasks on nodes, graphs and configurations of points, with wide success in practice.

Sign and Basis Invariant Networks for Spectral Graph Representation Learning

2 code implementations25 Feb 2022 Derek Lim, Joshua Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, Stefanie Jegelka

We introduce SignNet and BasisNet -- new neural architectures that are invariant to two key symmetries displayed by eigenvectors: (i) sign flips, since if $v$ is an eigenvector then so is $-v$; and (ii) more general basis symmetries, which occur in higher dimensional eigenspaces with infinitely many choices of basis eigenvectors.

Graph Regression Graph Representation Learning

Training invariances and the low-rank phenomenon: beyond linear networks

no code implementations ICLR 2022 Thien Le, Stefanie Jegelka

The implicit bias induced by the training of neural networks has become a topic of rigorous study.

Robust Contrastive Learning against Noisy Views

1 code implementation CVPR 2022 Ching-Yao Chuang, R Devon Hjelm, Xin Wang, Vibhav Vineet, Neel Joshi, Antonio Torralba, Stefanie Jegelka, Yale Song

Contrastive learning relies on an assumption that positive pairs contain related views, e. g., patches of an image or co-occurring multimodal signals of a video, that share certain underlying information about an instance.

Binary Classification Contrastive Learning

Neural Extensions: Training Neural Networks with Set Functions

no code implementations29 Sep 2021 Nikolaos Karalias, Joshua David Robinson, Andreas Loukas, Stefanie Jegelka

Our framework includes well-known extensions such as the Lovasz extension of submodular set functions and facilitates the design of novel continuous extensions based on problem-specific considerations, including constraints.

Combinatorial Optimization Image Classification

Optimization and Adaptive Generalization of Three layer Neural Networks

no code implementations ICLR 2022 Khashayar Gatmiry, Stefanie Jegelka, Jonathan Kelner

While there has been substantial recent work studying generalization of neural networks, the ability of deep nets in automating the process of feature extraction still evades a thorough mathematical understanding.

Generalization Bounds

On the Effect of Input Perturbations for Graph Neural Networks

no code implementations29 Sep 2021 Behrooz Tahmasebi, Stefanie Jegelka

Our theoretical results imply constraints on the model for exploiting random node IDs, and, conversely, insights into the tolerance of a given model class for retaining discrimination with perturbations of node attributes.

Graph Neural Network

Scaling up Continuous-Time Markov Chains Helps Resolve Underspecification

no code implementations NeurIPS 2021 Alkis Gotovos, Rebekka Burkholz, John Quackenbush, Stefanie Jegelka

Modeling the time evolution of discrete sets of items (e. g., genetic mutations) is a fundamental problem in many biomedical applications.

Can contrastive learning avoid shortcut solutions?

1 code implementation NeurIPS 2021 Joshua Robinson, Li Sun, Ke Yu, Kayhan Batmanghelich, Stefanie Jegelka, Suvrit Sra

However, we observe that the contrastive loss does not always sufficiently guide which features are extracted, a behavior that can negatively impact the performance on downstream tasks via "shortcuts", i. e., by inadvertently suppressing important predictive features.

Contrastive Learning

What training reveals about neural network complexity

1 code implementation NeurIPS 2021 Andreas Loukas, Marinos Poiitis, Stefanie Jegelka

This work explores the Benevolent Training Hypothesis (BTH) which argues that the complexity of the function a deep neural network (NN) is learning can be deduced by its training dynamics.

Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth

no code implementations10 May 2021 Keyulu Xu, Mozhi Zhang, Stefanie Jegelka, Kenji Kawaguchi

Our results show that the training of GNNs is implicitly accelerated by skip connections, more depth, and/or a good label distribution.

Recursive Neighborhood Pooling for Graph Representation Learning

no code implementations1 Jan 2021 Behrooz Tahmasebi, Stefanie Jegelka

While Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power.

Graph Representation Learning

Counting Substructures with Higher-Order Graph Neural Networks: Possibility and Impossibility Results

no code implementations6 Dec 2020 Behrooz Tahmasebi, Derek Lim, Stefanie Jegelka

While message passing Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power.

Information Obfuscation of Graph Neural Networks

1 code implementation28 Sep 2020 Peiyuan Liao, Han Zhao, Keyulu Xu, Tommi Jaakkola, Geoffrey Gordon, Stefanie Jegelka, Ruslan Salakhutdinov

While the advent of Graph Neural Networks (GNNs) has greatly improved node and graph representation learning in many applications, the neighborhood aggregation scheme exposes additional vulnerabilities to adversaries seeking to extract node-level information about sensitive attributes.

Adversarial Defense Graph Representation Learning +2

How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks

3 code implementations ICLR 2021 Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S. Du, Ken-ichi Kawarabayashi, Stefanie Jegelka

Second, in connection to analyzing the successes and limitations of GNNs, these results suggest a hypothesis for which we provide theoretical and empirical evidence: the success of GNNs in extrapolating algorithmic tasks to new data (e. g., larger graphs or edge weights) relies on encoding task-specific non-linearities in the architecture or features.

Estimating Generalization under Distribution Shifts via Domain-Invariant Representations

1 code implementation6 Jul 2020 Ching-Yao Chuang, Antonio Torralba, Stefanie Jegelka

When machine learning models are deployed on a test distribution different from the training distribution, they can perform poorly, but overestimate their performance.

Domain Adaptation Model Selection

Debiased Contrastive Learning

1 code implementation NeurIPS 2020 Ching-Yao Chuang, Joshua Robinson, Lin Yen-Chen, Antonio Torralba, Stefanie Jegelka

A prominent technique for self-supervised representation learning has been to contrast semantically similar and dissimilar pairs of samples.

Contrastive Learning Generalization Bounds +2

IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method

no code implementations NeurIPS 2020 Yossi Arjevani, Joan Bruna, Bugra Can, Mert Gürbüzbalaban, Stefanie Jegelka, Hongzhou Lin

We introduce a framework for designing primal methods under the decentralized optimization setting where local functions are smooth and strongly convex.

Distributionally Robust Bayesian Optimization

no code implementations20 Feb 2020 Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, Andreas Krause

Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate.

Bayesian Optimization

Strength from Weakness: Fast Learning Using Weak Supervision

no code implementations ICML 2020 Joshua Robinson, Stefanie Jegelka, Suvrit Sra

Our theoretical results are reflected empirically across a range of tasks and illustrate how weak labels speed up learning on the strong task.

Weakly-supervised Learning

Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions

no code implementations10 Feb 2020 Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka, Ali Jadbabaie, Suvrit Sra

In particular, we study the class of Hadamard semi-differentiable functions, perhaps the largest class of nonsmooth functions for which the chain rule of calculus holds.

On the Complexity of Minimizing Convex Finite Sums Without Using the Indices of the Individual Functions

no code implementations9 Feb 2020 Yossi Arjevani, Amit Daniely, Stefanie Jegelka, Hongzhou Lin

Recent advances in randomized incremental methods for minimizing $L$-smooth $\mu$-strongly convex finite sums have culminated in tight complexity of $\tilde{O}((n+\sqrt{n L/\mu})\log(1/\epsilon))$ and $O(n+\sqrt{nL/\epsilon})$, where $\mu>0$ and $\mu=0$, respectively, and $n$ denotes the number of individual functions.

Adaptive Sampling for Stochastic Risk-Averse Learning

1 code implementation NeurIPS 2020 Sebastian Curi, Kfir. Y. Levy, Stefanie Jegelka, Andreas Krause

In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples.

Point Processes

The Role of Embedding Complexity in Domain-invariant Representations

1 code implementation13 Oct 2019 Ching-Yao Chuang, Antonio Torralba, Stefanie Jegelka

In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain.

Unsupervised Domain Adaptation

Perceptual Regularization: Visualizing and Learning Generalizable Representations

no code implementations25 Sep 2019 Hongzhou Lin, Joshua Robinson, Stefanie Jegelka

We propose a technique termed perceptual regularization that enables both visualization of the latent representation and control over the generality of the learned representation.

Are Girls Neko or Sh\=ojo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization

no code implementations ACL 2019 Mozhi Zhang, Keyulu Xu, Ken-ichi Kawarabayashi, Stefanie Jegelka, Jordan Boyd-Graber

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.

Cross-Lingual Word Embeddings Translation +2

Flexible Modeling of Diversity with Strongly Log-Concave Distributions

1 code implementation NeurIPS 2019 Joshua Robinson, Suvrit Sra, Stefanie Jegelka

We propose SLC as the right extension of SR that enables easier, more intuitive control over diversity, illustrating this via examples of practical importance.


Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization

1 code implementation4 Jun 2019 Mozhi Zhang, Keyulu Xu, Ken-ichi Kawarabayashi, Stefanie Jegelka, Jordan Boyd-Graber

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.

Cross-Lingual Word Embeddings Translation +2

Optimal approximation for unconstrained non-submodular minimization

1 code implementation ICML 2020 Marwa El Halabi, Stefanie Jegelka

Submodular function minimization is well studied, and existing algorithms solve it exactly or up to arbitrary accuracy.

Bayesian Optimization Sparse Learning

Distributionally Robust Optimization and Generalization in Kernel Methods

1 code implementation NeurIPS 2019 Matthew Staib, Stefanie Jegelka

We show that MMD DRO is roughly equivalent to regularization by the Hilbert norm and, as a byproduct, reveal deep connections to classic results in statistical learning.

Learning Generative Models across Incomparable Spaces

no code implementations14 May 2019 Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety.

Relational Reasoning

Provable Variational Inference for Constrained Log-Submodular Models

no code implementations NeurIPS 2018 Josip Djolonga, Stefanie Jegelka, Andreas Krause

Submodular maximization problems appear in several areas of machine learning and data science, as many useful modelling concepts such as diversity and coverage satisfy this natural diminishing returns property.

Variational Inference

Exponentiated Strongly Rayleigh Distributions

no code implementations NeurIPS 2018 Zelda E. Mariet, Suvrit Sra, Stefanie Jegelka

Strongly Rayleigh (SR) measures are discrete probability distributions over the subsets of a ground set.

Point Processes

Adversarially Robust Optimization with Gaussian Processes

no code implementations NeurIPS 2018 Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, Volkan Cevher

In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.

Gaussian Processes

How Powerful are Graph Neural Networks?

17 code implementations ICLR 2019 Keyulu Xu, Weihua Hu, Jure Leskovec, Stefanie Jegelka

Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures.

General Classification Graph Classification +3

Discrete Sampling using Semigradient-based Product Mixtures

no code implementations4 Jul 2018 Alkis Gotovos, Hamed Hassani, Andreas Krause, Stefanie Jegelka

We consider the problem of inference in discrete probabilistic models, that is, distributions over subsets of a finite ground set.

Point Processes

ResNet with one-neuron hidden layers is a Universal Approximator

1 code implementation NeurIPS 2018 Hongzhou Lin, Stefanie Jegelka

We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation functions can uniformly approximate any Lebesgue integrable function in $d$ dimensions, i. e. $\ell_1(\mathbb{R}^d)$.

Towards Optimal Transport with Global Invariances

no code implementations25 Jun 2018 David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola

Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images.

Translation Word Embeddings +1

Representation Learning on Graphs with Jumping Knowledge Networks

4 code implementations ICML 2018 Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, Stefanie Jegelka

Furthermore, combining the JK framework with models like Graph Convolutional Networks, GraphSAGE and Graph Attention Networks consistently improves those models' performance.

Graph Attention Node Classification +2

Robust GANs against Dishonest Adversaries

no code implementations27 Feb 2018 Zhi Xu, Chengtao Li, Stefanie Jegelka

We explore a notion of robustness for generative adversarial models that is pertinent to their internal interactive structure, and show that, perhaps surprisingly, the GAN in its original form is not robust.

Distributionally Robust Submodular Maximization

no code implementations14 Feb 2018 Matthew Staib, Bryan Wilder, Stefanie Jegelka

We also show compelling empirical evidence that DRO improves generalization to the unknown stochastic submodular function.

Structured Optimal Transport

no code implementations17 Dec 2017 David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka

Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation, sentence similarities to deep learning.

BIG-bench Machine Learning Domain Adaptation +1

Graph-Sparse Logistic Regression

1 code implementation15 Dec 2017 Alexander LeNail, Ludwig Schmidt, Johnathan Li, Tobias Ehrenberger, Karen Sachs, Stefanie Jegelka, Ernest Fraenkel

We introduce Graph-Sparse Logistic Regression, a new algorithm for classification for the case in which the support should be sparse but connected on a graph.

General Classification regression

Distributional Adversarial Networks

1 code implementation ICLR 2018 Chengtao Li, David Alvarez-Melis, Keyulu Xu, Stefanie Jegelka, Suvrit Sra

We propose a framework for adversarial training that relies on a sample rather than a single sample point as the fundamental unit of discrimination.

Domain Adaptation

Streaming Non-monotone Submodular Maximization: Personalized Video Summarization on the Fly

1 code implementation12 Jun 2017 Baharan Mirzasoleiman, Stefanie Jegelka, Andreas Krause

The need for real time analysis of rapidly producing data streams (e. g., video and image streams) motivated the design of streaming algorithms that can efficiently extract and summarize useful information from massive data "on the fly".

Data Structures and Algorithms Information Retrieval

Batched Large-scale Bayesian Optimization in High-dimensional Spaces

2 code implementations5 Jun 2017 Zi Wang, Clement Gehring, Pushmeet Kohli, Stefanie Jegelka

Bayesian optimization (BO) has become an effective approach for black-box function optimization problems when function evaluations are expensive and the optimum can be achieved within a relatively small number of queries.

Bayesian Optimization Diversity +1

Parallel Streaming Wasserstein Barycenters

1 code implementation NeurIPS 2017 Matthew Staib, Sebastian Claici, Justin Solomon, Stefanie Jegelka

Our method is even robust to nonstationary input distributions and produces a barycenter estimate that tracks the input measures over time.

Bayesian Inference

Polynomial Time Algorithms for Dual Volume Sampling

no code implementations NeurIPS 2017 Chengtao Li, Stefanie Jegelka, Suvrit Sra

We study dual volume sampling, a method for selecting k columns from an n x m short and wide matrix (n <= k <= m) such that the probability of selection is proportional to the volume spanned by the rows of the induced submatrix.

Experimental Design

Max-value Entropy Search for Efficient Bayesian Optimization

4 code implementations ICML 2017 Zi Wang, Stefanie Jegelka

We propose a new criterion, Max-value Entropy Search (MES), that instead uses the information about the maximum function value.

Robust Budget Allocation via Continuous Submodular Functions

no code implementations ICML 2017 Matthew Staib, Stefanie Jegelka

The optimal allocation of resources for maximizing influence, spread of information or coverage, has gained attention in the past years, in particular in machine learning and data mining.

Deep Metric Learning via Facility Location

1 code implementation CVPR 2017 Hyun Oh Song, Stefanie Jegelka, Vivek Rathod, Kevin Murphy

Learning the representation and the similarity metric in an end-to-end fashion with deep networks have demonstrated outstanding results for clustering and retrieval.

Clustering Metric Learning +2

Cooperative Graphical Models

no code implementations NeurIPS 2016 Josip Djolonga, Stefanie Jegelka, Sebastian Tschiatschek, Andreas Krause

We study a rich family of distributions that capture variable interactions significantly more expressive than those representable with low-treewidth or pairwise graphical models, or log-supermodular models.

Variational Inference

Fast Mixing Markov Chains for Strongly Rayleigh Measures, DPPs, and Constrained Sampling

no code implementations NeurIPS 2016 Chengtao Li, Stefanie Jegelka, Suvrit Sra

We consider the task of rapidly sampling from such constrained measures, and develop fast Markov chain samplers for them.

Point Processes

Focused Model-Learning and Planning for Non-Gaussian Continuous State-Action Systems

no code implementations26 Jul 2016 Zi Wang, Stefanie Jegelka, Leslie Pack Kaelbling, Tomás Lozano-Pérez

We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models.

Fast DPP Sampling for Nyström with Application to Kernel Methods

no code implementations19 Mar 2016 Chengtao Li, Stefanie Jegelka, Suvrit Sra

Its theoretical guarantees and empirical performance rely critically on the quality of the landmarks selected.

Point Processes regression

Gauss quadrature for matrix inverse forms with applications

no code implementations7 Dec 2015 Chengtao Li, Suvrit Sra, Stefanie Jegelka

We present a framework for accelerating a spectrum of machine learning algorithms that require computation of bilinear inverse forms $u^\top A^{-1}u$, where $A$ is a positive definite matrix and $u$ a given vector.

BIG-bench Machine Learning Point Processes

Auxiliary Image Regularization for Deep CNNs with Noisy Labels

no code implementations22 Nov 2015 Samaneh Azadi, Jiashi Feng, Stefanie Jegelka, Trevor Darrell

Precisely-labeled data sets with sufficient amount of samples are very important for training deep convolutional neural networks (CNNs).

Image Classification

Deep Metric Learning via Lifted Structured Feature Embedding

3 code implementations CVPR 2016 Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese

Additionally, we collected Online Products dataset: 120k images of 23k classes of online products for metric learning.

Metric Learning Structured Prediction

Optimization as Estimation with Gaussian Processes in Bandit Settings

1 code implementation21 Oct 2015 Zi Wang, Bolei Zhou, Stefanie Jegelka

Recently, there has been rising interest in Bayesian optimization -- the optimization of an unknown function with assumptions usually expressed by a Gaussian Process (GP) prior.

Bayesian Optimization Gaussian Processes

Efficient Sampling for k-Determinantal Point Processes

no code implementations4 Sep 2015 Chengtao Li, Stefanie Jegelka, Suvrit Sra

Our method takes advantage of the diversity property of subsets sampled from a DPP, and proceeds in two stages: first it constructs coresets for the ground set of items; thereafter, it efficiently samples subsets based on the constructed coresets.

Diversity Point Processes

Convex Optimization for Parallel Energy Minimization

no code implementations5 Mar 2015 K. S. Sesh Kumar, Alvaro Barbero, Stefanie Jegelka, Suvrit Sra, Francis Bach

By exploiting results from convex and submodular theory, we reformulate the quadratic energy minimization problem as a total variation denoising problem, which, when viewed geometrically, enables the use of projection and reflection based convex methods.


Inferring and Learning from Neuronal Correspondences

no code implementations23 Jan 2015 Ashish Kapoor, E. Paxon Frady, Stefanie Jegelka, William B. Kristan, Eric Horvitz

We introduce and study methods for inferring and learning from correspondences among neurons.

Decision Making

Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets

no code implementations NeurIPS 2014 Adarsh Prasad, Stefanie Jegelka, Dhruv Batra

To cope with the high level of ambiguity faced in domains such as Computer Vision or Natural Language processing, robust prediction methods often search for a diverse set of high-quality candidate solutions or proposals.

Diversity Sentence +1

Weakly-supervised Discovery of Visual Pattern Configurations

no code implementations NeurIPS 2014 Hyun Oh Song, Yong Jae Lee, Stefanie Jegelka, Trevor Darrell

The increasing prominence of weakly labeled data nurtures a growing demand for object detection methods that can cope with minimal supervision.

Object object-detection +1

On learning to localize objects with minimal supervision

no code implementations5 Mar 2014 Hyun Oh Song, Ross Girshick, Stefanie Jegelka, Julien Mairal, Zaid Harchaoui, Trevor Darrell

Learning to localize objects with minimal supervision is an important problem in computer vision, since large fully annotated datasets are extremely costly to obtain.

Weakly Supervised Object Detection

Graph Cuts with Interacting Edge Costs - Examples, Approximations, and Algorithms

no code implementations2 Feb 2014 Stefanie Jegelka, Jeff Bilmes

We study an extension of the classical graph cut problem, wherein we replace the modular (sum of edge weights) cost function by a submodular set function defined over graph edges.

BIG-bench Machine Learning

Reflection methods for user-friendly submodular optimization

no code implementations NeurIPS 2013 Stefanie Jegelka, Francis Bach, Suvrit Sra

A key component of our method is a formulation of the discrete submodular minimization problem as a continuous best approximation problem that is solved through a sequence of reflections, and its solution can be easily thresholded to obtain an optimal discrete solution.

Image Segmentation Semantic Segmentation

Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions

no code implementations NeurIPS 2013 Rishabh Iyer, Stefanie Jegelka, Jeff Bilmes

We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization).

Fast Semidifferential-based Submodular Function Optimization

no code implementations5 Aug 2013 Rishabh Iyer, Stefanie Jegelka, Jeff Bilmes

We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials).

Optimistic Concurrency Control for Distributed Unsupervised Learning

no code implementations NeurIPS 2013 Xinghao Pan, Joseph E. Gonzalez, Stefanie Jegelka, Tamara Broderick, Michael. I. Jordan

Research on distributed machine learning algorithms has focused primarily on one of two extremes - algorithms that obey strict concurrency constraints or algorithms that obey few or no such constraints.

BIG-bench Machine Learning Clustering

A Principled Deep Random Field Model for Image Segmentation

no code implementations CVPR 2013 Pushmeet Kohli, Anton Osokin, Stefanie Jegelka

We discuss a model for image segmentation that is able to overcome the short-boundary bias observed in standard pairwise random field based approaches.

Image Segmentation Segmentation +1

On fast approximate submodular minimization

no code implementations NeurIPS 2011 Stefanie Jegelka, Hui Lin, Jeff A. Bilmes

We are motivated by an application to extract a representative subset of machine learning training data and by the poor empirical performance we observe of the popular minimum norm algorithm.

BIG-bench Machine Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.