Search Results for author:
Found 94 papers, 36 papers with code
Understanding and Estimating the Adaptability of Domain-Invariant Representations
We also propose a method for estimating how well a model based on domain-invariant representations will perform on the target domain, without having seen any target labels.
Complexity of Finding Stationary Points of Nonconvex Nonsmooth Functions
Therefore, we introduce the notion of (delta, epsilon)-stationarity, a generalization that allows for a point to be within distance delta of an epsilon-stationary point and reduces to epsilon-stationarity for smooth functions.
The Exact Sample Complexity Gain from Invariances for Kernel Regression on Manifolds
For groups of positive dimension, the gain is observed by a reduction in the manifold's dimension, in addition to a factor proportional to the volume of the quotient space.
Tetris-inspired detector with neural network for radiation mapping
Here we present a computational framework using Tetris-inspired detector pixels and machine learning for radiation mapping.
Debiasing Vision-Language Models via Biased Prompts
Machine learning models have been shown to inherit biases from their training datasets, which can be particularly problematic for vision-language foundation models trained on uncurated datasets scraped from the internet.
Efficiently predicting high resolution mass spectra with graph neural networks
Identifying a small molecule from its mass spectrum is the primary open problem in computational metabolomics.
Optimal algorithms for group distributionally robust optimization and beyond
Distributionally robust optimization (DRO) can improve the robustness and fairness of learning methods.
InfoOT: Information Maximizing Optimal Transport
Optimal transport aligns samples across distributions by minimizing the transportation cost between them, e. g., the geometric distances.
Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networks
Understanding generalization and robustness of machine learning models fundamentally relies on assuming an appropriate metric on the data space.
On the generalization of learning algorithms that do not converge
To reduce this discrepancy between theory and practice, this paper focuses on the generalization of neural networks whose training dynamics do not necessarily converge to fixed points.
Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions
Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects.
Theory of Graph Neural Networks: Representation and Learning
Graph Neural Networks (GNNs), neural network architectures targeted to learning representations of graphs, have become a popular learning model for prediction tasks on nodes, graphs and configurations of points, with wide success in practice.
Sign and Basis Invariant Networks for Spectral Graph Representation Learning
We introduce SignNet and BasisNet -- new neural architectures that are invariant to two key symmetries displayed by eigenvectors: (i) sign flips, since if $v$ is an eigenvector then so is $-v$; and (ii) more general basis symmetries, which occur in higher dimensional eigenspaces with infinitely many choices of basis eigenvectors.
Ranked #4 on
Graph Regression
on ZINC-500k
Training invariances and the low-rank phenomenon: beyond linear networks
The implicit bias induced by the training of neural networks has become a topic of rigorous study.
Robust Contrastive Learning against Noisy Views
Contrastive learning relies on an assumption that positive pairs contain related views, e. g., patches of an image or co-occurring multimodal signals of a video, that share certain underlying information about an instance.
Optimization and Adaptive Generalization of Three layer Neural Networks
While there has been substantial recent work studying generalization of neural networks, the ability of deep nets in automating the process of feature extraction still evades a thorough mathematical understanding.
Neural Extensions: Training Neural Networks with Set Functions
Our framework includes well-known extensions such as the Lovasz extension of submodular set functions and facilitates the design of novel continuous extensions based on problem-specific considerations, including constraints.
On the Effect of Input Perturbations for Graph Neural Networks
Our theoretical results imply constraints on the model for exploiting random node IDs, and, conversely, insights into the tolerance of a given model class for retaining discrimination with perturbations of node attributes.
Scaling up Continuous-Time Markov Chains Helps Resolve Underspecification
Modeling the time evolution of discrete sets of items (e. g., genetic mutations) is a fundamental problem in many biomedical applications.
Can contrastive learning avoid shortcut solutions?
However, we observe that the contrastive loss does not always sufficiently guide which features are extracted, a behavior that can negatively impact the performance on downstream tasks via "shortcuts", i. e., by inadvertently suppressing important predictive features.
What training reveals about neural network complexity
This work explores the Benevolent Training Hypothesis (BTH) which argues that the complexity of the function a deep neural network (NN) is learning can be deduced by its training dynamics.
Measuring Generalization with Optimal Transport
Understanding the generalization of deep neural networks is one of the most important tasks in deep learning.
Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth
Our results show that the training of GNNs is implicitly accelerated by skip connections, more depth, and/or a good label distribution.
Recursive Neighborhood Pooling for Graph Representation Learning
While Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power.
Counting Substructures with Higher-Order Graph Neural Networks: Possibility and Impossibility Results
While message passing Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power.
Contrastive Learning with Hard Negative Samples
How can you sample good negative examples for contrastive learning?
Information Obfuscation of Graph Neural Networks
While the advent of Graph Neural Networks (GNNs) has greatly improved node and graph representation learning in many applications, the neighborhood aggregation scheme exposes additional vulnerabilities to adversaries seeking to extract node-level information about sensitive attributes.
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks
Second, in connection to analyzing the successes and limitations of GNNs, these results suggest a hypothesis for which we provide theoretical and empirical evidence: the success of GNNs in extrapolating algorithmic tasks to new data (e. g., larger graphs or edge weights) relies on encoding task-specific non-linearities in the architecture or features.
Testing Determinantal Point Processes
Determinantal point processes (DPPs) are popular probabilistic models of diversity.
Estimating Generalization under Distribution Shifts via Domain-Invariant Representations
When machine learning models are deployed on a test distribution different from the training distribution, they can perform poorly, but overestimate their performance.
Debiased Contrastive Learning
A prominent technique for self-supervised representation learning has been to contrast semantically similar and dissimilar pairs of samples.
IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method
We introduce a framework for designing primal methods under the decentralized optimization setting where local functions are smooth and strongly convex.
Distributionally Robust Bayesian Optimization
Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate.
Strength from Weakness: Fast Learning Using Weak Supervision
Our theoretical results are reflected empirically across a range of tasks and illustrate how weak labels speed up learning on the strong task.
Generalization and Representational Limits of Graph Neural Networks
We address two fundamental questions about graph neural networks (GNNs).
Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions
In particular, we study the class of Hadamard semi-differentiable functions, perhaps the largest class of nonsmooth functions for which the chain rule of calculus holds.
On the Complexity of Minimizing Convex Finite Sums Without Using the Indices of the Individual Functions
Recent advances in randomized incremental methods for minimizing $L$-smooth $\mu$-strongly convex finite sums have culminated in tight complexity of $\tilde{O}((n+\sqrt{n L/\mu})\log(1/\epsilon))$ and $O(n+\sqrt{nL/\epsilon})$, where $\mu>0$ and $\mu=0$, respectively, and $n$ denotes the number of individual functions.
Adaptive Sampling for Stochastic Risk-Averse Learning
In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples.
The Role of Embedding Complexity in Domain-invariant Representations
In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain.
Perceptual Regularization: Visualizing and Learning Generalizable Representations
We propose a technique termed perceptual regularization that enables both visualization of the latent representation and control over the generality of the learned representation.
Are Girls Neko or Sh\=ojo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization
Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.
Flexible Modeling of Diversity with Strongly Log-Concave Distributions
We propose SLC as the right extension of SR that enables easier, more intuitive control over diversity, illustrating this via examples of practical importance.
Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization
Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.
Optimal approximation for unconstrained non-submodular minimization
Submodular function minimization is well studied, and existing algorithms solve it exactly or up to arbitrary accuracy.
Distributionally Robust Optimization and Generalization in Kernel Methods
We show that MMD DRO is roughly equivalent to regularization by the Hilbert norm and, as a byproduct, reveal deep connections to classic results in statistical learning.
Learning Generative Models across Incomparable Spaces
Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety.
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery.
Exponentiated Strongly Rayleigh Distributions
Strongly Rayleigh (SR) measures are discrete probability distributions over the subsets of a ground set.
Provable Variational Inference for Constrained Log-Submodular Models
Submodular maximization problems appear in several areas of machine learning and data science, as many useful modelling concepts such as diversity and coverage satisfy this natural diminishing returns property.
Adversarially Robust Optimization with Gaussian Processes
In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.
How Powerful are Graph Neural Networks?
Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures.
Ranked #1 on
Graph Classification
on RE-M5K
Discrete Sampling using Semigradient-based Product Mixtures
We consider the problem of inference in discrete probabilistic models, that is, distributions over subsets of a finite ground set.
ResNet with one-neuron hidden layers is a Universal Approximator
We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation functions can uniformly approximate any Lebesgue integrable function in $d$ dimensions, i. e. $\ell_1(\mathbb{R}^d)$.
Towards Optimal Transport with Global Invariances
Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images.
Representation Learning on Graphs with Jumping Knowledge Networks
Furthermore, combining the JK framework with models like Graph Convolutional Networks, GraphSAGE and Graph Attention Networks consistently improves those models' performance.
Ranked #12 on
Node Classification
on PPI
Robust GANs against Dishonest Adversaries
We explore a notion of robustness for generative adversarial models that is pertinent to their internal interactive structure, and show that, perhaps surprisingly, the GAN in its original form is not robust.
Distributionally Robust Submodular Maximization
We also show compelling empirical evidence that DRO improves generalization to the unknown stochastic submodular function.
Structured Optimal Transport
Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation, sentence similarities to deep learning.
Graph-Sparse Logistic Regression
We introduce Graph-Sparse Logistic Regression, a new algorithm for classification for the case in which the support should be sparse but connected on a graph.
Distributional Adversarial Networks
We propose a framework for adversarial training that relies on a sample rather than a single sample point as the fundamental unit of discrimination.
Streaming Non-monotone Submodular Maximization: Personalized Video Summarization on the Fly
The need for real time analysis of rapidly producing data streams (e. g., video and image streams) motivated the design of streaming algorithms that can efficiently extract and summarize useful information from massive data "on the fly".
Data Structures and Algorithms Information Retrieval
Batched Large-scale Bayesian Optimization in High-dimensional Spaces
Bayesian optimization (BO) has become an effective approach for black-box function optimization problems when function evaluations are expensive and the optimum can be achieved within a relatively small number of queries.
Parallel Streaming Wasserstein Barycenters
Our method is even robust to nonstationary input distributions and produces a barycenter estimate that tracks the input measures over time.
Polynomial Time Algorithms for Dual Volume Sampling
We study dual volume sampling, a method for selecting k columns from an n x m short and wide matrix (n <= k <= m) such that the probability of selection is proportional to the volume spanned by the rows of the induced submatrix.
Batched High-dimensional Bayesian Optimization via Structural Kernel Learning
Optimization of high-dimensional black-box functions is an extremely challenging problem.
Max-value Entropy Search for Efficient Bayesian Optimization
We propose a new criterion, Max-value Entropy Search (MES), that instead uses the information about the maximum function value.
Robust Budget Allocation via Continuous Submodular Functions
The optimal allocation of resources for maximizing influence, spread of information or coverage, has gained attention in the past years, in particular in machine learning and data mining.
Deep Metric Learning via Facility Location
Learning the representation and the similarity metric in an end-to-end fashion with deep networks have demonstrated outstanding results for clustering and retrieval.
Cooperative Graphical Models
We study a rich family of distributions that capture variable interactions significantly more expressive than those representable with low-treewidth or pairwise graphical models, or log-supermodular models.
Fast Mixing Markov Chains for Strongly Rayleigh Measures, DPPs, and Constrained Sampling
We consider the task of rapidly sampling from such constrained measures, and develop fast Markov chain samplers for them.
Focused Model-Learning and Planning for Non-Gaussian Continuous State-Action Systems
We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models.
Fast Sampling for Strongly Rayleigh Measures with Application to Determinantal Point Processes
In this note we consider sampling from (non-homogeneous) strongly Rayleigh probability measures.
Fast DPP Sampling for Nyström with Application to Kernel Methods
Its theoretical guarantees and empirical performance rely critically on the quality of the landmarks selected.
Gauss quadrature for matrix inverse forms with applications
We present a framework for accelerating a spectrum of machine learning algorithms that require computation of bilinear inverse forms $u^\top A^{-1}u$, where $A$ is a positive definite matrix and $u$ a given vector.
Auxiliary Image Regularization for Deep CNNs with Noisy Labels
Precisely-labeled data sets with sufficient amount of samples are very important for training deep convolutional neural networks (CNNs).
Deep Metric Learning via Lifted Structured Feature Embedding
Additionally, we collected Online Products dataset: 120k images of 23k classes of online products for metric learning.
Optimization as Estimation with Gaussian Processes in Bandit Settings
Recently, there has been rising interest in Bayesian optimization -- the optimization of an unknown function with assumptions usually expressed by a Gaussian Process (GP) prior.
Efficient Sampling for k-Determinantal Point Processes
Our method takes advantage of the diversity property of subsets sampled from a DPP, and proceeds in two stages: first it constructs coresets for the ground set of items; thereafter, it efficiently samples subsets based on the constructed coresets.
Convex Optimization for Parallel Energy Minimization
By exploiting results from convex and submodular theory, we reformulate the quadratic energy minimization problem as a total variation denoising problem, which, when viewed geometrically, enables the use of projection and reflection based convex methods.
Inferring and Learning from Neuronal Correspondences
We introduce and study methods for inferring and learning from correspondences among neurons.
Parallel Double Greedy Submodular Maximization
Many machine learning problems can be reduced to the maximization of submodular functions.
Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
To cope with the high level of ambiguity faced in domains such as Computer Vision or Natural Language processing, robust prediction methods often search for a diverse set of high-quality candidate solutions or proposals.
Weakly-supervised Discovery of Visual Pattern Configurations
The increasing prominence of weakly labeled data nurtures a growing demand for object detection methods that can cope with minimal supervision.
On the Convergence Rate of Decomposable Submodular Function Minimization
Submodular functions describe a variety of discrete problems in machine learning, signal processing, and computer vision.
Learning Scalable Discriminative Dictionary with Sample Relatedness
We use sample relatedness information to improve the generalization of the learned dictionary.
On learning to localize objects with minimal supervision
Learning to localize objects with minimal supervision is an important problem in computer vision, since large fully annotated datasets are extremely costly to obtain.
Ranked #35 on
Weakly Supervised Object Detection
on PASCAL VOC 2007
Graph Cuts with Interacting Edge Costs - Examples, Approximations, and Algorithms
We study an extension of the classical graph cut problem, wherein we replace the modular (sum of edge weights) cost function by a submodular set function defined over graph edges.
Reflection methods for user-friendly submodular optimization
A key component of our method is a formulation of the discrete submodular minimization problem as a continuous best approximation problem that is solved through a sequence of reflections, and its solution can be easily thresholded to obtain an optimal discrete solution.
Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions
We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization).
Fast Semidifferential-based Submodular Function Optimization
We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials).
Optimistic Concurrency Control for Distributed Unsupervised Learning
Research on distributed machine learning algorithms has focused primarily on one of two extremes - algorithms that obey strict concurrency constraints or algorithms that obey few or no such constraints.
A Principled Deep Random Field Model for Image Segmentation
We discuss a model for image segmentation that is able to overcome the short-boundary bias observed in standard pairwise random field based approaches.
On fast approximate submodular minimization
We are motivated by an application to extract a representative subset of machine learning training data and by the poor empirical performance we observe of the popular minimum norm algorithm.
