Search Results for author: Greg Ver Steeg

Found 58 papers, 31 papers with code

All in the (Exponential) Family: Information Geometry and Thermodynamic Variational Inference

no code implementations ICML 2020 Rob Brekelmans, Vaden Masrani, Frank Wood, Greg Ver Steeg, Aram Galstyan

While the Evidence Lower Bound (ELBO) has become a ubiquitous objective for variational inference, the recently proposed Thermodynamic Variational Objective (TVO) leverages thermodynamic integration to provide a tighter and more general family of bounds.

Variational Inference

Implicit SVD for Graph Representation Learning

1 code implementation NeurIPS 2021 Sami Abu-El-Haija, Hesham Mostafa, Marcel Nassar, Valentino Crespi, Greg Ver Steeg, Aram Galstyan

Recent improvements in the performance of state-of-the-art (SOTA) methods for Graph Representational Learning (GRL) have come at the cost of significant computational resource requirements for training, e. g., for calculating gradients via backprop over many data epochs.

Graph Representation Learning

Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling

1 code implementation NeurIPS 2021 Greg Ver Steeg, Aram Galstyan

Auxiliary neural models can learn to speed up MCMC, but the overhead for training the extra model can be prohibitive.

Information-theoretic generalization bounds for black-box learning algorithms

1 code implementation NeurIPS 2021 Hrayr Harutyunyan, Maxim Raginsky, Greg Ver Steeg, Aram Galstyan

We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm.

Generalization Bounds

Attributing Fair Decisions with Attention Interventions

no code implementations8 Sep 2021 Ninareh Mehrabi, Umang Gupta, Fred Morstatter, Greg Ver Steeg, Aram Galstyan

The widespread use of Artificial Intelligence (AI) in consequential domains, such as healthcare and parole decision-making systems, has drawn intense scrutiny on the fairness of these methods.

Decision Making Fairness

Secure Neuroimaging Analysis using Federated Learning with Homomorphic Encryption

no code implementations7 Aug 2021 Dimitris Stripelis, Hamza Saleem, Tanmay Ghai, Nikhil Dhinagar, Umang Gupta, Chrysovalantis Anastasiou, Greg Ver Steeg, Srivatsan Ravi, Muhammad Naveed, Paul M. Thompson, Jose Luis Ambite

Federated learning (FL) enables distributed computation of machine learning models over various disparate, remote data sources, without requiring to transfer any individual data to a centralized location.

Federated Learning

q-Paths: Generalizing the Geometric Annealing Path using Power Means

1 code implementation1 Jul 2021 Vaden Masrani, Rob Brekelmans, Thang Bui, Frank Nielsen, Aram Galstyan, Greg Ver Steeg, Frank Wood

Many common machine learning methods involve the geometric annealing path, a sequence of intermediate densities between two distributions of interest constructed using the geometric average.

Bayesian Inference

Membership Inference Attacks on Deep Regression Models for Neuroimaging

no code implementations6 May 2021 Umang Gupta, Dimitris Stripelis, Pradeep K. Lam, Paul M. Thompson, José Luis Ambite, Greg Ver Steeg

In particular, we show that it is possible to infer if a sample was used to train the model given only access to the model prediction (black-box) or access to the model itself (white-box) and some leaked samples from the training data distribution.

Federated Learning

Fast Graph Learning with Unique Optimal Solutions

1 code implementation ICLR Workshop GTRL 2021 Sami Abu-El-Haija, Valentino Crespi, Greg Ver Steeg, Aram Galstyan

We consider two popular Graph Representation Learning (GRL) methods: message passing for node classification and network embedding for link prediction.

Graph Learning Graph Representation Learning +3

Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning

1 code implementation ICLR 2021 Elan Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi, Greg Ver Steeg, Aram Galstyan

We propose Graph Traversal via Tensor Functionals(GTTF), a unifying meta-algorithm framework for easing the implementation of diverse graph algorithms and enabling transparent and efficient scaling to large graphs.

Graph Representation Learning

Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation

1 code implementation11 Jan 2021 Umang Gupta, Aaron M Ferber, Bistra Dilkina, Greg Ver Steeg

Controlling bias in training datasets is vital for ensuring equal treatment, or parity, between different groups in downstream applications.


Likelihood Ratio Exponential Families

no code implementations NeurIPS Workshop DL-IG 2020 Rob Brekelmans, Frank Nielsen, Alireza Makhzani, Aram Galstyan, Greg Ver Steeg

The exponential family is well known in machine learning and statistical physics as the maximum entropy distribution subject to a set of observed constraints, while the geometric mixture path is common in MCMC methods such as annealed importance sampling.

Annealed Importance Sampling with q-Paths

1 code implementation NeurIPS Workshop DL-IG 2020 Rob Brekelmans, Vaden Masrani, Thang Bui, Frank Wood, Aram Galstyan, Greg Ver Steeg, Frank Nielsen

Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods, corresponding to importance sampling over a path of distributions between a tractable base and an unnormalized target.

Compressing Deep Neural Networks via Layer Fusion

no code implementations29 Jul 2020 James O' Neill, Greg Ver Steeg, Aram Galstyan

This paper proposes \textit{layer fusion} - a model compression technique that discovers which weights to combine and then fuses weights of similar fully-connected, convolutional and attention layers.

Exponential degradation Language Modelling +1

Robust Classification under Class-Dependent Domain Shift

no code implementations10 Jul 2020 Tigran Galstyan, Hrant Khachatrian, Greg Ver Steeg, Aram Galstyan

Investigation of machine learning algorithms robust to changes between the training and test distributions is an active area of research.

General Classification Robust classification

All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference

1 code implementation1 Jul 2020 Rob Brekelmans, Vaden Masrani, Frank Wood, Greg Ver Steeg, Aram Galstyan

We propose to choose intermediate distributions using equal spacing in the moment parameters of our exponential family, which matches grid search performance and allows the schedule to adaptively update over the course of training.

Variational Inference

Latent Embeddings of Point Process Excitations

no code implementations5 May 2020 Myrl G. Marmarelis, Greg Ver Steeg, Aram Galstyan

When specific events seem to spur others in their wake, marked Hawkes processes enable us to reckon with their statistics.

Point Processes

Discovery and Separation of Features for Invariant Representation Learning

no code implementations2 Dec 2019 Ayush Jaiswal, Rob Brekelmans, Daniel Moyer, Greg Ver Steeg, Wael Abd-Almageed, Premkumar Natarajan

Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization.

Representation Learning

Invariant Representations through Adversarial Forgetting

no code implementations11 Nov 2019 Ayush Jaiswal, Daniel Moyer, Greg Ver Steeg, Wael Abd-Almageed, Premkumar Natarajan

We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism.

Nearly-Unsupervised Hashcode Representations for Biomedical Relation Extraction

no code implementations IJCNLP 2019 Sahil Garg, Aram Galstyan, Greg Ver Steeg, Guillermo Cecchi

Recently, kernelized locality sensitive hashcodes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks.

Relation Extraction

Nearly-Unsupervised Hashcode Representations for Relation Extraction

no code implementations9 Sep 2019 Sahil Garg, Aram Galstyan, Greg Ver Steeg, Guillermo Cecchi

Recently, kernelized locality sensitive hashcodes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks.

Relation Extraction

Efficient Covariance Estimation from Temporal Data

2 code implementations30 May 2019 Hrayr Harutyunyan, Daniel Moyer, Hrant Khachatrian, Greg Ver Steeg, Aram Galstyan

Estimating the covariance structure of multivariate time series is a fundamental problem with a wide-range of real-world applications -- from financial modeling to fMRI analysis.

Time Series

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

2 code implementations30 Apr 2019 Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, Aram Galstyan

Existing popular methods for semi-supervised learning with Graph Neural Networks (such as the Graph Convolutional Network) provably cannot learn a general class of neighborhood mixing relationships.

Graph Convolutional Network Node Classification

Exact Rate-Distortion in Autoencoders via Echo Noise

1 code implementation NeurIPS 2019 Rob Brekelmans, Daniel Moyer, Aram Galstyan, Greg Ver Steeg

The noise is constructed in a data-driven fashion that does not require restrictive distributional assumptions.

Representation Learning

Scanner Invariant Representations for Diffusion MRI Harmonization

no code implementations10 Apr 2019 Daniel Moyer, Greg Ver Steeg, Chantal M. W. Tax, Paul M. Thompson

Purpose: In the present work we describe the correction of diffusion-weighted MRI for site and scanner biases using a novel method based on invariant representation.

Fairness Image Reconstruction

Identifying and Analyzing Cryptocurrency Manipulations in Social Media

1 code implementation4 Feb 2019 Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Fred Morstatter, Greg Ver Steeg, Aram Galstyan

Because of the speed and relative anonymity offered by social platforms such as Twitter and Telegram, social media has become a preferred platform for scammers who wish to spread false hype about the cryptocurrency they are trying to pump.

Measures of Tractography Convergence

no code implementations12 Jun 2018 Daniel Moyer, Paul M. Thompson, Greg Ver Steeg

In the present work, we use information theory to understand the empirical convergence rate of tractography, a widely-used approach to reconstruct anatomical fiber pathways in the living brain.

Invariant Representations without Adversarial Training

1 code implementation NeurIPS 2018 Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan

Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation.

Representation Learning

A Forest Mixture Bound for Block-Free Parallel Inference

no code implementations17 May 2018 Neal Lawton, Aram Galstyan, Greg Ver Steeg

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time.

Variational Inference

Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach

no code implementations26 Apr 2018 Sahil Garg, Irina Rish, Guillermo Cecchi, Palash Goyal, Sarik Ghazarian, Shuyang Gao, Greg Ver Steeg, Aram Galstyan

We also derive a novel lower bound on mutual information, used as a model-selection criterion favoring representations with better alignment between the utterances of participants in a collaborative dialogue setting, as well as higher predictability of the generated responses.

Dialogue Generation Model Selection

Auto-Encoding Total Correlation Explanation

no code implementations16 Feb 2018 Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan

Advances in unsupervised learning enable reconstruction and generation of samples from complex distributions, but this success is marred by the inscrutability of the representations learned.

Representation Learning

Stochastic Learning of Nonstationary Kernels for Natural Language Modeling

no code implementations11 Jan 2018 Sahil Garg, Greg Ver Steeg, Aram Galstyan

Natural language processing often involves computations with semantic or syntactic graphs to facilitate sophisticated reasoning based on structural relationships.

Language Modelling

Kernelized Hashcode Representations for Relation Extraction

1 code implementation10 Nov 2017 Sahil Garg, Aram Galstyan, Greg Ver Steeg, Irina Rish, Guillermo Cecchi, Shuyang Gao

Here we propose to use random subspaces of KLSH codes for efficiently constructing an explicit representation of NLP structures suitable for general classification methods.

General Classification Relation Extraction

Disentangled Representations via Synergy Minimization

1 code implementation10 Oct 2017 Greg Ver Steeg, Rob Brekelmans, Hrayr Harutyunyan, Aram Galstyan

Scientists often seek simplified representations of complex systems to facilitate prediction and understanding.

Unifying Local and Global Change Detection in Dynamic Networks

no code implementations9 Oct 2017 Wenzhe Li, Dong Guo, Greg Ver Steeg, Aram Galstyan

Many real-world networks are complex dynamical systems, where both local (e. g., changing node attributes) and global (e. g., changing network topology) processes unfold over time.

Change Detection

Unsupervised Learning via Total Correlation Explanation

no code implementations27 Jun 2017 Greg Ver Steeg

Learning by children and animals occurs effortlessly and largely without obvious supervision.

Fast structure learning with modular regularization

3 code implementations NeurIPS 2019 Greg Ver Steeg, Hrayr Harutyunyan, Daniel Moyer, Aram Galstyan

We also use our approach for estimating covariance structure for a number of real-world datasets and show that it consistently outperforms state-of-the-art estimators at a fraction of the computational cost.

Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge

1 code implementation TACL 2017 Ryan J. Gallagher, Kyle Reing, David Kale, Greg Ver Steeg

While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters.

Toward Interpretable Topic Discovery via Anchored Correlation Explanation

no code implementations22 Jun 2016 Kyle Reing, David C. Kale, Greg Ver Steeg, Aram Galstyan

Many predictive tasks, such as diagnosing a patient based on their medical chart, are ultimately defined by the decisions of human experts.

Variational Information Maximization for Feature Selection

1 code implementation NeurIPS 2016 Shuyang Gao, Greg Ver Steeg, Aram Galstyan

We demonstrate that approximations made by existing methods are based on unrealistic assumptions.

Sifting Common Information from Many Variables

1 code implementation7 Jun 2016 Greg Ver Steeg, Shuyang Gao, Kyle Reing, Aram Galstyan

Measuring the relationship between any pair of variables is a rich and active area of research that is central to scientific practice.

Dimensionality Reduction

Latent Space Model for Multi-Modal Social Data

no code implementations18 Oct 2015 Yoon-Sik Cho, Greg Ver Steeg, Emilio Ferrara, Aram Galstyan

We validate the proposed framework on two problems: prediction of social interactions from user attributes and behaviors, and behavior prediction exploiting network information.

The Information Sieve

2 code implementations8 Jul 2015 Greg Ver Steeg, Aram Galstyan

Each layer of the sieve recovers a single latent factor that is maximally informative about multivariate dependence in the data.

Understanding confounding effects in linguistic coordination: an information-theoretic approach

no code implementations1 Dec 2014 Shuyang Gao, Greg Ver Steeg, Aram Galstyan

We revisit some of the previous studies that reported strong signatures of stylistic accommodation, and find that a significant part of the observed coordination can be attributed to a simple confounding effect - length coordination.

Scalable Link Prediction in Dynamic Networks via Non-Negative Matrix Factorization

no code implementations13 Nov 2014 Linhong Zhu, Dong Guo, Junming Yin, Greg Ver Steeg, Aram Galstyan

We propose a scalable temporal latent space model for link prediction in dynamic social networks, where the goal is to predict links over time based on a sequence of previous graph snapshots.

Link Prediction

Efficient Estimation of Mutual Information for Strongly Dependent Variables

4 code implementations7 Nov 2014 Shuyang Gao, Greg Ver Steeg, Aram Galstyan

We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI.

Maximally Informative Hierarchical Representations of High-Dimensional Data

3 code implementations27 Oct 2014 Greg Ver Steeg, Aram Galstyan

We present bounds on how informative a representation is about input data.

Discovering Structure in High-Dimensional Data Through Correlation Explanation

3 code implementations NeurIPS 2014 Greg Ver Steeg, Aram Galstyan

We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective.

Phase Transitions in Community Detection: A Solvable Toy Model

no code implementations2 Dec 2013 Greg Ver Steeg, Cristopher Moore, Aram Galstyan, Armen E. Allahverdyan

It predicts a first-order detectability transition whenever $q > 2$, while the finite-temperature cavity method shows that this is the case only when $q > 4$.

Community Detection

Demystifying Information-Theoretic Clustering

no code implementations15 Oct 2013 Greg Ver Steeg, Aram Galstyan, Fei Sha, Simon DeDeo

We propose a novel method for clustering data which is grounded in information-theoretic principles and requires no parametric assumptions.

Information-Theoretic Measures of Influence Based on Content Dynamics

3 code implementations22 Aug 2012 Greg Ver Steeg, Aram Galstyan

The fundamental building block of social influence is for one person to elicit a response in another.

Social and Information Networks Physics and Society Applications

Information Transfer in Social Media

1 code implementation12 Oct 2011 Greg Ver Steeg, Aram Galstyan

Recent research has explored the increasingly important role of social media by examining the dynamics of individual and group behavior, characterizing patterns of information diffusion, and identifying influential individuals.

Social and Information Networks Physics and Society Applications

Cannot find the paper you are looking for? You can Submit a new open access paper.