Search Results for author: Zaid Harchaoui

Found 63 papers, 32 papers with code

JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models

1 code implementation13 Feb 2024 Jillian Fisher, Ximing Lu, JaeHun Jung, Liwei Jiang, Zaid Harchaoui, Yejin Choi

The permanence of online content combined with the enhanced authorship identification techniques calls for stronger computational methods to protect the identity and privacy of online authorship when needed, e. g., blind reviews for scientific papers, anonymous online reviews, or anonymous interactions in the mental health forums.

Distributionally Robust Optimization with Bias and Variance Reduction

no code implementations21 Oct 2023 Ronak Mehta, Vincent Roulet, Krishna Pillutla, Zaid Harchaoui

We consider the distributionally robust optimization (DRO) problem with spectral risk-based uncertainty set and $f$-divergence penalty.


Faith and Fate: Limits of Transformers on Compositionality

1 code implementation NeurIPS 2023 Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Sean Welleck, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, Yejin Choi

We formulate compositional tasks as computation graphs to systematically quantify the level of complexity, and break down reasoning steps into intermediate sub-procedures.

Modified Gauss-Newton Algorithms under Noise

no code implementations18 May 2023 Krishna Pillutla, Vincent Roulet, Sham Kakade, Zaid Harchaoui

Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing.

Structured Prediction

Confidence Sets under Generalized Self-Concordance

no code implementations31 Dec 2022 Lang Liu, Zaid Harchaoui

This paper revisits a fundamental problem in statistical inference from a non-asymptotic theoretical viewpoint $\unicode{x2013}$ the construction of confidence sets.

MAUVE Scores for Generative Models: Theory and Practice

1 code implementation30 Dec 2022 Krishna Pillutla, Lang Liu, John Thickstun, Sean Welleck, Swabha Swayamdipta, Rowan Zellers, Sewoong Oh, Yejin Choi, Zaid Harchaoui

We present MAUVE, a family of comparison measures between pairs of distributions such as those encountered in the generative modeling of text or images.


Stochastic Optimization for Spectral Risk Measures

1 code implementation10 Dec 2022 Ronak Mehta, Vincent Roulet, Krishna Pillutla, Lang Liu, Zaid Harchaoui

Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task.

Stochastic Optimization

Statistical and Computational Guarantees for Influence Diagnostics

1 code implementation8 Dec 2022 Jillian Fisher, Lang Liu, Krishna Pillutla, Yejin Choi, Zaid Harchaoui

Influence diagnostics such as influence functions and approximate maximum influence perturbations are popular in machine learning and in AI domain applications.

Stochastic optimization on matrices and a graphon McKean-Vlasov limit

no code implementations2 Oct 2022 Zaid Harchaoui, Sewoong Oh, Soumik Pal, Raghav Somani, Raghavendra Tripathi

The limiting curve of graphons is characterized by a family of stochastic differential equations with reflections and can be thought of as an extension of the classical McKean-Vlasov limit for interacting diffusions.

Stochastic Optimization

Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates

1 code implementation13 Jul 2022 Vincent Roulet, Siddhartha Srinivasa, Maryam Fazel, Zaid Harchaoui

We present the implementation of nonlinear control algorithms based on linear and quadratic approximations of the objective from a functional viewpoint.

Car Racing

Orthogonal Statistical Learning with Self-Concordant Loss

no code implementations30 Apr 2022 Lang Liu, Carlos Cinelli, Zaid Harchaoui

Orthogonal statistical learning and double machine learning have emerged as general frameworks for two-stage statistical prediction in the presence of a nuisance component.

Flat minima generalize for low-rank matrix recovery

no code implementations7 Mar 2022 Lijun Ding, Dmitriy Drusvyatskiy, Maryam Fazel, Zaid Harchaoui

Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance.

Matrix Completion

Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates

no code implementations31 Dec 2021 Nicholas J. Irons, Meyer Scetbon, Soumik Pal, Zaid Harchaoui

Triangular flows, also known as Kn\"{o}the-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation, including popular autoregressive flow models such as real-valued non-volume preserving transformation models (Real NVP).

Density Estimation

Entropy Regularized Optimal Transport Independence Criterion

1 code implementation31 Dec 2021 Lang Liu, Soumik Pal, Zaid Harchaoui

We introduce an independence criterion based on entropy regularized optimal transport.


Federated Learning with Superquantile Aggregation for Heterogeneous Data

1 code implementation17 Dec 2021 Krishna Pillutla, Yassine Laguel, Jérôme Malick, Zaid Harchaoui

We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data.

Federated Learning

Target Propagation via Regularized Inversion

1 code implementation2 Dec 2021 Vincent Roulet, Zaid Harchaoui

Target Propagation (TP) algorithms compute targets instead of gradients along neural networks and propagate them backward in a way that is similar yet different than gradient back-propagation (BP).

Stochastic Optimization under Distributional Drift

1 code implementation NeurIPS 2021 Joshua Cutler, Dmitriy Drusvyatskiy, Zaid Harchaoui

We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself.

Stochastic Optimization valid

Score-Based Change Detection for Gradient-Based Learning Machines

1 code implementation27 Jun 2021 Lang Liu, Joseph Salmon, Zaid Harchaoui

The widespread use of machine learning algorithms calls for automatic change detection algorithms to monitor their behavior over time.

BIG-bench Machine Learning Change Detection +1

MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers

3 code implementations NeurIPS 2021 Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, Zaid Harchaoui

As major progress is made in open-ended text generation, measuring how close machine-generated text is to human language remains a critical open problem.

Text Generation

Differentiable Programming à la Moreau

no code implementations31 Dec 2020 Vincent Roulet, Zaid Harchaoui

The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning.

BIG-bench Machine Learning

Faster Policy Learning with Continuous-Time Gradients

3 code implementations12 Dec 2020 Samuel Ainsworth, Kendall Lowrey, John Thickstun, Zaid Harchaoui, Siddhartha Srinivasa

We study the estimation of policy gradients for continuous-time systems with known dynamics.

Asymptotics of Discrete Schrödinger Bridges via Chaos Decomposition

no code implementations17 Nov 2020 Zaid Harchaoui, Lang Liu, Soumik Pal

We consider instead in this paper the problem where each matching is endowed with a Gibbs probability weight proportional to the exponential of the negative total cost of that matching.

First-order Optimization for Superquantile-based Supervised Learning

1 code implementation30 Sep 2020 Yassine Laguel, Jérôme Malick, Zaid Harchaoui

Classical supervised learning via empirical risk (or negative log-likelihood) minimization hinges upon the assumption that the testing distribution coincides with the training distribution.

BIG-bench Machine Learning regression

Harmonic Decompositions of Convolutional Networks

no code implementations ICML 2020 Meyer Scetbon, Zaid Harchaoui

We present a description of the function space and the smoothness class associated with a convolutional network using the machinery of reproducing kernel Hilbert spaces.

A Spectral Analysis of Dot-product Kernels

no code implementations28 Feb 2020 Meyer Scetbon, Zaid Harchaoui

We present eigenvalue decay estimates of integral operators associated with compositional dot-product kernels.

Device Heterogeneity in Federated Learning: A Superquantile Approach

1 code implementation arXiv preprint 2020 Yassine Laguel, Krishna Pillutla, Jérôme Malick, Zaid Harchaoui

We propose a federated learning framework to handle heterogeneous client devices which do not conform to the population data distribution.

Federated Learning

An Elementary Approach to Convergence Guarantees of Optimization Algorithms for Deep Networks

no code implementations20 Feb 2020 Vincent Roulet, Zaid Harchaoui

We present an approach to obtain convergence guarantees of optimization algorithms for deep networks based on elementary arguments and computations.

BIG-bench Machine Learning

Robust Aggregation for Federated Learning

2 code implementations arXiv preprint 2019 Krishna Pillutla, Sham M. Kakade, Zaid Harchaoui

We present a robust aggregation approach to make federated learning robust to settings when a fraction of the devices may be sending corrupted updates to the server.

Additive models Federated Learning +1

Discriminative Clustering with Representation Learning with any Ratio of Labeled to Unlabeled Data

1 code implementation30 Dec 2019 Corinne Jones, Vincent Roulet, Zaid Harchaoui

We present a discriminative clustering approach in which the feature representation can be learned from data and moreover leverage labeled data.

Clustering Representation Learning

A Statistical Investigation of Long Memory in Language and Music

1 code implementation8 Apr 2019 Alexander Greaves-Tunnell, Zaid Harchaoui

Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data.

Language Modelling Time Series Analysis

Kernel-based Translations of Convolutional Networks

1 code implementation19 Mar 2019 Corinne Jones, Vincent Roulet, Zaid Harchaoui

Convolutional Neural Networks, as most artificial neural networks, are commonly viewed as methods different in essence from kernel-based methods.


Coupled Recurrent Models for Polyphonic Music Composition

no code implementations20 Nov 2018 John Thickstun, Zaid Harchaoui, Dean P. Foster, Sham M. Kakade

This paper introduces a novel recurrent model for music composition that is tailored to the structure of polyphonic music.

Time Series Analysis

Adaptive Denoising of Signals with Local Shift-Invariant Structure

no code implementations11 Jun 2018 Zaid Harchaoui, Anatoli Juditsky, Arkadi Nemirovski, Dmitrii Ostrovskii

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter.


Efficient First-Order Algorithms for Adaptive Signal Denoising

1 code implementation ICML 2018 Dmitrii Ostrovskii, Zaid Harchaoui

Our second contribution is a computational complexity analysis of the proposed procedures, which takes into account their statistical nature and the related notion of statistical accuracy.


Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice

1 code implementation15 Dec 2017 Hongzhou Lin, Julien Mairal, Zaid Harchaoui

One of the keys to achieve acceleration in theory and in practice is to solve these sub-problems with appropriate accuracy by using the right stopping criterion and the right warm-start strategy.

Invariances and Data Augmentation for Supervised Music Transcription

1 code implementation13 Nov 2017 John Thickstun, Zaid Harchaoui, Dean Foster, Sham M. Kakade

This paper explores a variety of models for frame-based music transcription, with an emphasis on the methods needed to reach state-of-the-art on human recordings.

Data Augmentation Music Transcription +1

Catalyst Acceleration for Gradient-Based Non-Convex Optimization

no code implementations31 Mar 2017 Courtney Paquette, Hongzhou Lin, Dmitriy Drusvyatskiy, Julien Mairal, Zaid Harchaoui

We introduce a generic scheme to solve nonconvex optimization problems using gradient-based algorithms originally designed for minimizing convex functions.

Learning Features of Music from Scratch

2 code implementations29 Nov 2016 John Thickstun, Zaid Harchaoui, Sham Kakade

This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research.

BIG-bench Machine Learning Multi-Label Classification +1

An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration

1 code implementation4 Oct 2016 Hongzhou Lin, Julien Mairal, Zaid Harchaoui

We propose an inexact variable-metric proximal point algorithm to accelerate gradient-based optimization algorithms.

Fast and Simple Optimization for Poisson Likelihood Models

no code implementations3 Aug 2016 Niao He, Zaid Harchaoui, Yichen Wang, Le Song

Since almost all gradient-based optimization algorithms rely on Lipschitz-continuity, optimizing Poisson likelihood models with a guarantee of convergence can be challenging, especially for large-scale problems.

Time Series Time Series Analysis

Structure-Blind Signal Recovery

1 code implementation NeurIPS 2016 Dmitry Ostrovsky, Zaid Harchaoui, Anatoli Juditsky, Arkadi Nemirovski

We consider the problem of recovering a signal observed in Gaussian noise.

Statistics Theory Statistics Theory

Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm

no code implementations2 Jul 2016 Yury Maximov, Massih-Reza Amini, Zaid Harchaoui

We propose Rademacher complexity bounds for multiclass classifiers trained with a two-step semi-supervised model.


Convolutional Patch Representations for Image Retrieval: an Unsupervised Approach

no code implementations1 Mar 2016 Mattis Paulin, Julien Mairal, Matthijs Douze, Zaid Harchaoui, Florent Perronnin, Cordelia Schmid

Convolutional neural networks (CNNs) have recently received a lot of attention due to their ability to model local stationary structures in natural images in a multi-scale fashion, when learning all model parameters with supervision.

Image Classification Image Retrieval +1

Beat-Event Detection in Action Movie Franchises

no code implementations15 Aug 2015 Danila Potapov, Matthijs Douze, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid

While important advances were recently made towards temporally localizing and recognizing specific human actions or activities in videos, efficient detection and classification of long video chunks belonging to semantically defined categories such as "pursuit" or "romance" remains challenging. We introduce a new dataset, Action Movie Franchises, consisting of a collection of Hollywood action movie franchises.

Classification Event Detection +1

Semi-proximal Mirror-Prox for Nonsmooth Composite Minimization

no code implementations NeurIPS 2015 Niao He, Zaid Harchaoui

We propose a new first-order optimisation algorithm to solve high-dimensional non-smooth composite minimisation problems.

Learning to track for spatio-temporal action localization

no code implementations ICCV 2015 Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid

We present experimental results for spatio-temporal localization on the UCF-Sports, J-HMDB and UCF-101 action localization datasets, where our approach outperforms the state of the art with a margin of 15%, 7% and 12% respectively in mAP.

Spatio-Temporal Action Localization Temporal Action Localization +1

Learning to Detect Motion Boundaries

no code implementations CVPR 2015 Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid

We compare the results obtained with several state-of-the-art optical flow approaches and study the impact of the different cues used in the random forest. Furthermore, we introduce a new dataset, the YouTube Motion Boundaries dataset (YMB), that comprises 60 sequences taken from real-world videos with manually annotated motion boundaries.

Boundary Detection Optical Flow Estimation

Label-Embedding for Image Classification

2 code implementations30 Mar 2015 Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid

Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce.

Attribute Classification +4

Convolutional Kernel Networks

no code implementations NeurIPS 2014 Julien Mairal, Piotr Koniusz, Zaid Harchaoui, Cordelia Schmid

An important goal in visual recognition is to devise image representations that are invariant to particular transformations.

Image Classification

Transformation Pursuit for Image Classification

no code implementations CVPR 2014 Mattis Paulin, Jerome Revaud, Zaid Harchaoui, Florent Perronnin, Cordelia Schmid

We propose a principled algorithm – Image Transformation Pursuit (ITP) – for the automatic selection of a compact set of transformations.

Classification General Classification +1

Fast and Robust Archetypal Analysis for Representation Learning

1 code implementation CVPR 2014 Yuansi Chen, Julien Mairal, Zaid Harchaoui

We revisit a pioneer unsupervised learning technique called archetypal analysis, which is related to successful data analysis methods such as sparse coding and non-negative matrix factorization.

General Classification Representation Learning

On learning to localize objects with minimal supervision

no code implementations5 Mar 2014 Hyun Oh Song, Ross Girshick, Stefanie Jegelka, Julien Mairal, Zaid Harchaoui, Trevor Darrell

Learning to localize objects with minimal supervision is an important problem in computer vision, since large fully annotated datasets are extremely costly to obtain.

Weakly Supervised Object Detection

Label-Embedding for Attribute-Based Classification

no code implementations CVPR 2013 Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid

The label embedding framework offers other advantages such as the ability to leverage alternative sources of information in addition to attributes (e. g. class hierarchies) or to transition smoothly from zero-shot learning to learning with large quantities of data.

Attribute Classification +3

Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization

no code implementations10 Feb 2013 Zaid Harchaoui, Anatoli Juditsky, Arkadi Nemirovski

Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone $K$, a norm $\|\cdot\|$ and a smooth convex function $f$, we want either 1) to minimize the norm over the intersection of the cone and a level set of $f$, or 2) to minimize over the cone the sum of $f$ and a multiple of the norm.

Cannot find the paper you are looking for? You can Submit a new open access paper.