Search Results for author: Martin Takáč

Found 69 papers, 13 papers with code

Parallel Coordinate Descent Methods for Big Data Optimization

no code implementations4 Dec 2012 Peter Richtárik, Martin Takáč

In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function.

Alternating Maximization: Unifying Framework for 8 Sparse PCA Formulations and Efficient Parallel Codes

1 code implementation17 Dec 2012 Peter Richtárik, Majid Jahani, Selin Damla Ahipaşaoğlu, Martin Takáč

Given a multivariate data set, sparse principal component analysis (SPCA) aims to extract several linear combinations of the variables that together explain the variance in the data as much as possible, while controlling the number of nonzero loadings in these combinations.

Distributed Coordinate Descent Method for Learning with Big Data

no code implementations8 Oct 2013 Peter Richtárik, Martin Takáč

In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data.

On Optimal Probabilities in Stochastic Coordinate Descent Methods

no code implementations13 Oct 2013 Peter Richtárik, Martin Takáč

We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly.

TOP-SPIN: TOPic discovery via Sparse Principal component INterference

no code implementations4 Nov 2013 Martin Takáč, Selin Damla Ahipaşaoğlu, Ngai-Man Cheung, Peter Richtárik

Our approach attacks the maximization problem in sparse PCA directly and is scalable to high-dimensional data.

Fast Distributed Coordinate Descent for Non-Strongly Convex Losses

no code implementations21 May 2014 Olivier Fercoq, Zheng Qu, Peter Richtárik, Martin Takáč

We propose an efficient distributed randomized coordinate descent method for minimizing regularized non-strongly convex loss functions.

mS2GD: Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting

no code implementations17 Oct 2014 Jakub Konečný, Jie Liu, Peter Richtárik, Martin Takáč

Our method first performs a deterministic step (computation of the gradient of the objective function at the starting point), followed by a large number of stochastic steps.

SDNA: Stochastic Dual Newton Ascent for Empirical Risk Minimization

no code implementations8 Feb 2015 Zheng Qu, Peter Richtárik, Martin Takáč, Olivier Fercoq

We propose a new algorithm for minimizing regularized empirical loss: Stochastic Dual Newton Ascent (SDNA).

Adding vs. Averaging in Distributed Primal-Dual Optimization

1 code implementation12 Feb 2015 Chenxin Ma, Virginia Smith, Martin Jaggi, Michael. I. Jordan, Peter Richtárik, Martin Takáč

Distributed optimization methods for large-scale machine learning suffer from a communication bottleneck.

Distributed Optimization

Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting

no code implementations16 Apr 2015 Jakub Konečný, Jie Liu, Peter Richtárik, Martin Takáč

Our method first performs a deterministic step (computation of the gradient of the objective function at the starting point), followed by a large number of stochastic steps.

Linear Convergence of the Randomized Feasible Descent Method Under the Weak Strong Convexity Assumption

no code implementations8 Jun 2015 Chenxin Ma, Rachael Tappenden, Martin Takáč

We show that the famous SDCA algorithm for optimizing the SVM dual problem, or the stochastic coordinate descent method for the LASSO problem, fits into the framework of RC-FDM.

Distributed Mini-Batch SDCA

no code implementations29 Jul 2015 Martin Takáč, Peter Richtárik, Nathan Srebro

We present an improved analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization (i. e. SVM and SVM-type objectives).

Partitioning Data on Features or Samples in Communication-Efficient Distributed Optimization?

no code implementations22 Oct 2015 Chenxin Ma, Martin Takáč

In this paper we study the effect of the way that the data is partitioned in distributed optimization.

Distributed Optimization

Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization

no code implementations22 Oct 2015 Xi He, Martin Takáč

This work is motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method, however, we allow a non-uniform selection of "dual" coordinates in SDCA.

Distributed Optimization with Arbitrary Local Solvers

1 code implementation13 Dec 2015 Chenxin Ma, Jakub Konečný, Martin Jaggi, Virginia Smith, Michael. I. Jordan, Peter Richtárik, Martin Takáč

To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally, and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods.

Distributed Optimization

Primal-Dual Rates and Certificates

no code implementations16 Feb 2016 Celestine Dünner, Simone Forte, Martin Takáč, Martin Jaggi

We propose an algorithm-independent framework to equip existing optimization methods with primal-dual certificates.

BIG-bench Machine Learning

Distributed Inexact Damped Newton Method: Data Partitioning and Load-Balancing

no code implementations16 Mar 2016 Chenxin Ma, Martin Takáč

In this paper we study inexact dumped Newton method implemented in a distributed environment.

Distributed Optimization

A Multi-Batch L-BFGS Method for Machine Learning

no code implementations NeurIPS 2016 Albert S. Berahas, Jorge Nocedal, Martin Takáč

The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature.

BIG-bench Machine Learning Distributed Computing

Applying Deep Learning to the Newsvendor Problem

2 code implementations7 Jul 2016 Afshin Oroojlooyjadid, Lawrence Snyder, Martin Takáč

However, approximating the probability distribution is not easy and is prone to error; therefore, the resulting solution to the newsvendor problem may be not optimal.

SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient

no code implementations ICML 2017 Lam M. Nguyen, Jie Liu, Katya Scheinberg, Martin Takáč

In this paper, we propose a StochAstic Recursive grAdient algoritHm (SARAH), as well as its practical variant SARAH+, as a novel approach to the finite-sum minimization problems.

BIG-bench Machine Learning

Stochastic Recursive Gradient Algorithm for Nonconvex Optimization

no code implementations20 May 2017 Lam M. Nguyen, Jie Liu, Katya Scheinberg, Martin Takáč

In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses.

Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory

no code implementations4 Jun 2017 Peter Richtárik, Martin Takáč

We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem.

Stochastic Optimization

A Robust Multi-Batch L-BFGS Method for Machine Learning

no code implementations26 Jul 2017 Albert S. Berahas, Martin Takáč

This paper describes an implementation of the L-BFGS method designed to deal with two adversarial situations.

BIG-bench Machine Learning Binary Classification +1

A Deep Q-Network for the Beer Game: A Deep Reinforcement Learning algorithm to Solve Inventory Optimization Problems

no code implementations20 Aug 2017 Afshin Oroojlooyjadid, MohammadReza Nazari, Lawrence Snyder, Martin Takáč

The game is a decentralized, multi-agent, cooperative problem that can be modeled as a serial supply chain network in which agents cooperatively attempt to minimize the total cost of the network even though each agent can only observe its own local information.

Management Transfer Learning

Stock-out Prediction in Multi-echelon Networks

no code implementations20 Sep 2017 Afshin Oroojlooyjadid, Lawrence Snyder, Martin Takáč

In multi-echelon inventory systems the performance of a given node is affected by events that occur at many other nodes and in many other time periods.

Fast and Safe: Accelerated gradient methods with optimality certificates and underestimate sequences

no code implementations10 Oct 2017 Majid Jahani, Naga Venkata C. Gudapati, Chenxin Ma, Rachael Tappenden, Martin Takáč

In this work we introduce the concept of an Underestimate Sequence (UES), which is motivated by Nesterov's estimate sequence.

An Accelerated Communication-Efficient Primal-Dual Optimization Framework for Structured Machine Learning

1 code implementation14 Nov 2017 Chenxin Ma, Martin Jaggi, Frank E. Curtis, Nathan Srebro, Martin Takáč

In this paper, an accelerated variant of CoCoA+ is proposed and shown to possess a convergence rate of $\mathcal{O}(1/t^2)$ in terms of reducing suboptimality.

BIG-bench Machine Learning Distributed Optimization

SGD and Hogwild! Convergence Without the Bounded Gradients Assumption

no code implementations ICML 2018 Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk, Peter Richtárik, Katya Scheinberg, Martin Takáč

In (Bottou et al., 2016), a new analysis of convergence of SGD is performed under the assumption that stochastic gradients are bounded with respect to the true gradient norm.

BIG-bench Machine Learning

Reinforcement Learning for Solving the Vehicle Routing Problem

4 code implementations NeurIPS 2018 Mohammadreza Nazari, Afshin Oroojlooy, Lawrence V. Snyder, Martin Takáč

Our model represents a parameterized stochastic policy, and by applying a policy gradient algorithm to optimize its parameters, the trained model produces the solution as a sequence of consecutive actions in real time, without the need to re-train for every new problem instance.

Combinatorial Optimization reinforcement-learning +1

Active Metric Learning for Supervised Classification

no code implementations28 Mar 2018 Krishnan Kumaran, Dimitri Papageorgiou, Yutong Chang, Minhan Li, Martin Takáč

We present mixed-integer optimization approaches to find optimal distance metrics that generalize the Mahalanobis metric extensively studied in the literature.

Active Learning Classification +3

Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy

no code implementations26 Oct 2018 Majid Jahani, Xi He, Chenxin Ma, Aryan Mokhtari, Dheevatsa Mudigere, Alejandro Ribeiro, Martin Takáč

In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory statistical accuracy.

New Convergence Aspects of Stochastic Gradient Algorithms

no code implementations10 Nov 2018 Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik, Katya Scheinberg, Martin Takáč, Marten van Dijk

We show the convergence of SGD for strongly convex objective function without using bounded gradient assumption when $\{\eta_t\}$ is a diminishing sequence and $\sum_{t=0}^\infty \eta_t \rightarrow \infty$.

Inexact SARAH Algorithm for Stochastic Optimization

no code implementations25 Nov 2018 Lam M. Nguyen, Katya Scheinberg, Martin Takáč

We develop and analyze a variant of the SARAH algorithm, which does not require computation of the exact gradient.

Stochastic Optimization

Distributed Learning with Compressed Gradient Differences

no code implementations26 Jan 2019 Konstantin Mishchenko, Eduard Gorbunov, Martin Takáč, Peter Richtárik

Our analysis of block-quantization and differences between $\ell_2$ and $\ell_{\infty}$ quantization closes the gaps in theory and practice.

Distributed Computing Quantization

Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample

1 code implementation28 Jan 2019 Albert S. Berahas, Majid Jahani, Peter Richtárik, Martin Takáč

We present two sampled quasi-Newton methods (sampled LBFGS and sampled LSR1) for solving empirical risk minimization problems that arise in machine learning.

Benchmarking BIG-bench Machine Learning +3

Multi-Agent Image Classification via Reinforcement Learning

1 code implementation13 May 2019 Hossein K. Mousavi, MohammadReza Nazari, Martin Takáč, Nader Motee

We investigate a classification problem using multiple mobile agents capable of collecting (partial) pose-dependent observations of an unknown environment.

Classification General Classification +3

Scaling Up Quasi-Newton Algorithms: Communication Efficient Distributed SR1

no code implementations30 May 2019 Majid Jahani, MohammadReza Nazari, Sergey Rusakov, Albert S. Berahas, Martin Takáč

In this paper, we present a scalable distributed implementation of the Sampled Limited-memory Symmetric Rank-1 (S-LSR1) algorithm.

Don't Forget Your Teacher: A Corrective Reinforcement Learning Framework

no code implementations30 May 2019 Mohammadreza Nazari, Majid Jahani, Lawrence V. Snyder, Martin Takáč

Therefore, we propose a student-teacher RL mechanism in which the RL (the "student") learns to maximize its reward, subject to a constraint that bounds the difference between the RL policy and the "teacher" policy.

reinforcement-learning Reinforcement Learning (RL) +1

A Layered Architecture for Active Perception: Image Classification using Deep Reinforcement Learning

no code implementations20 Sep 2019 Hossein K. Mousavi, Guangyi Liu, Weihang Yuan, Martin Takáč, Héctor Muñoz-Avila, Nader Motee

We propose a planning and perception mechanism for a robot (agent), that can only observe the underlying environment partially, in order to solve an image classification problem.

General Classification Image Classification +2

FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methods

no code implementations28 Oct 2019 Nur Sila Gulgec, Zheng Shi, Neil Deshmukh, Shamim Pakzad, Martin Takáč

Discovering the underlying physical behavior of complex systems is a crucial, but less well-understood topic in many engineering disciplines.

Distributed Fixed Point Methods with Compressed Iterates

no code implementations20 Dec 2019 Sélim Chraibi, Ahmed Khaled, Dmitry Kovalev, Peter Richtárik, Adil Salim, Martin Takáč

We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed.

Federated Learning

Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations

no code implementations2 Jun 2020 Zheng Shi, Nur Sila Gulgec, Albert S. Berahas, Shamim N. Pakzad, Martin Takáč

Discovering the underlying behavior of complex systems is an important topic in many science and engineering disciplines.

Constrained Combinatorial Optimization with Reinforcement Learning

no code implementations22 Jun 2020 Ruben Solozabal, Josu Ceberio, Martin Takáč

This paper presents a framework to tackle constrained combinatorial optimization problems using deep Reinforcement Learning (RL).

Combinatorial Optimization reinforcement-learning +1

DynNet: Physics-based neural architecture design for linear and nonlinear structural response modeling and prediction

no code implementations3 Jul 2020 Soheil Sadeghi Eshkevari, Martin Takáč, Shamim N. Pakzad, Majid Jahani

Data-driven models for predicting dynamic responses of linear and nonlinear systems are of great importance due to their wide application from probabilistic analysis to inverse problems such as system identification and damage diagnosis.

Decentralized Personalized Federated Learning for Min-Max Problems

no code implementations14 Jun 2021 Ekaterina Borodich, Aleksandr Beznosikov, Abdurakhmon Sadiev, Vadim Sushko, Nikolay Savelyev, Martin Takáč, Alexander Gasnikov

Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of training data.

Distributed Optimization Personalized Federated Learning

Random-reshuffled SARAH does not need a full gradient computations

no code implementations26 Nov 2021 Aleksandr Beznosikov, Martin Takáč

The StochAstic Recursive grAdient algoritHm (SARAH) algorithm is a variance reduced variant of the Stochastic Gradient Descent (SGD) algorithm that needs a gradient of the objective function from time to time.

Stochastic Gradient Methods with Preconditioned Updates

no code implementations1 Jun 2022 Abdurakhmon Sadiev, Aleksandr Beznosikov, Abdulla Jasem Almansoori, Dmitry Kamzolov, Rachael Tappenden, Martin Takáč

There are several algorithms for such problems, but existing methods often work poorly when the problem is badly scaled and/or ill-conditioned, and a primary goal of this work is to introduce methods that alleviate this issue.

On Scaled Methods for Saddle Point Problems

no code implementations16 Jun 2022 Aleksandr Beznosikov, Aibek Alanov, Dmitry Kovalev, Martin Takáč, Alexander Gasnikov

Methods with adaptive scaling of different features play a key role in solving saddle point problems, primarily due to Adam's popularity for solving adversarial machine learning problems, including GANS training.

Suppressing Poisoning Attacks on Federated Learning for Medical Imaging

1 code implementation15 Jul 2022 Naif Alkhunaizi, Dmitry Kamzolov, Martin Takáč, Karthik Nandakumar

Federated Learning (FL) is a promising solution that enables collaborative training through exchange of model parameters instead of raw data.

Federated Learning Outlier Detection

SP2: A Second Order Stochastic Polyak Method

no code implementations17 Jul 2022 Shuang Li, William J. Swartworth, Martin Takáč, Deanna Needell, Robert M. Gower

We take a step further and develop a method for solving the interpolation equations that uses the local second-order approximation of the model.

Matrix Completion Second-order methods

FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences

no code implementations18 Oct 2022 Artem Agafonov, Brahim Erraji, Martin Takáč

In the recent paper FLECS (Agafonov et al, FLECS: A Federated Learning Second-Order Framework via Compression and Sketching), the second-order framework FLECS was proposed for the Federated Learning problem.

Federated Learning Stochastic Optimization

Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points

no code implementations2 Nov 2022 Rachael Tappenden, Martin Takáč

This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on the gradients.

Partial Disentanglement with Partially-Federated GANs (PaDPaF)

1 code implementation7 Dec 2022 Abdulla Jasem Almansoori, Samuel Horváth, Martin Takáč

Federated learning has become a popular machine learning paradigm with many potential real-life applications, including recommendation systems, the Internet of Things (IoT), healthcare, and self-driving cars.

Disentanglement Federated Learning +2

Learning Confident Classifiers in the Presence of Label Noise

no code implementations2 Jan 2023 Asma Ahmed Hashmi, Aigerim Zhumabayeva, Nikita Kotelevskii, Artem Agafonov, Mohammad Yaqub, Maxim Panov, Martin Takáč

We evaluate the proposed method on a series of classification tasks such as noisy versions of MNIST, CIFAR-10, Fashion-MNIST datasets as well as CIFAR-10N, which is real-world dataset with noisy human annotations.

Image Segmentation Medical Image Segmentation +2

Stochastic Gradient Descent with Preconditioned Polyak Step-size

1 code implementation3 Oct 2023 Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov, Martin Takáč

Stochastic Gradient Descent (SGD) is one of the many iterative optimization methods that are widely used in solving machine learning problems.

Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networks

no code implementations18 Dec 2023 Nikita Kotelevskii, Samuel Horváth, Karthik Nandakumar, Martin Takáč, Maxim Panov

This paper presents a new approach to federated learning that allows selecting a model from global and personalized ones that would perform better for a particular input point.

Personalized Federated Learning Uncertainty Quantification

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms

1 code implementation28 Dec 2023 Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov, Robert Gower, Martin Takáč

Adaptive optimization methods are widely recognized as among the most popular approaches for training Deep Neural Networks (DNNs).

Federated Learning Can Find Friends That Are Beneficial

no code implementations7 Feb 2024 Nazarii Tupitsa, Samuel Horváth, Martin Takáč, Eduard Gorbunov

In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges.

Federated Learning

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size

no code implementations7 Feb 2024 Petr Ostroukhov, Aigerim Zhumabayeva, Chulu Xiang, Alexander Gasnikov, Martin Takáč, Dmitry Kamzolov

To substantiate the efficacy of our method, we experimentally show, how the introduction of adaptive step size and adaptive batch size gradually improves the performance of regular SGD.

FRESCO: Federated Reinforcement Energy System for Cooperative Optimization

no code implementations27 Mar 2024 Nicolas Mauricio Cuadrado, Roberto Alejandro Gutierrez, Martin Takáč

The rise in renewable energy is creating new dynamics in the energy grid that promise to create a cleaner and more participative energy grid, where technology plays a crucial part in making the required flexibility to achieve the vision of the next-generation grid.

Generalized Policy Learning for Smart Grids: FL TRPO Approach

no code implementations27 Mar 2024 Yunxiang Li, Nicolas Mauricio Cuadrado, Samuel Horváth, Martin Takáč

The smart grid domain requires bolstering the capabilities of existing energy management systems; Federated Learning (FL) aligns with this goal as it demonstrates a remarkable ability to train models on heterogeneous datasets while maintaining data privacy, making it suitable for smart grid applications, which often involve disparate data distributions and interdependencies among features that hinder the suitability of linear models.

Cannot find the paper you are looking for? You can Submit a new open access paper.