Search Results for author: Johan A. K. Suykens

Found 48 papers, 8 papers with code

Tensor Network Kalman Filtering for Large-Scale LS-SVMs

no code implementations26 Oct 2021 Maximilian Lucassen, Johan A. K. Suykens, Kim Batselier

Least squares support vector machines are a commonly used supervised learning method for nonlinear regression and classification.

Tensor Networks

On the Double Descent of Random Features Models Trained with SGD

no code implementations13 Oct 2021 Fanghui Liu, Johan A. K. Suykens, Volkan Cevher

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD).

Towards Deterministic Diverse Subset Sampling

no code implementations28 May 2021 Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens

Determinantal point processes (DPPs) are well known models for diverse subset selection problems, including recommendation tasks, document summarization and image search.

Document Summarization Image Retrieval +1

Latent Space Exploration Using Generative Kernel PCA

no code implementations28 May 2021 David Winant, Joachim Schreurs, Johan A. K. Suykens

This connection has led to insights on how to use kernel PCA in a generative procedure, called generative kernel PCA.

Leverage Score Sampling for Complete Mode Coverage in Generative Adversarial Networks

no code implementations6 Apr 2021 Joachim Schreurs, Hannes De Meulemeester, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens

A generative model may overlook underrepresented modes that are less frequent in the empirical data distribution.

Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine

1 code implementation16 Feb 2021 Francesco Tonin, Arun Pandey, Panagiotis Patrinos, Johan A. K. Suykens

Detecting out-of-distribution (OOD) samples is an essential requirement for the deployment of machine learning systems in the real world.

Out-of-Distribution Detection

Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints

no code implementations25 Nov 2020 Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

We introduce Constr-DRKM, a deep kernel method for the unsupervised learning of disentangled data representations.

Determinantal Point Processes Implicitly Regularize Semi-parametric Regression Problems

no code implementations13 Nov 2020 Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy.

Point Processes Time Series

Towards a Unified Quadrature Framework for Large-Scale Kernel Machines

no code implementations3 Nov 2020 Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens

In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical integration representation.

Numerical Integration

Kernel regression in high dimensions: Refined analysis beyond double descent

no code implementations6 Oct 2020 Fanghui Liu, Zhenyu Liao, Johan A. K. Suykens

In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depending on whether the number of training data n exceeds the feature dimension d. By establishing a bias-variance decomposition of the expected excess risk, we show that, while the bias is (almost) independent of d and monotonically decreases with n, the variance depends on n, d and can be unimodal or monotonically decreasing under different regularization schemes.

Outlier detection in non-elliptical data by kernel MRCD

1 code implementation5 Aug 2020 Joachim Schreurs, Iwein Vranckx, Mia Hubert, Johan A. K. Suykens, Peter J. Rousseeuw

The minimum regularized covariance determinant method (MRCD) is a robust estimator for multivariate location and scatter, which detects outliers by fitting a robust covariance matrix to the data.

Outlier Detection

A Theoretical Framework for Target Propagation

2 code implementations NeurIPS 2020 Alexander Meulemans, Francesco S. Carzaniga, Johan A. K. Suykens, João Sacramento, Benjamin F. Grewe

Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization.

Ensemble Kernel Methods, Implicit Regularization and Determinantal Point Processes

no code implementations24 Jun 2020 Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens

By using the framework of Determinantal Point Processes (DPPs), some theoretical results concerning the interplay between diversity and regularization can be obtained.

Point Processes

The Bures Metric for Generative Adversarial Networks

no code implementations16 Jun 2020 Hannes De Meulemeester, Joachim Schreurs, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens

However, under certain circumstances, the training of GANs can lead to mode collapse or mode dropping, i. e. the generative models not being able to sample from the entire probability distribution.

Disentangled Representation Learning and Generation with Manifold Optimization

no code implementations12 Jun 2020 Arun Pandey, Michael Fanuel, Joachim Schreurs, Johan A. K. Suykens

Our analysis shows that such a construction promotes disentanglement by matching the principal directions in latent space with the directions of orthogonal variation in data space.

Representation Learning Stochastic Optimization

Analysis of Regularized Least Squares in Reproducing Kernel Krein Spaces

no code implementations1 Jun 2020 Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens

In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS).

Fast Learning in Reproducing Kernel Krein Spaces via Signed Measures

no code implementations30 May 2020 Fanghui Liu, Xiaolin Huang, Yingyi Chen, Johan A. K. Suykens

In this paper, we attempt to solve a long-lasting open question for non-positive definite (non-PD) kernels in machine learning community: can a given non-PD kernel be decomposed into the difference of two PD kernels (termed as positive decomposition)?

Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond

no code implementations23 Apr 2020 Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens

This survey may serve as a gentle introduction to this topic, and as a users' guide for practitioners interested in applying the representative algorithms and understanding theoretical results under various technical assumptions.

Diversity sampling is an implicit regularization for kernel methods

no code implementations20 Feb 2020 Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

The Nystr\"om approximation -- based on a subset of landmarks -- gives a low rank approximation of the kernel matrix, and is known to provide a form of implicit regularization.

Point Processes

Wasserstein Exponential Kernels

2 code implementations5 Feb 2020 Henri De Plaen, Michaël Fanuel, Johan A. K. Suykens

In the context of kernel methods, the similarity between data points is encoded by the kernel function which is often defined thanks to the Euclidean distance, a common example being the squared exponential kernel.

Robust Generative Restricted Kernel Machines using Weighted Conjugate Feature Duality

no code implementations4 Feb 2020 Arun Pandey, Joachim Schreurs, Johan A. K. Suykens

Experiments show that the weighted RKM is capable of generating clean images when contamination is present in the training data.

Random Fourier Features via Fast Surrogate Leverage Weighted Sampling

no code implementations20 Nov 2019 Fanghui Liu, Xiaolin Huang, Yudong Chen, Jie Yang, Johan A. K. Suykens

In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation.

Generative Restricted Kernel Machines: A Framework for Multi-view Generation and Disentangled Feature Learning

no code implementations19 Jun 2019 Arun Pandey, Joachim Schreurs, Johan A. K. Suykens

This paper introduces a novel framework for generative models based on Restricted Kernel Machines (RKMs) with joint multi-view generation and uncorrelated feature learning, called Gen-RKM.

Nyström landmark sampling and regularized Christoffel functions

no code implementations29 May 2019 Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

Selecting diverse and important items, called landmarks, from a large set is a problem of interest in machine learning.

Point Processes

Efficient hinging hyperplanes neural network and its application in nonlinear system identification

no code implementations15 May 2019 Jun Xu, Qinghua Tao, Zhen Li, Xiangming Xi, Johan A. K. Suykens, Shuning Wang

It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, which was also proposed based on the model of HH and find good applications in system identification.

Variable Selection

Two-stage Best-scored Random Forest for Large-scale Regression

no code implementations9 May 2019 Hanyuan Hang, Yingyi Chen, Johan A. K. Suykens

We propose a novel method designed for large-scale regression problems, namely the two-stage best-scored random forest (TBRF).

Ensemble Learning

Spatio-temporal Stacked LSTM for Temperature Prediction in Weather Forecasting

no code implementations15 Nov 2018 Zahra Karevan, Johan A. K. Suykens

Subsequently, the input of the second LSTM layer is formed based on the combination of the hidden states of the first layer LSTM models.

Time Series Time Series Prediction +1

Generalization Properties of hyper-RKHS and its Applications

no code implementations26 Sep 2018 Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view.

Learning Theory

Positive semi-definite embedding for dimensionality reduction and out-of-sample extensions

no code implementations20 Nov 2017 Michaël Fanuel, Antoine Aspeel, Jean-Charles Delvenne, Johan A. K. Suykens

In machine learning or statistics, it is often desirable to reduce the dimensionality of a sample of data points in a high dimensional space $\mathbb{R}^d$.

Dimensionality Reduction

Solving $\ell^p\!$-norm regularization with tensor kernels

no code implementations18 Jul 2017 Saverio Salzo, Johan A. K. Suykens, Lorenzo Rosasco

In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of $\ell^p$ regularized learning methods.

Indefinite Kernel Logistic Regression with Concave-inexact-convex Procedure

no code implementations6 Jul 2017 Fanghui Liu, Xiaolin Huang, Chen Gong, Jie Yang, Johan A. K. Suykens

Since the concave-convex procedure has to solve a sub-problem in each iteration, we propose a concave-inexact-convex procedure (CCICP) algorithm with an inexact solving scheme to accelerate the solving process.

Modified Frank-Wolfe Algorithm for Enhanced Sparsity in Support Vector Machine Classifiers

no code implementations19 Jun 2017 Carlos M. Alaíz, Johan A. K. Suykens

This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Cand\`es et al. and on the equivalence between Lasso and SVM shown recently by Jaggi.

A Statistical Learning Approach to Modal Regression

no code implementations20 Feb 2017 Yunlong Feng, Jun Fan, Johan A. K. Suykens

However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view.

Robust Classification of Graph-Based Data

no code implementations21 Dec 2016 Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens

A graph-based classification method is proposed for semi-supervised learning in the case of Euclidean data and for classification in the case of graph data.

Classification General Classification +1

Parallelized Tensor Train Learning of Polynomial Classifiers

1 code implementation20 Dec 2016 Zhongming Chen, Kim Batselier, Johan A. K. Suykens, Ngai Wong

In pattern classification, polynomial classifiers are well-studied methods as they are capable of generating complex decision surfaces.

General Classification

Kernel Density Estimation for Dynamical Systems

no code implementations13 Jul 2016 Hanyuan Hang, Ingo Steinwart, Yunlong Feng, Johan A. K. Suykens

We study the density estimation problem with observations generated by certain dynamical systems that admit a unique underlying invariant Lebesgue density.

Density Estimation

Learning theory estimates with observations from general stationary stochastic processes

no code implementations10 May 2016 Hanyuan Hang, Yunlong Feng, Ingo Steinwart, Johan A. K. Suykens

We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a unified treatment on analyzing the learning schemes with various mixing processes can be conducted and a sharp oracle inequality for generic regularized empirical risk minimization schemes can be established.

Learning Theory

Generalized support vector regression: duality and tensor-kernel representation

no code implementations18 Mar 2016 Saverio Salzo, Johan A. K. Suykens

In this paper we study the variational problem associated to support vector regression in Banach function spaces.

Fast and Scalable Lasso via Stochastic Frank-Wolfe Methods with a Convergence Guarantee

1 code implementation24 Oct 2015 Emanuele Frandi, Ricardo Nanculef, Stefano Lodi, Claudio Sartori, Johan A. K. Suykens

Frank-Wolfe (FW) algorithms have been often proposed over the last few years as efficient solvers for a variety of optimization problems arising in the field of Machine Learning.

Kernel Spectral Clustering and applications

no code implementations3 May 2015 Rocco Langone, Raghvendra Mall, Carlos Alzate, Johan A. K. Suykens

This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics.

Model Selection Semantic Segmentation +2

Higher order Matching Pursuit for Low Rank Tensor Learning

no code implementations7 Mar 2015 Yuning Yang, Siamak Mehrkanoon, Johan A. K. Suykens

In this paper, we propose higher order matching pursuit for low rank tensor learning problems with a convex or a nonconvex cost function, which is a generalization of the matching pursuit type methods.

Fast Prediction with SVM Models Containing RBF Kernels

1 code implementation4 Mar 2014 Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor

We present an approximation scheme for support vector machine models that use an RBF kernel.

A Robust Ensemble Approach to Learn From Positive and Unlabeled Data Using SVM Base Models

1 code implementation13 Feb 2014 Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor

The included benchmark comprises three settings with increasing label noise: (i) fully supervised, (ii) PU learning and (iii) PU learning with false positives.

Learning Tensors in Reproducing Kernel Hilbert Spaces with Multilinear Spectral Penalties

no code implementations18 Oct 2013 Marco Signoretto, Lieven De Lathauwer, Johan A. K. Suykens

We present a general framework to learn functions in tensor product reproducing kernel Hilbert spaces (TP-RKHSs).

Transfer Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.