no code implementations • 21 Dec 2016 • Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens
A graph-based classification method is proposed for semi-supervised learning in the case of Euclidean data and for classification in the case of graph data.
no code implementations • 19 Jun 2017 • Carlos M. Alaíz, Johan A. K. Suykens
This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Cand\`es et al. and on the equivalence between Lasso and SVM shown recently by Jaggi.
no code implementations • 14 May 2015 • Xiaolin Huang, Lei Shi, Ming Yan, Johan A. K. Suykens
The one-sided $\ell_1$ loss and the linear loss are two popular loss functions for 1bit-CS.
no code implementations • 18 Jul 2017 • Saverio Salzo, Johan A. K. Suykens, Lorenzo Rosasco
In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of $\ell^p$ regularized learning methods.
no code implementations • 6 Jul 2017 • Fanghui Liu, Xiaolin Huang, Chen Gong, Jie Yang, Johan A. K. Suykens
Since the concave-convex procedure has to solve a sub-problem in each iteration, we propose a concave-inexact-convex procedure (CCICP) algorithm with an inexact solving scheme to accelerate the solving process.
no code implementations • 18 Mar 2016 • Saverio Salzo, Johan A. K. Suykens
In this paper we study the variational problem associated to support vector regression in Banach function spaces.
no code implementations • 20 Feb 2017 • Yunlong Feng, Jun Fan, Johan A. K. Suykens
However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view.
no code implementations • 21 Oct 2016 • Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens
In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem.
no code implementations • 13 Jul 2016 • Hanyuan Hang, Ingo Steinwart, Yunlong Feng, Johan A. K. Suykens
We study the density estimation problem with observations generated by certain dynamical systems that admit a unique underlying invariant Lebesgue density.
no code implementations • 10 May 2016 • Hanyuan Hang, Yunlong Feng, Ingo Steinwart, Johan A. K. Suykens
We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a unified treatment on analyzing the learning schemes with various mixing processes can be conducted and a sharp oracle inequality for generic regularized empirical risk minimization schemes can be established.
no code implementations • 3 May 2015 • Rocco Langone, Raghvendra Mall, Carlos Alzate, Johan A. K. Suykens
This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics.
no code implementations • 7 Mar 2015 • Yuning Yang, Siamak Mehrkanoon, Johan A. K. Suykens
In this paper, we propose higher order matching pursuit for low rank tensor learning problems with a convex or a nonconvex cost function, which is a generalization of the matching pursuit type methods.
no code implementations • 5 Feb 2015 • Emanuele Frandi, Ricardo Nanculef, Johan A. K. Suykens
Frank-Wolfe algorithms have recently regained the attention of the Machine Learning community.
no code implementations • 18 Oct 2013 • Marco Signoretto, Lieven De Lathauwer, Johan A. K. Suykens
We present a general framework to learn functions in tensor product reproducing kernel Hilbert spaces (TP-RKHSs).
no code implementations • 26 Sep 2018 • Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view.
no code implementations • 15 Nov 2018 • Zahra Karevan, Johan A. K. Suykens
Subsequently, the input of the second LSTM layer is formed based on the combination of the hidden states of the first layer LSTM models.
no code implementations • 9 May 2019 • Hanyuan Hang, Yingyi Chen, Johan A. K. Suykens
We propose a novel method designed for large-scale regression problems, namely the two-stage best-scored random forest (TBRF).
no code implementations • 15 May 2019 • Jun Xu, Qinghua Tao, Zhen Li, Xiangming Xi, Johan A. K. Suykens, Shuning Wang
It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, which was also proposed based on the model of HH and find good applications in system identification.
no code implementations • 29 May 2019 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens
In this context, we propose a deterministic and a randomized adaptive algorithm for selecting landmark points within a training data set.
no code implementations • 19 Jun 2019 • Arun Pandey, Joachim Schreurs, Johan A. K. Suykens
This paper introduces a novel framework for generative models based on Restricted Kernel Machines (RKMs) with joint multi-view generation and uncorrelated feature learning, called Gen-RKM.
no code implementations • 20 Nov 2019 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Jie Yang, Johan A. K. Suykens
In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation.
no code implementations • 4 Feb 2020 • Arun Pandey, Joachim Schreurs, Johan A. K. Suykens
Experiments show that the weighted RKM is capable of generating clean images when contamination is present in the training data.
no code implementations • 20 Feb 2020 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens
The Nystr\"om approximation -- based on a subset of landmarks -- gives a low rank approximation of the kernel matrix, and is known to provide a form of implicit regularization.
no code implementations • 23 Apr 2020 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens
This survey may serve as a gentle introduction to this topic, and as a users' guide for practitioners interested in applying the representative algorithms and understanding theoretical results under various technical assumptions.
no code implementations • 30 May 2020 • Fanghui Liu, Xiaolin Huang, Yingyi Chen, Johan A. K. Suykens
In this paper, we attempt to solve a long-lasting open question for non-positive definite (non-PD) kernels in machine learning community: can a given non-PD kernel be decomposed into the difference of two PD kernels (termed as positive decomposition)?
no code implementations • 1 Jun 2020 • Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens
In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS).
no code implementations • 12 Jun 2020 • Arun Pandey, Michael Fanuel, Joachim Schreurs, Johan A. K. Suykens
Our analysis shows that such a construction promotes disentanglement by matching the principal directions in the latent space with the directions of orthogonal variation in data space.
no code implementations • 16 Jun 2020 • Hannes De Meulemeester, Joachim Schreurs, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens
However, under certain circumstances, the training of GANs can lead to mode collapse or mode dropping, i. e. the generative models not being able to sample from the entire probability distribution.
no code implementations • 24 Jun 2020 • Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens
By using the framework of Determinantal Point Processes (DPPs), some theoretical results concerning the interplay between diversity and regularization can be obtained.
no code implementations • 6 Oct 2020 • Fanghui Liu, Zhenyu Liao, Johan A. K. Suykens
In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depending on whether the number of training data n exceeds the feature dimension d. By establishing a bias-variance decomposition of the expected excess risk, we show that, while the bias is (almost) independent of d and monotonically decreases with n, the variance depends on n, d and can be unimodal or monotonically decreasing under different regularization schemes.
no code implementations • 3 Nov 2020 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens
In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical integration representation.
no code implementations • 13 Nov 2020 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens
Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy.
no code implementations • 25 Nov 2020 • Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
We introduce Constr-DRKM, a deep kernel method for the unsupervised learning of disentangled data representations.
no code implementations • 6 Apr 2021 • Joachim Schreurs, Hannes De Meulemeester, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens
A generative model may overlook underrepresented modes that are less frequent in the empirical data distribution.
no code implementations • 28 May 2021 • David Winant, Joachim Schreurs, Johan A. K. Suykens
This connection has led to insights on how to use kernel PCA in a generative procedure, called generative kernel PCA.
no code implementations • 28 May 2021 • Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens
Determinantal point processes (DPPs) are well known models for diverse subset selection problems, including recommendation tasks, document summarization and image search.
no code implementations • 13 Oct 2021 • Fanghui Liu, Johan A. K. Suykens, Volkan Cevher
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.
no code implementations • 26 Oct 2021 • Maximilian Lucassen, Johan A. K. Suykens, Kim Batselier
Least squares support vector machines are a commonly used supervised learning method for nonlinear regression and classification.
no code implementations • 3 Feb 2022 • Mingzhen He, Fan He, Lei Shi, Xiaolin Huang, Johan A. K. Suykens
Asymmetric kernels naturally exist in real life, e. g., for conditional probability and directed graphs.
no code implementations • 18 Jun 2022 • Qinghua Tao, Li Li, Xiaolin Huang, Xiangming Xi, Shuning Wang, Johan A. K. Suykens
To apply PWLNN methods, both the representation and the learning have long been studied.
no code implementations • 24 Jan 2023 • Arun Pandey, Hannes De Meulemeester, Bart De Moor, Johan A. K. Suykens
In this paper, we propose a kernel principal component analysis model for multi-variate time series forecasting, where the training and prediction schemes are derived from the multi-view formulation of Restricted Kernel Machines.
1 code implementation • 31 Jan 2023 • Sonny Achten, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs.
no code implementations • 4 Mar 2023 • Jiani Liu, Qinghua Tao, Ce Zhu, Yipeng Liu, Johan A. K. Suykens
Multitask learning (MTL) can utilize the relatedness between multiple tasks for performance improvement.
no code implementations • 12 Jun 2023 • Qinghua Tao, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
We describe a nonlinear extension of the matrix Singular Value Decomposition through asymmetric kernels, namely KSVD.
no code implementations • 12 Jun 2023 • Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
In the context of deep learning with kernel machines, the deep Restricted Kernel Machine (DRKM) framework allows multiple levels of kernel PCA (KPCA) and Least-Squares Support Vector Machines (LSSVM) to be combined into a deep architecture using visible and hidden units.
no code implementations • 19 Jul 2023 • Henri De Plaen, Johan A. K. Suykens
In this paper, we characterize Probabilistic Principal Component Analysis in Hilbert spaces and demonstrate how the optimal solution admits a representation in dual space.
1 code implementation • 30 Aug 2023 • Jiani Liu, Qinghua Tao, Ce Zhu, Yipeng Liu, Xiaolin Huang, Johan A. K. Suykens
In contrast to previous MTL frameworks, our decision function in the dual induces a weighted kernel function with a task-coupling term characterized by the similarities of the task-specific factors, better revealing the explicit relations across tasks in MTL.
1 code implementation • 8 Oct 2023 • Fan He, Mingzhen He, Lei Shi, Xiaolin Huang, Johan A. K. Suykens
To enhance kernel flexibility, this paper introduces the concept of Locally-Adaptive-Bandwidths (LAB) as trainable parameters to enhance the Radial Basis Function (RBF) kernel, giving rise to the LAB RBF kernel.
no code implementations • 5 Jan 2024 • Zhongjie Shi, Jun Fan, Linhao Song, Ding-Xuan Zhou, Johan A. K. Suykens
With the rapid development of deep learning in various fields of science and technology, such as speech recognition, image classification, and natural language processing, recently it is also widely applied in the functional data analysis (FDA) with some empirical success.
no code implementations • 24 Jan 2024 • Zhongjie Shi, Fanghui Liu, Yuan Cao, Johan A. K. Suykens
Adversarial training is a widely used method to improve the robustness of deep neural networks (DNNs) over adversarial perturbations.
no code implementations • 2 Feb 2024 • Yingyi Chen, Qinghua Tao, Francesco Tonin, Johan A. K. Suykens
In this work, we propose Kernel-Eigen Pair Sparse Variational Gaussian Processes (KEP-SVGP) for building uncertainty-aware self-attention where the asymmetry of attention kernels is tackled by Kernel SVD (KSVD) and a reduced complexity is acquired.
no code implementations • 13 Feb 2024 • Qinghua Tao, Xiangming Xi, Jun Xu, Johan A. K. Suykens
For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with its convex counterparts.
1 code implementation • 20 Nov 2017 • Michaël Fanuel, Antoine Aspeel, Jean-Charles Delvenne, Johan A. K. Suykens
In machine learning or statistics, it is often desirable to reduce the dimensionality of a sample of data points in a high dimensional space $\mathbb{R}^d$.
2 code implementations • 26 May 2023 • Sonny Achten, Arun Pandey, Hannes De Meulemeester, Bart De Moor, Johan A. K. Suykens
We propose a unifying setting that combines existing restricted kernel machine methods into a single primal-dual multi-view framework for kernel principal component analysis in both supervised and unsupervised settings.
1 code implementation • 5 Feb 2020 • Henri De Plaen, Michaël Fanuel, Johan A. K. Suykens
In the context of kernel methods, the similarity between data points is encoded by the kernel function which is often defined thanks to the Euclidean distance, a common example being the squared exponential kernel.
1 code implementation • 5 Aug 2020 • Joachim Schreurs, Iwein Vranckx, Mia Hubert, Johan A. K. Suykens, Peter J. Rousseeuw
The minimum regularized covariance determinant method (MRCD) is a robust estimator for multivariate location and scatter, which detects outliers by fitting a robust covariance matrix to the data.
1 code implementation • 20 Dec 2016 • Zhongming Chen, Kim Batselier, Johan A. K. Suykens, Ngai Wong
In pattern classification, polynomial classifiers are well-studied methods as they are capable of generating complex decision surfaces.
1 code implementation • 24 Oct 2015 • Emanuele Frandi, Ricardo Nanculef, Stefano Lodi, Claudio Sartori, Johan A. K. Suykens
Frank-Wolfe (FW) algorithms have been often proposed over the last few years as efficient solvers for a variety of optimization problems arising in the field of Machine Learning.
1 code implementation • 22 Feb 2023 • Francesco Tonin, Qinghua Tao, Panagiotis Patrinos, Johan A. K. Suykens
Principal Component Analysis (PCA) and its nonlinear extension Kernel PCA (KPCA) are widely used across science and industry for data analysis and dimensionality reduction.
1 code implementation • 9 Jun 2023 • Francesco Tonin, Alex Lambert, Panagiotis Patrinos, Johan A. K. Suykens
The goal of this paper is to revisit Kernel Principal Component Analysis (KPCA) through dualization of a difference of convex functions.
1 code implementation • 23 Jul 2022 • Qinghua Tao, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
In our method, the dual variables, playing the role of hidden features, are shared by all views to construct a common latent space, coupling the views by learning projections from view-specific spaces.
1 code implementation • 16 Feb 2021 • Francesco Tonin, Arun Pandey, Panagiotis Patrinos, Johan A. K. Suykens
Detecting out-of-distribution (OOD) samples is an essential requirement for the deployment of machine learning systems in the real world.
Out-of-Distribution Detection Out of Distribution (OOD) Detection
1 code implementation • 4 Mar 2014 • Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor
We present an approximation scheme for support vector machine models that use an RBF kernel.
1 code implementation • CVPR 2023 • Henri De Plaen, Pierre-François De Plaen, Johan A. K. Suykens, Marc Proesmans, Tinne Tuytelaars, Luc van Gool
The approach is well suited for GPU implementation, which proves to be an advantage for large-scale models.
1 code implementation • NeurIPS 2023 • Yingyi Chen, Qinghua Tao, Francesco Tonin, Johan A. K. Suykens
To the best of our knowledge, this is the first work that provides a primal-dual representation for the asymmetric kernel in self-attention and successfully applies it to modeling and optimization.
Ranked #2 on Offline RL on D4RL
1 code implementation • 13 Feb 2014 • Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor
The included benchmark comprises three settings with increasing label noise: (i) fully supervised, (ii) PU learning and (iii) PU learning with false positives.
2 code implementations • NeurIPS 2020 • Alexander Meulemans, Francesco S. Carzaniga, Johan A. K. Suykens, João Sacramento, Benjamin F. Grewe
Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization.
1 code implementation • 25 Jul 2022 • Yingyi Chen, Xi Shen, Yahui Liu, Qinghua Tao, Johan A. K. Suykens
In this paper, we explore solving jigsaw puzzle as a self-supervised auxiliary loss in ViT for image classification, named Jigsaw-ViT.
Ranked #1 on Learning with noisy labels on ANIMAL
1 code implementation • 28 Apr 2021 • Yingyi Chen, Xi Shen, Shell Xu Hu, Johan A. K. Suykens
On Clothing1M, our approach obtains 74. 9% accuracy which is slightly better than that of DivideMix.
Ranked #12 on Image Classification on Clothing1M (using extra training data)
1 code implementation • 27 Jun 2022 • Yingyi Chen, Shell Xu Hu, Xi Shen, Chunrong Ai, Johan A. K. Suykens
This decomposition provides three insights: (i) it shows that over-fitting is indeed an issue for learning with noisy labels; (ii) through an information bottleneck formulation, it explains why the proposed feature compression helps in combating label noise; (iii) it gives explanations on the performance boost brought by incorporating compression regularization into Co-teaching.
Ranked #10 on Image Classification on Clothing1M (using extra training data)