no code implementations • 13 Feb 2024 • Qinghua Tao, Xiangming Xi, Jun Xu, Johan A. K. Suykens

For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with its convex counterparts.

no code implementations • 2 Feb 2024 • Yingyi Chen, Qinghua Tao, Francesco Tonin, Johan A. K. Suykens

In this work, we propose Kernel-Eigen Pair Sparse Variational Gaussian Processes (KEP-SVGP) for building uncertainty-aware self-attention where the asymmetry of attention kernels is tackled by Kernel SVD (KSVD) and a reduced complexity is acquired.

no code implementations • 24 Jan 2024 • Zhongjie Shi, Fanghui Liu, Yuan Cao, Johan A. K. Suykens

Adversarial training is a widely used method to improve the robustness of deep neural networks (DNNs) over adversarial perturbations.

no code implementations • 5 Jan 2024 • Zhongjie Shi, Jun Fan, Linhao Song, Ding-Xuan Zhou, Johan A. K. Suykens

With the rapid development of deep learning in various fields of science and technology, such as speech recognition, image classification, and natural language processing, recently it is also widely applied in the functional data analysis (FDA) with some empirical success.

1 code implementation • 8 Oct 2023 • Fan He, Mingzhen He, Lei Shi, Xiaolin Huang, Johan A. K. Suykens

To enhance kernel flexibility, this paper introduces the concept of Locally-Adaptive-Bandwidths (LAB) as trainable parameters to enhance the Radial Basis Function (RBF) kernel, giving rise to the LAB RBF kernel.

1 code implementation • 30 Aug 2023 • Jiani Liu, Qinghua Tao, Ce Zhu, Yipeng Liu, Xiaolin Huang, Johan A. K. Suykens

In contrast to previous MTL frameworks, our decision function in the dual induces a weighted kernel function with a task-coupling term characterized by the similarities of the task-specific factors, better revealing the explicit relations across tasks in MTL.

no code implementations • 19 Jul 2023 • Henri De Plaen, Johan A. K. Suykens

In this paper, we characterize Probabilistic Principal Component Analysis in Hilbert spaces and demonstrate how the optimal solution admits a representation in dual space.

1 code implementation • CVPR 2023 • Henri De Plaen, Pierre-François De Plaen, Johan A. K. Suykens, Marc Proesmans, Tinne Tuytelaars, Luc van Gool

The approach is well suited for GPU implementation, which proves to be an advantage for large-scale models.

no code implementations • 12 Jun 2023 • Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

In the context of deep learning with kernel machines, the deep Restricted Kernel Machine (DRKM) framework allows multiple levels of kernel PCA (KPCA) and Least-Squares Support Vector Machines (LSSVM) to be combined into a deep architecture using visible and hidden units.

no code implementations • 12 Jun 2023 • Qinghua Tao, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

We describe a nonlinear extension of the matrix Singular Value Decomposition through asymmetric kernels, namely KSVD.

1 code implementation • 9 Jun 2023 • Francesco Tonin, Alex Lambert, Panagiotis Patrinos, Johan A. K. Suykens

The goal of this paper is to revisit Kernel Principal Component Analysis (KPCA) through dualization of a difference of convex functions.

1 code implementation • NeurIPS 2023 • Yingyi Chen, Qinghua Tao, Francesco Tonin, Johan A. K. Suykens

To the best of our knowledge, this is the first work that provides a primal-dual representation for the asymmetric kernel in self-attention and successfully applies it to modeling and optimization.

Ranked #2 on Offline RL on D4RL

2 code implementations • 26 May 2023 • Sonny Achten, Arun Pandey, Hannes De Meulemeester, Bart De Moor, Johan A. K. Suykens

We propose a unifying setting that combines existing restricted kernel machine methods into a single primal-dual multi-view framework for kernel principal component analysis in both supervised and unsupervised settings.

no code implementations • 4 Mar 2023 • Jiani Liu, Qinghua Tao, Ce Zhu, Yipeng Liu, Johan A. K. Suykens

Multitask learning (MTL) can utilize the relatedness between multiple tasks for performance improvement.

1 code implementation • 22 Feb 2023 • Francesco Tonin, Qinghua Tao, Panagiotis Patrinos, Johan A. K. Suykens

Principal Component Analysis (PCA) and its nonlinear extension Kernel PCA (KPCA) are widely used across science and industry for data analysis and dimensionality reduction.

1 code implementation • 31 Jan 2023 • Sonny Achten, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs.

no code implementations • 24 Jan 2023 • Arun Pandey, Hannes De Meulemeester, Bart De Moor, Johan A. K. Suykens

In this paper, we propose a kernel principal component analysis model for multi-variate time series forecasting, where the training and prediction schemes are derived from the multi-view formulation of Restricted Kernel Machines.

1 code implementation • 25 Jul 2022 • Yingyi Chen, Xi Shen, Yahui Liu, Qinghua Tao, Johan A. K. Suykens

In this paper, we explore solving jigsaw puzzle as a self-supervised auxiliary loss in ViT for image classification, named Jigsaw-ViT.

Ranked #1 on Learning with noisy labels on ANIMAL

1 code implementation • 23 Jul 2022 • Qinghua Tao, Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

In our method, the dual variables, playing the role of hidden features, are shared by all views to construct a common latent space, coupling the views by learning projections from view-specific spaces.

1 code implementation • 27 Jun 2022 • Yingyi Chen, Shell Xu Hu, Xi Shen, Chunrong Ai, Johan A. K. Suykens

This decomposition provides three insights: (i) it shows that over-fitting is indeed an issue for learning with noisy labels; (ii) through an information bottleneck formulation, it explains why the proposed feature compression helps in combating label noise; (iii) it gives explanations on the performance boost brought by incorporating compression regularization into Co-teaching.

Ranked #10 on Image Classification on Clothing1M (using extra training data)

no code implementations • 18 Jun 2022 • Qinghua Tao, Li Li, Xiaolin Huang, Xiangming Xi, Shuning Wang, Johan A. K. Suykens

To apply PWLNN methods, both the representation and the learning have long been studied.

no code implementations • 3 Feb 2022 • Mingzhen He, Fan He, Lei Shi, Xiaolin Huang, Johan A. K. Suykens

Asymmetric kernels naturally exist in real life, e. g., for conditional probability and directed graphs.

no code implementations • 26 Oct 2021 • Maximilian Lucassen, Johan A. K. Suykens, Kim Batselier

Least squares support vector machines are a commonly used supervised learning method for nonlinear regression and classification.

no code implementations • 13 Oct 2021 • Fanghui Liu, Johan A. K. Suykens, Volkan Cevher

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.

no code implementations • 28 May 2021 • David Winant, Joachim Schreurs, Johan A. K. Suykens

This connection has led to insights on how to use kernel PCA in a generative procedure, called generative kernel PCA.

no code implementations • 28 May 2021 • Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens

Determinantal point processes (DPPs) are well known models for diverse subset selection problems, including recommendation tasks, document summarization and image search.

1 code implementation • 28 Apr 2021 • Yingyi Chen, Xi Shen, Shell Xu Hu, Johan A. K. Suykens

On Clothing1M, our approach obtains 74. 9% accuracy which is slightly better than that of DivideMix.

Ranked #12 on Image Classification on Clothing1M (using extra training data)

1 code implementation • 6 Apr 2021 • Joachim Schreurs, Hannes De Meulemeester, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens

A generative model may overlook underrepresented modes that are less frequent in the empirical data distribution.

1 code implementation • 16 Feb 2021 • Francesco Tonin, Arun Pandey, Panagiotis Patrinos, Johan A. K. Suykens

Detecting out-of-distribution (OOD) samples is an essential requirement for the deployment of machine learning systems in the real world.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

no code implementations • 25 Nov 2020 • Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens

We introduce Constr-DRKM, a deep kernel method for the unsupervised learning of disentangled data representations.

no code implementations • 13 Nov 2020 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy.

no code implementations • 3 Nov 2020 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens

In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical integration representation.

no code implementations • 6 Oct 2020 • Fanghui Liu, Zhenyu Liao, Johan A. K. Suykens

In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depending on whether the number of training data n exceeds the feature dimension d. By establishing a bias-variance decomposition of the expected excess risk, we show that, while the bias is (almost) independent of d and monotonically decreases with n, the variance depends on n, d and can be unimodal or monotonically decreasing under different regularization schemes.

1 code implementation • 5 Aug 2020 • Joachim Schreurs, Iwein Vranckx, Mia Hubert, Johan A. K. Suykens, Peter J. Rousseeuw

The minimum regularized covariance determinant method (MRCD) is a robust estimator for multivariate location and scatter, which detects outliers by fitting a robust covariance matrix to the data.

2 code implementations • NeurIPS 2020 • Alexander Meulemans, Francesco S. Carzaniga, Johan A. K. Suykens, João Sacramento, Benjamin F. Grewe

Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization.

no code implementations • 24 Jun 2020 • Joachim Schreurs, Michaël Fanuel, Johan A. K. Suykens

By using the framework of Determinantal Point Processes (DPPs), some theoretical results concerning the interplay between diversity and regularization can be obtained.

no code implementations • 16 Jun 2020 • Hannes De Meulemeester, Joachim Schreurs, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens

However, under certain circumstances, the training of GANs can lead to mode collapse or mode dropping, i. e. the generative models not being able to sample from the entire probability distribution.

no code implementations • 12 Jun 2020 • Arun Pandey, Michael Fanuel, Joachim Schreurs, Johan A. K. Suykens

Our analysis shows that such a construction promotes disentanglement by matching the principal directions in the latent space with the directions of orthogonal variation in data space.

no code implementations • 1 Jun 2020 • Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens

In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS).

no code implementations • 30 May 2020 • Fanghui Liu, Xiaolin Huang, Yingyi Chen, Johan A. K. Suykens

In this paper, we attempt to solve a long-lasting open question for non-positive definite (non-PD) kernels in machine learning community: can a given non-PD kernel be decomposed into the difference of two PD kernels (termed as positive decomposition)?

no code implementations • 23 Apr 2020 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Johan A. K. Suykens

This survey may serve as a gentle introduction to this topic, and as a users' guide for practitioners interested in applying the representative algorithms and understanding theoretical results under various technical assumptions.

no code implementations • 20 Feb 2020 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

The Nystr\"om approximation -- based on a subset of landmarks -- gives a low rank approximation of the kernel matrix, and is known to provide a form of implicit regularization.

1 code implementation • 5 Feb 2020 • Henri De Plaen, Michaël Fanuel, Johan A. K. Suykens

In the context of kernel methods, the similarity between data points is encoded by the kernel function which is often defined thanks to the Euclidean distance, a common example being the squared exponential kernel.

no code implementations • 4 Feb 2020 • Arun Pandey, Joachim Schreurs, Johan A. K. Suykens

Experiments show that the weighted RKM is capable of generating clean images when contamination is present in the training data.

no code implementations • 20 Nov 2019 • Fanghui Liu, Xiaolin Huang, Yudong Chen, Jie Yang, Johan A. K. Suykens

In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation.

no code implementations • 19 Jun 2019 • Arun Pandey, Joachim Schreurs, Johan A. K. Suykens

This paper introduces a novel framework for generative models based on Restricted Kernel Machines (RKMs) with joint multi-view generation and uncorrelated feature learning, called Gen-RKM.

no code implementations • 29 May 2019 • Michaël Fanuel, Joachim Schreurs, Johan A. K. Suykens

In this context, we propose a deterministic and a randomized adaptive algorithm for selecting landmark points within a training data set.

no code implementations • 15 May 2019 • Jun Xu, Qinghua Tao, Zhen Li, Xiangming Xi, Johan A. K. Suykens, Shuning Wang

It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, which was also proposed based on the model of HH and find good applications in system identification.

no code implementations • 9 May 2019 • Hanyuan Hang, Yingyi Chen, Johan A. K. Suykens

We propose a novel method designed for large-scale regression problems, namely the two-stage best-scored random forest (TBRF).

no code implementations • 15 Nov 2018 • Zahra Karevan, Johan A. K. Suykens

Subsequently, the input of the second LSTM layer is formed based on the combination of the hidden states of the first layer LSTM models.

no code implementations • 26 Sep 2018 • Fanghui Liu, Lei Shi, Xiaolin Huang, Jie Yang, Johan A. K. Suykens

This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view.

1 code implementation • 20 Nov 2017 • Michaël Fanuel, Antoine Aspeel, Jean-Charles Delvenne, Johan A. K. Suykens

In machine learning or statistics, it is often desirable to reduce the dimensionality of a sample of data points in a high dimensional space $\mathbb{R}^d$.

no code implementations • 18 Jul 2017 • Saverio Salzo, Johan A. K. Suykens, Lorenzo Rosasco

In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of $\ell^p$ regularized learning methods.

no code implementations • 6 Jul 2017 • Fanghui Liu, Xiaolin Huang, Chen Gong, Jie Yang, Johan A. K. Suykens

Since the concave-convex procedure has to solve a sub-problem in each iteration, we propose a concave-inexact-convex procedure (CCICP) algorithm with an inexact solving scheme to accelerate the solving process.

no code implementations • 19 Jun 2017 • Carlos M. Alaíz, Johan A. K. Suykens

This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Cand\`es et al. and on the equivalence between Lasso and SVM shown recently by Jaggi.

no code implementations • 20 Feb 2017 • Yunlong Feng, Jun Fan, Johan A. K. Suykens

However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view.

no code implementations • 21 Dec 2016 • Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens

A graph-based classification method is proposed for semi-supervised learning in the case of Euclidean data and for classification in the case of graph data.

1 code implementation • 20 Dec 2016 • Zhongming Chen, Kim Batselier, Johan A. K. Suykens, Ngai Wong

In pattern classification, polynomial classifiers are well-studied methods as they are capable of generating complex decision surfaces.

no code implementations • 21 Oct 2016 • Carlos M. Alaíz, Michaël Fanuel, Johan A. K. Suykens

In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem.

no code implementations • 13 Jul 2016 • Hanyuan Hang, Ingo Steinwart, Yunlong Feng, Johan A. K. Suykens

We study the density estimation problem with observations generated by certain dynamical systems that admit a unique underlying invariant Lebesgue density.

no code implementations • 10 May 2016 • Hanyuan Hang, Yunlong Feng, Ingo Steinwart, Johan A. K. Suykens

We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a unified treatment on analyzing the learning schemes with various mixing processes can be conducted and a sharp oracle inequality for generic regularized empirical risk minimization schemes can be established.

no code implementations • 18 Mar 2016 • Saverio Salzo, Johan A. K. Suykens

In this paper we study the variational problem associated to support vector regression in Banach function spaces.

1 code implementation • 24 Oct 2015 • Emanuele Frandi, Ricardo Nanculef, Stefano Lodi, Claudio Sartori, Johan A. K. Suykens

Frank-Wolfe (FW) algorithms have been often proposed over the last few years as efficient solvers for a variety of optimization problems arising in the field of Machine Learning.

no code implementations • 14 May 2015 • Xiaolin Huang, Lei Shi, Ming Yan, Johan A. K. Suykens

The one-sided $\ell_1$ loss and the linear loss are two popular loss functions for 1bit-CS.

no code implementations • 3 May 2015 • Rocco Langone, Raghvendra Mall, Carlos Alzate, Johan A. K. Suykens

This is a major advantage compared to classical spectral clustering where the determination of the clustering parameters is unclear and relies on heuristics.

no code implementations • 7 Mar 2015 • Yuning Yang, Siamak Mehrkanoon, Johan A. K. Suykens

In this paper, we propose higher order matching pursuit for low rank tensor learning problems with a convex or a nonconvex cost function, which is a generalization of the matching pursuit type methods.

no code implementations • 5 Feb 2015 • Emanuele Frandi, Ricardo Nanculef, Johan A. K. Suykens

Frank-Wolfe algorithms have recently regained the attention of the Machine Learning community.

1 code implementation • 4 Mar 2014 • Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor

We present an approximation scheme for support vector machine models that use an RBF kernel.

1 code implementation • 13 Feb 2014 • Marc Claesen, Frank De Smet, Johan A. K. Suykens, Bart De Moor

The included benchmark comprises three settings with increasing label noise: (i) fully supervised, (ii) PU learning and (iii) PU learning with false positives.

no code implementations • 18 Oct 2013 • Marco Signoretto, Lieven De Lathauwer, Johan A. K. Suykens

We present a general framework to learn functions in tensor product reproducing kernel Hilbert spaces (TP-RKHSs).

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.