Search Results for author:
Found 122 papers, 15 papers with code
Deep Forest
This study opens the door of deep learning based on non-differentiable modules, and exhibits the possibility of constructing deep models without using backpropagation.
Isolation Kernel and Its Effect on SVM
This paper investigates data dependent kernels that are derived directly from data.
Multi-Layered Gradient Boosting Decision Trees
Multi-layered representation is believed to be the key ingredient of deep neural networks especially in cognitive tasks like computer vision.
Tunneling Neural Perception and Logic Reasoning through Abductive Learning
Perception and reasoning are basic human abilities that are seamlessly connected as part of human intelligence.
Bridging Machine Learning and Logical Reasoning by Abductive Learning
In the area of artificial intelligence (AI), the two abilities are usually realised by machine learning and logic programming, respectively.
AutoEncoder by Forest
Auto-encoding is an important task which is typically realized by deep neural networks (DNNs) such as convolutional neural networks (CNN).
Semi-Supervised Regression with Co-Training
In many practical machine learning and data min-ing applications, unlabeled training examples arereadily available but labeled ones are fairly expen-sive to obtain.
Isolation Distributional Kernel: A New Tool for Point & Group Anomaly Detection
Existing approaches based on kernel mean embedding, which convert a point kernel to a distributional kernel, have two key issues: the point kernel employed has a feature map with intractable dimensionality; and it is {\em data independent}.
Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization
Specifically, we introduce novel online algorithms that can exploit smoothness and replace the dependence on $T$ in dynamic regret with problem-dependent quantities: the variation in gradients of loss functions, the cumulative loss of the comparator sequence, and the minimum of these two terms.
Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder
In this work, we consider one challenging training time attack by modifying training data with bounded perturbation, hoping to manipulate the behavior (both targeted or non-targeted) of any corresponding trained classifier during test time when facing clean samples.
Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval
Moreover, on general image retrieval datasets, SCDA achieves comparable retrieval results with state-of-the-art general image retrieval approaches.
Efficient Training for Positive Unlabeled Learning
Positive unlabeled (PU) learning is useful in various practical situations, where there is a need to learn a classifier for a class of interest from an unlabeled data set, which may contain anomalies as well as samples from unknown classes.
Evaluating the Robustness of Nearest Neighbor Classifiers: A Primal-Dual Perspective
Furthermore, we show that dual solutions for these QP problems could give us a valid lower bound of the adversarial perturbation that can be used for formal robustness verification, giving us a nice view of attack/verification for NN models.
$\ell_1$-regression with Heavy-tailed Distributions
In this paper, we consider the problem of linear regression with heavy-tailed distributions.
Dynamic Regret of Strongly Adaptive Methods
To cope with changing environments, recent developments in online learning have introduced the concepts of adaptive regret and dynamic regret independently.
Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraud
This model can block fraud transactions in a large amount of money each day.
Matrix Co-completion for Multi-label Classification with Missing Features and Labels
We consider a challenging multi-label classification problem where both feature matrix $\X$ and label matrix $\Y$ have missing entries.
Unorganized Malicious Attacks Detection
This work considers a different attack style: unorganized malicious attacks, where attackers individually utilize a small number of user profiles to attack different items without any organizer.
On the Resistance of Nearest Neighbor to Random Noisy Labels
The theoretical results show that, for asymmetric noises, k-nearest neighbor is robust enough to classify most data correctly, except for a handful of examples, whose labels are totally misled by random noises.
Learning with Feature Evolvable Streams
To benefit from the recovered features, we develop two ensemble methods.
Maximizing Submodular or Monotone Approximately Submodular Functions by Multi-objective Evolutionary Algorithms
To provide a general theoretical explanation of the behavior of EAs, it is desirable to study their performance on general classes of combinatorial optimization problems.
Improved Dynamic Regret for Non-degenerate Functions
When multiple gradients are accessible to the learner, we first demonstrate that the dynamic regret of strongly convex functions can be upper bounded by the minimum of the path-length and the squared path-length.
A Unified View of Multi-Label Performance Measures
Multi-label classification deals with the problem where each instance is associated with multiple class labels.
Theoretical Foundation of Co-Training and Disagreement-Based Algorithms
Disagreement-based approaches generate multiple classifiers and exploit the disagreement among them with unlabeled data to improve learning performance.
Crowdsourcing with Unsure Option
One of the fundamental problems in crowdsourcing is the trade-off between the number of the workers needed for high-accuracy aggregation and the budget to pay.
Unsupervised Object Discovery and Co-Localization by Deep Descriptor Transforming
Reusable model design becomes desirable with the rapid expansion of computer vision and machine learning applications.
Ranked #11 on
Single-object discovery
on COCO_20k
Distribution-Free One-Pass Learning
Moreover, as the whole data volume is unknown when constructing the model, it is desired to scan each data item only once with a storage independent with the data volume.
Deep Descriptor Transforming for Image Co-Localization
Reusable model design becomes desirable with the rapid expansion of machine learning applications.
Effects of the optimisation of the margin distribution on generalisation in deep architectures
Despite being so vital to success of Support Vector Machines, the principle of separating margin maximisation is not used in deep learning.
Multi-Label Learning with Global and Local Label Correlation
In fact, in the real-world applications, both cases may occur that some label correlations are globally applicable and some are shared only in a local group of instances.
Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models
In order to bridge the gap between panel attributes and the composition within each panel, we also propose a recursive page splitting algorithm to generate the panel layout for a poster.
Sparse Learning for Large-scale and High-dimensional Data: A Randomized Convex-concave Optimization Approach
In this paper, we develop a randomized algorithm and theory for learning a sparse model from large-scale and high-dimensional data, which is usually formulated as an empirical risk minimization problem with a sparsity-inducing regularizer.
A Lower Bound Analysis of Population-based Evolutionary Algorithms for Pseudo-Boolean Functions
Our results imply that the increase of population size, while usually desired in practice, bears the risk of increasing the lower bound of the running time and thus should be carefully considered.
Classification under Streaming Emerging New Classes: A Solution using Completely Random Trees
This is the first time, as far as we know, that completely random trees are used as a single common core to solve all three sub problems: unsupervised learning, supervised learning and model update in data streams.
One-Pass Learning with Incremental and Decremental Features
In many real tasks the features are evolving, with some features being vanished and some other features augmented.
Optimal Margin Distribution Machine
Support vector machine (SVM) has been one of the most popular learning algorithms, with the central idea of maximizing the minimum margin, i. e., the smallest distance from the instances to the classification boundary.
Learning to Generate Posters of Scientific Papers
Then, given inferred layout and attributes, composition of graphical elements within each panel is synthesized.
Minimal Gated Unit for Recurrent Neural Networks
Recently recurrent neural networks (RNN) has been very successful in handling sequence data.
Stochastic Proximal Gradient Descent for Nuclear Norm Regularization
In this paper, we utilize stochastic optimization to reduce the space complexity of convex composite optimization with a nuclear norm regularizer, where the variable is a matrix of size $m \times n$.
Transductive Optimization of Top k Precision
This paper introduces a new approach, Transductive Top K (TTK), that seeks to minimize the hinge loss over all training instances under the constraint that exactly $k$ test instances are predicted as positive.
Online Stochastic Linear Optimization under One-bit Feedback
In this paper, we study a special bandit setting of online stochastic linear optimization, where only one-bit of information is revealed to the learner at each round.
Multi-Label Active Learning from Crowds
Multi-label active learning is a hot topic in reducing the label cost by optimally choosing the most valuable instance to query its label from an oracle.
Scalable Stochastic Alternating Direction Method of Multipliers
There exists only one stochastic method, called SA-ADMM, which can achieve convergence rate $O(1/T)$ on general convex problems.
Analysis of Nuclear Norm Regularization for Full-rank Matrix Completion
To the best of our knowledge, this is first time such a relative bound is proved for the regularized formulation of matrix completion.
CUR Algorithm for Partially Observed Matrices
In particular, the proposed algorithm computes the low rank approximation of the target matrix based on (i) the randomly sampled rows and columns, and (ii) a subset of observed entries that are randomly sampled from the matrix.
Top Rank Optimization in Linear Time
Recent efforts of bipartite ranking are focused on optimizing ranking accuracy at the top of the ranked list.
On the Consistency of AUC Pairwise Optimization
Based on this result, we prove that exponential loss and logistic loss are consistent with AUC, but hinge loss is inconsistent.
Dropout Rademacher Complexity of Deep Neural Networks
Great successes of deep neural networks have been witnessed in various real applications.
Large Margin Distribution Machine
In this paper, we propose the Large margin Distribution Machine (LDM), which tries to achieve a better generalization performance by optimizing the margin distribution.
Inductive Logic Boosting
Structure learning of these systems is an intersection area of Inductive Logic Programming (ILP) and statistical learning (SL).
Analyzing Evolutionary Optimization in Noisy Environments
On a representative problem where the noise has a strong negative effect, we examine two commonly employed mechanisms in EAs dealing with noise, the re-evaluation and the threshold selection strategies.
Fast Multi-Instance Multi-Label Learning
Although the MIML problem is complicated, MIMLfast is able to achieve excellent performance by exploiting label relations with shared space and discovering sub-concepts for complicated labels.
On the Doubt about Margin Explanation of Boosting
Margin theory provides one of the most popular explanations to the success of \texttt{AdaBoost}, where the central point lies in the recognition that \textit{margin} is the key for characterizing the performance of \texttt{AdaBoost}.
Convex and Scalable Weakly Labeled SVMs
In this paper, we study the problem of learning from weakly labeled data, where labels of the training examples are incomplete.
One-Pass AUC Optimization
AUC is an important performance measure and many algorithms have been devoted to AUC optimization, mostly by minimizing a surrogate convex loss on a training data set.
Handling Concept Drift via Model Reuse
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature.
Adaptive Online Learning in Dynamic Environments
In this paper, we study online convex optimization in dynamic environments, and aim to bound the dynamic regret with respect to any sequence of comparators.
Learning with Interpretable Structure from Gated RNN
With the learned FSA and via experiments on artificial and real datasets, we find that FSA is more trustable than the RNN from which it learned, which gives FSA a chance to substitute RNNs in applications involving humans' lives or dangerous facilities.
\ell_1-regression with Heavy-tailed Distributions
In this paper, we consider the problem of linear regression with heavy-tailed distributions.
Preference Based Adaptation for Learning Objectives
In many real-world learning tasks, it is hard to directly optimize the true performance measures, meanwhile choosing the right surrogate objectives is also difficult.
Subset Selection under Noise
The problem of selecting the best $k$-element subset from a universe is involved in many applications.
What Makes Objects Similar: A Unified Multi-Metric Learning Approach
In UM2L, a type of combination operator is introduced for distance characterization from multiple perspectives, and thus can introduce flexibilities for representing and utilizing both spatial and semantic linkages.
Subset Selection by Pareto Optimization
Selecting the optimal subset from a large set of variables is a fundamental problem in various learning tasks such as feature selection, sparse regression, dictionary learning, etc.
Speedup Matrix Completion with Side Information: Application to Multi-Label Learning
In standard matrix completion theory, it is required to have at least $O(n\ln^2 n)$ observed entries to perfectly recover a low-rank matrix $M$ of size $n\times n$, leading to a large number of observations when $n$ is large.
Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison
Both random Fourier features and the Nyström method have been successfully applied to efficient kernel learning.
Active Learning by Querying Informative and Representative Examples
Most active learning approaches select either informative or representative unlabeled instances to query their labels.
Multi-View Active Learning in the Non-Realizable Case
The sample complexity of active learning under the realizability assumption has been well-studied.
Multi-Class Optimal Margin Distribution Machine
It still remains open for multi-class classification, and due to the complexity of margin for multi-class classification, optimizing its distribution by mean and variance can also be difficult.
Rectify Heterogeneous Models with Semantic Mapping
On the way to the robust learner for real-world applications, there are still great challenges, including considering unknown environments with limited data.
Optimal margin Distribution Network
Recent research about margin theory has proved that maximizing the minimum margin like support vector machines does not necessarily lead to better performance, and instead, it is crucial to optimize the margin distribution.
Improving Generalization of Deep Neural Networks by Leveraging Margin Distribution
We utilize a convex margin distribution loss function on the deep neural networks to validate our theoretical results by optimizing the margin ratio.
Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the $O(1/T)$ Convergence Rate
Finally, we emphasize that our proof is constructive and each risk bound is equipped with an efficient stochastic algorithm attaining that bound.
Improving deep forest by confidence screening
Most studies about deep learning are based on neural network models, where many layers of parameterized nonlinear differentiable modules are trained by back propagation.
Reliable Weakly Supervised Learning: Maximize Gain and Maintain Safeness
We guide the optimization of label quality through a small amount of validation data, and to ensure the safeness of performance while maximizing performance gain.
Adaptive Regret of Convex and Smooth Functions
We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure.
Prediction with Unpredictable Feature Evolution
Learning with feature evolution studies the scenario where the features of the data streams can evolve, i. e., old features vanish and new features emerge.
Forest Representation Learning Guided by Margin Distribution
In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction.
Reinforcement Learning Experience Reuse with Policy Residual Representation
Experience reuse is key to sample-efficient reinforcement learning.
Joint Semantic Domain Alignment and Target Classifier Learning for Unsupervised Domain Adaptation
Unsupervised domain adaptation aims to transfer the classifier learned from the source domain to the target domain in an unsupervised manner.
Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive Regret of Convex Functions
Along this line of research, this paper presents the first universal algorithm for minimizing the adaptive regret of convex functions.
Bandit Convex Optimization in Non-stationary Environments
In this paper, we investigate BCO in non-stationary environments and choose the \emph{dynamic regret} as the performance measure, which is defined as the difference between the cumulative loss incurred by the algorithm and that of any feasible comparator sequence.
Imitation Learning from Pixel-Level Demonstrations by HashReward
One of the key issues for imitation learning lies in making policy learned from limited samples to generalize well in the whole state-action space.
Bifurcation Spiking Neural Network
Inspired by this insight, by enabling the spike generation function to have adaptable eigenvalues rather than parametric control rates, we develop the Bifurcation Spiking Neural Network (BSNN), which has an adaptive firing rate and is insensitive to the setting of control rates.
An Unbiased Risk Estimator for Learning with Augmented Classes
This paper studies the problem of learning with augmented classes (LAC), where augmented classes unobserved in the training data might emerge in the testing phase.
A Refined Margin Distribution Analysis for Forest Representation Learning
In this paper, we formulate the forest representation learning approach called \textsc{CasDF} as an additive model which boosts the augmented feature instead of the prediction.
Multi-Label Learning with Deep Forest
In multi-label learning, each instance is associated with multiple labels and the crucial task is how to leverage label correlations in building models.
Model Reuse with Reduced Kernel Mean Embedding Specification
Given a publicly available pool of machine learning models constructed for various tasks, when a user plans to build a model for her own machine learning application, is it possible to build upon models in the pool such that the previous efforts on these existing models can be reused rather than starting from scratch?
Exploratory Machine Learning with Unknown Unknowns
In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to the known labels.
AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going Online
The framework consists of an evaluator that generalizes to evaluate recommendations involving the context, and a generator that maximizes the evaluator score by reinforcement learning, and a discriminator that ensures the generalization of the evaluator.
Flexible Transmitter Network
To exhibit its power and potential, we present the Flexible Transmitter Network (FTNet), which is built on the most common fully-connected feed-forward architecture taking the FT model as the basic building block.
Soft Gradient Boosting Machine
Gradient Boosting Machine has proven to be one successful function approximator and has been widely used in a variety of areas.
Dynamic Regret of Convex and Smooth Functions
We investigate online convex optimization in non-stationary environments and choose the dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence.
Storage Fit Learning with Feature Evolvable Streams
Our framework is able to fit its behavior to different storage budgets when learning with feature evolvable streams with unlabeled data.
Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data
Deep semi-supervised learning (SSL) has been shown very effectively.
Learning with Feature and Distribution Evolvable Streams
Besides the feature space evolving, it is noteworthy that the data distribution often changes in streaming data.
Cost-effectively Identifying Causal Effect When Only Response Variable Observable
In many real tasks, we care about how to make decisions other than mere predictions on an event, e. g. how to increase the revenue next month instead of knowing it will drop.
Towards Convergence Rate Analysis of Random Forests for Classification
We get a convergence rate O(n^{-{1}/(d+2)}(\ln n)^{{1}/(d+2)}) for the variant of random forests, which reaches the minimax rate, except for a factor (\ln n)^{{1}/(d+2)}, of the optimal plug-in classifier under the L-Lipschitz assumption.
Non-stationary Online Learning with Memory and Non-stochastic Control
We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions and thus captures temporal effects of learning problems.
Isolation forest
Most existing model-based approaches to anomaly detection construct a profile of normal instances, then identify instances that do not conform to the normal profile as anomalies.
Anomaly Detection
Unsupervised Anomaly Detection with Specified Settings -- 0.1% anomaly
+4
Towards an Understanding of Benign Overfitting in Neural Networks
Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss; yet surprisingly, they possess near-optimal prediction performance, contradicting classical learning theory.
Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning
In many real-world imitation learning tasks, the demonstrator and the learner have to act under different observation spaces.
Towards Understanding Theoretical Advantages of Complex-Reaction Networks
Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks.
LIFE: Learning Individual Features for Multivariate Time Series Prediction with Missing Values
Multivariate time series (MTS) prediction is ubiquitous in real-world fields, but MTS data often contains missing values.
Result Diversification by Multi-objective Evolutionary Algorithms with Theoretical Guarantees
Experiments on the applications of web-based search, multi-label feature selection and document summarization show the superior performance of the GSEMO over the state-of-the-art algorithms (i. e., the greedy algorithm and local search) under both static and dynamic environments.
Theoretical Exploration of Flexible Transmitter Model
Neural network models generally involve two important components, i. e., network architecture and neuron model.
ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions
Mimicking and learning the long-term memory of efficient markets is a fundamental problem in the interaction between machine learning and financial economics to sequential data.
Actively Identifying Causal Effects with Latent Variables Given Only Response Variable Observable
In many real tasks, it is generally desired to study the causal effect on a specific target (response variable) only, with no need to identify the thorough causal effects involving all variables.
No-Regret Learning in Time-Varying Zero-Sum Games
Learning from repeated play in a fixed two-player zero-sum game is a classic problem in game theory and online learning.
Corralling a Larger Band of Bandits: A Case Study on Switching Regret for Linear Bandits
The CORRAL algorithm of Agarwal et al. (2017) and its variants (Foster et al., 2020a) achieve this goal with a regret overhead of order $\widetilde{O}(\sqrt{MT})$ where $M$ is the number of base algorithms and $T$ is the time horizon.
Open-environment Machine Learning
With the great success of machine learning, nowadays, more and more practical tasks, particularly those involving open-environment scenarios where important factors are subject to change, called open-environment machine learning (Open ML) in this article, are present to the community.
Adapting to Online Label Shift with Provable Guarantees
In this paper, we formulate and investigate the problem of \emph{online label shift} (OLaS): the learner trains an initial model from the labeled offline data and then deploys it to an unlabeled online environment where the underlying label distribution changes over time but the label-conditional density does not.
On the Intrinsic Structures of Spiking Neural Networks
Initially, we unveil two pivotal components of intrinsic structures: the integration operation and firing-reset mechanism, by elucidating their influence on the expressivity of SNNs.
Dynamic Regret of Online Markov Decision Processes
For these three models, we propose novel online ensemble algorithms and establish their dynamic regret guarantees respectively, in which the results for episodic (loop-free) SSP are provably minimax optimal in terms of time horizon and certain non-stationarity measure.
Learnware: Small Models Do Big
There are complaints about current machine learning techniques such as the requirement of a huge amount of training data and proficient training skills, the difficulty of continual learning, the risk of catastrophic forgetting, the leaking of data privacy/proprietary, etc.
Stochastic Approximation Approaches to Group Distributionally Robust Optimization
First, we formulate GDRO as a stochastic convex-concave saddle-point problem, and demonstrate that stochastic mirror descent (SMD), using $m$ samples in each iteration, achieves an $O(m (\log m)/\epsilon^2)$ sample complexity for finding an $\epsilon$-optimal solution, which matches the $\Omega(m/\epsilon^2)$ lower bound up to a logarithmic factor.
Revisiting Weighted Strategy for Non-stationary Parametric Bandits
We propose a refined analysis framework, which simplifies the derivation and importantly produces a simpler weight-based algorithm that is as efficient as window/restart-based algorithms while retaining the same regret as previous studies.
A Theoretical Perspective of Machine Learning with Computational Resource Concerns
Conventional theoretical machine learning studies generally assume explicitly or implicitly that there are enough or even infinitely supplied computational resources.
Weakly Supervised AUC Optimization: A Unified Partial AUC Approach
Since acquiring perfect supervision is usually difficult, real-world machine learning tasks often confront inaccurate, incomplete, or inexact supervision, collectively referred to as weak supervision.
Universal Online Learning with Gradient Variations: A Multi-layer Online Ensemble Approach
Our approach is based on a multi-layer online ensemble framework incorporating novel ingredients, including a carefully designed optimism for unifying diverse function types and cascaded corrections for algorithmic stability.
Efficient Methods for Non-stationary Online Learning
In this paper, we present efficient methods for optimizing dynamic regret and adaptive regret, which reduce the number of projections per round from $\mathcal{O}(\log T)$ to $1$.
Beimingwu: A Learnware Dock System
The learnware paradigm proposed by Zhou [2016] aims to enable users to reuse numerous existing well-trained models instead of building machine learning models from scratch, with the hope of solving new user tasks even beyond models' original purposes.
Improved Algorithm for Adversarial Linear Mixture MDPs with Bandit Feedback and Unknown Transition
We study reinforcement learning with linear function approximation, unknown transition, and adversarial losses in the bandit feedback setting.
