Search Results for author:
Found 79 papers, 34 papers with code
Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections
Anomaly Detection (AD) involves identifying deviations from normal data distributions and is critical in fields such as medical diagnostics and industrial defect detection.
Many-to-Many Matching via Sparsity Controlled Optimal Transport
The first component is the matching budget constraints on each row and column of a transport plan, which specify how many points can be matched to a point at most.
Efficient Model Editing with Task Vector Bases: A Theoretical Framework and Scalable Approach
Task vectors, which are derived from the difference between pre-trained and fine-tuned model weights, enable flexible task adaptation and model merging through arithmetic operations such as addition and negation.
On Verbalized Confidence Scores for LLMs
The rise of large language models (LLMs) and their tight integration into our daily life make it essential to dedicate efforts towards their trustworthiness.
Learning Structured Representations with Hyperbolic Embeddings
Most real-world datasets consist of a natural hierarchy between classes or an inherent label structure that is either already available or can be constructed cheaply.
Fast unsupervised ground metric learning with tree-Wasserstein distance
The performance of unsupervised methods such as clustering depends on the choice of distance metric between features, or ground metric.
Towards the Effect of Examples on In-Context Learning: A Theoretical Case Study
To better understand how ICL integrates the examples with the knowledge learned by the LLM during pre-training (i. e., pre-training knowledge) and how the examples impact ICL, this paper conducts a theoretical study in binary classification tasks.
Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transport
To address this limitation, under the CPCC framework, we propose to use the Earth Mover's Distance (EMD) to measure the pairwise distances among classes in the feature space.
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents.
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis
In this work, we aim to explore the temporal prediction hypothesis from the perspective of self-supervised learning.
Parameter-free Clipped Gradient Descent Meets Polyak
We numerically validated our convergence results using a synthetic function and demonstrated the effectiveness of our proposed methods using LSTM, Nano-GPT, and T5.
An Empirical Study of Self-supervised Learning with Wasserstein Distance
We find that the model performance depends on the combination of TWD and probability model, and that the Jeffrey divergence regularization helps in model training.
Embarrassingly Simple Text Watermarks
LLMs can generate texts that cannot be distinguished from human-written texts.
Necessary and Sufficient Watermark for Large Language Models
Although existing watermarking methods have successfully detected texts generated by LLMs, they significantly degrade the quality of the generated texts.
Implicit neural representation for change detection
We outperform the previous methods by a margin of 10% in the intersection over union metric.
Nystrom Method for Accurate and Scalable Implicit Differentiation
The essential difficulty of gradient-based bilevel optimization using implicit differentiation is to estimate the inverse Hessian vector product with respect to neural network parameters.
Optimal Transport for Change Detection on LiDAR Point Clouds
The method is based on unbalanced optimal transport and can be generalised to any change detection problem with LiDAR data.
Structural Explanations for Graph Neural Networks using HSIC
More specifically, we extend the GraphLIME method for node explanation with a group lasso and a fused lasso-based node explanation method.
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
In this study, we propose Momentum Tracking, which is a method with momentum whose convergence rate is proven to be independent of data heterogeneity.
Twin Papers: A Simple Framework of Causal Inference for Citations via Coupling
The main difficulty in investigating the effects is that we need to know counterfactual results, which are not available in reality.
Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images
Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation.
Scale dependant layer for self-supervised nuclei encoding
Recent developments in self-supervised learning give us the possibility to further reduce human intervention in multi-step pipelines where the focus evolves around particular objects of interest.
Approximating 1-Wasserstein Distance with Trees
In this paper, we aim to approximate the 1-Wasserstein distance by the tree-Wasserstein distance (TWD), where TWD is a 1-Wasserstein distance with tree-based embedding and can be computed in linear time with respect to the number of nodes on a tree.
Feature Selection for Discovering Distributional Treatment Effect Modifiers
Finding the features relevant to the difference in treatment effects is essential to unveil the underlying causal mechanisms.
Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization
However, the convergence rate of the ECL is provided only when the objective function is convex, and has not been shown in a standard machine learning setting where the objective function is non-convex.
Communication Compression for Decentralized Learning with Operator Splitting Methods
Moreover, we demonstrate that the C-ECL is more robust to heterogeneous data than the Gossip-based algorithms.
Nys-Newton: Nyström-Approximated Curvature for Stochastic Optimization
However, the secant equation becomes insipid in approximating the Newton step owing to its use of the first-order derivatives.
Adversarial Regression with Doubly Non-negative Weighting Matrices
In this paper, we propose a novel and coherent scheme for kernel-reweighted regression by reparametrizing the sample weights using a doubly non-negative matrix.
Fixed Support Tree-Sliced Wasserstein Barycenter
By contrast, the Wasserstein distance on a tree, called the tree-Wasserstein distance, can be computed in linear time and allows for the fast comparison of a large number of distributions.
Re-evaluating Word Mover's Distance
The original study on WMD reported that WMD outperforms classical baselines such as bag-of-words (BOW) and TF-IDF by significant margins in various datasets.
Computationally Efficient Wasserstein Loss for Structured Labels
The problem of estimating the probability distribution of labels has been widely studied as a label distribution learning (LDL) problem, whose applications include age estimation, emotion analysis, and semantic segmentation.
Dynamic Sasvi: Strong Safe Screening for Norm-Regularized Least Squares
A recently introduced technique for a sparse optimization problem called "safe screening" allows us to identify irrelevant variables in the early stage of optimization.
Supervised Tree-Wasserstein Distance
Specifically, we rewrite the Wasserstein distance on the tree metric by the parent-child relationships of a tree and formulate it as a continuous optimization problem using a contrastive loss.
Feature-Robust Optimal Transport for High-Dimensional Data
To show the effectiveness of FROT, we propose using the FROT algorithm for the layer selection problem in deep neural networks for semantic correspondence.
Adaptive Tree Wasserstein Minimization for Hierarchical Generative Modeling
In this work, we consider OT over tree metrics, which is more general than the sliced Wasserstein and includes the sliced Wasserstein as a special case, and we propose a fast minimization algorithm in $O(n)$ for the optimal Wasserstein-1 transport plan between two distributions in the tree structure.
Post-selection inference with HSIC-Lasso
Detecting influential features in non-linear and/or high-dimensional data is a challenging and increasingly important task in machine learning.
Poincare: Recommending Publication Venues via Treatment Effect Estimation
We use a bias correction method to estimate the potential impact of choosing a publication venue effectively and to recommend venues based on the potential impact of papers in each venue.
Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search
Building upon tree-Wasserstein (TW), which is a negative definite variant of OT, we develop a novel discrepancy for neural architectures, and demonstrate it within a Gaussian process surrogate model for the sequential NAS settings.
Neural Methods for Point-wise Dependency Estimation
Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variables.
Fast Unbalanced Optimal Transport on a Tree
This study examines the time complexities of the unbalanced optimal transport problems from an algorithmic perspective for the first time.
Feature Robust Optimal Transport for High-dimensional Data
To show the effectiveness of FROT, we propose using the FROT algorithm for the layer selection problem in deep neural networks for semantic correspondence.
Volumization as a Natural Generalization of Weight Decay
We propose a novel regularization method, called \textit{volumization}, for neural networks.
Fast local linear regression with anchor regularization
Regression is an important task in machine learning and data mining.
Random Features Strengthen Graph Neural Networks
Through experiments, we show that the addition of random features enables GNNs to solve various problems that normal GNNs, including the graph convolutional networks (GCNs) and graph isomorphism networks (GINs), cannot solve.
Fast and Robust Comparison of Probability Measures in Heterogeneous Spaces
Building on \cite{memoli-2011}, who proposed to represent each point in each distribution as the 1D distribution of its distances to all other points, we introduce in this paper the Anchor Energy (AE) and Anchor Wasserstein (AW) distances, which are respectively the energy and Wasserstein distances instantiated on such representations.
FsNet: Feature Selection Network on High-dimensional Biological Data
Because a large number of parameters in the selection and reconstruction layers can easily result in overfitting under a limited number of samples, we use two tiny networks to predict the large, virtual weight matrices of the selection and reconstruction layers.
GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks
In this paper, we propose GraphLIME, a local interpretable model explanation for graphs using the Hilbert-Schmidt Independence Criterion (HSIC) Lasso, which is a nonlinear feature selection method.
Transformer Dissection: An Unified Understanding for Transformer's Attention via the Lens of Kernel
This new formulation gives us a better way to understand individual components of the Transformer{'}s attention, such as the better way to integrate the positional embedding.
Kernel Stein Tests for Multiple Model Comparison
The first test, building on the post selection inference framework, provably controls the number of best models that are wrongly declared worse (false positive rate).
More Powerful Selective Kernel Tests for Feature Selection
An approach for addressing this is via conditioning on the selection procedure to account for how we have used the data to generate our hypotheses, and prevent information to be used again after selection.
Flow-based Alignment Approaches for Probability Measures in Different Spaces
By leveraging a tree structure, we propose to align \textit{flows} from a root to each support instead of pair-wise tree metrics of supports, i. e., flows from a support to another, in GW.
Tree-Wasserstein Barycenter for Large-Scale Multilevel Clustering and Scalable Bayes
We study in this paper a variant of Wasserstein barycenter problem, which we refer to as tree-Wasserstein barycenter, by leveraging a specific class of ground metrics, namely tree metrics, for Wasserstein distance.
LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport
To estimate the mutual information from data, a common practice is preparing a set of paired samples $\{(\mathbf{x}_i,\mathbf{y}_i)\}_{i=1}^n \stackrel{\mathrm{i. i. d.
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel
This new formulation gives us a better way to understand individual components of the Transformer's attention, such as the better way to integrate the positional embedding.
Approximation Ratios of Graph Neural Networks for Combinatorial Problems
We theoretically demonstrate that the most powerful GNN can learn approximation algorithms for the minimum dominating set problem and the minimum vertex cover problem with some approximation ratios with the aid of the theory of distributed local algorithms.
Topological Bayesian Optimization with Persistence Diagrams
In this paper, we propose the topological Bayesian optimization, which can efficiently find an optimal solution from structured data using \emph{topological information}.
Learning to Sample Hard Instances for Graph Algorithms
We propose HiSampler, the hard instance sampler, to model the hard instance distribution of graph algorithms.
Tree-Sliced Variants of Wasserstein Distances
Optimal transport (\OT) theory defines a powerful set of tools to compare probability distributions.
Constant Time Graph Neural Networks
The recent advancements in graph neural networks (GNNs) have led to state-of-the-art performances in various applications, including chemo-informatics, question-answering systems, and recommender systems.
Learning Unsupervised Word Translations Without Adversaries
Word translation, or bilingual dictionary induction, is an important capability that impacts many multilingual language processing tasks.
Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator
In the paper, we propose a post selection inference (PSI) framework for divergence measure, which can select a set of statistically significant features that discriminate two distributions.
Selecting the Best in GANs Family: a Post Selection Inference Framework
"Which Generative Adversarial Networks (GANs) generates the most plausible images?"
"Dependency Bottleneck" in Auto-encoding Architectures: an Empirical Study
To address this issue, we propose to measure the dependency instead of MI between layers in DNNs.
Persistence Fisher Kernel: A Riemannian Manifold Kernel for Persistence Diagrams
To deal with it, an emerged approach is to use kernel methods, and an appropriate geometry for PDs is an important factor to measure the similarity of PDs.
Deep Matching Autoencoders
In this paper we introduce Deep Matching Autoencoders (DMAE), which learn a common latent space and pairing from unpaired multi-modal data.
Convex Coupled Matrix and Tensor Completion
We propose a set of convex low rank inducing norms for a coupled matrices and tensors (hereafter coupled tensors), which shares information between matrices and tensors through common modes.
Interpreting Outliers: Localized Logistic Regression for Density Ratio Estimation
We propose an inlier-based outlier detection method capable of both identifying the outliers and explaining why they are outliers, by identifying the outlier-specific features.
Multi-view Anomaly Detection via Robust Probabilistic Latent Variable Models
With the proposed model, all views of a non-anomalous instance are assumed to be generated from a single latent vector.
Post Selection Inference with Kernels
We propose a novel kernel based post selection inference (PSI) algorithm, which can not only handle non-linearity in data but also structured output such as multi-dimensional and multi-label outputs.
Ultra High-Dimensional Nonlinear Feature Selection for Big Biological Data
However, sophisticated learning models are computationally unfeasible for data with millions of features.
Localized Lasso for High-Dimensional Regression
We introduce the localized Lasso, which is suited for learning models that are both interpretable and have a high predictive power in problems with high dimensionality $d$ and small sample size $n$.
Convex Factorization Machine for Regression
We propose the convex factorization machine (CFM), which is a convex variant of the widely used Factorization Machines (FMs).
Consistent Collective Matrix Completion under Joint Low Rank Structure
We address the collective matrix completion problem of jointly recovering a collection of matrices with shared structure from partial (and potentially noisy) observations.
Multi-view Anomaly Detection via Probabilistic Latent Variable Models
We propose a nonparametric Bayesian probabilistic latent variable model for multi-view anomaly detection, which is the task of finding instances that have inconsistent views.
N$^3$LARS: Minimum Redundancy Maximum Relevance Feature Selection for Large and High-dimensional Data
We propose a feature selection method that finds non-redundant features from a large and high-dimensional data in nonlinear way.
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation
The objective of change-point detection is to discover abrupt property changes lying behind time-series data.
High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso
We first show that, with particular choices of kernel functions, non-redundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures.
Relative Density-Ratio Estimation for Robust Distribution Comparison
Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity test.
