Search Results for author:
Found 59 papers, 16 papers with code
Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity
One of the most intriguing findings in the structure of neural network landscape is the phenomenon of mode connectivity: For two typical global minima, there exists a path connecting them without barrier.
A Duality Analysis of Kernel Ridge Regression in the Noiseless Regime
In this paper, we conduct a comprehensive analysis of generalization properties of Kernel Ridge Regression (KRR) in the noiseless regime, a scenario crucial to scientific computing, where data are often generated via computer simulations.
The Implicit Bias of Gradient Noise: A Symmetry Perspective
For one class of symmetry, SGD naturally converges to solutions that have a balanced and aligned gradient noise.
DocReal: Robust Document Dewarping of Real-Life Images via Attention-Enhanced Control Point Prediction
Document image dewarping is a crucial task in computer vision with numerous practical applications.
The Local Landscape of Phase Retrieval Under Limited Samples
We next consider the one-point strong convexity and show that as long as $n=\omega(d)$, with high probability, the landscape is one-point strongly convex in the local annulus: $\{w\in\mathbb{R}^d: o_d(1)\leqslant \|w-w^*\|\leqslant c\}$, where $w^*$ is the ground truth and $c$ is an absolute constant.
Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling
Inspired by this analysis, we propose a novel algorithm called Progressive Rescaling Gradient Descent (PRGD) and show that PRGD can maximize the margin at an {\em exponential rate}.
A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent
In this paper, we provide a theoretical study of noise geometry for minibatch stochastic gradient descent (SGD), a phenomenon where noise aligns favorably with the geometry of local landscape.
The $L^\infty$ Learnability of Reproducing Kernel Hilbert Spaces
We prove that if $\beta$ is independent of the input dimension $d$, then functions in the RKHS can be learned efficiently under the $L^\infty$ norm, i. e., the sample complexity depends polynomially on $d$.
The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI
Clinical monitoring of metastatic disease to the brain can be a laborious and time-consuming process, especially in cases involving multiple metastases when the assessment is performed manually.
Embedding Inequalities for Barron-type Spaces
To this end, researchers have introduced the Barron space $\mathcal{B}_s(\Omega)$ and the spectral Barron space $\mathcal{F}_s(\Omega)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $\Omega\subset\mathbb{R}^d$ denotes the input domain.
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent
By contrast, for gradient descent (GD), the stability imposes a similar constraint but only on the largest eigenvalue of Hessian.
Theoretical Analysis of Inductive Biases in Deep Convolutional Networks
To this end, we compare the performance of CNNs, locally-connected networks (LCNs), and fully-connected networks (FCNs) on a simple regression task, where LCNs can be viewed as CNNs without weight sharing.
A duality framework for generalization analysis of random feature models and two-layer neural networks
The first application is to study learning functions in $\mathcal{F}_{p,\pi}$ with RFMs.
Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes
Coarse labels do not capture the long-tailed spectrum of brain disorder phenotypes, which leads to a loss of generalizability of the model that makes them less useful in diagnostic settings.
Towards Improving Operation Economics: A Bilevel MIP-Based Closed-Loop Predict-and-Optimize Framework for Prescribing Unit Commitment
Generally, system operators conduct the economic operation of power systems in an open-loop predict-then-optimize process: the renewable energy source (RES) availability and system reserve requirements are first predicted; given the predictions, system operators solve optimization models such as unit commitment (UC) to determine the economical operation plans accordingly.
RZCR: Zero-shot Character Recognition via Radical-based Reasoning
The long-tail effect is a common issue that limits the performance of deep learning models on real-world datasets.
The alignment property of SGD noise and how it helps select flat minima: A stability analysis
In this paper, we provide an explanation of this striking phenomenon by relating the particular noise structure of SGD to its \emph{linear stability} (Wu et al., 2018).
HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images
In this study, we propose a novel ROI-free model for breast cancer diagnosis in ultrasound images with interpretable feature representations.
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes
Numerically, we observe that neural network loss functions possesses a multiscale structure, manifested in two ways: (1) in a neighborhood of minima, the loss mixes a continuum of scales and grows subquadratically, and (2) in a larger region, the loss shows several separate scales clearly.
Exploiting the Potential of Datasets: A Data-Centric Approach for Model Robustness
Robustness of deep neural networks (DNNs) to malicious perturbations is a hot topic in trustworthy AI.
Learning a Single Neuron for Non-monotonic Activation Functions
Specifically, when the input distribution is the standard Gaussian, we show that mild conditions on $\sigma$ (e. g., $\sigma$ has a dominating linear part) are sufficient to guarantee the learnability in polynomial time and polynomial samples.
Efficient Medical Image Segmentation Based on Knowledge Distillation
To deal with this problem, we propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network.
A spectral-based analysis of the separation between two-layer neural networks and linear methods
We propose a spectral-based approach to analyze how two-layer neural networks separate from linear methods in terms of approximating high-dimensional functions.
Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data
We show that self-supervised models are not as robust as expected based on their results in natural imaging benchmarks and can be outperformed by supervised learning with dropout.
Hands-on Guidance for Distilling Object Detectors
Knowledge distillation can lead to deploy-friendly networks against the plagued computational complexity problem, but previous methods neglect the feature hierarchy in detectors.
Photon-jet events as a probe of axion-like particles at the LHC
Axion-like particles (ALPs) are predicted by many extensions of the Standard Model (SM).
High Energy Physics - Phenomenology High Energy Physics - Experiment
On self-supervised multi-modal representation learning: An application to Alzheimer's disease
In this paper, we introduce a way to exhaustively consider multimodal architectures for contrastive self-supervised fusion of fMRI and MRI of AD patients and controls.
Self-Supervised Multimodal Domino: in Search of Biomarkers for Alzheimer's Disease
Sensory input from multiple sources is crucial for robust and coherent human perception.
New Strong Bounds on sub-GeV Dark Matter from Boosted and Migdal Effects
Due to the low nuclear recoils, sub-GeV dark matter (DM) is usually beyond the sensitivity of the conventional DM direct detection experiments.
High Energy Physics - Phenomenology Cosmology and Nongalactic Astrophysics
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning.
Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms
The approach is motivated by approximating the general activation functions with one-dimensional ReLU networks, which reduces the problem to the complexity controls of ReLU networks.
A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms
The dynamic behavior of RMSprop and Adam algorithms is studied through a combination of careful numerical experiments and theoretical explanations.
The Slow Deterioration of the Generalization Error of the Random Feature Model
The random feature model exhibits a kind of resonance behavior when the number of parameters is close to the training sample size.
The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models
A numerical and phenomenological study of the gradient descent (GD) algorithm for training two-layer neural network models is carried out for different parameter regimes when the target function can be accurately approximated by a relatively small number of neurons.
Beyond the Virus: A First Look at Coronavirus-themed Mobile Malware
To facilitate future research, we have publicly released all the well-labelled COVID-19 themed apps (and malware) to the research community.
Cryptography and Security
Calibrating the dynamic Huff model for business analysis using location big data
In this research, we present a time-aware dynamic Huff model (T-Huff) for location-based market share analysis and calibrate this model using large-scale store visit patterns based on mobile phone location data across ten most populated U. S. cities.
Social and Information Networks H.1
Reinterpretation of LHC Results for New Physics: Status and Recommendations after Run 2
We report on the status of efforts to improve the reinterpretation of searches and measurements at the LHC in terms of models for new physics, in the context of the LHC Reinterpretation Forum.
High Energy Physics - Phenomenology High Energy Physics - Experiment
Machine learning based non-Newtonian fluid model with molecular fidelity
We introduce a machine-learning-based framework for constructing continuum non-Newtonian fluid dynamics model directly from a micro-scale description.
Machine Learning from a Continuous Viewpoint
We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations.
The Generalization Error of the Minimum-norm Solutions for Over-parameterized Neural Networks
We study the generalization properties of minimum-norm solutions for three over-parametrized machine learning models including the random feature model, the two-layer neural network model and the residual network model.
Global Convergence of Gradient Descent for Deep Linear Residual Networks
We analyze the global convergence of gradient descent for deep linear residual networks by proposing a new initialization: zero-asymmetric (ZAS) initialization.
EVulHunter: Detecting Fake Transfer Vulnerabilities for EOSIO's Smart Contracts at Webassembly-level
Unfortunately, current tools are web-application oriented and cannot be applied to EOSIO WebAssembly code directly, which makes it more difficult to detect vulnerabilities from those smart contracts.
Cryptography and Security
The Barron Space and the Flow-induced Function Spaces for Neural Network Models
We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space.
Exploring and Enhancing the Transferability of Adversarial Examples
State-of-the-art deep neural networks are vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs.
A Priori Estimates of the Generalization Error for Two-layer Neural Networks
These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model.
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Minima and Regularization Effects
Along this line, we theoretically study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics.
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections
In addition, it is also shown that the GD path is uniformly close to the functions given by the related random feature model.
A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics
In the over-parametrized regime, it is shown that gradient descent dynamics can achieve zero training loss exponentially fast regardless of the quality of the labels.
How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective
The question of which global minima are accessible by a stochastic gradient decent (SGD) algorithm with specific learning rate and batch size is studied from the perspective of dynamical stability.
A Priori Estimates of the Population Risk for Two-layer Neural Networks
New estimates for the population risk are established for two-layer neural networks.
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects
Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics.
Understanding and Enhancing the Transferability of Adversarial Examples
State-of-the-art deep neural networks are known to be vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs.
Enhancing the Transferability of Adversarial Examples with Noise Reduced Gradient
Deep neural networks provide state-of-the-art performance for many applications of interest.
Dual Long Short-Term Memory Networks for Sub-Character Representation Learning
To build a concrete study and substantiate the efficiency of our neural architecture, we take Chinese Word Segmentation as a research case example.
Effective Neural Solution for Multi-Criteria Word Segmentation
We present a simple yet elegant solution to train a single joint model on multi-criteria corpora for Chinese Word Segmentation (CWS).
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
It is widely observed that deep learning models with learned parameters generalize well, even with much more model parameters than the number of training samples.
SOM: Semantic Obviousness Metric for Image Quality Assessment
We propose to extract two types of features, one to measure the semantic obviousness of the image and the other to discover local characteristic.
Image Quality Estimation
No-Reference Image Quality Assessment
+1
LIFT : Multi-Label Learning with Label-Specific Features
Existing approaches learn from multi-label data by manipulating with identical feature set, i. e. the very instance representation of each example is employed in the discrimination processes of all class labels.
Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering
Learning distance functions with side information plays a key role in many machine learning and data mining applications.
