Search Results for author: Stanley Osher

Found 29 papers, 14 papers with code

Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals

1 code implementation28 Aug 2023 Hong Ye Tan, Stanley Osher, Wuchen Li

The score term is given in closed form by a regularized Wasserstein proximal, using a kernel convolution that is approximated by sampling.

Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data

2 code implementations1 Jan 2023 Hien Dang, Tho Tran, Stanley Osher, Hung Tran-The, Nhat Ho, Tan Nguyen

Modern deep neural networks have achieved impressive performance on tasks from image classification to natural language processing.

Image Classification

Taming Hyperparameter Tuning in Continuous Normalizing Flows Using the JKO Scheme

1 code implementation30 Nov 2022 Alexander Vidal, Samy Wu Fung, Luis Tenorio, Stanley Osher, Levon Nurbekyan

Instead of tuning $\alpha$, we repeatedly solve the optimization problem for a fixed $\alpha$ effectively performing a JKO update with a time-step $\alpha$.

Density Estimation

Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature

1 code implementation28 Nov 2022 Khang Nguyen, Hieu Nong, Vinh Nguyen, Nhat Ho, Stanley Osher, Tan Nguyen

Graph Neural Networks (GNNs) had been demonstrated to be inherently susceptible to the problems of over-smoothing and over-squashing.

Improving Generative Flow Networks with Path Regularization

no code implementations29 Sep 2022 Anh Do, Duy Dinh, Tan Nguyen, Khuong Nguyen, Stanley Osher, Nhat Ho

Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability proportional to a given reward function.

Active Learning

Neural ODE Control for Trajectory Approximation of Continuity Equation

no code implementations18 May 2022 Karthik Elamvazhuthi, Bahman Gharesifard, Andrea Bertozzi, Stanley Osher

As a corollary to this result, we establish that the continuity equation of the neural ODE is approximately controllable on the set of compactly supported probability measures that are absolutely continuous with respect to the Lebesgue measure.

How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies

no code implementations13 Oct 2021 Bao Wang, Hedi Xia, Tan Nguyen, Stanley Osher

As case studies, we consider how momentum can improve the architecture design for recurrent neural networks (RNNs), neural ordinary differential equations (ODEs), and transformers.

Computational Efficiency

GRAND++: Graph Neural Diffusion with A Source Term

no code implementations ICLR 2022 Matthew Thorpe, Tan Minh Nguyen, Hedi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, Bao Wang

We propose GRAph Neural Diffusion with a source term (GRAND++) for graph deep learning with a limited number of labeled nodes, i. e., low-labeling rate.

Graph Learning

JFB: Jacobian-Free Backpropagation for Implicit Networks

2 code implementations23 Mar 2021 Samy Wu Fung, Howard Heaton, Qiuwei Li, Daniel Mckenzie, Stanley Osher, Wotao Yin

Unlike traditional networks, implicit networks solve a fixed point equation to compute inferences.

Wasserstein Proximal of GANs

no code implementations ICLR 2019 Alex Tong Lin, Wuchen Li, Stanley Osher, Guido Montufar

We introduce a new method for training generative adversarial networks by applying the Wasserstein-2 metric proximal on the generators.

A Neural Network Approach Applied to Multi-Agent Optimal Control

1 code implementation9 Nov 2020 Derek Onken, Levon Nurbekyan, Xingjian Li, Samy Wu Fung, Stanley Osher, Lars Ruthotto

Our approach is grid-free and scales efficiently to dimensions where grids become impractical or infeasible.

Optimization and Control

Wasserstein-based Projections with Applications to Inverse Problems

2 code implementations5 Aug 2020 Howard Heaton, Samy Wu Fung, Alex Tong Lin, Stanley Osher, Wotao Yin

To bridge this gap, we present a new algorithm that takes samples from the manifold of true data as input and outputs an approximation of the projection operator onto this manifold.

Differentially Private Federated Learning with Laplacian Smoothing

no code implementations1 May 2020 Zhicong Liang, Bao Wang, Quanquan Gu, Stanley Osher, Yuan YAO

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users.

Federated Learning

A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems

1 code implementation4 Dec 2019 Lars Ruthotto, Stanley Osher, Wuchen Li, Levon Nurbekyan, Samy Wu Fung

State-of-the-art numerical methods for solving such problems utilize spatial discretization that leads to a curse-of-dimensionality.

BIG-bench Machine Learning

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

1 code implementation2 Nov 2019 Bao Wang, Difan Zou, Quanquan Gu, Stanley Osher

As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling.

Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets

no code implementations ICLR 2019 Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, Jack Xin

We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss.


Unnormalized Optimal Transport

1 code implementation9 Feb 2019 Wilfrid Gangbo, Wuchen Li, Stanley Osher, Michael Puthawala

We propose an extension of the computational fluid mechanics approach to the Monge-Kantorovich mass transfer problem, which was developed by Benamou-Brenier.

Optimization and Control

Decentralized Multi-Agents by Imitation of a Centralized Controller

no code implementations6 Feb 2019 Alex Tong Lin, Mark J. Debord, Katia Estabridis, Gary Hewer, Guido Montufar, Stanley Osher

In order to obtain multi-agents that act in a decentralized manner, we introduce a novel algorithm under the popular framework of centralized training, but decentralized execution.

Imitation Learning Multi-agent Reinforcement Learning +2

Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks

no code implementations15 Aug 2018 Penghang Yin, Shuai Zhang, Jiancheng Lyu, Stanley Osher, Yingyong Qi, Jack Xin

We introduce the notion of coarse gradient and propose the blended coarse gradient descent (BCGD) algorithm, for training fully quantized neural networks.

Binarization Quantization

Laplacian Smoothing Gradient Descent

1 code implementation17 Jun 2018 Stanley Osher, Bao Wang, Penghang Yin, Xiyang Luo, Farzin Barekat, Minh Pham, Alex Lin

We propose a class of very simple modifications of gradient descent and stochastic gradient descent.

BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights

2 code implementations19 Jan 2018 Penghang Yin, Shuai Zhang, Jiancheng Lyu, Stanley Osher, Yingyong Qi, Jack Xin

We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights.


Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for $k$-means Clustering

no code implementations21 Oct 2017 Penghang Yin, Minh Pham, Adam Oberman, Stanley Osher

In this paper, we propose an implicit gradient descent algorithm for the classic $k$-means problem.


Deep Relaxation: partial differential equations for optimizing deep neural networks

no code implementations17 Apr 2017 Pratik Chaudhari, Adam Oberman, Stanley Osher, Stefano Soatto, Guillaume Carlier

In this paper we establish a connection between non-convex optimization methods for training deep neural networks and nonlinear partial differential equations (PDEs).

Scalable low dimensional manifold model in the reconstruction of noisy and incomplete hyperspectral images

no code implementations18 May 2016 Wei Zhu, Zuoqiang Shi, Stanley Osher

We present a scalable low dimensional manifold model for the reconstruction of noisy and incomplete hyperspectral images.

A Multiphase Image Segmentation Based on Fuzzy Membership Functions and L1-norm Fidelity

no code implementations9 Apr 2015 Fang Li, Stanley Osher, Jing Qin, Ming Yan

In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity.

Image Segmentation Semantic Segmentation

Sparse Recovery via Differential Inclusions

1 code implementation30 Jun 2014 Stanley Osher, Feng Ruan, Jiechao Xiong, Yuan YAO, Wotao Yin

In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics.

Cannot find the paper you are looking for? You can Submit a new open access paper.