Search Results for author:
Found 79 papers, 26 papers with code
A First Look at creating mock catalogs with machine learning techniques
We investigate machine learning (ML) techniques for predicting the number of galaxies (N_gal) that occupy a halo, given the halo's properties.
Cosmology and Nongalactic Astrophysics
Fast Distribution To Real Regression
We study the problem of distribution to real-value regression, where one aims to regress a mapping $f$ that takes in a distribution input covariate $P\in \mathcal{I}$ (for a non-parametric family of distributions $\mathcal{I}$) and outputs a real-valued response $Y=f(P) + \epsilon$.
FuSSO: Functional Shrinkage and Selection Operator
We present the FuSSO, a functional analogue to the LASSO, that efficiently finds a sparse set of functional input covariates to regress a real-valued response against.
Two-stage Sampled Learning Theory on Distributions
To the best of our knowledge, the only existing method with consistency guarantees for distribution regression requires kernel density estimation as an intermediate step (which suffers from slow convergence issues in high dimensions), and the domain of the distributions to be compact Euclidean.
Nonparametric Estimation of Renyi Divergence and Friends
We consider nonparametric estimation of $L_2$, Renyi-$\alpha$ and Tsallis-$\alpha$ divergences between continuous distributions.
A Machine Learning Approach for Dynamical Mass Measurements of Galaxy Clusters
In the conventional method, we use a standard M(sigma_v) power law scaling relation to infer cluster mass, M, from line-of-sight (LOS) galaxy velocity dispersion, sigma_v.
Cosmology and Nongalactic Astrophysics
Fast Function to Function Regression
Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping between two separate types of distributions.
On Estimating $L_2^2$ Divergence
We give a comprehensive theoretical characterization of a nonparametric estimator for the $L_2^2$ divergence between two continuous distributions.
Learning Theory for Distribution Regression
In this paper, we study a simple, analytically computable, ridge regression-based alternative to distribution regression, where we embed the distributions to a reproducing kernel Hilbert space, and learn the regressor from the embeddings to the outputs.
Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations
We propose and analyze estimators for statistical functionals of one or more distributions under nonparametric assumptions.
On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives
The current literature is split into two kinds of tests - those which are consistent without any assumptions about how the distributions may differ (\textit{general} alternatives), and those which are designed to specifically test easier alternatives, like a difference in means (\textit{mean-shift} alternatives).
High Dimensional Bayesian Optimisation and Bandits via Additive Models
We prove that, for additive functions the regret has only linear dependence on $D$ even though the function depends on all $D$ dimensions.
An Analysis of Active Learning With Uniform Feature Noise
For larger $\sigma$, the \textit{unflattening} of the regression function on convolution with uniform noise, along with its local antisymmetry around the threshold, together yield a behaviour where noise \textit{appears} to be beneficial.
Bayesian Nonparametric Kernel-Learning
In this paper we introduce Bayesian nonparmetric kernel-learning (BaNK), a generic, data-driven framework for scalable learning of kernels.
Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing
We formally characterize the power of popular tests for GDA like the Maximum Mean Discrepancy with the Gaussian kernel (gMMD) and bandwidth-dependent variants of the Energy Distance with the Euclidean norm (eED) in the high-dimensional MDA regime.
Boolean Matrix Factorization and Noisy Completion via Message Passing
Boolean matrix factorization and Boolean matrix completion from noisy observations are desirable unsupervised data-analysis methods due to their interpretability, but hard to perform due to their NP-hardness.
Nonparametric von Mises Estimators for Entropies, Divergences and Mutual Informations
We propose and analyse estimators for statistical functionals of one or moredistributions under nonparametric assumptions. Our estimators are derived from the von Mises expansion andare based on the theory of influence functions, which appearin the semiparametric statistics literature. We show that estimators based either on data-splitting or a leave-one-out techniqueenjoy fast rates of convergence and other favorable theoretical properties. We apply this framework to derive estimators for several popular informationtheoretic quantities, and via empirical evaluation, show the advantage of thisapproach over existing estimators.
Stochastic Neural Networks with Monotonic Activation Functions
We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise.
Stochastic Variance Reduction for Nonconvex Optimization
We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient (SVRG) methods for them.
Fast Incremental Method for Nonconvex Optimization
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form $\min_x \sum_i f_i(x)$.
Multi-fidelity Gaussian Process Bandit Optimisation
However, in many cases, cheap approximations to $f$ may be obtainable.
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization
This paper builds upon our recent series of papers on fast stochastic methods for smooth nonconvex optimization [22, 23], with a novel analysis for nonconvex and nonsmooth functions.
Stochastic Frank-Wolfe Methods for Nonconvex Optimization
Finally, we show that the faster convergence rates of our variance reduced methods also translate into improved convergence rates for the stochastic setting.
Enabling Dark Energy Science with Deep Generative Models of Galaxy Images
To this end, we study the application of deep conditional generative models in generating realistic galaxy images.
Annealing Gaussian into ReLU: a New Sampling Strategy for Leaky-ReLU RBM
Due to numerical stability and quantifiability of the likelihood, RBM is commonly used with Bernoulli units.
Deep Learning with Sets and Point Clouds
We introduce a simple permutation equivariant layer for deep learning with set structure. This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set.
Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex.
Variance Reduction in Stochastic Gradient Langevin Dynamics
In this paper, we present techniques for reducing variance in stochastic gradient Langevin dynamics, yielding novel stochastic Monte Carlo methods that improve performance by reducing the variance in the stochastic gradient.
Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations
However, in many cases, cheap approximations to $\func$ may be obtainable.
Hypothesis Transfer Learning via Transformation Functions
We consider the Hypothesis Transfer Learning (HTL) problem where one incorporates a hypothesis trained on the source domain into the learning procedure of the target domain.
Equivariance Through Parameter-Sharing
We propose to study equivariance in deep neural networks through parameter symmetries.
The Statistical Recurrent Unit
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications.
CMU DeepLens: Deep Learning For Automatic Image-based Galaxy-Galaxy Strong Lens Finding
We find on our simulated data set that for a rejection rate of non-lenses of 99%, a completeness of 90% can be achieved for lenses with Einstein radii larger than 1. 4" and S/N larger than 20 on individual $g$-band LSST exposures.
Instrumentation and Methods for Astrophysics Cosmology and Nongalactic Astrophysics Astrophysics of Galaxies
Deep Sets
Our main theorem characterizes the permutation invariant functions and provides a family of functions to which any permutation invariant objective function must belong.
Multi-fidelity Bayesian Optimisation with Continuous Approximations
Bandit methods for black-box optimisation, such as Bayesian optimisation, are used in a variety of applications including hyper-parameter tuning and experiment design.
One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models
On the other hand, traditional methods using signal priors can be used in all linear inverse problems but often have worse performance on challenging tasks.
Data-driven Random Fourier Features using Stein Effect
Large-scale kernel approximation is an important problem in machine learning research.
Asynchronous Parallel Bayesian Optimisation via Thompson Sampling
We design and analyse variations of the classical Thompson sampling (TS) procedure for Bayesian optimisation (BO) in settings where function evaluations are expensive, but can be performed in parallel.
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape.
Recurrent Estimation of Distributions
After, an RNN is used to compute the conditional distributions of the latent covariates.
A Generic Approach for Escaping Saddle points
A central challenge to using first-order methods for optimizing nonconvex problems is the presence of saddle points.
One Network to Solve Them All -- Solving Linear Inverse Problems Using Deep Projection Models
While deep learning methods have achieved state-of-the-art performance in many challenging inverse problems like image inpainting and super-resolution, they invariably involve problem-specific training of the networks.
Estimating Cosmological Parameters from the Dark Matter Distribution
A major approach to estimating the cosmological parameters is to use the large-scale matter distribution of the Universe.
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i. e., $f(\mathbf{Z}, \mathbf{w}, \mathbf{a}) = \sum_j a_j\sigma(\mathbf{w}^T\mathbf{Z}_j)$, in which both the convolutional weights $\mathbf{w}$ and the output weights $\mathbf{a}$ are parameters to be learned.
Neural Architecture Search with Bayesian Optimisation and Optimal Transport
A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model.
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent
A major challenge in understanding the generalization of deep learning is to explain why (stochastic) gradient descent can exploit the network architecture to find solutions that have good generalization performance when using high capacity models.
Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming
We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal.
Subject2Vec: Generative-Discriminative Approach from a Set of Image Patches to a Vector
Our model consists of three mutually dependent modules which regulate each other: (1) a discriminative network that learns a fixed-length representation from local features and maps them to disease severity; (2) an attention mechanism that provides interpretability by focusing on the areas of the anatomy that contribute the most to the prediction task; and (3) a generative network that encourages the diversity of the local latent features.
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities.
End-to-End Physics Event Classification with CMS Open Data: Applying Image-Based Deep Learning to Detector Data for the Direct Classification of Collision Events at the LHC
This paper describes the construction of novel end-to-end image-based classifiers that directly leverage low-level simulated detector data to discriminate signal and background processes in pp collision events at the Large Hadron Collider at CERN.
Gradient Descent Provably Optimizes Over-parameterized Neural Networks
One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth.
Point Cloud GAN
In this paper, we first show a straightforward extension of existing GAN algorithm is not applicable to point clouds, because the constraint required for discriminators is undefined for set data.
Hallucinating Point Cloud into 3D Sculptural Object
Our approach extends DeepDream from images to 3D point clouds.
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Our method is based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.
ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum.
A Robust and Efficient Deep Learning Method for Dynamical Mass Measurements of Galaxy Clusters
Our first model, CNN$_\text{1D}$, infers cluster mass directly from the distribution of member galaxy line-of-sight velocities.
Cosmology and Nongalactic Astrophysics
End-to-End Jet Classification of Quarks and Gluons with the CMS Open Data
We describe the construction of end-to-end jet image classifiers based on simulated low-level detector data to discriminate quark- vs. gluon-initiated jets with high-fidelity simulated CMS Open Data.
The Role of Machine Learning in the Next Decade of Cosmology
In recent years, machine learning (ML) methods have remarkably improved how cosmologists can interpret data.
Instrumentation and Methods for Astrophysics Cosmology and Nongalactic Astrophysics
Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO.
Competence-based Curriculum Learning for Neural Machine Translation
In this paper, we propose a curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance.
A Deep Reinforcement Learning Approach for Global Routing
At the heart of the proposed method is deep reinforcement learning that enables an agent to produce an optimal policy for routing based on the variety of problems it is presented with leveraging the conjoint optimization mechanism of deep reinforcement learning.
ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations
In applications such as molecule design or drug discovery, it is desirable to have an algorithm which recommends new candidate molecules based on the results of past tests.
Developing Creative AI to Generate Sculptural Objects
We explore the intersection of human and machine creativity by generating sculptural objects through machine learning.
Autonomous discovery of battery electrolytes with robotic experimentation and machine-learning
Innovations in batteries take years to formulate and commercialize, requiring extensive experimentation during the design and optimization phases.
RotationOut as a Regularization Method for Neural Network
We further use a noise analysis method to interpret the difference between RotationOut and Dropout in co-adaptation reduction.
Learning Local Search Heuristics for Boolean Satisfiability
We present an approach to learn SAT solver heuristics from scratch through deep reinforcement learning with a curriculum.
Learned Interpolation for 3D Generation
The typical approach for incorporating this creative process is to interpolate in a learned latent space so as to avoid the problem of generating unrealistic instances by exploiting the model's learned structure.
Unsupervised Program Synthesis for Images By Sampling Without Replacement
However, they struggle with the inherent sparsity of meaningful programs in the search space.
Optimal Exact Matrix Completion Under new Parametrization
We study the problem of exact completion for $m \times n$ sized matrix of rank $r$ with the adaptive sampling method.
Adaptive Sampling Distributed Stochastic Variance Reduced Gradient for Heterogeneous Distributed Datasets
It often leads to the dependence of convergence rate on maximum Lipschitz constant of gradients across the devices.
Robust Density Estimation under Besov IPM Losses
We study minimax convergence rates of nonparametric density estimation in the Huber contamination model, in which a proportion of the data comes from an unknown outlier distribution.
Politeness Transfer: A Tag and Generate Approach
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
Covariate Distribution Aware Meta-learning
Based on this model we propose a computationally feasible meta-learning algorithm by introducing meaningful relaxations in our final objective.
Nonlinear ISA with Auxiliary Variables for Learning Speech Representations
This paper extends recent work on nonlinear Independent Component Analysis (ICA) by introducing a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables.
Robust Handwriting Recognition with Limited and Noisy Data
Despite the advent of deep learning in computer vision, the general handwriting recognition problem is far from solved.
Modeling Task Effects on Meaning Representation in the Brain via Zero-Shot MEG Prediction
These results suggest that only the end of semantic processing of a word is task-dependent, and pose a challenge for future research to formulate new hypotheses for earlier task effects as a function of the task and stimuli.
Improving Molecule Properties Through 2-Stage VAE
Variational autoencoder (VAE) is a popular method for drug discovery and there had been a great deal of architectures and pipelines proposed to improve its performance.
Objective-Agnostic Enhancement of Molecule Properties via Multi-Stage VAE
In this paper, we explore applying a multi-stage VAE approach, that can improve manifold recovery on a synthetic dataset, to the field of drug discovery.
Task-Based MoE for Multitask Multilingual Machine Translation
Mixture-of-experts (MoE) architecture has been proven a powerful method for diverse tasks in training deep models in many applications.
