Search Results for author:
Found 63 papers, 24 papers with code
Document Entity Retrieval with Massive and Noisy Pre-training
Visually-Rich Document Entity Retrieval (VDER) is a type of machine learning task that aims at recovering text spans in the documents for each of the entities in question.
LambdaBeam: Neural Program Search with Higher-Order Functions and Lambdas
Search is an important technique in program synthesis that allows for adaptive strategies such as focusing on particular search directions based on execution results.
SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL
One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases.
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
Combinatorial optimization (CO) problems are often NP-hard and thus out of reach for exact algorithms, making them a tempting domain to apply machine learning methods.
Universal Self-adaptive Prompting
A hallmark of modern large language models (LLMs) is their impressive general zero-shot and few-shot abilities, often elicited through prompt-based and/or in-context learning.
Better Zero-Shot Reasoning with Self-Adaptive Prompting
Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans.
Gradient-Free Structured Pruning with Unlabeled Data
By only using the weights of the pre-trained model and unlabeled data, in a matter of a few minutes on a single GPU, up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.
Learning Universal Policies via Text-Guided Video Generation
The proposed policy-as-video formulation can further represent environments with different state and action spaces in a unified space of images, which, for example, enables learning and generalization across a variety of robot manipulation tasks.
Score-based Continuous-time Discrete Diffusion Models
Score-based modeling through stochastic differential equations (SDEs) has provided a new perspective on diffusion models, and demonstrated superior performance on continuous data.
Learning to Optimize with Stochastic Dominance Constraints
In real-world decision-making, uncertainty is important yet difficult to handle.
Optimal Scaling for Locally Balanced Proposals in Discrete Spaces
Optimal scaling has been well studied for Metropolis-Hastings (M-H) algorithms in continuous spaces, but a similar understanding has been lacking in discrete spaces.
Annealed Training for Combinatorial Optimization on Graphs
However, learning neural networks for CO problems is notoriously difficult in lack of the labeled data as the training is easily trapped at local optima.
Does GNN Pretraining Help Molecular Representation?
Extracting informative representations of molecules using Graph neural networks (GNNs) is crucial in AI-driven drug discovery.
Discrete Langevin Sampler via Wasserstein Gradient Flow
It is known that gradient-based MCMC samplers for continuous spaces, such as Langevin Monte Carlo (LMC), can be derived as particle versions of a gradient flow that minimizes KL divergence on a Wasserstein manifold.
CrossBeam: Learning to Search in Bottom-Up Program Synthesis
Many approaches to program synthesis perform a search within an enormous space of programs to find one that satisfies a given specification.
Neural Stochastic Dual Dynamic Programming
Stochastic dual dynamic programming (SDDP) is a state-of-the-art method for solving multi-stage stochastic optimization, widely used for modeling real-world process optimization tasks.
Towards understanding retrosynthesis by energy-based models
In this paper, we propose a framework that unifies sequence- and graph-based methods as energy-based models (EBMs) with different energy functions.
SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs
There are two important reasoning tasks on KGs: (1) single-hop knowledge graph completion, which involves predicting individual links in the KG; and (2), multi-hop reasoning, where the goal is to predict which KG entities satisfy a given logical query.
Path Auxiliary Proposal for MCMC in Discrete Space
Energy-based Model (EBM) offers a powerful approach for modeling discrete structure, but both inference and learning of EBM are hard as it involves sampling from discrete distributions.
CodeTrek: Flexible Modeling of Code using an Extensible Relational Representation
Designing a suitable representation for code-reasoning tasks is challenging in aspects such as the kinds of program information to model, how to combine them, and how much context to consider.
Combiner: Full Attention Transformer with Sparse Computation Cost
However, the key limitation of transformers is their quadratic memory and time complexity $\mathcal{O}(L^2)$ with respect to the sequence length in attention layers, which restricts application in extremely long sequences.
Ranked #2 on
Language Modelling
on Wiki-40B
SpreadsheetCoder: Formula Prediction from Semi-structured Context
In this work, we present the first approach for synthesizing spreadsheet formulas from tabular context, which includes both headers and semi-structured tabular data.
Generative Fairness Teaching
We propose a generative fairness teaching framework that provides a model with not only real samples but also synthesized samples to compensate the data biases during training.
A Framework For Differentiable Discovery Of Graph Algorithms
Recently there is a surge of interests in using graph neural networks (GNNs) to learn algorithms.
PolyRetro: Few-shot Polymer Retrosynthesis via Domain Adaptation
We demonstrate that our method is able to propose high-quality polymerization plans for a dataset of 52 real-world polymers, of which a significant portion successfully recovers the currently-in-used polymerization processes in the real world.
Molecule Optimization by Explainable Evolution
Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science and drug discovery.
Differentiable Top-k with Optimal Transport
Finding the k largest or smallest elements from a collection of scores, i. e., top-k operation, is an important model component widely used in information retrieval, machine learning, and data mining.
Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
In this paper we propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search.
Polymers for Extreme Conditions Designed Using Syntax-Directed Variational Autoencoders
The design/discovery of new materials is highly non-trivial owing to the near-infinite possibilities of material candidates, and multiple required property/performance objectives.
Differentiable Top-$k$ with Optimal Transport
The top-$k$ operation, i. e., finding the $k$ largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.
BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration
Program synthesis is challenging largely because of the difficulty of search in a large space of programs.
Energy-based View of Retrosynthesis
Retrosynthesis -- the process of identifying a set of reactants to synthesize a target molecule -- is of vital importance to material design and drug discovery.
Ranked #1 on
Single-step retrosynthesis
on USPTO-50k
Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search
Retrosynthetic planning is a critical task in organic chemistry which identifies a series of reactions that can lead to the synthesis of a target product.
Ranked #5 on
Multi-step retrosynthesis
on USPTO-190
Scalable Deep Generative Modeling for Sparse Graphs
Based on this, we develop a novel autoregressive model, named BiGG, that utilizes this sparsity to avoid generating the full adjacency matrix, and importantly reduces the graph generation time complexity to $O((n + m)\log n)$.
Learning to Stop While Learning to Predict
Similar to algorithms, the optimal depth of a deep architecture may be different for different input instances, either to avoid ``over-thinking'', or because we want to compute less for operations converged already.
HOPPITY: LEARNING GRAPH TRANSFORMATIONS TO DETECT AND FIX BUGS IN PROGRAMS
We present a learning-based approach to detect and fix a broad range of bugs in Javascript programs.
Energy-Based Processes for Exchangeable Data
Recently there has been growing interest in modeling sets with exchangeability such as point clouds.
Differentiable Top-k Operator with Optimal Transport
The top-k operation, i. e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.
Retrosynthesis Prediction with Conditional Graph Logic Network
Retrosynthesis is one of the fundamental problems in organic chemistry.
Ranked #10 on
Single-step retrosynthesis
on USPTO-50k
Learning Transferable Graph Exploration
We propose a `learning to explore' framework where we learn a policy from a distribution of environments.
Cooperative neural networks (CoNN): Exploiting prior independence structure for improved classification
We propose a new approach, called cooperative neural networks (CoNN), which uses a set of cooperatively trained neural networks to capture latent representations that exploit prior given independence structure.
Learning a Meta-Solver for Syntax-Guided Program Synthesis
Our framework consists of three components: 1) an encoder, which embeds both the logical specification and grammar at the same time using a graph neural network; 2) a grammar adaptive policy network which enables learning a transferable policy; and 3) a reinforcement learning algorithm that jointly trains the specification and grammar embedding and adaptive policy.
Exponential Family Estimation via Adversarial Dynamics Embedding
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks.
Particle Flow Bayes' Rule
We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation.
CompILE: Compositional Imitation Learning and Execution
We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data.
Learning Loop Invariants for Program Verification
A fundamental problem in program verification concerns inferring loop invariants.
Coupled Variational Bayes via Optimization Embedding
This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution.
Kernel Exponential Family Estimation via Doubly Dual Embedding
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space.
Learning Steady-States of Iterative Algorithms over Graphs
Many graph analytics problems can be solved via iterative algorithms where the solutions are often characterized by a set of steady-state conditions.
Adversarial Attack on Graph Structured Data
Deep learning on graph structures has shown exciting results in various applications.
KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
Our model learns to reason with neural embeddings of both knowledge graphs.
Syntax-Directed Variational Autoencoder for Structured Data
Deep generative models have been enjoying success in modeling continuous data.
Variational Reasoning for Question Answering with Knowledge Graph
Knowledge graph (KG) is known to be helpful for the task of question answering (QA), since it provides well-structured relational information between entities, and allows one to further infer indirect facts.
Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs
The occurrence of a fact (edge) is modeled as a multivariate point process whose intensity function is modulated by the score for that fact computed based on the learned entity embeddings.
Learning Combinatorial Optimization Algorithms over Graphs
The design of good heuristics or approximation algorithms for NP-hard combinatorial optimization problems often requires significant specialized knowledge and trial-and-error.
Deep Coevolutionary Network: Embedding User and Item Features for Recommendation
DeepCoevolve use recurrent neural network (RNN) over evolving networks to define the intensity function in point processes, which allows the model to capture complex mutual influence between users and items, and the feature evolution over time.
Discriminative Embeddings of Latent Variable Models for Structured Data
Kernel classifiers and regressors designed for structured data, such as sequences, trees and graphs, have significantly advanced a number of interdisciplinary areas such as computational biology and drug design.
M-Statistic for Kernel Change-Point Detection
Detecting the emergence of an abrupt change-point is a classic problem in statistics and machine learning.
Online Supervised Subspace Tracking
The optimization problem for OSDR is non-convex and hard to analyze in general; we provide convergence analysis of OSDR in a simple linear regression setting.
Scan $B$-Statistic for Kernel Change-Point Detection
A novel theoretical result of the paper is the characterization of the tail probability of these statistics using the change-of-measure technique, which focuses on characterizing the tail of the detection statistics rather than obtaining its asymptotic distribution under the null distribution.
Provable Bayesian Inference via Particle Mirror Descent
Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters.
Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks
Click prediction is one of the fundamental problems in sponsored search.
