Search Results for author: Hanjun Dai

Found 63 papers, 24 papers with code

Document Entity Retrieval with Massive and Noisy Pre-training

no code implementations15 Jun 2023 Lijun Yu, Jin Miao, Xiaoyu Sun, Jiayi Chen, Alexander G. Hauptmann, Hanjun Dai, Wei Wei

Visually-Rich Document Entity Retrieval (VDER) is a type of machine learning task that aims at recovering text spans in the documents for each of the entities in question.

Entity Retrieval Few-Shot Learning +1

LambdaBeam: Neural Program Search with Higher-Order Functions and Lambdas

no code implementations3 Jun 2023 Kensen Shi, Hanjun Dai, Wen-Ding Li, Kevin Ellis, Charles Sutton

Search is an important technique in program synthesis that allows for adaptive strategies such as focusing on particular search directions based on execution results.

Program Synthesis

SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL

no code implementations26 May 2023 Ruoxi Sun, Sercan O. Arik, Hootan Nakhost, Hanjun Dai, Rajarishi Sinha, Pengcheng Yin, Tomas Pfister

One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases.

Language Modelling Large Language Model +1

Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets

no code implementations26 May 2023 Dinghuai Zhang, Hanjun Dai, Nikolay Malkin, Aaron Courville, Yoshua Bengio, Ling Pan

Combinatorial optimization (CO) problems are often NP-hard and thus out of reach for exact algorithms, making them a tempting domain to apply machine learning methods.

Combinatorial Optimization

Universal Self-adaptive Prompting

no code implementations24 May 2023 Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Martin Eisenschlos, Sercan O. Arik, Tomas Pfister

A hallmark of modern large language models (LLMs) is their impressive general zero-shot and few-shot abilities, often elicited through prompt-based and/or in-context learning.

Natural Language Understanding Text Generation +1

Better Zero-Shot Reasoning with Self-Adaptive Prompting

no code implementations23 May 2023 Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan O. Arik, Tomas Pfister

Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans.

Gradient-Free Structured Pruning with Unlabeled Data

no code implementations7 Mar 2023 Azade Nova, Hanjun Dai, Dale Schuurmans

By only using the weights of the pre-trained model and unlabeled data, in a matter of a few minutes on a single GPU, up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.

Model Compression

Learning Universal Policies via Text-Guided Video Generation

no code implementations31 Jan 2023 Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, Pieter Abbeel

The proposed policy-as-video formulation can further represent environments with different state and action spaces in a unified space of images, which, for example, enables learning and generalization across a variety of robot manipulation tasks.

Decision Making Image Generation +3

Score-based Continuous-time Discrete Diffusion Models

no code implementations30 Nov 2022 Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, Hanjun Dai

Score-based modeling through stochastic differential equations (SDEs) has provided a new perspective on diffusion models, and demonstrated superior performance on continuous data.

Optimal Scaling for Locally Balanced Proposals in Discrete Spaces

1 code implementation16 Sep 2022 Haoran Sun, Hanjun Dai, Dale Schuurmans

Optimal scaling has been well studied for Metropolis-Hastings (M-H) algorithms in continuous spaces, but a similar understanding has been lacking in discrete spaces.

Annealed Training for Combinatorial Optimization on Graphs

no code implementations23 Jul 2022 Haoran Sun, Etash K. Guha, Hanjun Dai

However, learning neural networks for CO problems is notoriously difficult in lack of the labeled data as the training is easily trapped at local optima.

Combinatorial Optimization

Does GNN Pretraining Help Molecular Representation?

no code implementations13 Jul 2022 Ruoxi Sun, Hanjun Dai, Adams Wei Yu

Extracting informative representations of molecules using Graph neural networks (GNNs) is crucial in AI-driven drug discovery.

Drug Discovery molecular representation

Discrete Langevin Sampler via Wasserstein Gradient Flow

no code implementations29 Jun 2022 Haoran Sun, Hanjun Dai, Bo Dai, Haomin Zhou, Dale Schuurmans

It is known that gradient-based MCMC samplers for continuous spaces, such as Langevin Monte Carlo (LMC), can be derived as particle versions of a gradient flow that minimizes KL divergence on a Wasserstein manifold.

CrossBeam: Learning to Search in Bottom-Up Program Synthesis

1 code implementation ICLR 2022 Kensen Shi, Hanjun Dai, Kevin Ellis, Charles Sutton

Many approaches to program synthesis perform a search within an enormous space of programs to find one that satisfies a given specification.

Program Synthesis Structured Prediction

Neural Stochastic Dual Dynamic Programming

no code implementations ICLR 2022 Hanjun Dai, Yuan Xue, Zia Syed, Dale Schuurmans, Bo Dai

Stochastic dual dynamic programming (SDDP) is a state-of-the-art method for solving multi-stage stochastic optimization, widely used for modeling real-world process optimization tasks.

Stochastic Optimization

Towards understanding retrosynthesis by energy-based models

no code implementations NeurIPS 2021 Ruoxi Sun, Hanjun Dai, Li Li, Steven Kearnes, Bo Dai

In this paper, we propose a framework that unifies sequence- and graph-based methods as energy-based models (EBMs) with different energy functions.

Drug Discovery Retrosynthesis

SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs

1 code implementation28 Oct 2021 Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Denny Zhou, Jure Leskovec, Dale Schuurmans

There are two important reasoning tasks on KGs: (1) single-hop knowledge graph completion, which involves predicting individual links in the KG; and (2), multi-hop reasoning, where the goal is to predict which KG entities satisfy a given logical query.


Path Auxiliary Proposal for MCMC in Discrete Space

no code implementations ICLR 2022 Haoran Sun, Hanjun Dai, Wei Xia, Arun Ramamurthy

Energy-based Model (EBM) offers a powerful approach for modeling discrete structure, but both inference and learning of EBM are hard as it involves sampling from discrete distributions.

CodeTrek: Flexible Modeling of Code using an Extensible Relational Representation

1 code implementation ICLR 2022 Pardis Pashakhanloo, Aaditya Naik, Yuepeng Wang, Hanjun Dai, Petros Maniatis, Mayur Naik

Designing a suitable representation for code-reasoning tasks is challenging in aspects such as the kinds of program information to model, how to combine them, and how much context to consider.

Exception type Variable misuse

Combiner: Full Attention Transformer with Sparse Computation Cost

2 code implementations NeurIPS 2021 Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, Bo Dai

However, the key limitation of transformers is their quadratic memory and time complexity $\mathcal{O}(L^2)$ with respect to the sequence length in attention layers, which restricts application in extremely long sequences.

Image Generation Language Modelling

SpreadsheetCoder: Formula Prediction from Semi-structured Context

1 code implementation26 Jun 2021 Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou

In this work, we present the first approach for synthesizing spreadsheet formulas from tabular context, which includes both headers and semi-structured tabular data.

Program Synthesis

Generative Fairness Teaching

no code implementations1 Jan 2021 Rongmei Lin, Hanjun Dai, Li Xiong, Wei Wei

We propose a generative fairness teaching framework that provides a model with not only real samples but also synthesized samples to compensate the data biases during training.


PolyRetro: Few-shot Polymer Retrosynthesis via Domain Adaptation

no code implementations1 Jan 2021 Binghong Chen, Chengtao Li, Hanjun Dai, Rampi Ramprasad, Le Song

We demonstrate that our method is able to propose high-quality polymerization plans for a dataset of 52 real-world polymers, of which a significant portion successfully recovers the currently-in-used polymerization processes in the real world.

Domain Adaptation Retrosynthesis

Molecule Optimization by Explainable Evolution

no code implementations ICLR 2021 Binghong Chen, Tianzhe Wang, Chengtao Li, Hanjun Dai, Le Song

Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science and drug discovery.

Drug Discovery

Differentiable Top-k with Optimal Transport

no code implementations NeurIPS 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

Finding the k largest or smallest elements from a collection of scores, i. e., top-k operation, is an important model component widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration

no code implementations NeurIPS 2020 Hanjun Dai, Rishabh Singh, Bo Dai, Charles Sutton, Dale Schuurmans

In this paper we propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search.

Language Modelling

Polymers for Extreme Conditions Designed Using Syntax-Directed Variational Autoencoders

no code implementations4 Nov 2020 Rohit Batra, Hanjun Dai, Tran Doan Huan, Lihua Chen, Chiho Kim, Will R. Gutekunst, Le Song, Rampi Ramprasad

The design/discovery of new materials is highly non-trivial owing to the near-infinite possibilities of material candidates, and multiple required property/performance objectives.


Differentiable Top-$k$ with Optimal Transport

no code implementations NeurIPS Workshop LMCA 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

The top-$k$ operation, i. e., finding the $k$ largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

Energy-based View of Retrosynthesis

no code implementations14 Jul 2020 Ruoxi Sun, Hanjun Dai, Li Li, Steven Kearnes, Bo Dai

Retrosynthesis -- the process of identifying a set of reactants to synthesize a target molecule -- is of vital importance to material design and drug discovery.

Drug Discovery Retrosynthesis +1

Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search

1 code implementation ICML 2020 Binghong Chen, Chengtao Li, Hanjun Dai, Le Song

Retrosynthetic planning is a critical task in organic chemistry which identifies a series of reactions that can lead to the synthesis of a target product.

Multi-step retrosynthesis

Scalable Deep Generative Modeling for Sparse Graphs

1 code implementation ICML 2020 Hanjun Dai, Azade Nazi, Yujia Li, Bo Dai, Dale Schuurmans

Based on this, we develop a novel autoregressive model, named BiGG, that utilizes this sparsity to avoid generating the full adjacency matrix, and importantly reduces the graph generation time complexity to $O((n + m)\log n)$.

Graph Generation

Learning to Stop While Learning to Predict

1 code implementation ICML 2020 Xinshi Chen, Hanjun Dai, Yu Li, Xin Gao, Le Song

Similar to algorithms, the optimal depth of a deep architecture may be different for different input instances, either to avoid ``over-thinking'', or because we want to compute less for operations converged already.



1 code implementation ICLR 2020 Elizabeth Dinella, Hanjun Dai, Ziyang Li, Mayur Naik, Le Song, Ke Wang

We present a learning-based approach to detect and fix a broad range of bugs in Javascript programs.

Energy-Based Processes for Exchangeable Data

1 code implementation ICML 2020 Mengjiao Yang, Bo Dai, Hanjun Dai, Dale Schuurmans

Recently there has been growing interest in modeling sets with exchangeability such as point clouds.

Denoising Point Cloud Generation

Differentiable Top-k Operator with Optimal Transport

no code implementations16 Feb 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

The top-k operation, i. e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

Learning Transferable Graph Exploration

no code implementations NeurIPS 2019 Hanjun Dai, Yujia Li, Chenglong Wang, Rishabh Singh, Po-Sen Huang, Pushmeet Kohli

We propose a `learning to explore' framework where we learn a policy from a distribution of environments.

Efficient Exploration

Cooperative neural networks (CoNN): Exploiting prior independence structure for improved classification

no code implementations NeurIPS 2018 Harsh Shrivastava, Eugene Bart, Bob Price, Hanjun Dai, Bo Dai, Srinivas Aluru

We propose a new approach, called cooperative neural networks (CoNN), which uses a set of cooperatively trained neural networks to capture latent representations that exploit prior given independence structure.

General Classification text-classification +1

Learning a Meta-Solver for Syntax-Guided Program Synthesis

no code implementations ICLR 2019 Xujie Si, Yuan Yang, Hanjun Dai, Mayur Naik, Le Song

Our framework consists of three components: 1) an encoder, which embeds both the logical specification and grammar at the same time using a graph neural network; 2) a grammar adaptive policy network which enables learning a transferable policy; and 3) a reinforcement learning algorithm that jointly trains the specification and grammar embedding and adaptive policy.

Meta-Learning Program Synthesis

Exponential Family Estimation via Adversarial Dynamics Embedding

1 code implementation NeurIPS 2019 Bo Dai, Zhen Liu, Hanjun Dai, Niao He, Arthur Gretton, Le Song, Dale Schuurmans

We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks.

Particle Flow Bayes' Rule

2 code implementations2 Feb 2019 Xinshi Chen, Hanjun Dai, Le Song

We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation.

Bayesian Inference Meta-Learning +1

CompILE: Compositional Imitation Learning and Execution

3 code implementations4 Dec 2018 Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Alvaro Sanchez-Gonzalez, Edward Grefenstette, Pushmeet Kohli, Peter Battaglia

We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data.

Continuous Control Imitation Learning

Coupled Variational Bayes via Optimization Embedding

1 code implementation NeurIPS 2018 Bo Dai, Hanjun Dai, Niao He, Weiyang Liu, Zhen Liu, Jianshu Chen, Lin Xiao, Le Song

This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution.

Variational Inference

Kernel Exponential Family Estimation via Doubly Dual Embedding

1 code implementation6 Nov 2018 Bo Dai, Hanjun Dai, Arthur Gretton, Le Song, Dale Schuurmans, Niao He

We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space.

Learning Steady-States of Iterative Algorithms over Graphs

no code implementations ICML 2018 Hanjun Dai, Zornitsa Kozareva, Bo Dai, Alex Smola, Le Song

Many graph analytics problems can be solved via iterative algorithms where the solutions are often characterized by a set of steady-state conditions.

Variational Reasoning for Question Answering with Knowledge Graph

1 code implementation12 Sep 2017 Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J. Smola, Le Song

Knowledge graph (KG) is known to be helpful for the task of question answering (QA), since it provides well-structured relational information between entities, and allows one to further infer indirect facts.

Knowledge Graphs Question Answering +1

Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs

2 code implementations ICML 2017 Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song

The occurrence of a fact (edge) is modeled as a multivariate point process whose intensity function is modulated by the score for that fact computed based on the learned entity embeddings.

Entity Embeddings Knowledge Graphs +1

Learning Combinatorial Optimization Algorithms over Graphs

8 code implementations NeurIPS 2017 Hanjun Dai, Elias B. Khalil, Yuyu Zhang, Bistra Dilkina, Le Song

The design of good heuristics or approximation algorithms for NP-hard combinatorial optimization problems often requires significant specialized knowledge and trial-and-error.

Combinatorial Optimization Graph Embedding

Deep Coevolutionary Network: Embedding User and Item Features for Recommendation

no code implementations13 Sep 2016 Hanjun Dai, Yichen Wang, Rakshit Trivedi, Le Song

DeepCoevolve use recurrent neural network (RNN) over evolving networks to define the intensity function in point processes, which allows the model to capture complex mutual influence between users and items, and the feature evolution over time.

Activity Prediction Network Embedding +2

Discriminative Embeddings of Latent Variable Models for Structured Data

1 code implementation17 Mar 2016 Hanjun Dai, Bo Dai, Le Song

Kernel classifiers and regressors designed for structured data, such as sequences, trees and graphs, have significantly advanced a number of interdisciplinary areas such as computational biology and drug design.

M-Statistic for Kernel Change-Point Detection

no code implementations NeurIPS 2015 Shuang Li, Yao Xie, Hanjun Dai, Le Song

Detecting the emergence of an abrupt change-point is a classic problem in statistics and machine learning.

Change Point Detection

Online Supervised Subspace Tracking

no code implementations1 Sep 2015 Yao Xie, Ruiyang Song, Hanjun Dai, Qingbin Li, Le Song

The optimization problem for OSDR is non-convex and hard to analyze in general; we provide convergence analysis of OSDR in a simple linear regression setting.

Dimensionality Reduction regression +2

Scan $B$-Statistic for Kernel Change-Point Detection

no code implementations5 Jul 2015 Shuang Li, Yao Xie, Hanjun Dai, Le Song

A novel theoretical result of the paper is the characterization of the tail probability of these statistics using the change-of-measure technique, which focuses on characterizing the tail of the detection statistics rather than obtaining its asymptotic distribution under the null distribution.

Change Point Detection

Provable Bayesian Inference via Particle Mirror Descent

no code implementations9 Jun 2015 Bo Dai, Niao He, Hanjun Dai, Le Song

Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters.

Bayesian Inference Gaussian Processes

Cannot find the paper you are looking for? You can Submit a new open access paper.