Search Results for author:
Found 44 papers, 16 papers with code
Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs
Identifying how much a model ${\widehat{p}}_{\theta}(Y|X)$ knows about the stochastic real-world process $p(Y|X)$ it was trained on is important to ensure it avoids producing incorrect or "hallucinated" answers or taking unsafe actions.
R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents
Large language models show impressive results at predicting structured text such as code, but also commonly introduce errors and hallucinations in their output.
A Library for Representing Python Programs as Graphs for Machine Learning
Graph representations of programs are commonly a central element of machine learning for code research.
Learning to Improve Code Efficiency
To automatically learn these hints from the dataset, we propose a novel discrete variational auto-encoder, where each discrete latent variable represents a different learned category of code-edit that increases performance.
Repository-Level Prompt Generation for Large Language Models of Code
With the success of large language models (LLMs) of code and their use as code assistants (e. g. Codex used in GitHub Copilot), techniques for introducing domain-specific knowledge in the prompt design process become important.
Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptions
This presents an interesting machine learning challenge: can we predict runtime errors in a "static" setting, where program execution is not possible?
PLUR: A Unifying, Graph-Based View of Program Learning, Understanding, and Repair
Machine learning for understanding and editing source code has recently attracted significant interest, with many developments in new models, new code representations, and new tasks. This proliferation can appear disparate and disconnected, making each approach seemingly unique and incompatible, thus obscuring the core machine learning challenges and contributions. In this work, we demonstrate that the landscape can be significantly simplified by taking a general approach of mapping a graph to a sequence of tokens and pointers. Our main result is to show that 16 recently published tasks of different shapes can be cast in this form, based on which a single model architecture achieves near or above state-of-the-art results on nearly all tasks, outperforming custom models like code2seq and alternative generic models like Transformers. This unification further enables multi-task learning and a series of cross-cutting experiments about the importance of different modeling choices for code understanding and repair tasks. The full framework, called PLUR, is easily extensible to more tasks, and will be open-sourced (https://github. com/google-research/plur).
Learning Generalized Gumbel-max Causal Mechanisms
To perform counterfactual reasoning in Structural Causal Models (SCMs), one needs to know the causal mechanisms, which provide factorizations of conditional distributions into noise sources and deterministic functions mapping realizations of noise to samples.
Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models
Denoising diffusion probabilistic models (DDPMs) have shown impressive results on sequence generation by iteratively corrupting each example and then learning to map corrupted versions back to the original.
Structured Denoising Diffusion Models in Discrete State-Spaces
Here, we introduce Discrete Denoising Diffusion Probabilistic Models (D3PMs), diffusion-like generative models for discrete data that generalize the multinomial diffusion model of Hoogeboom et al. 2021, by going beyond corruption processes with uniform transition probabilities.
Learning to Combine Per-Example Solutions for Neural Program Synthesis
Most learning-based approaches try to find a program that satisfies all examples at once.
Learning to Extend Program Graphs to Work-in-Progress Code
Source code spends most of its time in a broken or incomplete state during software development.
Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks
More practically, we evaluate these models on the task of learning to execute partial programs, as might arise if using the model as a heuristic function in program synthesis.
On-the-Fly Adaptation of Source Code Models
The ability to adapt to unseen, local contexts is an important challenge that successful models of source code must overcome.
Learning Graph Structure With A Finite-State Automaton Layer
In practice, edges are used both to represent intrinsic structure (e. g., abstract syntax trees of programs) and more abstract relations that aid reasoning for a downstream task (e. g., results of relevant program analyses).
Software Engineering Event Modeling using Relative Time in Temporal Knowledge Graphs
We present a multi-relational temporal Knowledge Graph based on the daily interactions between artifacts in GitHub, one of the largest social coding platforms.
Gradient Estimation with Stochastic Softmax Tricks
The Gumbel-Max trick is the basis of many relaxed gradient estimators.
On-the-Fly Adaptation of Source Code Models using Meta-Learning
The ability to adapt to unseen, local contexts is an important challenge that successful models of source code must overcome.
Learning to Fix Build Errors with Graph2Diff Neural Networks
A diff specifies how to modify the code's abstract syntax tree, represented in the neural network as a sequence of tokens and of pointers to code locations.
Fast Training of Sparse Graph Neural Networks on Dense Hardware
Graph neural networks have become increasingly popular in recent years due to their ability to naturally encode relational input data and their ability to scale to large graphs by operating on a sparse representation of graph adjacency matrices.
Learning Execution through Neural Code Fusion
In this work, we propose a new approach to use GNNs to learn fused representations of general source code and its execution.
Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spaces
A main benefit of DirPG algorithms is that they allow the insertion of domain knowledge in the form of upper bounds on return-to-go at training time, like is used in heuristic search, while still directly computing a policy gradient.
Neural Networks for Modeling Source Code Edits
In this work, we instead treat source code as a dynamic object and tackle the problem of modeling the edits that software developers make to source code files.
Graph Partition Neural Networks for Semi-Supervised Classification
We present graph partition neural networks (GPNN), an extension of graph neural networks (GNNs) able to handle extremely large graphs.
AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
Through an implementation on multi-core CPUs, we show that AMP training converges to the same accuracy as conventional synchronous training algorithms in a similar number of epochs, but utilizes the available hardware more efficiently even for small minibatch sizes, resulting in significantly shorter overall training times.
Batch Policy Gradient Methods for Improving Neural Conversation Models
We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain.
Summary - TerpreT: A Probabilistic Programming Language for Program Induction
A TerpreT model is composed of a specification of a program representation and an interpreter that describes how programs map inputs to outputs.
Differentiable Programs with Neural Libraries
We develop a framework for combining differentiable programming languages with neural networks.
Differentiable Functional Program Interpreters
Recent work on differentiable interpreters relaxes the discrete space of programs into a continuous space so that search over programs can be performed using gradient-based optimization.
DeepCoder: Learning to Write Programs
We develop a first line of attack for solving programming competition-style problems from input-output examples using deep learning.
TerpreT: A Probabilistic Programming Language for Program Induction
TerpreT is similar to a probabilistic programming language: a model is composed of a specification of a program representation (declarations of random variables) and an interpreter describing how programs map inputs to outputs (a model connecting unknowns to observations).
Fits Like a Glove: Rapid and Reliable Hand Shape Personalization
We present a fast, practical method for personalizing a hand shape basis to an individual user's detailed hand shape using only a small set of depth images.
Gated Graph Sequence Neural Networks
Graph-structured data appears frequently in domains including chemistry, natural language semantics, social networks, and knowledge bases.
Ranked #1 on
Graph Classification
on IPC-grounded
Candidate Constrained CRFs for Loss-Aware Structured Prediction
The result is that we can use loss-aware prediction methodology to improve performance of the highly tuned pipeline system.
Just-In-Time Learning for Fast and Flexible Inference
Much of research in machine learning has centered around the search for inference algorithms that are both general-purpose and efficient.
A* Sampling
The problem of drawing samples from a discrete distribution can be converted into a discrete optimization problem.
Consensus Message Passing for Layered Graphical Models
Generative models provide a powerful framework for probabilistic reasoning.
Empirical Minimum Bayes Risk Prediction: How to Extract an Extra Few % Performance from Vision Models with Just Three More Parameters
When building vision systems that predict structured objects such as image segmentations or human poses, a crucial concern is performance under task-specific evaluation measures (e. g. Jaccard Index or Average Precision).
Structured Generative Models of Natural Source Code
We study the problem of building generative models of natural source code (NSC); that is, source code written and understood by humans.
Detecting Parameter Symmetries in Probabilistic Models
Probabilistic models often have parameters that can be translated, scaled, permuted, or otherwise transformed without changing the model.
Learning to Pass Expectation Propagation Messages
Expectation Propagation (EP) is a popular approximate posterior inference algorithm that often provides a fast and accurate alternative to sampling-based methods.
Tighter Linear Program Relaxations for High Order Graphical Models
In this work, we study the LP relaxations that result from enforcing additional consistency constraints between the HOP and the rest of the model.
Exploring Compositional High Order Pattern Potentials for Structured Output Learning
In this work, we study the learning of a general class of pattern-like high order potential, which we call Compositional High Order Pattern Potentials (CHOPPs).
Cardinality Restricted Boltzmann Machines
The Restricted Boltzmann Machine (RBM) is a popular density model that is also good for extracting features.
