Search Results for author:
Found 54 papers, 27 papers with code
Learning to Progressively Plan
For problem solving, making reactive decisions based on problem description is fast but inaccurate, while search-based planning using heuristics gives better solutions but could be exponentially slow.
BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual Pragmatic Knowledge
BioCoder incorporates a fuzz-testing framework for evaluation, and we have applied it to evaluate many models including InCoder, CodeGen, CodeGen2, SantaCoder, StarCoder, StarCoder+, InstructCodeT5+, and ChatGPT.
Large Language Models as Tool Makers
Our approach consists of two key phases: 1) tool making: an LLM acts as the tool maker that crafts tools for given tasks, where a tool is implemented as a Python utility function.
Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
Sparse Mixture-of-Experts (MoE) is a neural architecture design that can be utilized to add learnable parameters to Large Language Models (LLMs) without increasing inference cost.
Symbol tuning improves in-context learning in language models
We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e. g., "positive/negative sentiment") are replaced with arbitrary symbols (e. g., "foo/bar").
Teaching Large Language Models to Self-Debug
In particular, we demonstrate that Self-Debugging can teach the large language model to perform rubber duck debugging; i. e., without any feedback on the code correctness or error messages, the model is able to identify its mistakes by explaining the generated code in natural language.
DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection
Combining our new dataset with previous datasets, we present an analysis of the challenges and promising research directions of using deep learning for detecting software vulnerabilities.
Larger language models do in-context learning differently
We next study semantically-unrelated label ICL (SUL-ICL), in which labels are semantically unrelated to their inputs (e. g., foo/bar instead of negative/positive), thereby forcing language models to learn the input-label mappings shown in in-context exemplars in order to perform the task.
Online Learning and Optimization for Queues with Unknown Demand Curve and Service Distribution
A major drawback of PTO is that its solution accuracy can often be highly sensitive to the parameter estimation errors because PTO is unable to properly link these errors (step 1) to the quality of the optimized solutions (step 2).
Large Language Models Can Be Easily Distracted by Irrelevant Context
We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included.
Data-pooling Reinforcement Learning for Personalized Healthcare Intervention
Motivated by the emerging needs of personalized preventative intervention in many healthcare applications, we consider a multi-stage, dynamic decision-making problem in the online setting with unknown model parameters.
Benchmarking Language Models for Code Syntax Understanding
Our key observation is that existing language models pretrained on code still lack the understanding of code syntax.
Scaling Instruction-Finetuned Language Models
We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).
Ranked #1 on
Coreference Resolution
on WSC
Cross-Lingual Question Answering
Multi-task Language Understanding
+1
Compositional Semantic Parsing with Large Language Models
Humans can reason compositionally when presented with new tasks.
Ranked #1 on
Semantic Parsing
on CFQ
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment
To deal with this problem, we modify a number of state-of-the-art models to train on the segmented data of Spider-SS, and we show that this method improves the generalization performance.
Competition-Level Code Generation with AlphaCode
Programming is a powerful and ubiquitous problem-solving tool.
Ranked #1 on
Code Generation
on CodeContests
Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations
With the forced decomposition, we show that the performance upper bounds of LSTM and Transformer in learning CFL are close: both of them can simulate a stack and perform stack operation along with state transitions.
SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs
There are two important reasoning tasks on KGs: (1) single-hop knowledge graph completion, which involves predicting individual links in the KG; and (2), multi-hop reasoning, where the goal is to predict which KG entities satisfy a given logical query.
Natural SQL: Making SQL Easier to Infer from Natural Language Specifications
Addressing the mismatch between natural language descriptions and the corresponding SQL queries is a key challenge for text-to-SQL translation.
Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization
Recently, there has been significant progress in studying neural networks for translating text descriptions into SQL queries under the zero-shot cross-domain setting.
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context
Creating effective visualization is an important part of data analytics.
Latent Execution for Neural Program Synthesis
While recent works demonstrated limited success on domain-specific languages (DSL), it remains highly challenging to apply them to real-world programming languages, such as C. Due to complicated syntax and token variation, there are three major challenges: (1) unlike many DSLs, programs in languages like C need to compile first and are not executed via interpreters; (2) the program search space grows exponentially when the syntax and semantics of the programming language become more complex; and (3) collecting a large-scale dataset of real-world programs is non-trivial.
SpreadsheetCoder: Formula Prediction from Semi-structured Context
In this work, we present the first approach for synthesizing spreadsheet formulas from tabular context, which includes both headers and semi-structured tabular data.
Understanding the Spread of COVID-19 Epidemic: A Spatio-Temporal Point Process View
In this paper, we model the propagation of the COVID-19 as spatio-temporal point processes and propose a generative and intensity-free model to track the spread of the disease.
Towards Robustness of Text-to-SQL Models against Synonym Substitution
We observe that the accuracy dramatically drops by eliminating such explicit correspondence between NL questions and table schemas, even if the synonyms are not adversarially selected to conduct worst-case adversarial attacks.
Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages
Program synthesis from input-output (IO) examples has been a long-standing challenge.
A High-fidelity, Machine-learning Enhanced Queueing Network Simulation Model for Hospital Ultrasound Operations
To address the issue, we propose a novel two-level routing component to the queueing network model.
Understanding Robustness in Teacher-Student Setting: A New Perspective
Extensive experiments show that student specialization correlates strongly with model robustness in different scenarios, including student trained via standard training, adversarial training, confidence-calibrated adversarial training, and training with robust feature dataset.
N-Bref : A High-fidelity Decompiler Exploiting Programming Structures
In software development, decompilation aims to reverse engineer binary executables.
Perturbation Type Categorization for Multiple $\ell_p$ Bounded Adversarial Robustness
In addition, we demonstrate the realization of this trade-off in deep networks by adding random noise to the model input at test time, enabling enhanced robustness against strong adaptive attacks.
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance.
Towards Defending Multiple $\ell_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
In this paper, we observe that different $\ell_p$ bounded adversarial perturbations induce different statistical properties that can be separated and characterized by the statistics of Batch Normalization (BN).
An online learning approach to dynamic pricing and capacity sizing in service systems
In this work we propose an online learning framework designed for solving this problem which does not require the system's scale to increase.
Compositional Generalization via Neural-Symbolic Stack Machines
Despite achieving tremendous success, existing deep learning models have exposed limitations in compositional generalization, the capability to learn compositional rules and apply them to unseen cases in a systematic manner.
Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis
The use of deep learning techniques has achieved significant progress for program synthesis from input-output examples.
Spatiotemporal Attacks for Embodied Agents
Adversarial attacks are valuable for providing insights into the blind-spots of deep learning models and help improve their robustness.
Deep Symbolic Superoptimization Without Human Knowledge
Deep symbolic superoptimization refers to the task of applying deep learning methods to simplify symbolic expressions.
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches are hard to scale to more domains or more complex reasoning.
Ranked #5 on
Question Answering
on DROP Test
Coda: An End-to-End Neural Program Decompiler
Furthermore, Coda outperforms the sequence-to-sequence model with attention by a margin of 70% program accuracy.
REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data
The experimental results demonstrate that our fine-tuning based watermark removal attacks could pose real threats to the copyright of pre-trained models, and thus highlight the importance of further investigating the watermarking problem and proposing more robust watermark embedding schemes against the attacks.
Infinite-horizon Off-Policy Policy Evaluation with Multiple Behavior Policies
We consider off-policy policy evaluation when the trajectory data are generated by multiple behavior policies.
A Neural-based Program Decompiler
Reverse engineering of binary executables is a critical problem in the computer security domain.
Execution-Guided Neural Program Synthesis
Most existing neural program synthesis approaches employ an encoder-decoder architecture, which uses an encoder to compute the embedding of the given input-output examples, as well as a decoder to generate the program from the embedding following a given syntax.
Learning to Perform Local Rewriting for Combinatorial Optimization
Search-based methods for hard combinatorial optimization are often guided by heuristics.
Tree-to-tree Neural Networks for Program Translation
We observe that program translation is a modular procedure, in which a sub-tree of the source tree is translated into the corresponding target sub-tree at each step.
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor.
Fooling Vision and Language Models Despite Localization and Attention Mechanism
Our work sheds new light on understanding adversarial attacks on vision systems which have a language component and shows that attention, bounding box localization, and compositional internal structures are vulnerable to adversarial attacks.
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
We ask whether a strong defense can be created by combining multiple (possibly weak) defenses.
Towards Synthesizing Complex Programs from Input-Output Examples
In our evaluation, we show that using our novel approach, neural parsing programs can be learned to achieve 100% test accuracy on test inputs that are 500x longer than the training samples.
Delving into Transferable Adversarial Examples and Black-box Attacks
In this work, we are the first to conduct an extensive study of the transferability over large models and a large scale dataset, and we are also the first to study the transferability of targeted adversarial examples with their target labels.
Latent Attention For If-Then Program Synthesis
Automatic translation from natural language descriptions into programs is a longstanding challenging problem.
A General Retraining Framework for Scalable Adversarial Classification
We propose the first systematic and general-purpose retraining framework which can: a) boost robustness of an \emph{arbitrary} learning algorithm, in the face of b) a broader class of adversarial models than any prior methods.
