Search Results for author: Xinyun Chen

Found 54 papers, 27 papers with code

Learning to Progressively Plan

no code implementations ICLR 2019 Xinyun Chen, Yuandong Tian

For problem solving, making reactive decisions based on problem description is fast but inaccurate, while search-based planning using heuristics gives better solutions but could be exponentially slow.

reinforcement-learning Reinforcement Learning (RL) +1

BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual Pragmatic Knowledge

1 code implementation31 Aug 2023 Xiangru Tang, Bill Qian, Rick Gao, Jiakang Chen, Xinyun Chen, Mark Gerstein

BioCoder incorporates a fuzz-testing framework for evaluation, and we have applied it to evaluate many models including InCoder, CodeGen, CodeGen2, SantaCoder, StarCoder, StarCoder+, InstructCodeT5+, and ChatGPT.

Code Generation

Large Language Models as Tool Makers

1 code implementation26 May 2023 Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou

Our approach consists of two key phases: 1) tool making: an LLM acts as the tool maker that crafts tools for given tasks, where a tool is implemented as a Python utility function.

Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models

no code implementations24 May 2023 Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou

Sparse Mixture-of-Experts (MoE) is a neural architecture design that can be utilized to add learnable parameters to Large Language Models (LLMs) without increasing inference cost.

Symbol tuning improves in-context learning in language models

no code implementations15 May 2023 Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc V. Le

We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e. g., "positive/negative sentiment") are replaced with arbitrary symbols (e. g., "foo/bar").

Teaching Large Language Models to Self-Debug

no code implementations11 Apr 2023 Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou

In particular, we demonstrate that Self-Debugging can teach the large language model to perform rubber duck debugging; i. e., without any feedback on the code correctness or error messages, the model is able to identify its mistakes by explaining the generated code in natural language.

Code Generation Language Modelling +3

DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection

1 code implementation1 Apr 2023 Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, David Wagner

Combining our new dataset with previous datasets, we present an analysis of the challenges and promising research directions of using deep learning for detecting software vulnerabilities.

Feature Engineering Vulnerability Detection

Larger language models do in-context learning differently

no code implementations7 Mar 2023 Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, Tengyu Ma

We next study semantically-unrelated label ICL (SUL-ICL), in which labels are semantically unrelated to their inputs (e. g., foo/bar instead of negative/positive), thereby forcing language models to learn the input-label mappings shown in in-context exemplars in order to perform the task.

Online Learning and Optimization for Queues with Unknown Demand Curve and Service Distribution

no code implementations6 Mar 2023 Xinyun Chen, Yunan Liu, Guiyu Hong

A major drawback of PTO is that its solution accuracy can often be highly sensitive to the parameter estimation errors because PTO is unable to properly link these errors (step 1) to the quality of the optimized solutions (step 2).

Large Language Models Can Be Easily Distracted by Irrelevant Context

1 code implementation31 Jan 2023 Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, Denny Zhou

We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included.

Arithmetic Reasoning Language Modelling

Data-pooling Reinforcement Learning for Personalized Healthcare Intervention

no code implementations16 Nov 2022 Xinyun Chen, Pengyi Shi, Shanwen Pu

Motivated by the emerging needs of personalized preventative intervention in many healthcare applications, we consider a multi-stage, dynamic decision-making problem in the online setting with unknown model parameters.

Decision Making Management +2

Benchmarking Language Models for Code Syntax Understanding

1 code implementation26 Oct 2022 Da Shen, Xinyun Chen, Chenguang Wang, Koushik Sen, Dawn Song

Our key observation is that existing language models pretrained on code still lack the understanding of code syntax.


Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment

1 code implementation Findings (NAACL) 2022 Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver

To deal with this problem, we modify a number of state-of-the-art models to train on the segmented data of Spider-SS, and we show that this method improves the generalization performance.


Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations

1 code implementation16 Dec 2021 Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao

With the forced decomposition, we show that the performance upper bounds of LSTM and Transformer in learning CFL are close: both of them can simulate a stack and perform stack operation along with state transitions.

SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs

1 code implementation28 Oct 2021 Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Denny Zhou, Jure Leskovec, Dale Schuurmans

There are two important reasoning tasks on KGs: (1) single-hop knowledge graph completion, which involves predicting individual links in the KG; and (2), multi-hop reasoning, where the goal is to predict which KG entities satisfy a given logical query.


Natural SQL: Making SQL Easier to Infer from Natural Language Specifications

1 code implementation Findings (EMNLP) 2021 Yujian Gan, Xinyun Chen, Jinxia Xie, Matthew Purver, John R. Woodward, John Drake, Qiaofu Zhang

Addressing the mismatch between natural language descriptions and the corresponding SQL queries is a key challenge for text-to-SQL translation.

Text-To-SQL Translation

Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization

1 code implementation EMNLP 2021 Yujian Gan, Xinyun Chen, Matthew Purver

Recently, there has been significant progress in studying neural networks for translating text descriptions into SQL queries under the zero-shot cross-domain setting.

Text-To-SQL Translation

RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

1 code implementation11 Sep 2021 Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H. S. Torr, DaCheng Tao

Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).

Adversarial Robustness Benchmarking +2

Latent Execution for Neural Program Synthesis

1 code implementation NeurIPS 2021 Xinyun Chen, Dawn Song, Yuandong Tian

While recent works demonstrated limited success on domain-specific languages (DSL), it remains highly challenging to apply them to real-world programming languages, such as C. Due to complicated syntax and token variation, there are three major challenges: (1) unlike many DSLs, programs in languages like C need to compile first and are not executed via interpreters; (2) the program search space grows exponentially when the syntax and semantics of the programming language become more complex; and (3) collecting a large-scale dataset of real-world programs is non-trivial.

Program Synthesis

SpreadsheetCoder: Formula Prediction from Semi-structured Context

1 code implementation26 Jun 2021 Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou

In this work, we present the first approach for synthesizing spreadsheet formulas from tabular context, which includes both headers and semi-structured tabular data.

Program Synthesis

Understanding the Spread of COVID-19 Epidemic: A Spatio-Temporal Point Process View

no code implementations24 Jun 2021 Shuang Li, Lu Wang, Xinyun Chen, Yixiang Fang, Yan Song

In this paper, we model the propagation of the COVID-19 as spatio-temporal point processes and propose a generative and intensity-free model to track the spread of the disease.

Imitation Learning Point Processes

Towards Robustness of Text-to-SQL Models against Synonym Substitution

1 code implementation ACL 2021 Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang

We observe that the accuracy dramatically drops by eliminating such explicit correspondence between NL questions and table schemas, even if the synonyms are not adversarially selected to conduct worst-case adversarial attacks.


Understanding Robustness in Teacher-Student Setting: A New Perspective

no code implementations25 Feb 2021 Zhuolin Yang, Zhaoxi Chen, Tiffany Cai, Xinyun Chen, Bo Li, Yuandong Tian

Extensive experiments show that student specialization correlates strongly with model robustness in different scenarios, including student trained via standard training, adversarial training, confidence-calibrated adversarial training, and training with robust feature dataset.

BIG-bench Machine Learning Data Augmentation

Perturbation Type Categorization for Multiple $\ell_p$ Bounded Adversarial Robustness

no code implementations1 Jan 2021 Pratyush Maini, Xinyun Chen, Bo Li, Dawn Song

In addition, we demonstrate the realization of this trade-off in deep networks by adding random noise to the model input at test time, enabling enhanced robustness against strong adaptive attacks.

Adversarial Robustness Vocal Bursts Type Prediction

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

no code implementations18 Dec 2020 Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance.

BIG-bench Machine Learning Data Poisoning

Towards Defending Multiple $\ell_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization

1 code implementation3 Dec 2020 Aishan Liu, Shiyu Tang, Xinyun Chen, Lei Huang, Haotong Qin, Xianglong Liu, DaCheng Tao

In this paper, we observe that different $\ell_p$ bounded adversarial perturbations induce different statistical properties that can be separated and characterized by the statistics of Batch Normalization (BN).

An online learning approach to dynamic pricing and capacity sizing in service systems

no code implementations7 Sep 2020 Xinyun Chen, Yunan Liu, Guiyu Hong

In this work we propose an online learning framework designed for solving this problem which does not require the system's scale to increase.

Compositional Generalization via Neural-Symbolic Stack Machines

no code implementations NeurIPS 2020 Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, Denny Zhou

Despite achieving tremendous success, existing deep learning models have exposed limitations in compositional generalization, the capability to learn compositional rules and apply them to unseen cases in a systematic manner.

Few-Shot Learning Machine Translation +1

Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis

1 code implementation NeurIPS 2020 Kavi Gupta, Peter Ebert Christensen, Xinyun Chen, Dawn Song

The use of deep learning techniques has achieved significant progress for program synthesis from input-output examples.

Program Synthesis

Spatiotemporal Attacks for Embodied Agents

1 code implementation ECCV 2020 Aishan Liu, Tairan Huang, Xianglong Liu, Yitao Xu, Yuqing Ma, Xinyun Chen, Stephen J. Maybank, DaCheng Tao

Adversarial attacks are valuable for providing insights into the blind-spots of deep learning models and help improve their robustness.


Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension

no code implementations ICLR 2020 Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, Quoc V. Le

Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches are hard to scale to more domains or more complex reasoning.

Data Augmentation Question Answering +1

REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data

1 code implementation17 Nov 2019 Xinyun Chen, Wenxiao Wang, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, Dawn Song

The experimental results demonstrate that our fine-tuning based watermark removal attacks could pose real threats to the copyright of pre-trained models, and thus highlight the importance of further investigating the watermarking problem and proposing more robust watermark embedding schemes against the attacks.

Infinite-horizon Off-Policy Policy Evaluation with Multiple Behavior Policies

no code implementations ICLR 2020 Xinyun Chen, Lu Wang, Yizhe Hang, Heng Ge, Hongyuan Zha

We consider off-policy policy evaluation when the trajectory data are generated by multiple behavior policies.

A Neural-based Program Decompiler

no code implementations28 Jun 2019 Cheng Fu, Huili Chen, Haolan Liu, Xinyun Chen, Yuandong Tian, Farinaz Koushanfar, Jishen Zhao

Reverse engineering of binary executables is a critical problem in the computer security domain.

Computer Security Malware Detection

Execution-Guided Neural Program Synthesis

no code implementations ICLR 2019 Xinyun Chen, Chang Liu, Dawn Song

Most existing neural program synthesis approaches employ an encoder-decoder architecture, which uses an encoder to compute the embedding of the given input-output examples, as well as a decoder to generate the program from the embedding following a given syntax.

Program Synthesis

Tree-to-tree Neural Networks for Program Translation

no code implementations ICLR 2018 Xinyun Chen, Chang Liu, Dawn Song

We observe that program translation is a modular procedure, in which a sub-tree of the source tree is translated into the corresponding target sub-tree at each step.


Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

1 code implementation15 Dec 2017 Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, Dawn Song

In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor.

Data Poisoning Face Recognition

Fooling Vision and Language Models Despite Localization and Attention Mechanism

no code implementations CVPR 2018 Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, Dawn Song

Our work sheds new light on understanding adversarial attacks on vision systems which have a language component and shows that attention, bounding box localization, and compositional internal structures are vulnerable to adversarial attacks.

Dense Captioning Natural Language Understanding +2

Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong

no code implementations15 Jun 2017 Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song

We ask whether a strong defense can be created by combining multiple (possibly weak) defenses.

Towards Synthesizing Complex Programs from Input-Output Examples

no code implementations ICLR 2018 Xinyun Chen, Chang Liu, Dawn Song

In our evaluation, we show that using our novel approach, neural parsing programs can be learned to achieve 100% test accuracy on test inputs that are 500x longer than the training samples.

Program Synthesis reinforcement-learning +1

Delving into Transferable Adversarial Examples and Black-box Attacks

1 code implementation8 Nov 2016 Yanpei Liu, Xinyun Chen, Chang Liu, Dawn Song

In this work, we are the first to conduct an extensive study of the transferability over large models and a large scale dataset, and we are also the first to study the transferability of targeted adversarial examples with their target labels.

Adversarial Attack Adversarial Defense +1

A General Retraining Framework for Scalable Adversarial Classification

no code implementations9 Apr 2016 Bo Li, Yevgeniy Vorobeychik, Xinyun Chen

We propose the first systematic and general-purpose retraining framework which can: a) boost robustness of an \emph{arbitrary} learning algorithm, in the face of b) a broader class of adversarial models than any prior methods.

Classification General Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.