Search Results for author:
Found 25 papers, 9 papers with code
Structure-Aware Fill-in-the-Middle Pretraining for Code
Fill-in-the-Middle (FIM) is a common pretraining method for code LLMs, where models complete code segments given surrounding context.
What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Traces
Code generation and understanding are critical capabilities for large language models (LLMs).
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Real-world enterprise text-to-SQL workflows often involve complex cloud or local data across various database systems, multiple SQL queries in various dialects, and diverse operations from data transformation to analytics.
Ranked #1 on
Text-To-SQL
on Spider 2.0
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
These tasks, derived from real-world use cases, evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry.
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
We introduce Syntax-Aware Fill-In-the-Middle (SAFIM), a new benchmark for evaluating Large Language Models (LLMs) on the code Fill-in-the-Middle (FIM) task.
Ranked #1 on
Code Completion
on SAFIM
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
We introduce DS-1000, a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas.
On Continual Model Refinement in Out-of-Distribution Data Streams
Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting.
InCoder: A Generative Model for Code Infilling and Synthesis
Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming.
Ranked #95 on
Code Generation
on MBPP
SILG: The Multi-domain Symbolic Interactive Language Grounding Benchmark
We hope SILG enables the community to quickly identify new methodolo- gies for language grounding that generalize to a diverse set of environments and their associated challenges.
Deep Natural Language Processing for LinkedIn Search
Building a successful search system requires a thorough understanding of textual data semantics, where deep learning based natural language processing techniques (deep NLP) can be of great help.
Deep Natural Language Processing for LinkedIn Search Systems
Many search systems work with large amounts of natural language data, e. g., search queries, user profiles and documents, where deep learning based natural language processing techniques (deep NLP) can be of great help.
Towards Understanding the Behaviors of Optimal Deep Active Learning Algorithms
Active learning (AL) algorithms may achieve better performance with fewer data because the model guides the data selection process.
Efficient Neural Query Auto Completion
On the candidate generation side, this system uses as much information as possible in unseen prefixes to generate relevant candidates, increasing the recall by a large margin.
DeText: A Deep Text Ranking Framework with BERT
Ranking is the most important component in a search system.
Memory-efficient Embedding for Recommendations
Specifically, we first proposed an end-to-end differentiable framework that can calculate the weights over various dimensions for feature fields in a soft and continuous manner with an AutoML based optimization algorithm; then we derive a hard and discrete embedding component architecture according to the maximal weights and retrain the whole recommender framework.
Pre-training via Paraphrasing
The objective noisily captures aspects of paraphrase, translation, multi-document summarization, and information retrieval, allowing for strong zero-shot performance on several tasks.
Interactive Classification by Asking Informative Questions
We study the potential for interaction in natural language classification.
Simple MAP Inference via Low-Rank Relaxations
We focus on the problem of maximum a posteriori (MAP) inference in Markov random fields with binary variables and pairwise interactions.
Altitude Training: Strong Bounds for Single-Layer Dropout
Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks.
Dropout Training as Adaptive Regularization
Dropout and other feature noising schemes control overfitting by artificially corrupting the training data.
