Search Results for author:
Found 8 papers, 2 papers with code
Fewer Truncations Improve Language Modeling
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens.
Bifurcated Attention for Single-Context Large-Batch Sampling
In our study, we present bifurcated attention, a method developed for language model inference in single-context batch sampling contexts.
Code Representation Learning At Scale
Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i. e., code generation.
Exploring Continual Learning for Code Generation Models
Large-scale code generation models such as Codex and CodeT5 have achieved impressive performance.
Towards Open-Domain Topic Classification
We introduce an open-domain topic classification system that accepts user-defined taxonomy in real time.
A Static Evaluation of Code Completion by Large Language Models
Large language models trained on code have shown great potential to increase productivity of software developers.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
SP-10K: A Large-scale Evaluation Set for Selectional Preference Acquisition
Selectional Preference (SP) is a commonly observed language phenomenon and proved to be useful in many natural language processing tasks.
