Search Results for author:
Found 40 papers, 24 papers with code
In-Context Learning for Text Classification with Many Labels
We analyze the performance across number of in-context examples and different model scales, showing that larger models are necessary to effectively and consistently make use of larger context lengths for ICL.
RepoFusion: Training Code Models to Understand Your Repository
We find these results to be a novel and compelling demonstration of the gains that training with repository context can bring.
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15. 5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention.
SantaCoder: don't reach for the stars!
The BigCode project is an open-scientific collaboration working on the responsible development of large language models for code.
Ranked #20 on
Code Generation
on HumanEval
(Pass@100 metric)
The Stack: 3 TB of permissively licensed source code
Large Language Models (LLMs) play an ever-increasing role in the field of Artificial Intelligence (AI)--not only for natural language processing but also for code understanding and generation.
On the Compositional Generalization Gap of In-Context Learning
In this work, we look at the gap between the in-distribution (ID) and out-of-distribution (OOD) performance of such models in semantic parsing tasks with in-context learning.
LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing
In this work, we show that better systematic generalization can be achieved by producing the meaning representation directly as a graph and not as a sequence.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Data augmentation is a widely employed technique to alleviate the problem of data scarcity.
Evaluating the Text-to-SQL Capabilities of Large Language Models
We perform an empirical evaluation of Text-to-SQL capabilities of the Codex language model.
Systematic Generalization with Edge Transformers
Recent research suggests that systematic generalization in natural language understanding remains a challenge for state-of-the-art neural models such as Transformers and Graph Neural Networks.
LAGr: Labeling Aligned Graphs for Improving Systematic Generalization in Semantic Parsing
In this work, we show that better systematic generalization can be achieved by producing the meaning representation (MR) directly as a graph and not as a sequence.
Compositional Generalization in Dependency Parsing
To test compositional generalization in semantic parsing, Keysers et al. (2020) introduced Compositional Freebase Queries (CFQ).
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
Large pre-trained language models for textual data have an unconstrained output space; at each decoding step, they can produce any of 10, 000s of sub-word tokens.
Ranked #1 on
Text-To-SQL
on SPIDER
Understanding by Understanding Not: Modeling Negation in Language Models
To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus.
Jointly Learning Truth-Conditional Denotations and Groundings using Parallel Attention
We present a model that jointly learns the denotations of words together with their groundings using a truth-conditional semantics.
DuoRAT: Towards Simpler Text-to-SQL Models
Recent neural text-to-SQL models can effectively translate natural language questions to corresponding SQL queries on unseen databases.
Towards Ecologically Valid Research on Language User Interfaces
To this end, we describe what we deem an ideal methodology for machine learning research on LUIs and categorize five common ways in which recent benchmarks deviate from it.
BabyAI 1.1
This increases reinforcement learning sample efficiency by up to 3 times and improves imitation learning performance on the hardest level from 77 % to 90. 4 %.
Combating False Negatives in Adversarial Imitation Learning
In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior.
CLOSURE: Assessing Systematic Generalization of CLEVR Models
In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.
Automated curriculum generation for Policy Gradients from Demonstrations
In this paper, we present a technique that improves the process of training an agent (using RL) for instruction following.
Systematic Generalization: What Is Required and Can It Be Learned?
Numerous models for grounded language understanding have been recently proposed, including (i) generic models that can be easily adapted to any given task and (ii) intuitively appealing modular models that require background knowledge to be instantiated.
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons, but given the poor data efficiency of the current learning methods, this goal may require substantial research efforts.
Learning to Understand Goal Specifications by Modelling Reward
Recent work has shown that deep reinforcement-learning agents can learn to follow language-like instructions from infrequent environment rewards.
Commonsense mining as knowledge base completion? A study on the impact of novelty
Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples.
Learning to Compute Word Embeddings On the Fly
Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare.
Ranked #48 on
Question Answering
on SQuAD1.1 dev
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity.
An Actor-Critic Algorithm for Sequence Prediction
We present an approach to training neural networks to generate sequences using actor-critic methods from reinforcement learning (RL).
Ranked #8 on
Machine Translation
on IWSLT2015 English-German
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Task Loss Estimation for Sequence Prediction
Our idea is that this score can be interpreted as an estimate of the task loss, and that the estimation error may be used as a consistent surrogate loss.
End-to-End Attention-based Large Vocabulary Speech Recognition
Many of the current state-of-the-art Large Vocabulary Continuous Speech Recognition Systems (LVCSR) are hybrids of neural networks and Hidden Markov Models (HMMs).
Attention-Based Models for Speech Recognition
Recurrent sequence generators conditioned on input data through an attention mechanism have recently shown very good performance on a range of tasks in- cluding machine translation, handwriting synthesis and image caption gen- eration.
Ranked #17 on
Speech Recognition
on TIMIT
Blocks and Fuel: Frameworks for deep learning
We introduce two Python frameworks to train neural networks on large datasets: Blocks and Fuel.
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes.
Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation
The authors of (Cho et al., 2014a) have shown that the recently introduced neural network translation systems suffer from a significant drop in translation quality when translating long sentences, unlike existing phrase-based translation systems.
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network.
Neural Machine Translation by Jointly Learning to Align and Translate
Neural machine translation is a recently proposed approach to machine translation.
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN).
Ranked #47 on
Machine Translation
on WMT2014 English-French
