Search Results for author:
Found 27 papers, 16 papers with code
Towards Reproducible Machine Learning Research in Natural Language Processing
While recent progress in the field of ML has been significant, the reproducibility of these cutting-edge results is often lacking, with many submissions lacking the necessary information in order to ensure subsequent reproducibility.
Sometimes We Want Ungrammatical Translations
Rapid progress in Neural Machine Translation (NMT) systems over the last few years has focused primarily on improving translation quality, and as a secondary focus, improving robustness to perturbations (e. g. spelling).
Efficient Tool Use with Chain-of-Abstraction Reasoning
LLM agents trained with our method also show more efficient tool use, with inference speed being on average ~1. 4x faster than baseline tool-augmented LLMs.
The ART of LLM Refinement: Ask, Refine, and Trust
In recent years, Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations?
Ranked #12 on
Arithmetic Reasoning
on GSM8K
Language model acceptability judgements are not always robust to context
In this paper, we investigate the stability of language models' performance on targeted syntactic evaluations as we vary properties of the input context: the length of the context, the types of syntactic phenomena it contains, and whether or not there are violations of grammaticality.
The Curious Case of Absolute Position Embeddings
Transformer language models encode the notion of word order using positional information.
State-of-the-art generalisation research in NLP: A taxonomy and review
We present a taxonomy for characterising and understanding generalisation research in NLP.
How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts
Neural Machine Translation systems built on top of Transformer-based architectures are routinely improving the state-of-the-art in translation quality according to word-overlap metrics.
Evaluating Gender Bias in Natural Language Inference
Gender-bias stereotypes have recently raised significant ethical concerns in natural language processing.
Sometimes We Want Translationese
Rapid progress in Neural Machine Translation (NMT) systems over the last few years has been driven primarily towards improving translation quality, and as a secondary focus, improved robustness to input perturbations (e. g. spelling and grammatical mistakes).
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines.
COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction
The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0. 742 for predicting an adverse event within 96 hours (compared to 0. 703 with supervised pretraining) and an AUC of 0. 765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0. 749 with supervised pretraining).
GraphLog: A Benchmark for Measuring Logical Generalization in Graph Neural Networks
In this work, we study the logical generalization capabilities of GNNs by designing a benchmark suite grounded in first-order logic.
UnNatural Language Inference
We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i. e. they are largely invariant to random word-order permutations.
A Closer Look at Codistillation for Distributed Training
Surprisingly, we find that even at moderate batch sizes, models trained with codistillation can perform as well as models trained with synchronous data-parallel methods, despite using a much weaker synchronization mechanism.
Measuring Systematic Generalization in Neural Proof Generation with Transformers
We observe that models that are not trained to generate proofs are better at generalizing to problems based on longer proofs.
Ideas for Improving the Field of Machine Learning: Summarizing Discussion from the NeurIPS 2019 Retrospectives Workshop
This report documents ideas for improving the field of machine learning, which arose from discussions at the ML Retrospectives workshop at NeurIPS 2019.
Probing Linguistic Systematicity
Recently, there has been much interest in the question of whether deep natural language understanding models exhibit systematicity; generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear.
Learning an Unreferenced Metric for Online Dialogue Evaluation
Evaluating the quality of a dialogue interaction between two agents is a difficult task, especially in open-domain chit-chat style dialogue.
Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)
Reproducibility, that is obtaining similar results as presented in a paper or talk, using the same code and data (when available), is a necessary step to verify the reliability of research findings.
Evaluating Logical Generalization in Graph Neural Networks
Recent research has highlighted the role of relational inductive biases in building learning agents that can generalize and reason in a compositional manner.
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way.
Inductive logic programming
Natural Language Understanding
+2
Compositional Language Understanding with Text-based Relational Reasoning
Neural networks for natural language reasoning have largely focused on extractive, fact-based question-answering (QA) and common-sense inference.
The RLLChatbot: a solution to the ConvAI challenge
This article presents in detail the RLLChatbot that participated in the 2017 ConvAI challenge.
Adversarial Gain
Adversarial examples can be defined as inputs to a model which induce a mistake - where the model output is different than that of an oracle, perhaps in surprising or malicious ways.
A Hierarchical Neural Attention-based Text Classifier
Deep neural networks have been displaying superior performance over traditional supervised classifiers in text classification.
Ethical Challenges in Data-Driven Dialogue Systems
The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm.
