Search Results for author:
Found 59 papers, 23 papers with code
GiBERT: Enhancing BERT with Linguistic Information using a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks.
Meta-Reasoning Improves Tool Use in Large Language Models
We present Tool selECTion via meta-reasONing (TECTON), a two-phase system that first reasons over a task using a custom fine-tuned LM head and outputs candidate tools.
DiffuseDef: Improved Robustness to Adversarial Attacks
Pretrained language models have significantly advanced performance across various natural language processing tasks.
SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)
In the first part, we review the literature on MIAs against LLMs.
Continuous Predictive Modeling of Clinical Notes and ICD Codes in Patient Health Records
Electronic Health Records (EHR) serve as a valuable source of patient information, offering insights into medical histories, treatments, and outcomes.
Prompting open-source and commercial language models for grammatical error correction of English learner text
Thanks to recent advances in generative AI, we are able to prompt large language models (LLMs) to produce texts which are fluent and grammatical.
Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models
First, we propose a procedure for the development and evaluation of document-level membership inference for LLMs by leveraging commonly used data sources for training and the model release date.
On the application of Large Language Models for language teaching and assessment technology
The recent release of very large language models such as PaLM and GPT-4 has made an unprecedented impact in the popular media and public consciousness, giving rise to a mixture of excitement and fear as to their capabilities and potential uses, and shining a light on natural language processing research which had not previously received so much attention.
Atomic Inference for NLI with Generated Facts as Atoms
With recent advances, neural models can achieve human-level performance on various natural language tasks.
Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation
Knowledge distillation optimises a smaller student model to behave similarly to a larger teacher model, retaining some of the performance benefits.
Finding the Needle in a Haystack: Unsupervised Rationale Extraction from Long Text Classifiers
Long-sequence transformers are designed to improve the representation of longer texts by language models and their performance on downstream document-level tasks.
Modelling Temporal Document Sequences for Clinical ICD Coding
Past studies on the ICD coding problem focus on predicting clinical codes primarily based on the discharge summary.
An Extended Sequence Tagging Vocabulary for Grammatical Error Correction
We extend a current sequence-tagging approach to Grammatical Error Correction (GEC) by introducing specialised tags for spelling correction and morphological inflection using the SymSpell and LemmInflect algorithms.
Probing for targeted syntactic knowledge through grammatical error detection
Targeted studies testing knowledge of subject-verb agreement (SVA) indicate that pre-trained language models encode syntactic information.
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
We can further improve model performance and span-level decisions by using the e-SNLI explanations during training.
Control Prefixes for Parameter-Efficient Text Generation
Prefix-tuning is a powerful lightweight technique for adapting a large pre-trained language model to a downstream application.
Ranked #1 on
Data-to-Text Generation
on WebNLG
Guiding Visual Question Generation
In traditional Visual Question Generation (VQG), most images have multiple concepts (e. g. objects and categories) for which a question could be generated, but models are trained to mimic an arbitrary choice of concept as given in their training data.
Contextual Sentence Classification: Detecting Sustainability Initiatives in Company Reports
We introduce the novel task of detecting sustainability initiatives in company reports.
Supervising Model Attention with Human Explanations for Robust Natural Language Inference
Natural Language Inference (NLI) models are known to learn from biases and artefacts within their training data, impacting how well they generalise to other unseen datasets.
Memorisation versus Generalisation in Pre-trained Language Models
State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data.
How Metaphors Impact Political Discourse: A Large-Scale Topic-Agnostic Study Using Neural Metaphor Detection
Metaphors are widely used in political rhetoric as an effective framing device.
Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers
We investigate how sentence-level transformers can be modified into effective sequence labelers at the token level without any direct supervision.
Visual Cues and Error Correction for Translation Robustness
Neural Machine Translation models are sensitive to noise in the input texts, such as misspelled words and ungrammatical constructions.
Grammatical error detection in transcriptions of spoken English
We describe the collection of transcription corrections and grammatical error annotations for the CrowdED Corpus of spoken English monologues on business topics.
Seeing Both the Forest and the Trees: Multi-head Attention for Joint Classification on Different Compositional Levels
In natural languages, words are used in association to construct sentences.
GiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks.
Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses
Evaluation of grammatical error correction (GEC) systems has primarily focused on essays written by non-native learners of English, which however is only part of the full spectrum of GEC applications.
Verbal Multiword Expressions for Identification of Metaphor
Metaphor is a linguistic device in which a concept is expressed by mentioning another.
Multidirectional Associative Optimization of Function-Specific Word Representations
We present a neural framework for learning associations between interrelated groups of words such as the ones found in Subject-Verb-Object (SVO) structures.
Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling
Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.
Modelling the interplay of metaphor and emotion through multitask learning
Metaphors allow us to convey emotion by connecting physical experiences and abstract concepts.
Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling
Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.
Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in Distributional Semantic Models
Word embeddings are an essential component in a wide range of natural language processing applications.
Neural and FST-based approaches to grammatical error correction
In this paper, we describe our submission to the BEA 2019 shared task on grammatical error correction.
Context is Key: Grammatical Error Detection with Contextual Word Representations
Grammatical error detection (GED) in non-native writing requires systems to identify a wide range of errors in text written by language learners.
CAMsterdam at SemEval-2019 Task 6: Neural and graph-based feature extraction for the identification of offensive tweets
We describe the CAMsterdam team entry to the SemEval-2019 Shared Task 6 on offensive language identification in Twitter data.
A Simple and Robust Approach to Detecting Subject-Verb Agreement Errors
While rule-based detection of subject-verb agreement (SVA) errors is sensitive to syntactic parsing errors and irregularities and exceptions to the main rules, neural sequential labelers have a tendency to overfit their training data.
Advance Prediction of Ventricular Tachyarrhythmias using Patient Metadata and Multi-Task Networks
We describe a novel neural network architecture for the prediction of ventricular tachyarrhythmias.
Jointly Learning to Label Sentences and Tokens
Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size.
Ranked #1 on
Grammatical Error Detection
on JFLEG
Sequence Classification with Human Attention
Learning attention functions requires large volumes of data, but many NLP tasks simulate human behavior, and in this paper, we show that human attention really does provide a good inductive bias on many attention functions in NLP.
Variable Typing: Assigning Meaning to Variables in Mathematical Text
Our results show that the best performing MIR models make use of our typed index, compared to a formula index only containing raw symbols, thereby demonstrating the usefulness of variable typing.
Scoring Lexical Entailment with a Supervised Directional Similarity Network
Experiments show excellent performance on scoring graded lexical entailment, raising the state-of-the-art on the HyperLex dataset by approximately 25%.
Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens
Can attention- or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels?
Neural Multi-task Learning in Automated Assessment
Grammatical error detection and automated essay scoring are two tasks in the area of automated assessment.
Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection
The ubiquity of metaphor in our everyday communication makes it an important problem for natural language understanding.
Neural Sequence-Labelling Models for Grammatical Error Correction
We propose an approach to N-best list reranking using neural sequence-labelling models.
An Error-Oriented Approach to Word Embedding Pre-Training
Additionally, extending the model with corrections provides further performance gains when data sparsity is an issue.
Auxiliary Objectives for Neural Error Detection Models
We investigate the utility of different auxiliary objectives and training strategies within a neural sequence labeling approach to error detection in learner writing.
Ranked #2 on
Grammatical Error Detection
on CoNLL-2014 A1
Detecting Off-topic Responses to Visual Prompts
Automated methods for essay scoring have made great progress in recent years, achieving accuracies very close to human annotators.
Artificial Error Generation with Machine Translation and Syntactic Patterns
Shortage of available training data is holding back progress in the area of automated error detection.
Ranked #3 on
Grammatical Error Detection
on FCE
Semi-supervised Multitask Learning for Sequence Labeling
We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset.
Ranked #4 on
Grammatical Error Detection
on FCE
Attending to Characters in Neural Sequence Labeling Models
Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words.
Ranked #7 on
Grammatical Error Detection
on FCE
Compositional Sequence Labeling Models for Error Detection in Learner Writing
In this paper, we present the first experiments using neural network models for the task of error detection in learner writing.
Ranked #3 on
Grammatical Error Detection
on CoNLL-2014 A1
Automatic Text Scoring Using Neural Networks
Automated Text Scoring (ATS) provides a cost-effective and consistent alternative to human marking.
Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays
We investigate the task of assessing sentence-level prompt relevance in learner essays.
A Joint Model for Word Embedding and Word Morphology
This paper presents a joint model for performing unsupervised morphological analysis on words, and learning a character-level composition function from morphemes to word embeddings.
Online Representation Learning in Recurrent Neural Language Models
We investigate an extension of continuous online learning in recurrent neural network language models.
