Search Results for author:
Found 57 papers, 40 papers with code
Train Hard, Finetune Easy: Multilingual Denoising for RDF-to-Text Generation
We describe our system for the RDF-to-text generation task of the WebNLG Challenge 2020.
GPT-2-based Human-in-the-loop Theatre Play Script Generation
We experiment with adapting generative language models for the generation of long coherent narratives in the form of theatre plays.
Text-in-Context: Token-Level Error Detection for Table-to-Text Generation
Our system can detect the errors automatically using a combination of a rule-based natural language generation (NLG) system and pretrained language models (LMs).
A Unifying View On Task-oriented Dialogue Annotation
In this paper, we present a new dataset, obtained by merging four publicly available annotated corpora for task-oriented dialogues in several domains (MultiWOZ 2. 2, CamRest676, DSTC2 and Schema-Guided Dialogue Dataset).
Large Language Models as Span Annotators
Until recently, span annotation was limited to human annotators or fine-tuned encoder models.
OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs
Large Language Models (LLMs) have demonstrated great potential as evaluators of NLG systems, allowing for high-quality, reference-free, and multi-aspect assessments.
Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices
Automatic metrics are extensively used to evaluate natural language processing systems.
Teaching LLMs at Charles University: Assignments and Activities
This paper presents teaching materials, particularly assignments and ideas for classroom activities, from a new course on large language models (LLMs) taught at Charles University.
factgenie: A Framework for Span-based Evaluation of Generated Texts
We present factgenie: a framework for annotating and visualizing word spans in textual model outputs.
Are Large Language Models Actually Good at Text Style Transfer?
We analyze the performance of large language models (LLMs) on Text Style Transfer (TST), specifically focusing on sentiment transfer and text detoxification across three languages: English, Hindi, and Bengali.
Multilingual Text Style Transfer: Datasets & Models for Indian Languages
Text style transfer (TST) involves altering the linguistic style of a text while preserving its core content.
Text Detoxification as Style Transfer in English and Hindi
This task contributes to safer and more respectful online communication and can be considered a Text Style Transfer (TST) task, where the text style changes while its content is preserved.
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
Natural Language Processing (NLP) research is increasingly focusing on the use of Large Language Models (LLMs), with some of the most popular ones being either fully or partially closed-source.
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation
We analyze the behaviors of open large language models (LLMs) on the task of data-to-text (D2T) generation, i. e., generating coherent and relevant text from structured data.
Balancing the Style-Content Trade-Off in Sentiment Transfer Using Polarity-Aware Denoising
Text sentiment transfer aims to flip the sentiment polarity of a sentence (positive to negative or vice versa) while preserving its sentiment-independent content.
LEEETs-Dial: Linguistic Entrainment in End-to-End Task-oriented Dialogue systems
Linguistic entrainment, or alignment, represents a phenomenon where linguistic patterns employed by conversational participants converge to one another.
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Our method does not need any changes to the underlying LM's architecture or training procedure and can thus be combined with any model and decoding operating on word probabilities.
Three Ways of Using Large Language Models to Evaluate Chat
This paper describes the systems submitted by team6 for ChatEval, the DSTC 11 Track 4 competition.
With a Little Help from the Authors: Reproducing Human Evaluation of an MT Error Detector
This work presents our efforts to reproduce the results of the human evaluation experiment presented in the paper of Vamvas and Sennrich (2022), which evaluated an automatic system detecting over- and undertranslations (translations containing more or less information than the original) in machine translation (MT) outputs.
Tackling Hallucinations in Neural Chart Summarization
Hallucinations in text generation occur when the system produces text that is not grounded in the input.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Are LLMs All You Need for Task-Oriented Dialogue?
Instructions-tuned Large Language Models (LLMs) gained recently huge popularity thanks to their ability to interact with users through conversation.
TabGenie: A Toolkit for Table-to-Text Generation
We present TabGenie - a toolkit which enables researchers to explore, preprocess, and analyze a variety of data-to-text generation datasets through the unified framework of table-to-text generation.
MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module
We present MooseNet, a trainable speech metric that predicts the listeners' Mean Opinion Score (MOS).
Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models
Pretrained language models (PLMs) for data-to-text (D2T) generation can use human-readable data labels such as column headings, keys, or relation names to generalize to out-of-domain examples.
Learning Interpretable Latent Dialogue Actions With Less Supervision
We present a novel architecture for explainable modeling of task-oriented dialogues with discrete latent variables to represent dialogue actions.
AARGH! End-to-end Retrieval-Generation for Task-Oriented Dialog
We introduce AARGH, an end-to-end task-oriented dialog system combining retrieval and generative approaches in a single model, aiming at improving dialog management and lexical diversity of outputs.
DialogueScript: Using Dialogue Agents to Produce a Script
We present a novel approach to generating scripts by using agents with different personality types.
Neural Pipeline for Zero-Shot Data-to-Text Generation
In data-to-text (D2T) generation, training on in-domain data leads to overfitting to the data representation and repeating training data noise.
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization
We show via data analysis that it's not only the models which are to blame: more than 27% of facts mentioned in the gold summaries of MiRANews are better grounded on assisting documents than in the main source articles.
Underreporting of errors in NLG output, and what to do about it
We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make.
Shades of BLEU, Flavours of Success: The Case of MultiWOZ
The MultiWOZ dataset (Budzianowski et al., 2018) is frequently used for benchmarking context-to-response abilities of task-oriented dialogue systems.
AGGGEN: Ordering and Aggregating while Generating
We present AGGGEN (pronounced 'again'), a data-to-text model which re-introduces two explicit sentence planning stages into neural data-to-text systems: input ordering and input aggregation.
THEaiTRE 1.0: Interactive generation of theatre play scripts
We present the first version of a system for interactive generation of theatre play scripts.
AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models
Our model substantially outperforms the baseline on the MultiWOZ data and shows competitive performance with state of the art in both automatic and human evaluation.
Ranked #3 on
End-To-End Dialogue Modelling
on MULTIWOZ 2.0
(using extra training data)
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Ranked #1 on
Extreme Summarization
on GEM-XSum
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference
A major challenge in evaluating data-to-text (D2T) generation is measuring the semantic accuracy of the generated text, i. e. checking if the output text contains all and only facts supported by the input data.
Data-to-Text Generation with Iterative Text Editing
Our approach maximizes the completeness and semantic accuracy of the output text while leveraging the abilities of recent pre-trained models for text editing (LaserTagger) and language modeling (GPT-2) to improve the text fluency.
SpeedySpeech: Efficient Neural Speech Synthesis
While recent neural sequence-to-sequence models have greatly improved the quality of speech synthesis, there has not been a system capable of fast training, fast inference and high-quality audio synthesis at the same time.
One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech
We introduce an approach to multilingual speech synthesis which uses the meta-learning concept of contextual parameter generation and produces natural-sounding multilingual speech using more languages and less training data than previous approaches.
THEaiTRE: Artificial Intelligence to Write a Theatre Play
We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts.
Semantic Noise Matters for Neural Natural Language Generation
Neural natural language generation (NNLG) systems are known for their pathological outputs, i. e. generating text which is unrelated to the input specification.
Ranked #3 on
Data-to-Text Generation
on Cleaned E2E NLG Challenge
Neural Generation for Czech: Data and Baselines
We present the first dataset targeted at end-to-end NLG in Czech in the restaurant domain, along with several strong baseline models using the sequence-to-sequence approach.
Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)
We present a recurrent neural network based system for automatic quality estimation of natural language generation (NLG) outputs, which jointly learns to assign numerical ratings to individual outputs and to provide pairwise rankings of two different outputs.
User Evaluation of a Multi-dimensional Statistical Dialogue System
We present the first complete spoken dialogue system driven by a multi-dimensional statistical dialogue manager.
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
Introducing novel automatic and human metrics, we compare 62 systems submitted by 17 institutions, covering a wide range of approaches, including machine learning architectures -- with the majority implementing sequence-to-sequence models (seq2seq) -- as well as systems based on grammatical rules and templates.
Neural Response Ranking for Social Conversation: A Data-Efficient Approach
Using a dataset of real conversations collected in the 2017 Alexa Prize challenge, we developed a neural ranker for selecting 'good' system responses to user utterances, i. e. responses which are likely to lead to long and engaging conversations.
Findings of the E2E NLG Challenge
This paper summarises the experimental setup and results of the first shared task on end-to-end (E2E) natural language generation (NLG) in spoken dialogue systems.
Ranked #5 on
Data-to-Text Generation
on E2E NLG Challenge
Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as the GloVe embedding similarity between the dialogue context and the generated response, (2) we filter our training corpora based on the measure of coherence to obtain topically coherent and lexically diverse context-response pairs, (3) we then train a response generator using a conditional variational autoencoder model that incorporates the measure of coherence as a latent variable and uses a context gate to guarantee topical consistency with the context and promote lexical diversity.
RankME: Reliable Human Ratings for Natural Language Generation
Human evaluation for natural language generation (NLG) often suffers from inconsistent user ratings.
An Ensemble Model with Ranking for Social Dialogue
Open-domain social dialogue is one of the long-standing goals of Artificial Intelligence.
Referenceless Quality Estimation for Natural Language Generation
Traditional automatic evaluation measures for natural language generation (NLG) use costly human-authored references to estimate the quality of a system output.
Why We Need New Evaluation Metrics for NLG
The majority of NLG evaluation relies on automatic metrics, such as BLEU .
Data-driven Natural Language Generation: Paving the Road to Success
We argue that there are currently two major bottlenecks to the commercial use of statistical machine learning approaches for natural language generation (NLG): (a) The lack of reliable automatic evaluation metrics for NLG, and (b) The scarcity of high quality in-domain corpora.
The E2E Dataset: New Challenges For End-to-End Generation
This paper describes the E2E data, a new dataset for training end-to-end, data-driven natural language generation systems in the restaurant domain, which is ten times bigger than existing, frequently used datasets in this area.
A Context-aware Natural Language Generator for Dialogue Systems
We present a novel natural language generation system for spoken dialogue systems capable of entraining (adapting) to users' way of speaking, providing contextually appropriate responses.
Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings
We present a natural language generator based on the sequence-to-sequence approach that can be trained to produce natural language strings as well as deep syntax dependency trees from input dialogue acts, and we use it to directly compare two-step generation with separate sentence planning and surface realization stages to a joint, one-step approach.
