Search Results for author:
Found 8 papers, 6 papers with code
TUDA-Reproducibility @ ReproGen: Replicability of Human Evaluation of Text-to-Text and Concept-to-Text Generation
This paper describes our contribution to the Shared Task ReproGen by Belz et al. (2021), which investigates the reproducibility of human evaluations in the context of Natural Language Generation.
Syntactic Language Change in English and German: Metrics, Parsers, and Convergences
Even though we have evidence that recent parsers trained on modern treebanks are not heavily affected by data 'noise' such as spelling changes and OCR errors in our historic data, we find that results of syntactic language change are sensitive to the parsers involved, which is a caution against using a single parser for evaluating syntactic language change as done in previous work.
NLLG Quarterly arXiv Report 09/23: What are the most influential current AI Papers?
Artificial Intelligence (AI) has witnessed rapid growth, especially in the subfields Natural Language Processing (NLP), Machine Learning (ML) and Computer Vision (CV).
NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers?
In particular, we compile a list of the 40 most popular papers based on normalized citation counts from the first half of 2023.
ChatGPT: A Meta-Analysis after 2.5 Months
ChatGPT, a chatbot developed by OpenAI, has gained widespread popularity and media attention since its release in November 2022.
Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End
Our human evaluation suggests that our best end-to-end system performs similarly to human authors (but arguably slightly worse).
MENLI: Robust Evaluation Metrics from Natural Language Inference
Recently proposed BERT-based evaluation metrics for text generation perform well on standard benchmarks but are vulnerable to adversarial attacks, e. g., relating to information correctness.
Reproducibility Issues for BERT-based Evaluation Metrics
We find that reproduction of claims and results often fails because of (i) heavy undocumented preprocessing involved in the metrics, (ii) missing code and (iii) reporting weaker results for the baseline metrics.
