no code implementations • EACL (HCINLP) 2021 • Nanna Inie, Leon Derczynski
The framework is constructed by following an interdisciplinary research-model (IDR), combining field-specific knowledge with existing work in the two fields.
1 code implementation • NoDaLiDa 2021 • Jeppe Nørregaard, Leon Derczynski
We present a dataset, DanFEVER, intended for multilingual misinformation research.
Ranked #1 on Fact Verification on DanFEVER
1 code implementation • EACL (BSNLP) 2021 • Kamil Saitov, Leon Derczynski
Abusive phenomena are commonplace in language on the web.
no code implementations • WS (NoDaLiDa) 2019 • Andreas Kirkedal, Barbara Plank, Leon Derczynski, Natalie Schluter
Danish is a North Germanic language spoken principally in Denmark, a country with a long tradition of technological and scientific innovation.
1 code implementation • WS (NoDaLiDa) 2019 • Leon Derczynski, Alex Speed Kjeldsen
This paper introduces language processing resources and tools for Bornholmsk, a language spoken on the island of Bornholm, with roots in Danish and closely related to Scanian.
2 code implementations • WS (NoDaLiDa) 2019 • Anders Edelbo Lillie, Emil Refsgaard Middelboe, Leon Derczynski
In our experiments, monolinugal scores reach stance-based veracity accuracy of 0. 83 (F1 0. 68); applying the model across languages predicts veracity of claims with an accuracy of 0. 82 (F1 0. 67).
1 code implementation • WS (NoDaLiDa) 2019 • Rasmus Lehmann, Leon Derczynski
Furthermore, three models based on an LSTM architecture are designed, implemented and optimized to perform the task of stance detection for the generated dataset.
1 code implementation • EMNLP (sustainlp) 2021 • Lucas Høyberg Puvis de Chavannes, Mads Guldborg Kjeldgaard Kongsbak, Timmie Rantzau, Leon Derczynski
Training large language models can consume a large amount of energy.
1 code implementation • 17 Jun 2024 • Nvidia, :, Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H. Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, Sirshak Das, Ayush Dattagupta, Olivier Delalleau, Leon Derczynski, Yi Dong, Daniel Egert, Ellie Evans, Aleksander Ficek, Denys Fridman, Shaona Ghosh, Boris Ginsburg, Igor Gitman, Tomasz Grzegorzek, Robert Hero, Jining Huang, Vibhu Jawa, Joseph Jennings, Aastha Jhunjhunwala, John Kamalu, Sadaf Khan, Oleksii Kuchaiev, Patrick Legresley, Hui Li, Jiwei Liu, Zihan Liu, Eileen Long, Ameya Sunil Mahabaleshwarkar, Somshubra Majumdar, James Maki, Miguel Martinez, Maer Rodrigues de Melo, Ivan Moshkov, Deepak Narayanan, Sean Narenthiran, Jesus Navarro, Phong Nguyen, Osvald Nitski, Vahid Noroozi, Guruprasad Nutheti, Christopher Parisien, Jupinder Parmar, Mostofa Patwary, Krzysztof Pawelec, Wei Ping, Shrimai Prabhumoye, Rajarshi Roy, Trisha Saar, Vasanth Rao Naik Sabavat, Sanjeev Satheesh, Jane Polak Scowcroft, Jason Sewall, Pavel Shamis, Gerald Shen, Mohammad Shoeybi, Dave Sizer, Misha Smelyanskiy, Felipe Soares, Makesh Narsimhan Sreedhar, Dan Su, Sandeep Subramanian, Shengyang Sun, Shubham Toshniwal, Hao Wang, Zhilin Wang, Jiaxuan You, Jiaqi Zeng, Jimmy Zhang, Jing Zhang, Vivienne Zhang, Yian Zhang, Chen Zhu
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward.
1 code implementation • 16 Jun 2024 • Leon Derczynski, Erick Galinkin, Jeffrey Martin, Subho Majumdar, Nanna Inie
As Large Language Models (LLMs) are deployed and integrated into thousands of applications, the need for scalable evaluation of how models respond to adversarial attacks grows rapidly.
1 code implementation • 18 Apr 2024 • Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller, Ram Gandikota, Agasthya Gangavarapu, Ananya Gangavarapu, James Gealy, Rajat Ghosh, James Goel, Usman Gohar, Sujata Goswami, Scott A. Hale, Wiebke Hutiri, Joseph Marvin Imperial, Surgan Jandial, Nick Judd, Felix Juefei-Xu, Foutse khomh, Bhavya Kailkhura, Hannah Rose Kirk, Kevin Klyman, Chris Knotz, Michael Kuchnik, Shachi H. Kumar, Srijan Kumar, Chris Lengerich, Bo Li, Zeyi Liao, Eileen Peters Long, Victor Lu, Sarah Luger, Yifan Mai, Priyanka Mary Mammen, Kelvin Manyeki, Sean McGregor, Virendra Mehta, Shafee Mohammed, Emanuel Moss, Lama Nachman, Dinesh Jinenhally Naganna, Amin Nikanjam, Besmira Nushi, Luis Oala, Iftach Orr, Alicia Parrish, Cigdem Patlak, William Pietri, Forough Poursabzi-Sangdeh, Eleonora Presani, Fabrizio Puletti, Paul Röttger, Saurav Sahay, Tim Santos, Nino Scherrer, Alice Schoenauer Sebag, Patrick Schramowski, Abolfazl Shahbazi, Vin Sharma, Xudong Shen, Vamsi Sistla, Leonard Tang, Davide Testuggine, Vithursan Thangarasa, Elizabeth Anne Watkins, Rebecca Weiss, Chris Welty, Tyler Wilbers, Adina Williams, Carole-Jean Wu, Poonam Yadav, Xianjun Yang, Yi Zeng, Wenhui Zhang, Fedor Zhdanov, Jiacheng Zhu, Percy Liang, Peter Mattson, Joaquin Vanschoren
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
no code implementations • 10 Nov 2023 • Nanna Inie, Jonathan Stray, Leon Derczynski
As a result, this paper presents a grounded theory of how and why people attack large language models: LLM red teaming in the wild.
no code implementations • 29 Jun 2023 • Ji-Ung Lee, Haritz Puerto, Betty van Aken, Yuki Arase, Jessica Zosa Forde, Leon Derczynski, Andreas Rücklé, Iryna Gurevych, Roy Schwartz, Emma Strubell, Jesse Dodge
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters.
2 code implementations • 31 Mar 2023 • Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, Saif Mohammad
However, there is no risk-centric framework for documenting the complexity of a landscape in which some risks are shared across models and contexts, while others are specific, and where certain conditions may be required for risks to manifest as harms.
no code implementations • 31 Aug 2022 • Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
no code implementations • 25 Aug 2022 • Manuel R. Ciosici, Leon Derczynski
Training large neural language models on large datasets is resource- and time-intensive.
no code implementations • 12 Aug 2022 • Jeppe Nørregaard, Leon Derczynski
Measuring inter-annotator agreement is important for annotation tasks, but many metrics require a fully-annotated set of data, where all annotators annotate all samples.
no code implementations • 17 Jun 2022 • Leon Derczynski, Annika Solveig Hedegaard Isfeldt, Signhild Djurhuus
This article documents a dataset of sentence pairs between Faroese and Danish, produced at ITU Copenhagen.
no code implementations • 8 Jun 2022 • Mateusz Jurewicz, Leon Derczynski
The task of learning to map an input set onto a permuted sequence of its elements is challenging for neural networks.
no code implementations • 6 May 2022 • Gcinizwe Dlamini, Imad Eddine Ibrahim Bekkouch, Adil Khan, Leon Derczynski
This allows us to rapidly achieve similar results for stance detection for the Zulu language, the target language in this work, as are found for English.
no code implementations • 29 Apr 2022 • Hannah Rose Kirk, Abeba Birhane, Bertie Vidgen, Leon Derczynski
Text data can pose a risk of harm.
1 code implementation • ACL 2021 • Philine Zeinert, Nanna Inie, Leon Derczynski
Online misogyny, a category of online abusive language, has serious and harmful social consequences.
Ranked #1 on Hate Speech Detection on bajer_danish_misogyny
no code implementations • 28 Jul 2021 • Erida Nurce, Jorgel Keci, Leon Derczynski
The ever growing usage of social media in the recent years has had a direct impact on the increased presence of hate speech and offensive speech in online platforms.
Ranked #1 on Hate Speech Detection on SHAJ
no code implementations • 16 Apr 2021 • Magnus Jacobsen, Mikkel H. Sørensen, Leon Derczynski
Improvement in machine learning-based NLP performance are often presented with bigger models and more complex code.
2 code implementations • EACL (VarDial) 2021 • René Haas, Leon Derczynski
Automatic language identification is a challenging problem.
no code implementations • COLING 2020 • Leon Derczynski, Arkaitz Zubiaga
Detecting and grounding false and misleading claims on the web has grown to form a substantial sub-field of NLP.
no code implementations • WS 2020 • Leon Derczynski, Julie Binau, Henri Schulte
We propose two measures for measuring the quality of constructed claims in the FEVER task.
no code implementations • SEMEVAL 2020 • Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, Çağrı Çöltekin
We present the results and main findings of SemEval-2020 Task 12 on Multilingual Offensive Language Identification in Social Media (OffensEval 2020).
1 code implementation • 12 Jun 2020 • Leon Derczynski
The power that machine learning models consume when making predictions can be affected by a model's architecture.
1 code implementation • LREC 2020 • Manuel R. Ciosici, Ira Assent, Leon Derczynski
We present efficient implementations of Brown clustering and the alternative Exchange clustering as well as a number of methods to accelerate the computation of both hierarchical and flat clusters.
no code implementations • 3 Apr 2020 • Bertie Vidgen, Leon Derczynski
Data-driven analysis and detection of abusive online content covers many different tasks, phenomena, contexts, and methodologies.
no code implementations • 11 Feb 2020 • Nanna Inie, Jeanette Falk Olesen, Leon Derczynski
Misinformation spread presents a technological and social threat to society.
no code implementations • LREC 2020 • Gudbjartur Ingi Sigurbergsson, Leon Derczynski
It contains user generated comments from various social media platforms, and to our knowledge, it is the first of its kind.
Ranked #1 on Hate Speech Detection on DKhate
1 code implementation • 27 Jun 2019 • Leon Derczynski
This technical note describes a set of baseline tools for automatic processing of Danish text.
no code implementations • NAACL 2019 • Manuel R. Ciosici, Leon Derczynski, Ira Assent
We show that increases in Average Mutual Information, the clustering algorithms{'} optimization goal, are highly correlated with improvements in encoding of morphosyntactic information.
no code implementations • SEMEVAL 2019 • Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, Leon Derczynski
Rumour verification is characterised by the need to consider evolving conversations and news updates to reach a verdict on a rumour{'}s veracity.
no code implementations • 18 Sep 2018 • Genevieve Gorrell, Kalina Bontcheva, Leon Derczynski, Elena Kochkina, Maria Liakata, Arkaitz Zubiaga
This is the proposal for RumourEval-2019, which will run in early 2019 as part of that year's SemEval event.
2 code implementations • 5 Sep 2018 • Nikita Lozhnikov, Leon Derczynski, Manuel Mazzara
As well as presenting this openly-available dataset, the first of its kind for Russian, the paper presents a baseline for stance prediction in the language.
Ranked #1 on Stance Detection on RuStance
1 code implementation • SEMEVAL 2018 • Sofia Reznikova, Leon Derczynski
This paper describes the IUCM entry at SemEval-2018 Task 11, on machine comprehension using commonsense knowledge.
1 code implementation • 29 Jan 2018 • Leon Derczynski, Kenny Meesters, Kalina Bontcheva, Diana Maynard
Messages are filtered for informativeness based on a definition of the concept drawn from prior research and crisis response experts.
1 code implementation • 22 Dec 2017 • Leon Derczynski, Matthew Rowe
Existing studies of how information diffuses across social networks have thus far concentrated on analysing and recovering the spread of deterministic innovations such as URLs, hashtags, and group membership.
no code implementations • WS 2017 • Leon Derczynski, Eric Nichols, Marieke van Erp, Nut Limsopatham
This shared task focuses on identifying unusual, previously-unseen entities in the context of emerging discussions.
2 code implementations • RANLP 2017 • Ahmet Aker, Leon Derczynski, Kalina Bontcheva
Stance classification determines the attitude, or stance, in a (typically short) text.
no code implementations • 11 Jan 2017 • Isabelle Augenstein, Leon Derczynski, Kalina Bontcheva
Unseen NEs, in particular, play an important role, which have a higher incidence in diverse genres such as social media than in more regular genres such as newswire.
no code implementations • WS 2016 • Bo Han, Afshin Rahimi, Leon Derczynski, Timothy Baldwin
This paper presents the shared task for English Twitter geolocation prediction in WNUT 2016.
no code implementations • COLING 2016 • Leon Derczynski, Kalina Bontcheva, Ian Roberts
One of the main obstacles, hampering method development and comparative evaluation of named entity recognition in social media, is the lack of a sizeable, diverse, high quality annotated corpus, analogous to the CoNLL{'}2003 news dataset.
no code implementations • COLING 2016 • Leon Derczynski
Determining the relative order of events and times described in text is an important problem in natural language processing.
no code implementations • 6 Aug 2016 • Leon Derczynski
A plethora of vector-space representations for words is currently available, which is growing.
no code implementations • LREC 2016 • Leon Derczynski
This paper addresses the problem of quantifying the differences between entity extraction systems, where in general only a small proportion a document should be selected.
1 code implementation • LREC 2016 • Leon Derczynski, Jannik Str{\"o}tgen, Diana Maynard, Mark A. Greenwood, Manuel Jung
GATE is a widely used open-source solution for text processing with a large user community.
no code implementations • WS 2015 • Leon Derczynski, Isabelle Augenstein, Kalina Bontcheva
This paper describes a pilot NER system for Twitter, comprising the USFD system entry to the W-NUT 2015 NER shared task.
no code implementations • 27 Oct 2014 • Leon Derczynski, Diana Maynard, Giuseppe Rizzo, Marieke van Erp, Genevieve Gorrell, Raphaël Troncy, Johann Petrak, Kalina Bontcheva
Applying natural language processing for mining and intelligent information access to tweets (a form of microblog) is a challenging, emerging research area.
no code implementations • LREC 2014 • Marta Sabou, Kalina Bontcheva, Leon Derczynski, Arno Scharl
Crowdsourcing is an emerging collaborative approach that can be used for the acquisition of annotated corpora and a wide range of other linguistic resources.
no code implementations • 19 Mar 2014 • Steven Bethard, Leon Derczynski, James Pustejovsky, Marc Verhagen
We describe the Clinical TempEval task which is currently in preparation for the SemEval-2015 evaluation exercise.
no code implementations • 26 Apr 2013 • Leon Derczynski, Hector Llorens, Naushad UzZaman
To unify the state of current resources, and to make progress toward easy adoption of its current incarnation ISO-TimeML, this paper introduces TimeML-strict: a valid, unambiguous, and easy-to-process subset of TimeML.
no code implementations • 26 Apr 2013 • Leon Derczynski, Richard Shaw, Ben Solway, Jun Wang
Question answering involves developing methods to extract useful information from large collections of documents.
2 code implementations • 22 Jun 2012 • Naushad UzZaman, Hector Llorens, James Allen, Leon Derczynski, Marc Verhagen, James Pustejovsky
We describe the TempEval-3 task which is currently in preparation for the SemEval-2013 evaluation exercise.
1 code implementation • LREC 2012 • Hector Llorens, Leon Derczynski, Robert Gaizauskas, Estela Saquete
In this paper, we present TIMEN, a community-driven tool for temporal expression normalisation.
Ranked #1 on Timex normalization on TimeBank
no code implementations • LREC 2012 • Leon Derczynski, H{\'e}ctor Llorens, Estela Saquete
Automatic annotation of temporal expressions is a research challenge of great interest in the field of information extraction.