Search Results for author:
Found 102 papers, 50 papers with code
BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter
This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task.
Towards better structured and less noisy Web data: Oscar with Register annotations
Web-crawled datasets are known to be noisy, as they feature a wide range of language use covering both user-generated and professionally edited content as well as noise originating from the crawling process.
A Relation Extraction Dataset for Knowledge Extraction from Web Tables
Most datasets are constructed using synthetic tables that lack valuable metadata information, or are limited in size to be considered as a challenging evaluation set.
The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
We describe the University of Edinburgh’s Bengali\leftrightarrowHindi constrained systems submitted to the WMT21 News Translation task.
Efficient Machine Translation with Model Pruning and Quantization
We participated in all tracks of the WMT 2021 efficient machine translation task: single-core CPU, multi-core CPU, and GPU hardware with throughput and latency conditions.
Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountability
As a call to action, we introduce DataRubrics, a structured framework for assessing the quality of both human- and model-generated datasets.
CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation
Cultural content poses challenges for machine translation systems due to the differences in conceptualizations between cultures, where language alone may fail to convey sufficient context to capture region-specific meanings.
From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs
Adapting cultural values in Large Language Models (LLMs) presents significant challenges, particularly due to biases and limited training data.
Understanding Cross-Lingual Inconsistency in Large Language Models
Large language models (LLMs) are demonstrably capable of cross-lingual transfer, but can produce inconsistent output when prompted with the same queries written in different languages.
Behind Maya: Building a Multilingual Vision Language Model
In recent times, we have seen a rapid development of large Vision-Language Models (VLMs).
Crosslingual Reasoning through Test-Time Scaling
Third, we discover an effective strategy to control the language of long CoT reasoning, and we observe that models reason better and more efficiently in high-resource languages.
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
We introduce softpick, a rectified, not sum-to-one, drop-in replacement for softmax in transformer attention mechanisms that eliminates attention sink and massive activations.
Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia
Beyond crowdsourcing, our initiative goes one step further in the exploration of the automatic collection of culturally relevant images through crawling and image generation.
SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection
The data instances are multi-labeled into six emotional classes, with additional datasets in 11 languages annotated for emotion intensity.
IteRABRe: Iterative Recovery-Aided Block Reduction
Large Language Models (LLMs) have grown increasingly expensive to deploy, driving the need for effective model compression techniques.
Llama-3.1-Sherkala-8B-Chat: An Open Large Language Model for Kazakh
Llama-3. 1-Sherkala-8B-Chat, or Sherkala-Chat (8B) for short, is a state-of-the-art instruction-tuned open generative large language model (LLM) designed for Kazakh.
Do Language Models Understand Honorific Systems in Javanese?
The Javanese language features a complex system of honorifics that vary according to the social status of the speaker, listener, and referent.
NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts
Indonesia is rich in languages and scripts.
TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
Reasoning is a fundamental capability of large language models (LLMs), enabling them to comprehend, analyze, and solve complex problems.
BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languages
In this paper, we present BRIGHTER-- a collection of multilabeled emotion-annotated datasets in 28 different languages.
Balanced Multi-Factor In-Context Learning for Multilingual Large Language Models
However, existing approaches address these factors independently, without explicitly disentangling their combined impact, leaving optimal example selection underexplored.
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI
To verify the generalizability of this finding across languages and domains, we perform an extensive case study to identify the upper bound of human detection accuracy.
QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
Fine-tuning large language models (LLMs) is often constrained by the computational costs of processing massive datasets.
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Extracting General-use Transformers for Low-resource Languages via Knowledge Distillation
Furthermore, we investigate additional steps during the distillation process that improves the soft-supervision of the target language, and provide a number of analyses and ablations to show the efficacy of the proposed method.
GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human
We present the GenAI Content Detection Task~1 -- a shared task on binary machine generated text detection, conducted as a part of the GenAI workshop at COLING 2025.
Can Multimodal LLMs do Visual Temporal Understanding and Reasoning? The answer is No!
Multimodal Large Language Models (MLLMs) have achieved significant advancements in tasks like Visual Question Answering (VQA) by leveraging foundational Large Language Models (LLMs).
Maya: An Instruction Finetuned Multilingual Multimodal Model
The rapid development of large Vision-Language Models (VLMs) has led to impressive results on academic benchmarks, primarily in widely spoken languages.
CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections
This three-step process allows us to harness the best of visual and textual foundation models, resulting in a powerful and efficient approach that surpasses state-of-the-art label-free classification methods.
Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
Error type information has been widely used to improve the performance of grammatical error correction (GEC) models, whether for generating corrections, re-ranking them, or combining GEC models.
Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Sense
In this paper, we demonstrate using false friends -- words that are orthographically similar but have completely different meanings in two languages -- as a possible approach to pinpoint the limitation of cross-lingual sense disambiguation in LLMs.
WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date.
Kalahi: A handcrafted, grassroots cultural LLM evaluation suite for Filipino
Multilingual large language models (LLMs) today may not necessarily provide culturally appropriate and relevant responses to its Filipino users.
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection
Category (iii) aims to detect attempts to obfuscate the fact that a text was machine-generated, while category (iv) looks for cases where the LLM was used to polish a human-written text, which is typically acceptable in academic writing, but not in education.
IndoToxic2024: A Demographically-Enriched Dataset of Hate Speech and Toxicity Types for Indonesian Language
To address this, we introduce IndoToxic2024, a comprehensive Indonesian hate speech and toxicity classification dataset.
The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation
Knowledge distillation (KD) has proven to be a successful strategy to improve the performance of smaller models in many NLP tasks.
Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting
We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy.
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
Southeast Asia (SEA) is a region rich in linguistic diversity and cultural variety, with over 1, 300 indigenous languages and a population of 671 million people.
MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding
Auto-regressive inference of transformers benefit greatly from Key-Value (KV) caching, but can lead to major memory bottlenecks as model size, batch size, and sequence length grow at scale.
CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge present in both visual and textual data.
Lessons from the Trenches on Reproducible Evaluation of Language Models
Third, we present the Language Model Evaluation Harness (lm-eval): an open source library for independent, reproducible, and extensible evaluation of language models that seeks to address these issues.
From Multiple-Choice to Extractive QA: A Case Study for English and Arabic
The rapid evolution of Natural Language Processing (NLP) has favoured major languages such as English, leaving a significant gap for many others due to limited resources.
SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection
The task attracted a large number of participants: subtask A monolingual (126), subtask A multilingual (59), subtask B (70), and subtask C (30).
Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning
While Large Language Models (LLMs) exhibit remarkable capabilities in zero-shot and few-shot scenarios, they often require computationally prohibitive sizes.
Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languages
To bridge this quality gap, we introduce Cendol, a collection of Indonesian LLMs encompassing both decoder-only and encoder-decoder architectures across a range of model sizes.
SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages
We present the first shared task on Semantic Textual Relatedness (STR).
Towards Measuring and Modeling "Culture" in LLMs: A Survey
We present a survey of more than 90 recent papers that aim to study cultural representation and inclusion in large language models (LLMs).
Daisy-TTS: Simulating Wider Spectrum of Emotions via Prosody Embedding Decomposition
This wide spectrum of emotions is well-studied in the structural model of emotions, which represents variety of emotions as derivative products of primary emotions with varying degrees of intensity.
Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
Large Language Models (LLMs) have demonstrated remarkable capabilities across various applications, fundamentally reshaping the landscape of natural language processing (NLP) research.
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels.
SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages
Exploring and quantifying semantic relatedness is central to representing language and holds significant implications across various NLP tasks.
LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
Pretrained language models (PLMs) have become remarkably adept at task and language generalization.
COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances
Unlike the previous Indonesian COPA dataset (XCOPA-ID), COPAL-ID incorporates Indonesian local and cultural nuances, and therefore, provides a more natural portrayal of day-to-day causal reasoning within the Indonesian cultural sphere.
QASiNa: Religious Domain Question Answering using Sirah Nabawiyah
This concludes Chat GPT is unsuitable for question answering task in religious domain especially for Islamic religion.
Low-Resource Clickbait Spoiling for Indonesian via Question Answering
Clickbait spoiling aims to generate a short text to satisfy the curiosity induced by a clickbait post.
NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages
We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets.
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs.
Style Over Substance: Evaluation Biases for Large Language Models
This study investigates the behavior of crowd-sourced and expert annotators, as well as LLMs, when comparing outputs from different models.
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
We validate the performance of our KAPING framework on the knowledge graph question answering task, that aims to answer the user's question based on facts over a knowledge graph, on which ours outperforms relevant zero-shot baselines by up to 48% in average, across multiple LLMs of various sizes.
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
This evidence-based position paper critiques current research practices within the language model pre-training literature.
Multi-lingual and Multi-cultural Figurative Language Understanding
Figurative language permeates human communication, but at the same time is relatively understudied in NLP.
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
These results show that the problem is far from solved and that there is a lot of room for improvement.
GlobalBench: A Benchmark for Global Progress in Natural Language Processing
Despite the major advances in NLP, significant disparities in NLP system performance across languages still exist.
Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation
However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages.
WebIE: Faithful and Robust Information Extraction on the Web
We evaluate the in-domain, out-of-domain, and zero-shot cross-lingual performance of generative IE models and find models trained on WebIE show better generalisability.
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited.
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Multilingual Large Language Models (LLMs) have recently shown great capabilities in a wide range of tasks, exhibiting state-of-the-art performance through zero-shot or few-shot prompting methods.
Direct Fact Retrieval from Knowledge Graphs without Entity Linking
There has been a surge of interest in utilizing Knowledge Graphs (KGs) for various natural language processing/understanding tasks.
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
We conclude by emphasizing the critical role of LLMs in guiding the future evolution of MT and offer a roadmap for future exploration in the sector.
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
The results demonstrate that our proposed LaMini-LM models are comparable to competitive baselines, while being much smaller in size.
Ranked #15 on
Word Sense Disambiguation
on Words in Context
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processing (NLP) research.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Code-Switching, a common phenomenon in written text and conversation, has been studied over decades by the natural language processing (NLP) research community.
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
We find language adaptation to be effective at improving zero-shot performance in new languages.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Crosslingual Generalization through Multitask Finetuning
We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus.
Ranked #1 on
Question Answering
on StoryCloze
Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering
We introduce Mintaka, a complex, natural, and multilingual dataset designed for experimenting with end-to-end question-answering models.
NusaCrowd: A Call for Open and Reproducible NLP Research in Indonesian Languages
At the center of the underlying issues that halt Indonesian natural language processing (NLP) research advancement, we find data scarcity.
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages
In this work, we focus on developing resources for languages in Indonesia.
ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair
We release our synthetic parallel paraphrase corpus across 17 languages: Arabic, Catalan, Czech, German, English, Spanish, Estonian, French, Hindi, Indonesian, Italian, Dutch, Romanian, Russian, Swedish, Vietnamese, and Chinese.
Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model.
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects.
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models.
Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
We perform knowledge distillation (KD) benchmark from task-specific BERT-base teacher models to various student models: BiLSTM, CNN, BERT-Tiny, BERT-Mini, and BERT-Small.
IndoNLI: A Natural Language Inference Dataset for Indonesian
The expert-annotated data is used exclusively as a test set.
Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
This paper explores augmenting monolingual data for knowledge distillation in neural machine translation.
Synthetic Source Language Augmentation for Colloquial Neural Machine Translation
Neural machine translation (NMT) is typically domain-dependent and style-dependent, and it requires lots of training data.
Costs to Consider in Adopting NLP for Your Business
Recent advances in Natural Language Processing (NLP) have largely pushed deep transformer-based models as the go-to state-of-the-art technique without much regard to the production and utilization cost.
Semi-Supervised Low-Resource Style Transfer of Indonesian Informal to Formal Language with Iterative Forward-Translation
In this work, we address a style-transfer from informal to formal Indonesian as a low-resource machine translation problem.
Compressing Neural Machine Translation Models with 4-bit Precision
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
In Neural Machine Translation, What Does Transfer Learning Transfer?
Transfer learning improves quality for low-resource machine translation, but it is unclear what exactly it transfers.
Edinburgh's Submissions to the 2020 Machine Translation Efficiency Task
We participated in all tracks of the Workshop on Neural Generation and Translation 2020 Efficiency Shared Task: single-core CPU, multi-core CPU, and GPU.
Benchmarking Multidomain English-Indonesian Machine Translation
In the context of Machine Translation (MT) from-and-to English, Bahasa Indonesia has been considered a low-resource language, and therefore applying Neural Machine Translation (NMT) which typically requires large training dataset proves to be problematic.
From Research to Production and Back: Ludicrously Fast Neural Machine Translation
Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models.
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
One way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients.
Neural Machine Translation with 4-Bit Precision and Beyond
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
Making Asynchronous Stochastic Gradient Descent Work for Transformers
Asynchronous stochastic gradient descent (SGD) is attractive from a speed perspective because workers do not wait for synchronization.
Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging
Previous work in Indonesian part-of-speech (POS) tagging are hard to compare as they are not evaluated on a common dataset.
Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation
In order to extract the best possible performance from asynchronous stochastic gradient descent one must increase the mini-batch size and scale the learning rate accordingly.
Marian: Fast Neural Machine Translation in C++
We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs.
Sparse Communication for Distributed Gradient Descent
Most configurations work on MNIST, whereas different configurations reduce convergence rate on the more complex translation task.
