Search Results for author:
Found 63 papers, 26 papers with code
Sparse Communication for Distributed Gradient Descent
Most configurations work on MNIST, whereas different configurations reduce convergence rate on the more complex translation task.
Marian: Fast Neural Machine Translation in C++
We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs.
Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation
In order to extract the best possible performance from asynchronous stochastic gradient descent one must increase the mini-batch size and scale the learning rate accordingly.
Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging
Previous work in Indonesian part-of-speech (POS) tagging are hard to compare as they are not evaluated on a common dataset.
Making Asynchronous Stochastic Gradient Descent Work for Transformers
Asynchronous stochastic gradient descent (SGD) is attractive from a speed perspective because workers do not wait for synchronization.
Neural Machine Translation with 4-Bit Precision and Beyond
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
From Research to Production and Back: Ludicrously Fast Neural Machine Translation
Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models.
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
One way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients.
Benchmarking Multidomain English-Indonesian Machine Translation
In the context of Machine Translation (MT) from-and-to English, Bahasa Indonesia has been considered a low-resource language, and therefore applying Neural Machine Translation (NMT) which typically requires large training dataset proves to be problematic.
Edinburgh's Submissions to the 2020 Machine Translation Efficiency Task
We participated in all tracks of the Workshop on Neural Generation and Translation 2020 Efficiency Shared Task: single-core CPU, multi-core CPU, and GPU.
In Neural Machine Translation, What Does Transfer Learning Transfer?
Transfer learning improves quality for low-resource machine translation, but it is unclear what exactly it transfers.
Compressing Neural Machine Translation Models with 4-bit Precision
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
Semi-Supervised Low-Resource Style Transfer of Indonesian Informal to Formal Language with Iterative Forward-Translation
In this work, we address a style-transfer from informal to formal Indonesian as a low-resource machine translation problem.
Costs to Consider in Adopting NLP for Your Business
Recent advances in Natural Language Processing (NLP) have largely pushed deep transformer-based models as the go-to state-of-the-art technique without much regard to the production and utilization cost.
Synthetic Source Language Augmentation for Colloquial Neural Machine Translation
Neural machine translation (NMT) is typically domain-dependent and style-dependent, and it requires lots of training data.
Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
This paper explores augmenting monolingual data for knowledge distillation in neural machine translation.
IndoNLI: A Natural Language Inference Dataset for Indonesian
The expert-annotated data is used exclusively as a test set.
Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
We perform knowledge distillation (KD) benchmark from task-specific BERT-base teacher models to various student models: BiLSTM, CNN, BERT-Tiny, BERT-Mini, and BERT-Small.
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models.
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects.
Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model.
ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair
We release our synthetic parallel paraphrase corpus across 17 languages: Arabic, Catalan, Czech, German, English, Spanish, Estonian, French, Hindi, Indonesian, Italian, Dutch, Romanian, Russian, Swedish, Vietnamese, and Chinese.
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages
In this work, we focus on developing resources for languages in Indonesia.
NusaCrowd: A Call for Open and Reproducible NLP Research in Indonesian Languages
At the center of the underlying issues that halt Indonesian natural language processing (NLP) research advancement, we find data scarcity.
Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering
We introduce Mintaka, a complex, natural, and multilingual dataset designed for experimenting with end-to-end question-answering models.
Crosslingual Generalization through Multitask Finetuning
We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus.
Ranked #1 on
Question Answering
on StoryCloze
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Code-Switching, a common phenomenon in written text and conversation, has been studied over decades by the natural language processing (NLP) research community.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
We find language adaptation to be effective at improving zero-shot performance in new languages.
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processing (NLP) research.
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
The results demonstrate that our proposed LaMini-LM models are comparable to competitive baselines, while being much smaller in size.
Ranked #15 on
Word Sense Disambiguation
on Words in Context
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
We conclude by emphasizing the critical role of LLMs in guiding the future evolution of MT and offer a roadmap for future exploration in the sector.
Direct Fact Retrieval from Knowledge Graphs without Entity Linking
There has been a surge of interest in utilizing Knowledge Graphs (KGs) for various natural language processing/understanding tasks.
WebIE: Faithful and Robust Information Extraction on the Web
We evaluate the in-domain, out-of-domain, and zero-shot cross-lingual performance of generative IE models and find models trained on WebIE show better generalisability.
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited.
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Multilingual Large Language Models (LLMs) have recently shown great capabilities in a wide range of tasks, exhibiting state-of-the-art performance through zero-shot or few-shot prompting methods.
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
These results show that the problem is far from solved and that there is a lot of room for improvement.
GlobalBench: A Benchmark for Global Progress in Natural Language Processing
Despite the major advances in NLP, significant disparities in NLP system performance across languages still exist.
Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation
However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages.
Multi-lingual and Multi-cultural Figurative Language Understanding
Figurative language permeates human communication, but at the same time is relatively understudied in NLP.
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
This evidence-based position paper critiques current research practices within the language model pre-training literature.
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
We validate the performance of our KAPING framework on the knowledge graph question answering task, that aims to answer the user's question based on facts over a knowledge graph, on which ours outperforms relevant zero-shot baselines by up to 48% in average, across multiple LLMs of various sizes.
Style Over Substance: Evaluation Biases for Large Language Models
This study investigates the behavior of crowd-sourced and expert annotators, as well as LLMs, when comparing outputs from different models.
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs.
NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages
We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets.
Low-Resource Clickbait Spoiling for Indonesian via Question Answering
Clickbait spoiling aims to generate a short text to satisfy the curiosity induced by a clickbait post.
QASiNa: Religious Domain Question Answering using Sirah Nabawiyah
This concludes Chat GPT is unsuitable for question answering task in religious domain especially for Islamic religion.
COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances
We present publicly available COPAL-ID, a novel Indonesian language common sense reasoning dataset.
LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
Pretrained language models (PLMs) have shown remarkable generalization toward multiple tasks and languages.
SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 14 Languages
Exploring and quantifying semantic relatedness is central to representing language.
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels.
Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
Large Language Models (LLMs) have demonstrated remarkable capabilities across various applications, fundamentally reshaping the landscape of natural language processing (NLP) research.
Daisy-TTS: Simulating Wider Spectrum of Emotions via Prosody Embedding Decomposition
This wide spectrum of emotions is well-studied in the structural model of emotions, which represents variety of emotions as derivative products of primary emotions with varying degrees of intensity.
Towards Measuring and Modeling "Culture" in LLMs: A Survey
We present a survey of 39 recent papers that aim to study cultural representation and inclusion in large language models.
SemEval Task 1: Semantic Textual Relatedness for African and Asian Languages
We present the first shared task on Semantic Textual Relatedness (STR).
Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languages
To bridge this quality gap, we introduce Cendol, a collection of Indonesian LLMs encompassing both decoder-only and encoder-decoder architectures across a range of model sizes.
A Relation Extraction Dataset for Knowledge Extraction from Web Tables
Most datasets are constructed using synthetic tables that lack valuable metadata information, or are limited in size to be considered as a challenging evaluation set.
BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter
This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task.
Towards better structured and less noisy Web data: Oscar with Register annotations
Web-crawled datasets are known to be noisy, as they feature a wide range of language use covering both user-generated and professionally edited content as well as noise originating from the crawling process.
The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
We describe the University of Edinburgh’s Bengali\leftrightarrowHindi constrained systems submitted to the WMT21 News Translation task.
Efficient Machine Translation with Model Pruning and Quantization
We participated in all tracks of the WMT 2021 efficient machine translation task: single-core CPU, multi-core CPU, and GPU hardware with throughput and latency conditions.
