Search Results for author:
Found 36 papers, 16 papers with code
A Relation Extraction Dataset for Knowledge Extraction from Web Tables
Most datasets are constructed using synthetic tables that lack valuable metadata information, or are limited in size to be considered as a challenging evaluation set.
The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
We describe the University of Edinburgh’s Bengali\leftrightarrowHindi constrained systems submitted to the WMT21 News Translation task.
Efficient Machine Translation with Model Pruning and Quantization
We participated in all tracks of the WMT 2021 efficient machine translation task: single-core CPU, multi-core CPU, and GPU hardware with throughput and latency conditions.
BERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter
This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task.
Towards better structured and less noisy Web data: Oscar with Register annotations
Web-crawled datasets are known to be noisy, as they feature a wide range of language use covering both user-generated and professionally edited content as well as noise originating from the crawling process.
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Code-Switching, a common phenomenon in written text and conversation, has been studied over decades by the natural language processing (NLP) research community.
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
The BLOOM model is a large open-source multilingual language model capable of zero-shot learning, but its pretraining was limited to 46 languages.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
We present NusaCrowd, a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Crosslingual Generalization through Multitask Finetuning
We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus.
Ranked #1 on
Cross-Lingual Transfer
on XCOPA
Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering
We introduce Mintaka, a complex, natural, and multilingual dataset designed for experimenting with end-to-end question-answering models.
NusaCrowd: A Call for Open and Reproducible NLP Research in Indonesian Languages
At the center of the underlying issues that halt Indonesian natural language processing (NLP) research advancement, we find data scarcity.
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages
In this work, we focus on developing resources for languages in Indonesia.
ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair
We release our synthetic parallel paraphrase corpus across 17 languages: Arabic, Catalan, Czech, German, English, Spanish, Estonian, French, Hindi, Indonesian, Italian, Dutch, Romanian, Russian, Swedish, Vietnamese, and Chinese.
Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model.
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects.
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models.
Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
We perform knowledge distillation (KD) benchmark from task-specific BERT-base teacher models to various student models: BiLSTM, CNN, BERT-Tiny, BERT-Mini, and BERT-Small.
IndoNLI: A Natural Language Inference Dataset for Indonesian
The expert-annotated data is used exclusively as a test set.
Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
This paper explores augmenting monolingual data for knowledge distillation in neural machine translation.
Synthetic Source Language Augmentation for Colloquial Neural Machine Translation
Neural machine translation (NMT) is typically domain-dependent and style-dependent, and it requires lots of training data.
Costs to Consider in Adopting NLP for Your Business
Recent advances in Natural Language Processing (NLP) have largely pushed deep transformer-based models as the go-to state-of-the-art technique without much regard to the production and utilization cost.
Semi-Supervised Low-Resource Style Transfer of Indonesian Informal to Formal Language with Iterative Forward-Translation
In this work, we address a style-transfer from informal to formal Indonesian as a low-resource machine translation problem.
Edinburgh's Submissions to the 2020 Machine Translation Efficiency Task
We participated in all tracks of the Workshop on Neural Generation and Translation 2020 Efficiency Shared Task: single-core CPU, multi-core CPU, and GPU.
Compressing Neural Machine Translation Models with 4-bit Precision
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
In Neural Machine Translation, What Does Transfer Learning Transfer?
Transfer learning improves quality for low-resource machine translation, but it is unclear what exactly it transfers.
Benchmarking Multidomain English-Indonesian Machine Translation
In the context of Machine Translation (MT) from-and-to English, Bahasa Indonesia has been considered a low-resource language, and therefore applying Neural Machine Translation (NMT) which typically requires large training dataset proves to be problematic.
From Research to Production and Back: Ludicrously Fast Neural Machine Translation
Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models.
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
One way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients.
Neural Machine Translation with 4-Bit Precision and Beyond
We empirically show that NMT models based on Transformer or RNN architecture can be compressed up to 4-bit precision without any noticeable quality degradation.
Making Asynchronous Stochastic Gradient Descent Work for Transformers
Asynchronous stochastic gradient descent (SGD) is attractive from a speed perspective because workers do not wait for synchronization.
Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging
Previous work in Indonesian part-of-speech (POS) tagging are hard to compare as they are not evaluated on a common dataset.
Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation
In order to extract the best possible performance from asynchronous stochastic gradient descent one must increase the mini-batch size and scale the learning rate accordingly.
Marian: Fast Neural Machine Translation in C++
We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs.
Sparse Communication for Distributed Gradient Descent
Most configurations work on MNIST, whereas different configurations reduce convergence rate on the more complex translation task.
