Search Results for author:
Found 75 papers, 26 papers with code
Overview of the 9th Workshop on Asian Translation
This paper presents the results of the shared tasks from the 9th workshop on Asian translation (WAT2022).
Overview of the 8th Workshop on Asian Translation
This paper presents the results of the shared tasks from the 8th workshop on Asian translation (WAT2021).
Bilingual Tabular Inference: A Case Study on Indic Languages
Existing research on Tabular Natural Language Inference (TNLI) exclusively examines the task in a monolingual setting where the tabular premise and hypothesis are in the same language.
Contact Relatedness can help improve multilingual NMT: Microsoft STCI-MT @ WMT20
We describe our submission for the English→Tamil and Tamil→English news translation shared task.
Overview of the 7th Workshop on Asian Translation
This paper presents the results of the shared tasks from the 7th workshop on Asian translation (WAT2020).
Pralekha: An Indic Document Alignment Evaluation Benchmark
To address the challenge of aligning documents using sentence and chunk-level alignments, we propose a novel scoring method, Document Alignment Coefficient (DAC).
BhasaAnuvaad: A Speech Translation Dataset for 13 Indian Languages
To this end, we introduce BhasaAnuvaad, the largest publicly available dataset for AST involving 13 out of 22 scheduled Indian languages and English spanning over 44, 400 hours and 17M text segments.
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
This would enable benchmarking of general-purpose multilingual LLMs and facilitate meta-evaluation of Evaluator LLMs.
An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
Language Models (LMs) excel in natural language processing tasks for English but show reduced performance in most other languages.
How Good is Zero-Shot MT Evaluation for Low Resource Indian Languages?
While machine translation evaluation has been studied primarily for high-resource languages, there has been a recent interest in evaluation for low-resource languages due to the increasing availability of data and models.
Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation
In this paper, we tackle the problem of code-mixed (Hinglish and Bengalish) to English machine translation.
IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages
We hope that the datasets, tools, and resources released as a part of this work will not only propel the research and development of Indic LLMs but also establish an open-source blueprint for extending such efforts to other languages.
Airavata: Introducing Hindi Instruction-tuned LLM
We announce the initial release of "Airavata," an instruction-tuned LLM for Hindi.
RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization
This study addresses the challenge of extending Large Language Models (LLMs) to non-English languages that use non-Roman scripts.
Bhasha-Abhijnaanam: Native-script and romanized Language Identification for 22 Indic languages
We create publicly available language identification (LID) datasets and models in all 22 Indian languages listed in the Indian constitution in both native-script and romanized text.
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
Prior to this work, there was (i) no parallel training data spanning all 22 languages, (ii) no robust benchmarks covering all these languages and containing content relevant to India, and (iii) no existing translation models which support all the 22 scheduled languages of India.
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation
We learn a regression model, CTQ Scorer (Contextual Translation Quality), that selects examples based on multiple features in order to maximize the translation quality.
Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models
This study investigates machine translation between related languages i. e., languages within the same family that share linguistic characteristics such as word order and lexical similarity.
A Comprehensive Analysis of Adapter Efficiency
However, adapters have not been sufficiently analyzed to understand if PEFT translates to benefits in training/deployment efficiency and maintainability/extensibility.
Natural Language Understanding
parameter-efficient fine-tuning
CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages
We address the task of machine translation (MT) from extremely low-resource language (ELRL) to English by leveraging cross-lingual transfer from 'closely-related' high-resource language (HRL).
Evaluating Inter-Bilingual Semantic Parsing for Indian Languages
Despite significant progress in Natural Language Generation for Indian languages (IndicNLP), there is a lack of datasets around complex structured tasks such as semantic parsing.
IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages
In this paper, we fill this gap by creating an MQM dataset consisting of 7000 fine-grained annotations, spanning 5 Indian languages and 7 MT systems, and use it to establish correlations between annotator scores and scores obtained using existing automatic metrics.
Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages
The dataset contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location, and, Organization) for 9 out of the 11 languages.
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages
Across languages and tasks, IndicXTREME contains a total of 105 evaluation sets, of which 52 are new contributions to the literature.
Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages
Significantly, we show that adding Shrutilipi to the training set of Wav2Vec models leads to an average decrease in WER of 5. 8\% for 7 languages on the IndicSUPERB benchmark.
IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages
We hope IndicSUPERB contributes to the progress of developing speech language understanding models for Indian languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users
Transliteration is very important in the Indian language context due to the usage of multiple scripts and the widespread use of romanized inputs.
IndicXNLI: Evaluating Multilingual Inference for Indian Languages
While Indic NLP has made rapid advances recently in terms of the availability of corpora and pre-trained models, benchmark datasets on standard NLU tasks are limited.
IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages
Natural Language Generation (NLG) for non-English languages is hampered by the scarcity of datasets in these languages.
Towards Building ASR Systems for the Next Billion Users
Second, using this raw speech data we pretrain several variants of wav2vec style models for 40 Indian languages.
An Empirical Investigation of Multi-bridge Multilingual NMT models
In this paper, we present an extensive investigation of multi-bridge, many-to-many multilingual NMT models (MB-M2M) ie., models trained on non-English language pairs in addition to English-centric language pairs.
IndicBART: A Pre-trained Model for Indic Natural Language Generation
We present IndicBART, a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English.
A Primer on Pretrained Multilingual Language Models
Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.}
Joint Multilingual Sentence Representations
Multilingual text classification
+5
Itihasa: A large-scale corpus for Sanskrit to English translation
This work introduces Itihasa, a large-scale translation dataset containing 93, 000 pairs of Sanskrit shlokas and their English translations.
Ranked #1 on
Machine Translation
on Itihasa
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages
We mine the parallel sentences from the web by combining many corpora, tools, and methods: (a) web-crawled monolingual corpora, (b) document OCR for extracting sentences from scanned documents, (c) multilingual representation models for aligning sentences, and (d) approximate nearest neighbor search for searching in a large collection of sentences.
A Large-scale Evaluation of Neural Machine Transliteration for Indic Languages
We take up the task of large-scale evaluation of neural machine transliteration between English and Indic languages, with a focus on multilingual transliteration to utilize orthographic similarity between Indian languages.
Multilingual Neural Machine Translation
The advent of neural machine translation (NMT) has opened up exciting research in building multilingual translation systems i. e. translation models that can handle more than one language pair.
IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages
These resources include: (a) large-scale sentence-level monolingual corpora, (b) pre-trained word embeddings, (c) pre-trained language models, and (d) multiple NLU evaluation datasets (IndicGLUE benchmark).
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages
We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2. 7 billion words for 10 Indian languages from two language families.
Learning Geometric Word Meta-Embeddings
We propose a geometric framework for learning meta-embeddings of words from different embedding sources.
Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent
To the best of our knowledge, this is the first large-scale study specifically devoted to utilizing language relatedness to improve translation between related languages.
A Comprehensive Survey of Multilingual Neural Machine Translation
We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years.
Overview of the 6th Workshop on Asian Translation
This paper presents the results of the shared tasks from the 6th workshop on Asian translation (WAT2019) including Ja↔En, Ja↔Zh scientific paper translation subtasks, Ja↔En, Ja↔Ko, Ja↔En patent translation subtasks, Hi↔En, My↔En, Km↔En, Ta↔En mixed domain subtasks and Ru↔Ja news commentary translation task.
A Brief Survey of Multilingual Neural Machine Translation
We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years.
Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages
To bridge this divergence, We propose to pre-order the assisting language sentence to match the word order of the source language and train the parent model.
McTorch, a manifold optimization library for deep learning
In this paper, we introduce McTorch, a manifold optimization library for deep learning that extends PyTorch.
Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach
Our approach decouples learning the transformation from the source language to the target language into (a) learning rotations for language-specific embeddings to align them to a common space, and (b) learning a similarity metric in the common space to model similarities between the embeddings.
Judicious Selection of Training Data in Assisting Language for Multilingual Neural NER
Multilingual learning for Neural Named Entity Recognition (NNER) involves jointly training a neural network for multiple languages.
Leveraging Orthographic Similarity for Multilingual Neural Transliteration
We address the task of joint training of transliteration models for multiple language pairs (multilingual transliteration).
Comparing Recurrent and Convolutional Architectures for English-Hindi Neural Machine Translation
In this paper, we empirically compare the two encoder-decoder neural machine translation architectures: convolutional sequence to sequence model (ConvS2S) and recurrent sequence to sequence model (RNNS2S) for English-Hindi language pair as part of IIT Bombay{'}s submission to WAT2017 shared task.
The IIT Bombay English-Hindi Parallel Corpus
We present the IIT Bombay English-Hindi Parallel Corpus.
Utilizing Lexical Similarity between Related, Low-resource Languages for Pivot-based SMT
We investigate pivot-based translation between related languages in a low resource, phrase-based SMT setting.
IIT Bombay's English-Indonesian submission at WAT: Integrating Neural Language Models with SMT
The Neural Probabilistic Language Model (NPLM) gave relatively high BLEU points for Indonesian to English translation system while the Neural Network Joint Model (NNJM) performed better for English to Indonesian direction of translation system.
Faster decoding for subword level Phrase-based SMT between related languages
The increase in length is also impacted by the specific choice of data format for representing the sentences as subwords.
Learning variable length units for SMT between related languages via Byte Pair Encoding
We explore the use of segments learnt using Byte Pair Encoding (referred to as BPE units) as basic units for statistical machine translation between related languages and compare it with orthographic syllables, which are currently the best performing basic units for this translation task.
Orthographic Syllable as basic unit for SMT between Related Languages
We explore the use of the orthographic syllable, a variable-length consonant-vowel sequence, as a basic unit of translation between related languages which use abugida or alphabetic scripts.
Shata-Anuvadak: Tackling Multiway Translation of Indian Languages
We present a compendium of 110 Statistical Machine Translation systems built from parallel corpora of 11 Indian languages belonging to both Indo-Aryan and Dravidian families.
When Transliteration Met Crowdsourcing : An Empirical Study of Transliteration via Crowdsourcing using Efficient, Non-redundant and Fair Quality Control
In contrast, we propose a low-cost QC mechanism which is fair to both workers and requesters.
Experiences in Resource Generation for Machine Translation through Crowdsourcing
The logistics of collecting resources for Machine Translation (MT) has always been a cause of concern for some of the resource deprived languages of the world.
