Search Results for author:
Found 91 papers, 47 papers with code
On the weak link between importance and prunability of attention heads
Given the success of Transformer-based models, two directions of study have emerged: interpreting role of individual attention heads and down-sizing the models for efficiency.
RomanLens: Latent Romanization and its role in Multilinguality in LLMs
For non-Latin script languages, we investigate the role of romanization - the representation of non-Latin scripts using Latin characters - as a bridge in multilingual processing.
Can Vision-Language Models Evaluate Handwritten Math?
To address this gap, we introduce FERMAT, a benchmark designed to assess the ability of VLMs to detect, localize and correct errors in handwritten mathematical content.
Pralekha: An Indic Document Alignment Evaluation Benchmark
To address the challenge of aligning documents using sentence and chunk-level alignments, we propose a novel scoring method, Document Alignment Coefficient (DAC).
Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation
The MUSHRA test is a promising alternative for evaluating multiple TTS systems simultaneously, but in this work we show that its reliance on matching human reference speech unduly penalises the scores of modern TTS systems that can exceed human speech quality.
ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams
We propose this methodology as a viable alternative for languages with limited access to high-quality data, enabling them to collectively benefit from shared resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
This would enable benchmarking of general-purpose multilingual LLMs and facilitate meta-evaluation of Evaluator LLMs.
Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling
In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi.
LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems
Hindi, one of the most spoken language of India, exhibits a diverse array of accents due to its usage among individuals from diverse linguistic origins.
Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings
We release Rasa, the first multilingual expressive TTS dataset for any Indian language, which contains 10 hours of neutral speech and 1-3 hours of expressive speech for each of the 6 Ekman emotions covering 3 languages: Assamese, Bengali, & Tamil.
Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategies
Publicly available TTS datasets for low-resource languages like Hindi and Tamil typically contain 10-20 hours of data, leading to poor vocabulary coverage.
An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
Language Models (LMs) excel in natural language processing tasks for English but show reduced performance in most other languages.
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists
Large Language Models (LLMs) are increasingly relied upon to evaluate text outputs of other LLMs, thereby influencing leaderboards and development decisions.
IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages
We hope that the datasets, tools, and resources released as a part of this work will not only propel the research and development of Indic LLMs but also establish an open-source blueprint for extending such efforts to other languages.
Airavata: Introducing Hindi Instruction-tuned LLM
We announce the initial release of "Airavata," an instruction-tuned LLM for Hindi.
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
Prior to this work, there was (i) no parallel training data spanning all 22 languages, (ii) no robust benchmarks covering all these languages and containing content relevant to India, and (iii) no existing translation models which support all the 22 scheduled languages of India.
Svarah: Evaluating English ASR Systems on Indian Accents
India is the second largest English-speaking country in the world with a speaker base of roughly 130 million.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Bhasha-Abhijnaanam: Native-script and romanized Language Identification for 22 Indic languages
We create publicly available language identification (LID) datasets and models in all 22 Indian languages listed in the Indian constitution in both native-script and romanized text.
Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR
In this paper, we focus on Indian languages, and make the case that diverse benchmarks are required to evaluate and improve ASR systems for Indian languages.
A Comprehensive Analysis of Adapter Efficiency
However, adapters have not been sufficiently analyzed to understand if PEFT translates to benefits in training/deployment efficiency and maintainability/extensibility.
Natural Language Understanding
parameter-efficient fine-tuning
IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages
In this paper, we fill this gap by creating an MQM dataset consisting of 7000 fine-grained annotations, spanning 5 Indian languages and 7 MT systems, and use it to establish correlations between annotator scores and scores obtained using existing automatic metrics.
Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages
The dataset contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location, and, Organization) for 9 out of the 11 languages.
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages
Across languages and tasks, IndicXTREME contains a total of 105 evaluation sets, of which 52 are new contributions to the literature.
Towards Building Text-To-Speech Systems for the Next Billion Users
We open-source all models on the Bhashini platform.
Ranked #1 on
Speech Synthesis - Hindi
on IndicTTS
Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages
Significantly, we show that adding Shrutilipi to the training set of Wav2Vec models leads to an average decrease in WER of 5. 8\% for 7 languages on the IndicSUPERB benchmark.
IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages
We hope IndicSUPERB contributes to the progress of developing speech language understanding models for Indian languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users
Transliteration is very important in the Indian language context due to the usage of multiple scripts and the widespread use of romanized inputs.
Joint Transformer/RNN Architecture for Gesture Typing in Indic Languages
Gesture typing is a method of typing words on a touch-based keyboard by creating a continuous trace passing through the relevant keys.
A Survey of Adversarial Defences and Robustness in NLP
In the past few years, it has become increasingly evident that deep neural networks are not resilient enough to withstand adversarial perturbations in input data, leaving them vulnerable to attack.
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons
In this work, we introduce Active Evaluation, a framework to efficiently identify the top-ranked system by actively choosing system pairs for comparison using dueling bandit algorithms.
IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages
Natural Language Generation (NLG) for non-English languages is hampered by the scarcity of datasets in these languages.
Towards Building ASR Systems for the Next Billion Users
Second, using this raw speech data we pretrain several variants of wav2vec style models for 40 Indian languages.
A Framework for Rationale Extraction for Deep QA models
As neural-network-based QA models become deeper and more complex, there is a demand for robust frameworks which can access a model's rationale for its prediction.
On the Prunability of Attention Heads in Multilingual BERT
Large multilingual models, such as mBERT, have shown promise in crosslingual transfer.
Perturbation CheckLists for Evaluating NLG Evaluation Metrics
Natural Language Generation (NLG) evaluation is a multifaceted task requiring assessment of multiple desirable criteria, e. g., fluency, coherency, coverage, relevance, adequacy, overall quality, etc.
IndicBART: A Pre-trained Model for Indic Natural Language Generation
We present IndicBART, a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English.
A Primer on Pretrained Multilingual Language Models
Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.}
Joint Multilingual Sentence Representations
Multilingual text classification
+5
A Tutorial on Evaluation Metrics used in Natural Language Generation
(iv) What are the criticisms and shortcomings of existing metrics?
The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT
There are two main challenges with existing methods for classification: (a) there are no standard scores across studies or across functional roles, and (b) these scores are often average quantities measured across sentences without capturing statistical significance.
Unsupervised Deep Video Denoising
This is advantageous because motion compensation is computationally expensive, and can be unreliable when the input data are noisy.
Ranked #5 on
Video Denoising
on Set8 sigma40
IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages
These resources include: (a) large-scale sentence-level monolingual corpora, (b) pre-trained word embeddings, (c) pre-trained language models, and (d) multiple NLU evaluation datasets (IndicGLUE benchmark).
Towards Interpreting BERT for Reading Comprehension Based QA
BERT and its variants have achieved state-of-the-art performance in various NLP tasks.
Evaluating a Generative Adversarial Framework for Information Retrieval
Recent advances in Generative Adversarial Networks (GANs) have resulted in its widespread applications to multiple domains.
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining
However, no such data is publicly available, and hence existing models are usually trained using a single relevant response and multiple randomly selected responses from other contexts (random negatives).
A Survey of Evaluation Metrics Used for NLG Systems
The expanding number of NLG models and the shortcomings of the current metrics has led to a rapid surge in the number of evaluation metrics proposed since 2014.
A Systematic Evaluation of Object Detection Networks for Scientific Plots
However, the performance drops drastically when evaluated at a stricter IOU of 0. 9 with the best model giving a mAP of 35. 70%.
On Incorporating Structural Information to improve Dialogue Response Generation
This requires capturing structural, sequential and semantic information from the conversation context and the background resources.
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages
We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2. 7 billion words for 10 Indian languages from two language families.
Towards Transparent and Explainable Attention Models
To make attention mechanisms more faithful and plausible, we propose a modified LSTM cell with a diversity-driven training objective that ensures that the hidden representations learned at different time steps are diverse.
Scene Graph based Image Retrieval -- A case study on the CLEVR Dataset
With the prolification of multimodal interaction in various domains, recently there has been much interest in text based image retrieval in the computer vision community.
PlotQA: Reasoning over Scientific Plots
However, in practice, this is an unrealistic assumption because many questions require reasoning and thus have real-valued answers which appear neither in a small fixed size vocabulary nor in the image.
Ranked #2 on
Chart Question Answering
on RealCQA
Let's Ask Again: Refine Network for Automatic Question Generation
It is desired that the generated question should be (i) grammatically correct (ii) answerable from the passage and (iii) specific to the given answer.
On Knowledge distillation from complex networks for response prediction
In order to overcome this, we use standard simple models which do not capture all pairwise interactions, but learn to emulate certain characteristics of a complex teacher network.
Graph Convolutional Network with Sequential Attention For Goal-Oriented Dialogue Systems
Domain specific goal-oriented dialogue systems typically require modeling three types of inputs, viz., (i) the knowledge-base associated with the domain, (ii) the history of the conversation, which is a sequence of utterances and (iii) the current utterance for which the response needs to be generated.
Dissecting an Adversarial framework for Information Retrieval
Recent advances in Generative Adversarial Networks facilitated by improvements to the framework and successful application to various problems has resulted in extensions to multiple domains.
Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples
We believe that the non-adversarial dataset created as a part of this work would complement the research on adversarial evaluation and give a more realistic assessment of the ability of RC models.
ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions
Specifically, it has gates which decide whether an option can be eliminated given the passage, question pair and if so it tries to make the passage representation orthogonal to this eliminated option (akin to ignoring portions of the passage corresponding to the eliminated option).
Efficient Video Classification Using Fewer Frames
We focus on building compute-efficient video classification models which process fewer frames and hence have less number of FLOPs.
Ranked #2 on
Video Classification
on YouTube-8M
Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses
ADEM(Lowe et al. 2017) formulated the automatic evaluation of dialogue systems as a learning problem and showed that such a model was able to predict responses which correlate significantly with human judgements, both at utterance and system level.
Studying the Plasticity in Deep Convolutional Neural Networks using Random Pruning
In this work, we report experiments which suggest that the comparable performance of the pruned network is not due to the specific criterion chosen but due to the inherent plasticity of deep neural networks which allows them to recover from the loss of pruned filters once the rest of the filters are fine-tuned.
On Controllable Sparse Alternatives to Softmax
Converting an n-dimensional vector to a probability distribution over n objects is a commonly used component in many machine learning tasks like multiclass classification, multilabel classification, attention mechanisms etc.
Towards Exploiting Background Knowledge for Building Conversation Systems
Existing dialog datasets contain a sequence of utterances and responses without any explicit background knowledge associated with them.
Towards a Better Metric for Evaluating Question Generation Systems
In particular, it is important to verify whether such metrics used for evaluating AQG systems focus on answerability of the generated question by preferring questions which contain all relevant information such as question type (Wh-types), entities, relations, etc.
A Dataset for Building Code-Mixed Goal Oriented Conversation Systems
("Can you help me in booking a table at this restaurant?").
FigureNet: A Deep Learning model for Question-Answering on Scientific Plots
Deep Learning has managed to push boundaries in a wide variety of tasks.
Fusion Graph Convolutional Networks
State-of-the-art models for node classification on such attributed graphs use differentiable recursive functions that enable aggregation and filtering of neighborhood information from multiple hops.
HOPF: Higher Order Propagation Framework for Deep Collective Classification
Given a graph where every node has certain attributes associated with it and some nodes have labels associated with them, Collective Classification (CC) is the task of assigning labels to every unlabeled node using information from the node as well as its neighbors.
I Have Seen Enough: A Teacher Student Network for Video Classification Using Fewer Frames
We then train a student network whose objective is to process only a small fraction of the frames in the video and still produce a representation which is very close to the representation computed by the teacher network.
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension
We propose DuoRC, a novel dataset for Reading Comprehension (RC) that motivates several new challenges for neural approaches in language understanding beyond those offered by existing RC datasets.
A Mixed Hierarchical Attention based Encoder-Decoder Approach for Standard Table Summarization
Structured data summarization involves generation of natural language summaries from structured input data.
Generating Descriptions from Structured Data Using a Bifocal Attention Mechanism and Gated Orthogonalization
For example, while generating descriptions from a table, a human would attend to information at two levels: (i) the fields (macro level) and (ii) the values within the field (micro level).
Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks
In this work, we report experiments which suggest that the comparable performance of the pruned network is not due to the specific criterion chosen but due to the inherent plasticity of deep neural networks which allows them to recover from the loss of pruned filters once the rest of the filters are fine-tuned.
Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph
Further, unlike existing large scale QA datasets which contain simple questions that can be answered from a single tuple, the questions in our dialogs require a larger subgraph of the KG.
Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model
To generate such QA pairs, we first extract a set of keywords from entities and relationships expressed in a triple stored in the knowledge graph.
A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation
However, there is no parallel training data available between X and Y but, training data is available between X & Z and Z & Y (as is often the case in many real world applications).
Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning
In this work, we address a real-world scenario where no direct parallel data is available between two views of interest (say, $V_1$ and $V_2$) but parallel data is available between each of these views and a pivot view ($V_3$).
Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain
Second, the agent should be able to selectively transfer, which is the ability to select and transfer from different and multiple source tasks for different parts of the state space of the target task.
Correlational Neural Networks
CCA based approaches learn a joint representation by maximizing correlation of the views when projected to the common subspace.
When Transliteration Met Crowdsourcing : An Empirical Study of Transliteration via Crowdsourcing using Efficient, Non-redundant and Fair Quality Control
In contrast, we propose a low-cost QC mechanism which is fair to both workers and requesters.
An Autoencoder Approach to Learning Bilingual Word Representations
Cross-language learning allows us to use training data from one language to build models for a different language.
Experiences in Resource Generation for Machine Translation through Crowdsourcing
The logistics of collecting resources for Machine Translation (MT) has always been a cause of concern for some of the resource deprived languages of the world.
