Search Results for author: Pratyush Kumar

Found 40 papers, 16 papers with code

On the weak link between importance and prunability of attention heads

no code implementations EMNLP 2020 Aakriti Budhraja, Madhura Pande, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra

Given the success of Transformer-based models, two directions of study have emerged: interpreting role of individual attention heads and down-sizing the models for efficiency.

IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages

1 code implementation11 Mar 2024 Mohammed Safi Ur Rahman Khan, Priyam Mehta, Ananth Sankar, Umashankar Kumaravelan, Sumanth Doddapaneni, Suriyaprasaad G, Varun Balan G, Sparsh Jain, Anoop Kunchukuttan, Pratyush Kumar, Raj Dabre, Mitesh M. Khapra

We hope that the datasets, tools, and resources released as a part of this work will not only propel the research and development of Indic LLMs but also establish an open-source blueprint for extending such efforts to other languages.

DSFormer: Effective Compression of Text-Transformers by Dense-Sparse Weight Factorization

no code implementations20 Dec 2023 Rahul Chand, Yashoteja Prabhu, Pratyush Kumar

Extensive experiments on multiple natural language understanding benchmarks demonstrate that DSFormer obtains up to 40% better compression than the state-of-the-art low-rank factorizers, leading semi-structured sparsity baselines and popular knowledge distillation approaches.

Knowledge Distillation Natural Language Understanding

IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages

2 code implementations25 May 2023 Jay Gala, Pranjal A. Chitale, Raghavan AK, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar, Janki Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, Anoop Kunchukuttan

Prior to this work, there was (i) no parallel training data spanning all 22 languages, (ii) no robust benchmarks covering all these languages and containing content relevant to India, and (iii) no existing translation models which support all the 22 scheduled languages of India.

Machine Translation Sentence +1

Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR

1 code implementation24 May 2023 Kaushal Santosh Bhogale, Sai Sundaresan, Abhigyan Raman, Tahir Javed, Mitesh M. Khapra, Pratyush Kumar

In this paper, we focus on Indian languages, and make the case that diverse benchmarks are required to evaluate and improve ASR systems for Indian languages.

Large Language Models Humanize Technology

no code implementations9 May 2023 Pratyush Kumar

Given this opportunity to humanize technology widely, we advocate for more widespread understanding of LLMs, tools and methods to simplify use of LLMs, and cross-cutting institutional capacity.

Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages

1 code implementation20 Dec 2022 Arnav Mhaske, Harshit Kedia, Sumanth Doddapaneni, Mitesh M. Khapra, Pratyush Kumar, Rudra Murthy V, Anoop Kunchukuttan

The dataset contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location, and, Organization) for 9 out of the 11 languages.

Named Entity Recognition Sentence

IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages

1 code implementation20 Dec 2022 Ananya B. Sai, Vignesh Nagarajan, Tanay Dixit, Raj Dabre, Anoop Kunchukuttan, Pratyush Kumar, Mitesh M. Khapra

In this paper, we fill this gap by creating an MQM dataset consisting of 7000 fine-grained annotations, spanning 5 Indian languages and 7 MT systems, and use it to establish correlations between annotator scores and scores obtained using existing automatic metrics.

Machine Translation

Efficient ML Models for Practical Secure Inference

no code implementations26 Aug 2022 Vinod Ganesan, Anwesh Bhattacharya, Pratyush Kumar, Divya Gupta, Rahul Sharma, Nishanth Chandran

For instance, the model provider could be a diagnostics company that has trained a state-of-the-art DenseNet-121 model for interpreting a chest X-ray and the user could be a patient at a hospital.

Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users

2 code implementations6 May 2022 Yash Madhani, Sushane Parthan, Priyanka Bedekar, Gokul NC, Ruchi Khapra, Anoop Kunchukuttan, Pratyush Kumar, Mitesh M. Khapra

Transliteration is very important in the Indian language context due to the usage of multiple scripts and the widespread use of romanized inputs.

Transliteration

Joint Transformer/RNN Architecture for Gesture Typing in Indic Languages

no code implementations COLING 2020 Emil Biju, Anirudh Sriram, Mitesh M. Khapra, Pratyush Kumar

Gesture typing is a method of typing words on a touch-based keyboard by creating a continuous trace passing through the relevant keys.

Transliteration

Input-specific Attention Subnetworks for Adversarial Detection

no code implementations Findings (ACL) 2022 Emil Biju, Anirudh Sriram, Pratyush Kumar, Mitesh M Khapra

We also demonstrate that our method (a) is more accurate for larger models which are likely to have more spurious correlations and thus vulnerable to adversarial attack, and (b) performs well even with modest training sets of adversarial examples.

Adversarial Attack

Towards Building ASR Systems for the Next Billion Users

no code implementations6 Nov 2021 Tahir Javed, Sumanth Doddapaneni, Abhigyan Raman, Kaushal Santosh Bhogale, Gowtham Ramesh, Anoop Kunchukuttan, Pratyush Kumar, Mitesh M. Khapra

Second, using this raw speech data we pretrain several variants of wav2vec style models for 40 Indian languages.

SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions

no code implementations10 Oct 2021 Vinod Ganesan, Gowtham Ramesh, Pratyush Kumar

Such models need to be deployed on devices across the cloud and the edge with varying resource and accuracy constraints.

Neural Architecture Search

Machine Learning approaches to do size based reasoning on Retail Shelf objects to classify product variants

no code implementations7 Oct 2021 Muktabh Mayank Srivastava, Pratyush Kumar

There has been a surge in the number of Machine Learning methods to analyze products kept on retail shelves images.

On the Prunability of Attention Heads in Multilingual BERT

no code implementations26 Sep 2021 Aakriti Budhraja, Madhura Pande, Pratyush Kumar, Mitesh M. Khapra

Large multilingual models, such as mBERT, have shown promise in crosslingual transfer.

Design and Scaffolded Training of an Efficient DNN Operator for Computer Vision on the Edge

no code implementations25 Aug 2021 Vinod Ganesan, Pratyush Kumar

The parameter efficiency of FuSeConv and its significant out-performance over depthwise separable convolutions on systolic arrays illustrates their promise as a strong solution on the edge.

Neural Architecture Search

VeRLPy: Python Library for Verification of Digital Designs with Reinforcement Learning

1 code implementation9 Aug 2021 Aebel Joe Shibu, Sadhana S, Shilpa N, Pratyush Kumar

Digital hardware is verified by comparing its behavior against a reference model on a range of randomly generated input signals.

reinforcement-learning Reinforcement Learning (RL)

FuSeConv: Fully Separable Convolutions for Fast Inference on Systolic Arrays

1 code implementation27 May 2021 Surya Selvam, Vinod Ganesan, Pratyush Kumar

The resultant computation is systolic and efficiently utilizes the systolic array with a slightly modified dataflow.

Neural Architecture Search

Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages

1 code implementation12 Apr 2021 Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Mahalakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Kumar, Mitesh Shantadevi Khapra

We mine the parallel sentences from the web by combining many corpora, tools, and methods: (a) web-crawled monolingual corpora, (b) document OCR for extracting sentences from scanned documents, (c) multilingual representation models for aligning sentences, and (d) approximate nearest neighbor search for searching in a large collection of sentences.

Machine Translation Multilingual NLP +3

The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT

1 code implementation22 Jan 2021 Madhura Pande, Aakriti Budhraja, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra

There are two main challenges with existing methods for classification: (a) there are no standard scores across studies or across functional roles, and (b) these scores are often average quantities measured across sentences without capturing statistical significance.

Sentence

Differential Euler: Designing a Neural Network approximator to solve the Chaotic Three Body Problem

no code implementations21 Jan 2021 Pratyush Kumar, Aishwarya Das, Debayan Gupta

In this paper, we propose a detailed experimental setup to determine the feasibility of using neural networks to solve the three body problem up to a certain number of time steps.

Experimental Design

A Systematic Evaluation of Object Detection Networks for Scientific Plots

no code implementations5 Jul 2020 Pritha Ganguly, Nitesh Methani, Mitesh M. Khapra, Pratyush Kumar

However, the performance drops drastically when evaluated at a stricter IOU of 0. 9 with the best model giving a mAP of 35. 70%.

object-detection Object Detection +1

AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages

2 code implementations30 Apr 2020 Anoop Kunchukuttan, Divyanshu Kakwani, Satish Golla, Gokul N. C., Avik Bhattacharyya, Mitesh M. Khapra, Pratyush Kumar

We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2. 7 billion words for 10 Indian languages from two language families.

Word Embeddings

Compact retail shelf segmentation for mobile deployment

no code implementations27 Apr 2020 Pratyush Kumar, Muktabh Mayank Srivastava

In this paper, we work on one such common problem in the retail industries - Shelf segmentation.

General Classification Segmentation +1

PlotQA: Reasoning over Scientific Plots

no code implementations3 Sep 2019 Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, Pratyush Kumar

However, in practice, this is an unrealistic assumption because many questions require reasoning and thus have real-valued answers which appear neither in a small fixed size vocabulary nor in the image.

Chart Question Answering Question Answering

Example Mining for Incremental Learning in Medical Imaging

no code implementations24 Jul 2018 Pratyush Kumar, Muktabh Mayank Srivastava

Incremental learning proves to be time as well as resource-efficient solution for deployment of deep learning algorithms in real world as the model can automatically and dynamically adapt to new data as and when annotated data becomes available.

Incremental Learning

Detection of Tooth caries in Bitewing Radiographs using Deep Learning

no code implementations20 Nov 2017 Muktabh Mayank Srivastava, Pratyush Kumar, Lalit Pradhan, Srikrishna Varadarajan

We develop a Computer Aided Diagnosis (CAD) system, which enhances the performance of dentists in detecting wide range of dental caries.

Cannot find the paper you are looking for? You can Submit a new open access paper.