Search Results for author:
Found 17 papers, 10 papers with code
Translate and Classify: Improving Sequence Level Classification for English-Hindi Code-Mixed Data
To translate English-Hindi code-mixed data to English, we use mBART, a pre-trained multilingual sequence-to-sequence model that has shown competitive performance on various low-resource machine translation pairs and has also shown performance gains in languages that were not in its pre-training corpus.
CoMeT: Towards Code-Mixed Translation Using Parallel Monolingual Sentences
Code-mixed languages are very popular in multilingual societies around the world, yet the resources lag behind to enable robust systems on such languages.
Towards Detecting Political Bias in Hindi News Articles
Political propaganda in recent times has been amplified by media news portals through biased reporting, creating untruthful narratives on serious issues causing misinformed public opinions with interests of siding and helping a particular political party.
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility.
Scaling Instructable Agents Across Many Simulated Worlds
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI.
Simple and Scalable Strategies to Continually Pre-train Large Language Models
In this work, we show that a simple and scalable combination of learning rate (LR) re-warming, LR re-decaying, and replay of previous data is sufficient to match the performance of fully re-training from scratch on all available data, as measured by the final loss and the average score on several language model (LM) evaluation benchmarks.
Continual Pre-Training of Large Language Models: How to (re)warm your model?
We study the warmup phase of models pre-trained on the Pile (upstream data, 300B tokens) as we continue to pre-train on SlimPajama (downstream data, 297B tokens), following a linear warmup and cosine decay schedule.
ARB: Advanced Reasoning Benchmark for Large Language Models
As a subset of ARB, we introduce a challenging set of math and physics problems which require advanced symbolic reasoning and domain knowledge.
Broken Neural Scaling Laws
Moreover, this functional form accurately models and extrapolates scaling behavior that other functional forms are incapable of expressing such as the non-monotonic transitions present in the scaling behavior of phenomena such as double descent and the delayed, sharp inflection points present in the scaling behavior of tasks such as arithmetic.
Data Augmentation for Automated Essay Scoring using Transformer Models
It has been explored for a number of years, and it remains partially solved.
MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation
We empirically demonstrate the effectiveness of self-supervised pre-training and data augmentation for zero-shot multi-lingual machine translation.
cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation
We propose a pipeline that utilizes English-only vision-language models to train a monolingual model for a target language.
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning
A slow stream that is recurrent in nature aims to learn a specialized and compressed representation, by forcing chunks of $K$ time steps into a single representation which is divided into multiple vectors.
Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble
Memes are one of the most popular types of content used to spread information online.
ViTA: Visual-Linguistic Translation by Aligning Object Tags
Multimodal Machine Translation (MMT) enriches the source text with visual information for translation.
Ranked #1 on
Multimodal Machine Translation
on Hindi Visual Genome (Test Set)
(using extra training data)
Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning
We fine-tune TAPAS (a model which extends BERT's architecture to capture tabular structure) for both the subtasks as it has shown state-of-the-art performance in various table understanding tasks.
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Metaphors are rhetorical use of words based on the conceptual mapping as opposed to their literal use.
