Search Results for author:
Found 83 papers, 18 papers with code
Stress Rules from Surface Forms: Experiments with Program Synthesis
Learning linguistic generalizations from only a few examples is a challenging task.
BERTologiCoMix: How does Code-Mixing interact with Multilingual BERT?
Models such as mBERT and XLMR have shown success in solving Code-Mixed NLP tasks even though they were not exposed to such text during pretraining.
”Diversity and Uncertainty in Moderation” are the Key to Data Selection for Multilingual Few-shot Transfer
Few-shot transfer often shows substantial gain over zero-shot transfer (CITATION), which is a practically useful trade-off between fully supervised and unsupervised learning approaches for multilingual pretained model-based systems.
SyMCoM - Syntactic Measure of Code Mixing A Study Of English-Hindi Code-Mixing
Code mixing is the linguistic phenomenon where bilingual speakers tend to switch between two or more languages in conversations.
Comparing Grammatical Theories of Code-Mixing
Code-mixed text generation systems have found applications in many downstream tasks, including speech recognition, translation and dialogue.
Language Patterns and Behaviour of the Peer Supporters in Multilingual Healthcare Conversational Forums
In this work, we conduct a quantitative linguistic analysis of the language usage patterns of multilingual peer supporters in two health-focused WhatsApp groups in Kenya comprising of youth living with HIV.
A Linguistic Annotation Framework to Study Interactions in Multilingual Healthcare Conversational Forums
In recent years, remote digital healthcare using online chats has gained momentum, especially in the Global South.
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages.
The Zeno's Paradox of `Low-Resource' Languages
The disparity in the languages commonly studied in Natural Language Processing (NLP) is typically reflected by referring to languages as low vs high-resourced.
[WIP] Jailbreak Paradox: The Achilles' Heel of LLMs
We introduce two paradoxes concerning jailbreak of foundation models: First, it is impossible to construct a perfect jailbreak classifier, and second, a weaker model cannot consistently detect whether a stronger (in a pareto-dominant sense) model is jailbroken or not.
Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting
We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy.
Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents
However, after manual evaluation of the validation set, we find that by correcting gold label errors and improving dataset annotation schema, GPT-4 with our prompts can achieve (1) 89. 6%-96. 8% accuracy in DST, and (2) more than 99% correct response generation across different languages.
From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences
To this end, we construct Cline - a dataset containing human acceptability judgements for English-Hindi (en-hi) code-mixed text.
"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations
Large language models (LLMs) have emerged as an integral part of modern societies, powering user-facing applications such as personal assistants and enterprise applications like recruitment tools.
Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in
Ethical reasoning is a crucial skill for Large Language Models (LLMs).
Towards Measuring and Modeling "Culture" in LLMs: A Survey
We present a survey of more than 90 recent papers that aim to study cultural representation and inclusion in large language models (LLMs).
Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test
This paper explores the moral judgment and moral reasoning abilities exhibited by Large Language Models (LLMs) across languages through the Defining Issues Test.
Evaluating Large Language Models for Health-related Queries with Presuppositions
As corporations rush to integrate large language models (LLMs) to their search offerings, it is critical that they provide factually accurate information that is robust to any presuppositions that a user may express.
Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
In this position paper, we argue that instead of morally aligning LLMs to specific set of ethical principles, we should infuse generic ethical reasoning capabilities into them so that they can handle value pluralism at a global scale.
Probing the Moral Development of Large Language Models through Defining Issues Test
In this study, we measure the moral reasoning ability of LLMs using the Defining Issues Test - a psychometric instrument developed for measuring the moral development stage of a person according to the Kohlberg's Cognitive Moral Development Model.
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top $20$, remains inadequate due to existing benchmarks and metrics limitations.
X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents
We create a new multilingual benchmark, X-RiSAWOZ, by translating the Chinese RiSAWOZ to 4 languages: English, French, Hindi, Korean; and a code-mixed English-Hindi language.
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies.
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited.
DUBLIN -- Document Understanding By Language-Image Network
Visual document understanding is a complex task that involves analyzing both the text and the visual elements in document images.
Ranked #1 on
Visual Question Answering (VQA)
on DeepForm
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer
Zero-shot cross-lingual transfer is promising, however has been shown to be sub-optimal, with inferior transfer performance across low-resource languages.
Fairness in Language Models Beyond English: Gaps and Challenges
With language models becoming increasingly ubiquitous, it has become essential to address their inequitable treatment of diverse demographic groups and factors.
Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Models
Leveraging shared learning through Massively Multilingual Models, state-of-the-art machine translation models are often able to adapt to the paucity of data for low-resource languages.
On the Calibration of Massively Multilingual Language Models
Massively Multilingual Language Models (MMLMs) have recently gained popularity due to their surprising effectiveness in cross-lingual transfer.
Generating Intermediate Steps for NLI with Next-Step Supervision
The Natural Language Inference (NLI) task often requires reasoning over multiple steps to reach the conclusion.
"Diversity and Uncertainty in Moderation" are the Key to Data Selection for Multilingual Few-shot Transfer
Few-shot transfer often shows substantial gain over zero-shot transfer~\cite{lauscher2020zero}, which is a practically useful trade-off between fully supervised and unsupervised learning approaches for multilingual pretrained model-based systems.
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models
Massively Multilingual Transformer based Language Models have been observed to be surprisingly effective on zero-shot transfer across languages, though the performance varies from language to language depending on the pivot language(s) used for fine-tuning.
On the Economics of Multilingual Few-shot Learning: Modeling the Cost-Performance Trade-offs of Machine Translated and Manual Data
Borrowing ideas from {\em Production functions} in micro-economics, in this paper we introduce a framework to systematically evaluate the performance and cost trade-offs between machine-translated and manually-created labelled data for task-specific fine-tuning of massively multilingual language models.
Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages
Although recent Massively Multilingual Language Models (MMLMs) like mBERT and XLMR support around 100 languages, most existing multilingual NLP benchmarks provide evaluation data in only a handful of these languages with little linguistic diversity.
Global Readiness of Language Technology for Healthcare: What would it Take to Combat the Next Pandemic?
The COVID-19 pandemic has brought out both the best and worst of language technology (LT).
Multilingual CheckList: Generation and Evaluation
We compare the TEA CheckLists with CheckLists created with different levels of human intervention.
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis
We introduce the first large-scale human-annotated Twitter sentiment dataset for the four most widely spoken languages in Nigeria (Hausa, Igbo, Nigerian-Pidgin, and Yor\`ub\'a ) consisting of around 30, 000 annotated tweets per language (and 14, 000 for Nigerian-Pidgin), including a significant fraction of code-mixed tweets.
LoNLI: An Extensible Framework for Testing Diverse Logical Reasoning Capabilities for NLI
Natural Language Inference (NLI) is considered a representative task to test natural language understanding (NLU).
Predicting the Performance of Multilingual NLP Models
Recent advancements in NLP have given us models like mBERT and XLMR that can serve over 100 languages.
Designing Language Technologies for Social Good: The Road not Taken
Development of speech and language technology for social good (LT4SG), especially those targeted at the welfare of marginalized communities and speakers of low-resource and under-served languages, has been a prominent theme of research within NLP, Speech, and the AI communities.
Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance
Multilingual language models achieve impressive zero-shot accuracies in many languages in complex tasks such as Natural Language Inference (NLI).
On the Universality of Deep Contextual Language Models
Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple tasks rapidly by pre-training a single model, followed by task-specific fine-tuning.
Trusting RoBERTa over BERT: Insights from CheckListing the Natural Language Inference Task
The recent state-of-the-art natural language understanding (NLU) systems often behave unpredictably, failing on simpler reasoning examples.
Sample-efficient Linguistic Generalizations through Program Synthesis: Experiments with Phonology Problems
Neural models excel at extracting statistical patterns from large amounts of data, but struggle to learn patterns or reason about language from only a few examples.
Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current Practices
Ethical aspects of research in language technologies have received much attention recently.
GCM: A Toolkit for Generating Synthetic Code-mixed Text
Code-mixing is common in multilingual communities around the world, and processing it is challenging due to the lack of labeled and unlabeled data.
TaxiNLI: Taking a Ride up the NLU Hill
Pre-trained Transformer-based neural architectures have consistently achieved state-of-the-art performance in the Natural Language Inference (NLI) task.
GLUECoS: An Evaluation Benchmark for Code-Switched NLP
We present results on all these tasks using cross-lingual word embedding models and multilingual models.
Code-mixed parse trees and how to find them
In this paper, we explore the methods of obtaining parse trees of code-mixed sentences and analyse the obtained trees.
Crowdsourcing Speech Data for Low-Resource Languages from Low-Income Workers
Unfortunately, collecting labelled speech data in any language is an expensive and resource-intensive task.
Understanding Script-Mixing: A Case Study of Hindi-English Bilingual Twitter Users
Our analysis shows that both intra-sentential and inter-sentential script-mixing are present on Twitter and show different behavior in different contexts.
GLUECoS : An Evaluation Benchmark for Code-Switched NLP
We present results on all these tasks using cross-lingual word embedding models and multilingual models.
The State and Fate of Linguistic Diversity and Inclusion in the NLP World
Language technologies contribute to promoting multilingualism and linguistic diversity around the world.
A New Dataset for Natural Language Inference from Code-mixed Conversations
Code-mixing is the use of more than one language in the same conversation or utterance, and is prevalent in multilingual communities all over the world.
Unsung Challenges of Building and Deploying Language Technologies for Low Resource Language Communities
In this paper, we examine and analyze the challenges associated with developing and introducing language technologies to low-resource language communities.
INMT: Interactive Neural Machine Translation Prediction
In this paper, we demonstrate an Interactive Machine Translation interface, that assists human translators with on-the-fly hints and suggestions.
Word Embeddings for Code-Mixed Language Processing
We compare three existing bilingual word embedding approaches, and a novel approach of training skip-grams on synthetic code-mixed text generated through linguistic models of code-mixing, on two tasks - sentiment analysis and POS tagging for code-mixed text.
Language Modeling for Code-Mixing: The Role of Linguistic Theory based Synthetic Data
Training language models for Code-mixed (CM) language is known to be a difficult problem because of lack of data compounded by the increased confusability due to the presence of more than one language.
Automatic Speech Recognition (ASR)
Language Identification
+3
Accommodation of Conversational Code-Choice
We find that the saliency or markedness of a language in context directly affects the degree of accommodation observed.
Phone Merging For Code-Switched Speech Recognition
Though the best performance gain of 1. 2{\%} WER was observed with manually merged phones, we show experimentally that the manual phone merge is not optimal.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Learnability of Learned Neural Networks
This paper explores the simplicity of learned neural networks under various settings: learned on real vs random data, varying size/architecture and using large minibatch size vs small minibatch size.
All that is English may be Hindi: Enhancing language identification through automatic ranking of the likeliness of word borrowing in social media
Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts.
All that is English may be Hindi: Enhancing language identification through automatic ranking of likeliness of word borrowing in social media
Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts.
Estimating Code-Switching on Twitter with a Novel Generalized Word-Level Language Detection Technique
Word-level language detection is necessary for analyzing code-switched text, where multiple languages could be mixed within a sentence.
Is this word borrowed? An automatic approach to quantify the likeliness of borrowing in social media
We first propose context based clustering method to sample a set of candidate words from the social media data. Next, we propose three novel and similar metrics based on the usage of these words by the users in different tweets; these metrics were used to score and rank the candidate words indicating their borrowed likeliness.
Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Models
We make one of the first attempts to build working models for intra-sentential code-switching based on the Equivalence-Constraint (Poplack 1980) and Matrix-Language (Myers-Scotton 1993) theories.
Functions of Code-Switching in Tweets: An Annotation Framework and Some Initial Experiments
Code-Switching (CS) between two languages is extremely common in communities with societal multilingualism where speakers switch between two or more languages when interacting with each other.
An Empirical Study of the Occurrence and Co-Occurrence of Named Entities in Natural Language Corpora
Named Entities (NEs) that occur in natural language text are important especially due to the advent of social media, and they play a critical role in the development of many natural language technologies.
Mining Hindi-English Transliteration Pairs from Online Hindi Lyrics
This paper describes a method to mine Hindi-English transliteration pairs from online Hindi song lyrics.
