Search Results for author: Rajiv Ratn Shah

Found 127 papers, 45 papers with code

A Preliminary Exploration of GANs for Keyphrase Generation

no code implementations EMNLP 2020 Avinash Swaminathan, Haimin Zhang, Debanjan Mahata, Rakesh Gosangi, Rajiv Ratn Shah, Amanda Stent

We observed that our model achieves state-of-the-art performance in the generation of abstractive keyphrases and is comparable to the best performing extractive techniques.

Keyphrase Generation

Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations

1 code implementation EMNLP 2020 Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa, Rajiv Ratn Shah

In the financial domain, risk modeling and profit generation heavily rely on the sophisticated and intricate stock movement prediction task.

Ranked #2 on Stock Market Prediction on stocknet (using extra training data)

Decision Making Graph Neural Network +1

A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media

no code implementations EMNLP 2020 Ramit Sawhney, Harshit Joshi, Saumya Gandhi, Rajiv Ratn Shah

Understanding the build-up of such ideation is critical for the identification of at-risk users and suicide prevention.

VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls

1 code implementation EMNLP 2020 Ramit Sawhney, Piyush Khanna, Arshiya Aggarwal, Taru Jain, Puneet Mathur, Rajiv Ratn Shah

Natural language processing has recently made stock movement forecasting and volatility forecasting advances, leading to improved financial forecasting.

Med-CoDE: Medical Critique based Disagreement Evaluation Framework

no code implementations21 Apr 2025 Mohit Gupta, Akiko Aizawa, Rajiv Ratn Shah

The emergence of large language models (LLMs) has significantly influenced numerous fields, including healthcare, by enhancing the capabilities of automated systems to process and generate human-like text.

Long-context Non-factoid Question Answering in Indic Languages

1 code implementation18 Apr 2025 Ritwik Mishra, Rajiv Ratn Shah, Ponnurangam Kumaraguru

Compared to the baseline of unshortened (long) contexts, our experiments on four Indic languages (Hindi, Tamil, Telugu, and Urdu) demonstrate that context-shortening techniques yield an average improvement of 4\% in semantic scores and 47\% in token-level scores when evaluated on three popular LLMs without fine-tuning.

coreference-resolution Open Information Extraction +1

Visual and Text Prompt Segmentation: A Novel Multi-Model Framework for Remote Sensing

no code implementations10 Mar 2025 Xing Zi, Kairui Jin, Xian Tao, Jun Li, Ali Braytee, Rajiv Ratn Shah, Mukesh Prasad

Pixel-level segmentation is essential in remote sensing, where foundational vision models like CLIP and Segment Anything Model(SAM) have demonstrated significant capabilities in zero-shot segmentation tasks.

Image Segmentation Segmentation +2

Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English

1 code implementation24 Dec 2024 Avinash Anand, Kritarth Prasad, Chhavi Kirtani, Ashwin R Nair, Manvendra Kumar Nema, Raj Jaiswal, Rajiv Ratn Shah

Adopting a bilingual approach that combines English and Hindi samples achieves results comparable to individual language models, demonstrating the capability to learn mathematical reasoning in both languages.

Mathematical Reasoning

Steps are all you need: Rethinking STEM Education with Prompt Engineering

no code implementations6 Dec 2024 Krishnasai Addala, Kabir Dev Paul Baghel, Chhavi Kirtani, Avinash Anand, Rajiv Ratn Shah

Few shot and Chain-of-Thought prompting have shown promise when applied to Physics Question Answering Tasks, but are limited by the lack of mathematical ability inherent to LLMs, and are prone to hallucination.

All Hallucination +3

Knowledge Graphs are all you need: Leveraging KGs in Physics Question Answering

no code implementations6 Dec 2024 Krishnasai Addala, Kabir Dev Paul Baghel, Dhruv Jain, Chhavi Kirtani, Avinash Anand, Rajiv Ratn Shah

This study explores the effectiveness of using knowledge graphs generated by large language models to decompose high school-level physics questions into sub-questions.

All Knowledge Graphs +1

Su-RoBERTa: A Semi-supervised Approach to Predicting Suicide Risk through Social Media using Base Language Models

no code implementations2 Dec 2024 Chayan Tank, Shaina Mehta, Sarthak Pol, Vinayak Katoch, Avinash Anand, Raj Jaiswal, Rajiv Ratn Shah

This paper is a study done on suicidal risk assessments using Reddit data leveraging Base language models to identify patterns from social media posts.

Data Augmentation

Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agents

no code implementations1 Dec 2024 Raj Jaiswal, Dhruv Jain, Harsh Parimal Popat, Avinash Anand, Abhishek Dharmadhikari, Atharva Marathe, Rajiv Ratn Shah

To address this, we introduce Mixture of Refinement Agents (MoRA), a novel agentic refinement framework that iteratively refines the LLM generated base solution by correcting the aforementioned errors, resulting in a significant performance improvement for open-source LLMs.

Mathematical Reasoning MMLU

JOOCI: a Framework for Learning Comprehensive Speech Representations

no code implementations14 Oct 2024 Hemant Yadav, Rajiv Ratn Shah, Sunayana Sitaram

Given the orthogonal nature of other and content information, attempting to optimize both within a single embedding results in suboptimal solutions.

Representation Learning Speech Representation Learning

RConE: Rough Cone Embedding for Multi-Hop Logical Query Answering on Multi-Modal Knowledge Graphs

1 code implementation21 Aug 2024 Mayank Kharbanda, Rajiv Ratn Shah, Raghava Mutharaju

However, to our knowledge, we are the first to introduce logical constructs in querying MMKGs and to answer queries that involve sub-entities of multi-modal entities as the answer.

Knowledge Graphs Link Prediction +3

Multilingual Non-Factoid Question Answering with Answer Paragraph Selection

1 code implementation20 Aug 2024 Ritwik Mishra, Sreeram Vennam, Rajiv Ratn Shah, Ponnurangam Kumaraguru

The APS model attained an accuracy of 80\% and 72\%, as well as a macro F1 of 72\% and 66\%, on the MuNfQuAD testset and the golden set, respectively.

Question Answering

Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities

no code implementations8 Jul 2024 Chayan Tank, Sarthak Pol, Vinayak Katoch, Shaina Mehta, Avinash Anand, Rajiv Ratn Shah

The proposed solutions demonstrate better results achieved by Proprietary and Open-source Large Language Models (LLMs), which achieved a Root Mean Square Error (RMSE) score of 3. 98 on Textual Modality, beating the AVEC 2019 challenge baseline results and current SOTA regression analysis architectures.

Depression Detection

Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs

no code implementations21 Jun 2024 Debnath Kundu, Atharva Mehta, Rajesh Kumar, Naman Lal, Avinash Anand, Apoorv Singh, Rajiv Ratn Shah

The transition to online examinations and assignments raises significant concerns about academic integrity.

DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbing

no code implementations13 Jun 2024 Neha Sahipjohn, Ashishkumar Gudmalwar, Nirmesh Shah, Pankaj Wasnik, Rajiv Ratn Shah

To this end, we propose a novel method, DubWise Multi-modal Large Language Model (LLM)-based Text-to-Speech (TTS), which can control the speech duration of synthesized speech in such a way that it aligns well with the speakers lip movements given in the reference video even when the spoken text is different or in a different language.

Language Modeling Language Modelling +4

VECL-TTS: Voice identity and Emotional style controllable Cross-Lingual Text-to-Speech

no code implementations12 Jun 2024 Ashishkumar Gudmalwar, Nirmesh Shah, Sai Akarsh, Pankaj Wasnik, Rajiv Ratn Shah

Despite the significant advancements in Text-to-Speech (TTS) systems, their full utilization in automatic dubbing remains limited.

text-to-speech Text to Speech

Teaching Human Behavior Improves Content Understanding Abilities Of LLMs

no code implementations2 May 2024 Somesh Singh, Harini S I, Yaman K Singla, Veeky Baths, Rajiv Ratn Shah, Changyou Chen, Balaji Krishnamurthy

Specifically, we show that training LLMs to predict the receiver behavior of likes and comments improves the LLM's performance on a wide variety of downstream content understanding tasks.

Context-Enhanced Language Models for Generating Multi-Paper Citations

no code implementations22 Apr 2024 Avinash Anand, Kritarth Prasad, Ujjwal Goel, Mohit Gupta, Naman Lal, Astha Verma, Rajiv Ratn Shah

This research underscores the potential of harnessing LLMs for citation generation, opening a compelling avenue for exploring the intricate connections between scientific documents.

Knowledge Graphs Sentence +1

Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks

1 code implementation19 Apr 2024 Avinash Anand, Mohit Gupta, Kritarth Prasad, Navya Singla, Sanjana Sanjeev, Jatin Kumar, Adarsh Raj Shivam, Rajiv Ratn Shah

Our experiments reveal that among the three models, MAmmoTH-13B emerges as the most proficient, achieving the highest level of competence in solving the presented mathematical problems.

Mathematical Problem-Solving

TC-OCR: TableCraft OCR for Efficient Detection & Recognition of Table Structure & Content

no code implementations16 Apr 2024 Avinash Anand, Raj Jaiswal, Pijush Bhuyan, Mohit Gupta, Siddhesh Bangar, Md. Modassir Imam, Rajiv Ratn Shah, Shin'ichi Satoh

Our proposed approach achieves an IOU of 0. 96 and an OCR Accuracy of 78%, showcasing a remarkable improvement of approximately 25% in the OCR Accuracy compared to the previous Table Transformer approach.

Information Retrieval Knowledge Graphs +3

KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models

no code implementations15 Apr 2024 Avinash Anand, Mohit Gupta, Kritarth Prasad, Ujjwal Goel, Naman Lal, Astha Verma, Rajiv Ratn Shah

Citation Text Generation (CTG) is a task in natural language processing (NLP) that aims to produce text that accurately cites or references a cited document within a source document.

Knowledge Graphs Text Generation +1

RanLayNet: A Dataset for Document Layout Detection used for Domain Adaptation and Generalization

1 code implementation15 Apr 2024 Avinash Anand, Raj Jaiswal, Mohit Gupta, Siddhesh S Bangar, Pijush Bhuyan, Naman Lal, Rajeev Singh, Ritika Jha, Rajiv Ratn Shah, Shin'ichi Satoh

To solve this problem, domain adaptation approaches have been developed that use a small quantity of labeled data to adjust the model to the target domain.

Domain Adaptation

Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning

no code implementations20 Mar 2024 Shivam Ratnakant Mhaskar, Nirmesh J. Shah, Mohammadi Zaki, Ashishkumar P. Gudmalwar, Pankaj Wasnik, Rajiv Ratn Shah

In this paper, we present the development of an isometric NMT system using Reinforcement Learning (RL), with a focus on optimizing the alignment of phoneme counts in the source and target language sentence pairs.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +8

Multilingual Coreference Resolution in Low-resource South Asian Languages

1 code implementation21 Feb 2024 Ritwik Mishra, Pooja Desur, Rajiv Ratn Shah, Ponnurangam Kumaraguru

We introduce a Translated dataset for Multilingual Coreference Resolution (TransMuCoRes) in 31 South Asian languages using off-the-shelf tools for translation and word-alignment.

coreference-resolution Word Alignment

Behavior Optimized Image Generation

no code implementations18 Nov 2023 Varun Khurana, Yaman K Singla, Jayakumar Subramanian, Rajiv Ratn Shah, Changyou Chen, Zhiqiang Xu, Balaji Krishnamurthy

We show that BoigLLM outperforms 13x larger models such as GPT-3. 5 and GPT-4 in this task, demonstrating that while these state-of-the-art models can understand images, they lack information on how these images perform in the real world.

Image Generation Marketing

Exploring Graph Neural Networks for Indian Legal Judgment Prediction

no code implementations19 Oct 2023 Mann Khatri, Mirza Yusuf, Yaman Kumar, Rajiv Ratn Shah, Ponnurangam Kumaraguru

We explored various embeddings as model features, while nodes such as time nodes and judicial acts were added and pruned to evaluate the model's performance.

Fairness Graph Neural Network +3

Long-Term Ad Memorability: Understanding & Generating Memorable Ads

no code implementations1 Sep 2023 Harini SI, Somesh Singh, Yaman K Singla, Aanisha Bhattacharyya, Veeky Baths, Changyou Chen, Rajiv Ratn Shah, Balaji Krishnamurthy

Finally, with the intent of memorable ad generation, we present a scalable method to build a high-quality memorable ad generation model by leveraging automatically annotated data.

Language Modelling Marketing +1

Certified Zeroth-order Black-Box Defense with Robust UNet Denoiser

no code implementations13 Apr 2023 Astha Verma, A V Subramanyam, Siddhesh Bangar, Naman Lal, Rajiv Ratn Shah, Shin'ichi Satoh

However, these methods suffer from high model variance with low performance on high-dimensional datasets due to the ineffective design of the denoiser and are limited in their utilization of ZO techniques.

Image Reconstruction

Emotionally Enhanced Talking Face Generation

1 code implementation21 Mar 2023 Sahil Goyal, Shagun Uppal, Sarthak Bhagat, Yi Yu, Yifang Yin, Rajiv Ratn Shah

To mitigate this, we build a talking face generation framework conditioned on a categorical emotion to generate videos with appropriate expressions, making them more realistic and convincing.

Talking Face Generation Talking Head Generation

On Comparing Fair Classifiers under Data Bias

1 code implementation12 Feb 2023 Mohit Sharma, Amit Deshpande, Rajiv Ratn Shah

In this paper, we consider a theoretical model for injecting data bias, namely, under-representation and label bias (Blum & Stangl, 2019).

Fairness Marketing

Emotional Talking Faces: Making Videos More Expressive and Realistic

no code implementations ACM Multimedia Asia 2022 Sahil Goyal, Shagun Uppal, Sarthak Bhagat, Dhroov Goel, Sakshat Mali, Yi Yu, Yifang Yin, Rajiv Ratn Shah

Lip synchronization and talking face generation have gained a specific interest from the research community with the advent and need of digital communication in different fields.

Talking Face Generation

A novel multimodal dynamic fusion network for disfluency detection in spoken utterances

no code implementations27 Nov 2022 Sreyan Ghosh, Utkarsh Tyagi, Sonal Kumar, Manan Suri, Rajiv Ratn Shah

Based on early-fusion and self-attention-based multimodal interaction between text and acoustic modalities, in this paper, we propose a novel multimodal architecture for disfluency detection from individual utterances.

multimodal interaction

Persuasion Strategies in Advertisements

1 code implementation20 Aug 2022 Yaman Kumar Singla, Rajat Jha, Arunim Gupta, Milan Aggarwal, Aditya Garg, Tushar Malyan, Ayush Bhardwaj, Rajiv Ratn Shah, Balaji Krishnamurthy, Changyou Chen

Motivated by persuasion literature in social psychology and marketing, we introduce an extensive vocabulary of persuasion strategies and build the first ad image corpus annotated with persuasion strategies.

Image Segmentation Marketing +2

Span Classification with Structured Information for Disfluency Detection in Spoken Utterances

1 code implementation30 Mar 2022 Sreyan Ghosh, Sonal Kumar, Yaman Kumar Singla, Rajiv Ratn Shah, S. Umesh

Existing approaches in disfluency detection focus on solving a token-level classification task for identifying and removing disfluencies in text.

Classification

Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets

no code implementations18 Dec 2021 Zaki Mustafa Farooqi, Sreyan Ghosh, Rajiv Ratn Shah

In the current era of the internet, where social media platforms are easily accessible for everyone, people often have to deal with threats, identity attacks, hate, and bullying due to their association with a cast, creed, gender, religion, or even acceptance or rejection of a notion.

Hate Speech Detection

Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees

1 code implementation17 Nov 2021 Yaman Kumar Singla, Sriram Krishna, Rajiv Ratn Shah, Changyou Chen

Automated Scoring (AS), the natural language processing task of scoring essays and speeches in an educational testing setting, is growing in popularity and being deployed across contexts from government examinations to companies providing language proficiency services.

Intent Classification Using Pre-trained Language Agnostic Embeddings For Low Resource Languages

no code implementations18 Oct 2021 Hemant Yadav, Akshat Gupta, Sai Krishna Rallabandi, Alan W Black, Rajiv Ratn Shah

We perform experiments across three different languages: English, Sinhala, and Tamil each with different data sizes to simulate high, medium, and low resource scenarios.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +5

DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances

1 code implementation14 Oct 2021 Sreyan Ghosh, Samden Lepcha, S Sakshi, Rajiv Ratn Shah, S. Umesh

We believe that our dataset would act as a benchmark for the relatively new and un-explored Spoken Language Processing task of detecting toxicity from spoken utterances and boost further research in this space.

Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks

no code implementations13 Oct 2021 Anuj Saraswat, Mehar Bhatia, Yaman Kumar Singla, Changyou Chen, Rajiv Ratn Shah

Recent studies in speech perception have been closely linked to fields of cognitive psychology, phonology, and phonetics in linguistics.

Lip Reading speech-recognition +1

MINIMAL: Mining Models for Data Free Universal Adversarial Triggers

1 code implementation25 Sep 2021 Swapnil Parekh, Yaman Singla Kumar, Somesh Singh, Changyou Chen, Balaji Krishnamurthy, Rajiv Ratn Shah

It is well known that natural language models are vulnerable to adversarial attacks, which are mostly input-specific in nature.

Natural Language Inference

AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses

no code implementations24 Sep 2021 Yaman Kumar Singla, Swapnil Parekh, Somesh Singh, Junyi Jessy Li, Rajiv Ratn Shah, Changyou Chen

This is in stark contrast to recent probing studies on pre-trained representation learning models, which show that rich linguistic features such as parts-of-speech and morphology are encoded by them.

Representation Learning

Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring

no code implementations30 Aug 2021 Yaman Kumar Singla, Avykat Gupta, Shaurya Bagga, Changyou Chen, Balaji Krishnamurthy, Rajiv Ratn Shah

In our technique, we take advantage of the fact that oral proficiency tests rate multiple responses for a candidate.

Multimodal Multi-Speaker Merger \& Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines

no code implementations ACL 2021 Ramit Sawhney, Mihir Goyal, Prakhar Goel, Puneet Mathur, Rajiv Ratn Shah

We introduce M3ANet, a baseline architecture that takes advantage of the multimodal multi-speaker input to forecast the financial risk associated with the M{\&}A calls.

Defending Touch-based Continuous Authentication Systems from Active Adversaries Using Generative Adversarial Networks

1 code implementation15 Jun 2021 Mohit Agrawal, Pragyan Mehrotra, Rajesh Kumar, Rajiv Ratn Shah

Previous studies have demonstrated that commonly studied (vanilla) touch-based continuous authentication systems (V-TCAS) are susceptible to population attack.

Generative Adversarial Network

Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures

1 code implementation NAACL 2021 Ramit Sawhney, Puneet Mathur, Taru Jain, Akash Kumar Gautam, Rajiv Ratn Shah

We show how for more domain-specific tasks related to sexual abuse disclosures such as sarcasm identification and dialogue act (refutation, justification, allegation) classification, homogeneous multitask learning is helpful, whereas for more general tasks such as stance and hate speech detection, heterogeneous multitask learning with emotion classification works better.

Classification Emotion Classification +2

Quantitative Day Trading from Natural Language using Reinforcement Learning

no code implementations NAACL 2021 Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah

It is challenging to design profitable and practical trading strategies, as stock price movements are highly stochastic, and the market is heavily influenced by chaotic data across sources like news and social media.

Deep Reinforcement Learning reinforcement-learning +2

An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls

1 code implementation NAACL 2021 Ramit Sawhney, Arshiya Aggarwal, Rajiv Ratn Shah

In this work, we present the first study to discover the gender bias in multimodal volatility prediction due to gender-sensitive audio features and fewer female executives in earnings calls of one of the world{'}s biggest stock indexes, the S{\&}P 500 index.

Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning

no code implementations NAACL 2021 Ramit Sawhney, Harshit Joshi, Rajiv Ratn Shah, Lucie Flek

Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners.

GupShup: An Annotated Corpus for Abstractive Summarization of Open-Domain Code-Switched Conversations

no code implementations17 Apr 2021 Laiba Mehnaz, Debanjan Mahata, Rakesh Gosangi, Uma Sushmitha Gunturi, Riya Jain, Gauri Gupta, Amardeep Kumar, Isabelle Lee, Anish Acharya, Rajiv Ratn Shah

Towards this objective, we introduce abstractive summarization of Hindi-English code-switched conversations and develop the first code-switched conversation summarization dataset - GupShup, which contains over 6, 831 conversations in Hindi-English and their corresponding human-annotated summaries in English and Hindi-English.

Abstractive Text Summarization Conversation Summarization

PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media

1 code implementation EACL 2021 Ramit Sawhney, Harshit Joshi, Lucie Flek, Rajiv Ratn Shah

Building on clinical studies, PHASE learns phase-like progressions in users{'} historical Plutchik-wheel-based emotions to contextualize suicidal intent.

FAST: Financial News and Tweet Based Time Aware Network for Stock Trading

no code implementations EACL 2021 Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah

Designing profitable trading strategies is complex as stock movements are highly stochastic; the market is influenced by large volumes of noisy data across diverse information sources like news and social media.

Learning-To-Rank

Factorization of Fact-Checks for Low Resource Indian Languages

no code implementations23 Feb 2021 Shivangi Singhal, Rajiv Ratn Shah, Ponnurangam Kumaraguru

The majority of studies on automatic fact-checking and fake news detection is restricted to English only.

Fact Checking Fake News Detection

Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings

1 code implementation10 Jan 2021 Sreyan Ghosh, Sonal Kumar, Harsh Jalan, Hemant Yadav, Rajiv Ratn Shah

This paper describes our proposed system for the AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides.

Exploring Semi-Supervised Learning for Predicting Listener Backchannels

no code implementations6 Jan 2021 Vidit Jain, Maitree Leekha, Rajiv Ratn Shah, Jainendra Shukla

To analyze our identification module's feasibility, we compared the backchannel prediction models trained on (a) manually-annotated and (b) semi-supervised labels.

What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure

no code implementations2 Jan 2021 Jui Shah, Yaman Kumar Singla, Changyou Chen, Rajiv Ratn Shah

In recent times, BERT based transformer models have become an inseparable part of the 'tech stack' of text processing models.

All

Towards Modelling Coherence in Spoken Discourse

no code implementations31 Dec 2020 Rajaswa Patil, Yaman Kumar Singla, Rajiv Ratn Shah, Mika Hama, Roger Zimmermann

While there has been significant progress towards modelling coherence in written discourse, the work in modelling spoken discourse coherence has been quite limited.

GPolS: A Contextual Graph-Based Language Model for Analyzing Parliamentary Debates and Political Cohesion

no code implementations COLING 2020 Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah

Parliamentary debates present a valuable language resource for analyzing comprehensive options in electing representatives under a functional, free society.

Language Modeling Language Modelling +1

LIFI: Towards Linguistically Informed Frame Interpolation

1 code implementation30 Oct 2020 Aradhya Neeraj Mathur, Devansh Batra, Yaman Kumar, Rajiv Ratn Shah, Roger Zimmermann

We also release several datasets to test computer vision video generation models of their speech understanding.

Video Generation

Conditional Hybrid GAN for Sequence Generation

no code implementations18 Sep 2020 Yi Yu, Abhishek Srivastava, Rajiv Ratn Shah

Conditional sequence generation aims to instruct the generation procedure by conditioning the model with additional context information, which is a self-supervised learning issue (a form of unsupervised learning with supervision information from data itself).

Attribute Relational Reasoning +2

Vyaktitv: A Multimodal Peer-to-Peer Hindi Conversations based Dataset for Personality Assessment

no code implementations31 Aug 2020 Shahid Nawaz Khan, Maitree Leekha, Jainendra Shukla, Rajiv Ratn Shah

Automatically detecting personality traits can aid several applications, such as mental health recognition and human resource management.

Management

Addressing the Cold-Start Problem in Outfit Recommendation Using Visual Preference Modelling

1 code implementation4 Aug 2020 Dhruv Verma, Kshitij Gulati, Rajiv Ratn Shah

Despite various cutting-edge solutions proposed in the past for personalising fashion recommendation, the technology is still limited by its poor performance on new entities, i. e. the cold-start problem.

Attribute Clustering

Unsupervised Paraphasia Classification in Aphasic Speech

no code implementations ACL 2020 Sharan Pai, Nikhil Sachdeva, Prince Sachdeva, Rajiv Ratn Shah

Aphasia is a speech and language disorder which results from brain damage, often characterized by word retrieval deficit (anomia) resulting in naming errors (paraphasia).

Classification General Classification +2

"Notic My Speech" -- Blending Speech Patterns With Multimedia

no code implementations12 Jun 2020 Dhruva Sahrawat, Yaman Kumar, Shashwat Aggarwal, Yifang Yin, Rajiv Ratn Shah, Roger Zimmermann

To close the gap between speech understanding and multimedia video applications, in this paper, we show the initial experiments by modelling the perception on visual speech and showing its use case on video compression.

speech-recognition Video Compression +1

audino: A Modern Annotation Tool for Audio and Speech

1 code implementation9 Jun 2020 Manraj Singh Grover, Pakhi Bamdev, Ratin Kumar Brala, Yaman Kumar, Mika Hama, Rajiv Ratn Shah

The tool allows audio data and their corresponding annotations to be uploaded and assigned to a user through a key-based API.

Action Detection Activity Detection +4

End-to-end Named Entity Recognition from English Speech

1 code implementation22 May 2020 Hemant Yadav, Sreyan Ghosh, Yi Yu, Rajiv Ratn Shah

Named entity recognition (NER) from text has been a widely studied problem and usually extracts semantic information from text.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Multi-modal Automated Speech Scoring using Attention Fusion

no code implementations17 May 2020 Manraj Singh Grover, Yaman Kumar, Sumit Sarin, Payman Vafaee, Mika Hama, Rajiv Ratn Shah

In this study, we propose a novel multi-modal end-to-end neural approach for automated assessment of non-native English speakers' spontaneous speech using attention fusion.

C3VQG: Category Consistent Cyclic Visual Question Generation

1 code implementation15 May 2020 Shagun Uppal, Anish Madan, Sarthak Bhagat, Yi Yu, Rajiv Ratn Shah

In this paper, we try to exploit the different visual cues and concepts in an image to generate questions using a variational autoencoder (VAE) without ground-truth answers.

Natural Questions Question Generation +1

COBRA: Contrastive Bi-Modal Representation Algorithm

1 code implementation7 May 2020 Vishaal Udandarao, Abhishek Maiti, Deepak Srivatsav, Suryatej Reddy Vyalla, Yifang Yin, Rajiv Ratn Shah

In this paper, we present a novel framework COBRA that aims to train two modalities (image and text) in a joint fashion inspired by the Contrastive Predictive Coding (CPC) and Noise Contrastive Estimation (NCE) paradigms which preserve both inter and intra-class relationships.

Cross-Modal Retrieval Image Captioning +3

An Annotated Dataset of Discourse Modes in Hindi Stories

no code implementations LREC 2020 Swapnil Dhanwal, Hritwik Dutta, Hitesh Nankani, Nilay Shrivastava, Yaman Kumar, Junyi Jessy Li, Debanjan Mahata, Rakesh Gosangi, Haimin Zhang, Rajiv Ratn Shah, Am Stent, a

In this paper, we present a new corpus consisting of sentences from Hindi short stories annotated for five different discourse modes argumentative, narrative, descriptive, dialogic and informative.

Descriptive Sentence

Touchless Typing using Head Movement-based Gestures

1 code implementation24 Jan 2020 Shivam Rustagi, Aakash Garg, Pranay Raj Anand, Rajesh Kumar, Yaman Kumar, Rajiv Ratn Shah

The modified GRU-based model outperforms the standard CNN-RNN and Conv3D models for three of the four scenarios.

Human-Computer Interaction I.2.7

#MeTooMA: Multi-Aspect Annotations of Tweets Related to the MeToo Movement

no code implementations14 Dec 2019 Akash Gautam, Puneet Mathur, Rakesh Gosangi, Debanjan Mahata, Ramit Sawhney, Rajiv Ratn Shah

In this paper, we present a dataset containing 9, 973 tweets related to the MeToo movement that were manually annotated for five different linguistic aspects: relevance, stance, hate speech, sarcasm, and dialogue acts.

Universal EEG Encoder for Learning Diverse Intelligent Tasks

no code implementations26 Nov 2019 Baani Leen Kaur Jolly, Palash Aggrawal, Surabhi S. Nath, Viresh Gupta, Manraj Singh Grover, Rajiv Ratn Shah

Brain Computer Interfaces (BCI) have become very popular with Electroencephalography (EEG) being one of the most commonly used signal acquisition techniques.

EEG General Classification +2

Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

1 code implementation26 Nov 2019 Osaid Rehman Nasir, Shailesh Kumar Jha, Manraj Singh Grover, Yi Yu, Ajit Kumar, Rajiv Ratn Shah

We then model the highly multi-modal problem of text to face generation as learning the conditional distribution of faces (conditioned on text) in same latent space.

Face Generation Face Reconstruction +1

Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings

no code implementations19 Oct 2019 Dhruva Sahrawat, Debanjan Mahata, Mayank Kulkarni, Haimin Zhang, Rakesh Gosangi, Amanda Stent, Agniv Sharma, Yaman Kumar, Rajiv Ratn Shah, Roger Zimmermann

In this paper, we formulate keyphrase extraction from scholarly articles as a sequence labeling task solved using a BiLSTM-CRF, where the words in the input text are represented using deep contextualized embeddings.

Keyphrase Extraction Word Embeddings

BHAAV- A Text Corpus for Emotion Analysis from Hindi Stories

1 code implementation9 Oct 2019 Yaman Kumar, Debanjan Mahata, Sagar Aggarwal, Anmol Chugh, Rajat Maheshwari, Rajiv Ratn Shah

In this paper, we introduce the first and largest Hindi text corpus, named BHAAV, which means emotions in Hindi, for analyzing emotions that a writer expresses through his characters in a story, as perceived by a narrator/reader.

Emotion Recognition Sentence

Keyphrase Generation for Scientific Articles using GANs

1 code implementation24 Sep 2019 Avinash Swaminathan, Raj Kuwar Gupta, Haimin Zhang, Debanjan Mahata, Rakesh Gosangi, Rajiv Ratn Shah

In this paper, we present a keyphrase generation approach using conditional Generative Adversarial Networks (GAN).

Keyphrase Generation

ARHNet - Leveraging Community Interaction for Detection of Religious Hate Speech in Arabic

no code implementations ACL 2019 Arijit Ghosh Chowdhury, Aniket Didolkar, Ramit Sawhney, Rajiv Ratn Shah

The rapid widespread of social media has lead to some undesirable consequences like the rapid increase of hateful content and offensive language.

Word Embeddings

\#YouToo? Detection of Personal Recollections of Sexual Harassment on Social Media

1 code implementation ACL 2019 Arijit Ghosh Chowdhury, Ramit Sawhney, Rajiv Ratn Shah, Debanjan Mahata

The availability of large-scale online social data, coupled with computational methods can help us answer fundamental questions relat- ing to our social lives, particularly our health and well-being.

Speak up, Fight Back! Detection of Social Media Disclosures of Sexual Harassment

no code implementations NAACL 2019 Arijit Ghosh Chowdhury, Ramit Sawhney, Puneet Mathur, Debanjan Mahata, Rajiv Ratn Shah

The {\#}MeToo movement is an ongoing prevalent phenomenon on social media aiming to demonstrate the frequency and widespread of sexual harassment by providing a platform to speak narrate personal experiences of such harassment.

Classification General Classification +5

MobiVSR: A Visual Speech Recognition Solution for Mobile Devices

no code implementations10 May 2019 Nilay Shrivastava, Astitwa Saxena, Yaman Kumar, Rajiv Ratn Shah, Debanjan Mahata, Amanda Stent

Visual speech recognition (VSR) is the task of recognizing spoken language from video input only, without any audio.

Lip Reading Quantization +2

Identifying Offensive Posts and Targeted Offense from Twitter

no code implementations19 Apr 2019 Haimin Zhang, Debanjan Mahata, Simra Shahid, Laiba Mehnaz, Sarthak Anand, Yaman Singla, Rajiv Ratn Shah, Karan Uppal

In this paper we present our approach and the system description for Sub-task A and Sub Task B of SemEval 2019 Task 6: Identifying and Categorizing Offensive Language in Social Media.

Suggestion Mining from Online Reviews using ULMFiT

1 code implementation19 Apr 2019 Sarthak Anand, Debanjan Mahata, Kartik Aggarwal, Laiba Mehnaz, Simra Shahid, Haimin Zhang, Yaman Kumar, Rajiv Ratn Shah, Karan Uppal

In this paper we present our approach and the system description for Sub Task A of SemEval 2019 Task 9: Suggestion Mining from Online Reviews and Forums.

General Classification Language Modeling +5

Harnessing GANs for Zero-shot Learning of New Classes in Visual Speech Recognition

1 code implementation29 Jan 2019 Yaman Kumar, Dhruva Sahrawat, Shubham Maheshwari, Debanjan Mahata, Amanda Stent, Yifang Yin, Rajiv Ratn Shah, Roger Zimmermann

To solve this problem, we present a novel approach to zero-shot learning by generating new classes using Generative Adversarial Networks (GANs), and show how the addition of unseen class samples increases the accuracy of a VSR system by a significant margin of 27% and allows it to handle speaker-independent out-of-vocabulary phrases.

speech-recognition Visual Speech Recognition +1

Kiki Kills: Identifying Dangerous Challenge Videos from Social Media

no code implementations2 Dec 2018 Nupur Baghel, Yaman Kumar, Paavini Nanda, Rajiv Ratn Shah, Debanjan Mahata, Roger Zimmermann

There has been upsurge in the number of people participating in challenges made popular through social media channels.

Did you take the pill? - Detecting Personal Intake of Medicine from Twitter

no code implementations3 Aug 2018 Debanjan Mahata, Jasper Friedrichs, Rajiv Ratn Shah, Jing Jiang

We believe that the developed classifier has direct uses in the areas of psychology, health informatics, pharmacovigilance and affective computing for tracking moods, emotions and sentiments of patients expressing intake of medicine in social media.

Pharmacovigilance

Theme-weighted Ranking of Keywords from Text Documents using Phrase Embeddings

no code implementations16 Jul 2018 Debanjan Mahata, John Kuriakose, Rajiv Ratn Shah, Roger Zimmermann, John R. Talburt

Keyword extraction is a fundamental task in natural language processing that facilitates mapping of documents to a concise set of representative single and multi-word phrases.

Keyword Extraction

A Multimodal Approach to Predict Social Media Popularity

no code implementations16 Jul 2018 Mayank Meghawat, Satyendra Yadav, Debanjan Mahata, Yifang Yin, Rajiv Ratn Shah, Roger Zimmermann

In this work, we propose a multimodal dataset consisiting of content, context, and social information for popularity prediction.

Prediction

Harnessing AI for Speech Reconstruction using Multi-view Silent Video Feed

no code implementations2 Jul 2018 Yaman Kumar, Mayank Aggarwal, Pratham Nawal, Shin'ichi Satoh, Rajiv Ratn Shah, Roger Zimmerman

Recently, research has started venturing into generating (audio) speech from silent video sequences but there have been no developments thus far in dealing with divergent views and poses of a speaker.

Sound Audio and Speech Processing

#phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter

no code implementations16 May 2018 Debanjan Mahata, Jasper Friedrichs, Hitkul, Rajiv Ratn Shah

Mining social media messages for health and drug related information has received significant interest in pharmacovigilance research.

Pharmacovigilance

Cannot find the paper you are looking for? You can Submit a new open access paper.