Search Results for author: Arindam Mitra

Found 29 papers, 6 papers with code

Deeply Embedded Knowledge Representation & Reasoning For Natural Language Question Answering: A Practitioner’s Perspective

no code implementations EMNLP (spnlp) 2020 Arindam Mitra, Sanjay Narayana, Chitta Baral

Successful application of Knowledge Representation and Reasoning (KR) in Natural Language Understanding (NLU) is largely limited by the availability of a robust and general purpose natural language parser.

Natural Language Understanding Question Answering

AgentInstruct: Toward Generative Teaching with Agentic Flows

no code implementations3 Jul 2024 Arindam Mitra, Luciano del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-Ge Chen, Olga Vrousgos, Corby Rosset, Fillipe Silva, Hamed Khanpour, Yash Lara, Ahmed Awadallah

We focus on using synthetic data for post-training, specifically creating data by powerful models to teach a new skill or behavior to another model, we refer to this setting as Generative Teaching.

GSM8K Reading Comprehension

LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models

1 code implementation23 Apr 2024 Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, Chitta Baral

Existing work investigating this reasoning ability of LLMs has focused only on a couple of inference rules (such as modus ponens and modus tollens) of propositional and first-order logic.

Logical Reasoning Question Answering

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

1 code implementation22 Apr 2024 Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Qin Cai, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Yen-Chun Chen, Yi-Ling Chen, Parul Chopra, Xiyang Dai, Allie Del Giorno, Gustavo de Rosa, Matthew Dixon, Ronen Eldan, Victor Fragoso, Dan Iter, Mei Gao, Min Gao, Jianfeng Gao, Amit Garg, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karampatziakis, Dongwoo Kim, Mahoud Khademi, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Ce Liu, Mengchen Liu, Weishung Liu, Eric Lin, Zeqi Lin, Chong Luo, Piyush Madan, Matt Mazzola, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Xin Wang, Lijuan Wang, Chunyu Wang, Yu Wang, Rachel Ward, Guanhua Wang, Philipp Witte, Haiping Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Sonali Yadav, Fan Yang, Jianwei Yang, ZiYi Yang, Yifan Yang, Donghan Yu, Lu Yuan, Chengruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, Xiren Zhou

We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.

Language Modelling

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

no code implementations4 Apr 2024 Corby Rosset, Ching-An Cheng, Arindam Mitra, Michael Santacroce, Ahmed Awadallah, Tengyang Xie

In this paper, we introduce Direct Nash Optimization (DNO), a provable and scalable algorithm that marries the simplicity and stability of contrastive learning with theoretical generality from optimizing general preferences.

Contrastive Learning

Orca-Math: Unlocking the potential of SLMs in Grade School Math

no code implementations16 Feb 2024 Arindam Mitra, Hamed Khanpour, Corby Rosset, Ahmed Awadallah

Ensembling provides a substantial boost in accuracy but at a significant cost increase with multiple calls to the model (e. g., Phi-GSM uses top-48 to boost the performance from 68. 2 to 81. 5).

Arithmetic Reasoning GSM8K +1

Teaching Language Models to Hallucinate Less with Synthetic Tasks

no code implementations10 Oct 2023 Erik Jones, Hamid Palangi, Clarisse Simões, Varun Chandrasekaran, Subhabrata Mukherjee, Arindam Mitra, Ahmed Awadallah, Ece Kamar

We also find that optimizing the system message rather than the model weights can be critical; fine-tuning the entire model on the synthetic task can counterintuitively increase hallucination.

Abstractive Text Summarization Hallucination +3

Orca: Progressive Learning from Complex Explanation Traces of GPT-4

3 code implementations5 Jun 2023 Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, Ahmed Awadallah

To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka. ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs.

Imitation Learning Knowledge Distillation

Instruction Tuned Models are Quick Learners

1 code implementation17 May 2023 Himanshu Gupta, Saurabh Arjun Sawant, Swaroop Mishra, Mutsumi Nakamura, Arindam Mitra, Santosh Mashetty, Chitta Baral

In the MTL setting, an instruction tuned model trained on only 6% of downstream training data achieve SOTA, while using 100% of the training data results in a 3. 69% points improvement (ROUGE-L 74. 68) over the previous SOTA.

In-Context Learning Multi-Task Learning +1

Improving Biomedical Information Retrieval with Neural Retrievers

no code implementations19 Jan 2022 Man Luo, Arindam Mitra, Tejas Gokhale, Chitta Baral

We show that BM25 and our method can complement each other, and a simple hybrid model leads to further gains in the large corpus setting.

Biomedical Information Retrieval Information Retrieval +4

Commonsense Reasoning with Implicit Knowledge in Natural Language

no code implementations AKBC 2021 Pratyay Banerjee, Swaroop Mishra, Kuntal Kumar Pal, Arindam Mitra, Chitta Baral

Two common approaches to this are (i) Use of well-structured commonsense present in knowledge graphs, and (ii) Use of progressively larger transformer language models.

Knowledge Graphs

Information leak and incompatibility of physical context: A modified approach

no code implementations15 Feb 2021 Arindam Mitra, Gautam Sharma, Sibasish Ghosh

The three primary limitations are that, (i) their approach is not positive operator-valued measurements and (ii), they restrict information-theoretic agents Alice, Eve and Bob to specific quantum operations and do not consider most general quantum operations i. e., quantum instruments and (iii), their measure of IPC can take negative values in specific cases in a more general scenario which implies the limitation of their information measure.

Quantum Physics Mathematical Physics Mathematical Physics

Layers of classicality in the compatibility of measurements

no code implementations14 Jan 2021 Arindam Mitra

In particular we show that: (i) none of the layers of classicality respect transitivity property, (ii) the concept like degree of broadcasting similar to degree of compatibility does not exist, (iii) there exist informationally incomplete POVMs that are not individually broadcastable, (iv) a set of broadcasting channels can be obtained through concatenation of broadcasting and non-disturbing channels, (v) unlike compatibility, other layers of classicality are not convex, in general.

Quantum Physics Mathematical Physics Mathematical Physics

Can Transformers Reason About Effects of Actions?

no code implementations17 Dec 2020 Pratyay Banerjee, Chitta Baral, Man Luo, Arindam Mitra, Kuntal Pal, Tran C. Son, Neeraj Varshney

A recent work has shown that transformers are able to "reason" with facts and rules in a limited setting where the rules are natural language expressions of conjunctions of conditions implying a conclusion.

Common Sense Reasoning Question Answering

Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks

no code implementations18 May 2020 Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Sachdeva, Chitta Baral

However, there exists a strong need for a benchmark which can evaluate the abilities of models, in performing question format independent numerical reasoning, as (i) the numerical reasoning capabilities we want to teach are not controlled by question formats, (ii) for numerical reasoning technology to have the best possible application, it must be able to process language and reason in a way that is not exclusive to a single format, task, dataset or domain.

Natural Language Inference Question Answering +1

Natural Language QA Approaches using Reasoning with External Knowledge

no code implementations6 Mar 2020 Chitta Baral, Pratyay Banerjee, Kuntal Kumar Pal, Arindam Mitra

The challenges inspired by Winograd's councilmen example, and recent developments such as the Rebooting AI book, various NLQA datasets, research on knowledge acquisition in the NLQA context, and their use in various NLQA models have brought the issue of NLQA using ``reasoning'' with external knowledge to the forefront.

Question Answering

How Additional Knowledge can Improve Natural Language Commonsense Question Answering?

no code implementations19 Sep 2019 Arindam Mitra, Pratyay Banerjee, Kuntal Kumar Pal, Swaroop Mishra, Chitta Baral

Recently several datasets have been proposed to encourage research in Question Answering domains where commonsense knowledge is expected to play an important role.

Language Modelling Multiple-choice +1

A Generate-Validate Approach to Answering Questions about Qualitative Relationships

no code implementations9 Aug 2019 Arindam Mitra, Chitta Baral, Aurgho Bhattacharjee, Ishan Shrivastava

Qualitative relationships describe how increasing or decreasing one property (e. g. altitude) affects another (e. g. temperature).

Question Answering Transfer Learning

Careful Selection of Knowledge to solve Open Book Question Answering

no code implementations ACL 2019 Pratyay Banerjee, Kuntal Kumar Pal, Arindam Mitra, Chitta Baral

Open book question answering is a type of natural language based QA (NLQA) where questions are expected to be answered with respect to a given set of open book facts, and common knowledge about a topic.

Information Retrieval Question Answering +2

Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge

no code implementations ACL 2019 Ashok Prakash, Arpit Sharma, Arindam Mitra, Chitta Baral

Our end-to-end system built in such a manner improves on the accuracy of two of the available language model based approaches by 5. 53{\%} and 7. 7{\%} respectively.

Language Modelling

Declarative Question Answering over Knowledge Bases containing Natural Language Text with Answer Set Programming

1 code implementation1 May 2019 Arindam Mitra, Peter Clark, Oyvind Tafjord, Chitta Baral

While in recent years machine learning (ML) based approaches have been the popular approach in developing end-to-end question answering systems, such systems often struggle when additional knowledge is needed to correctly answer the questions.

Logical Reasoning Natural Language Inference +1

Cannot find the paper you are looking for? You can Submit a new open access paper.