Search Results for author: Shikib Mehri

Found 28 papers, 11 papers with code

Data-Efficient Alignment of Large Language Models with Human Feedback Through Natural Language

no code implementations24 Nov 2023 Di Jin, Shikib Mehri, Devamanyu Hazarika, Aishwarya Padmakumar, Sungjin Lee, Yang Liu, Mahdi Namazifar

Learning from human feedback is a prominent technique to align the output of large language models (LLMs) with human expectations.

Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation

no code implementations27 Jan 2023 Jessica Huynh, Cathy Jiao, Prakhar Gupta, Shikib Mehri, Payal Bajaj, Vishrav Chaudhary, Maxine Eskenazi

The paper shows that the choice of datasets used for training a model contributes to how well it performs on a task as well as on how the prompt should be structured.

Question Answering

The DialPort tools

no code implementations SIGDIAL (ACL) 2022 Jessica Huynh, Shikib Mehri, Cathy Jiao, Maxine Eskenazi

The DialPort project http://dialport. org/, funded by the National Science Foundation (NSF), covers a group of tools and services that aim at fulfilling the needs of the dialog research community.

LAD: Language Models as Data for Zero-Shot Dialog

no code implementations SIGDIAL (ACL) 2022 Shikib Mehri, Yasemin Altun, Maxine Eskenazi

To facilitate zero-shot generalization in taskoriented dialog, this paper proposes Language Models as Data (LAD).

slot-filling Slot Filling +1

Interactive Evaluation of Dialog Track at DSTC9

no code implementations LREC 2022 Shikib Mehri, Yulan Feng, Carla Gordon, Seyed Hossein Alavi, David Traum, Maxine Eskenazi

Our track challenges participants to develop strong response generation models and explore strategies that extend them to back-and-forth interactions with real users.

Interactive Evaluation of Dialog Open-Domain Dialog +1

InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning

1 code implementation25 May 2022 Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, Jeffrey P. Bigham

We introduce InstructDial, an instruction tuning framework for dialogue, which consists of a repository of 48 diverse dialogue tasks in a unified text-to-text format created from 59 openly available dialogue datasets.

Dialogue Evaluation Dialogue Generation +3

Schema-Guided Paradigm for Zero-Shot Dialog

1 code implementation SIGDIAL (ACL) 2021 Shikib Mehri, Maxine Eskenazi

Developing mechanisms that flexibly adapt dialog systems to unseen tasks and domains is a major challenge in dialog research.

Memorization Transfer Learning

GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling

1 code implementation SIGDIAL (ACL) 2021 Shikib Mehri, Maxine Eskenazi

We instead achieve strong alignment by simultaneously modifying both the pre-trained model and the formulation of the downstream task, which is more efficient and preserves the scalability of transfer learning.

Open-Domain Dialog slot-filling +2

Reasoning Over History: Context Aware Visual Dialog

no code implementations EMNLP (nlpbt) 2020 Muhammad A. Shah, Shikib Mehri, Tejas Srinivasan

While neural models have been shown to exhibit strong performance on single-turn visual question answering (VQA) tasks, extending VQA to a multi-turn, conversational setting remains a challenge.

coreference-resolution Question Answering +2

STAR: A Schema-Guided Dialog Dataset for Transfer Learning

1 code implementation22 Oct 2020 Johannes E. M. Mosig, Shikib Mehri, Thomas Kober

We present STAR, a schema-guided task-oriented dialog dataset consisting of 127, 833 utterances and knowledge base queries across 5, 820 task-oriented dialogs in 13 domains that is especially designed to facilitate task and domain transfer learning in task-oriented dialog.

Transfer Learning Zero-shot Generalization

Example-Driven Intent Prediction with Observers

1 code implementation NAACL 2021 Shikib Mehri, Mihail Eric

Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances.

Attribute intent-classification +3

DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

1 code implementation28 Sep 2020 Shikib Mehri, Mihail Eric, Dilek Hakkani-Tur

A long-standing goal of task-oriented dialogue research is the ability to flexibly adapt dialogue models to new domains.

Ranked #5 on Multi-domain Dialogue State Tracking on MULTIWOZ 2.1 (using extra training data)

Domain Adaptation Multi-domain Dialogue State Tracking +1

``None of the Above'': Measure Uncertainty in Dialog Response Retrieval

no code implementations ACL 2020 Yulan Feng, Shikib Mehri, Maxine Eskenazi, Tiancheng Zhao

This paper discusses the importance of uncovering uncertainty in end-to-end dialog tasks and presents our experimental results on uncertainty classification on the processed Ubuntu Dialog Corpus.

General Classification Retrieval

Unsupervised Evaluation of Interactive Dialog with DialoGPT

2 code implementations SIGDIAL (ACL) 2020 Shikib Mehri, Maxine Eskenazi

It is important to define meaningful and interpretable automatic evaluation metrics for open-domain dialog research.

Dialogue Evaluation Text Generation

"None of the Above":Measure Uncertainty in Dialog Response Retrieval

no code implementations4 Apr 2020 Yulan Feng, Shikib Mehri, Maxine Eskenazi, Tiancheng Zhao

This paper discusses the importance of uncovering uncertainty in end-to-end dialog tasks, and presents our experimental results on uncertainty classification on the Ubuntu Dialog Corpus.

General Classification Object Detection +1

CMU GetGoing: An Understandable and Memorable Dialog System for Seniors

no code implementations3 Sep 2019 Shikib Mehri, Alan W. black, Maxine Eskenazi

Voice-based technologies are typically developed for the average user, and thus generally not tailored to the specific needs of any subgroup of the population, like seniors.

Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References

2 code implementations WS 2019 Prakhar Gupta, Shikib Mehri, Tiancheng Zhao, Amy Pavel, Maxine Eskenazi, Jeffrey P. Bigham

The aim of this paper is to mitigate the shortcomings of automatic evaluation of open-domain dialog systems through multi-reference evaluation.

Dialogue Evaluation

Structured Fusion Networks for Dialog

1 code implementation WS 2019 Shikib Mehri, Tejas Srinivasan, Maxine Eskenazi

Neural dialog models have exhibited strong performance, however their end-to-end nature lacks a representation of the explicit structure of dialog.

reinforcement-learning Reinforcement Learning (RL)

Middle-Out Decoding

no code implementations NeurIPS 2018 Shikib Mehri, Leonid Sigal

Despite being virtually ubiquitous, sequence-to-sequence models are challenged by their lack of diversity and inability to be externally controlled.

Video Captioning

Cannot find the paper you are looking for? You can Submit a new open access paper.