Search Results for author: Mari Ostendorf

Found 67 papers, 27 papers with code

OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Tracking

no code implementations16 Nov 2023 Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf

Large language models (LLMs) have revolutionized the landscape of Natural Language Processing systems, but are computationally expensive.

Dialogue State Tracking Sentence Embedding +1

DIALGEN: Collaborative Human-LM Generated Dialogues for Improved Understanding of Human-Human Conversations

no code implementations13 Jul 2023 Bo-Ru Lu, Nikita Haduong, Chia-Hsuan Lee, Zeqiu Wu, Hao Cheng, Paul Koester, Jean Utke, Tao Yu, Noah A. Smith, Mari Ostendorf

Applications that could benefit from automatic understanding of human-human conversations often come with challenges associated with private information in real-world data such as call center or clinical conversations.

Dialogue Generation Dialogue State Tracking +1

Building blocks for complex tasks: Robust generative event extraction for radiology reports under domain shifts

no code implementations15 Jun 2023 Sitong Zhou, Meliha Yetisgen, Mari Ostendorf

This paper explores methods for extracting information from radiology reports that generalize across exam modalities to reduce requirements for annotated data.

Domain Adaptation Event Extraction

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

no code implementations NeurIPS 2023 Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, Hannaneh Hajishirzi

We introduce Fine-Grained RLHF, a framework that enables training and learning from reward functions that are fine-grained in two respects: (1) density, providing a reward after every segment (e. g., a sentence) is generated; and (2) incorporating multiple reward models associated with different feedback types (e. g., factual incorrectness, irrelevance, and information incompleteness).

Language Modelling Long Form Question Answering +1

TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

1 code implementation ICCV 2023 Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, Noah A Smith

We introduce TIFA (Text-to-Image Faithfulness evaluation with question Answering), an automatic evaluation metric that measures the faithfulness of a generated image to its text input via visual question answering (VQA).

Language Modelling Object Counting +2

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

2 code implementations19 Dec 2022 Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu

Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets.

Information Retrieval Learning Word Embeddings +3

Binding Language Models in Symbolic Languages

1 code implementation6 Oct 2022 Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e. g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations.

Language Modelling Semantic Parsing +1

Generalizing through Forgetting -- Domain Generalization for Symptom Event Extraction in Clinical Notes

no code implementations20 Sep 2022 Sitong Zhou, Kevin Lybarger, Meliha Yetisgen, Mari Ostendorf

To reduce reliance on domain-specific features, we propose a domain generalization method that dynamically masks frequent symptoms words in the source domain.

Domain Generalization Event Extraction +2

Selective Annotation Makes Language Models Better Few-Shot Learners

1 code implementation5 Sep 2022 Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time.

Code Generation Retrieval

Unsupervised Learning of Hierarchical Conversation Structure

1 code implementation24 May 2022 Bo-Ru Lu, Yushi Hu, Hao Cheng, Noah A. Smith, Mari Ostendorf

Human conversations can evolve in many different ways, creating challenges for automatic understanding and summarization.

Automatic Dialect Density Estimation for African American English

no code implementations3 Apr 2022 Alexander Johnson, Kevin Everson, Vijay Ravi, Anissa Gladney, Mari Ostendorf, Abeer Alwan

In this paper, we explore automatic prediction of dialect density of the African American English (AAE) dialect, where dialect density is defined as the percentage of words in an utterance that contain characteristics of the non-standard dialect.

Density Estimation Language Modelling

In-Context Learning for Few-Shot Dialogue State Tracking

1 code implementation16 Mar 2022 Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

In this work, we propose an in-context learning (ICL) framework for zero-shot and few-shot learning DST, where a large pre-trained language model (LM) takes a test instance and a few exemplars as input, and directly decodes the dialogue state without any parameter updates.

Dialogue State Tracking Few-Shot Learning +3

CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning

no code implementations16 Dec 2021 Zeqiu Wu, Yi Luan, Hannah Rashkin, David Reitter, Hannaneh Hajishirzi, Mari Ostendorf, Gaurav Singh Tomar

Compared to standard retrieval tasks, passage retrieval for conversational question answering (CQA) poses new challenges in understanding the current user question, as each question needs to be interpreted within the dialogue context.

Conversational Question Answering Passage Retrieval +3

Dialogue State Tracking with a Language Model using Schema-Driven Prompting

1 code implementation EMNLP 2021 Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf

Task-oriented conversational systems often use dialogue state tracking to represent the user's intentions, which involves filling in values of pre-defined slots.

 Ranked #1 on Dialogue State Tracking on MULTIWOZ 2.1 (MultiWOZ (Joint Goal Acc) metric)

Dialogue State Tracking Language Modelling +1

DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization

1 code implementation EMNLP 2021 Zeqiu Wu, Bo-Ru Lu, Hannaneh Hajishirzi, Mari Ostendorf

Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation.

Response Generation

Representations for Question Answering from Documents with Tables and Text

no code implementations EACL 2021 Vicky Zayats, Kristina Toutanova, Mari Ostendorf

Tables in Web documents are pervasive and can be directly used to answer many of the queries searched on the Web, motivating their integration in question answering.

Natural Questions Question Answering

Extracting COVID-19 Diagnoses and Symptoms From Clinical Text: A New Annotated Corpus and Neural Event Extraction Framework

no code implementations2 Dec 2020 Kevin Lybarger, Mari Ostendorf, Matthew Thompson, Meliha Yetisgen

In a secondary use application, we explored the prediction of COVID-19 test results using structured patient data (e. g. vital signs and laboratory results) and automatically extracted symptom information.

Event Extraction

On the Role of Style in Parsing Speech with Neural Models

no code implementations8 Oct 2020 Trang Tran, Jiahong Yuan, Yang Liu, Mari Ostendorf

The differences in written text and conversational speech are substantial; previous parsers trained on treebanked text have given very poor results on spontaneous speech.

Analysis of Disfluency in Children's Speech

no code implementations8 Oct 2020 Trang Tran, Morgan Tinkler, Gary Yeung, Abeer Alwan, Mari Ostendorf

Disfluencies are prevalent in spontaneous speech, as shown in many studies of adult speech.

Extracting Summary Knowledge Graphs from Long Documents

1 code implementation19 Sep 2020 Zeqiu Wu, Rik Koncel-Kedziorski, Mari Ostendorf, Hannaneh Hajishirzi

Knowledge graphs capture entities and relations from long documents and can facilitate reasoning in many downstream applications.

Graph Learning Knowledge Graphs +1

A Controllable Model of Grounded Response Generation

1 code implementation1 May 2020 Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, Bill Dolan

Current end-to-end neural conversation models inherently lack the flexibility to impose semantic control in the response generation process, often resulting in uninteresting responses.

Informativeness Response Generation

Annotating Social Determinants of Health Using Active Learning, and Characterizing Determinants Using Neural Event Extraction

no code implementations11 Apr 2020 Kevin Lybarger, Mari Ostendorf, Meliha Yetisgen

The Social History Annotation Corpus (SHAC) includes 4, 480 social history sections with detailed annotation for 12 SDOH characterizing the status, extent, and temporal information of 18K distinct events.

Active Learning Decision Making +3

Automated Essay Scoring with Discourse-Aware Neural Models

no code implementations WS 2019 Farah Nadeem, Huy Nguyen, Yang Liu, Mari Ostendorf

Automated essay scoring systems typically rely on hand-crafted features to predict essay quality, but such systems are limited by the cost of feature engineering.

Automated Essay Scoring Feature Engineering

A Dynamic Speaker Model for Conversational Interactions

1 code implementation NAACL 2019 Hao Cheng, Hao Fang, Mari Ostendorf

Characterizing these differences can be useful in human-computer interaction, as well as analysis of human-human conversations.

Text Generation

Disfluencies and Human Speech Transcription Errors

no code implementations8 Apr 2019 Vicky Zayats, Trang Tran, Richard Wright, Courtney Mansfield, Mari Ostendorf

This paper explores contexts associated with errors in transcrip-tion of spontaneous speech, shedding light on human perceptionof disfluencies and other conversational speech phenomena.

A General Framework for Information Extraction using Dynamic Span Graphs

3 code implementations NAACL 2019 Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, Hannaneh Hajishirzi

We introduce a general framework for several information extraction tasks that share span representations using dynamically constructed span graphs.

 Ranked #1 on Relation Extraction on ACE 2004 (Cross Sentence metric)

Joint Entity and Relation Extraction Named Entity Recognition (NER)

Robust cross-domain disfluency detection with pattern match networks

no code implementations17 Nov 2018 Vicky Zayats, Mari Ostendorf

In this paper we introduce a novel pattern match neural network architecture that uses neighbor similarity scores as features, eliminating the need for feature engineering in a disfluency detection task.

Feature Engineering

Scientific Relation Extraction with Selectively Incorporated Concept Embeddings

no code implementations26 Aug 2018 Yi Luan, Mari Ostendorf, Hannaneh Hajishirzi

This paper describes our submission for the SemEval 2018 Task 7 shared task on semantic relation extraction and classification in scientific papers.

Classification General Classification +1

Domain Adversarial Training for Accented Speech Recognition

no code implementations7 Jun 2018 Sining Sun, Ching-Feng Yeh, Mei-Yuh Hwang, Mari Ostendorf, Lei Xie

In this paper, we propose a domain adversarial training (DAT) algorithm to alleviate the accented speech recognition problem.

Accented Speech Recognition Multi-Task Learning +1

Training Augmentation with Adversarial Examples for Robust Speech Recognition

no code implementations7 Jun 2018 Sining Sun, Ching-Feng Yeh, Mari Ostendorf, Mei-Yuh Hwang, Lei Xie

This paper explores the use of adversarial examples in training speech recognition systems to increase robustness of deep neural network acoustic models.

Data Augmentation Robust Speech Recognition +1

Estimating Linguistic Complexity for Science Texts

1 code implementation WS 2018 Farah Nadeem, Mari Ostendorf

Evaluation of text difficulty is important both for downstream tasks like text simplification, and for supporting educators in classrooms.

Feature Engineering Reading Comprehension +1

Personalized Language Model for Query Auto-Completion

4 code implementations ACL 2018 Aaron Jaech, Mari Ostendorf

Query auto-completion is a search engine feature whereby the system suggests completed queries as the user types.

Language Modelling

Community Member Retrieval on Social Media using Textual Information

1 code implementation NAACL 2018 Aaron Jaech, Shobhit Hathi, Mari Ostendorf

This paper addresses the problem of community membership detection using only text features in a scenario where a small number of positive labeled examples defines the community.


Real-Time Prediction of the Duration of Distribution System Outages

no code implementations3 Apr 2018 Aaron Jaech, Baosen Zhang, Mari Ostendorf, Daniel S. Kirschen

This paper addresses the problem of predicting duration of unplanned power outages, using historical outage records to train a series of neural network predictors.

Low-Rank RNN Adaptation for Context-Aware Language Modeling

1 code implementation TACL 2018 Aaron Jaech, Mari Ostendorf

A context-aware language model uses location, user and/or domain metadata (context) to adapt its predictions.

General Classification Language Modelling

A Factored Neural Network Model for Characterizing Online Discussions in Vector Space

1 code implementation EMNLP 2017 Hao Cheng, Hao Fang, Mari Ostendorf

We develop a novel factored neural model that learns comment embeddings in an unsupervised way leveraging the structure of distributional context in online discussion forums.

Feature Engineering

Language Based Mapping of Science Assessment Items to Skills

no code implementations WS 2017 Farah Nadeem, Mari Ostendorf

Knowledge of the association between assessment questions and the skills required to solve them is necessary for analysis of student learning.

General Classification text-classification +1

Scientific Information Extraction with Semi-supervised Neural Tagging

no code implementations EMNLP 2017 Yi Luan, Mari Ostendorf, Hannaneh Hajishirzi

This paper addresses the problem of extracting keyphrases from scientific articles and categorizing them as corresponding to a task, process, or material.

named-entity-recognition Named Entity Recognition +1

Improving Context Aware Language Models

1 code implementation21 Apr 2017 Aaron Jaech, Mari Ostendorf

Increased adaptability of RNN language models leads to improved predictions that benefit many applications.

General Classification Language Modelling

Reinforcement Learning with External Knowledge and Two-Stage Q-functions for Predicting Popular Reddit Threads

no code implementations20 Apr 2017 Ji He, Mari Ostendorf, Xiaodong He

This paper addresses the problem of predicting popularity of comments in an online discussion forum using reinforcement learning, particularly addressing two challenges that arise from having natural language state and action spaces.

Q-Learning reinforcement-learning +1

Conversation Modeling on Reddit using a Graph-Structured LSTM

no code implementations TACL 2018 Vicky Zayats, Mari Ostendorf

This paper presents a novel approach for modeling threaded discussions on social media using a graph-structured bidirectional LSTM which represents both hierarchical and temporal conversation structure.

Characterizing the Language of Online Communities and its Relation to Community Reception

no code implementations EMNLP 2016 Trang Tran, Mari Ostendorf

This work investigates style and topic aspects of language in online communities: looking at both utility as an identifier of the community and correlation with community reception of content.

Language Modelling TAG

Learning Latent Local Conversation Modes for Predicting Community Endorsement in Online Discussions

no code implementations16 Aug 2016 Hao Fang, Hao Cheng, Mari Ostendorf

Many social media platforms offer a mechanism for readers to react to comments, both positively and negatively, which in aggregate can be thought of as community endorsement.

Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads

1 code implementation EMNLP 2016 Ji He, Mari Ostendorf, Xiaodong He, Jianshu Chen, Jianfeng Gao, Lihong Li, Li Deng

We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space.

reinforcement-learning Reinforcement Learning (RL)

Disfluency Detection using a Bidirectional LSTM

no code implementations12 Apr 2016 Vicky Zayats, Mari Ostendorf, Hannaneh Hajishirzi

We introduce a new approach for disfluency detection using a Bidirectional Long-Short Term Memory neural network (BLSTM).

Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding

no code implementations1 Apr 2016 Aaron Jaech, Larry Heck, Mari Ostendorf

The goal of this paper is to use multi-task learning to efficiently scale slot filling models for natural language understanding to handle multiple target tasks or domains.

Domain Adaptation Multi-Task Learning +3

LSTM based Conversation Models

1 code implementation31 Mar 2016 Yi Luan, Yangfeng Ji, Mari Ostendorf

In this paper, we present a conversational model that incorporates both context and participant role for two-party conversations.

Language Modelling Text Generation

Deep Reinforcement Learning with a Natural Language Action Space

3 code implementations ACL 2016 Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, Mari Ostendorf

This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games.

Q-Learning reinforcement-learning +2

What Your Username Says About You

1 code implementation EMNLP 2015 Aaron Jaech, Mari Ostendorf

Experimental results on the two tasks demonstrate the effectiveness of the proposed morphological features compared to a character n-gram baseline.

Talking to the crowd: What do people react to in online discussions?

no code implementations EMNLP 2015 Aaron Jaech, Victoria Zayats, Hao Fang, Mari Ostendorf, Hannaneh Hajishirzi

This paper addresses the question of how language use affects community reaction to comments in online discussion forums, and the relative importance of the message vs. the messenger.

Leveraging Twitter for Low-Resource Conversational Speech Language Modeling

no code implementations9 Apr 2015 Aaron Jaech, Mari Ostendorf

In applications involving conversational speech, data sparsity is a limiting factor in building a better language model.

Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.