Search Results for author: Estevam Hruschka

Found 15 papers, 6 papers with code

Low-resource Entity Set Expansion: A Comprehensive Study on User-generated Text

1 code implementation Findings (NAACL) 2022 Yutong Shao, Nikita Bhutani, Sajjadur Rahman, Estevam Hruschka

Entity set expansion (ESE) aims at obtaining a more complete set of entities given a textual corpus and a seed set of entities of a concept.

Distilling Salient Reviews with Zero Labels

no code implementations FEVER (ACL) 2022 Chieh-Yang Huang, Jinfeng Li, Nikita Bhutani, Alexander Whedon, Estevam Hruschka, Yoshi Suhara

To alleviate this scarcity problem, we develop an unsupervised method, ZL-Distiller, which leverages contextual language representations of the reviews and their distributional patterns to identify salient sentences about entities.

Question Answering

Multi-Conditional Ranking with Large Language Models

1 code implementation30 Mar 2024 Pouya Pezeshkpour, Estevam Hruschka

Our analysis of LLMs using MCRank indicates a significant decrease in performance as the number and complexity of items and conditions grow.

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

no code implementations2 Feb 2024 Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani, Sajjadur Rahman, Tom Mitchell, Estevam Hruschka

We present a formal definition of reasoning capacity and illustrate its utility in identifying limitations within each component of the system.

Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks

1 code implementation10 Nov 2023 Pouya Pezeshkpour, Hayate Iso, Thom Lake, Nikita Bhutani, Estevam Hruschka

We meticulously craft this benchmark to cater to a wide array of HR tasks, including matching and explaining resumes to job descriptions, extracting skills and experiences from resumes, and editing resumes.

Language Modelling Large Language Model

Knowledge Graphs are not Created Equal: Exploring the Properties and Structure of Real KGs

no code implementations10 Nov 2023 Nedelina Teneva, Estevam Hruschka

Despite the recent popularity of knowledge graph (KG) related tasks and benchmarks such as KG embeddings, link prediction, entity alignment and evaluation of the reasoning abilities of pretrained language models as KGs, the structure and properties of real KGs are not well studied.

Entity Alignment Knowledge Graphs +1

Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks

no code implementations9 Nov 2023 Aditi Mishra, Sajjadur Rahman, Hannah Kim, Kushan Mitra, Estevam Hruschka

We consider the task of generating knowledge-guided rationalization in natural language by using expert-written examples in a few-shot manner.

Multiple-choice World Knowledge

Less is More for Long Document Summary Evaluation by LLMs

1 code implementation14 Sep 2023 Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, Estevam Hruschka

Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational costs and the Lost-in-the-Middle problem where important information in the middle of long documents is often overlooked.

Sentence Text Generation

Rethinking Language Models as Symbolic Knowledge Graphs

no code implementations25 Aug 2023 Vishwas Mruthyunjaya, Pouya Pezeshkpour, Estevam Hruschka, Nikita Bhutani

Despite these advancements, there is a void in comprehensively evaluating whether LMs can encompass the intricate topological and semantic attributes of KGs, attributes crucial for reasoning processes.

Knowledge Graphs Question Answering

Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

no code implementations22 Aug 2023 Pouya Pezeshkpour, Estevam Hruschka

Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting.

Multiple-choice

Towards Multifaceted Human-Centered AI

no code implementations9 Jan 2023 Sajjadur Rahman, Hannah Kim, Dan Zhang, Estevam Hruschka, Eser Kandogan

Human-centered AI workflows involve stakeholders with multiple roles interacting with each other and automated agents to accomplish diverse tasks.

MEGAnno: Exploratory Labeling for NLP in Computational Notebooks

no code implementations8 Jan 2023 Dan Zhang, Hannah Kim, Rafael Li Chen, Eser Kandogan, Estevam Hruschka

We present MEGAnno, a novel exploratory annotation framework designed for NLP researchers and practitioners.

Sentiment Analysis

Zero-shot Triplet Extraction by Template Infilling

1 code implementation21 Dec 2022 Bosung Kim, Hayate Iso, Nikita Bhutani, Estevam Hruschka, Ndapa Nakashole, Tom Mitchell

We propose a novel framework, ZETT (ZEro-shot Triplet extraction by Template infilling), that aligns the task objective to the pre-training objective of generative transformers to generalize to unseen relations.

Data Augmentation Language Modelling +2

TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration

1 code implementation24 Jun 2021 Dongjin Choi, Sara Evensen, Çağatay Demiralp, Estevam Hruschka

In this work, we extend the DPBD framework to span-level annotation tasks, arguably one of the most time-consuming NLP labeling tasks.

Active Learning Document Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.