Search Results for author:
Found 26 papers, 14 papers with code
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
To overcome these limitations, we introduce the BiGGen Bench, a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks.
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from various LMs.
HyperCLOVA X Technical Report
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding.
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.
EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
By simply composing prompts, developers can prototype novel generative applications with Large Language Models (LLMs).
Who Wrote this Code? Watermarking for Code Generation
Since the remarkable generation performance of large language models raised ethical and legal concerns, approaches to detect machine-generated text by embedding watermarks are being developed.
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2. 24% (Flan-T5 3B) and +2. 37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13. 98% margin.
Ranked #1 on
on BIG-bench (SNARKS)
Common Sense Reasoning
Common Sense Reasoning (Zero-Shot)
+7
Aligning Large Language Models through Synthetic Feedback
In human evaluation, our model is preferred to Alpaca and Dolly-v2, 55. 0% and 58. 5% of the time, respectively.
Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation
Most task-oriented dialogue (TOD) benchmarks assume users that know exactly how to use the system by constraining the user behaviors within the system's capabilities via strict user goals, namely "user familiarity" bias.
Towards Zero-Shot Functional Compositionality of Language Models
Large Pre-trained Language Models (PLM) have become the most desirable starting point in the field of NLP, as they have become remarkably good at solving many individual tasks.
Evaluating the Knowledge Dependency of Questions
They fail to evaluate the MCQ's ability to assess the student's knowledge of the corresponding target fact.
Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking
In this paper, we hypothesize that dialogue summaries are essentially unstructured dialogue states; hence, we propose to reformulate dialogue state tracking as a dialogue summarization problem.
Abstractive Dialogue Summarization
Dialogue State Tracking
+2
Pedagogical Word Recommendation: A novel task and dataset on personalized vocabulary acquisition for L2 learners
When learning a second language (L2), one of the most important but tedious components that often demoralizes students with its ineffectiveness and inefficiency is vocabulary acquisition, or more simply put, memorizing words.
KLUE: Korean Language Understanding Evaluation
We introduce Korean Language Understanding Evaluation (KLUE) benchmark.
Fast End-to-end Coreference Resolution for Korean
Recently, end-to-end neural network-based approaches have shown significant improvements over traditional pipeline-based models in English coreference resolution.
Attention over Parameters for Dialogue Systems
Dialogue systems require a great deal of different but complementary expertise to assist, inform, and entertain humans.
Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables
Despite the surging demands for multilingual task-oriented dialog systems (e. g., Alexa, Google Home), there has been less research done in multilingual or cross-lingual scenarios.
Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition
In countries that speak multiple main languages, mixing up different languages within a conversation is commonly called code-switching.
On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression
Despite their ubiquity in NLP tasks, Long Short-Term Memory (LSTM) networks suffer from computational inefficiencies caused by inherent unparallelizable recurrences, which further aggravates as LSTMs require more parameters for larger memory capacity.
MoEL: Mixture of Empathetic Listeners
Previous research on empathetic dialogue systems has mostly focused on generating responses given certain emotions.
Understanding the Shades of Sexism in Popular TV Series
[Multiple-submission] In the midst of a generation widely exposed to and influenced by media entertainment, the NLP research community has shown relatively little attention on the sexist comments in popular TV series.
CAiRE: An Empathetic Neural Chatbot
In this paper, we present an end-to-end empathetic conversation agent CAiRE.
Generating Empathetic Responses by Looking Ahead the User's Sentiment
Hence, in this paper, we propose Sentiment Look-ahead, which is a novel perspective for empathy that models the future user emotional state.
CAiRE_HKUST at SemEval-2019 Task 3: Hierarchical Attention for Dialogue Emotion Classification
Detecting emotion from dialogue is a challenge that has not yet been extensively surveyed.
CAiRE\_HKUST at SemEval-2019 Task 3: Hierarchical Attention for Dialogue Emotion Classification
Detecting emotion from dialogue is a challenge that has not yet been extensively surveyed.
Reducing Gender Bias in Abusive Language Detection
In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures.
