Search Results for author:
Found 26 papers, 17 papers with code
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Recognizing the need for a comprehensive evaluation of LMM chart understanding, we also propose a MultiModal Chart Benchmark (\textbf{MMC-Benchmark}), a comprehensive human-annotated benchmark with nine distinct tasks evaluating reasoning capabilities over charts.
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in creating advanced web agents.
TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs
We construct a hierarchical task tree encompassing 7 major areas covering over 200 categories and over 800 tasks, which covers diverse capabilities such as question answering, reasoning, multiturn dialogue, and text generation, to evaluate LLMs in a comprehensive and in-depth manner.
InFoBench: Evaluating Instruction Following Ability in Large Language Models
This paper introduces the Decomposed Requirements Following Ratio (DRFR), a new metric for evaluating Large Language Models' (LLMs) ability to follow instructions.
Salience Allocation as Guidance for Abstractive Summarization
Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance.
Ranked #7 on
Abstractive Text Summarization
on CNN / Daily Mail
Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era
Therefore, in this paper, we introduce Usable XAI in the context of LLMs by analyzing (1) how XAI can benefit LLMs and AI systems, and (2) how LLMs can contribute to the advancement of XAI.
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
We then train a model to identify semantic equivalence between a target word in context and one of its glosses using these aligned inventories, which exhibits strong transfer capability to many WSD tasks.
Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations
To evaluate our approach, we introduce a cold-start recommendation benchmark, and the results demonstrate that the enhanced small language models can achieve comparable cold-start recommendation performance to that of large models with only $17\%$ of the inference time.
Efficient Zero-shot Event Extraction with Context-Definition Alignment
We argue that using the static embedding of the event type name might not be enough because a single word could be ambiguous, and we need a sentence to define the type semantics accurately.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
And with our pretrained reader, the entire system improves by up to 4% in exact match.
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations.
Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations
It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space.
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks.
Ranked #2 on
Visual Commonsense Tests
on ViComTe-color
Weakly-supervised Fine-grained Event Recognition on Social Media Texts for Disaster Management
People increasingly use social media to report emergencies, seek help or share information during disasters, which makes social networks an important tool for disaster management.
NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization
To encourage research in this direction, we propose NarraSum, a large-scale narrative summarization dataset.
ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base.
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
In this work, we investigate how the instruction tuning adjusts pre-trained models with a focus on intrinsic changes.
Temporal Event Knowledge Acquisition via Identifying Narratives
Inspired by the double temporality characteristic of narrative texts, we propose a novel approach for acquiring rich temporal "before/after" event knowledge across sentences in narrative stories.
Using Context Events in Neural Network Models for Event Temporal Status Identification
Focusing on the task of identifying event temporal status, we find that events directly or indirectly governing the target event in a dependency tree are most important contexts.
A Weakly Supervised Approach to Train Temporal Relation Classifiers and Acquire Regular Event Pairs Simultaneously
Capabilities of detecting temporal relations between two events can benefit many applications.
Online Deception Detection Refueled by Real World Data Collection
The lack of large realistic datasets presents a bottleneck in online deception detection studies.
Weakly Supervised Subevent Knowledge Acquisition
We first obtain the initial set of event pairs that are likely to have the subevent relation, by exploiting two observations that 1) subevents are temporally contained by the parent event, and 2) the definitions of the parent event can be used to further guide the identification of subevents.
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
In this paper, we develop a novel semi-parametric language model architecture, Knowledge-in-Context (KiC), which empowers a parametric text-to-text language model with a knowledge-rich external memory.
Ranked #5 on
Question Answering
on StoryCloze
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57. 6% of the correctly detected hallucinations.
MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning
Our approach uniquely considers the various annotation formats as different "views" and leverages them in training the model.
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance.
