Search Results for author: Manling Li

Found 39 papers, 19 papers with code

Coreference by Appearance: Visually Grounded Event Coreference Resolution

no code implementations CRAC (ACL) 2021 Liming Wang, Shengyu Feng, Xudong Lin, Manling Li, Heng Ji, Shih-Fu Chang

Event coreference resolution is critical to understand events in the growing number of online news with multiple modalities including text, video, speech, etc.

coreference-resolution Event Coreference Resolution +2

New Frontiers of Information Extraction

no code implementations NAACL (ACL) 2022 Muhao Chen, Lifu Huang, Manling Li, Ben Zhou, Heng Ji, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI and ML technologies for structural information extraction (IE) from unstructured textual sources.

Event Schema Induction with Double Graph Autoencoders

no code implementations NAACL 2022 Xiaomeng Jin, Manling Li, Heng Ji

To induce event schemas from historical events, previous work uses an event-by-event scheme, ignoring the global structure of the entire schema graph.

COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System

1 code implementation ACL 2022 Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-shyuan Chiang, Tuan Lai, Pengfei Yu, Zixuan Zhang, Heng Ji

To tackle the challenge of accurate and timely communication regarding the COVID-19 pandemic, we present a COVID-19 Claim Radar to automatically extract supporting and refuting claims on a daily basis.

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

1 code implementation EMNLP 2021 Manling Li, Tengfei Ma, Mo Yu, Lingfei Wu, Tian Gao, Heng Ji, Kathleen McKeown

Timeline Summarization identifies major events from a news collection and describes them following temporal order, with key dates tagged.

Timeline Summarization

Text-Based Reasoning About Vector Graphics

no code implementations9 Apr 2024 Zhenhailong Wang, Joy Hsu, Xingyao Wang, Kuan-Hao Huang, Manling Li, Jiajun Wu, Heng Ji

By casting an image to a text-based representation, we can leverage the power of language models to learn alignment from SVG to visual primitives and generalize to unseen question-answering tasks.

Descriptive Language Modelling +2

Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate

no code implementations12 Feb 2024 Kyungha Kim, Sangyun Lee, Kung-Hsiang Huang, Hou Pong Chan, Manling Li, Heng Ji

Fact-checking research has extensively explored verification but less so the generation of natural-language explanations, crucial for user trust.

Fact Checking Text Generation

InfoPattern: Unveiling Information Propagation Patterns in Social Media

no code implementations27 Nov 2023 Chi Han, Jialiang Xu, Manling Li, Hanning Zhang, Tarek Abdelzaher, Heng Ji

Social media play a significant role in shaping public opinion and influencing ideological communities through information propagation.

Stance Detection

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

1 code implementation22 Nov 2023 Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

We adopt a weakly-supervised approach to directly generate visual event structures from captions for ViStruct training, capitalizing on abundant image-caption pairs from the web.

Defining a New NLP Playground

no code implementations31 Oct 2023 Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi R. Fung, Charles Yu, Joel R. Tetreault, Eduard H. Hovy, Heng Ji

The recent explosion of performance of large language models (LLMs) has changed the field of Natural Language Processing (NLP) more abruptly and seismically than any other shift in the field's 80-year history.

HallE-Control: Controlling Object Hallucination in Large Multimodal Models

2 code implementations3 Oct 2023 Bohan Zhai, Shijia Yang, Chenfeng Xu, Sheng Shen, Kurt Keutzer, Chunyuan Li, Manling Li

Current Large Multimodal Models (LMMs) achieve remarkable progress, yet there remains significant uncertainty regarding their ability to accurately apprehend visual details, that is, in performing detailed captioning.

Attribute Hallucination +2

Non-Sequential Graph Script Induction via Multimedia Grounding

1 code implementation27 May 2023 Yu Zhou, Sha Li, Manling Li, Xudong Lin, Shih-Fu Chang, Mohit Bansal, Heng Ji

To automate the induction of such graph scripts for given tasks, we propose to take advantage of loosely aligned videos of people performing the tasks.

LM-Switch: Lightweight Language Model Conditioning in Word Embedding Space

no code implementations22 May 2023 Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek Abdelzaher, Heng Ji

As pre-training and fine-tuning are costly and might negatively impact model performance, it is desired to efficiently adapt an existing model to different conditions such as styles, sentiments or narratives, when facing different audiences or scenarios.

Language Modelling Word Embeddings

SmartBook: AI-Assisted Situation Report Generation

1 code implementation25 Mar 2023 Revanth Gangi Reddy, Yi R. Fung, Qi Zeng, Manling Li, Ziqi Wang, Paul Sullivan, Heng Ji

Further, experiments show that expert analysts tend to add more information into the SmartBook reports, with only 2. 3% of the existing tokens being deleted, meaning SmartBook can serve as a useful foundation for analysts to build upon when creating intelligence reports.

Decision Making

ADEPT: A DEbiasing PrompT Framework

1 code implementation10 Nov 2022 Ke Yang, Charles Yu, Yi Fung, Manling Li, Heng Ji

Despite this, relatively few efforts have been made to debias PLMs by prompt tuning with continuous prompts compared to its discrete counterpart.

Attribute Language Modelling +1

Video Event Extraction via Tracking Visual States of Arguments

no code implementations3 Nov 2022 Guang Yang, Manling Li, Jiajie Zhang, Xudong Lin, Shih-Fu Chang, Heng Ji

Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles.

Event Extraction

Learning to Decompose Visual Features with Latent Textual Prompts

no code implementations9 Oct 2022 Feng Wang, Manling Li, Xudong Lin, Hairong Lv, Alexander G. Schwing, Heng Ji

Recent advances in pre-training vision-language models like CLIP have shown great potential in learning transferable visual representations.


Multimedia Generative Script Learning for Task Planning

1 code implementation25 Aug 2022 Qingyun Wang, Manling Li, Hou Pong Chan, Lifu Huang, Julia Hockenmaier, Girish Chowdhary, Heng Ji

Goal-oriented generative script learning aims to generate subsequent steps to reach a particular goal, which is an essential task to assist robots or humans in performing stereotypical activities.

Contrastive Learning Descriptive +3

Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval

1 code implementation CVPR 2023 Xudong Lin, Simran Tiwari, Shiyuan Huang, Manling Li, Mike Zheng Shou, Heng Ji, Shih-Fu Chang

We surprisingly find that discrete text tokens coupled with a pretrained contrastive text model yields the best performance, which can even outperform state-of-the-art on the iVQA and How2QA datasets without additional training on millions of video-text data.

Retrieval Sentence +2

Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners

1 code implementation22 May 2022 Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, ZiYi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji

The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.

Attribute Automatic Speech Recognition +6

Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning

1 code implementation MMMPIE (COLING) 2022 Zhenhailong Wang, Hang Yu, Manling Li, Han Zhao, Heng Ji

While much literature has been devoted to exploring alternative optimization strategies, we identify another essential aspect towards effective few-shot transfer learning, task sampling, which is previously only be viewed as part of data pre-processing in MAML.

Few-Shot Learning Transfer Learning

CLIP-Event: Connecting Text and Images with Event Structures

1 code implementation CVPR 2022 Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang

Vision-language (V+L) pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text.

Contrastive Learning Event Extraction +2

MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding

2 code implementations20 Dec 2021 Revanth Gangi Reddy, Xilin Rui, Manling Li, Xudong Lin, Haoyang Wen, Jaemin Cho, Lifu Huang, Mohit Bansal, Avirup Sil, Shih-Fu Chang, Alexander Schwing, Heng Ji

Specifically, the task involves multi-hop questions that require reasoning over image-caption pairs to identify the grounded visual object being referred to and then predicting a span from the news body text to answer the question.

Answer Generation Data Augmentation +2

Event-Centric Natural Language Processing

no code implementations ACL 2021 Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI technologies that help machines understand natural language text, particularly real-world events described in the text.

The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction

1 code implementation EMNLP 2021 Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, Clare Voss

We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.

GAIA: A Fine-grained Multimedia Knowledge Extraction System

no code implementations ACL 2020 Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman

We present the first comprehensive, open source multimedia knowledge extraction system that takes a massive stream of unstructured, heterogeneous multimedia data from various sources and languages as input, and creates a coherent, structured knowledge base, indexing entities, relations, and events, following a rich, fine-grained ontology.

Cross-media Structured Common Space for Multimedia Event Extraction

no code implementations ACL 2020 Manling Li, Alireza Zareian, Qi Zeng, Spencer Whitehead, Di Lu, Heng Ji, Shih-Fu Chang

We introduce a new task, MultiMedia Event Extraction (M2E2), which aims to extract events and their arguments from multimedia documents.

Event Extraction

Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization

no code implementations ACL 2019 Manling Li, Lingyu Zhang, Heng Ji, Richard J. Radke

Transcripts of natural, multi-person meetings differ significantly from documents like news articles, which can make Natural Language Generation models for generating summaries unfocused.

Meeting Summarization Text Generation

Multilingual Entity, Relation, Event and Human Value Extraction

no code implementations NAACL 2019 Manling Li, Ying Lin, Joseph Hoover, Spencer Whitehead, Clare Voss, Morteza Dehghani, Heng Ji

This paper demonstrates a state-of-the-art end-to-end multilingual (English, Russian, and Ukrainian) knowledge extraction system that can perform entity discovery and linking, relation extraction, event extraction, and coreference.

Event Extraction Relation +1

Path-Based Attention Neural Model for Fine-Grained Entity Typing

no code implementations29 Oct 2017 Denghui Zhang, Pengshan Cai, Yantao Jia, Manling Li, Yuanzhuo Wang, Xue-Qi Cheng

Fine-grained entity typing aims to assign entity mentions in the free text with types arranged in a hierarchical structure.

Entity Typing

Efficient Parallel Translating Embedding For Knowledge Graphs

1 code implementation30 Mar 2017 Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, Xue-Qi Cheng

Knowledge graph embedding aims to embed entities and relations of knowledge graphs into low-dimensional vector spaces.

Knowledge Graph Embedding Knowledge Graphs +2

Cannot find the paper you are looking for? You can Submit a new open access paper.