Search Results for author:
Found 39 papers, 19 papers with code
GENE: Global Event Network Embedding
Current methods for event representation ignore related events in a corpus-level global context.
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling
Event schemas can guide our understanding and ability to make predictions with respect to what might happen next.
Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport
Timeline Summarization identifies major events from a news collection and describes them following temporal order, with key dates tagged.
Coreference by Appearance: Visually Grounded Event Coreference Resolution
Event coreference resolution is critical to understand events in the growing number of online news with multiple modalities including text, video, speech, etc.
COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System
To tackle the challenge of accurate and timely communication regarding the COVID-19 pandemic, we present a COVID-19 Claim Radar to automatically extract supporting and refuting claims on a daily basis.
Event Schema Induction with Double Graph Autoencoders
To induce event schemas from historical events, previous work uses an event-by-event scheme, ignoring the global structure of the entire schema graph.
New Frontiers of Information Extraction
This tutorial targets researchers and practitioners who are interested in AI and ML technologies for structural information extraction (IE) from unstructured textual sources.
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios
We introduce RESIN-11, a new schema-guided event extraction&prediction framework that can be applied to a large variety of newsworthy scenarios.
Text-Based Reasoning About Vector Graphics
By casting an image to a text-based representation, we can leverage the power of language models to learn alignment from SVG to visual primitives and generalize to unseen question-answering tasks.
Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models
We present a Chain-of-Action (CoA) framework for multimodal and retrieval-augmented Question-Answering (QA).
Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
Fact-checking research has extensively explored verification but less so the generation of natural-language explanations, crucial for user trust.
InfoPattern: Unveiling Information Propagation Patterns in Social Media
Social media play a significant role in shaping public opinion and influencing ideological communities through information propagation.
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation
We adopt a weakly-supervised approach to directly generate visual event structures from captions for ViStruct training, capitalizing on abundant image-caption pairs from the web.
Defining a New NLP Playground
The recent explosion of performance of large language models (LLMs) has changed the field of Natural Language Processing (NLP) more abruptly and seismically than any other shift in the field's 80-year history.
HallE-Control: Controlling Object Hallucination in Large Multimodal Models
Current Large Multimodal Models (LMMs) achieve remarkable progress, yet there remains significant uncertainty regarding their ability to accurately apprehend visual details, that is, in performing detailed captioning.
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification
Event schemas are a form of world knowledge about the typical progression of events.
Non-Sequential Graph Script Induction via Multimedia Grounding
To automate the induction of such graph scripts for given tasks, we propose to take advantage of loosely aligned videos of people performing the tasks.
LM-Switch: Lightweight Language Model Conditioning in Word Embedding Space
As pre-training and fine-tuning are costly and might negatively impact model performance, it is desired to efficiently adapt an existing model to different conditions such as styles, sentiments or narratives, when facing different audiences or scenarios.
SmartBook: AI-Assisted Situation Report Generation
Further, experiments show that expert analysts tend to add more information into the SmartBook reports, with only 2. 3% of the existing tokens being deleted, meaning SmartBook can serve as a useful foundation for analysts to build upon when creating intelligence reports.
ADEPT: A DEbiasing PrompT Framework
Despite this, relatively few efforts have been made to debias PLMs by prompt tuning with continuous prompts compared to its discrete counterpart.
Video Event Extraction via Tracking Visual States of Arguments
Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles.
Learning to Decompose Visual Features with Latent Textual Prompts
Recent advances in pre-training vision-language models like CLIP have shown great potential in learning transferable visual representations.
Multimedia Generative Script Learning for Task Planning
Goal-oriented generative script learning aims to generate subsequent steps to reach a particular goal, which is an essential task to assist robots or humans in performing stereotypical activities.
Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval
We surprisingly find that discrete text tokens coupled with a pretrained contrastive text model yields the best performance, which can even outperform state-of-the-art on the iVQA and How2QA datasets without additional training on millions of video-text data.
Ranked #1 on
Video Question Answering
on iVQA
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.
Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning
While much literature has been devoted to exploring alternative optimization strategies, we identify another essential aspect towards effective few-shot transfer learning, task sampling, which is previously only be viewed as part of data pre-processing in MAML.
CLIP-Event: Connecting Text and Images with Event Structures
Vision-language (V+L) pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text.
MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding
Specifically, the task involves multi-hop questions that require reasoning over image-caption pairs to identify the grounded visual object being referred to and then predicting a span from the news body text to answer the question.
Joint Multimedia Event Extraction from Video and Article
We introduce the new task of Video MultiMedia Event Extraction (Video M2E2) and propose two novel components to build the first system towards this task.
Event-Centric Natural Language Processing
This tutorial targets researchers and practitioners who are interested in AI technologies that help machines understand natural language text, particularly real-world events described in the text.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System
We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video).
The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction
We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation
To combat COVID-19, both clinicians and scientists need to digest vast amounts of relevant biomedical knowledge in scientific literature to understand the disease mechanism and related biological functions.
GAIA: A Fine-grained Multimedia Knowledge Extraction System
We present the first comprehensive, open source multimedia knowledge extraction system that takes a massive stream of unstructured, heterogeneous multimedia data from various sources and languages as input, and creates a coherent, structured knowledge base, indexing entities, relations, and events, following a rich, fine-grained ontology.
Cross-media Structured Common Space for Multimedia Event Extraction
We introduce a new task, MultiMedia Event Extraction (M2E2), which aims to extract events and their arguments from multimedia documents.
Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization
Transcripts of natural, multi-person meetings differ significantly from documents like news articles, which can make Natural Language Generation models for generating summaries unfocused.
Multilingual Entity, Relation, Event and Human Value Extraction
This paper demonstrates a state-of-the-art end-to-end multilingual (English, Russian, and Ukrainian) knowledge extraction system that can perform entity discovery and linking, relation extraction, event extraction, and coreference.
Path-Based Attention Neural Model for Fine-Grained Entity Typing
Fine-grained entity typing aims to assign entity mentions in the free text with types arranged in a hierarchical structure.
Efficient Parallel Translating Embedding For Knowledge Graphs
Knowledge graph embedding aims to embed entities and relations of knowledge graphs into low-dimensional vector spaces.
Ranked #1 on
Link Prediction
on WN18 (filtered)
