Search Results for author:
Found 36 papers, 10 papers with code
Investigating Negation in Pre-trained Vision-and-language Models
Pre-trained vision-and-language models have achieved impressive results on a variety of tasks, including ones that require complex reasoning beyond object recognition.
Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation
Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects, which is essential for full scene understanding.
Film Trailer Generation via Task Decomposition
Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie.
A Temporal Variational Model for Story Generation
Recent language models can generate interesting and grammatically correct text in story generation but often lack plot development and long-term coherence.
Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories
Measuring event salience is essential in the understanding of stories.
A New Split for Evaluating True Zero-Shot Action Recognition
We benchmark several recent approaches on the proposed True Zero-Shot(TruZe) Split for UCF101 and HMDB51, with zero-shot and generalized zero-shot evaluation.
CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition
Theproblem can be seen as learning a function which general-izes well to instances of unseen classes without losing dis-crimination between classes.
Movie Summarization via Sparse Graph Construction
We summarize full-length movies by creating shorter videos containing their most informative scenes.
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads
Here, we propose a novel fully unsupervised parsing approach that extracts constituency trees from PLM attention heads.
Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation
Suspense is a crucial ingredient of narrative fiction, engaging readers and making stories compelling.
Screenplay Summarization Using Latent Narrative Structure
Most general-purpose extractive summarization models are trained on news articles, which are short and present all important information upfront.
Movie Plot Analysis via Turning Point Identification
According to screenwriting theory, turning points (e. g., change of plans, major setback, climax) are crucial narrative moments within a screenplay: they define the plot structure, determine its progression and segment the screenplay into thematic units (e. g., setup, complications, aftermath).
An Imitation Learning Approach to Unsupervised Parsing
In our work, we propose an imitation learning approach to unsupervised parsing, where we transfer the syntactic knowledge induced by the PRPN to a Tree-LSTM model with discrete parsing actions.
Cross-lingual Visual Verb Sense Disambiguation
We extend this line of work to the more challenging task of cross-lingual verb sense disambiguation, introducing the MultiSense dataset of 9, 504 images annotated with English, German, and Spanish verbs.
Character-based Surprisal as a Model of Reading Difficulty in the Presence of Error
Also, transpositions are more difficult than misspellings, and a high error rate increases difficulty for all words, including correct ones.
Dependency Grammar Induction with a Neural Variational Transition-based Parser
Transition-based models enable faster inference with $O(n)$ time complexity, but their performance still lags behind.
Modeling Task Effects in Human Reading with Neural Attention
We propose a neural architecture that combines an attention module (deciding whether to skip words) and a task module (memorizing the input).
An Evaluation of Image-Based Verb Prediction Models against Human Eye-Tracking Data
Recent research in language and vision has developed models for predicting and disambiguating verbs from images.
Extreme clicking for efficient object annotation
We crowd-source extreme point annotations for PASCAL VOC 2007 and 2012 and show that (1) annotation time is only 7s per box, 5x faster than the traditional way of drawing boxes [62]; (2) the quality of the boxes is as good as the original ground-truth drawn the traditional way; (3) detectors trained on our annotations are as accurate as those trained on the original ground-truth.
Image Pivoting for Learning Multilingual Multimodal Representations
In this paper we propose a model to learn multimodal multilingual representations for matching images and sentences in different languages, with the aim of advancing multilingual versions of image search and image understanding.
An Analysis of Action Recognition Datasets for Language and Vision Tasks
A large amount of recent research has focused on tasks that combine language and vision, resulting in a proliferation of datasets and methods.
Training object class detectors with click supervision
Training object class detectors typically requires a large set of images with objects annotated by bounding boxes.
Multiple Instance Learning
Weakly-Supervised Object Localization
Cross-lingual Transfer of Correlations between Parts of Speech and Gaze Features
Several recent studies have shown that eye movements during reading provide information about grammatical and syntactic processing, which can assist the induction of NLP models.
Modeling Human Reading with Neural Attention
When humans read text, they fixate some words and skip others.
Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings
We introduce a new task, visual sense disambiguation for verbs: given an image and a verb, assign the correct sense of the verb, i. e., the one that describes the action depicted in the image.
We don't need no bounding-boxes: Training object class detectors using only human verification
Training object class detectors typically requires a large set of images in which objects are annotated by bounding-boxes.
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities.
Incremental Tree Substitution Grammar for Parsing and Sentence Prediction
In this paper, we present the first incremental parser for Tree Substitution Grammar (TSG).
