Search Results for author:
Found 101 papers, 26 papers with code
MTEQA at WMT21 Metrics Shared Task
In this paper, we describe our submission to the WMT 2021 Metrics Shared Task.
KUL@SMM4H’22: Template Augmented Adaptive Pre-training for Tweet Classification
This paper describes models developed for the Social Media Mining for Health (SMM4H) 2022 shared tasks.
Online Continual Learning from Imbalanced Data
Motivated by this remark, we aim to evaluate memory population methods that are used in online continual learning, when dealing with highly imbalanced and temporally correlated streams of data.
Just Ask! Evaluating Machine Translation by Asking and Answering Questions
In this paper, we show that automatically-generated questions and answers can be used to evaluate the quality of Machine Translation (MT) systems.
Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities
The second phase tunes the feature learner to attend to neural activation patterns most informative for visual reconstruction with guidance from an image auto-encoder.
Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps
In this work, we conduct a systematic study of exposure bias in diffusion models and, intriguingly, we find that the exposure bias could be alleviated with a new sampling method, without retraining the model.
A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information
Existing question answering methods often assume that the input content (e. g., documents or videos) is always accessible to solve the task.
An Information Extraction Study: Take In Mind the Tokenization!
Current research on the advantages and trade-offs of using characters, instead of tokenized text, as input for deep learning models, has evolved substantially.
Generating multiple-choice questions for medical question answering with distractors and cue-masking
Medical multiple-choice question answering (MCQA) is particularly difficult.
Implicit Temporal Reasoning for Evidence-Based Fact-Checking
Our findings show that the presence of temporal information and the manner in which timelines are constructed greatly influence how fact-checking models determine the relevance and supporting or refuting character of evidence documents.
Optimizing ship detection efficiency in SAR images
The integration of the optimizations with the object detection model leads to a trade-off between speed and performance.
Layout-aware Dreamer for Embodied Referring Expression Grounding
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction.
Bidirectional Representations for Low Resource Spoken Language Understanding
Class attention can be used to visually explain the predictions of our model, which goes a long way in understanding how the model makes predictions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multitask Learning for Low Resource Spoken Language Understanding
We explore the benefits that multitask learning offer to speech processing as we train models on dual objectives with automatic speech recognition and intent classification or sentiment classification.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
Probing neural language models for understanding of words of estimative probability
and assess whether language models can predict whether the WEP consensual probability level is close to p. Secondly, we construct a dataset of WEP-based probabilistic reasoning, to test whether language models can reason with WEP compositions.
Weakly Supervised Face Naming with Symmetry-Enhanced Contrastive Loss
We revisit the weakly supervised cross-modal face-name alignment task; that is, given an image and a caption, we label the faces in the image with the names occurring in the caption.
Students taught by multimodal teachers are superior action recognizers
Our approach is based on multimodal knowledge distillation, featuring a multimodal teacher (in the current experiments trained only using object detections, optical flow and RGB frames) and a unimodal student (using only RGB frames as input).
How Relevant is Selective Memory Population in Lifelong Language Learning?
Lifelong language learning seeks to have models continuously learn multiple tasks in a sequential order without suffering from catastrophic forgetting.
Anti-Overestimation Dialogue Policy Learning for Task-Completion Dialogue System
This problem is detrimental to RL-based dialogue policy learning.
Entropy-based Stability-Plasticity for Lifelong Learning
The ability to continuously learn remains elusive for deep learning models.
Evaluation Benchmarks for Spanish Sentence Representations
Due to the success of pre-trained language models, versions of languages other than English have been released in recent years.
Finding Structural Knowledge in Multimodal-BERT
In this work, we investigate the knowledge learned in the embeddings of multimodal-BERT models.
Find a Way Forward: a Language-Guided Semantic Map Navigator
In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map.
Modeling Coreference Relations in Visual Dialog
Visual dialog is a vision-language task where an agent needs to answer a series of questions grounded in an image based on the understanding of the dialog history and the image.
Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (VQA) is a vision-language task that requires an agent to correctly answer image-related questions using knowledge that is not presented in the given image.
Discrete and continuous representations and processing in deep learning: Looking forward
Motivated by these insights, in this paper we argue that combining discrete and continuous representations and their processing will be essential to build systems that exhibit a general form of intelligence.
Two-phase training mitigates class imbalance for camera trap image classification with CNNs
As a result, they obtain good performance for a few majority classes but poor performance for many minority classes.
Analysis and Prediction of NLP Models Via Task Embeddings
Task embeddings are low-dimensional representations that are trained to capture task properties.
Predicting Physical World Destinations for Commands Given to Self-Driving Cars
However, surveys have shown that giving more control to an AI in self-driving cars is accompanied by a degree of uneasiness by passengers.
Revisiting spatio-temporal layouts for compositional action recognition
Recognizing human actions is fundamentally a spatio-temporal reasoning problem, and should be, at least to some extent, invariant to the appearance of the human and the objects involved.
Ranked #32 on
Action Classification
on Charades
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations
Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level.
Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning
Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is limited.
Like Article, Like Audience: Enforcing Multimodal Correlations for Disinformation Detection
The latent representations of news articles and user-generated content allow that during training the model is guided by the profile of users who prefer content similar to the news article that is evaluated, and this effect is reinforced if that content is shared among different users.
Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win
The lottery ticket hypothesis states that sparse subnetworks exist in randomly initialized dense networks that can be trained to the same accuracy as the dense network they reside in.
Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?
This paper proposes a model that detects uncertain situations when a command is given and finds the visual objects causing it.
LIIR at SemEval-2021 task 6: Detection of Persuasion Techniques In Texts and Images using CLIP features
We describe our approach for SemEval-2021 task 6 on detection of persuasion techniques in multimodal content (memes).
Autoregressive Reasoning over Chains of Facts with Transformers
This paper proposes an iterative inference algorithm for multi-hop explanation regeneration, that retrieves relevant factual evidence in the form of text snippets, given a natural language question and its answer.
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks
Understating spatial semantics expressed in natural language can become highly complex in real-world applications.
Decoding Language Spatial Relations to 2D Spatial Arrangements
We address the problem of multimodal spatial understanding by decoding a set of language-expressed spatial relations to a set of 2D spatial arrangements in a multi-object and multi-relationship setting.
Towards Extracting Absolute Event Timelines From English Clinical Reports
Temporal information extraction is a challenging but important area of automatic natural language understanding.
Natural Language Understanding
Temporal Information Extraction
Are scene graphs good enough to improve Image Captioning?
Overall, we find no significant difference between models that use scene graph features and models that only use object detection features across different captioning metrics, which suggests that existing scene graph generation models are still too noisy to be useful in image captioning.
Commands 4 Autonomous Vehicles (C4AV) Workshop Summary
In this work, we deviate from recent, popular task settings and consider the problem under an autonomous vehicle scenario.
Ranked #2 on
Referring Expression Comprehension
on Talk2Car
Checkworthiness in Automatic Claim Detection Models: Definitions and Analysis of Datasets
Public, professional and academic interest in automated fact-checking has drastically increased over the past decade, with many aiming to automate one of the first steps in a fact-check procedure: the selection of so-called checkworthy claims.
Structured (De)composable Representations Trained with Neural Networks
The paper proposes a novel technique for representing templates and instances of concept classes.
Convolutional Generation of Textured 3D Meshes
A key contribution of our work is the encoding of the mesh and texture as 2D representations, which are semantically aligned and can be easily modeled by a 2D convolutional GAN.
A Survey on Temporal Reasoning for Temporal Information Extraction from Text (Extended Abstract)
Time is deeply woven into how people perceive, and communicate about the world.
Natural Language Understanding
Temporal Information Extraction
LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification
We adapt and fine-tune the BERT and Multilingual Bert models made available by Google AI for English and non-English languages respectively.
Giving Commands to a Self-driving Car: A Multimodal Reasoner for Visual Grounding
We propose a new spatial memory module and a spatial reasoner for the Visual Grounding (VG) task.
Ranked #9 on
Referring Expression Comprehension
on Talk2Car
Binary and Multitask Classification Model for Dutch Anaphora Resolution: Die/Dat Prediction
Drawing on previous research conducted on neural context-dependent dt-mistake correction models (Heyman et al. 2018), this study constructs the first neural network model for Dutch demonstrative and relative pronoun resolution that specifically focuses on the correction and part-of-speech prediction of these two pronouns.
User Profiling Using Hinge-loss Markov Random Fields
In this paper, we propose a mechanism to infer a variety of user characteristics, such as, age, gender and personality traits, which can then be compiled into a user profile.
Distance-based Composable Representations with Neural Networks
We introduce a new deep learning technique that builds individual and class representations based on distance estimates to randomly generated contextual dimensions for different modalities.
Talk2Car: Taking Control of Your Self-Driving Car
Or more specifically, we consider the problem in an autonomous driving setting, where a passenger requests an action that can be associated with an object found in a street scene.
Attention-based Fusion for Outfit Recommendation
This paper describes an attention-based fusion method for outfit recommendation which fuses the information in the product image and description to capture the most important, fine-grained product features into the item representation.
Justifying Diagnosis Decisions by Deep Neural Networks
This textual representation is decoded into a diagnosis and the associated textual justification that will help a clinician evaluate the outcome.
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs
We learn a shared multilingual embedding space for a variable number of languages by incrementally adding new languages one by one to the current multilingual space.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+4
Evaluating Textual Representations through Image Generation
The method is illustrated on a medical dataset where the correct representation of spatial information and shorthands are of particular importance.
Temporal Information Extraction by Predicting Relative Time-lines
The current leading paradigm for temporal information extraction from text consists of three phases: (1) recognition of events and temporal expressions, (2) recognition of temporal relations among them, and (3) time-line construction from the temporal relations.
Word-Level Loss Extensions for Neural Temporal Relation Classification
In this work, we extend our classification model's task loss with an unsupervised auxiliary loss on the word-embedding level of the model.
A Flexible and Easy-to-use Semantic Role Labeling Framework for Different Languages
This paper presents a flexible and open source framework for deep semantic role labeling.
Do Neural Network Cross-Modal Mappings Really Bridge Modalities?
Feed-forward networks are widely used in cross-modal applications to bridge modalities by mapping distributed vectors of one modality to the other, or to a shared space.
Generating Continuous Representations of Medical Texts
During training the input to the system is a dataset of captions for medical X-Rays.
Learning Representations Specialized in Spatial Knowledge: Leveraging Language and Vision
Here, we move one step forward in this direction and learn such representations by leveraging a task consisting in predicting continuous 2D spatial arrangements of objects given object-relationship-object instances (e. g., {``}cat under chair{''}) and a simple neural network model that learns the task from annotated images.
Acquiring Common Sense Spatial Knowledge through Implicit Spatial Templates
In contrast with prior work that restricts spatial templates to explicit spatial prepositions (e. g., "glass on table"), here we extend this concept to implicit spatial language, i. e., those relationships (generally actions) for which the spatial arrangement of the objects is only implicitly implied (e. g., "man riding horse").
Fast Top-k Area Topics Extraction with Knowledge Base
We leverage a large-scale knowledge base (Wikipedia) to generate topic embeddings using neural networks and use this kind of representations to help capture the representativeness of topics for given areas.
KULeuven-LIIR at SemEval-2017 Task 12: Cross-Domain Temporal Information Extraction from Clinical Records
In this paper, we describe the system of the KULeuven-LIIR submission for Clinical TempEval 2017.
Speech-Based Visual Question Answering
This paper introduces speech-based visual question answering (VQA), the task of generating an answer given an image and a spoken question.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments
Implicit semantic role labeling (iSRL) is the task of predicting the semantic roles of a predicate that do not appear as explicit arguments, but rather regard common sense knowledge or are mentioned earlier in the discourse.
Structured Learning for Temporal Relation Extraction from Clinical Records
We propose a scalable structured learning model that jointly predicts temporal relations between events and temporal expressions (TLINKS), and the relation between these events and the document creation time (DCTR).
Temporal Information Extraction
Temporal Relation Extraction
Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations
We study the problem of bilingual lexicon induction (BLI) in a setting where some translation resources are available, but unknown translations are sought for certain, possibly domain-specific terminology.
Learning to Recognize Animals by Watching Documentaries: Using Subtitles as Weak Supervision
We investigate animal recognition models learned from wildlife video documentaries by using the weak supervision of the textual subtitles.
Learning to Predict: A Fast Re-constructive Method to Generate Multimodal Embeddings
Integrating visual and linguistic information into a single multimodal representation is an unsolved problem with wide-reaching applications to both natural language processing and computer vision.
Is an Image Worth More than a Thousand Words? On the Fine-Grain Semantic Differences between Visual and Linguistic Representations
Human concept representations are often grounded with visual information, yet some aspects of meaning cannot be visually represented or are better described with language.
Facing the most difficult case of Semantic Role Labeling: A collaboration of word embeddings and co-training
We present a successful collaboration of word embeddings and co-training to tackle in the most difficult test case of semantic role labeling: predicting out-of-domain and unseen semantic frames.
A Dataset for Multimodal Question Answering in the Cultural Heritage Domain
In this paper, we introduce the construction of a golden standard dataset that will aid research of multimodal question answering in the cultural heritage domain.
Semi-automatically Alignment of Predicates between Speech and OntoNotes data
We still lack suitable corpora of transcribed speech annotated with semantic roles that can be used for semantic role labeling (SRL), which is not the case for written data.
Deep Embedding for Spatial Role Labeling
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description.
Bilingual Distributed Word Representations from Document-Aligned Comparable Data
We propose a new model for learning bilingual word representations from non-parallel document-aligned data.
HiEve: A Corpus for Extracting Event Hierarchies from News Stories
In this work, we present HiEve, a corpus for recognizing relations of spatiotemporal containment between events.
Annotating Story Timelines as Temporal Dependency Structures
We present an approach to annotating timelines in stories where events are linked together by temporal relations into a temporal dependency tree.
