Search Results for author:
Found 52 papers, 32 papers with code
Verifying Cross-modal Entity Consistency in News using Vision-language Models
The identification of inconsistent cross-modal information, in particular entities such as persons, locations, and events, is critical to detect disinformation.
Unraveling the Impact of Visual Complexity on Search as Learning
The reported results shed light on the impact of visual complexity on learning-oriented searches, informing the design of more effective IR systems for educational contexts.
Saliency Detection in Educational Videos: Analyzing the Performance of Current Models, Identifying Limitations and Advancement Directions
In this paper, we address this gap by evaluating four state-of-the-art saliency detection approaches for educational videos.
Multimodal Misinformation Detection using Large Vision-Language Models
Few approaches consider evidence retrieval as part of the misinformation detection but rely on fine-tuning models.
On the Influence of Reading Sequences on Knowledge Gain during Web Search
We use publicly available study data from a web-based learning task to examine the relationship between our feature set and the participants' test scores.
Classification of Visualization Types and Perspectives in Patents
In this paper, we adopt state-of-the-art deep learning methods for the classification of visualization types and perspectives in patent images.
Improving Generalization for Multimodal Fake News Detection
To reduce the bias and improve model generalization, we suggest training data augmentation to conduct more meaningful experiments for fake news detection on social media.
Predicting Knowledge Gain for MOOC Video Consumption
Informal learning on the Web using search engines as well as more structured learning on MOOC platforms have become very popular in recent years.
MM-Locate-News: Multimodal Focus Location Estimation in News
In this paper, a novel dataset called Multimodal Focus Location of News (MM-Locate-News) is introduced.
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
TVCalib: Camera Calibration for Sports Field Registration in Soccer
Sports field registration in broadcast videos is typically interpreted as the task of homography estimation, which provides a mapping between a planar field and the corresponding visible area of the image.
Semi-supervised Human Pose Estimation in Art-historical Images
In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection.
Ranked #1 on
Semi-Supervised Human Pose Estimation
on PoPArt
MM-Claims: A Dataset for Multimodal Claim Detection in Social Media
In recent years, the problem of misinformation on the web has become widespread across languages, countries, and various social media platforms.
TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes
The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis.
SaL-Lightning Dataset: Search and Eye Gaze Behavior, Resource Interactions and Knowledge Gain during Web Search
The emerging research field Search as Learning investigates how the Web facilitates learning through modern information retrieval systems.
Combining Textual Features for the Detection of Hateful and Offensive Language
The detection of offensive, hateful and profane language has become a critical challenge since many users in social networks are exposed to cyberbullying activities on a daily basis.
Extraction of Positional Player Data from Broadcast Soccer Videos
A comprehensive experimental evaluation is presented for the individual modules as well as the entire pipeline to identify the influence of errors to subsequent modules and the overall result.
A Unified Taxonomy and Multimodal Dataset for Events in Invasion Games
To address these issues, we present (1) a universal taxonomy that covers a wide range of low and high-level events for invasion games and is exemplarily refined to soccer and handball, and (2) release two multi-modal datasets comprising video and positional data with gold-standard annotations to foster research in fine-grained and ball-centered event spotting.
iART: A Search Engine for Art-Historical Images to Support Research in the Humanities
In this paper, we introduce iART: an open Web platform for art-historical research that facilitates the process of comparative vision.
Evaluation of Automated Image Descriptions for Visually Impaired Students
Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students.
Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention
The recognition of handwritten mathematical expressions in images and video frames is a difficult and unsolved problem yet.
A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods
Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts.
Predicting Knowledge Gain during Web Search based on Multimedia Resource Consumption
Related work in this field, also called search as learning, has focused on behavioral or text resource features to predict learning outcome and knowledge gain.
Citation Recommendation for Research Papers via Knowledge Graphs
Citation recommendation for research papers is a valuable task that can help researchers improve the quality of their work by suggesting relevant related work.
Unsupervised Video Summarization via Multi-source Features
Our evaluation shows that we obtain state-of-the-art results on both datasets, while also highlighting the shortcomings of previous work with regard to the evaluation methodology.
Interpretable Semantic Photo Geolocation
State-of-the-art methods treat the task as a classification problem, where the choice of the classes, that is the partitioning of the world map, is crucial for the performance.
GeoWINE: Geolocation based Wiki, Image,News and Event Retrieval
The first module is a state-of-the-art model for geolocation estimation of images.
QuTI! Quantifying Text-Image Consistency in Multimodal Documents
The World Wide Web and social media platforms have become popular sources for news and information.
Supervised Video Summarization via Multiple Feature Sets with Parallel Attention
The proposed architecture utilizes an attention mechanism before fusing motion features and features representing the (static) visual content, i. e., derived from an image classification model.
Ranked #1 on
Supervised Video Summarization
on TvSum
On the Role of Images for Analyzing Claims in Social Media
Fake news is a severe problem in social media.
Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers
Our approach outperforms the state of the art on full paper datasets significantly while being on par for datasets consisting of abstracts.
Analysing the Requirements for an Open Research Knowledge Graph: Use Cases, Quality Requirements and Construction Strategies
Current science communication has a number of drawbacks and bottlenecks which have been subject of discussion lately: Among others, the rising number of published articles makes it nearly impossible to get a full overview of the state of the art in a certain field, or reproducibility is hampered by fixed-length, document-based publications which normally cannot cover all details of a research work.
Coreference Resolution in Research Papers from Multiple Domains
We present the following contributions: (1) We annotate a corpus for coreference resolution that comprises 10 different scientific disciplines from Science, Technology, and Medicine (STM); (2) We propose transfer learning for automatic coreference resolution in research papers; (3) We analyse the impact of coreference resolution on knowledge graph (KG) population; (4) We release a research KG that is automatically populated from 55, 485 papers in 10 STM domains.
Ranked #1 on
Coreference Resolution
on STM-coref
(using extra training data)
Ontology-driven Event Type Classification in Images
Event classification can add valuable information for semantic search and the increasingly important topic of fact validation in news.
Classification of Important Segments in Educational Videos using Multimodal Features
Our experiments investigate the impact of visual and temporal information, as well as the combination of multimodal features on importance prediction.
Monocular Depth Estimation via Listwise Ranking using the Plackett-Luce Model
In many real-world applications, the relative depth of objects in an image is crucial for scene understanding.
MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities
We demonstrate the value of the resource in developing novel applications in the digital humanities with a motivating use case and specify a benchmark set of tasks to retrieve modalities and locate entities in the dataset.
Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Features
In this digital age of news consumption, a news reader has the ability to react, express and share opinions with others in a highly interactive and fast manner.
A Feature Analysis for Multimodal News Retrieval
Content-based information retrieval is based on the information contained in documents rather than using metadata such as keywords.
Requirements Analysis for an Open Research Knowledge Graph
Current science communication has a number of drawbacks and bottlenecks which have been subject of discussion lately: Among others, the rising number of published articles makes it nearly impossible to get an overview of the state of the art in a certain field, or reproducibility is hampered by fixed-length, document-based publications which normally cannot cover all details of a research work.
Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency
In this paper, we introduce a novel task of cross-modal consistency verification in real-world news and present a multimodal approach to quantify the entity coherence between image and text.
The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources
We introduce the STEM (Science, Technology, Engineering, and Medicine) Dataset for Scientific Entity Extraction, Classification, and Resolution, version 1. 0 (STEM-ECR v1. 0).
SlideImages: A Dataset for Educational Image Classification
We have reserved all the actual educational images as a test dataset in order to ensure that the approaches using this dataset generalize well to new educational images, and potentially other domains.
Domain-independent Extraction of Scientific Concepts from Research Articles
We examine the novel task of domain-independent scientific concept extraction from abstracts of scholarly articles and present two contributions.
Ranked #1 on
Scientific Concept Extraction
on STM-corpus
Visual Summarization of Scholarly Videos using Word Embeddings and Keyphrase Extraction
Effective learning with audiovisual content depends on many factors.
"Does 4-4-2 exist?" -- An Analytics Approach to Understand and Classify Football Team Formations in Single Match Situations
For this purpose, a visual summary is utilized that summarizes the team formation in a match segment.
Investigating Correlations of Inter-coder Agreement and Machine Annotation Performance for Historical Video Data
Video indexing approaches such as visual concept classification and person recognition are essential to enable fine-grained semantic search in large-scale video archives such as the historical video collection of former German Democratic Republic (GDR) maintained by the German Broadcasting Archive (DRA).
Understanding, Categorizing and Predicting Semantic Image-Text Relations
Two modalities are often used to convey information in a complementary and beneficial manner, e. g., in online news, videos, educational resources, or scientific publications.
"Is this an example image?" -- Predicting the Relative Abstractness Level of Image and Text
In this paper, we present an approach to predict the (cross-modal) relative abstractness level of a given image-text pair, that is whether the image is an abstraction of the text or vice versa.
Geolocation Estimation of Photos using a Hierarchical Model and Scene Classification
While the successful estimation of a photo's geolocation enables a number of interesting applications, it is also a very challenging task.
Ranked #1 on
Photo geolocation estimation
on Im2GPS
(Street level (1 km) metric)
Finding Person Relations in Image Data of the Internet Archive
In this paper, we introduce a system for person recognition in image content of web news stored in the Internet Archive.
Recommending Scientific Videos based on Metadata Enrichment using Linked Open Data
The amount of available videos in the Web has significantly increased not only for entertainment etc., but also to convey educational or scientific information in an effective way.
Optical Character Recognition
Optical Character Recognition (OCR)
+2
