Search Results for author:
Found 12 papers, 8 papers with code
BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks
We use an efficient data curation process to ensure our data is high-quality and license-permissive.
WebQuest: A Benchmark for Multimodal QA on Web Page Sequences
In contrast to existing UI benchmarks that focus on multi-step web navigation and task completion, our dataset evaluates information extraction, multimodal retrieval and composition of information from many web pages.
ScreenAI: A Vision-Language Model for UI and Infographics Understanding
At the heart of this mixture is a novel screen annotation task in which the model has to identify the type and location of UI elements.
Ranked #3 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
Towards Better Semantic Understanding of Mobile Interfaces
Improving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users.
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
We introduce ScreenQA, a novel benchmarking dataset designed to advance screen content understanding through question answering.
A Unified Approach to Entity-Centric Context Tracking in Social Conversations
Second, we open-source a neural network architecture for context tracking.
UIBert: Learning Generic Multimodal Representations for UI Understanding
Our key intuition is that the heterogeneous features in a UI are self-aligned, i. e., the image and text features of UI components, are predictive of each other.
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces
Our methodology is designed to leverage visual, linguistic and domain-specific features in user interaction traces to pre-train generic feature representations of UIs and their components.
MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines
We also benchmark a few state of the art dialogue state tracking models on the corrected dataset to facilitate comparison for future work.
Schema-Guided Dialogue State Tracking Task at DSTC8
The goal of this task is to develop dialogue state tracking models suitable for large-scale virtual assistants, with a focus on data-efficient joint modeling across domains and zero-shot generalization to new APIs.
The Eighth Dialog System Technology Challenge
This paper introduces the Eighth Dialog System Technology Challenge.
Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset
In this work, we introduce the the Schema-Guided Dialogue (SGD) dataset, containing over 16k multi-domain conversations spanning 16 domains.
