Search Results for author: Xiaoxue Zang

Found 9 papers, 3 papers with code

UIBert: Learning Generic Multimodal Representations for UI Understanding

no code implementations29 Jul 2021 Chongyang Bai, Xiaoxue Zang, Ying Xu, Srinivas Sunkara, Abhinav Rastogi, Jindong Chen, Blaise Aguera y Arcas

Our key intuition is that the heterogeneous features in a UI are self-aligned, i. e., the image and text features of UI components, are predictive of each other.

Multimodal Icon Annotation For Mobile Applications

no code implementations9 Jul 2021 Xiaoxue Zang, Ying Xu, Jindong Chen

Annotating user interfaces (UIs) that involves localization and classification of meaningful UI elements on a screen is a critical step for many mobile applications such as screen readers and voice control of devices.

Object Classification Object Detection

PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling

no code implementations ACL 2021 Xiaoxue Zang, Lijuan Liu, Maria Wang, Yang song, Hao Zhang, Jindong Chen

Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context.

Image Retrieval

ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces

no code implementations22 Dec 2020 Zecheng He, Srinivas Sunkara, Xiaoxue Zang, Ying Xu, Lijuan Liu, Nevan Wichers, Gabriel Schubiner, Ruby Lee, Jindong Chen, Blaise Agüera y Arcas

Our methodology is designed to leverage visual, linguistic and domain-specific features in user interaction traces to pre-train generic feature representations of UIs and their components.

Schema-Guided Dialogue State Tracking Task at DSTC8

1 code implementation2 Feb 2020 Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, Pranav Khaitan

The goal of this task is to develop dialogue state tracking models suitable for large-scale virtual assistants, with a focus on data-efficient joint modeling across domains and zero-shot generalization to new APIs.

Data Augmentation Dialogue State Tracking

Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset

1 code implementation12 Sep 2019 Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, Pranav Khaitan

In this work, we introduce the the Schema-Guided Dialogue (SGD) dataset, containing over 16k multi-domain conversations spanning 16 domains.

Dialogue State Tracking Language understanding +1

Cannot find the paper you are looking for? You can Submit a new open access paper.