CoVA: Context-aware Visual Attention for Webpage Information Extraction

Webpage information extraction (WIE) is an important step to create knowledge bases. For this, classical WIE methods leverage the Document Object Model (DOM) tree of a website. However, use of the DOM tree poses significant challenges as context and appearance are encoded in an abstract manner. To address this challenge we propose to reformulate WIE as a context-aware Webpage Object Detection task. Specifically, we develop a Context-aware Visual Attention-based (CoVA) detection pipeline which combines appearance features with syntactical structure from the DOM tree. To study the approach we collect a new large-scale dataset of e-commerce websites for which we manually annotate every web element with four labels: product price, product title, product image and background. On this dataset we show that the proposed CoVA approach is a new challenging baseline which improves upon prior state-of-the-art methods.

PDF Abstract ECNLP (ACL) 2022 PDF ECNLP (ACL) 2022 Abstract

Datasets


Introduced in the Paper:

CoVA

Results from the Paper


 Ranked #1 on Webpage Object Detection on CoVA (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Webpage Object Detection CoVA CoVA++ Cross Domain Price Accuracy 96.1 # 1
Cross Domain Title Accuracy 96.7 # 1
Cross Domain Image Accuracy 99.6 # 1
Webpage Object Detection CoVA CoVA Cross Domain Price Accuracy 95.5 # 2
Cross Domain Title Accuracy 95.7 # 2
Cross Domain Image Accuracy 98.8 # 2

Methods


Convolution • CoVA • Fast R-CNN • GAT • RoIPool • Softmax