InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis

16 Feb 2023  ยท  Kevin Scaria, Himanshu Gupta, Siddharth Goyal, Saurabh Arjun Sawant, Swaroop Mishra, Chitta Baral ยท

We introduce InstructABSA, an instruction learning paradigm for Aspect-Based Sentiment Analysis (ABSA) subtasks. Our method introduces positive, negative, and neutral examples to each training sample, and instruction tune the model (Tk-Instruct) for ABSA subtasks, yielding significant performance improvements. Experimental results on the Sem Eval 2014, 15, and 16 datasets demonstrate that InstructABSA outperforms the previous state-of-the-art (SOTA) approaches on Term Extraction (ATE), Sentiment Classification(ATSC) and Sentiment Pair Extraction (ASPE) subtasks. In particular, InstructABSA outperforms the previous state-of-the-art (SOTA) on the Rest14 ATE subtask by 5.69% points, the Rest15 ATSC subtask by 9.59% points, and the Lapt14 AOPE subtask by 3.37% points, surpassing 7x larger models. We also get competitive results on AOOE, AOPE, and AOSTE subtasks indicating strong generalization ability to all subtasks. Exploring sample efficiency reveals that just 50% train data is required to get competitive results with other instruction tuning approaches. Lastly, we assess the quality of instructions and observe that InstructABSA's performance experiences a decline of ~10% when adding misleading examples.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Aspect-Based Sentiment Analysis (ABSA) SemEval-2014 Task-4 InstructABSA Restaurant (Acc) 89.76 # 3
Laptop (Acc) 88.37 # 1
Mean Acc (Restaurant + Laptop) 89.065 # 1
Aspect Extraction SemEval-2014 Task-4 InstructABSA Laptop (F1) 92.30 # 1
Mean F1 (Laptop + Restaurant) 92.53 # 1
Restaurant (F1) 92.76 # 1
Aspect-Based Sentiment Analysis (ABSA) SemEval 2014 Task 4 Laptop InstructABSA F1 79.34 # 1
Aspect Extraction SemEval 2014 Task 4 Sub Task 1 InstructABSA Laptop (F1) 92.30 # 1
Aspect-Based Sentiment Analysis (ABSA) SemEval 2014 Task 4 Sub Task 1 InstructABSA Laptop (F1) 92.30 # 1
Restaurant (F1) 92.76 # 1
Sentiment Analysis SemEval 2014 Task 4 Subtask 1+2 InstructABSA F1 79.34 # 1
Aspect-Based Sentiment Analysis (ABSA) SemEval 2014 Task 4 Subtask 1+2 InstructABSA F1 79.34 # 2

Methods


No methods listed for this paper. Add relevant methods here