Skip to content

This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Notifications You must be signed in to change notification settings

zhijing-jin/ARTS_TestSet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is the repository for our paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis" (EMNLP 2020).

Authors: Xiaoyu Xing*, Zhijing Jin*, Di Jin, Bingning Wang, Qi Zhang, and Xuanjing Huang.

Data

We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).

Data Generation Process

We generate our new probing test set by three automatic strategies:

  • RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
  • RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
  • AddDiff (sentence with a blue background): Add new aspects with Different sentiment.

method_illustration.png

Aspect Probing Results

We probed nine ABSA models (as mentioned in our paper).

  • Their outputs on SemEval 2014 are in the output folder.

How to Use Our Code

If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.

python code/main.py -dataset_name laptop

Dependencies

All Trained Models

If needed, see a dump of all the trained models and output files here.

More Questions

If you have more questions, please feel free to submit a GitHub issue.

About

This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages