| 1st | 2nd | 3rd | 4th | 5th | 6th | Final | |
|---|---|---|---|---|---|---|---|
| Presentation | 27.04. | 11.05. | 25.05. | 08.06. | 22.06. | 06.07. | 20.07. |
- @ge39dor (ge39dor@mytum.de)
- @ga58nav (florian.angermeir@tum.de)
The final report can be found here.
- Apply the given classifier method for attribute and entity classification on SemEval 2016 task 5 and the organic dataset.
- The idea for SemEval is to show that the classifier is actually working for the task of attribute/entity classification, since the dataset is well defined. Additionally, it would be interesting to analyze the reasoning of the classification to see which are coherent and which are incoherent attribute/entity classes.
- If the classifier can be used for the task, we want to investigate the organic dataset. The problem here is that the annotations are not reliable, i.e. some annotators have different annotation standards than others. Please use the techniques from the paper to explain
- which annotators decrease model performance (see the "Annotator" column from the annotated organic dataset),
- which are the coherently or reliably annotated classes and which are the other classes.
- The main research question to be solved here is how the SS3 classification model can be utilized for our denoted use case. Come up with own ideas in that regard! One option would be to generate the illustrations from figure 6 from the SS3 paper by iteratively adding annotations from one annotators after the other to see who decreases and who increases the confidence values. There will be many other options as well!