Home ➤ Publications ➤ peiris2023shade

SHADE: Semantic Hypernym Annotator for Domain-Specific Entities-Dungeons and Dragons Domain Use Case

Akila Peiris, Nisansa de Silva
2023 IEEE 17th International Conference on Industrial and Information Systems (ICIIS)

Manual data annotation is an important NLP task but one that takes a considerable amount of resources and effort. In spite of the costs, labelling and categorizing entities are essential for NLP tasks such as semantic evaluation. Even though annotation can be done by non-experts in most cases, due to the fact that this requires human labour, the process is costly. Another major challenge encountered in data annotation is maintaining annotation consistency. Annotation efforts are typically carried out by teams of multiple annotators. The annotations need to maintain consistency in relation to both the domain truth and annotation format while reducing human errors. Annotating a specialized domain that deviates significantly from the general domain, such as fantasy literature, will see a significant amount of human error and annotator disagreement. So it is vital that proper guidelines and error reduction mechanisms are enforced. One way to enforce these constraints is by using a specialized application. Such an app can ensure that the notations are consistent, and the labels can be pre-defined or restricted reducing the room for errors. In this paper, we present SHADE, an annotation software that can be used to annotate entities in the high fantasy literature domain Dungeons and Dragons extracted from the Forgotten Realms Fandom Wiki.

Keywords: Natural Language Processing | DnD | Data Annotation | Information Extraction |

@inproceedings{peiris2023shade,
  title={{SHADE: Semantic Hypernym Annotator for Domain-Specific Entities-Dungeons and Dragons Domain Use Case}},
  author={Peiris, Akila and de Silva, Nisansa},
  booktitle={2023 IEEE 17th International Conference on Industrial and Information Systems (ICIIS)},
  pages={1--6},
  year={2023},
  organization={IEEE},
  doi={10.1109/ICIIS58898.2023.10253606},
  abstract={Manual data annotation is an important NLP task but one that takes a considerable amount of resources and effort. In spite of the costs, labelling and categorizing entities are essential for NLP tasks such as semantic evaluation. Even though annotation can be done by non-experts in most cases, due to the fact that this requires human labour, the process is costly. Another major challenge encountered in data annotation is maintaining annotation consistency. Annotation efforts are typically carried out by teams of multiple annotators. The annotations need to maintain consistency in relation to both the domain truth and annotation format while reducing human errors. Annotating a specialized domain that deviates significantly from the general domain, such as fantasy literature, will see a significant amount of human error and annotator disagreement. So it is vital that proper guidelines and error reduction mechanisms are enforced. One way to enforce these constraints is by using a specialized application. Such an app can ensure that the notations are consistent, and the labels can be pre-defined or restricted reducing the room for errors. In this paper, we present SHADE, an annotation software that can be used to annotate entities in the high fantasy literature domain Dungeons and Dragons extracted from the Forgotten Realms Fandom Wiki.},
  misc={https://ieeexplore.ieee.org/document/10253606,NLP:DND,https://goo.gl/iY6aTr}
}



Acknowledgement

This work was partially supported by the following grants:

2022
10
-
2023
09
Data Sets and Benchmark for Interactive Collaborative Storytelling Text Generation
Rs 300,000 - SRC/ST/2022/01
We propose to create annotated data sets for the purpose of establishing a benchmark for interactive collaborative storytelling text generation. (Specifically in the TTRPG Dungeons & Dragons).


Related Projects

NLP for D&D

Active

PI: Nisansa de Silva

This project attempts to generate adventures, images, and encounters for Dungeons and Dragons, the world's most popular Table Top Role Playing Game.