Home ➤ Publications ➤ sivaganeshan2023fine

Fine Tuning Named Entity Extraction Models for the Fantasy Domain

Aravinth Sivaganeshan, Nisansa De Silva
2023 Moratuwa Engineering Research Conference (MERCon)

Named Entity Recognition (NER) is a sequence classification Natural Language Processing task where entities are identified in the text and classified into predefined categories. It acts as a foundation for most information extraction systems. Dungeons and Dragons (D\&D) is an open-ended tabletop fantasy game with its own diverse lore. DnD entities are domain-specific and are thus unrecognizable by even the state-of-the-art off-the-shelf NER systems as the NER systems are trained on general data for pre-defined categories such as: person (PERS), location (LOC), organization (ORG), and miscellaneous (MISC). For meaningful extraction of information from fantasy text, the entities need to be classified into domain-specific entity categories as well as the models be fine-tuned on a domain-relevant corpus. This work uses available lore of monsters in the D\&Ddomain to fine-tune Trankit, which is a prolific NER framework that uses a pre-trained model for NER. Upon this training, the system acquires the ability to extract monster names from relevant domain documents under a novel NER tag. This work compares the accuracy of the monster name identification against; the zero-shot Trankit model and two FLAIR models. The fine-tuned Trankit model achieves an 87.86\% F1 score surpassing all the other considered models.

Keywords: Natural Language Processing | DnD | Named Entity Recognition | 

@inproceedings{sivaganeshan2023fine,
  title={{Fine Tuning Named Entity Extraction Models for the Fantasy Domain}},
  author={Sivaganeshan, Aravinth and De Silva, Nisansa},
  booktitle={2023 Moratuwa Engineering Research Conference (MERCon)},
  pages={346--351},
  year={2023},
  organization={IEEE},
  doi={10.1109/MERCon60487.2023.10355501},
  abstract={Named Entity Recognition (NER) is a sequence classification Natural Language Processing task where entities are identified in the text and classified into predefined categories. It acts as a foundation for most information extraction systems. Dungeons and Dragons (D\&D) is an open-ended tabletop fantasy game with its own diverse lore. DnD entities are domain-specific and are thus unrecognizable by even the state-of-the-art off-the-shelf NER systems as the NER systems are trained on general data for pre-defined categories such as: person (PERS), location (LOC), organization (ORG), and miscellaneous (MISC). For meaningful extraction of information from fantasy text, the entities need to be classified into domain-specific entity categories as well as the models be fine-tuned on a domain-relevant corpus. This work uses available lore of monsters in the D\&Ddomain to fine-tune Trankit, which is a prolific NER framework that uses a pre-trained model for NER. Upon this training, the system acquires the ability to extract monster names from relevant domain documents under a novel NER tag. This work compares the accuracy of the monster name identification against; the zero-shot Trankit model and two FLAIR models. The fine-tuned Trankit model achieves an 87.86\% F1 score surpassing all the other considered models.},
  misc={https://ieeexplore.ieee.org/abstract/document/10355501,NLP:DND,https://goo.gl/iY6aTr}
}

Artefacts Released

GitHub
DnDMonsters

Related Projects