Text Infilling: Missing Text Generation

Text infilling or missing text generation (Huang et al., 2019; Zhu et al., 2019) is the NLP task of predicting/ generating spans of missing text in a larger text segment when the preceding and subsequent text of the missing span is given. Text infilling can be considered a generalization (Donahue et al., 2020) of a cloze task (Taylor, 1953) and as such requires contextual understanding which makes this NLP task a complex one. Text infilling has many practical applications including mixed initiative story generation (Ippolito et al., 2019), text revision (Shih et al., 2019), and restoring ancient documents that are missing content (Assael et al., 2019). With the recent paradigm shift of using pre-trained language models bringing NLP to new heights (Qiu et al., 2020), the text infilling area has seen a rise (Ou et al., 2021; Huang et al., 2019; Donahue et al., 2020). Test infilling research is gaining popularity in recent times, especially with pre-trained large language models. We plan on finetuning a language model that has been previously fine-tuned using data extracted from the Forgotten Realms Fandom wiki to perform text infilling tasks. This will ultimately map with creating a story/ free text generator based in the Dungeons and Dragons domain.