Introduction Natural Language Processing is a field widely growing in popularity these days. A large number of companies worldwide are leveraging the power of Natural language processing and the innovations in the field to extract meaningful insights from text and to generate text. In the past couple of years, with Google Brain’s Attention is All You Need paper, the transformers architecture has revolutionized this field even further. In almost all the classic NLP tasks like Machine translation, Question Answering, Reading Comprehension, Common Sense Reasoning and Summarization, a Transformer-based architecture beat the State-of-the-art. Since then, all tech giants like Google, Facebook, OpenAI, Microsoft have been experimenting with transformers in various applications. One such application that made headlines was the Language Generation task, wherein Transformers were able to generate meaningful text given a prompt. One of the first headliners was HuggingFace with their Talk to Transformers web page, where anyone could generate their own AI-generated text by giving a prompt. Here, we will explore how transformers are used in Language generation. Also later in the blog, we will share code for how to train a transformer langauge model on your own corpus. We trained a GPT-2 model on Harry Potter books. The … Continue reading Train a GPT-2 Transformer to write Harry Potter Books!