Enhancing Language Modelling with RNN and LSTMbased Next Word Prediction

Prakhar  Mathur; Khushi  Sharma; Shubhankar  Kavya; Aman  Sharma; Shazia  Haque

Abstract

A growing field of study, natural language processing is utilised in many different contexts. We frequently text each other, and we have noticed that whenever we type a message, a recommendation pops up to try to guess what word we will write next. The task of anticipating the next word is known as next word prediction or language modelling. This makes them highly effective for tasks such as language modelling, speech recognition, sentiment analysis, and machine translation. In this study, we will provide a comprehensive overview of RNNs. Additionally, we will explore the limitations and difficulties of RNNs, as well as recent advances such as attention mechanisms that have enhanced their performance. The process for language modelling on a collection of texts now uses artificial neural networks. Recurrent neural networks (RNNs) can use all of the words that have come before to predict the next word, unlike feed-forward networks that can only use a predetermined context length of words.

Abstract

Keywords