Website Summarization Using Deep Learning

Guda Pranay Netha; M.S.S Manohar; M. Sai Amartya Maruth; Ganjikunta Ganesh Kumar

Abstract

They are a lot of websites, articles and blogs on the internet with a lot of textual information it is sometimes impossible to read all of them, without knowing the underlying summary of the whole text. The whole concept is to reduce or minimize the entire Textual information into a small summary of important information present in the documents so that we get an idea of what the blog or article exactly consists of and we can decide whether to read it or not. They have been several methods implemented in the past to summarize the vast textual information such as Machine Learning Algorithms which were mostly centric on the frequency of words in the sentences and made a decision based on the frequency of words, while some machine learning models worked fine with extractive text summarization they failed in abstractive text summarization the results produced by the model was illogical and dint followed the basic grammar rules, this issue was overcome by using Deep Learning, although in the past deep learning models such as recurrent neural networks, Long Short Term Memory Neural Network, and Encoder-Decoder Architecture were powerful yet they were able to summarize the only short amount of text and failed extremely on a large set of documents. In this paper, we will see advancements in Natural language processing using deep learning and learn about the latest researched Transformers and Attention mechanisms and create a python application that can summarize a website given its URL and verify the fluency and correctness of the text generated by the application.

Abstract

Keywords