A Novel Framework for Next Word Prediction Using Long-Short Term Memory Networks

Date

2026-01-01

Author

Deveci, Ali
Erkan, Mehmet Ali
Medeni, Ihsan Tolga
Medeni, Tunç Durmuş

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

60
views

24
downloads

Cite This

Natural Language Processing (NLP) has become a cornerstone in various fields, revolutionizing how machines interpret and process human language. Among its diverse applications, next-word prediction emerges as a highly practical and impactful example of generative AI. This research focuses on the use of Long Short-Term Memory (LSTM) models—an innovative class of Recurrent Neural Network (RNN)—for predictive text generation. LSTMs excel in capturing sequential and contextual information, making them ideal for language tasks. While transformer models dominate accuracy benchmarks, this work addresses the critical need for efficient alternatives in resource-constrained deployment scenarios. This study presents a novel LSTM-based framework enhanced with hybrid architecture and advanced regularization techniques, trained on a carefully curated dataset of 15,000 English sentences. The proposed model achieves superior performance with 84.2% training accuracy, 79.6% test accuracy, and a perplexity score of 2.41, significantly outperforming traditional approaches. The methodology addresses overfitting through dropout regularization, batch normalization, and adaptive learning rate strategies while effectively capturing long-term contextual dependencies. This research contributes to the advancement of neural language modeling by providing a robust framework that bridges the gap between computational efficiency and prediction accuracy in real-world NLP applications.

URI

https://hdl.handle.net/11511/118684

Collections

Department of Statistics, Article