Bengali poetry, with its rich metaphorical and rhythmic qualities, poses significant challenges for natural language processing due to its linguistic complexity and cultural depth. This study explores the effectiveness of four architectures, LSTM, BiLSTM, Transformer Encoder, and fine-tuned BanglaBERT, for next-word prediction in Bengali poetry using a curated dataset of 6,706 poems from 137 poets. Trained over 700 epochs, the models were evaluated on accuracy, perplexity, BLEU scores, and human-assessed semantic correctness, stylistic appropriateness, and fluency. The LSTM and BiLSTM models achieved validation accuracies of 0.71 and 0.74, respectively, but struggled with long-range dependencies. The Transformer Encoder, with a validation accuracy of 0.77 and a BLEU score of 0.52, better captured poetic structures. BanglaBERT outperformed all, attaining a validation accuracy of 0.89, a perplexity of 10.8, and a BLEU score of 0.68, with human evaluations confirming its superior stylistic and semantic fidelity (4.6–4.8/5). These findings highlight the potential of pretrained transformer models for complex generative tasks in low-resource languages, offering insights for assistive writing and poetic generation in Bengali literature.