Semantic text similarity estimation between two long Bangla texts is a critical task issue in the field of NLP. Current similarity measuring methods such as Bert is not quite reasonable for the long texts’ similarity due to the limitation of tokens(512). Due to that limitation of Bert, it cannot capture fully semantic information from the composite complex structure of long texts, turning the loss of information into a loss of accuracy in the similarity score. In this paper, we have proposed a method that is a combination of our custom transformer encoder and Bert that can directly contribute to the better accuracy of similarity scores between two long Bangla texts. Preliminary results demonstrate that our proposed method is more accurate in the Bangla long texts semantic similarity measurement task, achieving an F1 Score of 0.9919 and a Test Loss of 0.0224 during the evaluation stage, compared to the only Bert approach with common evaluation data. These evaluation insights highlight that our proposed similarity calculation framework serves as a dominant power in semantic similarity calculation tasks.