In today’s society, the amount of information we need to process daily from sources such as news, videos, and literature is relatively high. The primary strategy to decrease the workload is to use effective summarization techniques, either through extractive (where the summary is made up of extracts from the source itself) or abstractive methods. Traditional summarization models often rely on extensive human-annotated data, which is usually quite costly. This research proposes an approach leveraging transformer models to optimize and affordably augment small datasets, enhancing the performance of summarization models. Using sentence clustering and pre-trained models on tasks such as summarization or paraphrasing, we explore whether such an approach can yield better results across various summarization datasets that target different formats, such as video conference transcripts and news articles.