Domain specialization methods for large language models (LLMs) significantly enhance their performance in specialized tasks. One such task is competitive programming, which requires a deep understanding of algorithmic paradigms and the ability to generate precise, executable solutions. This study explores domain adaptation techniques for LLMs tailored to various classes of competitive programming problems, including dynamic programming, greedy algorithms, data structures, graph algorithms, and others. We implement a combination of approaches: generation of synthetic problem–solution pairs with explanations, fine-tuning using Low-Rank Adaptation (LoRA), and integration of external knowledge via Retrieval-Augmented Generation (RAG). Experiments conducted on the CodeContests dataset show consistent improvements in the pass@1 metric across all problem domains when using RAG. At the same time, we observe a decrease in pass@10 variance, indicating a narrowing of the solution space. Further analysis of problem difficulty distributions suggests a shift in the model’s behavior—from broad but shallow exploration toward focused and expert-level specialization. The results demonstrate the potential for building adaptive, multi-domain agent systems based on LLMs with dynamic control over solution generation strategies.