We introduce a large language model UMamba specifically trained for the Ukrainian language using the novel state-space Mamba architecture. This paper addresses the underrepresentation of Ukrainian in current multilingual models and aims to enhance performance on Ukrainian text. We start by pretraining a state-space model (SSM) on the extensive Malyuk Ukrainian text dataset. Evaluations on perplexity and cross-entropy suggest that the model could be very useful for different downstream tasks, demonstrating significant improvements over comparably sized multilingual models and achieving performance on par with much larger models. Notably, the model is highly capable of handling extremely long contexts, making it well-suited for tasks requiring extended reasoning or document-level understanding. Additionally, due to the efficiency of the SSM, it requires significantly less computational power compared to standard transformer-based models, enabling more cost-effective deployment. Building on this foundation, we perform a simple instruction tuning alignment using a translated version of the Alpaca dataset, allowing the model to better follow instructions. These results underscore the benefits of dedicated monolingual pretraining and targeted alignment, demonstrating that our approach effectively enhances language-specific performance while maintaining efficiency.