This study presents the first unified speech and speaker recognition system for the Kui language. Kui is predominantly spoken in the western region of Odisha, India. It is primarily an oral language and does not have its own script. The Odia script is used for written communication. We did not find any open resources or speech recognition systems in Kui. Our proposed system follows a Bi-modular pipeline: first, a baseline ASR model is established using a Transformer-based encoder. To further improve recognition accuracy, we employ two multilingual pre-trained models and fine-tune them using an in-house continuous Kui speech dataset using transfer learning. Next, we integrate a speaker identification module within the framework, enabling the system to simultaneously recognize spoken content and attribute it to individual speakers. The experimental results demonstrate that the large multilingual pre-trained transfer learning approach significantly reduces both Word Error Rate (WER) and Character Error Rate (CER), while the speaker identification module provides robust speaker recognition. This work highlights the potential of combining multilingual transfer learning and speaker recognition to advance ASR research in low-resource languages such as Kui.