Speech-to-Speech Translation for Unwritten Language
摘要
The Tulu-to-English speech-to-speech translation system bridges language barriers by converting spoken Tulu phrases into fluent English speech through an end-to-end pipeline. The system integrates automatic speech recognition (ASR) for Tulu speech transcription, a sequence-to-sequence translation model for Tulu-to-English text conversion, and text-to-speech (TTS) synthesis for generating natural-sounding English speech. The model employs a sequential architecture consisting of an embedding layer (256-dimensional), an LSTM layer to capture sequential dependencies, and a time distributed dense layer with softmax activation for word prediction. The model is trained on 3000 datasets over 150 epochs, achieving an accuracy of 78% and a loss of 1%. Optimization is performed using the Adam optimizer and sparse categorical cross-entropy loss function, with a BLEU score of 64.35%, reflecting strong semantic and syntactic alignment, and a word error rate (WER) of 14.28%, suggesting areas for further improvement in word-level accuracy.