An AI company partnered with Toptal to extend its speech-to-text platform by training a deep learning speech recognition model capable of recognizing different accents, translating speech into English, and supporting voice-preserving speech synthesis.
The client wanted to make its speech-to-text platform more useful across a wider range of speakers without sacrificing the natural qualities of translated speech. Accomplishing that meant solving several interconnected AI problems, including multilingual speech recognition and voice-preserving speech synthesis.
Built an audio preprocessing pipeline that segmented client-provided speech into labeled training samples, normalized audio, and prepared datasets for supervised speech recognition model fine-tuning.
Fine-tuned an open-source Whisper automatic speech recognition (ASR) model using parameter-efficient fine-tuning (PEFT) techniques to improve transcription accuracy while minimizing training time and GPU resource requirements.
Implemented an automated speech translation pipeline that integrated Google Translate APIs into the transcription workflow, enabling low-latency translation of ASR outputs into English.
Developed a voice-preserving text-to-speech (TTS) synthesis pipeline that generated translated speech while maintaining speaker characteristics through voice cloning techniques.
The engagement delivered a working pipeline that processed multilingual speech, from transcription to translated audio, while preserving the original speaker’s identity.
The solution positioned the platform for expansion to additional languages and accents.
