Voice AI Engineer - Optimize Latency & Voice Quality (ElevenLabs + Telnyx)

UpworkFRNot specifiedexpertScore: 63
AI DevelopmentAI Model TrainingNatural Language ProcessingConversational AIVoIPAPI IntegrationWebsocketsArtificial IntelligenceAudio EngineeringElevenLabs
We built a voice AI product using ElevenLabs for TTS and Telnyx for telephony. The core pipeline works, but we're getting consistent user feedback on two issues: **response latency is too high** and **the voice sounds too robotic**. We need an experienced voice AI engineer to audit our current setup and deliver measurable improvements on both fronts. Scope of Work *Latency Optimization:* - Audit the full voice pipeline (STT → LLM → TTS → Telnyx delivery) and identify bottlenecks - Implement audio streaming/chunking optimizations (e.g., streaming TTS chunks as they're generated rather than waiting for full response) - Optimize Telnyx media handling and codec configuration - Reduce end-to-end latency to under 1 second (or propose a realistic target with justification) *Voice Quality Tuning:* - Fine-tune ElevenLabs voice settings (stability, similarity boost, style, speaker boost) - Evaluate and recommend the optimal ElevenLabs model for our use case - Improve prosody, pacing, and natural speech patterns - If needed, recommend or create a custom/cloned voice that fits our brand **Deliverables** - Written audit of current pipeline with identified bottlenecks - Implemented optimizations (merged into our codebase) - Before/after latency benchmarks - Before/after voice quality comparison (audio samples) - Documentation of all changes and recommended settings **Ideal Candidate** - Proven experience with ElevenLabs API (voice tuning, streaming, model selection) - Experience with Telnyx or similar VoIP/SIP platforms (Twilio, Vonage, etc.) - Strong understanding of real-time audio streaming and WebSocket/RTP protocols - Experience optimizing conversational AI latency (sub-second response times) - Bonus: experience with voice cloning, SSML, or alternative TTS providers To Apply, Please Include: 1. Examples of voice AI projects you've optimized (latency numbers and/or audio samples) 2. Your initial thoughts on common latency bottlenecks in an ElevenLabs + Telnyx stack 3. Proposed timeline and fixed-price estimate
View Original Listing
Unlock AI Intelligence, score breakdowns, and real-time alerts
Upgrade to Pro — $29.99/mo

Client

Spent: $52,337.93Rating: 4.9Verified