Voice AI Engineer - Optimize Latency & Voice Quality (ElevenLabs + Telnyx)
UpworkFRNot specifiedexpertScore: 63
AI DevelopmentAI Model TrainingNatural Language ProcessingConversational AIVoIPAPI IntegrationWebsocketsArtificial IntelligenceAudio EngineeringElevenLabs
We built a voice AI product using ElevenLabs for TTS and Telnyx for telephony. The core pipeline works, but we're getting consistent user feedback on two issues: **response latency is too high** and **the voice sounds too robotic**. We need an experienced voice AI engineer to audit our current setup and deliver measurable improvements on both fronts.
Scope of Work
*Latency Optimization:*
- Audit the full voice pipeline (STT → LLM → TTS → Telnyx delivery) and identify bottlenecks
- Implement audio streaming/chunking optimizations (e.g., streaming TTS chunks as they're generated rather than waiting for full response)
- Optimize Telnyx media handling and codec configuration
- Reduce end-to-end latency to under 1 second (or propose a realistic target with justification)
*Voice Quality Tuning:*
- Fine-tune ElevenLabs voice settings (stability, similarity boost, style, speaker boost)
- Evaluate and recommend the optimal ElevenLabs model for our use case
- Improve prosody, pacing, and natural speech patterns
- If needed, recommend or create a custom/cloned voice that fits our brand
**Deliverables**
- Written audit of current pipeline with identified bottlenecks
- Implemented optimizations (merged into our codebase)
- Before/after latency benchmarks
- Before/after voice quality comparison (audio samples)
- Documentation of all changes and recommended settings
**Ideal Candidate**
- Proven experience with ElevenLabs API (voice tuning, streaming, model selection)
- Experience with Telnyx or similar VoIP/SIP platforms (Twilio, Vonage, etc.)
- Strong understanding of real-time audio streaming and WebSocket/RTP protocols
- Experience optimizing conversational AI latency (sub-second response times)
- Bonus: experience with voice cloning, SSML, or alternative TTS providers
To Apply, Please Include:
1. Examples of voice AI projects you've optimized (latency numbers and/or audio samples)
2. Your initial thoughts on common latency bottlenecks in an ElevenLabs + Telnyx stack
3. Proposed timeline and fixed-price estimate
Unlock AI Intelligence, score breakdowns, and real-time alerts
Upgrade to Pro — $29.99/moClient
Spent: $52,337.93Rating: 4.9Verified