Custom voices
The Voices API lets you design a voice from a text prompt or clone one from an audio sample. Use a ready voice to generate audio-only speech or assign it to an Avatar.
Voice design
Section titled “Voice design”Create a voice by describing the characteristics you want. The prompt should include details like tone, accent, pacing, and personality.
import RunwayML from '@runwayml/sdk';
const client = new RunwayML();
const voice = await client.voices.create({ name: 'Brand Ambassador', from: { type: 'text', prompt: 'A warm, friendly voice with a slight British accent. Speaks at a measured pace with a professional yet approachable tone.', model: 'eleven_ttv_v3', },});
console.log('Voice created:', voice.id);from runwayml import RunwayML
client = RunwayML()
voice = client.voices.create( name='Brand Ambassador', from_={ 'type': 'text', 'prompt': 'A warm, friendly voice with a slight British accent. Speaks at a measured pace with a professional yet approachable tone.', 'model': 'eleven_ttv_v3', },)
print('Voice created:', voice.id)| Parameter | Type | Description |
|---|---|---|
name | string | A name for the voice (max 100 characters). |
from.type | "text" | Indicates voice design from a text prompt. |
from.prompt | string | A description of the desired voice characteristics. Must be at least 20 characters. |
from.model | string | The voice design model. Use eleven_ttv_v3 (latest) or eleven_multilingual_ttv_v2. |
Voice cloning
Section titled “Voice cloning”Clone a voice from an audio sample. Provide a clear recording with minimal background noise and varied tone for best results.
const voice = await client.voices.create({ name: 'Cloned Narrator', from: { type: 'audio', audio: 'https://example.com/voice-sample.mp3', },});
console.log('Voice created:', voice.id);voice = client.voices.create( name='Cloned Narrator', from_={ 'type': 'audio', 'audio': 'https://example.com/voice-sample.mp3', },)
print('Voice created:', voice.id)The audio sample must be between 10 seconds and 5 minutes long. You can pass a public HTTPS URL or a runway:// upload URI (up to 10 MB), or a data:audio/... data URI (up to 5 MB).
Voice status
Section titled “Voice status”Voice creation is asynchronous. After calling create, poll the voice until its status is READY.
const voice = await client.voices.retrieve(voiceId);
if (voice.status === 'READY') { console.log('Preview:', voice.previewUrl);}voice = client.voices.retrieve(id=voice_id)
if voice.status == 'READY': print('Preview:', voice.preview_url)| Status | Description |
|---|---|
PROCESSING | Voice is being generated. Poll until ready. |
READY | Voice is ready. A previewUrl is available. |
FAILED | Generation failed. Check failureReason. |
Generating speech with a custom voice
Section titled “Generating speech with a custom voice”Once a voice is READY, use its ID with the text to speech endpoint. Custom voices work with eleven_v4, eleven_v3, and eleven_multilingual_v2.
Send the following request body to POST /v1/text_to_speech, replacing the example ID with the id returned by the Voices API:
{ "model": "eleven_v4", "promptText": "Welcome! Let me walk you through what is new.", "voice": { "type": "custom", "id": "550e8400-e29b-41d4-a716-446655440000" }}The voice must belong to the same API project as the request. Voices that are still PROCESSING, have failed, or have been deleted cannot be used for speech generation.
The response contains a task ID. Retrieve GET /v1/tasks/{id} until its status is SUCCEEDED, then read the generated audio URL from output. See task polling and SDK helpers.
Seed Audio uses a reference-audio clip instead of a saved voice ID. The speech to speech endpoint accepts preset voices only.
Assigning a custom voice to an Avatar
Section titled “Assigning a custom voice to an Avatar”Once the voice is ready, assign it to an Avatar by setting the voice type to custom and providing the voice ID.
await client.avatars.update(avatarId, { voice: { type: 'custom', id: voice.id, },});client.avatars.update( avatar_id, voice={ 'type': 'custom', 'id': voice.id, },)You can also create a new Avatar with a custom voice directly:
const avatar = await client.avatars.create({ name: 'Support Agent', referenceImage: 'https://example.com/avatar.png', voice: { type: 'custom', id: voice.id, }, personality: 'You are a helpful customer support agent...',});avatar = client.avatars.create( name='Support Agent', reference_image='https://example.com/avatar.png', voice={ 'type': 'custom', 'id': voice.id, }, personality='You are a helpful customer support agent...',)Listing voices
Section titled “Listing voices”Retrieve all custom voices for your project.
const voices = await client.voices.list();
for await (const voice of voices) { console.log(voice.name, voice.status);}for voice in client.voices.list(): print(voice.name, voice.status)You can also manage custom voices through the Developer Portal. See the API reference for all available endpoints.