ElevenLabs Agents

Use Runway Characters with an ElevenLabs agent when you already have one and want a custom Character to deliver it. ElevenLabs handles speech recognition, reasoning, and text-to-speech; Runway lip-syncs that audio to your Character over WebRTC and does not run its own conversation model for the session.

Before you start

You’ll need:

• A Runway API key
• A custom Character your API key owns and is ready to use
• An ElevenLabs API key with ElevenAgents read permission
• An ElevenLabs agent with Advanced → User input audio format → PCM 16000Hz

Set these server-side environment variables (see the Next.js example for a full .env.example):

RUNWAYML_API_SECRET=...
RUNWAY_AVATAR_ID=...
ELEVENLABS_API_KEY=...
ELEVENLABS_AGENT_ID=...

Server-side only

Keep your Runway API secret and ElevenLabs API key on the server. The browser uses the Characters SDK for WebRTC only — no ElevenLabs SDK, and the signed WebSocket URL never goes to the client.

Guide

1. Install the Characters SDK

   npm install @runwayml/avatars @runwayml/avatars-react

   Server helpers such as createElevenLabsSession live in @runwayml/avatars/api. The React package re-exports them for convenience, but your API route should import from the core package.

2. Create a server route

   createElevenLabsSession from @runwayml/avatars/api fetches an ElevenLabs signed URL, creates a Runway session with integration: { type: "elevenlabs", signedUrl }, polls until READY, and returns sessionId, sessionKey, avatarId, and baseUrl for AvatarCall.

   app/api/avatar/connect/route.ts

   // app/api/avatar/connect/route.ts
   import { createElevenLabsSession } from '@runwayml/avatars/api';
   
   export async function POST() {
   const session = await createElevenLabsSession({
     runwayApiSecret: process.env.RUNWAYML_API_SECRET!,
     avatarId: process.env.RUNWAY_AVATAR_ID!,
     elevenLabsApiKey: process.env.ELEVENLABS_API_KEY!,
     elevenLabsAgentId: process.env.ELEVENLABS_AGENT_ID!,
   });
   
   return Response.json(session);
   }

   Also exported from @runwayml/avatars-react/api if you already depend on the React package only.

   Session create constraints

   Use a custom avatarId your key can access. Do not pass personality, startScript, or tools — the ElevenLabs agent owns conversation config.

   Without the SDK, run the same three steps manually: fetch the signed URL, create the session, poll until ready. See Building your integration for session lifecycle patterns.

3. Connect from the client

   Pass the JSON from your connect route to AvatarCall:

   'use client';
   
   import { AvatarCall } from '@runwayml/avatars-react';
   import '@runwayml/avatars-react/styles.css';
   
   export function Conversation({ sessionId, sessionKey, avatarId, baseUrl }) {
   return (
     <AvatarCall
       avatarId={avatarId}
       sessionId={sessionId}
       sessionKey={sessionKey}
       baseUrl={baseUrl}
     />
   );
   }

   createElevenLabsSession echoes avatarId and baseUrl so you can forward the server response without rebuilding those fields.

4. Test it

   Trigger your connect route, then confirm Character video appears while your ElevenLabs agent responds. Mic audio goes Runway → worker → ElevenLabs; agent audio drives lip sync on the way back.

Full working app: Next.js + ElevenLabs example.

End sessions promptly

Runway bills realtime Character sessions while the Character worker is active. End calls via AvatarCall onEnd and handle errors on your connect route so sessions are not left running.

Troubleshooting

Symptom	Fix
404 Could not find Avatar	Use a Character ID scoped to the same API key (see Custom Avatars)
401 or inactive API key	Check your Runway API key in the Developer Portal
Session stuck NOT_READY	Retry; confirm your environment has the ElevenLabs integration deployed
Silent agent / no audio	Set PCM 16000Hz input format on the ElevenLabs agent
personality… cannot be used with integration	Remove personality, startScript, and tools from the session create request
ElevenLabs signed URL fails	Check xi-api-key, agent ID, and ElevenAgents read permission

See Troubleshooting for general Characters SDK debugging.

Learn more

Next.js + ElevenLabs example Full working app with createElevenLabsSession and AvatarCall.

ElevenLabs agent authentication Signed URLs and server-side API key handling for ElevenLabs agents.

LiveKit Agents Bring your own agent via LiveKit instead of ElevenLabs.

Building your integration Default Runway-owned sessions and custom React UI patterns.