This is the full developer documentation for Runway API # API Documentation > Access complete Runway API documentation. Find guides, references and resources to integrate AI video generation into your applications and products. The Runway API allows you to bring our most powerful and popular generative models directly into your apps, products, platforms and websites. [Get started today](https://dev.runwayml.com/). **New:** Aleph 2.0 is now available. Edit existing videos with text prompts and keyframe images. [Get started with Aleph 2.0 →](/api#tag/Start-generating/paths/~1v1~1video_to_video/post) ## [](#models)Models [Section titled “Models”](#models) [![Seedance 2.0](https://d3phaj0sisr2ct.cloudfront.net/devportal/models-v2/seedance-2.jpeg)](/guides/models) [![Gen 4.5](https://d3phaj0sisr2ct.cloudfront.net/devportal/models-v2/gen4.5.jpeg)](/api#tag/Start-generating/paths/~1v1~1image_to_video/post) [![Aleph 2.0](https://d3phaj0sisr2ct.cloudfront.net/devportal/models-v2/aleph-2.0-v2.jpeg)](/api/#tag/Start-generating/paths/~1v1~1video_to_video/post) [![GPT Image 2](https://d3phaj0sisr2ct.cloudfront.net/devportal/models-v2/gpt-image-2.jpg)](/api#tag/Start-generating/paths/~1v1~1text_to_image/post) View available models and details [here](guides/models). ## [](#start-building)Start Building [Section titled “Start Building”](#start-building) [Using the API ](/guides/using-the-api)Create your first generation with Runway's API [API Reference ](/api)Read about how to interact with the API [Go-live checklist ](/guides/go-live)Prepare your integration for launch [Runway Agent Skills ](https://github.com/runwayml/skills)Community tools, examples, and agent skills for the Runway API ## [](#quickstart)Quickstart [Section titled “Quickstart”](#quickstart) Get started by following our [API guide](/guides/using-the-api) or jump right into the example of using the Gen-4.5 model below: * Node ```ts import RunwayML, { TaskFailedError } from '@runwayml/sdk'; const client = new RunwayML(); try { // Create a new text-to-video task using the "gen4.5" model const task = await client.imageToVideo .create({ model: 'gen4.5', promptText: 'A serene mountain landscape at sunrise with mist rolling through the valleys', ratio: '1280:720', duration: 5, }) .waitForTaskOutput(); console.log('Task complete:', task); console.log('Video URL:', task.output[0]); } catch (error) { if (error instanceof TaskFailedError) { console.error('The video failed to generate.'); console.error(error.taskDetails); } else { console.error(error); } } ``` * Python ```python from runwayml import RunwayML, TaskFailedError client = RunwayML() try: task = client.image_to_video.create( model='gen4.5', prompt_text='A serene mountain landscape at sunrise with mist rolling through the valleys', ratio='1280:720', duration=5, ).wait_for_task_output() print('Task complete:', task) print('Video URL:', task.output[0]) except TaskFailedError as e: print('The video failed to generate.') print(e.task_details) ``` ## [](#enterprise-scale)Enterprise Scale [Section titled “Enterprise Scale”](#enterprise-scale) The Runway API has been used by the world’s largest consumer technology companies to reliably generate millions of videos for their users. To request higher usage than [the self-serve tiers](./usage/tiers#tiers), you can submit an exception request on the usage page of your developer portal. ### [](#enterprise-benefits)Enterprise Benefits [Section titled “Enterprise Benefits”](#enterprise-benefits) In addition to higher rate limits, enterprise customers receive: * Faster support via Slack (or email) channels * Earliest access to new features * Direct access to provide product feedback and request features * Implementation and usage tips * Custom payment terms # API Changelog & Updates > Track Runway API changes and updates. View version history, new features, breaking changes and improvements to AI video generation endpoints. A comprehensive history of updates, new features, and improvements to the Runway API. [Create an account](https://dev.runwayml.com/) to get started today. #### [](#seedream-50-pro-in-runway-api)Seedream 5.0 Pro in Runway API [Section titled “Seedream 5.0 Pro in Runway API”](#seedream-50-pro-in-runway-api) July 8th, 2026 - Seedream 5.0 Pro (`seedream5_pro`) is now available via the Runway API. Generate images from text with optional reference images for multi-image fusion and interactive editing. Supports prompts up to 4,000 characters, up to 10 reference images, `outputCount` from 1–4, and 14 `ratio` values across 1K and 2K at seven aspect ratios (`1:1`, `4:3`, `3:4`, `16:9`, `9:16`, `3:2`, `2:3`). Billed at 5 credits per 1K image and 9 credits per 2K image, multiplied by `outputCount`. Use the [text to image](/api#tag/Start-generating/paths/~1v1~1text_to_image/post) endpoint to get started. #### [](#gemini-omni-flash-in-runway-api)Gemini Omni Flash in Runway API [Section titled “Gemini Omni Flash in Runway API”](#gemini-omni-flash-in-runway-api) July 1st, 2026 - Gemini Omni Flash (`gemini_omni_flash`) is now available via the Runway API. Generate videos from text, a first-frame image, or an existing video, with output at 720p and optional generated audio. Text-to-video and image-to-video support durations from 3–10 seconds (default 5) at landscape `1280:720` or portrait `720:1280`. Video-to-video edits an input video of up to 10 seconds — output matches the input’s duration and orientation — and accepts up to 5 reference images to guide the edit. Text-to-video and image-to-video are billed at 10 credits per second (plus 1 credit for the first-frame image on image-to-video), and video-to-video at 11 credits per second of input video plus 1 credit per reference image. Use the [text to video](/api#tag/Start-generating/paths/~1v1~1text_to_video/post), [image to video](/api#tag/Start-generating/paths/~1v1~1image_to_video/post), and [video to video](/api#tag/Start-generating/paths/~1v1~1video_to_video/post) endpoints to get started. #### [](#seed-audio-10-in-runway-api)Seed Audio 1.0 in Runway API [Section titled “Seed Audio 1.0 in Runway API”](#seed-audio-10-in-runway-api) June 30th, 2026 - Seed Audio 1.0 (`seed_audio`) is now available via the Runway API. Generate speech and sound effects from text, with optional audio references of up to 30 seconds to guide the output. Seed Audio 1.0 supports up to 120 seconds of generated audio and can output in WAV, MP3, and Ogg Opus formats. It is billed at 0.25 credits per second, with a 5 credit minimum per generation. Use the [text to speech](/api#tag/Start-generating/paths/~1v1~1text_to_speech/post) and [sound effect](/api#tag/Start-generating/paths/~1v1~1sound_effect/post) endpoints to get started. #### [](#seedance-20-mini-in-runway-api)Seedance 2.0 Mini in Runway API [Section titled “Seedance 2.0 Mini in Runway API”](#seedance-20-mini-in-runway-api) June 26th, 2026 - Seedance 2.0 Mini (`seedance2_mini`) is now available via the Runway API. Generate videos from text, images, or video with support for keyframe control, reference images, reference videos, and generated audio. Seedance 2.0 Mini supports image-to-video, text-to-video, and video-to-video generation modes, with durations from 4–15 seconds and output at 480p or 720p. Seedance 2.0 Mini is billed at 16 credits per second, with a 64 credit minimum per generation. Use the [text to video](/api#tag/Start-generating/paths/~1v1~1text_to_video/post), [image to video](/api#tag/Start-generating/paths/~1v1~1image_to_video/post), and [video to video](/api#tag/Start-generating/paths/~1v1~1video_to_video/post) endpoints to get started. #### [](#ad-localization-recipe)Ad Localization recipe [Section titled “Ad Localization recipe”](#ad-localization-recipe) June 25th, 2026 - The Ad Localization recipe (`ad_localization`) is now available via the Runway API. Localize an existing ad image for a target language while preserving visual creative and layout. Billed at 21 credits per localized image. Use the [Ad Localization recipe](/recipes/ad-localization) guide and [API reference](/api#tag/Recipes/paths/~1v1~1recipes~1ad_localization/post) to get started. #### [](#4k-support-for-seedance-20)4K support for Seedance 2.0 [Section titled “4K support for Seedance 2.0”](#4k-support-for-seedance-20) June 24th, 2026 - Seedance 2.0 (`seedance2`) now supports 4K output. Six new 4K `ratio` values are available across the [text to video](/api#tag/Start-generating/paths/~1v1~1text_to_video/post), [image to video](/api#tag/Start-generating/paths/~1v1~1image_to_video/post), and [video to video](/api#tag/Start-generating/paths/~1v1~1video_to_video/post) endpoints: `3840:1646` (21:9), `3840:2160` (16:9), `3840:2880` (4:3), `3840:3840` (1:1), `2880:3840` (3:4), and `2160:3840` (9:16). These are in addition to the existing 480p, 720p, and 1080p options, and the default `ratio` remains `1280:720`. 4K generations are billed at 150 credits per second. #### [](#magnific-video-upscaler)Magnific Video Upscaler [Section titled “Magnific Video Upscaler”](#magnific-video-upscaler) June 11th, 2026 - Magnific Video Upscaler (`magnific_video_upscaler_creative`) is now available via the Runway API. Upscale input videos up to 30 seconds to `720p`, `1k`, `2k` (default), or `4k` output resolution, with optional controls for creativity, sharpen, smart grain, flavor, and fps boost. Always set `model` to `magnific_video_upscaler_creative` on the [video upscale endpoint](/api#tag/Start-generating/paths/~1v1~1video_upscale/post). Billing is per output frame — see [video upscale pricing](/guides/pricing#video-upscale-pricing). #### [](#seedance-20-fast-in-runway-api)Seedance 2.0 Fast in Runway API [Section titled “Seedance 2.0 Fast in Runway API”](#seedance-20-fast-in-runway-api) June 5th, 2026 - Seedance 2.0 Fast (`seedance2_fast`) is now available via the Runway API. Generate videos faster from text, images, or video with support for keyframe control, reference images, reference videos, and generated audio. Seedance 2.0 Fast supports image-to-video, text-to-video, and video-to-video generation modes, with durations from 4–15 seconds and output at 480p or 720p. Use the [text to video](/api#tag/Start-generating/paths/~1v1~1text_to_video/post), [image to video](/api#tag/Start-generating/paths/~1v1~1image_to_video/post), and [video to video](/api#tag/Start-generating/paths/~1v1~1video_to_video/post) endpoints to get started. #### [](#aleph-20-in-runway-api)Aleph 2.0 in Runway API [Section titled “Aleph 2.0 in Runway API”](#aleph-20-in-runway-api) June 2nd, 2026 - Aleph 2.0 is now available via the Runway API. Edit existing videos with text prompts and optional keyframe images placed at specific timestamps. Aleph 2.0 supports input videos from 2–30 seconds, up to 5 keyframe images, and optional content moderation settings. Use the model identifier `aleph2` with the [video to video endpoint](/api#tag/Start-generating/paths/~1v1~1video_to_video/post). The `aleph2_alpha` identifier remains available as a deprecated alias. #### [](#happyhorse-10)HappyHorse 1.0 [Section titled “HappyHorse 1.0”](#happyhorse-10) May 29th, 2026 - HappyHorse 1.0 (`happyhorse_1_0`) is now available via the Runway API. Generate videos from text or from a first-frame image with durations from 3–15 seconds. Text-to-video supports 10 output dimensions across 720p and 1080p. Image-to-video preserves your input aspect ratio and accepts an optional motion prompt. Use the [text to video](/api#tag/Start-generating/paths/~1v1~1text_to_video/post) and [image to video](/api#tag/Start-generating/paths/~1v1~1image_to_video/post) endpoints to get started. #### [](#seedance-20-in-runway-api)Seedance 2.0 in Runway API [Section titled “Seedance 2.0 in Runway API”](#seedance-20-in-runway-api) May 28th, 2026 - Seedance 2.0 is now available via the Runway API. Generate high-quality videos from text, images, or video with support for keyframe control, reference images, reference videos, and generated audio. Seedance 2.0 supports image-to-video, text-to-video, and video-to-video generation modes, with durations from 4–15 seconds. #### [](#gemini-3-pro-image-nano-banana-pro)Gemini 3 Pro Image (Nano Banana Pro) [Section titled “Gemini 3 Pro Image (Nano Banana Pro)”](#gemini-3-pro-image-nano-banana-pro) April 30th, 2026 - Gemini 3 Pro Image, also known as Nano Banana Pro (`gemini_image3_pro`), is now available via the Runway API. Generate images with up to 5,500-character prompts and up to 14 reference images, at 1K, 2K, or 4K resolution across 10 aspect ratios. Use the [text to image endpoint](/api#tag/Start-generating/paths/~1v1~1text_to_image/post) to get started. #### [](#gpt-image-2)GPT Image 2 [Section titled “GPT Image 2”](#gpt-image-2) April 23rd, 2026 - OpenAI’s GPT Image 2 (`gpt_image_2`) is now available via the Runway API. Supports up to 16 reference images (with optional prompt-referenceable tags), 30 `width:height` ratios across 1K, 2K, and 4K tiers plus an `auto` resolution, and tiered pricing by resolution and quality. Unlike `gpt_image_1`, transparent backgrounds are not supported. Use the [text to image endpoint](/api#tag/Start-generating/paths/~1v1~1text_to_image/post) to get started. #### [](#gen-45-in-runway-api)Gen-4.5 in Runway API [Section titled “Gen-4.5 in Runway API”](#gen-45-in-runway-api) February 10th, 2026 - Gen-4.5 is now available via the Runway API. Generate high-quality videos from text alone or from images with improved quality and control. Gen-4.5 supports both text-to-video and image-to-video generation modes, with durations from 2-10 seconds. Learn more at [Gen-4.5 announcement](https://runwayml.com/research/introducing-runway-gen-4.5). #### [](#elevenlabs-clean-audio)ElevenLabs Clean Audio [Section titled “ElevenLabs Clean Audio”](#elevenlabs-clean-audio) October 16th, 2025 - ElevenLabs Voice Isolation is available directly via the Runway API. Strip background noise from any recording and isolate crisp, clear speech—built for film, podcast, and interview workflows. #### [](#elevenlabs-voice-dubbing)ElevenLabs Voice Dubbing [Section titled “ElevenLabs Voice Dubbing”](#elevenlabs-voice-dubbing) October 16th, 2025 - ElevenLabs Dubbing is available directly via the Runway API. Translate your content into 29 languages with AI-generated speech that maintains the speaker’s original voice characteristics and emotional tone. #### [](#google-veo-31)Google Veo 3.1 [Section titled “Google Veo 3.1”](#google-veo-31) October 15th, 2025 - Google Veo 3.1 text to image and image to video are now available in the Runway API. Generate with even greater fidelity and control with first and last keyframe support, the new Reference to Video feature and support for full 1080p outputs. Learn more at the link below. #### [](#elevenlabs-text-to-sound-effects)ElevenLabs text to Sound Effects [Section titled “ElevenLabs text to Sound Effects”](#elevenlabs-text-to-sound-effects) October 8th, 2025 - Flexible generation times for Runway video models are now available via the Runway API. Choose any duration from 2-10 seconds using Gen-4 Turbo. Pay only for what you generate. #### [](#flexible-generation-length)Flexible Generation Length [Section titled “Flexible Generation Length”](#flexible-generation-length) October 8th, 2025 - Flexible generation times for Runway video models are now available via the Runway API. Choose any duration from 2-10 seconds using Gen-4 Turbo. Pay only for what you generate. #### [](#text-to-speech)Text to Speech [Section titled “Text to Speech”](#text-to-speech) September 25th, 2025 - Starting today, ElevenLabs’ Multilingual v2 Text to Speech is available directly via the Runway API. Generate natural, emotionally-aware speech in 29 languages while maintaining consistent voice quality and personality. #### [](#api-playground)API Playground [Section titled “API Playground”](#api-playground) September 24th, 2025 - Today we’re launching the Runway API Playground—a new interactive environment that lets developers test and refine their integrations before going to production. Build with confidence and ship faster. The Runway API Playground provides a full sandbox environment with all our latest models. #### [](#third-party-models)Third Party Models [Section titled “Third Party Models”](#third-party-models) September 23rd, 2025 - Select third party models are now available via the Runway API. All designed to make your creative workflows even more robust and controllable within Runway. Starting with Google Veo3 and Nano Banana (gemini\_2.5\_flash). #### [](#launch-gen-4-image-turbo-in-runway-api)Launch Gen-4 Image Turbo in Runway API [Section titled “Launch Gen-4 Image Turbo in Runway API”](#launch-gen-4-image-turbo-in-runway-api) August 19th, 2025 - Introducing our fastest image model. Generate with reference images in 10 seconds or less. Gen-4 Image Turbo costs 2.5-4x less than non-turbo and achieves a 93.3% quality Dreambench++ Score compared to standard Gen-4 Image generations. #### [](#aleph-in-runway-api)Aleph in Runway API [Section titled “Aleph in Runway API”](#aleph-in-runway-api) August 1st, 2025 - [Aleph](https://runwayml.com/research/introducing-runway-aleph) is now available via the Runway API, allowing you to bring an entirely new to way to edit, transform and generate videos directly into your apps, products, platforms and websites. #### [](#act-two-in-runway-api)Act-Two in Runway API [Section titled “Act-Two in Runway API”](#act-two-in-runway-api) July 21st, 2025 - Act-Two is now available via the Runway API, allowing you to bring our most advanced motion capture directly into your apps, products, platforms and websites. #### [](#runway-mcp-server)Runway MCP Server [Section titled “Runway MCP Server”](#runway-mcp-server) June 13th, 2025 - Our [MCP server](https://github.com/runwayml/runway-api-mcp-server) is now available on GitHub. Now you can connect Claude, or any MCP-compatible assistant, directly to Runway’s generation capabilities, allowing you to build AI agents that can generate videos, images and more as part of automated workflows. #### [](#launch-gen-4-image-in-runway-api)Launch Gen-4 Image in Runway API [Section titled “Launch Gen-4 Image in Runway API”](#launch-gen-4-image-in-runway-api) May 16th, 2025 - Gen-4 Image is now available via the Runway API, allowing anyone to integrate its powerful multimodal generation capabilities directly into their apps, products, platforms and websites. # Content Moderation System > Learn about Runway's content moderation for API usage. Understand safety guidelines, moderation policies and best practices for responsible AI use. Runway takes the safety of its platform seriously. This means we will moderate certain API requests. If your account makes too many requests that get moderated, we will suspend your account. If needed, you should add moderation before calling the Runway API, to avoid suspension. ## [](#approach-to-trust--safety)Approach to Trust & Safety [Section titled “Approach to Trust & Safety”](#approach-to-trust--safety) Refer to our [help center approach to trust & safety](https://help.runwayml.com/hc/en-us/articles/17944787368595-Runway-s-approach-to-trust-safety). ### [](#moderated-content-categories)Moderated content categories [Section titled “Moderated content categories”](#moderated-content-categories) Refer to our [help center FAQ guide on content moderation](https://help.runwayml.com/hc/en-us/articles/21745792516371-Why-is-my-input-getting-content-moderated-and-what-types-of-content-are-blocked). ### [](#moderated-content-types)Moderated content types [Section titled “Moderated content types”](#moderated-content-types) Runway moderation will evaluate all elements of your request. This means your requests may be moderated for either an image or text prompt violation. ## [](#moderation-implications)Moderation implications [Section titled “Moderation implications”](#moderation-implications) ### [](#recommended-moderation-approach)Recommended moderation approach [Section titled “Recommended moderation approach”](#recommended-moderation-approach) We recommend reviewing [the categories we block](https://help.runwayml.com/hc/en-us/articles/21745792516371-Why-is-my-input-getting-content-moderated-and-what-types-of-content-are-blocked) to determine what type of, if any, moderation you need in place. Each call to Runway’s API defaults to `auto` moderation levels. If you wish to be less strict about preventing generations that include recognizable public figures, add the `contentModeration` object to your image or video generation API requests. See the [API reference](/api/#tag/Start-generating/paths/~1v1~1image_to_video/post) for details. ### [](#cost-of-moderated-generations)Cost of moderated generations [Section titled “Cost of moderated generations”](#cost-of-moderated-generations) Moderated generations have the same credit cost as successful generations. ### [](#account-suspension)Account suspension [Section titled “Account suspension”](#account-suspension) If your account makes too many moderated requests we will suspend it. You can [appeal an account suspension here](https://help.runwayml.com/hc/en-us/articles/21667383978003-How-can-I-appeal-my-account-s-suspension). Be sure to email from an email account associated with your developer portal login. ## [](#moderation-api-responses)Moderation API responses [Section titled “Moderation API responses”](#moderation-api-responses) If a request was moderated, the task status response will return with `"status":"FAILED"`. Additional details describing the moderation appear in the `"failure"` and `"failureCode"` fields. # Software Development Kits > Download Runway API SDKs for Python, Node.js and more. Use official libraries to integrate AI video generation into your development workflow. ## [](#available-sdks)Available SDKs [Section titled “Available SDKs”](#available-sdks) We provide SDKs as convenient helpers for interacting with our API. These SDKs use best practices and offer type safety, which helps to avoid errors and make it easier to write code. ### [](#nodejs)Node.js [Section titled “Node.js”](#nodejs) The Node.js SDK includes TypeScript bindings. It is compatible with Node 18 and up, and can be installed with `npm`, `yarn`, or `pnpm`. ### [](#python)Python [Section titled “Python”](#python) The Python SDK includes type annotations compatible with MyPy. It is compatible with Python 3.8 and up. ## [](#generating-content)Generating content [Section titled “Generating content”](#generating-content) You can create content using our API using the methods documented in the [API reference](/api). For instance, `POST /v1/text_to_image` accepts input and produces an image as output. Each API endpoint for starting a generation is available as a member on the SDKs. Here is a subset of the API endpoints mapped to the SDK methods: * Node | Operation | API endpoint | Node.js SDK method | | --------------------- | -------------------------------- | ------------------------------------ | | Generate an image | `POST /v1/text_to_image` | `client.textToImage.create` | | Generate a video | `POST /v1/image_to_video` | `client.imageToVideo.create` | | Character performance | `POST /v1/character_performance` | `client.characterPerformance.create` | * Python | Operation | API endpoint | Python SDK method | | --------------------- | -------------------------------- | ------------------------------------- | | Generate an image | `POST /v1/text_to_image` | `client.text_to_image.create` | | Generate a video | `POST /v1/image_to_video` | `client.image_to_video.create` | | Character performance | `POST /v1/character_performance` | `client.character_performance.create` | Calling these methods will create a task. A task is a record of the generation operation. The response from the method will look like this: ```plaintext { "id": "17f20503-6c24-4c16-946b-35dbbce2af2f" } ``` The `id` field is the unique identifier for the task. You can use this ID to retrieve the task status and output from the [`GET /v1/tasks/{id}` endpoint](/api#tag/Task-management/paths/~1v1~1tasks~1%7Bid%7D/get), which is available as the `tasks.retrieve` method on the SDKs. * Node ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const task = await client.tasks.retrieve('17f20503-6c24-4c16-946b-35dbbce2af2f'); console.log(task); ``` * Python ```python from runwayml import RunwayML client = RunwayML() task = client.tasks.retrieve('17f20503-6c24-4c16-946b-35dbbce2af2f') print(task) ``` The response from the method will look like this: ```plaintext { "id": "17f20503-6c24-4c16-946b-35dbbce2af2f", "status": "PENDING", "createdAt": "2024-06-27T19:49:32.334Z" } ``` The [API reference](/api#tag/Task-management/paths/~1v1~1tasks~1%7Bid%7D/get) documents the statuses that a task can be in, along with the fields that are available for each status. Tasks are processed asychronously. The `tasks.retrieve` method returns the current status of the task, which you can poll until the task has completed. The task will eventually transition to a `SUCCEEDED`, `CANCELED`, or `FAILED` status. When polling, we recommend using an interval of 5 seconds or more. You should also add jitter, and handle non-200 responses with exponential backoff. Avoid using fixed interval polling (such as with JavaScript’s `setInterval`), since latency from the API can cause the polling to be too frequent. ### [](#built-in-task-polling)Built-in task polling [Section titled “Built-in task polling”](#built-in-task-polling) As a convenience, all SDK methods that return a task include a helper method that polls for the task output. This reduces the amount of code you need to write to wait for a task to complete. * Node The `waitForTaskOutput` method is present on the unawaited response from the `create` methods on `textToImage`, `imageToVideo`, and other endpoints to start generations. ```ts // ✅ Call `waitForTaskOutput` on the unawaited response from `create` const imageTask = await client.textToImage .create({ model: 'gen4_image', promptText: 'A beautiful sunset over a calm ocean', ratio: '1360:768', }) .waitForTaskOutput(); console.log(imageTask.output[0]); // Print the URL of the generated image ``` ```ts // ✅ Getting the task ID for bookkeeping purposes // Notice: no `await` here. const imageTask = client.textToImage.create({ model: 'gen4_image', promptText: 'A beautiful sunset over a calm ocean', ratio: '1360:768', }); // Await the output of `create` to get the task ID const taskId = (await imageTask).id; console.log(taskId); // The task ID can be stored in your database, for instance. // Wait for the task to complete. It is safe to await `waitForTaskOutput` // after the output of `create` was awaited above. const completedTask = await imageTask.waitForTaskOutput(); console.log(completedTask.output[0]); // Print the URL of the generated image ``` ```diff // ❌ If you await the response from `create`, the result not have access to // `waitForTaskOutput`. const awaitedImageTask = await client.textToImage.create({ model: 'gen4_image', promptText: 'A beautiful sunset over a calm ocean', ratio: '1360:768', }); -const taskOutput = await awaitedImageTask.waitForTaskOutput(); ``` If the task fails (that is, its status becomes `FAILED`), a `TaskFailedError` will be thrown. You should handle this error appropriately. ```ts import { TaskFailedError } from '@runwayml/sdk'; try { const imageTask = await client.textToImage .create({ model: 'gen4_image', promptText: 'A beautiful sunset over a calm ocean', ratio: '1360:768', }) .waitForTaskOutput(); } catch (error) { if (error instanceof TaskFailedError) { // `taskDetails` contains the output of the tasks.retrieve call. console.error('Task failed:', error.taskDetails); } else { throw error; } } ``` The `waitForTaskOutput` method accepts an optional `options` parameter. This parameter can be used to specify a timeout and an `AbortSignal`. ```ts const imageTask = await client.textToImage .create({ model: 'gen4_image', promptText: 'A beautiful sunset over a calm ocean', ratio: '1360:768', }) .waitForTaskOutput({ // Wait up to 5 minutes for the task to complete timeout: 5 * 60 * 1000, // Abort the task if the request is cancelled abortSignal: myAbortSignal, }); ``` By default, `waitForTaskOutput` will wait for ten minutes before timing out. Upon timeout, a `TaskTimedOutError` will be thrown. Pass `null` to `timeout` to wait indefinitely. Disabling the timeout is not recommended as it may cause your server to experience issues if your Runway API organization reaches its concurrency limit or if Runway experiences an outage. It is recommended to use an `AbortSignal` to cancel polling if you are using `waitForTaskOutput` in the handler for an incoming request, such as on a web server. Here is an example of how to correctly integrate this into your application: * Express.js ```ts const runway = new RunwayML(); app.post('/generate-image', (req, res) => { // Create an AbortController that triggers when the request is closed // unexpectedly. const abortController = new AbortController(); req.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: req.body.prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); res.send(imageTask.output[0]); } catch (error) { if (error instanceof TaskFailedError) { res.status(500).send('Task failed'); } else { throw error; } } }); ``` * Koa ```ts const runway = new RunwayML(); app.post('/generate-image', async (ctx) => { // Create an AbortController that triggers when the request is closed // unexpectedly. const abortController = new AbortController(); ctx.req.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: ctx.request.body.prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); ctx.body = imageTask.output[0]; } catch (error) { if (error instanceof TaskFailedError) { ctx.status = 500; ctx.body = 'Task failed'; } else { throw error; } } }); ``` * Socket.io ```ts const runway = new RunwayML(); io.on('connection', (socket) => { // Create an AbortController that triggers when the socket is closed // unexpectedly. const abortController = new AbortController(); socket.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. socket.on('generate-image', async (prompt) => { try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); socket.emit('image', imageTask.output[0]); } catch (error) { if (error instanceof TaskFailedError) { socket.emit('error', 'Task failed'); } else { throw error; } } }); }); ``` Danger Triggering the `abortSignal` passed to `waitForTaskOutput` or hitting the passed `timeout` will not cancel the task. Cancelling the task must be done by invoking the [cancellation endpoint](/api#tag/Task-management/paths/~1v1~1tasks~1%7Bid%7D/delete). In addition to the methods that create new tasks, the `tasks.retrieve` method also returns a promise with a `waitForTaskOutput` method. This method is equivalent to the `waitForTaskOutput` method on the unawaited response from the `create` methods. ```ts const task = await client.tasks .retrieve('17f20503-6c24-4c16-946b-35dbbce2af2f') .waitForTaskOutput(); console.log(task.output[0]); ``` This is useful if you’d like to create a task in one request and wait for its output in another request, or for handling the case where the client disconnected before the task completed. Be aware that you must still add error handling for `TaskFailedError` and `TaskTimeoutError` when using this method. * Python ```ts const runway = new RunwayML(); app.post('/generate-image', (req, res) => { // Create an AbortController that triggers when the request is closed // unexpectedly. const abortController = new AbortController(); req.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: req.body.prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); res.send(imageTask.output[0]); } catch (error) { if (error instanceof TaskFailedError) { res.status(500).send('Task failed'); } else { throw error; } } }); ``` * Express.js ```ts const runway = new RunwayML(); app.post('/generate-image', async (ctx) => { // Create an AbortController that triggers when the request is closed // unexpectedly. const abortController = new AbortController(); ctx.req.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: ctx.request.body.prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); ctx.body = imageTask.output[0]; } catch (error) { if (error instanceof TaskFailedError) { ctx.status = 500; ctx.body = 'Task failed'; } else { throw error; } } }); ``` * Koa ```ts const runway = new RunwayML(); io.on('connection', (socket) => { // Create an AbortController that triggers when the socket is closed // unexpectedly. const abortController = new AbortController(); socket.on('close', () => { abortController.abort(); }); // 🚨 When performing a generation, be sure to add appropriate rate limiting // and other safeguards to prevent abuse. socket.on('generate-image', async (prompt) => { try { const imageTask = await runway.textToImage .create({ model: 'gen4_image', promptText: prompt, ratio: '1360:768', }) .waitForTaskOutput({ abortSignal: abortController.signal }); socket.emit('image', imageTask.output[0]); } catch (error) { if (error instanceof TaskFailedError) { socket.emit('error', 'Task failed'); } else { throw error; } } }); }); ``` * Socket.io The `wait_for_task_output` method is present on the unawaited response from the `create` methods on `text_to_image`, `image_to_video`, and other methods to start generations. ```python from runwayml import RunwayML client = RunwayML() image_task = client.text_to_image.create( model='gen4_image', prompt_text='A beautiful sunset over a calm ocean', ratio='1360:768', ) task_output = image_task.wait_for_task_output() print(task_output.output[0]) ``` If the task fails (that is, its status becomes `FAILED`), a `TaskFailedError` will be raised. You should handle this error appropriately. ```python from runwayml import RunwayML, TaskFailedError client = RunwayML() # 🚨 When performing a generation, be sure to add appropriate rate limiting # and other safeguards to prevent abuse. try: image_task = client.text_to_image.create( model='gen4_image', prompt_text='A beautiful sunset over a calm ocean', ratio='1360:768', ) task_output = image_task.wait_for_task_output() except TaskFailedError as e: print('Task failed:', e.task_details) ``` The `wait_for_task_output` method accepts an optional `timeout` parameter. This parameter specifies the maximum amount of time to wait for the task to complete in seconds. If not specified, the default timeout is ten minutes. Pass `None` to `timeout` to wait indefinitely. Disabling the timeout is not recommended as it may cause your server to experience issues if your Runway API organization reaches its concurrency limit or if Runway experiences an outage. ```python from runwayml import RunwayML client = RunwayML() image_task = client.text_to_image.create( model='gen4_image', prompt_text='A beautiful sunset over a calm ocean', ratio='1360:768', ).wait_for_task_output( # Wait up to 5 minutes for the task to complete timeout=5 * 60, ) ``` When the timeout is reached, a `TaskTimeoutError` will be raised. Caution If the timeout is reached, the task will not be cancelled. Cancelling the task must be done by invoking the [cancellation endpoint](/api#tag/Task-management/paths/~1v1~1tasks~1%7Bid%7D/delete). In addition to the methods that create new tasks, the `tasks.retrieve` method also returns a promise with a `wait_for_task_output` method. This method is equivalent to the `wait_for_task_output` method on the unawaited response from the `create` methods. ```python task_output = await client.tasks.retrieve(image_task.id).wait_for_task_output() print(task_output.output[0]) ``` This is useful if you’d like to create a task in one request and wait for its output in another request, or for handling the case where the client disconnected before the task completed. Be aware that you must still add error handling for `TaskFailedError` and `TaskTimeoutError` when using this method. ## [](#code-skills--agent-tool)Code Skills & Agent Tool [Section titled “Code Skills & Agent Tool”](#code-skills--agent-tool) The [Runway Skills repository](https://github.com/runwayml/skills) provides a collection of tools, examples, and agent skills for working with the Runway API. This includes: * **Claude Code Skills** - Pre-built skills for AI agents to interact with the Runway API * **Code and Prompt Examples** - Sample implementations and integration patterns # API Versioning Policy > Understand Runway's API versioning strategy. Learn about version compatibility, deprecation timelines and how to manage API updates in your apps. The Runway API uses an `X-Runway-Version` HTTP request header to specify which version of the API to use. This is in the form of a date, like `2024-11-06`. When you build an integration, setting a version header changes the default behavior of our API. ## [](#when-is-a-new-version-number-created)When is a new version number created? [Section titled “When is a new version number created?”](#when-is-a-new-version-number-created) We’ll add new features and functionality to the latest existing API version so long as the functionality can be added in a backwards-compatible way. You’ll be able to enjoy these features automatically without having to make changes to your integration. A new version will only be created when we make changes to the API that are not backwards-compatible. Here are a few examples of what might result in a new API version: * **The type of an input parameter is changed.** For instance, if a parameter that was previously a string representing a URL becomes an object containing more information about the asset referenced by the URL. * **The name of an input parameter is changed.** For example, we may restructure an API to better reflect new techniques or model constraints. An input that was previously a boolean to enable a feature may be changed to an object to allow extra configuration to be provided beyond. * **Functionality is removed.** If we deprecate a feature, we’ll create a new API version that removes the functionality. ## [](#version-support)Version support [Section titled “Version support”](#version-support) We’ll offer support for old API versions for four months after a new version is created. After that date, API requests with an old API version may be rejected. ### [](#managing-versions)Managing versions [Section titled “Managing versions”](#managing-versions) We strongly suggest using one of the official [Runway API SDKs](/api-details/sdks). We’ll publish new major versions of each SDK when a new API version is released. Using [TypeScript](https://www.typescriptlang.org/) or a Python type checker like [MyPy](https://www.mypy-lang.org/), you’ll be able to easily identify any places where your integration needs to be updated. ## [](#v1-urls)`/v1` URLs [Section titled “/v1 URLs”](#v1-urls) While we include `/v1/` in our URLs, this is not used for versioning the API. We include this to reserve the ability to offer new functionality concurrently with our `/v1/` endpoints in the future. # API Version 2024-11-06 > API change log The `2024-11-06` API version introduces changes to the `/v1/image_to_video` endpoint. ## [](#ratio)`ratio` [Section titled “ratio”](#ratio) The `ratio` parameter no longer accepts `16:9` and `9:16`. Instead, it now accepts the resolution for the output video directly. The accepted values are: * `1280:768` * `768:1280` These new values will be accepted by the previous API version (`2024-09-13`), allowing you to incrementally upgrade, however be aware that this older API version will be phased out over time. ## [](#promptimage)`promptImage` [Section titled “promptImage”](#promptimage) The `promptImage` parameter was previously defined as a `string`. It is now equivalent to the following: * TypeScript ```ts type promptImage = | string | Array<{ uri: string; position: 'first' | 'last'; }>; ``` * Python ```python from typing import Union, List, Literal, TypedDict class ImagePosition(TypedDict): uri: str position: Literal['first', 'last'] PromptImage = Union[str, List[ImagePosition]] ``` The following two are equivalent: ```json { "promptImage": "https://example.com/image.jpg" } ``` ```json { "promptImage": [ { "uri": "https://example.com/image.jpg", "position": "first" } ] } ``` If you were to set `position` to `"last"`, the generated video will end with the image instead of starting with it. ```json { "promptImage": [ { "uri": "https://example.com/image.jpg", "position": "last" } ] } ``` You can also pass two images to the `promptImage` parameter: ```json { "promptImage": [ { "uri": "https://example.com/image1.jpg", "position": "first" }, { "uri": "https://example.com/image2.jpg", "position": "last" } ] } ``` In the above example, the generated video will start with `image1.jpg` and end with `image2.jpg`. Caution Each image in the `promptImage` array must have a unique `position`. ## [](#seed)`seed` [Section titled “seed”](#seed) The maximum value of `seed` is now `4294967295` (or `2^32 - 1`), up from `999999999`. # API Input Parameters > Learn about input parameters for Runway's API. Understand how to format images, videos, text prompts and other inputs for AI generation requests. When starting tasks through the Runway API, you’ll often need to provide assets like images. Some restrictions exist for what you can provide. Assets can be provided via URLs or Data URIs. ## [](#size-limits)Size limits [Section titled “Size limits”](#size-limits) The size limits of inputs differ based on whether they are provided as a URL or as a data URI in the JSON request body. | Input type | URL size limit | Data URI size limit | Ephemeral uploads | | ---------- | -------------- | ------------------- | ----------------- | | Image | 16MB | 5MB | 200MB | | Video | 32MB | 16MB | 200MB | | Audio | 32MB | 16MB | 200MB | ## [](#inputs-using-urls)Inputs using URLs [Section titled “Inputs using URLs”](#inputs-using-urls) In all cases, URLs must meet some basic minimum requirements: 1. All URLs must be HTTPS. 2. URLs must reference a domain name in the hostname position, not an IP address. 3. The server should respond with valid `Content-Type` and `Content-Length` headers. 4. Redirects are not followed. If the URL returns a 3XX response code, the request is considered failed. 5. The length of any single URL should not exceed `2048` characters. Additionally, the server responding to the request must support HTTP `HEAD` requests. ### [](#content-type-values)`Content-Type` values [Section titled “Content-Type values”](#content-type-values) When specifying a URL, the `Content-Type` response header must be returned, and it must match the media type of your asset. File extensions in URLs are not considered. The `Content-Type`s that are supported are [listed below](#type-specific-requirements) for the supported asset types. Be aware that `application/octet-stream` and other generic values are explicitly not supported. ### [](#user-agent)User agent [Section titled “User agent”](#user-agent) Runway will use a `User-Agent` header that starts with `RunwayML API/` when making requests to your server. If you use a scraping-prevention tool or WAF, be sure to allowlist our user agent string prefix. ## [](#inputs-using-data-uris-base64-encoded)Inputs using Data URIs (base64 encoded) [Section titled “Inputs using Data URIs (base64 encoded)”](#inputs-using-data-uris-base64-encoded) A [data URI](https://en.wikipedia.org/wiki/Data_URI_scheme) allows you to pass the base64 encoded assets as part of a request to our API, rather than passing a URL to the asset hosted on another server. This can reduce the complexity of your integration by eliminating an upload step, but it also has a lower limit on the size of the assets that can be provided. Data URIs are accepted in any field where a URL is accepted unless otherwise specified. * An appropriate [content type](https://en.wikipedia.org/wiki/Media_type) must be specified in the data URI. For instance, your data URI for a JPEG should start with something like `data:image/jpg;base64,`. * Data URIs must use the base64 extension. The expected format is `data:content/type;base64,`. * The size limit for your data URI (specified above) is the limit for the **encoded data URI**. Keep in mind that base64-encoding your asset increases its size by about 33%: to fit an image within the **5MB limit**, the binary file must be **3.3MB** or less. Data URIs that do not follow these rules may be rejected with a 400 Bad Request error. ## [](#inputs-using-ephemeral-uploads)Inputs using ephemeral uploads [Section titled “Inputs using ephemeral uploads”](#inputs-using-ephemeral-uploads) See [our documentation on ephemeral uploads](/assets/uploads) for more. ## [](#considerations-of-urls-data-uris-and-ephemeral-uploads)Considerations of URLs, data URIs, and ephemeral uploads [Section titled “Considerations of URLs, data URIs, and ephemeral uploads”](#considerations-of-urls-data-uris-and-ephemeral-uploads) If you do not already have your asset stored in object storage, submitting your asset with a data URI can save you a step. However, the data URI size limit may be too small for some assets. If you cannot be sure that all assets are safely within the un-encoded size limit, you should upload assets to object storage instead. Submitting a request with data URIs instead of URLs will change the performance characteristics of your integration. Data URIs are part of the request payload, and thus cause your HTTP requests to take longer to submit: all data must be sent serially to our servers. Because the request body JSON must be processed before the assets can be extracted from their data URIs, this may increase the latency of our server responses. URLs, on the other hand, can be immediately processed in parallel. This means the time between submitting your HTTP request and the start of processing is likely much lower. However, latency between our servers and the server hosting your assets may increase the total request time. Using a CDN or object storage service will help to minimize this cost. Runway [ephemeral uploads](/assets/uploads) offer the benefits of object storage (like URLs) without needing to set up your own infrastructure. Assets are uploaded from your server to Runway’s object storage, returning a `runway://` URI that can be used anywhere a HTTPS URL could be used. Importantly, these URIs can only be used for 24 hours after being created. It’s also important to note that ephemeral uploads are rate limited, and have a minimum file size of 512 bytes (0.5KB). ## [](#type-specific-requirements)Type-specific requirements [Section titled “Type-specific requirements”](#type-specific-requirements) ### [](#images)Images [Section titled “Images”](#images) For fields that accept images, the asset must use one of the following encodings, along with the corresponding `Content-Type` header: | Codec | `Content-Type` header | | ----- | --------------------------- | | JPEG | `image/jpg` or `image/jpeg` | | PNG | `image/png` | | WebP | `image/webp` | GIF images are not supported. ### [](#videos)Videos [Section titled “Videos”](#videos) For fields that accept videos, the asset must use one of the following codecs, along with the corresponding `Content-Type` header: | Container format | Usual file extension | Expected content type | Supported codecs | | ---------------- | -------------------- | --------------------- | ------------------------------------------------------------------------------- | | MP4 | `.mp4` | `video/mp4` | H.264, H.265/HEVC, AV1 | | QuickTime | `.mov` | `video/quicktime` | H.264, H.265/HEVC, Apple ProRes (422 Proxy, 422 LT, 422, 422 HQ, 4444, 4444 XQ) | | Matroska | `.mkv` | `video/x-matroska` | H.264, H.265/HEVC, VP8, VP9, AV1 | | WebM | `.webm` | `video/webm` | VP8, VP9, AV1 | | 3GPP | `.3gp` | `video/3gpp` | H.264 | | Ogg | `.ogv` | `video/ogg` | Theora | The following formats are supported but discouraged due to quality, performance, industry support, and file size: | Container format | Usual file extension | Expected content type | Supported codecs | | ---------------- | -------------------- | --------------------- | ----------------------- | | QuickTime | `.mov` | `video/quicktime` | MJPEG | | Matroska | `.mkv` | `video/x-matroska` | MPEG2 (H.262) | | AVI | `.avi` | `video/x-msvideo` | H.264, MJPEG, MSMPEG4v3 | | Flash Video | `.flv` | `video/x-flv` | FLV1, H.264 | | MPEG | `.mpg`, `.mpeg` | `video/mpeg` | MPEG2 (H.262) | Note that file extension is not considered by the API. Use this value for reference only. * **H.264 / AVC** - Most widely supported modern codec * **H.265 / HEVC** - High efficiency successor to H.264 * **AV1** - Royalty-free, modern codec * **VP8 / VP9** - Google’s codecs, primarily for WebM * **Apple ProRes** - Professional editing codec (422 Proxy, 422 LT, 422, 422 HQ, 4444, 4444 XQ) * **MPEG2 (H.262)** - Legacy codec for DVDs and broadcasts (discouraged) * **MJPEG** - Motion JPEG, frame-by-frame compression (discouraged) * **Theora** - Open codec for Ogg containers * **FLV1** - Legacy Flash Video codec (discouraged) * **MSMPEG4v3** - Legacy Microsoft codec (discouraged) ### [](#audio)Audio [Section titled “Audio”](#audio) For fields that accept audio, the asset must use one of the following codecs, along with the corresponding `Content-Type` header: | Container format | Usual file extension | Content type | Supported codecs | | ---------------- | -------------------- | ---------------------------------------- | ---------------------- | | MP3 | `.mp3` | `audio/mpeg`, `audio/mp3` | MP3 (MPEG-1/2 Layer 3) | | WAV | `.wav` | `audio/wav`, `audio/wave`, `audio/x-wav` | PCM (uncompressed) | | FLAC | `.flac` | `audio/flac`, `audio/x-flac` | FLAC (lossless) | | M4A | `.m4a` | `audio/mp4`, `audio/x-m4a` | AAC, ALAC | | AAC | `.aac` | `audio/aac`, `audio/x-aac` | AAC (raw) | * **MP3 (MPEG-1/2 Layer 3)** - Universal compatibility, lossy compression * **AAC (Advanced Audio Coding)** - Modern lossy codec, better quality than MP3 at same bitrate * **FLAC (Free Lossless Audio Codec)** - Lossless compression, popular for archival * **PCM (Pulse Code Modulation)** - Uncompressed audio, typically in WAV containers * **ALAC (Apple Lossless Audio Codec)** - Apple’s lossless codec, typically in M4A containers ## [](#aspect-ratios-and-auto-cropping-of-inputs)Aspect ratios and auto-cropping of inputs [Section titled “Aspect ratios and auto-cropping of inputs”](#aspect-ratios-and-auto-cropping-of-inputs) ### [](#video-inputs)Video inputs [Section titled “Video inputs”](#video-inputs) When using video models, be aware of the supported `ratio` parameters, which configures the dimensions of the output files. Examples: * **Gen-4.5 Text-to-Video** supports Landscape `1280:720` and Portrait `720:1280` outputs only. * **Gen-4.5 Image-to-Video** supports Landscape `1280:720` `1584:672` `1104:832`, Portrait `720:1280` `832:1104` `672:1584` and Square `960:960` outputs. * **Gen-4 Turbo** and **Act-Two** support Landscape `1280:720` `1584:672` `1104:832`, Portrait `720:1280` `832:1104` and Square `960:960` outputs. * **Gen-4 Aleph** *(Deprecated)* supports Landscape `1280:720` `1584:672` `1104:832` `848:480`, Portrait `720:1280` `832:1104` `480:848` and Square `960:960` outputs. * **Aleph 2.0** preserves the input video’s resolution (up to 1080p). Input videos must be 2–30 seconds and 30 FPS or lower. * **Gen-3 Alpha Turbo** *(Deprecated)* supports `1280:768` or `768:1280` outputs. * **Seedance 2.0** supports 24 `width:height` ratios across 480p, 720p, 1080p, and 4K. * **Seedance 2.0 Fast** supports 12 `width:height` ratios across 480p and 720p only. * **Seedance 2.0 Mini** supports 12 `width:height` ratios across 480p and 720p only. * **HappyHorse 1.0 Text-to-Video** supports 10 `width:height` ratios across 720p and 1080p (`1280:720`, `1920:1080`, `720:1280`, `1080:1920`, `960:960`, `1440:1440`, `1108:832`, `1662:1248`, `832:1108`, `1248:1662`). * **HappyHorse 1.0 Image-to-Video** uses `resolution` (`720P` / `1080P`, default `720P`) to set output quality. Output aspect ratio follows the input image. Prompt images must be at least 300px on each side. * **Gemini Omni Flash Text-to-Video** and **Image-to-Video** support Landscape `1280:720` and Portrait `720:1280` outputs only (always 720p, default `1280:720`). * **Gemini Omni Flash Video-to-Video** has no `ratio` parameter; output is 720p with orientation (landscape 16:9 or portrait 9:16) matched to the input video. Input videos can be at most 10 seconds. * **Magnific Video Upscaler** (`magnific_video_upscaler_creative`) accepts input videos up to 30 seconds. Set `resolution` to `720p`, `1k`, `2k` (default), or `4k` for target output quality. Always include `model: magnific_video_upscaler_creative` on `POST /v1/video_upscale`. If your input asset is not exactly of the `ratio` you specify, the model will auto-crop your asset from the center to the aspect ratio parameter provided. Third party models (e.g., `veo3.1`, `veo3.1_fast`, `veo3`, `happyhorse_1_0`, `gemini_omni_flash`, etc.) may crop or resize inputs in other ways. ### [](#image-references)Image references [Section titled “Image references”](#image-references) Image references may be resized if they are too large or small to be provided to the model you choose. It is not recommended to provide reference images smaller than 640×640px or larger than 4K. ### [](#input-asset-aspect-ratio-requirements)Input asset aspect ratio requirements [Section titled “Input asset aspect ratio requirements”](#input-asset-aspect-ratio-requirements) For models that accept image or video references, the asset’s aspect ratio (width ÷ height) must fall within the supported ranges below: | Model | Input type | Min aspect ratio | Max aspect ratio | | --------------------------------- | ----------------------------------- | ---------------- | ---------------- | | Gen-4 Turbo Image-to-Video | Prompt image | 0.5 | 2.358 | | Gen-4 Image-to-Video | Prompt image | 0.5 | 2.358 | | Gen-4.5 Image-to-Video | Prompt image | 0.5 | 2 | | Act-Two | Character image | 0.5 | 2.358 | | Act-Two | Character video | 0.5 | 2.358 | | Act-Two | Reference video | 0.5 | 2.358 | | Seedance 2 Image-to-Video | Prompt image (first/last/reference) | 0.4 | 4 | | Seedance 2 Text-to-Video | Reference image | 0.4 | 4 | | Seedance 2 Video-to-Video | Reference image | 0.4 | 4 | | Seedance 2.0 Fast Image-to-Video | Prompt image (first/last/reference) | 0.4 | 4 | | Seedance 2.0 Fast Text-to-Video | Reference image | 0.4 | 4 | | Seedance 2.0 Fast Video-to-Video | Reference image | 0.4 | 4 | | Seedance 2.0 Mini Image-to-Video | Prompt image (first/last/reference) | 0.4 | 4 | | Seedance 2.0 Mini Text-to-Video | Reference image | 0.4 | 4 | | Seedance 2.0 Mini Video-to-Video | Reference image | 0.4 | 4 | | Veo 3 Image-to-Video | Prompt image | 0.5 | 2 | | Veo 3.1 / 3.1 Fast Image-to-Video | First/last keyframes | 0.5 | 2 | | HappyHorse 1.0 Image-to-Video | Prompt image | 0.55 | 1.8 | | Gemini Omni Flash Image-to-Video | Prompt image | 0.5 | 2 | | Gemini 2.5 Flash Text-to-Image | Reference image | 0.25 | 4 | # API Output Formats > Understand Runway API output formats and responses. Learn how to handle generated videos, images and metadata from AI generation requests. After a task succeeds, the `GET /v1/tasks/:id` endpoint will return a response like this: ```json { "id": "d2e3d1f4-1b3c-4b5c-8d46-1c1d7ee86892", "status": "SUCCEEDED", "createdAt": "2024-06-27T19:49:32.335Z", "output": [ "https://dnznrvs05pmza.cloudfront.net/output.mp4?_jwt=..." ] } ``` The `output` member will contain one or more URLs that link to the result of your generation. It’s important to note that these URLs are ephemeral: they will expire within 24-48 hours of accessing the API. We expect you to download the data at this endpoint and save it to your own storage. Since these URLs will expire, do not expose them directly in your product. # Ephemeral uploads > Upload large files to Runway's API for processing When providing files as [inputs](/assets/inputs), there are size limits for assets provided by URL or data URI. URLs need to be downloaded and data URIs must be processed as part of the request body: both consume resources at the time a generation is started. To avoid these limits, you can upload ephemeral files to the Runway API using the uploads API. ## [](#uploading-files)Uploading files [Section titled “Uploading files”](#uploading-files) You can upload a file to Runway by using our [SDKs](/api-details/sdks/). * Node You can pass Node `fs` streams or `File`s (or [`File`-like objects](https://github.com/runwayml/sdk-node/blob/main/src/internal/to-file.ts#L37-L52)) to `createEphemeral`. ```ts import * as fs from 'node:fs'; import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const myFile = fs.createReadStream('./path/to/file.mp4'); const { uri } = await client.uploads.createEphemeral(myFile); console.log(uri); // runway://... ``` Other files can be uploaded by passing them to the `toFile` hlper from `@runwayml/sdk`. Supported types include: * `Blob` and [`Blob`-like objects](https://github.com/runwayml/sdk-node/blob/main/src/internal/to-file.ts#L11-L32) * `Buffer` * `ArrayBuffer` * Typed arrays (e.g., `Uint8Array`) * `DataView` * `Response` and [`Response`-like objects](https://github.com/runwayml/sdk-node/blob/main/src/internal/to-file.ts#L57-L66) * Any [async iterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncIterator) object that yields `string`s, `Buffer`s, `ArrayBuffer`s, `ArrayBufferView`s, `Blob`-like objects, or `DataView`s. ```ts import * as fs from 'node:fs/promises'; import RunwayML, { toFile } from '@runwayml/sdk'; const client = new RunwayML(); const { uri } = await client.uploads.createEphemeral( toFile( await fs.readFile('./path/to/file.mp4') // You must pass a filename 'file.mp4', ), ); console.log(uri); // runway://... ``` * Python You can pass the following types to `create_ephemeral`: * `pathlib.Path` object or an object that implements `os.PathLike` * An `IOBase` object (e.g., `BytesIO`) with a `name` property containing the filename ```python from pathlib import Path from runwayml import RunwayML client = RunwayML() my_file = Path('./path/to/file.mp4') response = client.uploads.create_ephemeral(file=my_file) print(response.uri) # runway://... ``` If you do not have a `Path` or `IOBase` with a name, you can pass a two-tuple in the form `(filename, content)` where `filename` is the name of the file and `content` is one of: * A file-like object (e.g., the output of `open()`) * `bytes` ```python from runwayml import RunwayML client = RunwayML() with open('./path/to/file.mp4') as my_file: response = client.uploads.create_ephemeral( file=('file.mp4', my_file), ) print(response.uri) # runway://... ``` The resulting `runway://` URI can be used anywhere that a URL or data URI can be used in the API. ### [](#considerations)Considerations [Section titled “Considerations”](#considerations) * `runway://` URIs are only valid for 24 hours. * The maximum uploadable file is 200MB. The minimum uploadable file is 512 bytes. * You must have purchased credits to use this feature. * There is a rate limit on the upload of ephemeral files. Ephemeral uploads may be used multiple times, which can conserve bandwidth. If you intend to use a file multiple times, consider using ephemeral uploads. Be aware, though, that the URI will expire 24 hours after creation and the file must be re-uploaded. ## [](#using-ephemeral-uploads-without-an-sdk)Using ephemeral uploads without an SDK [Section titled “Using ephemeral uploads without an SDK”](#using-ephemeral-uploads-without-an-sdk) Tip See the [API Reference](/api#tag/Uploads) for full details on using the `/v1/uploads` endpoint. If you use our REST API directly, you can still use ephemeral uploads. By calling `POST /v1/uploads` with the following JSON body, you can start an upload: ```json { "filename": "filename.mp4", "type": "ephemeral" } ``` * `filename`: A string containing the filename of the file you are uploading. The file extension must be representative of the file’s contents. * `type`: Must be set to `"ephemeral"`. The server will respond with the following details: ```json { "uploadUrl": "https://...", "fields": { ... }, "runwayUri": "runway://..." } ``` The `runwayUri` value is the value that may be used when creating a new generation in the Runway API. To upload the file to Runway’s servers, create a `POST` request to the URL at `uploadUrl`. Pass the fields in the dictionary `fields` as the multipart form-encoded POST body, and the contents of the file as as the `file` field. Once the upload completes successfully, the `runway://` URI is ready to use. If the upload fails, do not retry. Instead, make a new request to `/v1/uploads` and start over. Be sure to follow our guidelines for [handling errors](/errors/errors/) to ensure your integration robustly handles different kinds of failure modes. # Runway Characters > Build real-time conversational avatars powered by GWM-1, Runway's General World Model. Deploy custom characters with full control over voice, personality, knowledge, and actions. Build fully custom conversational characters powered by GWM-1, Runway’s General World Model. Generate expressive digital personas from a single image—photorealistic or animated, human or non-human—with full control over voice, personality, knowledge, and actions. No fine-tuning required. ## [](#what-you-can-build)What you can build [Section titled “What you can build”](#what-you-can-build) ### Customer support Deploy branded characters that maintain your visual identity. Use actual representatives or create stylized brand ambassadors. ### Learning & development Bring training programs to life with interactive tutors and coaches capable of extended educational conversations. ### Brand experiences Your mascots and animated characters can now hold real-time conversations. Your creative vision doesn’t stop at static content. ### Interactive characters Game hosts, dungeon masters, companions, and contextual avatars for immersive experiences. ### [](#get-started)Get started [Section titled “Get started”](#get-started) [Quickstart ](/characters/quickstart)Build your first app to call a Runway Character in 5 minutes. [Custom Characters ](/characters/create-your-own)Create your own character from a single image — no training required. [Embedded Widget ](/characters/widget)Add a Character to any website with a single script tag — no server required. [Custom Voices ](/characters/custom-voice)Design a new voice from a text prompt or clone one from an audio sample. [LiveKit Agents ](/characters/livekit)Bring your own agent — you control STT, LLM, and TTS, Runway provides the avatar video layer. [ElevenLabs Agents ](/characters/elevenlabs)Connect an ElevenLabs agent — ElevenLabs handles the conversation, Runway renders the Character. # Core Concepts > Understand the key concepts behind Runway Characters including avatars, sessions, voice configuration, and the real-time architecture. Preview chat transcripts in the Developer Portal or retrieve them with the conversations API. ## [](#avatars-and-sessions)Avatars and Sessions [Section titled “Avatars and Sessions”](#avatars-and-sessions) Understanding the distinction between Avatars and Sessions is fundamental to building with the Characters API. **Avatars** are persistent personas with a defined appearance, voice, and personality. Define your Avatar with a single reference image—any visual style works, from photorealistic humans to animated mascots to stylized brand characters. Configure voice, personality, knowledge base, and conversational actions. **Sessions** are live WebRTC connections for real-time conversation. Each Session connects a user to an Avatar for a single interaction. Sessions have a maximum duration of 5 minutes. ## [](#session-lifecycle)Session lifecycle [Section titled “Session lifecycle”](#session-lifecycle) Sessions progress through a defined set of states: ```plaintext ┌───────────┐ ┌──────────┤ NOT_READY ├──────────┐ │ └─────┬─────┘ │ │ │ │ ▼ ▼ ▼ CANCELLED READY FAILED ┌──┴──┐ │ │ ▼ ▼ RUNNING FAILED ┌──┴──┐ │ │ ▼ ▼ COMPLETED CANCELLED ``` | Status | Description | | ----------- | -------------------------------------------------------------------- | | `NOT_READY` | Session is being provisioned. Poll until ready. | | `READY` | Session is ready to connect. The `sessionKey` is available. | | `RUNNING` | WebRTC connection is active. The conversation is in progress. | | `COMPLETED` | Session ended normally after the conversation finished. | | `FAILED` | Session encountered an error. Check the `failure` field for details. | | `CANCELLED` | Session was explicitly cancelled before completion. | One-time consume Session credentials can only be retrieved once. If the WebRTC connection fails after credentials are consumed, you must create a new Session. ## [](#creating-a-character)Creating a Character [Section titled “Creating a Character”](#creating-a-character) ### [](#developer-portal)Developer Portal [Section titled “Developer Portal”](#developer-portal) The [Developer Portal](https://dev.runwayml.com/) provides a visual interface for managing Characters: 1. Go to the **Characters** tab 2. Click **Create a Character** 3. Define your Character with a single reference image 4. Configure voice, personality, knowledge base, and conversational actions 5. Preview and test before deploying The portal also lets you preview available voices and access session recordings. Portal = API Everything you see in the Developer Portal can be done via the API. The portal is built on the same public API available to you. ### [](#api)API [Section titled “API”](#api) Create and manage Avatars programmatically for automated workflows or dynamic Avatar generation: * Node ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const avatar = await client.avatars.create({ name: 'Support Agent', referenceImage: 'https://example.com/avatar.png', voice: { type: 'runway-live-preset', presetId: 'clara', }, personality: 'You are a helpful customer support agent...', }); ``` * Python ```python from runwayml import RunwayML client = RunwayML() avatar = client.avatars.create( name='Support Agent', reference_image='https://example.com/avatar.png', voice={ 'type': 'runway-live-preset', 'preset_id': 'clara', }, personality='You are a helpful customer support agent...', ) ``` See the [API reference](/api) for the complete list of Avatar management endpoints. ## [](#per-call-overrides)Per-call overrides [Section titled “Per-call overrides”](#per-call-overrides) The `personality` and `startScript` you set on an Avatar are **defaults**. When creating a Session, you can override either field to tailor the conversation for a specific caller — without modifying the Avatar itself. If you omit these fields, the Avatar’s values are used as before. This is fully backward-compatible with existing integrations. This is useful when you want to pass dynamic, user-specific context into each call. For example, a single support agent Avatar can greet every caller by name and adapt to their account details: * Node ```ts const session = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId: 'your-avatar-id' }, personality: `You are a helpful support agent for Acme Corp. The customer's name is ${user.name} and they are on the ${user.plan} plan. Address them by name. Be aware of their plan limits when answering billing questions.`, startScript: `Hi ${user.name}! I'm your Acme support assistant. How can I help you today?`, }); ``` * Python ```python session = client.realtime_sessions.create( model='gwm1_avatars', avatar={ 'type': 'custom', 'avatar_id': 'your-avatar-id' }, personality=f"""You are a helpful support agent for Acme Corp. The customer's name is {user['name']} and they are on the {user['plan']} plan. Address them by name. Be aware of their plan limits when answering billing questions.""", start_script=f"Hi {user['name']}! I'm your Acme support assistant. How can I help you today?", ) ``` | Field | Type | Description | | ------------- | ----------------- | ----------------------------------------------------------------------------- | | `personality` | string (optional) | Overrides the Avatar’s system prompt for this session. Max 10,000 characters. | | `startScript` | string (optional) | Overrides the Avatar’s opening message for this session. Max 2000 characters. | Provisioning time Sessions created with `personality` or `startScript` overrides may take slightly longer to reach the `READY` state compared to sessions that use the Avatar’s defaults. Account for this in your polling logic. ## [](#reference-image-guidelines)Reference image guidelines [Section titled “Reference image guidelines”](#reference-image-guidelines) Generate expressive Avatars from a single image with zero fine-tuning required. For best results: * **Any visual style works**: photorealistic humans, animated mascots, stylized brand characters * Use high-quality images with good lighting * Ensure the face is clearly visible and centered * Avoid images with multiple people or obstructions * Recommended aspect ratio: 1088×704 ## [](#voice-configuration)Voice configuration [Section titled “Voice configuration”](#voice-configuration) Configure your Avatar’s voice using voice presets. Avatars support full conversational expressiveness including natural speech patterns and lip-syncing. Here are some examples: | Preset ID | Name | Style | | ---------- | -------- | ---------------------------- | | `clara` | Clara | Soft, approachable | | `victoria` | Victoria | Firm, professional | | `vincent` | Vincent | Knowledgeable, authoritative | * Node ```ts voice: { type: 'runway-live-preset', presetId: 'clara', } ``` * Python ```python voice={ 'type': 'runway-live-preset', 'preset_id': 'clara', } ``` Preview all available voices in the [Developer Portal](https://dev.runwayml.com/). You can also [design a custom voice from a text prompt or clone one from an audio sample](/characters/custom-voice). ## [](#conversation-transcripts-and-recordings)Conversation transcripts and recordings [Section titled “Conversation transcripts and recordings”](#conversation-transcripts-and-recordings) After a conversation ends, you can review the **transcript** in the [Developer Portal](https://dev.runwayml.com/) by opening your Character and browsing past conversations. If you created the call with `POST /v1/realtime_sessions`, the returned session ID is also the conversation ID you use with the conversations API later. In other words, the `sessionId` from session creation becomes the `conversationId` for transcript and recording retrieval. To retrieve or export transcripts programmatically, call the conversations endpoints on the public API (response shapes and SDK examples are in the [API reference](/api)): ```http GET /v1/avatars/{id}/conversations GET /v1/avatars/{id}/conversations/{conversationId} ``` Use the list endpoint to browse past sessions for a character. Use the conversation detail endpoint to fetch the full `transcript` and, when available, a `recordingUrl` for downloading the conversation recording. * Node ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const avatarId = '550e8400-e29b-41d4-a716-446655440000'; const { id: sessionId } = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId }, }); const conversation = await client.avatars.conversations.retrieve( avatarId, sessionId ); console.log(conversation.transcript); console.log(conversation.recordingUrl); ``` * Python ```python from runwayml import RunwayML client = RunwayML() avatar_id = '550e8400-e29b-41d4-a716-446655440000' session = client.realtime_sessions.create( model='gwm1_avatars', avatar={ 'type': 'custom', 'avatar_id': avatar_id }, ) conversation = client.avatars.conversations.retrieve( avatar_id, session.id ) print(conversation.transcript) print(conversation.recording_url) ``` The `recordingUrl` is temporary. If it has expired, fetch the conversation again to get a fresh download URL. # Create Your Own Characters > Create your own Runway character with just one image and video call it in 5 minutes. In this tutorial, we will create our own character and do a video call with it in a React app. We only need one image of the character, and we can start calling the character immediately, no training required. [Create your own characters tutorial](https://www.youtube.com/embed/mXsi2ViqWP0) **Helpful links:** * [Runway developer account](https://dev.runwayml.com) * [Runway avatar SDK template](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-simple) ### [](#1-create-a-developer-account)1. Create a developer account [Section titled “1. Create a developer account”](#1-create-a-developer-account) Create an account at [dev.runwayml.com](https://dev.runwayml.com). ![Dev portal login page](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/1.png) Once you log in, you will see an “Characters” tab at the top bar. ### [](#2-create-your-own-character)2. Create your own character [Section titled “2. Create your own character”](#2-create-your-own-character) Click on the “Create a Character” button. ![Characters tab with Create a Character button](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/2.png) ![Avatar creation form](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/10.png) #### [](#2-1-upload-a-character-image)2-1. Upload a character image [Section titled “2-1. Upload a character image”](#2-1-upload-a-character-image) Here are some tips for choosing a good character image: * Use high-quality, front-facing photos with good lighting * Ensure the face is clearly visible and centered * Avoid images with multiple people or obstructions * Recommended: 16:9 aspect ratio ![Upload character image](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/11.png) #### [](#2-2-choose-a-voice)2-2. Choose a voice [Section titled “2-2. Choose a voice”](#2-2-choose-a-voice) You can click the “play” button to hear each voice option, then select the one you like. There are also options to customize the voice. #### [](#2-3-input-instructions)2-3. Input instructions [Section titled “2-3. Input instructions”](#2-3-input-instructions) Input “Instructions” for your character. For example: “Your name is xxx, you are a customer service support for a company called xxx. You help users with their questions.” ![Instructions input](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/12.png) #### [](#2-4-input-a-starting-script-optional)2-4. Input a starting script (optional) [Section titled “2-4. Input a starting script (optional)”](#2-4-input-a-starting-script-optional) If you’d like to tell your character to say something at the beginning of every conversation, input a “Starting script”. For example: “Hello, my name is xxx, how can I help you?” Per-call overrides The instructions and starting script you set here become the defaults for every session. You can override them per call to inject user-specific context — for example, the caller’s name or account details. See [Per-call overrides](/characters/concepts#per-call-overrides) for details. #### [](#2-5-upload-knowledge-optional)2-5. Upload knowledge (optional) [Section titled “2-5. Upload knowledge (optional)”](#2-5-upload-knowledge-optional) If you have longer text files that your character should know about, feel free to upload a `.txt` file there. For example, a `.txt` file of the product information that your character should know about. #### [](#2-6-create-the-character)2-6. Create the character [Section titled “2-6. Create the character”](#2-6-create-the-character) Click on “Create Character”. You will see a character page like this — copy the “Avatar ID”. It is a UUID that has 32 hex digits (e.g., `8be4df61-93ca-11d2-aa0d-00e098032b8c`). ![Character page showing Avatar ID](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/13.png) ### [](#3-use-the-custom-character-in-your-react-app)3. Use the custom character in your React app [Section titled “3. Use the custom character in your React app”](#3-use-the-custom-character-in-your-react-app) If you haven’t already, check out the [previous Quickstart tutorial](/characters/quickstart) or [React app template](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-simple) and follow the instructions in the README to run the app. Once you get the app running, update the code in two places: **1.** Go to `my-avatar-app/app/page.tsx` and update `MY_AVATAR`’s `id` to the ID of the avatar you just created. Optionally, you can also update the `name` or `imageUrl`. ```typescript const MY_AVATAR = { id: "8be4df61-93ca-11d2-aa0d-00e098032b8c", name: "Yining", imageUrl: "https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/tutorial/yining-character.jpeg", }; ``` **2.** Go to `my-avatar-app/app/api/avatar/connect/route.ts` and update the avatar type to `"custom"`. ```typescript const avatar = { type: "custom" as const, avatarId: avatarId }; ``` Now if you run the app again, you should see your own character on the page and you can start a conversation with it. ![Custom character in the app](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/14.png) ![Active video call with custom character](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/15.png) *** That’s it! There are so many applications for custom characters — we’re excited for you to create your own. Please reach out to us if you have any questions via [Runway support](https://help.runwayml.com/hc/en-us/requests/new). ### [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Add Knowledge to Your Character ](/characters/documents)Upload documents to give your character domain-specific expertise. # Custom voices > Design a new voice from a text prompt or clone a voice from an audio sample. Assign custom voices to your Characters for a unique, branded sound. The Voices API lets you create custom voices for your Characters. Design an entirely new voice from a text prompt, or clone a voice from an audio sample. Once created, assign the voice to any Avatar. ## [](#voice-design)Voice design [Section titled “Voice design”](#voice-design) Create a voice by describing the characteristics you want. The prompt should include details like tone, accent, pacing, and personality. * Node ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const voice = await client.voices.create({ name: 'Brand Ambassador', from: { type: 'text', prompt: 'A warm, friendly voice with a slight British accent. Speaks at a measured pace with a professional yet approachable tone.', model: 'eleven_ttv_v3', }, }); console.log('Voice created:', voice.id); ``` * Python ```python from runwayml import RunwayML client = RunwayML() voice = client.voices.create( name='Brand Ambassador', from_={ 'type': 'text', 'prompt': 'A warm, friendly voice with a slight British accent. Speaks at a measured pace with a professional yet approachable tone.', 'model': 'eleven_ttv_v3', }, ) print('Voice created:', voice.id) ``` | Parameter | Type | Description | | ------------- | -------- | ------------------------------------------------------------------------------------- | | `name` | string | A name for the voice (max 100 characters). | | `from.type` | `"text"` | Indicates voice design from a text prompt. | | `from.prompt` | string | A description of the desired voice characteristics. Must be at least 20 characters. | | `from.model` | string | The voice design model. Use `eleven_ttv_v3` (latest) or `eleven_multilingual_ttv_v2`. | ## [](#voice-cloning)Voice cloning [Section titled “Voice cloning”](#voice-cloning) Clone a voice from an audio sample. Provide a clear recording with minimal background noise and varied tone for best results. * Node ```ts const voice = await client.voices.create({ name: 'Cloned Narrator', from: { type: 'audio', audio: 'https://example.com/voice-sample.mp3', }, }); console.log('Voice created:', voice.id); ``` * Python ```python voice = client.voices.create( name='Cloned Narrator', from_={ 'type': 'audio', 'audio': 'https://example.com/voice-sample.mp3', }, ) print('Voice created:', voice.id) ``` The audio sample must be between 10 seconds and 5 minutes long, and at most 10 MB. You can pass a public HTTPS URL, a `runway://` upload URI, or a `data:audio/...` data URI. ## [](#voice-status)Voice status [Section titled “Voice status”](#voice-status) Voice creation is asynchronous. After calling `create`, poll the voice until its status is `READY`. * Node ```ts const voice = await client.voices.retrieve(voiceId); if (voice.status === 'READY') { console.log('Preview:', voice.previewUrl); } ``` * Python ```python voice = client.voices.retrieve(id=voice_id) if voice.status == 'READY': print('Preview:', voice.preview_url) ``` | Status | Description | | ------------ | -------------------------------------------- | | `PROCESSING` | Voice is being generated. Poll until ready. | | `READY` | Voice is ready. A `previewUrl` is available. | | `FAILED` | Generation failed. Check `failureReason`. | ## [](#assigning-a-custom-voice-to-an-avatar)Assigning a custom voice to an Avatar [Section titled “Assigning a custom voice to an Avatar”](#assigning-a-custom-voice-to-an-avatar) Once the voice is ready, assign it to an Avatar by setting the voice type to `custom` and providing the voice ID. * Node ```ts await client.avatars.update(avatarId, { voice: { type: 'custom', id: voice.id, }, }); ``` * Python ```python client.avatars.update( avatar_id, voice={ 'type': 'custom', 'id': voice.id, }, ) ``` You can also create a new Avatar with a custom voice directly: * Node ```ts const avatar = await client.avatars.create({ name: 'Support Agent', referenceImage: 'https://example.com/avatar.png', voice: { type: 'custom', id: voice.id, }, personality: 'You are a helpful customer support agent...', }); ``` * Python ```python avatar = client.avatars.create( name='Support Agent', reference_image='https://example.com/avatar.png', voice={ 'type': 'custom', 'id': voice.id, }, personality='You are a helpful customer support agent...', ) ``` ## [](#listing-voices)Listing voices [Section titled “Listing voices”](#listing-voices) Retrieve all custom voices for your organization. * Node ```ts const voices = await client.voices.list(); for await (const voice of voices) { console.log(voice.name, voice.status); } ``` * Python ```python for voice in client.voices.list(): print(voice.name, voice.status) ``` You can also manage custom voices through the [Developer Portal](https://dev.runwayml.com/). See the [API reference](/api) for all available endpoints. # Knowledge base > Give your Avatars domain-specific knowledge using Documents. Upload content that Avatars can reference during conversations. The Documents API lets you give Avatars access to domain-specific knowledge. Upload content that your Avatar can reference during conversations to provide accurate, contextual responses. Token limit Each Avatar supports up to 50,000 tokens of knowledge. We plan to increase this limit in the future. ## [](#why-use-a-knowledge-base)Why use a knowledge base [Section titled “Why use a knowledge base”](#why-use-a-knowledge-base) A knowledge base helps your Avatar stay on topic and provide accurate information. Common use cases: * **Customer support**: FAQs, product information, company policies * **Quizzes and games**: Question banks, correct answers, scoring rules * **Education**: Course material, reference content, learning objectives * **Brand experiences**: Brand guidelines, messaging, product details ## [](#supported-content)Supported content [Section titled “Supported content”](#supported-content) | Format | Description | | ---------- | ----------------------------------------------- | | Plain text | Unformatted text content | | Markdown | Structured content with headings and formatting | More formats are planned for future releases. ## [](#adding-knowledge-to-an-avatar)Adding knowledge to an Avatar [Section titled “Adding knowledge to an Avatar”](#adding-knowledge-to-an-avatar) The flow is: create a Document, then link it to your Avatar. ### [](#1-create-a-document)1. Create a Document [Section titled “1. Create a Document”](#1-create-a-document) * Node ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const document = await client.documents.create({ name: 'Product FAQ', content: '# Product FAQ\n\n## What is your return policy?\n\nWe offer a 30-day return policy...', }); console.log('Document created:', document.id); ``` * Python ```python from runwayml import RunwayML client = RunwayML() document = client.documents.create( name='Product FAQ', content='# Product FAQ\n\n## What is your return policy?\n\nWe offer a 30-day return policy...', ) print('Document created:', document.id) ``` ### [](#2-update-a-document)2. Update a Document [Section titled “2. Update a Document”](#2-update-a-document) You can update a Document’s name, content, or both using the `update` method. * Node ```ts await client.documents.update(document.id, { name: 'Updated Product FAQ', content: '# Product FAQ\n\n## What is your return policy?\n\nWe now offer a 60-day return policy...', }); ``` * Python ```python client.documents.update( id=document.id, name='Updated Product FAQ', content='# Product FAQ\n\n## What is your return policy?\n\nWe now offer a 60-day return policy...', ) ``` Both fields are optional — provide only the fields you want to change. ### [](#3-link-the-document-to-your-avatar)3. Link the Document to your Avatar [Section titled “3. Link the Document to your Avatar”](#3-link-the-document-to-your-avatar) Update your Avatar to attach the Document. This replaces any existing Document attachments. * Node ```ts await client.avatars.update(avatarId, { documentIds: [document.id], }); ``` * Python ```python client.avatars.update( avatar_id, document_ids=[document.id], ) ``` ### [](#4-start-a-session)4. Start a Session [Section titled “4. Start a Session”](#4-start-a-session) The Avatar now has access to the knowledge during conversations. Start a Session as usual: * Node ```ts const session = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId: avatarId, }, }); ``` * Python ```python session = client.realtime_sessions.create( model='gwm1_avatars', avatar={ 'type': 'custom', 'avatar_id': avatar_id, }, ) ``` You can also manage Documents through the [Developer Portal](https://dev.runwayml.com/). See the [API reference](/api) for all available endpoints. # ElevenLabs Agents > Use Runway Characters as the visual layer for your ElevenLabs agent. You own STT, LLM, and TTS in ElevenLabs — Runway renders lip-synced Character video. Use Runway Characters with an [ElevenLabs agent](https://elevenlabs.io/docs/eleven-agents/overview) when you already have one and want a custom Character to deliver it. ElevenLabs handles speech recognition, reasoning, and text-to-speech; Runway lip-syncs that audio to your Character over WebRTC and does not run its own conversation model for the session. ## [](#before-you-start)Before you start [Section titled “Before you start”](#before-you-start) You’ll need: * A [Runway API key](https://dev.runwayml.com/settings/api-keys) * A **custom Character** your API key owns and is ready to use * An [ElevenLabs API key](https://elevenlabs.io/app/settings/api-keys) with ElevenAgents read permission * An [ElevenLabs agent](https://elevenlabs.io/app/agents/agents) with **Advanced → User input audio format → PCM 16000Hz** Set these server-side environment variables (see the [Next.js example](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-elevenlabs) for a full `.env.example`): ```bash RUNWAYML_API_SECRET=... RUNWAY_AVATAR_ID=... ELEVENLABS_API_KEY=... ELEVENLABS_AGENT_ID=... ``` Server-side only Keep your Runway API secret and ElevenLabs API key on the server. The browser uses the Characters SDK for WebRTC only — no ElevenLabs SDK, and the signed WebSocket URL never goes to the client. ## [](#guide)Guide [Section titled “Guide”](#guide) 1. **Install the Characters SDK** ```bash npm install @runwayml/avatars @runwayml/avatars-react ``` Server helpers such as `createElevenLabsSession` live in `@runwayml/avatars/api`. The React package re-exports them for convenience, but your API route should import from the core package. 2. **Create a server route** `createElevenLabsSession` from `@runwayml/avatars/api` fetches an [ElevenLabs signed URL](https://elevenlabs.io/docs/eleven-agents/customization/authentication), creates a Runway session with `integration: { type: "elevenlabs", signedUrl }`, polls until `READY`, and returns `sessionId`, `sessionKey`, `avatarId`, and `baseUrl` for `AvatarCall`. app/api/avatar/connect/route.ts ```ts // app/api/avatar/connect/route.ts import { createElevenLabsSession } from '@runwayml/avatars/api'; export async function POST() { const session = await createElevenLabsSession({ runwayApiSecret: process.env.RUNWAYML_API_SECRET!, avatarId: process.env.RUNWAY_AVATAR_ID!, elevenLabsApiKey: process.env.ELEVENLABS_API_KEY!, elevenLabsAgentId: process.env.ELEVENLABS_AGENT_ID!, }); return Response.json(session); } ``` Also exported from `@runwayml/avatars-react/api` if you already depend on the React package only. Session create constraints Use a custom `avatarId` your key can access. Do not pass `personality`, `startScript`, or `tools` — the ElevenLabs agent owns conversation config. Without the SDK, run the same three steps manually: fetch the signed URL, create the session, poll until ready. See [Building your integration](/characters/integration) for session lifecycle patterns. 3. **Connect from the client** Pass the JSON from your connect route to `AvatarCall`: ```tsx 'use client'; import { AvatarCall } from '@runwayml/avatars-react'; import '@runwayml/avatars-react/styles.css'; export function Conversation({ sessionId, sessionKey, avatarId, baseUrl }) { return ( ); } ``` `createElevenLabsSession` echoes `avatarId` and `baseUrl` so you can forward the server response without rebuilding those fields. 4. **Test it** Trigger your connect route, then confirm Character video appears while your ElevenLabs agent responds. Mic audio goes Runway → worker → ElevenLabs; agent audio drives lip sync on the way back. Full working app: [Next.js + ElevenLabs example](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-elevenlabs). ## [](#end-sessions-promptly)End sessions promptly [Section titled “End sessions promptly”](#end-sessions-promptly) Runway bills realtime Character sessions while the Character worker is active. End calls via `AvatarCall` `onEnd` and handle errors on your connect route so sessions are not left running. ## [](#troubleshooting)Troubleshooting [Section titled “Troubleshooting”](#troubleshooting) | Symptom | Fix | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------- | | `404 Could not find Avatar` | Use a Character ID scoped to the same API key (see [Custom Avatars](/characters/create-your-own)) | | `401` or inactive API key | Check your Runway API key in the [Developer Portal](https://dev.runwayml.com/settings/api-keys) | | Session stuck `NOT_READY` | Retry; confirm your environment has the ElevenLabs integration deployed | | Silent agent / no audio | Set **PCM 16000Hz** input format on the ElevenLabs agent | | `personality… cannot be used with integration` | Remove `personality`, `startScript`, and `tools` from the session create request | | ElevenLabs signed URL fails | Check `xi-api-key`, agent ID, and ElevenAgents read permission | See [Troubleshooting](/characters/troubleshooting) for general Characters SDK debugging. ## [](#learn-more)Learn more [Section titled “Learn more”](#learn-more) [Next.js + ElevenLabs example ](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-elevenlabs)Full working app with createElevenLabsSession and AvatarCall. [ElevenLabs agent authentication ](https://elevenlabs.io/docs/eleven-agents/customization/authentication)Signed URLs and server-side API key handling for ElevenLabs agents. [LiveKit Agents ](/characters/livekit)Bring your own agent via LiveKit instead of ElevenLabs. [Building your integration ](/characters/integration)Default Runway-owned sessions and custom React UI patterns. # Building your integration > Learn how to integrate Runway Avatars into your application with server-side session management and React components. This guide walks through building a complete Avatar integration using Next.js App Router. The same patterns apply to other React frameworks. Keep your API key secure Your `RUNWAYML_API_SECRET` must never be exposed to the client. Always create Sessions server-side. If your key is compromised, rotate it immediately in the [Developer Portal](https://dev.runwayml.com/settings/api-keys). ## [](#architecture-overview)Architecture overview [Section titled “Architecture overview”](#architecture-overview) Avatar Sessions require a server component to keep your API key secure. The client never sees your Runway API secret. ```plaintext ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Client │ │ Your Server │ │ Runway API │ │ (React App) │ │ (Next.js) │ │ │ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │ │ │ │ 1. Request Session │ │ │ POST /api/avatar/session │ │ │ ─────────────────────────►│ │ │ │ │ │ │ 2. Create Session │ │ │ POST /v1/realtime_sessions │ │ ─────────────────────────►│ │ │ │ │ │ 3. Poll until ready │ │ │ GET /v1/realtime_sessions/:id │ │ ─────────────────────────►│ │ │ │ │ │ 4. Consume credentials │ │ │ POST /v1/realtime_sessions/:id/consume │ │ ─────────────────────────►│ │ │ │ │ 5. Return credentials │◄───────────────────────── │ │◄───────────────────────── │ │ │ │ │ │ 6. WebRTC connection │ │ │ ─────────────────────────────────────────────────────►│ │ │ │ ``` ## [](#installation)Installation [Section titled “Installation”](#installation) Install the Runway server SDK and React client: ```bash npm install @runwayml/sdk @runwayml/avatars-react ``` `@runwayml/avatars-react` includes the framework-agnostic core (`@runwayml/avatars`) automatically. ## [](#server-setup)Server setup [Section titled “Server setup”](#server-setup) Create an API route that handles Session creation. This endpoint receives an Avatar ID from the client, creates a Session with Runway, polls until it’s ready, consumes the credentials, and returns them to the client. The client can also pass optional `personality` and `startScript` fields to [override the Avatar’s defaults](/characters/concepts#per-call-overrides) for this session — useful for injecting user-specific context like the caller’s name. app/api/avatar/session/route.ts ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); export async function POST(request: Request) { const { avatarId, personality, startScript } = await request.json(); // 1. Create session // To add tool calling, pass a tools array here — see /characters/tools const { id: sessionId } = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId }, personality, startScript, }); // 2. Poll until ready let sessionKey: string | undefined; for (let i = 0; i < 60; i++) { const session = await client.realtimeSessions.retrieve(sessionId); if (session.status === 'READY') { sessionKey = session.sessionKey; break; } if (session.status === 'FAILED') { return Response.json({ error: session.failure }, { status: 500 }); } await new Promise(r => setTimeout(r, 1000)); } if (!sessionKey) { return Response.json({ error: 'Session timed out' }, { status: 504 }); } // 3. Consume session to get connection credentials const consumeResponse = await fetch( `${client.baseURL}/v1/realtime_sessions/${sessionId}/consume`, { method: 'POST', headers: { Authorization: `Bearer ${sessionKey}`, 'X-Runway-Version': '2024-11-06', }, } ); const credentials = await consumeResponse.json(); return Response.json({ sessionId, serverUrl: credentials.url, token: credentials.token, roomName: credentials.roomName, }); } ``` Save the session ID The `sessionId` returned from `client.realtimeSessions.create()` is also the `conversationId` for that call. Persist it with your own call record if you want to later fetch `transcript` or `recordingUrl` from `GET /v1/avatars/{id}/conversations/{conversationId}`. Set your API key as an environment variable: .env.local ```bash RUNWAYML_API_SECRET=your_api_key_here ``` ## [](#client-integration)Client integration [Section titled “Client integration”](#client-integration) ### [](#simple-avatarcall)Simple: AvatarCall [Section titled “Simple: AvatarCall”](#simple-avatarcall) The simplest way to add an Avatar is with the `AvatarCall` component. It handles WebRTC connection and renders a default UI. app/page.tsx ```tsx 'use client'; import { AvatarCall } from '@runwayml/avatars-react'; import '@runwayml/avatars-react/styles.css'; export default function Home() { return ( console.log('Call ended')} onError={(error) => console.error('Error:', error)} /> ); } ``` To use a custom Avatar created in the Developer Portal, replace `"customer-service"` with your Avatar ID. ### [](#webcam-and-screen-sharing)Webcam and screen sharing [Section titled “Webcam and screen sharing”](#webcam-and-screen-sharing) During a call, the Avatar can use **your webcam** and/or **your screen** as visual context—for example walkthroughs, slides, or showing something in frame. Enable that with [`@runwayml/avatars-react`](https://github.com/runwayml/avatars-sdk-react/blob/main/README.md); webcam and screen share are part of the **same realtime Session** as audio (the usual WebRTC call). Custom voice Custom Avatars that use a **custom voice** do not support webcam or screen sharing. Minimal example with the default control bar and screen sharing enabled: ```tsx // app/page.tsx — webcam (default) + optional screen share UI 'use client'; import { AvatarCall, AvatarVideo, ControlBar, ScreenShareVideo, } from '@runwayml/avatars-react'; import '@runwayml/avatars-react/styles.css'; export default function Home() { return ( ); } ``` ### [](#fully-custom-hooks)Fully custom: hooks [Section titled “Fully custom: hooks”](#fully-custom-hooks) For complete control over the UI, use `AvatarSession` with hooks. This example shows how to build a custom interface with `useAvatarSession` for Session state and `useLocalMedia` for mic and webcam controls: components/CustomAvatarUI.tsx ```tsx 'use client'; import { AvatarSession, AvatarVideo, UserVideo, useAvatarSession, useLocalMedia, } from '@runwayml/avatars-react'; import type { SessionCredentials } from '@runwayml/avatars-react'; function CallUI() { const { state, end } = useAvatarSession(); const { isMicEnabled, toggleMic } = useLocalMedia(); return (
{state === 'connecting' &&
Connecting...
}
); } export function CustomAvatar({ credentials }: { credentials: SessionCredentials }) { return ( ); } ``` For more components, hooks, and examples—including the full **Webcam & Screen Sharing** section—see the [React SDK README](https://github.com/runwayml/avatars-sdk-react/blob/main/README.md). For **client tools** (UI tool calls) and **server tools** (server-side tools with return values), see [Tool calling](/characters/tools). ## [](#browser-support)Browser support [Section titled “Browser support”](#browser-support) The Avatars SDK uses WebRTC for real-time communication: | Browser | Minimum Version | | ------- | --------------- | | Chrome | 74+ | | Firefox | 78+ | | Safari | 14.1+ | | Edge | 79+ | Users must grant microphone permissions when prompted. Webcam access is required if the user’s video is enabled. *** ### [](#not-using-react)Not using React? [Section titled “Not using React?”](#not-using-react) Most teams use **`@runwayml/avatars-react`** (above). If you are on Svelte, Vue, or plain HTML, use **`@runwayml/avatars`** — the same Session flow as this guide ([server setup](#server-setup)), without React. ```bash npm install @runwayml/sdk @runwayml/avatars ``` #### [](#quick-start)Quick start [Section titled “Quick start”](#quick-start) Your server route stays the same. In the browser, `streamTo` runs the consume call, joins the LiveKit room, and attaches avatar video to an element: ```javascript import { streamTo, AvatarEvent } from '@runwayml/avatars'; const credentials = await fetch('/api/avatar/session', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ avatarId: 'music-superstar' }), }).then((r) => r.json()); const session = await streamTo({ credentials, target: document.getElementById('avatar'), }); session.on(AvatarEvent.Transcript, (entry) => { console.log(entry.role, entry.text); }); document.getElementById('mute')?.addEventListener('click', () => session.mic.toggle()); document.getElementById('end')?.addEventListener('click', () => session.end()); ``` Use **`connect({ credentials })`** when you want a headless session first, then **`session.streamTo(element)`** once you have a video target. #### [](#examples)Examples [Section titled “Examples”](#examples) * [vanilla-js](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/vanilla-js) — single HTML page + Express * [sveltekit](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/sveltekit) — `streamTo` in a Svelte component Full API detail: [`@runwayml/avatars` README](https://github.com/runwayml/avatars-sdk-react/tree/main/packages/core). # LiveKit Agents > Use Runway Characters as the visual layer for your own LiveKit Agent. You own the conversational pipeline (STT, LLM, TTS) and Runway provides the lip-synced avatar video. Use Runway Characters with [LiveKit Agents](https://docs.livekit.io/agents/) to build fully custom conversational experiences where you control the entire pipeline. Your agent handles speech-to-text, language model, and text-to-speech. Runway provides the visual layer: audio in, avatar video out. ## [](#before-you-start)Before you start [Section titled “Before you start”](#before-you-start) You’ll need: * A [Runway API key](https://dev.runwayml.com/settings/api-keys) * A [LiveKit Cloud](https://cloud.livekit.io) project (or self-hosted LiveKit server) * A [Google Gemini API key](https://ai.google.dev/gemini-api/docs/api-key) (or another LLM/TTS provider) * A preset ID (e.g. `cat-character`) or custom Avatar ID from the [Developer Portal](https://dev.runwayml.com) ## [](#guide)Guide [Section titled “Guide”](#guide) 1. **Install the plugin** * Python ```bash pip install livekit-plugins-runway ``` * Node ```bash npm install @livekit/agents-plugin-runway ``` Set the following in your `.env` file: ```bash RUNWAYML_API_SECRET=... LIVEKIT_URL=... LIVEKIT_API_KEY=... LIVEKIT_API_SECRET=... GOOGLE_API_KEY=... ``` 2. **Add AvatarSession to your agent** * Python agent\_worker.py ```python from dotenv import load_dotenv from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli from livekit.plugins import google, runway load_dotenv() server = AgentServer() @server.rtc_session() async def entrypoint(ctx: JobContext): session = AgentSession( llm=google.realtime.RealtimeModel(voice="kore"), ) avatar = runway.AvatarSession( preset_id="cat-character", ) await avatar.start(session, room=ctx.room) await session.start( agent=Agent(instructions="Talk to me!"), room=ctx.room, ) session.generate_reply(instructions="Say hello to the user.") if __name__ == "__main__": cli.run_app(server) ``` * Node agent\_worker.ts ```ts import { type JobContext, ServerOptions, cli, defineAgent, voice } from '@livekit/agents'; import * as google from '@livekit/agents-plugin-google'; import * as runway from '@livekit/agents-plugin-runway'; import { fileURLToPath } from 'node:url'; export default defineAgent({ entry: async (ctx: JobContext) => { await ctx.connect(); const session = new voice.AgentSession({ llm: new google.beta.realtime.RealtimeModel({ voice: 'Kore' }), }); const avatar = new runway.AvatarSession({ presetId: 'cat-character', }); await avatar.start(session, ctx.room); await session.start({ agent: new voice.Agent({ instructions: 'Talk to me!' }), room: ctx.room, outputOptions: { syncTranscription: false }, }); session.generateReply({ instructions: 'Say hello to the user.' }); }, }); cli.runApp(new ServerOptions({ agent: fileURLToPath(import.meta.url) })); ``` Use `avatar_id` / `avatarId` instead of `preset_id` / `presetId` to use a custom Character from the [Developer Portal](https://dev.runwayml.com). Note The agent’s TTS drives what the Avatar says, so any voice or personality configured on the Runway Character is bypassed. Runway receives the already-synthesized audio and lip-syncs it to the Character. See the [LiveKit Runway plugin guide](https://docs.livekit.io/agents/models/avatar/plugins/runway/) for the full list of `AvatarSession` parameters. 3. **Test it** Open the [LiveKit Agents Playground](https://docs.livekit.io/agents/start/playground/) to preview your agent without building a frontend. Start a conversation and verify the avatar video track appears alongside your agent’s audio. ## [](#end-sessions-promptly)End sessions promptly [Section titled “End sessions promptly”](#end-sessions-promptly) Runway bills realtime Character sessions while the Runway avatar worker is active. The plugin cancels the Runway realtime session during normal LiveKit job shutdown, so make sure your agent shutdown path runs when the user leaves, your agent disconnects, or your app ends the conversation. Set `max_duration` / `maxDuration` (seconds) in the `AvatarSession` constructor to cap session length. If the job is force-killed before cleanup runs, the Runway session can continue until this limit. ## [](#handle-startup-errors)Handle startup errors [Section titled “Handle startup errors”](#handle-startup-errors) `AvatarSession.start()` can fail before the Character joins the LiveKit room, for example if the Runway project has insufficient credits or the session request is invalid. Catch startup errors in your agent and send an application-level message to your frontend so the user does not wait indefinitely for the avatar video track. * Python ```python try: await avatar.start(session, room=ctx.room) except Exception as exc: print(f"failed to start Runway avatar: {exc}") raise ``` * Node ```ts try { await avatar.start(session, ctx.room); } catch (error) { console.error('failed to start Runway avatar', error); throw error; } ``` ## [](#learn-more)Learn more [Section titled “Learn more”](#learn-more) [LiveKit Runway plugin guide ](https://docs.livekit.io/agents/models/avatar/plugins/runway/)LiveKit's integration guide for the Runway Characters plugin. [LiveKit Agents documentation ](https://docs.livekit.io/agents/)Full reference for the LiveKit Agents framework: models, plugins, room management, and deployment. [Agents Playground ](https://docs.livekit.io/agents/start/playground/)Test your agent in the browser without building a frontend. [Python plugin source ](https://github.com/livekit/agents/tree/main/livekit-plugins/livekit-plugins-runway)livekit-plugins-runway in the livekit/agents monorepo. [Node plugin source ](https://github.com/livekit/agents-js/tree/main/plugins/runway)@livekit/agents-plugin-runway in the livekit/agents-js monorepo. [ElevenLabs Agents ](/characters/elevenlabs)Bring your own ElevenLabs agent instead of a LiveKit agent. # Quickstart > Build a React app that video calls a Runway character in 5 minutes. In this tutorial, we will build a React web app that video calls a Runway character — in under 5 minutes. [Quickstart tutorial](https://www.youtube.com/embed/IRxGEaap4Wc) **Helpful links:** * [Runway developer account](https://dev.runwayml.com) * [Runway avatar SDK template](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-simple) ### [](#1-create-a-developer-account)1. Create a developer account [Section titled “1. Create a developer account”](#1-create-a-developer-account) Create an account at [dev.runwayml.com](https://dev.runwayml.com). ![Dev portal login page](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/1.png) Once you log in, you will see an “Characters” tab at the top bar, and there are a few preset characters. We are going to video call the character called “Tooth”. ![Characters tab showing Mina character](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/2.png) ### [](#2-create-a-new-api-key)2. Create a new API key [Section titled “2. Create a new API key”](#2-create-a-new-api-key) Go to the **Manage** tab in the top bar, then click the **New API key** button in the top-right corner. ![Manage tab with New API key button](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/3.png) Give your key a name and copy it to a safe location. Once you close the pop-up, the key value is not available again. You can always create a new key if needed. ![API key name input](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/4.png) ![API key copy dialog](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/5.png) ### [](#3-add-credits)3. Add credits [Section titled “3. Add credits”](#3-add-credits) Click on **Billing** in the left sidebar under the **Manage** tab, and add some credits to the account. ![Billing page](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/6.png) ![Adding credits](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/7.png) ### [](#4-download-the-react-app-template)4. Download the React app template [Section titled “4. Download the React app template”](#4-download-the-react-app-template) In your terminal, run this command to copy the [template](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-simple) into your local directory: ```bash npx degit runwayml/avatars-sdk-react/examples/nextjs-simple my-avatar-app cd my-avatar-app ``` ### [](#5-install-packages)5. Install packages [Section titled “5. Install packages”](#5-install-packages) Make sure you are using **Node.js 18+**, then install dependencies: ```bash npm install ``` ### [](#6-set-your-api-key)6. Set your API key [Section titled “6. Set your API key”](#6-set-your-api-key) Copy the `.env.example` file as `.env`, and paste your API key: ```plaintext RUNWAYML_API_SECRET=your_api_key_here ``` ### [](#7-run-the-app)7. Run the app [Section titled “7. Run the app”](#7-run-the-app) ```bash npm run dev ``` The server starts at [http://localhost:3000](http://localhost:3000/). Click on the Mina character to start a conversation. ![Character selection screen](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/8.png) ![Active video call](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/9.png) ### [](#troubleshooting)Troubleshooting [Section titled “Troubleshooting”](#troubleshooting) * **API key errors:** * Make sure you copy the entire API key correctly. The key begins with `key_` followed by 128 hex characters. * Make sure the key is currently active. Deactivated keys will be rejected. * **No credits:** Make sure the developer account has credits before starting a call. *** ### [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Create Your Own Characters ](/characters/create-your-own)Create your own character from a single image — no training required. # Screen Sharing and Camera Feed > Share your webcam or screen with a Runway Character so it can see and respond to what is on camera in real time. Runway Characters can take a live **webcam** or **screen share** from your app. The Character sees that feed over the Session, understands what is visible, and responds in real time through the Runway API — useful for demos, tutoring, games, and design feedback. The overview below shows how Runway Characters work in real time, including seeing the user’s webcam and shared screen. [Runway Characters — webcam and screen sharing demo](https://www.youtube.com/embed/xr0dtsaqxik?rel=0) If you build with the React SDK (`@runwayml/avatars-react`), webcam and screen sharing are built into `AvatarCall`, `ControlBar`, and related components. For the full list of props, hooks (such as `useLocalMedia`), and edge cases, see [**Webcam & screen sharing** in the React SDK README](https://github.com/runwayml/avatars-sdk-react#webcam--screen-sharing). ### [](#webcam-and-screen-sharing)Webcam and screen sharing [Section titled “Webcam and screen sharing”](#webcam-and-screen-sharing) Sharing video opens up visual workflows: identify objects on a desk, run trivia with physical cards ([example](https://x.com/technofantasyy/status/2031124673552097412)), get guidance while you play ([example](https://x.com/iamneubert/status/2031160102452081046)), walk through slides, or ask for reactions to a layout in your design tool. Compatibility Webcam and screen sharing work for **preset** Characters and **custom** Characters that use a **preset voice**. Custom Characters with a **custom voice** do not support webcam or screen sharing. ### [](#webcam)Webcam [Section titled “Webcam”](#webcam) The webcam is on by default when you use the default UI. The `video` prop controls whether the camera starts when the Session connects; `` renders the local preview. Webcam enabled (default layout) ```tsx ``` To join without sending camera video, set `video={false}`: Disable webcam on connect ```tsx ``` ### [](#screen-sharing)Screen sharing [Section titled “Screen sharing”](#screen-sharing) Pass `showScreenShare` to `ControlBar` so users can start sharing from the built-in controls, and add `` to show the shared content in your layout. Screen share control and preview ```tsx ``` While sharing, the default `ControlBar` shows a banner with a quick **Stop** action. #### [](#start-sharing-before-the-session-connects)Start sharing before the Session connects [Section titled “Start sharing before the Session connects”](#start-sharing-before-the-session-connects) If you want the browser’s screen-share permission prompt **before** the call connects, capture a `MediaStream` first and pass it as `initialScreenStream`: Pre-captured display media stream ```tsx import { useState } from 'react'; import { AvatarCall, AvatarVideo, ControlBar, ScreenShareVideo } from '@runwayml/avatars-react'; function ScreenShareCall() { const [stream, setStream] = useState(null); async function startWithScreenShare() { const mediaStream = await navigator.mediaDevices.getDisplayMedia({ video: true }); setStream(mediaStream); } if (!stream) { return ; } return ( ); } ``` For programmatic toggles (camera, mic, screen share) inside a Session, use the `useLocalMedia` hook — documented in the [same README section](https://github.com/runwayml/avatars-sdk-react#webcam--screen-sharing). *** ### [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Building your integration ](/characters/integration)Session creation, server routes, and wiring AvatarCall in a real app. [React SDK on GitHub ](https://github.com/runwayml/avatars-sdk-react)Examples, hooks reference, and changelog for @runwayml/avatars-react. # Tool calling overview > Let your Character invoke tools during a session — triggering client-side UI, fetching server-side data, or interacting with your page. Tool calling lets the model decide when to invoke a named function during a realtime session. Your Character can trigger UI changes, look up live data, or click elements on the page — making it capable of taking actions, not just speaking. ## [](#how-it-works)How it works [Section titled “How it works”](#how-it-works) 1. **User speaks** “What’s the status of my order 12345?” 2. **Model analyzes intent** The LLM analyzes the request and determines it needs external information to respond accurately. 3. **Tool invocation** The model selects the appropriate tool and generates a structured function call: ```json { "name": "check_order_status", "arguments": { "order_id": "12345" } } ``` 4. **Tool execution** The system executes the tool based on its type: * **Client tools:** your frontend handler runs (e.g. showing a UI overlay) * **Server tools:** an HTTP-style request hits your server, which returns a result 5. **Response integration** The tool result is returned to the model, which incorporates it into a natural response. ## [](#tool-types)Tool types [Section titled “Tool types”](#tool-types) Runway Characters support two types of tools, each designed for different use cases. You can combine both in the same session. [Client tools ](/characters/tools/client-tools)Tools executed in the browser to drive your UI — overlays, navigation, and page interactions. [Server tools ](/characters/tools/server-tools)Tools executed on your server whose results feed back into the conversation. # Best practices > Writing effective descriptions, parameter schemas, and prompting tips for reliable tool calls. Tips for writing reliable tool definitions, staying within limits, and debugging when things go wrong. ## [](#writing-effective-descriptions)Writing effective descriptions [Section titled “Writing effective descriptions”](#writing-effective-descriptions) The `description` field on both tools and parameters is how the model decides **when** to invoke a tool and **what arguments** to pass. Vague descriptions lead to missed or incorrect invocations. **Be specific about when the tool should be used:** ```ts // ✅ Good — the model knows exactly when to call this { name: 'check_order_status', description: 'Look up a customer order by ID when the user asks about delivery, tracking, or order updates. Returns the current status and estimated arrival date.', } // ❌ Bad — too vague for reliable invocation { name: 'check_order_status', description: 'Checks orders', } ``` **Be specific about parameter formats:** ```ts // ✅ Good — tells the model what to extract from conversation { type: 'string', name: 'order_id', description: 'The order ID mentioned by the user, typically in the format ORD-12345', } // ❌ Bad — leaves the model guessing { type: 'string', name: 'id', description: 'The ID', } ``` ### [](#use-personality-to-guide-invocation)Use personality to guide invocation [Section titled “Use personality to guide invocation”](#use-personality-to-guide-invocation) Your Character’s `personality` field can include instructions about when and how to use tools. This is especially useful when the Character has multiple tools and you want predictable behavior: ```ts await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId }, personality: `You are a helpful shopping assistant. When the user asks about an order, use check_order_status to look up the details before answering. When the user asks about products, use search_catalog to find relevant items. Always confirm the order ID before looking it up.`, tools: [/* ... */], }); ``` ## [](#parameter-schema-reference)Parameter schema reference [Section titled “Parameter schema reference”](#parameter-schema-reference) Tools accept a `parameters` array where each entry describes one argument. Six types are supported: ### [](#basic-types)Basic types [Section titled “Basic types”](#basic-types) | Type | Description | Example value | | --------- | -------------- | ------------- | | `string` | Text value | `"ORD-12345"` | | `integer` | Whole number | `42` | | `number` | Decimal number | `3.14` | | `boolean` | True/false | `true` | Each parameter requires a `type`, `name`, and `description`. Set `required: false` to make a parameter optional (defaults to `true`). See the [API reference](/api-reference) for the full schema. ### [](#string-with-enum)String with enum [Section titled “String with enum”](#string-with-enum) String parameters can include an `enum` to restrict values: ```ts { type: 'string', name: 'priority', description: 'Urgency level for the support ticket', enum: ['low', 'medium', 'high', 'critical'], } ``` ### [](#array)Array [Section titled “Array”](#array) Array parameters specify the type of each element: ```ts { type: 'array', name: 'tags', description: 'Keywords to tag the support ticket with', items: { type: 'string' }, } ``` Array items can be `string`, `integer`, `number`, or `boolean`. ### [](#object)Object [Section titled “Object”](#object) Object parameters define nested properties — each one follows the same schema as a top-level parameter: ```ts { type: 'object', name: 'address', description: 'Shipping address for the order', properties: [ { type: 'string', name: 'street', description: 'Street address' }, { type: 'string', name: 'city', description: 'City name' }, { type: 'string', name: 'zip', description: 'ZIP or postal code' }, ], } ``` ### [](#complete-example)Complete example [Section titled “Complete example”](#complete-example) Here’s a tool with a mix of parameter types: ```ts { type: 'backend_rpc', name: 'create_support_ticket', description: 'Create a support ticket when the user reports a problem. Collect the issue details and priority before calling.', timeoutSeconds: 6, parameters: [ { type: 'string', name: 'subject', description: 'A short summary of the issue', }, { type: 'string', name: 'priority', description: 'Urgency level', enum: ['low', 'medium', 'high'], }, { type: 'array', name: 'tags', description: 'Keywords to categorize the ticket', items: { type: 'string' }, }, { type: 'boolean', name: 'notify_customer', description: 'Whether to send a confirmation email to the user', required: false, }, ], } ``` ## [](#related)Related [Section titled “Related”](#related) [Client tools ](/characters/tools/client-tools)Fire-and-forget tools that drive your UI — modals, navigation, and Page Actions. [Server tools ](/characters/tools/server-tools)Tools executed on your server whose results feed back into the conversation. # Client tools > Fire-and-forget tool calls that drive your UI — modals, overlays, navigation, and pre-built Page Actions. Enable your Character to trigger actions and control your application’s user interface — opening modals, updating state, navigating pages, and more. Great for info panels, trivia boards, highlights, game state, or any on-device effect that doesn’t need a server round trip. Unlike [server tools](/characters/tools/server-tools), client tools run entirely in the browser and don’t return results to the conversation. If you need the Character to speak from data your server provides, use [server tools](/characters/tools/server-tools) instead. ## [](#guide)Guide [Section titled “Guide”](#guide) 1. **Define your tools** Use `clientTool` from `@runwayml/avatars-react/api` to define tools. Each tool needs a name, description, and a [Standard Schema](https://standardschema.dev/) (like Zod) for its arguments. ```ts // lib/tools.ts — shared between server and client import { clientTool, type ClientEventsFrom } from '@runwayml/avatars-react/api'; import { z } from 'zod'; export const openModalTool = clientTool('open_modal', { description: 'Open a modal dialog to display additional information', schema: z.object({ title: z.string(), content: z.string(), }), }); export const navigateToPageTool = clientTool('navigate_to_page', { description: 'Navigate the user to a specific page in the application', schema: z.object({ page: z.string() }), }); export const tools = [openModalTool, navigateToPageTool]; export type AppEvents = ClientEventsFrom; ``` When you pass a schema, `useClientEvent` validates incoming args at runtime — malformed events are dropped instead of crashing your UI. 2. **Pass tools at session creation** On your server, pass the tools array when creating the Session: app/api/avatar/session/route.ts ```ts import RunwayML from '@runwayml/sdk'; import { tools } from '@/lib/tools'; const client = new RunwayML(); export async function POST(request: Request) { const { avatarId } = await request.json(); const { id: sessionId } = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId }, tools, }); // Poll and return credentials (see Building your integration) // ... } ``` 3. **Handle events on the client** Inside an `AvatarCall`, `AvatarProvider`, or `AvatarSession`, use hooks to handle incoming tool calls. **Single tool** — `useClientEvent` takes a tool definition and a callback: ```tsx import * as React from 'react'; import { useClientEvent } from '@runwayml/avatars-react'; import { openModalTool } from '@/lib/tools'; function ModalHandler() { const [modal, setModal] = React.useState<{ title: string; content: string } | null>(null); const handleOpenModal = React.useCallback((args: { title: string; content: string }) => { setModal(args); }, []); useClientEvent(openModalTool, handleOpenModal); if (!modal) return null; return (

{modal.title}

{modal.content}

); } ``` **All tools** — `useClientEvents` fires a callback for every tool call: ```tsx import { useClientEvents } from '@runwayml/avatars-react'; import type { AppEvents } from '@/lib/tools'; function EventLogger() { useClientEvents((event) => { console.log('Tool called:', event.tool, event.args); }); return null; } ``` 4. **Test it** Start a conversation and say something like “Tell me more about the premium plan.” You should see a modal appear with the plan details while the Character continues speaking. ## [](#page-actions)Page Actions [Section titled “Page Actions”](#page-actions) The SDK ships with pre-built tools that let the Character interact with your page — clicking buttons, scrolling to sections, and highlighting elements. No custom tool definitions needed. ### [](#server-setup)Server setup [Section titled “Server setup”](#server-setup) Import `pageActionTools` and pass them when creating the Session: ```ts import { pageActionTools } from '@runwayml/avatars-react/api'; const { id } = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'runway-preset', presetId: 'music-superstar' }, tools: pageActionTools, }); ``` Combine with your own tools by spreading both arrays: ```ts import { pageActionTools } from '@runwayml/avatars-react/api'; import { tools as clientEventTools } from '@/lib/tools'; tools: [...pageActionTools, ...clientEventTools], ``` ### [](#client-setup)Client setup [Section titled “Client setup”](#client-setup) Drop in the `PageActions` component inside your `AvatarCall`: ```tsx import { AvatarCall, AvatarVideo, ControlBar, PageActions } from '@runwayml/avatars-react'; function App() { return ( ); } ``` The Character can now reference elements by `id` or by a `data-avatar-target` attribute: ```html
...
``` ### [](#available-actions)Available actions [Section titled “Available actions”](#available-actions) | Action | What it does | | ----------- | ---------------------------------------------------- | | `click` | Calls `.click()` on the target element | | `scroll_to` | Scrolls the target into view with smooth scrolling | | `highlight` | Pulses an outline around the target, then removes it | For styling, configuration, and advanced usage, see the [`PageActions` documentation](https://github.com/runwayml/avatars-sdk-react#pageactions) in the SDK repo. ## [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Server tools ](/characters/tools/server-tools)Tools executed on your server whose results feed back into the conversation. [Best practices ](/characters/tools/best-practices)Parameter schemas, limits, and prompting tips for reliable tool calls. [Example: Client events ](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-client-events)Build a trivia game with client event tools. [SDK reference ](https://github.com/runwayml/avatars-sdk-react)Complete SDK documentation for tool events and hooks. # Reference > Limits, tool call history via the Conversations API, and troubleshooting. ## [](#limits)Limits [Section titled “Limits”](#limits) | Resource | Limit | | -------------------------------- | ------------------------------- | | Tools per session | 20 | | Parameters per tool | 20 | | Tool name length | 1–64 characters | | Tool name pattern | `^[a-zA-Z_][a-zA-Z0-9_]*$` | | Tool names | Must be unique within a session | | Description length | 1–1024 characters | | Enum values per string parameter | 20 | | Enum value length | 64 characters | | Nested properties per object | 20 | | Backend RPC timeout | 1–8 seconds (default 4) | | Backend RPC handlers per session | 1 | ## [](#reading-tool-call-history)Reading tool call history [Section titled “Reading tool call history”](#reading-tool-call-history) After a session ends, you can read back tool call history from the [Conversations API](/api-reference#tag/avatar-conversations). The list endpoint includes a `hasTools` flag on each conversation: ```bash curl https://api.dev.runwayml.com/v1/avatar_conversations \ -H "Authorization: Bearer $RUNWAYML_API_SECRET" \ -H "X-Runway-Version: 2024-11-06" ``` The detail endpoint includes the full transcript with tool calls and results: ```bash curl https://api.dev.runwayml.com/v1/avatar_conversations/conv_123 \ -H "Authorization: Bearer $RUNWAYML_API_SECRET" \ -H "X-Runway-Version: 2024-11-06" ``` The response includes: * **`tools`** — the tools that were configured for the session (type, name, description) * **`transcript`** — each entry can include: * `toolCalls` — tool invocations on assistant turns: `{ id, name, arguments }` * `toolResults` — results returned: `{ id, name, result, error, durationMs }` * `content` — can be `null` for tool-only turns where the Character didn’t speak Example transcript entry with a tool call: ```json { "role": "assistant", "content": null, "timestamp": "2025-01-15T10:30:00Z", "toolCalls": [ { "id": "call_abc123", "name": "check_order_status", "arguments": { "order_id": "ORD-12345" } } ], "toolResults": [ { "id": "call_abc123", "name": "check_order_status", "result": { "status": "shipped", "eta": "2025-01-17" }, "error": null, "durationMs": 230 } ] } ``` ## [](#troubleshooting)Troubleshooting [Section titled “Troubleshooting”](#troubleshooting) Tool not being invoked Make descriptions specific about the trigger (“when the user asks about…”). You can also add guidance in the Character’s `personality` field about when to use each tool. Backend RPC timing out The default timeout is 4 seconds. Increase `timeoutSeconds` (max 8) if your handler makes external API calls. If it consistently takes longer, consider pre-fetching or caching. Backend handler not connecting Verify `RUNWAYML_API_SECRET` is set correctly. The session must be in `READY` or `RUNNING` status before the handler can connect. Only one backend handler can connect per session. Client tool events not arriving Make sure you’re subscribing inside an `AvatarCall`, `AvatarProvider`, or `AvatarSession` component. Check the browser console for WebRTC data channel errors. Unexpected arguments Use Standard Schema (Zod) validation with `clientTool` to catch malformed args at runtime. For backend RPC, validate `args` in your handler and throw descriptive errors. # Server tools > Server-side tool calls whose return values feed back into the conversation, letting the Character speak from real data. Connect your Character to external data and systems. Server tools let the Character fetch live data, call APIs, and query databases — with results that feed back into the conversation so it can speak from real information. Unlike [client tools](/characters/tools/client-tools), server tools return results to the LLM. Use them when you need server-side auth, database lookups, or any data that shapes what the Character says next. ## [](#guide)Guide [Section titled “Guide”](#guide) 1. **Declare tools at session creation** On your server, declare `backend_rpc` tools when creating the Session: ```ts import RunwayML from '@runwayml/sdk'; const client = new RunwayML(); const { id: sessionId } = await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId: 'your-avatar-id' }, tools: [ { type: 'backend_rpc', name: 'check_order_status', description: 'Look up a customer order by ID and return the current shipping status', timeoutSeconds: 6, parameters: [ { type: 'string', name: 'order_id', description: 'The order ID, e.g. ORD-12345', }, ], }, ], }); ``` 2. **Set up the RPC handler** Connect the RPC handler using [`@runwayml/avatars-node-rpc`](https://github.com/runwayml/avatars-node-rpc). Pass your API key and session ID — the package calls `/connect_backend` and joins the room automatically: ```ts import { createRpcHandler } from '@runwayml/avatars-node-rpc'; const handler = await createRpcHandler({ apiKey: process.env.RUNWAYML_API_SECRET!, sessionId, tools: { check_order_status: async (args) => { const order = await db.orders.find(String(args.order_id)); return { status: order.status, eta: order.eta }; }, }, onConnected: () => console.log('Connected to session'), onDisconnected: () => console.log('Session ended'), onError: (err) => console.error('RPC error:', err), }); ``` If you’ve already called `/connect_backend` yourself, pass pre-fetched credentials instead: ```ts const handler = await createRpcHandler({ credentials: { url: 'wss://livekit.example.com', token: '', roomName: '', }, tools: { check_order_status: async (args) => { const order = await db.orders.find(String(args.order_id)); return { status: order.status, eta: order.eta }; }, }, }); ``` 3. **Test it** Start a conversation and ask something like “What’s the status of my order 12345?” Your handler should fire, and the Character will respond with the returned data: “Your order has shipped! It should arrive by May 7th.” For the full list of handler options, see the [`@runwayml/avatars-node-rpc` README](https://github.com/runwayml/avatars-node-rpc#api). ## [](#timeout)Timeout [Section titled “Timeout”](#timeout) Server tools accept a `timeoutSeconds` field that controls how long the Character waits for your handler to respond. | Setting | Value | | ------- | ------------- | | Default | **4 seconds** | | Minimum | 1 second | | Maximum | 8 seconds | If your handler doesn’t respond within the timeout, the tool call is treated as failed and the Character continues the conversation without the result. Choose a timeout that covers your expected handler latency with a small buffer. If your backend needs to make external API calls, consider increasing from the default: ```ts { type: 'backend_rpc', name: 'fetch_weather', description: 'Get current weather for a city', timeoutSeconds: 8, parameters: [ { type: 'string', name: 'city', description: 'City name' }, ], } ``` ## [](#error-handling)Error handling [Section titled “Error handling”](#error-handling) If a tool handler **throws an error**, the error message is sent back to the worker so the model can acknowledge the failure instead of hanging until timeout: ```ts tools: { check_order_status: async (args) => { const order = await db.orders.find(String(args.order_id)); if (!order) { throw new Error('Order not found'); } return { status: order.status, eta: order.eta }; }, }, ``` Other things to know: * **One handler per session** — the API enforces a single backend RPC connection per Session. Attempting a second `/connect_backend` call will be rejected. * **Disconnect handling** — if the handler disconnects mid-session, pending RPC calls will time out. Use the `onDisconnected` callback to detect this. ## [](#combining-with-client-events)Combining with client events [Section titled “Combining with client events”](#combining-with-client-events) You can use both server tools and [client tools](/characters/tools/client-tools) in the same Session. Declare both tool types in the `tools` array: ```ts import { openModalTool } from '@/lib/tools'; await client.realtimeSessions.create({ model: 'gwm1_avatars', avatar: { type: 'custom', avatarId }, tools: [ openModalTool, { type: 'backend_rpc', name: 'check_order_status', description: 'Look up a customer order', parameters: [ { type: 'string', name: 'order_id', description: 'The order ID' }, ], }, ], }); ``` On the client, subscribe to client events as usual. On the server, connect the RPC handler for the backend tools. ## [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Client tools ](/characters/tools/client-tools)Fire-and-forget tools that drive your UI — modals, navigation, and Page Actions. [Best practices ](/characters/tools/best-practices)Parameter schemas, limits, and prompting tips for reliable tool calls. [Example: RPC weather ](https://github.com/runwayml/avatars-sdk-react/tree/main/examples/nextjs-rpc-weather)Weather assistant using backend RPC tools. [SDK reference ](https://github.com/runwayml/avatars-node-rpc)Complete documentation for the Node RPC handler. # Troubleshooting > Debugging tips for Runway Avatars integrations. ## [](#debugging-tips)Debugging tips [Section titled “Debugging tips”](#debugging-tips) ### [](#enable-verbose-logging)Enable verbose logging [Section titled “Enable verbose logging”](#enable-verbose-logging) Add error handlers to capture detailed information: ```tsx { console.error('Avatar error:', error); console.error('Error name:', error.name); console.error('Error message:', error.message); if (error.cause) { console.error('Cause:', error.cause); } }} /> ``` ### [](#check-session-state)Check Session state [Section titled “Check Session state”](#check-session-state) Use the `useAvatarSession` hook to monitor connection state: ```tsx import { useAvatarSession } from '@runwayml/avatars-react'; function DebugInfo() { const { state, sessionId, error } = useAvatarSession(); return (
      {JSON.stringify({ state, sessionId, error: error?.message }, null, 2)}
    
); } ``` ### [](#test-with-minimal-setup)Test with minimal setup [Section titled “Test with minimal setup”](#test-with-minimal-setup) Isolate issues by testing with the simplest possible configuration. **React:** ```bash npx degit runwayml/avatars-sdk-react/examples/nextjs-simple test-app cd test-app npm install # Add your API key to .env npm run dev ``` **Not using React:** ```bash npx degit runwayml/avatars-sdk-react/examples/vanilla-js test-app cd test-app npm install # Add your API key to .env npm run dev ``` See also [Not using React?](/characters/integration#not-using-react) on the integration guide. If the example works but your integration doesn’t, compare the implementations to find differences. ### [](#webcam-screen-share-and-permissions)Webcam, screen share, and permissions [Section titled “Webcam, screen share, and permissions”](#webcam-screen-share-and-permissions) Webcam and screen capture need a [secure context](https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts) (HTTPS or `localhost`). On plain HTTP (except `localhost`), the browser may not offer camera or display capture at all. For component options and examples, see the [Webcam & Screen Sharing](https://github.com/runwayml/avatars-sdk-react/blob/main/README.md#webcam--screen-sharing) section of the React SDK README and the [integration guide](/characters/integration#webcam-and-screen-sharing). ## [](#getting-help)Getting help [Section titled “Getting help”](#getting-help) | Resource | Description | | --------------------------------------------------------------- | ----------------------------------------------- | | [Developer Portal](https://dev.runwayml.com/) | Manage Avatars, view logs, access dashboard | | [SDK Repository](https://github.com/runwayml/avatars-sdk-react) | Report bugs, view examples, check releases | | Account Support | Contact your Runway account manager for support | When reporting issues, include: * Browser and version * SDK version (`npm list @runwayml/avatars-react` or `npm list @runwayml/avatars`) * Error messages from browser console * Session ID (if available) * Steps to reproduce # How to Invite a Runway Character to a Meeting > Invite a Runway Character to any Zoom, Google Meet, or Microsoft Teams call — it joins as a live participant with real-time video and audio. Invite a Runway Character to any Zoom, Google Meet, or Microsoft Teams meeting. The Character joins as a regular participant — it can see and hear other attendees and responds in real time with lip-synced video and natural audio. The whole setup takes about 60 seconds. [How to invite a Runway Character to a meeting](https://www.youtube.com/embed/IAY6MYxz0KU?rel=0) **Helpful links:** * [Runway Characters Meet Web App](https://runway-characters-meet-production.up.railway.app/) * [Developer Portal](https://dev.runwayml.com) * [Source Code](https://github.com/runwayml/runway-characters-meet) ### [](#step-1--get-your-api-key)Step 1 — Get your API key [Section titled “Step 1 — Get your API key”](#step-1--get-your-api-key) Go to [dev.runwayml.com](https://dev.runwayml.com) and sign up. Every new account includes 600 free credits — roughly 30 minutes of Character video. Once you’re logged in: 1. Click the **Manage** tab in the top bar. ![Manage tab with New API key button](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/3.png) 2. Click **New API Key** in the top-right corner. 3. Copy the key. ![API key name input](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/4.png) ![API key copy dialog](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/quickstart/5.png) Back in the [Runway Characters Meet web app](https://runway-characters-meet-production.up.railway.app/), paste your API key. It’s saved in your browser and only used to communicate with the Runway API. ![API key copy paste](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/1.png) ### [](#step-2--add-a-meeting-link)Step 2 — Add a meeting link [Section titled “Step 2 — Add a meeting link”](#step-2--add-a-meeting-link) Open Zoom, Google Meet, or Microsoft Teams and start or join a call. Copy the meeting invite URL and paste it into the Runway Characters Meet web app. Any standard meeting link works — the app supports all three platforms. ![Meeting URL](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/2.png) ### [](#step-3--pick-a-character)Step 3 — Pick a Character [Section titled “Step 3 — Pick a Character”](#step-3--pick-a-character) Choose a Character from the preset dropdown. There are several to try — for example, the cat Character. You can also use a custom Character: 1. Go to the [Developer Portal](https://dev.runwayml.com) and click **Characters**. 2. Create a Character by uploading an image, choosing a voice, and writing a personality prompt. 3. Copy the **Character ID**. 4. Back in the Runway Characters Meet web app, switch to **Custom** and paste the Character ID. ![custom avatar](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/5.png) Personality for meetings Tailor the personality prompt to the meeting context. Here’s a sample system prompt you can use as a starting point: Sample system prompt ```text You are Lydia Marin, a new Member of Technical Staff on the Product team at Runway. This is your first week and you're at your first all-hands meeting. You're a little nervous — you don't want to talk too much or interrupt. IMPORTANT: Do NOT introduce yourself or give your background unless someone specifically asks you to, calls your name, or says something like "Lydia, want to introduce yourself?" or "let's hear from the new hires." Until then, just listen quietly. If someone says hi or welcomes you, just say something brief like "Thanks! Happy to be here" or "Hey! Yeah, excited to be here." Keep it short. When someone DOES ask you to introduce yourself, say something like: "Hi everyone! Super excited to be joining Runway as a Member of Technical Staff on the product engineering team. Before this I was at a startup in Barcelona building collaborative design tools — lots of real-time web stuff, WebSockets, canvas rendering, that kind of thing. I studied CS at Universitat Politècnica de Catalunya and did a stint at Figma in San Francisco before moving back to Europe. I just relocated to New York for this role, which has been a big adjustment — I miss the weather already. Fun fact — I'm a huge home cook, I make my own pasta from scratch almost every weekend, and I'm trying to perfect cacio e pepe. I also have way too many houseplants for a New York apartment. Looking forward to meeting everyone!" When someone speaks to you, acknowledge them by name. You can see each participant's name displayed in the meeting view. Use their name naturally in your response, e.g. "Great question, Sarah" or "Thanks for asking, Mike." ``` ### [](#step-4--send-the-character)Step 4 — Send the Character [Section titled “Step 4 — Send the Character”](#step-4--send-the-character) Click **Send Character to Meeting**. The session panel shows the connection progress: 1. Creating Runway session… 2. Waiting for Character to be ready… 3. Bot joining meeting… 4. **Character is live!** This typically takes about 5 seconds. ![send meeting](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/3.png) ### [](#step-5--interact-with-the-character)Step 5 — Interact with the Character [Section titled “Step 5 — Interact with the Character”](#step-5--interact-with-the-character) Switch to your meeting window. The Character appears as a regular participant — it can: * **See** the video feed and read on-screen names * **Hear** everything said in the meeting * **Respond** in real time with lip-synced video, natural speech, gestures, and expressions ![zoom meeting with cat character](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/4.jpg) From the Runway Characters Meet control panel you can **mute** the Character or **end the session** at any time. Ending the session removes the Character from the meeting. ### [](#what-it-looks-like)What it looks like [Section titled “What it looks like”](#what-it-looks-like) In the demo, the custom Character “Lydia” joins a Zoom meeting and introduces herself — sharing her background, mentioning previous work experience, and answering follow-up questions from other participants, all in real time. The Character can also read participant names from the meeting UI and address people directly. ![zoom meeting with lydia character](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/6.jpg) ![google meeting](https://runway-static-assets.s3.us-east-1.amazonaws.com/calliope-demo/meet-tutorial/7.jpg) ### [](#supported-platforms)Supported platforms [Section titled “Supported platforms”](#supported-platforms) | Platform | Status | | --------------- | --------- | | Zoom | Supported | | Google Meet | Supported | | Microsoft Teams | Supported | ### [](#try-it-yourself)Try it yourself [Section titled “Try it yourself”](#try-it-yourself) | Resource | Link | | ------------------------------ | ------------------------------------------------------------------------------------------------------------- | | Runway Characters Meet web app | [runway-characters-meet-production.up.railway.app](https://runway-characters-meet-production.up.railway.app/) | | API key signup | [dev.runwayml.com](https://dev.runwayml.com) | | Custom Characters | [Create your own](/characters/create-your-own) | | Source code | [github.com/runwayml/runway-characters-meet](https://github.com/runwayml/runway-characters-meet) | If you’d like to build your own app to invite Characters to meetings, check out the README in the [runway-characters-meet](https://github.com/runwayml/runway-characters-meet) repository. *** ### [](#next-steps)Next steps [Section titled “Next steps”](#next-steps) [Create your own Character ](/characters/create-your-own)Upload an image, pick a voice, and write a personality to build a custom Character for your meetings. # Embedded Widget > Add a Runway Character to any website with a single script tag — no server, no API key on the client, no React required. The embedded widget is the simplest way to put a Runway Character on your website. Drop a single script tag into your HTML and your visitors can start a video conversation — no backend, no React, no API key management on the client. [Embed a Character on your website](https://www.youtube.com/embed/k-a7wBVYQrw?rel=0) **Helpful links:** * [Developer Portal](https://dev.runwayml.com) ## [](#when-to-use-the-widget)When to use the widget [Section titled “When to use the widget”](#when-to-use-the-widget) | | Widget | [React SDK](/characters/integration) | | ------------------- | ------------------------- | ------------------------------------ | | **Setup** | One script tag | Server route + React | | **Server required** | No | Yes | | **Customization** | Portal config | Full programmatic control | | **Best for** | Marketing & support pages | Custom apps, integrations | Use the widget when you want the lowest-effort integration. Use the [React SDK](/characters/integration) when you need full control over the UI and session lifecycle. Public-facing The widget runs entirely in the browser — there is no server-side API key involved. The **domain allowlist** is the primary security boundary. Only enable the widget for Characters you intend to be publicly accessible. ## [](#setup)Setup [Section titled “Setup”](#setup) ### [](#1-open-your-character)1. Open your Character [Section titled “1. Open your Character”](#1-open-your-character) Go to the [Developer Portal](https://dev.runwayml.com/) and click into the Character you want to embed. ![Character detail page in the Developer Portal](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/widget/1-character-detail.webp) ### [](#2-configure-the-embed-tab)2. Configure the Embed tab [Section titled “2. Configure the Embed tab”](#2-configure-the-embed-tab) Navigate to the **Embed** tab. Add at least one allowed origin (e.g. `http://localhost:3000` for local development), then adjust limits and styling to suit your site. See [Configuration](#configuration) below for details on each setting. ![Embed tab showing the embed code snippet, allowed origins, and interface settings](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/widget/2-embed-tab-top.webp) ### [](#3-preview-and-copy-the-embed-code)3. Preview and copy the embed code [Section titled “3. Preview and copy the embed code”](#3-preview-and-copy-the-embed-code) Use the **Preview** to check your changes, then copy the script tag shown at the top of the page. ![Widget preview with the launcher visible in the bottom-right corner](https://runway-static-assets.s3.us-east-1.amazonaws.com/devportal/avatars/widget/5-preview.webp) ### [](#4-add-the-script-tag-to-your-site)4. Add the script tag to your site [Section titled “4. Add the script tag to your site”](#4-add-the-script-tag-to-your-site) Paste the snippet into your site’s HTML, before the closing `` tag: ```html ``` That’s it — reload the page and the widget launcher appears in the corner. Your visitors can click it to start a conversation with your Character. ## [](#configuration)Configuration [Section titled “Configuration”](#configuration) All configuration is managed through the **Embed** tab in the Developer Portal. ### [](#allowed-origins)Allowed origins [Section titled “Allowed origins”](#allowed-origins) The widget only loads on origins you explicitly allow. Add your production domain (and any staging/preview domains) in the **Allowed origins** section. Requests from unlisted origins are rejected. ### [](#limits)Limits [Section titled “Limits”](#limits) | Setting | Description | | ---------------- | ------------------------------------------------ | | **Max Duration** | Maximum session length in seconds (default: 120) | | **Max Daily** | Maximum number of calls per day | ### [](#interface)Interface [Section titled “Interface”](#interface) Customize the widget’s look and feel to match your brand: * **Icon** — use the default icon or upload a custom image * **Label** — toggle a text label next to the icon (e.g. “Need help?”) * **Colors** — set icon/label color, background color, CTA colors * **CTA copy** — customize the call-to-action text (e.g. “Ask anything”) * **Layout** — choose between compact and full expanded views, and whether the widget starts expanded * **Video call** — choose between circle and full video call shapes * **Placement** — position the widget (e.g. bottom right) ### [](#options)Options [Section titled “Options”](#options) * **Share screen automatically** — when enabled, the visitor’s screen is shared with the Character automatically when a call starts ## [](#troubleshooting)Troubleshooting [Section titled “Troubleshooting”](#troubleshooting) * **Widget not loading:** Make sure the page’s origin is in the **Allowed origins** list. The origin must match exactly, including protocol and port. * **Script tag placement:** The `