Client tools

Enable your Character to trigger actions and control your application’s user interface — opening modals, updating state, navigating pages, and more. Great for info panels, trivia boards, highlights, game state, or any on-device effect that doesn’t need a server round trip.

Unlike server tools, client tools run entirely in the browser and don’t return results to the conversation. If you need the Character to speak from data your server provides, use server tools instead.

Guide

1. Define your tools

   Use clientTool from @runwayml/avatars-react/api to define tools. Each tool needs a name, description, and a Standard Schema (like Zod) for its arguments.

   // lib/tools.ts — shared between server and client
   import { clientTool, type ClientEventsFrom } from '@runwayml/avatars-react/api';
   import { z } from 'zod';
   
   export const openModalTool = clientTool('open_modal', {
     description: 'Open a modal dialog to display additional information',
     schema: z.object({
       title: z.string(),
       content: z.string(),
     }),
   });
   
   export const navigateToPageTool = clientTool('navigate_to_page', {
     description: 'Navigate the user to a specific page in the application',
     schema: z.object({ page: z.string() }),
   });
   
   export const tools = [openModalTool, navigateToPageTool];
   export type AppEvents = ClientEventsFrom<typeof tools>;

   When you pass a schema, useClientEvent validates incoming args at runtime — malformed events are dropped instead of crashing your UI.

2. Pass tools at session creation

   On your server, pass the tools array when creating the Session:

   app/api/avatar/session/route.ts

   import RunwayML from '@runwayml/sdk';
   import { tools } from '@/lib/tools';
   
   const client = new RunwayML();
   
   export async function POST(request: Request) {
     const { avatarId } = await request.json();
   
     const { id: sessionId } = await client.realtimeSessions.create({
       model: 'gwm1_avatars',
       avatar: { type: 'custom', avatarId },
       tools,
     });
   
     // Poll and return credentials (see Building your integration)
     // ...
   }

3. Handle events on the client

   Inside an AvatarCall, AvatarProvider, or AvatarSession, use hooks to handle incoming tool calls.

   Single tool — useClientEvent takes a tool definition and a callback:

   import * as React from 'react';
   import { useClientEvent } from '@runwayml/avatars-react';
   import { openModalTool } from '@/lib/tools';
   
   function ModalHandler() {
     const [modal, setModal] = React.useState<{ title: string; content: string } | null>(null);
   
     const handleOpenModal = React.useCallback((args: { title: string; content: string }) => {
       setModal(args);
     }, []);
   
     useClientEvent(openModalTool, handleOpenModal);
   
     if (!modal) return null;
   
     return (
       <dialog open>
         <h2>{modal.title}</h2>
         <p>{modal.content}</p>
       </dialog>
     );
   }

   All tools — useClientEvents fires a callback for every tool call:

   import { useClientEvents } from '@runwayml/avatars-react';
   import type { AppEvents } from '@/lib/tools';
   
   function EventLogger() {
     useClientEvents<AppEvents>((event) => {
       console.log('Tool called:', event.tool, event.args);
     });
     return null;
   }

4. Test it

   Start a conversation and say something like “Tell me more about the premium plan.” You should see a modal appear with the plan details while the Character continues speaking.

Page Actions

The SDK ships with pre-built tools that let the Character interact with your page — clicking buttons, scrolling to sections, and highlighting elements. No custom tool definitions needed.

Server setup

Import pageActionTools and pass them when creating the Session:

import { pageActionTools } from '@runwayml/avatars-react/api';

const { id } = await client.realtimeSessions.create({
  model: 'gwm1_avatars',
  avatar: { type: 'runway-preset', presetId: 'music-superstar' },
  tools: pageActionTools,
});

Combine with your own tools by spreading both arrays:

import { pageActionTools } from '@runwayml/avatars-react/api';
import { tools as clientEventTools } from '@/lib/tools';

tools: [...pageActionTools, ...clientEventTools],

Client setup

Drop in the PageActions component inside your AvatarCall:

import { AvatarCall, AvatarVideo, ControlBar, PageActions } from '@runwayml/avatars-react';

function App() {
  return (
    <AvatarCall avatarId="music-superstar" connectUrl="/api/avatar/connect">
      <AvatarVideo />
      <ControlBar />
      <PageActions />
    </AvatarCall>
  );
}

The Character can now reference elements by id or by a data-avatar-target attribute:

<button id="signup">Sign Up</button>
<section data-avatar-target="pricing">...</section>

Available actions

Action	What it does
click	Calls .click() on the target element
scroll_to	Scrolls the target into view with smooth scrolling
highlight	Pulses an outline around the target, then removes it

For styling, configuration, and advanced usage, see the PageActions documentation in the SDK repo.

Next steps

Server tools Tools executed on your server whose results feed back into the conversation.

Best practices Parameter schemas, limits, and prompting tips for reliable tool calls.

Example: Client events Build a trivia game with client event tools.

SDK reference Complete SDK documentation for tool events and hooks.