Agents

ElevenLabs Agents Platform

Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding.

Setup, before writing any code: if ELEVENLABS_API_KEY isn't set for this project, run npx -y @elevenlabs/cli@latest onboard init from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the Installation Guide for CLI and SDK setup.

Quick Start with CLI

The ElevenLabs CLI is the recommended way to create and manage agents:

# Install CLI and authenticate
npm install -g @elevenlabs/cli
elevenlabs auth login

# Initialize project and create an agent
elevenlabs agents init
elevenlabs agents add "My Assistant" --template complete

# Push to ElevenLabs platform
elevenlabs agents push

Available templates: complete, minimal, voice-only, text-only, customer-service, assistant

Python

from elevenlabs import ElevenLabs

client = ElevenLabs()

agent = client.conversational_ai.agents.create(
    name="My Assistant",
    conversation_config={
        "agent": {
            "first_message": "Hello! How can I help?",
            "language": "en",
            "prompt": {
                "prompt": "You are a helpful assistant. Be concise and friendly.",
                "llm": "gemini-2.0-flash",
                "temperature": 0.7
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

JavaScript

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
const client = new ElevenLabsClient();

const agent = await client.conversationalAi.agents.create({
  name: "My Assistant",
  conversationConfig: {
    agent: {
      firstMessage: "Hello! How can I help?",
      language: "en",
      prompt: {
        prompt: "You are a helpful assistant.",
        llm: "gemini-2.0-flash",
        temperature: 0.7
      }
    },
    tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" }
  }
});

CLI

The CLI reads ELEVENLABS_API_KEY from the environment automatically:

elevenlabs agents create \
  --json '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'

Starting Conversations

Authenticated WebRTC: Request a session token from your backend. The response includes both the token and the conversation ID:

session = client.conversational_ai.conversations.get_webrtc_token(
    agent_id="your-agent-id",
)
print(session.token, session.conversation_id)

Server-side (Python): Get signed URL for client connection:

signed_url = client.conversational_ai.conversations.get_signed_url(
    agent_id="your-agent-id",
    environment="staging",
)

Client-side (JavaScript):

import { Conversation } from "@elevenlabs/client";

const conversation = await Conversation.startSession({
  agentId: "your-agent-id",
  environment: "staging",
  overrides: { asr: { keywords: ["ElevenLabs", "TechCorp"] } },
  onMessage: (msg) => console.log("Agent:", msg.message),
  onUserTranscript: (t) => console.log("User:", t.message),
  onPing: (event) => console.log("Estimated latency:", event.ping_ms),
  onContextUsage: ({ model, context_tokens, context_limit_tokens }) =>
    console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`),
  onError: (e) => console.error(e)
});

React Hook: Wrap hook consumers in ConversationProvider. Prefer granular hooks such as useConversationControls and useConversationStatus for session controls and UI state; useConversation remains available as the convenience all-in-one hook. Pass provider-level callbacks such as onError when you want React to handle conversation errors in one place.

import {
  ConversationProvider,
  useConversationControls,
  useConversationStatus,
} from "@elevenlabs/react";

function Agent({ signedUrl }: { signedUrl: string }) {
  const { startSession, endSession } = useConversationControls();
  const { status } = useConversationStatus();

  if (status === "connected") {
    return <button onClick={endSession}>End conversation</button>;
  }

  return (
    <button onClick={() => startSession({ signedUrl })}>
      Start conversation
    </button>
  );
}

function App({ signedUrl }: { signedUrl: string }) {
  return (
    <ConversationProvider
      onError={(error) => console.error("Conversation error:", error)}
      onPing={(event) => console.log("Estimated latency:", event.ping_ms)}
      onContextUsage={({ model, context_tokens, context_limit_tokens }) =>
        console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`)
      }
    >
      <Agent signedUrl={signedUrl} />
    </ConversationProvider>
  );
}

Configuration

Use GET /v1/convai/llm/list to inspect the current model catalog, including deprecation state, token/context limits, capability flags such as image-input support, and model-specific reasoning effort support.

To run an agent on an ElevenLabs Realtime model instead of a separate LLM and TTS model (conversation_config.realtime_model), use the realtime skill.

Popular voices: JBFqnCBsd6RMkjVDRZzb (George), EXAVITQu4vr4xnSDxMaL (Sarah), onwK4e9ZLuTAKqWW03F9 (Daniel), XB0fDUnXU5powFXDhCwa (Charlotte)

Turn eagerness: patient (waits longer for user to finish), normal, or eager (responds quickly)

See Agent Configuration for all options.

System Prompt Structure

Section the prompt with markdown headings — the model prioritizes and interprets instructions more reliably (prompting guide):

# Personality   – named character, 2-3 traits
# Environment   – where they work, who they talk to
# Tone          – vocal style as 4-5 bullets
# Goal          – what success looks like (numbered for multi-step flows)

Keep instructions short and action-based. Mark critical steps with "This step is important." For critical refusal/safety rules, include concise instructions in the prompt and also configure independent custom Guardrails via platform_settings.guardrails (see Guardrails).

Tools

Extend agents with webhook, client, or built-in system tools. Tools are defined inside conversation_config.agent.prompt:

Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as {{system__conversation_history}} can pass full conversation context into tool calls when needed.

"prompt": {
    "prompt": "You are a helpful assistant that can check the weather.",
    "llm": "gemini-2.0-flash",
    "tools": [
        # Webhook: server-side API call
        {"type": "webhook", "name": "get_weather", "description": "Get weather",
         "api_schema": {"url": "https://api.example.com/weather", "method": "POST",
             "request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}},
        # Client: runs in the browser
        {"type": "client", "name": "show_product", "description": "Display a product",
         "parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}}
    ],
    "built_in_tools": {
        "end_call": {},
        "transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}
    }
}

Client tools run in browser:

clientTools: {
  show_product: async ({ productId }) => {
    document.getElementById("product").src = `/products/${productId}`;
    return { success: true };
  }
}

See Client Tools Reference for complete documentation.

Built-in System Tools

Set under conversation_config.agent.prompt.built_in_tools. {} enables defaults; provide description to customize; omit to disable.

start_procedure and end_procedure are not configured here. The platform adds them automatically whenever the agent has at least one procedure (see Procedures).

run_subagent is a system tool for delegating a task to another configured agent. Add it to conversation_config.agent.prompt.tools with params.system_tool_type: "run_subagent" and an agents array. Each entry requires agent_id and description; branch_id and a JSON-schema parameters object are optional.

knowledge_base is a system tool for letting the model choose how to inspect attached knowledge. Add it to conversation_config.agent.prompt.tools with type: "system", a name, and params.system_tool_type: "knowledge_base". Use enabled_strategies to expose any combination of cat, keyword, semantic, and ls:

{
  "type": "system",
  "name": "knowledge_base",
  "description": "Search the attached knowledge base.",
  "params": {
    "system_tool_type": "knowledge_base",
    "enabled_strategies": ["semantic", "keyword"]
  }
}

Integration Tools

Pre-built connectors managed by the platform. Create a connection with credentials, then attach via tool_ids:

Three-step flow: POST /v1/convai/api-integrations/{id}/connections → GET /v1/convai/api-integrations/{id}/tools → POST /v1/convai/tools with api_integration_id and api_integration_connection_id. Attach to the agent with "prompt": {"tool_ids": ["tool_xxxx"]}. Inline tools and tool_ids can coexist — prefer an integration over a duplicate custom webhook.

Public-API Webhook Examples

No-auth APIs useful for prototypes (URLs must be HTTPS):

Workflows

Route conversations through discrete steps with branching logic. Define under the agent's top-level workflow field. Reference: Agent Workflows.

Node types: start (ID must be "start_node"), end (outcome records the conversation as success, the default, or failure), override_agent (subagent step with label + additional_prompt), dispatch_tool (executes a tool with success/failure routing), agent_transfer, transfer_to_number.

Edge types: unconditional, llm (natural-language condition), expression (deterministic data check). Tool nodes have separate success/failure edges.

Scope tools per step with additional_tool_ids on a node — prevents the wrong tool firing at the wrong step. Set additional_tool_ids: [] on conversational routing nodes such as greeting and classify_intent so they only converse:

{
  "type": "override_agent",
  "label": "Book Appointment",
  "additional_prompt": "Discuss preferred dates and doctors. Show the booking form once agreed.",
  "entry_behavior": "wait_for_user",
  "additional_tool_ids": ["show_booking_form", "display_appointment_card"],
  "position": {"x": 0, "y": 400}
}

Include position ({x, y}) on every node so the editor renders cleanly. Start at y=0, put end at the bottom, and space branches horizontally at x=-150 and x=150; suggested spacing is 200px vertical between levels and 300px horizontal between branches. Keep workflows to 4-7 nodes and always have a path to end.

Use entry_behavior on override_agent nodes to choose whether a sub-agent speaks immediately (generate_immediately), waits for user input (wait_for_user), or lets the platform decide (auto).

For nested agent transfers, set enable_nesting on a standalone_agent node and return_when_nested on an end node that should return control to the parent workflow.

Procedures

Reusable instruction blocks an agent runs when a trigger matches. A procedure is free_form (markdown guidance the agent adapts, and the only type that can reference knowledge base documents) or deterministic (called "structured" in the dashboard: typed steps that run in a fixed order, for flows that must happen the same way every time). See Using the Procedure API for the full CLI and SDK flow, and Writing Procedures for the step reference, validation rules, and authoring guidance.

Procedures live on an agent branch, and every write stages a per-user draft:

Semantics worth knowing before writing any of these calls:

• Nothing reaches the live agent until you publish. Publishing is not a procedure endpoint; one PATCH on the agent versions every changed procedure draft on the branch.

• GET .../procedures/{procedure_id} reads branch HEAD and returns 404 until that procedure's first publish. Read the /draft variant to see a procedure you just created; do not retry the create.

• Turning a structured procedure's steps into the form the agent executes is called compiling. The platform compiles every structured procedure when the agent draft is saved or the agent is published; authors never compile anything themselves. Both draft save and publish validate every structured procedure on the branch. A failed operation writes nothing. To validate without publishing, save an agent draft with POST /v1/convai/agents/{agent_id}/drafts?branch_id=..., sending the agent's current name, conversation_config, platform_settings, and workflow unchanged, then discard that agent draft with DELETE on the same path; see Using the Procedure API.

• If a structured procedure is invalid, the publish or draft save returns 400 with status procedure_validation_failed and errors keyed by procedure ID, each entry carrying the path of the offending field and a message. Nothing is written; repair the procedure draft and publish again.

• A draft update replaces the whole body. Read the draft first, then resend name, type, and trigger alongside the new content. type cannot change after creation.

• content is markdown for a free_form procedure, and a JSON-encoded object with a steps array for a deterministic one. The trigger is the top-level trigger field in both cases, not part of content. Serialize it; do not hand-escape quotes.

• Manually authored workflow edges cannot connect to or from a structured procedure. Use a sub_procedure step to chain structured procedures; entry from a matching trigger and completion are platform-managed.

• A third type, folder, groups procedures in the dashboard. Folders carry no content or trigger; move a procedure into one with POST .../procedures/{procedure_id}/move.

• The agent keeps the five most recently started procedures in context. When more have been started, the content, inline tools, and knowledge base documents of the oldest free-form ones drop out of the prompt, although the procedures remain active. Keep procedures focused and use sub-procedures so that few are active at once.

• Routing is driven by the trigger text, not the procedure name. Write concrete, non-overlapping triggers that cover the phrasings a user would actually say. The model sees only a numbered menu of triggers; it never sees procedure names or IDs.

• To restrict the starting agent to selected procedures for one conversation, enable platform_settings.overrides.enable_procedure_ids_from_client, then pass their IDs as procedure_ids in conversation initiation data. An empty list disables all procedures for that starting agent. The same setting allows start_procedure_ids: up to five free-form procedures that start with the conversation, for when the user's task is already known (for example from an IVR choice or the page hosting the widget).

• Procedure APIs require elevenlabs (Python) or @elevenlabs/elevenlabs-js at 2.60.0 or newer.

Guardrails

Layered safety enforcement that runs independently of the LLM — configured under platform_settings.guardrails, not in the system prompt. Reference: Guardrails.

"platform_settings": {
  "guardrails": {
    "version": "1",
    "focus": {"is_enabled": true},
    "prompt_injection": {"is_enabled": true},
    "content": {"config": {"harassment": {"is_enabled": true, "threshold": 0.5}}},
    "custom": {
      "config": {
        "configs": [{
          "is_enabled": true,
          "name": "No medical diagnoses",
          "prompt": "Block the agent from providing medical diagnoses or treatment advice.",
          "execution_mode": "blocking",
          "model": "gemini-3.1-flash-lite",
          "history_message_count": 1,
          "trigger_action": {"type": "retry", "feedback": "Reason: {{trigger_reason}}"}
        }]
      }
    }
  }
}

Types: focus (on-topic), prompt_injection (manipulation defense), content (category filters), custom (LLM-evaluated domain rules). Content categories include harassment, profanity, sexual, violence, self_harm, and medical_and_legal_information — threshold range 0.0–1.0 (default 0.3). Custom rules use execution_mode: "blocking" with a model, history_message_count, and trigger_action (e.g., retry with feedback). Custom guardrails evaluate in parallel and fail-open.

Per vertical: healthcare/finance/legal → enable medical_and_legal_information; education/youth → sexual/violence/self_harm/profanity; support/sales → harassment/profanity. All agents benefit from focus + prompt_injection + 2-4 custom rules.

Testing Agents

Three test types via POST /v1/convai/agent-testing/create, then attached with PATCH on the agent. Reference: Agent Testing.

// Tool-call test (snake_case throughout; chat_history role is "user" or "agent")
{
  "name": "Books with correct doctor and date",
  "type": "tool",
  "chat_history": [
    {"role": "user", "message": "Dr. Smith on March 5 at 2pm", "time_in_call_secs": 10}
  ],
  "tool_call_parameters": {
    "referenced_tool": {"id": "show_booking_form", "type": "client"},
    "parameters": [
      {"path": "doctor_name", "eval": {"type": "llm", "description": "Should reference Dr. Smith"}},
      {"path": "date", "eval": {"type": "regex", "pattern": "2025-03-05|March 5"}}
    ]
  }
}

Eval strategies: exact, regex, llm. Prompt evaluation criteria can use binary scoring or numeric scoring with scoring_mode: "numeric_uniform", max_score, and score_instructions; numeric scores are normalized into the aggregate conversation success percentage. Attach via an agent update:

elevenlabs agents update --agent-id "your-agent-id" \
  --json '{"platform_settings": {"testing": {"attached_tests": [{"test_id": "test_xxxx"}]}}}'

Run selected tests with POST /v1/convai/agents/{agent_id}/run-tests. The request body requires tests and accepts repeat_count from 1 to 50 for repeated runs. Simulation tests can define up to 30 success_conditions prompts; all criteria are evaluated and merged into the final result. Simulation tests can also define tool_mock_overrides, keyed by tool ID, to replace shared response mocks for one test. Each override is an array of mocks with a required mock_result; set is_error: true to exercise a tool-failure path. Overrides only apply to tools enabled for mocking through tool_mock_config. For completed conversations, rerun one evaluation criterion with POST /v1/convai/conversations/{conversation_id}/analysis/evaluations/run and a request body containing evaluation_id.

Widget Embedding

<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>

Customize with attributes: avatar-image-url, action-text, start-call-text, end-call-text.

See Widget Embedding Reference for all options.

Outbound Calls

Make outbound phone calls using your agent via Twilio or Exotel integration:

The examples below use Twilio. See the reference for Exotel usage.

Python

response = client.conversational_ai.twilio.outbound_call(
    agent_id="your-agent-id",
    agent_phone_number_id="your-phone-number-id",
    to_number="+1234567890",
    call_recording_enabled=True
)
print(f"Call initiated: {response.conversation_id}")

JavaScript

const response = await client.conversationalAi.twilio.outboundCall({
  agentId: "your-agent-id",
  agentPhoneNumberId: "your-phone-number-id",
  toNumber: "+1234567890",
  callRecordingEnabled: true,
});

CLI

elevenlabs agents twilio outbound_call \
  --agent-id "your-agent-id" \
  --agent-phone-number-id "your-phone-number-id" \
  --to-number "+1234567890" \
  --call-recording-enabled true

See Outbound Calls Reference for provider-specific endpoints, configuration overrides, and dynamic variables.

Managing Agents

Using CLI (Recommended)

# List agents and check status
elevenlabs agents list
elevenlabs agents status

# Import agents from platform to local config
elevenlabs agents pull                      # Import all agents
elevenlabs agents pull --agent <agent-id>   # Import specific agent

# Push local changes to platform
elevenlabs agents push              # Upload configurations
elevenlabs agents push --dry-run    # Preview changes first

# Add tools
elevenlabs tools add-webhook "Weather API"
elevenlabs tools add-client "UI Tool"

Project Structure

The CLI creates a project structure for managing agents:

your_project/
├── agents.json       # Agent definitions
├── tools.json        # Tool configurations
├── tests.json        # Test configurations
├── agent_configs/    # Individual agent configs
├── tool_configs/     # Individual tool configs
└── test_configs/     # Individual test configs

SDK Examples

# List
agents = client.conversational_ai.agents.list()

# Get
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")

# Update (partial - only include fields to change)
client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name")
client.conversational_ai.agents.update(agent_id="your-agent-id",
    conversation_config={
        "agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}}
    })

# Delete
client.conversational_ai.agents.delete(agent_id="your-agent-id")

See Agent Configuration for all configuration options and SDK examples.

Error Handling

try:
    agent = client.conversational_ai.agents.create(...)
except Exception as e:
    print(f"API error: {e}")

Common errors: 401 (invalid key), 404 (not found), 422 (invalid config), 429 (rate limit)

References

• Installation Guide - SDK setup and migration

• Agent Configuration - All config options and CRUD examples

• Client Tools - Webhook, client, and system tools

• Using the Procedure API - Procedure CLI and SDK flow, drafts and publish

• Writing Procedures - Trigger and content authoring, structured step reference and rules

• Widget Embedding - Website integration

• Outbound Calls - Phone call integrations