letta-server/fern/pages/agents/json_mode.mdx

---
title: JSON Mode & Structured Output
subtitle: Get structured JSON responses from your Letta agents
slug: guides/agents/json-mode
---

Letta provides two ways to get structured JSON output from agents: **Structured Generation through Tools** (recommended) and the `response_format` parameter.

## Quick Comparison

<Note>
**Recommended**: Use **Structured Generation through Tools** - works with all providers (Anthropic, OpenAI, Google, etc.) and integrates naturally with Letta's tool-calling architecture.
</Note>

<Info>
**Structured Generation through Tools**:
- ✅ Universal provider compatibility
- ✅ Both reasoning AND structured output
- ✅ Per-message control
- ✅ Works even as "dummy tool" for pure formatting
</Info>

<Warning>
**`response_format` parameter**:
- ⚠️ OpenAI-compatible providers only (NOT Anthropic)
- ⚠️ Persistent agent state (affects all future responses)

- ✅ Built-in provider schema enforcement
</Warning>

## Structured Generation through Tools (Recommended)

Create a tool that defines your desired response format. The tool arguments become your structured data, and you can extract them from the tool call.

### Creating a Structured Generation Tool

<CodeGroup>
```typescript TypeScript maxLines=100
import { LettaClient } from '@letta-ai/letta-client'

// Create client (Letta Cloud)
const client = new LettaClient({ token: "LETTA_API_KEY" });

// Or for self-hosted
// const client = new LettaClient({ baseUrl: "http://localhost:8283" });

// First create the tool
const toolCode = `def generate_rank(rank: int, reason: str):
    """Generate a ranking with explanation.

    Args:
        rank (int): The numerical rank from 1-10.
        reason (str): The reasoning behind the rank.
    """
    print("Rank generated")
    return`;

const tool = await client.tools.create({
    sourceCode: toolCode,
    sourceType: "python"
});

// Create agent with the structured generation tool
const agentState = await client.agents.create({
    model: "openai/gpt-4o-mini",
    embedding: "openai/text-embedding-3-small",
    memoryBlocks: [
        {
            label: "human",
            value: "The human's name is Chad. They are a food enthusiast who enjoys trying different cuisines."
        },
        {
            label: "persona",
            value: "I am a helpful food critic assistant. I provide detailed rankings and reviews of different foods and restaurants."
        }
    ],
    toolIds: [tool.id]
});
```

```python title="python" maxLines=100
from letta_client import Letta

# Create client (Letta Cloud)
client = Letta(token="LETTA_API_KEY")

# Or for self-hosted
# client = Letta(base_url="http://localhost:8283")

def generate_rank(rank: int, reason: str):
    """Generate a ranking with explanation.

    Args:
        rank (int): The numerical rank from 1-10.
        reason (str): The reasoning behind the rank.
    """
    print("Rank generated")
    return

# Create the tool
tool = client.tools.create(func=generate_rank)

# Create agent with the structured generation tool
agent_state = client.agents.create(
    model="openai/gpt-4o-mini",
    embedding="openai/text-embedding-3-small",
    memory_blocks=[
        {
            "label": "human",
            "value": "The human's name is Chad. They are a food enthusiast who enjoys trying different cuisines."
        },
        {
            "label": "persona",
            "value": "I am a helpful food critic assistant. I provide detailed rankings and reviews of different foods and restaurants."
        }
    ],
    tool_ids=[tool.id]
)
```
</CodeGroup>

### Using the Structured Generation Tool

<CodeGroup>
```typescript TypeScript maxLines=100
// Send message and instruct agent to use the tool
const response = await client.agents.messages.create(
    agentState.id, {
        messages: [
            {
                role: "user",
                content: "How do you rank sushi as a food? Please use the generate_rank tool to provide your response."
            }
        ]
    }
);

// Extract structured data from tool call
for (const message of response.messages) {
    if (message.messageType === "tool_call_message") {
        const args = JSON.parse(message.toolCall.arguments);
        console.log(`Rank: ${args.rank}`);
        console.log(`Reason: ${args.reason}`);
    }
}

// Example output:
// Rank: 8
// Reason: Sushi is a highly regarded cuisine known for its fresh ingredients...
```

```python title="python" maxLines=100
# Send message and instruct agent to use the tool
response = client.agents.messages.create(
    agent_id=agent_state.id,
    messages=[
        {
            "role": "user",
            "content": "How do you rank sushi as a food? Please use the generate_rank tool to provide your response."
        }
    ]
)

# Extract structured data from tool call
for message in response.messages:
    if message.message_type == "tool_call_message":
        import json
        args = json.loads(message.tool_call.arguments)
        rank = args["rank"]
        reason = args["reason"]
        print(f"Rank: {rank}")
        print(f"Reason: {reason}")

# Example output:
# Rank: 8
# Reason: Sushi is a highly regarded cuisine known for its fresh ingredients...
```
</CodeGroup>

The agent will call the tool, and you can extract the structured arguments:

```json
{
  "rank": 8,
  "reason": "Sushi is a highly regarded cuisine known for its fresh ingredients, artistic presentation, and cultural significance."
}
```

## Using `response_format` for Provider-Native JSON Mode

The `response_format` parameter enables structured output/JSON mode from LLM providers that support it. This approach is fundamentally different from tools because **`response_format` becomes a persistent part of the agent's state** - once set, all future responses from that agent will follow the format until explicitly changed.

Under the hood, `response_format` constrains the agent's assistant messages to follow the specified schema, but it doesn't affect tools - those continue to work normally with their original schemas.

<Warning>
**Requirements for `response_format`:**
- Only works with providers that support structured outputs (like OpenAI) - NOT Anthropic or other providers

</Warning>

### Basic JSON Mode

<CodeGroup>
```typescript TypeScript maxLines=100
import { LettaClient } from '@letta-ai/letta-client'

// Create client (Letta Cloud)
const client = new LettaClient({ token: "LETTA_API_KEY" });

// Create agent with basic JSON mode (OpenAI/compatible providers only)
const agentState = await client.agents.create({
    model: "openai/gpt-4o-mini",
    embedding: "openai/text-embedding-3-small",
    memoryBlocks: [
        {
            label: "human",
            value: "The human's name is Chad. They work as a data analyst and prefer clear, organized information."
        },
        {
            label: "persona",
            value: "I am a helpful assistant who provides clear and well-organized responses."
        }
    ],
    responseFormat: { type: "json_object" }
});

// Send message expecting JSON response
const response = await client.agents.messages.create(
    agentState.id, {
        messages: [
            {
                role: "user",
                content: "How do you rank sushi as a food? Please respond in JSON format with rank and reason fields."
            }
        ]
    }
);

for (const message of response.messages) {
    console.log(message);
}
```

```python title="python" maxLines=100
from letta_client import Letta

# Create client (Letta Cloud)
client = Letta(token="LETTA_API_KEY")

# Create agent with basic JSON mode (OpenAI/compatible providers only)
agent_state = client.agents.create(
    model="openai/gpt-4o-mini",
    embedding="openai/text-embedding-3-small",
    memory_blocks=[
        {
            "label": "human",
            "value": "The human's name is Chad. They work as a data analyst and prefer clear, organized information."
        },
        {
            "label": "persona",
            "value": "I am a helpful assistant who provides clear and well-organized responses."
        }
    ],
    response_format={"type": "json_object"}
)

# Send message expecting JSON response
response = client.agents.messages.create(
    agent_id=agent_state.id,
    messages=[
        {
            "role": "user",
            "content": "How do you rank sushi as a food? Please respond in JSON format with rank and reason fields."
        }
    ]
)

for message in response.messages:
    print(message)
```
</CodeGroup>

### Advanced JSON Schema Mode

For more precise control, you can use OpenAI's `json_schema` mode with strict validation:

<CodeGroup>
```typescript TypeScript maxLines=100
import { LettaClient } from '@letta-ai/letta-client'

const client = new LettaClient({ token: "LETTA_API_KEY" });

// Define structured schema (from OpenAI structured outputs guide)
const responseFormat = {
    type: "json_schema",
    jsonSchema: {
        name: "food_ranking",
        schema: {
            type: "object",
            properties: {
                rank: {
                    type: "integer",
                    minimum: 1,
                    maximum: 10
                },
                reason: {
                    type: "string"
                },
                categories: {
                    type: "array",
                    items: {
                        type: "object",
                        properties: {
                            name: { type: "string" },
                            score: { type: "integer" }
                        },
                        required: ["name", "score"],
                        additionalProperties: false
                    }
                }
            },
            required: ["rank", "reason", "categories"],
            additionalProperties: false
        },
        strict: true
    }
};

// Create agent
const agentState = await client.agents.create({
    model: "openai/gpt-4o-mini",
    embedding: "openai/text-embedding-3-small",
    memoryBlocks: []
});

// Update agent with response format
const updatedAgent = await client.agents.update(
    agentState.id,
    { responseFormat }
);

// Send message
const response = await client.agents.messages.create(
    agentState.id, {
        messages: [
            { role: "user", content: "How do you rank sushi? Include categories for taste, presentation, and value." }
        ]
    }
);

for (const message of response.messages) {
    console.log(message);
}
```

```python title="python" maxLines=100
from letta_client import Letta

client = Letta(token="LETTA_API_KEY")

# Define structured schema (from OpenAI structured outputs guide)
response_format = {
    "type": "json_schema",
    "json_schema": {
        "name": "food_ranking",
        "schema": {
            "type": "object",
            "properties": {
                "rank": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 10
                },
                "reason": {
                    "type": "string"
                },
                "categories": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": { "type": "string" },
                            "score": { "type": "integer" }
                        },
                        "required": ["name", "score"],
                        "additionalProperties": False
                    }
                }
            },
            "required": ["rank", "reason", "categories"],
            "additionalProperties": False
        },
        "strict": True
    }
}

# Create agent
agent_state = client.agents.create(
    model="openai/gpt-4o-mini",
    embedding="openai/text-embedding-3-small",
    memory_blocks=[]
)

# Update agent with response format
agent_state = client.agents.update(
    agent_id=agent_state.id,
    response_format=response_format
)

# Send message
response = client.agents.messages.create(
    agent_id=agent_state.id,
    messages=[
        {"role": "user", "content": "How do you rank sushi? Include categories for taste, presentation, and value."}
    ]
)

for message in response.messages:
    print(message)
```
</CodeGroup>

With structured JSON schema, the agent's response will be strictly validated:

```json
{
  "rank": 8,
  "reason": "Sushi is highly regarded for its fresh ingredients and artful presentation",
  "categories": [
    {"name": "taste", "score": 9},
    {"name": "presentation", "score": 10},
    {"name": "value", "score": 6}
  ]
}
```


## Updating Agent Response Format

You can update an existing agent's response format:

<CodeGroup>
```typescript TypeScript maxLines=100
// Update agent to use JSON mode (OpenAI/compatible only)
await client.agents.update(agentState.id, {
    responseFormat: { type: "json_object" }
});

// Or remove JSON mode
await client.agents.update(agentState.id, {
    responseFormat: null
});
```

```python title="python" maxLines=100
# Update agent to use JSON mode (OpenAI/compatible only)
client.agents.update(
    agent_id=agent_state.id,
    response_format={"type": "json_object"}
)

# Or remove JSON mode
client.agents.update(
    agent_id=agent_state.id,
    response_format=None
)
```
</CodeGroup>