---
title: OpenAI-compatible endpoint
slug: guides/server/providers/openai-proxy
---

<Warning>
OpenAI proxy endpoints are not officially supported and you are likely to encounter errors.
We strongly recommend using providers directly instead of via proxy endpoints (for example, using the Anthropic API directly instead of Claude through OpenRouter).
For questions and support you can chat with the dev team and community on our [Discord server](https://discord.gg/letta).
</Warning>

<Note>
To use OpenAI-compatible (`/v1/chat/completions`) endpoints with Letta, those endpoints must support function/tool calling.
</Note>

You can configure Letta to use OpenAI-compatible `ChatCompletions` endpoints by setting `OPENAI_API_BASE` in your environment variables (in addition to setting `OPENAI_API_KEY`).

## OpenRouter example

Create an account on [OpenRouter](https://openrouter.ai), then [create an API key](https://openrouter.ai/settings/keys).

Once you have your API key, set both `OPENAI_API_KEY` and `OPENAI_API_BASE` in your environment variables.

## Using Letta Server via Docker
Simply set the environment variables when you use `docker run`:
```bash
# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OPENAI_API_BASE="https://openrouter.ai/api/v1" \
  -e OPENAI_API_KEY="your_openai_api_key" \
  letta/letta:latest
```

## Using the Letta CLI
First we need to export the variables into our environment:
```sh
export OPENAI_API_KEY="sk-..."  # your OpenRouter API key
export OPENAI_API_BASE="https://openrouter.ai/api/v1"  # the OpenRouter OpenAI-compatible endpoint URL
```

Now, when we run `letta run` in the CLI, we can select OpenRouter models from the list of available models:
```
% letta run

? Would you like to select an existing agent? No

🧬 Creating new agent...
? Select LLM model: (Use arrow keys)
 » letta-free [type=openai] [ip=https://inference.letta.com]
   google/gemini-pro-1.5-exp [type=openai] [ip=https://openrouter.ai/api/v1]
   google/gemini-flash-1.5-exp [type=openai] [ip=https://openrouter.ai/api/v1]
   google/gemini-flash-1.5-8b-exp [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-11b-vision-instruct:free [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-1b-instruct:free [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-3b-instruct:free [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.1-8b-instruct:free [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-1b-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-3b-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   google/gemini-flash-1.5-8b [type=openai] [ip=https://openrouter.ai/api/v1]
   mistralai/mistral-7b-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   mistralai/mistral-7b-instruct-v0.3 [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3-8b-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.1-8b-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   meta-llama/llama-3.2-11b-vision-instruct [type=openai] [ip=https://openrouter.ai/api/v1]
   google/gemini-flash-1.5 [type=openai] [ip=https://openrouter.ai/api/v1]
   deepseek/deepseek-chat [type=openai] [ip=https://openrouter.ai/api/v1]
   openai/gpt-4o-mini [type=openai] [ip=https://openrouter.ai/api/v1]
   openai/gpt-4o-mini-2024-07-18 [type=openai] [ip=https://openrouter.ai/api/v1]
   mistralai/mistral-nemo [type=openai] [ip=https://openrouter.ai/api/v1]
   ...
```

For information on how to configure the Letta server or Letta Python SDK to use OpenRouter or other OpenAI-compatible endpoints providers, refer to [our guide on using OpenAI](/models/openai).
