---
title: Groq
slug: guides/server/providers/groq
---


<Tip>To use Letta with Groq, set the environment variable `GROQ_API_KEY=...`</Tip>

You can use Letta with Groq if you have a Groq account and API key. Once you have set your `GROQ_API_KEY` in your environment variables, you can select what model and configure the context window size.

## Enabling Groq as a provider
To enable the Groq provider, you must set the `GROQ_API_KEY` environment variable. When this is set, Letta will use available LLM models running on Groq.

### Using the `docker run` server with Groq
To enable Groq models, simply set your `GROQ_API_KEY` as an environment variable:
```bash
# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e GROQ_API_KEY="your_groq_api_key" \
  letta/letta:latest
```

<Accordion icon="square-terminal" title="CLI (pypi only)">
### Using `letta run` and `letta server` with Groq
To chat with an agent, run:
```bash
export GROQ_API_KEY="gsk-..."
letta run
```
This will prompt you to select a model:
```bash
? Select LLM model: (Use arrow keys)
 » letta-free [type=openai] [ip=https://inference.letta.com]
   llama-3.2-11b-text-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   gemma-7b-it [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.1-8b-instant [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-guard-3-8b [type=openai] [ip=https://api.groq.com/openai/v1]
   whisper-large-v3-turbo [type=openai] [ip=https://api.groq.com/openai/v1]
   llama3-70b-8192 [type=openai] [ip=https://api.groq.com/openai/v1]
   gemma2-9b-it [type=openai] [ip=https://api.groq.com/openai/v1]
   llama3-groq-8b-8192-tool-use-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   llama3-8b-8192 [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.2-1b-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   mixtral-8x7b-32768 [type=openai] [ip=https://api.groq.com/openai/v1]
   llava-v1.5-7b-4096-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.2-3b-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   distil-whisper-large-v3-en [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.2-90b-text-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   llama3-groq-70b-8192-tool-use-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.1-70b-versatile [type=openai] [ip=https://api.groq.com/openai/v1]
   llama-3.2-11b-vision-preview [type=openai] [ip=https://api.groq.com/openai/v1]
   whisper-large-v3 [type=openai] [ip=https://api.groq.com/openai/v1]
```
To run the Letta server, run:
```bash
export GROQ_API_KEY="gsk-..."
letta server
```
To select the model used by the server, use the dropdown in the ADE or specify a `LLMConfig` object in the Python SDK.
</Accordion>