Fimeg/letta-server
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add 'apps/core/' from commit 'ea2a7395f4023f5b9fab03e6273db3b64a1181d5'

Browse Source 
git-subtree-dir: apps/core
git-subtree-mainline: a8963e11e7a5a0059acbc849ce768e1eee80df61
git-subtree-split: ea2a7395f4023f5b9fab03e6273db3b64a1181d5
This commit is contained in:
Shubham Naik
2024-12-22 20:31:22 -08:00
commit 5a743d1dc4
478 changed files with 65642 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,39 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**Please describe your setup**
- [ ] How did you install letta?
- `pip install letta`? `pip install letta-nightly`? `git clone`?
- [ ] Describe your setup
- What's your OS (Windows/MacOS/Linux)?
- How are you running `letta`? (`cmd.exe`/Powershell/Anaconda Shell/Terminal)
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Additional context**
Add any other context about the problem here.
**Letta Config**
Please attach your `~/.letta/config` file or copy paste it below.
---
If you're not using OpenAI, please provide additional information on your local LLM setup:
**Local LLM details**
If you are trying to run Letta with local LLMs, please provide the following information:
- [ ] The exact model you're trying to use (e.g. `dolphin-2.1-mistral-7b.Q6_K.gguf`)
- [ ] The local LLM backend you are using (web UI? LM Studio?)
- [ ] Your hardware for the local LLM backend (local computer? operating system? remote RunPod?)

20
.github/ISSUE_TEMPLATE/feature_request.md vendored Normal file
View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

17
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,17 @@
**Please describe the purpose of this pull request.**
Is it to add a new feature? Is it to fix a bug?
**How to test**
How can we test your PR during review? What commands should we run? What outcomes should we expect?
**Have you tested this PR?**
Have you tested the latest commit on the PR? If so please provide outputs from your tests.
**Related issues or PRs**
Please link any related GitHub [issues](https://github.com/letta-ai/letta/issues) or [PRs](https://github.com/letta-ai/letta/pulls).
**Is your PR over 500 lines of code?**
If so, please break up your PR into multiple smaller PRs so that we can review them quickly, or provide justification for its length.
**Additional context**
Add any other context or screenshots about the PR here.

62
.github/workflows/check_for_new_prints.yml vendored Normal file
View File

@@ -0,0 +1,62 @@
name: Check for Print Statements
on:
pull_request:
paths:
- '**.py'
jobs:
check-print-statements:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Check for new print statements
run: |
# Get the files changed in this PR
git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }} > changed_files.txt
# Filter for only Python files, excluding tests directory
grep "\.py$" changed_files.txt | grep -v "^tests/" > python_files.txt || true
# Initialize error flag
ERROR=0
# Check each changed Python file
while IFS= read -r file; do
if [ "$file" == "letta/main.py" ]; then
echo "Skipping $file for print statement checks."
continue
fi
if [ -f "$file" ]; then
echo "Checking $file for new print statements..."
# Get diff and look for added lines containing print statements
NEW_PRINTS=$(git diff ${{ github.event.pull_request.base.sha }} ${{ github.sha }} "$file" | \
grep "^+" | \
grep -v "^+++" | \
grep -E "(^|\s)print\(" || true)
if [ ! -z "$NEW_PRINTS" ]; then
echo "❌ Found new print statements in $file:"
echo "$NEW_PRINTS"
ERROR=1
fi
fi
done < python_files.txt
# Exit with error if print statements were found
if [ $ERROR -eq 1 ]; then
echo "::error::New print statements were found in the changes"
exit 1
fi
echo "✅ No new print statements found"

22
.github/workflows/close_stale_issues.yml vendored Normal file
View File

@@ -0,0 +1,22 @@
name: Close inactive issues
on:
schedule:
- cron: "30 1 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v5
with:
days-before-issue-stale: 30
days-before-issue-close: 14
stale-issue-label: "stale"
stale-issue-message: "This issue is stale because it has been open for 30 days with no activity."
close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
days-before-pr-stale: -1
days-before-pr-close: -1
repo-token: ${{ secrets.GITHUB_TOKEN }}

50
.github/workflows/code_style_checks.yml vendored Normal file
View File

@@ -0,0 +1,50 @@
name: Code Style Checks
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
style-checks:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.12"] # Adjust Python version matrix if needed
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }} # Checkout the PR branch
fetch-depth: 0 # Fetch all history for all branches and tags
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: ${{ matrix.python-version }}
poetry-version: "1.8.2"
install-args: "-E dev -E postgres -E external-tools -E tests" # Adjust as necessary
- name: Validate PR Title
if: github.event_name == 'pull_request'
uses: amannn/action-semantic-pull-request@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Run Pyright
uses: jakebailey/pyright-action@v2
with:
python-version: ${{ matrix.python-version }}
level: "error"
continue-on-error: true
- name: Run isort
run: poetry run isort --profile black --check-only --diff .
- name: Run Black
run: poetry run black --check .
- name: Run Autoflake
run: poetry run autoflake --remove-all-unused-imports --remove-unused-variables --in-place --recursive --ignore-init-module-imports .

27
.github/workflows/docker-image-nightly.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
name: Docker Image CI (nightly)
on:
schedule:
- cron: '35 10 * * *' # 10:35am UTC, 2:35am PST, 5:35am EST
release:
types: [published]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: actions/checkout@v3
- name: Build and push the Docker image (letta)
run: |
docker build . --file Dockerfile --tag letta/letta:nightly
docker push letta/letta:nightly

41
.github/workflows/docker-image.yml vendored Normal file
View File

@@ -0,0 +1,41 @@
name: Docker Image CI
on:
release:
types: [published]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Extract version number
id: extract_version
run: echo "CURRENT_VERSION=$(awk -F '\"' '/version =/ { print $2 }' pyproject.toml | head -n 1)" >> $GITHUB_ENV
- name: Build and push
uses: docker/build-push-action@v6
with:
platforms: linux/amd64,linux/arm64
push: true
tags: |
letta/letta:${{ env.CURRENT_VERSION }}
letta/letta:latest
memgpt/letta:${{ env.CURRENT_VERSION }}
memgpt/letta:latest

65
.github/workflows/docker-integration-tests.yaml vendored Normal file
View File

@@ -0,0 +1,65 @@
name: Run Docker integration tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Set permissions for log directory
run: |
mkdir -p /home/runner/.letta/logs
sudo chown -R $USER:$USER /home/runner/.letta/logs
chmod -R 755 /home/runner/.letta/logs
- name: Build and run docker dev server
env:
LETTA_PG_DB: letta
LETTA_PG_USER: letta
LETTA_PG_PASSWORD: letta
LETTA_PG_PORT: 8888
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: docker compose -f dev-compose.yaml up --build -d
#- name: "Setup Python, Poetry and Dependencies"
# uses: packetcoders/action-setup-cache-python-poetry@v1.2.0
# with:
# python-version: "3.12"
# poetry-version: "1.8.2"
# install-args: "--all-extras"
- name: Wait for service
run: bash scripts/wait_for_service.sh http://localhost:8283 -- echo "Service is ready"
- name: Run tests with pytest
env:
LETTA_PG_DB: letta
LETTA_PG_USER: letta
LETTA_PG_PASSWORD: letta
LETTA_PG_PORT: 8888
LETTA_SERVER_PASS: test_server_token
LETTA_SERVER_URL: http://localhost:8283
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PYTHONPATH: ${{ github.workspace }}:${{ env.PYTHONPATH }}
run: |
pipx install poetry==1.8.2
poetry install -E dev -E postgres
poetry run pytest -s tests/test_client_legacy.py
- name: Print docker logs if tests fail
if: failure()
run: |
echo "Printing Docker Logs..."
docker compose -f dev-compose.yaml logs

81
.github/workflows/integration_tests.yml vendored Normal file
View File

@@ -0,0 +1,81 @@
name: Integration Tests
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
E2B_API_KEY: ${{ secrets.E2B_API_KEY }}
E2B_SANDBOX_TEMPLATE_ID: ${{ secrets.E2B_SANDBOX_TEMPLATE_ID }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
integ-run:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
integration_test_suite:
- "integration_test_summarizer.py"
- "integration_test_tool_execution_sandbox.py"
- "integration_test_offline_memory_agent.py"
- "integration_test_agent_tool_graph.py"
- "integration_test_o1_agent.py"
services:
qdrant:
image: qdrant/qdrant
ports:
- 6333:6333
postgres:
image: pgvector/pgvector:pg17
ports:
- 5432:5432
env:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_DB: postgres
POSTGRES_USER: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python, Poetry, and Dependencies
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E postgres -E external-tools -E tests -E cloud-tool-sandbox"
- name: Migrate database
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
run: |
psql -h localhost -U postgres -d postgres -c 'CREATE EXTENSION vector'
poetry run alembic upgrade head
- name: Run core unit tests
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
LETTA_SERVER_PASS: test_server_token
run: |
poetry run pytest -s -vv tests/${{ matrix.integration_test_suite }}

42
.github/workflows/letta-web-openapi-saftey.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: "Letta Web OpenAPI Compatibility Checker"
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
validate-openapi:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev"
- name: Checkout letta web
uses: actions/checkout@v4
with:
repository: letta-ai/letta-web
token: ${{ secrets.PULLER_TOKEN }}
path: letta-web
- name: Run OpenAPI schema generation
run: |
bash ./letta/server/generate_openapi_schema.sh
- name: Setup letta-web
working-directory: letta-web
run: npm ci
- name: Copy OpenAPI schema
working-directory: .
run: cp openapi_letta.json letta-web/libs/letta-agents-api/letta-agents-openapi.json
- name: Validate OpenAPI schema
working-directory: letta-web
run: |
npm run agents-api:generate
npm run type-check

85
.github/workflows/letta-web-safety.yml vendored Normal file
View File

@@ -0,0 +1,85 @@
name: "Letta Web Compatibility Checker"
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
cypress-run:
runs-on: ubuntu-latest
environment: Deployment
# Runs tests in parallel with matrix strategy https://docs.cypress.io/guides/guides/parallelization
# https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
# Also see warning here https://github.com/cypress-io/github-action#parallel
strategy:
fail-fast: false # https://github.com/cypress-io/github-action/issues/48
matrix:
containers: [ 1 ]
services:
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
postgres:
image: postgres
ports:
- 5433:5432
env:
POSTGRES_DB: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Checkout letta web
uses: actions/checkout@v4
with:
repository: letta-ai/letta-web
token: ${{ secrets.PULLER_TOKEN }}
path: letta-web
- name: Turn on Letta agents
env:
LETTA_PG_DB: letta
LETTA_PG_USER: letta
LETTA_PG_PASSWORD: letta
LETTA_PG_PORT: 8888
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: docker compose -f dev-compose.yaml up --build -d
- name: Cypress run
uses: cypress-io/github-action@v6
with:
working-directory: letta-web
build: npm run build:e2e
start: npm run start:e2e
project: apps/letta
wait-on: 'http://localhost:3000' # Waits for above
record: false
parallel: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CYPRESS_PROJECT_KEY: 38nemh
DATABASE_URL: postgres://postgres:postgres@localhost:5433/postgres
REDIS_HOST: localhost
REDIS_PORT: 6379
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
CYPRESS_GOOGLE_CLIENT_ID: ${{ secrets.CYPRESS_GOOGLE_CLIENT_ID }}
CYPRESS_GOOGLE_CLIENT_SECRET: ${{ secrets.CYPRESS_GOOGLE_CLIENT_SECRET }}
CYPRESS_GOOGLE_REFRESH_TOKEN: ${{ secrets.CYPRESS_GOOGLE_REFRESH_TOKEN }}
LETTA_AGENTS_ENDPOINT: http://localhost:8283
NEXT_PUBLIC_CURRENT_HOST: http://localhost:3000
IS_CYPRESS_RUN: yes
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

25
.github/workflows/manually_clear_old_issues.yml vendored Normal file
View File

@@ -0,0 +1,25 @@
name: Clear Old Issues
on:
workflow_dispatch:
jobs:
cleanup-old-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v5
with:
days-before-issue-stale: 60
days-before-issue-close: 0
stale-issue-label: "auto-closed"
stale-issue-message: ""
close-issue-message: "This issue has been automatically closed due to 60 days of inactivity."
days-before-pr-stale: -1
days-before-pr-close: -1
exempt-issue-labels: ""
only-issue-labels: ""
remove-stale-when-updated: true
operations-per-run: 1000
repo-token: ${{ secrets.GITHUB_TOKEN }}

44
.github/workflows/migration-test.yml vendored Normal file
View File

@@ -0,0 +1,44 @@
name: Alembic Migration Tester
on:
pull_request:
paths:
- '**.py'
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
services:
postgres:
image: pgvector/pgvector:pg17
ports:
- 5432:5432
env:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_DB: postgres
POSTGRES_USER: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v4
- run: psql -h localhost -U postgres -d postgres -c 'CREATE EXTENSION vector'
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "--all-extras"
- name: Test alembic migration
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
run: |
poetry run alembic upgrade head
poetry run alembic check

62
.github/workflows/poetry-publish-nightly.yml vendored Normal file
View File

@@ -0,0 +1,62 @@
name: poetry-publish-nightly
on:
schedule:
- cron: '35 10 * * *' # 10:35am UTC, 2:35am PST, 5:35am EST
release:
types: [published]
workflow_dispatch:
jobs:
# nightly release check from https://stackoverflow.com/a/67527144
check-date:
runs-on: ubuntu-latest
outputs:
should_run: ${{ steps.should_run.outputs.should_run }}
steps:
- uses: actions/checkout@v4
- name: print latest_commit
run: echo ${{ github.sha }}
- id: should_run
continue-on-error: true
name: check latest commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: test -z $(git rev-list --after="24 hours" ${{ github.sha }}) && echo "::set-output name=should_run::false"
build-and-publish-nightly:
name: Build and Publish to PyPI (nightly)
if: github.repository == 'letta-ai/letta' # TODO: if the repo org ever changes, this must be updated
runs-on: ubuntu-latest
needs: check-date
steps:
- name: Check out the repository
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.11"
poetry-version: "1.7.1"
- name: Set release version
run: |
# Extract the version number from pyproject.toml using awk
CURRENT_VERSION=$(awk -F '"' '/version =/ { print $2 }' pyproject.toml | head -n 1)
# Export the CURRENT_VERSION with the .dev and current date suffix
NIGHTLY_VERSION="${CURRENT_VERSION}.dev$(date +%Y%m%d%H%M%S)"
# Overwrite pyproject.toml with nightly config
sed -i "0,/version = \"${CURRENT_VERSION}\"/s//version = \"${NIGHTLY_VERSION}\"/" pyproject.toml
sed -i 's/name = "letta"/name = "letta-nightly"/g' pyproject.toml
sed -i "s/__version__ = '.*'/__version__ = '${NIGHTLY_VERSION}'/g" letta/__init__.py
cat pyproject.toml
cat letta/__init__.py
- name: Configure poetry
env:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN}}
run: poetry config pypi-token.pypi "$PYPI_TOKEN"
- name: Build the Python package
run: poetry build
- name: Publish the package to PyPI
run: poetry publish

32
.github/workflows/poetry-publish.yml vendored Normal file
View File

@@ -0,0 +1,32 @@
name: poetry-publish
on:
release:
types: [published]
workflow_dispatch:
jobs:
build-and-publish:
name: Build and Publish to PyPI
if: github.repository == 'letta-ai/letta' # TODO: if the repo org ever changes, this must be updated
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.11"
poetry-version: "1.7.1"
- name: Configure poetry
env:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
run: |
poetry config pypi-token.pypi "$PYPI_TOKEN"
- name: Build the Python package
run: poetry build
- name: Publish the package to PyPI
run: poetry publish

23
.github/workflows/test-pip-install.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: Test Package Installation
on: [push, pull_request, workflow_dispatch]
jobs:
test-install:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"] # Adjust Python versions as needed
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install package with extras
run: pip install '.[external-tools,postgres,dev,server,ollama]' # Replace 'all' with the key that includes all extras
- name: Check package installation
run: pip list # Or any other command to verify successful installation

102
.github/workflows/test_anthropic.yml vendored Normal file
View File

@@ -0,0 +1,102 @@
name: Anthropic Claude Opus 3 Capabilities Test
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E external-tools"
- name: Test first message contains expected function call and inner monologue
id: test_first_message
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_returns_valid_first_message
echo "TEST_FIRST_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model sends message with keyword
id: test_keyword_message
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_returns_keyword
echo "TEST_KEYWORD_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses external tool correctly
id: test_external_tool
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_uses_external_tool
echo "TEST_EXTERNAL_TOOL_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model recalls chat memory
id: test_chat_memory
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_recall_chat_memory
echo "TEST_CHAT_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses 'archival_memory_search' to find secret
id: test_archival_memory
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_archival_memory_retrieval
echo "TEST_ARCHIVAL_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model can edit core memories
id: test_core_memory
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_claude_opus_3_edit_core_memory
echo "TEST_CORE_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Summarize test results
if: always()
run: |
echo "Test Results Summary:"
echo "Test first message: $([[ $TEST_FIRST_MESSAGE_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
echo "Test model sends message with keyword: $([[ $TEST_KEYWORD_MESSAGE_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
echo "Test model uses external tool: $([[ $TEST_EXTERNAL_TOOL_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
echo "Test model recalls chat memory: $([[ $TEST_CHAT_MEMORY_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
echo "Test model uses 'archival_memory_search' to find secret: $([[ $TEST_ARCHIVAL_MEMORY_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
echo "Test model can edit core memories: $([[ $TEST_CORE_MEMORY_EXIT_CODE -eq 0 ]] && echo ✅ || echo ❌)"
# Check if any test failed
if [[ $TEST_FIRST_MESSAGE_EXIT_CODE -ne 0 || \
$TEST_KEYWORD_MESSAGE_EXIT_CODE -ne 0 || \
$TEST_EXTERNAL_TOOL_EXIT_CODE -ne 0 || \
$TEST_CHAT_MEMORY_EXIT_CODE -ne 0 || \
$TEST_ARCHIVAL_MEMORY_EXIT_CODE -ne 0 || \
$TEST_CORE_MEMORY_EXIT_CODE -ne 0 ]]; then
echo "Some tests failed."
exit 78
fi

111
.github/workflows/test_azure.yml vendored Normal file
View File

@@ -0,0 +1,111 @@
name: Azure OpenAI GPT-4o Mini Capabilities Test
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E external-tools"
- name: Test first message contains expected function call and inner monologue
id: test_first_message
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_returns_valid_first_message
echo "TEST_FIRST_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model sends message with keyword
id: test_keyword_message
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_returns_keyword
echo "TEST_KEYWORD_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses external tool correctly
id: test_external_tool
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_uses_external_tool
echo "TEST_EXTERNAL_TOOL_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model recalls chat memory
id: test_chat_memory
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_recall_chat_memory
echo "TEST_CHAT_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses 'archival_memory_search' to find secret
id: test_archival_memory
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_archival_memory_retrieval
echo "TEST_ARCHIVAL_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model can edit core memories
id: test_core_memory
env:
AZURE_API_KEY: ${{ secrets.AZURE_API_KEY }}
AZURE_BASE_URL: ${{ secrets.AZURE_BASE_URL }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_azure_gpt_4o_mini_edit_core_memory
echo "TEST_CORE_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Summarize test results
if: always()
run: |
echo "Test Results Summary:"
# If the exit code is empty, treat it as a failure (❌)
echo "Test first message: $([[ -z $TEST_FIRST_MESSAGE_EXIT_CODE || $TEST_FIRST_MESSAGE_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model sends message with keyword: $([[ -z $TEST_KEYWORD_MESSAGE_EXIT_CODE || $TEST_KEYWORD_MESSAGE_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model uses external tool: $([[ -z $TEST_EXTERNAL_TOOL_EXIT_CODE || $TEST_EXTERNAL_TOOL_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model recalls chat memory: $([[ -z $TEST_CHAT_MEMORY_EXIT_CODE || $TEST_CHAT_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model uses 'archival_memory_search' to find secret: $([[ -z $TEST_ARCHIVAL_MEMORY_EXIT_CODE || $TEST_ARCHIVAL_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model can edit core memories: $([[ -z $TEST_CORE_MEMORY_EXIT_CODE || $TEST_CORE_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
# Check if any test failed (either non-zero or unset exit code)
if [[ -z $TEST_FIRST_MESSAGE_EXIT_CODE || $TEST_FIRST_MESSAGE_EXIT_CODE -ne 0 || \
-z $TEST_KEYWORD_MESSAGE_EXIT_CODE || $TEST_KEYWORD_MESSAGE_EXIT_CODE -ne 0 || \
-z $TEST_EXTERNAL_TOOL_EXIT_CODE || $TEST_EXTERNAL_TOOL_EXIT_CODE -ne 0 || \
-z $TEST_CHAT_MEMORY_EXIT_CODE || $TEST_CHAT_MEMORY_EXIT_CODE -ne 0 || \
-z $TEST_ARCHIVAL_MEMORY_EXIT_CODE || $TEST_ARCHIVAL_MEMORY_EXIT_CODE -ne 0 || \
-z $TEST_CORE_MEMORY_EXIT_CODE || $TEST_CORE_MEMORY_EXIT_CODE -ne 0 ]]; then
echo "Some tests failed."
exit 78
fi
continue-on-error: true

67
.github/workflows/test_cli.yml vendored Normal file
View File

@@ -0,0 +1,67 @@
name: Test CLI
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test-cli:
runs-on: ubuntu-latest
timeout-minutes: 15
services:
qdrant:
image: qdrant/qdrant
ports:
- 6333:6333
postgres:
image: pgvector/pgvector:pg17
ports:
- 5432:5432
env:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_DB: postgres
POSTGRES_USER: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E postgres -E tests"
- name: Migrate database
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
run: |
psql -h localhost -U postgres -d postgres -c 'CREATE EXTENSION vector'
poetry run alembic upgrade head
- name: Test `letta run` up until first message
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
LETTA_SERVER_PASS: test_server_token
run: |
poetry run pytest -s -vv tests/test_cli.py::test_letta_run_create_new_agent

69
.github/workflows/test_examples.yml vendored Normal file
View File

@@ -0,0 +1,69 @@
name: Examples (documentation)
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Set permissions for log directory
run: |
mkdir -p /home/runner/.letta/logs
sudo chown -R $USER:$USER /home/runner/.letta/logs
chmod -R 755 /home/runner/.letta/logs
- name: Build and run docker dev server
env:
LETTA_PG_DB: letta
LETTA_PG_USER: letta
LETTA_PG_PASSWORD: letta
LETTA_PG_PORT: 8888
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: docker compose -f dev-compose.yaml up --build -d
#- name: "Setup Python, Poetry and Dependencies"
# uses: packetcoders/action-setup-cache-python-poetry@v1.2.0
# with:
# python-version: "3.12"
# poetry-version: "1.8.2"
# install-args: "--all-extras"
- name: Wait for service
run: bash scripts/wait_for_service.sh http://localhost:8283 -- echo "Service is ready"
- name: Run tests with pytest
env:
LETTA_PG_DB: letta
LETTA_PG_USER: letta
LETTA_PG_PASSWORD: letta
LETTA_PG_PORT: 8888
LETTA_SERVER_PASS: test_server_token
LETTA_SERVER_URL: http://localhost:8283
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PYTHONPATH: ${{ github.workspace }}:${{ env.PYTHONPATH }}
run: |
pipx install poetry==1.8.2
poetry install -E dev -E postgres -E external-tools
poetry run python examples/docs/agent_advanced.py
poetry run python examples/docs/agent_basic.py
poetry run python examples/docs/memory.py
poetry run python examples/docs/rest_client.py
poetry run python examples/docs/tools.py
- name: Print docker logs if tests fail
if: failure()
run: |
echo "Printing Docker Logs..."
docker compose -f dev-compose.yaml logs

31
.github/workflows/test_memgpt_hosted.yml vendored Normal file
View File

@@ -0,0 +1,31 @@
name: Endpoint (Letta)
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev"
- name: Test LLM endpoint
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_llm_endpoint_letta_hosted
continue-on-error: true
- name: Test embedding endpoint
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_embedding_endpoint_letta_hosted

87
.github/workflows/test_ollama.yml vendored Normal file
View File

@@ -0,0 +1,87 @@
name: Endpoint (Ollama)
env:
OLLAMA_BASE_URL: "http://localhost:11434"
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Install Ollama
run: |
set -e
set -x
curl -vfsSL https://ollama.com/install.sh -o install.sh
chmod +x install.sh
bash -x install.sh
if ! command -v ollama; then
echo "Ollama binary not found in PATH after installation."
exit 1
fi
echo "Ollama installed successfully."
- name: Start Ollama Server
run: |
set -e
set -x
ollama serve >ollama_server.log 2>&1 &
sleep 15
if ! curl -v http://localhost:11434; then
echo "Server logs (if available):"
[ -f ollama_server.log ] && cat ollama_server.log || echo "No logs found."
exit 1
fi
echo "Ollama server started successfully."
- name: Pull Models
run: |
set -e
set -x
for attempt in {1..3}; do
ollama pull thewindmom/hermes-3-llama-3.1-8b && break || sleep 5
done
for attempt in {1..3}; do
ollama pull mxbai-embed-large && break || sleep 5
done
- name: Debug Logs on Failure
if: failure()
run: |
echo "Debugging logs on failure:"
[ -f ollama_server.log ] && cat ollama_server.log || echo "No server logs available."
- name: Setup Python, Poetry, and Dependencies
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev"
- name: Test LLM Endpoint
run: |
set -e
set -x
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_llm_endpoint_ollama
- name: Test Embedding Endpoint
run: |
set -e
set -x
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_embedding_endpoint_ollama
- name: Test Provider
run: |
set -e
set -x
poetry run pytest -s -vv tests/test_providers.py::test_ollama

82
.github/workflows/test_openai.yml vendored Normal file
View File

@@ -0,0 +1,82 @@
name: OpenAI GPT-4 Capabilities Test
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E external-tools"
- name: Test first message contains expected function call and inner monologue
id: test_first_message
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_returns_valid_first_message
- name: Test model sends message with keyword
id: test_keyword_message
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_returns_keyword
- name: Test model uses external tool correctly
id: test_external_tool
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_uses_external_tool
- name: Test model recalls chat memory
id: test_chat_memory
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_recall_chat_memory
- name: Test model uses 'archival_memory_search' to find secret
id: test_archival_memory_search
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_archival_memory_retrieval
- name: Test model uses 'archival_memory_insert' to insert archival memories
id: test_archival_memory_insert
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_archival_memory_insert
- name: Test model can edit core memories
id: test_core_memory
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_openai_gpt_4o_edit_core_memory
- name: Test embedding endpoint
id: test_embedding_endpoint
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_embedding_endpoint_openai

105
.github/workflows/test_together.yml vendored Normal file
View File

@@ -0,0 +1,105 @@
name: Together Llama 3.1 70b Capabilities Test
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E external-tools"
- name: Test first message contains expected function call and inner monologue
id: test_first_message
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_returns_valid_first_message
echo "TEST_FIRST_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model sends message with keyword
id: test_keyword_message
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_returns_keyword
echo "TEST_KEYWORD_MESSAGE_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses external tool correctly
id: test_external_tool
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_uses_external_tool
echo "TEST_EXTERNAL_TOOL_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model recalls chat memory
id: test_chat_memory
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_recall_chat_memory
echo "TEST_CHAT_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model uses 'archival_memory_search' to find secret
id: test_archival_memory
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_archival_memory_retrieval
echo "TEST_ARCHIVAL_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Test model can edit core memories
id: test_core_memory
env:
TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
run: |
poetry run pytest -s -vv tests/test_model_letta_perfomance.py::test_together_llama_3_70b_edit_core_memory
echo "TEST_CORE_MEMORY_EXIT_CODE=$?" >> $GITHUB_ENV
continue-on-error: true
- name: Summarize test results
if: always()
run: |
echo "Test Results Summary:"
# If the exit code is empty, treat it as a failure (❌)
echo "Test first message: $([[ -z $TEST_FIRST_MESSAGE_EXIT_CODE || $TEST_FIRST_MESSAGE_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model sends message with keyword: $([[ -z $TEST_KEYWORD_MESSAGE_EXIT_CODE || $TEST_KEYWORD_MESSAGE_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model uses external tool: $([[ -z $TEST_EXTERNAL_TOOL_EXIT_CODE || $TEST_EXTERNAL_TOOL_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model recalls chat memory: $([[ -z $TEST_CHAT_MEMORY_EXIT_CODE || $TEST_CHAT_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model uses 'archival_memory_search' to find secret: $([[ -z $TEST_ARCHIVAL_MEMORY_EXIT_CODE || $TEST_ARCHIVAL_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
echo "Test model can edit core memories: $([[ -z $TEST_CORE_MEMORY_EXIT_CODE || $TEST_CORE_MEMORY_EXIT_CODE -ne 0 ]] && echo ❌ || echo ✅)"
# Check if any test failed (either non-zero or unset exit code)
if [[ -z $TEST_FIRST_MESSAGE_EXIT_CODE || $TEST_FIRST_MESSAGE_EXIT_CODE -ne 0 || \
-z $TEST_KEYWORD_MESSAGE_EXIT_CODE || $TEST_KEYWORD_MESSAGE_EXIT_CODE -ne 0 || \
-z $TEST_EXTERNAL_TOOL_EXIT_CODE || $TEST_EXTERNAL_TOOL_EXIT_CODE -ne 0 || \
-z $TEST_CHAT_MEMORY_EXIT_CODE || $TEST_CHAT_MEMORY_EXIT_CODE -ne 0 || \
-z $TEST_ARCHIVAL_MEMORY_EXIT_CODE || $TEST_ARCHIVAL_MEMORY_EXIT_CODE -ne 0 || \
-z $TEST_CORE_MEMORY_EXIT_CODE || $TEST_CORE_MEMORY_EXIT_CODE -ne 0 ]]; then
echo "Some tests failed."
exit 78
fi
continue-on-error: true

84
.github/workflows/tests.yml vendored Normal file
View File

@@ -0,0 +1,84 @@
name: Unit Tests
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
on:
push:
branches: [ main ]
pull_request:
jobs:
unit-run:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
test_suite:
- "test_vector_embeddings.py"
- "test_client.py"
- "test_client_legacy.py"
- "test_server.py"
- "test_v1_routes.py"
- "test_local_client.py"
- "test_managers.py"
- "test_base_functions.py"
- "test_tool_schema_parsing.py"
- "test_tool_rule_solver.py"
- "test_memory.py"
- "test_utils.py"
- "test_stream_buffer_readers.py"
services:
qdrant:
image: qdrant/qdrant
ports:
- 6333:6333
postgres:
image: pgvector/pgvector:pg17
ports:
- 5432:5432
env:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_DB: postgres
POSTGRES_USER: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python, Poetry, and Dependencies
uses: packetcoders/action-setup-cache-python-poetry@main
with:
python-version: "3.12"
poetry-version: "1.8.2"
install-args: "-E dev -E postgres -E external-tools -E tests"
- name: Migrate database
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
run: |
psql -h localhost -U postgres -d postgres -c 'CREATE EXTENSION vector'
poetry run alembic upgrade head
- name: Run core unit tests
env:
LETTA_PG_PORT: 5432
LETTA_PG_USER: postgres
LETTA_PG_PASSWORD: postgres
LETTA_PG_DB: postgres
LETTA_PG_HOST: localhost
LETTA_SERVER_PASS: test_server_token
run: |
poetry run pytest -s -vv tests/${{ matrix.test_suite }}

63
.github/workflows/warn_poetry_updates.yml vendored Normal file
View File

@@ -0,0 +1,63 @@
name: Check Poetry Dependencies Changes
on:
pull_request:
paths:
- 'poetry.lock'
- 'pyproject.toml'
jobs:
check-poetry-changes:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for poetry.lock changes
id: check-poetry-lock
run: |
if git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }} | grep -q "poetry.lock"; then
echo "poetry_lock_changed=true" >> $GITHUB_OUTPUT
else
echo "poetry_lock_changed=false" >> $GITHUB_OUTPUT
fi
- name: Check for pyproject.toml changes
id: check-pyproject
run: |
if git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }} | grep -q "pyproject.toml"; then
echo "pyproject_changed=true" >> $GITHUB_OUTPUT
else
echo "pyproject_changed=false" >> $GITHUB_OUTPUT
fi
- name: Create PR comment
if: steps.check-poetry-lock.outputs.poetry_lock_changed == 'true' || steps.check-pyproject.outputs.pyproject_changed == 'true'
uses: actions/github-script@v7
with:
script: |
const poetryLockChanged = ${{ steps.check-poetry-lock.outputs.poetry_lock_changed }};
const pyprojectChanged = ${{ steps.check-pyproject.outputs.pyproject_changed }};
let message = '📦 Dependencies Alert:\n\n';
if (poetryLockChanged && pyprojectChanged) {
message += '- Both `poetry.lock` and `pyproject.toml` have been modified\n';
} else if (poetryLockChanged) {
message += '- `poetry.lock` has been modified\n';
} else if (pyprojectChanged) {
message += '- `pyproject.toml` has been modified\n';
}
message += '\nPlease review these changes carefully to ensure they are intended (cc @sarahwooders @cpacker).';
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: message
});