349 lines
10 KiB
Markdown
349 lines
10 KiB
Markdown
# Matrix Bridge - Future Ideas & TODO
|
|
|
|
## Multi-Bot Room Support (Not Implemented - ACTIVE CONSIDERATION)
|
|
|
|
### Problem
|
|
|
|
If multiple bridges (Ani, Jean Luc, Sebastian) join the same room, they will respond to each other's messages in an infinite loop:
|
|
|
|
```
|
|
Jean Luc: "Hello"
|
|
↓ Ani receives → Letta →
|
|
Ani: "Hi Jean Luc!"
|
|
↓ Jean Luc receives → Letta →
|
|
Jean Luc: "Hello back!"
|
|
↓ Ani receives...
|
|
[loop forever]
|
|
```
|
|
|
|
### Proposed Solutions
|
|
|
|
#### Option A: Ignore Other Bot Messages (Simple)
|
|
|
|
Add configuration to ignore messages from known bots:
|
|
|
|
```bash
|
|
# .env
|
|
IGNORED_BOTS=@jeanluc:wiuf.net,@sebastian:wiuf.net
|
|
```
|
|
|
|
Implementation:
|
|
```python
|
|
IGNORED_BOTS = set(os.getenv("IGNORED_BOTS", "").split(","))
|
|
|
|
async def on_message(self, evt):
|
|
if str(evt.sender) in IGNORED_BOTS:
|
|
log.info(f"Ignoring message from bot {evt.sender}")
|
|
return
|
|
```
|
|
|
|
#### Option B: @Mention-Only Mode
|
|
|
|
Only respond when @mentioned:
|
|
|
|
```bash
|
|
# .env
|
|
RESPOND_ONLY_WHEN_MENTIONED=1
|
|
```
|
|
|
|
#### Option C: Primary/Observer Roles
|
|
|
|
Designated primary agent always responds, others stay silent unless they choose to interject via MCP tool:
|
|
|
|
```bash
|
|
# Ani (primary):
|
|
AGENT_ROLE=primary
|
|
|
|
# Jean Luc, Sebastian (observers):
|
|
AGENT_ROLE=observer
|
|
```
|
|
|
|
**Observer behavior:**
|
|
- Reads all messages (build context)
|
|
- Can interject via `matrix-send-message` MCP tool
|
|
- Tags response: `**Jean Luc**: <message>`
|
|
- Normal Letta response NOT sent (silent)
|
|
|
|
---
|
|
|
|
## Access Control (Not Implemented)
|
|
|
|
### Current State
|
|
- Bridge auto-accepts ALL invites
|
|
- Responds to ANYONE in joined rooms
|
|
- No sender filtering
|
|
- Invitation to random users not prevented
|
|
|
|
### Proposed Implementation
|
|
|
|
#### 1. Configuration Variables
|
|
```bash
|
|
AUTO_ACCEPT_INVITES=0 # Disable auto-join
|
|
ALLOWED_USERS=@casey:wiuf.net,@xzaviar:wiuf.net # Comma-separated allowlist
|
|
```
|
|
|
|
#### 2. New Instance Variables
|
|
```python
|
|
self.pending_invites: list = [] # Track pending invitations
|
|
self.allowed_users: set = set() # Allowed Matrix user IDs
|
|
```
|
|
|
|
#### 3. Modified `handle_member` (Invite Handler)
|
|
```python
|
|
@self.client.on(EventType.ROOM_MEMBER)
|
|
async def handle_member(evt: StateEvent):
|
|
# Handle invites
|
|
if (evt.state_key == str(self.user_id) and
|
|
evt.content.membership == Membership.INVITE):
|
|
|
|
inviter = str(evt.sender) if evt.sender else "unknown"
|
|
|
|
# Check if auto-accept enabled and inviter allowed
|
|
if AUTO_ACCEPT and inviter in ALLOWED_USERS:
|
|
await self.client.join_room(evt.room_id)
|
|
log.info(f"Auto-accepted invite from {inviter}")
|
|
else:
|
|
# Track pending invite
|
|
self.pending_invites.append({
|
|
"room_id": str(evt.room_id),
|
|
"inviter": inviter,
|
|
"timestamp": datetime.now().isoformat()
|
|
})
|
|
log.info(f"Pending invite from {inviter} to {evt.room_id}")
|
|
# Agent can decide via API
|
|
```
|
|
|
|
#### 4. Modified `on_message` (Message Handler)
|
|
```python
|
|
async def on_message(self, evt):
|
|
# Ignore messages during initial sync
|
|
if not self.initial_sync_done:
|
|
return
|
|
|
|
# Ignore old messages (more than 60 seconds old)
|
|
event_time = datetime.fromtimestamp(evt.timestamp / 1000)
|
|
message_age = datetime.now() - event_time
|
|
if message_age > timedelta(seconds=60):
|
|
return
|
|
|
|
# Ignore own messages
|
|
if evt.sender == self.user_id:
|
|
return
|
|
|
|
# NEW: Check if sender is allowed
|
|
sender = str(evt.sender)
|
|
if sender not in self.allowed_users:
|
|
log.info(f"Ignoring message from disallowed user {sender}")
|
|
return
|
|
|
|
# Process message normally...
|
|
```
|
|
|
|
#### 5. API Endpoints for Invite Management
|
|
|
|
| Endpoint | Method | Purpose |
|
|
|----------|--------|---------|
|
|
| `GET /api/invites` | List pending invitations |
|
|
| `POST /api/invites/{room_id}/accept` | Accept specific invitation |
|
|
| `POST /api/invites/{room_id}/decline` | Decline invitation |
|
|
| `POST /api/invites/{room_id}/leave` | Leave room (cleanup) |
|
|
|
|
#### 6. MCP Usage Pattern
|
|
```python
|
|
# Agent checks for pending invites
|
|
invites = requests.post("http://localhost:8284/api/invites").json()
|
|
|
|
# Agent decides to accept an invite
|
|
if invites["pending"] and should_accept(invites[0]["inviter"]):
|
|
requests.post(f"http://localhost:8284/api/invites/{invites[0]['room_id']}/accept")
|
|
```
|
|
|
|
---
|
|
|
|
## Migrate to mautrix.util.formatter (IN PROGRESS - Copy Branch)
|
|
|
|
### Status: Design Complete, Awaiting Implementation
|
|
|
|
**Working File:** `ani_e2ee_bridge.py` (refactor branch)
|
|
**Original:** `bridge-e2ee.py` (preserved as safety)
|
|
|
|
### Current State - UPDATED 2026-02-07
|
|
|
|
**Current Implementation:**
|
|
- Using `python-markdown` library with manual patches (~100 lines)
|
|
- Regex-based `{color|text}` and `||spoiler||` processing
|
|
- False code block detection is manual (keeps breaking)
|
|
- HTML pass-through is patched
|
|
|
|
**Already Completed (✅):**
|
|
- Extracted `_handle_success_response()` method to eliminate duplication between `on_message()` and `process_queue()`
|
|
- Reduced ~50 lines of duplicated SUCCESS handler code
|
|
|
|
**Findings from mautrix-python docs:**
|
|
- `parse_html()` is **async** - returns coroutine, requires await
|
|
- `MarkdownString.format(EventType.ROOM_MESSAGE)` is **sync** but requires EntityType argument
|
|
- `MatrixParser.parse()` is **async**
|
|
|
|
**Impact:** ~25 call sites need to be updated to await async format_html()
|
|
|
|
### Why Migrate?
|
|
|
|
- Native Matrix formatting support
|
|
- Proper EntityType enum (COLOR, SPOILER, etc.)
|
|
- Built-in mention, pill, and room reference handling
|
|
- More robust HTML→Markdown round-tripping
|
|
- ~100 fewer lines of maintenance code
|
|
|
|
### API Documentation References
|
|
|
|
```python
|
|
from mautrix.util.formatter import parse_html, MarkdownString, MatrixParser
|
|
from mautrix.types import EventType
|
|
|
|
# HTML → Plain Text (ASYNC)
|
|
plain_text = await parse_html(html_input)
|
|
|
|
# Markdown → HTML (SYNC)
|
|
markdown = MarkdownString("**Bold** and ||spoiler||")
|
|
html_output = markdown.format(EventType.ROOM_MESSAGE)
|
|
|
|
# Mentions and pills (ASYNC)
|
|
parser = MatrixParser()
|
|
formatted = await parser.parse("Hello @user:example.com")
|
|
```
|
|
|
|
### Implementation Plan
|
|
|
|
**See `BRIDGE_DESIGN.md` for detailed refactor plan including:**
|
|
|
|
1. Phase 1: Infrastructure (2h) - async format_html(), color syntax helper
|
|
2. Phase 2: Update Call Sites (1h) - ~25 await statements needed
|
|
3. Phase 3: Remove Dead Code (30m) - delete manual patches
|
|
4. Phase 4: Testing (1h) - formatting regression tests
|
|
|
|
**Estimated total: ~4.5h**
|
|
|
|
### Detailed Implementation (FROM DOCS - UPDATED)
|
|
|
|
#### New async format_html()
|
|
|
|
```python
|
|
async def format_html(text: str) -> tuple[str, str]:
|
|
"""
|
|
Format text using mautrix native formatter.
|
|
|
|
Args:
|
|
text: Response from Letta (markdown or HTML)
|
|
|
|
Returns:
|
|
(plain_text, html_body) tuple
|
|
"""
|
|
try:
|
|
# Strip whitespace
|
|
text = text.strip()
|
|
|
|
# Convert emoji shortcodes (keep existing behavior)
|
|
text = normalize_emoji_shortcodes(text)
|
|
text = emoji.emojize(text, language='en')
|
|
|
|
# HTML path → parse to plain (ASYNC)
|
|
if text.startswith('<') and '>' in text:
|
|
# Pre-process {color|text} - mautrix doesn't handle this
|
|
text = _apply_color_syntax(text.strip())
|
|
plain = await parse_html(text)
|
|
return plain, text
|
|
|
|
# Markdown path → use MarkdownString (SYNC)
|
|
md = MarkdownString(text)
|
|
# Pre-process {color|text} syntax
|
|
processed_md = _apply_color_syntax(md.text)
|
|
md.text = processed_md
|
|
|
|
# Format to HTML (SYNC)
|
|
html = md.format(EventType.ROOM_MESSAGE)
|
|
|
|
# Generate plain text (ASYNC)
|
|
plain = await parse_html(html)
|
|
|
|
return plain, html
|
|
|
|
except Exception as e:
|
|
log.warning(f"HTML formatting failed: {e}")
|
|
return emoji.emojize(text), emoji.emojize(text)
|
|
|
|
def _apply_color_syntax(text: str) -> str:
|
|
"""Convert {color|text} to HTML spans (adapter for existing syntax)."""
|
|
def replace_color(match):
|
|
color = match.group(1)
|
|
content = match.group(2)
|
|
hex_color = MATRIX_COLORS.get(color, color)
|
|
return f'<font color="{hex_color}" data-mx-color="{hex_color}">{content}</font>'
|
|
return re.sub(r'\{([a-zA-Z0-9_#]+)\|([^}]+)\}', replace_color, text)
|
|
```
|
|
|
|
#### Updated send_message()
|
|
|
|
```python
|
|
async def send_message(self, room_id: RoomID, text: str) -> str | None:
|
|
"""
|
|
Send a formatted message to a room (auto-encrypts if needed).
|
|
"""
|
|
# Format text as HTML with full markdown and emoji support
|
|
plain_text, html_body = await format_html(text) # NOW AWAIT
|
|
|
|
# Create content with both plain text and formatted HTML
|
|
content = {
|
|
"msgtype": "m.text",
|
|
"body": plain_text,
|
|
"format": "org.matrix.custom.html",
|
|
"formatted_body": html_body,
|
|
}
|
|
|
|
event_id = await self.client.send_message_event(room_id, EventType.ROOM_MESSAGE, content)
|
|
return str(event_id) if event_id else None
|
|
```
|
|
|
|
---
|
|
|
|
## Other Ideas
|
|
|
|
### Room Metadata in Context
|
|
Let the agent know room type (DM vs public), member count, room name
|
|
|
|
```python
|
|
# In on_message(), add room metadata
|
|
room_info = await self.client.get_state_event(room_id, EventType.ROOM_NAME)
|
|
member_count = await self.get_member_count(room_id)
|
|
is_dm = member_count == 2
|
|
|
|
# Build richer context for Letta
|
|
context = f"[Room: {room_name} ({member_count} members, {'DM' if is_dm else 'group'})]"
|
|
```
|
|
|
|
### Unified Context Option
|
|
Share conversations between specific rooms (e.g., DM + heartbeat room)
|
|
|
|
```python
|
|
# Map multiple room IDs to same conversation ID
|
|
ROOM_CONTEXT_MAP = {
|
|
"!dm-room": "shared-context-1",
|
|
"!heartbeat-room": "shared-context-1",
|
|
"!public-room": "shared-context-2"
|
|
}
|
|
```
|
|
|
|
### Feedback Loop Improvements
|
|
- Store which messages got positive/negative reactions
|
|
- Let agent see reaction patterns over time
|
|
- Use feedback for preference learning
|
|
|
|
### Multi-Agent Bridge Support
|
|
- Allow multiple agents in same room with different personae
|
|
- Route messages to specific agents based on @mentions or keywords
|
|
- Agent-to-agent conversation capability
|
|
|
|
---
|
|
|
|
**Last Updated**: 2026-02-05
|
|
**Status**: Ideas not yet implemented - safe to work on in future sessions
|