matrix-bridge-legacy/ideasmaybe.md

Matrix Bridge - Future Ideas & TODO

Multi-Bot Room Support (Not Implemented - ACTIVE CONSIDERATION)

Problem

If multiple bridges (Ani, Jean Luc, Sebastian) join the same room, they will respond to each other's messages in an infinite loop:

Jean Luc: "Hello"
  ↓ Ani receives → Letta →
Ani: "Hi Jean Luc!"
  ↓ Jean Luc receives → Letta →
Jean Luc: "Hello back!"
  ↓ Ani receives...
[loop forever]

Proposed Solutions

Option A: Ignore Other Bot Messages (Simple)

Add configuration to ignore messages from known bots:

# .env
IGNORED_BOTS=@jeanluc:wiuf.net,@sebastian:wiuf.net

Implementation:

IGNORED_BOTS = set(os.getenv("IGNORED_BOTS", "").split(","))

async def on_message(self, evt):
    if str(evt.sender) in IGNORED_BOTS:
        log.info(f"Ignoring message from bot {evt.sender}")
        return

Option B: @Mention-Only Mode

Only respond when @mentioned:

# .env
RESPOND_ONLY_WHEN_MENTIONED=1

Option C: Primary/Observer Roles

Designated primary agent always responds, others stay silent unless they choose to interject via MCP tool:

# Ani (primary):
AGENT_ROLE=primary

# Jean Luc, Sebastian (observers):
AGENT_ROLE=observer

Observer behavior:

• Reads all messages (build context)
• Can interject via matrix-send-message MCP tool
• Tags response: **Jean Luc**: <message>
• Normal Letta response NOT sent (silent)

---

Access Control (Not Implemented)

Current State

• Bridge auto-accepts ALL invites
• Responds to ANYONE in joined rooms
• No sender filtering
• Invitation to random users not prevented

Proposed Implementation

1. Configuration Variables

AUTO_ACCEPT_INVITES=0                                   # Disable auto-join
ALLOWED_USERS=@casey:wiuf.net,@xzaviar:wiuf.net        # Comma-separated allowlist

2. New Instance Variables

self.pending_invites: list = []        # Track pending invitations
self.allowed_users: set = set()        # Allowed Matrix user IDs

3. Modified handle_member (Invite Handler)

@self.client.on(EventType.ROOM_MEMBER)
async def handle_member(evt: StateEvent):
    # Handle invites
    if (evt.state_key == str(self.user_id) and
        evt.content.membership == Membership.INVITE):

        inviter = str(evt.sender) if evt.sender else "unknown"

        # Check if auto-accept enabled and inviter allowed
        if AUTO_ACCEPT and inviter in ALLOWED_USERS:
            await self.client.join_room(evt.room_id)
            log.info(f"Auto-accepted invite from {inviter}")
        else:
            # Track pending invite
            self.pending_invites.append({
                "room_id": str(evt.room_id),
                "inviter": inviter,
                "timestamp": datetime.now().isoformat()
            })
            log.info(f"Pending invite from {inviter} to {evt.room_id}")
            # Agent can decide via API

4. Modified on_message (Message Handler)

async def on_message(self, evt):
    # Ignore messages during initial sync
    if not self.initial_sync_done:
        return

    # Ignore old messages (more than 60 seconds old)
    event_time = datetime.fromtimestamp(evt.timestamp / 1000)
    message_age = datetime.now() - event_time
    if message_age > timedelta(seconds=60):
        return

    # Ignore own messages
    if evt.sender == self.user_id:
        return

    # NEW: Check if sender is allowed
    sender = str(evt.sender)
    if sender not in self.allowed_users:
        log.info(f"Ignoring message from disallowed user {sender}")
        return

    # Process message normally...

5. API Endpoints for Invite Management

Endpoint	Method	Purpose
GET /api/invites	List pending invitations
POST /api/invites/{room_id}/accept	Accept specific invitation
POST /api/invites/{room_id}/decline	Decline invitation
POST /api/invites/{room_id}/leave	Leave room (cleanup)

6. MCP Usage Pattern

# Agent checks for pending invites
invites = requests.post("http://localhost:8284/api/invites").json()

# Agent decides to accept an invite
if invites["pending"] and should_accept(invites[0]["inviter"]):
    requests.post(f"http://localhost:8284/api/invites/{invites[0]['room_id']}/accept")

---

Migrate to mautrix.util.formatter (IN PROGRESS - Copy Branch)

Status: Design Complete, Awaiting Implementation

Working File: ani_e2ee_bridge.py (refactor branch) Original: bridge-e2ee.py (preserved as safety)

Current State - UPDATED 2026-02-07

Current Implementation:

• Using python-markdown library with manual patches (~100 lines)
• Regex-based {color|text} and ||spoiler|| processing
• False code block detection is manual (keeps breaking)
• HTML pass-through is patched

Already Completed (✅):

• Extracted _handle_success_response() method to eliminate duplication between on_message() and process_queue()
• Reduced ~50 lines of duplicated SUCCESS handler code

Findings from mautrix-python docs:

• parse_html() is async - returns coroutine, requires await
• MarkdownString.format(EventType.ROOM_MESSAGE) is sync but requires EntityType argument
• MatrixParser.parse() is async

Impact: ~25 call sites need to be updated to await async format_html()

Why Migrate?

• Native Matrix formatting support
• Proper EntityType enum (COLOR, SPOILER, etc.)
• Built-in mention, pill, and room reference handling
• More robust HTML→Markdown round-tripping
• ~100 fewer lines of maintenance code

API Documentation References

from mautrix.util.formatter import parse_html, MarkdownString, MatrixParser
from mautrix.types import EventType

# HTML → Plain Text (ASYNC)
plain_text = await parse_html(html_input)

# Markdown → HTML (SYNC)
markdown = MarkdownString("**Bold** and ||spoiler||")
html_output = markdown.format(EventType.ROOM_MESSAGE)

# Mentions and pills (ASYNC)
parser = MatrixParser()
formatted = await parser.parse("Hello @user:example.com")

Implementation Plan

See BRIDGE_DESIGN.md for detailed refactor plan including:

1. Phase 1: Infrastructure (2h) - async format_html(), color syntax helper
2. Phase 2: Update Call Sites (1h) - ~25 await statements needed
3. Phase 3: Remove Dead Code (30m) - delete manual patches
4. Phase 4: Testing (1h) - formatting regression tests

Estimated total: ~4.5h

Detailed Implementation (FROM DOCS - UPDATED)

New async format_html()

async def format_html(text: str) -> tuple[str, str]:
    """
    Format text using mautrix native formatter.

    Args:
        text: Response from Letta (markdown or HTML)

    Returns:
        (plain_text, html_body) tuple
    """
    try:
        # Strip whitespace
        text = text.strip()

        # Convert emoji shortcodes (keep existing behavior)
        text = normalize_emoji_shortcodes(text)
        text = emoji.emojize(text, language='en')

        # HTML path → parse to plain (ASYNC)
        if text.startswith('<') and '>' in text:
            # Pre-process {color|text} - mautrix doesn't handle this
            text = _apply_color_syntax(text.strip())
            plain = await parse_html(text)
            return plain, text

        # Markdown path → use MarkdownString (SYNC)
        md = MarkdownString(text)
        # Pre-process {color|text} syntax
        processed_md = _apply_color_syntax(md.text)
        md.text = processed_md

        # Format to HTML (SYNC)
        html = md.format(EventType.ROOM_MESSAGE)

        # Generate plain text (ASYNC)
        plain = await parse_html(html)

        return plain, html

    except Exception as e:
        log.warning(f"HTML formatting failed: {e}")
        return emoji.emojize(text), emoji.emojize(text)

def _apply_color_syntax(text: str) -> str:
    """Convert {color|text} to HTML spans (adapter for existing syntax)."""
    def replace_color(match):
        color = match.group(1)
        content = match.group(2)
        hex_color = MATRIX_COLORS.get(color, color)
        return f'<font color="{hex_color}" data-mx-color="{hex_color}">{content}</font>'
    return re.sub(r'\{([a-zA-Z0-9_#]+)\|([^}]+)\}', replace_color, text)

Updated send_message()

async def send_message(self, room_id: RoomID, text: str) -> str | None:
    """
    Send a formatted message to a room (auto-encrypts if needed).
    """
    # Format text as HTML with full markdown and emoji support
    plain_text, html_body = await format_html(text)  # NOW AWAIT

    # Create content with both plain text and formatted HTML
    content = {
        "msgtype": "m.text",
        "body": plain_text,
        "format": "org.matrix.custom.html",
        "formatted_body": html_body,
    }

    event_id = await self.client.send_message_event(room_id, EventType.ROOM_MESSAGE, content)
    return str(event_id) if event_id else None

---

Other Ideas

Room Metadata in Context

Let the agent know room type (DM vs public), member count, room name

# In on_message(), add room metadata
room_info = await self.client.get_state_event(room_id, EventType.ROOM_NAME)
member_count = await self.get_member_count(room_id)
is_dm = member_count == 2

# Build richer context for Letta
context = f"[Room: {room_name} ({member_count} members, {'DM' if is_dm else 'group'})]"

Unified Context Option

Share conversations between specific rooms (e.g., DM + heartbeat room)

# Map multiple room IDs to same conversation ID
ROOM_CONTEXT_MAP = {
    "!dm-room": "shared-context-1",
    "!heartbeat-room": "shared-context-1",
    "!public-room": "shared-context-2"
}

Feedback Loop Improvements

• Store which messages got positive/negative reactions
• Let agent see reaction patterns over time
• Use feedback for preference learning

Multi-Agent Bridge Support

• Allow multiple agents in same room with different personae
• Route messages to specific agents based on @mentions or keywords
• Agent-to-agent conversation capability

---

Last Updated: 2026-02-05 Status: Ideas not yet implemented - safe to work on in future sessions