feat(backend): MemoryEnvelope metadata model, scoped retrieval, and memory hardening

[ntindle]

Why / What / How

Why: CoPilot's Graphiti memory system needed structured metadata to distinguish memory types (rules, procedures, facts, preferences), support scoped retrieval, enable targeted deletion, and track memory costs under the AutoPilot billing account separately from the platform.

What: Adds the MemoryEnvelope metadata model, structured rule/procedure memory types, a derived-finding lane for assistant-distilled knowledge, two-step forget tools, scope-aware retrieval filtering, AutoPilot-dedicated API key routing, and several reliability fixes (streaming socket leaks, event-loop-scoped caches, ingestion hardening).

How: MemoryEnvelope wraps every stored episode with typed metadata (source_kind, memory_kind, scope, status, confidence) serialized as JSON. Retrieval filters by scope at the context layer. The forget flow uses a search-then-confirm two-step pattern. Ingestion queues and client caches are scoped per event loop via WeakKeyDictionary to prevent cross-loop RuntimeErrors in multi-worker deployments. API key resolution falls back to AutoPilot-dedicated keys (CHAT_API_KEY, CHAT_OPENAI_API_KEY) before platform-wide keys.

Changes 🏗️

New: MemoryEnvelope metadata model (memory_model.py)

• Typed memory categories: fact, preference, rule, finding, plan, event, procedure
• Source tracking: user_asserted, assistant_derived, tool_observed
• Scope namespacing: real:global, project:<name>, book:<title>, session:<id>
• Status lifecycle: active, tentative, superseded, contradicted
• Structured RuleMemory and ProcedureMemory models for complex instructions

New: Targeted forget tools (graphiti_forget.py)

• memory_forget_search: returns candidate facts with UUIDs for user confirmation
• memory_forget_confirm: deletes specific edges by UUID after confirmation

New: Architecture test (architecture_test.py)

• Validates no new @cached(...) usage around event-loop-bound async clients
• Allowlists pre-existing violations for future cleanup

Enhanced: memory_store tool (graphiti_store.py)

• Accepts MemoryEnvelope metadata fields (source_kind, scope, memory_kind, rule, procedure)
• Wraps content in MemoryEnvelope before ingestion

Enhanced: memory_search tool (graphiti_search.py)

• Scope-aware retrieval with hard filtering on group_id

Enhanced: Ingestion pipeline (ingest.py)

• Derived-finding lane: distills substantive assistant responses into tentative findings
• Event-loop-scoped queues and workers via WeakKeyDictionary (fixes multi-worker RuntimeError)
• Improved error handling and dropped-episode reporting

Enhanced: Client cache (client.py)

• Per-loop client cache and lock via WeakKeyDictionary (fixes "Future attached to a different loop")

Enhanced: Warm context (context.py)

• Filters out non-global-scope episodes from warm context

Fix: Streaming socket leak (baseline/service.py)

• try/finally around async stream iteration to release httpx connections on early exit

Config: AutoPilot key routing (config.py, .env.default)

• LLM key fallback: GRAPHITI_LLM_API_KEY → CHAT_API_KEY → OPEN_ROUTER_API_KEY
• Embedder key fallback: GRAPHITI_EMBEDDER_API_KEY → CHAT_OPENAI_API_KEY → OPENAI_API_KEY
• Backwards-compatible: existing behavior unchanged until new keys are provisioned

Checklist 📋

For code changes:

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • poetry run pytest backend/copilot/graphiti/config_test.py — 16 tests pass (key fallback priority)
  • poetry run pytest backend/copilot/tools/graphiti_store_test.py — store envelope tests pass
  • poetry run pytest backend/copilot/graphiti/ingest_test.py — ingestion tests pass
  • poetry run pytest backend/util/architecture_test.py — structural validation passes
  • Verify memory store/retrieve/forget cycle via copilot chat
  • Run AgentProbe multi-session memory benchmark (31 scenarios x3 repeats)
  • Confirm no CLOSE_WAIT socket accumulation under sustained streaming load
  • Verify multi-worker deployment doesn't produce loop-binding errors

For configuration changes:

• .env.default is updated or already compatible with my changes
• docker-compose.yml is updated or already compatible with my changes
• Configuration changes:
  • New optional env var CHAT_OPENAI_API_KEY — AutoPilot-dedicated OpenAI key for Graphiti embeddings (falls back to OPENAI_API_KEY if not set)
  • CHAT_API_KEY now used as first fallback for Graphiti LLM calls (was OPEN_ROUTER_API_KEY)
  • Infra action needed: add CHAT_OPENAI_API_KEY sealed secret in autogpt-shared-config values (dev + prod)

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches Graphiti memory ingestion/retrieval and introduces hard-delete capabilities plus event-loop–scoped caching/queues; failures could affect memory correctness or delete the wrong edges. Also changes streaming resource cleanup and key routing, which could surface as connection or billing/cost attribution issues if misconfigured.

Overview
Graphiti memory is upgraded from plain text episodes to a structured JSON MemoryEnvelope. memory_store now wraps content with typed metadata (source, kind, scope, status) and optional structured rule/procedure payloads, and ingestion supports JSON episodes.

Memory retrieval and lifecycle controls are expanded. memory_search adds optional scope hard-filtering to prevent cross-scope leakage, warm-context formatting drops non-global scoped episodes (and avoids empty wrappers), and new two-step tools (memory_forget_search → memory_forget_confirm) enable targeted soft- or hard-deletion of specific graph edges by UUID.

Reliability and multi-worker safety improvements. Graphiti client caching and ingestion worker registries are now per-event-loop (avoiding cross-loop Future errors), streaming chat completions explicitly close async streams to prevent CLOSE_WAIT socket leaks, warm-context is injected into the first user message to keep the system prompt cacheable, and a new architecture_test.py blocks future process-wide caching of event-loop–bound async clients. Config updates route Graphiti LLM/embedder keys to AutoPilot-specific env vars first, and OpenAPI schema exports include the new memory response types.

^{Reviewed by Cursor Bugbot for commit 5fb4bd0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[ntindle]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved in merge commit 8455a13. There was one conflict in service.py — the warm-context injection (our branch) overlapped with the transcript seeding addition from dev. Both changes are independent and have been kept in the correct order.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e7ae7b5. Configure here.}