fix(tll, schemas): address PR #18 review v2–v5 follow-ups

Squashed across four review rounds on PR #18, all of which surfaced after
the initial wave of fixes (#1–#11). Each item below maps to a finding in
the PR's review threads.

Search path

- hydrateChainMemories now returns fully-shaped SearchResults via SELECT *
  + normalizeMemoryRow. The previous projection set similarity: null and
  omitted source_site / score / summary / observed_at, crashing the
  buildInjection formatter (`memory.similarity.toFixed(2)`) the first
  time TLL augmentation actually fired against a populated chain. The
  `as unknown as SearchResult` cast was hiding it from tsc. (review v2 #1)
- Hydration query now uses `unnest($2::uuid[]) WITH ORDINALITY ... ORDER
  BY req.ord` so the chronological order chainsFor returns through the
  augmentation pipeline survives. (review v4 #2)
- Workspace isolation: hydrateChainMemories filters `m.workspace_id IS NULL`
  to match the gate behavior performSearch's postProcessResults already
  applies. Without it, a workspace memory chained from a global memory's
  entity could surface in a global response. (review v4 #1)
- Defensive `relevance: 1.0` on hydrated rows locks in the chain-membership
  bypass invariant against future filter drift. The augmented rows are
  appended after applySearchRelevanceFilter today, but `similarity: 0`
  + `score: 0` would make them load-bearing on `relevance` if any future
  filter past appendTllAugmentation checked `memory.relevance >= threshold`.
  Regression test drives performSearch with a high
  retrievalOptions.relevanceThreshold and confirms the augmented row
  survives. (review v5 #2)

Repository

- TLL chain reads (chain, chainEventsForEntities) now derive chronological
  position and predecessor via `ROW_NUMBER()` and `LAG()` window functions
  ordered by observation_date ASC (with stored position_in_chain as a
  deterministic tiebreaker for events sharing an observation_date). The
  stored predecessor_memory_id and position_in_chain columns become
  insertion-order audit metadata; the API surface returns chronological
  ordering. Backfilled out-of-order events surface in their true position
  with chronologically-correct predecessors. (review v3 #1)
- chainEventsForEntities adds `m.workspace_id IS NULL` for the same
  reason as the search-path fix above — the global event-chains HTTP
  endpoint must not surface workspace memories. (review v4 #1)

Schema

- FirstMentionsExtractBodySchema validates memory_ids_by_turn_id values
  as UUIDs so a non-UUID returns 400 (schema layer) instead of leaking a
  Postgres "invalid input syntax for type uuid" as 500 from the route.
  (review v3 #2)
- New SearchResult.retrieval_signal optional field tags chain-augmented
  rows so observability and any future ranker can distinguish them from
  similarity-ranked candidates. (review v2 #1, plumbed through v4)

Refactor

- Extracted maybeExpandViaTLL, hydrateChainMemories, appendTllAugmentation
  out of memory-search.ts into a sibling tll-augmentation.ts module. The
  search file dropped from 551 → 385 LOC, back under the 400-LOC project
  cap. Shared internal types (PostProcessedSearch, RelevanceFilterSummary)
  pulled into memory-search-types.ts so the two consumers don't duplicate.
  (review v5 #4)

Test coverage

- New integration test (services/__tests__/tll-augmentation-integration.test.ts)
  drives performSearch end-to-end through appendTllAugmentation, with
  cases for: rendering augmented rows through buildInjection without
  crashing, the SQL contract (unnest ORDINALITY + ORDER BY req.ord),
  workspace-leak prevention, the relevance-1.0 bypass invariant, and
  no-augmentation for non-TLL queries.
- New repository tests for backfill chronological ordering of chain()
  and chainEventsForEntities() and for chainEventsForEntities workspace
  isolation.
- New route test asserts 400 (not 500) on non-UUID memory_ids_by_turn_id.

Verification: `npx tsc --noEmit` clean · 1286/1286 vitest pass against the
test DB · `npx fallow audit --no-cache` exit 0.

Deferrals (parked durably in the research repo's tech-debt log at
Atomicmemory-research/docs/core-repo/tech-debt.md):
- predecessor_memory_id ON DELETE CASCADE vs SET NULL — design call,
  contested between two reviewers; current default kept.
- process.env.ALLOWED_ORIGINS direct read in src/routes/memories.ts —
  pre-existing CLAUDE.md violation, out of scope for this PR.
- shouldUseTLL non-adjacent "after did" — low operational risk;
  tightening the regex was the explicit goal of review #9 and
  re-broadening risks reintroducing false positives.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: ethanj

src/db/__tests__/repository-tll.test.ts

@@ -66,6 +66,25 @@ describe('TllRepository', () => {
     return { id, date };
   }
 
+  /**
+   * Store a workspace-scoped memory. Used to verify that the global
+   * `chainEventsForEntities` endpoint never surfaces workspace memories.
+   * Any non-null UUID for `workspaceId` is sufficient for the
+   * `workspace_id IS NULL` filter to drop the row.
+   */
+  async function makeWorkspaceMemory(content: string, seed: number, date: Date): Promise<SeededMemory> {
+    const id = await memoryRepo.storeMemory({
+      userId: TEST_USER,
+      content,
+      embedding: unitVector(seed),
+      importance: 0.6,
+      sourceSite: 'test',
+      workspaceId: '00000000-0000-0000-0000-00000000ffff',
+      agentId: '00000000-0000-0000-0000-0000000000aa',
+    });
+    return { id, date };
+  }
+
   /** Count rows in TLL for a (user, entity, memory) triple. */
   async function countRows(entityId: string, memoryId: string): Promise<number> {
     const r = await pool.query<{ c: string }>(
@@ -169,6 +188,22 @@ describe('TllRepository', () => {
       const events = await tllRepo.chain(TEST_USER, ent.id);
       expect(events).toEqual([]);
     });
+
+    it('orders by observation_date — backfilled out-of-order events get chronological positions and predecessors', async () => {
+      const ent = await makeEntity('BackfillChain', 56);
+      const middle = await makeMemory('mid', 57, new Date('2026-02-01'));
+      const latest = await makeMemory('late', 58, new Date('2026-03-01'));
+      const earliest = await makeMemory('early', 59, new Date('2026-01-01'));
+
+      await tllRepo.append(TEST_USER, middle.id, [ent.id], middle.date);
+      await tllRepo.append(TEST_USER, latest.id, [ent.id], latest.date);
+      await tllRepo.append(TEST_USER, earliest.id, [ent.id], earliest.date);
+
+      const events = await tllRepo.chain(TEST_USER, ent.id);
+      expect(events.map((e) => e.memoryId)).toEqual([earliest.id, middle.id, latest.id]);
+      expect(events.map((e) => e.positionInChain)).toEqual([0, 1, 2]);
+      expect(events.map((e) => e.predecessorMemoryId)).toEqual([null, earliest.id, middle.id]);
+    });
   });
 
   describe('chainsFor()', () => {
@@ -261,5 +296,39 @@ describe('TllRepository', () => {
       expect(result).toHaveLength(1);
       expect(result[0].events.map((e) => e.memoryId)).toEqual([live.id]);
     });
+
+    it('orders backfilled events by observation_date with chronological predecessors', async () => {
+      const ent = await makeEntity('Backfill', 111);
+      // Insertion order: middle, latest, earliest. Observation order: jan, feb, mar.
+      const middle = await makeMemory('middle', 112, new Date('2026-02-01'));
+      const latest = await makeMemory('latest', 113, new Date('2026-03-01'));
+      const earliest = await makeMemory('earliest', 114, new Date('2026-01-01'));
+
+      await tllRepo.append(TEST_USER, middle.id, [ent.id], middle.date);
+      await tllRepo.append(TEST_USER, latest.id, [ent.id], latest.date);
+      await tllRepo.append(TEST_USER, earliest.id, [ent.id], earliest.date);
+
+      const result = await tllRepo.chainEventsForEntities(TEST_USER, [ent.id]);
+      expect(result).toHaveLength(1);
+      const events = result[0].events;
+      expect(events.map((e) => e.memoryId)).toEqual([earliest.id, middle.id, latest.id]);
+      expect(events.map((e) => e.positionInChain)).toEqual([0, 1, 2]);
+      expect(events[0].predecessorMemoryId).toBeNull();
+      expect(events[1].predecessorMemoryId).toBe(earliest.id);
+      expect(events[2].predecessorMemoryId).toBe(middle.id);
+    });
+
+    it('omits events whose memory has a non-null workspace_id (global endpoint isolation)', async () => {
+      const ent = await makeEntity('GlobalEntity', 121);
+      const global = await makeMemory('global memory', 122, new Date('2026-01-01'));
+      const workspaceMem = await makeWorkspaceMemory('workspace memory', 123, new Date('2026-02-01'));
+
+      await tllRepo.append(TEST_USER, global.id, [ent.id], global.date);
+      await tllRepo.append(TEST_USER, workspaceMem.id, [ent.id], workspaceMem.date);
+
+      const result = await tllRepo.chainEventsForEntities(TEST_USER, [ent.id]);
+      expect(result).toHaveLength(1);
+      expect(result[0].events.map((e) => e.memoryId)).toEqual([global.id]);
+    });
   });
 });

src/db/repository-tll.ts

@@ -102,22 +102,40 @@ export class TllRepository {
   }
 
   /**
-   * Get the full event chain for an entity, ordered chronologically.
-   * Used for "list events in order" (EO) and "how did X evolve" (TR/MSR).
+   * Get the full event chain for an entity, ordered by observation_date
+   * (the conversation timestamp). Used for EO/TR/MSR queries that need
+   * chronological order — a backfilled event with an earlier
+   * observation_date surfaces in its true chronological position even
+   * though it was inserted last.
+   *
+   * `positionInChain` returned here is the 0-based chronological rank,
+   * derived via `ROW_NUMBER()` over the chronological order. The stored
+   * `position_in_chain` column is insertion-order audit metadata and is
+   * not exposed by this API. `predecessorMemoryId` is the immediate
+   * chronologically-prior memory (via `LAG()`) — also chronological,
+   * not the position-tip predecessor recorded at insert time.
+   *
+   * Tiebreaker for events sharing an `observation_date` (e.g. the same
+   * conversation timestamp) is the stored `position_in_chain` (insertion
+   * order), keeping the result deterministic.
    */
   async chain(userId: string, entityId: string): Promise<TLLEvent[]> {
     const result = await this.pool.query(
-      `SELECT memory_id, predecessor_memory_id, observation_date, position_in_chain
+      `SELECT memory_id,
+              observation_date,
+              LAG(memory_id) OVER w AS chronological_predecessor,
+              ROW_NUMBER() OVER w - 1 AS chronological_position
        FROM temporal_linkage_list
        WHERE user_id = $1 AND entity_id = $2
-       ORDER BY position_in_chain ASC`,
+       WINDOW w AS (ORDER BY observation_date ASC, position_in_chain ASC)
+       ORDER BY observation_date ASC, position_in_chain ASC`,
       [userId, entityId],
     );
     return result.rows.map((row) => ({
       memoryId: row.memory_id,
-      predecessorMemoryId: row.predecessor_memory_id,
+      predecessorMemoryId: row.chronological_predecessor ?? null,
       observationDate: row.observation_date,
-      positionInChain: row.position_in_chain,
+      positionInChain: Number(row.chronological_position),
     }));
   }
 
@@ -144,7 +162,11 @@ export class TllRepository {
    * by EO-shaped read paths that need content alongside chain position.
    *
    * Returns one entry per entity; entities with no events are dropped.
-   * Within an entity, events are ordered by position_in_chain ASC.
+   * Within an entity, events are ordered by `observation_date` (the
+   * conversation timestamp), not insertion order — see `chain()` for
+   * the rationale and the `LAG`/`ROW_NUMBER` derivation of
+   * `predecessorMemoryId` and `positionInChain`. Tiebreaker for events
+   * sharing an observation_date is the stored insertion `position_in_chain`.
    */
   async chainEventsForEntities(
     userId: string,
@@ -161,20 +183,28 @@ export class TllRepository {
   }>> {
     if (entityIds.length === 0) return [];
     const unique = [...new Set(entityIds)];
+    // `m.workspace_id IS NULL` — this is the global event-chain endpoint;
+    // workspace-scoped memories must not surface here even if they share
+    // an entity with a global memory.
     const result = await this.pool.query(
       `SELECT t.entity_id,
               t.memory_id,
-              t.predecessor_memory_id,
               t.observation_date,
-              t.position_in_chain,
+              LAG(t.memory_id) OVER w AS chronological_predecessor,
+              ROW_NUMBER() OVER w - 1 AS chronological_position,
               m.content
        FROM temporal_linkage_list t
        JOIN memories m ON m.id = t.memory_id
        WHERE t.user_id = $1
          AND t.entity_id = ANY($2::uuid[])
          AND m.deleted_at IS NULL
          AND m.status = 'active'
-       ORDER BY t.entity_id, t.position_in_chain ASC`,
+         AND m.workspace_id IS NULL
+       WINDOW w AS (
+         PARTITION BY t.entity_id
+         ORDER BY t.observation_date ASC, t.position_in_chain ASC
+       )
+       ORDER BY t.entity_id, t.observation_date ASC, t.position_in_chain ASC`,
       [userId, unique],
     );
     const grouped = new Map<string, Array<{
@@ -190,8 +220,8 @@ export class TllRepository {
         memoryId: row.memory_id,
         content: row.content,
         observationDate: row.observation_date,
-        positionInChain: row.position_in_chain,
-        predecessorMemoryId: row.predecessor_memory_id,
+        positionInChain: Number(row.chronological_position),
+        predecessorMemoryId: row.chronological_predecessor ?? null,
       });
       grouped.set(row.entity_id, list);
     }

src/db/repository-types.ts

@@ -169,6 +169,12 @@ export interface SearchResult extends MemoryRow {
   matched_facts?: string[];
   matched_fact_ids?: string[];
   retrieval_layer?: 'memory' | 'atomic_fact';
+  /**
+   * Origin of the row in the candidate pool. Absent for similarity-ranked
+   * rows; set to `'tll-chain'` for rows hydrated by TLL chain expansion
+   * after the relevance gate (see `hydrateChainMemories`).
+   */
+  retrieval_signal?: 'tll-chain';
 }
 
 export type AtomicFactType = 'preference' | 'project' | 'knowledge' | 'person' | 'plan';

src/routes/__tests__/event-chains-and-first-mentions.test.ts

@@ -272,6 +272,19 @@ describe('POST /memories/first-mentions/extract — schema validation', () => {
     });
     expect(error).toMatch(/source_site/i);
   });
+
+  it('returns 400 (not 500) when a memory_ids_by_turn_id value is not a UUID', async () => {
+    // Without schema-layer UUID validation, a bad value reaches Postgres
+    // and crashes with "invalid input syntax for type uuid", which the
+    // route handler maps to 500. The schema must reject it as 400 instead.
+    const { error } = await postExpecting400({
+      user_id: TEST_USER,
+      conversation_text: 'hi',
+      source_site: 'beam',
+      memory_ids_by_turn_id: { '1': INVALID_UUID },
+    });
+    expect(error).toMatch(/memory_ids_by_turn_id/i);
+  });
 });
 
 // ---------------------------------------------------------------------------

src/schemas/memories.ts

@@ -586,7 +586,11 @@ export const FirstMentionsExtractBodySchema = z
     user_id: z.string().min(1),
     conversation_text: z.string().min(1).max(MAX_CONVERSATION_LENGTH),
     source_site: z.string().min(1),
-    memory_ids_by_turn_id: z.record(z.string(), z.string()),
+    // Values must be UUIDs — they're inserted into a UUID column
+    // (`first_mention_events.memory_id`). Validate at the schema layer so a
+    // bad value returns 400 instead of leaking a Postgres "invalid input
+    // syntax for type uuid" error as a 500.
+    memory_ids_by_turn_id: z.record(z.string(), z.string().uuid()),
   })
   .transform(b => {
     const map = new Map<number, string>();