jaclang 0.13.4 serializer hash change causes mass edge loss on existing deployments

[udithishanka]

Bug

jaclang 0.13.4 introduced topology_index_data in the serializer (jac/jaclang/runtimelib/impl/serializer.impl.jac). The _compute_hash function was changed to exclude this field from the hash. This causes all existing anchors in MongoDB to appear dirty (hash mismatch) on every request, triggering full graph rewrites on every walker invocation.

Impact

On jac-builder (production IDE with 76 users, 140 projects):

• All 79 UserProfile → Project edges were erased within minutes of deploying with jaclang 0.13.4
• Every walker rewrites every anchor it touches (because hashes don't match)
• With 36 concurrent requests on IDE page load, 36 simultaneous full-graph rewrites occur
• Last-writer-wins → edges silently erased
• New projects also lose their edges immediately after creation

Root Cause

1. _compute_hash in serializer.impl.jac now excludes topology_index_data from the hash
2. Existing anchors in MongoDB were hashed WITH the old algorithm (no topology_index_data field existed)
3. New code computes a different hash for the same data → every anchor looks "modified"
4. The persistence layer commits every "dirty" anchor back to MongoDB on every request
5. Concurrent walkers each read the graph, compute work, and write back their (stale) snapshot
6. Last writer wins → edges from other concurrent walkers are erased

Before vs After

• jaclang 0.13.3: Only actually-modified anchors get rewritten. Race condition exists but is rare (most walkers are read-only).
• jaclang 0.13.4: Every anchor gets rewritten on every request due to hash mismatch. Race condition is guaranteed on any page with concurrent requests.

Reproduction

1. Deploy a jac-scale app with MongoDB persistence on jaclang 0.13.3
2. Create users and projects (graph edges: Root → UserProfile → Project)
3. Upgrade to jaclang 0.13.4
4. Open any page that fires 3+ concurrent walker requests
5. Observe: edges disappear from MongoDB

Workaround

Reverted to jaclang 0.13.3 (pre-topology_index_data changes). Edges stop disappearing.

Suggested Fix

Either:

1. Add a migration step that recomputes hashes for all existing anchors on first startup with 0.13.4
2. Don't change the hash computation in a minor release — treat it as a breaking change
3. Add per-anchor optimistic locking (version field) so concurrent writes don't silently overwrite each other (fixes the underlying race condition from Graph persistence race condition: concurrent walker commits cause node/edge loss #5446)

Related

• Graph persistence race condition: concurrent walker commits cause node/edge loss #5446 — graph persistence race condition (last-writer-wins)
• Commits: 2e4ae438a (release 0.13.4), serializer changes in jac/jaclang/runtimelib/impl/serializer.impl.jac

[udithishanka]

Root cause PR identified: #5396 (feat: persist topology index across Serializer round-trips, commit 9e7260c09).

This PR changed _compute_hash() in serializer.impl.jac to exclude topology_index_data from the hash. On existing deployments with MongoDB data created before this change, every anchor's hash now differs from what's stored — making every anchor appear "dirty" and triggering a full rewrite on every walker request. With 36 concurrent requests on IDE page load, this causes mass edge loss via last-writer-wins.

[udithishanka]

Detailed Root Cause Analysis of PR #5396

The bug is in how _compute_hash interacts with the serializer changes. Here's the exact mechanism:

What PR #5396 changed in serializer.impl.jac:

1. _serialize_value now adds topology_index_data to root NodeAnchors:

if val.topology_index_data is not None and isinstance(val.archetype, Root) {
    result['topology_index_data'] = b64encode(val.topology_index_data).decode('ascii');
}

2. _deserialize_anchor now sets topology_index_data = None on ALL NodeAnchors:

anchor.topology_index_data = None;
if isinstance(anchor.__dict__.get('archetype'), Root) {
    if (topo_b64 := data.get('topology_index_data')) {
        anchor.topology_index_data = b64decode(topo_b64);
    }
}

3. _compute_hash pops topology_index_data before hashing:

data = Serializer.serialize(anchor, include_type=True);
if isinstance(data, dict) {
    data.pop('topology_index_data', None);
}
return hash(dumps(data, sort_keys=True));

Why this causes hash mismatch on existing data:

Old anchors in MongoDB (created with jaclang ≤ 0.13.3):

• Were hashed with _compute_hash that called Serializer.serialize() — which did NOT produce topology_index_data in the output
• Stored hash = hash(json_without_topology_field)

New runtime (jaclang 0.13.4):

• Loads anchor from MongoDB → _deserialize_anchor sets anchor.topology_index_data = None on every NodeAnchor
• At commit time, _compute_hash calls Serializer.serialize() → the serialize path now checks val.topology_index_data is not None — it's None, so the field is NOT included in output
• BUT: the deserialization step may have subtly changed the anchor's in-memory state (e.g., the topology_index_data attribute now EXISTS on the object even as None, which wasn't there before)
• The serialized output may differ from the original due to other fields being affected by the round-trip

Even if the serialized JSON is byte-identical, the hash stored in MongoDB was computed by the OLD _compute_hash (which didn't have the data.pop('topology_index_data', None) line). If the pop operation changes the dict structure in any way (e.g., dict ordering after pop), the hash changes.

The amplification effect:

Once the hash mismatches:

1. Every anchor looks "dirty" to the persistence layer
2. Every walker invocation commits ALL touched anchors back to MongoDB
3. With 36+ concurrent requests on page load, 36 simultaneous full-graph rewrites race
4. Last-writer-wins → edges silently erased

Suggested fix:

The hash computation needs to be backward-compatible with existing stored hashes. Options:

1. Don't modify _compute_hash at all — let topology_index_data be included in the hash. It's None for existing anchors, so the serialize output won't include it, and the hash stays the same.

2. Add a hash migration — on first load after upgrade, recompute and store new hashes for all anchors so the dirty-check baseline is reset.

3. Use a version field in the serialized data so old vs new format is distinguishable, and hashing uses the appropriate algorithm for each version.

The simplest fix is option 1 — remove the data.pop('topology_index_data', None) from _compute_hash. The topology index data is only present on root anchors with the feature enabled, so including it in the hash is fine (it won't affect non-root anchors at all, and root anchors with the feature enabled SHOULD be re-persisted when the index changes).