Skip to content

Commit 7ae80fe

Browse files
bendvcbendvc
authored
feat(mrt-utilities): add development data-store with dist export (#384)
* Initial Commit * Don't overload development condition * Update docs * Refactor error types, make backwards compatible
1 parent ca921ae commit 7ae80fe

10 files changed

Lines changed: 577 additions & 138 deletions

File tree

‎.changeset/mrt-utilities-dev-data-store.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@salesforce/mrt-utilities': patch
3+
---
4+
5+
Add a development-mode pseudo data-store implementation for `@salesforce/mrt-utilities/data-store` with environment-variable-backed defaults, while preserving the existing public API and production behavior.

‎docs/guide/mrt-utilities.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,37 @@ import { MetricsSender } from '@salesforce/mrt-utilities/metrics';
149149

150150
Use when you need to emit metrics from the same process that serves requests (e.g. custom middleware or request processor).
151151

152+
## Data Store In Development
153+
154+
The production MRT data store is not available during local development because it depends on deployed runtime infrastructure. `@salesforce/mrt-utilities` provides an equivalent development data-store implementation for local use.
155+
156+
To use that local equivalent, import from the `data-store` subpath and run Node with the `dev-data-store` condition:
157+
158+
```bash
159+
node --conditions dev-data-store server.js
160+
```
161+
162+
```typescript
163+
import {DataStore} from '@salesforce/mrt-utilities/data-store';
164+
165+
const store = DataStore.getDataStore();
166+
const entry = await store.getEntry('custom-global-preferences');
167+
```
168+
169+
Provide local data-store values through environment variables:
170+
171+
- `SFNEXT_DATA_STORE_DEFAULTS`: JSON map of data-store keys to object values
172+
- `SFNEXT_DATA_STORE_WARN_ON_MISSING`: set to `false` to suppress missing-key warnings
173+
174+
Example:
175+
176+
```bash
177+
export SFNEXT_DATA_STORE_DEFAULTS='{"custom-global-preferences":{"featureFlag":true}}'
178+
export SFNEXT_DATA_STORE_WARN_ON_MISSING=true
179+
```
180+
181+
The development pseudo store keeps production parity for missing keys and throws `DataStoreNotFoundError` when a key is not found.
182+
152183
## Related
153184

154185
- [MRT CLI commands](/cli/mrt) — manage MRT projects, environments, and bundles from the CLI.

‎packages/mrt-utilities/DEV-DATA-STORE-PLAN.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Development Data Store Plan (`@salesforce/mrt-utilities`)
2+
3+
## Goal
4+
5+
Add a pseudo local data-store implementation for development mode so local runtimes do not fail when DynamoDB-backed MRT data store is unavailable.
6+
7+
This behavior should be activated through the existing package export condition:
8+
9+
- `@salesforce/mrt-utilities/data-store` + `--conditions development`
10+
11+
## Key Requirement
12+
13+
The pseudo local implementation must read default entry values from environment variables (as implied by the provided prototype):
14+
15+
- `SFNEXT_DATA_STORE_DEFAULTS` (JSON object map of key -> value object)
16+
- `SFNEXT_DATA_STORE_WARN_ON_MISSING` (`"false"` disables warnings; default is warning enabled)
17+
18+
## Current State
19+
20+
- `./data-store` already has a `development` export condition in `package.json`, but it currently points to `src`.
21+
- Desired update: point `development` to a built `dist` pseudo-local data-store output.
22+
- Current `DataStore` implementation is DynamoDB-based and throws `DataStoreUnavailableError` when required MRT environment variables are missing.
23+
- Existing tests primarily validate production/DynamoDB behavior.
24+
25+
## Proposed Design
26+
27+
## 1) Split data-store implementations
28+
29+
Create two implementation modules:
30+
31+
- **Production implementation** (existing behavior)
32+
- DynamoDB-backed
33+
- Preserves existing errors:
34+
- `DataStoreUnavailableError`
35+
- `DataStoreNotFoundError`
36+
- `DataStoreServiceError`
37+
- **Development implementation** (new behavior)
38+
- No AWS dependency
39+
- Uses local defaults from env var JSON
40+
- Warns once per missing key (configurable)
41+
- Uses strict parity with production semantics by default for missing keys (throws not-found)
42+
- Optional lenient mode can be introduced as explicit opt-in for `{}` fallback during local experimentation
43+
44+
## 2) Preserve stable public API
45+
46+
Keep consumer import surface unchanged:
47+
48+
- `DataStore.getDataStore()`
49+
- `DataStore#isDataStoreAvailable()`
50+
- `DataStore#getEntry(key)`
51+
- Existing error classes remain exported
52+
53+
This allows existing projects to adopt dev behavior without refactoring imports.
54+
55+
## 3) Route development exports
56+
57+
Use conditional exports to load the development implementation for local dev from built artifacts:
58+
59+
- `development` -> built dev pseudo-local data-store module in `dist`
60+
- `import` / `require` -> production built outputs in `dist`
61+
62+
## 4) Environment variable behavior in dev store
63+
64+
### `SFNEXT_DATA_STORE_DEFAULTS`
65+
66+
- Parse as JSON object.
67+
- Expected shape:
68+
- `{ "<entry-key>": { ...objectValue } }`
69+
- On invalid JSON:
70+
- fall back to empty defaults
71+
- warn once with clear message
72+
73+
### `SFNEXT_DATA_STORE_WARN_ON_MISSING`
74+
75+
- If unset: warnings enabled
76+
- If set to `"false"` (case-insensitive): disable missing-key warnings
77+
- Any other value: warnings enabled
78+
79+
### Missing key semantics (dev mode)
80+
81+
- If key exists in parsed defaults and value is object: return that value.
82+
- If key missing or invalid value type:
83+
- by default, throw `DataStoreNotFoundError` (production parity)
84+
- optionally warn once for that key before throwing
85+
- optional future opt-in lenient mode may return `{}` instead (must be off by default)
86+
87+
## 5) Tests
88+
89+
Add/adjust tests to cover both modes:
90+
91+
- **Production tests**
92+
- Keep current behavior assertions unchanged.
93+
- **Development tests**
94+
- Reads defaults from `SFNEXT_DATA_STORE_DEFAULTS`
95+
- Throws `DataStoreNotFoundError` when key is absent (default behavior)
96+
- Warns once per missing key when warnings enabled
97+
- Does not warn when `SFNEXT_DATA_STORE_WARN_ON_MISSING=false`
98+
- Handles invalid JSON safely
99+
- (If lenient mode is added) returns `{}` only when explicitly enabled
100+
101+
## 6) Documentation updates
102+
103+
Update `packages/mrt-utilities/README.md` (or docs page if preferred) with:
104+
105+
- How to enable dev behavior (`node --conditions development`)
106+
- Env var configuration examples for default data-store values
107+
- Differences between dev and production data-store semantics
108+
109+
## Implementation Steps
110+
111+
1. Add a new dev data-store module in `src/data-store/`.
112+
2. Move/keep current DynamoDB implementation as production module.
113+
3. Ensure build output emits both implementations to `dist` (esm/cjs + types).
114+
4. Update `package.json` exports so `development` resolves to the built dev pseudo-local module in `dist` (not `src`), while `import`/`require` continue resolving to production built outputs.
115+
5. Add development-focused tests.
116+
6. Run validation:
117+
- `pnpm --filter @salesforce/mrt-utilities run test:agent`
118+
- `pnpm --filter @salesforce/mrt-utilities run lint:agent`
119+
- `pnpm --filter @salesforce/mrt-utilities run typecheck:agent`
120+
7. Add a changeset for `@salesforce/mrt-utilities` if this is considered user-facing behavior.
121+
122+
## Risks / Notes
123+
124+
- Strict production parity in dev is the default to avoid masking missing-key issues.
125+
- Any lenient `{}` fallback behavior must be explicit opt-in and clearly documented.
126+
- Existing export stripping behavior is already understood and is not changed by this plan.
127+
128+
## Acceptance Criteria
129+
130+
- Local development using `--conditions development` no longer fails due to missing DynamoDB/MRT runtime vars.
131+
- Dev data-store entries are sourced from `SFNEXT_DATA_STORE_DEFAULTS`.
132+
- Missing-key behavior is predictable and configurable via `SFNEXT_DATA_STORE_WARN_ON_MISSING`.
133+
- Production behavior and API remain backward-compatible.
134+
- No breaking public interface changes: existing import paths, exported symbols, and type surface for `@salesforce/mrt-utilities` and `@salesforce/mrt-utilities/data-store` remain intact (except correcting the `development` export target to built `dist` output).
135+
- Default dev missing-key semantics match production (`DataStoreNotFoundError`), with no implicit `{}` fallback.

‎packages/mrt-utilities/README.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,3 +47,32 @@ export const createApp = (): Express => {
4747
// Cleans up any remaining headers and sets any remaining values
4848
app.use(createMRTCleanUpMiddleware());
4949
```
50+
51+
## Development data-store usage
52+
53+
Use the `data-store` subpath with Node's `dev-data-store` condition to load the pseudo local data-store implementation:
54+
55+
```bash
56+
node --conditions dev-data-store your-app.js
57+
```
58+
59+
```ts
60+
import {DataStore} from '@salesforce/mrt-utilities/data-store';
61+
62+
const store = DataStore.getDataStore();
63+
const entry = await store.getEntry('custom-global-preferences');
64+
```
65+
66+
Configure local values with environment variables:
67+
68+
- `SFNEXT_DATA_STORE_DEFAULTS`: JSON map of key to object value
69+
- `SFNEXT_DATA_STORE_WARN_ON_MISSING`: set to `false` to suppress missing-key warnings
70+
71+
Example:
72+
73+
```bash
74+
export SFNEXT_DATA_STORE_DEFAULTS='{"custom-global-preferences":{"featureFlag":true}}'
75+
export SFNEXT_DATA_STORE_WARN_ON_MISSING=true
76+
```
77+
78+
By default, missing keys still throw `DataStoreNotFoundError` in development (matching production semantics).

‎packages/mrt-utilities/package.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,16 @@
5252
}
5353
},
5454
"./data-store": {
55+
"dev-data-store": {
56+
"import": {
57+
"types": "./dist/esm/data-store/development.d.ts",
58+
"default": "./dist/esm/data-store/development.js"
59+
},
60+
"require": {
61+
"types": "./dist/cjs/data-store/development.d.ts",
62+
"default": "./dist/cjs/data-store/development.js"
63+
}
64+
},
5565
"development": "./src/data-store/index.ts",
5666
"import": {
5767
"types": "./dist/esm/data-store/index.d.ts",

‎packages/mrt-utilities/src/data-store/development.ts‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
/*
2+
* Copyright (c) 2025, Salesforce, Inc.
3+
* SPDX-License-Identifier: Apache-2
4+
* For full license text, see the license.txt file in the repo root or http://www.apache.org/licenses/LICENSE-2.0
5+
*/
6+
7+
import {DataStoreNotFoundError} from './errors.js';
8+
9+
export {DataStoreNotFoundError, DataStoreServiceError, DataStoreUnavailableError} from './errors.js';
10+
11+
/**
12+
* Development-only pseudo data store backed by environment variables.
13+
*
14+
* This class mirrors the public DataStore API while avoiding DynamoDB access.
15+
*/
16+
export class DataStore {
17+
private defaults: Record<string, Record<string, unknown>>;
18+
19+
private warnOnMissing: boolean;
20+
21+
private warnedKeys: Set<string>;
22+
23+
private static _instance: DataStore | null = null;
24+
25+
private constructor() {
26+
this.defaults = readDefaultsFromEnv();
27+
this.warnOnMissing = readWarnOnMissingFromEnv();
28+
this.warnedKeys = new Set<string>();
29+
}
30+
31+
/**
32+
* Get or create the singleton DataStore instance.
33+
*
34+
* @returns The singleton DataStore instance
35+
*/
36+
static getDataStore(): DataStore {
37+
if (!DataStore._instance) {
38+
DataStore._instance = new DataStore();
39+
}
40+
41+
return DataStore._instance;
42+
}
43+
44+
/**
45+
* Whether the data store can be used in the current environment.
46+
*
47+
* The development pseudo store is always available when loaded.
48+
*
49+
* @returns true
50+
*/
51+
isDataStoreAvailable(): boolean {
52+
return true;
53+
}
54+
55+
/**
56+
* Fetch an entry from the pseudo data store.
57+
*
58+
* @param key The data store entry's key
59+
* @returns An object containing the entry's key and value
60+
* @throws {DataStoreNotFoundError} An entry with the given key cannot be found
61+
*/
62+
async getEntry(key: string): Promise<Record<string, unknown> | undefined> {
63+
const value = this.defaults[key];
64+
if (value && typeof value === 'object') {
65+
return {key, value};
66+
}
67+
68+
if (this.warnOnMissing && !this.warnedKeys.has(key)) {
69+
this.warnedKeys.add(key);
70+
console.warn(`Local data-store provider did not find '${key}'.`);
71+
}
72+
73+
throw new DataStoreNotFoundError(`Data store entry '${key}' not found.`);
74+
}
75+
}
76+
77+
function readDefaultsFromEnv(): Record<string, Record<string, unknown>> {
78+
const raw = process.env.SFNEXT_DATA_STORE_DEFAULTS;
79+
if (!raw) {
80+
return {};
81+
}
82+
83+
try {
84+
const parsed: unknown = JSON.parse(raw);
85+
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {
86+
return parsed as Record<string, Record<string, unknown>>;
87+
}
88+
} catch (error) {
89+
console.warn('Failed to parse SFNEXT_DATA_STORE_DEFAULTS JSON.', error);
90+
}
91+
92+
return {};
93+
}
94+
95+
function readWarnOnMissingFromEnv(): boolean {
96+
const raw = process.env.SFNEXT_DATA_STORE_WARN_ON_MISSING;
97+
if (!raw) {
98+
return true;
99+
}
100+
101+
return raw.toLowerCase() !== 'false';
102+
}

‎packages/mrt-utilities/src/data-store/errors.ts‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
/*
2+
* Copyright (c) 2025, Salesforce, Inc.
3+
* SPDX-License-Identifier: Apache-2
4+
* For full license text, see the license.txt file in the repo root or http://www.apache.org/licenses/LICENSE-2.0
5+
*/
6+
7+
export class DataStoreNotFoundError extends Error {
8+
constructor(message: string) {
9+
super(message);
10+
this.name = 'DataStoreNotFoundError';
11+
Object.setPrototypeOf(this, DataStoreNotFoundError.prototype);
12+
}
13+
}
14+
15+
export class DataStoreServiceError extends Error {
16+
constructor(message: string) {
17+
super(message);
18+
this.name = 'DataStoreServiceError';
19+
Object.setPrototypeOf(this, DataStoreServiceError.prototype);
20+
}
21+
}
22+
23+
export class DataStoreUnavailableError extends Error {
24+
constructor(message: string) {
25+
super(message);
26+
this.name = 'DataStoreUnavailableError';
27+
Object.setPrototypeOf(this, DataStoreUnavailableError.prototype);
28+
}
29+
}

0 commit comments

Comments
 (0)