Merge pull request #133 from Harmanpreet-Microsoft/fabric-byo-enhancements

chore: Enhance deployment guides and improve OneLake data source creation

Author: Roopan-Microsoft

docs/deploymentguide.md

@@ -191,6 +191,42 @@ Edit `infra/main.bicepparam` or set environment variables:
 # var fabricWorkspacePreset = 'none'
 ```
 
+#### Reusing an Existing Fabric Capacity and Workspace (BYO mode)
+
+If you already have a Fabric capacity and workspace, set `byo` mode so the deployment skips creating new ones. The bicepparam variables are driven by environment variables, so the recommended approach is to set them with `azd env set` before running `azd up`:
+
+**Step 1 — Set the mode in `infra/main.bicepparam`** (or leave the default `byo` unchanged):
+
+```bicep
+// infra/main.bicepparam
+var fabricCapacityPreset = readEnvironmentVariable('fabricCapacityMode', 'byo')
+```
+
+The `fabricCapacityMode` env variable controls both capacity and workspace preset (they are tied together). Set it explicitly if the checked-in default has been changed:
+
+```powershell
+azd env set fabricCapacityMode byo
+```
+
+**Step 2 — Supply the existing resource identifiers:**
+
+```powershell
+# ARM resource ID of the existing Fabric capacity
+azd env set fabricCapacityResourceId "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Fabric/capacities/<capacity-name>"
+
+# GUID of the existing Fabric workspace (from the workspace URL or Fabric portal)
+azd env set FABRIC_WORKSPACE_ID "<workspace-guid>"
+
+# Display name of the existing workspace (used for naming/UX; optional but recommended)
+azd env set FABRIC_WORKSPACE_NAME "<workspace-display-name>"
+```
+
+> **How to find the workspace GUID:** Open the workspace in [app.fabric.microsoft.com](https://app.fabric.microsoft.com), copy the URL. The segment after `/groups/` is the workspace GUID (e.g., `https://app.fabric.microsoft.com/groups/e9c7ed61-0cdc-4356-a239-9d49cc755fe0/...` → `e9c7ed61-0cdc-4356-a239-9d49cc755fe0`).
+
+> **How to find the capacity resource ID:** In Azure Portal, open the Fabric capacity resource → **Properties** → copy **Resource ID**. It follows the pattern `/subscriptions/.../providers/Microsoft.Fabric/capacities/<name>`.
+
+After setting these variables, run `azd up` normally. The deployment will attach to your existing capacity and workspace instead of creating new ones.
+
 </details>
 
 <details>

docs/parameter_guide.md

@@ -8,12 +8,80 @@ This guide focuses on configuration concepts for the **AI Landing Zone**.
 > - AI Landing Zone submodule parameters file (if you deploy it directly): `submodules/ai-landing-zone/main.parameters.json`
 >
 > **Fabric options in this repo** are configured in `infra/main.bicepparam` via:
-> - `fabricCapacityPreset` (`create` | `byo` | `none`)
-> - `fabricWorkspacePreset` (`create` | `byo` | `none`)
-> - BYO inputs: `fabricCapacityResourceId`, `fabricWorkspaceId`, `fabricWorkspaceName`
+> - `fabricCapacityPreset` (`create` | `byo` | `none`) — driven by the `fabricCapacityMode` env variable
+> - `fabricWorkspacePreset` (`create` | `byo` | `none`) — mirrors `fabricCapacityPreset` by default
+> - BYO inputs: `fabricCapacityResourceId` (env), `FABRIC_WORKSPACE_ID` (env), `FABRIC_WORKSPACE_NAME` (env)
 
 > **Deployment flow**: This repo deploys the AI Landing Zone submodule from `submodules/ai-landing-zone/main.bicep` during the preprovision hook. The single source of truth for parameters is `infra/main.bicepparam`.
 
+## Fabric Configuration
+
+### Modes: create, byo, none
+
+| Mode | Description |
+|------|-------------|
+| `create` | Provisions a new Fabric capacity (Bicep) and workspace (postprovision script) |
+| `byo` | Reuses an existing Fabric capacity and workspace — no new resources created |
+| `none` | Disables all Fabric automation; OneLake indexing will be skipped |
+
+Both capacity and workspace modes are controlled by the same `fabricCapacityMode` environment variable (they are tied together in `infra/main.bicepparam`).
+
+### Setting Mode via azd env
+
+The recommended way to configure Fabric mode is with `azd env set` — these values are read directly by `infra/main.bicepparam` at provision time:
+
+```powershell
+# Choose one:
+azd env set fabricCapacityMode create   # create new capacity + workspace
+azd env set fabricCapacityMode byo      # reuse existing capacity + workspace (default)
+azd env set fabricCapacityMode none     # disable all Fabric automation
+```
+
+### Reusing Existing Fabric Resources (BYO)
+
+When `fabricCapacityMode` is `byo`, supply the identifiers of your existing resources:
+
+```powershell
+# ARM resource ID of the existing Fabric capacity
+azd env set fabricCapacityResourceId "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Fabric/capacities/<capacity-name>"
+
+# GUID of the existing Fabric workspace (from the workspace URL)
+azd env set FABRIC_WORKSPACE_ID "<workspace-guid>"
+
+# Display name of the existing workspace (optional, used for naming/UX)
+azd env set FABRIC_WORKSPACE_NAME "<workspace-display-name>"
+```
+
+> **How to find the workspace GUID:** Open the workspace in [app.fabric.microsoft.com](https://app.fabric.microsoft.com). The URL segment after `/groups/` is the GUID (e.g., `https://app.fabric.microsoft.com/groups/e9c7ed61-0cdc-4356-a239-9d49cc755fe0/...`).
+>
+> **How to find the capacity resource ID:** Azure Portal → Fabric capacity resource → **Properties** → **Resource ID**.
+
+You can also set these directly in `infra/main.bicepparam` if you prefer source-controlled values:
+
+```bicep
+// infra/main.bicepparam
+var fabricCapacityPreset = 'byo'
+param fabricCapacityResourceId = '/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Fabric/capacities/<name>'
+param fabricWorkspaceId = '<workspace-guid>'
+param fabricWorkspaceName = '<workspace-display-name>'
+```
+
+> **Note:** Values set via `azd env set` take precedence over hardcoded bicepparam values because `readEnvironmentVariable(...)` is evaluated at deploy time.
+
+### Creating New Fabric Resources
+
+When `fabricCapacityMode` is `create`, you must provide at least one admin principal:
+
+```bicep
+// infra/main.bicepparam
+param fabricCapacityAdmins = ['user@contoso.com']
+param fabricCapacitySku = 'F2'  // adjust SKU as needed
+```
+
+> **Permission requirement:** The identity running `azd` must have the **Fabric Administrator** role (or Power BI tenant admin) to call the workspace admin APIs used during postprovision.
+
+---
+
 ## Table of Contents
 1. [Basic Parameters](#basic-parameters)
 2. [Deployment Toggles](#deployment-toggles)

infra/main.bicepparam

@@ -236,11 +236,11 @@ var fabricWorkspacePreset = fabricCapacityPreset
 param deployFabricCapacity = fabricCapacityPreset != 'none'
 
 param fabricCapacityMode = fabricCapacityPreset
-param fabricCapacityResourceId = '' // required when fabricCapacityPreset='byo'
+param fabricCapacityResourceId = readEnvironmentVariable('fabricCapacityResourceId', '') // required when fabricCapacityPreset='byo'
 
 param fabricWorkspaceMode = fabricWorkspacePreset
-param fabricWorkspaceId = '' // required when fabricWorkspacePreset='byo'
-param fabricWorkspaceName = '' // optional (helpful for naming/UX)
+param fabricWorkspaceId = readEnvironmentVariable('FABRIC_WORKSPACE_ID', readEnvironmentVariable('fabricWorkspaceId', '')) // required when fabricWorkspacePreset='byo'
+param fabricWorkspaceName = readEnvironmentVariable('FABRIC_WORKSPACE_NAME', readEnvironmentVariable('fabricWorkspaceName', '')) // optional (helpful for naming/UX)
 
 // Fabric capacity SKU.
 param fabricCapacitySku = 'F8'

scripts/automationScripts/FabricWorkspace/CreateWorkspace/create_lakehouses.ps1

@@ -302,6 +302,13 @@ if ($names -contains "bronze") {
         } else {
           Set-Content -Path $workspaceEnvPath -Value $lakehouseExports
         }
+
+        # Persist lakehouse IDs in azd env so downstream hooks and reruns can find them
+        try { azd env set FABRIC_LAKEHOUSE_ID $bronzeLakehouse.id } catch {}
+        foreach ($lakehouse in $existingLakehouses.value) {
+          $lhName = if ($null -ne $lakehouse.PSObject.Properties['displayName']) { $lakehouse.displayName } else { $lakehouse.name }
+          try { azd env set "FABRIC_LAKEHOUSE_${lhName}_ID" $lakehouse.id } catch {}
+        }
 
       } catch {
         Warn "Failed to export lakehouse IDs: $($_.Exception.Message)"

scripts/automationScripts/OneLakeIndex/04_create_onelake_datasource.ps1

@@ -86,6 +86,19 @@ if (-not $subscription) { $subscription = $env:AZURE_SUBSCRIPTION_ID }
 if (-not $workspaceId) { $workspaceId = $env:FABRIC_WORKSPACE_ID }
 if (-not $lakehouseId) { $lakehouseId = $env:FABRIC_LAKEHOUSE_ID }
 
+# Try azd outputs (Bicep emits fabricWorkspaceIdOut for BYO mode)
+if (-not $workspaceId -and $outputs) {
+    if ($outputs.fabricWorkspaceIdOut -and $outputs.fabricWorkspaceIdOut.value) { $workspaceId = $outputs.fabricWorkspaceIdOut.value }
+    elseif ($outputs.fabricWorkspaceId -and $outputs.fabricWorkspaceId.value) { $workspaceId = $outputs.fabricWorkspaceId.value }
+}
+if (-not $lakehouseId -and $outputs) {
+    if ($outputs.fabricLakehouseId -and $outputs.fabricLakehouseId.value) { $lakehouseId = $outputs.fabricLakehouseId.value }
+}
+
+# Try azd env store (persisted by create_lakehouses.ps1)
+if (-not $workspaceId) { try { $val = & azd env get-value FABRIC_WORKSPACE_ID 2>$null; if ($val) { $workspaceId = $val.ToString().Trim() } } catch {} }
+if (-not $lakehouseId) { try { $val = & azd env get-value FABRIC_LAKEHOUSE_ID 2>$null; if ($val) { $lakehouseId = $val.ToString().Trim() } } catch {} }
+
 # Try temp fabric_workspace.env (from create_fabric_workspace.ps1)
 if ((-not $workspaceId -or -not $lakehouseId) -and (Test-Path (Join-Path ([IO.Path]::GetTempPath()) 'fabric_workspace.env'))) {
     Get-Content (Join-Path ([IO.Path]::GetTempPath()) 'fabric_workspace.env') | ForEach-Object {