update docs and post deployment steps

Author: Mike Swantek

README.md

@@ -22,7 +22,7 @@ This accelerator extends the [AI Landing Zone](https://github.com/Azure/ai-landi
 
 ### Solution Architecture
 
-| ![Architecture](./img/Architecture/Deploy-AI-App-in-Prod-Architecture_final.png) |
+| ![Architecture](./img/Architecture/Depoly-AI-App-in-Prod-Architecture-final.png) |
 |---|
 
 ### Key Components
@@ -31,6 +31,7 @@ This accelerator extends the [AI Landing Zone](https://github.com/Azure/ai-landi
 |-----------|---------|
 | **Azure AI Foundry** | Unified platform for AI development, testing, and deployment with playground, prompt flow, and publishing |
 | **Microsoft Fabric** | Data foundation with lakehouses (bronze/silver/gold) for document storage and OneLake indexing |
+| **Azure Database for PostgreSQL** | Optional operational data source that can be prepared for Microsoft Fabric mirroring after deployment |
 | **Azure AI Search** | Retrieval backbone enabling RAG (Retrieval-Augmented Generation) chat experiences |
 | **Microsoft Purview** | Governance layer for cataloging, scans, and Data Security Posture Management |
 | **Private Networking** | All traffic secured via private endpoints—no public internet exposure |
@@ -61,6 +62,9 @@ This accelerator extends the [AI Landing Zone](https://github.com/Azure/ai-landi
   - **Integrated data-to-AI pipeline** <br/>
   Connect Fabric lakehouses → OneLake indexer → AI Search → Foundry playground for grounded chat experiences.
 
+  - **PostgreSQL-to-Fabric mirroring path** <br/>
+  Provision Azure Database for PostgreSQL, prepare it for Fabric mirroring, create the Fabric connection, and mirror operational data into OneLake for downstream analytics and AI scenarios.
+
   - **Governance built-in** <br/>
   Microsoft Purview integration for cataloging, scoped scans, and Data Security Posture Management (DSPM).
 
@@ -169,6 +173,7 @@ After deployment, you'll have a complete, enterprise-ready platform that unifies
 |-------|-----------------|----------------|
 | **AI Platform** | Azure AI Foundry with OpenAI models, playground, and prompt flow | Build, test, and publish AI chat applications without managing infrastructure |
 | **Data Foundation** | Microsoft Fabric with bronze/silver/gold lakehouses and OneLake indexing | Store documents at scale and automatically feed them into your AI workflows |
+| **Operational Data Mirroring** | Azure Database for PostgreSQL prepared for Fabric mirroring | Bring PostgreSQL operational data into Fabric with a documented connection and mirror setup path |
 | **Search & Retrieval** | Azure AI Search with vector and semantic search | Enable RAG (Retrieval-Augmented Generation) for grounded, accurate AI responses |
 | **Governance** | Microsoft Purview with cataloging, scans, and DSPM | Track data lineage, enforce policies, and maintain compliance visibility |
 | **Security** | Private endpoints, managed identities, RBAC, network isolation | Zero public internet exposure—all traffic stays on the Microsoft backbone |
@@ -186,6 +191,9 @@ After deployment, you'll have a complete, enterprise-ready platform that unifies
   - **Fabric-powered retrieval workflows**
     <br/>Land documents in a Fabric lakehouse, index them with OneLake + Azure AI Search, and wire the index into the Foundry playground for grounded chat experiences.
 
+  - **Fabric mirroring for PostgreSQL**
+    <br/>Prepare Azure Database for PostgreSQL for Fabric mirroring, create the Fabric connection, and mirror source data into Fabric using the documented post-deployment flow.
+
   - **Governed data and agent operations**
     <br/>Integrate Microsoft Purview for cataloging, scoped scans, and Data Security Posture Management (DSPM) so compliance teams can monitor the same assets the app consumes.
 
@@ -208,6 +216,16 @@ After deployment, you'll have a complete, enterprise-ready platform that unifies
 5. **Publish application** → Deploy the chat experience to end users
 6. **Monitor governance** → Review data lineage and security posture in Purview
 
+### PostgreSQL Mirroring Setup
+
+If you deploy Azure Database for PostgreSQL, the repo also supports a documented Fabric mirroring path after deployment:
+
+1. Prepare the PostgreSQL server and mirroring user with the provided automation.
+2. Create the Fabric PostgreSQL connection using the `fabric_user` credentials stored in Key Vault.
+3. Start the mirror in Fabric so PostgreSQL data lands in OneLake.
+
+See the detailed steps in [docs/postgresql_mirroring.md](./docs/postgresql_mirroring.md) and the shorter checklist in [docs/post_deployment_steps.md](./docs/post_deployment_steps.md).
+
 <br/>
 
 <!------------------------------------------>

docs/PARAMETER_GUIDE.md

@@ -444,10 +444,22 @@ Use these in `infra/main.bicepparam` when deploying via this repo. `postgreSqlNe
 ```bicep-params
 param deployPostgreSql = true
 param postgreSqlNetworkIsolation = networkIsolation
+param postgreSqlMirrorConnectionMode = 'fabricUser'
+param postgreSqlAuthConfig = {
+  activeDirectoryAuth: 'Enabled'
+  passwordAuth: 'Enabled'
+}
 ```
 
 When `postgreSqlNetworkIsolation` is `false`, PostgreSQL uses public access and does not create private endpoints or private DNS resources.
 
+`postgreSqlAuthConfig` should remain set to both authentication modes enabled if you plan to configure Fabric mirroring after deployment. This ensures the server is created with password authentication available for the `fabric_user` connection instead of relying on a later hook to change the auth mode.
+
+`postgreSqlMirrorConnectionMode` controls which credential the manual Fabric PostgreSQL connection should use after deployment:
+
+- `fabricUser` uses the dedicated least-privilege mirroring user and `postgres-fabric-user-password`. This is the production-oriented default.
+- `admin` uses the PostgreSQL admin login and `postgres-admin-password`. This is intended for demo automation scenarios where you want to avoid creating a separate mirroring user.
+
 ### Storage Account
 
 ```json

docs/post_deployment_steps.md

@@ -46,18 +46,22 @@ az fabric capacity resume --capacity-name <capacity-name> --resource-group <rg-n
    - **gold** — Curated analytics-ready data
 
 5. Open the **bronze** lakehouse and verify the `Files/documents` folder structure exists
+6. In the workspace, check each lakehouse (**bronze**, **silver**, **gold**) and confirm the **Sensitivity label** matches the value set in the parameter file.
 
 ### PostgreSQL Mirroring (if enabled)
 
-Use these short steps to create the Fabric connection and enable mirroring. For full details and troubleshooting, see [PostgreSQL mirroring](./postgresql_mirroring.md).
+Use these short steps to verify the automatic Fabric connection and mirroring flow. For full details and troubleshooting, see [PostgreSQL mirroring](./postgresql_mirroring.md).
 
-0. In **Azure Portal** → **Key Vault** → your vault → **Networking**, set **Public access** to **Allow public access from specific virtual networks and IP addresses**, add your client IP, then **Apply**. This lets you read the `fabric_user` password from the vault.
+0. In **Azure Portal** → **Key Vault** → your vault → **Networking**, set **Public access** to **Allow public access from specific virtual networks and IP addresses**, add your client IP, then **Apply**. This lets you read the PostgreSQL connection password from the vault.
    After you retrieve the secret, remove your IP and **Apply** again to re-lock the vault.
-1. In Fabric, open the workspace, then select **Connections** → **New** → **PostgreSQL**.
-2. Use the PostgreSQL server name, database name, and the `fabric_user` credentials stored in Key Vault.
-3. Test the connection and **Save**.
-4. In the workspace, select **New** → **Data pipeline** → **Mirror database**.
-5. Pick the PostgreSQL connection, select the target database, and **Start mirroring**.
+1. Check the resolved mirroring identity instead of hardcoding it:
+   - `azd env get-value postgreSqlMirrorConnectionModeOut`
+   - `azd env get-value postgreSqlMirrorConnectionUserNameOut`
+   - `azd env get-value postgreSqlMirrorConnectionSecretNameOut`
+2. Run `pwsh ./scripts/automationScripts/FabricWorkspace/Mirror/create_postgresql_mirror.ps1` if Stage 7.5 did not already complete it.
+3. Verify `azd env get-value fabricPostgresConnectionId` now returns a Fabric connection ID.
+4. In Fabric, confirm the PostgreSQL connection exists under **Connections** and that the mirrored database is running.
+5. If your PostgreSQL source requires a Fabric VNet gateway, set `azd env set-value fabricPostgresGatewayId "<gateway-id>"` and rerun the script.
 
 ---
 

docs/postgresql_mirroring.md

@@ -7,13 +7,8 @@ This guide explains how to complete PostgreSQL mirroring in Microsoft Fabric aft
 What is automated today:
 
 - PostgreSQL server prep (roles, grants, seed table, parameters).
-- Mirror creation **after** a Fabric connection exists (scripted).
-
-What is still manual and why:
-
-- Fabric connection creation is **portal-only** today. The public Fabric API does not currently expose a supported endpoint to create PostgreSQL connections, so the connection must be created in the UI to obtain a `connectionId`.
-
-Once Fabric exposes a supported API for connection creation, this step can be fully automated.
+- Fabric connection creation or reuse for PostgreSQL mirroring.
+- Mirror creation after the Fabric connection is resolved.
 
 ## Why a Fabric Connection Is Required
 
@@ -25,16 +20,17 @@ The Fabric mirroring API requires a Fabric "connection" object that stores the P
 - You can sign in to Fabric (app.fabric.microsoft.com) with access to the workspace.
 - PostgreSQL authentication mode is **PostgreSQL and Microsoft Entra authentication** (password auth enabled).
 - You have access to the Key Vault that stores the PostgreSQL secrets.
+- Decide which connection mode you are using: `fabricUser` (default) or `admin` via `postgreSqlMirrorConnectionMode`.
 
 ## Step 1: Confirm PostgreSQL Details
 
 Get the PostgreSQL server FQDN and database name:
 
 - FQDN: from `azd env get-value postgreSqlServerFqdn`
 - Database name: `postgres` (default) or your custom DB
-- Admin login: `pgadmin`
-- Fabric login: `fabric_user` (used by Fabric)
-- Fabric password: Key Vault secret `postgres-fabric-user-password`
+- Connection mode: from `azd env get-value postgreSqlMirrorConnectionModeOut`
+- Fabric login: from `azd env get-value postgreSqlMirrorConnectionUserNameOut`
+- Fabric password secret name: from `azd env get-value postgreSqlMirrorConnectionSecretNameOut`
 
 ## Step 2: Prepare the Database (Automated by Default)
 
@@ -56,10 +52,10 @@ $env:POSTGRES_TEMP_ENABLE_KV_PUBLIC_ACCESS = 'true'
 
 What it does now:
 
-- Creates or validates the `fabric_user` role.
+- Creates or validates the `fabric_user` role when mode is `fabricUser`.
 - Ensures PostgreSQL auth modes are enabled (password + Entra).
 - Grants `azure_cdc_admin` and database permissions.
-- Creates a seed table: `public.fabric_mirror_seed` (owned by the mirroring user when created as `fabric_user`).
+- Creates a seed table: `public.fabric_mirror_seed` (owned by the mirroring identity, either `fabric_user` or `pgadmin`).
 - Uses `psql` fallback when `rdbms-connect` cannot install.
 
 ### Manual (only if automation fails)
@@ -92,19 +88,53 @@ az keyvault secret set --vault-name <keyvault-name> --name postgres-fabric-user-
 
 > Ownership note: Fabric requires the mirror user to own tables. If you create tables as `pgadmin`, change ownership to `fabric_user`.
 
-## Step 3: Create the Fabric Connection (UI)
+## Step 3: Create or Reuse the Fabric Connection (Automated by Default)
+
+Run:
+
+```powershell
+pwsh ./scripts/automationScripts/FabricWorkspace/Mirror/create_postgresql_mirror.ps1
+```
+
+What the script does now:
+
+- Reuses `fabricPostgresConnectionId` when it is already stored in `azd`.
+- Otherwise resolves the connection login from `postgreSqlMirrorConnectionUserNameOut`.
+- Resolves the connection password secret name from `postgreSqlMirrorConnectionSecretNameOut`.
+- Reads the chosen secret from Key Vault, creates or reuses the Fabric PostgreSQL connection, and stores the resulting `fabricPostgresConnectionId` back into `azd`.
+- Creates the mirrored database after the connection is available.
+
+If your PostgreSQL server is reachable only through a Fabric VNet data gateway, set the gateway ID before rerunning the script:
+
+```powershell
+azd env set-value fabricPostgresGatewayId "<fabric-vnet-gateway-id>"
+```
+
+Without `fabricPostgresGatewayId`, the script creates a standard cloud connection.
+
+### Manual fallback
+
+If you need to create the Fabric connection manually, do not hardcode `fabric_user`, `pgadmin`, or the secret name. Read the values from the deployment outputs first:
+
+```powershell
+azd env get-value postgreSqlMirrorConnectionModeOut
+azd env get-value postgreSqlMirrorConnectionUserNameOut
+azd env get-value postgreSqlMirrorConnectionSecretNameOut
+```
+
+Then in Fabric:
 
 1. Open the Fabric workspace.
 2. Go to **Settings** -> **Manage connections and gateways**.
 3. Select **New connection** -> **PostgreSQL**.
 4. Enter:
    - Server: PostgreSQL FQDN (example: `pg-<env>.postgres.database.azure.com`)
    - Database: `postgres` (or your custom DB)
-   - User: `fabric_user` (example: `fabric_user`)
-   - Password: value from Key Vault secret `postgres-fabric-user-password`
+   - User: the value from `postgreSqlMirrorConnectionUserNameOut`
+   - Password: the Key Vault secret value stored under `postgreSqlMirrorConnectionSecretNameOut`
 5. Save and copy the **Connection ID**.
 
-## Step 4: Set the Connection ID in azd
+## Step 4: Persist the Connection ID in azd (only if you created it manually)
 
 ```powershell
 azd env set-value fabricPostgresConnectionId "<connection-id>"
@@ -113,7 +143,7 @@ azd env set-value POSTGRES_DATABASE_NAME "postgres"
 
 ## Step 5: Create the Mirror
 
-Run the mirror script (this is the automation step after the connection exists):
+If the previous script already created the connection automatically, re-running it is safe and idempotent. If you created the connection manually, run it once now:
 
 ```powershell
 ./scripts/automationScripts/FabricWorkspace/Mirror/create_postgresql_mirror.ps1
@@ -126,16 +156,22 @@ Run the mirror script (this is the automation step after the connection exists):
 
 ## Notes
 
-- The deployment now skips the mirror step until a valid Fabric connection exists, so `azd up` will no longer fail on this step.
+- The deployment now attempts to create or reuse the Fabric PostgreSQL connection automatically before creating the mirror.
+- If automatic connection creation cannot reach Key Vault or the source database, the script exits without failing the entire deployment and leaves a manual fallback path.
 - If you rotate passwords, update the Fabric connection in the workspace.
 
 ## Troubleshooting
 
 ### Invalid credentials
 
 - Ensure PostgreSQL auth is **PostgreSQL and Microsoft Entra authentication** (password auth enabled).
-- Use `fabric_user` in the Fabric connection.
-- Verify the Key Vault secret matches the role password. Automation sets it unless it failed.
+- Use the login from `postgreSqlMirrorConnectionUserNameOut` in the Fabric connection.
+- Verify the Key Vault secret named by `postgreSqlMirrorConnectionSecretNameOut` matches the chosen connection credential.
+
+### Private networking or gateway-required sources
+
+- If the PostgreSQL server is private-only, set `fabricPostgresGatewayId` in `azd` before rerunning the script so the connection is created under the Fabric VNet gateway.
+- If the gateway ID is not set, the automation uses a shareable cloud connection.
 
 ### Must be owner of table