Commit e24bb86
feat: Adding AI Landing Zone and Fabric workspace automation to provisioning (#101)
* feat: streamlined azd deployment using AI Landing Zone submodule
- Remove all local Bicep modules (infra/modules/*) to eliminate duplication
- Create minimal main.bicep (160 lines) that directly calls AI Landing Zone submodule
- Add comprehensive main.parameters.json with deployment toggles and service configuration
- Add QUICKSTART.md for 5-minute deployment instructions
- Add detailed docs/AZD_DEPLOYMENT.md with full parameter reference
- Configure for azd CLI deployment workflow
- Enable AI Foundry with GPT-4o and text-embedding-3-small models
- All services deployed with private endpoints for security
- Type-safe parameters using AI Landing Zone type definitions
* docs: add comprehensive deployment summary
* docs: add comprehensive parameter customization guide
* feat: add modern .bicepparam file with type safety and IntelliSense
- Add infra/main.bicepparam with full type safety and validation
- Remove 'role' properties from vNet subnets (use standard AVM schema)
- Add delegation for Container Apps subnet
- Add comprehensive inline documentation and comments
- Add docs/BICEP_PARAMETERS.md explaining both formats
- Keep main.parameters.json for backward compatibility
- Benefits: type safety, IntelliSense, compile-time validation, better DX
* docs: update QUICKSTART to recommend bicepparam file
* fix: align deployToggles with AI Landing Zone defaults and add clarifying comments
- Keep current practical defaults (core services enabled, optional disabled)
- Add note that AI LZ example has everything set to true
- Add helpful inline comments explaining when to enable each toggle
- Clarify NSG toggles should match their corresponding service toggles
- Improve developer guidance for customization
* fix: enable Bastion and Jump VM by default for accessing private endpoints
BREAKING: Resources now use private endpoints without public access
REQUIRED: Bastion + Jump VM needed to manage and access services
- Enable bastionHost: true (required for accessing private resources)
- Enable jumpVm: true (Windows jump box accessed via Bastion)
- Enable bastionNsg: true and jumpboxNsg: true (required NSGs)
- Add AzureBastionSubnet (10.0.5.0/26) to vNet
- Add snet-jumpbox (10.0.6.0/28) to vNet
This is the correct default for a secure, production deployment with
private endpoints - you MUST have a way to access the resources!
* docs: add comprehensive guide for accessing private resources
- Add docs/ACCESSING_PRIVATE_RESOURCES.md with detailed access instructions
- Explain Bastion + Jump VM requirement for private endpoints
- Document cost implications (~75/month for secure access)
- Provide cost optimization strategies (stop VM when not in use)
- Add alternative access methods (VPN, Build VM, public endpoints)
- Include security best practices and troubleshooting
- Update QUICKSTART.md to list Bastion in deployment output
* docs: add prominent warnings about ARM 4MB template size limit
- Add warning at top of AZD_DEPLOYMENT.md about RequestContentTooLarge error
- Add 'If Deployment Fails' section to QUICKSTART.md with immediate fix
- Document that default config with Bastion enabled will likely fail
- Provide clear fix: disable Bastion initially, add later via idempotent redeployment
- Add comprehensive troubleshooting section in AZD_DEPLOYMENT.md
- Explain trade-offs: public vs private endpoints, cost implications
- Keep all parameters flexible - users choose their configuration
This addresses the immediate issue users will face with default configuration
while maintaining flexibility and documenting the upgrade path.
* feat: integrate AI Landing Zone submodule with Template Spec support
- Add AI Landing Zone as git submodule for clean architecture
- Create minimal wrapper (160 lines) that calls AI Landing Zone main.bicep
- Implement preprovision scripts (PowerShell + Bash) for Template Spec creation
- Configure AI Landing Zone defaults (192.168.0.0/22 network, proper subnet names)
- Template Specs bypass ARM 4MB deployment limit
- Clean azure.yaml with only preprovision hook (no legacy sample app scripts)
- Successful deployment of full stack: VNet, AI Services, GPT-4o, Cosmos DB, AI Search, Container Apps, Bastion
* chore: update AI Landing Zone submodule pointer
* feat: Complete 5-stage modular deployment with conditional toggles
- Created main-orchestrator.bicep with 5-stage deployment pattern
- Implemented centralized deployToggles parameter in main-orchestrator.bicepparam
- Built Stage 1 (Networking): VNet + 5 NSGs with conditional deployment
- Built Stage 2 (Monitoring): Log Analytics + App Insights with conditionals
- Built Stage 3 (Security): KeyVault + Bastion + Jump VM with conditionals
- Built Stage 4 (Data): Storage, Cosmos, Search, ACR + private endpoints with conditionals
- Built Stage 5 (Compute/AI): Container Apps Environment + AI Foundry with conditionals
- All stages use AI Landing Zone wrappers and follow exact naming patterns
- All 11 deployment toggles functional and set to true for complete deployment
- Updated azure.yaml to use main-orchestrator as deployment entry point
- Successfully tested full deployment with all resources deployed
* chore: Remove unused deployment files
- Deleted infra/main.bicep (old monolithic deployment)
- Deleted infra/main.bicepparam (old parameter file)
- Deleted infra/main.parameters.json (old JSON parameters)
- Deleted infra/orchestrators/main-modular.bicep (earlier iteration)
- Deleted infra/params/ directory (no longer needed)
All deployments now use main-orchestrator.bicep with 5-stage modular architecture
* chore: Remove unused scripts and Python requirements
Deleted Template Spec scripts (modular deployment doesn't need them):
- scripts/preprovision-integrated.ps1
- scripts/preprovision-integrated.sh
Deleted Python/sample data scripts (not using Python):
- scripts/install_python.ps1
- scripts/process_sample_data.ps1
- scripts/process_sample_data.sh
- scripts/index_scripts/ (entire directory)
- scripts/auth_init.py
- scripts/auth_update.py
- requirements.txt
- requirements-dev.txt
Deleted connection/testing scripts (infrastructure only):
- scripts/set_conns_env_vars.ps1
- scripts/set_conns_env_vars.sh
- scripts/test_azure_resource_conns.ps1
Kept scripts (still useful):
- scripts/auth_init.ps1 / auth_init.sh (basic auth)
- scripts/loadenv.ps1 / loadenv.sh (environment variables)
- scripts/postprovision.ps1 / postprovision.sh (post-deployment)
- scripts/quota_check.sh (quota checking)
* chore: Remove outdated documentation files
- Deleted DEPLOYMENT_SUMMARY.md (refers to old feature/azd-submodule-deployment branch)
- Deleted QUICKSTART.md (outdated, refers to non-existent main.bicepparam)
Current documentation:
- QUICKSTART_MODULAR.md - Quick start for modular deployment
- docs/MODULAR_DEPLOYMENT.md - Full modular deployment documentation
- README.md - Updated with deployment options
* Fix subnet layout and Application Gateway private IP to match AI Landing Zone. Resolves Azure Firewall deployment failures due to subnet overlap.
* Apply AI Landing Zone variable pattern to all 5 stages. Resolves Bicep conditional output errors.
* Mark as internal development branch
* Add Stage 6 (Fabric Capacity) and import automation scripts from fabric-purview-domain-integration
- Added Stage 6: Microsoft Fabric Capacity deployment with AVM module v0.1.2
- Added all parameters for future phases: Fabric, Purview, AI Services, Lakehouses
- Imported 37 automation scripts from fabric-purview-domain-integration repo:
* Fabric_Purview_Automation/ - Fabric workspace, domain, lakehouse automation
* OneLakeIndex/ - OneLake document indexing with AI Search
* SecurityModule.ps1 - Centralized token security
* cleanup/ - Workspace cleanup utilities
* monitoring/ - Workflow telemetry
- Fixed stage3-security.bicep bastion module path (deploy/wrappers)
- Updated main-orchestrator.bicep with organized parameter sections
- Updated main-orchestrator.bicepparam with all new parameters
- Ready for next phase: Fabric workspace and Purview integration automation
* feat: Add Fabric private networking with automated public access control
Major enhancements:
- Added Stage 7: Fabric Private Networking infrastructure (DNS zones, VNet links)
- Created setup_fabric_private_link.ps1 with auto-approved shared private link creation
- Automated workspace communication policy to deny public access via Fabric REST API
- Added 11 deployment outputs to main-orchestrator.bicep for script automation
- Enhanced azure.yaml with 17 postprovision automation stages
- Created comprehensive documentation (fabric-onelake-private-networking.md, automation-outputs-mapping.md)
* feat: Add cross-subscription Purview support and fix deployment issues
CROSS-SUBSCRIPTION PURVIEW INTEGRATION:
- Added purviewSubscriptionId and purviewResourceGroup parameters to main-orchestrator.bicep
- Added corresponding outputs for script consumption
- Updated create_purview_collection.ps1 to handle cross-subscription Purview accounts
- Updated trigger_purview_scan_for_fabric_workspace.ps1 with subscription/RG resolution
- Tested successfully with Purview account in different subscription (48ab3756-f962-40a8-b0cf-b33ddae744bb)
DEPLOYMENT FIXES:
- Fixed VM disk type mismatches (Premium_LRS → Standard_LRS) in stage3-security.bicep and stage5-compute-ai.bicep
* Azure doesn't allow changing disk SKU on existing VMs through ARM deployment
* Changed to Standard_LRS to match already-deployed infrastructure
- Added environment variable fallbacks to create_fabric_workspace.ps1 for azd output variables
* Added fallback for $env:desiredFabricWorkspaceName
* Added fallback for $env:fabricCapacityId
* Scripts now work correctly with azd hooks
FABRIC WORKSPACE AUTOMATION:
- Successfully created workspace 'workspace002' and assigned to capacity FE509DCC-0864-4EBD-B69E-576E4E286AC5
- Successfully created domain 'datadomain002' and assigned workspace
- Workspace admin configured: admin@MngEnv282784.onmicrosoft.com
TESTING STATUS:
✅ Infrastructure deploys without errors
✅ Fabric workspace automation working
✅ Purview collection creation working (cross-subscription)
1 parent 6ed2592 commit e24bb86
File tree
159 files changed
+14850
-112091
lines changed- .github/workflows
- data
- docs
- examples
- images
- re_use_log
- readme
- img
- Architecture
- provisioning
- infra
- modules
- ai-foundry-project
- avm/cognitive-services
- modules
- cognitive-services
- vscode_web
- scripts
- automationScripts
- FabricPurviewAutomation
- shell
- FabricWorkspace
- CreateWorkspace
- SecureWorkspace
- OneLakeIndex
- cleanup
- monitoring
- index_scripts
- submodules
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
159 files changed
+14850
-112091
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
| 1 | + | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | 9 | | |
11 | | - | |
| 10 | + | |
12 | 11 | | |
13 | 12 | | |
14 | | - | |
| 13 | + | |
15 | 14 | | |
16 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
17 | 23 | | |
18 | | - | |
19 | | - | |
20 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
21 | 36 | | |
22 | 37 | | |
23 | 38 | | |
24 | 39 | | |
25 | 40 | | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
| 41 | + | |
| 42 | + | |
44 | 43 | | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
| 44 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
5 | 35 | | |
6 | 36 | | |
7 | 37 | | |
| |||
0 commit comments