Skip to content

Commit e6ff115

Browse files
Prashant-MicrosoftPrashant-Microsoft
committed
Merge remote-tracking branch 'origin/hotfix' into psl-backend-unit-test
2 parents 1bfbef0 + 28f55d5 commit e6ff115

84 files changed

Lines changed: 2183 additions & 1250 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.flake8‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
max-line-length = 88
33
extend-ignore = E501
44
exclude = .venv, frontend
5-
ignore = E203, W503, G004, G200,B008,ANN,D100,D101,D102,D103,D104,D105,D106,D107
5+
ignore = E203, W503, G004, G200,B008,ANN,D100,D101,D102,D103,D104,D105,D106,D107,D205,D400,D401,D200

‎.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
<!--
2+
IF SUFFICIENT INFORMATION IS NOT PROVIDED VIA THE FOLLOWING TEMPLATE THE ISSUE MIGHT BE CLOSED WITHOUT FURTHER CONSIDERATION OR INVESTIGATION
3+
-->
4+
> Please provide us with the following information:
5+
> ---------------------------------------------------------------
6+
7+
### This issue is for a: (mark with an `x`)
8+
```
9+
- [ ] bug report -> please search issues before submitting
10+
- [ ] feature request
11+
- [ ] documentation issue or request
12+
- [ ] regression (a behavior that used to work and stopped in a new release)
13+
```
14+
15+
### Minimal steps to reproduce
16+
>
17+
18+
### Any log messages given by the failure
19+
>
20+
21+
### Expected/desired behavior
22+
>
23+
24+
### OS and Version?
25+
> Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
26+
27+
### azd version?
28+
> run `azd version` and copy paste here.
29+
30+
### Versions
31+
>
32+
33+
### Mention any other details that might be useful
34+
35+
> ---------------------------------------------------------------
36+
> Thanks! We'll be in touch soon.

‎.github/workflows/deploy.yml‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
name: Validate Deployment
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
schedule:
8+
- cron: '0 5,17 * * *' # Runs at 5:00 AM and 5:00 PM GMT
9+
10+
jobs:
11+
deploy:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Checkout Code
15+
uses: actions/checkout@v3
16+
17+
- name: Setup Azure CLI
18+
run: |
19+
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
20+
az --version # Verify installation
21+
22+
- name: Login to Azure
23+
run: |
24+
az login --service-principal -u ${{ secrets.AZURE_CLIENT_ID }} -p ${{ secrets.AZURE_CLIENT_SECRET }} --tenant ${{ secrets.AZURE_TENANT_ID }}
25+
26+
- name: Install Bicep CLI
27+
run: az bicep install
28+
29+
- name: Generate Resource Group Name
30+
id: generate_rg_name
31+
run: |
32+
echo "Generating a unique resource group name..."
33+
TIMESTAMP=$(date +%Y%m%d%H%M%S)
34+
COMMON_PART="ci-mycsa"
35+
UNIQUE_RG_NAME="${COMMON_PART}${TIMESTAMP}"
36+
echo "RESOURCE_GROUP_NAME=${UNIQUE_RG_NAME}" >> $GITHUB_ENV
37+
echo "Generated Resource_GROUP_PREFIX: ${UNIQUE_RG_NAME}"
38+
39+
40+
- name: Check and Create Resource Group
41+
id: check_create_rg
42+
run: |
43+
set -e
44+
echo "Checking if resource group exists..."
45+
rg_exists=$(az group exists --name ${{ env.RESOURCE_GROUP_NAME }})
46+
if [ "$rg_exists" = "false" ]; then
47+
echo "Resource group does not exist. Creating..."
48+
az group create --name ${{ env.RESOURCE_GROUP_NAME }} --location northcentralus || { echo "Error creating resource group"; exit 1; }
49+
else
50+
echo "Resource group already exists."
51+
fi
52+
53+
54+
- name: Deploy Bicep Template
55+
id: deploy
56+
run: |
57+
set -e
58+
az deployment group create \
59+
--resource-group ${{ env.RESOURCE_GROUP_NAME }} \
60+
--template-file infra/main.bicep \
61+
--parameters ResourcePrefix=codegen AiLocation=northcentralus
62+
63+
64+
- name: Send Notification on Failure
65+
if: failure()
66+
run: |
67+
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
68+
69+
# Construct the email body
70+
EMAIL_BODY=$(cat <<EOF
71+
{
72+
"body": "<p>Dear Team,</p><p>We would like to inform you that the Modernize-your-code-solution-accelerator Automation process has encountered an issue and has failed to complete successfully.</p><p><strong>Build URL:</strong> ${RUN_URL}<br> ${OUTPUT}</p><p>Please investigate the matter at your earliest convenience.</p><p>Best regards,<br>Your Automation Team</p>"
73+
}
74+
EOF
75+
)
76+
77+
# Send the notification
78+
curl -X POST "${{ secrets.LOGIC_APP_URL }}" \
79+
-H "Content-Type: application/json" \
80+
-d "$EMAIL_BODY" || echo "Failed to send notification"
81+
82+
83+
- name: Get Log Analytics Workspace from Resource Group
84+
id: get_log_analytics_workspace
85+
run: |
86+
87+
set -e
88+
echo "Fetching Log Analytics workspace from resource group ${{ env.RESOURCE_GROUP_NAME }}..."
89+
90+
# Run the az monitor log-analytics workspace list command to get the workspace name
91+
log_analytics_workspace_name=$(az monitor log-analytics workspace list --resource-group ${{ env.RESOURCE_GROUP_NAME }} --query "[0].name" -o tsv)
92+
93+
if [ -z "$log_analytics_workspace_name" ]; then
94+
echo "No Log Analytics workspace found in resource group ${{ env.RESOURCE_GROUP_NAME }}."
95+
exit 1
96+
else
97+
echo "LOG_ANALYTICS_WORKSPACE_NAME=${log_analytics_workspace_name}" >> $GITHUB_ENV
98+
echo "Log Analytics workspace name: ${log_analytics_workspace_name}"
99+
fi
100+
101+
102+
- name: List KeyVaults and Store in Array
103+
id: list_keyvaults
104+
run: |
105+
106+
set -e
107+
echo "Listing all KeyVaults in the resource group ${RESOURCE_GROUP_NAME}..."
108+
109+
# Get the list of KeyVaults in the specified resource group
110+
keyvaults=$(az resource list --resource-group ${{ env.RESOURCE_GROUP_NAME }} --query "[?type=='Microsoft.KeyVault/vaults'].name" -o tsv)
111+
112+
if [ -z "$keyvaults" ]; then
113+
echo "No KeyVaults found in resource group ${RESOURCE_GROUP_NAME}."
114+
echo "KEYVAULTS=[]" >> $GITHUB_ENV # If no KeyVaults found, set an empty array
115+
else
116+
echo "KeyVaults found: $keyvaults"
117+
118+
# Format the list into an array with proper formatting (no trailing comma)
119+
keyvault_array="["
120+
first=true
121+
for kv in $keyvaults; do
122+
if [ "$first" = true ]; then
123+
keyvault_array="$keyvault_array\"$kv\""
124+
first=false
125+
else
126+
keyvault_array="$keyvault_array,\"$kv\""
127+
fi
128+
done
129+
keyvault_array="$keyvault_array]"
130+
131+
# Output the formatted array and save it to the environment variable
132+
echo "KEYVAULTS=$keyvault_array" >> $GITHUB_ENV
133+
fi
134+
135+
- name: Purge log analytics workspace
136+
id: log_analytics_workspace
137+
run: |
138+
139+
set -e
140+
# Purge Log Analytics Workspace
141+
echo "Purging the Log Analytics Workspace..."
142+
if ! az monitor log-analytics workspace delete --force --resource-group ${{ env.RESOURCE_GROUP_NAME }} --workspace-name ${{ env.LOG_ANALYTICS_WORKSPACE_NAME }} --yes --verbose; then
143+
echo "Failed to purge Log Analytics workspace: ${{ env.LOG_ANALYTICS_WORKSPACE_NAME }}"
144+
else
145+
echo "Purged the Log Analytics workspace: ${{ env.LOG_ANALYTICS_WORKSPACE_NAME }}"
146+
fi
147+
148+
echo "Log analytics workspace resource purging completed successfully"
149+
150+
151+
- name: Delete Bicep Deployment
152+
if: success()
153+
run: |
154+
set -e
155+
echo "Checking if resource group exists..."
156+
rg_exists=$(az group exists --name ${{ env.RESOURCE_GROUP_NAME }})
157+
if [ "$rg_exists" = "true" ]; then
158+
echo "Resource group exist. Cleaning..."
159+
az group delete \
160+
--name ${{ env.RESOURCE_GROUP_NAME }} \
161+
--yes \
162+
--no-wait
163+
echo "Resource group deleted... ${{ env.RESOURCE_GROUP_NAME }}"
164+
else
165+
echo "Resource group does not exists."
166+
fi
167+
168+
169+
- name: Wait for resource deletion to complete
170+
run: |
171+
172+
# List of keyvaults
173+
KEYVAULTS="${{ env.KEYVAULTS }}"
174+
175+
# Remove the surrounding square brackets, if they exist
176+
stripped_keyvaults=$(echo "$KEYVAULTS" | sed 's/\[\|\]//g')
177+
178+
# Convert the comma-separated string into an array
179+
IFS=',' read -r -a resources_to_check <<< "$stripped_keyvaults"
180+
181+
# Append new resources to the array
182+
resources_to_check+=("${{ env.LOG_ANALYTICS_WORKSPACE_NAME }}")
183+
184+
echo "List of resources to check: ${resources_to_check[@]}"
185+
186+
# Maximum number of retries
187+
max_retries=3
188+
189+
# Retry intervals in seconds (30, 60, 120)
190+
retry_intervals=(30 60 120)
191+
192+
# Retry mechanism to check resources
193+
retries=0
194+
while true; do
195+
resource_found=false
196+
197+
# Get the list of resources in YAML format again on each retry
198+
resource_list=$(az resource list --resource-group ${{ env.RESOURCE_GROUP_NAME }} --output yaml)
199+
200+
# Iterate through the resources to check
201+
for resource in "${resources_to_check[@]}"; do
202+
echo "Checking resource: $resource"
203+
if echo "$resource_list" | grep -q "name: $resource"; then
204+
echo "Resource '$resource' exists in the resource group."
205+
resource_found=true
206+
else
207+
echo "Resource '$resource' does not exist in the resource group."
208+
fi
209+
done
210+
211+
# If any resource exists, retry
212+
if [ "$resource_found" = true ]; then
213+
retries=$((retries + 1))
214+
if [ "$retries" -gt "$max_retries" ]; then
215+
echo "Maximum retry attempts reached. Exiting."
216+
break
217+
else
218+
# Wait for the appropriate interval for the current retry
219+
echo "Waiting for ${retry_intervals[$retries-1]} seconds before retrying..."
220+
sleep ${retry_intervals[$retries-1]}
221+
fi
222+
else
223+
echo "No resources found. Exiting."
224+
break
225+
fi
226+
done
227+
228+
229+
- name: Purging the Resources
230+
if: success()
231+
run: |
232+
233+
set -e
234+
# List of keyvaults
235+
KEYVAULTS="${{ env.KEYVAULTS }}"
236+
237+
# Remove the surrounding square brackets, if they exist
238+
stripped_keyvaults=$(echo "$KEYVAULTS" | sed 's/\[\|\]//g')
239+
240+
# Convert the comma-separated string into an array
241+
IFS=',' read -r -a keyvault_array <<< "$stripped_keyvaults"
242+
243+
echo "Using KeyVaults Array..."
244+
for keyvault_name in "${keyvault_array[@]}"; do
245+
echo "Processing KeyVault: $keyvault_name"
246+
# Check if the KeyVault is soft-deleted
247+
deleted_vaults=$(az keyvault list-deleted --query "[?name=='$keyvault_name']" -o json --subscription ${{ secrets.AZURE_SUBSCRIPTION_ID }})
248+
249+
# If the KeyVault is found in the soft-deleted state, purge it
250+
if [ "$(echo "$deleted_vaults" | jq length)" -gt 0 ]; then
251+
echo "KeyVault '$keyvault_name' is soft-deleted. Proceeding to purge..."
252+
# Purge the KeyVault
253+
if az keyvault purge --name "$keyvault_name" --no-wait; then
254+
echo "Successfully purged KeyVault '$keyvault_name'."
255+
else
256+
echo "Failed to purge KeyVault '$keyvault_name'."
257+
fi
258+
else
259+
echo "KeyVault '$keyvault_name' is not soft-deleted. No action taken."
260+
fi
261+
done

‎README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ The sample data used in this repository is synthetic and generated using Azure O
6161

6262
This diagram double-clicks into the agentic framework for the code conversion process. The conversion uses an agentic approach with each agent playing a specialized role in the process. The system gets a list of SQL files which are targeted for conversion. 
6363

64-
**Step 1:** The system loops through the list of SQL files, converting each file, starting by passing the SQL to the Migrator agent. This agent will create several candidate SQL files that should be equivalent. It does this to ensure that the system acknowledges that most of these queries could be converted in a number of different ways. *Note that the processing time can vary depending on OpenAI and cloud services.*
64+
**Step 1:** The system loops through the list of SQL files, converting each file, starting by passing the SQL to the Migrator agent. This agent will create several candidate SQL files that should be equivalent. It does this to ensure that the system acknowledges that most of these queries could be converted in a number of different ways. *Note that the processing time can vary depending on Azure OpenAI service and cloud services.*
6565

6666
**Step 2:** The Picker agent then examines these various possibilities and picks the one it believes is best using criteria such as simplicity, clarity of syntax, etc.
6767

@@ -106,12 +106,12 @@ When you start the deployment, most parameters will have **default values**, but
106106
|------------|----------------| ------------|
107107
| **Azure Region** | The region where resources will be created. | East US|
108108
| **Resource Prefix** | Prefix for all resources created by this template. This prefix will be used to create unique names for all resources. The prefix must be unique within the resource group. | None |
109-
| **Ai Location** | Location for all Ai services resources. This location can be different from the resource group location | None |
110-
| **Capacity** | Configure capacity for **GPT models**. | 5k |
109+
| **AI Location** | Location for all AI services resources. This location can be different from the resource group location | None |
110+
| **Capacity** | Configure capacity for **gpt-4o**. | 5k |
111111

112112
### [Optional] Quota Recommendations
113113
By default, the **GPT model capacity** in deployment is set to **5k tokens**.
114-
> **We recommend increasing the capacity to 30k tokens for optimal performance.**
114+
> **We recommend increasing the capacity to 200k tokens for optimal performance.**
115115
116116
To adjust quota settings, follow these [steps](./docs/AzureGPTQuotaSettings.md)
117117

@@ -262,7 +262,7 @@ You can try the [Azure pricing calculator](https://azure.microsoft.com/en-us/pri
262262
* Azure AI Foundry: Free tier. [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/)
263263
* Azure Storage Account: Standard tier, LRS. Pricing is based on storage and operations. [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/)
264264
* Azure Key Vault: Standard tier. Pricing is based on the number of operations. [Pricing](https://azure.microsoft.com/pricing/details/key-vault/)
265-
* Azure AI Services: S0 tier, defaults to gpt-4o-mini and text-embedding-ada-002 models. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/)
265+
* Azure AI Services: S0 tier, defaults to gpt-4o-mini. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/)
266266
* Azure Container App: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. [Pricing](https://azure.microsoft.com/pricing/details/container-apps/)
267267
* Azure Container Registry: Basic tier. [Pricing](https://azure.microsoft.com/pricing/details/container-registry/)
268268
* Log analytics: Pay-as-you-go tier. Costs based on data ingested. [Pricing](https://azure.microsoft.com/pricing/details/monitor/)

‎docs/AzureGPTQuotaSettings.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
## How to Check & Update Quota
2+
3+
1. **Navigate** to the [Azure AI Foundry portal](https://ai.azure.com/).
4+
2. **Select** the AI Project associated with this accelerator.
5+
3. **Go to** the `Management Center` from the bottom-left navigation menu.
6+
4. Select `Quota`
7+
- Click on the `GlobalStandard` dropdown.
8+
- Select the required **GPT model** (`GPT-4, GPT-4o`).
9+
- Choose the **region** where the deployment is hosted.
10+
5. Request More Quota or delete any unused model deployments as needed.

‎docs/CustomizingAzdParameters.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
By default this template will use the environment name as the prefix to prevent naming collisions within Azure. The parameters below show the default values. You only need to run the statements below if you need to change the values.
2+
3+
4+
> To override any of the parameters, run `azd env set <key> <value>` before running `azd up`. On the first azd command, it will prompt you for the environment name. Be sure to choose 3-20 charaters alphanumeric unique name.
5+
6+
Change the Model Deployment Type (allowed values: Standard, GlobalStandard)
7+
8+
```shell
9+
azd env set AZURE_ENV_MODEL_DEPLOYMENT_TYPE Standard
10+
```
11+
12+
Set the Model Name (allowed values: gpt-4)
13+
14+
```shell
15+
azd env set AZURE_ENV_MODEL_NAME gpt-4
16+
```
17+
18+
Change the Model Capacity (choose a number based on available GPT model capacity in your subscription)
19+
20+
```shell
21+
azd env set AZURE_ENV_MODEL_CAPACITY 30
22+
```
23+

0 commit comments

Comments
 (0)