diff --git a/README.md b/README.md index 90018fe7..78f922d8 100644 --- a/README.md +++ b/README.md @@ -1,295 +1,213 @@ # Modernize your code solution accelerator -MENU: [**USER STORY**](#user-story) \| [**QUICK DEPLOY**](#quick-deploy) \| [**SUPPORTING DOCUMENTATION**](#supporting-documentation) - -

-
-User story -

- -### Overview - Welcome to the *Modernize your code* solution accelerator, designed to help customers transition their SQL queries to new environments quickly and efficiently. This accelerator is particularly useful for organizations modernizing their data estates, as it simplifies the process of translating SQL queries from various dialects. When dealing with legacy code, users often face significant challenges, including the absence of proper documentation, loss of knowledge of outdated languages, and missing business logic that explains functional requirements. -The *Modernize your code* solution accelerator allows users to specify a group of SQL queries and the target SQL dialect for translation. It then initiates a batch process where each query is translated using a group of Large Language Model (LLM) agents. This automation not only saves time but also ensures accuracy and consistency in query translation. - -### Technical Key features - - - - - KeyFeatures - - -
-
- -Below is an image of the solution accelerator: - -image - -
- -### Use case / scenario - -Companies maintaining and modernizing their data estates often face large migration projects. They may have volumes of files in various dialects, which need to be translated into a modern alternative. Some of the challenges they face include: +The Modernize your code solution accelerator allows users to specify a group of SQL queries and the target SQL dialect for translation. It then initiates a batch process where each query is translated using a group of Large Language Model (LLM) agents. This automation not only saves time but also ensures accuracy and consistency in query translation. +
- +
+ +[**SOLUTION OVERVIEW**](#solution-overview) \| [**QUICK DEPLOY**](#quick-deploy) \| [**BUSINESS SCENARIO**](#business-scenario) \| [**SUPPORTING DOCUMENTATION**](#supporting-documentation) -By using the *Modernize your code* solution accelerator, users can automate this process, ensuring that all queries are accurately translated and ready for use in the new modern environment. +
+
-For an in-depth look at the applicability of using multiple agents for this code modernization use case, please see the [supporting AI Research paper](./documentation/modernize_report.pdf). +

+Solution overview +

-The sample data used in this repository is synthetic and generated using Azure Open AI service. The data is intended for use as sample data only. +The solution leverages Azure AI Foundry, Azure OpenAI Service, Azure Container Apps, Azure Cosmos DB, and Azure Storage to create an intelligent code modernization pipeline. It uses a multi-agent approach where specialized AI agents work together to translate, validate, and optimize SQL queries for the target environment. ### Solution architecture +|![image](./docs/images/read_me/solArchitecture.png)| +|---| -image +### Agentic architecture +|![image](./docs/images/read_me/agentArchitecture.png)| +|---| -
+### How to customize +If you'd like to customize the solution accelerator, here are some common areas to start: -### Agentic architecture +[Custom scenario](./docs/CustomizingScenario.md) -image
-This diagram double-clicks into the agentic framework for the code conversion process. The conversion uses an agentic approach with each agent playing a specialized role in the process. The system gets a list of SQL files which are targeted for conversion.  - -**Step 1:** The system loops through the list of SQL files, converting each file, starting by passing the SQL to the Migrator agent. This agent will create several candidate SQL files that should be equivalent. It does this to ensure that the system acknowledges that most of these queries could be converted in a number of different ways. *Note that the processing time can vary depending on Azure OpenAI service and cloud services.* +### Additional resources -**Step 2:** The Picker agent then examines these various possibilities and picks the one it believes is best using criteria such as simplicity, clarity of syntax, etc. +[Azure AI Foundry documentation](https://learn.microsoft.com/en-us/azure/ai-studio/) -**Step 3:** This query is sent to the Syntax checker agent which, using a command line tool designed to validate SQL syntax, checks to make sure the query should run without error. -- **Step 3n:** If the Syntax checker agent finds potential errors, it then in Step 3n sends the query to a Fixer agent which will attempt to fix the problem. The Fixer agent then sends the fixed query back to the Syntax checker agent again. If there are still errors, the Syntax checker agent sends back to the Fixer agent to make another attempt. This iteration continues until, either there are no errors found, or a max number of allowed iterations is reached. If the max number is hit, error logs are generated for that query and stored in its Cosmos DB metadata.  +[Semantic Kernel Agent Framework](https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/?pivots=programming-language-python) -**Step 4:** Once the SQL is found to run without errors, it is sent for a final check to the Semantic checker agent. This agent makes sure that the query in the new syntax will have the same logical effects as the old query, with no extra effects. It can find edge cases which don’t apply to most scenarios, so, if it finds an issue, this issue is sent to the query logs, and the query is generated and the file will be present in storage, but its state will be listed as “warning”.  If no semantic issues are found, the query is generated and placed into Azure storage with a state of success. +[Azure OpenAI Service Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data) -


-QUICK DEPLOY -

-### **Prerequisites** +### Key features +
+ Click to learn more about the key features this solution enables -To deploy this solution accelerator, ensure you have access to an [Azure subscription](https://azure.microsoft.com/free/) with the necessary permissions to create **resource groups and resources**. Follow the steps in [Azure Account Set Up](./docs/AzureAccountSetUp.md) + - **Code language modernization**
+ Modernizing outdated code ensures compatibility with current technologies, reduces reliance on legacy expertise, and keeps businesses competitive. + + - **Summary and review of new code**
+ Generating summaries and translating code files keeps humans in the loop, enhances their understanding, and facilitates timely interventions, ensuring the files are ready to export. -Check the [Azure Products by Region](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=all®ions=all) page and select a **region** where the following services are available: + - **Business logic analysis**
+ Leveraging AI to decipher business logic from legacy code helps minimizes the risk of human error. -- Azure AI Foundry -- Azure OpenAI Service -- GPT Model Capacity + - **Efficient code transformation**
+ Streamlining the process of analyzing, converting, and iterative error testing reduces time and effort required to modernize the systems. -Here are some example regions where the services are available: East US, East US2, Japan East, UK South, Sweden Central. +
-### ⚠️ Important: Check Azure OpenAI Quota Availability +

+

+Quick deploy +

+ +### How to install or deploy +Follow the quick deploy steps on the deployment guide to deploy this solution to your own Azure subscription. -➡️ To ensure sufficient quota is available in your subscription, please follow **[Quota check instructions guide](./docs/quota_check.md)** before you deploy the solution. +[Click here to launch the deployment guide](./docs/DeploymentGuide.md) +

| [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Modernize-your-Code-Solution-Accelerator) | [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Modernize-your-Code-Solution-Accelerator) | |---|---| -### **Configurable Deployment Settings** - -When you start the deployment, most parameters will have **default values**, but you can update the following settings: +
-| **Setting** | **Description** | **Default value** | -|------------|----------------| ------------| -| **Azure Region** | The region where resources will be created. | East US| -| **Resource Prefix** | Prefix for all resources created by this template. This prefix will be used to create unique names for all resources. The prefix must be unique within the resource group. | None | -| **AI Location** | Location for all AI services resources. This location can be different from the resource group location | None | -| **Capacity** | Configure capacity for **gpt-4o**. | 5k | +> ⚠️ **Important: Check Azure OpenAI Quota Availability** +
To ensure sufficient quota is available in your subscription, please follow [quota check instructions guide](./docs/quota_check.md) before you deploy the solution. -This accelerator can be configured to use authentication. +
-* To use authentication the installer must have the rights to create and register an application identity in their Azure environment. -After installation is complete, follow the directions in the [App Authentication](./docs/AddAuthentication.md) document to enable authentication. -* Note: If you enable authentication, all processing history and current processing will be performed for your specific user. Without authentication, all batch history from the tool will be visible to all users. +### Prerequisites and Costs -### [Optional] Quota Recommendations -By default, the **GPT model capacity** in deployment is set to **5k tokens**. -> **We recommend increasing the capacity to 200k tokens for optimal performance.** +To deploy this solution accelerator, ensure you have access to an [Azure subscription](https://azure.microsoft.com/free/) with the necessary permissions to create **resource groups, resources, app registrations, and assign roles at the resource group level**. This should include Contributor role at the subscription level and Role Based Access Control role on the subscription and/or resource group level. Follow the steps in [Azure Account Set Up](./docs/AzureAccountSetUp.md). -To adjust quota settings, follow these [steps](./docs/AzureGPTQuotaSettings.md) +Check the [Azure Products by Region](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=all®ions=all) page and select a **region** where the following services are available: Azure AI Foundry, Azure OpenAI Service, and GPT Model Capacity. -### Deployment Options -Pick from the options below to see step-by-step instructions for: GitHub Codespaces, VS Code Dev Containers, Local Environments, and Bicep deployments. +Here are some example regions where the services are available: East US, East US2, Japan East, UK South, Sweden Central. -
- Deploy in GitHub Codespaces +Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. However, Azure Container Registry has a fixed cost per registry per day. -### GitHub Codespaces +Use the [Azure pricing calculator](https://azure.microsoft.com/en-us/pricing/calculator) to calculate the cost of this solution in your subscription. -You can run this solution using GitHub Codespaces. The button will open a web-based VS Code instance in your browser: +| Product | Description | Cost | +|---|---|---| +| [Azure AI Foundry](https://learn.microsoft.com/azure/ai-studio/) | Used for AI agent orchestration and management | [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/) | +| [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) | Powers the AI agents for code translation | [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) | +| [Azure Container Apps](https://learn.microsoft.com/azure/container-apps/) | Hosts the web application frontend | [Pricing](https://azure.microsoft.com/pricing/details/container-apps/) | +| [Azure Cosmos DB](https://learn.microsoft.com/azure/cosmos-db/) | Stores metadata and processing results | [Pricing](https://azure.microsoft.com/pricing/details/cosmos-db/) | +| [Azure Storage Account](https://learn.microsoft.com/azure/storage/) | Stores SQL files and processing artifacts | [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/) | +| [Azure Container Registry](https://learn.microsoft.com/azure/container-registry/) | Stores container images for deployment | [Pricing](https://azure.microsoft.com/pricing/details/container-registry/) | -1. Open the solution accelerator (this may take several minutes): +
- [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Modernize-your-Code-Solution-Accelerator) -2. Accept the default values on the create Codespaces page -3. Open a terminal window if it is not already open -4. Continue with the [deploying steps](#deploying) +>⚠️ **Important:** To avoid unnecessary costs, remember to take down your app if it's no longer in use, +either by deleting the resource group in the Portal or running `azd down`. -
+

+

+Business Scenario +

-
- Deploy in VS Code +|![image](./docs/images/read_me/webappHero.png)| +|---| - ### VS Code Dev Containers +
-You can run this solution in VS Code Dev Containers, which will open the project in your local VS Code using the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers): +Companies maintaining and modernizing their data estates often face large migration projects. They may have volumes of files in various dialects, which need to be translated into a modern alternative. Some of the challenges they face include: -1. Start Docker Desktop (install it if not already installed) -2. Open the project: +- Difficulty analyzing and maintaining legacy systems due to missing documentation +- Time-consuming process to manually update legacy code and extract missing business logic +- High risk of errors from manual translations, which can lead to incorrect query results and data integrity issues +- Lack of available knowledge and expertise for legacy languages creates additional effort, cost, and reliance on niche skills - [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Modernize-your-Code-Solution-Accelerator) +By using the *Modernize your code* solution accelerator, users can automate this process, ensuring that all queries are accurately translated and ready for use in the new modern environment. +For an in-depth look at the applicability of using multiple agents for this code modernization use case, please see the [supporting AI Research paper](./docs/modernize_report.pdf). -3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window. -4. Continue with the [deploying steps](#deploying) +The sample data used in this repository is synthetic and generated using Azure Open AI service. The data is intended for use as sample data only. -
+⚠️ The sample data used in this repository is synthetic and generated using Azure OpenAI service. The data is intended for use as sample data only. +### Business value
- Deploy in your local environment - - ### Local environment + Click to learn more about what value this solution provides -If you're not using one of the above options for opening the project, then you'll need to: + - **Accelerated Migration**
+ Automate the translation of SQL queries, significantly reducing migration time and effort. -1. Make sure the following tools are installed: + - **Error Reduction**
+ Multi-agent validation ensures accurate translations and maintains data integrity. - * [Azure Developer CLI (azd)](https://aka.ms/install-azd) - * [Python 3.9+](https://www.python.org/downloads/) - * [Docker Desktop](https://www.docker.com/products/docker-desktop/) - * [Git](https://git-scm.com/downloads) + - **Knowledge Preservation**
+ Captures and preserves business logic during the modernization process. -2. Download the project code: + - **Cost Efficiency**
+ Reduces reliance on specialized legacy system expertise and manual translation efforts. - ```shell - azd init -t microsoft/Modernize-your-Code-Solution-Accelerator/ - ``` - -3. Open the project folder in your terminal or editor. - -4. Continue with the [deploying steps](#deploying). + - **Standardization**
+ Ensures consistent query translation across the organization.
-### Deploying - -Once you've opened the project in [Codespaces](#github-codespaces) or in [Dev Containers](#vs-code-dev-containers) or [locally](#local-environment), you can deploy it to Azure following the following steps. - -To change the azd parameters from the default values, follow the steps [here](./docs/CustomizingAzdParameters.md). - - -1. Login to Azure: - - ```shell - azd auth login - ``` - - #### Note: To authenticate with Azure Developer CLI (`azd`) to a specific tenant, use the previous command with your **Tenant ID**: +

- ```sh - azd auth login --tenant-id - ``` - -2. Provision and deploy all the resources: - - ```shell - azd up - ``` - -3. Provide an `azd` environment name (like "cmsaapp") -4. Select a subscription from your Azure account, and select a location which has quota for all the resources. - * This deployment will take *6-9 minutes* to provision the resources in your account and set up the solution with sample data. - * If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the resources. - -5. Once the deployment has completed successfully, open the [Azure Portal](https://portal.azure.com/), go to the deployed resource group, find the container app with "frontend" in the name, and get the app URL from `Application URI`. - -6. You can now delete the resources by running `azd down`, when you have finished trying out the application. - -

-Additional Steps +

+Supporting documentation

-1. **Deleting Resources After a Failed Deployment** - - Follow steps in [Delete Resource Group](./docs/DeleteResourceGroup.md) If your deployment fails and you need to clean up the resources. - -1. **Add App Authentication** - - If you chose to enable authentication for the deployment, follow the steps in [App Authentication](./docs/AddAuthentication.md) - -## Running the application - -To help you get started, sample Informix queries have been included in the data/informix/functions and data/informix/simple directories. You can choose to upload these files to test the application. - -

-Responsible AI Transparency FAQ -

- -Please refer to [Transparency FAQ](./TRANSPARENCY_FAQ.md) for responsible AI transparency details of this solution accelerator. - -

-Supporting Documentation -

+### Security guidelines -### Costs +This template uses Azure Key Vault for use by AI Foundry. -Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. -The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. -However, Azure Container Registry has a fixed cost per registry per day. +This template uses [Managed Identity](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/overview) for all Azure service communication. -You can try the [Azure pricing calculator](https://azure.microsoft.com/en-us/pricing/calculator) for the resources: +To ensure continued best practices in your own repository, we recommend that anyone creating solutions based on our templates ensure that the [Github secret scanning](https://docs.github.com/code-security/secret-scanning/about-secret-scanning) setting is enabled. -* Azure AI Foundry: Free tier. [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/) -* Azure Storage Account: Standard tier, LRS. Pricing is based on storage and operations. [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/) -* Azure Key Vault: Standard tier. Pricing is based on the number of operations. [Pricing](https://azure.microsoft.com/pricing/details/key-vault/) -* Azure AI Services: S0 tier, defaults to gpt-4o-mini. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/) -* Azure Container App: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. [Pricing](https://azure.microsoft.com/pricing/details/container-apps/) -* Azure Container Registry: Basic tier. [Pricing](https://azure.microsoft.com/pricing/details/container-registry/) -* Log analytics: Pay-as-you-go tier. Costs based on data ingested. [Pricing](https://azure.microsoft.com/pricing/details/monitor/) -* Azure Cosmos DB: [Pricing](https://azure.microsoft.com/en-us/pricing/details/cosmos-db/autoscale-provisioned/) +You may want to consider additional security measures, such as: -⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, -either by deleting the resource group in the Portal or running `azd down`. +* Enabling Microsoft Defender for Cloud to [secure your Azure resources](https://learn.microsoft.com/en-us/azure/defender-for-cloud/). +* Protecting the Azure Container Apps instance with a [firewall](https://learn.microsoft.com/azure/container-apps/waf-app-gateway) and/or [Virtual Network](https://learn.microsoft.com/azure/container-apps/networking?tabs=workload-profiles-env%2Cazure-cli). -### Security guidelines +
-This installs Azure Key Vault for use by AI Foundry. +### Cross references +Check out similar solution accelerators -This template uses [Managed Identity](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/overview) for all Azure service communication. +| Solution Accelerator | Description | +|---|---| +| [Documen Knowledge Mining](https://github.com/microsoft/Document-Knowledge-Mining-Solution-Accelerator) | Extract structured information from unstructured documents using AI | +| [Multi Agent Custom Automation Engine Solution Acceleratorr](https://github.com/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator/tree/main) | An AI-driven orchestration system that manages a group of AI agents to accomplish tasks based on user input | +| [Conversation Knowledge Mining](https://github.com/microsoft/Conversation-Knowledge-Mining-Solution-Accelerator) | Enable organizations to derive insights from volumes of conversational data using generative AI | -To ensure continued best practices in your own repository, we recommend that anyone creating solutions based on our templates ensure that the [Github secret scanning](https://docs.github.com/code-security/secret-scanning/about-secret-scanning) setting is enabled. +
-You may want to consider additional security measures, such as: +## Provide feedback -* Enabling Microsoft Defender for Cloud to [secure your Azure resources](https://learn.microsoft.com/azure/security-center/defender-for-cloud). -* Protecting the Azure Container Apps instance with a [firewall](https://learn.microsoft.com/azure/container-apps/waf-app-gateway) and/or [Virtual Network](https://learn.microsoft.com/azure/container-apps/networking?tabs=workload-profiles-env%2Cazure-cli). +Have questions, find a bug, or want to request a feature? [Submit a new issue](https://github.com/microsoft/Modernize-your-Code-Solution-Accelerator/issues) on this repo and we'll connect. -**Additional resources** +
-- [Azure AI Foundry documentation](https://learn.microsoft.com/en-us/azure/ai-studio/) -- [Semantic Kernel Agent Framework](https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/?pivots=programming-language-python) -- [Azure Cosmos DB Documentation](https://learn.microsoft.com/en-us/azure/cosmos-db/) -- [Azure OpenAI Service - Documentation, quickstarts, API reference - Azure AI services | Microsoft Learn](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data) -- [Azure Container Apps documentation](https://learn.microsoft.com/en-us/azure/container-apps/) +## Responsible AI Transparency FAQ +Please refer to [Transparency FAQ](./TRANSPARENCY_FAQS.md) for responsible AI transparency details of this solution accelerator. +
## Disclaimers -To the extent that the Software includes components or code used in or derived from Microsoft products or services, including without limitation Microsoft Azure Services (collectively, “Microsoft Products and Services”), you must also comply with the Product Terms applicable to such Microsoft Products and Services. You acknowledge and agree that the license governing the Software does not grant you a license or other right to use Microsoft Products and Services. Nothing in the license or this ReadMe file will serve to supersede, amend, terminate or modify any terms in the Product Terms for any Microsoft Products and Services. +To the extent that the Software includes components or code used in or derived from Microsoft products or services, including without limitation Microsoft Azure Services (collectively, "Microsoft Products and Services"), you must also comply with the Product Terms applicable to such Microsoft Products and Services. You acknowledge and agree that the license governing the Software does not grant you a license or other right to use Microsoft Products and Services. Nothing in the license or this ReadMe file will serve to supersede, amend, terminate or modify any terms in the Product Terms for any Microsoft Products and Services. You must also comply with all domestic and international export laws and regulations that apply to the Software, which include restrictions on destinations, end users, and end use. For further information on export restrictions, visit https://aka.ms/exporting. -You acknowledge that the Software and Microsoft Products and Services (1) are not designed, intended or made available as a medical device(s), and (2) are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment. Customer is solely responsible for displaying and/or obtaining appropriate consents, warnings, disclaimers, and acknowledgements to end users of Customer’s implementation of the Online Services. +You acknowledge that the Software and Microsoft Products and Services (1) are not designed, intended or made available as a medical device(s), and (2) are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment. Customer is solely responsible for displaying and/or obtaining appropriate consents, warnings, disclaimers, and acknowledgements to end users of Customer's implementation of the Online Services. You acknowledge the Software is not subject to SOC 1 and SOC 2 compliance audits. No Microsoft technology, nor any of its component technologies, including the Software, is intended or made available as a substitute for the professional advice, opinion, or judgement of a certified financial services professional. Do not use the Software to replace, substitute, or provide professional financial advice or judgment. -BY ACCESSING OR USING THE SOFTWARE, YOU ACKNOWLEDGE THAT THE SOFTWARE IS NOT DESIGNED OR INTENDED TO SUPPORT ANY USE IN WHICH A SERVICE INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE COULD RESULT IN THE DEATH OR SERIOUS BODILY INJURY OF ANY PERSON OR IN PHYSICAL OR ENVIRONMENTAL DAMAGE (COLLECTIVELY, “HIGH-RISK USE”), AND THAT YOU WILL ENSURE THAT, IN THE EVENT OF ANY INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE, THE SAFETY OF PEOPLE, PROPERTY, AND THE ENVIRONMENT ARE NOT REDUCED BELOW A LEVEL THAT IS REASONABLY, APPROPRIATE, AND LEGAL, WHETHER IN GENERAL OR IN A SPECIFIC INDUSTRY. BY ACCESSING THE SOFTWARE, YOU FURTHER ACKNOWLEDGE THAT YOUR HIGH-RISK USE OF THE SOFTWARE IS AT YOUR OWN RISK. +BY ACCESSING OR USING THE SOFTWARE, YOU ACKNOWLEDGE THAT THE SOFTWARE IS NOT DESIGNED OR INTENDED TO SUPPORT ANY USE IN WHICH A SERVICE INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE COULD RESULT IN THE DEATH OR SERIOUS BODILY INJURY OF ANY PERSON OR IN PHYSICAL OR ENVIRONMENTAL DAMAGE (COLLECTIVELY, "HIGH-RISK USE"), AND THAT YOU WILL ENSURE THAT, IN THE EVENT OF ANY INTERRUPTION, DEFECT, ERROR, OR OTHER FAILURE OF THE SOFTWARE, THE SAFETY OF PEOPLE, PROPERTY, AND THE ENVIRONMENT ARE NOT REDUCED BELOW A LEVEL THAT IS REASONABLY, APPROPRIATE, AND LEGAL, WHETHER IN GENERAL OR IN A SPECIFIC INDUSTRY. BY ACCESSING THE SOFTWARE, YOU FURTHER ACKNOWLEDGE THAT YOUR HIGH-RISK USE OF THE SOFTWARE IS AT YOUR OWN RISK. \ No newline at end of file diff --git a/docs/CustomizingAzdParameters.md b/docs/CustomizingAzdParameters.md index d8200cbf..ee72294c 100644 --- a/docs/CustomizingAzdParameters.md +++ b/docs/CustomizingAzdParameters.md @@ -1,7 +1,9 @@ +## [Optional]: Customizing resource names + By default this template will use the environment name as the prefix to prevent naming collisions within Azure. The parameters below show the default values. You only need to run the statements below if you need to change the values. -> To override any of the parameters, run `azd env set ` before running `azd up`. On the first azd command, it will prompt you for the environment name. Be sure to choose 3-20 charaters alphanumeric unique name. +> To override any of the parameters, run `azd env set ` before running `azd up`. On the first azd command, it will prompt you for the environment name. Be sure to choose 3-20 characters alphanumeric unique name. Change the Model Deployment Type (allowed values: Standard, GlobalStandard) diff --git a/docs/CustomizingScenario.md b/docs/CustomizingScenario.md new file mode 100644 index 00000000..bc7c69f4 --- /dev/null +++ b/docs/CustomizingScenario.md @@ -0,0 +1,15 @@ +## [Optional]: Customizing scenario + +This template pattern can be used for other types of conversions requiring the same or similar agent workflows. This document provides a suggested path to modifying the template to support a new scenario - for example an infrastructure as code template conversion. Generally the API backend is modified with the API used to support a new user experience / UI. This document will focus on necessary backend changes. + +The first step is to determine the overall architecture for the system, how the agents will interact, and details regarding the step by step architecture. If the conversion needs to be validated by a tool or tested in an environment, full details on how to configure and run this are also necessary. After this, follow the steps below to quickly create a proof of concept for the new system. + +1. Copy the agent workflow folder (sql_agents) into a new sibling folder within src/backend and name it as appropriate to your scenario +1. Modify the agent folder and file names as appropriate to support new agent types +1. Modify the agent response class to represent the structured response needed from the agent +1. Modify the agents prompting in the associated prompt.txt file. Note that changing the conversion inputs and outputs will also require changes to agent_config.py as well as src/backend/api/api_routes in the definition of start-processing. +1. If workflow modification is necessary, those changes would take place in the src/backend/sql_agents/helper/comms_manager.py file as well as the src/backend/sql_agents/convert_script.py file. +1. There are two primary ways of messaging state changes to the front end. The first results from state storage in Cosmos. This is updated primarily in the convert_script.py file with the creation of file logs. The second is for transitory state changes that are communicated through websockets to the client. These are also primarily in the convert_script.py file. +1. Create a function to validate conversions using a test environment or utility. Provide this function to an agent to perform the validation role and iterate with another agent which can attempt to fix any issues. You can follow the plug in example within the current Syntax checker agent. + +Agent code in src/backend/agents including agent_base.py, agent_factory.py, and agent_config.py is designed to be largely reused in any scenario. Code in sql_agents/helpers is aso designed for reuse. \ No newline at end of file diff --git a/docs/DeploymentGuide.md b/docs/DeploymentGuide.md new file mode 100644 index 00000000..8c592880 --- /dev/null +++ b/docs/DeploymentGuide.md @@ -0,0 +1,156 @@ +## **Deployment Guide** + +### **Pre-requisites** + +To deploy this solution accelerator, ensure you have access to an [Azure subscription](https://azure.microsoft.com/free/) with the necessary permissions to create **resource groups and resources**. Follow the steps in [Azure Account Set Up](./docs/AzureAccountSetUp.md) + +Check the [Azure Products by Region](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=all®ions=all) page and select a **region** where the following services are available: + +- Azure AI Foundry +- Azure OpenAI Service +- GPT Model Capacity + +Here are some example regions where the services are available: East US, East US2, Japan East, UK South, Sweden Central. + +### ⚠️ Important: Check Azure OpenAI Quota Availability + +➡️ To ensure sufficient quota is available in your subscription, please follow **[Quota check instructions guide](../docs/quota_check.md)** before you deploy the solution. + +| [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Modernize-your-Code-Solution-Accelerator) | [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Modernize-your-Code-Solution-Accelerator) | +|---|---| + +### **Configurable Deployment Settings** + +When you start the deployment, most parameters will have **default values**, but you can update the following settings by following the steps [here](../docs/CustomizingAzdParameters.md): + +| **Setting** | **Description** | **Default value** | +|------------|----------------| ------------| +| **Azure Region** | The region where resources will be created. | East US| +| **Resource Prefix** | Prefix for all resources created by this template. This prefix will be used to create unique names for all resources. The prefix must be unique within the resource group. | None | +| **AI Location** | Location for all AI services resources. This location can be different from the resource group location | None | +| **Capacity** | Configure capacity for **gpt-4o**. | 5k | + +This accelerator can be configured to use authentication. + +* To use authentication the installer must have the rights to create and register an application identity in their Azure environment. +After installation is complete, follow the directions in the [App Authentication](../docs/AddAuthentication.md) document to enable authentication. +* Note: If you enable authentication, all processing history and current processing will be performed for your specific user. Without authentication, all batch history from the tool will be visible to all users. + +### [Optional] Quota Recommendations +By default, the **GPT model capacity** in deployment is set to **5k tokens**. +> **We recommend increasing the capacity to 200k tokens for optimal performance.** + +To adjust quota settings, follow these [steps](../docs/AzureGPTQuotaSettings.md) + +### Deployment Options +Pick from the options below to see step-by-step instructions for: GitHub Codespaces, VS Code Dev Containers, Local Environments, and Bicep deployments. + +
+ Deploy in GitHub Codespaces + +### GitHub Codespaces + +You can run this solution using GitHub Codespaces. The button will open a web-based VS Code instance in your browser: + +1. Open the solution accelerator (this may take several minutes): + + [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/Modernize-your-Code-Solution-Accelerator) +2. Accept the default values on the create Codespaces page +3. Open a terminal window if it is not already open +4. Continue with the [deploying steps](#deploying) + +
+ +
+ Deploy in VS Code + + ### VS Code Dev Containers + +You can run this solution in VS Code Dev Containers, which will open the project in your local VS Code using the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers): + +1. Start Docker Desktop (install it if not already installed) +2. Open the project: + + [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/Modernize-your-Code-Solution-Accelerator) + + +3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window. +4. Continue with the [deploying steps](#deploying) + +
+ +
+ Deploy in your local environment + + ### Local environment + +If you're not using one of the above options for opening the project, then you'll need to: + +1. Make sure the following tools are installed: + + * [Azure Developer CLI (azd)](https://aka.ms/install-azd) + * [Python 3.9+](https://www.python.org/downloads/) + * [Docker Desktop](https://www.docker.com/products/docker-desktop/) + * [Git](https://git-scm.com/downloads) + +2. Download the project code: + + ```shell + azd init -t microsoft/Modernize-your-Code-Solution-Accelerator/ + ``` + +3. Open the project folder in your terminal or editor. + +4. Continue with the [deploying steps](#deploying). + +
+ +### Deploying + +Once you've opened the project in [Codespaces](#github-codespaces) or in [Dev Containers](#vs-code-dev-containers) or [locally](#local-environment), you can deploy it to Azure following the following steps. + +To change the azd parameters from the default values, follow the steps [here](../docs/CustomizingAzdParameters.md). + + +1. Login to Azure: + + ```shell + azd auth login + ``` + + #### Note: To authenticate with Azure Developer CLI (`azd`) to a specific tenant, use the previous command with your **Tenant ID**: + + ```sh + azd auth login --tenant-id + ``` + +2. Provision and deploy all the resources: + + ```shell + azd up + ``` + +3. Provide an `azd` environment name (like "cmsaapp") +4. Select a subscription from your Azure account, and select a location which has quota for all the resources. + * This deployment will take *6-9 minutes* to provision the resources in your account and set up the solution with sample data. + * If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the resources. + +5. Once the deployment has completed successfully, open the [Azure Portal](https://portal.azure.com/), go to the deployed resource group, find the container app with "frontend" in the name, and get the app URL from `Application URI`. + +6. You can now delete the resources by running `azd down`, when you have finished trying out the application. + +

+Additional Steps +

+ +1. **Deleting Resources After a Failed Deployment** + + Follow steps in [Delete Resource Group](../docs/DeleteResourceGroup.md) If your deployment fails and you need to clean up the resources. + +1. **Add App Authentication** + + If you chose to enable authentication for the deployment, follow the steps in [App Authentication](../docs/AddAuthentication.md) + +## Running the application + +To help you get started, sample Informix queries have been included in the data/informix/functions and data/informix/simple directories. You can choose to upload these files to test the application. \ No newline at end of file diff --git a/docs/images/read_me/business-scenario.png b/docs/images/read_me/business-scenario.png new file mode 100644 index 00000000..017032cc Binary files /dev/null and b/docs/images/read_me/business-scenario.png differ diff --git a/docs/images/read_me/quick-deploy.png b/docs/images/read_me/quick-deploy.png new file mode 100644 index 00000000..421c0c1f Binary files /dev/null and b/docs/images/read_me/quick-deploy.png differ diff --git a/docs/images/read_me/solution-overview.png b/docs/images/read_me/solution-overview.png new file mode 100644 index 00000000..483dbfcd Binary files /dev/null and b/docs/images/read_me/solution-overview.png differ diff --git a/docs/images/read_me/supporting-documentation.png b/docs/images/read_me/supporting-documentation.png new file mode 100644 index 00000000..b498805c Binary files /dev/null and b/docs/images/read_me/supporting-documentation.png differ