Skip to content

Commit f3d7473

Browse files
brown9804brown9804
authored
Merge aa64f2d into d3517ab 
2 parents d3517ab + aa64f2d commit f3d7473

3 files changed

Lines changed: 358 additions & 0 deletions

File tree

  • 0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/0_serverlessSQLPool/README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Serverless SQL Pool: Dynamically Remove Space
2+
3+
Costa Rica
4+
5+
[![GitHub](https://badgen.net/badge/icon/github?icon=github&label)](https://github.com)
6+
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
7+
[brown9804](https://github.com/brown9804)
8+
9+
Last updated: 2025-01-24
10+
11+
----------
12+
13+
## Wiki
14+
15+
<details>
16+
<summary><b>List of References </b> (Click to expand)</summary>
17+
18+
19+
</details>
20+
21+
## Content
22+
23+
<details>
24+
<summary><b>Table of Content </b> (Click to expand)</summary>
25+
26+
27+
</details>
28+
29+
30+
<div align="center">
31+
<h3 style="color: #4CAF50;">Total Visitors</h3>
32+
<img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
33+
</div>

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/1_dedicatedSQLPoolStoreProc/README.md‎

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
# Dedicated SQL Pool: Store Procedure Dynamically Remove Space
2+
3+
Costa Rica
4+
5+
[![GitHub](https://badgen.net/badge/icon/github?icon=github&label)](https://github.com)
6+
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
7+
[brown9804](https://github.com/brown9804)
8+
9+
Last updated: 2025-01-24
10+
11+
----------
12+
13+
## Content
14+
15+
<details>
16+
<summary><b>Table of Content </b> (Click to expand)</summary>
17+
18+
- [Content](#content)
19+
- [Demo](#demo)
20+
- [Set Up a Dedicated SQL Pool](#set-up-a-dedicated-sql-pool)
21+
- [Create a Dedicated SQL Pool](#create-a-dedicated-sql-pool)
22+
- [Create Tables with Spaces in Names and Columns](#create-tables-with-spaces-in-names-and-columns)
23+
- [Create Views with Modified Tables/Column Names](#create-views-with-modified-tablescolumn-names)
24+
25+
</details>
26+
27+
## Demo
28+
29+
### Set Up a Dedicated SQL Pool
30+
31+
1. **Sign in to the Azure Portal**: Go to the Azure Portal and sign in with your Azure account.
32+
2. **Navigate to Your Synapse Workspace**: In the Azure Portal, search for your Synapse workspace or create a new one if you don't have one.
33+
34+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/92a5e451-1868-47e2-b32b-858591c306ee" />
35+
36+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/51e3b091-855d-4481-89e0-623705e3cf2a" />
37+
38+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/7d03bfa8-e1e3-4706-970f-a89c7b8cd904" />
39+
40+
### Create a Dedicated SQL Pool
41+
42+
1. **Launch Synapse Studio**: From the Synapse workspace overview, click on the `Open Synapse Studio` button.
43+
44+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/302b1fd8-49a6-427e-93dc-8e952f1667e6" />
45+
46+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/a368036b-c859-47fc-a1c5-d045b6910790" />
47+
48+
2. **Create a Dedicated SQL Pool**:
49+
- In Synapse Studio, go to the `Manage` hub by clicking on the `Manage` icon in the left navigation pane.
50+
- Under `Analytics pools`, select `SQL pools` and click on the `+ New` button.
51+
52+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/6d96ac07-57f1-4efd-917b-200a43091311" />
53+
54+
- Enter the following details:
55+
- **SQL pool name**: Enter a name for your SQL pool (e.g., `SQLPOOL1`).
56+
- **Performance level**: Choose a performance level (e.g., `DW1000c`).
57+
- Click `Review + create` and then `Create` to provision the dedicated SQL pool.
58+
59+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/89ca427b-20f1-4df5-ae20-847b76cdd9a7" />
60+
61+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/b8b2c94b-76e6-4ced-8812-c386c6a55f32" />
62+
63+
### Create Tables with Spaces in Names and Columns
64+
65+
1. **Open the SQL Script Editor**:
66+
- In Synapse Studio, go to the `Develop hub` by clicking on the `Develop` icon in the left navigation pane.
67+
- Click on `+ New SQL script` to open the SQL script editor.
68+
69+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/3382ea4a-06eb-4e32-93d9-569cef7fc2f5" />
70+
71+
2. **Create Sample Tables**: Use the following script to create tables with spaces in their names and columns. Click [here to see the .sql file]().
72+
73+
```sql
74+
-- Create sample tables with spaces in names
75+
CREATE TABLE [Employee Records] (
76+
[Employee ID] INT,
77+
[Employee Name] NVARCHAR(255),
78+
[Employee Address] NVARCHAR(255)
79+
);
80+
81+
CREATE TABLE [Sales Data] (
82+
[Sale ID] INT,
83+
[Sale Date] DATE,
84+
[Employee ID] INT,
85+
[Sale Amount] DECIMAL(10, 2)
86+
);
87+
88+
CREATE TABLE [Inventory Details] (
89+
[Item ID] INT,
90+
[Item Name] NVARCHAR(255),
91+
[Item Category] NVARCHAR(255),
92+
[Item Price] DECIMAL(10, 2)
93+
);
94+
95+
-- Insert sample data into the tables
96+
INSERT INTO [Employee Records] ([Employee ID], [Employee Name], [Employee Address])
97+
VALUES (1, 'Alice Johnson', '789 Pine St');
98+
99+
INSERT INTO [Employee Records] ([Employee ID], [Employee Name], [Employee Address])
100+
VALUES (2, 'Bob Brown', '101 Maple St');
101+
102+
INSERT INTO [Sales Data] ([Sale ID], [Sale Date], [Employee ID], [Sale Amount])
103+
VALUES (1, '2023-02-01', 1, 200.00);
104+
105+
INSERT INTO [Sales Data] ([Sale ID], [Sale Date], [Employee ID], [Sale Amount])
106+
VALUES (2, '2023-02-02', 2, 250.00);
107+
108+
INSERT INTO [Inventory Details] ([Item ID], [Item Name], [Item Category], [Item Price])
109+
VALUES (1, 'Gadget', 'Electronics', 49.99);
110+
111+
INSERT INTO [Inventory Details] ([Item ID], [Item Name], [Item Category], [Item Price])
112+
VALUES (2, 'Tool', 'Hardware', 29.99);
113+
```
114+
115+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/6f8a154d-2fcb-4aa5-bc9f-27f6b8334017" />
116+
117+
3. **Run the Script**: Execute the script in the SQL script editor to create the tables and insert sample data.
118+
119+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/1d02fd74-7246-4aa6-8247-b98619d70c47" />
120+
121+
> [!NOTE]
122+
> Once you refresh, the tables will be visible:
123+
124+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/3e6fc8a1-ad34-4a0b-8940-ae27b303190d">
125+
126+
### Create Views with Modified Tables/Column Names
127+
128+
1. **Create a Stored Procedure to Remove Spaces from Column Names**: Use the following script to create a stored procedure that removes spaces from column names and creates views. Click [here to see the .sql file]().
129+
130+
```sql
131+
CREATE PROCEDURE RemoveSpacesFromColumnNames
132+
AS
133+
BEGIN
134+
DECLARE @tableName NVARCHAR(255)
135+
DECLARE @columnName NVARCHAR(255)
136+
DECLARE @sql NVARCHAR(MAX)
137+
138+
-- Temporary table to store table names
139+
CREATE TABLE #TableNames (TABLE_NAME NVARCHAR(255))
140+
INSERT INTO #TableNames
141+
SELECT TABLE_NAME
142+
FROM INFORMATION_SCHEMA.TABLES
143+
WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_SCHEMA = 'dbo'
144+
145+
-- Loop through each table
146+
WHILE EXISTS (SELECT 1 FROM #TableNames)
147+
BEGIN
148+
SELECT TOP 1 @tableName = TABLE_NAME FROM #TableNames
149+
150+
-- Print the table name for debugging
151+
PRINT 'Processing table: ' + @tableName
152+
153+
SET @sql = 'CREATE VIEW dbo.vw' + REPLACE(@tableName, ' ', '') + ' AS SELECT '
154+
155+
-- Drop the temporary table if it exists
156+
IF OBJECT_ID('tempdb..#ColumnNames') IS NOT NULL
157+
DROP TABLE #ColumnNames
158+
159+
-- Temporary table to store column names
160+
CREATE TABLE #ColumnNames (COLUMN_NAME NVARCHAR(255))
161+
INSERT INTO #ColumnNames
162+
SELECT COLUMN_NAME
163+
FROM INFORMATION_SCHEMA.COLUMNS
164+
WHERE TABLE_NAME = @tableName
165+
166+
-- Loop through each column
167+
WHILE EXISTS (SELECT 1 FROM #ColumnNames)
168+
BEGIN
169+
SELECT TOP 1 @columnName = COLUMN_NAME FROM #ColumnNames
170+
171+
-- Print the column name for debugging
172+
PRINT 'Processing column: ' + @columnName
173+
174+
-- Remove all spaces from column names
175+
IF (SELECT COUNT(*) FROM #ColumnNames) = 1
176+
BEGIN
177+
SET @sql = @sql + 'REPLACE([' + @columnName + '], '' '', '''') AS [' + REPLACE(@columnName, ' ', '') + '] '
178+
END
179+
ELSE
180+
BEGIN
181+
SET @sql = @sql + 'REPLACE([' + @columnName + '], '' '', '''') AS [' + REPLACE(@columnName, ' ', '') + '], '
182+
END
183+
184+
DELETE FROM #ColumnNames WHERE COLUMN_NAME = @columnName
185+
END
186+
187+
-- Remove the trailing comma and space if any
188+
IF RIGHT(@sql, 2) = ', '
189+
BEGIN
190+
SET @sql = LEFT(@sql, LEN(@sql) - 2)
191+
END
192+
193+
SET @sql = @sql + ' FROM [' + @tableName + '];'
194+
195+
-- Print the dynamic SQL for debugging
196+
PRINT 'Generated SQL: ' + @sql
197+
198+
-- Execute the dynamic SQL
199+
BEGIN TRY
200+
EXEC sp_executesql @sql
201+
END TRY
202+
BEGIN CATCH
203+
PRINT 'Error: ' + ERROR_MESSAGE()
204+
END CATCH
205+
206+
DELETE FROM #TableNames WHERE TABLE_NAME = @tableName
207+
END
208+
209+
-- Clean up temporary tables
210+
DROP TABLE #TableNames
211+
DROP TABLE #ColumnNames
212+
END
213+
```
214+
215+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/13b3b5f2-3142-448f-b03a-3eeae00a1509" />
216+
217+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/575905c1-bcf7-4800-981b-6ff4ab2b3302" />
218+
219+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/a7244b21-c787-4f64-8689-fe66670aa86a" />
220+
221+
2. **Execute the Stored Procedure**: Click on `Run`, to create the stored procedure.
222+
223+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/17be4e92-9816-43ff-9cc3-23d9737a9056" />
224+
225+
| Before | After |
226+
| --- | --- |
227+
| <img width="550" alt="image" src="https://github.com/user-attachments/assets/d081329c-b5f6-452f-826b-50a2dd614a1c" /> | <img width="550" alt="image" src="https://github.com/user-attachments/assets/588aa482-ac5c-4013-b926-e01db58d1733" /> |
228+
229+
3. Run the stored procedure to create views with modified column names.
230+
231+
```sql
232+
EXEC RemoveSpacesFromColumnNames
233+
```
234+
235+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/015b9c07-10f6-4895-bdba-4945e3277923" />
236+
237+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/1ccf430e-1dff-4968-980c-a4f3913a8369" />
238+
239+
| Before | After |
240+
| --- | --- |
241+
| <img width="360" alt="image" src="https://github.com/user-attachments/assets/b0de5118-bf67-4f75-9dd7-2ae5ba33cb28" /> | <img width="360" alt="image" src="https://github.com/user-attachments/assets/ef9c6d3a-1d45-4d37-9977-431cf8b774d0" /> |
242+
243+
> [!NOTE]
244+
> Once you refresh, the views will be visible:
245+
246+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/e0191419-29cc-448a-a14f-a76da221b015" />
247+
248+
249+
<div align="center">
250+
<h3 style="color: #4CAF50;">Total Visitors</h3>
251+
<img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
252+
</div>

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/README.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Azure Synapse: Dynamically Remove Space - Overview
2+
3+
Costa Rica
4+
5+
[![GitHub](https://badgen.net/badge/icon/github?icon=github&label)](https://github.com)
6+
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
7+
[brown9804](https://github.com/brown9804)
8+
9+
Last updated: 2025-01-24
10+
11+
----------
12+
13+
> [!NOTE]
14+
> Both the `table name and column name fields will be reviewed`. We'll remove any blank spaces to create the view.
15+
16+
## Wiki
17+
18+
<details>
19+
<summary><b>List of References </b> (Click to expand)</summary>
20+
21+
- [What is Azure Synapse Analytics?](https://learn.microsoft.com/en-us/azure/synapse-analytics/overview-what-is)
22+
- [What is dedicated SQL pool (formerly SQL DW) in Azure Synapse Analytics?](https://learn.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is)
23+
- [Serverless SQL pool in Azure Synapse Analytics](https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/on-demand-workspace-overview)
24+
- [Create and use views using serverless SQL pool in Azure Synapse Analytics](https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/create-use-views)
25+
26+
</details>
27+
28+
## Content
29+
30+
<details>
31+
<summary><b>Table of Content </b> (Click to expand)</summary>
32+
33+
- [Wiki](#wiki)
34+
- [Content](#content)
35+
- [Overview](#overview)
36+
- [SQL Pools](#sql-pools)
37+
- [Synapse Views](#synapse-views)
38+
- [How to Dynamically Remove Space in field name](#how-to-dynamically-remove-space-in-field-name)
39+
40+
</details>
41+
42+
## Overview
43+
44+
> `Azure Synapse Analytics` is an integrated analytics service that accelerates time to insight across data warehouses and big data systems.
45+
> It combines the `best of SQL technologies used in enterprise data warehousing`, Spark technologies for big data, and data integration
46+
> capabilities to provide a unified experience for data professionals
47+
48+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/9a753e36-e229-4da7-8a87-a5c21ddd70f9"> <br/>
49+
From [Microsoft Official Documentation](https://learn.microsoft.com/en-us/azure/synapse-analytics/overview-what-is)
50+
51+
### SQL Pools
52+
53+
SQL pools in Azure Synapse Analytics come in two main types:
54+
55+
| Type | Description | Characteristics |
56+
| --- | --- | --- |
57+
| Serverless SQL Pool (built in) | This is a built-in, on-demand query service in Azure Synapse Analytics. You don't need to provision or manage any infrastructure. You simply run your queries, and you're billed based on the amount of data processed. It's great for ad-hoc querying and exploring data in your data lake without worrying about the underlying infrastructure. | - A query service over data in your data lake. <br/> - No infrastructure setup or clusters to maintain. <br/> - Pay-per-use model, charging only for the data processed by queries. <br/> - Suitable for ad-hoc querying and data exploration. |
58+
| Dedicated SQL Pool (formerly SQL DW) | This requires you to provision and manage a set of resources (Data Warehousing Units or DWUs). You have control over the performance and scale of your data warehouse, but it also means you need to manage the infrastructure. It's ideal for large-scale, high-performance analytics and data warehousing. | - A collection of analytic resources provisioned when using Synapse SQL. <br/> - Uses Data Warehousing Units (DWU) to determine size and performance. <br/> - Ideal for large-scale data warehousing and high-performance analytics. |
59+
60+
### Synapse Views
61+
62+
> **Synapse views** are `virtual tables created by querying data from one or more tables in Azure Synapse Analytics.
63+
> They don't store data themselves but provide a way to simplify complex queries and reuse them`. Views can be created over both dedicated and serverless SQL pools.
64+
65+
## How to Dynamically Remove Space in field name
66+
67+
- For an overview of Serverless SQL Pool (built-in), please read this overview.
68+
- For information on Dedicated SQL Pool (formerly SQL DW), click here.
69+
70+
<div align="center">
71+
<h3 style="color: #4CAF50;">Total Visitors</h3>
72+
<img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
73+
</div>

0 commit comments

Comments
 (0)