Skip to content

Commit acebe52

Browse files
brown9804brown9804
authored
Merge ac3a403 into 871cabd 
2 parents 871cabd + ac3a403 commit acebe52

4 files changed

Lines changed: 157 additions & 4 deletions

File tree

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/0_serverlessSQLPool/README.md‎

Lines changed: 146 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Costa Rica
66
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
77
[brown9804](https://github.com/brown9804)
88

9-
Last updated: 2025-01-24
9+
Last updated: 2025-01-25
1010

1111
----------
1212

@@ -26,7 +26,152 @@ Last updated: 2025-01-24
2626

2727
</details>
2828

29+
## Overview
2930

31+
> [!IMPORTANT]
32+
> The serverless SQL pool in Azure Synapse Analytics `does not support internal tables. It primarily supports external tables and temporary tables`.
33+
> For `internal tables,` you would need to use a `dedicated SQL pool` in Synapse Analytics, which allows you to `create and manage internal tables with local storage`.
34+
35+
| **Table Type** | **Description** | **Use Cases** |
36+
|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
37+
| **External Tables** | - Reference data stored in external sources like Azure Data Lake Storage, Azure Blob Storage, Azure Cosmos DB, etc. <br> - Data is not physically stored in the SQL pool. <br> - Useful for querying large datasets without loading them into the SQL pool. | - Querying large datasets stored externally. <br> - Performing data analysis on data stored in various formats. |
38+
| **Temporary Tables**| - Created and used within the scope of a session. <br> - Data is stored temporarily and is dropped when the session ends. <br> - Useful for intermediate data processing and transformations. | - Storing intermediate results during complex queries. <br> - Performing temporary data transformations and aggregations. |
39+
40+
41+
## Demo
42+
43+
### Set Up a Synapse Workspace
44+
45+
1. **Sign in to the Azure Portal**: Go to the Azure Portal and sign in with your Azure account.
46+
2. **Navigate to Your Synapse Workspace**: In the Azure Portal, search for your Synapse workspace or create a new one if you don't have one.
47+
48+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/92a5e451-1868-47e2-b32b-858591c306ee" />
49+
50+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/51e3b091-855d-4481-89e0-623705e3cf2a" />
51+
52+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/7d03bfa8-e1e3-4706-970f-a89c7b8cd904" />
53+
54+
3. **Launch Synapse Studio**: From the Synapse workspace overview, click on the `Open Synapse Studio` button.
55+
56+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/302b1fd8-49a6-427e-93dc-8e952f1667e6" />
57+
58+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/a368036b-c859-47fc-a1c5-d045b6910790" />
59+
60+
### Upload Sample Data to Storage Account
61+
62+
> [!IMPORTANT]
63+
> For this demo, `we'll be using the same storage account created with the synpase workspace`.
64+
> `CREATE EXTERNAL DATA SOURCE is not supported in the master database of the serverless` SQL pool. Instead, you `need to create a user database and perform the operations there`.
65+
66+
1. Create a Container in the Storage Account:
67+
- Go to the Azure portal and navigate to your storage account.
68+
- In the left-hand menu, select `Containers`.
69+
- Click on `+ Container` to create a new container.
70+
- Enter a name for the container, such as `sample-tables-container`.
71+
- Set the `Public access level` to your preference (e.g., Private).
72+
- Click `Create`.
73+
74+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/3060045f-a515-4a27-a4c6-80bd2ba3d9ab" />
75+
76+
2. Navigate to the Container:
77+
- In the Azure portal, go to your storage account.
78+
- Select `Containers` from the left-hand menu.
79+
- Click on the container you created (e.g., `sample-tables-container`).
80+
3. Upload the Sample CSV File:
81+
- Click on the `Upload` button.
82+
- In the upload blade, click on `Browse` to select the sample CSV file from your local machine.
83+
- Choose the file, click `Upload` to upload the file to the container.
84+
85+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/4fa8d1c8-e5a0-4d65-a3b5-83bf7c7118c1" />
86+
87+
### Create User Database
88+
89+
1. First, create a new database in your Synapse workspace.
90+
91+
```sql
92+
CREATE DATABASE {User Database Name};
93+
```
94+
95+
<img width="884" alt="image" src="https://github.com/user-attachments/assets/f24a4d45-6bee-46ea-bccc-b5f893044c01" />
96+
97+
2. Switch to the User Database: Use the newly created database.
98+
3. Create an External Data Source: Integrate this task with the previous step by establishing the external data source within the user database.
99+
100+
```sql
101+
USE {User Database Name};
102+
103+
CREATE EXTERNAL DATA SOURCE {Data Source Name}
104+
WITH (
105+
LOCATION = 'https://<your-storage-account>.dfs.core.windows.net/<your-container>'
106+
);
107+
```
108+
109+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/13f22e3b-7c68-4ed2-8a56-2752a5033869" />
110+
111+
112+
4. **Create an External File Format**: As part of the same flow, define the format of the CSV file.
113+
114+
```sql
115+
CREATE EXTERNAL FILE FORMAT {File Format Name}
116+
WITH (
117+
FORMAT_TYPE = DELIMITEDTEXT,
118+
FORMAT_OPTIONS (
119+
FIELD_TERMINATOR = ',',
120+
STRING_DELIMITER = '"',
121+
FIRST_ROW = 2
122+
)
123+
);
124+
```
125+
126+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/40a4c6a2-bc51-49ba-9361-bec6eb0edb9e" />
127+
128+
129+
5. **Create an External Table**: Create an external table that references the sample data.
130+
131+
```sql
132+
CREATE EXTERNAL TABLE [Table With Spaces] (
133+
[ID] INT,
134+
[Name With Spaces] NVARCHAR(50),
135+
[Date With Spaces] DATE
136+
)
137+
WITH (
138+
LOCATION = '<file-name>.csv',
139+
DATA_SOURCE = {Data Source Name},
140+
FILE_FORMAT = {File Format Name}
141+
);
142+
```
143+
144+
145+
146+
6. **Query the External Table**: You can now query the external table to see the sample data.
147+
```sql
148+
SELECT * FROM [Table With Spaces];
149+
```
150+
151+
152+
153+
154+
155+
1. **Open the SQL Script Editor**:
156+
- In Synapse Studio, go to the `Develop hub` by clicking on the `Develop` icon in the left navigation pane.
157+
- Click on `+ New SQL script` to open the SQL script editor.
158+
159+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/3382ea4a-06eb-4e32-93d9-569cef7fc2f5" />
160+
161+
162+
2. Create an External Data Source: Define an external data source that points to your SQL database.
163+
164+
```sql
165+
CREATE EXTERNAL DATA SOURCE MySQLDataSource
166+
WITH (
167+
TYPE = RDBMS,
168+
LOCATION = '<your-sql-server-name>.database.windows.net',
169+
DATABASE_NAME = '<your-database-name>',
170+
CREDENTIAL = MyCredential
171+
);
172+
```
173+
174+
30175
<div align="center">
31176
<h3 style="color: #4CAF50;">Total Visitors</h3>
32177
<img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/0_serverlessSQLPool/_samples-files/employee data with spaces.csv‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Employee ID, Employee Name, Hire Date
2+
101, Alice Johnson, 2024-05-15
3+
102, Bob Smith, 2024-06-20
4+
103, Charlie Brown, 2024-07-25

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/0_serverlessSQLPool/_samples-files/product data with spaces.csv‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
ProductID, Product Name, Release Date
2+
201, Gadget Pro, 2023-11-01
3+
202, Widget Plus, 2023-12-15
4+
203, Device Max, 2024-01-10

‎0_Azure/2_AzureAnalytics/1_SynapseAnalytics/demos/2_synapse_views_dynamically_remove_space/1_dedicatedSQLPoolStoreProc/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Costa Rica
66
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
77
[brown9804](https://github.com/brown9804)
88

9-
Last updated: 2025-01-24
9+
Last updated: 2025-01-26
1010

1111
----------
1212

@@ -17,7 +17,7 @@ Last updated: 2025-01-24
1717

1818
- [Content](#content)
1919
- [Demo](#demo)
20-
- [Set Up a Dedicated SQL Pool](#set-up-a-dedicated-sql-pool)
20+
- [Set Up a Synapse Workspace](#set-up-a-synapse-workspace)
2121
- [Create a Dedicated SQL Pool](#create-a-dedicated-sql-pool)
2222
- [Create Tables with Spaces in Names and Columns](#create-tables-with-spaces-in-names-and-columns)
2323
- [Create Views with Modified Tables/Column Names](#create-views-with-modified-tablescolumn-names)
@@ -26,7 +26,7 @@ Last updated: 2025-01-24
2626

2727
## Demo
2828

29-
### Set Up a Dedicated SQL Pool
29+
### Set Up a Synapse Workspace
3030

3131
1. **Sign in to the Azure Portal**: Go to the Azure Portal and sign in with your Azure account.
3232
2. **Navigate to Your Synapse Workspace**: In the Azure Portal, search for your Synapse workspace or create a new one if you don't have one.

0 commit comments

Comments
 (0)