Skip to content

Commit

Permalink
Merge pull request #28 from microsoft/structure-changes
Browse files Browse the repository at this point in the history
Structure changes
  • Loading branch information
itsnotaboutthecell authored Jan 30, 2025
2 parents 194eb9a + c89de77 commit c31496a
Show file tree
Hide file tree
Showing 142 changed files with 7,658 additions and 8,021 deletions.
44 changes: 40 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,44 @@
# Fabric Toolbox
<p align="center">
<img src="./media/Fabric.svg" />
</p>

Fabric toolbox is a repository of tools, tempalates, queries, and other useful artifacts that can help make you successful with Microsoft Fabric. This repository will be ever-growing as we develop new tools and accelerators.
# Welcome to the Fabric Toolbox

## Contributing
Fabric toolbox is a repository of tools, accelerators, scripts, and samples to help you work with Microsoft Fabric. This repository is brought to you by the Fabric Customer Advisory Team (CAT) and will continue to grow as we develop new tools and accelerators.

These assets should be treated as examples that you can use to create the solutions that are appropriate for your use case. If you have any issues, please use the [issues](https://github.com/microsoft/fabric-toolbox/issues) tab of this repository and we will work to address issues on a best effort basis.

## Accelerators
- [Data Warehouse Backup and Recovery](./accelerators/data-warehouse-backup-and-recovery)
- [Power BI to Fabric Data Warehouse Modernization](./accelerators/power-bi-to-fabric-data-warehouse-modernization)
- [Real Time Intelligence Eventhouse](./accelerators/real-time-intelligence_eventhouse)
- [Real Time Intelligence Eventstream](./accelerators/real-time-intelligence_eventstream)

## Samples
- $${\color{green}NEW!}$$ [Open Mirroring](./samples/open-mirroring)
- [Advanced Data Pipeline Scheduleing Specific_Day](./samples/Advanced_Data_Pipeline_Scheduleing_Specific_Day)
- [Polling Storage Event Trigger](./samples/polling-storage-even-trigger)
- $${\color{green}NEW!}$$ [Refresh tables in SQL Endpoint](./samples/refresh-tables-in-sql-endpoint)
- [Visualizing Linked Table Dataflows](./samples/visualizing-linked-table-dataflows)
- $${\color{green}NEW!}$$ [Azure Policies: Pause or Delete Capacity](./samples/azure-policy)
- $${\color{green}NEW!}$$ [Notebook: List Dedicated Pool Connections](./samples/notebook-list-dedicated-pool-connections)
- $${\color{green}NEW!}$$ [Notebook: Create, List, Delete a Data Warehouse](./samples/notebook-create-list-delete-warehouse)
- $${\color{green}NEW!}$$ [Notebook: Workspace Size](./samples/notebook-workspace-size)

## Scripts
- [DW Active Requests](./scripts/dw-active-requests)
- [DW Copy DMV to Table](./scripts/dw-copy-dmv-to-table)
- [DW Queries Running at Timepoint](./scripts/dw-queries-running-at-timepoint)
- [DW SP Kill Queries](./scripts/dw-sp-kill-queries)

## Tools
- $${\color{green}NEW!}$$ [Copy Warehouse](./tools/copy-warehouse)
- [TPCH Benchmarking](./tools/tpch-benchmarking)

## Monitoring
$${\color{green}NEW!}$$ [Workspace Monitoring Dashboards](./monitoring/workspace-monitoring-dashboards)

# Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
Expand All @@ -16,7 +52,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.

## Trademarks
# Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must follow
Expand Down
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
![Datamart Modernization](./media/Datamart%20Modernization.png)
![Datamart Modernization](../media/Datamart%20Modernization.png)

# Power BI Datamart Modernization

Expand Down
49 changes: 49 additions & 0 deletions accelerators/real-time-intelligence_eventhouse/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# FabricRTI_Accelerator
## Introduction
Real-Time Intelligence is a powerful service in Microsoft Fabric that empowers everyone in your organization to extract insights and visualize their streaming and high granularity data in real time. It offers an end-to-end solution for event-driven scenarios, streaming data, and data logs. Whether dealing with gigabytes or petabytes, all organizational data in motion converges in the Real-Time Hub. It seamlessly connects time-based data from various sources using no-code connectors, enabling immediate visual insights, geospatial analysis, and trigger-based reactions that are all part of an organization-wide data catalog.
Once you seamlessly connect any stream of data, the entire SaaS solution becomes accessible. Real-Time Intelligence handles data ingestion, transformation, storage, analytics, visualization, tracking, AI, and real-time actions. Your data remains protected, governed, and integrated across your organization, seamlessly aligning with all Fabric offerings. Real-Time Intelligence transforms your data into a dynamic, actionable resource that drives value across the entire organization.

[Read more](https://learn.microsoft.com/en-us/fabric/real-time-intelligence/overview)
## Overview
This project aims to provide easy, Low code and automated way of deploying Real-Time intelligence workload and its entities in Microsoft Fabric. This removes the hassle of manually creating each item and entities when you want to move from one environment or workspace to another. Users can run the PowerShell script along with required parameters to provision automatically -
* Eventhouse, KQL Database and it's entities like Tables, Functions and Materialized Views.
* Eventstream with definition of source, processing and destination.

[Full List of Fabric APIs](https://learn.microsoft.com/en-us/rest/api/fabric/articles/)

Currently Eventhouse and Eventstream accelerators are added in this project. Look out for accelerators for Activator as soon as the APIs are available.
Jump to the relavent section by using the links below
* [Eventhouse](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator#eventhouse)
* [Eventstream](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator#eventstream)

# Eventhouse

[Eventhouse and KQL DB APIs Blog](https://blog.fabric.microsoft.com/en-us/blog/using-apis-with-fabric-real-time-analytics/)
## Instructions
To create Eventhouse and its entities, run the PowerShell script with following parameters

|Parameter|Mandatory?|Description|
|--------|--------|----------|
|tenantId|Yes|Tenant ID of your Fabric tenant|
|workspaceId| Yes |Workspace ID where you want to create Eventhouse|
|eventhouseName | Yes| Display name of your Eventhouse|
|kqlDBName | No| Name of your KQL DB. If not provided, default KQL DB in Eventhouse will be used|
|dbScriptName | No| Database script with creation script of all entities|

* Make sure you have Azure modules installed in Powershell.
```
Install-Module Az
```
* Make sure you have the database script in the same folder as the Powershell script. A test database script is provided here - TestPS_KQL.csl . This script creates a simple table testPS3 and a function GetNames()._
* The script will open an interactive login window and then ask for a selection of tenant and subscription (Yes! You must select subscription although you are working with Fabric). You can set any default subscription for subsequent runs
```
Update-AzConfig -DefaultSubscriptionForLogin "My_Subscription_Name"
```
* Run the script
```
.\createEventhouse.ps1 -tenantId 'xxxxxx-yyyy-aaaa-bbbb-aaaaaaaaa' -workspaceId 'xxxxx-yyyy-yyyy-aaaa-aaaaaaaaaaaa' -eventhouseName 'Eventhouse_001' -kqlDBName 'KQLDB_001' -dbScriptName 'CreateDB_script.csl'
```

* Example of Eventhouse, KQL Database, table and a function created using this Powershell script

![Eventhouse](../media/Created_Entities.png)
Original file line number Diff line number Diff line change
Expand Up @@ -16,38 +16,6 @@ Jump to the relavent section by using the links below
* [Eventhouse](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator#eventhouse)
* [Eventstream](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator#eventstream)

# Eventhouse

[Eventhouse and KQL DB APIs Blog](https://blog.fabric.microsoft.com/en-us/blog/using-apis-with-fabric-real-time-analytics/)
## Instructions
To create Eventhouse and its entities, run the PowerShell script with following parameters

|Parameter|Mandatory?|Description|
|--------|--------|----------|
|tenantId|Yes|Tenant ID of your Fabric tenant|
|workspaceId| Yes |Workspace ID where you want to create Eventhouse|
|eventhouseName | Yes| Display name of your Eventhouse|
|kqlDBName | No| Name of your KQL DB. If not provided, default KQL DB in Eventhouse will be used|
|dbScriptName | No| Database script with creation script of all entities|

* Make sure you have Azure modules installed in Powershell.
```
Install-Module Az
```
* Make sure you have the database script in the same folder as the Powershell script. A test database script is provided here - TestPS_KQL.csl . This script creates a simple table testPS3 and a function GetNames()._
* The script will open an interactive login window and then ask for a selection of tenant and subscription (Yes! You must select subscription although you are working with Fabric). You can set any default subscription for subsequent runs
```
Update-AzConfig -DefaultSubscriptionForLogin "My_Subscription_Name"
```
* Run the script
```
.\createEventhouse.ps1 -tenantId 'xxxxxx-yyyy-aaaa-bbbb-aaaaaaaaa' -workspaceId 'xxxxx-yyyy-yyyy-aaaa-aaaaaaaaaaaa' -eventhouseName 'Eventhouse_001' -kqlDBName 'KQLDB_001' -dbScriptName 'CreateDB_script.csl'
```

* Example of Eventhouse, KQL Database, table and a function created using this Powershell script

![Eventhouse](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator/blob/main/Assets/Created_Entities.png)

# Eventstream

[Eventstream APIs Docs](https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/eventstream-rest-api#create-eventstream-item-with-definition)
Expand All @@ -74,5 +42,5 @@ Please follow the steps below in order to create the right payload for Eventstre

* Example of Eventstream created along with defined topology

![Eventstream](https://github.com/SuryaTejJosyula/FabricRTI_Accelerator/blob/main/Assets/Eventstream_Created.png)
![Eventstream](../media/Eventstream_Created.png)

11 changes: 0 additions & 11 deletions data-integration/README.md

This file was deleted.

14 changes: 0 additions & 14 deletions data-warehouse/README.md

This file was deleted.

Empty file.
94 changes: 0 additions & 94 deletions data-warehouse/connectivity/jdbc/pom.xml

This file was deleted.

This file was deleted.

This file was deleted.

Binary file not shown.
Binary file not shown.

This file was deleted.

Loading

0 comments on commit c31496a

Please sign in to comment.