Real-Time Intelligence: Real-time data in Microsoft Fabric
In today’s business world, there is an increasing focus on basing decisions and processes on a solid foundation of data. The technical solution is usually a combination of data warehouses and dashboards that compile company data and present it in a visual format that is easy for everyone to understand and use.
Implementation often relies on batch processing, whereby data is collected and prepared automatically — for instance, once a day or less frequently, such as every hour or every few minutes.
This approach works well for many applications. However, it has its limitations when it comes to analysing information ‘in real time’ with a delay of a few seconds at most. Here are a few examples:
- Production: monitoring sensor data to prevent machine breakdowns (‘predictive maintenance’).
- Supply chain: tracking location data and weather events to recognise delivery delays early on.
- Finance: Monitoring and analysing share prices in real time.
- IT: analysing log data to detect problems immediately after updates.
- Marketing: analysing social media posts during live events.
None of this is new, but previous solutions were often challenging to implement and required considerable expertise.
This is precisely where Microsoft stepped in: in 2024, the Fabric platform was expanded to include Real-Time Intelligence and various building blocks that can be used to bypass much of the complexity and swiftly develop functioning solutions.
In future articles, we will closely examine the most important of these ‘fabric items‘. Here is a brief overview:
Eventstream
Event streams continuously receive real-time data (events) from various sources. This data is transformed as required and ultimately forwarded to a destination responsible for storing it. Typically, this destination is an event hub. No code is required.
Eventhouse
An event house is an optimised data storage facility designed specifically for events. It contains at least one KQL database, which stores event data in table form.
Real-Time Dashboard
Real-time dashboards are similar to Power BI reports, but they are a standalone solution independent of Power BI. A real-time dashboard contains tiles with visualisations, such as diagrams or tables. These are interactive, and you can apply filters, for example. Each visual retrieves the necessary data via a database query formulated in KQL (Kusto Query Language), typically from an event hub.
Activator
Activator enables you to automatically perform an action based on certain conditions, such as a real-time dashboard or a KQL query. The simplest action is sending a message via email or Teams, but you can also trigger a Power Automate flow.
So what do I need to learn to implement a solution with Real-Time Intelligence?,
It essentially boils down to KQL and a basic understanding of the aforementioned fabric items. A lot of it is no-code. KQL is less common than SQL, but the basics are easy to learn, and feel natural after a short time.
We’ll be back soon with more posts on Fabric Real-Time Intelligence in our blog, delving deeper into the various topics.
Author
Rupert Schneider
Fabric Data Engineer at scieneers GmbH
rupert.schneider@scieneers.de