Here we go in further detail describing how we implemented the different process steps, with the objective of creating a comprehensive and real-time building management system starting from the project BIM file.
REAL-TIME IOT DATA, MICROSOFT FABRIC AND VCAD
REAL-TIME IOT DATA, MICROSOFT FABRIC AND VCAD #3
REAL-TIME IOT DATA, MICROSOFT FABRIC AND VCAD #4
REAL-TIME IOT DATA, MICROSOFT FABRIC AND VCAD #5
REAL-TIME IOT DATA, MICROSOFT FABRIC AND VCAD #6
Implementation
We can divide the process previously described in 3 main phases:
- Creation of the device instances on the IoT Hub
- Management of the device real-time data
- Displaying of the real-time data
In particular, for the management of the real-time device data, we used the new Microsoft Fabric platform.
Microsoft Fabric is a SaaS (Software as a Service) Analytics Platform that covers end-to-end business requirements. It is built upon the Power BI platform and extends the capabilities of Azure Synapse Analytics to all analytics workloads.
In the following paragraphs we will detail these phases.
Creation of digital device twins
DPS devices enrollment.
The process begins by gathering device details in two JSON files: a definition file and a device instance file. These files hold vital information about the devices and their properties. Azure CLI, a powerful command-line interface, aids in managing Azure resources and services. Utilizing the Microsoft PowerShell SDK library, we can seamlessly interact with Azure services programmatically.
Figure 1 Device definition
Figure 2 Device instances
First, we authenticate the PowerShell script with Azure credentials to access the Azure Device Provisioning Service (DPS). Next, the script parses the JSON files, extracting device attributes like device ID, key, and other essential data. With this information at hand, the PowerShell script interacts with Azure CLI to create device enrollments dynamically within the DPS.
Figure 3 Devices bulk enrollment on DPS
By automating the device enrollment process, we eliminate manual errors, reduce deployment time, and ensure a scalable and efficient IoT solution. This approach not only enhances productivity but also ensures seamless integration between BIM projects and Azure DPS, optimizing the overall IoT ecosystem. As a result, organizations can unlock the full potential of their IoT devices with Azure’s secure and reliable platform.
Figure 4 DPS devices enrollment
IoT Hub device registration
Registering a device enrolled in Azure DPS on the IoT Hub involves a series of steps to establish a connection and securely provision the device.
Below is an outline of the process:
Retrieve Enrollment Information: access the enrollment information of the device from Azure DPS. This information includes the unique device ID and the authentication keys.
Install Azure IoT SDK: ensure that you have the necessary Azure IoT SDK installed on the device to establish communication with the IoT Hub.
Register the Device: use the Azure IoT SDK to register the device with the IoT Hub. During the registration process, the device is assigned a unique connection string that it will use to securely communicate with the IoT Hub.
Start Sending Data: with the device registered and connected to the IoT Hub, it can now start sending data to the cloud for processing and analysis.
By following these steps, we have seamlessly registered the devices enrolled in Azure DPS onto the IoT Hub, enabling them to be part of a broader IoT ecosystem and benefiting from Azure’s comprehensive suite of services for IoT solutions.
Figure 5 Devices registered on the IoT Hub
Devices simulator
The steps described in the previous paragraph were performed by building a windows form application in c#. The first phase is the configuration of the application, which is obtained by loading the json files previously used for devices enrollment on DPS.
Figure 6 Simulator configuration
After the configuration, we can proceed with registering the devices on the IoT hub, and then with sending the data.
Figure 7 Send data to devices
This kind of approach is very flexible and allows you to simulate data for any number and type of sensors. However, this solution requires an installation on an ad hoc machine.
A different approach, though less flexible in terms of configuration, that allows deployment and scheduling on cloud servers, is the use of the Node-RED tool.
Node-RED is an open-source flow-based development tool that is used for visual programming and creating applications by wiring together different nodes. It is often used for building Internet of Things (IoT) applications and integrating various devices, APIs, and online services.
Node-RED provides a graphical user interface (GUI) that allows users to create workflows (flows) by connecting nodes in a flowchart-like manner.
Figure 8 Node-RED flow
While building the flow, we used the “Azure IoT Hub” Node-RED’s component, which allows the flow to simulate data submissions as if it were the device sending them.
As stated, the advantage of this approach is that it can be deployed in a cloud solution and scheduled to send data as required.
Real-time data management
Sending real-time data from Azure IoT Hub to Power BI involves a multi-step process, leveraging Microsoft technologies like Azure Event Hubs and Azure Data Explorer (formerly known as Kusto). We experimented using the new Microsoft Fabric platform. Let’s explore the step-by-step process to achieve this:
Device Data Ingestion: IoT devices send data to the IoT Hub using protocols like MQTT, AMQP, or HTTPS. The IoT Hub acts as a message broker and routes the incoming data to different endpoints for further processing.
Azure Event Hubs: we have created an Azure Event Hub, which can be used to receive and temporarily store high-throughput, real-time event data from the IoT Hub. The Event Hub provides a buffer between the IoT Hub and downstream data consumers, enabling smooth data flow even during spikes in data volume.
Eventstream Job: we have set up a Microsoft Fabric Eventstream job that takes the data from the Event Hub as the input source. Eventstream is a real-time data stream processing service that send this data to different endpoints.
Data Ingestion: we have set up an Azure Data Explorer (ADX) database, formerly known as Kusto DB, as the endpoint of the Eventstream job. Data collected by the Eventstream from the Event Hub is sent to ADX and stored in a table. ADX is a fast and scalable data analytics service. ADX is well-suited for ingesting, analyzing, and visualizing large volumes of data in real-time. A materialized view has been created on this table that selects the latest ingested data and transforms them as required.
Power BI Integration: to enable real-time visualization of the data, whe have created a Power BI dataset and streaming data source. Power BI supports real-time data streaming, allowing you to visualize the data as it arrives.
Real-Time Data Ingestion: with the EventStream in place, it continuously ingests the data from the Event Hub into the ADX database, ensuring real-time updates with minimal latency.
Conclusions
In summary, by routing data from Azure IoT Hub to an Azure Event Hub and leveraging ADX for real-time ingestion and storage, organizations can build powerful real-time analytics solutions. With Power BI integration, users gain access to dynamic visualizations, empowering them to make data-driven decisions instantly. This end-to-end solution enables businesses to harness the power of real-time data analytics and gain valuable insights from their IoT devices in a seamless and efficient manner.
