SORACOM Developers

Documents

Using the SORACOM Air metadata service to obtain IMSI information and connect with a Microsoft Azure Event Hub

Introduction

Here, we will show you how to use the SORACOM metadata service (hereon “metadata service”) to connect with Microsoft Azure Event Hubs. In this article, we want to take data from a sensor attached to a Raspberry PI and process it with Power BI.

The data from the sensor attached to the Raspberry PI will be sent to Microsoft Azure via SORACOM. Then, the data will be shaped and sent on to Power BI.

This process has the following steps.

  1. Use the metadata service to obtain IMSI information
  2. Send the acquired IMSI information and sensor data to an Event Hub in JSON format
  3. Use the transmitted data in Power BI as data per unit of time

Sending the data to a server (EventHub) with the IMSI information included makes it possible to identify the device on the server side. We will use the metadata service to enable identification of the device.

Use the metadata service to retrieve IMSI information

You can easily obtain IMSI information using the metadata service. If you run the following command from a device that uses an Air SIM card (the Raspberry PI), you can obtain information from the Air SIM card such as the card’s ISMI, name, tag, and speed class in JSON format, as pictured below.

$ curl -s http://metadata.soracom.io/v1/subscriber
{"imsi":"44010XXXXXXXXXX","msisdn":"81XXXXXXXXXX","ipAddress":"10.XXX.XXX.XX","apn":"soracom.io","type":"s1.standard","groupId":"XXXXXXXXXXXXXXXXXXXXXXXXXX","createdAt":1444126013510,"lastModifiedAt":1452656126137,"expiredAt":null,"terminationEnabled":false,"status":"active","tags":{"name":"noriyuki01"},"sessionStatus":{"lastUpdatedAt":1452656126137,"imei":"XXXXXXXXXXXX","location":null,"ueIpAddress":"10.XXX.XXX.XXX","dnsServers":["100.127.0.53","100.127.1.53"],"online":true},"speedClass":"s1.standard","moduleType":"nano","plan":0,"expiryTime":null,"createdTime":1444126013510,"operatorId":"OPXXXXXXXX","lastModifiedTime":1452656126137}

Using API access requires authentication via a token. When using the metadata service, however, there are no tokens involved. Authentication is handled on the level of the SIM card, enabling you to obtain the Air SIM card’s information simply over HTTP.

Send the acquired IMSI information and sensor data to an Event Hub in JSON format

Next, we send the sensor data from the Raspberry PI to an Event hub. Create an Event Hub, then create an application to send data from the Raspberry PI.

Creating an Event Hub

You can create an Event Hub from the Azure portal. Because an Event Hub is part of a Service Bus, you must first create a Service Bus.

eventhub

create an Event Hub from the Service Bus namespace you just created.

eventhub

choose the Event hubs of the ENTITIES of the dashboard menu. eventhub

When the Event Hub is created, you can configure the shared access policies. We use SendRule as the policy to receive data from the Raspberry PI. As the access policy from Stream Analytics, we use ReceiveRule. The permissions are as follows.

  1. SendRule: Send
  2. ReceiveRule: Manage, Send, Listen

When you are done, do not forget to click the “Save” button at the bottom of the screen. You have now created an Event Hub.

eventhub

Making an application to send data from the Raspberry PI

Now we make an application to send data from the Raspberry PI to the Azure Event Hub. Our current Raspberry PI uses Linux and Python. Install additional packages and such as pictured below.

#!/bin/sh
sudo apt-get install python-pip
sudo pip install urllib3
sudo pip install azure

The Python runtime environment is ready, so we can create the application as follows.

Line 15 in the application serves to obtain the IMSI information from the metadata service, and use it as the DeviceID within JSON. This makes it possible to manage sensor data sent on to Azure Event Hub by using the SIM card’s IMSI information, even if the same image is included with every Raspberry PI that is shipped.

#coding: utf-8
from azure.servicebus import ServiceBusService
import json, urllib3
import datetime

#azure service bus
key_name = 'SendRule' # SharedAccessKeyName from Azure portal
key_value = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # SharedAccessKey from Azure portal
service_namespace = 'sora-ns'
sbs = ServiceBusService(service_namespace,
                        shared_access_key_name=key_name,
                        shared_access_key_value=key_value)

#values
url = 'http://metadata.soracom.io/v1/subscriber'
http = urllib3.PoolManager()
r = http.request('GET', url)

root = json.loads(r.data)
devid = root['imsi']
dtime = datetime.datetime.today().strftime("%F %H:%M:%S %z").strip()

#from sensors
temp = "27.4"
pres = "1000.1"
hum = "68.0"
hilux = "100.0"
lowlux = "10.0"

#json fot event hub
json = '{ "Date":"' + dtime + '", "DeviceId":"' + devid + '", "Temp":"' + temp + '", "Pressure":"' + pres + '","Hum":"' + hum + '", "HiLux":"' + hilux + '","LowLux":"' + lowlux + '" }'
sbs.send_event('sora', json)

The shared access key on line 8 can be obtained from the Event Hub configuration page.

eventhub

The program we just created can, via a cron job, regularly send information obtained from the sensor to the Event Hub.

Verifying reception in the Event Hub

You can check the data received by the Event Hub in the Event Hub dashboard.

eventhub

Use the transmitted data in Power BI as data per unit of time time

Using Stream Analytics, we obtain data from the Event Hub, calculate average values per 30-second interval, and send the results of that calculation on to Power BI.

Create a Stream Analytics job

We will now create a Stream Analytics job from the Azure portal.

eventhub

After creating the job, configure the settings in the following order. First create an Input source, then an Output destination, and finally a Query. Click each in the correct order in Job Topology, as pictured below.

eventhub

For the Input source, use the ReceiveRule policy that you configured for the Event Hub you create earlier. The alias name is a character string that we will use in the Stream Analytics Query (explained later), so avoid using the “-” character. When you are done, click “Create.”

eventhub

Next, we create an Output destination.

eventhub

Finally, we configure the query. For this exercise, we group data by DeviceID and average values per 30-second interval. Enter the Event Hub input alias under FROM, and the Power BI output alias under INTO.

The query is as follows.

SELECT
    System.Timestamp as Time,
    DeviceId,
    CAST(AVG(frompis.Temp) AS float) AS Temp,
    CAST(AVG(frompis.Pressure) AS float) AS Pressure,
    CAST(AVG(frompis.Hum) AS float) AS Hum,
    CAST(AVG(frompis.HiLux) AS float) AS HiLux,
    CAST(AVG(frompis.LowLux) AS float) AS LowLux
INTO
    powerbi
FROM
    raspisensors TIMESTAMP BY Date
GROUP BY
    DeviceId,
    TumblingWindow(second, 30)

When you have finished all settings, click “Start” to start the Stream Analytics job. eventhub

Verifying the data received by Power BI

The data from the Stream Analytics job will arrive in Power BI as a dataset. You can create a report from this data and have it displayed on the dashboard. eventhub

Now you can verify the data that were sent from the Raspberry PI sensor to Microsoft Azure, via SORACOM.

This concludes “Using the SORACOM Air metadata service to obtain IMSI information and connect with a Microsoft Azure Event Hub.” Thank you.

This guide has created kekekekenta the original’s permission, on the basis of the following entries. http://kentablog.cluscore.com/2015/10/soracomazureiot.html

Mr. kekekekenta, Thank you.

Getting Started

SORACOM Air for Cellular

SORACOM Air for Sigfox

SORACOM Beam

SORACOM Canal/Direct/Door

SORACOM Endorse

SORACOM Funnel

SORACOM Gate

SORACOM Harvest

SORACOM Inventory

SORACOM Junction

SORACOM Krypton

SORACOM Lagoon

Service Detail

Developer Tools

pagetop