Month: May 2018

Last time we looked at how to create your first channel to receive data from your connected IoT device.  This week we’re going to take a closer look at the IoT Core Rule that was created for you last time and then go on to create the Pipeline that will connect this Channel to a Data store where all your data will be available for query.

First though, let’s remind ourselves which Channel we’re going to be working with today by visiting the Channels page in the IoT Analytics Console.

We’ll be using this vibration_channel to collect data from our vibration sensor project.

Last week I mentioned that using the topic filter in the Channel creation flow would automatically create a Rule for you in IoT Core, so let’s mosey on over to the IoT Core Console, select Act from the menu on the left and take a look at what rules we now have.

Cool – we have a rule called IoTAnalytics_vibration_channel. Note now any rules created automatically like this are prefixed by the IoTAnalytics keyword to make them easier to locate. Let’s click on that and take a look at the rule in some more detail.

As you can see, for this rule we are taking all messages from the wildcard topic filter rtjm/# and sending them to the vibration_channel using the IoT Analytics Channel. We can use this screen to edit the topic filter, edit the action or add another action. This last point is worth remembering, you may want to send messages to multiple destinations and adding multiple actions in the Rule is one of the ways of doing this.

For now, let’s head back to the IoT Analytics Channel and create the Pipeline to join everything together.

STEP ONE

We’ll start by naming the Pipeline and selecting the Channel we created time as the source. Every Pipeline needs to get its data from somewhere and so the first mandatory step is to connect the Pipeline to the message source.

Again, you can choose to optionally tag your resources, but for now we’ll just click through with Next.

STEP TWO

On this screen, the console will attempt to find any existing messages in the channel and show you a sample of attributes that have been found. This helper can be useful for some of the later activity steps, but it’s completely optional and I generally just click through with Next to get straight to the next step.

STEP THREE

On this screen you can configure any pipeline activities to transform your messages. I typically go back and do that later if I need to and find it faster to just click on Next to get straight to the final stage. If you already know exactly what you want to do though, you can add various processing activities for your data preparation stage here and we’ll discuss the activities in more detail another time.

STEP FOUR

A Pipeline need to be connected to a Data store in order to process any messages and so this final step requires you to select an existing data store if you have one, or create a new data store if you need to. We don’t have any data stores yet so we will click on create a new data store to finish setting that up.

Give your data store a sensible name for this project as we’ll be needing to refer to it later on.

Clicking on Create data store will create the store for you and take you to the final screen in the Create Pipeline flow.

Finally, after clicking on Create pipeline, we’ll see that everything is setup and the pipeline is complete.

In conclusion, we’ve seen in part one and part two how to create a Channel to receive IoT messages from AWS IoT Core via Rules and how to create a processing Pipeline to connect the Channel to a Data store. Any messages we’re now sending on the MQTT topics we’re listening to will now be stored in the Data store where we can run analysis on them later – but that’s for a future topic.

I have a new IoT project at home to figure out if I can do some frequency analysis on the vibrations coming from our furnace motor to identify possible future failure once I’ve replaced the belt between the motor and the blower fan. I’m going to send the data to IoT Analytics so let’s use this as an example of how to get started.

LAUNCH THE IOT ANALYTICS CONSOLE

Head over to the AWS Console and find IoT Analytics either by searching in the services list or scroll down and you’ll find a collection of IoT solutions grouped together on the right.

If you’re new to IoT Analytics, you’ll see a first time user experience screen that looks like this.

STEP ONE

Create a Channel to receive your messages. You can put messages into a Channel either by using an IoT Rule to send messages arriving to the AWS IoT gateway to an IoT Analytics Channel or you can send data directly using the batch-put-message API if your workflow supports that instead.

So let’s go ahead and click on Create a channel and give our channel an identifier we will use to refer to it later. As this project is all about vibration data, I’ll just call it vibration_channel. Note that you can’t change the identifier of a channel later on but you can tag the resource and use the tags to group and organise your channels and other resources once you start to have a lot of them.

Now scroll down and click on NEXT. We will use the default data-retention period for our raw data which means that we will, in addition to the processed data that you can query, keep the original raw data in case we want to re-process it at a later date. This is one of the more advanced use cases for IoT Analytics and for now we will just proceed with the default settings.

STEP TWO

After clicking Next, we’ll be on the final screen in the Channel creation flow which looks like this.

It’s important to say that you can simply go straight to clicking on Create Channel at this point, especially if you are using batch-put-message to send your data. As a convenience however, if you are sending messages to IoT Core, you can type a topic filter and IoT Analytics will create a Rule in IoT Core for you. You can even see the messages coming in live to that MQTT topic to check that you’re listening to the right topic or to have a look at the shape of the messages.

Let’s try that first and enter a topic and click on See Messages.

You’ll see I’ve used a wildcard topic filter in this case so the first message I’ve received is nothing to do with this particular project, but you get the idea.

STEP THREE

The next thing to point out is super important though. You’ll see that you need to choose or create a role to grant IoT Core the permissions needed to put messages into your channel. Typically I will use create new role here, but you need to be aware that the role that is created is scoped down to just this individual channel. This is important because you might be tempted to re-use this role for future channels – but this will not work as the channels are different. Of course one way around this is to use the IAM console to create a role scoped to any of your channels for example, but that’s for another day.

For this example, I’ll create a new role and then click on Create Channel to complete the process. This will take us back to the Channels page which lists all the Channels we have so far.

Let’s recap what we’ve done so far

1. We’ve created a channel called vibration_channel to collect our data.
2. We’ve created a new IAM role to grant IoT Core permissions to send data to the channel.
3. By creating a topic filter, we’ve automatically created a Rule in IoT Core which will route messages matching the filter to our newly created Channel.

Note that if you were using the AWS CLI or SDK to create the channel, the Rule is not created for you – this is assistance that is only provided by the console (but of course it’s easy to setup the Rule with the CLI or SDK as well).

We’re now ready to send data, but we’ll pick that up in part two.