Monday, January 2, 2012

Half-Baked MQTT Publisher for Netduino Plus

UPDATE:  This has been moved to GitHub: https://github.com/danielan/NetduinoMQTT

I received a Netduino Plus for Christmas this year. So, I needed a project to put this to work. After working my way through the O'Reilly/Make book, "Getting Started with the Internet of Things" book, I decided to implement a MQTT publisher for my Netduino Plus.

Now, MQTT is a really cool protocol that can be used for all sorts of things and supports all sorts of neat features.  This isn't that sort of program.  :)

All this does, in a weak way, is connect to a message broker, publish a message without QoS and then disconnect.  This is NOT how you should do it.  This doesn't work for long topics, long messages, etc.  It doesn't retry if there is a problem, etc.

This is intended only for my use - ultimately publishing a temperature from a sensor over and over - but I'm back to work tomorrow, so this will probably be idle for some time.

I hope this helps - but don't use it expecting it to be useful without more work. 

So, enough disclaimers... :)

MQTT

I learned of MQTT from this post at AdaFruit: http://www.adafruit.com/blog/2011/11/08/ibm-open-sources-potential-internet-of-things-protocol/

And the learned more from this Series on "ReadWriteWeb":
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-negotiating-th.php
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-part-2-how-mqt.php
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-part-3-the-tax.php

There's also a web site at: http://mqtt.org/

And the quite useful official specification is over here: http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html

Demoing MQTT

To use MQTT you need a "Broker" - I used "Really Small Message Broker" from here: https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=d5bedadd-e46f-4c97-af89-22d65ffee070

To demo this, download rsmb from there and unzip it.  Then open 4 "cmd" windows and make your working directory in the folder "windows" (assuming Windows) in the rsmb directory you just extracted.

In one cmd window run: broker.exe

In the next cmd window run: stdinpub.exe test
In another cmd window run: stdoutsub.exe test --clientid one
In the last cmd window run: stdoutsub.exe test --clientid two

"test" is the name of our topic. The client ids are necessary because, I assume, there is a bug in the windows stdoutsub.exe binary - the names are supposed to be unique by default*, but aren't.

* "Help" for stdoutsub.exe: "--clientid <clientid> (default is hostname+timestamp)"  But in reality they seem to default to "stdout-subscriber" - I'll see if I can find someone to report this to.

Now, if you type something into the stdinpub.exe window - you should see it show up in the two stdoutsub.exe windows.  This is, simplistically, what MQTT is for (lightweight publisher/subscriber model comms).

(These windows will also be useful to see if our program is working.)

The Code

There's a library available for MQTT on all sorts of languages and software, but I couldn't find anything for the Netduino Plus (although maybe the .NET one would work?)

It would probably be a good idea to read through the standard first.  I used hex for flags and numbers where they made more sense.  I started to do constants, but didn't get finished yet...

(Note - the flags are easier to decipher if you remember that for hex you use 4 bits per hex digit.)

While keeping in mind that this is not appropriate for actual use....

Here is the code for the "library".

using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Microsoft.SPOT;
using Socket = System.Net.Sockets.Socket;

namespace Netduino_MQTT_Client_Library
{
    static class Constants
    {
        public const int MQTTVERSION = 3;
    }

    public static class MQTTClient
    {
        // From Sample Code (SockClient.cs) Copyright Microsoft
        public static Socket ConnectSocket(String server, Int32 port)
        {
            // Get server's IP address.
            IPHostEntry hostEntry = Dns.GetHostEntry(server);

            // Create socket and connect to the server's IP address and port
            Socket socket = new Socket(AddressFamily.InterNetwork,
                SocketType.Stream, ProtocolType.Tcp);
            socket.Connect(new IPEndPoint(hostEntry.AddressList[0], port));
            return socket;
        }
        // End of Microsoft Copyright sample code

        public static void ConnectMQTT(Socket mySocket, String clientID)
        {

            int index=0;
            byte[] buffer = null;
            buffer = new byte[16 + clientID.Length];
            // Fixed Header (2.1)
            buffer[index++] = 0x10; // Fixed header flags
            buffer[index++] = (byte)(14 + clientID.Length); // Remaining Length
            // End Fixed Header
            // Connect (3.1) 
            // Protocol Name
            buffer[index++] = 0;       // String (MQIsdp) Length MSB
            buffer[index++] = 6;       // Length LSB
            buffer[index++] = (byte)'M';  // M
            buffer[index++] = (byte)'Q';  // Q
            buffer[index++] = (byte)'I';  // I
            buffer[index++] = (byte)'s';  // s
            buffer[index++] = (byte)'d';  // d
            buffer[index++] = (byte)'p';  // p
            // Protocol Version
            buffer[index++] = Constants.MQTTVERSION;
            // Connect Flags
            buffer[index++] = 0x02;
            //Keep alive (20 seconds)
            buffer[index++] = 0;     // Keep Alive MSB
            buffer[index++] = 20;    // Keep Alive LSB
            // ClientID
            buffer[index++] = 0;                        // Length MSB
            buffer[index++] = (byte)clientID.Length;    // Length LSB
            for (var i = 0; i < clientID.Length; i++) 
            {
                buffer[index++] = (byte)clientID[i]; 
            }
            mySocket.Send(buffer, index, 0);

        }

        public static void PublishMQTT(Socket mySocket, String topic, String message)
        {

            int index = 0;
            byte[] buffer = null;
            buffer = new byte[4 + topic.Length + message.Length];
            // Fixed header
            //      Publish (3.3) fixed header flags
            buffer[index++] = 0x30;
            //      Remaining Length (variable header + payload)
            buffer[index++] = (byte)(2 + topic.Length + message.Length);
            // End of fixed header 
            // Variable header (topic)
            buffer[index++] = 0;                   // Length MSB
            buffer[index++] = (byte)topic.Length;  // Length LSB
            // End of variable header
            for (var i = 0; i < topic.Length; i++)
            {
                buffer[index++] = (byte)topic[i];
            }
            // Message (Length is accounted for in the fixed header)
            for (var i = 0; i < message.Length; i++)
            {
                buffer[index++] = (byte)message[i];
            }
            mySocket.Send(buffer, buffer.Length, 0);
        }

        public static void DisconnectMQTT(Socket mySocket)
        {
            byte[] buffer = null;
            buffer = new byte[2];
            buffer[0] = 0xe0;
            buffer[1] = 0x00;
            mySocket.Send(buffer, buffer.Length, 0);
        }
    }
}

Here is the code to use this library (Don't forget to change to your IP (where broker.exe is running)).


using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.NetduinoPlus;
using Netduino_MQTT_Client_Library;

namespace NetduinoMQTT
{
    public class Program
    {
        public static void Main()
        {
            // This is where you would do your work
            Socket mySocket = MQTTClient.ConnectSocket("192.168.1.1",1883);
            MQTTClient.ConnectMQTT(mySocket,"tester123");
            MQTTClient.PublishMQTT(mySocket, "test", "testme12");
            Thread.Sleep(1000);
            MQTTClient.DisconnectMQTT(mySocket);
        }
    }
}

Deploy this to your Netduino Plus and you should see the message, "testme12" appear in your stdoutsub.exe windows (and some connection messages in your broker.exe window).


To Do

If you wanted to use this for anything, probably at a minimum you should account for connect failures, longer topics, longer messages, maybe QoS, maybe authentication, maybe use the SSL version, etc.  Since it is a "library" it would be good to support the whole subscription thing, etc.

Conclusion

So, that's it!  VERY simplistic MQTT publishing from a Netduino Plus to RSMB.  When I get around to setting up my temperature sensor maybe I'll make this a bit more robust and re-visit this.

I hope you enjoyed this post.

9 comments:

  1. Hey! Really nice work. I'll put a link to this on the mqtt.org Software page... I only have Arduinos myself, no Netduinos (although I do have the O'Reilly Getting Started with the IoT book) so I can't test it out.

    Worth noting that you're only doing QoS 0 here (same as the equivalent Arduino library) and no retained publication by the look of it. But it's a nice start, good work - looking forward to see how this gets used!

    ReplyDelete
  2. Hi, Thanks for the encouragement!

    Now I feel guilty for posting this in such a state. :)

    I'll try to get it cleaned up a bit this weekend. I'll at least get the 128 byte limit removed (do multi-byte remaining length right)...

    ReplyDelete
  3. I got full length message support and a little bit of error checking added tonight.

    Tomorrow night I'll take care of long topic names.

    Then I'll update this post.

    ReplyDelete
  4. There are contact details for RSMB in the text when you run it if you want to report the bugs you found.

    I also would like to do some blatant advertising - you may find my MQTT broker mosquitto provides a nicer Windows experience than RSMB and the command line clients don't have the client id bug either :)

    If you get chance I think it'd be very valuable to know if the existing .net clients work on the netduino. I'm not suggesting you stop your own work though!

    ReplyDelete
  5. Hi, Yes, I found where they had cleverly, conspicuously hidden this prominently in the usage message - about 15 minutes after I found Ian's contact info via developerWorks. :)

    I'll give mosquitto a try. Without the words "really small" in the name, I figured it might be more difficult. :)

    You're right - I'll try those other libs one day. Not sure when - I have a couple projects going on.

    ReplyDelete
  6. I did a bunch of updates tonight and moved this to github. https://github.com/danielan/NetduinoMQTT

    ReplyDelete
  7. I'm leaving the mqtt.org software page pointing at this post for now since it provides the best "documentation" but since it points at the code in github so that seems to work :-) thanks.

    ReplyDelete
  8. Fair enough. I'll try and get a "page" here and a decent README for GitHub together.

    I think I still want to get the user/password and maybe the "will" stuff setup. After that, I should have some time to work on docs.

    ReplyDelete
  9. So, it turns out that the "will" stuff is QoS > 0 so I'm skipping that for now.

    I added a bunch of other stuff though, connack, pings, UTF8, etc. I also made a "page" here:

    http://diabolicalws.blogspot.com/p/netduinomqtt.html

    Please link to there if you would.

    ReplyDelete