Tunneling MQTT over SSH

Tunnel image from http://www.imageafter.com/

One approach that can be used to secure protocols that do not have native encryption is to tunnel them over a known secure protocol.

This can be a VPN protocol like IPSec or it can be a secure general purpose protocol like HTTPS/TLS or SSH.

In this post, we will discuss tunneling MQTT over SSH in a simple and a more advanced configuration.

What is SSH?
SSH is a great protocol that is essentially a drop-in replace for many older, insecure protocols.

For example, SSH can(should) replace telnet, FTP, rlogin, rsh, rcp, etc, etc. It's like the secure Swiss Army knife of protocols.

Simple Configuration with Interactive Password
For the most basic implementation of SSH tunneling all you need is SSH installed on both devices.  The end with the service that you want to connect to must offer the SSH service and you need an SSH client on the end you wish to connect from.  You will also, obviously need an account on the server side.

Once you have this and have tested that the basic functionality works, we will need to set up the tunnel.  For this section we will assume that you are using the default ports for SSH (22) and MQTT (1883).  We will also need to select another port for the local tunnel endpoint - let's use 22883 (which is officially unassigned (IANA) yet, you should check your local machine first to make sure it is not in use (i.e., netstat -an)).

Now we need to enter only one command to set this up:

ssh -f -L 22883:127.0.0.1:1883 SSHusername@MQTT -N

'-f' means to fork into the background.
'-L' means to do a local tunnel.
'22883' is our local port number.
'127.0.0.1' is the destination host from the perspective of the tunnel destination.
'1883' is the MQTT port on the destination server.
'SSHusername' is the username that you use to login to the MQTT server's SSH service.
'MQTT' is the name or IP address of the MQTT server.
'-N' tells ssh not to run a command on the remote server.

You will be prompted for your password (for the SSH connection).

Now, if you connect to MQTT from this system using the localhost address and a port of 22883 like this:

mosquitto_sub -h 127.0.0.1 -p 22883 -u dan -P password -t 'test/#'

You will connect to the MQTT broker on your MQTT server - but this time over a secure (encrypted and authenticated) SSH connection.

The problem with what we have so far it that we need to be at the client to set up the tunnel - which is fine if it is our PC - but if it is a server, we probably need a solution that doesn't need an interactive password - but one that also does not sacrifice security.  This is what we will look at next.

Configuration to Support Key-based Authentication
SSH supports a wide variety of authentication natively and many more via the PAM (plugable authentication) interface.  We are going to enable the public/private key based authentication so that we can login to our MQTT server over SSH without providing a password - or leaving a password stored in plain-text.

Since our purpose here is to make this work in an unattended manner, we will not give a passphrase for our private key - keep in mind that this can lead to security problems - do not re-use this key for other purposes/servers and apply additional server side controls.

To generate our key, on the client side, run:

ssh-keygen

Then accept the defaults. If you want a passphrase - this would be the time to add one. Now run:

ssh-copy-id username@host

This copies your key into the .ssh/authorized_keys file.

If you didn't enter a passphrase - as I mentioned before - you'll want to provide additional security on the server side.

Some of those additional server-side controls might be to lock down what logins that use this key can do.  To do this, you can go to the server and edit the .ssh/authorized_keys file adding some options to the beginning of it:

from="192.168.1.*",no-pty,permitopen="127.0.0.1:1883",command="/bin/false" ssh-rsa AAAAB....

This limits logins to the 192.168.1.x network, declines to assign a pty, limits the forwarding to the localhost on the MQTT port, and runs the command "/bin/false" on login.  The point here being to lock down what the passwordless key can do to those things that you want that key to do anyway (no interactive logins, no sftp logins, no tunneling to other servers, no reusing the key on other servers, etc.   The ssh/sshd man pages have more information on other settings that may be useful for enhancing your security.

Once this is all done and you are satisfied with the security - just go back and run your ssh comand from above and you will not be prompted for a password (unless you set a key passphrase).  Now you should be able to use the tunnel as before, but with no password required.

Half-Baked MQTT Publisher for Netduino Plus

UPDATE:  This has been moved to GitHub: https://github.com/danielan/NetduinoMQTT

I received a Netduino Plus for Christmas this year. So, I needed a project to put this to work. After working my way through the O'Reilly/Make book, "Getting Started with the Internet of Things" book, I decided to implement a MQTT publisher for my Netduino Plus.

Now, MQTT is a really cool protocol that can be used for all sorts of things and supports all sorts of neat features.  This isn't that sort of program.  :)

All this does, in a weak way, is connect to a message broker, publish a message without QoS and then disconnect.  This is NOT how you should do it.  This doesn't work for long topics, long messages, etc.  It doesn't retry if there is a problem, etc.

This is intended only for my use - ultimately publishing a temperature from a sensor over and over - but I'm back to work tomorrow, so this will probably be idle for some time.

I hope this helps - but don't use it expecting it to be useful without more work. 

So, enough disclaimers... :)

MQTT

I learned of MQTT from this post at AdaFruit: http://www.adafruit.com/blog/2011/11/08/ibm-open-sources-potential-internet-of-things-protocol/

And the learned more from this Series on "ReadWriteWeb":
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-negotiating-th.php
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-part-2-how-mqt.php
http://www.readwriteweb.com/hack/2011/11/ibms-andy-piper-part-3-the-tax.php

There's also a web site at: http://mqtt.org/

And the quite useful official specification is over here: http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html

Demoing MQTT

To use MQTT you need a "Broker" - I used "Really Small Message Broker" from here: https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=d5bedadd-e46f-4c97-af89-22d65ffee070

To demo this, download rsmb from there and unzip it.  Then open 4 "cmd" windows and make your working directory in the folder "windows" (assuming Windows) in the rsmb directory you just extracted.

In one cmd window run: broker.exe

In the next cmd window run: stdinpub.exe test
In another cmd window run: stdoutsub.exe test --clientid one
In the last cmd window run: stdoutsub.exe test --clientid two

"test" is the name of our topic. The client ids are necessary because, I assume, there is a bug in the windows stdoutsub.exe binary - the names are supposed to be unique by default*, but aren't.

* "Help" for stdoutsub.exe: "--clientid <clientid> (default is hostname+timestamp)"  But in reality they seem to default to "stdout-subscriber" - I'll see if I can find someone to report this to.

Now, if you type something into the stdinpub.exe window - you should see it show up in the two stdoutsub.exe windows.  This is, simplistically, what MQTT is for (lightweight publisher/subscriber model comms).

(These windows will also be useful to see if our program is working.)

The Code

There's a library available for MQTT on all sorts of languages and software, but I couldn't find anything for the Netduino Plus (although maybe the .NET one would work?)

It would probably be a good idea to read through the standard first.  I used hex for flags and numbers where they made more sense.  I started to do constants, but didn't get finished yet...

(Note - the flags are easier to decipher if you remember that for hex you use 4 bits per hex digit.)

While keeping in mind that this is not appropriate for actual use....

Here is the code for the "library".

using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Microsoft.SPOT;
using Socket = System.Net.Sockets.Socket;

namespace Netduino_MQTT_Client_Library
{
    static class Constants
    {
        public const int MQTTVERSION = 3;
    }

    public static class MQTTClient
    {
        // From Sample Code (SockClient.cs) Copyright Microsoft
        public static Socket ConnectSocket(String server, Int32 port)
        {
            // Get server's IP address.
            IPHostEntry hostEntry = Dns.GetHostEntry(server);

            // Create socket and connect to the server's IP address and port
            Socket socket = new Socket(AddressFamily.InterNetwork,
                SocketType.Stream, ProtocolType.Tcp);
            socket.Connect(new IPEndPoint(hostEntry.AddressList[0], port));
            return socket;
        }
        // End of Microsoft Copyright sample code

        public static void ConnectMQTT(Socket mySocket, String clientID)
        {

            int index=0;
            byte[] buffer = null;
            buffer = new byte[16 + clientID.Length];
            // Fixed Header (2.1)
            buffer[index++] = 0x10; // Fixed header flags
            buffer[index++] = (byte)(14 + clientID.Length); // Remaining Length
            // End Fixed Header
            // Connect (3.1) 
            // Protocol Name
            buffer[index++] = 0;       // String (MQIsdp) Length MSB
            buffer[index++] = 6;       // Length LSB
            buffer[index++] = (byte)'M';  // M
            buffer[index++] = (byte)'Q';  // Q
            buffer[index++] = (byte)'I';  // I
            buffer[index++] = (byte)'s';  // s
            buffer[index++] = (byte)'d';  // d
            buffer[index++] = (byte)'p';  // p
            // Protocol Version
            buffer[index++] = Constants.MQTTVERSION;
            // Connect Flags
            buffer[index++] = 0x02;
            //Keep alive (20 seconds)
            buffer[index++] = 0;     // Keep Alive MSB
            buffer[index++] = 20;    // Keep Alive LSB
            // ClientID
            buffer[index++] = 0;                        // Length MSB
            buffer[index++] = (byte)clientID.Length;    // Length LSB
            for (var i = 0; i < clientID.Length; i++) 
            {
                buffer[index++] = (byte)clientID[i]; 
            }
            mySocket.Send(buffer, index, 0);

        }

        public static void PublishMQTT(Socket mySocket, String topic, String message)
        {

            int index = 0;
            byte[] buffer = null;
            buffer = new byte[4 + topic.Length + message.Length];
            // Fixed header
            //      Publish (3.3) fixed header flags
            buffer[index++] = 0x30;
            //      Remaining Length (variable header + payload)
            buffer[index++] = (byte)(2 + topic.Length + message.Length);
            // End of fixed header 
            // Variable header (topic)
            buffer[index++] = 0;                   // Length MSB
            buffer[index++] = (byte)topic.Length;  // Length LSB
            // End of variable header
            for (var i = 0; i < topic.Length; i++)
            {
                buffer[index++] = (byte)topic[i];
            }
            // Message (Length is accounted for in the fixed header)
            for (var i = 0; i < message.Length; i++)
            {
                buffer[index++] = (byte)message[i];
            }
            mySocket.Send(buffer, buffer.Length, 0);
        }

        public static void DisconnectMQTT(Socket mySocket)
        {
            byte[] buffer = null;
            buffer = new byte[2];
            buffer[0] = 0xe0;
            buffer[1] = 0x00;
            mySocket.Send(buffer, buffer.Length, 0);
        }
    }
}

Here is the code to use this library (Don't forget to change to your IP (where broker.exe is running)).

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.NetduinoPlus;
using Netduino_MQTT_Client_Library;

namespace NetduinoMQTT
{
    public class Program
    {
        public static void Main()
        {
            // This is where you would do your work
            Socket mySocket = MQTTClient.ConnectSocket("192.168.1.1",1883);
            MQTTClient.ConnectMQTT(mySocket,"tester123");
            MQTTClient.PublishMQTT(mySocket, "test", "testme12");
            Thread.Sleep(1000);
            MQTTClient.DisconnectMQTT(mySocket);
        }
    }
}

Deploy this to your Netduino Plus and you should see the message, "testme12" appear in your stdoutsub.exe windows (and some connection messages in your broker.exe window).


To Do

If you wanted to use this for anything, probably at a minimum you should account for connect failures, longer topics, longer messages, maybe QoS, maybe authentication, maybe use the SSL version, etc.  Since it is a "library" it would be good to support the whole subscription thing, etc.

Conclusion

So, that's it!  VERY simplistic MQTT publishing from a Netduino Plus to RSMB.  When I get around to setting up my temperature sensor maybe I'll make this a bit more robust and re-visit this.

I hope you enjoyed this post.