The “Internet of Things” (IoT) is not only a vision for future communication and interaction systems, but an active and emerging field of research in both academia and industry. IoT is not just a new term for “remote controlling a device over the Internet”; IoT is a concept of multiple devices communicating and interacting with each other and the environment mostly without human interaction. There can be human interaction, but it is considered to be a subset of the interaction with the environment. This introduces new requirements on the communication protocol, such as machine-interpretable data and – especially if small embedded devices shall be included into the network – resource restrictions. This article is focused on the implications of the latter, and shows a possible solution based on two small examples.

The Drawbacks of HTTP

Thinking of “the Internet”, many people – including those with a technical background – think of the Hypertext Transfer Protocol (HTTP) that is used to transport information mostly in HTML- and XML-Files. HTTP is great for the communication between powerful devices from Smartphones up to large Serverfarms. But, HTTP has its limitations: As it has been presented in an earlier Blog-Post, the resource usage and overhead is not negligible, especially not in an embedded setting. When an Ethernet-based protocol shall be used to send and receive live information data that is characterized through frequent but very small messages, the ratio between overhead and intended Information gets even worse. So HTTP is not in every case the protocol of your choice.

About MQTT

The MQ Telemetry Transport (MQTT) Protocol is a lightweight TCP/IP based messaging system. It is mainly characterized through it’s simple, yet powerful subscribe-publish mechanism: Every client can subscribe to topics that are published on the network. Each time, a client publishes a message to a specific topic, all subscribers receive this message and are able to use the information to their needs. Besides the clients, a server – “broker” in the MQTT terms – manages the topics and the subscribers. This enables dynamic adding and removing of clients. As the message headers are only a few bytes, the communication overhead is reduced to a minimum. This leads to a minimal network load and client resource usage. Furthermore, MQTT supports different encryption and authentication mechanisms, as well as message delivery control by applying a QoS-Flag (Quality of Service) to each message.

These characteristics qualify MQTT to be seriously taken into consideration when designing Internet of Things applications.

There are several implementations of MQTT available that target different applications and architectures. Below, two examples of the usage of existing implementations are shown. Additionally MQTT is supported by several middleware solutions, such as RabbitMQ or Spring.

Example 1: MQTT on an Arduino

In a first example, an implementation of the MQTT protocol on an Arduino Ethernet R3 shall be illustrated. For this, the PubSubClient by Nick O’Leary can easily be used. This Client, combined with the standard Arduino Ethernet API, lets you implement a responsive client in just a few lines of code that uses only a fraction of the very limited resources. An implementation of the same functionality with a HTTP-Server would have used about half the Arduino’s resources. When you add just a little functionality, a HTTP-Server runs into the limits of an embedded platform.

Example 2: MQTT on a Raspberry Pi

Although a Raspberry Pi offers enough resources to host a Webserver, MQTT can also be used there. For example, when you use a Raspberry to collect information from Arduino-based sensors for logging and user presentation. Assuming the Raspberry Pi is using Rasbian or another Linux-Distribution, mosquitto is a useful framework for the usage of MQTT. With the C++-Wrapper class mosqpp::mosquittopp, you are given the full functionality of the MQTT protocol. Just derive a class from the base class and implement the client-specific behaviour, and you’re ready to go.

In a small Investigation, Noser Engineering explored the possibilities of using an Arduino embedded platform to host a HTTP-Service in order to interact with a remote system. The Webserver shall be able to read Files from an SD-Card and send them to the client. The default index.htm-Page shall enable the presentation of live information and manipulation options of the used IO pins.

Hardware

For the implementation an Arduino Ethernet Board R3 and an off-the-shelf SD card were used. For IO an LED was connected to digital/PWM output 6 and a potentiometer was connected to analog input 0. The LED shall blink in a defined frequency and a defined brightness. The brightness is set over a corresponding PWM percentage by the potentiometer. The frequency can be modified using the web interface.

Software (Arduino)

General

The server software was not implemented using the Arduino IDE and .ino sketches, but the Arduino Core Library as well as the Ethernet, SPI and SD Library have been ported for C++ development with Eclipse as described in the Arduino and Eclipse Tutorial. When the non-core libraries are linked, the linking order is crucial for successful builds, due to dependencies. This order seems to work well, while other orders did not link correctly:

-lArduinoEthernet -lArduinoSD -lArduinoSPI –lArduinoCore –lm

For SD file access the existing Arduino SD Library has been chosen and as basis for the webserver the Webduino framework was used.  Therefore the following headers and global definitions are needed

Setup Routine

First the usual stuff is initialized during setup(). In order to have this run correctly, the correct output pin has to be selected for the SD SPI access. As the whole program is on the edge of the Arduino Ethernet’s capability, you can save about 4-5kB of program memory (and some SRAM as well), when the Arduino is equipped with a fixed IP address instead of calling the DHCP routines. After the network has been initialized the request handlers are linked to the server object and the server is started. Finally, the interrupt frequency is defined.
For SD file access the existing Arduino SD Library has been chosen and as basis for the webserver the Webduino framework was used. Therefore the following headers and global definitions are needed:

Interrupt Service Routine

For the blinking LED an interrupt service routine (ISR) is defined, such that the blinking frequency is always correct and not infringed by e.g. heavy loads on the Ethernet communication. The period length dt is used for logging/user output purposes, otherwise the ISR itself is rather simple and straight forward.

Main Loop Function

During the main loop function loop(), the analogue input value from the potentiometer is read and mapped to the corresponding PWM value, as well as the HTTP request processing is generally handled by the function provided by Webduino.

Http Request processing

For the actual output of the data, to handling functions are linked to the server. HttpIndexCmd() is called when the index page is requested, HttpDataCmd() is called by requests to the “file” data.xml, which is generated by the server based on the IO data and not read from SD card. For all other requests, HttpDefaultCmd() is called.

Index

The index function directly opens the helper function, that handles the file access and data transmission HttpReadFile() with the hard-coded filename “www/index.htm”.

IO Data

The data response function first processes POST calls and then generates the response XML (for both GET and POST requests). In the POST processing, firstly the parameters are read individually and then compared against the known commands. In this case “F_blink” is the only known command. It resets the interrupt timer to match the new blinking frequency.
During the XML generation, alternatingly mark-up and data is written to the client, after a corresponding HTTP header is sent.

Other Resources

If any other resource is called, the function HttpDefaultCmd() first generates the full path and filename to the corresponding resource on the SD card, and then hands this path over to the helper function. In order to create the full path first the number of items (subdirectories plus file name) in the URL is counted and then a string to be filled with the full URL is initialized with the prefix, which corresponds to the directory in the SD Card, where the accessible files are stored. (In this case “www/”). Then the full URL is generated by reading all URL items. If the last item is empty, this means that a directory was requested, and therefore “index.htm” is appended to the filename.

SD File Access

The function HttpReadFile() reads the requested file from the SD Card and sends it to the Client. As a first step, the requested file is opened. If that fails, a 404 response is sent. Otherwise, the file type is determined from the file name suffix and a corresponding http-header is sent to the client. Afterwards, the file is read in chunks of 32 bytes, which then are sent directly to the client. Once the file transmission is completed, the file is closed and released again.
Note: All file and directory names have to follow the 8.3 naming pattern. Further, the SD card has to be formatted in FAT16 or FAT32. Otherwise, any SD file access will fail.

Modifications to the Webduino Framework

For the program to work completely and in an efficient manner, some adaptations to the Webduino framework have been made.
As the function WebServer::httpSuccess() only accepts parameters from RAM, this can lead to high memory usage, especially when long additional headers shall be transmitted in order to control the caching behaviour of connecting clients. Therefore a function WebServer::httpSuccessP() has been added, that reads the parameters from program memory instead of the SRAM. In contrast to the function WebServer::httpSuccess() the new function forwards the parameter to WebServer::printP() instead of WebServer::print(). This happens for both parameters, so both have to be defined using the P() macro. Further contentType has been declared as a required parameter, whereas the default value for extraHeaders is still defined as NULL.

As the webpages will be loaded from SD Card instead of generation by the program, the functions WebServer::outputCheckboxOrRadio(), WebServer::checkBox() and WebServer::radioButton() are not used and can be deleted from – or commented out of – the Webduino source code in order to save some memory. To further save RAM, in all Serial.print() respectively Serial.println() calls the F() macro can be added if the content is static.
Further a function has been added to send 404 errors to the client, if a file cannot be read from the SD card.

Web Interface

In order to reduce the Ethernet load and RAM-usage on the Arduino, and because the server only provides live data in XML and not in HTML form, a web interface is implemented in a simple AJAX-Script, that regularly requests the Live-Data-XML and parses the data for presentation on client side.

Conclusion

With the Webduino-Framework Noser Engineering implemented a HTTP-Server, that has a footprint of less than 32kB and uses less than 2kB of RAM. Besides the actual Web-server, the Implemented program is also able to perform soft-real-time I/O-Operations in parallel. The largest limitations is, that the capabilities of the chosen Arduino-Platform are completely used and not further expandable.