Linux User & Developer - - Contents -

While Ar­duinos are mainly used for con­trol­ling hard­ware, they can do a lot more – find out how to use an Ar­duino to cre­ate your own web server which can dou­ble as an IOT de­vice

Ar­duinos are de­signed for hard­ware, but the right model can still power through a lot of com­put­ing

Es­tab­lish­ing an on­line pres­ence is a task eas­ily ac­com­plished with a per­sonal web­site. While web­sites are a great plat­form to show­case your skills, there’s no rea­son we can’t show off our Ar­duino prow­ess and avoid pay­ing money to host con­tent on a server at the same time. This month we are go­ing to be­gin a se­ries of Ar­duino tu­to­ri­als that take the mi­cro­con­troller board out of its com­fort zone and we start by trans­form­ing it into a sim­ple HTTP web server.

Alarm bells should be ring­ing in your head right now: aren’t Ar­duinos meant for hard­ware projects? Well, yes en­tirely. Ar­duinos also have a max­i­mum pro­gram size and a rel­a­tively small amount of mem­ory, mak­ing this dif­fi­cult. We could also do this cheaply with an ac­tual, al­beit small, com­puter such as the Raspberry Pi. We also won’t be able to do much (if any) server-side pro­cess­ing – so no data­base queries. Th­ese are three strong ar­gu­ments against us­ing an Ar­duino for this. How­ever, if you have an Ar­duino Yún then this isn’t the case at all, and if you have a Mega the pro­gram size isn’t an is­sue and ex­tra fea­tures can be added as needed.

The ben­e­fit of us­ing an Ar­duino and pro­gram­ming your own server is that you can also serve other com­mands of your choos­ing. This al­lows your web server to dou­ble up as an IOT de­vice, con­trol­ling items around your home. You could be boil­ing a ket­tle, open­ing the garage door, or even con­trol­ling mo­tors on some kind of ro­bot you built your­self, all feed­ing back to the user on their browser. An Ar­duino server can be­come the gate­way into your world.

Set up the hard­ware

You’ll need an Ar­duino board, a way of con­nect­ing it to your in­ter­net router and SD card read­ing ca­pa­bil­ity. For this tu­to­rial, we used the Ar­duino Leonardo and the Eth­er­net & SD card shield. Im­me­di­ately, you’ll need to in­sert the shield into the Ar­duino’s header pins and con­nect your shield to the router with an Eth­er­net cable. You’ll also need to power your Ar­duino – but for now, plug it into your com­puter with a USB cable.

You should log into your router, iden­tify your de­vice’s lo­cal IP ad­dress and check that you have con­fig­ured your router cor­rectly. If you don’t have ac­cess to the ad­min­is­tra­tor lo­gin, you might be able to de­ter­mine its net­work ad­dress us­ing nmap or arp -a. At the top of the ex­am­ple ‘WEBSERVER’ sketch (un­der Eth­er­net Li­brary sketches), you’ll need to change your IP and MAC ad­dresses to match.

Up­load this sketch, as is, to your Ar­duino. If all is well, you should be able to con­nect to your de­vice from any browser and it should dis­play a sen­sor read­ing. As you haven’t con­nected any­thing to the in­put pin, it’s prob­a­bly go­ing to be zero or just garbage, but that’s okay, we’re go­ing to get rid of it any­way.

Read HTTP re­quests

Open the se­rial mon­i­tor within the Ar­duino IDE and con­nect to your de­vice on your browser. A stream of text should ap­pear. This is the HTTP re­quest that the browser is send­ing to the Ar­duino. At the be­gin­ning is the re­quest method in all caps, fol­lowed by the re­source re­quested. Briefly, the browser is send­ing a re­quest to ask the server to GET a re­source, such as a web­page

(in this case just ‘/’). A web server should then send a re­sponse mes­sage af­ter de­ter­min­ing the na­ture of the re­quest and as­sess­ing re­sources. For ex­am­ple, if a page is not found, it should re­ply ‘HTTP/1.1 404 Page Not Found’. This is a header and would be a re­sponse to a HEAD re­quest. This can be fol­lowed by a mes­sage, such as the main body of a web­page.

This al­lows your web server to dou­ble up as an In­ter­net of Things de­vice, con­trol­ling items around your home

What we are go­ing to do here is em­u­late the server be­hav­iour by re­spond­ing ap­pro­pri­ately to cer­tain HTTP re­quest types. To be­gin, you’ll need to write a func­tion to read the HTTP re­quest let­ter-by-let­ter, sav­ing to a char­ac­ter ar­ray, and paus­ing when you re­ceive a white­space char­ac­ter. You should then con­vert that char­ac­ter ar­ray to a string and com­pare the re­ceived re­quest to the meth­ods your Ar­duino is go­ing to serve.

For now, we’ll deal with GET and HEAD. You’ll also want to store the re­source name.

Serve HTTP re­quests

The GET re­quest is just like the HEAD re­quest, but with an ad­di­tional body that fol­lows the header in­for­ma­tion. To save on pro­gram space, the GET re­quest should call a func­tion that re­turns the header in­for­ma­tion and then finishes off with the mes­sage body. Be­gin by mak­ing a func­tion which can re­ply with the HEAD in­for­ma­tion, as in the first four client.println() calls in the ex­am­ple code. De­pend­ing on the avail­abil­ity of re­sources on the SD card and the re­quest sent, we will want to change this code by pass­ing it in as an ar­gu­ment, but for now we’ll just use the ‘200 – OK’ code.

Now add an­other func­tion which can process GET re­quests. This func­tion should check the SD card for a file with the same path as the re­quested re­source and re­turn an OK code if present, or an er­ror code if not, fol­lowed by the con­tent of the re­source (or an er­ror page). This check can be done with SD.ex­ists(), but to ini­tialise the SD card you’ll need to add a line at the top of the sketch:

#in­clude <SD.h>

To re­turn the con­tent, you also need to de­clare and open the file, read the con­tents (see the SD card ex­am­ple sketches), then print the file con­tents back to the client.

Cre­ate the web­site files

Be­fore we can test the server, you’ll need to cre­ate a set of HTML files for the Ar­duino to send to the client. Th­ese days, us­ing just HTML and CSS alone, it is pos­si­ble to quite quickly make a some­what in­ter­ac­tive, el­e­gant web­site au­to­mat­i­cally com­pat­i­ble with dif­fer­ent dis­play sizes. Be­cause we are deal­ing with GET re­quests ap­pro­pri­ately rather than just send­ing a sin­gle page ev­ery time, as some peo­ple have done, we are able to use the iframe and ob­ject tags and host our own im­ages. How­ever, be­cause you can’t in­stall soft­ware or run scripts on the Ar­duino server, some of the tricks with PHP, JAVASCRIPT and other lan­guages won’t nec­es­sar­ily work. There are also is­sues with speed. Send­ing a 10MB image to the client will take a while. How­ever, cre­at­ing a for­ward-facing min­i­mal­ist web­site is pos­si­ble and can be done in a few hours.

When sav­ing your files, be sure to not use more than three char­ac­ters in the ex­ten­sion and eight char­ac­ters

in the file­name. The SD li­brary won’t al­low more, and it’s very easy to think your code isn’t work­ing rather than as­sume it’s some­thing as sim­ple as this – your au­thor wasted a day on this prob­lem: use .htm. When done, in­sert the SD card into the shield and re­set your Ar­duino.

Test and fix your server

In the main loop you should now be able to com­pare the string com­pris­ing of the re­quested method and the meth­ods served by the Ar­duino and then call the ap­pro­pri­ate func­tion. Up­load this new sketch to the board and try out your web­site. You should be able to use your lo­cal IP ad­dress fol­lowed by the file­name, for ex­am­ple The server isn’t yet fully op­er­a­tional, but we’re get­ting there. As you nav­i­gate, sev­eral prob­lems should be­come ap­par­ent.

First of all, re­gard­less of what hap­pens, the ‘200 – OK’ code is re­turned. Sec­ond, the server is al­ways re­spond­ing with ‘type/html’, even if the Ar­duino is ac­tu­ally send­ing over JPEGs or plain text. Also, if any im­ages are large, you’ll al­ready be ex­pe­ri­enc­ing per­for­mance is­sues and, if too large, you can be de­nied ser­vice al­to­gether.

To fix th­ese prob­lems, you’ll need to cre­ate a func­tion which de­ter­mines if the file is on the SD card and, if so, look at the ex­ten­sion and re­turn a string for the re­sponse code and the re­source type. This should be used in the HEAD method when called. You should also re­turn a 404 er­ror if there is no file, and a 405 er­ror if the method isn’t served by the Ar­duino. Im­ages should be com­pressed and data sent in batches us­ing a buf­fer.

Prune your pro­gram

If you’ve fixed the above prob­lems then pat your­self on the back: you’ve man­aged to cre­ate a minis­erver which can host a ba­sic web­site that, with the right router con­fig­u­ra­tion, can be ac­cessed from the out­side world. Any cool fea­tures, like CSS animations, fil­ters and scripts, will need to be done client side, and any data­base lookups just can’t be done. The good news is that we at least com­ply with some of the HTTP pro­to­col and, for most viewers, this will just seem like a nor­mal web­site. Nev­er­the­less, our func­tion­al­ity is still quite re­stricted. Let’s sup­pose we want some­one to be able to leave com­ments, or use this to run a per­sonal blog. We’re go­ing to need to add in an­other HTTP method: POST.

How­ever, as those us­ing an Ar­duino Uno or Leonardo have prob­a­bly no­ticed, we’re al­ready run­ning up against the max­i­mum pro­gram size, which is lim­ited to 28KB.

The SPI, SD and Eth­er­net libraries take up a lot of the pro­gram space, but we’ve con­trib­uted our fair share too. It’s time to get rid of any un­nec­es­sary print state­ments (although keep print­ing the in­com­ing re­quest to screen),

Cre­at­ing a for­ward­fac­ing min­i­mal­ist web­site is pos­si­ble and can be done in a few hours

and clean out the code. We only need to make a lit­tle bit of room for an ex­tra func­tion and an ex­tra if con­di­tion in the main loop.

Serv­ing POST re­quests

POST re­quests tell the server that a new re­source needs to be cre­ated. On an HTML page th­ese re­quests can be gen­er­ated us­ing the form and in­put tags. Th­ese al­low you to spec­ify the method (e.g. POST) as well as the ac­tion (des­ig­nat­ing the page to be loaded af­ter sub­mis­sion). When a user sub­mits, the HTTP re­quest sends the method, the re­source to load and, at the very end af­ter two line breaks, the mes­sage sub­mit­ted. This mes­sage is in the for­mat ‘field=mes­sage data’.

To serve POST re­quests, you will want to iden­tify the re­quested method and call a new func­tion. This func­tion should read the name of the re­source spec­i­fied by the ac­tion in the form and save it to a spare buf­fer. It should then open a file on the SD card, read from client and save every­thing af­ter the dou­ble-line break. The func­tion

should fin­ish by call­ing the GET func­tion (which also calls HEAD) and retriev­ing the stored re­source name. You may need to mod­ify the ex­ist­ing code and will want the file­name to in­cre­ment. If you still have pro­gram space, you may also want to split the field name from the data in the mes­sage body by check­ing for an = sign and use the field name as the file­name for the new re­source cre­ated.

Load sub­mit­ted posts

To fin­ish the blog or com­ments sec­tion, you’re go­ing to want to load all of the in­for­ma­tion sub­mit­ted to the server and dis­play it on the web­page. One might in­stinc­tively think of stor­ing the in­for­ma­tion in a data­base and ac­cess­ing en­tries as needed. The only prob­lem is, we can’t in­stall soft­ware or try PHP and MYSQL. In­stead, if you’ve got an Ar­duino with larger pro­gram stor­age, you might want to make a cus­tom method which acts like a data­base query. If you’ve got an ‘or­di­nary’ Ar­duino, every­thing’s go­ing to have to be done on the client’s side.

One of your au­thor’s many short­com­ings is not learn­ing JAVASCRIPT, but now is ex­actly the time. In the ex­am­ple web­site is a snip­pet of code within an HTML script-tag which it­er­ates be­tween 0 and 4 and cre­ates a se­ries of ob­ject-tags. Th­ese ob­ject tags point to text files con­tain­ing sub­mit­ted posts. Th­ese are in­serted into a para­graph in the HTML body. When the page is loaded, the browser re­quests five ob­jects sep­a­rately to the .htm file. It’s a cheap trick, and causes other lay­out prob­lems, but the server can cope and the page loads. It would be bet­ter to load the con­tent and lay­out ap­pro­pri­ately – it might even be bet­ter to have most re­cent first. If you know how, do it.

Share with the world

The web server is fin­ished. You can send and re­ceive web­pages, text, im­ages, GIFs even. It should even run rea­son­ably quickly. With a static IP or Dy­nam­icDNS, you could buy a do­main and let peo­ple con­nect to your web­site with­out handing out a se­ries of num­bers.

How­ever, there is still the is­sue of send­ing and re­ceiv­ing ob­scenely large im­ages. This will al­ways be slow – and is even the case with some pop­u­lar web­sites. For now, we can en­sure that we only host small im­ages, but if we re­ally need to we could al­ways link to the re­source on a faster cloud stor­age ser­vice in­stead.

Sadly, there’s no way of stop­ping some­one at­tack­ing our server by up­load­ing a 32GB file – that’s just not been pro­grammed in. If there’s room for it in the pro­gram, you could count the num­ber of char­ac­ters re­ceived and check it doesn’t ex­ceed a limit. If that limit is ex­ceeded, stop sav­ing and re­turn a bad re­quest code. If there’s no space, you could do a client-side check in JAVASCRIPT.

But there are a hun­dred-and-one ways some­one could deny all ser­vice to the Ar­duino server. We’ve also added no se­cu­rity for data stored on the de­vice. How­ever, at least there’s no way of some­one delet­ing in­for­ma­tion or us­ing the Ar­duino to con­nect to other lo­cal de­vices.

Add hard­ware meth­ods

That was Ar­duino truly out of its com­fort zone! Now the web server is com­plete, let’s bring the de­vice back to what it’s good at: in­ter­fac­ing with hard­ware. We can use the web server to act as the in­fra­struc­ture for con­trol­ling the Ar­duino re­motely. To do this you’ll need to cre­ate a new re­quest com­par­i­son con­di­tion within the loop func­tion, as with the other HTTP re­quests. To con­trol the de­vice, you can just send this ar­bi­trary key­word and any fur­ther in­struc­tions over Tel­net (or equiv­a­lent) to the server ad­dress to trig­ger an ac­tion for the Ar­duino to per­form. You could even cre­ate a very sim­ple smart­phone app that sends a pre­set com­mand at the touch of a but­ton.

And what to do with it? At this point the task could be any­thing at all, from turn­ing on a light to wa­ter­ing some flow­ers. De­pend­ing on the size of the web server pro­gram, it might be use­ful to use this Ar­duino as a base sta­tion which then re­lays the com­mand via ra­dio to other Ar­duinos scat­tered around the house. The big­ger the project, the more likely you are go­ing to need to up­grade to the larger Ar­duino. Al­ter­na­tively, you could pig­gy­back off of the POST method and put the world in charge of your hard­ware us­ing a sim­ple HTML form as a con­troller.

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.