Net­work­ing Soft­ware net­works..........

The Dark Lord of Net­work Op­er­a­tions, Tim Arm­strong, sheds some light on soft­ware-de­fined net­work­ing and open net­work switches. Fear his power!

Linux Format - - CONTENTS -

See how Tim Arm­strong makes ex­ceed­ingly light work of soft­ware-de­fined net­work­ing.

The rise of open net­work switches and Soft­ware De­fined Net­work­ing (SDN) has be­gun a par­a­digm shift in dat­a­cen­tres, and with a global short­age of net­work en­gi­neers now is the time to get on the band­wagon.

Deep in the heart of both the Linux ker­nel team and the en­gi­neer­ing depart­ment at Mel­lanox, en­gi­neers have been work­ing hard to bring the big­gest change to Linux net­work­ing since In­tel opened the source code for the e1000 driver. And just like that driver changed the face of Linux net­work­ing 10 years ago, SwitchDev does it again. It em­bod­ies the be­gin­ning to true SDN to the Linux com­mu­nity, en­abling com­plete con­trol of a switch’s hard­ware with the tools you use ev­ery day for man­ag­ing any Linux ma­chine’s net­work stack.

SwitchDev pro­vides an ab­strac­tion layer over the switch’s hard­ware, mak­ing it pos­si­ble to con­fig­ure your switches as if they’re noth­ing more than a Linux server with an enor­mous num­ber of NICs. All the heavy lift­ing is han­dled for you by the ker­nel: bridges con­verted into FIBs, in­ter­faces into ports, VLANs are… well VLANs, and all the for­ward­ing and rout­ing is off-loaded onto the hard­ware au­to­mat­i­cally.

Net­work em­u­la­tion with GNS3

To open this tu­to­rial up to ev­ery­one who doesn’t hap­pen to have an £8,000 Mel­lanox Switch just sit­ting around do­ing noth­ing, we’re go­ing to use GNS3 and some vir­tu­alised ap­pli­ances. Be­cause Mel­lanox’s driv­ers have been up­streamed to the Linux ker­nel tree our Open Net­work Switch ap­pli­ance and all the ker­nel we’ll be us­ing is up to date and com­pat­i­ble with the ma­jor­ity of fea­tures in the Mel­lanox Spec­trum ASIC.

Specif­i­cally we con­fig­ured the switch ap­pli­ance to be as close as pos­si­ble to a Mel­lanox SN2100 as is cur­rently pos­si­ble in GNS3. As for the dis­tro that we’re go­ing to use on the ap­pli­ances, we’re go­ing to go with De­vaun be­cause it’s light­weight, sta­ble and se­cure.

In­struc­tions for in­stalling GNS3 on pretty much any OS are avail­able on, but for sim­plic­ity we’ve in­cluded the Ubuntu in­struc­tions here: $ sudo add-apt-repos­i­tory ppa:gns3/ppa $ sudo apt-get up­date $ sudo apt-get install gns3-gui gns3-server

Be­cause the im­ages we’re us­ing are KVM/QEMU, you’ll also need to install qemu and lib­virtd again. We’ve in­cluded the in­struc­tions for Ubuntu be­low, but you can find in­struc­tions for your pre­ferred dis­tro quite eas­ily on­line. $ sudo apt-get install qemu-kvm qemu-sys­tem-x84 qemu $ sudo apt-get install lib­virt-bin

Now GNS3 is in­stalled we need to import our ap­pli­ances: a GNS3 com­pat­i­ble fac­sim­ile of the Mel­lanox SN2100, a sim­ple De­vuan server, and a De­vuan desk­top. These will be down­loaded au­to­mat­i­cally. When you open GNS3 for the first time you’ll be prompted with a wiz­ard that will guide you through im­port­ing ap­pli­ances, and be­cause this is in­cred­i­bly straight­for­ward we won’t du­pli­cate the in­struc­tions here.

Plug­ging and play­ing

For this tu­to­rial we want to keep things sim­ple so as to keep the fo­cus on Linux, or rather more specif­i­cally SwitchDev, with­out go­ing off into an over­whelm­ing dis­ser­ta­tion on the mer­its and is­sues of dif­fer­ent topolo­gies. To this end we’ll use only one switch, two clients and one server.

To do this sim­ply se­lect the switch drawer on the left and drag an OpenNet­workSwitch into the mid­dle of the work area. Next, drag a cou­ple of De­vuan Desk­tops and a De­vuan Server from the End de­vices drawer into the work area. Ar­range your de­vices around the switch how­ever you feel com­fort­able.

To cre­ate a con­nec­tion from Server to the Switch se­lect the Add a Link tool from the bar on the left, then click thse server, se­lect eno1, and then click the switch se­lect­ing SW1P1. Now do the same for Desk­tops 1 and 2 se­lect­ing SW1P11 and SW1P12, re­spec­tively. Then click the “Add a Link” tool again to dis­able it.

Once all the de­vices are con­nected cor­rectly we can start them up. Press the play but­ton on the top tool­bar. The VMs will take a short time to launch; you can mon­i­tor the progress of the boot by open­ing the OpenNet­workSwitch’s con­sole. Open the con­sole by right-click­ing the de­vice icon and se­lect­ing Con­sole. Once booted you can lo­gin with the user root and the pass­word nauved. Once logged in you can see how the in­ter­faces are pre­sented with the ip com­mand: $ ip link 1: lo: <LOOP­BACK,UP,LOWER_UP> mtu 65536 qdisc

no­queue state UN­KNOWN mode DE­FAULT group de­fault

link/loop­back 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: mgmt0: <BROAD­CAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DE­FAULT group de­fault qlen 1000

link/ether 00:75:f0:86:64:00 brd ff:ff:ff:ff:ff:ff 3: sw1p1: <BROAD­CAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfi­fo_­fast mas­ter br0 state UP mode DE­FAULT group de­fault qlen 1000

link/ether 00:75:f0:86:64:01 brd ff:ff:ff:ff:ff:ff ... 18: sw1p16: <BROAD­CAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DE­FAULT group de­fault qlen 1000

link/ether 00:75:f0:86:64:10 brd ff:ff:ff:ff:ff:ff

Be­cause Mel­lanox’s Spec­trum series ASIC sup­ports ev­ery­thing from 100Gbps down to 1Gbps it’s nor­mally help­ful to set the port speed be­fore ac­ti­vat­ing any in­ter­faces or in­clud­ing them in a bridge. In this case we’re us­ing a vir­tu­alised en­vi­ron­ment so this isn’t re­quired. How­ever, if for ex­am­ple we were go­ing to use a 1Gbps con­nec­tion for each desk­top and a 10Gbps con­nec­tion for the server, then we could con­fig­ure that by set­ting the fol­low­ing: $ eth­tool -s sw1p1 speed 100000 au­toneg off $ eth­tool -s sw1p11 speed 1000 au­toneg off $ eth­tool -s sw1p12 speed 1000 au­toneg off

Bridg­ing the gap

Now that the ports are all set to the right speeds we can cre­ate the bridge. This is largely the same as cre­at­ing any Linux bridge on De­bian-based sys­tems (such as De­vuan). This is achieved by edit­ing /etc/net­work/in­ter­faces.

In or­der to as­so­ciate a switch port with a bridge you need to de­cide on what type of as­so­ci­a­tion it has with the bridge. At the time of writ­ing Linux’s VLAN-aware bridge sys­tem isn’t par­tic­u­larly user friendly when it comes to at­tach­ing ac­cess ports, there­fore it’s rec­om­mended that you use clas­sic bridges. If a given in­ter­face needs to be a trunk port you also need to de­fine each VLAN at the port level and at­tach it to each of the VLANs (bridges). In this ex­am­ple how­ever, we’re keep­ing it sim­ple, so we won’t be us­ing any trunk ports.

Let’s set up one bridge for each of the ports we’re us­ing, so that we have the op­tion to at­tach more de­vices to each sub­net at a later date with­out hav­ing to bring ex­ist­ing con­nec­tions down. Ap­pend the fol­low­ing to the /etc/ net­work/in­ter­faces file: auto br0 iface br0 inet static bridge-ports sw1p1 bridge-stp off auto br1 iface br1 inet static bridge-ports sw1p11 bridge-stp off auto br2 iface br2 inet static bridge-ports sw1p12 bridge-stp off

Each bridge acts as a vir­tual seg­ment in the switch and en­slaves the ports we’re us­ing to the seg­ment. This au­to­mat­i­cally al­ters the port-map­ping and FIB.

Bring­ing each bridge up now would en­able Layer 2 switch­ing on that seg­ment, which would be an ac­cept­able so­lu­tion if we wanted a Layer 2 so­lu­tion, but Layer 2 net­works aren’t very scal­able, and no-one buys a £8,000 switch to do some­thing a £800 switch can do. Be­cause we want to work with mod­ern ar­chi­tec­tures, we’ll need to en­able Layer 3 switch­ing and In­ter-VLAN/cross-seg­ment rout­ing. On a SwitchDev plat­form it’s as sim­ple as adding the rel­e­vant IP def­i­ni­tions on the bridges. To do this just mod­ify /etc/ net­work/in­ter­faces: auto br0 iface br0 inet static bridge-ports sw1p1 bridge-stp off ad­dress net­mask 24 auto br1 iface br1 inet static bridge-ports sw1p11 bridge-stp off ad­dress net­mask 24 auto br2 iface br2 inet static bridge-ports sw1p12 bridge-stp off ad­dress net­mask 24

Now we can start bring­ing in­ter­faces up. $ ifup br0 $ ifup br1 $ ifup br2

Ugh... clients

Now the switch is con­fig­ured it’s time to con­fig­ure the server and the clients. This is fairly stan­dard boil­er­plate but for ver­bosity it bears re­peat­ing. On the server you’ll want to edit /etc/net­work/in­ter­faces, set­ting the eth0 IP in the range of to and the gate­way to In CIDR no­ta­tion this would be (ex­clud­ing the net­work ad­dress [0], broad­cast [255]). This would re­sult in a con­fig block sim­i­lar to the fol­low­ing: auto eth0 iface eth0 inet static ad­dress net­mask 24 gate­way

Next, we need to do the same for the two desk­tops, mod­i­fy­ing the IP ad­dresses ap­pro­pri­ately. For ex­am­ple, the con­fig block for Desk­top 1 would be as fol­lows: auto eth0 iface eth0 inet static ad­dress net­mask 24 gate­way Once we have both the server and the two desk­tops con­fig­ured we can bring the net­works up as nor­mal: $ ifup eth0 Fi­nally, we can test the con­nec­tion from each end-de­vice to the switch. From the Server to the switch: $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.359 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.493 ms From Desk­top 1 to the switch: $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.513 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.471 ms From Desk­top 2 to the switch: $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.314 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.499 ms Fi­nally from the Server to the each Desk­top: $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.712 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.520 ms $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.849 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.591 ms

You should now see all the con­nec­tions are func­tion­ing cor­rectly and our switch is han­dling the rout­ing for us nicely.

So what’s re­ally hap­pen­ing here? Well, in our sim­u­la­tion, noth­ing much: we ef­fec­tively just cre­ated a Linux-based router, but had this been a SwitchDev com­pat­i­ble switch it would be a very dif­fer­ent story. We just did some­thing that a few years back would have re­quired two NDAs and an SLA just to get the SDK. We con­fig­ured a switch, with Linux, us­ing the tools we’ve known for years, with­out a sin­gle bi­nary blob in sight. Not even a shim, just pure open source Linux. Time to stand back and ad­mire your hand­i­work.

En­ter­ing the real world

For those who have con­fig­ured Linux bridges be­fore, all this will have seemed very fa­mil­iar, but that’s the point. SwitchDev is sup­posed to make things easy, by re­duc­ing com­pli­cated switches CLIs, APIs and SDKs into sim­plis­tic Linux com­mands that just work. You install your dis­tro of choice and then get on with it.

What’s more, you can fully au­to­mate the process us­ing your favourite tools such as Chef, An­si­ble or Pup­pet. If it can con­fig­ure a Linux net­work stack it can con­fig­ure a SwitchDev com­pat­i­ble switch. Thanks to the en­gi­neers at Mel­lanox and the Linux Ker­nel De­vel­op­ers, SDN is no longer com­pli­cated, ex­pen­sive or re­liant on spe­cialised black box con­troller units.

Un­like other SDN tech­nolo­gies such as OpenFlow, SAI and OpenNSL, SwitchDev is keep­ing the brains in the box, re­sult­ing in net­work la­ten­cies in the range of tenths of a mil­lisec­ond − all while sim­pli­fy­ing the im­ple­men­ta­tion of mas­sively re­dun­dant net­work topolo­gies.

Fur­ther­more, thanks to SwitchDev the con­ver­gence of Switches and Routers be­comes pos­si­ble, re­sult­ing in a sim­pli­fied net­work edge where one de­vice sup­ports both func­tions. Adding BGP sup­port to your switch is as sim­ple as in­stalling Quagga and hey presto, you have all the most pop­u­lar rout­ing pro­to­cols avail­able.

With any luck the pi­o­neer­ing work done by the Linux Ker­nel de­vel­op­ers and the en­gi­neers at Mel­lanox will be the be­gin­ning of the end for the lay­ers of bu­reau­cracy and li­cens­ing re­quired to gain ac­cess to Switch APIs.

Be­yond the ba­sics

Now that all the ba­sics are up and run­ning, let’s tackle a more re­al­is­tic net­work, split­ting the re­spon­si­bil­i­ties of the server(s) and the desk­tops into a closer ap­prox­i­ma­tion of a mid-size com­pany net­work (not quite as com­plex as an En­ter­prise net­work, but it’s got most of the build­ing blocks in place).

To start off, be­cause we’re us­ing KVM/QEMU as our hy­per­vi­sor we need to shut down our ap­pli­ances be­fore we can re­wire our net­work in GNS3. To do this hit the big stop

but­ton on the top tool­bar in GNS3. Next, drag an­other switch from the ap­pro­pri­ate drawer on the left onto the workspace.

Now click the path be­tween the first switch and the server and press the delete key. Then pull up the Add a Link tool, con­nect the server to the new switch on sw1p10, and con­nect

sw1p1 from the first switch to sw1p1 the sec­ond switch. Then start the Ap­pli­ances again with the big play but­ton.

Re­con­fig­ur­ing ports

Af­ter the switches have booted up it’s time to re­con­fig­ure the in­ter­faces. First we need to ad­just the IP range of the VLAN as­so­ci­ated with sw1p1 on the orig­i­nal switch so that we don’t have to re­con­fig­ure the server also. Let’s have the switches talk to each other over the range. To do this, edit the br0 def­i­ni­tion in /etc/net­work/in­ter­faces: auto br0 iface br0 inet static bridge-ports sw1p1 bridge-stp off ad­dress net­mask 24

Then sim­ply restart the in­ter­face. $ if­down br0; ifup br0

Now let’s take a look at the con­fig­u­ra­tion for the new server switch. We con­nected sw1p1 from the desk­top switch to the sw1p1 port on the server switch. So we need to add a con­fig for that port that’s in the same range, for ex­am­ple: auto br0 iface br0 inet static bridge-ports sw1p1 bridge-stp off ad­dress net­mask 24

Next we need to con­fig­ure the server switch port. We’ll cre­ate a VLAN-aware bridge and en­slave the port so that the server is con­nected to the bridge, and then en­able L3 on that VLAN. auto br1 iface br1 inet static bridge-ports sw1p10 bridge-stp on ad­dress net­mask 24

Once we’ve saved that con­fig we can bring those in­ter­faces up and test the con­nec­tion to the server: $ ifup br0 $ ifup br1 $ ping PING ( 56(84) bytes of data. 64 bytes from icm­p_seq=1 ttl=64 time=0.312 ms 64 bytes from icm­p_seq=2 ttl=64 time=0.314 ms

You might have no­ticed we haven’t put any routes in the switches. As a re­sult we can’t ping from the server to the desk­tops or vice versa. Be­cause we don’t want to end up man­ag­ing a grow­ing mess of static routes, we’re go­ing to use OSPF to safely dis­trib­ute and syn­chro­nise our in­ter­nal rout­ing ta­bles. As we weren’t us­ing VRFs in this tu­to­rial, set­ting up OSPF via quagga is quite sim­ple. Start by en­ter­ing the quagga ter­mi­nal: $ vtysh

The brings us into a read-only mode, where we can use the show com­mand to in­spect the var­i­ous rout­ing ta­bles and links. To en­ter a write mode so that we can add our OSPF con­fig, run the fol­low­ing: $ con­fig­ure ter­mi­nal

Now we’re in the con­fig­ure mode en­abling OSPF and dis­tribut­ing our lo­cally at­tached routes is as sim­ple as: $ router ospf $ ospf router-id $ net­work area $ re­dis­tribute con­nected

Then we need to exit the router con­fig, con­fig­ure mode, and vty so run the exit com­mand three times. $ exit De­vuan-NOS(con­fig)# $ exit De­vuan-NOS# $ exit

Now re­peat this on the client switch, re­plac­ing the routerid with the ip ad­dress of the br0 bridge, and we should be able to ping from the server to the desk­tops again.

This may seem a lit­tle overkill for a net­work with only one server and two switches, but we have laid the ground­work for build­ing a fully meshed Clos net­work. This set-up, with only a few mi­nor changes, is scal­able to the ex­tent of even the most de­mand­ing en­ter­prise en­vi­ron­ments with­out sig­nif­i­cant in­creases in la­tency or de­creases in through­put.

Af­ter around 30 sec­onds the rout­ing ta­ble should be syn­chro­nised and you’ll be able to test the con­nec­tions. This is best done with a nice cuppa and a few choice bis­cuits.

Dou­ble-click­ing a de­vice that’s booted up opens the ap­pro­pri­ate ter­mi­nal.

Tim Arm­strong is the net­work ar­chi­tect at Nerdal­ize. He de­signs and im­ple­ments dat­a­cen­tre and ISP net­works. He’s a real con­trol freak when it all comes to Bits.

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.