A Guide to Run­ning OpenS­tack on AWS

OpenS­tack is a cloud op­er­at­ing sys­tem that con­trols vast com­put­ing re­sources through a data cen­tre, while AWS (Ama­zon Web ser­vices) of­fers re­li­able, scal­able and in­ex­pen­sive cloud com­put­ing ser­vices. The in­stal­la­tion of OpenS­tack on AWS is an in­stance of

OpenSource For You - - Contents -

This ar­ti­cle will guide you on in­stalling OpenS­tack on top of AWS EC2. In­stalling OpenS­tack on a nested hy­per­vi­sor en­vi­ron­ment is not a big deal when a QEMU em­u­la­tor is used for launch­ing vir­tual ma­chines in­side the vir­tual ma­chine (VM). How­ever, un­like the usual nested hy­per­vi­sor set-up, in­stalling OpenS­tack on AWS EC2 in­stances has a few re­stric­tions on the net­work­ing side, for the OpenS­tack set-up to work prop­erly. This ar­ti­cle outlines the lim­i­ta­tions and the so­lu­tions to run OpenS­tack on top of the AWS EC2 VM.


The AWS en­vi­ron­ment will al­low the pack­ets to flow in their net­work only when the MAC ad­dress is known or reg­is­tered in the AWS net­work en­vi­ron­ment. Also, the MAC ad­dress and the IP ad­dress are tightly mapped. So, the AWS en­vi­ron­ment will not al­low packet flow if the MAC ad­dress reg­is­tered for the given IP ad­dress is dif­fer­ent.

You may won­der whether the above re­stric­tions will im­pact the OpenS­tack set-up on AWS EC2.

Yes, they cer­tainly will! While con­fig­ur­ing Neu­tron net­work­ing, we cre­ate a vir­tual bridge (say, br-ex) for the provider net­work, where all the VM’s traf­fic will reach the In­ter­net via the ex­ter­nal bridge, fol­lowed by the ac­tual phys­i­cal NIC (say, eth1). In that case, we usu­ally con­fig­ure the ex­ter­nal in­ter­face (NIC) with a spe­cial type of con­fig­u­ra­tion, as given be­low.

The provider in­ter­face uses a spe­cial con­fig­u­ra­tion with­out an IP ad­dress as­signed to it. Con­fig­ure the sec­ond in­ter­face as the provider in­ter­face. Re­place INTERFACE_NAME with the ac­tual in­ter­face name, for ex­am­ple, eth1 or ens224.

Next, edit the /etc/net­work/in­ter­faces file to con­tain the fol­low­ing:

# The provider net­work in­ter­face auto INTERFACE_NAME iface INTERFACE_NAME inet man­ual up ip link set dev $IFACE up down ip link set dev $IFACE down

You can go to http://docs.opens­tack.org/mi­taka/in­stall­guide-ubuntu/en­vi­ron­ment-net­work­ing-con­troller.html for more de­tails.

Due to this spe­cial type of in­ter­face con­fig­u­ra­tion, the re­stric­tion in AWS will hit OpenS­tack net­work­ing. In the main­stream OpenS­tack set-up, the above-men­tioned provider A Guide to Run­ning OpenS­tack on AWS

in­ter­face is con­fig­ured with a spe­cial NIC con­fig­u­ra­tion that will have no IP for that in­ter­face, and will al­low all pack­ets via that spe­cially con­fig­ured NIC. More­over, the VM pack­ets reach­ing the In­ter­net via this spe­cially con­fig­ured NIC will have the OpenS­tack ten­ant router’s gate­way IP ad­dress as the source in each packet.

As I men­tioned ear­lier, in the lim­i­ta­tions above, AWS will only al­low the packet flow when the MAC ad­dress is known/ reg­is­tered in its en­vi­ron­ment. Also, the IP ad­dress must match the MAC ad­dress.

In our case, the packet from the above-men­tioned OpenS­tack ten­ant router will have the IP ad­dress of the router’s gate­way in ev­ery sin­gle packet, and the packet source’s MAC ad­dress will be the MAC ad­dress of the router’s in­ter­face.

Note: These de­tails are avail­able if you use ip netns show fol­lowed by ip netns exec qr­<router_ID> if­con­fig com­mands in the OpenS­tack con­troller’s ter­mi­nal. Since the MAC ad­dress is un­known or not reg­is­tered in the AWS en­vi­ron­ment, the pack­ets will be dropped when they reach the AWS switch. To al­low the VM pack­ets to reach the In­ter­net via the AWS switch, we need to use some tricks/ hacks in our OpenS­tack set-up.

Mak­ing use of what we have

One pos­si­ble way is to reg­is­ter the router’s MAC ad­dress and its IP ad­dress with the AWS en­vi­ron­ment. How­ever, this is not fea­si­ble. As of now, AWS does not have the fea­ture of reg­is­ter­ing any ran­dom MAC ad­dress and IP ad­dress in­side the VPC. More­over, al­low­ing this type of func­tion­al­ity will be a se­vere se­cu­rity threat to the en­vi­ron­ment.

The other method is to make use of what we have. Since we have used a spe­cial type of in­ter­face con­fig­u­ra­tion for the provider NIC, you will note that the IP ad­dress as­signed to the provider NIC (say, eth1) is left un­used. We could use this avail­able/un­used IP ad­dress for the OpenS­tack router’s gate­way. The com­mand given be­low will do the trick:

neu­tron router-gate­way-set router provider --fixed-ip ip_ ad­dress=<Regis­tered_IP_ad­dress*>

IP ad­dress and MAC ad­dress mis­matches

Af­ter con­fig­ur­ing the router gate­way with the AWS reg­is­tered IP ad­dress, each packet from the router’s gate­way will have the AWS reg­is­tered IP ad­dress as the source IP ad­dress, but with the un­reg­is­tered MAC ad­dress gen­er­ated by the OVS. As I men­tioned ear­lier, while dis­cussing the lim­i­ta­tions of AWS, the IP ad­dress must match the MAC ad­dress reg­is­tered; else, all the pack­ets with the mis­matched MAC and IP ad­dress will be dropped by the AWS switch. To make the reg­is­tered MAC ad­dress match with the IP ad­dress, we need to change the MAC ad­dress of the router’s in­ter­face.

The fol­low­ing steps will do the magic. Step 1) In­stall the mac­cha­nger tool.

Step 2) Note down the ac­tual/orig­i­nal MAC ad­dress of the provider NIC (eth1).

Step 3) Change the MAC ad­dress of the provider NIC (eth1).

Step 4) Change the MAC ad­dress of the router’s gate­way in­ter­face to the orig­i­nal MAC ad­dress of eth1.

Step 5) Now, try to ping from the router names­pace.

If you get a suc­cess­ful ping re­sponse, then we are done with the Cloud-on-the-Cloud set-up.

Key points to re­mem­ber

Here are some im­por­tant things that one needs to keep track of.

Chang­ing the MAC ad­dress: In my case, I had used the Ubuntu 14.04 LTS server, with which there was no is­sue in chang­ing the MAC ad­dress us­ing the mac­cha­nger tool. How­ever, when I tried the Ubuntu 16.04 LTS, I got an er­ror say­ing, “No per­mis­sion to mod­ify the MAC ad­dress.” I sus­pect the er­ror was due to the cloud-init tool not al­low­ing the MAC ad­dress’ con­fig­u­ra­tion. So, be­fore set­ting up OpenS­tack, try chang­ing the MAC ad­dress of the NIC.

Float­ing IP dis­abled: As­so­ci­at­ing a float­ing IP to any of OpenS­tack’s VMs will send the packet via the router’s gate­way with the source IP ad­dress as the float­ing IP’s IP ad­dress. This will make the pack­ets hit the AWS switch with a non-reg­is­tered IP and MAC ad­dress, which re­sults in the pack­ets be­ing dropped. So, I could not use the float­ing IP func­tion­al­ity in this set-up. How­ever, I could ac­cess the VM pub­licly us­ing the fol­low­ing NAT process.

Us­ing NAT to ac­cess OpenS­tack’s VM: As men­tioned ear­lier, I could ac­cess the OpenS­tack VM pub­licly us­ing the reg­is­tered IP ad­dress that was as­signed to the router’s gate­way. Use the fol­low­ing NAT com­mand to ac­cess the OpenS­tack VM us­ing the AWS EC2 in­stance’s elas­tic IP:

$ ip netns exec qrouter-f85bxxxx-61b2-xxxx-xxxx-xxxx­ba0xxxx ipt­a­bles -t nat -A PREROUTING -p tcp -d --dport 522 -j DNAT --to-des­ti­na­tion

Note: In the above com­mand, I had NAT for­ward­ing for all pack­ets for with Port 522. Us­ing the above NAT com­mand, all the pack­ets reach­ing with Port num­ber 522 are for­warded to

Here, is the reg­is­tered IP ad­dress of the AWS EC2 in­stance which was as­signed to the router’s gate­way. is the lo­cal IP of the OpenS­tack VM. No­tably, al­ready has NAT with the AWS elas­tic IP, which means all the traf­fic that comes to the elas­tic IP (pub­lic IP) will be for­warded to this VPC

lo­cal IP (

In short, [Elas­tic IP]:522 π π

This means that you can SSH the OpenS­tack VM glob­ally by us­ing the elas­tic IP ad­dress and the re­spec­tive port num­ber.

Elas­tic IP ad­dress: For this type of cus­tomised OpenS­tack in­stal­la­tion, we re­quire at least two NICs for an AWS EC2 in­stance. One is for ac­cess­ing the VM ter­mi­nal for in­stalling and ac­cess­ing the dash­board. In short, it acts as a man­age­ment net­work, a VM tun­nel net­work or an API net­work. The last one is for an ex­ter­nal net­work with a unique type of in­ter­face con­fig­u­ra­tion and is mapped with the provider net­work bridge (say br-ex with eth1).

AWS will not al­low any pack­ets to travel out of the

VPC un­less the elas­tic IP is at­tached to that IP ad­dress. To over­come this prob­lem, we must at­tach the elas­tic IP for this NIC. This is so that the pack­ets of the OpenS­tack’s VM reach the OpenS­tack router’s gate­way and from the gate­way, the pack­ets get em­bed­ded with the reg­is­tered MAC ad­dress. Then, the match­ing IP ad­dress will reach the AWS switch (VPC en­vi­ron­ment) via br-ex and eth1 (a spe­cial type of in­ter­face con­fig­u­ra­tion), and then hit the AWS ac­tual VPC gate­way. From there, the pack­ets will reach the In­ter­net.

Other cloud plat­forms

In my anal­y­sis, I no­ticed that most of the cloud providers like Dreamhost and Auro-cloud have the same lim­i­ta­tions for OpenS­tack net­work­ing. So we could use the tricks/hacks men­tioned above in any of those cloud providers to run an OpenS­tack cloud on top of them.

Note: Since we are us­ing the QEMU em­u­la­tor with­out KVM for the nested hy­per­vi­sor en­vi­ron­ment, the VM’s per­for­mance will be slow.

If you want to try OpenS­tack on AWS, you can reg­is­ter for CloudEn­abler’s Cloud Lab-as-a-Ser­vice of­fer­ing avail­able at https://claas.corestack.io/ which pro­vides a con­sis­tent and on-de­mand lab en­vi­ron­ment for OpenS­tack in AWS.

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.