Big Data on a Desk­top: A Vir­tual Ma­chine in an OpenS­tack Cloud OpenS­tack is a world­wide col­lab­o­ra­tion be­tween de­vel­op­ers and cloud com­put­ing tech­nol­o­gists aimed at de­vel­op­ing the cloud com­put­ing plat­form for pub­lic and pri­vate clouds. Let’s in­stall it on

OpenSource For You - - EXPLORING SOFTWARE GUEST COLUMN -

In­stalling OpenS­tack us­ing Pack­stack is very sim­ple. Af­ter a test in­stal­la­tion in a vir­tual ma­chine, you will find that the ba­sic op­er­a­tions for cre­at­ing and us­ing vir­tual ma­chines are now quite sim­ple when us­ing a Web in­ter­face.

The en­vi­ron­ment

It is im­por­tant to un­der­stand the vir­tual en­vi­ron­ment. While ev­ery­thing is run­ning on a desk­top, the setup con­sists of mul­ti­ple log­i­cal net­works in­ter­con­nected via vir­tual routers and switches. You need to make sure that the routes are de­fined prop­erly be­cause oth­er­wise, you will not be able to ac­cess the vir­tual ma­chines you cre­ate.

On the desk­top, the virt-man­ager cre­ates a NAT-based net­work by de­fault. NAT as­sures that if your desk­top can ac­cess the In­ter­net, so can the vir­tual ma­chine. The In­ter­net ac­cess had been used when the OpenS­tack dis­tri­bu­tion was in­stalled in the vir­tual ma­chine.

The Pack­stack in­stal­la­tion process cre­ates a vir­tual pub­lic net­work for use by the var­i­ous net­works cre­ated within the cloud en­vi­ron­ment. The vir­tual ma­chine on which OpenS­tack is in­stalled is the gate­way to the phys­i­cal net­work. Vir­tual Net­work on the Desk­top (virbr0 in­ter­face): 192.168.122.0/32 IP ad­dress of eth0 in­ter­face on OpenS­tack VM: 192.168.122.54 Pub­lic Vir­tual Net­work cre­ated by pack­stack on OpenS­tack VM: 172.24.4.224/28 IP ad­dress of the br-ex in­ter­face OpenS­tack VM: 172.24.4.225

Test­ing the en­vi­ron­ment

In the OpenS­tack VM con­sole, ver­ify the net­work ad­dresses. In my case, I had to ex­plic­itly give an ip to the br-ex in­ter­face, as fol­lows: # if­con­fig # ip addr add 172.24.4.225/28 dev br-ex

On the desk­top, add a route to the pub­lic vir­tual net­work on OpenS­tack VM: # route add -net 172.24.4.224 net­mask 255.255.255.240 gw 192.168.122.54

Now, browse http://192.168.122.54/dash­board and cre­ate a new project and a user as­so­ci­ated with the project. 1. Sign in as the ad­min. 2. Un­der the Iden­tity panel, cre­ate a user (youser) and a project (Big­data). Sign out and sign in as youser to cre­ate and test a cloud VM. 3. Cre­ate a pri­vate net­work for the project un­der Project/Net­work/Net­works: • Cre­ate the pri­vate net­work 192.168.10.0/24 with the gate­way 192.168.10.254 • Cre­ate a router and set a gate­way to the pub­lic net­work. Add an in­ter­face to the pri­vate net­work and ip ad­dress 192.168.10.254. 4. To be able to sign in us­ing ssh, un­der the Project/ Com­pute/ Ac­cess & Se­cu­rity, in the Se­cu­rity Groups tab, add the fol­low­ing rules to the de­fault se­cu­rity group: • Al­low ssh ac­cess: Cus­tom TCP Rule for al­low­ing traf­fic on Port 22. • Al­low icmp ac­cess: Cus­tom ICMP Rule with Type and Code value -1. 5. For pass­word-less sign­ing into the VM, un­der the Project/Com­pute/Ac­cess & Se­cu­rity, in the Key Pairs tab the fol­low­ing: • Se­lect the Im­port Key Pair op­tion and give it a name, e.g., ‘desk­top user lo­gin’. • In your desk­top ter­mi­nal win­dow, use ssh-key­gen to cre­ate a pub­lic/pri­vate key pair in case you don't al­ready have one. • Copy the con­tents of ~/.ssh/id_rsa.pub from your desk­top ac­count and paste them in the pub­lic key. 6. Al­lo­cate a pub­lic IP for ac­cess­ing the VM un­der Project/Com­pute/Ac­cess & Se­cu­rity in the Float­ing Ips tab, and al­lo­cate IP to the project. You may get a value like 172.24.4.229 7. Now launch the in­stance un­der Project/Com­pute/

In­stance:

• Give it a name - test and choose the m1tiny flavour. • Se­lect the boot source as ‘Boot from image' with the image name ‘cir­ros', a very small image in­cluded in the in­stal­la­tion. • Once it is launched, as­so­ciate the float­ing ip ob­tained above with this in­stance. Now, you are ready to log in to the VM cre­ated in your lo­cal cloud. In a ter­mi­nal win­dow, type:

ssh cir­[email protected]

You should be signed into the vir­tual ma­chine with­out need­ing a pass­word.

You can ex­per­i­ment with im­port­ing the Fe­dora VM image you used for the OpenS­tack VM and launch­ing it in the cloud. Whether you suc­ceed or not will de­pend on the re­sources avail­able in the OpenS­tack VM.

In­stalling only the needed OpenS­tack ser­vices

You will have ob­served that OpenS­tack comes with a very wide range of ser­vices, some of which are not likely to be very use­ful for your ex­per­i­ments on the desk­top, e.g., the ad­di­tional net­works and router cre­ated in the tests above. Here is a part of the di­a­logue for in­stalling the re­quired ser­vices on the desk­top: [[email protected] ~]# pack­stack Wel­come to In­staller setup util­ity En­ter the path to your ssh Pub­lic key to in­stall on servers: Pack­stack changed given value to re­quired value /root/. ssh/id_rsa.pub Should Pack­stack in­stall MySQL DB [y|n] [y] : y Should Pack­stack in­stall OpenS­tack Image Ser­vice (Glance) [y|n] [y] : y Should Pack­stack in­stall OpenS­tack Block Stor­age (Cin­der) ser­vice [y|n] [y] : n Should Pack­stack in­stall OpenS­tack Com­pute (Nova) ser­vice [y|n] [y] : y Should Pack­stack in­stall OpenS­tack Net­work­ing (Neu­tron) ser­vice [y|n] [y] : n Should Pack­stack in­stall OpenS­tack Dash­board (Hori­zon) [y|n] [y] : y Should Pack­stack in­stall OpenS­tack Ob­ject Stor­age (Swift) [y|n] [y] : n Should Pack­stack in­stall OpenS­tack Me­ter­ing (Ceilome­ter) [y|n] [y] : n Should Pack­stack in­stall OpenS­tack Or­ches­tra­tion (Heat) [y|n] [n] : n Should Pack­stack in­stall OpenS­tack client tools [y|n] [y] :y

The an­swers to the other ques­tions will de­pend on the net­work in­ter­face and the IP ad­dress of your desk­top, but there is no am­bi­gu­ity here. You should an­swer with the in­ter­face ‘lo' for CONFIG_NOVA_COMPUTE_PRIVIF and CONFIG_NOVA_NETWORK_PRIVIF. You don't need an ex­tra phys­i­cal in­ter­face as the com­pute ser­vices are run­ning on the same server.

Now, you are ready to test your OpenS­tack in­stal­la­tion on the desk­top. You may want to cre­ate a project and add a user to the project. Un­der Project/ Com­pute/Ac­cess & Se­cu­rity, you will need to add fire­wall rules and key pairs, as above.

How­ever, you will not need to cre­ate any ad­di­tional pri­vate net­work or a router.

Im­port a ba­sic cloud image, e.g., from http://fe­do­rapro­ject. org/get-fe­dora#clouds un­der Project/Com­pute/Im­ages.

You may want to cre­ate an ad­di­tional flavour for a vir­tual ma­chine. The m1.tiny flavour has 512MB of RAM and 4GB of disk and is too small for run­ning Hadoop. The m1.small flavour has 2GB of RAM and 20GB of disk, which will re­strict the num­ber of vir­tual ma­chines you can run for test­ing Hadoop. Hence, you may cre­ate a mini flavour with 1GB of RAM and 10GB of disk. This will need to be done as the ad­min user.

Now, you can cre­ate an in­stance of the ba­sic cloud image. The de­fault user is fe­dora and your setup is ready for ex­plo­ration of Hadoop data.

Fig­ure 1: Sim­pli­fied net­work di­a­gram

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.