Big Data on a Desktop: A Virtual Machine in an OpenStack Cloud OpenStack is a worldwide collaboration between developers and cloud computing technologists aimed at developing the cloud computing platform for public and private clouds. Let’s install it on
Installing OpenStack using Packstack is very simple. After a test installation in a virtual machine, you will find that the basic operations for creating and using virtual machines are now quite simple when using a Web interface.
The environment
It is important to understand the virtual environment. While everything is running on a desktop, the setup consists of multiple logical networks interconnected via virtual routers and switches. You need to make sure that the routes are defined properly because otherwise, you will not be able to access the virtual machines you create.
On the desktop, the virt-manager creates a NAT-based network by default. NAT assures that if your desktop can access the Internet, so can the virtual machine. The Internet access had been used when the OpenStack distribution was installed in the virtual machine.
The Packstack installation process creates a virtual public network for use by the various networks created within the cloud environment. The virtual machine on which OpenStack is installed is the gateway to the physical network. Virtual Network on the Desktop (virbr0 interface): 192.168.122.0/32 IP address of eth0 interface on OpenStack VM: 192.168.122.54 Public Virtual Network created by packstack on OpenStack VM: 172.24.4.224/28 IP address of the br-ex interface OpenStack VM: 172.24.4.225
Testing the environment
In the OpenStack VM console, verify the network addresses. In my case, I had to explicitly give an ip to the br-ex interface, as follows: # ifconfig # ip addr add 172.24.4.225/28 dev br-ex
On the desktop, add a route to the public virtual network on OpenStack VM: # route add -net 172.24.4.224 netmask 255.255.255.240 gw 192.168.122.54
Now, browse http://192.168.122.54/dashboard and create a new project and a user associated with the project. 1. Sign in as the admin. 2. Under the Identity panel, create a user (youser) and a project (Bigdata). Sign out and sign in as youser to create and test a cloud VM. 3. Create a private network for the project under Project/Network/Networks: • Create the private network 192.168.10.0/24 with the gateway 192.168.10.254 • Create a router and set a gateway to the public network. Add an interface to the private network and ip address 192.168.10.254. 4. To be able to sign in using ssh, under the Project/ Compute/ Access & Security, in the Security Groups tab, add the following rules to the default security group: • Allow ssh access: Custom TCP Rule for allowing traffic on Port 22. • Allow icmp access: Custom ICMP Rule with Type and Code value -1. 5. For password-less signing into the VM, under the Project/Compute/Access & Security, in the Key Pairs tab the following: • Select the Import Key Pair option and give it a name, e.g., ‘desktop user login’. • In your desktop terminal window, use ssh-keygen to create a public/private key pair in case you don't already have one. • Copy the contents of ~/.ssh/id_rsa.pub from your desktop account and paste them in the public key. 6. Allocate a public IP for accessing the VM under Project/Compute/Access & Security in the Floating Ips tab, and allocate IP to the project. You may get a value like 172.24.4.229 7. Now launch the instance under Project/Compute/
Instance:
• Give it a name - test and choose the m1tiny flavour. • Select the boot source as ‘Boot from image' with the image name ‘cirros', a very small image included in the installation. • Once it is launched, associate the floating ip obtained above with this instance. Now, you are ready to log in to the VM created in your local cloud. In a terminal window, type:
ssh cirros@172.24.4.229
You should be signed into the virtual machine without needing a password.
You can experiment with importing the Fedora VM image you used for the OpenStack VM and launching it in the cloud. Whether you succeed or not will depend on the resources available in the OpenStack VM.
Installing only the needed OpenStack services
You will have observed that OpenStack comes with a very wide range of services, some of which are not likely to be very useful for your experiments on the desktop, e.g., the additional networks and router created in the tests above. Here is a part of the dialogue for installing the required services on the desktop: [root@amd ~]# packstack Welcome to Installer setup utility Enter the path to your ssh Public key to install on servers: Packstack changed given value to required value /root/. ssh/id_rsa.pub Should Packstack install MySQL DB [y|n] [y] : y Should Packstack install OpenStack Image Service (Glance) [y|n] [y] : y Should Packstack install OpenStack Block Storage (Cinder) service [y|n] [y] : n Should Packstack install OpenStack Compute (Nova) service [y|n] [y] : y Should Packstack install OpenStack Networking (Neutron) service [y|n] [y] : n Should Packstack install OpenStack Dashboard (Horizon) [y|n] [y] : y Should Packstack install OpenStack Object Storage (Swift) [y|n] [y] : n Should Packstack install OpenStack Metering (Ceilometer) [y|n] [y] : n Should Packstack install OpenStack Orchestration (Heat) [y|n] [n] : n Should Packstack install OpenStack client tools [y|n] [y] :y
The answers to the other questions will depend on the network interface and the IP address of your desktop, but there is no ambiguity here. You should answer with the interface ‘lo' for CONFIG_NOVA_COMPUTE_PRIVIF and CONFIG_NOVA_NETWORK_PRIVIF. You don't need an extra physical interface as the compute services are running on the same server.
Now, you are ready to test your OpenStack installation on the desktop. You may want to create a project and add a user to the project. Under Project/ Compute/Access & Security, you will need to add firewall rules and key pairs, as above.
However, you will not need to create any additional private network or a router.
Import a basic cloud image, e.g., from http://fedoraproject. org/get-fedora#clouds under Project/Compute/Images.
You may want to create an additional flavour for a virtual machine. The m1.tiny flavour has 512MB of RAM and 4GB of disk and is too small for running Hadoop. The m1.small flavour has 2GB of RAM and 20GB of disk, which will restrict the number of virtual machines you can run for testing Hadoop. Hence, you may create a mini flavour with 1GB of RAM and 10GB of disk. This will need to be done as the admin user.
Now, you can create an instance of the basic cloud image. The default user is fedora and your setup is ready for exploration of Hadoop data.