DevOps Se­ries An­si­ble De­ploy­ment of the Aerospike NoSQL Data­base

In this 13th ar­ti­cle in the DevOps se­ries, we will in­stall and set up an Aerospike NoSQL data­base clus­ter.

OpenSource For You - - Contents - By: Shak­thi Kan­nan The au­thor is a free soft­ware en­thu­si­ast and blogs at shak­thi­maan.com.

Aerospike is a free and open source, dis­trib­uted NoSQL data­base writ­ten in C. It is pri­mar­ily de­signed for high through­put and low la­tency ap­pli­ca­tions. It is a key-value store op­ti­mised for Flash stor­age de­vices and runs on *nix plat­forms. It can also be used as an in-mem­ory data­base re­sid­ing en­tirely on RAM. Fol­low­ing the CAP (Con­sis­tency, High Avail­abil­ity, Par­ti­tion Tol­er­ance) the­o­rem, at present, the data­base can be op­er­ated in AP mode. The data­base func­tions in three lay­ers — the data stor­age layer, the clus­ter­ing and dis­tri­bu­tion layer, and the client layer. As in many dis­trib­uted data­bases, the server sup­ports rolling up­grades and failover mech­a­nisms in the clus­ter.

A num­ber of client li­braries for var­i­ous pro­gram­ming lan­guage en­vi­ron­ments like C, Java, Go, Python, Node.js, etc, are avail­able to cre­ate ap­pli­ca­tions that can com­mu­ni­cate with the Aerospike server. The ba­sic data types that are sup­ported are in­te­ger, string, bytes, dou­ble, list, map, GeoJSON and blobs (lan­guage se­ri­alised). Sup­port for user de­fined func­tions (UDF) also ex­ists, us­ing the Lua pro­gram­ming lan­guage. The Aerospike tool­ing ecosys­tem has a num­ber of con­nec­tors for the Spring frame­work, Hadoop, Kafka, etc. The Aerospike Man­age­ment Con­sole (AMC) is a Web-based ap­pli­ca­tion used to mon­i­tor the Aerospike clus­ter. The Aerospike server was first launched in 2012, and is re­leased un­der the Af­fero Gen­eral Pub­lic Li­cense.

GNU/Linux

A host sys­tem run­ning Ubuntu 16.04.3 LTS (Xe­nial) with Va­grant and Vir­tu­alBox is used to launch three Cen­tOS

6.9 vir­tual ma­chines, on which the Aerospike server will be in­stalled and set up. The Va­grant file to launch the three nodes is shown be­low:

BOX_IMAGE = “bento/cen­tos-6.9” NODE_COUNT = 3 Va­grant.con­fig­ure(“2”) do |con­fig|

(1..NODE_COUNT).each do |i|

# Cre­ate Aerospike server node con­fig.vm.de­fine “as#{i}” do |node| node.vm.box = BOX_IMAGE node.ssh.for­ward_a­gent = true # https://github.com/mitchellh/va­grant/is­sues/4967 node.ssh.in­sert_key = false node.ssh.user­name = ‘va­grant’ node.ssh.pass­word = ‘va­grant’ node.vm.net­work :pri­vate_net­work, ip: “10.0.0.#{i + 10}” node.vm.host­name = “as#{i}” node.vm.provider “vir­tu­albox” do |vb| vb.name = “Aerospike­server­#{i}” vb.mem­ory = “2048” end end end end

The three in­stances are named ‘as1’, ‘as2’ and ‘as3’, and are as­signed pri­vate IP ad­dresses that can be ac­cessed from the host sys­tem. The ma­chines are brought up us­ing the fol­low­ing com­mand:

$ va­grant up

The ver­sion of An­si­ble used on the host sys­tem is 2.2.0.0. A project direc­tory struc­ture is cre­ated to store the An­si­ble play­books and in­ven­tory, as fol­lows:

an­si­ble/in­ven­tory/vbox/ /play­books/con­fig­u­ra­tion/

An ‘in­ven­tory’ file is cre­ated in­side the in­ven­tory/vbox

folder, which con­tains ac­cess in­for­ma­tion for each node:

[nodes] as1 an­si­ble_host=10.0.0.11 an­si­ble_­con­nec­tion=ssh an­si­ble_ user=va­grant an­si­ble_ssh_pri­vate_key_­file=~/.va­grant.d/ in­se­cure_pri­vate_key as2 an­si­ble_host=10.0.0.12 an­si­ble_­con­nec­tion=ssh an­si­ble_ user=va­grant an­si­ble_ssh_pri­vate_key_­file=~/.va­grant.d/ in­se­cure_pri­vate_key as3 an­si­ble_host=10.0.0.13 an­si­ble_­con­nec­tion=ssh an­si­ble_ user=va­grant an­si­ble_ssh_pri­vate_key_­file=~/.va­grant.d/ in­se­cure_pri­vate_key

We also cre­ate ‘in­ven­tory/vbox/group_­vars/all/all.yml’ with the fol­low­ing vari­ables:

--aerospike_edi­tion: “com­mu­nity” aerospike_ver­sion: “lat­est”

am­c_ver­sion: 3.6.13

In­stalling Aerospike

The en­tire set of play­books is avail­able in the play­books/ con­fig­u­ra­tion/aerospike.yml file. The first step is to in­stall the Aerospike server on all the three nodes us­ing An­si­ble, as shown be­low:

--name: Con­fig­ure Aerospike clus­ter node hosts: nodes be­come: yes be­come_method: sudo gath­er_­facts: true tags: [server]

tasks:

­ name: Cre­ate down­loads direc­tory file: path=”/home/{{ an­si­ble_user }}/down­loads” state=direc­tory

­ name: Cre­ate in­stalls direc­tory file: path=”/home/{{ an­si­ble_user }}/in­stalls” state=direc­tory

­ name: Down­load Aerospike com­mu­nity server (RPM) get_url: url=”http://aerospike.com/down­load/server/{{ aerospike_ver­sion }}/ar­ti­fact/el6” dest=”/home/{{ an­si­ble_ user }}/down­loads/” regis­ter: __­com­mu­ni­ty_server_­file

­ set_­fact: server_ver­sion: “{{ __­com­mu­ni­ty_server_­file.dest | base­name }}” ­ name: Ex­tract Aerospike server com­mand: tar xzvf “/home/{{ an­si­ble_user }}/down­loads/{{ server_ver­sion }}” ­C in­stalls

­ name: Yum up­date yum: name=* up­date_­cache=yes state=present

­ name: In­stall RPM de­pen­den­cies be­come: yes be­come_method: sudo yum: name: “{{ item }}” state: lat­est with­_items:

­ python

­ lib­selinux­python

­ name: Run asin­stall shell: cd “/home/{{ an­si­ble_user }}/in­stalls/{{ server_ ver­sion | regex_re­place(‘\.tgz’, ‘’) }}” && sudo ./asin­stall

­ name: Start Aerospike server (RPM) shell: sudo ser­vice aerospike start

­ name: Wait for server to be up and run­ning wait­_­for: port: 3000 de­lay: 5 state: started

The ‘down­loads’ and ‘in­stalls’ di­rec­to­ries are first cre­ated. The Aerospike server pack­age for Cen­tOS 6 is then down­loaded and ex­tracted. The YUM up­date com­mand is used to up­date the soft­ware repos­i­tory, fol­low­ing which the de­pen­den­cies and the Aerospike server are in­stalled. The data­base server is then started, and the play­book waits for the server to lis­ten on port 3000. The above play­book can be run as fol­lows:

$ an­si­ble­play­book ­i in­ven­tory/vbox/in­ven­tory play­books/ con­fig­u­ra­tion/aerospike.yml ­­tags server

Set­ting up a mul­ti­cast clus­ter

The Aerospike clus­ter can be set up ei­ther in multi-cast or mesh mode. The fol­low­ing play­book con­fig­ures the clus­ter in mul­ti­cast mode:

­ name: Cre­ate mul­ti­cast clus­ter hosts: nodes gath­er_­facts: true sudo: yes tags: [mul­ti­cast]

vars:

ip_ad­dress: “{{ an­si­ble_eth1[‘ipv4’][‘ad­dress’] }}” line_ad­dress: “ad­dress {{ ip_ad­dress }}”

tasks:

­ name: Stop aerospike server com­mand: ser­vice aerospike stop

­ name: Spec­ify mul­ti­cast con­fig­u­ra­tion file pa­ram­e­ters line­in­file: dest: /etc/aerospike/aerospike.conf in­sertafter: “{{ item.in­sertafter }}” line: “{{ item.line }}” state: present with­_items:

­ { in­sertafter: ‘proto­fd­max 15000’, line: “node­id­in­ter­face eth1” }

­ { in­sertafter: ‘port 9918’, line: “{{ line_ad­dress }}” }

­ { in­sertafter: ‘fab­ric’, line: “{{ line_ad­dress }} # Mul­ti­cast: Spec­ify pri­vate IP ad­dress” }

­ name: Up­date net­work ser­vice ad­dress re­place: name: /etc/aerospike/aerospike.conf reg­exp: ‘ad­dress any’ re­place: “ad­dress {{ an­si­ble_eth1[‘ipv4’][‘ad­dress’] }}”

­ name: Start aerospike server com­mand: ser­vice aerospike start

The above play­book can be ex­e­cuted us­ing the fol­low­ing com­mand:

$ an­si­ble­play­book ­i in­ven­tory/vbox/in­ven­tory play­books/ con­fig­u­ra­tion/aerospike.yml ­­tags mul­ti­cast

You can ver­ify the multi-cast clus­ter setup by log­ging into one of the vir­tual ma­chines, and us­ing the Aerospike Ad­min (asadm) tool as il­lus­trated be­low:

[va­grant@as1 ~]$ asadm

Aerospike In­ter­ac­tive Shell, ver­sion 0.1.15

Found 3 nodes

On­line: 10.0.0.11:3000, 10.0.0.13:3000, 10.0.0.12:3000

Ad­min> exit [va­grant@as1 ~]$

In­stalling the Aerospike Man­age­ment Con­sole

The Aerospike Man­age­ment Con­sole (AMC) dash­board pro­vides a graph­i­cal user in­ter­face (GUI) for the Aerospike server. Its in­stal­la­tion play­book is as fol­lows: ­ name: In­stall Aerospike Man­age­ment Con­sole hosts: nodes be­come: yes be­come_method: sudo gath­er_­facts: true tags: [amc]

tasks:

­ name: Down­load AMC Com­mu­nity RPM pack­age get_url: url=”http://www.aerospike.com/down­load/amc/{{ am­c_ver­sion }}/ar­ti­fact/el6” dest=”/home/{{ an­si­ble_user }}/ down­loads/” regis­ter: __­com­mu­ni­ty_am­c_­file

­ set_­fact: am­c_­file: “{{ __­com­mu­ni­ty_am­c_­file.dest }}”

­ name: In­stall RPM de­pen­den­cies yum: name: “{{ item }}” state: lat­est with­_items:

- gcc

­ python­de­vel

­ name: In­stall RPM amc

com­mand: rpm ­ivh “{{ am­c_­file }}” ­ name: Start AMC server (RPM)

shell: sudo ser­vice amc start

The in­vo­ca­tion of the above play­book and a sam­ple ex­e­cu­tion run is shown be­low:

$ an­si­ble­play­book ­i in­ven­tory/vbox/in­ven­tory play­books/ con­fig­u­ra­tion/aerospike.yml ­­tags amc

PLAY [Con­fig­ure Aerospike clus­ter node] ********************** TASK [setup] ************************************************ ok: [as2] ok: [as1] ok: [as3]

PLAY [Cre­ate mul­ti­cast clus­ter] *****************************

TASK [setup] ************************************************ ok: [as1] ok: [as2] ok: [as3]

PLAY [In­stall Aerospike Man­age­ment Con­sole] ***************** TASK [setup] ************************************************ ok: [as1] ok: [as2] ok: [as3]

TASK [Down­load AMC Com­mu­nity RPM pack­age] ******************* changed: [as2] changed: [as1] changed: [as3]

TASK [set_­fact] ********************************************* ok: [as1] ok: [as2] ok: [as3]

TASK [In­stall RPM de­pen­den­cies] ***************************** changed: [as1] => (item=[u’gcc’, u’python­de­vel’]) changed: [as2] => (item=[u’gcc’, u’python­de­vel’]) changed: [as3] => (item=[u’gcc’, u’python­de­vel’])

TASK [In­stall RPM amc] ************************************** changed: [as1] changed: [as2] changed: [as3]

TASK [Start AMC server (RPM)] ******************************* changed: [as1] changed: [as2] changed: [as3]

PLAY RE­CAP ************************************************** as1 : ok=8 changed=4 un­reach­able=0 failed=0 as2 : ok=8 changed=4 un­reach­able=0 failed=0 as3 : ok=8 changed=4 un­reach­able=0 failed=0

You can now open http://10.0.0.1:8081 in a browser on the host sys­tem, and you will be prompted with a modal win­dow, as shown in Fig­ure 1.

You can in­put ‘lo­cal­host’ for the ‘Host Name’, and the AMC dash­board opens as shown in Fig­ure 2.

Unin­stalling Aerospike

An unin­stall play­book is writ­ten to stop the ser­vices, and unin­stalls both the Aerospike Man­age­ment Con­sole (AMC) and the Aerospike server, as fol­lows: ­ name: Unin­stall hosts: nodes be­come: yes be­come_method: sudo gath­er_­facts: true tags: [unin­stall]

tasks:

­ name: Stop AMC server ser­vice: name: amc state: stopped ­ name: Unin­stall RPM AMC com­mand: yum re­move aerospike­amc­* ­y

­ name: Stop server com­mand: ser­vice aerospike stop

­ name: Unin­stall server shell: yum re­move aerospike­* ­y

The above play­book can be run us­ing the fol­low­ing com­mand:

$ an­si­ble­play­book ­i in­ven­tory/vbox/in­ven­tory play­books/ con­fig­u­ra­tion/aerospike.yml ­­tags unin­stall

You can read the doc­u­men­ta­tion at https://www.aerospike. com/docs/ to learn more about Aerospike.

Fig­ure 2: AMC dash­board

Fig­ure 1: AMC modal win­dow

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.