A NoSQL Data­base for Real-Time Ap­pli­ca­tions

OpenSource For You - - Developers - The au­thor is the MD of Magma Re­search and Con­sul­tancy and is as­so­ci­ated with var­i­ous aca­demic in­sti­tutes, where he de­liv­ers lec­tures and con­ducts tech­ni­cal work­shops on the lat­est tech­nolo­gies and tools. You can con­tact him by email at ku­mar­gau­

Re­thinkDB is an open source, scal­able data­base that makes build­ing real-time apps dra­mat­i­cally eas­ier.

With the het­ero­ge­neous as well as un­struc­tured data on the World Wide Web (WWW) in­creas­ing in leaps and bounds, tra­di­tional data­base en­gines are fac­ing nu­mer­ous is­sues re­lated to schema man­age­ment, con­cur­rency, data­base in­tegrity, se­cu­rity, par­al­lel read-write op­er­a­tions, re­source op­ti­mi­sa­tion and many oth­ers. Real-time ap­pli­ca­tions re­ceive lots of traf­fic from dif­fer­ent chan­nels in­clud­ing so­cial me­dia, mail groups, satel­lites, the In­ter­net of Things (IoT), etc. Such traf­fic is gen­er­ally un­struc­tured and voluminous, and can­not be han­dled by clas­si­cal re­la­tional data­base man­age­ment sys­tems (RDBMS). To cope with such is­sues of Big Data, which in­clude ve­loc­ity, vol­ume and va­ri­ety (the 3Vs of

Big Data), Not Only SQL (NoSQL) data­bases that could han­dle un­struc­tured and het­ero­ge­neous data with higher per­for­mance came into ex­is­tence.

To­day, there are dif­fer­ent types of NoSQL data­bases in di­verse cat­e­gories to help spe­cific ap­pli­ca­tions achieve a higher de­gree of per­for­mance and ac­cu­racy (Ta­ble 1).

Re­thinkDB as a doc­u­ment-ori­ented NoSQL data­base

Re­thinkDB (­ is a high-per­for­mance NoSQL data­base for real-time ap­pli­ca­tions with an ef­fec­tive Web based ad­min­is­tra­tive panel and user in­ter­face. The data­base sys­tem pro­vides ex­cel­lent fea­tures for doc­u­ment stor­age based ap­pli­ca­tions, and is termed as a doc­u­men­to­ri­ented data­base. Re­thinkDB pro­vides fea­tures for stor­age, re­trieval and man­age­ment of semi-struc­tured data to achieve the min­i­mum de­lay in data­base op­er­a­tions. The key con­cept and par­a­digm be­hind this doc­u­ment-ori­ented data­base is the stor­age and han­dling of doc­u­ments in un­struc­tured or semistruc­tured for­mats. All doc­u­ment for­mats can be en­cap­su­lated with the en­cod­ing of data in doc­u­ment-ori­ented NoSQL data­bases. The en­cod­ing of data in­cludes YAML, BSON, XML and JSON, along with the other bi­nary forms in­clud­ing PDF, MS Word, spread­sheets and many oth­ers.

The real-time push ar­chi­tec­ture of Re­thinkDB

Real-time Web ap­pli­ca­tions con­sume re­sources in par­al­lel from dif­fer­ent sources, in­clud­ing data­base en­gines, live up­date scripts, band­width, con­cur­rent con­nec­tions, se­cu­rity mod­ules, etc. To ac­cess and up­date real-time data in Web ap­pli­ca­tions, there is a need to in­te­grate high per­for­mance tech­nolo­gies so that the live stream­ing data can be in­serted or up­dated with min­i­mum de­lay and re­source con­sump­tion. For ex­am­ple, live polling or live chat based ap­pli­ca­tions may be de­layed be­cause of a high num­ber of par­al­lel users on the same chan­nel, and this can in­crease the wait­ing time. With Re­thinkDB, this de­lay can be de­creased as it has been de­vel­oped specif­i­cally for real-time Web ap­pli­ca­tions, with the re­quired scal­a­bil­ity and ease of use.

Re­thinkDB is the first data­base to be scal­able as well as open source (with the base of JSON) for real-time Web ap­pli­ca­tions. Us­ing Re­thinkDB, the de­vel­oper can di­rect the data­base en­gine to push the up­dated query re­sults in real-time to the run­ning ap­pli­ca­tion, so that max­i­mum up­dates can be done with­out de­lay. The real-time push ar­chi­tec­ture of Re­thinkDB dras­ti­cally re­duces the ef­fort, time and re­sources for the real-time ap­pli­ca­tions.

The ar­chi­tec­ture of Re­thinkDB is highly ad­van­ta­geous in real-time Web ap­pli­ca­tions, high traf­fic e-com­merce ap­pli­ca­tions, real-time mo­bile apps, shar­ing in­for­ma­tion be­tween con­nected de­vices, the In­ter­net of Things (IoT), the Cloud of Things (CoT), multi-player real-time games, satel­lite chan­nels and wire­less sig­nal an­a­lyt­ics.

Fea­tures and ad­van­tages of Re­thinkDB

Re­thinkDB has a flex­i­ble and high-per­for­mance query lan­guage, ReQL, to pro­vide real-time push based up­dates to mul­ti­ple users in par­al­lel. In ad­di­tion, it pro­vides in­tu­itive op­er­a­tions with the APIs for mon­i­tor­ing real-time changes, which are easy to set up and un­der­stand for pro­gram­mers. It is be­ing used by as­sorted cor­po­rate gi­ants, For­tune 500 companies, as well as star­tups, to man­age live ap­pli­ca­tions.

Re­thinkDB has a vi­brant as well as large com­mu­nity of more than 100,000 soft­ware de­vel­op­ers and trou­bleshoot­ing ex­perts across the world. These de­vel­op­ers and con­trib­u­tors keep on cus­tomis­ing the code and shar­ing their work on on­line por­tals to en­rich the over­all com­mu­nity of Re­thinkDB pro­gram­mers.

Some of the key users of Re­thinkDB are:

Jive Soft­ware

Nar­ra­tive Clip







In­stal­la­tion of Re­thinkDB

The of­fi­cial pack­ages of Re­thinkDB for Ubuntu Linux, OS X, Cen­tOS, De­bian and Win­dows are avail­able. In ad­di­tion, com­mu­nity-sup­ported pack­ages are avail­able on re­thinkdb. com for Arch Linux, OpenSUSE, Fe­dora, Linux Mint, Rasp­bian and Gen­too.

Of­fi­cial Re­thinkDB client driv­ers are avail­able for JavaScript, Ruby, Python and Java, while com­mu­ni­ty­sup­port driv­ers are avail­able for C#, Clo­jure, Del­phi, Go, Lua, PHP, Swift, R, Com­mon Lisp, Elixir, Haskell, Nim, C++, Dart, Er­lang, JS Neu­mino, Perl and Rust. Driv­ers with lim­ited fea­tures are also avail­able for Ob­jec­tive-C and Scala.

In­stalling Re­thinkDB on Win­dows

The 64-bit bi­nar­ies of Re­thinkDB for Win­dows 7 and its ver­sions are avail­able on the of­fi­cial URL of Re­thinkDB­­stall/win­dows/. To in­stall Re­thinkDB on Win­dows, the 64-bit flavour is re­quired.

A ZIP ar­chive is down­loaded and un­packed in any

direc­tory. In the direc­tory of Re­thinkDB, there is a file called re­thinkdb.exe. On dou­ble-click­ing it, the server gets ini­ti­ated and is then avail­able for fur­ther data­base op­er­a­tions.

After start­ing the Re­thinkDB server, the ad­min­is­tra­tion con­sole needs to be opened on the Web browser with the URL http://, so that dif­fer­ent op­tions can be viewed for data­base ad­min­is­tra­tion.

The op­tions for cre­at­ing the data­base and ta­bles with dif­fer­ent priv­i­leges are avail­able in the ad­min­is­tra­tive panel of the Re­thinkDB server. The con­nected servers, ta­bles, in­dexes and re­sources can be viewed in the panel, as shown in Fig­ure 3.

To use a spe­cific direc­tory for the stor­age and log­ging of data, the fol­low­ing in­struc­tion is used:

Win­dowsDirec­tory:\>re­thinkdb.exe -d c:\Re­thinkDB\data\

To spec­ify a par­tic­u­lar server name and clus­ter, use the com­mand given be­low:

Win­dowsDirec­tory:\>re­thinkdb.exe -n MyDo­main -j my­clus­ter.

In the ad­min­is­tra­tive con­sole, there is the op­tion to cre­ate a new data­base with any name. This data­base name can be called in the front-end ap­pli­ca­tions for real-time ap­pli­ca­tions.

Once the data­base is cre­ated, it is vis­i­ble in the ad­min­is­tra­tion con­sole. In a sim­i­lar way, the ta­ble in a par­tic­u­lar data­base can be cre­ated with the spec­i­fi­ca­tion of a pri­mary key, along with the ac­knowl­edge­ment.

After cre­at­ing the ta­ble in a spe­cific data­base, the Re­thinkDB en­gine re­turns the mes­sage in JSON for­mat so that the con­fir­ma­tion of the write op­er­a­tion can be dis­played with dif­fer­ent val­ues in­serted in the data­base ta­ble.

In­stalling Re­thinkDB on Ubuntu

For the in­stal­la­tion of Re­thinkDB on Ubuntu Linux, both 32bit and 64-bit flavours are avail­able.

$ source /etc/lsb-re­lease && echo “deb http://down­load. re­ $DISTRIB_CODENAME main” | sudo tee /etc/ apt/sources.list.d/re­thinkdb.list

$ wget -qO- https://down­­­key.gpg | sudo apt-key add $ sudo apt-get up­date

$ sudo apt-get in­stall re­thinkdb

Com­pi­la­tion and in­stal­la­tion from source

To com­pile and in­stall from the source, use the fol­low­ing com­mands:

$ sudo apt-get in­stall build-es­sen­tial pro­to­buf-com­piler python \ libpro­to­buf-dev libcurl4-openssl-dev \ lib­boost-all-dev lib­n­curses5-dev \ lib­je­mal­loc-dev wget m4

$ wget https://down­­­thinkdb-ver­sion. tgz

$ tar xf re­thinkdb- ver­sion.tgz

$ cd re­thinkdb- ver­sion

$./con­fig­ure --al­low-fetch

$ make

$ sudo make in­stall

To start the server on Ubuntu Linux, ex­e­cute the fol­low­ing in­struc­tion from the ter­mi­nal win­dow:

$ re­thinkdb

Cre­at­ing a ta­ble us­ing the Data Ex­plorer tab in Ex­plorer

In the Re­thinkDB ad­min­is­tra­tive con­sole, there is a Data Ex­plorer tab, which pro­vides the panel for ex­e­cut­ing the com­mands that will work on the data­bases. In the fol­low­ing ex­am­ple, a new ta­ble ti­tled chat_users is cre­ated in the data­base ti­tled my­database.

r.db(‘my­database’).tableCreate(‘chat_users ‘)

On click­ing the Run but­ton at the bot­tom-right of the ad­min­is­tra­tion con­sole, the query is ex­e­cuted and the op­er­a­tion is im­ple­mented. In ad­di­tion, the key com­bi­na­tion Shift+En­ter can be used to run the query en­tered.

To in­sert records in the form of JSON in the ta­ble chat_ users, use the fol­low­ing com­mands:

r.ta­ble(‘chat_users).in­sert([{ name: ‘User­name-1’, age: 17 }, { name: ‘User­name-2’, age: 35 }])

To count the num­ber of records in the ta­ble, give the fol­low­ing com­mand:


To dis­play the records of ap­pli­ca­tion users aged 20 years and above from the ta­ble, use the fol­low­ing com­mand:


Pro­gram­ming with Re­thinkDB us­ing Python

Python is a free and open source pro­gram­ming lan­guage that has in­ter­faces for al­most all NoSQL data­bases. It is widely used for cloud ap­pli­ca­tions and as­sorted real-time high­per­for­mance Web ap­pli­ca­tions. In Python, Re­thinkDB can be eas­ily in­stalled and mapped us­ing Pip, us­ing the fol­low­ing Fig­ure 9: Ex­e­cu­tion of Re­thinkDB in­struc­tions in the Python IDLE shell com­mand: Win­dowsDirec­tory:\> python -m pip in­stall re­thinkdb

The pack­age in­staller of Python au­to­mat­i­cally fetches the li­braries of Re­thinkDB and maps them with the ex­ist­ing in­stal­la­tion of Python.

After map­ping Re­thinkDB with Python, the IDLE shell can use the meth­ods and APIs of Re­thinkDB.

>> im­port re­thinkdb as <Han­dle>

Us­ing the Python IDLE shell, or within the script, the con­nec­tion can be opened with the fol­low­ing com­mand, which in­cludes a client driver with 28015 as the port num­ber:

>> r.con­nect( “lo­cal­host”, 28015).repl()

The repl com­mand sets and pre­pares the de­fault con­nec­tion with the Python shell.

The func­tion table_cre­ate() is used to add a new ta­ble in the par­tic­u­lar data­base, as shown in Fig­ure 9. By: Dr Gau­rav Ku­mar

Fig­ure 3: Dash­board and con­trol panel of Re­thinkDB

Fig­ure 1: Of­fi­cial por­tal of Re­thinkDB

Fig­ure 2: Start­ing the Re­thinkDB server

Fig­ure 7: Re­turned mes­sage after cre­at­ing a ta­ble on the dash­board of Re­thinkDB

Fig­ure 5: View the data­base after cre­ation

Fig­ure 6: Adding a ta­ble in a spe­cific data­base

Fig­ure 4: Cre­at­ing a new data­base

Fig­ure 8: In­stal­la­tion of Re­thinkDB with Python

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.