An In­tro­duc­tion to Deep­learn­ing4j, the Dis­trib­uted Deep Learn­ing Li­brary

There are many deep learn­ing li­braries that are be­com­ing pop­u­lar among the de­vel­oper com­mu­nity such as Theano, Torch, Caffe, etc. Deep­learn­ing4J is an open source and dis­trib­uted deep learn­ing li­brary tar­geted at Java Vir­tual Ma­chine (JVM). This ar­ti­cle p

OpenSource For You - - Developers - By: Dr K.S. Kup­pusamy The au­thor is an as­sis­tant pro­fes­sor of com­puter science, School of En­gi­neer­ing and Tech­nol­ogy, Pondicherry Cen­tral Univer­sity. He has more than 12 years of ex­pe­ri­ence in academia and in­dus­try. He can be con­tacted at

Ma­chine learn­ing has led to a tremen­dous shift in the way we cre­ate al­go­rithms to solve prob­lems. It has made de­vel­op­ers and re­searchers shift away from the tra­di­tional step-by-step so­lu­tions ap­proach to a holis­tic model, which in many as­pects mim­ics the meth­ods that bi­o­log­i­cal or­gan­isms em­ploy while solv­ing prob­lems or ac­quir­ing a new skill. These days, ma­chine learn­ing mod­els are em­ployed in all those sec­tors that were orig­i­nally dif­fi­cult to deal with by the tra­di­tional al­go­rith­mic ap­proaches. Real-time ob­ject de­tec­tion from videos, clas­si­fi­ca­tion and pre­dic­tion sys­tems are some of the tasks per­formed with the help of ma­chine learn­ing. The pop­u­lar quote by Prof. Tom Mitchell, who is a pi­o­neer in shap­ing the do­main, clearly de­fines ma­chine learn­ing as fol­lows: “A com­puter pro­gram is said to learn from ex­pe­ri­ence E with re­spect to some class of tasks T and per­for­mance mea­sure P if its per­for­mance at tasks in T, as mea­sured by P, im­proves with ex­pe­ri­ence E.”

Deep learn­ing

Deep learn­ing is a re­cent im­prove­ment in the ma­chine learn­ing do­main. A ba­sic re­quire­ment of ma­chine learn­ing is to iden­tify and build a fea­ture set. This task is gen­er­ally car­ried out by ex­perts in that do­main, man­u­ally. So, for each prob­lem do­main, cus­tomised fea­tures need to be built by ex­pert hu­mans. This of­ten cre­ates a bot­tle­neck in the over­all process flow.

Deep learn­ing dif­fers from tra­di­tional ma­chine learn­ing in the way fea­tures are built. It at­tempts to build the fea­tures au­to­mat­i­cally from the given large data set. Note the em­pha­sis on the word ‘large’. For deep learn­ing to work at a de­cent ac­cu­racy level, we need con­sid­er­ably large sized data sets. This is sig­nif­i­cant, be­cause the ma­chine re­quires a fairly large amount of data in or­der to de­tect the dis­crim­i­na­tive fea­tures of a set. Fea­ture en­gi­neer­ing is an im­por­tant com­po­nent for the suc­cess­ful im­ple­men­ta­tion of any ma­chine learn­ing re­lated project.

Deep learn­ing frame­works/li­braries

There are many pop­u­lar li­braries that can be used to per­form deep learn­ing tasks. Some of them are listed be­low:






A de­tailed com­par­i­son of deep learn­ing frame­works is avail­able at https://deep­learn­­pare-dl4j­torch7-pylearn.

Deep­learn­ing4j (DL4J)

This ar­ti­cle ex­plores the Deep­learn­ing4J (DL4J) li­brary. DL4J has been de­vel­oped in Java and is tar­geted at

Java Vir­tual Ma­chine (JVM). An in­ter­est­ing fea­ture of Deep­learn­ing4J is the abil­ity to build fast pro­to­types. The at­tributes of DL4J are listed be­low.

Dis­trib­uted: The train­ing process can be ac­cel­er­ated be­cause DL4J is dis­trib­uted by na­ture. It can har­ness mul­ti­ple GPUs, and the per­for­mance is on par with other ma­jor li­braries such as Caffe.

Open Source: DL4J is a pro­duc­tion qual­ity open source li­brary avail­able for per­form­ing deep learn­ing tasks. The ac­tive com­mu­nity of de­vel­op­ers keeps DL4J fresh. In­ter­op­er­abil­ity: DL4J is writ­ten in Java, and hence all the JVM based lan­guages such as Scala, Clo­jure and Kotlin are com­pat­i­ble. In­ter­op­er­abil­ity with other ma­jor frame­works such as Caffe and Torch can be achieved through Keras.

DL4J fea­tures

The ma­jor fea­tures of DL4J are:

Java, Scala and Python APIs

Par­al­lel train­ing through it­er­a­tive re­duce

Scal­able with Hadoop

Dis­trib­uted CPU and GPU sup­port

DL4J in­cor­po­rates both a dis­trib­uted, multi-threaded deep learn­ing frame­work and a sin­gle-threaded deep learn­ing frame­work. An­other im­por­tant fea­ture of DL4J is that it is the first deep learn­ing frame­work adopted for a mi­croser­vice ar­chi­tec­ture.


The pre­req­ui­sites to start de­vel­op­ment with DL4J are listed be­low:

Java 1.7 or above (DL4J sup­ports only the

64-bit ver­sions).

Apache Maven: This is a de­pen­dency man­age­ment and build tool. The Maven ver­sion in your sys­tem can be checked with the fol­low­ing com­mand:

mvn --ver­sion

If you are new to Maven, an ex­cel­lent ‘get­ting started’ guide (Maven in Five Min­utes) is avail­able at http://­ting-started/maven-in-fiveminutes.html

The of­fi­cial doc­u­ment rec­om­mends the use of Inel­liJ IDE or Eclipse. The com­mu­nity edi­tion of Inel­liJ IDE can be down­loaded from the of­fi­cial web­site.

Git: Get the latest ver­sion with the fol­low­ing com­mand:

$ git clone git://git.ker­

Us­ing the DL4J ex­am­ples

To down­load and use the ex­am­ples from DL4J, use the fol­low­ing com­mands:

$ git clone­learn­ing4j/dl4j-ex­am­ples. git

$ cd dl4j-ex­am­ples/

$ mvn clean in­stall

Run In­tel­liJ. Choose the Im­port Project op­tion. Select the folder ‘dl4j-ex­am­ple’.

Select ‘Im­port Project from ex­ter­nal model’ and make sure that Maven is cho­sen.

Fol­low the wizard’s in­struc­tions. Click Fin­ish to com­plete. It might take some time to get all the de­pen­den­cies in the lo­cal sys­tem.

4. Select a par­tic­u­lar ex­am­ple and right-click the file to run it.

The deep neu­ral net­works are made up of mul­ti­ple lay­ers. The Multi Layer Con­fig­u­ra­tion with pa­ram­e­ters is cus­tomised to suit the re­quire­ments of the cur­rent prob­lem. The hy­per­pa­ram­e­ter vari­able de­cides the learn­ing be­hav­iour of the net­work. These pa­ram­e­ters in­clude the fol­low­ing:

Num­ber of times to up­date the weights of the model Ini­tial­i­sa­tion mech­a­nism for these weights

The type of ac­ti­va­tion func­tion to link with the nodes

The spe­cific op­ti­mi­sa­tion al­go­rithm to be used

The speed with which the net­work should learn

A sam­ple con­fig­u­ra­tion is shown in the fol­low­ing code snip­pet:

Multi Layer Con­fig­u­ra­tion conf= new Neu­ral Net Con­fig­u­ra­tion. Builder()


.weight In it( Weight In it. XAVIER)

.ac­ti­va­tion(“relu”) .op­ti­miza­tion Al go( Op­ti­miza­tion Al­go­rithm. STOCHASTIC_ GRADIENT_DESCENT)


// ... other hy­per­pa­ram­e­ters




A new layer can be added by in­vok­ing layer() on Neu­ral Net Con­fig­u­ra­tion. Builder (). In this process, we need to spec­ify the fol­low­ing:

Lo­ca­tion (or­der) where the layer has to be added

The num­ber of in­put and out­put nodes (nIn and nOut) The type of layer

layer(0, new DenseLayer.Builder().nIn(784).nOut(250) .build())

After com­plet­ing the con­fig­u­ra­tion process, the train­ing of the model can be car­ried out with the fol­low­ing com­mand:

DL4J’s neu­ral net­works

DL4J sup­ports many pow­er­ful neu­ral net­works. Some of them are listed be­low:

Re­stricted Boltz­mann ma­chines

Con­vo­lu­tional nets

Re­cur­rent nets

Deep-be­lief net­works

Stacked de­nois­ing au­toen­coders

The rea­son for choos­ing JVM

Most of the deep learn­ing/ma­chine learn­ing li­braries are in Python. But DL4J is based on JVM. One may ask why this de­lib­er­ate choice was made. The of­fi­cial doc­u­men­ta­tion lists the ma­jor rea­sons for choos­ing the Java Plat­form:

Many large gov­ern­ment or­gan­i­sa­tions and com­pa­nies have Java as their ma­jor plat­form. There are mil­lions of Java de­vel­op­ers and Java is the largest pro­gram­ming lan­guage, as of now.

Java is the ba­sis of many pow­er­ful tech­niques/tools such as Hadoop, Elas­ticSearch, Lucene and Hive.

One prime com­plaint against Python is its slow speed. Java is def­i­nitely quicker than Python. Hence, when han­dling projects with mas­sive data sets, Java may be a bet­ter choice.

Java is in­her­ently se­cure and cross-plat­form. It can be eas­ily used on all ma­jor plat­forms such as Linux, Win­dows, OSX and An­droid.

DL4J on An­droid

Deep­learn­ing4j can be used with An­droid mo­bile phones as well. The pre­req­ui­sites for run­ning it on An­droid are: Emu­la­tor (with API level 21 or above) or a de­vice An­droid Stu­dio 2.2 or later

The fol­low­ing de­pen­den­cies must be added to the build. gra­dle file:

com­pile ‘org.deep­learn­ing4j:deep­learn­ing4j-core:0.8.0’ com­pile ‘org.nd4j:nd4j-na­tive:0.8.0’ com­pile ‘org.nd4j:nd4j-na­tive:0.8.0:an­droid-x86’ com­pile ‘org.nd4j:nd4j-na­tive:0.8.0:an­droid-arm’ com­pile ‘org.byt­edeco.javacpp-pre­sets:open­blas:­droid-x86’

com­pile ‘org.byt­edeco.javacpp-pre­sets:open­blas:­droid-arm’

A clean and easy-to-fol­low tu­to­rial is avail­able at https:// deep­learn­­droid.

This ar­ti­cle has pro­vided an in­tro­duc­tion to the Deep­learn­ing4J li­brary. If you are in­ter­ested in ex­plor­ing more about deep learn­ing, there are many in­ter­est­ing on­line cour­ses and re­sources. A com­piled list of links is avail­able at https:// deep­learn­­ing­for­be­gin­ners.html.


[1] Deep­learn­ing4j of­fi­cial home­page: https://deep­learn­ [2] Maven:­ting-started/ maven-in-five-min­utes.html

Fig­ure 2: DL4J fea­tures

Fig­ure 1: Pop­u­lar deep learn­ing li­braries

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.