DeepLearning4j and PyTorch: Two Powerful Deep Learning Tools
This article gives an introduction to two free and open source tools for deep learning and knowledge discovery–DL4J and PyTorch.
Machine learning (ML) is a prominent area of research in the fields of knowledge discovery and the identification of hidden patterns in data sets. ML integrates predictive mining and the optimisation of results in multiple applications including bioinformatics, biometrics, computer vision, computational anatomy, criminal forensics, face detection, document forgery, fraud detection and many others. A comprehensive list of such applications is available on the Internet. ML focuses on the training of intelligent models so that the prediction can be accurate and fast.
Machines learn from the data and then get trained for prediction. For example, the preferences of customers are fetched from their existing records of interests.
Deep learning
Deep learning (DL) is a more accurate and performanceaware flavour of machine learning, in which the error rate is very small compared to the latter. DL is also known as deep structured learning and hierarchical learning.
Let’s look at speech recognition, for instance. If we install speech recognition software in a mobile phone for disaster/emergency purposes, when selected words like ‘help’ or ‘kidnap’ are uttered, a disaster message will be sent to chosen family members or friends. In this case, ML
will detect only these two words and will respond as per the data it has been trained on. In the case of DL, the messages analogous to the training data will also be processed, like ‘snatching’, ‘cannot see’ and similar words. This is the beauty and power of DL – it enables machines to learn on their own, from the training data set.
As shown in Figure 1, the process of feature extraction from the input is a separate one in the case of machine learning. Researchers need to create a separate method for feature extraction in the case of ML, using different mathematical formulae and scientific computations.
The methods and approaches for feature extraction are integrated in the case of deep learning, which makes it more accurate and smart.
DeepLearning4J: Deep learning in Java
DeepLearning4j (DL4J) (https://deeplearning4j.org/) is a Java based library that provides comprehensive and high performance implementations for DL in multiple applications. It is available as a free and open source deep learning library, with features like distributed computing for Java platforms, and can be used for knowledge discovery and deep predictive mining on graphics processing units (GPUs) as well as CPUs. The GPU is used for high performance and fast operations with a display of images, videos, animations and related multimedia objects.
Using Eclipse DeepLearning4j, Big Data analytics can be performed along with Apache Spark and Hadoop, using Java as well as Scala programming. DL4J integrates the techniques and algorithms of artificial intelligence (AI), which can be used for business intelligence, cyber forensics, robotic process automation (RBA), network intrusion detection and prevention, recommender systems, predictive analysis, regression, face recognition, natural language processing, anomaly detection and many others.
In addition to the inherent high performance features, DL4J enables the import of models from other prominent DL frameworks including TensorFlow, Theano, Caffe and Keras. DL4J provides the interface for both Java and Python programming without any compatibility issues.
The key features embedded in DeepLearning4j include the following:
Scalability on Hadoop for Big Data
Microservices architecture
Parallel computing and training
Massive amounts of data can be processed using clusters Support for GPUs for scalability on Amazon Web
Services (AWS) cloud
APIs for Java, Python and Scala
Distributed architecture with multi-threading
Support to CPUs and GPUs
DL4J has the following components and libraries for assorted applications, in order to support multiple functions. ND4J: This is the integration of NumPy for Java Virtual Machine (JVM). ND4J can be used to call the libraries for fast processing of matrix data on the processing units. It is a library for numerical computations and processing with performance-aware execution of multi-dimensional objects for scientific applications, including signal processing, linear algebra, optimisation, transformations,
gradient descent and many others. It is similar to scikitlearn and NumPy in Python.
JavaCPP: This is the bridge and interface tool for Java and C++. It provides the interface and APIs to access C++ code in Java without any intermediate or third-party application. DataVec: This is the machine learning tool to extract, transform and load (ETL). The key function of DataVec is to transform the raw data to vector format with preprocessing to make it compatible with training in the ML implementations. It supports multiple formats including comma-separated values (CSVs), images, video, binary, text and many others.
RL4J: This refers to reinforcement learning for Java platforms with the implementation of Deep Q Learning, asynchronous actor-critic agents (A3C) and many other algorithms for reinforcement learning for JVM.
Arbiter: This is a tool to evaluate and test the algorithms of ML. It supports searching for hyper-parameters to extract the best fit configuration of neural networks.
The following are the prerequisites to work with DeepLearning4j:
Java Development Kit (JDK)
Eclipse IDE or IntelliJ IDEA
Git
Apache Maven
After installation of the required software tools, the examples and use cases from GitHub can be downloaded for testing and further customisation of the code.
$ git clone https://github.com/deeplearning4j/dl4j-examples.git $ cd dl4j-examples/
$ mvn clean install
From IntelliJ, the option of Import Project is selected, along with the directory ‘dl4j-examples’.
Select ‘Import project from external model’ and Maven as shown in Figure 3.
Select the Deeplearning4j example for execution. In this example, MLPClassifierLinear is used and run. A series of scores will be displayed at the bottom.
Figure 6 depicts the graph to show the classified data using multi-layer perceptron (MLP).
PyTorch: Deep learning in Python
PyTorch (http://pytorch.org/) is a free and open source framework for DL using Python programming. Dynamic models and neural networks can be programmed using PyTorch. It supports CPU as well as GPU based computing for DL algorithms.
PyTorch provides the following features for high performance DL:
Tensor processing and computation with GPU support Deep neural networks
Reusability of Python scientific packages on demand like SciPy, NumPy, scikit-learn and Cython
Minimal overhead and complexities
Fast processing regardless of the size of the neural network in computation
Integration with acceleration and speed-up libraries including Intel MKL and NVIDIA to escalate the speed and minimise delay
The list of organisations, universities and companies working on developing PyTorch is shown in Figure 7.
The installation process of PyTorch varies depending upon the operating system and the package manager, as specified in the official portal of PyTorch. The developers can install PyTorch by selecting the OS, package manager, Python version and CUDA support, as shown in Figure 8.
Example of the classifier using deep learning in PyTorch
The features of deep learning are presented with a classification, using a benchmark data set CIFAR-10 as shown in Figure 9 (https://www.cs.toronto.edu/~kriz/cifar.html). This data set is used for training the model of deep learning. In this data set, multiple similar images are labelled. Using PyTorch, the image that does not exist in the data set can be predicted under a specific class and label category.
The output will be fetched as ‘plane horse cat bird’ because of the feature extraction and deep learning, based on the properties of these objects extracted from the training data set. Despite the fact that none of the images in Figure 10 exist in the training data, the deep learning algorithm is able to identify similar patterns and assign the class or label to the appropriate category.
Scope for research and development
Deep learning libraries can be used for R&D in assorted applications including criminal face detection, flying objects analysis, intrusion detection at international borders, gesture recognition for criminal investigation, video forgery analysis, image tampering detection and many others, in which the identification of specific patterns is required. Deep learning libraries provide many models and approaches for prediction, with the maximum accuracy.