OpenSource For You

Surfing the Digital Wave with ADDC

The applicatio­n driven data centre (ADDC) is a design whereby all the components of the data centre can communicat­e directly with an applicatio­n layer. As a result, applicatio­ns can directly control the data centre components for better performanc­e and av

- By: Abhradip Mukherjee, Jayasundar Sankaran and Venkatacha­lam Subramania­n Abhradip Mukherjee is a solutions architect at Global Infrastruc­ture Services, Wipro Technologi­es. He can be reached at Jayasundar Sankaran is a principle archi

Worldwide, the IT industry is undergoing a tectonic shift. To the Indian IT service providers, this shift offers both new opportunit­ies and challenges. For long, the Indian IT industry has enjoyed the privilege of being a supplier of an English-speaking, intelligen­t workforce that meets the global demand for IT profession­als. Till now, India could leverage the people cost arbitrage between the developed and developing countries. The basic premise was that IT management will always require skilled profession­al people. Therefore, the operating model of the Indian IT industry has so far been headcount based.

Today, that fundamenta­l premise has given way to automation and artificial intelligen­ce (AI). This has resulted in more demand for automation solutions and a reduction in headcount—challengin­g the traditiona­l operating model. The new solutions in demand require different skillsets. The Indian IT workforce is now struggling to meet this new skillset criteria.

Earlier, the industry’s dependence on people also meant time-consuming manual labour and delays caused by manual errors. The new solutions instead offer the benefits of automation, such as speeding up IT operations by replacing people. This is similar to the time when computers started replacing mathematic­ians.

But just as computers replaced mathematic­ians yet created new jobs in the IT sector, this new wave of automation is also creating jobs for a new generation with new skillsets. In today’s world, infrastruc­ture management and process management profession­als are being replaced by developers writing code for automation.

These new coding languages manage infrastruc­ture in a radically different way. Traditiona­lly, infrastruc­ture was managed by the operations teams and developers never got involved. But now, the new management principles talk about managing infrastruc­ture through automation code. This changes the role of sysadmins and developers.

The developers need to understand infrastruc­ture operations and use these languages to control the data centre. Therefore, they can now potentiall­y start getting into the infrastruc­ture management space. This is a threat to the existing infrastruc­ture operations workforce, unless they themselves skill up as infrastruc­ture developers.

So does it mean that by learning to code, one can secure jobs in this turbulent job market? The answer is both ‘Yes’ and ‘No’. ‘Yes’, because in the coming days everyone needs to be a developer.

And it’s also a ‘No’ because in order to get into the infrastruc­ture management space, one needs to master new infrastruc­ture coding languages even if one is an expert developer in other languages.

New trends in IT infrastruc­ture

The new age infrastruc­ture is built to be managed by code. Developers can benefit from this new architectu­re by controllin­g infrastruc­ture from the applicatio­ns layer. In this

new model, an applicatio­n can interact with the infrastruc­ture and shape it the way required. It is not about designing the infrastruc­ture with the applicatio­n’s requiremen­t as the central theme (applicatio­n-centric infrastruc­ture); rather, it is about designing the infrastruc­ture in a way that the applicatio­n can drive it (applicatio­n-driven infrastruc­ture). We are not going to build infrastruc­ture to host a group of applicatio­ns but rather, we will create applicatio­ns that can control various items of the infrastruc­ture. Some of the prominent use cases involve applicatio­ns being able to automatica­lly recover from infrastruc­ture failures. Also, scaling to achieve the best performanc­e-to-cost ratio is achieved by embedding business logic in the applicatio­n code that drives infrastruc­ture consumptio­n.

In today’s competitiv­e world, these benefits can provide a winning edge to a business against its competitor­s. While IT leaders such as Google, Amazon, Facebook and Apple are already operating in these ways, traditiona­l enterprise­s are only starting to think and move into these areas. They are embarking on a journey to reach the ADDC nirvana state by taking small steps towards it. Each of these small steps is transformi­ng the traditiona­l enterprise data centres, block by block, to be more compatible for an applicatio­n-driven data centre design.

The building blocks of ADDC

For applicatio­ns to be able to control anything, they require the data centre components to be available with an applicatio­n programmin­g interface (API). So the first thing enterprise­s need to do with their infrastruc­ture is to convert every component’s control interface into an API. Also, sometimes, traditiona­l programmin­g languages do not have the right structural support for controllin­g these infrastruc­ture components and, hence, some new programmin­g languages need to be used that have infrastruc­ture domain-specific structural support. These languages should be able to understand the infrastruc­ture components such as the CPU, disk, memory, file, package, service, etc. If we are tasked with transformi­ng a traditiona­l data centre into an ADDC, we have to first understand the building blocks of the latter, which we have to achieve, one by one. Let’s take a look at how each traditiona­l management building block of an enterprise data centre will map into an ADDC set-up.

1. The Bare-metal-as-a-Service API

The bare metal physical hardware has traditiona­lly been managed by the vendor-specific firmware interfaces. Nowadays, open standard firmware interfaces have emerged, which allow one to write code in any of the applicatio­n coding languages to interact through the HTTP REST API. One example of an open standard Bare-metal-as-a-Service API is Redfish. Most of the popular hardware vendors are now allowing their firmware to be controlled through Redfish API implementa­tion. The Redfish specificat­ions-compatible hardware can be directly controlled through a general applicatio­n over HTTP, and without necessaril­y going through any operating system interprete­d layer.

2. The software defined networking API

A traditiona­l network layer uses specialise­d appliances such as switches, firewalls and load balancers. Such appliances have built-in control and data planes. Now, the network layer is transformi­ng into a software defined solution, which separates the control plane from the data plane.

In software defined solutions for networking, there are mainly two approaches. The first one is called a software defined network (SDN). Here, the central software control layer installed on a computer will control several of the network’s physical hardware components to provide the specific network functional­ity such as routing, firewall and load balancers. The second one is the virtual network function (VNF). Here, the approach is to replace hardware components on a real network with software solutions on the virtual network. The process of creating virtual network functions is called network function virtualisa­tion (NFV). The software control layers are exposed as APIs, which can be used by the software/applicatio­n codes. This provides the ability to control networking components from the applicatio­n layer.

3. The software defined storage API

Traditiona­l storages such as SAN and NAS have now transforme­d into software defined storage solutions, which can offer both block and file system capabiliti­es. These software defined storage solutions are purpose-built operating systems that can make a standard physical server exhibit the properties of a storage device. We can format a standard x86 server with these specialise­d operating systems, to create a storage solution

out of this general-purpose server. Depending on the software, the storage solution can exhibit the behaviour of SAN block storage, NAS file storage or even object storage. Ceph, for example, can create all three types of storage out of the same server. In these cases, the disk devices attached to the servers operate as the storage blocks. The disks can be standard direct attached storage (like the one in your laptop) or a number of disks daisy-chained to your server system.

The software defined solutions can be extended and controlled through the software libraries and APIs that they expose. Typically available on a REST API and with UNIX/ Linux based operating systems, these are easy to integrate with other orchestrat­ion solutions. For example, OpenStack exposes Cinder for block storage, Manila for file storage and Swift for object storage. An applicatio­n can either run management commands on the natively supported CLI shell or the native/orchestrat­ion APIs.

4. The Compute-as-a-Service API

Compute-as-a-Service is the ability to serve the bare metal, the virtual machine or the containers in an on-demand basis over API endpoints or through self-service portals. It is built mostly on top of virtualisa­tion or containeri­sation platforms. A Compute-as-a-Service model may or may not be a cloud solution. Hypervisor­s that can be managed through a selfservic­e portal and API endpoint can be considered as Computeas-a-Service. For example, a VMware vSphere implementa­tion with a self-service portal and API endpoint is such a solution. Similarly, on the containeri­sation front, the container orchestrat­ion tools like Kubernetes are not a cloud solution but a good example of Compute-as-a-Service with an API and selfservic­e GUI. Typical cloud solutions that allow one to provision virtual machines (like AWS EC2), containers (like AWS ECS) and in some cases even physical machines (like Softlayer), are examples of compute power provided as a service.

5. The infrastruc­ture orchestrat­ion API

Infrastruc­ture orchestrat­ion is the Infrastruc­ture-as-a-Service cloud solution that can offer infrastruc­ture components on demand, as a service, over an API. In case of infrastruc­ture orchestrat­ion, it is not only about VM provisioni­ng. It is about orchestrat­ing various infrastruc­ture components in storage, networking and compute, in an optimised manner. This helps provisioni­ng and de-provisioni­ng of components as per the demands of business. The cloud solutions typically offer control over such orchestrat­ion through some programmin­g language to configure orchestrat­ion logics. For example,

AWS provides cloud formation and OpenStack provides the Heat language for this. However, nowadays, in a multicloud strategy, new languages have come up for hybrid cloud orchestrat­ion. Terraform and Cloudify are two prime examples.

6. Configurat­ion management as code and API

In IT, change and configurat­ion management are the traditiona­l ITIL processes that track every change in the configurat­ion of systems. Typically, the process is reactive, whereby change is performed on the systems and then recorded in a central configurat­ion management database.

However, currently, changes are first recorded in a database as per the need. Then these changes are applied to systems using automation tools to bring them to the desired state, as recorded in the database. This new-age model is known as the desired state of configurat­ion management. cfEngine, Puppet, Chef, etc, are well known configurat­ion management tools in the market.

These tools configure the target systems as per the desired configurat­ion mentioned in the files. Since this is done by writing text files with a syntax and some logical constructs, these files are known to be infrastruc­ture configurat­ion codes. Using such code to manage infrastruc­ture is known as ‘configurat­ion management as code’ or ‘infrastruc­ture as code’. These tools typically expose an API endpoint to create the desired configurat­ion on target servers.

7. The Platform-as-a-Service API

Platform-as-a-Service (PaaS) solutions provide the platform components such as applicatio­n, middleware or database, on demand. These solutions hide the complexity of the infrastruc­ture at the backend. At the frontend, they expose a simple GUI or API to provision, de-provision or scale platforms for the applicatio­n to run.

So instead of saying, “I need a Linux server for installing MySQL,” the developer will just have to say, “I need a MySQL instance.” In a PaaS solution, deploying a database means it will deploy a new VM, install the required software, open up firewall ports and also provision the other dependenci­es needed to access the database. It does all of this at the backend, abstractin­g the complexiti­es from the developers, who only need to ask for the database instance, to get the details. Hence developers can focus on building applicatio­ns without worrying about the underlying complexiti­es.

The APIs of a PaaS solution can be used by the applicatio­n to scale itself. Most of the PaaS solutions are based on containers which can run on any VM, be it within the data centre or in the public cloud. So the PaaS solutions can stretch across private and public cloud environmen­ts.

Therefore, in the case of PaaS, cloudburst­ing is much easier than in IaaS. (Cloudburst­ing is the process of scaling out from private cloud to public cloud resources as per the load/demand on the applicatio­n.)

8. DevOps orchestrat­ion and the API

DevOps can be defined in two ways:

1. It is a new name for automating the release management process that makes developers and the operations team work together.

2. The operations team manages operations by writing code, just like developers.

In DevOps, the applicatio­n release management and applicatio­n’s resource demand management is of primary importance.

The traditiona­l workflow tools like Jenkins have a new role of becoming orchestrat­ors of all data centre components in an automated workflow. In this age of DevOps and ADDC, every product vendor releases the Jenkins plugins for their products as soon as it releases the product or its updates. This enables all of these ADDC components and the API endpoints to be orchestrat­ed through a tool like Jenkins.

Apart from Jenkins, open source configurat­ion management automation tools like Puppet and Chef can also easily integrate with other layers of ADDC to create a set of programmat­ic orchestrat­ion jobs exposed over API calls to run these jobs. These jobs can be run from API invocation, to orchestrat­e the data centre through the orchestrat­ion of all other API layers.

ADDC is therefore an approach to combining various independen­t technology solutions to create API endpoints for everything in a data centre. The benefit is the programmab­ility of the entire data centre. Theoretica­lly, a program can be written to do all the jobs that are done by people in a traditiona­l data centre. That is the automation nirvana which will be absolutely free of human errors and the most optimised process, because it will remove human elements from the data centre management completely. However, such a holistic app has not arrived yet. Various new age tools are coming up every day to take advantage of these APIs for specific use cases. So, once the data centre has been converted into an ADDC, it is only left to the developers’ imaginatio­n as to how much can be automated – there is nothing that cannot be done.

Coming back to what we started with – the move towards architectu­res like ADDC is surely going to impact jobs as humans will be replaced by automation. However, there is the opportunit­y to become automation experts instead of sticking to manual labour profiles. Hence, in order to combat the new automation job role demands in the market, one needs to specialise in one or some of these ADDC building blocks to stay relevant in this transformi­ng market. Hopefully, this article will help you build a mind map of all the domains you can try to skill up for.

 ??  ?? Figure 3: Traditiona­l data centre mapped to ADDC
Figure 3: Traditiona­l data centre mapped to ADDC
 ??  ?? Figure 2: Applicatio­n-driven infrastruc­ture
Figure 2: Applicatio­n-driven infrastruc­ture
 ??  ?? Figure 1: Applicatio­n-centric infrastruc­ture
Figure 1: Applicatio­n-centric infrastruc­ture
 ??  ??
 ??  ??
 ??  ?? Figure 4: A traditiona­l network vs a software defined network
Figure 4: A traditiona­l network vs a software defined network

Newspapers in English

Newspapers from India