Get Started with Jenk­ins 2.0: Pipeline as Code

With the Pipeline suite of plu­g­ins, the mantra of Jenk­ins 2.0—of ‘Pipeline as Code’ —is gain­ing mo­men­tum. This is ev­i­dent from the grow­ing num­ber of or­gan­i­sa­tions em­brac­ing this new way of cod­i­fy­ing build flow. This ar­ti­cle’s ob­jec­tive is to lower the ent

OpenSource For You - - Contents - By: Ra­manathan Mutha­iah The au­thor works for a cy­ber se­cu­rity firm in Ben­galuru. He en­joys the out­doors when he’s not churn­ing out Jenk­ins Pipeline code.

For many years, Jenk­ins has been the most pre­ferred choice when it comes to prac­tis­ing au­to­mated con­tin­u­ous in­te­gra­tion. The tra­di­tional (also known as freestyle) way of set­ting up new projects in Jenk­ins has been (and still is) through the graph­i­cal user in­ter­face. With the help of the GUI, pre- and post-build tasks, in­clud­ing han­dling pa­ram­e­ters and in­ject­ing cus­tom en­vi­ron­ment vari­ables, can be glued to­gether in a sin­gle project.

With the re­lease of Jenk­ins 2.0, there has been a shift in the strat­egy to set up new projects in Jenk­ins. Al­most all the build tasks are treated as Pipeline as Code, us­ing the suite of Pipeline plu­g­ins, which means that the com­plete build flow can be cod­i­fied. There­fore, it can be tracked in ver­sion con­trol, thus en­abling col­lab­o­ra­tion across teams. For a quick overview and high­lights of the fea­tures in Jenk­ins 2.0, visit https://jenk­

Jenk­ins­file is where Pipeline as Code is main­tained, and it should be placed in the root di­rec­tory of the source code.

Par­ti­tion­ing CI/CD (Con­tin­u­ous In­te­gra­tion/Con­tin­u­ous De­ploy­ment) into log­i­cally unique but in­ter­con­nected parts, and the vis­ual rep­re­sen­ta­tion of this en­tire chain of events, is termed as build flow and this is what makes Pipeline at­trac­tive.

Pipeline is grad­u­ally evolv­ing into the new way to pro­gram­mat­i­cally con­struct a log­i­cal flow of tasks — for ex­am­ple, com­pile source code, gen­er­ate ar­ti­facts and de­ploy them – all by us­ing a va­ri­ety of plu­g­ins and third-party tools. How­ever, it should be em­pha­sised that the build sys­tem (make, ant, or a mix of scripts in what­ever lan­guage you choose) does not need to be re­vamped and can be reused as is.

Pre­req­ui­sites or as­sump­tions

Be­fore we pro­ceed fur­ther, make sure you have ac­cess to Jenk­ins (v2.3x and above) with stan­dard plu­g­ins, and are able to set up a test project. You should also have the ba­sic fa­mil­iar­ity to set up new projects in Jenk­ins.

This ar­ti­cle will be based on the syn­tax of scripted Pipeline, a gen­eral-pur­pose DSL that’s based on the Groovy pro­gram­ming lan­guage. How­ever, the declar­a­tive syn­tax can be used too to achieve the same re­sults with mi­nor mod­i­fi­ca­tions.

The de­fin­i­tive place to start read­ing about Jenk­ins

Pipeline is https://jenk­

The link https://jenk­ is the ref­er­ence for var­i­ous steps sup­ported in the Pipeline en­vi­ron­ment. A com­par­i­son of the two syn­taxes (im­per­a­tive and declar­a­tive) is doc­u­mented at https://jenk­ pipeline/syn­tax/#com­pare.


Let’s be­gin with analysing Pipeline as Code by fo­cus­ing on the in­di­vid­ual stages, one at a time. Groovy code runs in a JVM en­vi­ron­ment in a build agent (not called slaves any more) and in a tightly con­trolled en­vi­ron­ment, i.e., a Groovy sand­box. This sand­box en­vi­ron­ment en­sures that Pipeline as Code can­not be used to in­voke sys­tem calls. These calls are made avail­able by Jenk­ins for ad­min­is­tra­tive pur­poses – so use them with cau­tion!

An easy and quick ref­er­ence on how to auto-gen­er­ate code snip­pets for most of the Pipeline steps is avail­able with

the op­tion Pipeline Syn­tax (as shown in Fig­ure 1), which can be ac­cessed from the Jenk­ins project’s land­ing page, if the project is set up us­ing Pipeline-style. This will launch a new tab/win­dow, as per the browser’s con­fig­u­ra­tion, pro­vid­ing the drop­down list of the var­i­ous steps avail­able, along with the op­tions and a small text box to dis­play the gen­er­ated code snip­pet (as seen in Fig­ure 2) for the se­lected step.

Here is a snip­pet of Pipeline as Code: #!groovy node(‘la­bel’) { stage(‘Prep’) {

git url:’ git’, branch: ‘mas­ter’



It’s a good prac­tice to start the script with the she­bang header #!groovy, which helps in code high­light­ing. Some of the best prac­tices in writ­ing Jenk­ins Pipeline are listed at­ti­cles/top-10-best-prac­tices-for-jenk­in­spipeline?from­rel=true. The ‘node’ block helps to group the var­i­ous stages, and ex­e­cute the steps de­fined in the stages in the build agents that have the ‘la­bel’. stage block groups dif­fer­ent in­di­vid­ual steps, which are ex­e­cuted on the build agent. A few ex­am­ples of the com­monly used steps are git, sh and echo.

Sin­gle-line com­ments are pre­ceded by two for­ward slashes ( // ), and com­ments span­ning mul­ti­ple lines start with /* and end with */.

This is a multi-line com­ment to high­light its us­age and it ends here.

Build flow

Let’s con­struct an ex­am­ple of a scripted pipeline to: i) pre­pare the workspace and down­load the source code, ii) com­pile the source code (a small pro­gram writ­ten in C lan­guage), and iii) ar­chive the ar­ti­facts. These tasks will be cod­i­fied us­ing scripted Pipeline syn­tax. For the pur­poses of this dis­cus­sion, the Jenk­ins project has been cre­ated and named ‘OSFY-pipeline demo’.

Jenk­ins­file in the link,­manathan/ osfy­pipeline­demo/blob/mas­ter/pipeline_bin/Jenk­ins­file., is the com­plete source (writ­ten in Groovy) for this demo project.

Though the Jenk­ins doc­u­men­ta­tion man­dates that this file be named Jenk­ins­file and placed at the root of the repos­i­tory, Fig­ure 3: Script path in our ex­am­ple, it is named Jenk­ins­file.groovy and kept un­der the folder pipeline_bin. This con­ven­tion of plac­ing the Jenk­ins­file at the root of the repo should be fol­lowed while set­ting up the project con­fig­u­ra­tion in Jenk­ins.

A clean start

To en­sure that ev­ery trig­ger of the OSFY project starts a fresh build job with a clean workspace, we use the step deleteDir(). While node and stage blocks are cov­ered, a new step named dir is in­tro­duced here, and deleteDir is wrapped in­side this step.

node(‘linux’) { stage(‘Clean Start’) { dir(‘demo-sources’) { deleteDir()



The con­tents of the folder, demo-sources, if present, will be re­moved. In the sub­se­quent stages, the same folder will be reused to main­tain con­tin­ued ac­cess to the down­loaded source code.

Down­load the source code

The git step ref­er­ences the mas­ter branch to down­load source code from the spec­i­fied GitHub repo that’s ref­er­enced us­ing the URL. This step can in­clude other op­tions, like us­ing the cre­den­tials which may or may not be dif­fer­ent from that set-up in the Jenk­ins mas­ter. The con­tents of the repo are down­loaded and stored in demo-sources.

stage(‘Pre­pare Workspace’) { dir(‘demo-sources’) { git url: ‘­manathan/osfy-pipelinedemo.git’, branch: ‘mas­ter’


While the git step is enough to han­dle most of the sce­nar­ios to down­load the source code, sit­u­a­tions might arise where the down­load (which is es­sen­tially a Git clone) could fail if the source code repos­i­tory is quite large, i.e., not just in terms of source code but other as­sets, like, bi­nary files, sys­tem im­ages, build pack­ages, and ex­ten­sive meta data, like branches and tags. The rem­edy for this is avail­able as an­other step— check­out.

Com­pil­ing the source code

From the dir, demo-sources, we will call the make util­ity pro­gram to com­pile the code and gen­er­ate the ob­ject ar­ti­fact. The make util­ity uses a make­file that helps to build a tree of de­pen­den­cies, de­fine tar­gets and vari­ables, and per­form ac­tions. This make­file is also avail­able in the root of the GitHub repo, men­tioned in the sec­tion ti­tled ‘Build Flow’.

The source code is in­side the src di­rec­tory at the root of the GitHub repos­i­tory. It is a tiny lit­tle pro­gram writ­ten in C lan­guage:

stage(‘Gen­er­ate Bi­nary’) { dir(‘demo-sources’) { sh “make pip­im­age”



To en­able the build ar­ti­fact to be down­loaded and used for fur­ther test­ing, we use the ar­chive step to save the out­put from the pre­vi­ous com­pile stage. Once archived, the ar­ti­fact can be down­loaded from the project page in Jenk­ins for sim­ple build projects. Jenk­ins sup­ports in­te­gra­tion with third-party ar­ti­fac­tory tools, like, for ex­am­ple, JFrog and Maven, which help to ver­sion and pro­mote the ar­ti­facts.

stage(‘Ar­chive’) { dir(‘demo-sources’) { ar­chive ex­cludes: ‘*.txt’, in­cludes: ‘pip­im­age’

} } }

The build ar­ti­fact, thus gen­er­ated, can be down­loaded by click­ing on the down ar­row but­ton that’s avail­able upon com­plet­ing the build. Its de­tails can also be checked on the project page.

Stage view

For ev­ery job run of the project that’s trig­gered either man­u­ally or au­to­mat­i­cally, the Pipeline plugin presents a vis­ual rep­re­sen­ta­tion of all the com­pleted and failed stages. If the build is suc­cess­ful, then the stages are shown in green, else, the failed stage is shown in red and the build stops there with­out ex­e­cut­ing any fur­ther stages in red.

If the Jenk­ins project is con­fig­ured to trig­ger au­to­matic builds on new com­mits in the con­fig­ured GitHub repo, which can be en­abled via We­bHooks, then the com­mit in­for­ma­tion will be dis­played along with the stages.

With this strat­egy to cod­ify the en­tire build flow by break­ing it up into its log­i­cal, con­stituent blocks and group­ing them to­gether, Jenk­ins Pipeline is in per­fect align­ment with the cur­rent trend to in­stru­ment the in­fra­struc­ture us­ing code. Of course, with the sup­port of 1000+ plu­g­ins avail­able in the ecosys­tem, there is a mul­ti­tude of other pos­si­bil­i­ties to ex­tend Jenk­ins ca­pa­bil­i­ties, such as, to de­tect and set the build sta­tuses, as well as send out no­ti­fi­ca­tions to Slack and HipChat. With BlueOcean, the re­vamped user in­ter­face (that of­fers the ca­pa­bil­ity to write Jenk­ins­file us­ing declar­a­tive syn­tax), Jenk­ins is cer­tainly evolv­ing with the times.

For big­ger projects, global Pipeline shared li­braries re­duce the main­te­nance bur­den by ab­stract­ing away the com­mon tasks from the main Jenk­ins­file. This ap­proach brings down the en­try bar­rier for en­gi­neer­ing teams adapt­ing or switch­ing to Jenk­ins Pipeline. Re­fer to the Jenk­ins doc­u­men­ta­tion at https://jenk­­braries/.

Fig­ure 2: Snip­pet gen­er­a­tor Steps Sam­ple Step archiveAr­ti­facts: Ar­chive the ar­ti­facts Files to ar­chive Ad­vanced... Gen­er­ate Pipeline Script

Fig­ure 1: Pipeline syn­tax

Script Path pipeline_bin/Jenk­ins­file.groovy

Fig­ure 8: Com­mit info

Fig­ure 7: Pipeline stages

Fig­ure 6: Ar­ti­fact down­load

Fig­ure 5: Build ar­ti­fact

Fig­ure 4: make­file

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.