CI/CD Pipeline With Github,Jenkins & kubernetes using Static & Dynamic cluster in Jenkins.

5 min readJun 4, 2020

An amazing trio……

In this article we will create a complete CD(continuous delivery) & CI(continuous Integration ) pipeline through which as soon as developer push the code , it will directly launch in the production environment , without Human intervention with the help of SCM(source code management ) tool : GitHub, Continuous Integration tool: jenkins, Containerization tool : Docker & Container Orchestration tool: kubernetes .

Here we are also using the powerful concept of jenkins “Master-slave architecture” to run job in jenkins. Jenkins slave can be of two types:static or dynamic. In case of static slave , we have to create a node earlier before run any job , But dynamic slave or dynamic cluster launches as soon as requirement come ,run job inside cluster & terminate it ,if job complete.

LET’s START :-

1. Developer’s Job :

Developer (take an example)write the Application code & Dockerfile :which create the image to launch the container to run the application, It will have all the necessary software to run the application.
Here we are taking the example of web application so image should have httpd services started & It’s configuration and all required things so developer write the code accordingly.

As soon as developer commit the code it pushes all the files automatically to the github because we are using post-commit hooks here.

automatically pushing all the files in GitHub after commit

In GitHub we are using webhooks which will send the event or a response to jenkins [using ngrok for communicating with jenkins’s private IP] ,& thus jenkins will trigger ,for run the job-1 which are using github-hook triggers method as a Build Trigger Option.

2. Jenkins’s Jobs:

The first job will be run on slave node [rhel-8-CLI node in my case ] , This job will build the Dockerfile which it pulls from github & push this image to the docker hub account.
Understand , Here images is created after the job-1 is run , so all the files related to web application will be copy at the time of image build , so if there is any change in code the image will be change & web-application also which is accessible to our client.

Here we have to set up label of static-cluster in order to run job-1 on another slave node[ rhel8 in my case]

For build & push image I am using execute shell option , you can use the by default option available in jenkins for building & pushing an image in docker hub.

Before pushing image in docker hub ,not forget to set your credentials for docker-hub account.

output of job-1 [pushing the httpd image in docker hub]

now If job-1 will be successfully run, then it will trigger the job-2 which will launch the dynamic slave using docker container.
For launching the dynamic cluster we have to do following setting to create a cloud node in jenkins:

→ Here we need to access the docker-daemon remotely , so we need to enable the tcp socket & do the following settings, Take reference from here.

→ In docker client you need to export DOCKER_HOST variable.

→ we also need a image to launch a container , here because we will be launch our application inside the k8s cluster & we are launching dynamic cluster or slave so the image should consists following things

sshd service running on port 22.[If using ssh method for starting slave agent in slave node]
Jenkins user with password.
java installed
All the required application dependencies for the build. For example here for running k8s we need to install & configure kubectl inside image.

kube image(name of my image) to launch docker container as a dynamic slave of jenkins

→ push this image to the your docker -hub account.

→ For launching docker container using jenkins job we need Docker plugins, go in Dashboard –> Manage Jenkins –> Manage Plugins. & install docker plugin from there.

→ Now configure cloud Node:

here I am using ssh key for authentication in order to launching slave agent

Note : This whole task of setting job-2 & cloud node should be done earlier before launching the job1.

For job-2 we need to set label of dynamic cluster.

set label which was given to the cloud node

command in execute shell option in order to launch deployment

Now the job-2 will be triggered by job-1,first of all It will launch the slave agent in dynamic slave node.

Here we can see that in remote system docker container is launched.

Now after agent launched it will run the job-2 commands given in execute shell, which will create a deployment using httpd image having our application code.

website running inside k8s cluster inside a pod

This website is running inside the pod launched by deployment , It is so powerful because if pod is deleted by any reason , deployment will launch the new pod again , Now we easily can do rolling updates or roll out , scaling & load-balancing of pods.
let’s take the scenario if developer change the code now , so in production environment it also have change without any down-time here roll-out come in role , look at this :
code change so developer again commit -> push the code on github -> job-1 run -> create new image with new code -> job-2 run -> now the pods is already exist -> this time roll-out of deployment will be done, because we give this command using shell scripting in execute shell option.