Configure Hadoop Cluster Using Ansible Playbook..

In this blog I will tell you how we can configure the target nodes as data node and master node using ansible playbook that will create a Hadoop Cluster..

Let’s start..

Step 1:

• Install the ansible in controller node.
• Create Configuration file in it.

• Create Inventory files with the IP, login name and password of managed node or target node, In my case I have two target nodes , one for master node and other for data node that will create a Hadoop cluster.

Step 2:

• Create Configuration files for data nodes and master nodes. We can use modules like lineinfile or replace module that will directory configure the files in data node and master nodes, but for easiness I am creating these files in ansible controller node using jinja templates and will copy in target nodes using template module.
• Template module will parse these files and will replace the variables.

core-site.xml file for name node or master node

hdfs-site.xml file for master node

hdfs-site.xml file for data node and core-site.xml will be same as master node.

Step 3:

• Ansible-playbook for configure Hadoop-cluster.

• Run the ansible playbook.

Step 4 :

• Check the target nodes.

Target Node — 1: Data node is running can see via jps command .

Target Node — 2 : Name node is running.

• Can see the data node present in Hadoop cluster via this command that is used to check the report : Information of data nodes, storage details and many other things regarding Hadoop cluster.

hadoop dfsadmin -report

Showing one data node in cluster, which we configured using ansible.

Hurrah !! Task completed..

Thanks for Scrolling. :-)