Thursday, November 15, 2012

Hadoop File System Metadata Replication By Using Secondary Name Node

FSIMAGE and EDITS are the two most important files of name node. FSIMAGE is having snapshot of all the metadata of the Hadoop cluster whereas EDITS contains the incremental information of metadata. The reason for maintaining the incremental information in another file because it requires the write operation whereas FSIMAGE is served directly from the RAM. Over the time EDITS log file grows and in case of failure of name node, it could take a longer time to load the stuff in FSIMAGE.

How to eliminate the problem in case of failure of primary name node?

The solution would be secondary name node. The logic is as simple as we do replace the two numbers by using the temp variable. Below are the messages exchange between primary name node and secondary name node.

1.Secondary name node sends a message to primary name node to start writing the incremental messages in EDITS.NEW file.
2.Secondary name node copies the FSIMAGE and EDITS file from the primary name node.
3.Secondary name node adds the EDITS to the FSIMAGE and make a new FSIMAGE file.
4.Sends it to the primary name node.
5.Primary name node renames the EDITS.NEW file to EDITS.

People who read this post also read :

1 comment:

Jhon Abraham said...

You have done a great job by explaining what is hadoop file system metadata is all about. Thanks for sharing this in here.

Big Data Course in Chennai