Thursday, November 15, 2012

Hadoop File System Metadata Replication By Using Secondary Name Node


FSIMAGE and EDITS are the two most important files of name node. FSIMAGE is having snapshot of all the metadata of the Hadoop cluster whereas EDITS contains the incremental information of metadata. The reason for maintaining the incremental information in another file because it requires the write operation whereas FSIMAGE is served directly from the RAM. Over the time EDITS log file grows and in case of failure of name node, it could take a longer time to load the stuff in FSIMAGE.

How to eliminate the problem in case of failure of primary name node?

The solution would be secondary name node. The logic is as simple as we do replace the two numbers by using the temp variable. Below are the messages exchange between primary name node and secondary name node.

1.Secondary name node sends a message to primary name node to start writing the incremental messages in EDITS.NEW file.
2.Secondary name node copies the FSIMAGE and EDITS file from the primary name node.
3.Secondary name node adds the EDITS to the FSIMAGE and make a new FSIMAGE file.
4.Sends it to the primary name node.
5.Primary name node renames the EDITS.NEW file to EDITS.

People who read this post also read :



13 comments:

Jhon Abraham said...

You have done a great job by explaining what is hadoop file system metadata is all about. Thanks for sharing this in here.

Big Data Course in Chennai

priya said...

i gathered knowledge info from your blog about hadoop.
Hadoop Training in chennai

sarah taylor said...

Thanks for your informative guide on SQL technology. As we all know, SQL is most used database tool and your article offer clear insight about the technology. Your blog is recommended for freshers and experienced ones looking for ideal blog to learn SQL technology.
Java Training in Chennai
.Net Training in Chennai
PHP Training in Chennai

robin singh said...

The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling. Hadoop Training in Chennai | Big Data Training in Chennai

murali karthik said...

Hi, JAVA is not at all difficult all you need is to clear your concepts mainly because the concepts of JAVA programming have been taken from day to day life examples. J2EE Training in Chennai | JAVA Training in Chennai

savitha singh said...

Software testing industry offer huge career opportunity for talented professionals. software testing training in Chennai

murali karthik said...

For management and troubleshooting of enormous databases, corporations square measure searching for qualified and licensed dispersions Hadoop experts for the duty. Hadoop Training in Chennai

murali karthik said...

Thanks for your informative article. Android SDK allows you to create stunning mobile application loaded with more features and enhanced priority. With basis on Java coding language, you can create stunning mobile application with ease. Android Training Institutes in Chennai | Android Training in Chennai

savitha singh said...

Informative post on Android mobile application development!!! As the usage of iPhones keep on increasing, there is massive demand for best performing Apps and Games for iOS platform. Best IOS Training in Chennai

kovalan J said...

Thanks for sharing detailed information of unified functional testing automation tool. QTP Course in Chennai | QTP training

Yasmin Priya said...

Thanks for sharing informative post. Big data is a term that portrays the substantial volume of information; both organized and unstructured that immerses a business on an everyday premise. To know more details please visit Big Data Training in Chennai | Primavera Training in Chennai

kovalan Jayamurugan said...

Thanks for sharing informative article on java application development. Your post helped to understand the career in Java. JAVA Training in Chennai

jhansi joe said...

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.
Regards,
Hadoop Training in Chennai|Big Data Training in Chennai