Step 14: Start up the Secondary NameNode (if used)
At this point, you should be able to start the Secondary NameNode if you are using one:
$ sudo service hadoop-hdfs-secondarynamenode start
If you are using HDFS HA, do not use the Secondary NameNode. See Configuring HDFS High Availability for instructions on configuring and deploying the Standby NameNode.
You'll see some extra information in the logs such as:
10/10/26 12:03:18 INFO security.UserGroupInformation: Login successful for user hdfs/fully.qualified.domain.name@YOUR-REALM using keytab file /etc/hadoop/conf/hdfs.keytab
12/05/23 18:33:06 INFO http.HttpServer: Adding Kerberos (SPNEGO) filter to getimage 12/05/23 18:33:06 INFO http.HttpServer: Jetty bound to port 50090 12/05/23 18:33:06 INFO mortbay.log: jetty-6.1.26 12/05/23 18:33:06 INFO server.KerberosAuthenticationHandler: Login using keytab /etc/hadoop/conf/hdfs.keytab, for principal HTTP/fully.qualified.domain.name@YOUR-REALM.COM 12/05/23 18:33:06 INFO server.KerberosAuthenticationHandler: Initialized, principal [HTTP/fully.qualified.domain.name@YOUR-REALM.COM] from keytab [/etc/hadoop/conf/hdfs.keytab]
You should make sure that the Secondary NameNode not only starts, but that it is successfully checkpointing.
If you're using the service command to start the Secondary NameNode from the /etc/init.d scripts, Cloudera recommends setting the property fs.checkpoint.period in the hdfs-site.xml file to a very low value (such as 5), and then monitoring the Secondary NameNode logs for a successful startup and checkpoint. Once you are satisfied that the Secondary NameNode is checkpointing properly, you should reset the fs.checkpoint.period to a reasonable value, or return it to the default, and then restart the Secondary NameNode.
You can make the Secondary NameNode perform a checkpoint by doing the following:
$ sudo -u hdfs hdfs secondarynamenode -checkpoint force
Note that this will not cause a running Secondary NameNode to checkpoint, but rather will start up a Secondary NameNode that will immediately perform a checkpoint and then shut down. This can be useful for debugging.
If you encounter errors during Secondary NameNode checkpointing, it may be helpful to enable Kerberos debugging output. For instructions, see Appendix D - Enabling Debugging Output for the Sun Kerberos Classes.