Enable WebHDFS

Elena Kostyuchenko

WebHDFS is a service that provides HTTP REST API access to HDFS.

In ADH, WebHDFS is enabled by default. For more information on how to connect to HDFS using HTTP REST API, see the Connect to HDFS article.

For secure clusters, configure additional parameters in hdfs-site.xml:

  1. Create a new principal for the HTTP service:

    $ addprinc -randkey HTTP/$<HOST>@$<REALM>.COM

    Where <HOST> is the FDQN or IP address of the NameNode’s host, and <REALM> is the name of the Kerberos realm.

  2. Generate the keytab for the HTTP service:

    $ xst -norandkey -k /etc/security/spnego.service.keytab HTTP/$<HOST>

    Where <HOST> is the FDQN or IP address of the NameNode’s host.

  3. Make sure that the keytab is connected to the service:

    $ klist –k -t /etc/security/spnego.service.keytab

  4. Set the principal created earlier as the dfs.web.authentication.kerberos.principal parameter’s value and the keytab as the dfs.web.authentication.kerberos.keytab. The result should look like this:

    <property>
        <name>dfs.web.authentication.kerberos.principal</name>
        <value>HTTP/$<HOST>@$<REALM>.COM</value>
</property>
<property>
        <name>dfs.web.authentication.kerberos.keytab</name>
        <value>/etc/security/spnego.service.keytab</value>
</property>

  5. Reboot the NameNode and DataNode services.

Found a mistake? Seleсt text and press Ctrl+Enter to report it