Local read
Local read, or the "short-circuit" read, is an HDFS setting that enables reading files directly from the source, bypassing the DataNode. This approach speeds up data reading, which can result in a substantial performance boost for your applications.
In ADH, local read is turned on by default. To be able to use it, the client must be in the same location as the data.
In hdfs-site.xml, the default local reads parameters are:
-
dfs.client.read.shortcircuit—True, -
dfs.domain.socket.path—/var/lib/hadoop-hdfs/dn_socket.
If necessary, you can change these values via ADCM:
-
On the Clusters page, select the desired cluster.
-
Go to the Services tab and click at HDFS.
-
Find a parameter, change its value, and click Apply.
-
Confirm changes to HDFS configuration by clicking Save.
-
In the Actions drop-down menu, select Restart, make sure the Apply configs from ADCM option is set to true and click Run.
To view local reads statistics, open the Metrics page of the DataNode UI and find the ReadsFromLocalClient parameter. It indicates the number of local reads.