开发者

The right place for "io.sort.mb" in Hadoop?

开发者 https://www.devze.com 2023-04-07 10:59 出处:网络
I am a bit confused, in the Hadoop cluster setup, in section \"Real-World Cluster Configurations\", an example is given where properties like io.sort.mb & io.sort.factor goes in core-site.xml. But

I am a bit confused, in the Hadoop cluster setup, in section "Real-World Cluster Configurations", an example is given where properties like io.sort.mb & io.sort.factor goes in core-site.xml. But in the default configuration files these properties appears in mapred-site.xml!!! which one I should follow? Am i free to put it anywhere? If yes, what is the difference be开发者_如何学JAVAtween the three configuration file (core, mapred, and hdfs .xml)?

Is there any complete list of all possible properties?

Any help is much appreciated..

EDIT: Trial and error I found io.sort.mb belongs to mapred-site.xml.


core-default.html, hdfs-default.html and mapred-default.html have all the properties with their defaults.

According to the 'Hadoop : The Definitive Guide'

core-site.xml has 'Configuration settings for Hadoop Core, such as I/O settings that are common to HDFS and MapReduce.'

hdfs-site.xml has 'Configuration settings for HDFS daemons: the namenode, the secondary namenode, and the datanodes.'

mapred-site.xml had 'Configuration settings for MapReduce daemons: the jobtracker, and the tasktrackers.'

Find more information about the configuration files here.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号