开发者

How to control file assignation in different slave in hadoop distributed system?

开发者 https://www.devze.com 2023-04-06 04:18 出处:网络
How to control file assignation in different slave in hadoop distributed system? Is it possible to write 2 or more file in hadoo开发者_运维知识库p as map reduce task Simultaneously?
  1. How to control file assignation in different slave in hadoop distributed system?
  2. Is it possible to write 2 or more file in hadoo开发者_运维知识库p as map reduce task Simultaneously?

I am new to hadoop.It will be really helpful to me. If you know please answer.


This is my answer for your #1:

You can't directly control where map tasks go in your cluster or where files get sent in your cluster. The JobTracker and the NameNode handle these, respectively. The JobTracker will try to send the map tasks to be data local to improve performance. (I had to guess what you meant for your question , if I didn't get it right, please elaborate)

This is my answer for your #2:

MultipleOutputs is what you are looking for when you want to write multiple files out from a single reducer.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号