开发者

Add Entire Files Text as Map Key in Hadoop

开发者 https://www.devze.com 2023-03-21 19:42 出处:网络
I\'m looking for a way to loa开发者_StackOverflow中文版d an entire file text into my map.Not a single line at a time like TextInputFormat does.

I'm looking for a way to loa开发者_StackOverflow中文版d an entire file text into my map. Not a single line at a time like TextInputFormat does.

So that when I do value.toString in my map it gives me the entire input to work with.


You have to put every line into a StringBuilder until you've reached the end of file. Or you override your own RecordReader that provides this functionality. But I would not recommend this.


I would path name of the file to the mapper, and then will be free to load is entirely or do some kind of streaming processing.

0

精彩评论

暂无评论...
验证码 换一张
取 消