开发者

Executing Pig on another framework

开发者 https://www.devze.com 2023-04-13 00:06 出处:网络
I understand that Pig Latin is a data flow language. In that sense it should be theoretically possible to execute Pig Latin in any framework though currently and it is meant to be executed in a Hadoop

I understand that Pig Latin is a data flow language. In that sense it should be theoretically possible to execute Pig Latin in any framework though currently and it is meant to be executed in a Hadoop enviornment. How hard would it be to switch Pig Latin to run on a different framework? Are there any extension points for this if at all or is开发者_StackOverflow中文版 Pig Latin is tightly coupled to Hadoop?


PigLatin is tightly coupled to Hadoop. Basically, the PigLatin script is interpreted into a series of Hadoop MapReduce jobs and there isn't really much way of swapping out that underlying data computation layer.

You can write a custom loader and custom writer to load from and store to custom data stores, but in the end it is still starting a Hadoop MapReduce job.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号