Site Sponsors:
Man -v- Pig 
Whenever something is slow, unwieldy & wallowing in muck, colloquial English has reserved an animal cat-call for us to use... Never as responsive as stand-alone Map Reduce (well, in production at least!) such is obviously the legacy for Hadoop Pig.

Yet while Pig is indeed getting faster with each and every release (Gotta love C/C++), sometimes piggy-job slowdowns can get downright scary. Indeed, those who use HUE and / or the Hortonworks VM via the Pig-Page will feel that we have encountered a new performance-low.

Not to worry though - When working from the VM Command line, those 26 & 48 second 3-line pig-scripts will get back to running in well under 2 - 3 seconds. (Really!)

The way is easy: Simply log-into your VM via the console prompt so as to execute Pig directly:
pig -x local

Of course, once running Pig, the massive amount of information and warning messages can also seem positively ludicrous.

Here again, in the Man -over -Pig struggle at Hadoop Acres, a local configuration file is all that we need:

Useful for both R&D as well impromptu Production spelunking, here be the content for log4j.prop:

The cut-and-paste fodder for starting up our quiet-pig pen is simply:
pig -x local -4 log4j.prop

You will still be able to get those all-important errors:

Finally, simply leave-off that -4 clause when you want all of that glorious detail back again.

Enjoy the Journey!



Add Comment
Comments are not available for this entry.