Site Sponsors:
Man -v- Pig 
Whenever something is slow, unwieldy & wallowing in muck, colloquial English has reserved an animal cat-call for us to use... Never as responsive as stand-alone Map Reduce (well, in production at least!) such is obviously the legacy for Hadoop Pig.



Yet while Pig is indeed getting faster with each and every release (Gotta love C/C++), sometimes piggy-job slowdowns can get downright scary. Indeed, those who use HUE and / or the Hortonworks VM via the Pig-Page will feel that we have encountered a new performance-low.

Not to worry though - When working from the VM Command line, those 26 & 48 second 3-line pig-scripts will get back to running in well under 2 - 3 seconds. (Really!)

The way is easy: Simply log-into your VM via the console prompt so as to execute Pig directly:
pig -x local

Of course, once running Pig, the massive amount of information and warning messages can also seem positively ludicrous.



Here again, in the Man -over -Pig struggle at Hadoop Acres, a local configuration file is all that we need:



Useful for both R&D as well impromptu Production spelunking, here be the content for log4j.prop:
log4j.logger.org.apache.pig=ERROR
log4j.logger.org.apache.hadoop=ERROR

The cut-and-paste fodder for starting up our quiet-pig pen is simply:
pig -x local -4 log4j.prop

You will still be able to get those all-important errors:



Finally, simply leave-off that -4 clause when you want all of that glorious detail back again.


Enjoy the Journey!

-Rn

[ add comment ] ( 1553 views )   |  permalink  |  related link
New Project: Hadoop9000 
For the benefit of folks who will either be taking our Hadoop training, or are interested in having a copy of the latest stable Hadoop (a copy that they can debug / tweak / re-build on-their-own), then we just created the "Hadoop 9000" Project.

Weighing in at 3.5GB, the VirtualBox Virtual Machine (OVA Appliance) will be uploaded sometime this week.

Cheers,

-Rn


[ add comment ] ( 1154 views )   |  permalink  |  related link

<<First <Back | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | Next> Last>>