Site Sponsors:
HUE File Browser Exception / Error 
Those who are using HUE on the HDP 2 (and probably elsewhere) will - from time to time - see the annoying "Error 113 - No route to host" Exception:

WebHdfsException at /filebrowser/

<urlopen error [Errno 113] No route to host>

Request Method: GET
Request URL: http://192.111.1.237:8000/filebrowser/
Django Version: 1.2.3
Exception Type: WebHdfsException
Exception Value:

<urlopen error [Errno 113] No route to host>

Exception Location: /usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py in _stats, line 209
Python Executable: /usr/bin/python2.6
Python Version: 2.6.6
Python Path: ['', '/usr/lib/hue/build/env/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Babel-0.9.6-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/BabelDjango-0.2.2-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.7.2-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Markdown-2.0.3-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/MarkupSafe-0.9.3-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Paste-1.7.2-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/PyYAML-3.09-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Pygments-1.3.1-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/South-0.7-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/Spawning-0.9.6-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/avro-1.5.0-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/configobj-4.6.0-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/django_auth_ldap-1.0.7-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/django_extensions-0.5-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/django_nose-0.5-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/elementtree-1.2.6_20050316-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/enum-0.4.4-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/eventlet-0.9.14-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/greenlet-0.3.1-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/happybase-0.6-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/kerberos-1.1.1-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/lockfile-0.8-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/lxml-2.2.2-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/moxy-1.0.0-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pam-0.1.3-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pyOpenSSL-0.13-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pycrypto-2.6-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pysqlite-2.5.5-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/python_daemon-1.5.1-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/python_ldap-2.3.13-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/pytidylib-0.2.1-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/sasl-0.1.1-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/sh-1.08-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/simplejson-2.0.9-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/threadframe-0.2-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/thrift-0.9.0-py2.6-linux-x86_64.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/urllib2_kerberos-0.1.6-py2.6.egg', '/usr/lib/hue/build/env/lib/python2.6/site-packages/xlrd-0.9.0-py2.6.egg', '/usr/lib/hue/desktop/core/src', '/usr/lib/hue/desktop/libs/hadoop/src', '/usr/lib/hue/desktop/libs/liboozie/src', '/usr/lib/hue/build/env/lib/python2.6/site-packages', '/usr/lib/hue/apps/about/src', '/usr/lib/hue/apps/beeswax/src', '/usr/lib/hue/apps/filebrowser/src', '/usr/lib/hue/apps/hcatalog/src', '/usr/lib/hue/apps/help/src', '/usr/lib/hue/apps/jobbrowser/src', '/usr/lib/hue/apps/jobsub/src', '/usr/lib/hue/apps/oozie/src', '/usr/lib/hue/apps/pig/src', '/usr/lib/hue/apps/proxy/src', '/usr/lib/hue/apps/shell/src', '/usr/lib/hue/apps/useradmin/src', '/usr/lib/hue/build/env/bin', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/lib/hue/apps/beeswax/gen-py', '/usr/lib/hue', '/usr/lib64/python26.zip', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/lib/hue/apps/beeswax/src/beeswax/../../gen-py', '/usr/lib/hue/apps/jobbrowser/src/jobbrowser/../../gen-py', '/usr/lib/hue/apps/proxy/src/proxy/../../gen-py']
Server time: Sat, 14 Dec 2013 01:23:12 -0800

While disconcerting, rest assured that there is nothing wrong with your VM. This is a connectivity thing!



Repeat: Until a fix is available for this buglet, there is no need to re-install your VM! (You might, however, want to bump-up your VM response time by manually cranking-up the processor speed on the VM Host.)

Environment:

Request Method: GET
Request URL: http://192.111.1.237:8000/filebrowser/
Django Version: 1.2.3
Python Version: 2.6.6
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'django_extensions',
'south',
'babeldjango',
'desktop',
'about',
'beeswax',
'filebrowser',
'hcatalog',
'help',
'jobbrowser',
'jobsub',
'oozie',
'pig',
'proxy',
'shell',
'useradmin']
Installed Middleware:
['desktop.middleware.DatabaseLoggingMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.common.CommonMiddleware',
'desktop.middleware.SessionOverPostMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'desktop.middleware.SpnegoMiddleware',
'desktop.middleware.HueRemoteUserMiddleware',
'django.middleware.locale.LocaleMiddleware',
'babeldjango.middleware.LocaleMiddleware',
'desktop.middleware.AjaxMiddleware',
'desktop.middleware.LoginAndPermissionMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'desktop.middleware.NotificationMiddleware',
'desktop.middleware.JFrameMiddleware',
'desktop.middleware.ExceptionMiddleware',
'desktop.middleware.ClusterMiddleware',
'desktop.middleware.AppSpecificMiddleware',
'django.middleware.transaction.TransactionMiddleware',
'desktop.middleware.XFrameOptionsMiddleware']


Traceback:
File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/hue/apps/filebrowser/src/filebrowser/views.py" in index
91. if not request.fs.isdir(path):
File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py" in isdir
224. sb = self._stats(path)
File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py" in _stats
209. raise ex

Exception Type: WebHdfsException at /filebrowser/
Exception Value: <urlopen error [Errno 113] No route to host>

Whenever we see a Error 113, what needs to happen is a simple re-start.

Restarting the Sandbox


To clear the problem we need to:

(1) Close all browser windows, as well as browser sessions (important!)

(2) Re-start the VM.



NOTE: To re-start your Sandbox properly, simply log into the OS as root via the main HDP console so as to enter shutdown -r 0 at the main console prompt.

(No, shutdown will not blink like that - Except maybe around Christmas ;oP )

The Unix / Linux Manual


For more information on the Shutdown command (or just about any other common command utility), the next time you are at the Console enter man shutdown, where 'man' is the mnemonic for manual, and shutdown can be replaced with any other command you might be interested in (for example, man tar, man bash, or even man man :)


Obvious pun??


If things scroll too fast, try:

man shutdown | more


[ view entry ] ( 1147 views )   |  permalink
Hadoop 2.2.0 in the Hortonworks Sandbox (HDP 2) 
For the benefit of my fellow Hadoop-ies, as I work thru the evolving coolness of the Hadoop User Experience (HUE) under HDP 2 I wanted to pass in a few observations by-value.

Use Java 6


While we could use Java 1.7, note that the following message means that we had better not use the same as the default compatibility mode:


The google-candy here is:

Java MapReduce
MaxTemperature : Unsupported major.minor version 51.0


What the run time is trying to tell us is to use Java 1.6.

Logging into the VM will - of course - verify as much:



The solution is to simply re-deploy as a JDK 1.6 rendition. Even the Job Designer's "Job Design" need not change.

Moving the source code from the 3rd edition of O'Reilly's "Hadoop: The Definitive Guide" by Tom White to Hadoop 2.2.0, I am also bumping into allot of compatibility warnings and issues.

Stay tuned!!

[ view entry ] ( 53895 views )   |  permalink
PC Signaling & IPC Efficiencies 
Linkedin: Observations on Qt Signalling:

Qt Design Lamentations


I still do not understand what was wrong with simply using structs... Indeed, even when it comes to IPC, savvy consultants have always liked marshalling such things into XDR/RPC packages.... Allot like "Bundles" or "Parsables" on Android, once a standards-based out-of-process marshalling has been ensured, data can pretty much go everywhere. The only benefit to not having so many struct-version compile-time changes is to ensure that any time-frozen stringy-application will break someday?

Data Interopt


When it comes to IPC, savvy consultants have also always liked XDR/RPC. The cornerstone of CORBA, many are gratified that - years after Microsoft upset the apple-cart with their COM/DCOM nightmare - that places like Google's GWT are finally abandoning the fat-and-chatty XML (and even JSON!) world for things like good 'old XDR.

The CORBA/DCOM War


Why did we ever get into this nightmare world of passing strings? Because - at the time - Microsoft did not like things like XDR because the transmission favoured Motorola (natural order / big-endian) format over that strange INTEL (little-endian) format.
(click to enlarge)

Many of my friends at MS felt like they were being picked on. Today however, with AMD / INTEL computing speeds being what they are in multicore, who the heck cares?

Trade-Offs


Looking at messages more as octets than double-byte characters results in transmissions that are ever-so marvellously faster, as well as often laughingly more efficient & secure than these "string things." While understanding protocols from the wire-up is tougher on the commodity-developer, taking the time to learn about things like binary efficiencies is invariably massively easier on the computer. --Many feel that weighing system performance (as well as genuine security!) over developer-ease and industry convention (homogeneity is - by very definition - insecure) is one of many long forgotten absolute necessities of designing larger-scale enterprise systems.

Lean, Mean ... and Green?


Of course, better performance means allot less switches, routers, network traffic, and computers... Such effective weight loss surely spells less rack-space, lower utility costs ... and is therefore allot easier on this spaceship that we call earth, too?

While knowledge is indeed power, perhaps wisdom might yet save our planet? :-)


Indeed, when it comes to why-the-heck use anything other than structs or XDR under QT, I remember wondering why folks ever used such silly & unsafe string-mechanism as that Qt signal tomfoolery. --For decades, marshaling structs between things like CORBA skeletons / signatures have been doing much the same thing, yet providing full-speed end-to-end type safety via delivery systems such as RPC. When it comes to stark type-safety & efficiency, why not use it between process, as well as between computers? While a bit overkill for local signal-signatures, rather than being so string-generic even a humble struct surely might be used?

CORBA Problems


Of course, with the CORBA type safety comes a problem of routing / scaling / and using generic turnstiles such as what we see in Spring Integration. Resolving those problems are what the Internet Inter-ORB Protocol (IIOP) and CORBA Services are all about.

Olde-School Bestfficiencies


Yet, come to discussing Java in a C/C++ forum (bad idea?), it is fun to note that both generic (common message header -w- size fields (etc)) - as well as all competent security (bit-level field encoding, etc.) remains ever in the realm of the absolute efficiency folded into those 1970-someting transactions... -For more secured transactions, even today it seems that only a blast to that transactional-past can be allot more readily 'rollingly obfuscated... as well as a whole lot more efficient?

[ view entry ] ( 1909 views )   |  permalink  |  related link

<<First <Back | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | Next> Last>>