Skip to content Skip to sidebar Skip to footer
Showing posts with the label Hadoop

Hadoop Streaming: Where Are Application Logs?

My question is similar to : hadoop streaming: how to see application logs? (The link in the answer … Read more Hadoop Streaming: Where Are Application Logs?

Encountered Ioexception While Registering Python Udf In Pig. File Helloworld.py Does Not Exist

Pytjon UDF : @outputSchema('word:chararray') def helloworld(): return 'Hello, World&#… Read more Encountered Ioexception While Registering Python Udf In Pig. File Helloworld.py Does Not Exist

Is It Possible To Save Files In Hadoop Without Saving Them In Local File System?

Is it possible to save files in Hadoop without saving them in local file system? I would like to do… Read more Is It Possible To Save Files In Hadoop Without Saving Them In Local File System?

Subprocess Popen To Run Commands (hdfs/hadoop)

I am trying to use subprocess.popen to run commands on my machine. This is what I have so far cmdve… Read more Subprocess Popen To Run Commands (hdfs/hadoop)

Running Hadoop Jar Using Luigi Python

I need to run a Hadoop jar job using Luigi from python. I searched and found examples of writing ma… Read more Running Hadoop Jar Using Luigi Python

Reading And Writing From Hive Tables With Spark After Aggregation

We have a hive warehouse, and wanted to use spark for various tasks (mainly classification). At tim… Read more Reading And Writing From Hive Tables With Spark After Aggregation

Running A Job Using Hadoop Streaming And Mrjob: Pipemapred.waitoutputthreads(): Subprocess Failed With Code 1

Hey I'm fairly new to the world of Big Data. I came across this tutorial on http://musicmachin… Read more Running A Job Using Hadoop Streaming And Mrjob: Pipemapred.waitoutputthreads(): Subprocess Failed With Code 1

Loading A Defaultdict In Hadoop Using Pickle And Sys.stdin

I posted a similar question about an hour ago, but have since deleted it after realising I was aski… Read more Loading A Defaultdict In Hadoop Using Pickle And Sys.stdin