Comments on: Securing a Hadoop Cluster Through a Gateway http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/ Hadoop and Cloudera's Products and Services Fri, 10 Feb 2012 20:11:24 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Jeff http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-16069 Jeff Fri, 17 Jun 2011 06:07:25 +0000 http://www.cloudera.com/blog/?p=140#comment-16069 I think you made a mistake in the post. property hadoop.rpc.socket.factory.class.default should been set to org.apache.hadoop.net.SocksSocketFactory rather than org.apache.hadoop.net.StandardSocketFactory I think you made a mistake in the post.

property hadoop.rpc.socket.factory.class.default should been set to org.apache.hadoop.net.SocksSocketFactory rather than org.apache.hadoop.net.StandardSocketFactory

]]> By: Quora http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-14563 Quora Fri, 10 Dec 2010 17:12:42 +0000 http://www.cloudera.com/blog/?p=140#comment-14563 <strong>If I'm running a Hadoop job on Amazon Elastic MapReduce, how can I see how far along the job is?...</strong> In the past, I have created a SOCKS proxy by ssh'ing to the master node in the cluster.  Once you have done this, you can view the hadoop web ui for the job tracker, name node, etc.  The web ui offers job and task-level progress details as well as acc... If I’m running a Hadoop job on Amazon Elastic MapReduce, how can I see how far along the job is?…

In the past, I have created a SOCKS proxy by ssh’ing to the master node in the cluster.  Once you have done this, you can view the hadoop web ui for the job tracker, name node, etc.  The web ui offers job and task-level progress details as well as acc…

]]> By: Aaron http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-13300 Aaron Wed, 25 Aug 2010 23:47:16 +0000 http://www.cloudera.com/blog/?p=140#comment-13300 Hawksury, After you've got your cluster configured and have your local system configured to talk to it, you can copy your data up to HDFS, and then operate on it with Hive as usual. You may want to watch our training video on how to use Hive: http://www.cloudera.com/videos/introduction_to_hive - Aaron Hawksury,

After you’ve got your cluster configured and have your local system configured to talk to it, you can copy your data up to HDFS, and then operate on it with Hive as usual.

You may want to watch our training video on how to use Hive: http://www.cloudera.com/videos/introduction_to_hive

- Aaron

]]> By: Hawksury http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-13297 Hawksury Wed, 25 Aug 2010 17:37:03 +0000 http://www.cloudera.com/blog/?p=140#comment-13297 Great post mate, I have few questions would much appreciate if you could help/answer. The data I want to process is on a local machine (3TB in size) i want to use EC2 machines to process this data by submitting Hive Queries from a local machine that is outside of Amazon EC2. Could you please help or point me to the correct post where the process is explained. Many Thanks, Hawksury Great post mate, I have few questions would much appreciate if you could help/answer.

The data I want to process is on a local machine (3TB in size) i want to use EC2 machines to process this data by submitting Hive Queries from a local machine that is outside of Amazon EC2.

Could you please help or point me to the correct post where the process is explained.

Many Thanks,
Hawksury

]]> By: Andy http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-4981 Andy Fri, 11 Sep 2009 01:01:11 +0000 http://www.cloudera.com/blog/?p=140#comment-4981 Please ignore my previous comment. Wrong configuration parameter was used ( dfs.web.ugi ). Please ignore my previous comment. Wrong configuration parameter was used ( dfs.web.ugi ).

]]> By: Andy http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-4979 Andy Fri, 11 Sep 2009 00:42:07 +0000 http://www.cloudera.com/blog/?p=140#comment-4979 Great post! Very helpful and much appreciated. You mention you use hadoop.job.ugi to set the username/group for submitted jobs. Should that configuration also work for basic hdfs operations as well ( e.g., -copyFromLocal )? My quick attempt ( with hadoop 0.20 from a windows vista client ) did not seem to honor hadoop.job.ugi for 'hadoop dfs' commands. Great post! Very helpful and much appreciated.

You mention you use hadoop.job.ugi to set the username/group for submitted jobs. Should that configuration also work for basic hdfs operations as well ( e.g., -copyFromLocal )? My quick attempt ( with hadoop 0.20 from a windows vista client ) did not seem to honor hadoop.job.ugi for ‘hadoop dfs’ commands.

]]> By: Aaron http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-3855 Aaron Tue, 04 Aug 2009 17:31:26 +0000 http://www.cloudera.com/blog/?p=140#comment-3855 yes. yes.

]]> By: Steven Wong http://www.cloudera.com/blog/2008/12/securing-a-hadoop-cluster-through-a-gateway/comment-page-1/#comment-3394 Steven Wong Wed, 15 Jul 2009 01:15:51 +0000 http://www.cloudera.com/blog/?p=140#comment-3394 Question: "an HTTP Forwarding Proxy on the gateway node" -- are you referring to a reverse proxy? Question: “an HTTP Forwarding Proxy on the gateway node” — are you referring to a reverse proxy?

]]>