Comments on: Parallel LZO: Splittable Compression for Hadoop http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/ Hadoop and Cloudera's Products and Services Fri, 10 Feb 2012 20:11:24 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Active Intelligence http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-16407 Active Intelligence Thu, 06 Oct 2011 06:44:59 +0000 http://www.cloudera.com/blog/?p=902#comment-16407 [...] Parallel LZO: Splittable Compression… - cloudera.com [...] [...] Parallel LZO: Splittable Compression… - cloudera.com [...]

]]> By: WhatIf http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-16296 WhatIf Thu, 25 Aug 2011 13:38:18 +0000 http://www.cloudera.com/blog/?p=902#comment-16296 Do you intend to test snappy on Hadoop ? (using an external container format, as you mention) While at it, would it seem sensible to also compare with other BSD compression algorithms, such as LZF and LZ4 ? Best Rgds Do you intend to test snappy on Hadoop ? (using an external container format, as you mention)
While at it, would it seem sensible to also compare with other BSD compression algorithms, such as LZF and LZ4 ?
Best Rgds

]]> By: Todd Lipcon http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-16249 Todd Lipcon Wed, 10 Aug 2011 21:05:11 +0000 http://www.cloudera.com/blog/?p=902#comment-16249 Hi, Snappy itself doesn't provide a file format, just a compression scheme. That contrasts with LZO, which has a file format (LZOP) that has a block-based format that can be indexed, etc. With Snappy we recommend using a container format like SequenceFile or Avro DataFile. Regarding performance and compression ratios, look out for a blog post soon that will have some details. -Todd Hi,

Snappy itself doesn’t provide a file format, just a compression scheme. That contrasts with LZO, which has a file format (LZOP) that has a block-based format that can be indexed, etc.

With Snappy we recommend using a container format like SequenceFile or Avro DataFile.

Regarding performance and compression ratios, look out for a blog post soon that will have some details.

-Todd

]]> By: Gabriel http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-16245 Gabriel Wed, 10 Aug 2011 14:35:34 +0000 http://www.cloudera.com/blog/?p=902#comment-16245 I want to know the same is snappy spplitable ? I want to know the same is snappy spplitable ?

]]> By: ali8zake http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-16235 ali8zake Sun, 07 Aug 2011 04:26:47 +0000 http://www.cloudera.com/blog/?p=902#comment-16235 what is the differencde between lzo and snappy ,in hadooop compressway ,which is better ?tnanks. what is the differencde between lzo and snappy ,in hadooop compressway ,which is better ?tnanks.

]]> By: twitter engineering « Febryadi.com http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-11820 twitter engineering « Febryadi.com Mon, 10 May 2010 14:13:54 +0000 http://www.cloudera.com/blog/?p=902#comment-11820 [...] available to do real work. For more discussion of LZO, complete with performance comparisons, see this Cloudera blog post we did a while [...] [...] available to do real work. For more discussion of LZO, complete with performance comparisons, see this Cloudera blog post we did a while [...]

]]> By: Hadoop at Twitter (part 1): Splittable LZO Compression » Cloudera Hadoop & Big Data Blog http://www.cloudera.com/blog/2009/06/parallel-lzo-splittable-compression-for-hadoop/comment-page-1/#comment-7154 Hadoop at Twitter (part 1): Splittable LZO Compression » Cloudera Hadoop & Big Data Blog Tue, 17 Nov 2009 16:27:45 +0000 http://www.cloudera.com/blog/?p=902#comment-7154 [...] of Last.fm and soon to be joining us at Twitter (welcome, Johan!).  Chris Goffinet also wrote a blog post on the topic for Cloudera a while back, but many people we’ve talked to don’t [...] [...] of Last.fm and soon to be joining us at Twitter (welcome, Johan!).  Chris Goffinet also wrote a blog post on the topic for Cloudera a while back, but many people we’ve talked to don’t [...]

]]>