Pig Key Features
Simple Language:
Leverage the simple scripting language, Pig Latin, to perform complex data transformations, aggregations, and analysis. Queries are translated into MapReduce or Apache Spark jobs, making it easy for more users to process and analyze unlimited amounts of data.
Shared Data Structures:
Using HCatalog, a table and storage management layer for Hadoop, Pig can work directly with Hive metadata and existing tables, without the need to redefine schema or duplicate data. This flexibility allows users to easily read and write data without facing concerns about where the data is stored, its format, or redefining the structure for every processing tool.
Common Use Cases
With its simple scripting language, Pig makes Hadoop data accessible for a variety of batch processing workloads, including:
- Data preparation
- ETL
- Data mining
Integrated across the platform
As an integrated part of Cloudera’s platform, users can run batch processing workloads with Apache Pig, while also analyzing the same data for interactive SQL or machine learning workloads using tools like Impala or Apache Spark — all within a single platform. Pig also benefits from unified resource management (through YARN), simple deployment and administration (through Cloudera Manager) and shared compliance-ready security and governance (through Apache Sentry and Cloudera Navigator) — all critical for running in production.
The shift to Pig-on-Spark
Apache Spark is a powerful data processing engine that has quickly emerged as an open standard for Hadoop due to its added speed and greater flexibility. Together with the community, Cloudera has been working to evolve the tools currently built on MapReduce, including Hive and Pig, and migrate them to the Spark execution engine for faster processing.
Pig-on-Spark is available in alpha on Github.
Partnered with the ecosystem
Seamlessly integrate with the tools your business already uses by leveraging Cloudera’s 1,700+ partner ecosystem. With a robust partner certification program, we are continuously working to build out production-hardened integrations between Pig and the most popular third-party tools.
Expert support for Pig
Cloudera has Pig experts available across the globe ready to deliver world-class support 24/7. With more experience across more production customers, for more use cases, Cloudera is the leader in Pig support so you can focus on results.
