New Core Security Layer Brings Fine-Grained Access Controls and Dynamic Data Masking to Apache Spark, MapReduce, and More - Now Available as Public Beta
PALO ALTO, Calif., September 28, 2015 – Cloudera, the leader in enterprise analytic data management powered by Apache Hadoop™, today announced the public beta release of RecordService, a new high-performance security layer for Apache Hadoop that centrally enforces role-based access control policies across the platform. Complementing Apache Sentry (incubating), which provides unified policy definition, RecordService is the first solution to deliver complete row- and column-based security, and dynamic data masking, for every Hadoop access engine. A public beta of RecordService is immediately available under the Apache open source license, and will be transitioned to the Apache Software Foundation in the future.
Security has been a critical requirement for enterprises as their Hadoop deployments move from pilot to production. To ensure that sensitive data cannot fall into the wrong hands, a comprehensive security approach must include fine-grained access controls that define what data users can see and what they can do with it. However, for enterprises to take full advantage of the power of Hadoop, these access controls cannot limit the agility of the platform or the users. These access permissions must be the same for the user, regardless of their access engine of choice, down to the row and column-level.
Sentry, the standard for unified policy definition in Hadoop, plays a crucial role in addressing this by applying consistent policies across all the different access paths. However, some access paths support more granular restrictions than others. As the Hadoop ecosystem has expanded to include diverse access engines such as Apache Spark, Impala, and Apache Solr, it has been challenging to enforce these policies consistently without limiting access to the data itself. For example, while Impala supports granular row- and column-level controls, Spark and MapReduce only support file- or table-level controls. In modern architectures that use multiple access engines, this variance can lead to complicated workarounds or a reliance on the "lowest common denominator" of security, with users either having access to the entire data set or no access at all — ultimately leading to less agile and less secure Hadoop applications.
RecordService complements the policy definition of Sentry as a new layer that provides a single point of enforcement — simplifying security with unified row- and column-level controls for all access paths, including Spark and MapReduce. This fine-grained policy enforcement allows enterprises to take advantage of the full capabilities of Hadoop while eliminating the complex security workarounds that can often lead to security errors. With RecordService, all users can gain insights from their data, securely, using their tool of choice.
"There have been rapid improvements made to Hadoop security, addressing the growing need to store and analyze sensitive data in the platform. However, for Hadoop to continue to evolve and support the next generation of analytics for more users and access paths, security needs to become universal across the platform," said Eddie Garcia, chief security architect, Cloudera. "With RecordService, the Hadoop community fulfills the vision of unified fine-grained access controls across every Hadoop access path. Together, with leading security advances such as Sentry, RecordService enables enterprises to gain powerful insights from Hadoop with the confidence that a powerful, core security layer is protecting their most sensitive data."
By enabling the first, native, granular controls for Apache Spark specifically, RecordService plays a key role in advancing the development of Spark to meet enterprise requirements. With security being a core focus area of Spark development, as part of Cloudera’s recently announced One Platform Initiative, RecordService is a critical project to help Spark further progress towards becoming the next default processing engine for Hadoop.
As a new layer that sits between Hadoop’s storage and compute engines, RecordService not only consistently enforces the fine-grained, role-based access controls defined by Sentry, but it also provides dynamic data masking across Hadoop. This capability enables organizations to mask sensitive elements of data, protecting it as it is accessed in real-time. With dynamic data masking, more users can access and analyze data without needing permissions for the data itself. The platform offers advanced protection that has never before been available within core Hadoop security.
In regulated industries, advanced security is critical for protecting sensitive data without limiting the analytic agility necessary for competitive advantage. Capital One is one of the first early users of the beta of RecordService, and actively contributes to the project. RecordService is also being developed in collaboration with Cloudera partners, including Datameer and Platfora, and other Hadoop vendors.
"RecordService will enable our customers to enforce row- and column-level permissions on data in their large multi-tenant, multi-use, Hadoop clusters," said Raghu Thiagarajan, senior director, Product Management at Datameer. "This will allow data to be stored more efficiently, while ensuring each user role only gets access to what they need. We've worked closely with Cloudera to help improve Hadoop's security options for our customers, and are pleased to engage with the RecordService open source community."
"Security is at the core of enterprise Hadoop adoption. Cloudera continues to be a leader in driving Hadoop security forward, and RecordService is a key milestone for the open source community," said Denise Hemke, director of Product Management, Platfora. "We are excited to work with Cloudera on developing this new project, and bringing unified granular access controls to our mutual customers, many of whom are in compliance-sensitive markets such as Financial Services."
RecordService is a key advancement for Hadoop security, complementing existing authorization capabilities. As a new core security layer, native to the Hadoop ecosystem, RecordService enables the continued expansion of Hadoop for the enterprise, so that organizations can innovate without compromise.
Start Contributing - http://github.com/cloudera/recordservice
Download the Public Beta or Try the VM - www.cloudera.com/downloads
This information is not a commitment, promise or legal obligation to deliver any material, code or functionality. Cloudera does not guarantee that the beta software will be made generally available or that any individual feature in the beta version will be made generally available. Cloudera may make the beta software generally available, or not, in its sole discretion and without obligation to make any communication of any kind with regard to such availability.
Cloudera is revolutionizing enterprise data management by offering the first unified Platform for big data, an enterprise data hub built on Apache Hadoop. Cloudera offers enterprises one place to store, access, process, secure, and analyze all their data, empowering them to extend the value of existing investments while enabling fundamental new ways to derive value from their data. Cloudera's open source big data platform is the most widely adopted in the world, and Cloudera is the most prolific contributor to the open source Hadoop ecosystem. As the leading educator of Hadoop professionals, Cloudera has trained over 40,000 individuals worldwide. Over 1,700 partners and a seasoned professional services team help deliver greater time to value. Leading organizations in every industry plus top public sector organizations globally run Cloudera in production.
Connect With Cloudera
Follow us on Twitter: http://twitter.com/cloudera
Visit us on Facebook: http://www.facebook.com/cloudera
Join the Cloudera Community: http://cloudera.com/community
Cloudera, Cloudera's Platform for Big Data, Cloudera Enterprise Data Hub Edition, Cloudera Enterprise Flex Edition, Cloudera Enterprise Basic Editionand CDH are trademarks or registered trademarks of Cloudera Inc. in the United States, and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.