Enable real-time insights and extreme scalability with a high-performance stream processing engine
Cloudera Stream Processing offers the latest certified, secure, and governed Apache Kafka that provides the messaging backbone for real-time streaming use cases that require low latency ingestion and durability along with decentralized management across producers and consumers.
CSP also includes support for Kafka Streams for real-time analytics, Schema Registry for centralized schema management, Mirrormaker for Kafka replication, Cloudera Manager for cluster management and monitoring, and Apache Sentry for rich access control and security.
Cloudera Streams Management is the premium management module for enterprise Kafka. It includes:
- Streams Messaging Manager (SMM)
- Streams Replication Manager (SRM)
CSP is part of Cloudera DataFlow
Cloudera Stream Processing (CSP) is a key part of the Cloudera DataFlow (CDF) platform. While CDF enables the edge-to-enterprise journey for streaming data, CSP provides the messaging backbone for real-time streaming use cases.
Key benefits
IoT-level scale of streaming architectures
- Process millions of messages per second with Apache Kafka
- Achieve high scale between diverse producers and consumers with Kafka’s decoupled architecture
- Connect data from any data producer to consumer using Kafka’s publish-subscribe model
Messaging efficiency and data governance
- Re-use schemas, define relationship between schemas, and manage schema versions with Schema Registry
- Avoid attaching a schema with every message and boost message transport efficiency
- Leverage the integration of Schema Registry across Kafka and Apache NiFi by using the same schemas from end-to-end
Real-time insights and actionable intelligence
- Adopt a lightweight streaming analytics engine that is more developer-friendly
- Build out a microservices-driven architecture that takes advantage of the Kafka Streams library
- Detect complex patterns and identify trends in real-time streaming data with Kafka Streams
Kafka Management and Monitoring
- Take advantage of Cloudera Streams Management to manage and monitor enterprise Kafka clusters
- Cure “Kafka Blindness” by getting visibility into all your Kafka clusters with Streams Messaging Manager
- Manage enterprise Kafka data effectively for active-active cluster replication and disaster recovery use cases
Cloudera Streams Management
Cloudera Streams Management (CSM) includes the key management and monitoring capabilities you will need to maintain business continuity of your Kafka implementation. CSM is made up of the following two products:
Streams Messaging Manager (SMM)
Cure Kafka blindness with a single monitoring and management dashboard that lets you:
- Troubleshoot your Kafka environment to identify bottlenecks, throughputs, consumer patterns, traffic flow, and so on
- Visualize end-to-end flows and complete data lineage of message streams from producers to topics to consumers
- Optimize your Kafka environment based on the key performance insights gathered from various brokers and topics
Streams Replication Manager (SRM)
Next-gen Kafka replication powered by Mirrormaker 2.0 allows you to:
- Take advantage of significantly improved Kafka replication capabilities with Mirrormaker 2.0
- Extend your Kafka architecture to handle active-active clusters and disaster recovery scenarios with SRM
- Install an MM2 cluster easily, provide lifecycle management actions on the cluster (start, stop), and manage/monitor the replication flows across clusters
