What's New in Cloudera Distribution of Apache Kafka?
This section lists new features in Cloudera Distribution of Apache Kafka. The following links provide detailed information for each release:
New Features in Cloudera Distribution of Apache Kafka 2.1.0
- Rebase on Kafka 0.10
Cloudera Distribution of Apache Kafka 2.1.0 is rebased on Apache Kafka 0.10. See https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.0.0/RELEASE_NOTES.html .
- Sentry Authentication
Apache Sentry includes Kafka binding you can use to enable authorization in Kafka with Sentry. See Configuring Kafka to Use Sentry Authentication.
New Features in Cloudera Distribution of Apache Kafka 2.0.0
- Rebase on Kafka 0.9
Cloudera Distribution of Apache Kafka 2.0.0 is rebased on Apache Kafka 0.9. See https://www.apache.org/dyn/closer.cgi?path=/kafka/0.9.0.0/RELEASE_NOTES.html.
Cloudera Distribution of Apache Kafka 2.0.0 supports Kerberos authentication of connections from clients and other brokers, including to ZooKeeper.
Cloudera Distribution of Apache Kafka 2.0.0 supports wire encryption of communications from clients and other brokers using SSL.
- New Consumer API
Cloudera Distribution of Apache Kafka 2.0.0 includes a new Java API for consumers.
MirrorMaker is enhanced to help prevent data loss and improve reliability of cross-data center replication.
You can use per-user quotas to throttle producer and consumer throughput in a multitenant cluster. See Quotas.
New Features in Cloudera Distribution of Apache Kafka 1.4.0
- Cloudera Distribution of Apache Kafka 1.4.0 is distributed as a package as well as a parcel. See Cloudera Distribution of Apache Kafka Version and Packaging Information.
- RHEL 7.1
Kafka 1.3.2 supports RHEL 7.1. See Supported Operating Systems
New features in Cloudera Distribution of Apache Kafka 1.3.0
- Metrics Reporter
Cloudera Manager now displays Kafka metrics. Use the values to identify current performance issues and plan enhancements to handle anticipated changes in workload. See Viewing Kafka Metrics.
- MirrorMaker configuration
Cloudera Manager allows you to configure the Kafka MirrorMaker cross-cluster replication service. You can add a MirrorMaker role and use it to replicate to a machine in another cluster. See Kafka MirrorMaker.
New Features in Cloudera Distribution of Apache Kafka 1.1.0
- New producer
The producer added in Cloudera Distribution of Apache Kafka 1.1.0 combines features of the existing synchronous and asynchronous producers. Send requests are batched, allowing the new producer to perform as well as the asynchronous producer under load. Every send request returns a response object that can be used to retrieve status and exceptions.
- Ability to delete topics
You can now delete topics using the kafka-topics --delete command.
- Offset management
In previous versions, consumers that wanted to keep track of which messages were consumed did so by updating the offset of the last consumed message in ZooKeeper. With this new feature, Kafka itself tracks the offsets. Using offset management can significantly improve consumer performance.
- Automatic leader rebalancing
Each partition starts with a randomly selected leader replica that handles requests for that partition. When a cluster first starts, the leaders are evenly balanced among hosts. When a broker restarts, leaders from that broker are distributed to other brokers, which results in an unbalanced distribution. With this feature enabled, leaders are assigned to the original replica after a restart.
- Connection quotas
Kafka administrators can limit the number of connections allowed from a single IP address. By default, this limit is 10 connections per IP address. This prevents misconfigured or malicious clients from destabilizing a Kafka broker by opening a large number of connections and using all available file handles.