Instaclustr is pleased to announce the release of Apache Kafka version 2.8.1 on the Instaclustr Managed Platform. The new version is available paired with version 2.8.1 of Kafka Connect. Apache Kafka is the leading distributed event streaming platform and is open source under the Apache 2.0 License. Instaclustr provides our customers managed and supported open source software allowing them to avoid the risks of both vendor lock-in and unsupported software with open source that’s really open source.

Compared to the previous release, version 2.8.1 of Apache Kafka includes some new features, under the hood improvements, and many bug fixes. Following are some you might find interesting:

Apache Kafka:

• Topic identifiers (KIP-516):

In previous versions of Apache Kafka, topics could only be identified by their name. Now, each topic is assigned a unique topic identifier which is valid for its lifetime, and even after deletion of the topic. This change is applicable for all new topics as well as existing ones. Kafka 2.8.x clients are able to retrieve topic IDs through metadata responses.

• Limit broker connection creation rate (KIP-612):

Sometimes connection storms can occur from authorized connections (new deployment/testing effort), and each new connection adds an overhead to the broker. While KIP-306 mitigated the issue of connection by introducing a way of delaying response to a failed authentication, KIP-612 goes further and provides the ability to set a limit (both overall and per IP address) on the rate at which the broker accepts new connections. This can now be addressed by setting max.connection.creation.rate in the broker config. As always we recommend that this feature be tested on lower environments before implementing it on the production Kafka cluster.

• Replica leader election is too slow in the case of too many partitions (KAFKA-10794)

An issue where replica leader election was seen to be too slow in some cases (for example, due to larger number of partitions) is now fixed.

• Better error messages when consumer group coordinator is not available (KAFKA-10839)

More informative error messages have now been introduced for situations when a consumer encounters an issue involving a Group coordinator.

• Consumer group stuck in CompletingRebalance (KAFKA-12890)

An issue on previous versions where sometimes Consumer groups could get stuck in a rebalancing state has now been resolved.

Kafka Connect:

• Expose task configurations in Connect REST API (KIP-661):

With this change, Apache Kafka Connect exposes a REST API allowing you to determine the workload of running connectors by being able to view their configurations. Historically, it has been possible to view the nominal configuration, but not the actual resolved configurations used by the running tasks.

Full details of the release can be found in the release notes for Apache Kafka 2.8.0 and 2.8.1.

If you would like to try our Kafka or Kafka Connect service, log in to our Management Console, create a test Kafka Connect cluster, and associate it with your Instaclustr managed Kafka cluster. If you haven’t created a Kafka cluster on our platform, create a test Kafka cluster and use the 30-day free trial. If you’d also like a free trial of a Kafka Connect cluster, please contact our Sales team.