-
- Dev Rel
- Technical
Exploring Karapace—the Open Source Schema Registry for Apache Kafka®: Part 2—Apache Avro IDL, NOAA Tidal Example, POJOs, and Logical Types
This is the second part of our “Exploring Karapace—the Open Source Apache Kafka Schema Registry” blog series, where we continue to get up to speed with Platonic Forms (aka Schemas) in the form of Apache Avro, which is one of the Schema Types supported by Karapace. In this part we try out Avro IDL, come up with a Schema for some complex tidal data (and devise a scheme to generate a Schema from POJOs), and perfect our Schema with the addition of an Avro Logical Type for the Date field—thereby achieving close to Platonic perfection (but that’s just an idea).
Learn MorePaul Brebner
January 27, 2023 -
- Dev Rel
- Technical
Exploring Karapace—the Open Source Apache Kafka® Schema Registry: Part 1—Apache Avro Introduction with Platonic Solids
Despite Kafka being a schemaless open-source platform, Karapace offers support for schemas. The main scheme type offered by Karapace is Avro so In part 1 we are going to walk through Apache Avro to serialize and deserialize a Platonic Solids example.
Learn MorePaul Brebner
January 18, 2023 -
- Dev Rel
- Technical
Apache Kafka® KRaft Abandons the Zoo(Keeper): Part 3—Maximum Partitions and Conclusions
In this final part of the Kafka KRaft blog series we try to answer the final question (to Kafka, KRaft, Everything!) that has eluded us so far: Is there a limit to the number of partitions for a cluster? How many partitions can we create? And can we reach 1 Million or more partitions?! ...
Learn MorePaul Brebner
January 01, 2023 -
- Dev Rel
Apache Kafka® KRaft Abandons the Zoo(Keeper): Part 2—Partitions and Meta-Data Performance
In Part 1 of this blog series we explored the background to Kafka® and meta-data management, including the use of ZooKeeperTM and the recent move to KRaft to replace it. We revealed the results of our first experiment with Kafka ZooKeeper vs. KRaft, to measure the impact of increasing partitions on a data workload. Interestingly, there...
Learn MorePaul Brebner
December 08, 2022 -
- Dev Rel
- Technical
Apache Kafka® KRaft Abandons the Zoo(Keeper): Part 1— Partitions and Data Performance
In order to provide a fair comparison of ZooKeeper and KRaft, we decided to start with the managed service using ZooKeeper, and then modify it for KRaft until the general availability of KRaft is released.
Learn MorePaul Brebner
November 08, 2022 -
- Dev Rel
- Technical
Change Data Capture (CDC) With Kafka® Connect and the Debezium PostgreSQL® Source Connector
After learning how to capture change data from an Apache Cassandra® database and produce event streams in Apache Kafka®, But now we need to speed up PostgreSQL® to be as fast as a Kafka.
Learn MorePaul Brebner
August 09, 2022 -
- Dev Rel
- Technical
Spinning Your Drones With Cadence and Apache Kafka®: How Many Drones Can We Fly? Part 6
In our Cadence Drone demonstration application blog series we have come up with we have a “New and Improved” Cadence+Kafka integration pattern, and push the boundaries of Cadence scalability to discover how many drones we can fly at once.
Learn MorePaul Brebner
July 22, 2022 -
- Dev Rel
- Technical
Spinning Your Drones With Cadence and Apache Kafka – Integration Patterns and New Cadence Features – Part 5
Understand how the Drone and Order workflows interact, discover some extra Cadence+Kafka integration patterns, and explore some new Cadence features
Learn MorePaul Brebner
May 17, 2022