-
- Dev Rel
Why is Apache Kafka® Tiered Storage more like a dam than a fountain? Part 3: Kafka time and space
At a medieval feast where food is better! This picture illustrate the theme of this and the next blog: “more is often better”! (Source: Adobe Stock) In two recent blogs, I investigated the new Apache Kafka® Tiered Storage architecture which enables Kafka to stream more (essentially unlimited data) for less cost (by storing more older...
Learn MorePaul Brebner
January 07, 2025 -
- Dev Rel
It’s a ClickHouse® Christmas!
All I want for Christmas is…a new real-time stream processing technology: ClickHouse!
Learn MorePaul Brebner
December 18, 2024 -
- Dev Rel
- Technical
Why Is Apache Kafka® Tiered Storage More Like a Dam Than a Fountain? Part 2
In Part 1 of this blog, we introduced Apache Kafka® Tiered Storage and had an initial look at how it works in practice on a public preview version of Instaclustr’s managed Apache Kafka service. In this part, we will have a closer look at Performance, dam the river, and conclude. Kafka Tiered Storage Performance Mistaya...
Learn MorePaul Brebner
September 18, 2024 -
- Dev Rel
- Technical
Why is Apache Kafka® Tiered Storage More Like a Dam than a Fountain? Part 1
Instaclustr recently announced the availability of a public preview of Apache Kafka® tiered storage. This is a long-awaited “feature” (more a fundamental architectural shift really), proposed in KIP-405 in 2019, which finally appeared in Kafka version 3.6.0 late last year (KAFKA-7739). Tiered storage is still “early access” so is not currently recommended for use in...
Learn MorePaul Brebner
August 27, 2024 -
- Dev Rel
Get Your Apache Camel™ Kafka Connectors in a Row
A long camel train on the beach in Broome, Western Australia (Source: Adobe Stock by scottimage) Recently I wrote a blog called “Running the Apache Camel™ HTTP Kafka Source Connector on Instaclustr Managed Apache Kafka®” (check it out here). There are a lot of Apache Camel Kafka Connectors available, and I wondered how to “get your Camels...
Learn MorePaul Brebner
May 28, 2024 -
- Dev Rel
- Technical
Running the Apache Camel™ HTTP Kafka Source Connector on Instaclustr Managed Apache Kafka®
Camels drinking from a leaking pipe (Source: Wikimedia) A few years ago I built a “zero-code” data processing pipeline with Apache Kafka® and related open source technologies like OpenSearch®, PostgreSQL®, and Apache Superset™. Here’s the first blog in that series (there were 10 in all; the links to each can be found at the bottom...
Learn MorePaul Brebner
April 08, 2024 -
- Dev Rel
Tracing Apache Kafka® With OpenTelemetry – “Leave Lots of Traces”
A radio-controlled saltwater crocodile? No, a croc with telemetry – sensors + radio communication! (Source: Wikimedia) 1. Introduction It’s been 5 years since I looked at observability and distributed tracing for Apache Kafka® with open source OpenTracing and Jaeger in this blog. So, I decided it was worth taking another look at tracing Apache Kafka, particularly...
Learn MorePaul Brebner
March 15, 2024 -
- Dev Rel
- Technical
An Apache Kafka® and RisingWave Stream Processing Christmas Special
Something has gone amiss in Santa’s workshop! The “toy” Elves and the “box” Elves have stopped talking to each other and have set up 2 separate conveyor belts. The packing Elves therefore need some help to match toys and boxes. In this Christmas Special blog let's give Santa a hand and build a Kafka Elf Packing Assistance Machine. We'll use Apache Kafka combined with Apache Streams or a new open-source Streams processing technology, RisingWave.
Learn MorePaul Brebner
December 14, 2023