-
- Technical
Apache Kafka “Kongo” Part 4.2: Connecting Kafka to Cassandra with Kafka Connect
Here’s the Kongo code and sample connect property files for this blog. Kafka Connect is an API and ecosystem of 3rd party connectors that enables Kafka to be easily integrated with other heterogeneous systems without having to write any extra code. This blog focuses on a use case extending the Kongo IoT application to stream...
Learn MorePaul Brebner
May 25, 2018 -
- Technical
Apache Kafka “Kongo” Part 4.1: Connecting Kafka to Cassandra with Kafka Connect
Kafka Connect is an API and ecosystem of 3rd party connectors that enables Kafka to be easily integrated with other heterogeneous systems without having to write any extra code. This blog focuses on a use case extending the Kongo IoT application to stream events from Kafka to Apache Cassandra using a Kafka Connect Cassandra Sink....
Learn MorePaul Brebner
May 09, 2018 -
- Technical
Apache Kafka® Connect Architecture Overview
Kafka Connect is an API and ecosystem of 3rd party connectors that enables Apache Kafka to be scalable, reliable, and easily integrated with other heterogeneous systems (such as Cassandra, Spark, and Elassandra) without having to write any extra code. This blog is an overview of Kafka Connect Architecture with a focus on the main Kafka...
Learn MorePaul Brebner
May 09, 2018 -
- Technical
“Kongo” Part 3: Apache Kafka®: Kafkafying Kongo—Serialization, One or Many topics, Event Order Matters
Kafkafying: the transformation of a primitive monolithic program into a sophisticated scalable low-latency distributed streaming application (c.f. “An epidemic of a zombifying virus ravaged the country”) Steps for Kafkafying Kongo In the previous blog (“Kongo” Part 2: Exploring Apache Kafka application architecture: Event Types and Loose Coupling) we made a few changes to the original...
Learn MorePaul Brebner
April 26, 2018 -
- Technical
“Kongo” Part 2: Exploring Apache Kafka application architecture: Event Types and Loose Coupling
This is the second post in our series exploring designing and developing and example IOT application with Apache Kafka to illustrate typical design and implementation considerations and patterns. In the previous blog, we introduced our Instaclustr “Kongo” IoT Logistics Streaming Demo Application. The code for Version 1 of the Kongo application was designed as an initial...
Learn MorePaul Brebner
April 05, 2018 -
- Technical
“Kongo” Part 1: Apache Kafka®—IoT Logistics Streaming Demo Application
What’s a good name to give a demo IoT streaming application dealing with large scale logistics? How about a river… Maybe The “Amazon” application? That’s sort of taken. The Amazon is the longest river and has the most water flow, but what’s the 2nd ranking river? The Congo! The Congo is the 2nd biggest river...
Learn MorePaul Brebner
March 15, 2018 -
- Technical
Exploring the Apache Kafka® “Castle” Part B: Event Reprocessing
In this second part of the Apache Kafka Castle blog we contemplate the being or not being of Kafka Event Reprocessing, and speeding up time! Reprocessing Use Cases Reprocess: /riːˈprəʊsɛs/ verb Process (something, especially spent nuclear fuel) again or differently. Repeat event processing is called reprocessing (or sometimes replaying or rewinding), and some reprocessing use...
Learn MorePaul Brebner
January 18, 2018 -
- Technical
Exploring the Apache Kafka “Castle” Part A: Architecture and Semantics
NEWS FLASH Apache Kafka Coming Soon to Instaclustr’s Service Offering! (Source: Wikipedia) If you haven’t read Kafka’s “The Castle” (I haven’t) a few online observations are sufficient for a concise summary (and will save you the trouble of reading it): Time seems to have stopped in the village The story has no ending (Kafka died...
Learn MorePaul Brebner
January 12, 2018 -
- Technical
Apache Kafka® Christmas Tree Light Simulation – Seasons Greetings From Instaclustr
What would you do if you received a request like this? Looks like poor old Santa confused us with a company that produces Instant Forests. But maybe Santa knew I’d been trying out Decision Tree Machine Learning (including Random Forests) on our performance data (see previous blogs on Machine Learning with Spark)? But then I...
Learn MorePaul Brebner
December 19, 2017