-
- Technical
Building a Real-Time Tide Data Processing Pipeline: Using Apache Kafka®, Kafka Connect, Elasticsearch™, and Kibana™—Part 2
In Part 1 of this blog, we built a simple real-time data processing pipeline to take streaming tidal data from NOAA stations using Kafka connectors, and graph them in Elasticsearch and Kibana. We also tried viewing the data on a Kibana map but ran into a problem! In Part 2 we add the missing geo_points...
Learn MorePaul Brebner
November 11, 2020 -
- Technical
Building a Real-Time Tide Data Processing Pipeline: Using Apache Kafka®, Kafka Connect, Elasticsearch™, and Kibana™—Part 1
ApacheCon@Home is over for 2020 and was a resounding success, with close to 6,000 attendees from every continent. As a Platinum Sponsor, Instaclustr ran an ApacheCon Booth and this blog was originally presented on 30 September 2020 as a booth talk. I was one of the 1% attending from Australasia: This, unfortunately, meant I was...
Learn MorePaul Brebner
November 05, 2020 -
- Feature Releases
- Technical
Dedicated ZooKeeper for Apache Kafka®
Instaclustr is pleased to announce the release of dedicated Apache ZooKeeper nodes as an additional optional feature of our Managed Apache Kafka offering. Apache ZooKeeper is used for the management and coordination of nodes in Kafka.
Learn MorePaul Brebner
October 12, 2020 -
- Technical
Apache Kafka® Technology Advances: Kafka Summit 2020
The annual Kafka Summit 2020 went ahead this year (August 24-25) with a lot of topics. In the previous blog, I examined talks 1-3 from the perspective of challenging Kafka Use Cases. In this blog, I’ll focus on some of the more interesting Kafka technology advances from the remaining talks that I watched. Here is...
Learn MorePaul Brebner
October 08, 2020 -
- Technical
Building a Low-Latency Distributed Stock Broker Application: Part 4
In the fourth blog of the “Around the World ” series we built a prototype of the application, designed to run in two georegions. Recently I re-watched “Star Trek: The Motion Picture” (The original 1979 Star Trek Film). I’d forgotten how much like “2001: A Space Odyssey” the vibe was (a drawn out quest to...
Learn MorePaul Brebner
October 05, 2020 -
- Technical
Redis™ Java Clients and Client-Side Caching
Achieving sub-millisecond Redis latency with Java clients and client-side caching In my previous Redis blog, we discovered what Redis really is! It’s an open-source in-memory data structures server. And we discovered how fast it is! For a 6 node Instaclustr Managed Redis cluster latencies are under 20ms and throughput is in the millions of “gets”...
Learn MorePaul Brebner
September 24, 2020 -
- Technical
Apache Kafka® Use Cases from the 2020 Kafka Summit
The annual 2020 Kakfa summit went ahead this year (August 24-25), but this time in a virtual format. This turned out to be a good thing for me as I’m in the wrong time zone to watch the talks live, so I could take my time after the summit to catch up. Actually, this format...
Learn MorePaul Brebner
September 21, 2020 -
- Technical
It’s an In-Memory Key-Value Store! It’s a Database! It’s Redis™!
Look! Up in the sky! It’s an in-memory key-value store! It’s a database! It’s Redis! Faster than a speeding database! More powerful than an in-memory key-value store! Able to leap tall performance barriers at a single bound! “Yes, it’s Redis—strange visitor from another planet who came to Earth with powers and abilities far beyond those...
Learn MorePaul Brebner
September 09, 2020