Get Kafka-Nated: Biweekly Espresso July 24 - August 7, 2025
Your concentrated shot of Apache Kafka® community news—fresh, essential, and designed to keep you current with what's brewing in the streaming world.
Coming Up: Apache Iceberg™ & Streaming Data
Get Kafka-nated Episode 4 drops Friday, August 15 at 16:00 BST featuring Alex Merced from Dremio, co-author of "Iceberg: The Definitive Guide." We'll explore how Apache Iceberg™ is reshaping streaming data architectures, from direct streaming into Iceberg tables to schema evolution challenges and how it all fits with emerging diskless Kafka concepts. Register for the live show
Featured: KIP-1150 Deep Dive
Speaking of diskless topics, we just released Get Kafka-nated Episode 3 featuring Greg Harris, Staff Software Engineer and the driving force behind KIP-1150. We covered how this ambitious proposal aims to eliminate Kafka's dependence on local disk storage entirely—from the technical challenges of rerouting replication pathways to the elimination of the leadership model. Listen to the full conversation
KIP Watch - The last two weeks in Kafka improvement proposals
The past two weeks have seen significant progress on several key proposals:
Recently Completed:
KIP-1160: Enable returning supported features from a specific broker - Approved August 4, this addresses a growing pain point as Kafka adds version-specific features. Without proper feature detection, applications can fail unexpectedly when connecting to brokers with different capabilities.
Currently Voting:
- KIP-1197: Improve TopicBasedRemoteLogMetadataManager initialization - Critical fix for production teams running tiered storage who've experienced startup failures that completely break the feature. This addresses a significant operational pain point.
Active Discussions:
- KIP-1134: Virtual Clusters in Kafka - Could be the biggest change to Kafka since KRaft. This proposal would allow multiple isolated "tenants" on a single physical cluster, potentially reducing infrastructure costs by 60-80% for organizations running multiple small Kafka deployments.
- KIP-1187: Support remote log size via DescribeLogDirs RPC - Solves a major blind spot for operators running tiered storage who currently can't see remote storage usage through standard Kafka tools. Essential for capacity planning and cost management.
Release Radar
Kafka 4.0.1 is brewing—Christo Lolov published the first release candidate on July 25, packing over 40 fixes since 4.0.0. The community vote is currently active. Check the release plan
Technical Deep Dives Worth Your Time
"Understanding Apache Kafka Performance: Diskless Topics Deep Dive" - Our comprehensive analysis of diskless Kafka performance characteristics, including real-world benchmarks and configuration guides. Based on testing with Diskless Kafka 4.0.0-rc15, we found 500ms P50, 2s P99 latencies but with potential for 80% cost savings. Essential reading if you're evaluating object storage-backed streaming. Read the full analysis
"Why don't Kafka and Iceberg get along?" - Filip Yonov breaks down the costly friction between Kafka and Apache Iceberg, where every message can be billed up to four times before reaching an Iceberg table. The analysis reveals how a 1 GiB/s tiered cluster can burn $3.4M annually just in transfer fees, and proposes a vision for native, zero-copy Iceberg topics that could unify the streaming log with the data lake. Read the analysis
Next shot brewing: We'll be back in two weeks with more community updates, KIP voting results, and the technical deep dives worth your time.