-
Notifications
You must be signed in to change notification settings - Fork 309
/
Copy pathblog.html
930 lines (883 loc) · 105 KB
/
blog.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--#include virtual="includes/_header.htm" -->
<body class="page-project ">
<!--#include virtual="includes/_top.htm" -->
<div class="content">
<!--#include virtual="includes/_nav.htm" -->
<div class="right">
<h1 class="content-title">Blog</h1>
<article>
<h2 class="bullet">
<a id="apache_kafka_400_release_announcement"></a>
<a href="#apache_kafka_400_release_announcement">Apache Kafka 4.0.0 Release Announcement</a>
</h2>
18 March 2025 - David Jacot
<p>
We are proud to announce the release of Apache Kafka® 4.0. This release contains many new
features and improvements. This blog post will highlight some of the more prominent ones.
For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/4.0.0/RELEASE_NOTES.html">release notes</a>.
</p>
<p>
Apache Kafka 4.0 is a significant milestone, marking the first major release to operate entirely
without Apache ZooKeeper®. By running in KRaft mode by default, Kafka simplifies
deployment and management, eliminating the complexity of maintaining a separate ZooKeeper
ensemble. This change significantly reduces operational overhead, enhances scalability, and
streamlines administrative tasks. We want to take this as an opportunity to express our gratitude
to the ZooKeeper community and say thank you! ZooKeeper was the backbone of Kafka for
more than 10 years, and it did serve Kafka very well. Kafka would most likely not be what it is
today without it. We don’t take this for granted, and highly appreciate all of the hard work the
community invested to build ZooKeeper. Thank you!
</p>
<p>
Kafka 4.0 also brings the general availability of <a href="https://cwiki.apache.org/confluence/x/HhD1D">KIP-848</a>, introducing a powerful new consumer
group protocol designed to dramatically improve rebalance performance. This optimization
significantly reduces downtime and latency, enhancing the reliability and responsiveness of
consumer groups, especially in large-scale deployments.
</p>
<p>
Additionally, we are excited to offer <a href="https://cwiki.apache.org/confluence/x/i4ogF">early access</a> to
Queues for Kafka (<a href="https://cwiki.apache.org/confluence/x/4hA0Dw">KIP-932</a>), enabling Kafka to support traditional queue semantics directly. This
feature extends Kafka’s versatility, making it an ideal messaging platform for a wider range of
use cases, particularly those requiring point-to-point messaging patterns.
</p>
<p>
Kafka's major releases, such as version 4.0, remove APIs deprecated for at least 12 months to simplify the platform,
and encourage the adoption of newer features. Notably, in Kafka 4.0, Kafka Clients and Kafka Streams require Java 11,
while Kafka Brokers, Connect, and Tools, now require Java 17. This release also updates the minimum supported client
and broker versions (<a href="https://cwiki.apache.org/confluence/x/K5sODg">KIP-896</a>), and defines new baseline
requirements for supported upgrade paths. Further details are provided in subsequent sections of this document.
</p>
<p>
These highlights illustrate Kafka's continued evolution to meet the demands of modern,
scalable, and efficient data streaming and messaging architectures.
</p>
<p>
See the <a href="https://kafka.apache.org/documentation.html#upgrade_4_0_0">Upgrading to 4.0</a> section in the
documentation for the list of notable changes and detailed upgrade steps.
</p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li>
<a href="https://cwiki.apache.org/confluence/x/HhD1D">KIP-848: The Next Generation of the Consumer Rebalance Protocol</a>
<br>
Apache Kafka says goodbye to “stop-the-world” rebalances with the general availability of the
next generation of the consumer rebalance protocol. It increases the stability and the performance
of consumer groups while simplifying clients. The new protocol is enabled by default on the server
side. Consumers must opt-in by setting <code>group.protocol=consumer</code>.
See <a href="https://kafka.apache.org/40/documentation.html#consumer_rebalance_protocol">here</a> for details.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/B40ODg">KIP-890: Transactions Server-Side Defense</a>
<br>
The second phase of KIP-890 has been completed. It reduces the chances of “zombie transactions”
during producer failures. See <a href="https://kafka.apache.org/40/documentation.html#transaction_protocol">here</a> for details.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/4hA0Dw">KIP-932: Queues for Kafka (Early Access)</a>
<br>
This KIP introduces the concept of a share group as a way of enabling cooperative consumption
using Kafka topics. It does not add the concept of a “queue” to Kafka per se, but rather
introduces cooperative consumption to accommodate these queuing use-cases, using regular Kafka
topics. Share groups make this possible. You can think of a share group as roughly equivalent to a
“durable shared subscription” in existing systems.
Please see the <a href="https://cwiki.apache.org/confluence/x/i4ogF">Early Access Release Notes</a>
for limitations and configurations.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/mpOzDw">KIP-966: Eligible Leader Replicas (Preview)</a>
<br>
KIP-966 introduces Eligible Leader Replicas (ELR) in preview in 4.0. ELR is a subset of the ISR
replicas guaranteed to have complete data up to the high-watermark. ELRs are safe for leader
election, preventing data loss. See <a href="https://kafka.apache.org/40/documentation.html#eligible_leader_replicas">here</a> for details.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/MYtEE">KIP-996: Pre-Vote</a>
<br>
KIP-996 introduces a "Pre-Vote" mechanism to reduce unnecessary KRaft leader elections in Apache
Kafka clusters. By allowing nodes to check their eligibility for leadership before initiating an election,
this approach minimizes disruptions caused by network partitions or transient issues.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/XA-OEg">KIP-1076: Metrics for client applications KIP-714 extension</a>
<br>
With <a href="https://cwiki.apache.org/confluence/x/2xRRCg">KIP-714</a>, cluster operators can now collect
client metrics directly from brokers using a plugin. This makes it much easier to monitor client behavior.
However, KIP-714 only gathers metrics from Kafka clients (admin, consumer, and producer), and doesn't cover
application-level metrics for embedded clients, such as Kafka Streams. To fully understand performance,
it's important to collect both client metrics and application-specific metrics. This KIP introduces a way
for applications that use Kafka clients to include their own metrics alongside the existing client metrics.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/NIyMEw">KIP-1106: Add duration based offset reset option for consumer clients</a>
<br>
Kafka consumers support the <code>auto.offset.reset</code> config option, which is used when there is no initial
offset in Kafka, or if the current offset does not exist any more on the server. This config currently
supports earliest/latest/none options. Currently consumer resets might force applications to
reprocess large amounts of data from earlier offsets. With infinite storage, it's beneficial to have a
duration based offset reset strategy. This KIP allows applications to consume/initialize from a fixed
duration when there is no initial offset in Kafka.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/XoowEg">KIP-1043: Administration of groups</a>
<br>
KIP-848 and KIP-932 introduce new group types, consumer and share, respectively. The Admin Client
API has limitations when dealing with newer group types, sometimes returning errors or incorrectly
stating that groups don't exist. To address this challenge, a new command-line tool, <code>kafka-groups.sh</code>,
and updates to existing command line tools, <code>kafka-consumer-groups.sh</code> and <code>kafka-share-groups.sh</code>,
were made in Kafka 4.0 to enable users to view all groups in a cluster, along with their types and
protocols —- providing accurate information even when the Admin Client API fails.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/w4mMEw">KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group</a>
<br>
This KIP extends the <code>kafka-consumer-groups.sh</code> and the <code>kafka-share-groups.sh</code> command line tools to
provide more information related to consumer and share groups. This helps when it comes to
troubleshooting consumer and share groups, respectively introduced by KIP-848 and KIP-932.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/KYuMEw">KIP-1102: Enable clients to rebootstrap based on timeout or error code</a>
<br>
KIP-1102 enhances Kafka client resilience by proactively triggering metadata rebootstrap when no
updates occur within a timeout period, and allowing servers to explicitly signal clients to rebootstrap.
This addresses previous limitations in KIP-899, where clients could become stuck with outdated
metadata unless all brokers were unreachable.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/K5sODg">KIP-896: Remove old client protocol API versions in Kafka 4.0</a>
<br>
For the first time, old protocol API versions have been removed. Users should ensure brokers are
version 2.1 or higher before upgrading Java clients (including Connect and Streams) to 4.0. Similarly,
users should ensure their Java client version is 2.1 or higher before upgrading brokers to 4.0.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/y4kgF">KIP-1124: Providing a clear Kafka Client upgrade path for 4.x</a>
<br>
This KIP outlines the upgrade path for Kafka Clients, Streams and Connect to 4.0. It is a must read
before upgrading your clusters and clients.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/GCx4CQ">KIP-653: Upgrade log4j to log4j2</a>
<br>
The logging framework has been migrated from Log4j to Log4j2. Users can use the
<a href="https://logging.staged.apache.org/log4j/transform/cli.html">log4j-transform-cli</a> tool to
automatically convert their existing Log4j configuration files to Log4j2 format. Users can also keep
using their Log4j configurations, but with certain limitations. Check the
<a href="https://logging.apache.org/log4j/2.x/migrate-from-log4j1.html#limitations-of-the-log4j-1-x-bridge">migration guide</a> for details.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/RomNCg">KIP-724: Drop support for message formats v0 and v1</a>
<br>
The message formats v0 and v1 were deprecated in Apache Kafka 3.0. They have been removed in 4.0.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/P4vOCg">KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)</a> and
<a href="https://cwiki.apache.org/confluence/x/Bov5E">KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)</a>
<br>
Kafka Clients and Kafka Streams require Java 11, while Kafka Brokers, Connect and Tools now require Java 17.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/FAqpEQ">KIP-1030: Change constraints and default values for various configurations</a>
<br>
KIP-1030 has changed the default values of some configurations in order to provide better defaults to
users starting out using Apache Kafka. Check out the KIP for the details.
</li>
</ul>
<h3>Kafka Streams</h3>
<ul>
<li>
<a href="https://cwiki.apache.org/confluence/x/gIuMEw">KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins</a>
<br>
KIP-1104 enhances Kafka Streams by allowing foreign keys to be extracted directly from both
record keys and values, removing the need to duplicate keys into values for foreign-key joins.
This improvement simplifies joins, reduces storage overhead, and provides a more intuitive
developer experience.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/TZCMEw">KIP-1112: Allow custom processor wrapping</a>
<br>
KIP-1112 simplifies applying cross-cutting logic in Kafka Streams by introducing the
<code>ProcessorWrapper</code> interface, enabling seamless injection of custom logic around Processor API
and DSL processors. This eliminates previous redundancy and reduces the maintenance
overhead caused by manually integrating logic into each processor individually.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/LQ6TEg">KIP-1065: Add "retry" return-option to ProductionExceptionHandler</a>
<br>
KIP-1065 addresses persistent errors in Kafka Streams by allowing users to break retry loops
via a new "RETRY" option in the <code>ProductionExceptionHandler</code>. This enhancement provides
customizable error handling, enabling retries, graceful failures, or dropping problematic records
to continue processing efficiently.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/IgstEw">KIP-1091: Improved Kafka Streams operator metrics</a>
<br>
Kafka Streams exposes its metrics with KIP-1076. This KIP specifically adds a state metric for
each StreamThread and the client instance itself. It also provides detailed visibility into
application state.
</li>
</ul>
<h3>Kafka Connect</h3>
<ul>
<li>
<a href="https://cwiki.apache.org/confluence/x/rZWzDw">KIP-970: Deprecate and remove Connect's redundant task configurations endpoint</a>
<br>
The endpoint <code>GET /connectors/{connector}/tasks-config</code> added in KIP-661 was deprecated in
Apache Kafka 3.7 and is finally removed in 4.0. The endpoint
<code>GET /connectors/{connector}/tasks</code> must be used as a replacement.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/jA3OEg">KIP-1074: Allow the replication of user internal topics</a>
<br>
Previously, MirrorMaker 2 automatically excluded topics whose names ended with .internal or -internal, incorrectly classifying them
as internal topics. This behavior prevented legitimate business topics from being replicated unless users implemented a custom
replication policy. This KIP introduces a configurable option that allows users to replicate such topics without requiring custom code.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/UwotEw">KIP-1089: Allow disabling heartbeats replication in MirrorSourceConnector</a>
<br>
Previously, MirrorSourceConnector always replicated heartbeat topics, which could cause issues
when multiple connectors with different configurations replicated topics between the same clusters,
resulting in duplicate heartbeat replication. This KIP introduces a configurable option to disable
heartbeat topic replication, providing users greater flexibility, particularly when using multiple
connectors with distinct configurations.
</li>
<li>
<a href="https://cwiki.apache.org/confluence/x/uwrPEQ">KIP-1032: Upgrade to Jakarta and JavaEE 10 in Kafka 4.0</a>
<br>
Kafka Connect previously utilized outdated JavaEE APIs, restricting its compatibility with contemporary Jakarta EE
frameworks and applications. KIP-1032 addresses this by upgrading Kafka 4.0 to Jakarta EE and JavaEE 10 APIs,
introducing Java 17 as a required minimum version, and aligning Kafka with current standards for improved
maintainability.
</li>
</ul>
<h3>Summary</h3>
<p>Ready to get started with Apache Kafka 4.0.0? Check out all the details in the <a href="https://kafka.apache.org/documentation.html#upgrade_4_0_0">upgrade notes</a> and
the <a href="https://downloads.apache.org/kafka/4.0.0/RELEASE_NOTES.html">release notes</a>,
and <a href="https://kafka.apache.org/downloads">download</a> Apache Kafka 4.0.0.</p>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 175 contributors:<br>
A. Sophie Blee-Goldman, abhi-ksolves, Abhijeet Kumar, Abhinav Dixit, Abhishek Giri, Alieh Saeedi, Almog Gavra, Alyssa Huang,
Andrew Schofield, Anshul Goyal, Ao Li, Apoorv Mittal, Arnav Dadarya, Arpit Goyal, Artem Livshits, Ayoub Omari, bachmanity1,
bboyleonp666, Bill Bejeck, brenden20, Bruno Cadonna, Caio Guedes, Calvin Liu, Chengyan, Cheryl Simmons, Chia-Chuan Yu,
Chia-Ping Tsai, Chirag Wadhwa, Chris Egerton, Christo Lolov, Christopher L. Shannon, ClarkChen, Clay Johnson, Colin P. McCabe,
Colt McNealy, Danica Fine, Dániel Urbán, David Arthur, David Jacot, David Mao, David Schlosnagle, Dejan Stojadinović,
devanshikhatsuriya, Dimitar Dimitrov, Divij Vaidya, DL1231, Dmitry Werner, donaldzhu-cc, Dongnuo Lyu, dujian0068, Edoardo Comar,
Eric Chang, Federico Valeri, Frederik Rouleau, GangHuo, Gantigmaa Selenge, Gaurav Narula, Greg Harris, Guozhang Wang,
Hailey Ni, Hongten, Hyunsang Han, Ian McDonald, Igor Soarez, Ismael Juma, Ivan Yurchenko, Jakub Scholz, Jason Gustafson,
Jason Taylor, Jeff Kim, Jhen-Yung Hsu, Jim Galasyn, João Pedro Fonseca Dantas, John Huang, JohnHuang, Jonah Hooper,
José Armando García Sancio, Josep Prat, Jun Rao, Justin Lee, Justine Olshan, Kamal Chandraprakash, KApolinario1120, kartik-3513,
Kaushik Raina, Ken Huang, kevin-wu24, Kirk True, Kondrat Bertalan, Krishna Agarwal, Kuan-Po Tseng, kwonyonghyun, Laxman Ch,
Liam Miller-Cushon, Lianet Magrans, Linsiyuan9, Linu Shibu, Liu Zeyu, Logan Zhu, Loïc Greffier, Lucas Brutschy, Luke Chen,
lushilin, Mahsa Seifikar, Manikumar Reddy, mannoopj, Martin Sillence, Mason Chen, Matthias J. Sax, Mehari Beyene, Mickael Maison,
Ming-Yen Chung, mingdaoy, msureshpro, Murali Basani, Nancy, Nick Guo, Nick Telford, Oleg Bonar, Oleksandr K., Omnia Ibrahim,
Parker Chang, Patrik Marton, Paul R. Mellor, Peter Lee, Philip Nee, PoAn Yang, Rajini Sivaram, Ramin Gharib, Ritika Reddy,
Robert Young, Rohan, S.Y. Wang, Said Boudjelda, Sanskar Jhajharia, santhoshct, Sasaki Toru, Satish Duggana, Saxon Chen,
Scott Hendricks, Sean Quah, Sebastien Viale, Shivsundar R, snehashisp, Stanislav Knot, Steven Xu, stevenbooke, Stig Døssing,
Sushant Mahajan, Swikar Patel, TaiJuWu, tall15421542-lab, TapDang, Ted Yan, TengYao Chi, Thomas Thornton, Tim Fox,
tkuramoto33, Tom Duckering, Vedarth Sharma, Vikas Singh, Viktor Somogyi-Vass, Vincent Jiang, wperlichek, xijiu, Xuan-Zhang Gong,
yangjf2019, Yaroslav Kutsela, Yash Mayya, Yung, yungh, yx9o, Zhengke Zhou, Ziming Deng, 陳昱霖 (Yu-Lin Chen)
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_390_release_announcement"></a>
<a href="#apache_kafka_390_release_announcement">Apache Kafka 3.9.0 Release Announcement</a>
</h2>
6 November 2024 - Colin P. McCabe
<p>We are proud to announce the release of Apache Kafka 3.9.0. This is a major release, the final one in the 3.x line. This will also be the final major release to feature the deprecated Apache ZooKeeper mode. Starting in 4.0 and later, Kafka will always run without ZooKeeper.</p>
<h3>Dynamic KRaft Quorums</h3>
<p>When in KRaft mode, Kafka relies on a Raft quorum of controller processes to store its metadata log.
Previously, controller quorums were specified by a static configuration that needed to be set on all
brokers and controllers. This made changing the hostnames of controllers, or adding or removing them,
very difficult to do without downtime.</p>
<p><a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-853%3A+KRaft+Controller+Membership+Changes">KIP-853:
KRaft Controller Membership Changes</a> makes quorum membership dynamic. Administrators can now add and
remove controller nodes by running the <b>kafka-metadata-quorum.sh</b> tool or using the associated
AdminClient API.
<p>This feature has been eagerly awaited by the community, and we are excited to deliver it in 3.9. The
main limitation in 3.9 is that we do not support converting over static metadata quorums to dynamic
ones. However, this ability is coming soon.</p>
<p>For more information about using the new dynamic quorums feature, see <a
href="https://kafka.apache.org/39/documentation.html#kraft_reconfig">the KIP-853 documentation.</a></p>
<h3>Improved ZooKeeper Migration</h3>
<p>Users that are using ZooKeeper mode today need to go through a process called ZooKeeper migration
before they will be able to use KRaft mode. The migration process involves running a series of commands
to copy the metadata stored in ZooKeeper into a KRaft quorum.</p>
<p>ZK migration is not a new feature; indeed, it has been under development since Kafka 3.4. However, I
wanted to highlight that Kafka 3.9 is the final and best iteration of our ZK migration feature. As we
migrated thousands of clusters (big and small), we found and fixed many bugs. We also closed all of the
remaining feature gaps that kept some users tied to ZooKeeper.</p>
<p>Kafka 4.0 will be fully saying goodbye to ZooKeeper. There will be no support for running in ZK mode,
or migrating from ZK mode. This means that administrators that are still using the deprecated ZK mode
who need to upgrade to 4.0 and beyond will need to make a stop at a "bridge release."
For example, if you wanted to upgrade from Kafka 3.0 to Kafka 4.0, you might do the following:</p>
<ul>
<li>Upgrade to Kafka 3.9.</li>
<li>Perform ZK migration.</li>
<li>Upgrade to Kafka 4.0.</li>
</ul>
<p>In this example, Kafka 3.9 serves as the "bridge" to 4.0</p>
<p>As you migrate your older Kafka clusters, also keep in mind that Kafka 3.5 and later use a version of
ZooKeeper that is not wire-compatible with Kafka versions older than 2.4. Therefore, if you want to migrate
a cluster older than that, you will have to make an additional stop at a release with a ZK version
supported by both Kafkas. See <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-902%3A+Upgrade+Zookeeper+to+3.8.2">KIP-902</a>
for details.</p>
<p>As an example, if you wanted to migrate from Kafka 1.0 to Kafka 4.0, you might do the following:</p>
<ul>
<li>Upgrade to Kafka 3.4 (a newer version will not work due to ZK incompatibilites)</li>
<li>Upgrade to ZooKeeper 3.8</li>
<li>Upgrade to Kafka 3.9.</li>
<li>Perform ZK migration.</li>
<li>Upgrade to Kafka 4.0.</li>
</ul>
<p>These multi-step migrations should be quite rare. Running such an old Kafka version raises security concerns,
after all. However, I wanted to mention it for completeness.</p>
<h3>Tiered Storage</h3>
<p>Tiered storage is a feature that has been under development since Kafka 3.6. It allows Kafka to seamlessly offload data to pluggable external storage systems, such as cloud object stores. (See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage">KIP-405</a> for more details.)</p>
<p>Tiered storage is now production-ready in Kafka 3.9. In addition, we added the following
improvements:</p>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-950%3A++Tiered+Storage+Disablement">KIP-950: Tiered Storage Disablement</a>:
</b><br>There is now a mechanism for dynamically disabling tiered storage on a per-topic basis.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-956+Tiered+Storage+Quotas">KIP-956 Tiered Storage Quotas</a>:
</b><br>Administrators can now put upper bounds on tiered storage upload and download rates.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1005%3A+Expose+EarliestLocalOffset+and+TieredOffset">KIP-1005: Expose EarliestLocalOffset and TieredOffset</a>:
</b><br>Kafka now exposes the highest offset at which partition data is stored in remote storage.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1057%3A+Add+remote+log+metadata+flag+to+the+dump+log+tool">KIP-1057: Add remote log metadata flag to the dump log tool</a>:
</b><br>The kafka-dump-log.sh tool gained the ability to examine tiered storage records.
</li>
</ul>
<h3>Kafka Streams</h3>
<p>There are several Kafka Streams improvements in Apache Kafka 3.9.</p>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1049%3A+Add+config+log.summary.interval.ms+to+Kafka+Streams">KIP-1049: Add config log.summary.interval.ms to Kafka Streams</a>:
</b><br>Introduce the log.summary.interval.ms to control the frequency of summary logs.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+Kafka+Streams+exception+handler+for+exceptions+occurring+during+processing">KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing</a>:
</b><br>Improve exception handling in Kafka streams.
</li>
</ul>
<h3>Kafka Connect</h3>
<p>Kafka Connect received several improvements in 3.9 as well.</p>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1040%3A+Improve+handling+of+nullable+values+in+InsertField%2C+ExtractField%2C+and+other+transformations">KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations</a>:
</b><br>Add more configuration knobs for handling nulls.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1031%3A+Control+offset+translation+in+MirrorSourceConnector">KIP-1031: Control offset translation in MirrorSourceConnector</a>:
</b><br>Add the emit.offset-syncs.enabled configuration, which can be used to disable configuration synchronization.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect">KIP-1017: Health check endpoint for Kafka Connect</a>:
</b><br>Add a REST endpoint that can be used to determine if Kafka Connect workers are healthy.
</li>
</ul>
<h3>Conclusion</h3>
<p>I hope that this post has given you a sense of all the exciting things that are going on in Kafka 3.9. For more details, take a look at the <a href="https://archive.apache.org/dist/kafka/3.9.0/RELEASE_NOTES.html">release notes</a>, or simply <a href="https://kafka.apache.org/downloads">download</a> the release for yourself.
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
A. Sophie Blee-Goldman, abhi-ksolves, Abhijeet Kumar, Abhinav Dixit, Adrian Preston, Alieh Saeedi, Alyssa Huang, Anatoly Popov, Andras Katona, Andrew Schofield, Andy Wilkinson, Anna Sophie Blee-Goldman, Antoine Pourchet, Apoorv Mittal, Arnav Dadarya, Arnout Engelen, Arpit Goyal, Arun Mathew, Ayoub Omari, bachmanity1, Bill Bejeck, brenden20, Bruno Cadonna, Chia Chuan Yu, Chia-Ping Tsai, ChickenchickenLove, Chirag Wadhwa, Chris Egerton, Christo Lolov, Ming-Yen Chung, Colin P. McCabe, Cy, David Arthur, David Jacot, Demonic, dengziming, Dimitar Dimitrov, Dmitry Werner, Dongnuo Lyu, dujian0068, Edoardo Comar, Farbod Ahmadian, Federico Valeri, Fiore Mario Vitale, Florin Akermann, Francois Visconte, Ganesh Sadanala, Gantigmaa Selenge, Gaurav Narula, gongxuanzhang, Greg Harris, Gyeongwon Do, Harry Fallows, Hongten, Ian McDonald, Igor Soarez, Ismael Juma, Ivan Yurchenko, Jakub Scholz, Jason Gustafson, Jeff Kim, Jim Galasyn, Jinyong Choi, Johnny Hsu, José Armando García Sancio, Josep Prat, Jun Rao, Justine Olshan, Kamal Chandraprakash, Ken Huang, Kevin Wu, Kirk True, Kondrat Bertalan, Krishna Agarwal, KrishVora01, Kuan-Po (Cooper) Tseng, Lee Dongjin, Lianet Magrans, Logan Zhu, Loïc GREFFIER, Lucas Brutschy, Luke Chen, Maciej Moscicki, Manikumar Reddy, Mason Chen, Matthias J. Sax, Max Riedel, Mickael Maison, Murali Basani, Nancy, Nicolas Guyomar, Nikolay, Okada Haruki, Omnia Ibrahim, PaulRMellor, Pavel Pozdeev, Phuc-Hong-Tran, Piotr Rzysko, PoAn Yang, Ritika Reddy, Rohan, Ron Dagostino, Sanskar Jhajharia, Satish Duggana, Sean Quah, Sebastien Viale, Shawn Hsu, ShivsundarR, Sushant Mahajan, TaiJuWu, TengYao Chi, TingIāu "Ting" Ki, vamossagar12, Vedarth Sharma, Vikas Balani, Vikas Singh, Viktor Somogyi-Vass, Vinay Agarwal, Vincent Rose, Volk, Wang Xiaoqing, Xiduo You, xijiu, Xuan-Zhang Gong, Yash Mayya, Zhengke Zhou.
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_381_release_announcement"></a>
<a href="#apache_kafka_381_release_announcement">Apache Kafka 3.8.1 Release Announcement</a>
</h2>
29 October 2024 - Josep Prat (<a href="https://twitter.com/jlprat">@jlprat</a>)
<p>We are proud to announce the release of Apache Kafka 3.8.1. This is a bugfix release. For a full list of changes, be sure to check the <a href="https://downloads.apache.org/kafka/3.8.1/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/38/documentation.html#upgrade_3_8_1">Upgrading to 3.8.1 from any version 0.8.x through 3.7.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
Andrew Schofield, Apoorv Mittal, Bill Bejeck, Bruno Cadonna, Chia-Ping Tsai, Chris Egerton, Colin P. McCabe, David Arthur, Guillaume Mallet, Igor Soarez, Josep Prat, Justine Olshan, Ken Huang, Kondrat Bertalan, Kuan-Po Tseng, Luke Chen, Manikumar Reddy, Matthias J. Sax, Mickael Maison, PoAn Yang, Rohan, TengYao Chi, Vikas Singh
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_380_release_announcement"></a>
<a href="#apache_kafka_380_release_announcement">Apache Kafka 3.8.0 Release Announcement</a>
</h2>
29 July 2024 - Josep Prat (<a href="https://twitter.com/jlprat">@jlprat</a>)
<p>We are proud to announce the release of Apache Kafka 3.8.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://downloads.apache.org/kafka/3.8.0/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/documentation.html#upgrade_3_8_0">Upgrading to 3.8.0 from any version 0.8.x through 3.7.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<p>
In a previous release, 3.6,
<a href="https://kafka.apache.org/38/documentation.html#tiered_storage">tiered storage</a> was released as early access feature.
In this release, Tiered Storage now supports clusters configured with multiple log directories (i.e. JBOD feature). This feature still remains as early access.
</p>
<p>
In the last release, 3.7, <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft">KIP-858</a>
was released in early access. Since this version, JBOD in KRaft is no longer considered an early access feature.
</p>
<p>
Up until now, only the default compression level was used by Apache Kafka. From this version on, a configuration mechanism to specify compression level is included. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level">KIP-390</a> for more details.
</p>
<p>
<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol">KIP-848 The Next Generation of the Consumer Rebalance Protocol</a> is available as preview in 3.8. This version includes numerous bug
fixes and the community is encouraged to test and provide feedback. <a href="https://cwiki.apache.org/confluence/display/KAFKA/The+Next+Generation+of+the+Consumer+Rebalance+Protocol+%28KIP-848%29+-+Preview+Release+Notes">See the preview release notes for more information.</a>
</p>
<p>
The configuration value <code>offsets.commit.required.acks</code> is deprecated in this version and it will be removed in Kafka 4.0. See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794933">KIP-1041</a> for more details.
</p>
<p>
<i>Note: ZooKeeper is marked as deprecated since the 3.5.0 release. ZooKeeper is planned to be removed in Apache Kafka 4.0. For more information, please see the documentation for <a href="https://kafka.apache.org/documentation/#zk_depr">ZooKeeper Deprecation</a></i>.
</p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level">KIP-390: Support Compression Level</a>:
</b><br>Adds a mechanism to specify compression level instead of relying on the default one.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Deprecate+Log4J+Appender">KIP-719: Deprecate Log4J Appender</a>:
</b><br>Log4J Appender is now deprecated and it will be removed, most probably, in Kafka 4.0.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol">(Preview) KIP-848 The Next Generation of the Consumer Rebalance Protocol</a>:
</b><br>The new simplified Consumer Rebalance Protocol moves complexity away from the consumer and into the Group Coordinator within the broker and completely revamps the protocol to be incremental in nature. It provides the same guarantee as the current protocol––but better and more efficient, including no longer relying on a global synchronization barrier. <a href="https://cwiki.apache.org/confluence/display/KAFKA/The+Next+Generation+of+the+Consumer+Rebalance+Protocol+%28KIP-848%29+-+Preview+Release+Notes">See the preview release notes for more information.</a>
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-899%3A+Allow+producer+and+consumer+clients+to+rebootstrap">KIP-899: Allow producer and consumer clients to rebootstrap</a>:
</b><br>This KIP allows Kafka clients to repeat the bootstrap process when updating metadata if none of the known brokers are available.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-993%3A+Allow+restricting+files+accessed+by+File+and+Directory+ConfigProviders">KIP-993: Allow restricting files accessed by File and Directory ConfigProviders</a>:
</b><br>This KIP adds the ability to limit the files accessible to the file and directory configuration providers so the caller doesn't have unrestricted access.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker">KIP-974: Docker Image for GraalVM based Native Kafka Broker</a>:
</b><br>This update introduces the process to create Docker native image for Apache Kafka based onf GraalVM. The advantage of this Apache Kafka docker image that can launch brokers with sub-second startup time and minimal memory footprint by leveraging a GraalVM based native Kafka binary and runs in the Kraft mode.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1018%3A+Introduce+max+remote+fetch+timeout+config+for+DelayedRemoteFetch+requests">KIP-1018: Introduce max remote fetch timeout config for DelayedRemoteFetch requests</a>:
</b><br>Introduces a new timeout parameter, <code>remote.fetch.max.wait.ms</code>, to offer users the option to configure the timeout based on their workload.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1019%3A+Expose+method+to+determine+Metric+Measurability">KIP-1019: Expose method to determine Metric Measurability</a>:
</b><br>This KIP introduces a new method, <code>isMeasurable()</code>, to the <code>KafkaMetric</code> class, eliminating the need for accessing private fields or handling exceptions in case of non-measurable metrics.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Image+for+Apache+Kafka">KIP-1028: Docker Official Image for Apache Kafka</a>:
</b><br>This update introduces the official Docker image for Apache Kafka.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1036%3A+Extend+RecordDeserializationException+exception">KIP-1036: Extend RecordDeserializationException exception</a>:
</b><br><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87297793">KIP-334</a> introduced into the Consumer the RecordDeserializationException with offsets information. That is useful to skip a poison pill but as you do not have access to the Record, it still prevents easy implementation of dead letter queue or simply logging the faulty data.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1037%3A+Allow+WriteTxnMarkers+API+with+Alter+Cluster+Permission">KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission</a>:
</b><br>This KIP builds on <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging#KIP98ExactlyOnceDeliveryandTransactionalMessaging-5.2WriteTxnMarkerRequest">KIP-98</a> and <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-664%3A+Provide+tooling+to+detect+and+abort+hanging+transactions">KIP-664</a> allowing the <code>WriteTxnMarkers</code> API to be invoked with the Alter permission on the cluster to reflect that this API is an admin operation that can be called from the Kafka Admin Client.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794933">KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)</a>:
</b><br>The configuration <code>offsets.commit.required.acks</code> is deprecated and it will be removed in Kafka 4.0.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1047+Introduce+new+org.apache.kafka.tools.api.Decoder+to+replace+kafka.serializer.Decoder">KIP-1047 Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder</a>:
</b><br>This KIP deprecates <code>kafka.serializer.Decoder</code> and introduces <code>org.apache.kafka.tools.api.Decoder</code> in oder to allow the migration of tool related code from core to the tools module.
</li>
</ul>
<h3>Kafka Streams</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-989%3A+Improved+StateStore+Iterator+metrics+for+detecting+leaks">KIP-989: Improved StateStore Iterator metrics for detecting leaks</a>:
</b><br>This KIP introduces several new metrics to aid users in finding leaked Iterators, as well as identifying the cause of a high number of pinned blocks, or other kinds of memory leaks and performance problems.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-924%3A+customizable+task+assignment+for+Streams">KIP-924: customizable task assignment for Streams</a>:
</b><br>This KIP adds a new group of configurable interfaces for plugging custom behaviour into the Streams Partition Assignor. This configuration supplants the existing internal task assignor config. Additionally, it limits the scope of these configs to supplying a custom task assignor. This opens the door for future KIPs to add further configs in which a user can set to plug in custom behavior.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-813%3A+Shareable+State+Stores">KIP-813: Shareable State Stores</a>:
</b><br>This KIP adds Shareable State Stores, introducing the ability to use data within a state store across multiple applications without duplicating it on topic level.
</li>
</ul>
<h3>Kafka Connect</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-477%3A+Add+PATCH+method+for+connector+config+in+Connect+REST+API">KIP-477: Add PATCH method for connector config in Connect REST API</a>:
</b><br>This KIP adds the ability in Kafka Connect to understand <code>PATCH</code> methods in the Connect REST API, allowing partial configuration updates.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect">KIP-1004: Enforce tasks.max property in Kafka Connect</a>:
</b><br>This KIP changes Kafka Connect so it respects the value for the <code>tasks.max</code> property. Check the KIP description page for compatibility and migration plan.
</li>
</ul>
<h3>Summary</h3>
<p>Ready to get started with Apache Kafka 3.8.0? Check out all the details in the <a href="https://downloads.apache.org/kafka/3.8.0/RELEASE_NOTES.html">release notes</a> and <a href="https://kafka.apache.org/downloads">download</a> Apache Kafka 3.8.0.</p>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 196 contributors:<br>
Aadithya Chandra, Abhijeet Kumar, Abhinav Dixit, Adrian Preston, Afshin Moazami, Ahmed Najiub, Ahmed Sobeh, Akhilesh Chaganti, Almog Gavra, Alok Thatikunta, Alyssa Huang, Anatoly Popov, Andras Katona, Andrew Schofield, Anna Sophie Blee-Goldman, Antoine Pourchet, Anton Agestam, Anton Liauchuk, Anuj Sharma, Apoorv Mittal, Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj, Ayoub Omari, Bruno Cadonna, Calvin Liu, Cameron Redpath, charliecheng630, Cheng-Kai, Zhang, Cheryl Simmons, Chia Chuan Yu, Chia-Ping Tsai, ChickenchickenLove, Chris Egerton, Chris Holland, Christo Lolov, Christopher Webb, Colin P. McCabe, Colt McNealy, [email protected], Vedarth Sharma, Crispin Bernier, Daan Gerits, David Arthur, David Jacot, David Mao, dengziming, Divij Vaidya, DL1231, Dmitry Werner, Dongnuo Lyu, Drawxy, Dung Ha, Edoardo Comar, Eduwer Camacaro, Emanuele Sabellico, Erik van Oosten, Eugene Mitskevich, Fan Yang, Federico Valeri, Fiore Mario Vitale, flashmouse, Florin Akermann, Frederik Rouleau, Gantigmaa Selenge, Gaurav Narula, ghostspiders, gongxuanzhang, Greg Harris, Gyeongwon Do, Hailey Ni, Hao Li, Hector Geraldino, highluck, hudeqi, Hy (하이), IBeyondy, Iblis Lin, Igor Soarez, ilyazr, Ismael Juma, Ivan Vaskevych, Ivan Yurchenko, James Faulkner, Jamie Holmes, Jason Gustafson, Jeff Kim, jiangyuan, Jim Galasyn, Jinyong Choi, Joel Hamill, John Doe [email protected], John Roesler, John Yu, Johnny Hsu, Jorge Esteban Quilcate Otoya, Josep Prat, José Armando García Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Ken Huang, Kirk True, Kohei Nozaki, Krishna Agarwal, KrishVora01, Kuan-Po (Cooper) Tseng, Kvicii, Lee Dongjin, Leonardo Silva, Lianet Magrans, LiangliangSui, Linu Shibu, lixinyang, Lokesh Kumar, Loïc GREFFIER, Lucas Brutschy, Lucia Cerchie, Luke Chen, Manikumar Reddy, mannoopj, Manyanda Chitimbo, Mario Pareja, Matthew de Detrich, Matthias Berndt, Matthias J. Sax, Matthias Sax, Max Riedel, Mayank Shekhar Narula, Michael Edgar, Michael Westerby, Mickael Maison, Mike Lloyd, Minha, Jeong, Murali Basani, n.izhikov, Nick Telford, Nikhil Ramakrishnan, Nikolay, Octavian Ciubotaru, Okada Haruki, Omnia G.H Ibrahim, Ori Hoch, Owen Leung, Paolo Patierno, Philip Nee, Phuc-Hong-Tran, PoAn Yang, Proven Provenzano, Qichao Chu, Ramin Gharib, Ritika Reddy, Rittika Adhikari, Rohan, Ron Dagostino, runom, rykovsi, Sagar Rao, Said Boudjelda, sanepal, Sanskar Jhajharia, Satish Duggana, Sean Quah, Sebastian Marsching, Sebastien Viale, Sergio Troiano, Sid Yagnik, Stanislav Kozlovski, Stig Døssing, Sudesh Wasnik, TaiJuWu, TapDang, testn, TingIāu "Ting" Kì, vamossagar12, Vedarth Sharma, Victor van den Hoven, Vikas Balani, Viktor Somogyi-Vass, Vincent Rose, Walker Carlson, wernerdv, Yang Yu, Yash Mayya, yicheny, Yu-Chen Lai, yuz10, Zhifeng Chen, Zihao Lin, Ziming Deng, 谭九鼎
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_371_release_announcement"></a>
<a href="#apache_kafka_371_release_announcement">Apache Kafka 3.7.1 Release Announcement</a>
</h2>
28 June 2024 - Igor Soarez (<a href="https://www.linkedin.com/in/soarez/">@soarez</a>)
<p>We are proud to announce the release of Apache Kafka 3.7.1. This is a bugfix release. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.7.1/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/37/documentation.html#upgrade_3_7_1">Upgrading to 3.7.1 from any version 0.8.x through 3.6.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
Adrian Preston, Anatoly Popov, Andras Katona, Andrew Schofield, Anna Sophie Blee-Goldman, Anton Liauchuk, Apoorv Mittal, Ayoub Omari, Bruno Cadonna, Calvin Liu, Cameron Redpath, Cheng-Kai Zhang, Chia-Ping Tsai, Chris Egerton, Colin Patrick McCabe, David Arthur, David Jacot, Divij Vaidya, Dmitry Werner, Edoardo Comar, flashmouse, Florin Akermann, Gantigmaa Selenge, Gaurav Narula, Greg Harris, Igor Soarez, ilyazr, Ismael Juma, Jason Gustafson, Jeff Kim, jiangyuan, Joel Hamill, John Yu, Johnny Hsu, José Armando García Sancio, Josep Prat, Jun Rao, Justine Olshan, Kamal Chandraprakash, Ken Huang, Kuan-Po (Cooper) Tseng, Lokesh Kumar, Luke Chen, Manikumar Reddy, Mario Pareja, Matthias J. Sax, Mayank Shekhar Narula, Mickael Maison, Murali Basani, Omnia Ibrahim, Paolo Patierno, PoAn Yang, Sagar Rao, sanepal, Sean Quah, Sebastian Marsching, Stanislav Kozlovski, Vedarth Sharma, Walker Carlson, Yash Mayya
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_362_release_announcement"></a>
<a href="#apache_kafka_362_release_announcement">Apache Kafka 3.6.2 Release Announcement</a>
</h2>
4 April 2024 - Manikumar Reddy O. (<a href="https://www.linkedin.com/in/15mani/">@Manikumar</a>)
<p>We are proud to announce the release of Apache Kafka 3.6.2. This is a bugfix release. It contains 28 fixes. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.6.2/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/36/documentation.html#upgrade_3_6_2">Upgrading to 3.6.2 from any version 0.8.x through 3.5.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
Akhilesh Chaganti, Andrew Schofield, Anton Liauchuk, Bob Barrett, Bruno Cadonna, Cheng-Kai, Zhang, Chia-Ping Tsai, Chris Egerton, Colin P. McCabe, Colin Patrick McCabe, David Arthur, David Jacot, David Mao, Divij Vaidya, Edoardo Comar, Emma Humber, Gaurav Narula, Greg Harris, hudeqi, Ismael Juma, Jason Gustafson, Jim Galasyn, Joel Hamill, Johnny Hsu, José Armando García Sancio, Justine Olshan, Luke Chen, Manikumar Reddy, Matthias J. Sax, Mayank Shekhar Narula, Mickael Maison, Mike Lloyd, Paolo Patierno, PoAn Yang, Ron Dagostino, Sagar Rao, Stanislav Kozlovski, Walker Carlson, Yash Mayya
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_370_release_announcement"></a>
<a href="#apache_kafka_370_release_announcement">Apache Kafka 3.7.0 Release Announcement</a>
</h2>
27 February 2024 - Stanislav Kozlovski (<a href="https://twitter.com/BdKozlovski">@BdKozlovski</a>)
<p>We are proud to announce the release of Apache Kafka 3.7.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.7.0/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/documentation.html#upgrade_3_7_0">Upgrading to 3.7.0 from any version 0.8.x through 3.6.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<p>
In the last release, 3.6,
<a href="https://kafka.apache.org/documentation/#kraft_zk_migration">the ability to migrate Kafka clusters from a ZooKeeper metadata system</a>
to a KRaft metadata system was ready for usage in production environments with one caveat -- JBOD was not yet available for KRaft clusters.
In this release, we are shipping an early access release of JBOD in KRaft. (See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft">KIP-858</a> and the <a href="https://cwiki.apache.org/confluence/display/KAFKA/Kafka+JBOD+in+KRaft+Early+Access+Release+Notes">release notes</a> for details).
</p>
<p>
Additionally, client APIs released prior to Apache Kafka 2.1 are now marked deprecated in 3.7 and will be removed in Apache Kafka 4.0. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0">KIP-896</a> for details and RPC versions that are now deprecated.
</p>
<p>
Java 11 support for the Kafka broker is also marked deprecated in 3.7, and is planned to be removed in Kafka 4.0. See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=284789510">KIP-1013</a> for more details. Note that clients can continue to use JDK >= 11 to connect to Kafka brokers.
</p>
<p>
<i>Note: ZooKeeper is marked as deprecated since the 3.5.0 release. ZooKeeper is planned to be removed in Apache Kafka 4.0. For more information, please see the documentation for <a href="https://kafka.apache.org/documentation/#zk_depr">ZooKeeper Deprecation</a></i>.
</p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft">(Early Access) KIP-858 Handle JBOD broker disk failure in KRaft</a>:
</b><br>This update closes the gap on one of the last major missing features in KRaft by adding JBOD support in KRaft-based clusters. Note that it is not yet recommended for use in production environments. Please refer to the <a href="https://cwiki.apache.org/confluence/display/KAFKA/Kafka+JBOD+in+KRaft+Early+Access+Release+Notes">release notes</a> to help us test it and provide feedback at <a href="https://issues.apache.org/jira/browse/KAFKA-16061">KAFKA-16061</a>.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability">KIP-714 Client metrics and observability</a>:
</b><br>With KIP-714, operators get better visibility into the clients connecting to their cluster with broker-side support of client-level metrics via a standardized telemetry interface.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources">KIP-1000 List Client Metrics Configuration Resources</a>:
</b><br>KIP-1000 supports KIP-714 by introducing a way to create, read, update, and delete the client metrics configuration resources using the existing RPCs and the kafka-configs.sh tool.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol">(Early Access) KIP-848 The Next Generation of the Consumer Rebalance Protocol</a>:
</b><br>The new simplified Consumer Rebalance Protocol moves complexity away from the consumer and into the Group Coordinator within the broker and completely revamps the protocol to be incremental in nature. It provides the same guarantee as the current protocol––but better and more efficient, including no longer relying on a global synchronization barrier. <a href="https://cwiki.apache.org/confluence/display/KAFKA/The+Next+Generation+of+the+Consumer+Rebalance+Protocol+%28KIP-848%29+-+Early+Access+Release+Notes">See the early access release notes for more information.</a>
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-951%3A+Leader+discovery+optimisations+for+the+client">KIP-951 Leader discovery optimisations for the client</a>:
</b><br>KIP-951 optimizes the time it takes for a client to discover the new leader of a partition, leading to reduced end-to-end latency of produce/fetch requests in the presence of leadership changes (broker restarts, partition reassignments, etc.).
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka">KIP-975 Docker Image for Apache Kafka</a>:
</b><br>Introduces an official Apache Kafka Docker image, enabling quicker testing and deployment, as well as onboarding of developers.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-580%3A+Exponential+Backoff+for+Kafka+Clients">KIP-580 Exponential Backoff for Kafka Clients</a>:
</b><br>Changes the client’s retry backoff time used for retrying failed requests from a static one to an exponentially-increasing one. This should help reduce slow metadata convergence after broker failure due to overload.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-963%3A+Additional+metrics+in+Tiered+Storage">KIP-963 Additional metrics in Tiered Storage</a>:
</b><br>KIP-405 brought the early access of Tiered Storage, and with this update we’re introducing new metrics for the feature, allowing you to better monitor performance, troubleshoot, and prevent issues.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+processes">KIP-979 Allow independently stop KRaft processes</a>:
</b><br>Adds a way to independently stop KRaft processes in cases where operators are running in combined mode (a controller and broker in the same node). Previously, you could only stop both. The command line for stopping Kafka nodes now includes a pair of optional and mutually exclusive parameters "[--process-role]" OR "[--node-id]" to use with ./bin/kafka-server-stop.sh.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-890%3A+Transactions+Server-Side+Defense">KIP-890 Transactions Server-Side Defense</a>:
</b><br>Another part of this KIP shipped, this time adding transaction verification preventing hanging transactions for consumer offset partitions.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0">KIP-896 Remove old client protocol API versions in Kafka 4.0</a>:
</b><br>KIP-896 marks all client request versions older than Apache Kafka 2.1 as deprecated, and introduces new metrics to monitor the presence of such requests. Support for these requests will be dropped in Apache Kafka 4.0.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-919%3A+Allow+AdminClient+to+Talk+Directly+with+the+KRaft+Controller+Quorum+and+add+Controller+Registration">KIP-919 Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration</a>:
</b><br>Allows the AdminClient to talk directly with the KRaft Controller Quorum, which allows us to support operations like DESCRIBE_QUORUM and INCREMENTAL_ALTER_CONFIGS to dynamically change the log4j levels on a KRaft controller. This helps with debugging scenarios where other parts of the system are down.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263429128">KIP-978 Allow dynamic reloading of certificates with different DN / SANs</a>:
</b><br>Allows configurability to prevent checks that ensure DN (Distinguished Name) and SAN (Subject Alternative Names) are the same between old and new keystores when they’re being reload. This ultimately allows dynamic reloading of certificates that have different DN/SANs.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1001%3A+Add+CurrentControllerId+Metric">KIP-1001 Add CurrentControllerId Metric</a>:
</b><br>Adds a CurrentControllerId metric, allowing users to find the current controller by looking at the metrics of any Kafka broker/controller.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-938%3A+Add+more+metrics+for+measuring+KRaft+performance">KIP-938 Add more metrics for measuring KRaft performance</a>:
</b><br>We’ve added 11 new metrics for measuring KRaft-mode performance, including ActiveControllersCount, CurrentMetadataVersion and TimedOutBrokerHeartbeatCount.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=284789510">KIP-1013 Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)</a>:
</b><br>Java 11 is now deprecated for all the server and tools modules - kafka-server-common, kafka-server, kafka_2.13, kafka-storage, kafka-metadata, kafka-group-coordinator, kafka-raft, kafka-shell, kafka-tools. It will not be supported after Kafka 4.0. All other modules (including clients, streams and connect) will continue to require Java 11 (as specified by <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181308223">KIP-750</a>).
</li>
</ul>
<h3>Kafka Streams</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams">KIP-925 Rack aware task assignment in Kafka Streams</a>:
</b><br>In part one of KIP-925, the min_traffic assignment strategy for Kafka Streams was added. Part two finishes the KIP by introducing the second rack-aware assignment strategy: balanced_subtopology.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-954%3A+expand+default+DSL+store+configuration+to+custom+types">KIP-954 Expand default DSL store configuration to custom types</a>:
</b><br>KIP-954 builds on KIP-591 and allows users to provide a default state store provider for their custom stores. As part of this change, a new interface has been provided along with default support for RocksDB and in-memory state stores.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams">KIP-962 Relax non-null key requirement in Kafka Streams</a>:
</b><br>Kafka Streams treated records with null-keys as invalid input for joins and dropped them on the floor. KIP-962 relaxes this behavior for various left-joins and thus allows null-key records to be processed successfully.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-988%3A+Streams+Standby+Update+Listener">KIP-988 Streams Standby Update Listener</a>:
</b><br>Adds a new interface for handling cases where standby tasks have their a) state stores registered, b) load a batch of records and c) stop updates.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-960%3A+Support+single-key_single-timestamp+interactive+queries+%28IQv2%29+for+versioned+state+stores">KIP-960</a> / <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-968%3A+Support+single-key_multi-timestamp+interactive+queries+%28IQv2%29+for+versioned+state+stores">KIP-968</a> IQ support for Versioned State Stores:
</b><br>Version state stores were added in Apache Kafka 3.5 release (KIP-889), but it was not possible so far to query the new stores. KIP-960 and KIP-968 close this gap by adding new query types for IQv2 namely VersionedKeyQuery and MultiVersionedKeyQuery, respectively. Both queries allow you to do lookups for a single key, to ask for the most recent value, a historic value, or a range of historic values for the provided key.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-985%3A+Add+reverseRange+and+reverseAll+query+over+kv-store+in+IQv2">KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2</a>:
</b><br>IQv2 supports RangeQueries which allow you to query for a range of keys. However, the API did not provide any guarantee about the ordering of the returned result. With KIP-985 it’s now possible to request the result to be ordered (per partition) in either ascending or descending order, or leave the order unspecified.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-992%3A+Proposal+to+introduce+IQv2+Query+Types%3A+TimestampedKeyQuery+and+TimestampedRangeQuery">KIP-992 Introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery</a>:
</b><br>Finally, KIP-992 adds new timestamped-key and timestamped-range interactive queries for timestamped key-value state stores. This change improves the type safety of the IQv2 API. The existing RangeQuery now always returns only the value if issued against a timestamped key-value store.
</li>
</ul>
<h3>Kafka Connect</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-976%3A+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect">KIP-976 Cluster-wide dynamic log adjustment for Kafka Connect</a>:
</b><br>KIP-495 introduced the ability for users to dynamically change the log level of individual workers in their connect cluster. To address situations when it is not possible to target an individual worker, KIP-976 allows for dynamic log-level changes to be broadcast to all workers across the cluster in a single update.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state">KIP-980 Allow creating connectors in a stopped state</a>:
</b><br>This update allows users to create new Connectors in a STOPPED or PAUSED state, enabling use cases like migrating connectors. There is now a new optional "initial_state" field during connector creation.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-959%3A+Add+BooleanConverter+to+Kafka+Connect">KIP-959 Add BooleanConverter to Kafka Connect</a>:
</b><br>KIP-959 adds a BooleanConverter to Connect, supporting serializing and deserializing the boolean primitive in a Kafka Connect schema. This implements both org.apache.kafka.connect.storage.Converter and org.apache.kafka.connect.storage.HeaderConverter.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-970%3A+Deprecate+and+remove+Connect%27s+redundant+task+configurations+endpoint">KIP-970 Deprecate and remove Connect's redundant task configurations endpoint</a>:
</b><br>Deprecates a redundant endpoint for fetching task configurations, eventually to be removed in Apache Kafka 4.0.
</li>
</ul>
<h3>Summary</h3>
<p>Ready to get started with Apache Kafka 3.7.0? Check out all the details in the <a href="https://archive.apache.org/dist/kafka/3.7.0/RELEASE_NOTES.html">release notes</a> and <a href="https://kafka.apache.org/downloads">download</a> Apache Kafka 3.7.0.</p>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 146 contributors: <br>
Abhijeet Kumar, Akhilesh Chaganti, Alieh, Alieh Saeedi, Almog Gavra, Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew Schofield, Anna Sophie Blee-Goldman, Anton Agestam, Apoorv Mittal, Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj, ashwinpankaj, atu-sharm, bachmanity1, Bob Barrett, Bruno Cadonna, Calvin Liu, Cerchie, chern, Chris Egerton, Christo Lolov, Colin Patrick McCabe, Colt McNealy, Crispin Bernier, David Arthur, David Jacot, David Mao, Deqi Hu, Dimitar Dimitrov, Divij Vaidya, Dongnuo Lyu, Eaugene Thomas, Eduwer Camacaro, Eike Thaden, Federico Valeri, Florin Akermann, Gantigmaa Selenge, Gaurav Narula, gongzhongqiang, Greg Harris, Guozhang Wang, Gyeongwon, Do, Hailey Ni, Hanyu Zheng, Hao Li, Hector Geraldino, hudeqi, Ian McDonald, Iblis Lin, Igor Soarez, iit2009060, Ismael Juma, Jakub Scholz, James Cheng, Jason Gustafson, Jay Wang, Jeff Kim, Jim Galasyn, John Roesler, Jorge Esteban Quilcate Otoya, Josep Prat, José Armando García Sancio, Jotaniya Jeel, Jouni Tenhunen, Jun Rao, Justine Olshan, Kamal Chandraprakash, Kirk True, kpatelatwork, kumarpritam863, Laglangyue, Levani Kokhreidze, Lianet Magrans, Liu Zeyu, Lucas Brutschy, Lucia Cerchie, Luke Chen, maniekes, Manikumar Reddy, mannoopj, Maros Orsak, Matthew de Detrich, Matthias J. Sax, Max Riedel, Mayank Shekhar Narula, Mehari Beyene, Michael Westerby, Mickael Maison, Nick Telford, Nikhil Ramakrishnan, Nikolay, Okada Haruki, olalamichelle, Omnia G.H Ibrahim, Owen Leung, Paolo Patierno, Philip Nee, Phuc-Hong-Tran, Proven Provenzano, Purshotam Chauhan, Qichao Chu, Matthias J. Sax, Rajini Sivaram, Renaldo Baur Filho, Ritika Reddy, Robert Wagner, Rohan, Ron Dagostino, Roon, runom, Ruslan Krivoshein, rykovsi, Sagar Rao, Said Boudjelda, Satish Duggana, shuoer86, Stanislav Kozlovski, Taher Ghaleb, Tang Yunzi, TapDang, Taras Ledkov, tkuramoto33, Tyler Bertrand, vamossagar12, Vedarth Sharma, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Wuzhengyu97, Xavier Léauté, Xiaobing Fang, yangy0000, Ritika Reddy, Yanming Zhou, Yash Mayya, yuyli, zhaohaidao, Zihao Lin, Ziming Deng
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_352_release_announcement"></a>
<a href="#apache_kafka_352_release_announcement">Apache Kafka 3.5.2 Release Announcement</a>
</h2>
11 December 2023 - Luke Chen (<a href="https://twitter.com/showuon1">@showuon1</a>)
<p>We are proud to announce the release of Apache Kafka 3.5.2. This is a bugfix release. It contains many bug fixes including upgrades the Snappy and Rocksdb dependencies. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.5.2/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/documentation.html#upgrade_3_5_2">Upgrading to 3.5.2 from any version 0.8.x through 3.4.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
A. Sophie Blee-Goldman, Anna Sophie Blee-Goldman, atu-sharm, bachmanity1, Bill Bejeck, Calvin Liu, Chase Thomas, Chris Egerton, Christo Lolov, Colin P. McCabe, Colin Patrick McCabe, David Arthur, David Jacot, Divij Vaidya, Farooq Qaiser, Federico Valeri, flashmouse, Florin Akermann, Greg Harris, Guozhang Wang, Hao Li, hudeqi, Ismael Juma, Jason Gustafson, José Armando García Sancio, Josep Prat, Levani Kokhreidze, lixy, Lucas Brutschy, Luke Chen, Manikumar Reddy, Matthias J. Sax, Mickael Maison, Nick Telford, Okada Haruki, Omnia G.H Ibrahim, Philip Nee, Qichao Chu (@ex172000), Mickael Maison, Rajini Sivaram, Ritika Reddy, Robert Wagner, Rohan, Ron Dagostino, Sagar Rao, Said Boudjelda, sciclon2, Viktor Somogyi-Vass, Vincent Jiang, Xiaobing Fang, Yash Mayya
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_361_release_announcement"></a>
<a href="#apache_kafka_361_release_announcement">Apache Kafka 3.6.1 Release Announcement</a>
</h2>
7 December 2023 - Mickael Maison (<a href="https://twitter.com/MickaelMaison">@MickaelMaison</a>)
<p>We are proud to announce the release of Apache Kafka 3.6.1. This is a bugfix release. It contains 30 fixes including upgrades to the ZooKeeper and Netty dependencies. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.6.1/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/36/documentation.html#upgrade_3_6_1">Upgrading to 3.6.1 from any version 0.8.x through 3.5.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release:</br>
Anna Sophie Blee-Goldman, Arpit Goyal, atu-sharm, Bill Bejeck, Chris Egerton, Colin P. McCabe, David Arthur, David Jacot, Divij Vaidya, Federico Valeri, Greg Harris, Guozhang Wang, Hao Li, hudeqi, iit2009060, Ismael Juma, Jorge Esteban Quilcate Otoya, Josep Prat, Jotaniya Jeel, Justine Olshan, Kamal Chandraprakash, kumarpritam863, Levani Kokhreidze, Lucas Brutschy, Luke Chen, Manikumar Reddy, Matthias J. Sax, Mayank Shekhar Narula, Mickael Maison, Nick Telford, Philip Nee, Qichao Chu, Rajini Sivaram, Robert Wagner, Sagar Rao, Satish Duggana, Walker Carlson, Xiaobing Fang, Yash Mayya
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_360_release_announcement"></a>
<a href="#apache_kafka_360_release_announcement">Apache Kafka 3.6.0 Release Announcement</a>
</h2>
10 Oct 2023 - Satish Duggana (<a href="https://twitter.com/0xeed">@SatishDuggana</a>)
<p>We are proud to announce the release of Apache Kafka 3.6.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/36/documentation.html#upgrade_3_6_0">Upgrading to 3.6.0 from any version 0.8.x through 3.5.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<p>
The 3.6.0 release marked the first production ready release for the migration of Kafka
clusters from a ZooKeeper metadata system to a KRaft metadata system. Users wanting to
perform this migration are advised to first upgrade to 3.6.2 or 3.7.1 to receive important
bug fixes.
<p>
See the ZooKeeper to KRaft migration
<a href="https://kafka.apache.org/documentation/#kraft_zk_migration">operations documentation</a> for
details. Note that support for JBOD is still not available for KRaft clusters, therefore clusters
utilizing JBOD can not be migrated. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft">KIP-858</a>
for details regarding KRaft and JBOD.
</p>
<p>Support for Delegation Tokens in KRaft (<a href="https://issues.apache.org/jira/browse/KAFKA-15219">KAFKA-15219</a>) was completed in 3.6, further reducing the gap of features between ZooKeeper-based Kafka clusters and KRaft. Migration of delegation tokens from ZooKeeper to KRaft is also included in 3.6.</p>
<p><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage">Tiered Storage</a> is an early access feature. It is currently only suitable for testing in non production environments. See the <a href="https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Tiered+Storage+Early+Access+Release+Notes">Early Access Release Notes</a> for more details.</p>
<p><i>Note: ZooKeeper is marked as deprecated since 3.5.0 release. ZooKeeper is planned to be removed in Apache Kafka 4.0. For more information, please see the documentation for <a href="/documentation#zk_depr">ZooKeeper Deprecation</a></i><p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage">KIP-405</a>:
Kafka Tiered Storage (Early Access): </b><br>Introduces Tiered Storage to Kafka. Note that this
is an early access feature only advised for use in non-production environments (see the <a
href="https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Tiered+Storage+Early+Access+Release+Notes">early
access notes</a> for more information). This feature provides a separation of computation
and storage in the broker for pluggable storage tiering natively in Kafka Tiered Storage brings
a seamless extension of storage to remote objects with minimal operational changes.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-890%3A+Transactions+Server-Side+Defense">KIP-890</a>:
Transactions Server Side Defense (Part 1): </b><br>Hanging transactions can negatively impact
your read committed consumers and prevent compacted logs from being compacted. KIP-890 helps
address hanging transactions by verifying partition additions. Part 2 of KIP-890 will optimize
verification, which currently adds an extra hop.<br>
In 3.6.0, transaction verification will prevent hanging transactions on data partitions. In the next release, transactional offset commits will also be covered.
</li>
<li><b><a
href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195726330">KIP-797</a>:
Accept duplicate listener on port for IPv4/IPv6: </b><br>Until now, Kafka has not supported
duplicate listeners on the same port. This works when using only a single IP stack, but presents
an issue if you are working with both IPv4 and IPv6. With KIP-797, brokers can be configured
with listeners that have the same port on different IP stacks. This update does not affect
advertised listeners, which already have this feature.
</li>
<li><b><a
href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152035">KIP-863</a>:
Reduce CompletedFetch#parseRecord() memory copy: </b><br>Reduces memory allocation and improves
memory performance during record deserialization by using a ByteBuffer instead of byte[] for
deserialization, which improves efficiency. Updated public interfaces include the Deserializer
class, ByteBufferDeserializer class, and StringDeserializer class.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-868+Metadata+Transactions">KIP-868</a>:
Metadata Transactions: </b><br>Improves the overall durability of the KRaft layer by adding
metadata transactions that consist of:
<ul>
<li>BeginTransaction</li>
<li>Number of records</li>
<li>EndTransaction or AbortTransaction</li>
</ul>
KRaft uses record batches as a mechanism for atomicity. Typically, there was a limit to the
fetch size on the Raft consensus layer, and the controller could generate a set of atomic
records that exceeded this limit. This update introduces marker records that allow larger sets
of atomic records to be sent to the Raft consensus layer in multiple batches. This bypasses the
fetch limit.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-902%3A+Upgrade+Zookeeper+to+3.8.2">KIP-902</a>:
Upgrade Zookeeper to 3.8.2:</b><br>This upgrades the ZooKeeper version that is bundled with Kafka to
version 3.8.2. The new version includes several updates and security improvements.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-917%3A+Additional+custom+metadata+for+remote+log+segment">KIP-917</a>:
Additional custom metadata for remote log segment:</b><br> It introduces having an optional
custom metadata as part of remote log segment metadata. RemoteStorageManager returns the
optional custom metadata when copyLogSegmentData() is invoked. It will be passed along with
remote log segment metadata.
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-937%3A+Improve+Message+Timestamp+Validation">KIP-937</a>:
Improve Message Timestamp Validation: </b><br>It improves data integrity and prevents potential
pitfalls caused by inaccurate timestamp handling by adding more validation logic for message
timestamps. While past timestamps are a normal occurrence in Kafka, future timestamps might
represent an incorrectly formatted integer. KIP-937 rejects messages with future timestamps and
provides a descriptive exemption.
</li>
<li><b><a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-938%3A+Add+more+metrics+for+measuring+KRaft+performance">KIP-938</a>:
Add more metrics for measuring KRaft performance: </b><br>Adds new controller, loader, and snapshot emitter KRaft performance metrics.
</li>
</ul>
<h3>Kafka Streams</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-923%3A+Add+A+Grace+Period+to+Stream+Table+Join">KIP-923</a>: Add A Grace Period to Stream Table Join: </b><br>Adds a grace period to stream-table joins to improve table-side out-of-order data handling. The joined object has a new method called "withGracePeriod" that will cause the table side lookup to only happen after the grace period has passed.</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-941%3A+Range+queries+to+accept+null+lower+and+upper+bounds">KIP-941</a>: Range queries to accept null lower and upper bounds:</b><br>Previously, RangeQuery did not support null to specify “no upper/lower bound”.
KIP-941 allows users to pass null into withRange(...) for lower/upper bounds to specify a full or half-open range:
<ul>
<li>`withRange(null, null)` == `withNoBounds()`</li>
<li>`withRange(lower, null)` == `withLowerBound(lower)`</li>
<li>`withRange(null, upper)` == `withUpperBound(upper)`</li>
</ul>
</li>
</ul>
<h3>Kafka Connect</h3>
<ul>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-793%3A+Allow+sink+connectors+to+be+used+with+topic-mutating+SMTs">KIP-793</a>: Allow sink connectors to be used with topic-mutating SMTs: </b><br>Adds support for topic-mutating SMTs for async sink connectors. This is to address an incompatibility between sink connectors overriding the SinkTask::preCommit method and SMTs that mutate the topic field of a SinkRecord .</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect">KIP-875</a>: First-class offsets support in Kafka Connect: </b><br>Provides first-class admin support for offsets in Kafka Connect. KIP-875 Part 1 added endpoints to get offsets and a new STOPPED state for connectors. The alter offsets and reset offsets endpoints have now been added.
<table>
<tr>
<th>Action</th>
<th>Description</th>
</tr>
<tr>
<td>GET /connectors/{connector}/offsets</td>
<td>Retrieve the offsets for a connector; the connector must exist</td>
</tr>
<tr>
<td>PATCH /connectors/{connector}/offsets</td>
<td>Alter the offsets for a connector; the connector must exist, and must be in the STOPPED state
</td>
</tr>
<tr>
<td>DELETE /connectors/{connector}/offsets</td>
<td>Reset the offsets for a connector; the connector must exist, and must be in the STOPPED state
</td>
</tr>
<tr>
<td>PUT /connectors/{connector}/pause</td>
<td>Pause the connector; the connector must exist</td>
</tr>
</table>
</li>
<li><b><a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-898%3A+Modernize+Connect+plugin+discovery">KIP-898</a>: Modernize Connect plugin discovery: </b><br>With KIP-898, Connect workers can now read from ServiceLoader manifests and module info directly during startup for more efficient plugin class discovery. Note that this update allows connector developers to add service declarations to their plugins.</li>
</ul>
<h3>Summary</h3>
<p>Ready to get started with Apache Kafka 3.6.0? Check out all the details in the <a href="https://archive.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html">release notes</a> and <a href="https://kafka.apache.org/downloads">download</a> Apache Kafka 3.6.0.</p>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 139 contributors: <br>
A. Sophie Blee-Goldman, Aaron Ai, Abhijeet Kumar, aindriu-aiven, Akhilesh Chaganti, Alexandre Dupriez, Alexandre Garnier, Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew Schofield, Andrew Grant, Aneel Kumar, Anton Agestam, Artem Livshits, atu-sharm, bachmanity1, Bill Bejeck, Bo Gao, Bruno Cadonna, Calvin Liu, Chaitanya Mukka, Chase Thomas, Cheryl Simmons, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Clay Johnson, Colin P. McCabe, Colt McNealy, d00791190, Damon Xie, Danica Fine, Daniel Scanteianu, Daniel Urban, David Arthur, David Jacot, David Mao, dengziming, Deqi Hu, Dimitar Dimitrov, Divij Vaidya, DL1231, Dániel Urbán, Erik van Oosten, ezio, Farooq Qaiser, Federico Valeri, flashmouse, Florin Akermann, Gabriel Oliveira, Gantigmaa Selenge, Gaurav Narula, GeunJae Jeon, Greg Harris, Guozhang Wang, Hailey Ni, Hao Li, Hector Geraldino, hudeqi, hzh0425, Iblis Lin, iit2009060, Ismael Juma, Ivan Yurchenko, James Shaw, Jason Gustafson, Jeff Kim, Jim Galasyn, John Roesler, Joobi S B, Jorge Esteban Quilcate Otoya, Josep Prat, Joseph (Ting-Chou) Lin, José Armando García Sancio, Jun Rao, Justine Olshan, Kamal Chandraprakash, Keith Wall, Kirk True, Lianet Magrans, LinShunKang, Liu Zeyu, lixy, Lucas Bradstreet, Lucas Brutschy, Lucent-Wong, Lucia Cerchie, Luke Chen, Manikumar Reddy, Manyanda Chitimbo, Maros Orsak, Matthew de Detrich, Matthias J. Sax, maulin-vasavada, Max Riedel, Mehari Beyene, Michal Cabak (@miccab), Mickael Maison, Milind Mantri, minjian.cai, mojh7, Nikolay, Okada Haruki, Omnia G H Ibrahim, Owen Leung, Philip Nee, prasanthV, Proven Provenzano, Purshotam Chauhan, Qichao Chu, Rajini Sivaram, Randall Hauch, Renaldo Baur Filho, Ritika Reddy, Rittika Adhikari, Rohan, Ron Dagostino, Sagar Rao, Said Boudjelda, Sambhav Jain, Satish Duggana, sciclon2, Shekhar Rajak, Sungyun Hur, Sushant Mahajan, Tanay Karmarkar, tison, Tom Bentley, vamossagar12, Victoria Xia, Vincent Jiang, vveicc, Walker Carlson, Yash Mayya, Yi-Sheng Lien, Ziming Deng, 蓝士钦
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_351_release_announcement"></a>
<a href="#apache_kafka_351_release_announcement">Apache Kafka 3.5.1 Release Announcement</a>
</h2>
21 July 2023 - Divij Vaidya (<a href="https://twitter.com/divijvaidya">@DivijVaidya</a>)
<p>We are proud to announce the release of Apache Kafka 3.5.1. This is a security patch release. It upgrades the dependency, snappy-java, to a version which is not vulnerable to <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-34455">CVE-2023-34455</a>. You can find more information about the CVE at <a href="https://kafka.apache.org/cve-list#CVE-2023-3445">Kafka CVE list</a>. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.5.1/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/35/documentation.html#upgrade_3_5_1">Upgrading to 3.5.1 from any version 0.8.x through 3.4.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li>
Upgraded the dependency, snappy-java, to a version which is not vulnerable to
<a href="https://nvd.nist.gov/vuln/detail/CVE-2023-34455">CVE-2023-34455.</a>
You can find more information about the CVE at <a href="https://kafka.apache.org/cve-list#CVE-2023-34455">Kafka CVE list.</a>
</li>
<li>
Fixed a regression introduced in 3.3.0, which caused <code>security.protocol</code> configuration values to be restricted to
upper case only. After the fix, <code>security.protocol</code> values are case insensitive.
See <a href="https://issues.apache.org/jira/browse/KAFKA-15053">KAFKA-15053</a> for details.
</li>
</ul>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 22 authors. Please report an unintended omission.
<p>Alyssa Huang, Aman Singh, andymg3, Bo Gao, Calvin Liu, Chia-Ping Tsai, Chris Egerton, d00791190, Damon Xie, David Arthur, David Jacot, Divij Vaidya, DL1231, ezio, Manikumar Reddy, Manyanda Chitimbo, Mickael Maison, minjian.cai, Proven Provenzano, Sambhav Jain, vamossagar12, Yash Mayya</p>
</p>
</article>
<article>
<h2 class="bullet">
<a id="apache_kafka_350_release_announcement"></a>
<a href="#apache_kafka_350_release_announcement">Apache Kafka 3.5.0 Release Announcement</a>
</h2>
15 June 2023 - Mickael Maison (<a href="https://twitter.com/MickaelMaison">@MickaelMaison</a>)
<p>We are proud to announce the release of Apache Kafka 3.5.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/3.5.0/RELEASE_NOTES.html">release notes</a>.</p>
<p>See the <a href="https://kafka.apache.org/35/documentation.html#upgrade_3_5_0">Upgrading to 3.5.0 from any version 0.8.x through 3.4.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p>
<p>The ability to migrate Kafka clusters from ZK to KRaft mode with no downtime is still an early access feature. It is currently only suitable for testing in non production environments. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-866+ZooKeeper+to+KRaft+Migration">KIP-866</a> for more details.</p>
<p><i>Note: ZooKeeper is now marked deprecated in this release. ZooKeeper is planned to be removed in Apache Kafka 4.0. (Cf <a href="/documentation#zk_depr">ZooKeeper Deprecation</a>)</i><p>
<h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3>
<ul>
<li><b>KIP-881: Rack-aware Partition Assignment for Kafka Consumers</b>: Kafka 3.4.0 only contained the protocol changes for <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-881%3A+Rack-aware+Partition+Assignment+for+Kafka+Consumers">KIP-881</a>. The built-in assignors have now been updated to support rack-awareness.</li>
<li><b>KIP-887: Add ConfigProvider to make use of environment variables</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-887%3A+Add+ConfigProvider+to+make+use+of+environment+variables">KIP-887</a> introduces a new <code><a href="https://kafka.apache.org/35/javadoc/org/apache/kafka/common/config/provider/ConfigProvider.html">ConfigProvider</a></code> implementation, <code>EnvVarConfigProvider</code>, to retrieve configurations from environment variables.</li>
<li><b>KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for Kafka Brokers</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-900%3A+KRaft+kafka-storage.sh+API+additions+to+support+SCRAM+for+Kafka+Brokers">KIP-900</a> updates the <code>kafka-storage</code> tool and adds a mechanism to configure SCRAM for inter broker authentication with KRaft.</li>
<li><b>KIP-903: Replicas with stale broker epoch should not be allowed to join the ISR:</b> <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-903%3A+Replicas+with+stale+broker+epoch+should+not+be+allowed+to+join+the+ISR">KIP-903</a> addresses a limitation of the inter broker replication protocol which could lead to data loss in case of a broker failing while another broker had an unclean shutdown.</li>
</ul>
<h3>Kafka Streams</h3>
<ul>
<li><b>KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-399%3A+Extend+ProductionExceptionHandler+to+cover+serialization+exceptions">KIP-399</a> adds a method, <code>handleSerializationException()</code>, to the <code>ProductionExceptionHandler</code> interface to handle any serialization errors encountered while producing records.</li>
<li><b>KIP-889: Versioned State Stores</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores">KIP-889</a> introduces versioned state stores to improve the accuracy of joins when out of order records are processed.</li>
<li><b>KIP-907: Add Boolean Serde to public interface</b>: Kafka Streams includes built-in Serdes for most primitive types. <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-907%3A+Add+Boolean+Serde+to+public+interface">KIP-907</a> adds a new one for booleans.</li>
</ul>
<h3>Kafka Connect</h3>
<ul>
<li><b>KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+support+for+distributed+mode+in+dedicated+MirrorMaker+2.0+clusters">KIP-710</a> improves the dedicated mode of MirrorMaker. It now supports running multiple instances and handling automatic reconfigurations.</li>
<li><b>KIP-875: First-class offsets support in Kafka Connect</b>: <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect">KIP-875</a> adds REST API endpoints for managing offset of Connectors. 3.5.0 only contains endpoints for listing offsets. Endpoints for updating and deleting offsets will come in a future release.</li>
<li><b>KIP-894: Use incrementalAlterConfig for syncing topic configurations</b>: With <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations">KIP-894</a>, MirrorMaker can use the <code>IncrementalAlterConfig</code> API when mirroring topic configuration between clusters.</li>
<li><b>KAFKA-14021: MirrorMaker 2 should implement KIP-618 APIs</b>: <a href="https://issues.apache.org/jira/browse/KAFKA-14021">KAFKA-14021</a> adds support for exactly-once semantics to MirrorSourceConnector.</li>
</ul>
<h3>Summary</h3>
<p>This was a community effort, so thank you to everyone who contributed to this release, including all our users and our 103 authors:
A. Sophie Blee-Goldman, Akhilesh Chaganti, Alex Sorokoumov, Alexandre Dupriez, Alyssa Huang, Anastasia Vela, Andreas Maechler, andymg3, Artem Livshits, atu-sharm, bachmanity1, Bill Bejeck, Brendan Ribera, Calvin Liu, Chaitanya Mukka, Cheryl Simmons, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Colin P. McCabe, csolidum, Daniel Scanteianu, David Arthur, David Jacot, David Karlsson, David Mao, Dejan Stojadinović, Divij Vaidya, dorwi, drgnchan, Dániel Urbán, Edoardo Comar, egyedt, emilnkrastev, Eric Haag, Farooq Qaiser, Federico Valeri, Gantigmaa Selenge, Greg Harris, Guozhang Wang, Hao Li, Hector Geraldino, Himani Arora, Hoki Min, hudeqi, iamazy, Iblis Lin, Ismael Juma, Ivan Yurchenko, Jakub Scholz, Jason Gustafson, Jeff Kim, Jim Galasyn, Jorge Esteban Quilcate Otoya, Josep Prat, José Armando García Sancio, Juan José Ramos, Junyang Liu, Justine Olshan, Kamal Chandraprakash, Kirk True, Kowshik Prakasam, littlehorse-eng, liuzc9, Lucas Brutschy, Lucia Cerchie, Luke Chen, Manikumar Reddy, Manyanda Chitimbo, Matthew Wong, Matthias J. Sax, Matthias Seiler, Michael Marshall, Mickael Maison, nicolasguyomar, Nikolay, Paolo Patierno, Philip Nee, Pierangelo Di Pilato, Proven Provenzano, Purshotam Chauhan, Qing, Rajini Sivaram, RivenSun, Robert Young, Rohan, Roman Schmitz, Ron Dagostino, Ruslan Krivoshein, Satish Duggana, Shay Elkin, Shekhar Rajak, Simon Woodman, Spacrocket, stejani-cflt, Terry, Tom Bentley, vamossagar12, Victoria Xia, Viktor Somogyi-Vass, Vladimir Korenev, Yash Mayya, Zheng-Xian Li
</p>
</article>
<!--#include virtual="includes/_footer.htm" -->
<script>
// Show selected style on nav item
$(function() { $('.b-nav__project').addClass('selected'); });
</script>