[ Ссылка ] | Apache Kafka® 3.2 delivers new KIPs in three different areas of the Kafka ecosystem: Kafka Core, Kafka Streams, and Kafka Connect. On behalf of the Kafka community, Danica Fine (Senior Developer Advocate, Confluent), shares release highlights.

More than half of the KIPs in the new release concern Kafka Core. KIP-704 addresses unclean leader elections by allowing for further communication between the controller and the brokers. KIP-764 takes on the problem of a large number of client connections in a short period of time during preferred leader election by adding the configuration `socket.listen.backlog.size`. KIP-784 adds an error code field to the response of the `DescribeLogDirs` API, and KIP-788 improves network traffic by allowing you to set the pool size of network threads individually per listener on Kafka brokers. Finally, in accordance with the imminent KRaft protocol, KIP-801 introduces a built-in `StandardAuthorizer` that doesn't depend on ZooKeeper.

There are five KIPs related to Kafka Streams in the AK 3.2 release. KIP-708 brings rack-aware standby assignment by tag, which improves fault tolerance.

Then there are three projects related to Interactive Queries v2: KIP-796 specifies an improved interface for Interactive Queries; KIP-805 allows state to be queried over a specific range; and KIP-806 adds two implementations of the Query interface, `WindowKeyQuery` and `WindowRangeQuery`.

The final Kafka Streams project, KIP-791, enhances `StateStoreContext` with `recordMetadata`,which may be accessed from state stores.

Additionally, this Kafka release introduces Kafka Connect-related improvements, including KIP-769, which extends the `/connect-plugins` API, letting you list all available plugins, and not just connectors as before. KIP-779 lets `SourceTasks` handle producer exceptions according to `error.tolerance`, rather than instantly killing the entire connector by default. Finally, KIP-808 lets you specify precisions with respect to TimestampConverter single message transforms.

Tune in to learn more about the Apache Kafka 3.2 release!

EPISODE LINKS
► Apache Kafka 3.2 release notes: [ Ссылка ]
► Read the blog to learn more: [ Ссылка ]
► Download Apache Kafka 3.2.0: [ Ссылка ]
► Listen to the audio version: [ Ссылка ]

TIMESTAMPS
00:00 - Intro
0:28 - KIP-704: Send a hint to the partition leader to recover the partition
0:52 - KIP-764: Configurable backlog size for creating Acceptor
1:06 - KIP-784: Add top-level error code field to DescribeLogDirsResponse
1:23 - KIP-788: Allow configuring num.network.threads per listener
1:52 - KIP-798: Add possibility to write kafka headers in Kafka Console Producer
KIP-810:Allow producing records with null values in Kafka Console Producer
2:17 - KIP-800: Add reason to JoinGroupRequest and LeaveGroupRequest
02:40 - KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata
3:03 - KIP-814: Static membership protocol should let the leader skip assignment
3:34 - KIP-708: Rack awareness for Kafka Streams
3:52 - KIP-805: Add range and scan query over kv-store in IQv2
4:05 - KIP-806: Add session and window query over kv-store in IQv2
4:27 - KIP-791: Add Record Metadata to StateStoreContext
4:44 - KIP-769: Connect APIs to list all connector plugins and retrieve their configuration definitions
5:08 - KIP-808: Add support for different unix precisions in TimestampConverter SMT
5:25 - KIP-779: Allow Source Tasks to Handle Producer Exceptions
5:51 - That's a wrap

ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.

#apachekafka #kafka #confluent

свернуть