KAFKA-17298: Update upgrade notes for 4.0 KIP-848 #18756
Conversation
| @@ -89,6 +89,27 @@ <h5><a id="upgrade_400_notable" href="#upgrade_400_notable">Notable changes in 4 | |||
| </li> | |||
| <li><b>Broker</b> | |||
| <ul> | |||
| <li>The new Consumer Group Rebalance Protocol is now a production ready feature, enabled by default on the broker. | |||
There was a problem hiding this comment.
nit: Should we add (KIP-848) after Consumer Group Rebalance Protocol and link the KIP?
| @@ -89,6 +89,27 @@ <h5><a id="upgrade_400_notable" href="#upgrade_400_notable">Notable changes in 4 | |||
| </li> | |||
| <li><b>Broker</b> | |||
| <ul> | |||
| <li>The new Consumer Group Rebalance Protocol is now a production ready feature, enabled by default on the broker. | |||
| The supported group protocols are defined with the config <code>group.coordinator.rebalance.protocols=classic,consumer</code>, | |||
There was a problem hiding this comment.
We may want to explain the group.version feature flag. It will be automatically set when the upgrade to 4.0 is finalized with the kafka-features command. Until this is done, the new protocol is not enabled.
| with both protocols supported. Client applications should select the protocol to use (see Consumer section below). | ||
| <ul> | ||
| <li> | ||
| The supported partition assignment strategies are defined on the broker now, via the config <code>group.consumer.assignors</code>. |
There was a problem hiding this comment.
Should we mention the two assignors configured by default?
| The supported partition assignment strategies are defined on the broker now, via the config <code>group.consumer.assignors</code>. | ||
| It defaults to <code>org.apache.kafka.coordinator.group.assignor.UniformAssignor</code>, so this will be the assignor used unless the consumer selects a different one (Consumer section below). | ||
| See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217387038#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-AssignorSelection">Assignor selection</a> for more details. | ||
| </li> |
There was a problem hiding this comment.
I wonder if we should put a bullet point about the new server side configs (session, heartbeat interval). What do you think?
| <li> | ||
| A new config <code>group.coordinator.threads</code> is added to control the number of threads used by the new group coordinator. | ||
| It defaults to 1, and it should be adjusted as needed. | ||
| See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217387038#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-GroupCoordinator">New Group Coordinator</a> for more details. | ||
| </li> | ||
| </ul> |
There was a problem hiding this comment.
I wonder wether we should bring this as its own bullet point before the one about the new rebalance protocol because this is important regardless of the new protocol. What do you think? We could perhaps explain that 4.0 ships with a new group coordinator, etc.
| See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217387038#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-GroupCoordinator">New Group Coordinator</a> for more details. | ||
| </li> | ||
| </ul> | ||
| </li> |
There was a problem hiding this comment.
I wonder if we should put a few words about the downgrade path after the new protocol is enabled and used. What do you think?
| <li> | ||
| New metrics have been added to the Consumer when using the new rebalance protocol, mainly providing visibility over the improved threading model. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1068%3A+New+metrics+for+the+new+KafkaConsumer">New Consumer Metrics</a> | ||
| </li> | ||
| </ul> |
There was a problem hiding this comment.
- Should we mention the upgrade path too?
- Should we mention the limitations (e.g. client side assignors are not supported)?
Broker and Consumer notable changes