MINOR: Tweak default group coordinator config & upgrade notes (#18948)

This patch changes the default value of `group.coordinator.threads` to `4` and sets it priority to `HIGH`. This change makes it consistent with how we handle `num.network.threads` and `num.io.threads`. The patch also tweaks the upgrade notes.

Reviewers: Ismael Juma <ismael@juma.me.uk>

docs/upgrade.html

@@ -52,17 +52,13 @@
         </li>
         <li>
             Apache Kafka 4.0 ships with a brand-new group coordinator implementation (See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217387038#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-GroupCoordinator">here</a>.
-            Functionally speaking, it implements all the same APIs. However, the group coordinator runs in its own dedicated threads.
+            Functionally speaking, it implements all the same APIs. There are reasonable defaults, but the behavior of the new
-            The number of threads can be defined with the <code>group.coordinator.threads</code> configuration. It defaults to 1 and
+            group coordinator can be tuned by setting the configurations with prefix <code>group.coordinator</code>.
-            must certainly be adjusted for production deployments.
         </li>
         <li>
             The Next Generation of the Consumer Rebalance Protocol (<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol">KIP-848</a>)
             is now Generally Available (GA) in Apache Kafka 4.0. The protocol is automatically enabled on the server when the upgrade to 4.0 is finalized.
-            Note that once the new protocol is used by consumer groups, care must be taken to downgrade to a previous version. Downgrading
+            Note that once the new protocol is used by consumer groups, the cluster can only downgrade to version 3.4.1 or newer.
-            to version 3.4.1 or above is safe. Downgrading to an earlier version requires to ensure that all the records in the
-            <code>__consumer_offsets</code> are removed because those versions cannot handle them. This can be achieved by deleting all the
-            consumer groups using the new protocol, disabling the new protocol and letting compaction remove the tombstones.
             Check <a href="/{{version}}/documentation.html#consumer_rebalance_protocol">here</a> for details.
         </li>
         <li>

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

@@ -92,7 +92,7 @@ public class GroupCoordinatorConfig {
 
     public static final String GROUP_COORDINATOR_NUM_THREADS_CONFIG = "group.coordinator.threads";
     public static final String GROUP_COORDINATOR_NUM_THREADS_DOC = "The number of threads used by the group coordinator.";
-    public static final int GROUP_COORDINATOR_NUM_THREADS_DEFAULT = 1;
+    public static final int GROUP_COORDINATOR_NUM_THREADS_DEFAULT = 4;
 
     /** Consumer group configs */
     public static final String CONSUMER_GROUP_SESSION_TIMEOUT_MS_CONFIG = "group.consumer.session.timeout.ms";
@@ -229,7 +229,7 @@ public class GroupCoordinatorConfig {
     public static final ConfigDef NEW_GROUP_CONFIG_DEF = new ConfigDef()
             .define(GROUP_COORDINATOR_REBALANCE_PROTOCOLS_CONFIG, LIST, GROUP_COORDINATOR_REBALANCE_PROTOCOLS_DEFAULT,
                     ConfigDef.ValidList.in(Group.GroupType.documentValidValues()), MEDIUM, GROUP_COORDINATOR_REBALANCE_PROTOCOLS_DOC)
-            .define(GROUP_COORDINATOR_NUM_THREADS_CONFIG, INT, GROUP_COORDINATOR_NUM_THREADS_DEFAULT, atLeast(1), MEDIUM, GROUP_COORDINATOR_NUM_THREADS_DOC)
+            .define(GROUP_COORDINATOR_NUM_THREADS_CONFIG, INT, GROUP_COORDINATOR_NUM_THREADS_DEFAULT, atLeast(1), HIGH, GROUP_COORDINATOR_NUM_THREADS_DOC)
             .define(GROUP_COORDINATOR_APPEND_LINGER_MS_CONFIG, INT, GROUP_COORDINATOR_APPEND_LINGER_MS_DEFAULT, atLeast(0), MEDIUM, GROUP_COORDINATOR_APPEND_LINGER_MS_DOC)
             // Internal configuration used by integration and system tests.
             .defineInternal(NEW_GROUP_COORDINATOR_ENABLE_CONFIG, BOOLEAN, NEW_GROUP_COORDINATOR_ENABLE_DEFAULT, null, MEDIUM, NEW_GROUP_COORDINATOR_ENABLE_DOC);