Change the default celery worker_concurrency to 16 (#13612)

This change was unintentional -- apache/airflow#7205

That PR just changed it to work with breeze. Since we had `16` as default in 1.10.x
and to get better performance and keep in line with `dag_concurrency` and
`max_active_runs_per_dag` -- I think `16` makes more sense.

GitOrigin-RevId: 6c458f29c0eeadb1282e524e76fdd379d6436824

UPDATING.md

@@ -52,6 +52,13 @@ assists users migrating to a new version.
 
 ## Master
 
+### Default `[celery] worker_concurrency` is changed to `16`
+
+The default value for `[celery] worker_concurrency` was `16` for Airflow <2.0.0.
+However, it was unintentionally changed to `8` in 2.0.0.
+
+From Airflow 2.0.1, we revert to the old default of `16`.
+
 ## Airflow 2.0.0
 
 ### The experimental REST API is disabled by default

airflow/config_templates/config.yml

@@ -1356,7 +1356,7 @@
       version_added: ~
       type: string
       example: ~
-      default: "8"
+      default: "16"
     - name: worker_autoscale
       description: |
         The maximum and minimum concurrency that will be used when starting workers with the

airflow/config_templates/default_airflow.cfg

@@ -663,7 +663,7 @@ celery_app_name = airflow.executors.celery_executor
 # ``airflow celery worker`` command. This defines the number of task instances that
 # a worker will take, so size up your workers based on the resources on
 # your worker box and the nature of your tasks
-worker_concurrency = 8
+worker_concurrency = 16
 
 # The maximum and minimum concurrency that will be used when starting workers with the
 # ``airflow celery worker`` command (always keep minimum processes, but grow

scripts/in_container/entrypoint_ci.sh

@@ -152,6 +152,9 @@ unset AIRFLOW__CORE__UNIT_TEST_MODE
 mkdir -pv "${AIRFLOW_HOME}/logs/"
 cp -f "${IN_CONTAINER_DIR}/airflow_ci.cfg" "${AIRFLOW_HOME}/unittests.cfg"
 
+# Change the default worker_concurrency for tests
+export AIRFLOW__CELERY__WORKER_CONCURRENCY=8
+
 disable_rbac_if_requested
 
 set +e