Add latest changes from gitlab-org/gitlab@master

db/docs/project_statistics.yml

@@ -17,3 +17,6 @@ schema_inconsistencies:
 - type: missing_indexes
   object_name: index_project_statistics_on_storage_size_and_project_id
   introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/156005
+- type: missing_indexes
+  object_name: index_project_statistics_on_packages_size_and_project_id
+  introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/156008

db/post_migrate/20240611121818_remove_project_statistics_packages_size_and_project_id_index.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+class RemoveProjectStatisticsPackagesSizeAndProjectIdIndex < Gitlab::Database::Migration[2.2]
+  milestone '17.2'
+
+  INDEX_NAME = 'index_project_statistics_on_packages_size_and_project_id'
+  COLUMNS = %i[packages_size project_id]
+
+  # TODO: Index to be destroyed synchronously in https://gitlab.com/gitlab-org/gitlab/-/issues/466691
+  def up
+    return unless should_run?
+
+    prepare_async_index_removal :project_statistics, COLUMNS, name: INDEX_NAME
+  end
+
+  def down
+    return unless should_run?
+
+    unprepare_async_index :project_statistics, COLUMNS, name: INDEX_NAME
+  end
+
+  def should_run?
+    Gitlab.com_except_jh?
+  end
+end

db/schema_migrations/20240611121818

@@ -0,0 +1 @@
+a79d6464d995015034a5534f63bbed10ab899d6b01d5424b7e00d426b6dedd82

doc/user/application_security/api_fuzzing/troubleshooting.md

@@ -364,17 +364,72 @@ sudo: If sudo is running in a container, you may need to adjust the container co
 
 This issue can be worked around in the following ways:
 
-1. Run the container as the `root` user. This can be done by modifying the CICD configuration:
+- Run the container as the `root` user. It's recommended to test this configuration as it may not work in all cases. This can be done by modifying the CICD configuration and checking the job output to make sure that `whoami` returns `root` and not `gitlab`. If `gitlab` is displayed, use another workaround. Once tested the `before_script` can be removed.
 
    ```yaml
-   api_security:
+   apifuzzer_fuzz:
      image:
        name: $SECURE_ANALYZERS_PREFIX/$FUZZAPI_IMAGE:$FUZZAPI_VERSION$FUZZAPI_IMAGE_SUFFIX
        docker:
          user: root
+    before_script:
+      - whoami
    ```
 
-1. Change the GitLab Runner configuration, disabling the no-new-privileges flag.
+   _Example job console output:_
+
+   ```log
+   Executing "step_script" stage of the job script
+   Using docker image sha256:8b95f188b37d6b342dc740f68557771bb214fe520a5dc78a88c7a9cc6a0f9901 for registry.gitlab.com/security-products/api-security:5 with digest registry.gitlab.com/security-products/api-security@sha256:092909baa2b41db8a7e3584f91b982174772abdfe8ceafc97cf567c3de3179d1 ...
+   $ whoami
+   root
+   $ /peach/analyzer-api-fuzzing
+   17:17:14 [INF] API Security: Gitlab API Security
+   17:17:14 [INF] API Security: -------------------
+   17:17:14 [INF] API Security:
+   17:17:14 [INF] API Security: version: 5.7.0
+   ```
+
+- Wrap the container and add any dependencies at build time. This option has the benefit of running with lower privileges than root which may be a requirement for some customers.
+   1. Create a new `Dockerfile` that wraps the existing image.
+
+      ```yaml
+      ARG SECURE_ANALYZERS_PREFIX
+      ARG FUZZAPI_IMAGE
+      ARG FUZZAPI_VERSION
+      ARG FUZZAPI_IMAGE_SUFFIX
+      FROM $SECURE_ANALYZERS_PREFIX/$FUZZAPI_IMAGE:$FUZZAPI_VERSION$FUZZAPI_IMAGE_SUFFIX
+      USER root
+
+      RUN pip install ...
+      RUN apk add ...
+
+      USER gitlab
+      ```
+
+   1. Build the new image and push it to your local container registry before the API Fuzzing job starts. The image should be removed after the `` job has been completed.
+
+      ```shell
+      TARGET_NAME=apifuzz-$CI_COMMIT_SHA
+      docker build -t $TARGET_IMAGE \
+        --build-arg "SECURE_ANALYZERS_PREFIX=$SECURE_ANALYZERS_PREFIX" \
+        --build-arg "FUZZAPI_IMAGE=$APISEC_IMAGE" \
+        --build-arg "FUZZAPI_VERSION=$APISEC_VERSION" \
+        --build-arg "FUZZAPI_IMAGE_SUFFIX=$APISEC_IMAGE_SUFFIX" \
+        .
+      docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
+      docker push $TARGET_IMAGE
+      ```
+
+   1. Extend the `apifuzzer_fuzz` job and use the new image name.
+
+      ```yaml
+      apifuzzer_fuzz:
+        image: apifuzz-$CI_COMMIT_SHA
+      ```
+
+   1. Remove the temporary container from the registry. See [this documentation page for information on removing container images.](../../packages/container_registry/delete_container_registry_images.md)
+- Change the GitLab Runner configuration, disabling the no-new-privileges flag. This could have security implications and should be discussed with your operations and security teams.
 
 ## `Index was outside the bounds of the array.    at Peach.Web.Runner.Services.RunnerOptions.GetHeaders()`
 

doc/user/application_security/api_security_testing/troubleshooting.md

@@ -340,17 +340,72 @@ sudo: If sudo is running in a container, you may need to adjust the container co
 
 This issue can be worked around in the following ways:
 
-1. Run the container as the `root` user. This can be done by modifying the CICD configuration:
+- Run the container as the `root` user. You should test this configuration as it may not work in all cases. This can be done by modifying the CICD configuration and checking the job output to make sure that `whoami` returns `root` and not `gitlab`. If `gitlab` is displayed, use another workaround. After testing has confirmed the change is successful, the `before_script` can be removed.
 
    ```yaml
    api_security:
      image:
-       name: $SECURE_ANALYZERS_PREFIX/$DAST_API_IMAGE:$DAST_API_VERSION$DAST_API_IMAGE_SUFFIX
+       name: $SECURE_ANALYZERS_PREFIX/$APISEC_IMAGE:$APISEC_VERSION$APISEC_IMAGE_SUFFIX
        docker:
          user: root
+    before_script:
+      - whoami
    ```
 
-1. Change the GitLab Runner configuration, disabling the no-new-privileges flag.
+   _Example job console output:_
+
+   ```log
+   Executing "step_script" stage of the job script
+   Using docker image sha256:8b95f188b37d6b342dc740f68557771bb214fe520a5dc78a88c7a9cc6a0f9901 for registry.gitlab.com/security-products/api-security:5 with digest registry.gitlab.com/security-products/api-security@sha256:092909baa2b41db8a7e3584f91b982174772abdfe8ceafc97cf567c3de3179d1 ...
+   $ whoami
+   root
+   $ /peach/analyzer-api-security
+   17:17:14 [INF] API Security: Gitlab API Security
+   17:17:14 [INF] API Security: -------------------
+   17:17:14 [INF] API Security:
+   17:17:14 [INF] API Security: version: 5.7.0
+   ```
+
+- Wrap the container and add any dependencies at build time. This option has the benefit of running with lower privileges than root which may be a requirement for some customers.
+   1. Create a new `Dockerfile` that wraps the existing image.
+
+      ```yaml
+      ARG SECURE_ANALYZERS_PREFIX
+      ARG APISEC_IMAGE
+      ARG APISEC_VERSION
+      ARG APISEC_IMAGE_SUFFIX
+      FROM $SECURE_ANALYZERS_PREFIX/$APISEC_IMAGE:$APISEC_VERSION$APISEC_IMAGE_SUFFIX
+      USER root
+
+      RUN pip install ...
+      RUN apk add ...
+
+      USER gitlab
+      ```
+
+   1. Build the new image and push it to your local container registry before the API Security Testing job starts. The image should be removed after the `api_security` job has been completed.
+
+      ```shell
+      TARGET_NAME=apisec-$CI_COMMIT_SHA
+      docker build -t $TARGET_IMAGE \
+        --build-arg "SECURE_ANALYZERS_PREFIX=$SECURE_ANALYZERS_PREFIX" \
+        --build-arg "APISEC_IMAGE=$APISEC_IMAGE" \
+        --build-arg "APISEC_VERSION=$APISEC_VERSION" \
+        --build-arg "APISEC_IMAGE_SUFFIX=$APISEC_IMAGE_SUFFIX" \
+        .
+      docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
+      docker push $TARGET_IMAGE
+      ```
+
+   1. Extend the `api_security` job and use the new image name.
+
+      ```yaml
+      api_security:
+        image: apisec-$CI_COMMIT_SHA
+      ```
+
+   1. Remove the temporary container from the registry. See [this documentation page for information on removing container images.](../../packages/container_registry/delete_container_registry_images.md)
+- Change the GitLab Runner configuration, disabling the no-new-privileges flag. This could have security implications and should be discussed with your operations and security teams.
 
 ## `Index was outside the bounds of the array.    at Peach.Web.Runner.Services.RunnerOptions.GetHeaders()`
 

doc/user/project/merge_requests/approvals/rules.md

@@ -33,6 +33,8 @@ enforce a [minimum number of required approvers](settings.md) in the project's s
 Merge requests that target a different project, such as from a fork to the upstream project,
 use the default approval rules from the target (upstream) project, not the source (fork).
 
+Merge request approvals can be configured globally to apply across all (or a subset) projects with [policies](../../../application_security/policies/index.md). [Merge request approval policies](../../../application_security/policies/scan-result-policies.md) also provide additional flexibility with more granular configuration options.
+
 ## Add an approval rule
 
 > - Approval rules for all protected branches introduced in GitLab 15.3.

spec/migrations/20240611121818_remove_project_statistics_packages_size_and_project_id_index_spec.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+require_migration!
+
+RSpec.describe RemoveProjectStatisticsPackagesSizeAndProjectIdIndex, feature_category: :consumables_cost_management do
+  let(:migration) { described_class.new }
+  let(:postgres_async_indexes) { table(:postgres_async_indexes) }
+
+  describe '#up' do
+    subject(:up) { migration.up }
+
+    it 'does nothing when not on gitlab.com' do
+      expect { up }.not_to change { postgres_async_indexes.count }
+    end
+
+    it 'prepares async index removal when on gitlab.com', :saas do
+      expect { up }.to change { postgres_async_indexes.count }.from(0).to(1)
+    end
+  end
+
+  describe '#down' do
+    subject(:down) { migration.down }
+
+    before do
+      postgres_async_indexes.create!(
+        name: 'index_project_statistics_on_packages_size_and_project_id',
+        table_name: 'project_statistics',
+        definition: 'test index'
+      )
+    end
+
+    it 'does nothing when not on gitlab.com' do
+      expect { down }.not_to change { postgres_async_indexes.count }
+    end
+
+    it 'unprepares async index removal when on gitlab.com', :saas do
+      expect { down }.to change { postgres_async_indexes.count }.from(1).to(0)
+    end
+  end
+end