Add latest changes from gitlab-org/gitlab@master

.gitlab/ci/test-on-gdk/main.gitlab-ci.yml

@@ -65,6 +65,8 @@ workflow:
     GL_DOCKER_DIND_OPTIONS: "--registry-mirror=$GL_DOCKER_REGISTRY_MIRROR_ENDPOINT"
     QA_DOCKER_NETWORK: host
     QA_GITLAB_URL: http://gdk.test:3000
+    QA_RUN_IN_PARALLEL: "true"
+    QA_PARALLEL_PROCESSES: 4
     RSPEC_LAST_RUN_RESULTS_FILE: "$CI_PROJECT_DIR/qa/tmp/examples.txt"
     GIT_STRATEGY: "none"
   after_script:

.gitlab/issue_templates/AI Model Rollout.md

@@ -1,13 +1,10 @@
-<!-- Title suggestion: [Model] Rollout Plan -->
+<!-- Title suggestion: [AI Model] Rollout Plan -->
 
 <!-- Do not remove this confidential declaration and make sure to mark this issue as confidential! -->
 
 _The issue is marked confidential, as we'll share SAFE metrics in the comments._
 
 ## Overview
-
-_Briefly describe the new model. Mention why you're introducing it._
-
 <!-- Add relevant links. Example below: -->
 
 | Resource | Links |
@@ -17,27 +14,52 @@ _Briefly describe the new model. Mention why you're introducing it._
 | Feature Flag Rollout Issue | <!-- Required. --> |
 | Status updates | <!-- Optional, but recommended. --> |
 
-### Rollout success criteria
+### Checklist 
+- [ ] Update the following `yml` files to support customer personal model selection:
+  - [ ] Add a new configurable to [unit_primitives.yml](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/ai_gateway/model_selection/unit_primitives.yml)
+  - [ ] Add the new feature model to [model.yml](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/ai_gateway/model_selection/models.yml)
+- [ ] *Optional*: Pin the prompt version to rollout in the GitLab Rails client by overriding the prompt version [behind a feature flag](https://gitlab.com/gitlab-org/gitlab/-/blob/879066eaafb9a989ad8ca26e97602db7f40e47ca/ee/lib/gitlab/llm/chain/tools/identifier.rb#L152-155).
+- [ ] *Optional*: Configure the prompt registry with a non-stable prompt (i.e., `alpha`, `dev`, `rc`) once you fully roll out to internal/external users. [Prompt versioning conventions](https://docs.gitlab.com/development/ai_features/actions/#appendix-a-prompt-versioning-conventions)
+- [ ] Once the feature is fully rolled out, add a *stable* prompt version— non `dev`, `alpha` or `beta` prompt to the prompt registry.
 
-_Add a list of success criteria._
+### Evaluation Metrics
+
+Use this base table to share evaluation metrics with team members. Run metrics from either the [prompt-library](https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library) or [evaluation-runner](https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/evaluation-runner).
+
+## Evaluation Template
+**Langsmith Experiment**: [View Results](<insert_link>)
+
+| Evaluation Pipeline | Model | Quality Metric | P50 Latency | P99 Latency |
+|-------------------|-------|----------------|-------------|-------------|
+| [pipeline_link] | claude-3-7-sonnet (baseline) | context-qa-accuracy: X | 17.5s | 44s |
+| [pipeline_link] | claude-4-0-sonnet | context-qa-accuracy: X | 22s | 40s |
+| **Change** | | **XX change** | **+X% slower** | **XX% faster** |
+
+### Rollout Success Criteria
+
+1. Verify the prompt registry selects the correct prompt version in production. 
+   1. Filter the [Kibana link](https://log.gprd.gitlab.net/app/r/s/Pu5F3) with `json.jsonPayload.message: Returning prompt from the registry` to view all selected prompt version information. 
+2. Confirm Duo Chat streaming works correctly. Check our [Kibana Dashboard](https://log.gprd.gitlab.net/app/r/s/AxFq1) ReAct Agent streaming page [DC | Sidekiq] to verify the LLM follows ReAct instructions (V2 Duo Chat only - broken streaming)
+
+_Add your specific success criteria._
 
 ### Dashboard References
 
-_This can be the acceptance rate or latency dashboards filtered to the new model. Add as many dashboards as is relevant._
+_Filter acceptance rate or latency dashboards to the new model. Add all relevant dashboards._
 
-## Legal notes
+## Legal Notes
 
 _Add legal notes here_
 
-## Known issues
+## Known Issues
 
-_List the issues identified throughout the evaluation, implementation, and rollout of the model._
+_List issues you identified during evaluation, implementation, and rollout of the model._
 
 ## Rollout
 
 ### Timeline
 
-_Optional: Breifly describe the expected timeline._
+_Optional: Describe your expected timeline._
 
 <!-- Add a detailed timeline similar to the example below: -->
 
@@ -48,18 +70,18 @@ _Optional: Breifly describe the expected timeline._
 | ??? | 50% of all users | |
 | ??? | 100% of all users | |
 
-### Feedback from GitLab team members
+### Feedback from GitLab Team Members
 
 _Add link to the internal feedback issue._
 
 ### Persevere / Continue Criteria
 
-_Add specific criteria that indicates rollout is successful and should continue._
+_Define specific criteria that indicate successful rollout and continuation._
 
 <!-- example criteria: -->
-1. Latency remains within observed p50/90/95 ranges
-2. Success/acceptance rate remains within observed range or improves
-3. No blockers have been identified
+1. Maintain latency within observed p50/90/95 ranges
+2. Keep success/acceptance rate within observed range or improve it
+3. Identify no blockers
 
 <!-- Add supporting details as needed, for example: -->
 _Observed latency from [date] to [date]_
@@ -72,23 +94,31 @@ _Observed success/acceptance rate from [date] to [date]_
 
 ### Pivot / Pause / Rollback Criteria
 
-_Add specific criteria that indicates the rollout should be paused or rolled back._
+_Define specific criteria that require pausing or rolling back the rollout._
 
 <!-- example criteria: -->
-1. Requests are not using the new model as expected
-2. There is an increase or spike in latency for the new model vs the old model
-3. There is a decrease in success/acceptance rate compared to the old model
+1. Requests fail to use the new model as expected
+2. Latency increases or spikes for the new model vs the old model
+3. Success/acceptance rate decreases compared to the old model
 
 ## Mitigation and Rollback Plan
 
-_Describe how you will handle issues if they arise during rollout._
+_Describe how you will handle issues during rollout._
 
 <!-- Example plan description: -->
-We will use a [feature flag](https://docs.gitlab.com/operations/feature_flags/) to control the rollout. If we need to pause, pivot, or rollback the model, we will disable the feature flag, especially for external users, to investigate any potential issues.
+We will control the rollout using a [feature flag](https://docs.gitlab.com/operations/feature_flags/). If we need to pause, pivot, or rollback the model, we will disable the feature flag, especially for external users, to investigate potential issues.
+
+## Useful Documentation
+
+- [Prompt Versioning Guidelines](https://docs.gitlab.com/development/ai_features/actions/#appendix-a-prompt-versioning-conventions)
+
+- [How to add a new prompt version](https://docs.gitlab.com/development/ai_features/actions/#1-add-your-action-to-the-cloud-connector-feature-list)
+
+- [Model Selection Blueprint](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/ai_model_selection/)
 
 ## Release Announcement
 
-_Describe where to make announcements when the model is ready for rollout to external users._
+_Describe where you will announce the model when ready for rollout to external users._
 
 /confidential
 /confidential

GITALY_SERVER_VERSION

@@ -1 +1 @@
-a2d87e9fa4447c81916079cce9beb1785dd1d76c
+333befa213dffe9a756658ebd1d85da5d32e8b11

GITLAB_KAS_VERSION

@@ -1 +1 @@
-9f4dcab5a68db7c3fb30f29fa315b80011449cf0
+b9ee5f42724493046e4b49b6386393ebb714088e

app/assets/javascripts/ci/job_details/components/sidebar/sidebar_job_details_container.vue

@@ -86,7 +86,7 @@ export default {
       return this.job?.runner?.admin_path || '';
     },
     hasTestSummaryDetails() {
-      return Object.keys(this.testSummary).length > 0;
+      return Object.keys(this.testSummary).length > 0 && this.job?.test_suite_name;
     },
     testSummaryDescription() {
       let message;
@@ -106,7 +106,7 @@ export default {
     },
     testReportUrlWithJobName() {
       const urlParams = {
-        job_name: this.job.name,
+        job_name: this.job.test_suite_name,
       };
 
       return mergeUrlParams(urlParams, this.pipelineTestReportUrl);

app/models/commit_status.rb

@@ -367,6 +367,10 @@ class CommitStatus < Ci::ApplicationRecord
     'build'
   end
 
+  def test_suite_name
+    nil
+  end
+
   # For AiAction
   def resource_parent
     project

app/serializers/ci/job_entity.rb

@@ -6,6 +6,7 @@ module Ci
 
     expose :id
     expose :name
+    expose :test_suite_name
 
     expose :started?, as: :started
     expose :started_at, if: ->(job) { job.started? }

app/validators/json_schemas/duo_context_exclusion_settings.json

@@ -0,0 +1,14 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "description": "Duo Context Exclusion Settings",
+  "type": "object",
+  "additionalProperties": false,
+  "properties": {
+    "exclusion_rules": {
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    }
+  }
+}

db/migrate/20250528165343_add_duo_context_exclusion_to_project_settings.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+class AddDuoContextExclusionToProjectSettings < Gitlab::Database::Migration[2.3]
+  milestone '18.1'
+
+  def change
+    add_column :project_settings, :duo_context_exclusion_settings, :jsonb, null: false, default: {}
+  end
+end

db/post_migrate/20250602200712_remove_idx_ci_build_needs_on_partition_id_build_id.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class RemoveIdxCiBuildNeedsOnPartitionIdBuildId < Gitlab::Database::Migration[2.3]
+  milestone '18.1'
+
+  disable_ddl_transaction!
+
+  TABLE_NAME = :ci_build_needs
+  INDEX_NAME = :index_ci_build_needs_on_partition_id_build_id
+  COLUMNS = [:partition_id, :build_id]
+
+  def up
+    remove_concurrent_index_by_name TABLE_NAME, name: INDEX_NAME
+  end
+
+  def down
+    add_concurrent_index TABLE_NAME, COLUMNS, name: INDEX_NAME
+  end
+end

db/schema_migrations/20250528165343

@@ -0,0 +1 @@
+d729e72c0c49e5b1fc28186244cfb530935278e7993473877caa57ec5a77ca9b

db/schema_migrations/20250602200712

@@ -0,0 +1 @@
+d065def26c267f279718e31023ba82692d10ee870bae5a435bd88a18fc81b41a

db/structure.sql

@@ -21489,6 +21489,7 @@ CREATE TABLE project_settings (
     auto_duo_code_review_enabled boolean DEFAULT false NOT NULL,
     model_prompt_cache_enabled boolean,
     web_based_commit_signing_enabled boolean DEFAULT false NOT NULL,
+    duo_context_exclusion_settings jsonb DEFAULT '{}'::jsonb NOT NULL,
     CONSTRAINT check_1a30456322 CHECK ((char_length(pages_unique_domain) <= 63)),
     CONSTRAINT check_3a03e7557a CHECK ((char_length(previous_default_branch) <= 4096)),
     CONSTRAINT check_3ca5cbffe6 CHECK ((char_length(issue_branch_template) <= 255)),
@@ -34424,8 +34425,6 @@ CREATE UNIQUE INDEX index_chat_teams_on_namespace_id ON chat_teams USING btree (
 
 CREATE UNIQUE INDEX index_ci_build_needs_on_build_id_and_name ON ci_build_needs USING btree (build_id, name);
 
-CREATE INDEX index_ci_build_needs_on_partition_id_build_id ON ci_build_needs USING btree (partition_id, build_id);
-
 CREATE INDEX index_ci_build_needs_on_project_id ON ci_build_needs USING btree (project_id);
 
 CREATE UNIQUE INDEX index_ci_build_pending_states_on_build_id ON ci_build_pending_states USING btree (build_id);

doc/api/graphql/reference/_index.md

@@ -9703,7 +9703,8 @@ Input type: `ProjectSettingsUpdateInput`
 | Name | Type | Description |
 | ---- | ---- | ----------- |
 | <a id="mutationprojectsettingsupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
-| <a id="mutationprojectsettingsupdateduofeaturesenabled"></a>`duoFeaturesEnabled` | [`Boolean!`](#boolean) | Indicates whether GitLab Duo features are enabled for the project. |
+| <a id="mutationprojectsettingsupdateduocontextexclusionsettings"></a>`duoContextExclusionSettings` | [`DuoContextExclusionSettingsInput`](#duocontextexclusionsettingsinput) | Settings for excluding files from Duo context. |
+| <a id="mutationprojectsettingsupdateduofeaturesenabled"></a>`duoFeaturesEnabled` | [`Boolean`](#boolean) | Indicates whether GitLab Duo features are enabled for the project. |
 | <a id="mutationprojectsettingsupdatefullpath"></a>`fullPath` | [`ID!`](#id) | Full Path of the project the settings belong to. |
 
 #### Fields
@@ -26653,6 +26654,16 @@ Aggregated DORA score counts for projects for the last complete month.
 | <a id="doraperformancescorecountmetricname"></a>`metricName` | [`String!`](#string) | Name of the DORA metric. |
 | <a id="doraperformancescorecountnodataprojectscount"></a>`noDataProjectsCount` | [`Int`](#int) | Number of projects with no data for the metric. |
 
+### `DuoContextExclusionSettings`
+
+Settings for Duo context exclusion rules.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="duocontextexclusionsettingsexclusionrules"></a>`exclusionRules` | [`[String!]`](#string) | List of rules for excluding files from Duo context. |
+
 ### `DuoSettings`
 
 GitLab Duo settings.
@@ -38448,6 +38459,7 @@ Represents the source of a security policy belonging to a project.
 
 | Name | Type | Description |
 | ---- | ---- | ----------- |
+| <a id="projectsettingduocontextexclusionsettings"></a>`duoContextExclusionSettings` | [`DuoContextExclusionSettings`](#duocontextexclusionsettings) | Settings for excluding files from Duo context. |
 | <a id="projectsettingduofeaturesenabled"></a>`duoFeaturesEnabled` | [`Boolean`](#boolean) | Indicates whether GitLab Duo features are enabled for the project. |
 | <a id="projectsettingproject"></a>`project` | [`Project`](#project) | Project the settings belong to. |
 
@@ -50589,6 +50601,16 @@ Filter parameters for projects to be aggregated for DORA metrics.
 | ---- | ---- | ----------- |
 | <a id="doraprojectfilterinputtopic"></a>`topic` | [`[String!]`](#string) | Filter projects by topic. |
 
+### `DuoContextExclusionSettingsInput`
+
+Input for Duo context exclusion settings.
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="duocontextexclusionsettingsinputexclusionrules"></a>`exclusionRules` | [`[String!]!`](#string) | List of rules for excluding files from Duo context. |
+
 ### `EpicFilters`
 
 #### Arguments

locale/gitlab.pot

@@ -38823,7 +38823,7 @@ msgstr ""
 msgid "Microsoft|Client secret"
 msgstr ""
 
-msgid "Microsoft|Enable Microsoft Azure integration for this group"
+msgid "Microsoft|Enable Microsoft Azure integration"
 msgstr ""
 
 msgid "Microsoft|Graph API endpoint"

qa/qa/tools/ci/pipeline_creator.rb

@@ -25,7 +25,7 @@ module QA
         # @return [Hash]
         RUNTIME_COEFFICIENT = {
           test_on_cng: 0.7, # cng supports parallel_tests, so less jobs are needed to retain target runtime
-          test_on_gdk: 1.0,
+          test_on_gdk: 0.8, # gdk supports parallel_tests but is in general slower than other environments
           test_on_omnibus: 1.0,
           test_on_omnibus_nightly: 1.0
         }.freeze

spec/features/projects/jobs/user_browses_job_spec.rb

@@ -48,6 +48,30 @@ RSpec.describe 'User browses a job', :js, feature_category: :continuous_integrat
     end
   end
 
+  context 'with parallel job' do
+    let!(:build_1) do
+      create(:ci_build, :success, :report_results, name: 'rspec 1/2', pipeline: pipeline,
+        options: { parallel: { number: 1, total: 2 } })
+    end
+
+    let!(:build_2) do
+      create(:ci_build, :success, :report_results, name: 'rspec 2/2', pipeline: pipeline,
+        options: { parallel: { number: 2, total: 2 } })
+    end
+
+    let!(:artifact_1_trace) do
+      create(:ci_job_artifact, :trace, job: build_1)
+    end
+
+    it 'renders the correct test report link', :js do
+      visit(project_job_path(project, build_1))
+      wait_for_requests
+
+      href = find_by_testid('job-sidebar-value-link')[:href]
+      expect(href[-5..]).to eq('rspec')
+    end
+  end
+
   context 'with a failed job and live trace' do
     let!(:build) { create(:ci_build, :failed, :trace_live, pipeline: pipeline) }
 

spec/frontend/ci/job_details/components/sidebar/sidebar_job_details_container_spec.js

@@ -162,6 +162,9 @@ describe('Job Sidebar Details Container', () => {
       expect(findTestSummary().exists()).toBe(true);
       expect(findTestSummary().text()).toContain('Test summary');
       expect(findTestSummary().text()).toContain('1');
+      expect(findTestSummary().props('path')).toBe(
+        '/root/test-unit-test-reports/-/pipelines/512/test_report?job_name=test',
+      );
     });
 
     it('does not display the test summary section', async () => {

spec/frontend/ci/jobs_mock_data.js

@@ -949,6 +949,7 @@ export default {
     locked: false,
   },
   name: 'test',
+  test_suite_name: 'test',
   stage: 'build',
   build_path: '/root/ci-mock/-/jobs/4757',
   retry_path: '/root/ci-mock/-/jobs/4757/retry',

spec/requests/api/project_attributes.yml

@@ -201,6 +201,7 @@ project_setting:
     - protect_merge_request_pipelines
     - model_prompt_cache_enabled
     - web_based_commit_signing_enabled
+    - duo_context_exclusion_settings
 
 build_service_desk_setting: # service_desk_setting
   unexposed_attributes:

spec/tooling/lib/tooling/events/track_pipeline_events_spec.rb

@@ -8,10 +8,10 @@ require_relative '../../../../../tooling/lib/tooling/events/track_pipeline_event
 RSpec.describe Tooling::Events::TrackPipelineEvents, feature_category: :tooling do
   include StubENV
 
-  subject(:send_event) { described_class.new(logger: logger).send_event(event_name, **additional_properties) }
+  subject(:send_event) { described_class.new(logger: logger).send_event(event_name, **base_properties) }
 
   let(:event_name) { "e2e_tests_selected_for_execution_gitlab_pipeline" }
-  let(:additional_properties) { { label: 'label', value: 10, property: 'property' } }
+  let(:base_properties) { { label: 'label', value: 10, property: 'property' } }
   let(:access_token) { 'test-admin-token' }
   let(:logger) { instance_double(Logger, info: nil, error: nil) }
   let(:http_client) { instance_double(Net::HTTP, :use_ssl= => true, :request_post => response) }
@@ -33,6 +33,18 @@ RSpec.describe Tooling::Events::TrackPipelineEvents, feature_category: :tooling
 
   describe '#send_event' do
     context 'with API request' do
+      let(:event_properties) { base_properties }
+
+      let(:expected_request_body) do
+        {
+          event: event_name,
+          send_to_snowplow: true,
+          namespace_id: 1,
+          project_id: 2,
+          additional_properties: event_properties
+        }.to_json
+      end
+
       before do
         allow(Net::HTTP).to receive(:new).and_return(http_client)
       end
@@ -45,22 +57,34 @@ RSpec.describe Tooling::Events::TrackPipelineEvents, feature_category: :tooling
       end
 
       context 'when successful' do
-        let(:expected_request_body) do
-          {
-            event: event_name,
-            send_to_snowplow: true,
-            namespace_id: 1,
-            project_id: 2,
-            additional_properties: additional_properties
-          }.to_json
-        end
-
         it 'sends correct event parameters and success message' do
           send_event
 
           expect(http_client).to have_received(:request_post).with(api_path, expected_request_body, headers)
           expect(logger).to have_received(:info).with(
-            "Successfully sent data with properties: #{additional_properties}"
+            "Successfully sent data with properties: #{event_properties}"
+          )
+        end
+      end
+
+      context 'with extra properties' do
+        subject(:send_event) do
+          described_class.new(logger: logger).send_event(
+            event_name,
+            **base_properties,
+            extra_properties: extra_properties
+          )
+        end
+
+        let(:extra_properties) { { custom_key: 'value' } }
+        let(:event_properties) { base_properties.merge(extra_properties) }
+
+        it 'sends event with extra properties' do
+          send_event
+
+          expect(http_client).to have_received(:request_post).with(api_path, expected_request_body, headers)
+          expect(logger).to have_received(:info).with(
+            "Successfully sent data with properties: #{event_properties}"
           )
         end
       end

spec/tooling/lib/tooling/predictive_tests_spec.rb

@@ -55,7 +55,8 @@ RSpec.describe Tooling::PredictiveTests, feature_category: :tooling do
       'FRONTEND_FIXTURES_MAPPING_PATH' => fixtures_mapping.path,
       'RSPEC_MATCHING_JS_FILES_PATH' => matching_js_files.path,
       'RSPEC_TESTS_MAPPING_ENABLED' => "false",
-      'RSPEC_TESTS_MAPPING_PATH' => '/tmp/does-not-exist.out'
+      'RSPEC_TESTS_MAPPING_PATH' => '/tmp/does-not-exist.out',
+      'CI_JOB_ID' => '123'
     )
 
     # We write some data to later on verify that we only append to this file.
@@ -146,7 +147,8 @@ RSpec.describe Tooling::PredictiveTests, feature_category: :tooling do
             "glci_predictive_tests_count",
             label: "test-count",
             value: File.read(matching_tests.path).split(" ").length,
-            property: "described_class"
+            property: "described_class",
+            extra_properties: { ci_job_id: '123' }
           )
         end
       end

tooling/lib/tooling/events/track_pipeline_events.rb

@@ -18,11 +18,12 @@ module Tooling
       # @param label [String] Event attribute
       # @param value [Number] Numeric event attribute
       # @param property [String] Optional event attribute
+      # @param extra_properties [Hash] Additional custom properties
       # @return [Net::HTTPResponse]
-      def send_event(event_name, label:, value: nil, property: nil)
+      def send_event(event_name, label:, value: nil, property: nil, extra_properties: {})
         return log(:error, "Error: Cannot send event '#{event_name}'. Missing project access token.") unless api_token
 
-        properties = { label:, value:, property: }.compact
+        properties = { label:, value:, property:, **extra_properties }.compact
         body = {
           event: event_name,
           send_to_snowplow: true,

tooling/lib/tooling/predictive_tests.rb

@@ -74,7 +74,8 @@ module Tooling
         "glci_predictive_tests_count",
         label: "test-count",
         value: test_count,
-        property: predictive_tests_strategy
+        property: predictive_tests_strategy,
+        extra_properties: { ci_job_id: ENV["CI_JOB_ID"] }
       )
     end
   end