root
/
gitlab-ce
mirror of https://gitlab.com/free-releases/gitlab-ce.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

Add latest changes from gitlab-org/gitlab@master

This commit is contained in:
GitLab Bot 2024-11-08 03:16:09 +00:00
parent c7c4e203a0
commit fa3e6cffe7
28 changed files with 533 additions and 357 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
app/assets/javascripts/environments/components/confirm_rollback_modal.vue
View File

@ -149,7 +149,9 @@ export default {
cancelProps: {
text: __('Cancel'),
},
docsPath: helpPagePath('ci/environments/index.md', { anchor: 'retry-or-roll-back-a-deployment' }),
docsPath: helpPagePath('ci/environments/deployments.md', {
anchor: 'retry-or-roll-back-a-deployment',
}),
};
</script>
<template>

9
config/feature_flags/ops/omit_aggregated_db_log_fields.yml Normal file
View File

@ -0,0 +1,9 @@
---
name: omit_aggregated_db_log_fields
feature_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/500768#note_2179280865
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/170751
rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/501156
milestone: '17.6'
group: group::scalability
type: ops
default_enabled: false

34
data/deprecations/17-6-deprecate-chart-nginx-1-3-1-support.yml Normal file
View File

@ -0,0 +1,34 @@
- title: "GitLab chart use of NGINX controller image v1.3.1"
# The milestones for the deprecation announcement, and the removal.
removal_milestone: "18.0"
announcement_milestone: "17.6"
# Change breaking_change to false if needed.
breaking_change: true
# The stage and GitLab username of the person reporting the change,
# and a link to the deprecation issue
reporter: dorrino
stage: systems
issue_url: https://gitlab.com/gitlab-org/charts/gitlab/-/issues/5794
# Use the impact calculator https://gitlab-com.gitlab.io/gl-infra/breaking-change-impact-calculator/?
impact: small
scope: [instance]
resolution_role: [Admin]
manual_task: true
body: | # (required) Don't change this line.
In GitLab 17.6, the GitLab chart updated the default NGINX controller image to v1.11.2. This new version requires new RBAC rules that were added to our Ingress
NGINX-bundled subchart.
This change is being backported to 17.5.1, 17.4.3, and 17.3.6.
Some users prefer to manage RBAC rules themselves by setting the Helm key `nginx-ingress.rbac.create` to `false`. To give time for users who manage their own RBAC rules to
add the new required rules before they adopt the new v1.11.2 version, we've implemented a fallback mechanism to detect `nginx-ingress.rbac.create: false` and force the chart
to keep using NGINX image v1.3.1, which does not need the new RBAC rules.
If you manage your own NGINX RBAC rules, but you also want to take advantage of the new NGINX controller image v1.11.2 immediately:
1. Add the new RBAC rules to your cluster [like we did](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/3901/diffs?commit_id=93a3cbdb5ad83db95e12fa6c2145df0800493d8b).
1. Set `nginx-ingress.controller.image.disableFallback` to true.
We plan to remove this fallback support and support for NGINX controller image v1.3.1 in GitLab 18.0.
You can read more about it in the [charts release page](https://docs.gitlab.com/charts/releases/8_0.html#upgrade-to-86x-851-843-836).

2
db/docs/pm_cve_enrichment.yml
View File

@ -8,5 +8,5 @@ feature_categories:
description: Stores EPSS data (https://www.first.org/epss/).
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/168288
milestone: '17.6'
gitlab_schema: gitlab_sec
gitlab_schema: gitlab_pm
exempt_from_sharding: true # See discussion on keys for pm_ tables https://gitlab.com/gitlab-org/gitlab/-/issues/434988#note_1827421068

2
doc/api/deployments.md
View File

@ -491,7 +491,7 @@ Example responses:
NOTE:
Not all deployments can be associated with merge requests. See
[Track what merge requests were deployed to an environment](../ci/environments/index.md#track-newly-included-merge-requests-per-deployment)
[Track what merge requests were deployed to an environment](../ci/environments/deployments.md#track-newly-included-merge-requests-per-deployment)
for more information.
This API retrieves the list of merge requests shipped with a given deployment:

2
doc/ci/environments/deployment_safety.md
View File

@ -102,7 +102,7 @@ can be prevented in some circumstances.
> - Job retries for rollback deployments checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/410427) in GitLab 16.3.
You might need to quickly roll back to a stable, outdated deployment.
By default, pipeline job retries for [deployment rollback](index.md#deployment-rollback) are enabled.
By default, pipeline job retries for [deployment rollback](deployments.md#deployment-rollback) are enabled.
To disable pipeline retries, clear the **Allow job retries for rollback deployments** checkbox. You should disable pipeline retries in sensitive projects.

168
doc/ci/environments/deployments.md Normal file
View File

@ -0,0 +1,168 @@
---
stage: Deploy
group: Environments
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Deployments
DETAILS:
**Tier:** Free, Premium, Ultimate
**Offering:** GitLab.com, Self-managed, GitLab Dedicated
When you deploy a version of your code to an environment, you create a deployment.
There is usually only one active deployment per environment.
GitLab:
- Provides a full history of deployments to each environment.
- Tracks your deployments, so you always know what is deployed on your
servers.
If you have a deployment service like [Kubernetes](../../user/infrastructure/clusters/index.md)
associated with your project, you can use it to assist with your deployments.
After a deployment is created, you can roll it out to users.
## Configure manual deployments
You can create a job that requires someone to manually start the deployment.
For example:
```yaml
deploy_prod:
stage: deploy
script:
- echo "Deploy to production server"
environment:
name: production
url: https://example.com
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
```
The `when: manual` action:
- Exposes the **Run** (**{play}**) button for the job in the GitLab UI, with the text **Can be manually deployed to &lt;environment&gt;**.
- Means the `deploy_prod` job must be triggered manually.
You can find **Run** (**{play}**) in the pipelines, environments, deployments, and jobs views.
## Track newly included merge requests per deployment
GitLab can track newly included merge requests per deployment.
When a deployment succeeds, the system calculates commit-diffs between the latest deployment and the previous deployment.
You can fetch tracking information with the [Deployment API](../../api/deployments.md#list-of-merge-requests-associated-with-a-deployment)
or view it at a post-merge pipeline in [merge request pages](../../user/project/merge_requests/index.md).
To enable tracking configure your environment so either:
- The [environment name](../yaml/index.md#environmentname) doesn't use folders with `/` (long-lived or top-level environments).
- The [environment tier](index.md#deployment-tier-of-environments) is either `production` or `staging`.
Here are some example configurations using the [`environment` keyword](../yaml/index.md#environment) in `.gitlab-ci.yml`:
```yaml
# Trackable
environment: production
environment: production/aws
environment: development
# Non Trackable
environment: review/$CI_COMMIT_REF_SLUG
environment: testing/aws
```
Configuration changes apply only to new deployments. Existing deployment records do not have merge requests linked or unlinked from them.
## Check out deployments locally
A reference in the Git repository is saved for each deployment, so
knowing the state of your current environments is only a `git fetch` away.
In your Git configuration, append the `[remote "<your-remote>"]` block with an extra
fetch line:
```plaintext
fetch = +refs/environments/*:refs/remotes/origin/environments/*
```
## Archive old deployments
When a new deployment happens in your project,
GitLab creates [a special Git-ref to the deployment](#check-out-deployments-locally).
Since these Git-refs are populated from the remote GitLab repository,
you could find that some Git operations, such as `git-fetch` and `git-pull`,
become slower as the number of deployments in your project increases.
To maintain the efficiency of your Git operations, GitLab keeps
only recent deployment refs (up to 50,000) and deletes the rest of the old deployment refs.
Archived deployments are still available, in the UI or by using the API, for auditing purposes.
Also, you can still fetch the deployed commit from the repository
with specifying the commit SHA (for example, `git checkout <deployment-sha>`), even after archive.
NOTE:
GitLab preserves all commits as [`keep-around` refs](../../user/project/repository/repository_size.md#methods-to-reduce-repository-size)
so that deployed commits are not garbage collected, even if it's not referenced by the deployment refs.
## Deployment rollback
When you roll back a deployment on a specific commit,
a _new_ deployment is created. This deployment has its own unique job ID.
It points to the commit you're rolling back to.
For the rollback to succeed, the deployment process must be defined in
the job's `script`.
Only the [deployment jobs](../jobs/index.md#deployment-jobs) are run.
In cases where a previous job generates artifacts that must be regenerated
on deploy, you must manually run the necessary jobs from the pipelines page.
For example, if you use Terraform and your `plan` and `apply` commands are separated
into multiple jobs, you must manually run the jobs to deploy or roll back.
### Retry or roll back a deployment
If there is a problem with a deployment, you can retry it or roll it back.
To retry or roll back a deployment:
1. On the left sidebar, select **Search or go to** and find your project.
1. Select **Operate > Environments**.
1. Select the environment.
1. To the right of the deployment name:
- To retry a deployment, select **Re-deploy to environment**.
- To roll back to a deployment, next to a previously successful deployment, select **Rollback environment**.
NOTE:
If you have [prevented outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) in your project,
the rollback buttons might be hidden or disabled.
In this case, see [job retries for rollback deployments](deployment_safety.md#job-retries-for-rollback-deployments).
## Related topics
- [Environments](index.md)
- [Downstream pipelines for deployments](../pipelines/downstream_pipelines.md#downstream-pipelines-for-deployments)
- [Deploy to multiple environments with GitLab CI/CD (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/)
- [Review apps](../review_apps/index.md)
- [Track deployments of an external deployment tool](external_deployment_tools.md)
## Troubleshooting
When you work with deployments, you might encounter the following issues.
### Deployment refs are not found
GitLab [deletes old deployment refs](#archive-old-deployments)
to keep your Git repository performant.
If you have to restore archived Git-refs, ask an administrator of your self-managed GitLab instance
to execute the following command on Rails console:
```ruby
Project.find_by_full_path(<your-project-full-path>).deployments.where(archived: true).each(&:create_ref)
```
GitLab might drop this support in the future for the performance concern.
You can open an issue in [GitLab Issue Tracker](https://gitlab.com/gitlab-org/gitlab/-/issues/new)
to discuss the behavior of this feature.

4
doc/ci/environments/external_deployment_tools.md
View File

@ -18,11 +18,11 @@ For example, the following features are available by setting up tracking:
- [Filter merge requests by environment or deployment date](../../user/project/merge_requests/index.md#by-environment-or-deployment-date).
- [DevOps Research and Assessment (DORA) metrics](../../user/analytics/dora_metrics.md).
- [View environments and deployments](index.md#view-environments-and-deployments).
- [Track newly included merge requests per deployment](index.md#track-newly-included-merge-requests-per-deployment).
- [Track newly included merge requests per deployment](deployments.md#track-newly-included-merge-requests-per-deployment).
NOTE:
Some of the features are not available because GitLab can't authorize and leverage those external deployments, including
[Protected Environments](protected_environments.md), [Deployment Approvals](deployment_approvals.md), [Deployment safety](deployment_safety.md), and [Deployment rollback](index.md#deployment-rollback).
[Protected Environments](protected_environments.md), [Deployment Approvals](deployment_approvals.md), [Deployment safety](deployment_safety.md), and [Deployment rollback](deployments.md#deployment-rollback).
## How to set up deployment tracking

2
doc/ci/environments/incremental_rollouts.md
View File

@ -68,7 +68,7 @@ rollout 10%:
After the jobs are built, select **Run** (**{play}**) next to the job's name
to release each stage of pods. You can also rollback by running a lower percentage job. Once 100%
is reached, you cannot roll back using this method. To roll back a deployment, see [retry or roll back a deployment](../../ci/environments/index.md#retry-or-roll-back-a-deployment).
is reached, you cannot roll back using this method. To roll back a deployment, see [retry or roll back a deployment](../../ci/environments/deployments.md#retry-or-roll-back-a-deployment).
A [deployable application](https://gitlab.com/gl-release/incremental-rollout-example) is
available, demonstrating manually triggered incremental rollouts.

244
doc/ci/environments/index.md
View File

@ -4,7 +4,7 @@ group: Environments
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Environments and deployments
# Environments
DETAILS:
**Tier:** Free, Premium, Ultimate
@ -18,18 +18,6 @@ Environments connect GitLab to your infrastructure. An environment:
In addition, access to an environment can be controlled.
Each time [GitLab CI/CD](../index.md) deploys a version of code to an environment,
a deployment is created.
GitLab:
- Provides a full history of deployments to each environment.
- Tracks your deployments, so you always know what is deployed on your
servers.
If you have a deployment service like [Kubernetes](../../user/infrastructure/clusters/index.md)
associated with your project, you can use it to assist with your deployments.
## View environments and deployments
Prerequisites:
@ -82,139 +70,7 @@ For example:
With GitLab [Route Maps](../review_apps/index.md#route-maps), you can go directly
from source files to public pages in the environment set for review apps.
## Working with deployments
When you deploy a version of your code to an environment, you create a deployment.
There is usually only one active deployment per environment.
After a deployment is created, you can roll it out to users.
### Configure manual deployments
You can create a job that requires someone to manually start the deployment.
For example:
```yaml
deploy_prod:
stage: deploy
script:
- echo "Deploy to production server"
environment:
name: production
url: https://example.com
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
```
The `when: manual` action:
- Exposes the **Run** (**{play}**) button for the job in the GitLab UI, with the text **Can be manually deployed to &lt;environment&gt;**.
- Means the `deploy_prod` job must be triggered manually.
You can find **Run** (**{play}**) in the pipelines, environments, deployments, and jobs views.
### Track newly included merge requests per deployment
GitLab can track newly included merge requests per deployment.
When a deployment succeeds, the system calculates commit-diffs between the latest deployment and the previous deployment.
You can fetch tracking information with the [Deployment API](../../api/deployments.md#list-of-merge-requests-associated-with-a-deployment)
or view it at a post-merge pipeline in [merge request pages](../../user/project/merge_requests/index.md).
To enable tracking configure your environment so either:
- The [environment name](../yaml/index.md#environmentname) doesn't use folders with `/` (long-lived or top-level environments).
- The [environment tier](#deployment-tier-of-environments) is either `production` or `staging`.
Here are some example configurations using the [`environment` keyword](../yaml/index.md#environment) in `.gitlab-ci.yml`:
```yaml
# Trackable
environment: production
environment: production/aws
environment: development
# Non Trackable
environment: review/$CI_COMMIT_REF_SLUG
environment: testing/aws
```
Configuration changes apply only to new deployments. Existing deployment records do not have merge requests linked or unlinked from them.
### Check out deployments locally
A reference in the Git repository is saved for each deployment, so
knowing the state of your current environments is only a `git fetch` away.
In your Git configuration, append the `[remote "<your-remote>"]` block with an extra
fetch line:
```plaintext
fetch = +refs/environments/*:refs/remotes/origin/environments/*
```
### Archive old deployments
When a new deployment happens in your project,
GitLab creates [a special Git-ref to the deployment](#check-out-deployments-locally).
Since these Git-refs are populated from the remote GitLab repository,
you could find that some Git operations, such as `git-fetch` and `git-pull`,
become slower as the number of deployments in your project increases.
To maintain the efficiency of your Git operations, GitLab keeps
only recent deployment refs (up to 50,000) and deletes the rest of the old deployment refs.
Archived deployments are still available, in the UI or by using the API, for auditing purposes.
Also, you can still fetch the deployed commit from the repository
with specifying the commit SHA (for example, `git checkout <deployment-sha>`), even after archive.
NOTE:
GitLab preserves all commits as [`keep-around` refs](../../user/project/repository/repository_size.md#methods-to-reduce-repository-size)
so that deployed commits are not garbage collected, even if it's not referenced by the deployment refs.
### Deployment rollback
When you roll back a deployment on a specific commit,
a _new_ deployment is created. This deployment has its own unique job ID.
It points to the commit you're rolling back to.
For the rollback to succeed, the deployment process must be defined in
the job's `script`.
Only the [deployment jobs](../jobs/index.md#deployment-jobs) are run.
In cases where a previous job generates artifacts that must be regenerated
on deploy, you must manually run the necessary jobs from the pipelines page.
For example, if you use Terraform and your `plan` and `apply` commands are separated
into multiple jobs, you must manually run the jobs to deploy or roll back.
#### Retry or roll back a deployment
If there is a problem with a deployment, you can retry it or roll it back.
To retry or roll back a deployment:
1. On the left sidebar, select **Search or go to** and find your project.
1. Select **Operate > Environments**.
1. Select the environment.
1. To the right of the deployment name:
- To retry a deployment, select **Re-deploy to environment**.
- To roll back to a deployment, next to a previously successful deployment, select **Rollback environment**.
NOTE:
If you have [prevented outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) in your project,
the rollback buttons might be hidden or disabled.
In this case, see [job retries for rollback deployments](deployment_safety.md#job-retries-for-rollback-deployments).
## Working with environments
Environments describe where code is deployed.
Each environment has one of three states, depending on whether its [stop job](../../ci/yaml/index.md#environmenton_stop) has run:
- `available`: The environment exists. There might be a deployment.
- `stopping`: The _on stop job_ has started. This state does not apply when there is no on stop job defined.
- `stopped`: Either the _on stop job_ has run, or a user manually stopped the job.
### Types of environments
## Types of environments
An environment is either static or dynamic.
@ -230,11 +86,17 @@ Dynamic environments:
- Have dynamic names, usually based on the value of a CI/CD variable.
- Are a feature of [review apps](../review_apps/index.md).
### Create a static environment
An environment has one of three states, depending on whether its [stop job](../../ci/yaml/index.md#environmenton_stop) has run:
- `available`: The environment exists. There might be a deployment.
- `stopping`: The _on stop job_ has started. This state does not apply when there is no on stop job defined.
- `stopped`: Either the _on stop job_ has run, or a user manually stopped the job.
## Create a static environment
You can create a static environment in the UI or in your `.gitlab-ci.yml` file.
#### In the UI
### In the UI
Prerequisites:
@ -248,7 +110,7 @@ To create a static environment in the UI:
1. Complete the fields.
1. Select **Save**.
#### In your `.gitlab-ci.yml` file
### In your `.gitlab-ci.yml` file
Prerequisites:
@ -276,7 +138,7 @@ deploy_staging:
url: https://staging.example.com
```
### Create a dynamic environment
## Create a dynamic environment
To create a dynamic environment, you use [CI/CD variables](#cicd-variables) that are
unique to each pipeline.
@ -314,7 +176,7 @@ deploy_review_app:
- if: $CI_COMMIT_BRANCH
```
#### Set a dynamic environment URL
### Set a dynamic environment URL
Some external hosting platforms generate a random URL for each deployment, for example:
`https://94dd65b.amazonaws.com/qa-lambda-1234567`. That makes it difficult to reference the URL in
@ -387,7 +249,7 @@ For Windows runners, you should use the PowerShell `Add-Content` command to writ
Add-Content -Path deploy.env -Value "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL"
```
### Deployment tier of environments
## Deployment tier of environments
Sometimes, instead of using an [industry standard](https://en.wikipedia.org/wiki/Deployment_environment)
environment name, like `production`, you might want to use a code name, like `customer-portal`.
@ -411,7 +273,7 @@ By default, GitLab assumes a tier based on [the environment name](../yaml/index.
You cannot set an environment tier using the UI.
Instead, you can use the [`deployment_tier` keyword](../yaml/index.md#environmentdeployment_tier) to specify a tier.
#### Rename an environment
### Rename an environment
> - Renaming an environment by using the API was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/338897) in GitLab 15.9.
> - Renaming an environment with the API [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/338897) in GitLab 16.0.
@ -424,13 +286,13 @@ To achieve the same result as renaming an environment:
1. [Delete the existing environment](#delete-an-environment).
1. [Create a new environment](#create-a-static-environment) with the desired name.
### CI/CD variables
## CI/CD variables
To customize your environments and deployments, you can use any of the
[predefined CI/CD variables](../../ci/variables/predefined_variables.md),
and define custom CI/CD variables.
#### Limit the environment scope of a CI/CD variable
### Limit the environment scope of a CI/CD variable
By default, all [CI/CD variables](../variables/index.md) are available to all jobs in a pipeline.
If a test tool in a job becomes compromised, the tool could attempt to retrieve all
@ -469,7 +331,7 @@ You should not use environment-scoped variables with [`rules`](../yaml/index.md#
or [`include`](../yaml/index.md#include). The variables might not be defined when
GitLab validates the pipeline configuration at pipeline creation.
### Search environments
## Search environments
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10754) in GitLab 15.5.
> - [Searching environments within a folder](https://gitlab.com/gitlab-org/gitlab/-/issues/373850) was introduced in GitLab 15.7 with [Feature flag `enable_environments_search_within_folder`](https://gitlab.com/gitlab-org/gitlab/-/issues/382108). Enabled by default.
@ -487,7 +349,7 @@ To search environments by name:
- For example when the name is `review/test-app`, search term `test` matches `review/test-app`.
- Also searching with the folder name prefixed like `review/test` matches `review/test-app`.
### Group similar environments
## Group similar environments
You can group environments into collapsible sections in the UI.
@ -508,12 +370,12 @@ deploy_review:
name: review/$CI_COMMIT_REF_SLUG
```
### Stopping an environment
## Stopping an environment
Stopping an environment means its deployments are not accessible on the target server. You must stop
an environment before it can be deleted.
#### Stop an environment by using the UI
### Stop an environment by using the UI
NOTE:
To trigger an `on_stop` action and manually stop an environment from the
@ -527,7 +389,7 @@ To stop an environment in the GitLab UI:
1. Next to the environment you want to stop, select **Stop**.
1. On the confirmation dialog, select **Stop environment**.
#### Stop an environment when a branch is deleted
### Stop an environment when a branch is deleted
You can configure environments to stop when a branch is deleted.
@ -565,7 +427,7 @@ stop_review:
when: manual
```
#### Stop an environment when a merge request is merged or closed
### Stop an environment when a merge request is merged or closed
When you use the [merge request pipelines](../pipelines/merge_request_pipelines.md) configuration,
the `stop` trigger is automatically enabled.
@ -596,7 +458,7 @@ stop_review:
when: manual
```
#### Stop an environment after a certain time period
### Stop an environment after a certain time period
You can set an environment to stop automatically after a certain time period.
@ -636,7 +498,7 @@ stop_review_app:
when: manual
```
##### View an environment's scheduled stop date and time
#### View an environment's scheduled stop date and time
When a environment has been [scheduled to stop after a specified time period](#stop-an-environment-after-a-certain-time-period),
you can view its expiration date and time.
@ -649,7 +511,7 @@ To view an environment's expiration date and time:
The expiration date and time is displayed in the upper-left corner, next to the environment's name.
##### Override an environment's scheduled stop date and time
#### Override an environment's scheduled stop date and time
When a environment has been [scheduled to stop after a specified time period](#stop-an-environment-after-a-certain-time-period),
you can override its expiration.
@ -669,7 +531,7 @@ To override an environment's expiration in the `.gitlab-ci.yml`:
The `auto_stop_in` setting is overridden and the environment remains active until it's stopped
manually.
#### Clean up stale environments
### Clean up stale environments
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108616) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `stop_stale_environments`. Disabled by default.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112098) in GitLab 15.10. Feature flag `stop_stale_environments` removed.
@ -691,7 +553,7 @@ To clean up stale environments:
Active environments that haven't been updated after the specified date are stopped.
Protected environments are ignored and not stopped.
#### Run a pipeline job when environment is stopped
### Run a pipeline job when environment is stopped
> - Feature flag `environment_stop_actions_include_all_finished_deployments` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/435128) in GitLab 16.9. Disabled by default.
> - Feature flag `environment_stop_actions_include_all_finished_deployments` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/150932) in GitLab 17.0.
@ -743,7 +605,7 @@ stop_review_app:
action: stop
```
#### Multiple stop actions for an environment
### Multiple stop actions for an environment
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/358911) in GitLab 15.0. [Feature flag `environment_multiple_stop_actions`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86685) removed.
@ -795,7 +657,7 @@ teardown-cloud-b:
when: manual
```
#### Stop an environment without running the `on_stop` action
### Stop an environment without running the `on_stop` action
There may be times when you want to stop an environment without running the defined
[`on_stop`](../yaml/index.md#environmenton_stop) action. For example, you want to delete many
@ -805,7 +667,7 @@ To stop an environment without running the defined `on_stop` action, execute the
[Stop an environment API](../../api/environments.md#stop-an-environment) with the parameter
`force=true`.
#### Delete an environment
### Delete an environment
Delete an environment when you want to remove it and all its deployments.
@ -822,7 +684,7 @@ To delete an environment:
1. Next to the environment you want to delete, select **Delete environment**.
1. On the confirmation dialog, select **Delete environment**.
### Access an environment for preparation or verification purposes
## Access an environment for preparation or verification purposes
You can define a job that accesses an environment for various purposes, such as verification or preparation. This
effectively bypasses deployment creation, so that you can adjust your CD workflow more accurately.
@ -843,7 +705,7 @@ build:
This gives you access to environment-scoped variables, and can be used to protect builds from unauthorized access. Also,
it's effective to avoid the [prevent outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) feature.
### Environment incident management
## Environment incident management
Production environments can go down unexpectedly, including for reasons outside
of your control. For example, issues with external dependencies, infrastructure,
@ -858,7 +720,7 @@ or human error can cause major issues with an environment. Things like:
You can use [incident management](../../operations/incident_management/index.md)
to get alerts when there are critical issues that need immediate attention.
#### View the latest alerts for environments
### View the latest alerts for environments
DETAILS:
**Tier:** Ultimate
@ -873,10 +735,10 @@ severity is shown, so you can identify which environments need immediate attenti
When the issue that triggered the alert is resolved, it is removed and is no
longer visible on the environments page.
If the alert requires a [rollback](#retry-or-roll-back-a-deployment), you can select the
If the alert requires a [rollback](deployments.md#retry-or-roll-back-a-deployment), you can select the
deployment tab from the environment page and select which deployment to roll back to.
#### Auto Rollback
### Auto Rollback
DETAILS:
**Tier:** Ultimate
@ -908,27 +770,27 @@ GitLab Auto Rollback is turned off by default. To turn it on:
1. Select the checkbox for **Enable automatic rollbacks**.
1. Select **Save changes**.
### Environment permissions
## Environment permissions
Depending on your role, you can interact with environments in public
and private projects.
#### View environments
### View environments
- In public projects, anyone can view a list of environments, including non-members.
- In private projects, you must have at least the Reporter role to view a list of environments.
#### Create and update environments
### Create and update environments
- You must have at least the Developer role to create a new environment, or update an existing unprotected environment.
- If an existing environment is protected and you don't have access to it, you cannot update the environment.
#### Stop and delete environments
### Stop and delete environments
- You must have at least the Developer role to stop or delete an unprotected environment.
- If an environment is protected and you don't have access to it, you cannot stop or delete the environment.
#### Run deployment jobs in protected environments
### Run deployment jobs in protected environments
If you can push or merge to the protected branch:
@ -983,14 +845,10 @@ session and even a multiplexer like `screen` or `tmux`.
## Related topics
- [Dashboard for Kubernetes](kubernetes_dashboard.md)
- [Downstream pipelines for deployments](../pipelines/downstream_pipelines.md#downstream-pipelines-for-deployments)
- [Deploy to multiple environments with GitLab CI/CD (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/)
- [Review apps](../review_apps/index.md)
- [Deployments](deployments.md)
- [Protected environments](protected_environments.md)
- [Environments Dashboard](../environments/environments_dashboard.md)
- [Environments Dashboard](environments_dashboard.md)
- [Deployment safety](deployment_safety.md#restrict-write-access-to-a-critical-environment)
- [Track deployments of an external deployment tool](external_deployment_tools.md)
- [Configure Kubernetes deployments (deprecated)](configure_kubernetes_deployments.md)
## Troubleshooting
@ -1119,19 +977,3 @@ To fix this, use one of the following solutions:
script: deploy review app
environment: review/$CI_COMMIT_REF_SLUG
```
### Deployment refs are not found
GitLab [deletes old deployment refs](#archive-old-deployments)
to keep your Git repository performant.
If you have to restore archived Git-refs, ask an administrator of your self-managed GitLab instance
to execute the following command on Rails console:
```ruby
Project.find_by_full_path(<your-project-full-path>).deployments.where(archived: true).each(&:create_ref)
```
GitLab might drop this support in the future for the performance concern.
You can open an issue in [GitLab Issue Tracker](https://gitlab.com/gitlab-org/gitlab/-/issues/new)
to discuss the behavior of this feature.

2
doc/ci/jobs/index.md
View File

@ -305,7 +305,7 @@ For example, if you start rolling out new code and:
- Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%.
- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline
and [rolling](../environments/index.md#retry-or-roll-back-a-deployment) back to the last stable version.
and [rolling](../environments/deployments.md#retry-or-roll-back-a-deployment) back to the last stable version.
![A pipeline graph with a delayed job.](img/pipeline_delayed_job_v14_2.png)

2
doc/ci/jobs/job_control.md
View File

@ -55,7 +55,7 @@ the type of manual job can affect the trigger job's status while the pipeline ru
To run a manual job, you must have permission to merge to the assigned branch:
1. Go to the pipeline, job, [environment](../environments/index.md#configure-manual-deployments),
1. Go to the pipeline, job, [environment](../environments/deployments.md#configure-manual-deployments),
or deployment view.
1. Next to the manual job, select **Run** (**{play}**).

2
doc/ci/pipelines/index.md
View File

@ -198,7 +198,7 @@ allow you to require manual interaction before moving forward in the pipeline.
You can do this straight from the pipeline graph. Select **Run** (**{play}**) to execute that particular job.
For example, your pipeline can start automatically, but require a manual action to
[deploy to production](../environments/index.md#configure-manual-deployments).
[deploy to production](../environments/deployments.md#configure-manual-deployments).
In the example below, the `production` stage has a job with a manual action:
![Pipelines example](img/manual_pipeline_v14_2.png)

2
doc/topics/autodevops/cicd_variables.md
View File

@ -305,7 +305,7 @@ You can run the rollout jobs in any order. To scale down, rerun a
lower percentage job.
After you run the `rollout 100%` job, you cannot scale down, and must
[roll back your deployment](../../ci/environments/index.md#retry-or-roll-back-a-deployment).
[roll back your deployment](../../ci/environments/deployments.md#retry-or-roll-back-a-deployment).
### Example incremental rollout configurations

2
doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md
View File

@ -211,7 +211,7 @@ you to common environment tasks:
- **Terminal** (**{terminal}**) - Opens a [web terminal](../../../ci/environments/index.md#web-terminals-deprecated)
session inside the container where the application is running
- **Re-deploy to environment** (**{repeat}**) - For more information, see
[Retrying and rolling back](../../../ci/environments/index.md#retry-or-roll-back-a-deployment)
[Retrying and rolling back](../../../ci/environments/deployments.md#retry-or-roll-back-a-deployment)
- **Stop environment** (**{stop}**) - For more information, see
[Stopping an environment](../../../ci/environments/index.md#stopping-an-environment)

2
doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md
View File

@ -219,7 +219,7 @@ you to common environment tasks:
- **Terminal** (**{terminal}**) - Opens a [web terminal](../../../ci/environments/index.md#web-terminals-deprecated)
session inside the container where the application is running
- **Re-deploy to environment** (**{repeat}**) - For more information, see
[Retrying and rolling back](../../../ci/environments/index.md#retry-or-roll-back-a-deployment)
[Retrying and rolling back](../../../ci/environments/deployments.md#retry-or-roll-back-a-deployment)
- **Stop environment** (**{stop}**) - For more information, see
[Stopping an environment](../../../ci/environments/index.md#stopping-an-environment)

32
doc/update/deprecations.md
View File

@ -417,6 +417,38 @@ See also how to [prevent your runner registration workflow from breaking](https:
</div>
<div class="deprecation breaking-change" data-milestone="18.0">
### GitLab chart use of NGINX controller image v1.3.1
<div class="deprecation-notes">
- Announced in GitLab <span class="milestone">17.6</span>
- Removal in GitLab <span class="milestone">18.0</span> ([breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change))
- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/5794).
</div>
In GitLab 17.6, the GitLab chart updated the default NGINX controller image to v1.11.2. This new version requires new RBAC rules that were added to our Ingress
NGINX-bundled subchart.
This change is being backported to 17.5.1, 17.4.3, and 17.3.6.
Some users prefer to manage RBAC rules themselves by setting the Helm key `nginx-ingress.rbac.create` to `false`. To give time for users who manage their own RBAC rules to
add the new required rules before they adopt the new v1.11.2 version, we've implemented a fallback mechanism to detect `nginx-ingress.rbac.create: false` and force the chart
to keep using NGINX image v1.3.1, which does not need the new RBAC rules.
If you manage your own NGINX RBAC rules, but you also want to take advantage of the new NGINX controller image v1.11.2 immediately:
1. Add the new RBAC rules to your cluster [like we did](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/3901/diffs?commit_id=93a3cbdb5ad83db95e12fa6c2145df0800493d8b).
1. Set `nginx-ingress.controller.image.disableFallback` to true.
We plan to remove this fallback support and support for NGINX controller image v1.3.1 in GitLab 18.0.
You can read more about it in the [charts release page](https://docs.gitlab.com/charts/releases/8_0.html#upgrade-to-86x-851-843-836).
</div>
<div class="deprecation " data-milestone="18.0">
### Group vulnerability report by OWASP top 10 2017 is deprecated

42
lib/gitlab/metrics/subscribers/active_record.rb
View File

@ -76,19 +76,27 @@ module Gitlab
return {} unless Gitlab::SafeRequestStore.active?
{}.tap do |payload|
db_counter_keys.each do |key|
payload[key] = Gitlab::SafeRequestStore[key].to_i
end
if Feature.disabled?(:omit_aggregated_db_log_fields, :current_request, type: :ops)
db_counter_keys.each do |key|
payload[key] = Gitlab::SafeRequestStore[key].to_i
end
if ::Gitlab::SafeRequestStore.active?
load_balancing_metric_counter_keys.each do |counter|
load_balancing_roles_metric_counter_keys.each do |counter|
payload[counter] = ::Gitlab::SafeRequestStore[counter].to_i
end
load_balancing_metric_duration_keys.each do |duration|
load_balancing_roles_metric_duration_keys.each do |duration|
payload[duration] = ::Gitlab::SafeRequestStore[duration].to_f.round(3)
end
end
load_balancing_metric_counter_keys.each do |counter|
payload[counter] = ::Gitlab::SafeRequestStore[counter].to_i
end
load_balancing_metric_duration_keys.each do |duration|
payload[duration] = ::Gitlab::SafeRequestStore[duration].to_f.round(3)
end
end
end
@ -188,14 +196,34 @@ module Gitlab
end
end
def self.load_balancing_metric_keys(metrics)
def self.load_balancing_roles_metric_counter_keys
strong_memoize(:load_balancing_roles_metric_counter_keys) do
load_balancing_roles_metric_keys(DB_LOAD_BALANCING_COUNTERS)
end
end
def self.load_balancing_roles_metric_duration_keys
strong_memoize(:load_balancing_roles_metric_duration_keys) do
load_balancing_roles_metric_keys(DB_LOAD_BALANCING_DURATIONS)
end
end
def self.load_balancing_roles_metric_keys(metrics)
counters = []
metrics.each do |metric|
DB_LOAD_BALANCING_ROLES.each do |role|
counters << compose_metric_key(metric, role)
end
end
counters
end
def self.load_balancing_metric_keys(metrics)
counters = []
metrics.each do |metric|
::Gitlab::Database.database_base_models.keys.each do |config_name|
counters << compose_metric_key(metric, nil, config_name) # main / ci / geo
end

1
scripts/verify-tff-mapping
View File

@ -627,6 +627,7 @@ tests = [
ee/spec/lib/gitlab/database/desired_sharding_key_spec.rb
spec/db/docs_spec.rb
spec/lib/gitlab/database/dictionary_spec.rb
ee/spec/lib/gitlab/database/inclusion_of_tables_with_gitlab_sec_schema_spec.rb
spec/lib/gitlab/database/no_new_tables_with_gitlab_main_schema_spec.rb
spec/lib/gitlab/database/sharding_key_spec.rb
]

24
spec/initializers/lograge_spec.rb
View File

@ -207,16 +207,16 @@ RSpec.describe 'lograge', type: :request, feature_category: :observability do
context 'with db payload' do
let(:db_load_balancing_logging_keys) do
%w[
db_primary_wal_count
db_replica_wal_count
db_primary_wal_cached_count
db_replica_wal_cached_count
db_replica_count
db_replica_cached_count
db_primary_count
db_primary_cached_count
db_primary_duration_s
db_replica_duration_s
db_main_wal_count
db_main_wal_count
db_main_wal_cached_count
db_main_wal_cached_count
db_main_count
db_main_cached_count
db_main_count
db_main_cached_count
db_main_duration_s
db_main_duration_s
]
end
@ -228,7 +228,7 @@ RSpec.describe 'lograge', type: :request, feature_category: :observability do
it 'includes db counters' do
subscriber.process_action(event)
expect(log_data).to include("db_count" => a_value >= 1, "db_write_count" => 0, "db_cached_count" => 0)
expect(log_data).to include("db_main_count" => a_value >= 1, "db_main_write_count" => 0, "db_main_cached_count" => 0)
end
end
@ -236,7 +236,7 @@ RSpec.describe 'lograge', type: :request, feature_category: :observability do
it 'does not include db counters' do
subscriber.process_action(event)
expect(log_data).not_to include("db_count", "db_write_count", "db_cached_count")
expect(log_data).not_to include("db_main_count", "db_main_write_count", "db_main_cached_count")
end
end

11
spec/lib/gitlab/ci/pipeline/logger_spec.rb
View File

@ -57,16 +57,11 @@ RSpec.describe ::Gitlab::Ci::Pipeline::Logger, feature_category: :continuous_int
logger.instrument_once_with_sql(:expensive_operation, &operation)
end
def expected_data(count:, db_count: nil)
def expected_data
database_name = Ci::ApplicationRecord.connection.pool.db_config.name
total_db_count = count * db_count if db_count
{
"expensive_operation_duration_s" => a_kind_of(Numeric),
"expensive_operation_db_count" => total_db_count || a_kind_of(Numeric),
"expensive_operation_db_primary_count" => a_kind_of(Numeric),
"expensive_operation_db_primary_duration_s" => a_kind_of(Numeric),
"expensive_operation_db_#{database_name}_count" => a_kind_of(Numeric),
"expensive_operation_db_#{database_name}_duration_s" => a_kind_of(Numeric)
}
@ -81,7 +76,7 @@ RSpec.describe ::Gitlab::Ci::Pipeline::Logger, feature_category: :continuous_int
instrument_once_with_sql
expect(logger.observations_hash)
.to match(a_hash_including(expected_data(count: 1, db_count: 1)))
.to match(a_hash_including(expected_data))
end
end
@ -94,7 +89,7 @@ RSpec.describe ::Gitlab::Ci::Pipeline::Logger, feature_category: :continuous_int
instrument_once_with_sql
expect(logger.observations_hash)
.to match(a_hash_including(expected_data(count: 1, db_count: 2)))
.to match(a_hash_including(expected_data))
end
end

19
spec/lib/gitlab/instrumentation_helper_spec.rb
View File

@ -18,12 +18,6 @@ RSpec.describe Gitlab::InstrumentationHelper, :clean_gitlab_redis_repository_cac
described_class.init_instrumentation_data
end
it 'includes DB counts' do
subject
expect(payload).to include(db_count: 0, db_cached_count: 0, db_write_count: 0)
end
context 'when Gitaly calls are made' do
it 'adds Gitaly and Redis data' do
project = create(:project)
@ -182,14 +176,11 @@ RSpec.describe Gitlab::InstrumentationHelper, :clean_gitlab_redis_repository_cac
it 'includes DB counts' do
subject
expect(payload).to include(db_replica_count: 0,
db_replica_cached_count: 0,
db_primary_count: 0,
db_primary_cached_count: 0,
db_primary_wal_count: 0,
db_replica_wal_count: 0,
db_primary_wal_cached_count: 0,
db_replica_wal_cached_count: 0)
expect(payload).to include(db_main_cached_count: 0,
db_main_count: 0,
db_main_write_count: 0,
db_main_wal_count: 0,
db_main_wal_cached_count: 0)
end
context 'when replica caught up search was made' do

128
spec/lib/gitlab/metrics/subscribers/active_record_spec.rb
View File

@ -19,26 +19,18 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it 'has expected keys' do
expect(described_class.load_balancing_metric_counter_keys).to include(
:db_replica_count,
:db_primary_count,
:db_main_count,
:db_main_replica_count,
:db_ci_count,
:db_ci_replica_count,
:db_replica_cached_count,
:db_primary_cached_count,
:db_main_cached_count,
:db_main_replica_cached_count,
:db_ci_cached_count,
:db_ci_replica_cached_count,
:db_replica_wal_count,
:db_primary_wal_count,
:db_main_wal_count,
:db_main_replica_wal_count,
:db_ci_wal_count,
:db_ci_replica_wal_count,
:db_replica_wal_cached_count,
:db_primary_wal_cached_count,
:db_main_wal_cached_count,
:db_main_replica_wal_cached_count,
:db_ci_wal_cached_count,
@ -56,20 +48,12 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it 'has expected keys' do
expect(described_class.load_balancing_metric_counter_keys).to include(
:db_replica_count,
:db_primary_count,
:db_main_count,
:db_main_replica_count,
:db_replica_cached_count,
:db_primary_cached_count,
:db_main_cached_count,
:db_main_replica_cached_count,
:db_replica_wal_count,
:db_primary_wal_count,
:db_main_wal_count,
:db_main_replica_wal_count,
:db_replica_wal_cached_count,
:db_primary_wal_cached_count,
:db_main_wal_cached_count,
:db_main_replica_wal_cached_count,
:db_main_txn_count
@ -92,6 +76,46 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
end
end
describe '.load_balancing_roles_metric_counter_keys' do
context 'multiple databases' do
before do
skip_if_multiple_databases_not_setup(:ci)
end
it 'has expected keys' do
expect(described_class.load_balancing_roles_metric_counter_keys).to include(
:db_replica_count,
:db_primary_count,
:db_replica_cached_count,
:db_primary_cached_count,
:db_replica_wal_count,
:db_primary_wal_count,
:db_replica_wal_cached_count,
:db_primary_wal_cached_count
)
end
end
context 'single database' do
before do
skip_if_multiple_databases_are_setup
end
it 'has expected keys' do
expect(described_class.load_balancing_roles_metric_counter_keys).to include(
:db_replica_count,
:db_primary_count,
:db_replica_cached_count,
:db_primary_cached_count,
:db_replica_wal_count,
:db_primary_wal_count,
:db_replica_wal_cached_count,
:db_primary_wal_cached_count
)
end
end
end
describe '.load_balancing_metric_duration_keys' do
context 'multiple databases' do
before do
@ -100,8 +124,6 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it 'has expected keys' do
expect(described_class.load_balancing_metric_duration_keys).to include(
:db_replica_duration_s,
:db_primary_duration_s,
:db_main_duration_s,
:db_main_replica_duration_s,
:db_ci_duration_s,
@ -121,8 +143,6 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it 'has expected keys' do
expect(described_class.load_balancing_metric_duration_keys).to include(
:db_replica_duration_s,
:db_primary_duration_s,
:db_main_duration_s,
:db_main_replica_duration_s,
:db_main_txn_duration_s,
@ -141,6 +161,34 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
end
end
describe '.load_balancing_roles_metric_duration_keys' do
context 'multiple databases' do
before do
skip_if_multiple_databases_not_setup(:ci)
end
it 'has expected keys' do
expect(described_class.load_balancing_roles_metric_duration_keys).to include(
:db_replica_duration_s,
:db_primary_duration_s
)
end
end
context 'single database' do
before do
skip_if_multiple_databases_are_setup
end
it 'has expected keys' do
expect(described_class.load_balancing_roles_metric_duration_keys).to include(
:db_replica_duration_s,
:db_primary_duration_s
)
end
end
end
describe '#transaction', :request_store do
let(:web_transaction) { double('Gitlab::Metrics::WebTransaction') }
let(:background_transaction) { double('Gitlab::Metrics::WebTransaction') }
@ -163,7 +211,6 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it do
expect { subscriber.transaction(event) }
.to change { ::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_payload[:db_main_txn_count] }.by(1)
.and change { ::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_payload[:db_txn_count] }.by(1)
expect(::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_payload[:db_main_txn_duration_s]).to be >= 0
expect(::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_payload[:db_main_txn_max_duration_s]).to be >= 0
end
@ -301,6 +348,14 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it_behaves_like 'record ActiveRecord metrics'
it_behaves_like 'store ActiveRecord info in RequestStore', :primary
context 'when omit_aggregated_db_log_fields disabled' do
before do
stub_feature_flags(omit_aggregated_db_log_fields: false)
end
it_behaves_like 'store ActiveRecord info in RequestStore', :primary, include_aggregated: true
end
end
end
@ -377,6 +432,14 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it_behaves_like 'record ActiveRecord metrics', :replica
it_behaves_like 'store ActiveRecord info in RequestStore', :replica
context 'when omit_aggregated_db_log_fields disabled' do
before do
stub_feature_flags(omit_aggregated_db_log_fields: false)
end
it_behaves_like 'store ActiveRecord info in RequestStore', :replica, include_aggregated: true
end
end
context 'query using a connection to a primary' do
@ -394,6 +457,14 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
it_behaves_like 'record ActiveRecord metrics', :primary
it_behaves_like 'store ActiveRecord info in RequestStore', :primary
context 'when omit_aggregated_db_log_fields disabled' do
before do
stub_feature_flags(omit_aggregated_db_log_fields: false)
end
it_behaves_like 'store ActiveRecord info in RequestStore', :primary, include_aggregated: true
end
end
context 'query using a connection to an unknown source' do
@ -421,21 +492,6 @@ RSpec.describe Gitlab::Metrics::Subscribers::ActiveRecord do
subscriber.sql(event)
end
it 'does not store DB roles into into RequestStore' do
Gitlab::SafeRequestStore.ensure_request_store do
subscriber.sql(event)
expect(described_class.db_counter_payload).to include(
db_primary_cached_count: 0,
db_primary_count: 0,
db_primary_duration_s: 0,
db_replica_cached_count: 0,
db_replica_count: 0,
db_replica_duration_s: 0
)
end
end
end
end
end

4
spec/lib/gitlab/sidekiq_logging/structured_logger_spec.rb
View File

@ -293,12 +293,8 @@ RSpec.describe Gitlab::SidekiqLogging::StructuredLogger do
let(:expected_end_payload_with_db) do
expected_end_payload.merge(
'db_duration_s' => a_value >= 0.1,
'db_count' => a_value >= 1,
"db_#{db_config_name}_replica_count" => 0,
'db_replica_duration_s' => a_value >= 0,
'db_primary_count' => a_value >= 1,
"db_#{db_config_name}_count" => a_value >= 1,
'db_primary_duration_s' => a_value > 0,
"db_#{db_config_name}_duration_s" => a_value > 0
)
end

5
spec/services/merge_requests/mergeability/logger_spec.rb
View File

@ -90,11 +90,8 @@ RSpec.describe MergeRequests::Mergeability::Logger, :request_store, feature_cate
context 'when its a query' do
let(:extra_data) do
{
'mergeability.expensive_operation.db_count.values' => a_kind_of(Array),
'mergeability.expensive_operation.db_main_count.values' => a_kind_of(Array),
'mergeability.expensive_operation.db_main_duration_s.values' => a_kind_of(Array),
'mergeability.expensive_operation.db_primary_count.values' => a_kind_of(Array),
'mergeability.expensive_operation.db_primary_duration_s.values' => a_kind_of(Array)
'mergeability.expensive_operation.db_main_duration_s.values' => a_kind_of(Array)
}
end

1
spec/support/shared_contexts/lib/gitlab/sidekiq_logging/structured_logger_shared_context.rb
View File

@ -49,7 +49,6 @@ RSpec.shared_context 'structured_logger' do
metrics =
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_metric_counter_keys +
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_metric_duration_keys +
::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_keys +
[:db_duration_s]
metrics.each_with_object({}) do |key, result|

139
spec/support/shared_examples/metrics/active_record_subscriber_shared_examples.rb
View File

@ -1,6 +1,6 @@
# frozen_string_literal: true
RSpec.shared_examples 'store ActiveRecord info in RequestStore' do |db_role|
RSpec.shared_examples 'store ActiveRecord info in RequestStore' do |db_role, include_aggregated: false|
let(:db_config_name) do
db_config_name = ::Gitlab::Database.db_config_names(with_schema: :gitlab_shared).first
db_config_name += "_replica" if db_role == :secondary
@ -10,8 +10,16 @@ RSpec.shared_examples 'store ActiveRecord info in RequestStore' do |db_role|
let(:expected_payload_defaults) do
result = {}
metrics =
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_metric_counter_keys +
::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_keys
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_metric_counter_keys
if include_aggregated
metrics += ::Gitlab::Metrics::Subscribers::ActiveRecord.db_counter_keys +
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_roles_metric_counter_keys
::Gitlab::Metrics::Subscribers::ActiveRecord.load_balancing_roles_metric_duration_keys.each do |key|
result[key] = 0.0
end
end
metrics.each do |key|
result[key] = 0
@ -37,60 +45,77 @@ RSpec.shared_examples 'store ActiveRecord info in RequestStore' do |db_role|
Gitlab::SafeRequestStore.ensure_request_store do
subscriber.sql(event)
expected = case db_role
when :primary
transform_hash(expected_payload_defaults, {
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_primary_cached_count: record_cached_query ? 1 : 0,
"db_#{db_config_name}_cached_count": record_cached_query ? 1 : 0,
db_primary_count: record_query ? 1 : 0,
"db_#{db_config_name}_count": record_query ? 1 : 0,
db_primary_write_count: record_write_query ? 1 : 0,
"db_#{db_config_name}_write_count": record_write_query ? 1 : 0,
db_primary_duration_s: record_query ? 0.002 : 0.0,
"db_#{db_config_name}_duration_s": record_query ? 0.002 : 0.0,
db_primary_wal_count: record_wal_query ? 1 : 0,
"db_#{db_config_name}_wal_count": record_wal_query ? 1 : 0,
db_primary_wal_cached_count: record_wal_query && record_cached_query ? 1 : 0,
"db_#{db_config_name}_wal_cached_count": record_wal_query && record_cached_query ? 1 : 0
})
when :replica
transform_hash(expected_payload_defaults, {
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_replica_cached_count: record_cached_query ? 1 : 0,
"db_#{db_config_name}_cached_count": record_cached_query ? 1 : 0,
db_replica_count: record_query ? 1 : 0,
"db_#{db_config_name}_count": record_query ? 1 : 0,
db_replica_write_count: record_write_query ? 1 : 0,
"db_#{db_config_name}_write_count": record_write_query ? 1 : 0,
db_replica_duration_s: record_query ? 0.002 : 0.0,
"db_#{db_config_name}_duration_s": record_query ? 0.002 : 0.0,
db_replica_wal_count: record_wal_query ? 1 : 0,
"db_#{db_config_name}_wal_count": record_wal_query ? 1 : 0,
db_replica_wal_cached_count: record_wal_query && record_cached_query ? 1 : 0,
"db_#{db_config_name}_wal_cached_count": record_wal_query && record_cached_query ? 1 : 0
})
else
transform_hash(expected_payload_defaults, {
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_primary_cached_count: 0,
"db_#{db_config_name}_cached_count": 0,
db_primary_count: 0,
"db_#{db_config_name}_count": 0,
db_primary_duration_s: 0.0,
"db_#{db_config_name}_duration_s": 0.0,
db_primary_wal_count: 0,
"db_#{db_config_name}_wal_count": 0,
db_primary_wal_cached_count: 0,
"db_#{db_config_name}_wal_cached_count": 0
})
end
case db_role
when :primary
subscriber_hash = {
"db_#{db_config_name}_cached_count": record_cached_query ? 1 : 0,
"db_#{db_config_name}_count": record_query ? 1 : 0,
"db_#{db_config_name}_write_count": record_write_query ? 1 : 0,
"db_#{db_config_name}_duration_s": record_query ? 0.002 : 0.0,
"db_#{db_config_name}_wal_count": record_wal_query ? 1 : 0,
"db_#{db_config_name}_wal_cached_count": record_wal_query && record_cached_query ? 1 : 0
}
if include_aggregated
subscriber_hash = subscriber_hash.merge(
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_primary_cached_count: record_cached_query ? 1 : 0,
db_primary_count: record_query ? 1 : 0,
db_primary_write_count: record_write_query ? 1 : 0,
db_primary_duration_s: record_query ? 0.002 : 0.0,
db_primary_wal_count: record_wal_query ? 1 : 0,
db_primary_wal_cached_count: record_wal_query && record_cached_query ? 1 : 0
)
end
when :replica
subscriber_hash = {
"db_#{db_config_name}_cached_count": record_cached_query ? 1 : 0,
"db_#{db_config_name}_count": record_query ? 1 : 0,
"db_#{db_config_name}_write_count": record_write_query ? 1 : 0,
"db_#{db_config_name}_duration_s": record_query ? 0.002 : 0.0,
"db_#{db_config_name}_wal_count": record_wal_query ? 1 : 0,
"db_#{db_config_name}_wal_cached_count": record_wal_query && record_cached_query ? 1 : 0
}
if include_aggregated
subscriber_hash = subscriber_hash.merge({
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_replica_cached_count: record_cached_query ? 1 : 0,
db_replica_count: record_query ? 1 : 0,
db_replica_write_count: record_write_query ? 1 : 0,
db_replica_duration_s: record_query ? 0.002 : 0.0,
db_replica_wal_count: record_wal_query ? 1 : 0,
db_replica_wal_cached_count: record_wal_query && record_cached_query ? 1 : 0
})
end
else
subscriber_hash = {
"db_#{db_config_name}_cached_count": 0,
"db_#{db_config_name}_count": 0,
"db_#{db_config_name}_duration_s": 0.0,
"db_#{db_config_name}_wal_count": 0,
"db_#{db_config_name}_wal_cached_count": 0
}
if include_aggregated
subscriber_hash = subscriber_hash.merge({
db_count: record_query ? 1 : 0,
db_write_count: record_write_query ? 1 : 0,
db_cached_count: record_cached_query ? 1 : 0,
db_primary_cached_count: 0,
db_primary_count: 0,
db_primary_duration_s: 0.0,
db_primary_wal_count: 0,
db_primary_wal_cached_count: 0
})
end
end
expected = transform_hash(expected_payload_defaults, subscriber_hash)
expect(described_class.db_counter_payload).to eq(expected)
end

1
tests.yml
View File

@ -189,6 +189,7 @@ mapping:
- 'ee/spec/lib/gitlab/database/desired_sharding_key_spec.rb'
- 'spec/db/docs_spec.rb'
- 'spec/lib/gitlab/database/dictionary_spec.rb'
- 'ee/spec/lib/gitlab/database/inclusion_of_tables_with_gitlab_sec_schema_spec.rb'
- 'spec/lib/gitlab/database/no_new_tables_with_gitlab_main_schema_spec.rb'
- 'spec/lib/gitlab/database/sharding_key_spec.rb'