root
/
gitlab-ce
mirror of https://gitlab.com/free-releases/gitlab-ce.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

Add latest changes from gitlab-org/gitlab@master

This commit is contained in:
GitLab Bot 2021-04-15 06:09:30 +00:00
parent 1edce674dc
commit 17e561ffb8
17 changed files with 246 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.rubocop_manual_todo.yml
View File

@ -1884,6 +1884,7 @@ Gitlab/NamespacedClass:
- 'app/models/issue_email_participant.rb'
- 'app/models/issue_link.rb'
- 'app/models/issue_user_mention.rb'
- 'app/models/iteration.rb'
- 'app/models/jira_connect_installation.rb'
- 'app/models/jira_connect_subscription.rb'
- 'app/models/jira_import_state.rb'
@ -2695,7 +2696,6 @@ Gitlab/NamespacedClass:
- 'ee/app/models/issuable_metric_image.rb'
- 'ee/app/models/issuable_sla.rb'
- 'ee/app/models/issuables_analytics.rb'
- 'ee/app/models/iteration.rb'
- 'ee/app/models/iteration_note.rb'
- 'ee/app/models/ldap_group_link.rb'
- 'ee/app/models/ldap_key.rb'

2
GITALY_SERVER_VERSION
View File

@ -1 +1 @@
8752fc1097114316889560502579ddf477a20a2d
1724a973d59c30e92e73953f0b34701e34dff6eb

2
Gemfile
View File

@ -398,6 +398,8 @@ group :development, :test do
gem 'parallel', '~> 1.19', require: false
gem 'rblineprof', '~> 0.3.6', platform: :mri, require: false
gem 'test_file_finder', '~> 0.1.3'
end
group :development, :test, :danger do

3
Gemfile.lock
View File

@ -1233,6 +1233,8 @@ GEM
terser (1.0.2)
execjs (>= 0.3.0, < 3)
test-prof (0.12.0)
test_file_finder (0.1.3)
faraday (~> 1.0.1)
text (1.3.1)
thin (1.8.0)
daemons (~> 1.0, >= 1.0.9)
@ -1601,6 +1603,7 @@ DEPENDENCIES
sys-filesystem (~> 1.1.6)
terser (= 1.0.2)
test-prof (~> 0.12.0)
test_file_finder (~> 0.1.3)
thin (~> 1.8.0)
thrift (>= 0.14.0)
timecop (~> 0.9.1)

16
app/models/iteration.rb Normal file
View File

@ -0,0 +1,16 @@
# frozen_string_literal: true
# Placeholder class for model that is implemented in EE
class Iteration < ApplicationRecord
self.table_name = 'sprints'
def self.reference_prefix
'*iteration:'
end
def self.reference_pattern
nil
end
end
Iteration.prepend_if_ee('::EE::Iteration')

5
changelogs/unreleased/update-ruby-in-dockerfile-templates.yml Normal file
View File

@ -0,0 +1,5 @@
---
title: Update Ruby from 2.5 to 2.7 in Dockerfile templates
merge_request: 59345
author: Takuya Noguchi
type: changed

2
config/feature_flags/development/geo_pipeline_artifact_replication.yml
View File

@ -5,4 +5,4 @@ rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/326228
milestone: '13.11'
type: development
group: group::geo
default_enabled: false
default_enabled: true

2
doc/api/graphql/reference/index.md
View File

@ -3128,7 +3128,7 @@ Represents an external issue.
| `minimumReverificationInterval` | [`Int`](#int) | The interval (in days) in which the repository verification is valid. Once expired, it will be reverified. |
| `name` | [`String`](#string) | The unique identifier for this Geo node. |
| `packageFileRegistries` | [`PackageFileRegistryConnection`](#packagefileregistryconnection) | Package file registries of the GeoNode. |
| `pipelineArtifactRegistries` | [`PipelineArtifactRegistryConnection`](#pipelineartifactregistryconnection) | Find pipeline artifact registries on this Geo node. Available only when feature flag `geo_pipeline_artifact_replication` is enabled. |
| `pipelineArtifactRegistries` | [`PipelineArtifactRegistryConnection`](#pipelineartifactregistryconnection) | Find pipeline artifact registries on this Geo node. |
| `primary` | [`Boolean`](#boolean) | Indicates whether this Geo node is the primary. |
| `reposMaxCapacity` | [`Int`](#int) | The maximum concurrency of repository backfill for this secondary node. |
| `selectiveSyncNamespaces` | [`NamespaceConnection`](#namespaceconnection) | The namespaces that should be synced, if `selective_sync_type` == `namespaces`. |

18
doc/development/rake_tasks.md
View File

@ -152,6 +152,24 @@ To run several tests inside one directory:
- `bin/rspec spec/requests/api/` for the RSpec tests if you want to test API only
### Run RSpec tests which failed in Merge Request pipeline on your machine
If your Merge Request pipeline failed with RSpec test failures,
you can run all the failed tests on your machine with the following Rake task:
```shell
bin/rake spec:merge_request_rspec_failure
```
There are a few caveats for this Rake task:
- You need to be on the same branch on your machine as the source branch of the Merge Request.
- The pipeline must have been completed.
- You may need to wait for the test report to be parsed and retry again.
This Rake task depends on the [unit test reports](../ci/unit_test_reports.md) feature,
which only gets parsed when it is requested for the first time.
### Speed up tests, Rake tasks, and migrations
[Spring](https://github.com/rails/spring) is a Rails application pre-loader. It

81
doc/user/application_security/container_scanning/index.md
View File

@ -250,8 +250,85 @@ To allowlist specific vulnerabilities, follow these steps:
1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in
[overriding the container scanning template](#overriding-the-container-scanning-template).
1. Define the allowlisted vulnerabilities in a YAML file named `vulnerability-allowlist.yml`. This must use
the format described in the [allowlist example file](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/raw/master/testdata/vulnerability-allowlist.yml).
1. Add the `vulnerability-allowlist.yml` file to your project's Git repository.
the format described in [vulnerability-allowlist.yml data format](#vulnerability-allowlistyml-data-format).
1. Add the `vulnerability-allowlist.yml` file to the root folder of your project's Git repository.
#### vulnerability-allowlist.yml data format
The `vulnerability-allowlist.yml` file is a YAML file that specifies a list of CVE IDs of vulnerabilities that are **allowed** to exist, because they're _false positives_, or they're _not applicable_.
If a matching entry is found in the `vulnerability-allowlist.yml` file, the following happens:
- The vulnerability **is not included** when the analyzer generates the `gl-container-scanning-report.json` file.
- The Security tab of the pipeline **does not show** the vulnerability. It is not included in the JSON file, which is the source of truth for the Security tab.
Example `vulnerability-allowlist.yml` file:
```yaml
generalallowlist:
CVE-2019-8696:
CVE-2014-8166: cups
CVE-2017-18248:
images:
registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:
CVE-2018-4180:
your.private.registry:5000/centos:
CVE-2015-1419: libxml2
CVE-2015-1447:
```
This example excludes from `gl-container-scanning-report.json`:
1. All vulnerabilities with CVE IDs: _CVE-2019-8696_, _CVE-2014-8166_, _CVE-2017-18248_.
1. All vulnerabilities found in the `registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256` container image with CVE ID _CVE-2018-4180_.
1. All vulnerabilities found in `your.private.registry:5000/centos` container with CVE IDs _CVE-2015-1419_, _CVE-2015-1447_.
##### File format
- `generalallowlist` block allows you to specify CVE IDs globally. All vulnerabilities with matching CVE IDs are excluded from the scan report.
- `images` block allows you to specify CVE IDs for each container image independently. All vulnerabilities from the given image with matching CVE IDs are excluded from the scan report. The image name is retrieved from one of the environment variables used to specify the Docker image to be scanned, such as `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` or `DOCKER_IMAGE`. The image provided in this block **must** match this value and **must not** include the tag value. For example, if you specify the image to be scanned using `DOCKER_IMAGE=alpine:3.7`, then you would use `alpine` in the `images` block, but you cannot use `alpine:3.7`.
You can specify container image in multiple ways:
- as image name only (ie. `centos`).
- as full image name with registry hostname (ie. `your.private.registry:5000/centos`).
- as full image name with registry hostname and sha256 label (ie. `registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256`).
NOTE:
The string after CVE ID (`cups` and `libxml2` in the previous example) is an optional comment format. It has **no impact** on the handling of vulnerabilities. You can include comments to describe the vulnerability.
##### Container scanning job log format
You can verify the results of your scan and the correctness of your `vulnerability-allowlist.yml` file by looking
at the logs that are produced by the container scanning analyzer in `container_scanning` job details.
The log contains a list of found vulnerabilities as a table, for example:
```plainttext
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| STATUS | CVE SEVERITY | PACKAGE NAME | PACKAGE VERSION | CVE DESCRIPTION |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Approved | High CVE-2019-3462 | apt | 1.4.8 | Incorrect sanitation of the 302 redirect field in HTTP transport metho |
| | | | | d of apt versions 1.4.8 and earlier can lead to content injection by a |
| | | | | MITM attacker, potentially leading to remote code execution on the ta |
| | | | | rget machine. |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Unapproved | Medium CVE-2020-27350 | apt | 1.4.8 | APT had several integer overflows and underflows while parsing .deb pa |
| | | | | ckages, aka GHSL-2020-168 GHSL-2020-169, in files apt-pkg/contrib/extr |
| | | | | acttar.cc, apt-pkg/deb/debfile.cc, and apt-pkg/contrib/arfile.cc. This |
| | | | | issue affects: apt 1.2.32ubuntu0 versions prior to 1.2.32ubuntu0.2; 1 |
| | | | | .6.12ubuntu0 versions prior to 1.6.12ubuntu0.2; 2.0.2ubuntu0 versions |
| | | | | prior to 2.0.2ubuntu0.2; 2.1.10ubuntu0 versions prior to 2.1.10ubuntu0 |
| | | | | .1; |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Unapproved | Medium CVE-2020-3810 | apt | 1.4.8 | Missing input validation in the ar/tar implementations of APT before v |
| | | | | ersion 2.1.2 could result in denial of service when processing special |
| | | | | ly crafted deb files. |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
```
Vulnerabilities in the log are marked as `Approved` when the corresponding CVE ID is added to the `vulnerability-allowlist.yml` file.
### Running container scanning in an offline environment

10
lib/tasks/spec.rake
View File

@ -2,6 +2,8 @@
return if Rails.env.production?
require_relative '../../tooling/merge_request_rspec_failure_rake_task'
namespace :spec do
desc 'GitLab | RSpec | Run unit tests'
RSpec::Core::RakeTask.new(:unit, :rspec_opts) do |t, args|
@ -24,11 +26,19 @@ namespace :spec do
t.rspec_opts = args[:rspec_opts]
end
desc 'GitLab | RSpec | Run merge request RSpec failures'
Tooling::MergeRequestRspecFailureRakeTask.new(:merge_request_rspec_failure, :rspec_opts) do |t, args|
t.pattern = t.rspec_failures_on_merge_request
t.rspec_opts = args[:rspec_opts]
end
desc 'Run the code examples in spec/requests/api'
RSpec::Core::RakeTask.new(:api) do |t|
t.pattern = 'spec/requests/api/**/*_spec.rb'
end
private
def require_test_level
require_relative '../../tooling/quality/test_level'
end

2
qa/qa/resource/merge_request.rb
View File

@ -62,7 +62,7 @@ module QA
@labels = []
@file_name = "added_file-#{SecureRandom.hex(8)}.txt"
@file_content = "File Added"
@target_branch = "master"
@target_branch = project.default_branch
@target_new_branch = true
@no_preparation = false
@wait_for_merge = true

31
spec/tooling/merge_request_spec.rb Normal file
View File

@ -0,0 +1,31 @@
# frozen_string_literal: true
require 'webmock/rspec'
require_relative '../../tooling/merge_request'
require_relative '../support/helpers/next_instance_of'
RSpec.describe Tooling::MergeRequest do
let(:project_path) { 'gitlab-org/gitlab' }
let(:branch_name) { 'my-branch' }
let(:merge_request_iid) { 123 }
let(:merge_requests) { [{ 'iid' => merge_request_iid }] }
describe '.for' do
let(:stub_api) do
stub_request(:get, "https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/merge_requests")
.and_return(body: merge_requests)
end
before do
stub_api.with(query: { source_branch: branch_name, order_by: 'updated_at', sort: 'desc' })
end
it 'fetches merge request for local branch in the given GitLab project path' do
merge_request = described_class.for(branch: branch_name, project_path: project_path)
expect(merge_request.iid).to eq(merge_request_iid)
expect(stub_api).to have_been_requested.once
end
end
end

38
tooling/merge_request.rb Normal file
View File

@ -0,0 +1,38 @@
# frozen_string_literal: true
require 'faraday'
require 'faraday_middleware'
module Tooling
class MergeRequest
GITLAB_API_URL_TEMPLATE = 'https://gitlab.com/api/v4/projects/%{project_path}/merge_requests'
def self.for(branch:, project_path:)
url = format(GITLAB_API_URL_TEMPLATE, { project_path: URI.encode_www_form_component(project_path) })
conn = Faraday.new(url) do |conn|
conn.request :json
conn.response :json, content_type: /\bjson$/
conn.adapter Faraday.default_adapter
end
response = conn.get do |req|
req.params[:source_branch] = branch
req.params[:order_by] = 'updated_at'
req.params[:sort] = 'desc'
end
new(response.body.first)
end
attr_reader :merge_request
def initialize(merge_request)
@merge_request = merge_request
end
def iid
merge_request['iid']
end
end
end

37
tooling/merge_request_rspec_failure_rake_task.rb Normal file
View File

@ -0,0 +1,37 @@
# frozen_string_literal: true
require 'test_file_finder'
require_relative './merge_request'
module Tooling
class MergeRequestRspecFailureRakeTask < RSpec::Core::RakeTask
PROJECT_PATH = 'gitlab-org/gitlab'
def run_task(_verbose)
if pattern.empty?
puts "No rspec failures in the merge request."
return
end
super
end
def rspec_failures_on_merge_request
test_file_finder = TestFileFinder::FileFinder.new
test_file_finder.use TestFileFinder::MappingStrategies::GitlabMergeRequestRspecFailure.new(project_path: PROJECT_PATH, merge_request_iid: merge_request.iid)
test_file_finder.test_files
rescue TestFileFinder::TestReportError => e
abort e.message
end
private
def merge_request
@merge_request ||= Tooling::MergeRequest.for(branch: current_branch, project_path: PROJECT_PATH)
end
def current_branch
@current_branch ||= `git branch --show-current`.strip
end
end
end

2
vendor/Dockerfile/Ruby-alpine.Dockerfile vendored
View File

@ -1,4 +1,4 @@
FROM ruby:2.5-alpine
FROM ruby:2.7-alpine
# Edit with nodejs, mysql-client, postgresql-client, sqlite3, etc. for your needs.
# Or delete entirely if not needed.

2
vendor/Dockerfile/Ruby.Dockerfile vendored
View File

@ -1,4 +1,4 @@
FROM ruby:2.5
FROM ruby:2.7
# Edit with nodejs, mysql-client, postgresql-client, sqlite3, etc. for your needs.
# Or delete entirely if not needed.