Commit Graph

16388 Commits

Author SHA1 Message Date
Ash McKenzie 6cafa70027 Merge branch 'sh-support-csp-nonce' into 'master'
Add support for Content-Security-Policy

Closes #65330

See merge request gitlab-org/gitlab-ce!31402
2019-08-07 05:03:05 +00:00
Tiger Watson 36a01a88ce Use separate Kubernetes namespaces per environment
Kubernetes deployments on new clusters will now have
a separate namespace per project environment, instead
of sharing a single namespace for the project.

Behaviour of existing clusters is unchanged.

All new functionality is controlled by the
:kubernetes_namespace_per_environment feature flag,
which is safe to enable/disable at any time.
2019-08-07 04:40:29 +00:00
David Wilkins 467a411e88 Convert RestClient to Gitlab::HTTP for Prometheus Monitor
- Closes #60024

- Change PrometheusClient.new to accept a base url instead of an
  already created RestClient

- Use Gitlab::HTTP in PrometheusClient instead of creating RestClient
  in PrometheusService

- Move http_options from PrometheusService to
  PrometheusClient (follow_redirects: false)

- ensure that base urls don't have the trailing slash

- Created a `PrometheusClient#url` method that might not be strictly
  required

- Change rescued exceptions from RestClient::* to
  HTTParty::ResponseError where possible and StandardError for the
  rest
2019-08-07 02:42:20 +00:00
Stan Hu 5fbbd3dd6e
Add support for Content-Security-Policy
A nonce-based Content-Security-Policy thwarts XSS attacks by allowing
inline JavaScript to execute if the script nonce matches the header
value. Rails 5.2 supports nonce-based Content-Security-Policy headers,
so provide configuration to enable this and make it work.

To support this, we need to change all `:javascript` HAML filters to the
following form:

```
= javascript_tag nonce: true do
  :plain
    ...
```

We use `%script` throughout our HAML to store JSON and other text, but
since this doesn't execute, browsers don't appear to block this content
from being used and require the nonce value to be present.
2019-08-07 12:37:31 +10:00
Kelly Hair 4b91047c22 Update Packer.gitlab-ci.yml to use latest image 2019-08-06 13:00:55 -04:00
Bob Van Landuyt d61dab9147 Merge branch '65152-selective-highlight' into 'master'
Support selective highlighting of lines

See merge request gitlab-org/gitlab-ce!31361
2019-08-06 16:42:14 +00:00
Sean McGivern 2608732271 Merge branch 'make-issue-boards-importable' into 'master'
Make issue boards importable

See merge request gitlab-org/gitlab-ce!31434
2019-08-06 12:40:33 +00:00
Patrick Bajao 46631e1023 Support selective highlighting of lines
Instead of highlighting all lines when not all of them are
needed, only highlight specific lines.

The `BlobPresenter#highlight` method has been updated to
support `since` and `to` params. These params will be used to
limit the content to be highlighted.

Modify `Gitlab::Highlight` to support `since` param which will
then be used to determine the starting line number.
2019-08-06 09:32:29 +08:00
Gosia Ksionek 6c5e948013 Fix error on project name
Add project path to sql query to build proper path
2019-08-05 21:15:00 +00:00
Steve Abrams 3dbf3997bb Add group level container repository endpoints
API endpoints for requesting container repositories
and container repositories with their tag information
are enabled for users that want to specify the group
containing the repository rather than the specific project.
2019-08-05 20:00:50 +00:00
Douglas Barbosa Alexandre e9918b1a94 Merge branch 'sh-disable-redis-peek' into 'master'
Only track Redis calls if Peek is enabled

See merge request gitlab-org/gitlab-ce!31438
2019-08-05 19:09:48 +00:00
Stan Hu c368bae5b7 Merge branch 'add-missing-timezone-legacy-artifacts' into 'master'
Add missing timezone when copying legacy artifacts (ci_builds)

See merge request gitlab-org/gitlab-ce!31447
2019-08-05 17:33:39 +00:00
Jason Colyer f4ceff6e0c Make Issue boards importable
- Added space to li definiton
- Remove milestone from import_export.yml
2019-08-05 08:28:39 -05:00
Rémy Coutable f74387d298 Merge branch 'georgekoltsov/55474-outbound-setting-system-hooks' into 'master'
Add outbound setting for system hooks

See merge request gitlab-org/gitlab-ce!31177
2019-08-05 08:55:16 +00:00
Sean McGivern 5e20689d89 Merge branch 'GL-12757' into 'master'
Increase clair scanner from v11 to v12

See merge request gitlab-org/gitlab-ce!30809
2019-08-05 07:56:09 +00:00
Cameron Boulton 4b76d631e4 Set DOCKER_TLS_CERTDIR in CI job templates 2019-08-05 03:47:24 +00:00
David H. Wilkins afb3c3c1fb Add missing timezone to legacy artifacts (ci_builds)
- ci_builds.artifacts_expire_at are copied to
  ci_job_artifacts.expire_at with incorrect timestamps when the
  database timezone is NOT utc

- ci_builds.artifacts_expire_at is `timestamp without time zone` and
  ci_job_artifacts.expire_at is `timestamp with time zone` on
  postgresql

- Tests fail locally for `rspec
  ./spec/lib/gitlab/import_export/import_export_spec.rb` without this
  change
2019-08-04 12:33:37 -05:00
Stan Hu a43c84e6ab Only track Redis calls if Peek is enabled
In dev environments, Sidekiq was encountering the message:

Circular dependency detected while autoloading constant Gitlab::Profiler

This saves some overhead during normal usage.
2019-08-02 23:48:14 -07:00
mo 6a6be879e0
increase clair local scan version 2019-08-02 14:01:53 -06:00
mo e0d7df6561
increase clair from v11 to v12 2019-08-02 14:01:51 -06:00
Jason Colyer 87235d009c Make issue boards importable
- Added Importable to models/list.rb
- Did unless: :importable? on board validation
- Created changelog
- Modified haml to show issue boards are importable
- Added needed spec tests
- Modified project.json to include board information
- Added relevant models to all_models
- Added relevant models to import_export
- Added relevant models to safe_model_attributes
2019-08-02 14:35:10 -05:00
Dylan Griffith 5027979b9b Implement Helm ResetCommand for removing Tiller
Also creates specs

Only allow Helm to be uninstalled if it's the only app

- Remove Tiller leftovers after reser command
- Fixes specs and offenses

Adds changelog file

Fix reset_command specs
2019-08-02 19:02:57 +00:00
Sean McGivern c7de5403bd Merge branch 'make-needs-strong-connection' into 'master'
Make `needs:` to require a strong reference

Closes #65512

See merge request gitlab-org/gitlab-ce!31419
2019-08-02 18:48:52 +00:00
Kamil Trzciński 684751d3c2 Make needs: to require previous jobs
This changes `needs:` from weak reference
to have a strong reference.

This means that job will not be created
unless all needs are present as part of
a pipeline.
2019-08-02 18:41:40 +02:00
Robert Speicher 1bc2ac330e Merge branch '17276-breakage-in-displaying-svg-in-the-same-repository' into 'master'
Resolve "Breakage in displaying SVG in the same repository"

See merge request gitlab-org/gitlab-ce!31352
2019-08-02 16:32:40 +00:00
George Koltsov 5e3a208f58 Add backwards compatibility for legacy setting
`allow_local_requests_for_hooks_and_services` was renamed to
`allow_local_requests_for_web_hooks_and_services`.
2019-08-02 15:39:18 +01:00
George Koltsov 8abf920d1f Refactor SystemHookUrlValidator and specs
Simplify SystemHookUrlValidator to inherit from PublicUrlValidator
Refactor specs to move out shared examples to be used in both
system hooks and public url validators.
2019-08-02 15:39:18 +01:00
George Koltsov e5e1c907c0 Add outbound requests setting for system hooks
This MR adds new application setting to network section
`allow_local_requests_from_system_hooks`. Prior to this change
system hooks were allowed to do local network requests by default
and we are adding an ability for admins to control it.
2019-08-02 15:39:18 +01:00
Lukas '+ alert('Eipi') + ' Eipert fc9f099884 Prevent empty classes in ansi2html conversion
Currently we write out empty CSS classes (`class=""`) every time we
create a new tag. This adds 9 unnecessary bytes per span element. In a
recent trace, I have counted 11950 span elements. So we transported 105
unnecessary kilobytes!
2019-08-02 14:38:50 +00:00
Aleksei Lipniagov 1f9edb7c4a Call `GC::Profiler.clear` only in one place
Previously, both InfluxSampler and RubySampler were relying on the
`GC::Profiler.total_time` data which is the sum over the list
of captured GC events. Also, both samplers asynchronously called
`GC::Profiler.clear` which led to incorrect metric data because
each sampler has the wrong assumption it is the only object who calls
`GC::Profiler.clear` and thus could rely on the gathered results between
such calls.

We should ensure that `GC::Profiler.total_time` is called only in one
place making it possible to rely on accumulated data between such wipes.

Also, we need to track the amount of profiler reports we lost.
2019-08-02 09:04:32 +00:00
Marcel Amirault 98d6db4b43 Update regex for docs branches in ci
Changes * to + to be more precise with acceptable branch names
2019-08-02 06:24:07 +00:00
Heinrich Lee Yu 3b1da9be25 Fix inline rendering of SVGs from current repo
Changes generated URL to raw instead of blob
2019-08-02 06:34:20 +08:00
Stan Hu dbc6c24403 Merge branch '52123-issuable-actions-notesfinder-pderichs' into 'master'
Use NotesFinder to fetch notes on API and Controllers

Closes #52123

See merge request gitlab-org/gitlab-ce!31300
2019-08-01 16:27:02 +00:00
Douwe Maan c017dc578d Merge branch 'osw-avoid-errors-due-to-concurrent-calls' into 'master'
Add exclusive lease to mergeability check process

See merge request gitlab-org/gitlab-ce!31082
2019-08-01 14:58:14 +00:00
Kamil Trzciński e7ee84aad4 Add support for DAG
This implements the support for `needs:` keyword
as part of GitLab CI. That makes some of the jobs
to be run out of order.
2019-08-01 14:26:49 +00:00
Sean McGivern 4072c8e1bc Merge branch 'sh-backtrace-clean-sidekiq' into 'master'
Add additional paths to clean from backtrace

See merge request gitlab-org/gitlab-ce!31363
2019-08-01 09:44:06 +00:00
Sean McGivern 3186a3bfe3 Merge branch 'sh-fix-infite-loop-peek-bar' into 'master'
Fix SystemStackError when Peek bar is active with Rugged calls

Closes #65404

See merge request gitlab-org/gitlab-ce!31357
2019-08-01 09:42:28 +00:00
Patrick Derichs 0e99daae4a Use NotesFinder in IssuableActions module
Remove project from NotesFinder constructor

Add project parameter to specs

Also look for methods in private scope

Fix specs to match new NotesFinder constructor
2019-08-01 10:42:42 +02:00
Stan Hu 9225eb13ca Add additional paths to clean from backtrace
`Gitlab::Profiler` can also be used to profile Sidekiq jobs.
Add some call traces that can be omitted from the backtrace.
2019-07-31 23:07:42 -07:00
Stan Hu cfb7f11644 Merge branch '65363-add-feature-remove' into 'master'
Add Feature.remove

Closes #65363

See merge request gitlab-org/gitlab-ce!31315
2019-08-01 00:02:07 +00:00
Oswaldo Ferreira f4cd926cf3 Add exclusive lease to mergeability check process
Concurrent calls to UserMergeToRef RPC updating a single ref
can lead to an opaque fail that is being rescued at Gitaly.

So this commit adds an exclusive lease to the mergeability
check process with the key as the current MR ID.
2019-07-31 19:58:43 -03:00
Stan Hu 7a5c4cd0ca Fix SystemStackError when Peek bar is active with Rugged calls
Peek attempts to serialize results with `to_json`, which calls
`ActiveSupport::JSON`. If an object is passed to `to_json` that contains
instance variables, `ActiveSupport` will attempt to recursively traverse
all variables.

The problem is that we can get into an infinite loop if the instance
references to an instance that references to something else that points
back to the same instance.

To avoid this mess, we just call `to_s` on the object. It appears only
`Gitlab::Git::Repository` and `::Repository` are the culprits here.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65404
2019-07-31 15:47:19 -07:00
Ash McKenzie 31e419e945
Add new Feature.remove method 2019-08-01 08:29:14 +10:00
Lin Jen-Shin bea3d730c2 Merge branch 'add-project-scope-to-live-trace-feature-flag' into 'master'
Add project scope to live trace feature flag

See merge request gitlab-org/gitlab-ce!31325
2019-07-31 13:30:06 +00:00
Stan Hu 7bab7b6993 Merge branch 'fix-sidekiq-memory-killer-warn-message' into 'master'
Fix sidekiq memory killer warning message

Closes gitlab-com/gl-infra/infrastructure#7371

See merge request gitlab-org/gitlab-ce!31264
2019-07-31 13:04:46 +00:00
Andrew Newdigate ee828f09bf Adds Sidekiq scheduling latency structured logging field 2019-07-31 12:07:47 +00:00
drew 5eb3c4af38 Default dependency job stage index to Infinity, and correctly report it as undefined in prior stages 2019-07-31 12:06:01 +00:00
João Cunha d7c7ebf50a Allow knative do be uninstalled:
- After uninstalling the knative helm chart it's necessary to also
remove some leftover resources to allow the cluster to be clean
and knative to be reinstalleable.
- Adds knative uninstall disclaimer
- Uninstall ksvc before uninstalling knative

Make list of Knative and Ingres resources explicit

- To avoid deleting unwanted resources we are listing exact
which resources will be deleted rather than simply deleting any
resource that contains istio or knative words.
2019-07-31 11:58:38 +00:00
Sean McGivern fff0fc7b4c Merge branch '65317-adapt-tae-role-name-change' into 'master'
Fix Danger finding a test reviewer

Closes #65317

See merge request gitlab-org/gitlab-ce!31292
2019-07-31 11:32:13 +00:00
Shinya Maeda 759a3484ca Add project scope to live trace feature flag
Add project scope
2019-07-31 15:43:52 +07:00