Commit Graph

8326 Commits

Author SHA1 Message Date
Robert Speicher 4ec9009502 Merge branch 'ce-rd-extract-ee-specific-sidekiq-queue-config' into 'master'
Backport of gitlab-ee!8470

See merge request gitlab-org/gitlab-ce!23150
2018-11-26 17:55:21 +00:00
Bob Van Landuyt 67f2faa72d Don't specify `shell` for Gitaly write-ref
Use shelling out to git to write refs instead of rugged, hoping to
avoid creating invalid refs.

To update HEAD we switched to using `git symbolic-ref`.
2018-11-26 16:44:26 +01:00
Bob Van Landuyt 5e9921d547 Update gitaly and gitaly-proto 2018-11-26 16:44:26 +01:00
Stan Hu 1c1b7a820d Merge branch 'dm-batch-loader-key' into 'master'
Batch load only data from same repository when lazy object is accessed

See merge request gitlab-org/gitlab-ce!23309
2018-11-26 13:20:49 +00:00
Kamil Trzciński c150772edb Fix deadlock on ChunkedIO 2018-11-26 13:15:46 +01:00
Grzegorz Bizon f9c6134b60 Merge branch '40260-reduce-gitaly-calls-project-pipeline-status' into 'master'
Cache project HEAD to prevent unnecessary Gitaly calls

See merge request gitlab-org/gitlab-ce!23307
2018-11-26 12:14:35 +00:00
Grzegorz Bizon bf99a58852 Merge branch 'fix-null-variables' into 'master'
Allow to store null variables

Closes #54379

See merge request gitlab-org/gitlab-ce!23299
2018-11-26 11:53:01 +00:00
Sean McGivern f1a7e7fea1 Allow profiler to authenticate by stubbing users directly
Previously, we used a personal access token. This had a couple of
problems:

1. If the user didn't have a PAT, we couldn't impersonate them.
2. It depended on reading the raw PAT from the database.

Instead, we can monkey-patch the authentication methods on
ApplicationController (overriding the Devise ones), and remove them once
we're done. This does mean that profiles will not profile auth
correctly, so for that, use a PAT directly.
2018-11-26 11:18:03 +00:00
Douwe Maan 4b7e9eaacd
Treat Repository instances with the same disk path as the same hash key 2018-11-26 11:15:18 +01:00
Douwe Maan 5f0e4040ce
Batch load only data from same repository when lazy object is accessed
By specifying `key`, we get a different lazy batch loader for each
repository, which means that accessing a lazy object from one repository
will only result in that repository's objects being fetched, not those
of other repositories, saving us some unnecessary Gitaly lookups.
2018-11-26 11:15:18 +01:00
Heinrich Lee Yu 0c30052427
Use strong memoize utility 2018-11-26 17:41:41 +08:00
Heinrich Lee Yu 3b49a7948c
Prevent loading from cache if commit is nil 2018-11-26 17:41:40 +08:00
Heinrich Lee Yu 0047429a97
Cache project HEAD to prevent unnecessary Gitaly calls 2018-11-26 17:41:39 +08:00
Peter Leitzen b47a3a4085
Extract query variables into own module 2018-11-23 22:35:38 +01:00
Kamil Trzciński 9c98e2a922 Merge branch 'certmanager-temp' into 'master'
Deploy cert-manager to managed cluster for SSL certificates

See merge request gitlab-org/gitlab-ce!23036
2018-11-23 15:24:32 +00:00
Kamil Trzciński 56a01151d5 Allow to store null variables 2018-11-23 15:20:07 +01:00
Douwe Maan 1b8a791829 Clear BatchLoader context between Sidekiq jobs 2018-11-22 09:22:12 -08:00
Sean McGivern 2bc4a17dec Merge branch 'revert-fd6e3781' into 'master'
Revert "Merge branch 'revert-e2aa2177' into 'master'"

Closes #54267

See merge request gitlab-org/gitlab-ce!23237
2018-11-22 09:50:34 +00:00
Davy Defaud 6353a45b30 Fix typo in scheduled pipelines 2018-11-22 08:38:05 +00:00
Takuya Noguchi e902f462c2 Eliminate duplicated words
Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
2018-11-22 01:01:23 +09:00
Robert Speicher 9fe85710f6 Merge branch '49565-ssh-push-mirroring' into 'master'
SSH public-key authentication for push mirroring

Closes #49565

See merge request gitlab-org/gitlab-ce!22982
2018-11-20 13:43:24 +00:00
Rémy Coutable 2ea250d4bf Merge branch 'frozen-string-lib-gitlab-even-even-even-more' into 'master'
Enable even more frozen string for lib/gitlab

See merge request gitlab-org/gitlab-ce!23225
2018-11-20 13:05:40 +00:00
Zeger-Jan van de Weg 5c131dac5e
Fix bug where ID is not set
On HEAD~ we remove the ID from the class, which created a bug. Given we
don't need the ID anymore, it has been removed and simplified.
2018-11-20 13:40:45 +01:00
Zeger-Jan van de Weg 16346eb5b9 Revert "Merge branch 'revert-e2aa2177' into 'master'"
This reverts merge request !23229
2018-11-20 12:29:56 +00:00
Amit Rathi c3bad68d1c Merge branch 'master' into certmanager-temp 2018-11-20 17:58:10 +05:30
Sean McGivern fd6e3781ba Merge branch 'revert-e2aa2177' into 'master'
Revert "Merge branch 'zj-improve-gitaly-pb' into 'master'"

See merge request gitlab-org/gitlab-ce!23229
2018-11-20 11:53:18 +00:00
Amit Rathi 2945ab47d6 Fix usage data and install command spec 2018-11-20 16:48:23 +05:30
Douwe Maan 6f0ff56ef8 Merge branch 'fix/allow-saml2-for-2fa-bypass' into 'master'
saml/auth_hash: Allow 2FA bypass for SAML 2.0 responses

See merge request gitlab-org/gitlab-ce!22568
2018-11-20 11:07:59 +00:00
Douwe Maan e65e184269 Merge branch '3062-improve-project-cache' into 'master'
Change project cache key to depend on ID instead of full path

Closes #42191

See merge request gitlab-org/gitlab-ce!23135
2018-11-20 11:03:17 +00:00
Sean McGivern 2742b871fe Revert "Merge branch 'zj-improve-gitaly-pb' into 'master'"
This reverts merge request !23140
2018-11-20 10:36:52 +00:00
Steve Azzopardi 282a5b4c84
Merge branch 'master' of dev.gitlab.org:gitlab/gitlabhq 2018-11-20 10:11:42 +01:00
Sean McGivern e2aa2177a7 Merge branch 'zj-improve-gitaly-pb' into 'master'
Show what RPC is called in the performance bar

See merge request gitlab-org/gitlab-ce!23140
2018-11-20 06:42:49 +00:00
gfyoung f93f8f569d Enable even more frozen string for lib/gitlab
Enables frozen string for the following:

* lib/gitlab/patch/**/*.rb
* lib/gitlab/popen/**/*.rb
* lib/gitlab/profiler/**/*.rb
* lib/gitlab/project_authorizations/**/*.rb
* lib/gitlab/prometheus/**/*.rb
* lib/gitlab/query_limiting/**/*.rb
* lib/gitlab/quick_actions/**/*.rb
* lib/gitlab/redis/**/*.rb
* lib/gitlab/request_profiler/**/*.rb
* lib/gitlab/search/**/*.rb
* lib/gitlab/sherlock/**/*.rb
* lib/gitlab/sidekiq_middleware/**/*.rb
* lib/gitlab/slash_commands/**/*.rb
* lib/gitlab/sql/**/*.rb
* lib/gitlab/template/**/*.rb
* lib/gitlab/testing/**/*.rb
* lib/gitlab/utils/**/*.rb
* lib/gitlab/webpack/**/*.rb

Partially addresses gitlab-org/gitlab-ce#47424.
2018-11-19 18:24:22 -08:00
Stan Hu e392184e18 Merge branch 'frozen-string-lib-gitlab-even-even-more' into 'master'
Enable even more frozen string for lib/gitlab

See merge request gitlab-org/gitlab-ce!23161
2018-11-20 00:12:18 +00:00
Stan Hu 2e3674f7a8 Prevent templated services from being imported
Templated services should only be created by admins and does not
apply to project import/export.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54189
2018-11-19 15:41:41 -08:00
Rubén Dávila c02855f464 Backport of gitlab-ee!8470 2018-11-19 16:29:19 -05:00
Amit Rathi 2bb7573380 Updated schema.rb and remove extra char from install_command 2018-11-20 00:02:07 +05:30
Amit Rathi 1c7372ad56 Merge branch 'master' into certmanager-temp 2018-11-19 23:12:52 +05:30
Amit Rathi feee33e0dc Remove application flags 2018-11-19 22:47:59 +05:30
Douwe Maan af1ed0e2bb Merge branch '54046-fix-by-any-email-for-private-commit-emails' into 'master'
Match users better by their private commit email

Closes #54046

See merge request gitlab-org/gitlab-ce!23080
2018-11-19 16:53:14 +00:00
Sean McGivern 69f74582aa Merge branch 'improve-variables-support' into 'master'
Improve variables support

See merge request gitlab-org/gitlab-ce!23077
2018-11-19 15:57:53 +00:00
Kamil Trzciński c3bd3bfc6e Improve variables support
This ensures that variables accept only string,
alongside also improves kubernetes_namespace,
improving validation and default value being set.
2018-11-19 14:09:39 +01:00
Nick Thomas 59c4fb4ecb
Match users better by their private commit email
Private commit emails were introduced in !22560, but some parts of
GitLab were not updated to take account of them. This commit adds
support in places that were missed.
2018-11-19 12:45:07 +00:00
Nick Thomas f1bc7b6eb5
SSH public-key authentication for push mirroring 2018-11-19 11:46:39 +00:00
Fabio Busatto 294d15be3e Auto DevOps support for Group Security Dashboard 2018-11-19 11:22:19 +00:00
gfyoung 7ec8af5017 Enable even more frozen string for lib/gitlab
Enables frozen string for the following:

* lib/gitlab/hook_data/**/*.rb
* lib/gitlab/i18n/**/*.rb
* lib/gitlab/import/**/*.rb
* lib/gitlab/import_export/**/*.rb
* lib/gitlab/kubernetes/**/*.rb
* lib/gitlab/legacy_github_import/**/*.rb
* lib/gitlab/manifest_import/**/*.rb
* lib/gitlab/metrics/**/*.rb
* lib/gitlab/middleware/**/*.rb

Partially addresses gitlab-org/gitlab-ce#47424.
2018-11-16 17:41:14 -08:00
Gabriel Mazetto cb541aef89 Change project pipeline status cache key to follow project convention
Old cache key was: `projects/#{id}/pipeline_status/#{sha}`
New one is: `project:#{id}:pipeline_status:#{sha}`

This allows to delete any project related key just by scanning for:
`project:#{id}`.
2018-11-16 16:22:53 +01:00
Gabriel Mazetto 19440e6722 Change project cache key to depend on ID instead of full path
Previously, project cache used as part of the namespace the `full_path`,
which included namespace and project slug.

That meant that anytime a project was renamed or transfered to a
different namespace, we would lose the existing cache. This is not
necessary, nor desired.

I've prefixed cache key with `project:` to make it easy to find in redis
if necessary as well as make it possible to purge all project related
cache.

I've also switched the cache key type to go after the initial namespace
and not before.
2018-11-16 16:22:53 +01:00
Kamil Trzciński 5b75fcf6f2 Merge branch '49726-upgrade-helm-to-2-11' into 'master'
Resolve "Upgrade Helm Tiller Version Used By GitLab Managed Apps"

Closes #49726

See merge request gitlab-org/gitlab-ce!22693
2018-11-16 12:09:26 +00:00
Steve Azzopardi 6ac06e95e1 Merge branch '54011-all-files-named-index-have-their-content-rendered-as-if-they-were-text-files' into 'master'
Resolve "All files named `index.*` have their content rendered as if they were text files"

Closes #54011

See merge request gitlab-org/gitlab-ce!23063
2018-11-16 11:52:59 +00:00
Kamil Trzciński 348dd42d79 Merge branch 'sh-use-nokogiri-xml-backend' into 'master'
Use Nokogiri as the ActiveSupport XML backend

Closes #54068

See merge request gitlab-org/gitlab-ce!23136
2018-11-16 11:49:40 +00:00
Zeger-Jan van de Weg 35239a6aec
Show what RPC is called in the performance bar
Now only the data was shown of the service, which is not valueable at
times given some of those expose a lot of RPCs.
2018-11-16 12:16:25 +01:00
James Lopez a4ae5411f5
Fix rubocop warning 2018-11-16 12:09:18 +01:00
Dylan Griffith 1244533b77 Remove unnecessary --tiller-namespace 2018-11-16 10:57:48 +00:00
Dylan Griffith 9b1a99b03d Use helm version loop instead of sleep in ClientCommand 2018-11-16 10:57:27 +00:00
Dylan Griffith a71b3f6a7c Extract Helm::ClientCommand for shared commands 2018-11-16 10:57:20 +00:00
Grzegorz Bizon 875bae16d0 Merge branch 'allow-to-use-glob-for-ci-changes-detection' into 'master'
Add glob for CI changes detection

See merge request gitlab-org/gitlab-ce!23128
2018-11-16 10:35:18 +00:00
Sean McGivern 85ec04eaaa Merge branch 'gt-fix-typos-in-lib' into 'master'
Fix typos in lib

See merge request gitlab-org/gitlab-ce!23106
2018-11-16 10:32:31 +00:00
James Lopez 53b2c3011e
Remove redundant end anchors 2018-11-16 11:27:47 +01:00
Dylan Griffith fe1469e12f Upgrade helm to 2.11.0 and upgrade on every install 2018-11-16 10:22:40 +00:00
James Lopez f837281ff6
Update regex to use union 2018-11-16 11:03:45 +01:00
Douwe Maan a480c551a2 Merge branch 'fj-fix-bug-commit-check' into 'master'
Fix bug in Gitlab::Checks::CommitCheck

See merge request gitlab-org/gitlab-ce!23045
2018-11-16 08:53:58 +00:00
Stan Hu 198f2a0b61 Use Nokogiri as the ActiveSupport XML backend
This significantly improves performance and reduces memory consumption
when parsing XML files. On a test with 124 JUnit files from a CE build,
there was about a 4x reduction in processing time.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/54068
2018-11-16 00:23:36 -08:00
Kirill Zaitsev 9bea3c0fbb Add glob for CI changes detection 2018-11-15 22:04:47 +02:00
Stan Hu 6494467a19 Merge branch 'frozen-string-lib-gitlab-even-more' into 'master'
Enable even more frozen string in lib/gitlab

See merge request gitlab-org/gitlab-ce!22959
2018-11-15 19:50:53 +00:00
Kamil Trzciński 5059c15d06 Merge branch '53626-update-config-map-on-install-retry' into 'master'
Update config map if already present on install

Closes #53626

See merge request gitlab-org/gitlab-ce!22969
2018-11-15 12:33:40 +00:00
Amit Rathi 161e66890c Set appropriate RBAC flag for cluster applications 2018-11-15 17:52:41 +05:30
Chris Baumbauer 327892b23e Add knative client to kubeclient library 2018-11-15 11:39:43 +00:00
George Tsiolis 566ba126f5 Fix typos in lib 2018-11-15 12:15:43 +02:00
Thong Kuah 429d3e4952 Update config map if already present on install
When an application install fails, and the user retries install, the
configmap for the application will already exists. If so, we simply
update instead of create.
2018-11-15 22:39:42 +13:00
Grzegorz Bizon 7528b2d260 Merge branch 'osw-fallback-on-blank-refs' into 'master'
Avoid Gitaly RPC errors when fetching diff stats

Closes gitaly#1393

See merge request gitlab-org/gitlab-ce!22995
2018-11-15 09:16:30 +00:00
Grzegorz Bizon 1cccfca1dc Merge branch 'auto_devops_kubernetes_active' into 'master'
Switch kubernetes:active with checking $KUBECONFIG

Closes gitlab-ee#8311

See merge request gitlab-org/gitlab-ce!22929
2018-11-15 09:09:26 +00:00
Oswaldo Ferreira 039df0267e Avoid Gitaly RPC errors when fetching diff stats 2018-11-14 17:33:30 -02:00
James Lopez 3cd538c2e9
Fix text rendering of readme/index 2018-11-14 16:07:31 +01:00
Dylan Griffith ba7cfdc2d7 Track Kubernetes errors using Sentry 2018-11-14 11:10:36 +00:00
Rémy Coutable dff3ded57f Merge branch 'patch-31' into 'master'
Update comment in Auto-DevOps.gitlab-ci.yml

See merge request gitlab-org/gitlab-ce!23013
2018-11-14 10:22:00 +00:00
Francisco Javier López 3797cfbd39
Fix bug in CommitCheck 2018-11-14 09:55:57 +01:00
Amit Rathi 5c275490ec Merge branch 'master' into certmanager-temp 2018-11-14 10:14:50 +05:30
gfyoung ebf98f27c4 Enable even more frozen string in lib/gitlab
Enables frozen string for the following:

* lib/gitlab/fogbugz_import/**/*.rb
* lib/gitlab/gfm/**/*.rb
* lib/gitlab/git/**/*.rb
* lib/gitlab/gitaly_client/**/*.rb
* lib/gitlab/gitlab_import/**/*.rb
* lib/gitlab/google_code_import/**/*.rb
* lib/gitlab/gpg/**/*.rb
* lib/gitlab/grape_logging/**/*.rb
* lib/gitlab/graphql/**/*.rb
* lib/gitlab/graphs/**/*.rb
* lib/gitlab/hashed_storage/**/*.rb
* lib/gitlab/health_checks/**/*.rb

Partially address gitlab-org/gitlab-ce#47424.
2018-11-13 11:42:15 -08:00
Amit Rathi 631ab89885 Use email on the account 2018-11-13 20:36:06 +05:30
Thong Kuah 14b3033a0d Show HTTP response code for Kubernetes errors 2018-11-13 12:46:01 +00:00
Amit Rathi 0e4ad53c92 Remove logging from pod.rb 2018-11-13 17:42:34 +05:30
Amit Rathi 7be10a7c21 Remove logging changes 2018-11-13 17:40:38 +05:30
Amit Rathi 6dda43d67e Cleanup 2018-11-13 17:38:04 +05:30
Amit Rathi 0e8e75581b Merge branch 'master' into certmanager-temp 2018-11-13 17:00:43 +05:30
Amit Rathi 206f6747ec Updated logging 2018-11-13 16:57:49 +05:30
Tripple Moon b9ec3c2ffa Update Auto-DevOps.gitlab-ci.yml
Typo in comment INCREMENTAL_ROLLOUT_TYPE => INCREMENTAL_ROLLOUT_MODE
2018-11-13 07:27:22 +00:00
Rémy Coutable ec3712c201 Merge branch 'ce-3777-promote-to-epic' into 'master'
Refactoring Issues::MoveService (port of promote epics)

See merge request gitlab-org/gitlab-ce!22766
2018-11-12 20:22:57 +00:00
Rémy Coutable 33cd8efbd7 Merge branch 'sh-53180-append-path' into 'master'
Make sure there's only one slash as path separator

Closes #53180

See merge request gitlab-org/gitlab-ce!22954
2018-11-12 19:37:36 +00:00
Douwe Maan 56834e6bb2 Merge branch 'osw-comment-on-any-line-on-diffs-w-feature-flag' into 'master'
Comment on any expanded diff line on MRs (with feature-flag)

Closes #13950

See merge request gitlab-org/gitlab-ce!22914
2018-11-12 15:36:48 +00:00
Jarka Košanová 4af1712d01 Extract code from Issues::MoveService
Extract code to make it easier reusable
- introduce AttributesRewriter and ContentRewriter
  - support group entites when rewriting content
- make Uploader copy_to working for Namespaces
2018-11-12 08:57:25 +01:00
Amit Rathi c2407dab3e Merge branch 'master' into certmanager-temp 2018-11-12 12:06:59 +05:30
Philippe Lafoucrière a737c4fb5d Set an empty entrypoint for license_management job 2018-11-10 01:18:33 +00:00
Stan Hu 39f252254b Make sure there's only one slash as path separator
In Ruby 2.4, `URI.join("http://test//", "a").to_s` will
remove the double slash, however it's not the case in
Ruby 2.5. Using chomp should work better for the intention,
as we're not trying to allow things like ../ or / paths
resolution.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/53180
2018-11-09 15:36:45 -08:00
Stan Hu 6a91b28932 Remove circular dependency on Redactable in migration
Gitlab::BackgroundMigration::RedactLinks was prepending
EE::Gitlab::BackgroundMigration::RedactLinks, while
EE::Gitlab::BackgroundMigration::RedactLinks was trying to include
Gitlab::BackgroundMigration::RedactLinks::Redactable. Ruby 2.5.3 failed
with an uninitialized constant
(https://gitlab.com/gitlab-org/gitlab-ee/-/jobs/118388511).
2018-11-09 12:59:39 -08:00
Oswaldo Ferreira f9b4130bb7 Comment on any expanded diff line on MRs 2018-11-09 16:56:48 -02:00
Rémy Coutable 32bf2667d2 Merge branch 'frozen-string-lib-gitlab-more' into 'master'
Enable even more frozen string in lib/gitlab

See merge request gitlab-org/gitlab-ce!22863
2018-11-09 17:26:46 +00:00
Dylan Griffith da7836956e Improve Debugging Capabilities For Helm Application Installs 2018-11-09 15:03:24 +00:00
Thong Kuah 1e367810dc Use except:variables to fix multiple variable jobs
As `only:variables` has the effect of being satisfied by any one of
variable expression - this will create a situation where e.g. the dast
job will be active even if there's no cluster.

So use `except:variables which requires each and every variable
expression to be satisfied.
2018-11-09 17:01:35 +13:00
Thong Kuah f44b7ac6b7 Switch kubernetes:active with checking $KUBECONFIG
We have found an issue with `kubernetes:active` where it doesn't work
with multiple clusters. Switch with the similar $KUBECONFIG variable
check but this check actually gets the variable from the cluster
matching the environment name.

Fixes https://gitlab.com/gitlab-org/gitlab-ee/issues/8311
2018-11-09 13:45:41 +13:00