Commit Graph

3358 Commits

Author SHA1 Message Date
Douwe Maan 6c33981654 Merge branch 'sh-optimize-read-only-check' into 'master'
Optimize read-only middleware so that it does not consume as much CPU

Closes #40185 and gitlab-com/infrastructure#3240

See merge request gitlab-org/gitlab-ce!15504
2017-11-21 16:10:47 +00:00
Zeger-Jan van de Weg f9565e3039
Batchload blobs for diff generation
After installing a new gem, batch-loader, a construct can be used to
queue data to be fetched in bulk. The gem was also introduced in both
gitlab-org/gitlab-ce!14680 and gitlab-org/gitlab-ce!14846, but those mrs
are not merged yet.

For the generation of diffs, both the old blob and the new blob need to
be loaded. This for every file in the diff, too. Now we collect all
these so we do 1 fetch. Three `.allow_n_plus_1_calls` have been removed,
which I expect to be valid, but this needs to be confirmed by a full CI
run.

Possibly closes:
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37445
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37599
- https://gitlab.com/gitlab-org/gitlab-ce/issues/37431
2017-11-21 13:53:26 +01:00
digitalMoksha cba68d338b use `Gitlab::Routing.url_helpers` instead of `Rails.application.routes.url_helpers`
since `Rails.application.routes.url_helpers` creates a new anonymous module every time it's called
2017-11-21 13:29:57 +01:00
Stan Hu 3c52e2f06e Optimize read-only middleware so that it does not consume as much CPU
In !15082, we changed the behavior of the middleware to call
`Rails.application.routes.recognize_path` whenever a new route arrived.
However, this can be a CPU-intensive task because Rails needs to allocate
memory and compile 850+ different regular expressions, which are complicated
in GitLab.

As a short-term fix, we can do a lightweight string match before
we do the heavier comparison.

Closes #40185, gitlab-com/infrastructure#3240
2017-11-20 15:27:52 -08:00
Douwe Maan c043ba67c2 Merge branch '40292-bitbucket-import-hashed-storage' into 'master'
Fix bitbucket wiki import with hashed storage enabled

Closes #40292

See merge request gitlab-org/gitlab-ce!15490
2017-11-20 18:08:36 +00:00
Nick Thomas f2977c63a8
Fix bitbucket wiki import with hashed storage enabled 2017-11-20 16:14:41 +00:00
Douwe Maan 4b78f70fc4 Merge branch 'gitaly-delete-refs' into 'master'
Incorporate Gitaly's RefService.DeleteRefs RPC

Closes gitaly#740

See merge request gitlab-org/gitlab-ce!15460
2017-11-20 15:20:40 +00:00
Douwe Maan bd026b6a28 Merge branch 'bvl-dont-move-projects-using-hashed-storage' into 'master'
Don't move project repository/attachments when using hashed storage

Closes #40289

See merge request gitlab-org/gitlab-ce!15479
2017-11-20 13:48:20 +00:00
Bob Van Landuyt fa39e8a09c Don't move project repository/attachments when using hashed storage
When a project is using hashed storage, the repositories and
attachments wouldn't be saved on disk using the `full_path`. So the
migration would not do anything.

However: best to just skip moving when hashed storage is enabled.
2017-11-20 13:46:53 +01:00
Yorick Peterse 936e9e8950
Clean up schema of the "merge_requests" table
This adds various foreign keys and indexes to the "merge_requests" table
as outlined in https://gitlab.com/gitlab-org/gitlab-ce/issues/31825.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/31825
2017-11-20 12:45:33 +01:00
Douwe Maan 202ab62874 Merge branch 'fix-conflict-highlighting' into 'master'
Fix conflict highlighting

See merge request gitlab-org/gitlab-ce!15463
2017-11-20 10:17:43 +00:00
Rémy Coutable 4f09d099e9 Merge branch '18040-rubocop-line-break-after-guard-clause' into 'master'
Adds Rubocop rule for line break after guard clause

Closes #18040

See merge request gitlab-org/gitlab-ce!15188
2017-11-20 09:22:14 +00:00
Alejandro Rodríguez 3f0c9e9708 Fix Gitlab::Git::Repository#remote_tags using unexisting variable 2017-11-20 09:08:09 +00:00
Alejandro Rodríguez 38730a2d07 Incorporate Gitaly's RefService.DeleteRefs RPC 2017-11-17 18:08:50 -03:00
Sean McGivern 64a9e53bd1 Fix conflict highlighting
Conflicts used to take a `Repository` and pass that to
`Gitlab::Highlight.highlight`, which would call `#gitattribute` on the
repository. Now they use a `Gitlab::Git::Repository`, which didn't have that
method defined - but defining it on `Gitlab::Git::Repository` does make it
available on `Repository` through `method_missing`, so we can do that and both
cases will work.
2017-11-17 17:57:48 +00:00
Douwe Maan 371180a47d Merge branch 'mk-add-user-rate-limits' into 'master'
Add request rate limits

Closes #30053

See merge request gitlab-org/gitlab-ce!14708
2017-11-17 16:11:22 +00:00
Francisco Javier López c7cf68bd6f Changing OAuth lookup to be case insensitive 2017-11-17 14:24:25 +00:00
Sean McGivern 69dd59aec9 Merge branch 'dm-go-get-blank-enabled-protocol' into 'master'
Fix go-import meta data when enabled_git_access_protocol is a blank string

Closes #40256

See merge request gitlab-org/gitlab-ce!15448
2017-11-17 12:35:05 +00:00
Francisco Lopez 4188c10c07 Renaming AuthenticationException to AuthenticationError 2017-11-17 13:33:21 +01:00
Douwe Maan f767dd4a4d Fix go-import meta data when enabled_git_access_protocol is a blank string 2017-11-17 12:17:16 +01:00
Stan Hu 5cecff893d Convert migration to populate latest merge request ID into a background migration
This is to smear updates over a few hours to avoid causing excessive
replication lag as seen in https://gitlab.com/gitlab-com/infrastructure/issues/3235.
2017-11-17 10:23:48 +00:00
Francisco Lopez 1436598e49 Moved Exceptions to Gitlab::Auth 2017-11-17 10:02:11 +01:00
Francisco Lopez aa84ef1e1a Moving exceptions to UserAuthFinders 2017-11-17 10:02:11 +01:00
Francisco Lopez f189657523 Added some more comments 2017-11-17 10:02:11 +01:00
Francisco Lopez 130a9933fe Added UserAuthFinders spec 2017-11-17 10:02:11 +01:00
Francisco Lopez 8e57cc7e41 Added RequestAuthenticator spec 2017-11-17 10:02:11 +01:00
Jacopo 181cd299f9 Adds Rubocop rule for line break after guard clause
Adds a rubocop rule (with autocorrect) to ensure line break after guard clauses.
2017-11-16 17:58:29 +01:00
Rémy Coutable c5720382e6 Merge branch 'bvl-isolate-fork-network-background-migrations' into 'master'
Isolate the fork network background migrations

See merge request gitlab-org/gitlab-ce!15393
2017-11-16 16:08:46 +00:00
Rémy Coutable 05c10c9bf7 Add total_time_spent to the `changes` hash in issuable Webhook payloads
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-11-15 15:42:19 +01:00
Bob Van Landuyt f961744bde Isolate the fork network background migrations
Before the `PopulateForkNetworksRange` spec would also call the
`CreateForkNetworkMemberships` which we would count on in the spec.

With this, I'm isolating that, and counting only records created in
this particular migration instead.
2017-11-15 10:13:57 +01:00
Yorick Peterse 3730d89fe8 Merge branch 'bvl-fork-network-memberships-for-deleted-source' into 'master'
Don't try to create fork network memberships for forks of forks

Closes #40072

See merge request gitlab-org/gitlab-ce!15366
2017-11-14 13:43:04 +00:00
Bob Van Landuyt aaf18bb8c8 Don't try to create fork network memberships for forks of forks
In case the root project of a Fork-of-fork is deleted, the ForkNetwork
and the membership for that fork network is never created. In this
case we shouldn't try to create the membership, since the parent
membership will never be created.

This means that these fork networks will be lost.
2017-11-14 11:19:19 +01:00
Douwe Maan c9515ca541 Merge branch 'jej/fix-lfs-integrity-with-forks' into 'master'
Handle forks in Gitlab::Checks::LfsIntegrity

Closes #39902

See merge request gitlab-org/gitlab-ce!15279
2017-11-14 09:48:19 +00:00
Rémy Coutable 701577f0f2 Merge branch 'gitaly-wiki-get-all-pages' into 'master'
Incorporate Gitaly's WikiService.WikiGetAllPages RPC

Closes gitaly#722

See merge request gitlab-org/gitlab-ce!15307
2017-11-14 08:02:49 +00:00
Alejandro Rodríguez 282e7f8eab Incorporate Gitaly's WikiService.WikiGetAllPages RPC 2017-11-13 16:54:50 -03:00
Alejandro Rodríguez 5a38a9d86d Add spec examples for Gitlab::Gitaly::WikiService 2017-11-13 16:54:40 -03:00
Lin Jen-Shin (godfat) 258bf3e187 Add Gitlab::Utils::StrongMemoize 2017-11-13 15:27:30 +00:00
Jacob Vosmaer (GitLab) de301d13bb Prepare Repository#fetch_source_branch for migration 2017-11-10 14:45:23 +00:00
Sean McGivern 58dd9d73ab Merge branch 'cleanup-issues-schema' into 'master'
Clean up the schema of the table "issues"

Closes #31811

See merge request gitlab-org/gitlab-ce!15225
2017-11-10 11:26:32 +00:00
Yorick Peterse d825c9cb5d
Clean up schema of the "issues" table
This adds various foreign key constraints, indexes, missing NOT NULL
constraints, and changes some column types from timestamp to
timestamptz.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/31811
2017-11-09 18:00:30 +01:00
James Edwards-Jones ebd5174472 Handle forks in Gitlab::Checks::LfsIntegrity 2017-11-09 13:44:20 +00:00
Sean McGivern 43d98cd1ff Merge branch '39958-updatemergerequestsworker-are-failing-on-metric-tagging' into 'master'
Revert "add metrics tagging to the sidekiq middleware"

Closes #39958

See merge request gitlab-org/gitlab-ce!15285
2017-11-09 12:30:57 +00:00
Douwe Maan 760b2c75ef Merge branch 'github-rake-task-rate-limiting' into 'master'
Add GitHub enterprise support to the GitHub Rake task and better handle rate limiting being disabled

See merge request gitlab-org/gitlab-ce!15284
2017-11-09 08:37:41 +00:00
Douwe Maan 89bd78352e Merge branch 'ssrf-protections-round-2' into 'security-10-1'
Replace SSRF resolver with Addrinfo.getaddrinfo to include alternative localhost versions

See merge request gitlab/gitlabhq!2219

(cherry picked from commit 4a1e73783d)

1bffa0c3 Replace SSRF resolver with Addrinfo.getaddrinfo to include alternative localhost versions
2017-11-08 20:11:08 -08:00
micael.bergeron 7fd3ce417f Revert "add metrics tagging to the sidekiq middleware"
This reverts commit d5859bb9d5.
This reverts commit 2b7e03cf69.
This reverts commit 7799a9bc44.
2017-11-08 16:21:08 -05:00
Yorick Peterse f37fe2edc8
Support importing GH projects without rate limits
GitHub Enterprise disables rate limiting for the API, resulting in HTTP
404 errors when requesting rate limiting details. This changes
Gitlab::GithubImport::Client so it can deal with rate limiting being
disabled.
2017-11-08 21:37:01 +01:00
Stan Hu 0c3877a488 Merge branch 'fix-issues-api-list-performance' into 'master'
Fail when issuable_meta_data is called on an unlimited collection

Closes #39845

See merge request gitlab-org/gitlab-ce!15249
2017-11-08 19:37:21 +00:00
Yorick Peterse 2b886a7815
Restore Enterprise support in the GH importer
This was removed by accident as the old GitHub importer handled this
deep down the codebase, making it easy to miss.
2017-11-08 18:06:03 +01:00
Stan Hu 0232450c8a Fix Error 500 when pushing LFS objects with a write deploy key 2017-11-08 16:21:39 +00:00
James Edwards-Jones 78ea074f42 Moved LfsIntegrity specs to own file 2017-11-08 12:29:46 +00:00