Commit Graph

267 Commits

Author SHA1 Message Date
micael.bergeron cd88fa8f80 removed the #ensure_ref_fetched from all controllers
also, I refactored the MergeRequest#fetch_ref method to express
the side-effect that this method has.

  MergeRequest#fetch_ref -> MergeRequest#fetch_ref!
  Repository#fetch_source_branch -> Repository#fetch_source_branch!
2017-11-03 08:13:11 -04:00
Rémy Coutable 6a5e8f1046 Merge branch 'gitaly-ff-merge' into 'master'
Incorporate Gitaly's OperationService.UserFFBranch RPC

Closes gitaly#693

See merge request gitlab-org/gitlab-ce!15040
2017-11-02 16:11:53 +00:00
Kim Carlbäcker d6066870cc Fix encoding issue with Repository.ls_files 2017-11-02 08:29:03 +00:00
Douwe Maan e65bbc0ca4 Merge branch 'jej/lfs-change-detection' into 'master'
Detect changes to LFS pointers for pruning and integrity check

See merge request gitlab-org/gitlab-ce!14785
2017-11-01 17:33:41 +00:00
James Edwards-Jones fb3f9c6e50 Detect changes to LFS pointers for pruning and integrity check
Gitlab::Git::Blob.batch_lfs_metadata can be used to check for LFS pointers. It uses a lazy enumorator and filters by blob size
2017-11-01 16:41:05 +00:00
Alejandro Rodríguez 37cc50f843 Incorporate Gitaly's OperationService.UserFFBranch RPC 2017-10-31 16:28:09 -03:00
Lin Jen-Shin (godfat) 57d7ed05d9 Fetch the merged branches at once 2017-10-27 15:55:08 +00:00
Alejandro Rodríguez 42a45392a9 Fix the incorrect value being used to set GL_USERNAME on hooks 2017-10-25 14:01:57 -03:00
Rémy Coutable 53ab16f6ff Merge branch 'gitaly-ff-merge-preparation' into 'master'
Move all rugged operation for ff_merge inside Gitlab::Git

Closes gitaly#683

See merge request gitlab-org/gitlab-ce!15011
2017-10-25 12:42:25 +00:00
Alejandro Rodríguez a64601b929 Move all rugged operation for ff_merge inside Gitlab::Git
We also delete some unused code related to the aforementioned feature.
2017-10-24 14:31:52 -03:00
Douwe Maan 30a8d2def3 Merge branch 'gitaly-gl-username' into 'master'
Support `Gitaly::User`'s gl_username field

See merge request gitlab-org/gitlab-ce!15016
2017-10-24 13:59:44 +00:00
Alejandro Rodríguez 4969f486f2 Support `Gitaly::User`'s gl_username field
We also unify conversions from and to `Gitaly::User`s in the
`Gitlab::Git::User` class.
2017-10-23 17:31:48 -03:00
Bob Van Landuyt 705c15d7af Allow enabling the circuitbreaker using an env variable
That way we can enable the circuitbreaker for just one host at a time.
2017-10-23 13:53:49 +03:00
Bob Van Landuyt 591ee4e361 Perform the stat check multiple times when checking a storage
Instead of only checking once within a timeout, check multiple times
within a timeout.

That means with a timeout of 30 seconds and 3 retries. Each try would
be allowed 20 seconds.
2017-10-23 13:53:49 +03:00
Bob Van Landuyt 430e767139 Implement backoff for the circuitbreaker
The circuitbreaker now has 2 failure modes:

- Backing off: This will raise the `Gitlab::Git::Storage::Failing`
  exception. Access to the shard is blocked temporarily.
- Circuit broken: This will raise the
  `Gitlab::Git::Storage::CircuitBroken` exception. Access to the shard
  will be blocked until the failures are reset.
2017-10-23 12:02:23 +03:00
Rémy Coutable 89f5ede2a3 Merge branch 'bvl-do-not-use-redis-keys' into 'master'
Avoid using `Redis#keys`

See merge request gitlab-org/gitlab-ce!14889
2017-10-17 16:15:44 +00:00
Bob Van Landuyt c365dea887 Don't use `Redis#keys` in the circuitbreaker 2017-10-17 12:47:20 +02:00
Douwe Maan 4bbdab764d Merge branch 'bvl-circuitbreaker-improvements' into 'master'
Make the circuitbreaker configurable at runtime

See merge request gitlab-org/gitlab-ce!14842
2017-10-17 10:43:05 +00:00
Bob Van Landuyt 619021fd7a Read circuitbreaker settings from `Gitlab::CurrentSettings`
Instead of from the configuration file
2017-10-17 11:50:32 +02:00
Sean McGivern 980423f6da Merge branch 'git-env-handle-arrays' into 'master'
Add Gitlab::Git::Env.to_env_hash

See merge request gitlab-org/gitlab-ce!14841
2017-10-16 12:37:02 +00:00
Jacob Vosmaer e73af42239 Add Gitlab::Git::Env.to_env_hash 2017-10-16 13:19:44 +02:00
Sean McGivern b2553840e8 Merge branch 'conflict-resolution-refactor' into 'master'
Conflict resolution refactor

See merge request gitlab-org/gitlab-ce!14747
2017-10-16 10:36:06 +00:00
Andrew Newdigate f530261773 Popen with a timeout 2017-10-16 09:11:31 +00:00
Jacob Vosmaer 0aff29f96b Merge Merge Requests via Gitaly 2017-10-13 14:07:31 +02:00
Alejandro Rodríguez faa9bd402d Create a Gitlab::Git submodule for conlict-related files
Rename classes to (hopefully) clearer names while we're doing that.
2017-10-12 22:03:15 -03:00
Alejandro Rodríguez 3fcab51ebb Refactor conflict resolution to contain git ops within Gitlab::Git
This prepares the codebase for a Gitaly migration. See
https://gitlab.com/gitlab-org/gitaly/issues/553
2017-10-12 22:03:14 -03:00
Douwe Maan 05f46b820e Merge branch 'fix/rugged-alternate-paths' into 'master'
Fix the format of rugged alternate directory list

Closes #39046

See merge request gitlab-org/gitlab-ce!14840
2017-10-12 15:37:14 +00:00
Dmitriy Zaporozhets 400557f0ba Merge branch 'gitaly-fetch-remote' into 'master'
Add `Gitlab::Git::Repository#fetch` command

Closes gitaly#586

See merge request gitlab-org/gitlab-ce!14772
2017-10-12 14:53:32 +00:00
Ahmad Sherif a24abf39bf Fix the format of rugged alternate directory list
Fixes #39046
2017-10-12 14:20:04 +02:00
Ahmad Sherif 4378f56c7b Pass git object dir attributes as relative paths to Gitaly
Fixes gitaly#629
2017-10-11 15:46:55 +02:00
Alejandro Rodríguez 1731934398 Add `Gitlab::Git::Repository#fetch` command 2017-10-09 18:25:24 -03:00
Jacopo 0ce6785851 Replaces `tag: true` into `:tag` in the specs
Replaces all the explicit include metadata syntax in the specs (tag:
true) into the implicit one (:tag).
Added a cop to prevent future errors and handle autocorrection.
2017-10-07 13:57:54 +02:00
Jacob Vosmaer 147e2b21be Let fetch_ref pull from Gitaly instead of from disk 2017-10-04 12:57:41 +02:00
Douwe Maan c6e5a77e51 Merge branch 'gitaly-delete-branch' into 'master'
Implement OperationService.UserDeleteBranch Gitaly RPC

See merge request gitlab-org/gitlab-ce!14603
2017-10-04 07:34:33 +00:00
Alejandro Rodríguez 79719cf003 Add OperationService.UserDeleteBranch Gitaly RPC 2017-10-03 21:06:34 -03:00
Jacob Vosmaer (GitLab) bfd53aa6d0 Restore User.from_gitaly 2017-10-03 16:53:11 +00:00
Douwe Maan 4716e81f9d Merge branch 'revert-configurable-size-limits' into 'master'
Revert "Increase diff limits to 100 KB for collapse and 200 KB overall"

See merge request gitlab-org/gitlab-ce!14628
2017-10-02 14:39:55 +00:00
Sean McGivern a5d47bb063 Revert "Increase diff limits to 100 KB for collapse and 200 KB overall"
This reverts commit 1d3c33b57e.
2017-10-02 13:54:40 +01:00
Sean McGivern 0a69abdb28 Merge branch 'gitaly-add-branch' into 'master'
Implement OperationService.UserAddBranch Gitaly RPC

Closes gitaly#544

See merge request gitlab-org/gitlab-ce!14522
2017-10-02 09:27:23 +00:00
Alejandro Rodríguez a97ff8aae0 Add a factory for `Gitaly::GitCommit`s 2017-09-30 00:51:16 -03:00
David Turner dbcf48af8b Add username as GL_USERNAME in hooks (http)
When calling pre-receive, post-receive, and update hooks, add the GitLab
username as the GL_USERNAME environment variable.

This patch only handles cases where pushes are over http, or via
the web interface.  Later, we will address the ssh case.
2017-09-29 18:12:03 -04:00
Jacob Vosmaer (GitLab) 403712f06e Make Repository#has_visible_content more efficient 2017-09-29 13:08:44 +00:00
Sean McGivern b63e8d64c1 Handle error when fetching ref for MR with deleted source branch
If the ref doesn't exist, and the source branch is deleted, we can't get it back
easily. Previously, we ignored this error by shelling out, so replicate that
behaviour.
2017-09-28 11:11:10 +01:00
Kim "BKC" Carlbäcker 9c16958c30 Migrate Gitlab::Git::Repository#log to Gitaly 2017-09-25 11:34:10 +02:00
Douwe Maan 7cf8e0981a Merge branch '36549-circuit-breaker-handles-missing-storages' into 'master'
Allow the git circuit breaker to correctly handle missing repository storages

Closes #36549

See merge request gitlab-org/gitlab-ce!14417
2017-09-22 20:24:22 +00:00
Douwe Maan 408c595292 Merge branch 'sh-blob-raw-check' into 'master'
Ensure that Blob.raw returns always returns a valid blob object

See merge request gitlab-org/gitlab-ce!14412
2017-09-22 10:40:27 +00:00
Nick Thomas ba0ebbb510 Allow the git circuit breaker to correctly handle missing repository storages 2017-09-22 10:05:28 +01:00
Stan Hu 794b4c5579 Ensure that Blob.raw returns always returns a valid blob object
In gitlab-org/gitlab-ee!2976, we saw that a given OID could point
to a commit, which would cause the delta size check to fail.
Gitaly already returns nil if the OID isn't a blob, so this change
makes the Rugged implementation consistent.
2017-09-21 14:49:51 -04:00
Robin Bobbitt fbe205f92f Display full pre-receive and post-receive hook output in GitLab UI 2017-09-21 09:13:00 -04:00
Alejandro Rodríguez 34eeac6108 Use Gitlab::Git's Popen on that module's code
This allows the current Gitaly migration to depend on less code outside
of the Gitlab::Git module
2017-09-20 13:21:54 +02:00
Jacob Vosmaer 9e40baba1e Rename Gitlab::Git::Committer to User 2017-09-14 17:42:02 +02:00
Kim "BKC" Carlbäcker 25c34608b9 Migrate Git::CommitStats to Gitaly 2017-09-11 21:00:58 +02:00
Alejandro Rodríguez c2e99b40f7 Implement fix for n+1 issue on `flatten_tree` helper 2017-09-06 17:47:25 -03:00
Robert Speicher 86cbf60cbb Merge branch 'feature/migrate-branch-operations-to-gitaly' into 'master'
Migrate creating/deleting a branch to Gitaly

See merge request !13864
2017-09-06 17:26:05 +00:00
Ahmad Sherif 41ef94e777 Migrate creating/deleting a branch to Gitaly 2017-09-06 17:25:17 +02:00
micael.bergeron 46f6092a6d [ci-skip] spec for an instance method 2017-09-06 09:44:12 -04:00
micael.bergeron 5625b0e453 add .json_safe_diff spec 2017-09-06 09:01:53 -04:00
Ahmad Sherif e10437de40 Migrate Gitlab::Git::Repository#find_branch to Gitaly 2017-09-05 17:29:45 +00:00
Lin Jen-Shin c5553ce772 Use `git update-ref --stdin -z` to delete refs 2017-08-30 22:01:23 +08:00
Robert Speicher 51ceacb1fc Add missing third argument to `Git::Repository#initialize` in spec 2017-08-24 17:34:05 -04:00
Sean McGivern e46a3d2fda Merge branch 'git-operation-user' into 'master'
Move GitHooksService into Gitlab::Git

See merge request !13739
2017-08-24 14:32:17 +00:00
Andrew Newdigate fb49c94e49 Delegate Repository::branch_exists? and ref_exists? to Gitlab::Git 2017-08-24 09:20:04 +00:00
Douwe Maan 93ad83443a Merge branch 'sh-fix-branch-count' into 'master'
Fix inconsistent number of branches when remote branches are present

Closes #36934

See merge request !13784
2017-08-24 07:42:30 +00:00
Stan Hu 37904108b9 Fix inconsistent number of branches when remote branches are present
Users of project mirrors would see that the number of branches did not
match the number in the branch dropdown because remote branches were
counted when Rugged was in use. With Gitaly, only local branches
are counted.

Closes #36934
2017-08-23 22:07:32 -07:00
Jacob Vosmaer 961da7d0a7 Add tests for Committer#== 2017-08-23 17:23:28 +02:00
Jacob Vosmaer c47b947a73 Move GitHooksService tests 2017-08-23 10:45:20 +02:00
Jacob Vosmaer 65f83941c3 Make gl_repository a G::G::Repository attribute 2017-08-23 10:45:20 +02:00
Robert Speicher a6fbd8648e Stub `ForkedStorageCheck.storage_available?` by default in all specs
Add `:broken_storage` metadata to examples to disable this behavior only
when necessary.
2017-08-22 13:06:25 -04:00
Jacob Vosmaer faaed0aa1f Delete unused Gitlab::Git methods 2017-08-18 14:12:17 +02:00
Andrew Newdigate aaa887feba Client Implementation: RefService::RefExists 2017-08-18 09:43:58 +01:00
Rémy Coutable 4e0ee10a72 Merge branch 'bvl-circuitbreaker-settings-to-avoid-failures' into 'master'
Circuitbreaker settings to avoid spec-failures

Closes #36324

See merge request !13519
2017-08-16 13:38:00 +00:00
Bob Van Landuyt ee603a0089 Allow a `failure_wait_time` of 0 for storage access
This allows testing every storage attempt after a failure. Which could
be useful for tests
2017-08-16 14:15:50 +02:00
Douwe Maan 0da644072f Merge branch 'malformed-submodule' into 'master'
Handle missing .gitmodules when getting submodule urls

See merge request !13509
2017-08-15 09:00:44 +00:00
Douwe Maan a64760d6d0 Merge branch 'gitaly-404-commit-list-files' into 'master'
Migrate `Git::Repository.ls_files` to Gitaly

Closes gitaly#404

See merge request !13302
2017-08-14 15:44:36 +00:00
Jacob Vosmaer d9b6fd4fba Handle missing .gitmodules when getting submodule urls 2017-08-14 10:53:02 +02:00
Rémy Coutable 1b413bd62f Enable Timecop safe mode
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-11 14:53:17 +02:00
Robert Speicher c8b802471b Enable the RSpec/HookArgument cop and auto-correct offenses 2017-08-10 19:29:42 -04:00
Kim "BKC" Carlbäcker 7b10885046 Migrate Git::Repository.ls_files to Gitaly 2017-08-10 16:10:47 +02:00
Rémy Coutable c946ee1282
Enable the Layout/SpaceBeforeBlockBraces cop
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-09 11:52:22 +02:00
Alejandro Rodríguez 3ce6f03f14 Incorporate Gitaly's CommitService.FindCommit RPC 2017-08-07 23:33:43 -04:00
Alejandro Rodríguez e363fbf71a Move `deltas` and `diff_from_parents` logic to Gitlab::Git::Commit
This helps keep the abstraction layers simpler, and also keep the
interface of those methods consistent, in case of implementation
changes.
2017-08-07 23:33:40 -04:00
Alejandro Rodríguez c21ae07e33 Refactor Gitlab::Git::Commit to include a repository 2017-08-07 22:34:34 -04:00
Douwe Maan 942bd5b411 Merge branch 'blob-batch-methods' into 'master'
Add Gitlab::Git::Blob.batch method

See merge request !13225
2017-08-07 19:57:25 +00:00
Jacob Vosmaer 29a1c5a126 Rename 'limit' to 'blob_size_limit' 2017-08-07 16:26:50 +02:00
Douwe Maan 727b6d16e5 Merge branch 'master' into 'bvl-nfs-circuitbreaker'
# Conflicts:
#   app/models/repository.rb
#   spec/models/repository_spec.rb
2017-08-07 09:13:10 +00:00
Sean McGivern 9b21668663 Merge branch 'gitaly-renames-annotations' into 'master'
Gitaly deletions and annotations

See merge request !13280
2017-08-07 08:40:53 +00:00
Bob Van Landuyt 4b34720c0c Use ruby's `File.stat` to check storage availability 2017-08-04 15:38:50 +02:00
Bob Van Landuyt 0dd4c306ca Only track accessibility once 2017-08-04 15:38:50 +02:00
Bob Van Landuyt 022c38e63e Use `keys` instead of `scan_each` 2017-08-04 15:38:49 +02:00
Bob Van Landuyt 3899d07f9e Move hostname to Gitlab::Environment 2017-08-04 15:38:49 +02:00
Bob Van Landuyt 0fa94a0ddd Make sure the check works for paths with spaces. 2017-08-04 15:38:49 +02:00
Bob Van Landuyt 3598e60bf2 Add a Circuitbreaker for storage paths 2017-08-04 15:38:48 +02:00
Ahmad Sherif 03440eed20 Migrate blame loading to Gitaly
Closes gitaly#421
2017-08-03 19:20:46 +02:00
Jacob Vosmaer 3a8f4a3460 Remove unnecessary 'raw' alias 2017-08-03 15:52:14 +02:00
Jacob Vosmaer 29a07fd3a1 Rename 'commits_between' to 'rugged_commits_between' 2017-08-03 15:48:10 +02:00
Ahmad Sherif 215e0911e5 Migrate Gitlab::Git::Repository#count_commits to Gitaly
Closes gitaly#415
2017-08-02 00:33:04 +02:00
Sean McGivern 7a2509ea1f Merge branch 'blob-limit-test' into 'master'
Fix bug in blob test

See merge request !13224
2017-08-01 12:53:57 +00:00
Jacob Vosmaer f3569a25e4 Fix bug in blob test 2017-08-01 13:53:32 +02:00
Jacob Vosmaer 5e20e448ce Add Gitlab::Git::Blob.batch method 2017-08-01 13:49:57 +02:00
Jacob Vosmaer 67de82cf5f Add option to use CommitLanguages RPC 2017-08-01 10:48:46 +02:00