Commit Graph

432 Commits

Author SHA1 Message Date
Nick Thomas 65dbead796
Fix repository archive generation when hashed storage is enabled 2018-06-05 20:44:07 +01:00
Jacob Vosmaer (GitLab) a0808df0b6 Find and mark more Git disk access locations 2018-06-05 15:51:14 +00:00
Jacob Vosmaer (GitLab) 3a4ecfd0c9 Prevent Gitaly WriteConfig log noise 2018-06-05 12:55:41 +00:00
Jacob Vosmaer (GitLab) edb9db37ed Add "deny disk access" Gitaly feature (tripswitch) 2018-06-01 11:56:29 +00:00
Kim "BKC" Carlbäcker 0db994fedc Fix encoding error in Gitaly::Commit::TreeEntry 2018-05-30 17:33:17 +02:00
Grzegorz Bizon b113330fab Merge branch 'zj-workhorse-archive-mandatory' into 'master'
Move git archives downloading to Gitaly

See merge request gitlab-org/gitlab-ce!18990
2018-05-21 07:08:58 +00:00
Grzegorz Bizon ad9e00917f Merge branch 'xeodon/gitlab-ce-fix/45743-master-fix-gitaly-delete-refs' into 'master'
Fix error when deleting an empty list of refs

Closes #45743

See merge request gitlab-org/gitlab-ce!19053
2018-05-19 05:42:16 +00:00
Sergey Sinev 100c687cbc Fix error when deleting an empty list of refs
Closes #45743
2018-05-18 20:47:46 -07:00
Robert Speicher 076ab2e8e0 Merge branch 'fix/handle-large-commit-tag-message' into 'master'
Add handling for commit/tags with big messages

See merge request gitlab-org/gitlab-ce!17892
2018-05-18 19:28:25 +00:00
Ahmad Sherif 019f5e2469 Add handling for commit/tags with big messages 2018-05-18 20:51:10 +02:00
Nick Thomas 8bacfbd1cc Merge branch 'zj-calculate-checksum-mandator' into 'master'
Calculating repository checksums executed by Gitaly

Closes gitaly#1105 and #46293

See merge request gitlab-org/gitlab-ce!18907
2018-05-18 15:53:35 +00:00
Zeger-Jan van de Weg 18a8eb96b3
Calculating repository checksums executed by Gitaly
OPT_OUT status has been removed, and alternative implementation removed.
Also checks if the repository exists before executing the checksum RPC
to guard against NotFound errors.

Closes gitlab-org/gitaly#1105
2018-05-18 16:07:29 +02:00
Grzegorz Bizon 48877dfc7e Merge branch 'zj-workhorse-commit-patch-diff' into 'master'
Workhorse to send raw diff and patch for commits

Closes gitaly#1196

See merge request gitlab-org/gitlab-ce!18974
2018-05-17 08:12:33 +00:00
Zeger-Jan van de Weg 94b209b32c
Move git archives downloading to Gitaly 2018-05-17 09:31:50 +02:00
Zeger-Jan van de Weg dfdd881510
Workhorse to send raw diff and patch for commits
Prior to this change, this was done through unicorn. In theory this
could time out. Workhorse has been sending these raw patches and diffs
for a long time and is stable in doing so.

Added bonus is the fact that `Commit#to_patch` can be removed.

`Commit#to_diff` too, which closes
https://gitlab.com/gitlab-org/gitaly/issues/324

Closes https://gitlab.com/gitlab-org/gitaly/issues/1196
2018-05-16 11:52:43 +02:00
Zeger-Jan van de Weg 7fc185ab25
Ref contains oid check done by Gitaly
These two endpoints are in opt-out, and no one noticed. Has been running
on .com for 4 months now and considered stable.

Closes https://gitlab.com/gitlab-org/gitaly/issues/884
2018-05-14 15:55:27 +02:00
Rubén Dávila 5b584a0fd2 Backport some changes from gitlab-ee!5476
The lib/gitlab/git/repository.rb needs to have the same content between
gitlab-ce and gitlab-ee in order to have Gitaly working fine.
2018-05-09 13:20:28 -05:00
Douglas Barbosa Alexandre 17258b37c8
Raise InvalidRepository error for non-valid git repositories 2018-05-07 14:22:16 -03:00
Zeger-Jan van de Weg 713c7fa9a5
Client implementation GetRawChanges
Part of https://gitlab.com/gitlab-org/gitaly/issues/1138
2018-05-07 15:37:49 +02:00
Douwe Maan 9cf4e47341 Merge branch 'security-45689-fix-archive-cache-bug' into 'security-10-7'
Serve archive requests with the correct file in all cases (10.7)

See merge request gitlab/gitlabhq!2376
2018-04-30 15:00:14 -05:00
Zeger-Jan van de Weg ec4423665c
Gitlab::Shell works on shard name, not path
Direct disk access is done through Gitaly now, so the legacy path was
deprecated. This path was used in Gitlab::Shell however. This required
the refactoring in this commit.

Added is the removal of direct path access on the project model, as that
lookup wasn't needed anymore is most cases.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1111
2018-04-25 13:36:22 +02:00
Francisco Javier López c921e686b8 Moved committer and spec. Added some extra code to run hooks or not depending on the options 2018-04-23 10:56:46 +00:00
Francisco Javier López e8a27a67fa Fix Custom hooks are not triggered by UI wiki edit 2018-04-18 17:50:56 +00:00
Stan Hu 74e5ec198c Fix failing ./spec/lib/backup/repository_spec.rb by clearing the memoized value 2018-04-14 15:36:36 -07:00
Stan Hu b1f15dfa42 Memoize Git::Repository#has_visible_content?
This is called repeatedly when viewing a merge request, and this should
improve performance significantly by avoiding shelling out to git every time.

This should help https://gitlab.com/gitlab-com/infrastructure/issues/4027.
2018-04-14 14:37:28 -07:00
Douwe Maan db6854b2cd Merge branch 'backport-of-rd-3429-enabling-maximum-file-size-limit-in-repository-causes-pushes-to-fail' into 'master'
Backport of EE !4989

See merge request gitlab-org/gitlab-ce!18238
2018-04-12 08:46:36 +00:00
Rubén Dávila d28b1dfc46 Backport of EE !4989 2018-04-11 22:05:07 -05:00
Zeger-Jan van de Weg c830b8e3b7
Client implementation for InfoAttributes
Clients can now request the attributes from `$GIT_DIR/info/attributes`
through Gitaly. The Gitaly migration is described in gitlab-org/gitaly#1082.

The parser algorithm was implemented in a way it could handle both file
contents or a File handle, and both were already tested.

Other than that, using the boy scout rule, I've removed a class,
InfoAttributes, as it was delegating everything to the parser and
therefor wasn't really needed in my opinion.
2018-04-11 14:21:27 +02:00
Douglas Barbosa Alexandre 76cd2d7881
Move checksum calculation to Gitlab::Git::Repository 2018-04-06 13:25:07 -03:00
Douglas Barbosa Alexandre 13992ac92e
Checksum calculation is handled by Gitaly when feature is enabled 2018-04-06 13:23:23 -03:00
Stan Hu a18eea8c32 Automatically cleanup stale worktrees and lock files upon a push
git 2.16 will fail badly if there are stale worktrees.

Closes #44115
2018-04-06 08:17:07 -07:00
James Ramsay 07f517d441 Add new repository archive route
Repository archives are always named `<project>-<ref>-<sha>` even if
the ref is a commit. A consequence of always including the sha even
for tags is that packaging a release is more difficult because both
the ref and sha must be known by the packager.

- add `<project>/-/archive/<ref>/<filename>.<format>` route using the
`-` separator to prevent namespace collisions. If the filename is
`<project>-<ref>` or the ref is a sha, the sha will be omitted,
otherwise the default filename will be used.
- deprecate previous archive route `repository/<ref>/archive`
2018-04-06 08:45:17 -04:00
James Ramsay 0b1b9c409d Add option to suppress archive commit sha
Repository archives are always named `<project>-<ref>-<sha>` even if
the ref is a commit. A consequence of always including the sha even
for tags is that packaging a release is more difficult because both
the ref and sha must be known by the packager.

- add append_sha option (defaults true) to provide a method for
toggling this feature.

Support added to GitLab Workhorse by gitlab-org/gitlab-workhorse!232
2018-04-04 16:56:18 -04:00
Sean McGivern c353426650 Merge branch 'fix/use-shard-name-in-gitlab-projects-instead-of-shard-path' into 'master'
Use shard name in Git::GitlabProjects instead of shard path

Closes gitaly#1110

See merge request gitlab-org/gitlab-ce!18015
2018-04-04 09:20:19 +00:00
Douglas Barbosa Alexandre 563d38e81e
Backport Gitlab::Git::Checksum to CE 2018-04-03 15:36:43 -03:00
Ahmad Sherif ddfc661f79 Use shard name in Git::GitlabProjects instead of shard path
Closes gitaly#1110
2018-04-03 16:22:13 +02:00
Stan Hu 6fd1d07cfc Handle CR-LFs properly in .gitmodules file
Any submodule lines that had a CR-LF ending would be ignored by the parser.

Closes gitlab-org/gitlab-ce#2262
2018-03-30 15:23:44 -07:00
Jacob Vosmaer (GitLab) b942462788 Remove support for absolute dirs from Git::Env 2018-03-30 09:19:46 +00:00
Sean McGivern c113b70057 Merge branch 'dm-refs-contains-sha-encoding' into 'master'
Fix listing commit branch/tags that contain special characters

Closes #43212

See merge request gitlab-org/gitlab-ce!18023
2018-03-28 09:02:09 +00:00
Douwe Maan 39c911eaaa Fix listing commit branch/tags that contain special characters 2018-03-27 18:20:08 +02:00
Sean McGivern 70af1e2e03 Fix 500 error when trying to resolve non-ASCII conflicts in editor
When we added caching, this meant that calling `can_be_resolved_in_ui?` didn't
always call `lines`, which meant that we didn't get the benefit of the
side-effect from that, where it forced the conflict data itself to UTF-8.

To fix that, make this explicit by separating the `raw_content` (any encoding)
from the `content` (which is either UTF-8, or an exception is raised).
2018-03-23 18:33:14 +00:00
Zeger-Jan van de Weg 77f0906e4c
Change Gitlab::Shell#add_namespace to #create_namespace
Prior to this change, this method was called add_namespace, which broke
the CRUD convention and made it harder to grep for what I was looking
for. Given the change was a find and replace kind of fix, this was
changed without opening an issue and on another feature branch.

If more dynamic calls are made to add_namespace, these could've been
missed which might lead to incorrect bahaviour. However, going through
the commit log it seems thats not the case.
2018-03-14 09:56:22 +01:00
Nick Thomas 91ff0eaa13
Respect the configured git path for imported gitlab-shell operations 2018-03-12 16:03:00 +00:00
Nick Thomas 0649a1f835 Merge branch 'sh-make-prune-optional-in-git-fetch' into 'master'
Make --prune a configurable parameter in fetching a git remote

See merge request gitlab-org/gitlab-ce!17346
2018-03-07 02:02:24 +00:00
Rémy Coutable af84ccbd82 Merge branch '40187-project-branch-dashboard-with-active-stale-branches' into 'master'
Add overview of branches and filtered views for active/stale branches

Closes #40187 et #17293

See merge request gitlab-org/gitlab-ce!15402
2018-03-06 13:14:10 +00:00
Takuya Noguchi 580d895363 Add overview of branches and a filter for active/stale branches 2018-03-06 21:28:14 +09:00
Stan Hu e9fad3e501 Make --prune a configurable parameter in fetching a git remote
By default, --prune is added to the command-line of a `git fetch` operation,
but for repositories with many references this can take a long time to run. We
shouldn't need to run --prune the first time we fetch a new repository.
2018-03-06 00:10:04 -08:00
Douwe Maan 0736197f63 Merge branch 'gitaly-lfs-changes' into 'master'
Incorporate Gitaly's RPCs for Gitlab::Git::LfsChanges

Closes gitaly#1048

See merge request gitlab-org/gitlab-ce!16973
2018-03-05 15:43:26 +00:00
Douwe Maan 899967fbd8 Merge branch 'feature/add-support-for-all-option-in-count-find-commits' into 'master'
Add support for :all option to {count,find}_commits

See merge request gitlab-org/gitlab-ce!17464
2018-03-05 15:41:49 +00:00
Alejandro Rodríguez 0a4ee10eda Fix n+1 issue by not reloading fully loaded blobs 2018-03-05 11:14:43 -03:00
Alejandro Rodríguez a0a7b551ae Incorporate Gitaly's RPCs for Gitlab::Git::LfsChanges 2018-03-02 17:12:08 -03:00
Ahmad Sherif cd770946c4 Add support for :all option to {count,find}_commits 2018-03-02 14:55:20 +01:00
Sean McGivern 40c61acb6a Merge branch 'feature/fetch-commit-signs-from-gitaly-in-batch' into 'master'
Fetch commit signatures from Gitaly in batches

Closes gitaly#1046

See merge request gitlab-org/gitlab-ce!17456
2018-03-02 11:16:20 +00:00
Ahmad Sherif 03f3350f3f Fetch commit signatures from Gitaly in batches
Closes gitaly#1046
2018-03-01 18:31:31 +01:00
Sean McGivern 52e133d176 Merge branch 'zj-licensee-key' into 'master'
Client implementation for Licensee#key

See merge request gitlab-org/gitlab-ce!17449
2018-03-01 17:10:39 +00:00
Douwe Maan de454de9b1 Merge branch '42434-allow-commits-endpoint-to-work-over-all-commits' into 'master'
Resolve "Allow API method /projects/:id/repository/commits  to work over all commits"

Closes #42434

See merge request gitlab-org/gitlab-ce!17182
2018-03-01 15:54:43 +00:00
Zeger-Jan van de Weg 34fbade4ae
Client implementation for Licensee#key
Part of the migration as tracked in: gitlab-org/gitaly#1026
2018-03-01 16:41:08 +01:00
Tiago Botelho cd9daf644e Add specs 2018-03-01 14:38:35 +00:00
Zeger-Jan van de Weg c12b18c545
Encode revision for gitattributes ref
A field didn't call the needed encoding helper, thus some UTF-8 encoding
couldn't be encoded to ASCII. Using the helper method this was fixed.

Tests are now run against Gitaly and Rugged too, to ensure both remain
working correctly.

Fixes gitlab-org/gitaly#1032, gitlab-org/gitlab-ce#43278
2018-03-01 09:18:26 +01:00
Sean McGivern e351f67874 Suppress whitespace warnings in squash error messages
These are obscuring the real error, which is confusing for everyone.
2018-02-26 09:08:17 +00:00
Grzegorz Bizon eb421c88ee Merge branch 'fix-squash-with-renamed-files' into 'master'
Fix squash with renamed files

Closes gitlab-ee#4975

See merge request gitlab-org/gitlab-ce!17216
2018-02-22 11:35:34 +00:00
Douwe Maan e503efa320 Merge branch 'zj-branch-contains-git-message' into 'master'
Allow branchnames to be named the same as the commit it points to

See merge request gitlab-org/gitlab-ce!17231
2018-02-22 11:34:03 +00:00
Zeger-Jan van de Weg e70fe78281
Handle branch and tag names which are commit ids
Adds a test where a branch name is also a valid commit id. Git, the
binary should create an error message which is difficult to parse and
leading to errors later, as seen in: gitlab-org/gitlab-ce#43222

To catch these cases in the future,
gitlab-test@1942eed5cc108b19c7405106e81fa96125d0be22 was created. Which
a branch name matching the commit
2018-02-21 19:08:26 +01:00
Ken ed0a029d53 26039 - update github-linguist to 5.3.3 version
26039 - add merge request id


26039 change due to lingust source file updates
2018-02-21 03:29:04 +10:30
Zeger-Jan van de Weg f42bf66eef
Add unit tests for Gitaly types 2018-02-20 13:57:33 +01:00
Sean McGivern 01f5035bfe Fix squash with renamed files
We need to ignore the names for renamed files when configuring with sparse
checkout.
2018-02-19 15:11:10 +00:00
Stan Hu 8d32dfef9b Fix squash rebase not working when diff contained encoded data
When the applied diff contains UTF-8 or some other encoded data, the diff
returned back from the git process may be in ASCII-8BIT format. Writing this
data to stdin may fail if the data because stdin expects this data to be in
UTF-8. By switching the output to binmode, we ensure that the diff will
always be written as-is.

Closes gitlab-org/gitlab-ee#4960
2018-02-18 21:48:49 -08:00
Ahmad Sherif dabc703a29 Migrate Git::Blob.batch to Gitaly
Closes gitaly#985
2018-02-08 00:26:45 +01:00
Francisco Javier López d40912bb49 Removing gitaly flags 2018-02-07 16:33:12 +00:00
Sean McGivern 683984f2e3 Merge branch 'zj-refs-hash' into 'master'
Don't use rugged in Repository#refs_hash

Closes gitaly#880

See merge request gitlab-org/gitlab-ce!16827
2018-02-07 15:42:55 +00:00
Zeger-Jan van de Weg 73e78c4e15
Don't use rugged in Repository#refs_hash
The refs hash is used to determine what branches and tags have a commit
as head in the network graph. The previous implementation depended on
Rugged#references. The problem with this implementation was that it
depended on rugged, but also that it iterated over all references and
thus loading more data than needed if for example the project uses CI/CD
environments, Pipelines, or Merge Requests.

Given only refs are checked the network cares about the GraphHelper#refs
method has no need to reject those, simplifying the method.

Closes gitlab-org/gitaly#880
2018-02-07 15:07:03 +01:00
Robert Speicher 245d937514 Merge branch 'gitaly-lfs-client-prep' into 'master'
Refactor Gitlab::Git code related to LFS changes for Gitaly migration

Closes gitaly#980

See merge request gitlab-org/gitlab-ce!16835
2018-02-06 17:35:08 +00:00
Sean McGivern f81306e76f Merge branch 'jej/upload-file-tracks-lfs' into 'master'
File upload UI obeys LFS filter

Closes #29876

See merge request gitlab-org/gitlab-ce!16412
2018-02-06 16:24:54 +00:00
James Edwards-Jones d2a77094ae File upload UI obeys LFS filters
Uses Lfs::FileModificationHandler to coordinate LFS detection, creation of LfsObject, etc

Caveats:
 1. This isn't used by the multi-file editor / Web IDE
 2. This isn't used on rename. We'd need to be able to download LFS files
    and add them to the commit if they no longer match so not as simple.
 3. We only check the root .gitattributes file, so this should be improved
    to correctly check for nested .gitattributes files in subfolders.
2018-02-06 15:26:25 +00:00
Francisco Javier López 27c08a1689 Allow moving wiki pages from the UI 2018-02-05 17:17:21 +00:00
Rémy Coutable f5990e444a Merge branch 'gitaly-user-squash' into 'master'
Incorporate OperationService.UserSquash Gitaly RPC

Closes gitaly#991

See merge request gitlab-org/gitlab-ce!16899
2018-02-05 10:32:20 +00:00
Grzegorz Bizon 5aea8dc1a6 Merge branch 'fix/remove-duplicated-logic-between-model-and-lib-in-find-branch' into 'master'
Remove repo reloading logic from Repository#find_branch

Closes #42609

See merge request gitlab-org/gitlab-ce!16815
2018-02-03 21:47:28 +00:00
Alejandro Rodríguez e60bf2f256 Incorporate OperationService.UserSquash Gitaly RPC 2018-02-02 19:02:40 -03:00
Ahmad Sherif 05f17e4d72 Remove repo reloading logic from Repository#find_branch
Gitlab::Git::Repository#find_branch has a similar logic.

Fixes #42609
2018-02-02 22:24:54 +01:00
Alejandro Rodríguez 98affa75ed Refactor Gitlab::Git code related to LFS changes for Gitaly migration
We stop relying on Gitlab::Git::Env for the RevList class, and use
Gitlab::Git::Repository#run_git methods inteaad. The refactor also fixes
another issue, since we now top using "path_to_repo" (which is a
Repository model method).
2018-02-02 16:27:01 -03:00
Alejandro Rodríguez 658749ddd3 Fix an issue where sparse checkout wasn't configured correctly
This was affecting the performance of `Gitlab::Git::Repository#squash`.
2018-02-01 17:07:40 -03:00
Robert Speicher 2793e3e1cc Merge branch 'gitaly-970-repo-write-config' into 'master'
Migrate Gitlab::Git::Repository#write_config to Gitaly

Closes gitaly#970

See merge request gitlab-org/gitlab-ce!16730
2018-02-01 19:46:32 +00:00
Rémy Coutable 76b9f1a4c5 Merge branch '42497-rubocop-style-regexpliteral' into 'master'
Enable Style/RegexpLiteral cop

Closes #42497

See merge request gitlab-org/gitlab-ce!16752
2018-02-01 09:11:09 +00:00
Douwe Maan 283c0a9d28 Merge branch 'repository-log-disable-walk' into 'master'
Remove Rugged::Walk implementation of #log

See merge request gitlab-org/gitlab-ce!16798
2018-01-31 21:54:02 +00:00
Takuya Noguchi 2b6307f6ad Enable RuboCop Style/RegexpLiteral 2018-02-01 02:06:07 +09:00
Kim "BKC" Carlbäcker 28bbb4cb47 Migrate Gitlab::Git::Repository#write_config to Gitaly
- Add tests for Repository#write_config
2018-01-31 15:02:31 +01:00
Sean McGivern a0fc0fcc53 Merge branch 'rugged-reference-cache-rescue' into 'master'
Stop rescuing Rugged::Reference in Repository cache

See merge request gitlab-org/gitlab-ce!16770
2018-01-31 14:00:37 +00:00
Jacob Vosmaer afe665d9c0 Remove Rugged::Walk implementation of #log 2018-01-31 11:56:36 +01:00
Ahmad Sherif 1ebb781f6a Migrate Git::Repository#delete_refs to Gitaly
Closes gitaly#967
2018-01-30 23:48:51 +01:00
Jacob Vosmaer a6dbb85e97 Remove Rugged exception in cache rescue 2018-01-30 17:51:56 +01:00
Jacob Vosmaer (GitLab) 5c2fe2d3d7 Decouple more of Repository from Rugged 2018-01-30 16:21:55 +00:00
Jacob Vosmaer (GitLab) e407c2d18f Don't allow Repository#log with limit zero 2018-01-29 15:27:46 +00:00
Ahmad Sherif 2290b438f4 Fix rubocop offenses introduced in !16623 2018-01-25 20:44:16 +01:00
Robert Speicher af95ba2fb4 Merge branch 'feature/migrate-repo-restorer-to-gitaly' into 'master'
Migrate restoring repo from bundle to Gitaly

Closes gitaly#946

See merge request gitlab-org/gitlab-ce!16623
2018-01-25 19:03:51 +00:00
Grzegorz Bizon 9df130ff22 Merge branch 'fix/batch-lfs-pointers-accepting-lazy-enum' into 'master'
Fix .batch_lfs_pointers accepting a lazy enumerator

See merge request gitlab-org/gitlab-ce!16701
2018-01-25 14:28:48 +00:00
Ahmad Sherif 07b0beb004 Fix .batch_lfs_pointers accepting a lazy enumerator 2018-01-25 14:11:27 +01:00
Douwe Maan eab9ffeff6 Merge branch 'revert-430b3f0e' into 'master'
Revert "Merge branch 'remove-rugged-walk' into 'master'"

See merge request gitlab-org/gitlab-ce!16676
2018-01-25 11:15:07 +00:00
Rémy Coutable d2bf53942f Merge branch 'jej/gitattributes-check-at-ref' into 'master'
Can parse root .gitattributes file for a ref

See merge request gitlab-org/gitlab-ce!16418
2018-01-25 11:01:36 +00:00
Jacob Vosmaer (GitLab) 7363aa0b9d Revert "Merge branch 'remove-rugged-walk' into 'master'"
This reverts merge request !16525
2018-01-25 11:03:29 +01:00
Ahmad Sherif 29fa930baf Migrate .batch_lfs_pointers to Gitaly
Closes gitaly#921
2018-01-24 19:02:51 +01:00