Commit Graph

1040 Commits

Author SHA1 Message Date
Stan Hu f929411c6d Rename gl_project_name -> gl_project_path
Upgrade gitaly-proto to 1.10.0 to have this field.
2019-02-05 21:33:10 -08:00
Stan Hu 82b6e5378a Send project name with Gitaly repository requests
When hashed storage is in use, it's helpful to have the project
name associated with the request.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1394
2019-02-05 21:33:09 -08:00
Nick Thomas 067dc6540c
Validate bundle files before unpacking them 2019-01-10 21:33:36 +00:00
Francisco Javier López 5a3e6fdff9
Fixing image lfs bug and also displaying text lfs
This commit, introduced in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23812,
fixes a problem creating a displaying image diff notes when the image
is stored in LFS. The main problem was that `Gitlab::Diff::File` was
returning an invalid valid in `text?` for this kind of files.

It also fixes a rendering problem with other LFS files, like text
ones. They LFS pointer shouldn't be shown when LFS is enabled
for the project, but they were.
2018-12-27 16:51:07 +01:00
Kamil Trzciński f9fd9b1def Merge branch 'zj-pool-destruction' into 'master'
Leave object pools when destroying projects

Closes gitaly#1415

See merge request gitlab-org/gitlab-ce!23869
2018-12-19 13:03:54 +00:00
Zeger-Jan van de Weg 752e9c18a1
Leave object pools when destroying projects
This action doesn't lean on reduplication, so a short call can me made
to the Gitaly server to have the object pool remove its remote to the
project pending deletion.
https://gitlab.com/gitlab-org/gitaly/blob/f6cd55357/internal/git/objectpool/link.go#L58

When an object pool doesn't have members, this would invalidate the need
for a pool. So when a project leaves the pool, the pool will be
destroyed on the background.

Fixes: https://gitlab.com/gitlab-org/gitaly/issues/1415
2018-12-19 13:21:56 +01:00
Zeger-Jan van de Weg 50ce459118
Upgrade Gitaly to 1.9.0
This upgrade moves logic from gitlab-rails to Gitaly, which allowed code
removal too.
2018-12-17 09:26:09 +01:00
Zeger-Jan van de Weg 896c0bdbfb
Allow public forks to be deduplicated
When a project is forked, the new repository used to be a deep copy of everything
stored on disk by leveraging `git clone`. This works well, and makes isolation
between repository easy. However, the clone is at the start 100% the same as the
origin repository. And in the case of the objects in the object directory, this
is almost always going to be a lot of duplication.

Object Pools are a way to create a third repository that essentially only exists
for its 'objects' subdirectory. This third repository's object directory will be
set as alternate location for objects. This means that in the case an object is
missing in the local repository, git will look in another location. This other
location is the object pool repository.

When Git performs garbage collection, it's smart enough to check the
alternate location. When objects are duplicated, it will allow git to
throw one copy away. This copy is on the local repository, where to pool
remains as is.

These pools have an origin location, which for now will always be a
repository that itself is not a fork. When the root of a fork network is
forked by a user, the fork still clones the full repository. Async, the
pool repository will be created.

Either one of these processes can be done earlier than the other. To
handle this race condition, the Join ObjectPool operation is
idempotent. Given its idempotent, we can schedule it twice, with the
same effect.

To accommodate the holding of state two migrations have been added.
1. Added a state column to the pool_repositories column. This column is
managed by the state machine, allowing for hooks on transitions.
2. pool_repositories now has a source_project_id. This column in
convenient to have for multiple reasons: it has a unique index allowing
the database to handle race conditions when creating a new record. Also,
it's nice to know who the host is. As that's a short link to the fork
networks root.

Object pools are only available for public project, which use hashed
storage and when forking from the root of the fork network. (That is,
the project being forked from itself isn't a fork)

In this commit message I use both ObjectPool and Pool repositories,
which are alike, but different from each other. ObjectPool refers to
whatever is on the disk stored and managed by Gitaly. PoolRepository is
the record in the database.
2018-12-07 19:18:37 +01:00
Nick Thomas 9395d198f9
Use BFG object maps to clean projects 2018-12-06 18:58:00 +00:00
Bob Van Landuyt 67f2faa72d Don't specify `shell` for Gitaly write-ref
Use shelling out to git to write refs instead of rugged, hoping to
avoid creating invalid refs.

To update HEAD we switched to using `git symbolic-ref`.
2018-11-26 16:44:26 +01:00
Douwe Maan 4b7e9eaacd
Treat Repository instances with the same disk path as the same hash key 2018-11-26 11:15:18 +01:00
Douwe Maan 5f0e4040ce
Batch load only data from same repository when lazy object is accessed
By specifying `key`, we get a different lazy batch loader for each
repository, which means that accessing a lazy object from one repository
will only result in that repository's objects being fetched, not those
of other repositories, saving us some unnecessary Gitaly lookups.
2018-11-26 11:15:18 +01:00
Sean McGivern 2bc4a17dec Merge branch 'revert-fd6e3781' into 'master'
Revert "Merge branch 'revert-e2aa2177' into 'master'"

Closes #54267

See merge request gitlab-org/gitlab-ce!23237
2018-11-22 09:50:34 +00:00
Takuya Noguchi e902f462c2 Eliminate duplicated words
Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
2018-11-22 01:01:23 +09:00
Zeger-Jan van de Weg 16346eb5b9 Revert "Merge branch 'revert-e2aa2177' into 'master'"
This reverts merge request !23229
2018-11-20 12:29:56 +00:00
Sean McGivern 2742b871fe Revert "Merge branch 'zj-improve-gitaly-pb' into 'master'"
This reverts merge request !23140
2018-11-20 10:36:52 +00:00
Nick Thomas f1bc7b6eb5
SSH public-key authentication for push mirroring 2018-11-19 11:46:39 +00:00
Zeger-Jan van de Weg 35239a6aec
Show what RPC is called in the performance bar
Now only the data was shown of the service, which is not valueable at
times given some of those expose a lot of RPCs.
2018-11-16 12:16:25 +01:00
Stan Hu 6494467a19 Merge branch 'frozen-string-lib-gitlab-even-more' into 'master'
Enable even more frozen string in lib/gitlab

See merge request gitlab-org/gitlab-ce!22959
2018-11-15 19:50:53 +00:00
Oswaldo Ferreira 039df0267e Avoid Gitaly RPC errors when fetching diff stats 2018-11-14 17:33:30 -02:00
gfyoung ebf98f27c4 Enable even more frozen string in lib/gitlab
Enables frozen string for the following:

* lib/gitlab/fogbugz_import/**/*.rb
* lib/gitlab/gfm/**/*.rb
* lib/gitlab/git/**/*.rb
* lib/gitlab/gitaly_client/**/*.rb
* lib/gitlab/gitlab_import/**/*.rb
* lib/gitlab/google_code_import/**/*.rb
* lib/gitlab/gpg/**/*.rb
* lib/gitlab/grape_logging/**/*.rb
* lib/gitlab/graphql/**/*.rb
* lib/gitlab/graphs/**/*.rb
* lib/gitlab/hashed_storage/**/*.rb
* lib/gitlab/health_checks/**/*.rb

Partially address gitlab-org/gitlab-ce#47424.
2018-11-13 11:42:15 -08:00
Bob Van Landuyt 6fbdc5ed52 Apply patches when creating MR via email
This allows users to add patches as attachments to merge request
created via email.

When an email to create a merge request is sent, all the attachments
ending in `.patch` will be applied to the branch specified in the
subject of the email. If the branch did not exist, it will be created
from the HEAD of the repository.

When the patches could not be applied, the error message will be
replied to the user.

The patches can have a maximum combined size of 2MB for now.
2018-11-07 16:27:55 +01:00
Francisco Javier López 28cbb2acfe
Add submodule update API endpoint
This new endpoint allow users to update a submodule's reference.

The MR involves adding a new operation RPC operation in gitaly-proto
(see gitlab-org/gitaly-proto!233) and change Gitaly to use this
new version (see gitlab-org/gitaly!936).

See gitlab-org/gitlab-ce!20949
2018-11-07 14:03:30 +01:00
Nick Thomas 668e5d63fa
Stop Repository#fetch_remote from using Gitlab::Shell
This indirection doesn't provide any value, so remove it
2018-11-06 12:42:00 +00:00
Bob Van Landuyt 81f5955eb6 Move Repository#wrapped_gitaly_errors into concern
Having this in a concern allows us to reuse it for different single
purpose classes that call out to git without going through the
repository every time.
2018-10-30 15:53:46 +01:00
Tiago Botelho 084a8b6101
Adds tracing messages for slow git pushes
Whenever a git push takes more than 50 seconds the user
will receive a trace from each check performed along with
their timings
2018-10-25 10:10:45 +01:00
Alejandro Rodríguez c8ffd8b035 [ci skip] Fix documentation of Gitlab::Git::Diff#filter_diff_options flags 2018-10-17 17:39:16 -03:00
Nick Thomas c5bff77ea4
Remove a dependency on gitlab-gollum-lib
Inlining this code allows us to remove a dependency on gitlab_grit in
gitlab-ce. We can't stop maintaining gitlab_grit yet, since gitaly-ruby
still depends on this gem, but it moves us a step closer.
2018-10-16 12:53:30 +01:00
Nick Thomas c7be9f5cac
Remove a dependency on gitlab-gollum-lib
Removing this dependency also allows us to remove a transitive
dependency on gitlab_grit - which is the whole point of this exercise.

I don't think we can EOL gitlab_grit until it's removed as a dependency
from gitaly-ruby, but this at least gets it out of gitlab-ce.
2018-10-16 12:53:30 +01:00
Douwe Maan 3c15a20ba3 Merge branch 'bvl-merge-base-multiple-revisions' into 'master'
Allow getting the merge base of multiple revisions

Closes #52573

See merge request gitlab-org/gitlab-ce!22295
2018-10-15 16:18:21 +00:00
Zeger-Jan van de Weg 399056ed78 Remove dependencies on Linguist
This saves about 128 MB of baseline RAM usage per Unicorn and
Sidekiq process (!).

Linguist wasn't detecting languages anymore from CE/EE since
9ae8b57467. However, Linguist::BlobHelper
was still being depended on by BlobLike and others.

This removes the Linguist gem, given it isn't required anymore.
EscapeUtils were pulled in as dependency, but given Banzai depends on
it, it is now added explicitly.

Previously, Linguist was used to detect the best ACE mode. Instead,
we rely on ACE to guess the best mode based on the file extension.
2018-10-12 17:17:30 -07:00
Bob Van Landuyt 71d71afb3a Allow getting the merge base of multiple revisions
As we now support getting the merge base for multiple revisions in
gitaly, we can provide this functionality in our API
2018-10-12 11:41:22 +02:00
Zeger-Jan van de Weg 30b4ce940d
Remove Git circuit breaker
Was introduced in the time that GitLab still used NFS, which is not
required anymore in most cases. By removing this, the API it calls will
return empty responses. This interface has to be removed in the next
major release, expected to be 12.0.
2018-10-10 09:08:18 +02:00
Kamil Trzciński 9f9cac67ab Fix CE to EE merge (backport) 2018-10-05 17:39:42 +02:00
Kamil Trzciński 84c4e920cd Merge branch 'master' into 'feature/gb/pipeline-only-except-with-modified-paths'
# Conflicts:
#   app/models/ci/pipeline.rb
2018-10-04 16:52:48 +00:00
Douwe Maan a5cfacc281 Merge branch 'remove-rugged' into 'master'
Remove Gitlab::Git::Repository#rugged

See merge request gitlab-org/gitlab-ce!22039
2018-10-03 09:06:32 +00:00
Alejandro Rodríguez a99bf447a2 Remove Gitlab::Git::Repository#rugged and Gollum code
Cleanup code, and refactor tests that still use Rugged. After this, there should
be no Rugged code that access the instance's repositories on non-test
environments. There is still some rugged code for other tasks like the
repository import task, but since it doesn't access any repository storage path
it can stay.
2018-10-02 16:34:28 -03:00
Douwe Maan 57dc233325 Merge branch 'osw-configurable-single-diff-file-limit' into 'master'
Make single diff patch limit configurable

Closes #48027

See merge request gitlab-org/gitlab-ce!21886
2018-10-02 13:53:32 +00:00
Grzegorz Bizon 1c4187e38e Treat nil git push revisons as a blank Git SHA value 2018-10-02 13:01:55 +02:00
Oswaldo Ferreira 4fbca2a346 Make single diff patch limit configurable
- Creates a new column to hold the single patch limit value on
application_settings
- Allows updating this value through the application_settings API
- Calculates single diff patch collapsing limit based on
diff_max_patch_bytes column
- Updates diff limit documentation
- Adds documentation (with warning) as of how one can update this limit
2018-10-01 11:58:21 -03:00
Tiago Botelho a583f137dd Removes N+1 gitaly rpc call to fetch the last commit for path
Implements list_last_commits_for_tree to communicate with the
ListLastCommitsForTree Gitaly RPC

Bumps the Gitaly server version

Bumps the Gitaly-Proto gem version
2018-10-01 13:29:47 +01:00
Grzegorz Bizon 419d8cc7a2 Calculate modified paths of a git push operation 2018-09-28 15:13:04 +02:00
Grzegorz Bizon 4abba28944 Add specs for extracted git push class 2018-09-28 14:46:22 +02:00
Grzegorz Bizon 76d9e29a65 Extract git push from merge request refresh service 2018-09-28 12:20:51 +02:00
Grzegorz Bizon 3c2d6b4870 Add a class that represents a git push operation 2018-09-28 11:09:55 +02:00
Michael Kozono a54a5d9f39 Use `Gitlab::SafeRequestStore` in more places
Even if it doesn’t save lines of code, since people will tend to use
code they’ve seen. And `SafeRequestStore` is safer since you
don’t have to remember to check `RequestStore.active?`.
2018-09-24 12:11:27 -07:00
Michael Kozono f107bc69e3 Simplify by using Gitlab::SafeRequestStore
These are clear wins.
2018-09-24 12:11:26 -07:00
Oswaldo Ferreira 5dce096cf8 Use standalone diff stats RPC on every comparison view 2018-09-19 12:26:28 +00:00
Douwe Maan a2a00a5e6d Merge branch '51564-fix-commit-email-usage' into 'master'
Respect the user commit email in more places

Closes #51564

See merge request gitlab-org/gitlab-ce!21773
2018-09-18 12:07:52 +00:00
Douwe Maan bff85a4b1c Merge branch 'osw-gitaly-diff-stats-client' into 'master'
Add Gitaly diff stats RPC client

See merge request gitlab-org/gitlab-ce!21732
2018-09-17 16:29:42 +00:00
Oswaldo Ferreira c9d6c02f15 Add Gitaly diff stats RPC client
This adds a basic interface to fetch diff statistics given two SHAs.
It's a requirement for #49399 #20282 and #19232.
2018-09-17 12:27:01 -03:00
Nick Thomas f54cada7b7
Remove Gitlab::Git::Repository#user_to_committer 2018-09-17 14:44:24 +01:00
Nick Thomas fc0194b589 Resolve "Add functionality to change what email address online actions commit using" 2018-09-14 09:52:09 +00:00
Robert Speicher c7d1eef671 Merge branch 'rubocop-code-reuse' into 'master'
Add RuboCop cops to enforce code reusing rules

See merge request gitlab-org/gitlab-ce!21391
2018-09-13 14:53:05 +00:00
Alejandro Rodríguez d3c5b07962 Remove Rugged and shell code from Gitlab::Git 2018-09-12 12:32:32 -03:00
Yorick Peterse 2039c8280d
Disable existing offenses for the CodeReuse cops
This whitelists all existing offenses for the various CodeReuse cops, of
which most are triggered by the CodeReuse/ActiveRecord cop.
2018-09-11 17:32:00 +02:00
Douwe Maan 7f2b287f66 Merge branch 'osw-send-max-patch-bytes-to-gitaly' into 'master'
Send max_patch_bytes to Gitaly via Gitaly::CommitDiffRequest

Closes #50635

See merge request gitlab-org/gitlab-ce!21575
2018-09-07 14:02:17 +00:00
Oswaldo Ferreira 965ff14b6f Send max_patch_bytes to Gitaly via Gitaly::CommitDiffRequest
We used to apply this limitation on GitLab when using Rugged. Now that
we've shifted to Gitaly, this parameter needs to be sent via a RPC
request. Currently this value is hardcoded on Gitaly.
2018-09-06 19:18:27 -03:00
Douglas Barbosa Alexandre 460badd8cc
Add Gitlab::Git::Repository#find_remote_root_ref 2018-09-06 16:01:48 -03:00
Alejandro Rodríguez 4b87d80499 Fix merge requests not showing any diff files for big patches 2018-08-17 13:22:38 -03:00
Douwe Maan 5bf9f5b67b Merge branch 'zj-user-update-branch' into 'master'
Remove feature flags for UserUpdateBranch

Closes gitaly#1246 and gitaly#1310

See merge request gitlab-org/gitlab-ce!21238
2018-08-17 09:00:43 +00:00
Zeger-Jan van de Weg 295e582142
Remove feature flags for UserUpdateBranch
This has been tested on .com. So far no errors have been seen.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1310
Closes https://gitlab.com/gitlab-org/gitaly/issues/1246
2018-08-16 10:24:26 +02:00
Zeger-Jan van de Weg 31749779da
Remove feature flag for FindAllRemoteBranches
Acceptance testing done through:
https://gitlab.com/gitlab-org/gitaly/issues/1312

Relatively short AT period, but given its not a hard RPC, nor anything
funky is going on, I felt that this was decent enough to remove the
feature flag.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1243
2018-08-16 09:42:31 +02:00
Douwe Maan bbbcb9142e Merge branch 'zj-remove-feature-gate-list-commits' into 'master'
Remove feature gate for ListNewCommits

Closes gitaly#1286 and gitaly#1233

See merge request gitlab-org/gitlab-ce!21184
2018-08-14 09:14:23 +00:00
Zeger-Jan van de Weg 5ddc9ad0b9
Remove feature gate for ListNewCommits
Introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20087,
this has been tested on .com now, and is stable.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1286
Closes https://gitlab.com/gitlab-org/gitaly/issues/1233
2018-08-14 10:34:15 +02:00
Bob Van Landuyt 7466df872c Get the `merge-base` of 2 refs trough the API
This adds an endpoint to get the common ancestor of 2 refs from the API.
2018-08-13 11:58:54 +02:00
Rubén Dávila 6688a719c4 Backport of gitlab-org/gitlab-ee!6848 2018-08-08 16:18:37 -05:00
Rubén Dávila 5c748a0bd9 Backport some changes from gitlab-org/gitlab-ee!6767 2018-08-07 06:13:48 -05:00
Alejandro Rodríguez 01de2b5df8 Refactor gitlab:import:repos task to remove direct disk access 2018-07-31 16:28:44 -04:00
Jacob Vosmaer (GitLab) 3f0e6d9205 More Gitaly cleanup: fetch_ref, allow disk access blocks 2018-07-27 08:43:19 +00:00
Stan Hu c8ff6b7c73 Use limit parameter to retrieve Wikis from Gitaly
Without this parameter, every load of a Wiki page will load all the Wiki pages
in the repository for the sidebar. This is a significant performance penalty
that can significant slow the display of all Wiki pages.

Relates to #40101
2018-07-24 15:36:31 -07:00
Zeger-Jan van de Weg ad73953772
Make repository cleanup mandatory 2018-07-24 13:25:27 +02:00
Jacob Vosmaer (GitLab) 4147a2dbbd Annotate Gitlab::Git::Repository#fetch_ref 2018-07-24 06:33:55 +00:00
Alejandro Rodríguez d4c3a9ab41 Remove unused rugged code that is currently on Gitaly 2018-07-20 14:47:50 -04:00
Sean McGivern 256e7a6396 Merge branch 'zj-gitaly-migrate-old-migrations' into 'master'
Gitaly migrate old DB migrations

Closes gitaly#1106

See merge request gitlab-org/gitlab-ce!20708
2018-07-20 10:21:05 +00:00
Alejandro Rodríguez fa1ea656f7 Migrate add_remote, remove_remote, fetch_internal_remote to gitaly 2018-07-19 15:07:37 -04:00
Jacob Vosmaer (GitLab) 805de7fb8d Add missing Gitaly branch_update nil checks 2018-07-19 17:48:58 +00:00
Zeger-Jan van de Weg cfbb256b8b
MigrateProcessCommitWorkerJobs to use Gitaly
This old migration used Rugged to find a commit, while Gitaly is the
prefered way now. By migrating this to Gitaly, Gitaly is now a required
running component for this migration.

Part of https://gitlab.com/gitlab-org/gitaly/issues/1106
2018-07-19 13:34:10 +02:00
Alejandro Rodríguez 68b1e5a97c Incorporate Gitaly's RefService.FindAllRemoteBranches RPC 2018-07-18 16:48:47 -04:00
Zeger-Jan van de Weg 9348efc6ba
Client implementation for Repository#new_commits
After trying to remove the whole method in
8f69014af2902d8d53fe931268bec60f6858f160, this is a more gentle
approach to the method. :)

Prior to this change, new commit detection wasn't implemented in Gitaly,
this was done through: https://gitlab.com/gitlab-org/gitaly/merge_requests/779

As the new implemented got moved around a bit, the whole RevList class
got removed.

Part of https://gitlab.com/gitlab-org/gitaly/issues/1233
2018-07-18 14:46:27 +02:00
Jacob Vosmaer (GitLab) 5c651daa97 Remove unreachable Git code 2018-07-18 11:12:57 +00:00
Jacob Vosmaer (GitLab) 830940f422 Migrate merge base, merged branches, write ref 2018-07-17 09:00:32 +00:00
Jacob Vosmaer (GitLab) 37eca76583 Remove Repository#lookup and unreachable rugged code 2018-07-16 10:34:31 +00:00
Yorick Peterse 9451c80ac9 Merge branch 'sh-fix-stderr-pipe-consumption' into 'master'
Avoid process deadlock in popen by consuming input pipes

Closes gitlab-ee#6895

See merge request gitlab-org/gitlab-ce!20600
2018-07-13 13:14:34 +00:00
Stan Hu fd392cd725 Avoid process deadlock in popen by consuming input pipes
A process that spews a lot of output to stderr or stdout could stall out
due to the pipe buffer being full. As described in https://bugs.ruby-lang.org/issues/9082,
we can use the trick used in Ruby's capture3 function to read the pipes in separate
threads.

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6895
2018-07-12 16:21:30 -07:00
Jacob Vosmaer 9d93a61a70 Fix find_branch call sites 2018-07-12 18:06:31 +02:00
Jacob Vosmaer d1d7144b96 Make more ref RPC's mandatory 2018-07-12 15:12:00 +02:00
Jacob Vosmaer (GitLab) 62ffad0802 Remove Repository#path memoization 2018-07-12 09:49:25 +00:00
Jacob Vosmaer 723f74d49d Remove last flags from Blob and Workhorse 2018-07-11 12:59:15 +02:00
Rémy Coutable 7358211ece Merge branch '49057-fix-repository-typo' into 'master'
Fix a typo in Gitlab::Git::Repository

See merge request gitlab-org/gitlab-ce!20520
2018-07-10 13:32:20 +00:00
Douwe Maan 937d916b5f Merge branch 'gitaly-mandatory-20180709-jv' into 'master'
Use Gitaly for fetches and creating bundles

Closes gitaly#387, gitaly#874, gitaly#750, and gitaly#665

See merge request gitlab-org/gitlab-ce!20490
2018-07-10 10:22:05 +00:00
Lin Jen-Shin 1f28f4567b Fix a typo? gitaly_operation{-s-}_client 2018-07-10 16:52:34 +08:00
Sean McGivern 2a4039cf66 Merge branch 'update-rubocop' 2018-07-09 17:23:30 +01:00
Robert Speicher 2cedb243ee Merge branch 'gitaly-update-branch' into 'master'
Use Gitaly's OperationService.UserUpdateBranch RPC

Closes gitaly#1252

See merge request gitlab-org/gitlab-ce!20231
2018-07-09 16:12:18 +00:00
Lin Jen-Shin 9286f5b934 Use stable gitlab-styles and eliminate offenses 2018-07-09 21:13:08 +08:00
Lin Jen-Shin 4ee08b77bc Updates from `rubocop -a` 2018-07-09 21:13:08 +08:00
Jacob Vosmaer a4e75e7a83 Use Gitaly for fetches and creating bundles 2018-07-09 12:50:17 +02:00
Jacob Vosmaer (GitLab) 3082b7d1c2 Make blob and other RPC's mandatory 2018-07-09 10:02:02 +00:00
Douwe Maan 090f4d9e9f Merge branch 'gitaly-mandatory-20180706-jv' into 'master'
Remove gitaly_list_commits_by_oid, gitaly_commit_deltas, gitaly_create_repository flags

Closes gitaly#593, gitaly#566, and gitaly#670

See merge request gitlab-org/gitlab-ce!20432
2018-07-09 09:59:51 +00:00
Grzegorz Bizon 1a78483b49 Merge branch 'gitaly-diff-mandatory' into 'master'
Make all Gitaly diff flags mandatory

Closes gitaly#528 and gitaly#222

See merge request gitlab-org/gitlab-ce!20424
2018-07-09 09:42:30 +00:00
Alejandro Rodríguez 4a4338fcc6 Use Gitaly's OperationService.UserUpdateBranch RPC 2018-07-08 23:27:20 -04:00
Nick Thomas 7a17f8d85d Merge branch 'gitaly-set-delete-config' into 'master'
Add Repository#set_config and #delete_config

See merge request gitlab-org/gitlab-ce!20372
2018-07-06 15:55:10 +00:00
Jacob Vosmaer c770d2f96e Make more things mandatory 2018-07-06 14:44:54 +02:00
Jacob Vosmaer ff112ce641 Add Repository#set_config and #delete_config 2018-07-06 12:06:54 +02:00
Jacob Vosmaer b012abee8b Make all Gitaly diff flags mandatory 2018-07-06 12:01:15 +02:00
Jacob Vosmaer 9f8326995c Partially rever "Revert "Merge branch 'gitaly-mandatory-20180703-jv' into 'master'""
This partially reverts commit 64dfe2cba1.
2018-07-06 11:43:01 +02:00
Jacob Vosmaer (GitLab) 64dfe2cba1 Revert "Merge branch 'gitaly-mandatory-20180703-jv' into 'master'"
This reverts merge request !20339
2018-07-05 10:22:04 +00:00
Sean McGivern 7a355f6856 Merge branch 'gitaly-mandatory-20180703-jv-2' into 'master'
Remove more Gitaly feature flags

Closes gitaly#329, gitaly#320, gitaly#323, and gitaly#328

See merge request gitlab-org/gitlab-ce!20343
2018-07-04 10:39:59 +00:00
Jacob Vosmaer (GitLab) 78708b1932 Make more Gitaly features mandatory 2018-07-04 07:39:59 +00:00
Jacob Vosmaer 201802f723 Remove more feature flags 2018-07-03 17:39:08 +02:00
Jacob Vosmaer (GitLab) 15aadc665f Make OperationService RPC's mandatory 2018-07-03 09:12:03 +00:00
Douwe Maan e28896ce0d Merge branch 'zj-find-all-commits' into 'master'
FindAllCommits mandatory

Closes gitaly#326

See merge request gitlab-org/gitlab-ce!20242
2018-06-28 13:11:00 +00:00
Douwe Maan 37b1922237 Merge branch 'zj-commits-between-mandatory' into 'master'
Commits between in Gitaly only

Closes gitaly#315

See merge request gitlab-org/gitlab-ce!20239
2018-06-28 13:10:07 +00:00
Zeger-Jan van de Weg ec78d29a6e
FindAllCommits mandatory
Closes https://gitlab.com/gitlab-org/gitaly/issues/326
2018-06-28 13:42:10 +02:00
Zeger-Jan van de Weg f369da063f
Commits between in Gitaly only
Prior to this change, most the commits counted were done through Gitaly.
This removes the last point where this wasn't the case.

This makes the `rugged_count_commits` method obsolete, with its tests.

Closes https://gitlab.com/gitlab-org/gitaly/issues/315
2018-06-28 12:09:20 +02:00
Douwe Maan e38db19659 Merge branch 'gitaly-mandatory-20180627-jv-2' into 'master'
Make Gitaly signature RPC's mandatory

Closes gitaly#745, gitaly#1044, and gitaly#876

See merge request gitlab-org/gitlab-ce!20212
2018-06-28 07:45:31 +00:00
Jacob Vosmaer f4d6834786 Make Gitaly signature RPC's mandatory 2018-06-27 14:32:09 +02:00
Jacob Vosmaer be5e6d586f Make search and message RPC's mandatory 2018-06-27 13:46:51 +02:00
Sean McGivern 2452f1a73e Merge branch 'gitaly-tree-entry-dot-dot' into 'master'
Client-side fix for Gitaly TreeEntry bug

See merge request gitlab-org/gitlab-ce!20176
2018-06-27 09:04:34 +00:00
Jacob Vosmaer 33c950f293 Client-side fix for Gitaly TreeEntry bug 2018-06-26 14:12:08 +02:00
Alejandro Rodríguez c6ff7bfaec Move git calls for `Repository#update_branch` inside Gitlab::Git
This prepares this code for migration to Gitaly.
2018-06-25 19:49:55 -04:00
Douwe Maan 77fe416681 Merge branch 'zj-create-from-bundle-mandatory' into 'master'
Moves CreateFromBundle RPC to mandatory

Closes gitaly#944

See merge request gitlab-org/gitlab-ce!20144
2018-06-25 15:29:57 +00:00
Douwe Maan fe03ff90c2 Merge branch 'zj-is-anchestor' into 'master'
Ancestor check is Gitaly only now

Closes gitaly#789 and gitaly#308

See merge request gitlab-org/gitlab-ce!20095
2018-06-25 15:28:19 +00:00
Douwe Maan 246ce5b5f9 Merge branch 'zj-write-config' into 'master'
Write Config is mandatory

Closes gitaly#879

See merge request gitlab-org/gitlab-ce!20146
2018-06-25 15:27:41 +00:00
Douwe Maan 51f047aaad Merge branch 'zj-missing-part-linguist' into 'master'
Remove Languages feature gate

See merge request gitlab-org/gitlab-ce!20128
2018-06-25 15:23:25 +00:00
Zeger-Jan van de Weg 29b4d657be
Moves another RPC to mandatory
This specific one isn't used on most machines, therefor low risk.

Closes https://gitlab.com/gitlab-org/gitaly/issues/944
2018-06-25 15:55:22 +02:00
Zeger-Jan van de Weg 8354ae3cef
Update remote happens through Gitaly only
This change makes closes another migration! And remotes tests around it,
mostly due to stubbing this was needed. Also, these are tested on the
Gitaly side too:
- https://gitlab.com/gitlab-org/gitaly/issues/791
-
6dd74543f6/internal/service/repository/fetch_remote_test.go

Closes https://gitlab.com/gitlab-org/gitaly/issues/789
2018-06-25 15:47:42 +02:00
Zeger-Jan van de Weg 0855478726
Write Config is mandatory
Closes https://gitlab.com/gitlab-org/gitaly/issues/879
2018-06-25 15:08:20 +02:00
Zeger-Jan van de Weg af20d7816e
Remove feature gate 2018-06-25 13:51:06 +02:00
Zeger-Jan van de Weg e91b2b5dff
Gitaly tree entry is mandatory
Closes https://gitlab.com/gitlab-org/gitaly/issues/310
2018-06-25 13:48:28 +02:00
Zeger-Jan van de Weg 9d438edb34
Anchestor check is Gitaly only now
Closes https://gitlab.com/gitlab-org/gitaly/issues/308
2018-06-25 09:25:15 +02:00
Douwe Maan 70bc866563 Merge branch 'zj-commit-raw-diffs' into 'master'
Raw diffs are served from Gitaly

Closes gitaly#222

See merge request gitlab-org/gitlab-ce!20091
2018-06-22 16:00:02 +00:00
Robert Speicher 254e589de9 Merge branch 'rs-lib-gitlab-git-gitlab_projects-parity' into 'master'
CE-EE parity for lib/gitlab/git/gitlab_projects.rb

See merge request gitlab-org/gitlab-ce!20047
2018-06-22 15:25:36 +00:00
Douwe Maan 510e45fd1e Merge branch 'zj-find-ref-name' into 'master'
FindRefName RPC now mandatory

Closes gitaly#221

See merge request gitlab-org/gitlab-ce!20090
2018-06-22 13:35:49 +00:00
Zeger-Jan van de Weg 417a21a123
Raw diffs are served from Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/222
2018-06-22 13:40:38 +02:00
Zeger-Jan van de Weg 4e9491f7bb
Repository size is Gitaly only
Closes https://gitlab.com/gitlab-org/gitaly/issues/393
2018-06-22 11:53:18 +02:00
Zeger-Jan van de Weg 21c4ffe36f
FindRefName RPC now mandatory
Has never been in OPT_OUT, dispite being ages old. The time where
feature flags are still set by env vars.

Closes https://gitlab.com/gitlab-org/gitaly/issues/221
2018-06-22 11:41:07 +02:00
Zeger-Jan van de Weg 3247f47c8d
Git ls files is Gitaly only now
Moved to the front on the line through: https://gitlab.com/gitlab-org/gitlab-ce/issues/47893

Closes: https://gitlab.com/gitlab-org/gitlab-ce/issues/47893
Closes: https://gitlab.com/gitlab-org/gitaly/issues/327
2018-06-22 11:14:34 +02:00
Zeger-Jan van de Weg 2ac672280b
Raw changes calculation is Gitaly only now
Closes https://gitlab.com/gitlab-org/gitaly/issues/1138
2018-06-21 10:12:51 +02:00
Robert Speicher f87e702ece
CE-EE parity for lib/gitlab/git/gitlab_projects.rb 2018-06-20 15:08:11 -05:00
Mark Chao 937bd9e386 Fix possiblilty of branch not found
This now can happen because can_be_merged? is called
during MR merge_status transition to cannot_be_merged.
It is possible branch_name is invalid.
2018-06-20 23:38:13 +08:00
Douwe Maan a5245fd8b8 Merge branch 'zj-can-be-merged' into 'master'
Move mergablility check to Gitaly

Closes gitaly#889

See merge request gitlab-org/gitlab-ce!20019
2018-06-20 14:15:28 +00:00
Sean McGivern b60415c2b7 Merge branch '48105-git-version-detection-using-gitaly' into 'master'
Move git version detection to gitaly for usage ping data

Closes #48105 and charts/gitlab#530

See merge request gitlab-org/gitlab-ce!19993
2018-06-20 09:00:32 +00:00
Douwe Maan c67d8c2478 Merge branch 'zj-lfs-pointers' into 'master'
LFS changes are detected by Gitaly

Closes gitaly#935

See merge request gitlab-org/gitlab-ce!19995
2018-06-20 08:29:47 +00:00
Zeger-Jan van de Weg 46b56b18f5
Move mergablility check to Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/889
2018-06-20 09:11:55 +02:00
Douwe Maan c0f0ccf222 Merge branch 'zj-force-push-opt-out' into 'master'
Force push is handled by Gitaly now

Closes gitaly#348

See merge request gitlab-org/gitlab-ce!19992
2018-06-19 14:19:18 +00:00
Zeger-Jan van de Weg 64cda7746c
LFS changes are detected by Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/935
2018-06-19 15:59:27 +02:00
Zeger-Jan van de Weg c2cdefd441
Force push is handled by Gitaly now
Closes gitlab-org/gitaly#348
2018-06-19 15:35:17 +02:00
Balasankar "Balu" C 5607c57c41 Use gitaly to figure out git version for usage ping 2018-06-19 18:39:47 +05:30
Douwe Maan b0fadeee83 Merge branch 'zj-counting-commits' into 'master'
Counting commits is done by Gitaly

Closes gitaly#382

See merge request gitlab-org/gitlab-ce!19983
2018-06-19 12:45:49 +00:00
Douwe Maan 51334904f5 Merge branch 'zj-gitaly-squash' into 'master'
Rebase and Squash in progress are Gitaly only

Closes gitaly#865 and gitaly#1018

See merge request gitlab-org/gitlab-ce!19987
2018-06-19 09:56:33 +00:00
Douwe Maan 14dda26b25 Merge branch 'zj-info-attributes' into 'master'
Move info attributes to mandatory

Closes gitaly#1082

See merge request gitlab-org/gitlab-ce!19988
2018-06-19 09:55:48 +00:00
Douwe Maan 9e82c32b31 Merge branch 'zj-commit-count' into 'master'
Move count commit to mandatory

Closes gitaly#330

See merge request gitlab-org/gitlab-ce!19984
2018-06-19 09:55:38 +00:00
Zeger-Jan van de Weg c24e39ab88
Counting commits is done by Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/382
2018-06-19 10:37:51 +02:00
Zeger-Jan van de Weg 012948b849
Move info attributes to mandatory
Closes https://gitlab.com/gitlab-org/gitaly/issues/1082
2018-06-19 10:02:12 +02:00
Zeger-Jan van de Weg 5053dc84b0
Rebase and Squash in progress are Gitaly only
Closes https://gitlab.com/gitlab-org/gitaly/issues/865
Closes https://gitlab.com/gitlab-org/gitaly/issues/1018
2018-06-19 09:41:09 +02:00
Zeger-Jan van de Weg da8419a1ba
License detection happens soley through Gitaly now
Migration: https://gitlab.com/gitlab-org/gitaly/issues/1026

Conflicts:
	lib/gitlab/git/repository.rb
2018-06-19 09:35:15 +02:00
Zeger-Jan van de Weg 6e25935f73
Move count commit to mandatory
Closes https://gitlab.com/gitlab-org/gitaly/issues/330
2018-06-19 09:24:46 +02:00
Jacob Vosmaer (GitLab) 14000a561b Make Gitaly wiki RPC's mandatory 2018-06-18 08:19:11 +00:00
Zeger-Jan van de Weg 15f0cef1d0
Local branches go through Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/217
2018-06-14 10:44:33 +02:00
Zeger-Jan van de Weg 3ed4a1b3aa
HasLocalBranches check is done by Gitaly only
Closes https://gitlab.com/gitlab-org/gitaly/issues/217
2018-06-14 10:31:24 +02:00
Zeger-Jan van de Weg 89ab32c6b2
Branches are fully migrated to Gitaly
Closes: https://gitlab.com/gitlab-org/gitaly/issues/389
2018-06-14 10:25:28 +02:00
Zeger-Jan van de Weg 143a632ebe
Tags are migrated to Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/390
2018-06-14 10:18:45 +02:00
Zeger-Jan van de Weg 36d75be393
Move TagNames to mandatory through Gitaly
Closes: https://gitlab.com/gitlab-org/gitaly/issues/220
2018-06-14 10:13:15 +02:00
Zeger-Jan van de Weg 89095530bc
Tag names returned through Gitaly
Migration: https://gitlab.com/gitlab-org/gitaly/issues/220
2018-06-14 10:00:47 +02:00
Zeger-Jan van de Weg 4fc4600747
Default branch detection happens through Gitaly
Migration: https://gitlab.com/gitlab-org/gitaly/issues/220
2018-06-14 09:56:16 +02:00
Zeger-Jan van de Weg f195a7436d
RawBlame only called through Gitaly
Closes https://gitlab.com/gitlab-org/gitaly/issues/376
2018-06-13 16:44:59 +02:00
Sean McGivern 4666ef683c Merge branch 'zj-compare-source-branch' into 'master'
Migrate compare_source_branch to use only Gitaly

Closes gitaly#771

See merge request gitlab-org/gitlab-ce!19700
2018-06-13 08:26:00 +00:00
Douwe Maan 8b0d2283d7 Merge branch 'gitaly-disk-access-3' into 'master'
Find and mark more Git disk access locations, part 2

See merge request gitlab-org/gitlab-ce!19437
2018-06-12 13:32:46 +00:00
Jacob Vosmaer f376347f24 Find and mark more Git disk access locations, part 2 2018-06-12 14:33:37 +02:00
Zeger-Jan van de Weg d65db552af
Migrate compare_source_branch to use only Gitaly
Initially expected this to be a full blown migration, but that wasn't
the case as we could fetch to a reference that would be deleted after
the instanciation of the Diff object.

This is a racy RPC, as #delete_refs is called and garbage collections
runs right than, the commit can't be found later.

Closes https://gitlab.com/gitlab-org/gitaly/issues/771
2018-06-12 12:48:09 +02:00
Jacob Vosmaer (GitLab) 8749641838 Move Gitaly branch/tag/ref RPC's to opt-out 2018-06-12 09:24:05 +00:00
Jacob Vosmaer (GitLab) ea8dc10772 Don't use Gitlab::Utils.nlbr in Gitlab::Git 2018-06-11 10:42:09 +00:00
Jacob Vosmaer (GitLab) ab40c7e3c8 Move gitaly_blame, gitaly_count_commits, gitaly_commit_count to opt-out 2018-06-11 10:39:23 +00:00
Jacob Vosmaer 2120391441 Remove Gitlab::Popen dependency from lib/gitlab/git 2018-06-08 11:16:21 +02:00
Rubén Dávila c5d4f10b80 Pass the --in-commit-order arg to `git-rev-list`
@jamedjo has found a good improvement in the performance of the LFS
integrity check by passing this arg
2018-06-07 12:09:45 -05:00
Francisco Javier López ea2bd9cb4f Add git filter flag only if it is supported 2018-06-07 15:49:01 +00:00
Francisco Javier López 86d39016a2
Moving rev-list lfs options to Lfschanges 2018-06-06 21:55:32 +02:00
Francisco Javier López e8f49b4bee Support LFS objects when creating a project by import 2018-06-06 16:42:18 +00:00
Jacob Vosmaer e7ef91339e Simplify server feature flags 2018-06-06 11:20:41 +02:00
Zeger-Jan van de Weg f131757fc8 Set Gitaly Server feature flags from Rails
Gitaly itself hold very little state, other than the data on disk. This
limits the interfaces to set feature flags. Gitaly now has the ability
to interpret the request metadata to check for feature flags.
https://gitlab.com/gitlab-org/gitaly/merge_requests/704
This allows clients control on the Gitaly server, and given that Rails
has an internal chatops interface to set these values this was chosen as
solution.

Known limitation right now, is that this package doesn't support the opt
out that other Gitaly features do.
2018-06-06 11:20:41 +02:00
Nick Thomas 65dbead796
Fix repository archive generation when hashed storage is enabled 2018-06-05 20:44:07 +01:00
Jacob Vosmaer (GitLab) a0808df0b6 Find and mark more Git disk access locations 2018-06-05 15:51:14 +00:00
Jacob Vosmaer (GitLab) 3a4ecfd0c9 Prevent Gitaly WriteConfig log noise 2018-06-05 12:55:41 +00:00
Jacob Vosmaer (GitLab) edb9db37ed Add "deny disk access" Gitaly feature (tripswitch) 2018-06-01 11:56:29 +00:00
Douwe Maan d6d7abb1d0 Merge branch 'gitaly-repo-squash-in-progress-opt-out' into 'master'
Make Repo#squash_in_progress Opt-Out

See merge request gitlab-org/gitlab-ce!19088
2018-05-23 09:52:11 +00:00
Douwe Maan 91f3930cc1 Merge branch 'gitaly-wiki-get-formatted-data-opt-out' into 'master'
Move Wiki::GetFormattedData to OptOut

See merge request gitlab-org/gitlab-ce!19091
2018-05-23 09:51:37 +00:00
Douwe Maan 68b5cd71e0 Merge branch 'gitaly-import-project-opt-out' into 'master'
Move ImportProject to OptOut in Gitaly

See merge request gitlab-org/gitlab-ce!19090
2018-05-23 09:50:54 +00:00
Kim Carlbäcker f0ef31a114 Move Repository#info_attributes to OptOut in gitaly 2018-05-22 16:05:02 +00:00
Kim Carlbäcker cb2a541027 Move Wiki::GetFormattedData to OptOut 2018-05-22 15:58:36 +00:00
Kim Carlbäcker 2a781f7f5b Move ImportProject to OptOut in Gitaly 2018-05-22 15:52:54 +00:00
Kim Carlbäcker 536b32c8fa Make Repo#squash_in_progress Opt-Out 2018-05-22 15:44:42 +00:00
Grzegorz Bizon b113330fab Merge branch 'zj-workhorse-archive-mandatory' into 'master'
Move git archives downloading to Gitaly

See merge request gitlab-org/gitlab-ce!18990
2018-05-21 07:08:58 +00:00
Grzegorz Bizon ad9e00917f Merge branch 'xeodon/gitlab-ce-fix/45743-master-fix-gitaly-delete-refs' into 'master'
Fix error when deleting an empty list of refs

Closes #45743

See merge request gitlab-org/gitlab-ce!19053
2018-05-19 05:42:16 +00:00
Sergey Sinev 100c687cbc Fix error when deleting an empty list of refs
Closes #45743
2018-05-18 20:47:46 -07:00
Stan Hu 95dacabe4b Merge branch '46498-do-not-modify-strings' into 'master'
Resolve "Blob#batch can't handle frozen string paths"

Closes #46498

See merge request gitlab-org/gitlab-ce!19039
2018-05-19 00:03:49 +00:00
Robert Speicher 076ab2e8e0 Merge branch 'fix/handle-large-commit-tag-message' into 'master'
Add handling for commit/tags with big messages

See merge request gitlab-org/gitlab-ce!17892
2018-05-18 19:28:25 +00:00
Ahmad Sherif 019f5e2469 Add handling for commit/tags with big messages 2018-05-18 20:51:10 +02:00
Robert Speicher ee59fdf566 Merge branch 'gitaly-1182-search-by-content-name' into 'master'
Migrate Repo#search_by_{content,name}  to Gitaly

See merge request gitlab-org/gitlab-ce!18750
2018-05-18 18:28:32 +00:00
Nick Thomas 8bacfbd1cc Merge branch 'zj-calculate-checksum-mandator' into 'master'
Calculating repository checksums executed by Gitaly

Closes gitaly#1105 and #46293

See merge request gitlab-org/gitlab-ce!18907
2018-05-18 15:53:35 +00:00