Nick Thomas
4aa76dddec
Remove dead MySQL code
...
None of this code can be reached any more, so it can all be removed
2019-07-23 16:53:03 +01:00
Dominic Renner
ad29681cab
Replaced config access
...
Replaced Settings with Gitlab.config for uniformity
2019-03-18 17:26:18 +00:00
Frederic Van Espen
094f219506
trim empty line
2019-03-07 13:34:17 +01:00
Frederic Van Espen
31dfc31aaa
Merge branch 'master' into incremental-backups
2019-03-07 11:14:27 +01:00
Frederic Van Espen
562a1fc87d
fix option test
2019-03-07 09:51:53 +01:00
Frederic Van Espen
935b698384
introduce optional rsyncable flag
2019-03-07 09:38:14 +01:00
Stan Hu
b103f61450
Add convenience methods for creating project and Wiki repositories
...
This makes it easier to access other project arguments in the future.
2019-02-05 21:33:10 -08:00
Stan Hu
82b6e5378a
Send project name with Gitaly repository requests
...
When hashed storage is in use, it's helpful to have the project
name associated with the request.
Closes https://gitlab.com/gitlab-org/gitaly/issues/1394
2019-02-05 21:33:09 -08:00
Jason Colyer
c74820db07
Modify file restore to rectify tar issue
...
Fixed typo in run_pipeline! function
-> Open3.pieline to Open3.pipeline
2019-01-17 10:07:29 -06:00
Jason Colyer
f88b464bf3
Modify file restore to rectify tar issue
...
Adjusted run_pipeline! function
to match rubocop syntax requirements
Adjusted run_pipeline! function
to use variable instead of string interpolation
2019-01-17 10:07:29 -06:00
Jason Colyer
99b6f37fa5
Modify file restore to rectify tar issue
...
Changed run_pipeline! function to have use $ in regex
Changed run_pipeline! function to use Open3.pipeline
instead of Open3.pipeline_start
2019-01-17 10:07:06 -06:00
Jason Colyer
6de60ef295
Modified files.rb to handle tar error
...
Changed the run_pipeline! function to instead use Open3.pipeline_start
In doing so, the code is able to grab the last STDERR. In the case
where the error is the one from older tar versions (relating to ./),
it will not raise a Backup::Error. All other instances of command
failure will result in Backup::Error being raised.
2019-01-16 08:29:40 -06:00
Pepijn Van Eeckhoudt
1b3ffdf250
Add SSE-C key configuration option for Amazon S3 remote backups
2019-01-09 16:39:20 +00:00
Frederic Van Espen
3bb3f7ddd0
fix code style
2018-12-27 13:43:45 +01:00
Frederic Van Espen
e8419fb5bb
set the --rsyncable option for gzip
2018-12-26 15:35:20 +01:00
Frederic Van Espen
7a58eb2e16
Allow to override part of the backup filename
2018-12-26 15:33:11 +01:00
Michael Kozono
d2d85c31fe
Avoid extra storage bucket perm and query
...
Specifically, the `ListAllMyBuckets` permission.
This works if you know the directory exists.
See more:
* https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23981
* https://stackoverflow.com/a/12288581/1992201
2018-12-21 10:34:05 -08:00
Zeger-Jan van de Weg
89a407dc3b
Restore Object Pools when restoring an object pool
...
Pool repositories are persisted in the database, and when the DB is
restored, the data need to be restored on disk. This is done by
resetting the state machine and rescheduling the object pool creation.
This is not an exact replica of the state like at the time of the
creation of the backup. However, the data is consistent again.
Dumping isn't required as internally GitLab uses git bundles which
bundle all refs and include all objects in the bundle that they require,
reduplicating as more repositories get backed up. This does require more
data to be stored.
Fixes https://gitlab.com/gitlab-org/gitaly/issues/1355
2018-12-13 09:00:03 +01:00
gfyoung
c858f70d07
Enable frozen string for lib/gitlab/*.rb
2018-10-22 07:00:50 +00:00
Balasankar "Balu" C
91ef6989d1
Add installation type to backup information file
2018-10-05 19:34:22 +05:30
gfyoung
3836d69119
Enable frozen string in lib/api and lib/backup
...
Partially addresses #47424 .
Had to make changes to spec files because
stubbing methods on frozen objects is a mess
in RSpec and leads to failures:
https://github.com/rspec/rspec-mocks/issues/1190
2018-09-29 21:04:50 -07:00
Zeger-Jan van de Weg
d7afed34c4
Remove feature flags from lib/backup
...
Moved to OPT_OUT in 7d14b725a0 , Now, by
removing the feature gates, this is an mandatory feature.
Related issues:
- https://gitlab.com/gitlab-org/gitaly/issues/526
- https://gitlab.com/gitlab-org/gitaly/issues/1194
Closes https://gitlab.com/gitlab-org/gitaly/issues/749
Closes https://gitlab.com/gitlab-org/gitaly/issues/1212
Closes https://gitlab.com/gitlab-org/gitaly/issues/1195
2018-07-31 15:24:19 +02:00
Jacob Vosmaer
7d14b725a0
Move repo backup RPC's to opt-out
2018-06-25 16:21:54 +02:00
Ahmad Hassan
099eb8ae40
Restore custom hooks for wiki
2018-06-21 13:56:00 +02:00
Ahmad Hassan
9d7e73277e
Move backup custom hooks up a level
2018-06-20 11:43:59 +02:00
Ahmad Hassan
5308b51ef4
Output done for wiki and repo
2018-06-19 16:24:16 +02:00
Ahmad Hassan
69de7b42cf
Migrate repo backup to gitaly
2018-06-15 18:27:25 +02:00
Douwe Maan
f068479e63
Merge branch 'use-restore-custom-hooks-gitaly' into 'master'
...
Use RestoreCustomHooks RPC in restore rake task
See merge request gitlab-org/gitlab-ce!19370
2018-06-08 11:49:13 +00:00
Ahmad Hassan
59a8c79f05
Use RestoreCustomHooks RPC in restore rake task
2018-06-07 14:26:20 +02:00
Jacob Vosmaer
6dfb0b4b31
Consistently use tar lookup function
2018-06-07 13:39:34 +02:00
Jacob Vosmaer (GitLab)
a0808df0b6
Find and mark more Git disk access locations
2018-06-05 15:51:14 +00:00
Jacob Vosmaer (GitLab)
b529d04b69
Use Gitaly's DeleteAllRepositories RPC during backup restore
2018-05-28 09:58:14 +00:00
Douwe Maan
ba58a66a55
Merge branch 'migrate-restore-repo-to-gitaly' into 'master'
...
Use gitaly repository_service.CreateRepositoryFromBundle to restore (rake task)
Closes charts/gitlab#390
See merge request gitlab-org/gitlab-ce!18644
2018-05-24 18:28:23 +00:00
Stan Hu
e6fca03bbd
Fix backup creation and restore for specific Rake tasks
2018-05-24 14:58:25 +00:00
Ahmad Hassan
84e6ddb41f
Remove unneeded gitaly_migrate function
2018-05-24 14:16:31 +02:00
Ahmad Hassan
9c4f967785
Take restoring custom hooks and the skip flag out of scope
2018-05-23 13:47:56 +02:00
Ahmad Hassan
80197bdc62
Remove gitaly_migrate blocks and replace by early return
2018-05-21 11:45:41 +02:00
Ahmad Hassan
125be1865b
Move restore_custom_hooks to its seperate function
2018-05-18 11:55:54 +03:00
Ahmad Hassan
62f2988713
Return from prepare directory if gitaly skip is enabled
2018-05-17 15:36:58 +03:00
Ahmad Hassan
42cabcb1c4
Call path_to_repo only when using gitaly
2018-05-17 15:34:31 +03:00
DJ Mountney
889c62c1e3
Link to upstream gitaly issues for the missing restore paths
2018-05-16 13:28:15 +03:00
DJ Mountney
b8b82aa50e
Fix the paths for wiki restore
2018-05-15 16:43:04 +03:00
DJ Mountney
d5bd61e82f
Output project name before restoring each project repo
2018-05-15 16:43:04 +03:00
DJ Mountney
4ea9458bbe
Remove hooks restore that is no longer necessary
2018-05-15 16:43:04 +03:00
DJ Mountney
2bcc324f26
Add back some of the non-gitaly restore functionality behind gates
2018-05-15 16:43:04 +03:00
Ahmad Hassan
0734da16f6
Wrap create_from_bundle in a begin rescue block
2018-05-15 16:43:04 +03:00
Ahmad Hassan
8457262125
Better repo restore progress logging
2018-05-15 16:43:04 +03:00
Ahmad Hassan
fb54eb76ee
Remove useless print line
2018-05-15 16:43:04 +03:00
Ahmad Hassan
3b5e9f0608
Use gitaly repository_service.CreateFromBundle to restore
2018-05-15 16:43:04 +03:00
Bob Van Landuyt
e3ff928c75
Describe workaround when restore fails because of `Errno::EBUSY`
...
When `Errno::EBUSY` is raised during restore, this could indicate that
the directory being restored into is a mountpoint. In this case we
explain the user how to retry the restore.
2018-04-23 17:24:29 +02:00