Commit Graph

50 Commits

Author SHA1 Message Date
GitLab Bot 023e050d82 Add latest changes from gitlab-org/gitlab@master 2021-05-05 12:10:33 +00:00
GitLab Bot 1743439667 Add latest changes from gitlab-org/gitlab@master 2021-02-03 18:09:25 +00:00
GitLab Bot c6f0b221b7 Add latest changes from gitlab-org/gitlab@master 2020-05-12 00:10:11 +00:00
GitLab Bot 427a082f7d Add latest changes from gitlab-org/gitlab@master 2020-04-23 21:09:31 +00:00
GitLab Bot e33f87ac0f Add latest changes from gitlab-org/gitlab@master 2020-04-21 15:21:10 +00:00
GitLab Bot 83731155d9 Add latest changes from gitlab-org/gitlab@master 2020-03-06 18:08:08 +00:00
GitLab Bot 619d0b6922 Add latest changes from gitlab-org/gitlab@master 2020-02-26 18:09:24 +00:00
GitLab Bot 3888bc4261 Add latest changes from gitlab-org/gitlab@master 2019-12-24 00:07:31 +00:00
GitLab Bot 0c3f121493 Add latest changes from gitlab-org/gitlab@master 2019-11-11 21:06:20 +00:00
GitLab Bot 20450649ca Add latest changes from gitlab-org/gitlab@master 2019-09-27 09:06:26 +00:00
dineshpanda e908e11776 Avoid calling freeze on already frozen strings in lib/gitlab 2019-09-04 09:52:02 +05:30
Luke Duncalfe d18ee3faad LFS export records repository_type data
A project can have the same `LfsObject` linked with up to three
`LfsObjectsProject` records. Each of these records would be for a
different repository, recorded in the `repository_type` property. The
different repositories at time of writing are "project", "wiki", and
"design". See https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/13894

This change exports the list of `repository_type`s as a JSON mapping of
oid => repository_types, which are imported to recreate the correct
`LfsObjectsProject` records.

https://gitlab.com/gitlab-org/gitlab-ee/issues/11090
2019-07-24 11:23:51 +00:00
gfyoung c858f70d07 Enable frozen string for lib/gitlab/*.rb 2018-10-22 07:00:50 +00:00
James Lopez a2aa505805
Refactor code to remove object storage flag from Import/Export
Updated docs, refactor import/export code
Fix AvatarUploader path issue
Fix project export upload webhook error
2018-09-06 16:52:42 +02:00
James Lopez a2bf164154
Update Import/Export to use object storage (based on aa feature flag) 2018-07-06 15:46:18 +02:00
Grzegorz Bizon 90a99ff3aa Add pipeline stages position clean-up migration 2018-06-29 13:57:52 +00:00
Jan Provaznik 5bb421ecbb Cleanup after adding MR diff's commit_count (try 2)
* processes any pending records which are not migrated yet
* bumps import_export version because of new commits_count attribute
* removes commits_count fallback method
2018-04-25 10:53:37 +02:00
Jan Provaznik 7db225e480 Revert "Cleanup after adding MR diff's commit_count"
This reverts commit 0b7d108514.
2018-03-08 12:43:14 +01:00
Jan Provaznik 0b7d108514 Cleanup after adding MR diff's commit_count
* processes any pending records which are not migrated yet
* bumps import_export version because of new commits_count attribute
* removes commits_count fallback method
2018-03-05 15:37:01 +01:00
Grzegorz Bizon bf8c5643a9 Bump import/export version to 2.2.0
We need to bump import/export version because we introduced a new
object's hierarchy that is not backwards compatible.
2018-01-05 15:18:17 +01:00
Sean McGivern 4ebbfe5d3e Remove serialised diff and commit columns
The st_commits and st_diffs columns on merge_request_diffs historically held the
YAML-serialised data for a merge request diff, in a variety of formats.

Since 9.5, these have been migrated in the background to two new tables:
merge_request_diff_commits and merge_request_diff_files. That has the advantage
that we can actually query the data (for instance, to find out how many commits
we've stored), and that it can't be in a variety of formats, but must match the
new schema.

This is the final step of that journey, where we drop those columns and remove
all references to them. This is a breaking change to the importer, because we
can no longer import diffs created in the old format, and we cannot guarantee
the export will be in the new format unless it was generated after this commit.
2017-11-28 16:13:40 +00:00
Yorick Peterse 235b105c91
Finish migration to the new events setup
This finishes the procedure for migrating events from the old format
into the new format. Code no longer uses the old setup and the database
tables used during the migration process are swapped, with the old table
being dropped.

While the database migration can be reversed this will 1) take a lot of
time as data has to be coped around 2) won't restore data in the
"events.data" column as we have no way of restoring this.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/37241
2017-09-06 16:40:31 +02:00
Sean McGivern 9a73b634ab Add table for files in merge request diffs
This adds an ID-less table containing one row per file, per merge request
diff. It has a column for each attribute on Gitlab::Git::Diff that is serialised
currently, with the advantage that we can easily query the attributes of this
new table.

It does not migrate existing data, so we have fallback code when the legacy
st_diffs column is present instead. For a merge request diff to be valid, it
should have at most one of:

* Rows in this new table, with the correct merge_request_diff_id.
* A non-NULL st_diffs column.

It may have neither, if the diff is empty.
2017-06-16 18:30:01 +01:00
Zeger-Jan van de Weg 8df3997a92 Add Pipeline Schedules that supersedes experimental Trigger Schedule 2017-05-07 22:35:56 +00:00
Douwe Maan ad640bc5f9 Use Namespace#full_path instead of #path where appropriate 2017-02-23 17:55:01 -06:00
Douwe Maan b7d8df503c Enable Style/MutableConstant 2017-02-23 09:31:56 -06:00
Dmitriy Zaporozhets 6676b4f0dd
Use Namespace#full_path instead of Namespace#path
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2017-02-14 18:38:20 +02:00
James Lopez e589c7e848 Ignore encrypted attributes in Import/Export
* Regenerates tokens for all models that have them
* Remove variables, since they are basically just storing encrypted data
* Bumped version up to 0.1.6
* Updated related docs
2017-01-30 12:34:32 +01:00
James Lopez 77b7bfd463 Fix import/export labels to cope with project and group labels. Added relevant specs. 2016-10-19 14:58:25 -02:00
James Lopez 3c7feaf3b1 Fixed label color issue and added Import/Export versioning table 2016-09-19 09:18:37 +02:00
James Lopez e74b7d665b squashed - Fix DB exceptions raised importing some specific projects.
Better import of labels, milestones and protected branches. Updated relevant specs.
Loose pipeline validation on importing, so it does not fail when there are missing fields, which are not validated at DB level. Also, updated spec with relevant test.
2016-09-19 09:18:37 +02:00
James Lopez 6a0bbb5aa5 using shared path for project import uploads and refactored gitlab remove export worker 2016-08-04 12:51:55 +02:00
James Lopez aad0ae7162 squashed - fixed label and milestone association problems, updated specs and refactored reader class a bit 2016-08-01 09:57:40 +02:00
James Lopez bf1ea8c6fe Squashed - fix encoding issue
WIP - trying to replicate UTF-8 error

fix spec

fixing encoding issue and another spec, to do with MR diffs

fix issue and spec failure

Add changelog and bumped up I/E version

fix spec based on feedback - omitted target project
2016-07-19 09:59:54 +02:00
James Lopez 65549a5866 add project name and namespace to filename on project export
added changelog
2016-07-15 12:55:28 +02:00
James Lopez 5cd3d7c40a bump version - as old exports wont be compatible 2016-06-29 10:49:31 +02:00
James Lopez d5b3a266e8 use rails root join 2016-06-20 09:17:07 +02:00
James Lopez cee2a2dc66 fixed a couple of errors spotted in production 2016-06-20 08:58:43 +02:00
James Lopez ad68bc63b5 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import
# Conflicts:
#	app/models/project.rb
#	db/schema.rb
#	lib/gitlab/import_export/import_export_reader.rb
2016-06-13 12:43:25 +02:00
James Lopez 771f735109 few more modifications based on comments 2016-06-13 09:37:58 +02:00
James Lopez 1466997755 import uploads. Fixed a few things to do with members, triggers, etc... 2016-05-19 15:36:20 +02:00
James Lopez 504c186f71 fix version issue 2016-05-16 13:28:11 +02:00
James Lopez 5777ad9a1f adding versioning to export 2016-05-16 12:04:31 +02:00
James Lopez 2dff04f24a fixed TODOs left 2016-05-13 17:39:03 +02:00
James Lopez a5d59f075a added better error handling. Also refactored some of the code and fixed a few issues in project_tree_saver 2016-05-10 17:15:20 +02:00
James Lopez 21be0cae62 fixing merge issues 2016-05-09 18:33:48 +02:00
James Lopez 8ac53eb5d0 started refactoring import export reader - WIP 2016-05-06 17:55:06 +02:00
James Lopez ce598b0541 fixed and refactored a few things based on MR feedback 2016-05-06 13:40:02 +02:00
James Lopez 4256743686 refactored path stuff 2016-04-22 12:18:11 +02:00
James Lopez 0852f539aa refactored stuff, added a save and compress all class and moved mostly everything to lib 2016-04-14 16:57:25 +02:00