Commit Graph

238 Commits

Author SHA1 Message Date
James Lopez b3bb8dc46f added spec replicating the problem 2017-01-18 15:46:54 +01:00
James Lopez 35442766cf do not map usersat all unless admin 2017-01-18 14:13:01 +01:00
Ruben Davila 17196a2ff3 Backport backend work for time tracking. 2017-01-15 11:10:04 -05:00
James Lopez 5452747729 Fix error importing label priorities and added relevant spec 2016-12-21 10:33:20 +01:00
Rémy Coutable 8c8c4f8c5a Merge branch 'fix/import-export-mr-error' into 'master'
Fix import/export merge requests error

When MWBS is set and the MR exported/imported an error gets thrown. We should ignore this so it can succeed as the merge user may no longer exist in the target instance.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/24009 and https://gitlab.com/gitlab-com/support-forum/issues/1345

See merge request !8134
2016-12-19 18:03:06 +00:00
James Lopez 0631c7bcdc Fix MR issue to do with merge user 2016-12-19 16:52:38 +01:00
James Lopez 786457ca3e
Fix for missing service when importing from EE to CE
Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-12-19 16:32:26 +01:00
James Lopez d6edecdeec Fix duplicated build token problem and added relevant spec 2016-12-19 11:26:57 +01:00
Douwe Maan dc9b3db8b0 Merge branch 'fix/import-export-symlink-vulnerability' into 'security'
Fix symlink vulnerability in Import/Export

Replaces https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2018 made by @james

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23822

See merge request !2022

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-03 16:04:18 +01:00
Douwe Maan cfb511ea69 Merge branch 'fix/import-projectmember-security' into 'security'
Fix Import/Export foreign key issue to do with project members

Cleans-up any foreign keys in `ProjectMember` - same as we do with the rest of the models when importing.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23837 and https://gitlab.com/gitlab-org/gitlab-ce/issues/23739

See merge request !2020

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-11-03 16:04:18 +01:00
James Lopez 891e5f4851 Update specs to cope with new label types and priorities
Fixed all related specs and also changed the logic to handle edge cases. This includes exporting and exporting of group labels, which will get associated with the new group (if any) or they will become normal project labels otherwise.

Found other issues to do with not being able to import all labels at once in the beginning of the JSON - code was much simpler when we import all labels and milestones associated to a project first, then the associations will find the already created labels instead of creating them from the associations themselves.
2016-10-19 14:58:28 -02:00
Douglas Barbosa Alexandre 11d786e7da Use try instead of ternary operator on Gitlab::ImportExport::ProjectTreeRestorer 2016-10-19 14:58:25 -02:00
James Lopez 77b7bfd463 Fix import/export labels to cope with project and group labels. Added relevant specs. 2016-10-19 14:58:25 -02:00
James Lopez 848a146fc3 Fix import test 2016-10-19 14:58:25 -02:00
Douglas Barbosa Alexandre e036a72dca Add Lavel#type to methods in lib/gitlab/import_export/import_export.yml 2016-10-19 14:58:25 -02:00
Rémy Coutable 67aabcc764 Merge remote-tracking branch 'dev/master' 2016-10-06 08:34:39 +02:00
Rémy Coutable d51bb99a7e
Merge commit 'dev/security' into 'master'
Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-10-06 08:33:11 +02:00
James Lopez 958d9f11e8 fix export project file permissions issue 2016-09-30 16:10:48 +02:00
Douwe Maan 8a866bfce3 Merge branch 'fix/id-claim-import-issue' into 'master'
Prevent claiming associated model IDs via import

On the import side, we should be careful not to use any IDs as part of the JSON file that could have been manipulated.

Part of https://gitlab.com/gitlab-org/gitlab-ce/issues/20821


Things we already do (__before__ this fix):

1. Remove all primary keys
1. **Always** reassign some of the foreign keys, such as ALL project IDs and user IDs (so it would be difficult to impersonate or try to gain access to another project)
1. Ignore/reject attributes that do not exist in the model
1. If someone reassigns a foreign key `submodel_id`, and that object has another json as the submodel, the new submodel will reassign the `submodel_id` to the newly created submodel ID.

Things we should do:

1. Remove/nullify any other foreign keys that we don't reassign (checked this, and there aren't many, fortunately. In fact, I don't think much harm can be done at all - at the moment).

See merge request !1985
2016-09-30 07:31:02 +00:00
James Lopez 9e0b7c630f updated attribute cleaner to use allowed keyword and reject attributes 2016-09-27 16:12:08 +02:00
James Lopez 4c480be39b Prevent claiming associated model IDs via import and added relevant specs 2016-09-27 15:48:10 +02:00
James Lopez fc548207d5 fix model order in import/export config and 1to1 relation issue. Added relevant specs. 2016-09-27 14:56:33 +02:00
James Lopez a9f1e972ec Fixes issue with rails reserved keyword type exporting/importing services. Also fixed CustomIssueTrackerService title setter and added relevant specs. 2016-09-26 09:49:05 +02:00
James Lopez ad62a25e0f fix import/export security specs after merge 2016-09-21 15:47:11 -05:00
James Lopez 3c7feaf3b1 Fixed label color issue and added Import/Export versioning table 2016-09-19 09:18:37 +02:00
James Lopez e74b7d665b squashed - Fix DB exceptions raised importing some specific projects.
Better import of labels, milestones and protected branches. Updated relevant specs.
Loose pipeline validation on importing, so it does not fail when there are missing fields, which are not validated at DB level. Also, updated spec with relevant test.
2016-09-19 09:18:37 +02:00
Alejandro Rodríguez 02bc717f34 Update references to deprecated `repos_path` configuration key to avoid
errors on updates from older versions
2016-09-14 21:19:24 -03:00
Felipe Artur 892dea6771 Project tools visibility level 2016-09-01 11:47:59 -03:00
James Lopez c42f5f8b56 refactor parse_hash based on feedback 2016-08-11 21:42:34 +02:00
James Lopez efab1677a7 Fix attribute inclusion in import/export config ignored in some cases 2016-08-11 12:49:07 +02:00
James Lopez f8e8547980 fix MR source project assignment 2016-08-09 11:03:02 +02:00
Douwe Maan a77394a7e5 Merge branch 'rubocop/EmptyLinesAroundModuleBody' into 'master'
Enable Style/EmptyLinesAround Module/Class Body cop

## What does this MR do?
Enable a new rubocop cops as discussed here: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1877/diffs#note_13021078

## Are there points in the code the reviewer needs to double check?
May be a good idea to send a message to people to prevent `master` red because of the new cop.

## Why was this MR needed?
We want to improve code style and not waste endbosses time checking style manually

## What are the relevant issue numbers?

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [ ] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5692
2016-08-08 17:50:22 +00:00
James Lopez f04e9dad47 Support pending invitation project members importing projects 2016-08-08 17:25:17 +02:00
Gabriel Mazetto 5f6223cf9f Enable Style/EmptyLinesAroundClassBody cop 2016-08-06 03:52:24 +02:00
James Lopez f87eb25020 Fix Import/Export error checking versions 2016-08-03 13:00:34 +02:00
James Lopez f611b7b357 fix TODO comment [ci skip] 2016-08-02 12:05:05 +02:00
Rémy Coutable ab3dd9a106 Merge branch 'fix/importing-io-timing-issue' into 'master'
Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
2016-08-01 13:18:39 +00:00
James Lopez 81495528f9 refactored wait_for_archived_file method 2016-08-01 11:07:06 +02:00
James Lopez aad0ae7162 squashed - fixed label and milestone association problems, updated specs and refactored reader class a bit 2016-08-01 09:57:40 +02:00
James Lopez dad1d0b864 fix return value and spec 2016-08-01 09:15:11 +02:00
James Lopez 52bb564812 squashed - fix timing issues in prod importing projects
added changelog

fix specs

refactored code based on feedback

fix rubocop warning
2016-08-01 09:15:11 +02:00
James Lopez 32d8aa6d5e fix repo hooks missing on import
fix spec and added changelog
2016-07-28 17:41:17 +02:00
James Lopez 76771c2946 squashed - added avatar saver/restorer and specs
added spec for avatar saver

avatar saver!

added avatar restorer spec

fix spec

added avatar restorer class

fix export service

fix warnings, added changelog

fix spec

some refactoring based on feedback

fixed a few issues after testing i/e avatar
2016-07-19 13:06:06 +02:00
James Lopez bf1ea8c6fe Squashed - fix encoding issue
WIP - trying to replicate UTF-8 error

fix spec

fixing encoding issue and another spec, to do with MR diffs

fix issue and spec failure

Add changelog and bumped up I/E version

fix spec based on feedback - omitted target project
2016-07-19 09:59:54 +02:00
James Lopez 60a2b3eff2 allow empty repos on import/export 2016-07-18 11:02:07 +02:00
Douwe Maan 686381f7e6 Merge branch 'fix/project-export-filename' into 'master'
add project name and namespace to filename on project export

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/19848

 Export filename now includes project and namespace path

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5268
2016-07-15 19:21:35 +00:00
Douwe Maan 4859770e26 Merge branch 'fix/ee-to-ce-import' 2016-07-15 14:18:31 -05:00
James Lopez 9b86006ca1 Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into fix/project-export-filename 2016-07-15 17:05:05 +02:00
James Lopez 65549a5866 add project name and namespace to filename on project export
added changelog
2016-07-15 12:55:28 +02:00
James Lopez b1ab079fae fix updated_at not preserved after import - for GitLab projects 2016-07-14 16:03:00 +02:00
James Lopez 097706b458 fix EE => CE project import, and updated JSON spec 2016-07-14 12:07:54 +02:00
James Lopez feecb4afde Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into fix/import-event-error 2016-07-11 18:00:19 +02:00
James Lopez 0e5f0276ea squashed - refactor to cope with sub sub N relations probably using the sub_relations method recursively. 2016-07-11 17:21:22 +02:00
James Lopez 91a183bc57 fix log statements in import/export 2016-07-07 09:49:46 +02:00
James Lopez 09452715d3 fix typo 2016-07-06 11:09:07 +02:00
James Lopez 95f630daeb even more debug 2016-07-06 10:29:31 +02:00
Grzegorz Bizon 9e211091a8 Enable Style/EmptyLines cop, remove redundant ones 2016-07-01 21:56:17 +02:00
James Lopez f29c30475e use has_many relationship with events 2016-07-01 15:34:10 +02:00
James Lopez 3d2a736679 fixing events for import/export 2016-06-29 10:35:26 +02:00
James Lopez 46b89a270f Fix tmp file being deleted after the request plus some cleanup and improved erroring for this situation 2016-06-24 10:50:23 +02:00
James Lopez 16ed545e7c fix utc error 2016-06-22 14:12:50 +02:00
James Lopez 1b8a107325 fix wiki stuff 2016-06-16 10:59:36 +02:00
James Lopez 4bde59341f lots of refactoring again based on feedback. Changed the UI slightly and also fixed a small bug 2016-06-15 17:31:00 +02:00
James Lopez fc5f6943f8 yay finally importing working with the new services structure 2016-06-14 21:41:40 +02:00
James Lopez ff44198e17 few fixes after refactoring the whole UI stuff 2016-06-14 21:03:14 +02:00
James Lopez b53ed84843 adapted current services stuff to use new project import, plus fixes a few issues, updated routes, etc... 2016-06-14 20:32:19 +02:00
James Lopez 1d4243f5dd Merge branches 'feature/project-export-ui-experimental' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export-ui-experimental 2016-06-14 16:40:13 +02:00
James Lopez 862b359b9a fix merge issue 2016-06-14 16:37:41 +02:00
James Lopez 9ecebaaea1 adding notifications stuff and more refactoring for exporting projects 2016-06-14 16:31:03 +02:00
James Lopez fe370b1c39 new export stuff and view 2016-06-14 14:28:30 +02:00
James Lopez 3f7ed55011 lots of refactoring to reuse import service 2016-06-14 12:47:07 +02:00
James Lopez 1ea44ee750 Merge branches 'feature/project-export-ui-experimental' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export-ui-experimental 2016-06-14 11:06:22 +02:00
James Lopez 279412f90a updated relation_factory based on MR feedback 2016-06-14 10:55:24 +02:00
James Lopez 7779457907 fix merge 2016-06-14 10:20:47 +02:00
James Lopez 9ff1990422 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import 2016-06-14 10:18:03 +02:00
James Lopez f6ed7c8ff8 missed line break 2016-06-14 10:15:20 +02:00
James Lopez 4020b0f55f few changes based on MR feedback 2016-06-13 21:18:26 +02:00
James Lopez 903da37755 WIP - starting refactoring import/export to use services 2016-06-13 20:35:57 +02:00
James Lopez bd5c749f29 fix file.write 2016-06-13 18:24:21 +02:00
James Lopez f449eeb6f1 Merge branch 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export-ui-experimental 2016-06-13 17:56:50 +02:00
James Lopez 9fd35740b8 refactored notes logic 2016-06-13 17:09:21 +02:00
James Lopez 833dc3204d few more changes based on feedback 2016-06-13 16:55:51 +02:00
James Lopez f4d762d7c2 addressing MR feedback, few changes to members mapper 2016-06-13 16:12:40 +02:00
James Lopez c23aaf1318 Merge branches 'feature/project-export-ui-experimental' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export-ui-experimental 2016-06-13 15:40:16 +02:00
James Lopez e5cf4cd745 corrected a few warnings 2016-06-13 15:33:28 +02:00
James Lopez e83fc684a5 Merge branches 'feature/project-export-ui-experimental' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export-ui-experimental
# Conflicts:
#	Gemfile.lock
#	app/helpers/todos_helper.rb
#	app/models/todo.rb
#	app/views/projects/edit.html.haml
#	lib/gitlab/import_export/import_service.rb
2016-06-13 14:00:19 +02:00
James Lopez 6f0c503397 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import 2016-06-13 13:44:00 +02:00
James Lopez 86e4751011 fixed deprecation warning 2016-06-13 13:42:30 +02:00
James Lopez b07dc938b9 fixed specs and refactored a few things due to recent model changes and merge conflicts 2016-06-13 13:34:36 +02:00
James Lopez ad68bc63b5 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import
# Conflicts:
#	app/models/project.rb
#	db/schema.rb
#	lib/gitlab/import_export/import_export_reader.rb
2016-06-13 12:43:25 +02:00
James Lopez cc32260394 added comments to import export reader class 2016-06-13 12:32:15 +02:00
James Lopez 069bc26418 refactored loads of things due to commits to pipeline change 2016-06-13 10:55:54 +02:00
James Lopez 771f735109 few more modifications based on comments 2016-06-13 09:37:58 +02:00
James Lopez 8476f91a4e WIP - added missing notes, trying to fix specs 2016-06-03 17:28:08 +02:00
James Lopez 9d0038f2d7 started refactoring a bunch of stuff based on feedback 2016-06-03 12:56:29 +02:00
James Lopez 41c06c311b refactoring more things based on MR feedback 2016-06-02 14:44:59 +02:00
James Lopez a9fdf62b57 refactoring relation factory, changed from module to class 2016-06-02 14:07:09 +02:00
James Lopez 102074c801 more and more refactoring 2016-06-02 10:59:54 +02:00
James Lopez 7c8359b744 started refactoring some stuff based on MR feedback 2016-06-01 18:03:51 +02:00
James Lopez fa1884698d a few nice to have and updated changelog 2016-05-20 10:57:10 +02:00