Commit Graph

140 Commits

Author SHA1 Message Date
Kamil Trzciński e00e62c2c8 Merge branch 'backstage/gb/migrate-stages-statuses' into 'master'
Migrate CI/CD stages statuses

Closes #33453

See merge request !12584
2017-08-21 11:47:44 +00:00
Michael Kozono cb1e237583 Fix spec 2017-08-17 14:39:02 -07:00
Sean McGivern c3e0f31dfb Merge branch 'backstage/gb/migrations-tests-schema-version' into 'master'
Improve migrations / background migrations testing strategy

Closes #36303

See merge request !13589
2017-08-17 11:54:54 +00:00
Grzegorz Bizon d1cfa79dfc Merge branch 'backstage/gb/migrations-tests-schema-version' into backstage/gb/migrate-stages-statuses
* backstage/gb/migrations-tests-schema-version:
  Migrate down before each migration unit test
  Disable db_cleaner tables caching in migrations tests
  Extend migrations testing documentation
  Make it possible to provide schema version in tests
  Migrate db schema for context in :migration specs
2017-08-17 11:59:00 +02:00
Grzegorz Bizon 32d0983a4e Merge branch 'master' into backstage/gb/migrate-stages-statuses
* master: (1000 commits)
  Fix username autocomplete group name with no avatar alignment
  Fix 'Projected tags' typo in protected_tags_spec.rb
  Many Repo Fixes
  Repo Editor Fixes
  Docs: New index for permissions
  link article from CI index
  link tech articles from the landing page
  new articles come first
  fix relative link
  fix date format
  Fixed changed files dropdown not being shown
  Update publication date
  Remove deprecated field from workhorse API responses
  Fix API responses when dealing with txt files
  Make sure MySQL would not use CURRENT_TIMESTAMP
  Add two more project templates
  Allow usage of any_projects? with an Array
  Copyedit Artifactory and GitLab article
  Rename Artifactory and GitLab article file
  Display GPG status loading spinner only when Ajax request is made
  ...
2017-08-16 13:18:03 +02:00
Grzegorz Bizon e2030ca019 Extend migrations testing documentation 2017-08-16 13:15:41 +02:00
Rémy Coutable 7b68bdce15 Merge branch 'rs-trailingwhitespace-cop' into 'master'
Enable Layout/TrailingWhitespace cop and auto-correct offenses

See merge request !13573
2017-08-16 10:27:46 +00:00
Jarka Kadlecova 04ee970cc1 Don't create event in Merge Request Create Service 2017-08-16 09:41:52 +00:00
Robert Speicher 4edfad9678 Enable Layout/TrailingWhitespace cop and auto-correct offenses 2017-08-15 13:44:37 -04:00
Bob Van Landuyt 2ea8442ff3 Move the personal snippet uploads from `system` to `-/system`
Update the markdown unconditionally since the move might have been
done before, but the markdown not updated.
2017-08-11 18:17:45 +02:00
Bob Van Landuyt b8ae15397f Update migrations to move directly into the `-/system` folder 2017-08-11 18:15:02 +02:00
Bob Van Landuyt f0f4506775 Don't update upload paths twice
This will be done in
20170717150329_enqueue_migrate_system_uploads_to_new_folder.rb
instead.
2017-08-11 18:15:02 +02:00
Bob Van Landuyt 969ccec7bb Don't rename the system namespace 2017-08-11 18:15:02 +02:00
Dmitriy Zaporozhets 1439115183 Merge branch 'gitaly-find-commit' into 'master'
Incorporate Gitaly's CommitService.FindCommit RPC

Closes gitaly#402

See merge request !13094
2017-08-08 14:27:22 +00:00
Alejandro Rodríguez 3ce6f03f14 Incorporate Gitaly's CommitService.FindCommit RPC 2017-08-07 23:33:43 -04:00
Jarka Kadlecova 0f9bde41fc Store & use ConvDev percentages returned by Version app 2017-08-07 20:52:08 +02:00
Sean McGivern f2d50af917 Migrate MR commits and diffs to new tables
Previously, we stored these as serialised fields - `st_{commits,diffs}` - on the
`merge_request_diffs` table. These now have their own tables -
`merge_request_diff_{commits,diffs}` - with a column for each attribute of the
serialised data.

Add a background migration to go through the existing MR diffs and migrate them
to the new format. Ignore any contents that cannot be displayed. Assuming that
we have 5 million rows to migrate, and each batch of 2,500 rows can be
completed in 5 minutes, this will take about 7 days to migrate everything.
2017-08-03 13:20:26 +01:00
Robert Speicher 72a7b30c9f Change all `:empty_project` to `:project` 2017-08-02 17:47:31 -04:00
Robert Speicher 9513bd18c4 Ensure all project factories use `:repository` trait or `:empty_project` 2017-08-01 14:51:52 -04:00
Gabriel Mazetto 597e619080 Rename ensure_dir_exist -> ensure_storage_path_exist 2017-08-01 07:44:25 +02:00
Gabriel Mazetto abb878326c Rename many path_with_namespace -> full_path 2017-08-01 07:26:58 +02:00
Grzegorz Bizon 9c10683b01 Merge branch 'master' into backstage/gb/migrate-stages-statuses
* master:
  Fix background migration cleanup specs
  Fix JS; make buttons sr accessibile; fix overlay
  Isolate stage_id reference clean up migration
  Fix eslint
  Make sidebar accessible on mobile
  Add mobile navigation on project page
  Fix database schema version number
  Fix background migrations module specs
  Remove migration dependency from stage_id migration
  Remove obsolete argument from bg migrations code
  Add specs for stage_id reference cleanup migration
  Add pending set of specs for stage_id cleanup migration
  Rename stage_id reference clean up migration
  Implement build stage_id reference migration clean up
2017-07-20 12:42:01 +02:00
Yorick Peterse 001dd56e7b Merge branch 'backstage/gb/build-stage-id-ref-bg-migration-cleanup' into 'master'
Implement build stage_id reference migration clean up

Closes #34893

See merge request !12785
2017-07-20 10:10:36 +00:00
Grzegorz Bizon 54efa041d7 Merge branch 'master' into backstage/gb/migrate-stages-statuses
* master: (319 commits)
  remove redundant changelog entries
  Merge branch '24570-use-re2-for-user-supplied-regexp-9-3' into 'security-9-3'
  Merge branch '33303-404-for-unauthorized-project' into 'security-9-3'
  Merge branch '33359-pers-snippet-files-location' into 'security-9-3'
  Merge branch 'bvl-remove-appearance-symlink' into 'security-9-3'
  Hide description about protected branches to non-member
  Update CHANGELOG.md for 9.0.11
  Update CHANGELOG.md for 9.1.8
  Update CHANGELOG.md for 8.17.7
  Update CHANGELOG.md for 9.2.8
  Update CHANGELOG.md for 9.3.8
  Respect blockquote line breaks in markdown
  35209 Add wip message to new navigation preference section
  Add github imported projects count to usage data
  Add versions to Prometheus metrics doc
  Add Bulgarian translations of Pipeline Schedules
  Add Esperanto translations of Pipeline Schedules
  Add Traditional Chinese in HongKong translations of Pipeline Schedules
  Add Simplified Chinese translations of Pipeline Schedules
  Resolve "Clarify k8s service keys"
  ...

Conflicts:
	db/schema.rb
2017-07-20 11:48:23 +02:00
Grzegorz Bizon 841de9752f Fix background migration cleanup specs
We need to use a spy because an `after` RSpec hook is also going to call
the migration we want to test, so we need to use `have_received`
expectation. See gitlab-org/gitlab-ce#35351 for more details.
2017-07-20 10:53:52 +02:00
Sean McGivern 88df076fae Merge branch '33359-pers-snippet-files-location' into 'security-9-3'
Use uploads/system directory for personal snippets

See merge request !2123
2017-07-19 22:28:27 -05:00
Douwe Maan 3a7b724f6a Merge branch 'bvl-remove-appearance-symlink' into 'security-9-3'
Remove the `appearance` symlink that was previously missed

See merge request !2124
2017-07-19 22:28:17 -05:00
Grzegorz Bizon 98f5c00e7a Merge branch 'master' into backstage/gb/build-stage-id-ref-bg-migration-cleanup
* master: (59 commits)
  Resolve "Clarify k8s service keys"
  Add Portuguese Brazil translations of Commits Page & Pipeline Charts
  Add Japanese Translation to i18n
  Update Prometheus gem to version that explicitly calls `munmap`
  Simplify width for dropdown-menu on mobile
  Update CHANGELOG.md for 9.3.7
  Remove developer documentation about not describing symbols
  Incorporate Gitaly's Commits#between RPC
  Adapt to new Gitaly commit message format
  Remove transitions on nav link hover
  Provide option to trigger build only for official CE and EE repos in .com
  Fix queries duration sorting in Performance Bar
  Rename Project nav items
  Add structured logging for Rails processes
  Disable Rails logging in CI test environments
  Fix download artifacts button alignment
  Update avatar border to be opaque for better stacking
  Fixed typos
  Fix typos
  Fix external issue trackers redirect
  ...

Conflicts:
	db/schema.rb
2017-07-19 13:21:01 +02:00
Grzegorz Bizon a6d1e92d98 Isolate stage_id reference clean up migration
This addreses a review remarks discussed in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12785/diffs#note_35276344
2017-07-19 13:19:27 +02:00
Bob Van Landuyt 79f591df4d Move the `uploads/system` folder to `uploads/-/system`
Without downtime, so we need the symlinks
2017-07-18 15:38:54 +02:00
Grzegorz Bizon 2deeac5643 Use batching to clear orphans in head_pipeline migration 2017-07-18 12:07:12 +02:00
Grzegorz Bizon 73c7b96885 Remove migration dependency from stage_id migration 2017-07-18 11:51:47 +02:00
Grzegorz Bizon 5c3fd67075 Add specs for stage_id reference cleanup migration 2017-07-18 10:36:12 +02:00
Grzegorz Bizon fa3acb3bb6 Add pending set of specs for stage_id cleanup migration 2017-07-18 10:36:12 +02:00
Grzegorz Bizon bb67b4749b Update version number of stages statuses migration 2017-07-11 14:31:04 +02:00
Grzegorz Bizon 65b3c22009 Fix pipeline stages statuses migration specs 2017-07-11 12:28:04 +02:00
Grzegorz Bizon 1d087e0736 Freeze mutable constants in stages migration code 2017-07-11 12:05:04 +02:00
Paul Charlton cb3b4a15e6 Support multiple Redis instances based on queue type 2017-07-11 03:35:47 +00:00
Grzegorz Bizon 7082530d55 Schedule stages statuses bg migrations in batches 2017-07-10 15:42:19 +02:00
Grzegorz Bizon 6120dc261a Merge branch 'master' into backstage/gb/migrate-stages-statuses
* master: (587 commits)
  Increase GitLab QA screenshots size
  Fix QA test scenario for creating a new project
  Configure headless chrome screenshots correctly
  Allow passing exceptions when creating project services
  Also inject new route helpers into includers of GitlabRoutingHelper
  Remove many N+1 queries with merge requests API
  fix milestones finder failing spec
  Make wrong migrations idempotent
  Remove double border on last group row
  Update tests to test what the component renders and what the clicks calls
  Username and password are no longer stripped from import url on import
  Change `sign_out` usage to `gitlab_sign_out` in one spec
  Send Gitaly Repository with /api/internal/allowed
  Resolve "Liberation Mono weird font rendering on Fedora/openSUSE/other Linux distros"
  Force mobile view for admin projects
  fix spec
  Mark a subgroup-related spec as PostgreSQL-only
  Use vue .prevent to prevent the click
  Add members-list class to admin projects page
  Fix another typo in SHA attribute spec
  ...

Conflicts:
	db/post_migrate/20170628080858_migrate_stage_id_reference_in_background.rb
	db/schema.rb
	lib/gitlab/background_migration/migrate_build_stage_id_reference.rb
	spec/migrations/migrate_stage_id_reference_in_background_spec.rb
2017-07-10 13:40:11 +02:00
Grzegorz Bizon 320229e12a Do not schedule bg migration when it is not needed 2017-07-07 15:50:33 +02:00
Grzegorz Bizon c467451ea6 Schedule stage_id bg migrations in batches of 10 2017-07-07 15:45:32 +02:00
Grzegorz Bizon b41b4d2e6a Use new `each_batch` helper instead of custom one
In stage_id backgrond migration.
2017-07-07 15:19:43 +02:00
Grzegorz Bizon fb89ba2482 Schedule stage_id background migration in ranges 2017-07-07 15:08:15 +02:00
Grzegorz Bizon 24a4199ac5 Reduce a delay between stage_id scheduled migrations 2017-07-07 15:08:15 +02:00
Grzegorz Bizon 14e45c424f Do not compare float with integer in migrations specs 2017-07-07 15:08:15 +02:00
Grzegorz Bizon d953f1762e Improve readability of build stage id migration query 2017-07-07 15:08:15 +02:00
Grzegorz Bizon 01128b130b Use integers to schedule delayed background migrations 2017-07-07 15:08:15 +02:00
Grzegorz Bizon 825521539d Improve specs for background stage_id ref migration 2017-07-07 15:08:15 +02:00
Grzegorz Bizon 00ca748108 Remove unused background migrations matcher 2017-07-07 15:08:15 +02:00