Commit Graph

65 Commits

Author SHA1 Message Date
Grzegorz Bizon 5331a49d0b Merge branch 'master' into fix/gb/improve-updating-column-in-batches-helper
* master: (56 commits)
  File view buttons
  Don't reset the session when the example failed, because we need capybara-screenshot to have access to it
  Resolve "MR comment + system note highlight don't have the same width"
  Add feature spec for dashboard state filter tabs
  Wording of Mysql support.
  a new feature checklist and more elaborate documentation requirements
  Filter archived project in API v3 only if param present
  Revert to using links instead of buttons in Issuable Index tabs.
  Do not run the codeclimate job on docs-only changes
  Only show gray footer space if environment actions exist
  Migrate Gitlab::Git::Blob.find to Gitaly
  Backport filtered search lazy token consistent state fix
  Add a comment explaining how the branch clean up happens
  Fix Github::Representation::PullRequest#source_branch_exists?
  Add CHANGELOG
  Fix GitHub importer performance on branch existence check
  Rebuild the dynamic path before validating it
  Rename stage ref migration specs to match a class name
  Enable Style/DotPosition Rubocop 👮
  Revert "Merge branch 'winh-merge-request-related-issues' into 'master'"
  ...

Conflicts:
	db/post_migrate/20170526185921_migrate_build_stage_reference.rb
2017-06-22 12:37:38 +02:00
Grzegorz Bizon 0430b76441 Enable Style/DotPosition Rubocop 👮 2017-06-21 13:48:12 +00:00
Grzegorz Bizon e4d42a62d9 Raise if updating columns in batches within a transaction 2017-06-21 14:31:49 +02:00
Yorick Peterse 9a3bea6707
Put an upper limit on update batches
When using update_column_in_batches the upper limit on the batch size is
now 1000. This ensures that for very large tables we don't lock tens of
thousands of rows during the update. This in turn should reduce the
likelyhood of running into deadlocks.
2017-06-19 14:18:41 +02:00
blackst0ne bc00806a4e Add database helpers 'add_timestamps_with_timezone' and 'timestamps_with_timezone' 2017-06-13 22:44:13 +11:00
Nick Thomas eb21f93267 Only use DROP INDEX CONCURRENTLY on postgreql 9.2+ 2017-05-25 18:12:42 +01:00
Bob Van Landuyt f62f0c258d Fixes for the rename reserved paths helpers 2017-05-18 20:08:32 +00:00
Douwe Maan beec97f6cf Disallow NULL on renamed column after default has been set 2017-05-15 16:01:09 -05:00
Rémy Coutable ae8511df13 Merge branch 'rename-column-concurrently-defaults' into 'master'
Fix adding defaults for concurrent column renames

See merge request !11335
2017-05-15 07:50:44 +00:00
James Lopez 48ffec9e74 Update rename_base.rb 2017-05-12 16:00:18 +00:00
Bob Van Landuyt 3760a7c230 Only rename namespaces and projects with an invalid path
For exact matches, not namespaces that end with an invalid path
2017-05-12 17:46:16 +02:00
Yorick Peterse 3fc20c0117
Fix adding defaults for concurrent column renames
By adding the default value _after_ adding the column we avoid updating
all rows in a table, saving a lot of time and unnecessary work in the
process.
2017-05-12 16:27:30 +02:00
Bob Van Landuyt 49fb31db41 Add a new column before creating rename triggers
MySQL doesn't allow us to create a trigger for a column that doesn't
exist yet. Failing with this error:

```
Mysql2::Error: Unknown column 'build_events' in 'NEW': CREATE TRIGGER trigger_6a80c097c862_insert
BEFORE INSERT
ON `services`
FOR EACH ROW
SET NEW.`build_events` = NEW.`job_events`
```

Creating the new column before creating the trigger avoids this.
2017-05-11 16:29:42 +02:00
Bob Van Landuyt 29f2001102 Update comments 2017-05-02 13:49:09 +02:00
Bob Van Landuyt 2e2a63c866 Rename child namespaces in migrationhelpers 2017-05-01 11:14:24 +02:00
Bob Van Landuyt b85f2fa4c7 Clear html cache for a projects milestones 2017-05-01 11:14:24 +02:00
Bob Van Landuyt ce0102d8b7 Remove dependecy on `User` 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 99a03fd6e9 Move ReservedPathsMigration into V1 namespace 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 12735eefcd Minor style adjustments 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 39efd0c030 Clear cached markdown after renaming projects 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 389057f001 Rename Projects & Namespaces based on entire paths 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 27f54bebb2 Use objects for renaming namespaces and projects 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 579d8891d5 Rename projects in a migrationhelper 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 7508ee5667 Make renaming records in the database reusable
So we can use it for projects
2017-05-01 11:14:24 +02:00
Bob Van Landuyt 58bc628d30 Rename namespace-paths in a migration helper 2017-05-01 11:14:24 +02:00
Bob Van Landuyt 2f95e6a0c4 Move `replace_sql` into `Database::MigrationHelpers` 2017-05-01 11:14:24 +02:00
Douwe Maan 3584f7dd52 Merge branch 'zero-downtime-migrations' into 'master'
Prepare for zero downtime migrations

See merge request !9976
2017-04-12 20:43:12 +00:00
Yorick Peterse 223d8a3d26
Prepare for zero downtime migrations
Starting with GitLab 9.1.0 we will no longer allow downtime migrations
unless absolutely necessary. This commit updates the various developer
guides and adds code that is necessary to make zero downtime migrations
less painful.
2017-04-12 18:15:19 +02:00
Tiago Botelho c41df67676 removes redundant code from database.rb 2017-04-11 20:20:19 +01:00
blackst0ne 9997c58fc0 Add remove_concurrent_index to database helper 2017-04-06 09:53:57 +11:00
Douwe Maan 1fe7501b49 Revert "Prefer leading style for Style/DotPosition"
This reverts commit cb10b725c8929b8b4460f89c9d96c773af39ba6b.
2017-02-23 09:33:05 -06:00
Douwe Maan bdbc7d967a Revert "Enable Style/BarePercentLiterals"
This reverts commit 96bef54154e669f9a3e92c3a4bc76c0be3a52e48.
2017-02-23 09:32:42 -06:00
Douwe Maan 6023fbfd0f Enable Style/EmptyLineBetweenDefs 2017-02-23 09:32:22 -06:00
Douwe Maan 206953a430 Prefer leading style for Style/DotPosition 2017-02-23 09:32:22 -06:00
Douwe Maan baafd8de26 Enable Style/BarePercentLiterals 2017-02-23 09:31:57 -06:00
Yorick Peterse 79696f5b7a
Hash concurrent foreign key names similar to Rails
This was initially not implemented simply because I forgot about the
size limit of constraint names in PostgreSQL (63 bytes). Using the old
technique we can't add foreign keys for certain tables. For example,
adding a foreign key on
protected_branch_merge_access_levels.protected_branch_id would lead to
the following key name:

    fk_protected_branch_merge_access_levels_protected_branches_protected_branch_id

This key is 78 bytes long, thus violating the PostgreSQL size
requirements.

The hashing strategy is copied from Rails' foreign_key_name() method,
which unfortunately is private and subject to change without notice.
2017-02-21 15:07:02 +01:00
Yorick Peterse a97dcc077c
Add method for creating foreign keys concurrently
This method allows one to create foreign keys without blocking access to
the source table, but only on PostgreSQL.

When creating a regular foreign key the "ALTER TABLE" statement used for
this won't return until all data has been validated. This statement in
turn will acquire a lock on the source table. As a result this lock can
be held for quite a long amount of time, depending on the number of rows
and system load.

By breaking up the foreign key creation process in two steps (creation,
and validation) we can reduce the amount of locking to a minimum.
Locking is still necessary for the "ALTER TABLE" statement that adds the
constraint, but this is a fast process and so will only block access for
a few milliseconds.
2017-02-10 21:51:09 +01:00
James Lopez 69ecd951a9 refactor fetcher and fixed specs 2017-01-17 11:32:55 +01:00
James Lopez f8acc7ea77 fixing spec failures 2016-11-17 08:22:55 +01:00
James Lopez 1a4ff5d720 Added code events spec and logic. Also fixed SQL issues and refactored the code a bit. 2016-11-17 08:22:55 +01:00
James Lopez 470e39d64f WIP - refactored some arel queries 2016-11-17 08:22:54 +01:00
Timothy Andrew 244ec0a84c Implement fourth round of comments from @DouweM.
- Pluralize summary titles
- Remove the `run_query` method - always return sql strings from the
  `date_time_sql` methods
2016-09-21 09:57:14 +05:30
Timothy Andrew 6df2d57394 Improve indentation in `Gitlab::Database::Median` 2016-09-21 02:18:57 +05:30
Timothy Andrew 918e589c2b Implement a second round of review comments from @DouweM.
- Don't use `TableReferences` - using `.arel_table` is shorter!
- Move some database-related code to `Gitlab::Database`
- Remove the `MergeRequest#issues_closed` and
  `Issue#closed_by_merge_requests`  associations. They were either
  shadowing or were too similar to existing methods. They are not being
  used anywhere, so it's better to remove them to reduce confusion.
- Use Rails 3-style validations
- Index for `MergeRequest::Metrics#first_deployed_to_production_at`
- Only include `CycleAnalyticsHelpers::TestGeneration` for specs that
  need it.
- Other minor refactorings.
2016-09-21 00:47:37 +05:30
Timothy Andrew 231a9f5b87 Fix rubocop spec.
And `scss_lint`
2016-09-20 18:26:31 +05:30
Timothy Andrew 71d4bf721b Implement (some) comments from @DouweM's review.
- Move things common to `Issue` and `MergeRequest` into `Issuable`
- Move more database-specific functions into `Gitlab::Database`
- Indentation changes and other minor refactorings.
2016-09-20 18:13:11 +05:30
Timothy Andrew 8957293d9b Implement review comments from @yorickpeterse
1. Change multiple updates to a single `update_all`

2. Use cascading deletes

3. Extract an average function for the database median.

4. Move database median to `lib/gitlab/database`

5. Use `delete_all` instead of `destroy_all`

6. Minor refactoring
2016-09-20 16:05:25 +05:30
Drew Blessing e000f02bd3 Add support for column limits in add_column_with_default 2016-09-15 21:59:58 -05:00
Stan Hu f700f3ec5d Disable statement timeout outside of transaction and during adding concurrent index 2016-07-14 18:50:46 -07:00
Stan Hu e5c7e11840 Disable PostgreSQL statement timeout during migrations
Long-running migrations may take more than the timeout allowed by
the database. Disable the session's statement timeout to ensure
migrations don't get killed prematurely.
2016-07-14 18:40:54 -07:00