Commit Graph

539 Commits

Author SHA1 Message Date
GitLab Bot faec73b0fe Add latest changes from gitlab-org/gitlab@master 2021-10-22 12:09:12 +00:00
GitLab Bot 229c115027 Add latest changes from gitlab-org/gitlab@master 2021-10-22 00:13:18 +00:00
GitLab Bot b428f0ed8d Add latest changes from gitlab-org/gitlab@master 2021-10-21 21:14:18 +00:00
GitLab Bot 046d6f5277 Add latest changes from gitlab-org/gitlab@master 2021-10-11 15:13:01 +00:00
GitLab Bot 85c68f14bf Add latest changes from gitlab-org/gitlab@master 2021-10-06 00:11:56 +00:00
GitLab Bot cee701c964 Add latest changes from gitlab-org/gitlab@master 2021-09-29 15:11:47 +00:00
GitLab Bot b119503b70 Add latest changes from gitlab-org/gitlab@master 2021-09-14 15:12:05 +00:00
GitLab Bot ceb0c326ae Add latest changes from gitlab-org/gitlab@master 2021-09-02 03:09:04 +00:00
GitLab Bot 34d6e7c91b Add latest changes from gitlab-org/gitlab@master 2021-08-31 18:10:24 +00:00
GitLab Bot a56971e97f Add latest changes from gitlab-org/gitlab@master 2021-08-26 18:10:35 +00:00
GitLab Bot 0fd2296553 Add latest changes from gitlab-org/gitlab@master 2021-08-17 09:10:02 +00:00
GitLab Bot 7134e029c5 Add latest changes from gitlab-org/gitlab@master 2021-08-13 15:11:15 +00:00
GitLab Bot 65688a5092 Add latest changes from gitlab-org/gitlab@master 2021-08-09 09:22:41 +00:00
GitLab Bot 42d930072f Add latest changes from gitlab-org/gitlab@master 2021-08-03 09:15:56 +00:00
GitLab Bot 1930898566 Add latest changes from gitlab-org/gitlab@master 2021-08-02 09:10:09 +00:00
GitLab Bot f4d51a9f71 Add latest changes from gitlab-org/gitlab@master 2021-07-29 15:09:48 +00:00
GitLab Bot 112fe349cb Add latest changes from gitlab-org/gitlab@master 2021-07-28 21:08:53 +00:00
GitLab Bot a8f5aaa708 Add latest changes from gitlab-org/gitlab@master 2021-07-21 15:08:52 +00:00
GitLab Bot c13bf68077 Add latest changes from gitlab-org/gitlab@master 2021-07-21 00:09:07 +00:00
GitLab Bot dd1c093e28 Add latest changes from gitlab-org/gitlab@master 2021-07-12 18:09:09 +00:00
GitLab Bot 3462d7613f Add latest changes from gitlab-org/gitlab@master 2021-07-07 00:07:23 +00:00
GitLab Bot e5f2a04e9d Add latest changes from gitlab-org/gitlab@master 2021-06-29 12:08:48 +00:00
GitLab Bot be8b477154 Add latest changes from gitlab-org/gitlab@master 2021-06-29 03:07:32 +00:00
GitLab Bot e4632f4c63 Add latest changes from gitlab-org/gitlab@master 2021-06-28 18:08:39 +00:00
GitLab Bot 59469a1782 Add latest changes from gitlab-org/gitlab@master 2021-06-28 15:08:03 +00:00
GitLab Bot 03c5d7f2c1 Add latest changes from gitlab-org/gitlab@master 2021-06-15 03:10:34 +00:00
GitLab Bot ebc589e90c Add latest changes from gitlab-org/gitlab@master 2021-06-10 03:10:01 +00:00
GitLab Bot 0ebbf19f2d Add latest changes from gitlab-org/gitlab@master 2021-06-08 15:10:00 +00:00
GitLab Bot 79f98200f8 Add latest changes from gitlab-org/gitlab@master 2021-06-07 15:09:56 +00:00
GitLab Bot f7bc7dc5ea Add latest changes from gitlab-org/gitlab@master 2021-06-01 21:10:06 +00:00
GitLab Bot 389312d009 Add latest changes from gitlab-org/gitlab@master 2021-05-19 09:10:19 +00:00
GitLab Bot 49bb78aac3 Add latest changes from gitlab-org/gitlab@master 2021-05-17 18:10:42 +00:00
GitLab Bot cf05fd7f39 Add latest changes from gitlab-org/gitlab@master 2021-05-05 15:10:05 +00:00
GitLab Bot bc62085601 Add latest changes from gitlab-org/gitlab@master 2021-04-23 18:10:18 +00:00
GitLab Bot 10130901f1 Add latest changes from gitlab-org/gitlab@master 2021-04-15 15:09:11 +00:00
GitLab Bot a3dfd311f4 Add latest changes from gitlab-org/gitlab@master 2021-04-14 15:09:04 +00:00
GitLab Bot da07b341fd Add latest changes from gitlab-org/gitlab@master 2021-04-05 21:09:19 +00:00
GitLab Bot 05003789d9 Add latest changes from gitlab-org/gitlab@master 2021-04-01 15:08:54 +00:00
GitLab Bot cd40f83527 Add latest changes from gitlab-org/gitlab@master 2021-02-23 09:10:45 +00:00
GitLab Bot bf217da41b Add latest changes from gitlab-org/gitlab@master 2021-02-11 21:09:00 +00:00
GitLab Bot 02c3b2af44 Add latest changes from gitlab-org/gitlab@master 2021-02-10 21:09:24 +00:00
GitLab Bot ec0ecba05c Add latest changes from gitlab-org/gitlab@master 2021-02-10 12:09:45 +00:00
GitLab Bot 64f7eb2b37 Add latest changes from gitlab-org/gitlab@master 2021-02-05 12:09:31 +00:00
GitLab Bot bcd0f3a2f6 Add latest changes from gitlab-org/gitlab@master 2021-02-02 06:09:47 +00:00
GitLab Bot 10052df753 Add latest changes from gitlab-org/gitlab@master 2021-01-29 15:09:40 +00:00
GitLab Bot a4df3f0dbb Add latest changes from gitlab-org/gitlab@master 2021-01-29 12:09:29 +00:00
GitLab Bot 6bc190088e Add latest changes from gitlab-org/gitlab@master 2021-01-25 12:09:07 +00:00
GitLab Bot 852877d997 Add latest changes from gitlab-org/gitlab@master 2021-01-22 18:09:10 +00:00
GitLab Bot 16e3c34cac Add latest changes from gitlab-org/gitlab@master 2021-01-22 06:08:52 +00:00
GitLab Bot c2dcc63e6d Add latest changes from gitlab-org/gitlab@master 2021-01-11 09:10:46 +00:00
GitLab Bot 2c2b5aeac0 Add latest changes from gitlab-org/gitlab@master 2021-01-05 15:10:02 +00:00
GitLab Bot de64b03b15 Add latest changes from gitlab-org/gitlab@master 2020-12-23 06:10:22 +00:00
GitLab Bot b07852468f Add latest changes from gitlab-org/gitlab@master 2020-12-15 18:10:06 +00:00
GitLab Bot a8704bd33c Add latest changes from gitlab-org/gitlab@master 2020-12-11 15:10:04 +00:00
GitLab Bot cba8ff6440 Add latest changes from gitlab-org/gitlab@master 2020-12-10 18:10:16 +00:00
GitLab Bot 3e06afc4cd Add latest changes from gitlab-org/gitlab@master 2020-12-10 12:09:43 +00:00
GitLab Bot 109562e64e Add latest changes from gitlab-org/gitlab@master 2020-12-09 09:09:47 +00:00
GitLab Bot 707742e59c Add latest changes from gitlab-org/gitlab@master 2020-12-08 09:09:41 +00:00
GitLab Bot b4ddc24045 Add latest changes from gitlab-org/gitlab@master 2020-12-08 00:09:45 +00:00
GitLab Bot ccdcf4e139 Add latest changes from gitlab-org/gitlab@master 2020-11-27 00:09:42 +00:00
GitLab Bot 4fc6f62c16 Add latest changes from gitlab-org/gitlab@master 2020-11-25 00:09:24 +00:00
GitLab Bot df40cd1c38 Add latest changes from gitlab-org/gitlab@master 2020-11-19 21:09:07 +00:00
GitLab Bot 5f362c717e Add latest changes from gitlab-org/gitlab@master 2020-11-12 06:09:02 +00:00
GitLab Bot db53c7d425 Add latest changes from gitlab-org/gitlab@master 2020-11-08 21:08:49 +00:00
GitLab Bot b4b6bff01d Add latest changes from gitlab-org/gitlab@master 2020-10-13 18:08:58 +00:00
GitLab Bot 16515bdfcb Add latest changes from gitlab-org/gitlab@master 2020-10-13 15:08:53 +00:00
GitLab Bot 960d1d4c59 Add latest changes from gitlab-org/gitlab@master 2020-10-09 15:09:18 +00:00
GitLab Bot 33882b0ed1 Add latest changes from gitlab-org/gitlab@master 2020-10-08 09:08:40 +00:00
GitLab Bot b6724a211e Add latest changes from gitlab-org/gitlab@master 2020-10-07 06:09:03 +00:00
GitLab Bot 4d922922a9 Add latest changes from gitlab-org/gitlab@master 2020-10-06 15:08:33 +00:00
GitLab Bot 251d3d2b23 Add latest changes from gitlab-org/gitlab@master 2020-10-02 09:08:33 +00:00
GitLab Bot 0b4bb101ea Add latest changes from gitlab-org/gitlab@master 2020-10-01 18:10:20 +00:00
GitLab Bot f4fb4d5948 Add latest changes from gitlab-org/gitlab@master 2020-09-25 15:09:36 +00:00
GitLab Bot 6f15c2c272 Add latest changes from gitlab-org/gitlab@master 2020-09-24 12:09:37 +00:00
GitLab Bot eb08c8e6f8 Add latest changes from gitlab-org/gitlab@master 2020-09-22 15:09:37 +00:00
GitLab Bot fe53f9c358 Add latest changes from gitlab-org/gitlab@master 2020-09-22 00:09:59 +00:00
GitLab Bot 7985071975 Add latest changes from gitlab-org/gitlab@master 2020-09-21 12:09:34 +00:00
GitLab Bot cf9694dd84 Add latest changes from gitlab-org/gitlab@master 2020-09-18 00:09:39 +00:00
GitLab Bot bf1600d157 Add latest changes from gitlab-org/gitlab@master 2020-09-16 18:09:47 +00:00
GitLab Bot b296ffa543 Add latest changes from gitlab-org/gitlab@master 2020-09-09 21:08:33 +00:00
GitLab Bot 4e06ca9e7d Add latest changes from gitlab-org/gitlab@master 2020-09-02 18:10:40 +00:00
GitLab Bot 3827baae07 Add latest changes from gitlab-org/gitlab@master 2020-08-13 00:10:06 +00:00
GitLab Bot bdca097916 Add latest changes from gitlab-org/gitlab@master 2020-08-11 03:11:00 +00:00
GitLab Bot 2948e306c0 Add latest changes from gitlab-org/gitlab@master 2020-07-24 18:09:45 +00:00
GitLab Bot 8e42824b11 Add latest changes from gitlab-org/gitlab@master 2020-07-14 06:09:17 +00:00
GitLab Bot d9aac35d79 Add latest changes from gitlab-org/gitlab@master 2020-07-13 18:09:16 +00:00
GitLab Bot 83fc2f3dc8 Add latest changes from gitlab-org/gitlab@master 2020-07-08 21:09:09 +00:00
GitLab Bot 4c5468b408 Add latest changes from gitlab-org/gitlab@master 2020-06-24 15:08:50 +00:00
GitLab Bot 963838b90c Add latest changes from gitlab-org/gitlab@master 2020-06-05 09:08:05 +00:00
GitLab Bot 340fd2966e Add latest changes from gitlab-org/gitlab@master 2020-06-04 00:08:17 +00:00
GitLab Bot f3e7bc8060 Add latest changes from gitlab-org/gitlab@master 2020-06-02 18:08:32 +00:00
GitLab Bot 21539fe9ab Add latest changes from gitlab-org/gitlab@master 2020-05-22 00:08:07 +00:00
GitLab Bot aebabf8f5a Add latest changes from gitlab-org/gitlab@master 2020-05-15 00:08:06 +00:00
GitLab Bot 67cd2904c9 Add latest changes from gitlab-org/gitlab@master 2020-05-14 21:07:52 +00:00
GitLab Bot 87f286558d Add latest changes from gitlab-org/gitlab@master 2020-05-14 15:08:14 +00:00
GitLab Bot c7ad2610df Add latest changes from gitlab-org/gitlab@master 2020-05-14 09:07:53 +00:00
GitLab Bot bc692af988 Add latest changes from gitlab-org/gitlab@master 2020-05-09 00:09:39 +00:00
GitLab Bot 00a8c64ffd Add latest changes from gitlab-org/gitlab@master 2020-04-15 06:09:49 +00:00
GitLab Bot 254ec28f54 Add latest changes from gitlab-org/gitlab@master 2020-04-09 21:09:19 +00:00
GitLab Bot a9ced7da44 Add latest changes from gitlab-org/gitlab@master 2020-04-09 12:09:24 +00:00
GitLab Bot 1219a9dce9 Add latest changes from gitlab-org/gitlab@master 2020-04-01 15:07:45 +00:00
GitLab Bot 8f9beefac3 Add latest changes from gitlab-org/gitlab@master 2020-03-23 12:09:47 +00:00
GitLab Bot 1635eacd2a Add latest changes from gitlab-org/gitlab@master 2020-03-23 06:09:28 +00:00
GitLab Bot f2dfd9ee81 Add latest changes from gitlab-org/gitlab@master 2020-03-22 15:09:49 +00:00
GitLab Bot dc889678d1 Add latest changes from gitlab-org/gitlab@master 2020-03-12 15:09:39 +00:00
GitLab Bot a8de96bff5 Add latest changes from gitlab-org/gitlab@master 2020-03-05 18:08:19 +00:00
GitLab Bot 77237c5a6b Add latest changes from gitlab-org/gitlab@master 2020-03-05 00:07:49 +00:00
GitLab Bot 951616a26a Add latest changes from gitlab-org/gitlab@master 2020-02-25 18:09:02 +00:00
GitLab Bot a8c82a6395 Add latest changes from gitlab-org/gitlab@master 2020-02-25 03:08:49 +00:00
GitLab Bot a6c2be7cd2 Add latest changes from gitlab-org/gitlab@master 2020-02-21 21:08:57 +00:00
GitLab Bot 55733b19c5 Add latest changes from gitlab-org/gitlab@master 2020-02-11 06:09:46 +00:00
GitLab Bot a89cb5cbdd Add latest changes from gitlab-org/gitlab@master 2020-02-06 21:08:48 +00:00
GitLab Bot 0d6fa03312 Add latest changes from gitlab-org/gitlab@master 2020-02-06 18:08:54 +00:00
GitLab Bot 76623c12c1 Add latest changes from gitlab-org/gitlab@master 2020-02-05 21:09:02 +00:00
GitLab Bot 50ae406553 Add latest changes from gitlab-org/gitlab@master 2020-01-30 12:08:54 +00:00
GitLab Bot fedf978f9a Add latest changes from gitlab-org/gitlab@master 2020-01-28 15:08:36 +00:00
GitLab Bot bbf6581214 Add latest changes from gitlab-org/gitlab@master 2020-01-28 06:09:01 +00:00
GitLab Bot 7873bb3c1a Add latest changes from gitlab-org/gitlab@master 2020-01-11 00:08:28 +00:00
GitLab Bot b5571e6e22 Add latest changes from gitlab-org/gitlab@master 2019-12-19 18:07:48 +00:00
GitLab Bot 69d6d3ca20 Add latest changes from gitlab-org/gitlab@master 2019-12-16 21:08:00 +00:00
GitLab Bot a19a376bf3 Add latest changes from gitlab-org/gitlab@master 2019-12-02 21:06:51 +00:00
GitLab Bot eac0da9a47 Add latest changes from gitlab-org/gitlab@master 2019-11-29 15:06:43 +00:00
GitLab Bot ef326e805a Add latest changes from gitlab-org/gitlab@master 2019-10-28 21:06:24 +00:00
GitLab Bot ad1e4b8fb8 Add latest changes from gitlab-org/gitlab@master 2019-10-21 15:05:58 +00:00
GitLab Bot 5c521d1f9b Add latest changes from gitlab-org/gitlab@master 2019-10-18 15:06:05 +00:00
GitLab Bot 25989ab7ef Add latest changes from gitlab-org/gitlab@master 2019-10-18 11:11:44 +00:00
GitLab Bot 7b8ec6e718 Add latest changes from gitlab-org/gitlab@master 2019-10-15 18:06:01 +00:00
GitLab Bot 3d13802bc2 Add latest changes from gitlab-org/gitlab@master 2019-10-15 09:06:09 +00:00
GitLab Bot 185f428fa5 Add latest changes from gitlab-org/gitlab@master 2019-10-07 12:06:18 +00:00
GitLab Bot d46287cc16 Add latest changes from gitlab-org/gitlab@master 2019-09-20 12:05:52 +00:00
Ash McKenzie 124bad7bd8
Use stub_rails_env for Rails.env stubbing
RailsHelpers.stub_rails_env takes care of
stubbing Rails.env
2019-09-05 12:54:03 +10:00
Hordur Freyr Yngvason d3c5ff7b72 Squash project templates on update
As per https://gitlab.com/gitlab-org/gitlab-ce/issues/46043, project
templates should be squashed before updating, so that repositories
created from these templates don't include the full history of the
backing repository.
2019-08-15 20:20:08 +00:00
Sean McGivern 930026f49b Remove mail_google_schema_whitelisting spec
This task was removed in 2014! The spec wasn't, probably because it
didn't fail as it was never executed. See
0fc5c80207.
2019-08-01 11:07:28 +01:00
Sean McGivern c4a8f45f7d Rename broken spec files
RSpec only executes files ending in _spec.rb.
2019-08-01 11:07:27 +01:00
Hordur Freyr Yngvason 012fe3141e Fix broken update_project_templates rake task
This rake task had been broken for a while. This fixes the breakages,
adds a test to help avoid future breakages, and adds a few ergonomic
improvements to the task itself.
2019-07-30 13:52:28 +00:00
Imre Farkas 313f145b55
Rake task to cleanup expired ActiveSession lookup keys
In some cases ActiveSession.cleanup was not called after authentication,
so for some user ActiveSession lookup keys grew without ever cleaning
up. This Rake task manually iterates over the lookup keys and removes
ones without existing ActiveSession.
2019-07-22 17:15:57 +02:00
Gabriel Mazetto d8cad8837c Display the amount for Hashed Storage migration/rollback correctly
Rake script considers the specified range, if present, to calculate
the affected amount of projects correctly.
2019-07-10 22:14:50 +01:00
Toon Claes dabd91b2c8 Add rake task to clean orphan artifact files
This adds the rake task rake
gitlab:cleanup:orphan_job_artifact_files. This rake task cleans all
orphan job artifact files it can find on disk.

 It performs a search on the complete folder of all artifacts on
disk. Then it filters out all the job artifact ID for which it could
not find a record with matching ID in the database. For these, the
file is deleted from disk.
2019-06-28 10:02:18 +02:00
Drew Blessing 05d5504d07 Sanitize LDAP output in Rake tasks
The various LDAP check Rake tasks have long supported a SANITIZE
environment variable. When present, identifiable information is
obscured such as user names and project/group names. Until now,
the LDAP check did not honor this. Now it will only say how many
users were found. This should at least give the indication that
the LDAP configuration found something, but will not leak what
it is. Resolves #56131
2019-06-21 05:49:56 -05:00
Bob Van Landuyt 9a8955d3c0 Merge branch 'gitaly-version-v1.43.0' into 'master'
Upgrade Gitaly to v1.43.0

See merge request gitlab-org/gitlab-ce!28867
2019-05-31 14:16:54 +00:00
Shinya Maeda 387a4f4b2c Remove legacy artifact related code
We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.
2019-05-31 10:49:17 +07:00
Zeger-Jan van de Weg 951afba624
Remove hook directory requirement from Shell
It used to be the case that GitLab created symlinks for each repository
to one copy of the Git hooks, so these ran when required. This changed
to set the hooks dynamically on Gitaly when invoking Git.

The side effect is that we didn't need all these symlinks anymore, which
Gitaly doesn't create anymore either. Now that means that the tests in
GitLab-Rails should test for it either.

Related: https://gitlab.com/gitlab-org/gitaly/issues/1392#note_175619926
2019-05-30 20:04:08 +02:00
Dominic Renner ad29681cab Replaced config access
Replaced Settings with Gitlab.config for uniformity
2019-03-18 17:26:18 +00:00
Gabriel Mazetto 523712d116 Fix storage migration check for non-empty queues
The previous code works fine when running specs, but the API
does not exist outside that environment.
2019-03-17 00:27:38 +01:00
Gabriel Mazetto 337977776a Prevent storage migration and rollback running at the same time
This is a small polishing on the storage migration and storage rollback
rake tasks. By aborting a migration while a rollback is already
scheduled we want to prevent unexpected consequences.
2019-03-12 17:51:05 +01:00
Jacob Vosmaer 078f11a6b9 Prepare test suite for switch to Gitaly-embedded Git hooks 2019-02-27 10:58:21 +00:00
Rémy Coutable 5c583c8e87 Merge branch 'master' into '46448-add-timestamps-for-each-stage-of-gitlab-rake-gitlab-backup-restore'
# Conflicts:
#   spec/tasks/gitlab/backup_rake_spec.rb
2019-02-04 14:11:04 +00:00
Gabriel Mazetto 7bc16889df Refactor Storage Migration
Specs were reviewed and improved to better cover the current behavior.
There was some standardization done as well to facilitate the
implementation of the rollback functionality.

StorageMigratorWorker was extracted to HashedStorage namespace were
RollbackerWorker will live one as well.
2019-01-25 20:26:35 +01:00
Gabriel Mazetto c2c34eba62 Prepare rake task for storage rollback
We are keeping compatibility with existing scheduled jobs.
2019-01-25 20:26:35 +01:00
Michael Kozono 9f80f04059 Prevent admins from attempting hashed storage migration on read only DB 2018-12-19 21:10:00 +00:00
Zeger-Jan van de Weg 89a407dc3b
Restore Object Pools when restoring an object pool
Pool repositories are persisted in the database, and when the DB is
restored, the data need to be restored on disk. This is done by
resetting the state machine and rescheduling the object pool creation.

This is not an exact replica of the state like at the time of the
creation of the backup. However, the data is consistent again.

Dumping isn't required as internally GitLab uses git bundles which
bundle all refs and include all objects in the bundle that they require,
reduplicating as more repositories get backed up. This does require more
data to be stored.

Fixes https://gitlab.com/gitlab-org/gitaly/issues/1355
2018-12-13 09:00:03 +01:00
Nick Thomas 7329480412
Fix gitlab:web_hook tasks 2018-12-06 18:37:05 +00:00
Michael Kozono 6855e6b586 Extract system check rake task logic
These changes make the code more reusable, testable, and most
importantly, overrideable.
2018-12-03 13:51:46 -08:00
Gabriel Mazetto fe2e6c6dc0 Remove Site Statistic
This approach caused many different problems as we tightened
the query execution timeout.
2018-11-26 23:39:05 +01:00
Toon Claes 1c481b7aac Enhance performance of counting local Uploads
Add an index to the `store` column on `uploads`. This makes counting
local uploads faster.

Also, there is no longer need to check for objects with `store = NULL`.
See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18557

---

### Query plans

Query:

```sql
SELECT COUNT(*)
FROM "uploads"
WHERE ("uploads"."store" = ? OR "uploads"."store" IS NULL)
```

#### Without index

```
gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                  QUERY PLAN
---------------------------------------------------------------------------------------------------------------
 Seq Scan on uploads  (cost=0.00..601729.54 rows=578 width=272) (actual time=6.170..2308.256 rows=545 loops=1)
   Filter: ((store = 1) OR (store IS NULL))
   Rows Removed by Filter: 4411957
 Planning time: 38.652 ms
 Execution time: 2308.454 ms
(5 rows)
```

#### Add index

```
gitlabhq_production=# create index uploads_tmp1 on uploads (store);
CREATE INDEX
```

#### With index

```
gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                          QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on uploads  (cost=11.46..1238.88 rows=574 width=272) (actual time=0.155..0.577 rows=545 loops=1)
   Recheck Cond: ((store = 1) OR (store IS NULL))
   Heap Blocks: exact=217
   ->  BitmapOr  (cost=11.46..11.46 rows=574 width=0) (actual time=0.116..0.116 rows=0 loops=1)
         ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..8.74 rows=574 width=0) (actual time=0.095..0.095 rows=545 loops=1)
               Index Cond: (store = 1)
         ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..2.44 rows=1 width=0) (actual time=0.020..0.020 rows=0 loops=1)
               Index Cond: (store IS NULL)
 Planning time: 0.274 ms
 Execution time: 0.637 ms
(10 rows)
```

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6070
2018-11-07 11:29:31 +01:00
George Tsiolis 733ae94921 Fix typos in comments and specs 2018-11-01 08:59:20 +02:00
Valery Sizov 15d011d64d Geo: sync disabled wikis. Stage 2
We started syncing all the wiki regardless of the fact it's disabled or
not. We couldn't do that in one stage because of needing of smoth update
and deprecating things. This is the second stage that finally removes
unused columns in the geo_node_status table.
2018-09-25 17:49:39 +03:00
Yorick Peterse ec33016bcd Include post migrations when loading the schema
See merge request gitlab-org/gitlab-ce!21689
2018-09-19 12:29:25 +00:00
DJ Mountney 60747672bb Fix the schema load test
And added changelog
2018-09-12 15:55:26 -07:00
DJ Mountney 76cfe4f1fd Ensure the schema is loaded with post_migrations included
If doing a schema load, the post_migrations should also be marked as up,
even if SKIP_POST_DEPLOYMENT_MIGRATIONS was set, otherwise future
migration runs will be broken.
2018-09-11 18:41:14 -07:00
Zeger-Jan van de Weg 3aedccb17a
Port cleanup tasks to use Gitaly
Rake tasks cleaning up the Git storage were still using direct disk
access, which won't work if these aren't attached. To mitigate a
migration issue was created.

To port gitlab:cleanup:dirs, and gitlab:cleanup:repos, a new RPC was
required, ListDirectories. This was implemented in Gitaly, through
https://gitlab.com/gitlab-org/gitaly/merge_requests/868.

To be able to use the new RPC the Gitaly server was bumped to v0.120.

This is an RPC that will not use feature gates, as this doesn't scale on
.com so there is no way to test it at scale. Futhermore, we _know_ it
doesn't scale, but this might be a useful task for smaller instances.

Lastly, the tests are slightly updated to also work when the disk isn't
attached. Eventhough this is not planned, it was very little effort and
thus I applied the boy scout rule.

Closes https://gitlab.com/gitlab-org/gitaly/issues/954
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/40529
2018-09-07 13:58:31 +02:00
Gabriel Mazetto 1a54986c16 Refactor SiteStatistics to extract refresh logic into a rake task 2018-08-16 21:31:19 +02:00
Shinya Maeda 63091cfe64 Add rake command to migrate archived traces from local storage to object storage 2018-08-16 14:28:47 +00:00
Douwe Maan d9614e701a Merge branch 'gitaly-install-path' into 'master'
Remove storage path dependency of gitaly install task

See merge request gitlab-org/gitlab-ce!21101
2018-08-14 08:59:10 +00:00
Alejandro Rodríguez eb1a3798ad Remove storage path dependency of gitaly install task 2018-08-12 18:52:44 -03:00
Michael Kozono a4351ac077 Add object storage related tests for `gitlab:cleanup:project_uploads` task 2018-08-03 04:36:43 +00:00
Sean McGivern 056025f78e Merge branch 'zj-backup-migration-done' into 'master'
Remove feature flags from lib/backup

Closes gitaly#749, gitaly#1212, and gitaly#1195

See merge request gitlab-org/gitlab-ce!20854
2018-08-02 12:35:35 +00:00
Michael Kozono 3cbd8b1343 Add local project uploads cleanup task 2018-07-31 11:09:42 -07:00
Zeger-Jan van de Weg d7afed34c4
Remove feature flags from lib/backup
Moved to OPT_OUT in 7d14b725a0, Now, by
removing the feature gates, this is an mandatory feature.

Related issues:
- https://gitlab.com/gitlab-org/gitaly/issues/526
- https://gitlab.com/gitlab-org/gitaly/issues/1194

Closes https://gitlab.com/gitlab-org/gitaly/issues/749
Closes https://gitlab.com/gitlab-org/gitaly/issues/1212
Closes https://gitlab.com/gitlab-org/gitaly/issues/1195
2018-07-31 15:24:19 +02:00
Zeger-Jan van de Weg f1f7bfc06f
Remove git rake tasks
These tasks are happening through housekeeping right now, by default
ever 10th push. This removes the need for these tasks.

Side note, this removes one of my first contributions to GitLab, as back
than I introduced these tasks through: 54e6c0045b

Closes https://gitlab.com/gitlab-org/gitaly/issues/768
2018-07-31 14:31:50 +02:00
Robert Speicher f5da4815a5
Speed up spec/tasks/gitlab/git_rake_spec.rb
Because no Git repository was actually created at the temporary path we
were using, `git fsck` would traverse up until it found a repository,
which in our case was the CE or EE repository.
2018-07-03 17:20:51 -05:00
DJ Mountney 2efe4a13b6 Fix static analysis failure 2018-06-27 14:09:06 -07:00
DJ Mountney 1ef3b3efbd Update tests for gitlab:db:configure 2018-06-27 13:42:22 -07:00
Will Chandler cc41a77183 Add timestamps to gitlab-rake gitlab:backup:restore
Adds a new method 'puts_time' that prepends the time of a
message when printing it.  All instances of 'progress.puts'
in the gitlab:backup:restore tasks are replaced with puts_time.
Example output:
  2018-06-03 16:33:25 -0400 -- Restoring uploads ..
Closes #46448
2018-06-14 23:03:15 -04:00
Jacob Vosmaer f376347f24 Find and mark more Git disk access locations, part 2 2018-06-12 14:33:37 +02:00
Gabriel Mazetto 36c3376475 Resolve "Hashed Storage: Make possible to migrate single project" 2018-06-07 15:40:44 +00:00
Jacob Vosmaer (GitLab) a0808df0b6 Find and mark more Git disk access locations 2018-06-05 15:51:14 +00:00
Gabriel Mazetto 8c5ce1b07c Specs covering listing all gitlab:storage:* types 2018-05-29 02:53:21 +02:00
Stan Hu e6fca03bbd Fix backup creation and restore for specific Rake tasks 2018-05-24 14:58:25 +00:00
Zeger-Jan van de Weg ec4423665c
Gitlab::Shell works on shard name, not path
Direct disk access is done through Gitaly now, so the legacy path was
deprecated. This path was used in Gitlab::Shell however. This required
the refactoring in this commit.

Added is the removal of direct path access on the project model, as that
lookup wasn't needed anymore is most cases.

Closes https://gitlab.com/gitlab-org/gitaly/issues/1111
2018-04-25 13:36:22 +02:00
Micaël Bergeron 4f24a65e45 fix the rake task from being to strict about Group 2018-04-02 08:19:34 -04:00
Micaël Bergeron 316eb6ef60 apply feedback 2018-03-30 08:39:12 -04:00
Micaël Bergeron da4fc5b899 fix a future conflict 2018-03-29 12:26:55 -04:00
Micaël Bergeron ef10679418 fix rake migration task from ignoring Upload(store: nil) 2018-03-29 11:05:05 -04:00
Sean McGivern 835fe4d324 Merge branch '40781-os-to-ce' into 'master'
Bring Object Storage to CE

Closes #4171, #4163, #3370, #2841, and #29203

See merge request gitlab-org/gitlab-ce!17358
2018-03-27 08:20:03 +00:00
Micaël Bergeron b08a1cd2c4 fix the failing specs 2018-03-22 14:37:47 -04:00
Micaël Bergeron 764b5fdd16 Backport EE changes 2018-03-22 13:35:42 -04:00
Micaël Bergeron 9c6663ea07 Merge remote-tracking branch 'origin/master' into 40781-os-to-ce 2018-03-22 09:06:10 -04:00
Jacob Vosmaer 008a6a6ce6 Route path lookups through legacy_disk_path 2018-03-21 18:18:32 +01:00
Micaël Bergeron 6466739e2e Merge remote-tracking branch 'origin/master' into 40781-os-to-ce 2018-03-08 10:55:47 -05:00
Sean McGivern edbcde8877 [cherry-pick] '4862-verify-file-checksums'
See merge request gitlab-org/gitlab-ee!4753
2018-03-08 10:20:40 -05:00
Nick Thomas 4419d7ea1f
Implement foreground verification of CI artifacts 2018-03-07 17:28:15 +00:00
Shinya Maeda 0ac1322045 Fix tests by latest proposal changes 2018-03-06 21:44:15 +09:00
Shinya Maeda 824af79d64 Fix rake task to use corrrect SQL 2018-03-06 21:43:19 +09:00
Nick Thomas 6f945f20b4
Foreground verification of uploads and LFS objects 2018-03-02 14:16:17 +00:00
Micaël Bergeron a8df653fae another round of fixes 2018-03-01 10:36:24 -05:00
Kamil Trzciński a2f375e8f7 Merge remote-tracking branch 'origin/master' into object-storage-ee-to-ce-backport 2018-02-28 21:28:43 +01:00
Kamil Trzciński 729391fbfc Merge commit '8af23def1d6' into object-storage-ee-to-ce-backport 2018-02-28 21:11:53 +01:00
Sean McGivern a7dae52e9d Merge branch '4163-move-uploads-to-object-storage' into 'master'
Move uploads to object storage

Closes #4163

See merge request gitlab-org/gitlab-ee!3867
2018-02-28 20:58:15 +01:00
Kamil Trzciński 45d2c31643 Merge commit '4b92efd90cedaa0aff218d11fdce279701128bea' into object-storage-ee-to-ce-backport 2018-02-28 20:46:53 +01:00
Grzegorz Bizon 6ca02a4150 Merge branch 'zj-multiple-artifacts-ee' into 'master'
Multiple artifacts ee

See merge request gitlab-org/gitlab-ee!3276
2018-02-28 20:44:09 +01:00
Kamil Trzciński 965dc28691 Merge commit '7fabc892f251740dbd9a4755baede662e6854870' into object-storage-ee-to-ce-backport 2018-02-28 20:36:55 +01:00
Kamil Trzciński e3fafa7632 Merge commit 'f2f58a60b76acd479e37bdbc9246ec9f9b2bea82' into object-storage-ee-to-ce-backport 2018-02-28 20:36:07 +01:00
Kamil Trzciński e0401df121 Merge commit '11c67e7c2f992299ff5918ce67995b73d1e0be6d' into object-storage-ee-to-ce-backport 2018-02-28 20:35:22 +01:00
Douwe Maan bc76062774 Merge branch 'jej/lfs-object-storage' into 'master'
Can migrate LFS objects to S3 style object storage

Closes #2841

See merge request !2760
2018-02-28 20:29:37 +01:00
Horatiu Eugen Vlad 1ad5df49b1 Moved o_auth/saml/ldap modules under gitlab/auth 2018-02-28 16:53:02 +01:00
Ken c5e385bd5a 36847 - update toml-rb to 1.0.0
36847 - add changelog
2018-02-21 22:09:15 +10:30
Takuya Noguchi 2b6307f6ad Enable RuboCop Style/RegexpLiteral 2018-02-01 02:06:07 +09:00
Robert Speicher 1f309b69df Merge branch '41972-make-sure-no-warnings' into 'master'
Fail static-analysis if there's output to stderr

Closes #41972

See merge request gitlab-org/gitlab-ce!16648
2018-01-29 21:09:09 +00:00
Lin Jen-Shin ac3d56e0ff Pass BUNDLE_PATH to install to the same path for test 2018-01-26 21:24:57 +08:00
Lin Jen-Shin 8e87ecbf30 Eliminate the warnings from task helpers 2018-01-26 19:42:48 +08:00
Douglas Barbosa Alexandre 9afb77b6d4 Refactoring spec for the gitlab:uploads:check rake task 2018-01-08 19:43:53 -02:00
Douglas Barbosa Alexandre a75bbbd92a Make Rubocop happy 2018-01-08 19:40:23 -02:00
Douglas Barbosa Alexandre 4b1546159c Add spec for gitlab:uploads rake tasks 2018-01-08 19:25:32 -02:00
James Lopez de36a8e279 refactor spec, add docs 2018-01-04 11:22:43 +01:00
James Lopez 5b9e777376 add lock specs 2018-01-04 11:22:43 +01:00
James Lopez f8e1b44dc5 add locks chek 2018-01-04 11:22:43 +01:00
James Lopez 7721e8dfca fix spec 2018-01-04 11:22:43 +01:00
James Lopez 260935868a add new git fsck rake task and spec 2018-01-04 11:22:43 +01:00
Zeger-Jan van de Weg 03ac8d5d0b Remove Rugged::Repository#empty? 2017-12-07 15:33:30 +00:00
digitalMoksha 02dd3b4ed1 static analysis fix 2017-11-24 21:10:00 +01:00
digitalMoksha 17069a9547 ignore hashed repositories when doing rake gitlab:cleanup:dirs 2017-11-24 19:29:25 +01:00
Jacob Vosmaer (GitLab) 571f1dda17 Add FetchSourceBranch Gitaly call 2017-11-22 10:19:42 +00:00
Brett Walker 5e861a0525 ignore hashed repos (for now) when using `rake gitlab:cleanup:repos` 2017-11-21 20:26:53 +00:00
Alejandro Rodríguez c33ca9d807 Use `make install` for Gitaly setups in non-test environments 2017-11-21 17:45:36 +00:00
Jacob Vosmaer (GitLab) 6f1e9f7ef7 Fix slow gitaly dev test bundle 2017-11-21 12:28:02 +00:00
Douwe Maan cd3b4ba4ab Remove gitlab:users:clear_all_authentication_tokens rake task 2017-11-02 11:39:02 +01:00
Rémy Coutable 4774b6cfd2 Stub ENV in the LDAP task spec file
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-10-18 18:42:09 +02:00
Rémy Coutable 91553ae833 Stub ENV in the Gitaly task spec file
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-10-18 18:42:09 +02:00
Rémy Coutable 8b8bab118e Stub ENV in the backup task spec file
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-10-18 18:42:09 +02:00
Zeger-Jan van de Weg 4656283c5c
Gitaly namespace service enabled for GitLab 2017-10-05 14:11:32 +02:00
Gabriel Mazetto f4de14d71f
Add support to migrate existing projects to Hashed Storage async 2017-09-28 16:32:14 +01:00
Gabriel Mazetto c505a52347 Fixed few gitlab:check tasks that were failing with exception 2017-09-25 00:34:12 +02:00
Robert Speicher 4edfad9678 Enable Layout/TrailingWhitespace cop and auto-correct offenses 2017-08-15 13:44:37 -04:00
Rémy Coutable 004d335f26 Merge branch 'gitaly-test-bundle-path' into 'master'
Use existing BUNDLE_PATH for gitaly in local tests

See merge request !13493
2017-08-14 16:25:23 +00:00
Jacob Vosmaer fab0c1eb80 Use existing BUNDLE_PATH for gitaly in local tests 2017-08-11 15:49:50 +02:00
Robert Speicher c8b802471b Enable the RSpec/HookArgument cop and auto-correct offenses 2017-08-10 19:29:42 -04:00
Robert Speicher 420d835e5f Merge branch '36003-do-not-include-to-kernel' into 'master'
Don't include anything to Kernel only because of tasks

Closes #36003

See merge request !13380
2017-08-09 16:35:33 +00:00
Lin Jen-Shin 77f10d55de Properly set the expectation on the main object 2017-08-09 01:43:33 +08:00
Lin Jen-Shin 5d963fccab We shouldn't include utility methods everywhere 2017-08-08 23:14:29 +08:00
Jacob Vosmaer 5684528081 Unset BUNDLE_GEMFILE when installing Gitaly 2017-08-08 15:34:29 +02:00
Rémy Coutable feb8974cc8 Merge branch '35941-fix-testing-issue-following-gitaly-install-fix' into 'master'
Unset the `RUBYOPT` env variable before installing `gitaly-ruby`

Closes #35941 and #35967

See merge request !13313
2017-08-08 09:39:49 +00:00
Rémy Coutable 21937a157d
Print the setup steps and duration and fix an issue resulting in re-setuping GitLab Shell on each test run
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-04 17:27:50 +02:00
Rémy Coutable 201ce2cb55
Unset the RUBYOPT env variable before installing gitaly-ruby
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-04 16:10:52 +02:00
Rémy Coutable 805a28e6a1 Revert "Merge branch 'fix-gitaly-install' into 'master'"
This reverts commit 23c502b434, reversing
changes made to 1018ab0516.
2017-08-04 15:59:30 +02:00
Kim "BKC" Carlbäcker 4c68f22b0b fix tests 2017-08-04 13:54:41 +02:00
Rémy Coutable 532ad2e56e Don't call load_tasks as this would load the tasks twice
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-02 19:02:01 +02:00
Sean McGivern 90cb2aab38 Merge branch '28283-legacy-storage-format' into 'master'
[CE] Added Legacy Storage format

See merge request !13149
2017-08-01 12:24:24 +00:00
Jacob Vosmaer e99564568b CI fixes for gitaly-ruby 2017-08-01 10:48:46 +02:00
Gabriel Mazetto c6dee99803 Rename path_with_namespace -> disk_path when dealing with the filesystem 2017-08-01 07:26:58 +02:00