Many customers forget to include the gitlab-secrets.json file. This adds
a warning that both gitlab-secrets.json and gitlab.rb are not included
in the backup.
The `#reload` makes to load all objects into memory,
and the main purpose of `#reload` is to drop the association cache.
The `#reset` seems to solve exactly that case.
This backports EE specific changes for the Rake task `gitlab:env:info`,
wrapping them in a conditional. There is no way to inject code in the
middle of a Rake task in EE, so unfortunately this is the best we can
do.
In this commit, some methods that aren't being used
are removed from `Gitlab::Shell`. They are the ff:
- `#remove_keys_not_found_in_db`
- `#batch_read_key_ids`
- `#list_key_ids`
The corresponding methods in `Gitlab::Keys` have been
removed as well.
This is a small polishing on the storage migration and storage rollback
rake tasks. By aborting a migration while a rollback is already
scheduled we want to prevent unexpected consequences.
This brings back some of the changes in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20339.
For users using Gitaly on top of NFS, accessing the Git data directly
via Rugged is more performant than Gitaly. This merge request introduces
the feature flag `rugged_find_commit` to activate Rugged paths.
There are also Rake tasks `gitlab:features:enable_rugged` and
`gitlab:features:disable_rugged` to enable/disable these feature
flags altogether.
Part of four Rugged changes identified in
https://gitlab.com/gitlab-org/gitlab-ce/issues/57317.
Rollback is done similar to Migration for the Hashed Storage.
It also shares the same ExclusiveLease key to prevent both happening
at the same time.
All Hashed Storage related workers now share the same queue namespace
which allows for assigning dedicated workers easily.
If there are any clients connected to the DB, PostgreSQL won't let you
drop the database. It's possible that Sidekiq, Unicorn, or some other
client will be hanging onto a connection, preventing the DROP DATABASE
from working. To workaround this problem, this method cancels all the
connections so that the db:reset command will work.
Note that there's still a slight possibility a client connects after its
connection is terminated. If this is an issue, we could solve it by
revoking CONNECT access, but for now it seems this works.
Closes https://gitlab.com/gitlab-org/gitlab-development-kit/issues/450
Specs were reviewed and improved to better cover the current behavior.
There was some standardization done as well to facilitate the
implementation of the rollback functionality.
StorageMigratorWorker was extracted to HashedStorage namespace were
RollbackerWorker will live one as well.
To separate the different kinds of repositories we have at GitLab this
table will be renamed to pool_repositories. A project can, for now at
least, be member of none, or one of these. The table will get additional
columns in a later merge request where more logic is implemented for the
model.
Further included is a small refactor of logic around hashing ids for the
disk_path, mainly to ensure a previous implementation is reusable.
The disk_path for the pool_repositories table no longer has a NOT NULL
constraint, but given the hashing of the ID requires the DB to assign
the record an ID, an after_create hook is used to update the value.
A related MR is:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23143, adding
tables for 'normal' repositories and wiki_repositories.
Also, since Gitaly now takes care of checking for storage paths
existence/accessibility, we can remove those check from the
gitlab:gitlab_shell_check task and advance further into 0 direct disk
approach on gitlab-rails
We started syncing all the wiki regardless of the fact it's disabled or
not. We couldn't do that in one stage because of needing of smoth update
and deprecating things. This is the second stage that finally removes
unused columns in the geo_node_status table.
Fast lookup of authorized SSH keys in the database was ported to CE in
v10.4. This change adds the option to enable the setting via the restore
rake task and assumes yes if the force env variable is set.
If doing a schema load, the post_migrations should also be marked as up,
even if SKIP_POST_DEPLOYMENT_MIGRATIONS was set, otherwise future
migration runs will be broken.
Rake tasks cleaning up the Git storage were still using direct disk
access, which won't work if these aren't attached. To mitigate a
migration issue was created.
To port gitlab:cleanup:dirs, and gitlab:cleanup:repos, a new RPC was
required, ListDirectories. This was implemented in Gitaly, through
https://gitlab.com/gitlab-org/gitaly/merge_requests/868.
To be able to use the new RPC the Gitaly server was bumped to v0.120.
This is an RPC that will not use feature gates, as this doesn't scale on
.com so there is no way to test it at scale. Futhermore, we _know_ it
doesn't scale, but this might be a useful task for smaller instances.
Lastly, the tests are slightly updated to also work when the disk isn't
attached. Eventhough this is not planned, it was very little effort and
thus I applied the boy scout rule.
Closes https://gitlab.com/gitlab-org/gitaly/issues/954
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/40529
Ben Bodenmiller
Cindy Pallares
Rémy Coutable
Kamil Trzciński
Robert Speicher
Yorick Peterse
John Jarvis
Jan Provaznik
Wei-Meng Lee
Douglas Barbosa Alexandre
Patrick Bajao
Nick Thomas
Gabriel Mazetto
Ray Paik
Sean McGivern
Stan Hu
Jacob Vosmaer
Jose Torres
Michael Kozono
Zeger-Jan van de Weg
Alejandro Rodríguez
Michael Bisbjerg
Dylan Griffith
Valery Sizov
Balasankar "Balu" C
DJ Mountney
Douwe Maan
James Lopez