Instead of checking if a commit already exists in the upstream project
in its ProcessCommitWorker and bailing out if it does, we check the
existence of all commits in bulk in Git::BranchHooksService, so that we
can skip scheduling ProcessCommitWorker jobs for those commits
that already exist upstream entirely.
When a merge request is merged, it creates a commit with the
description of the MR, which may contain references and issue
closing references. As this will be handled in the PostMergeService
anyways, let's ignore merge commit generated from a MR.
By passing commit data to this worker we remove the need for querying
the Git repository for every job. This in turn reduces the time spent
processing each job.
The migration included migrates jobs from the old format to the new
format. For this to work properly it requires downtime as otherwise
workers may start producing errors until they're using a newer version
of the worker code.
This moves the code used for processing commits from GitPushService to
its own Sidekiq worker: ProcessCommitWorker.
Using a Sidekiq worker allows us to process multiple commits in
parallel. This in turn will lead to issues being closed faster and cross
references being created faster. Furthermore by isolating this code into
a separate class it's easier to test and maintain the code.
The new worker also ensures it can efficiently check which issues can be
closed, without having to run numerous SQL queries for every issue.
GitLab Bot
Douwe Maan
Thong Kuah
Sean McGivern
Micaël Bergeron
Robert Speicher
Grzegorz Bizon
Felipe Artur
James Edwards-Jones
mhasbini
Yorick Peterse