When a issue is moved from one project to another, all associated
Markdown text is rewritten in the context of the new project. If the
note contained a link to a commit URL, `CommitRewriter#rewrite` would
fail because `Commit#link_reference_pattern` would match `nil` `commit`
values in the HTML generated from the Markdown. These `nil` values were
passed along to `Project#commits_by` because `Commit#reference_valid?`
was always returning `true`.
To prevent this issue from happening, we tighten up the check for
`Commit#reference_valid?` to look for valid SHA values.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66666
We've seen a significant performance penalty when using
`BatchLoader#__replace_with!`. This defines methods on the batch loader
that proxy to the 'real' object using send. The alternative is
`method_missing`, which is slower. However, we've noticed that
`method_missing` can be faster if:
1. The objects being loaded have a large interface.
2. We don't call too many methods on the loaded object.
In production, we've seen the rendering times of the merge request
widget increase as a result of loading commit data. BatchLoader attempts
to call replace_methods on the lazy object, but this has a significant
performance penalty. Disabling this mode
(https://github.com/exAspArk/batch-loader/pull/45) appears to cut load
times by about 50% for MergeRequestsController#show.
Relates to https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/6941
This brings back some of the changes in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20176/diffs.
We discovered another N+1 that hits Gitaly `TreeEntry` via the
`RelativeLinkFilter`:
https://gitlab.com/gitlab-org/gitlab-ce/issues/58657. When a blob is
loaded with many relative links, `TreeEntry` is called for each link to
scan the URI type.
There are multiple paths that hit Gitaly `TreeEntry`, and
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25706 did not
cover all cases. This commit covers another common use case.
For users using Gitaly on top of NFS, accessing the Git data directly
via Rugged may be faster than going through than Gitaly. This merge
request introduces the feature flag `rugged_commit_tree_entry` to
activate the Rugged method.
Private commit emails were introduced in !22560, but some parts of
GitLab were not updated to take account of them. This commit adds
support in places that were missed.
Even if it doesn’t save lines of code, since people will tend to use
code they’ve seen. And `SafeRequestStore` is safer since you
don’t have to remember to check `RequestStore.active?`.
When displaying the pipelines of a project we now preload the following
data:
1. Authors of the commits that belong to these pipelines
2. The number of warnings per pipeline, which is used by
Ci::Pipeline#has_warnings?
== Commit Authors
Previously this data was queried for every Commit separately, leading to
20 SQL queries being executed in the worst case. With an average of 3 to
5 milliseconds per SQL query this could result in 100 milliseconds being
spent in _just_ getting Commit authors.
To preload this data Commit#author now uses BatchLoader (through
Commit#lazy_author), and a separate module
Gitlab::Ci::Pipeline::Preloader is used to ensure all authors are loaded
before they are used.
== Number of warnings
This changes Ci::Pipeline#has_warnings? so it supports preloading of the
number of warnings per pipeline. This removes the need for executing a
COUNT(*) query for every pipeline just to see if it has any warnings or
not.
Prior to this change, this was done through unicorn. In theory this
could time out. Workhorse has been sending these raw patches and diffs
for a long time and is stable in doing so.
Added bonus is the fact that `Commit#to_patch` can be removed.
`Commit#to_diff` too, which closes
https://gitlab.com/gitlab-org/gitaly/issues/324
Closes https://gitlab.com/gitlab-org/gitaly/issues/1196
Uses `list_commits_by_oid` on the CommitService, to request the needed
commits for pipelines. These commits are needed to display the user that
created the commit and the commit title.
This includes fixes for tests failing that depended on the commit
being `nil`. However, now these are batch loaded, this doesn't happen
anymore and the commits are an instance of BatchLoader.
We also try to use instance variable to cache the result if
RequestStore is not available, so we could keep the same logic,
using the same cache key. Also introduce a way to specify method
specific cache key
* upstream/master: (557 commits)
Fix wrong error message expectation in API::Commits spec
Move admin settings spinach feature to rspec
Encode when migrating ProcessCommitWorker jobs
Prevent overflow with vertical scroll when we have space to show content
Make rubocop happy
API: Ability to cherry-pick a commit
Be smarter when finding a sudoed user in API::Helpers
Backport hooks on group policies for the EE-specific implementation
API: Ability to get group's project in simple representation
Add AddLowerPathIndexToRoutes to setup_postgresql.rake
For single line git commit messages, the close quote should be on the same line as the open quote
added border-radius and padding to labels
Allow all alphanumeric characters in file names (!8002)
Add failing test for #20190
Don't allow blank MR titles in API
Replace static fixture for awards_handler_spec (!7661)
Crontab typo '* */6' -> '0 */6' (4x/day not 1x-per-min-for-1h 4x/day)
Fix test
Tweak style and add back wording
Clean up commit copy to clipboard and make consistent
...
By passing commit data to this worker we remove the need for querying
the Git repository for every job. This in turn reduces the time spent
processing each job.
The migration included migrates jobs from the old format to the new
format. For this to work properly it requires downtime as otherwise
workers may start producing errors until they're using a newer version
of the worker code.
A lot of git operations were being repeated, for example, to build a url
you would ask if the path was a Tree, which would call a recursive routine
in Gitlab::Git::Tree#where, then ask if the path was a Blob, which would
call a recursive routine at Gitlab::Git::Blob#find, making reference to
the same git objects several times. Now we call Rugged::Tree#path, which
allows us to determine the type of the path in one pass.
Some other minor improvement added, like saving commonly used references
instead of calculating them each time.
There are several changes to this module:
1. The use of an explicit stack in Participable#participants
2. Proc behaviour has been changed
3. Batch permissions checking
== Explicit Stack
Participable#participants no longer uses recursion to process "self" and
all child objects, instead it uses an Array and processes objects in
breadth-first order. This allows us to for example create a single
Gitlab::ReferenceExtractor instance and pass this to any Procs. Re-using
a ReferenceExtractor removes the need for running potentially many SQL
queries every time a Proc is called on a new object.
== Proc Behaviour Changed
Previously a Proc in Participable was expected to return an Array of
User instances. This has been changed and instead it's now expected that
a Proc modifies the Gitlab::ReferenceExtractor passed to it. The return
value of the Proc is ignored.
== Permissions Checking
The method Participable#participants uses
Ability.users_that_can_read_project to check if the returned users have
access to the project of "self" _without_ running multiple SQL queries
for every user.
GitLab Bot
Stan Hu
Heinrich Lee Yu
Thong Kuah
Kamil Trzciński
Nick Thomas
Rémy Coutable
George Tsiolis
Michael Kozono
Sean McGivern
yisiliang
Jacob Vosmaer
Yorick Peterse
Zeger-Jan van de Weg
Ahmad Sherif
Hiroyuki Sato
blackst0ne
Saverio Miroddi
Alejandro Rodríguez
Robert Speicher
Gabriel Mazetto
Lin Jen-Shin
Dmitriy Zaporozhets
Alexander Randa
Robert Speicher
Filipa Lacerda
YarNayar
jurre
Douwe Maan
Oswaldo Ferreira
tiagonbotelho
Clement Ho
Gabriel Mazetto
Kamil Trzcinski
Douglas Barbosa Alexandre
Valery Sizov
Valery Sizov
Jeroen van Baarsen
jubianchi