Preloading of project_features mitigates N+1 queries when checking
references in other projects.
When loading projects for resources referenced in comments it
makes sense to include also associated project_features because
in the following step (`can_read_reference?(user, projects[node],
node)`) project features is used for checking permissions for the given
project.
Even if it doesn’t save lines of code, since people will tend to use
code they’ve seen. And `SafeRequestStore` is safer since you
don’t have to remember to check `RequestStore.active?`.
Since !18150 abstract reference filter supports also `group`
as a resource parent (not only `project`). Some reference filters,
which inherit from abstract filter, depend on parent being `project`
when checking a reference.
This patch adds explicit check of parent class to the given filters.
This refactors the Markdown pipeline so it supports the rendering of
multiple documents that may belong to different projects. An example of
where this happens is when displaying the event feed of a group. In this
case we retrieve events for all projects in the group. Previously we
would group events per project and render these chunks separately, but
this would result in many SQL queries being executed. By extending the
Markdown pipeline to support this out of the box we can drastically
reduce the number of SQL queries.
To achieve this we introduce a new object to the pipeline:
Banzai::RenderContext. This object simply wraps two other objects: an
optional Project instance, and an optional User instance. On its own
this wouldn't be very helpful, but a RenderContext can also be used to
associate HTML documents with specific Project instances. This work is
done in Banzai::ObjectRenderer and allows us to reuse as many queries
(and results) as possible.
read_project can be prevented by a very expensive condition, which we want to
avoid, while still not writing manual SQL queries. read_project_for_iids is used
by read_issue_iid and read_merge_request_iid to satisfy both of those
constraints, and allow the declarative policy runner to use its normal caching
strategy.
The /unsubscribe slash command means that we check if the current user is
subscribed to the issuable without having an explicit subscription. That means
that we use the UserParser to find references to them in the notes.
The UserParser (and all parsers inheriting from BaseParser) use RequestStore to
cache ActiveRecord objects, so that we don't need to load the User object each
time, if we're parsing references a bunch of times in the same request.
However, it was always returning _all_ of the previously cached items, not just
the ones matching the IDs passed. This would mean that we did two runs through
with UserParser if you were mentioned in a comment, and then mentioned someone
else in your comment while using /unsubscribe:
1. Because /unsubscribe was used, we see if you were mentioned in any comments.
2. Because you mentioned someone, we find them - but we would also get back your
user, even if you didn't mention yourself. This would have the effect of
creating a mention or directly addressed todo for yourself incorrectly.
The fix is simple: only return values from the cache matching the IDs passed.
Example: for issues that are closed, the links will now show '[closed]'
following the issue number. This is done as post-process after the markdown has
been loaded from the cache as the status of the issue may change between
the cache being populated and the content being displayed.
In order to avoid N+1 queries problem when rendering notes ObjectRenderer
populates the cache of referenced issuables for all notes at once,
before the post processing phase.
As a part of this change, the Banzai BaseParser#grouped_objects_for_nodes
method has been refactored to return a Hash utilising the node itself as the
key, since this was a common pattern of usage for this method.
disable markdown in comments when referencing disabled features
fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/23548
This MR prevents the following references when tool is disabled:
- issues
- snippets
- commits - when repo is disabled
- commit range - when repo is disabled
- milestones
This MR does not prevent references to repository files, since they are just markdown links and don't leak
information.
See merge request !2011
Signed-off-by: Rémy Coutable <remy@rymai.me>
The method Ability.issues_readable_by_user takes a list of users and an
optional user and returns an Array of issues readable by said user. This
method in turn is used by
Banzai::ReferenceParser::IssueParser#nodes_visible_to_user so this
method no longer needs to get all the available abilities just to check
if a user has the "read_issue" ability.
To test this I benchmarked an issue with 222 comments on my development
environment. Using these changes the time spent in nodes_visible_to_user
was reduced from around 120 ms to around 40 ms.
This splits the Markdown rendering and reference extraction phases into
two distinct code bases. The reference extraction phase no longer relies
on the html-pipeline Gem (and any related code) and allows for
extracting of references from multiple HTML nodes in a single pass. This
means that if you want to extract user references from 200 comments you
no longer need to run 200 times N number of queries, instead only a
handful of queries may be needed.
GitLab Bot
Thong Kuah
Jan Provaznik
Michael Kozono
Sean McGivern
Reuben Pereira
Yorick Peterse
Bob Van Landuyt
blackst0ne
Jarka Kadlecova
Robert Speicher
Rémy Coutable
Adam Niedzielski
Grzegorz Bizon
Sean McGivern
Douwe Maan
Oswaldo Ferreira
DJ Mountney
Stan Hu
Adam Buckland
Felipe Artur
Douwe Maan
http://jneen.net/