Because we don't have any destroy callbacks (or other logic
triggered on event destroy), there is no reason for deleting events
inefficiently one by one, instead we can use :delete_all.
These are the structural changes for supporting the EE feature of moving
"code_owner_approval_required" state from existing on a project to being
on the protected branches individually, allowing for CODEOWNER
validation on push events.
Prior to 12.1, rebase status was looked up directly from Gitaly. In
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14417 , a DB
column was added to track the status instead. However, we couldn't stop
looking at the gitaly status immediately, since some rebases may been
running across the upgrade.
Now that we're in 12.3, it is safe to remove the direct-to-gitaly
lookup. This also happens to fix a 500 error that is seen when viewing
an MR for a fork where the source project has been removed.
We still look at the Gitaly status in the service, just in case Gitaly
and Sidekiq get out of sync - I assume this is possible, and it's a
relatively cheap check.
Since we atomically check and set `merge_requests.rebase_jid`, we
should never enqueue two `RebaseWorker` jobs in parallel.
Splits auto-refreshing of MR widget into 2 requests:
- the one which uses etag-caching and invalidates the fields on change
- the one without caching
The idea is to gradually move all the fields to etag-cached endpoint
Concurrent calls to UserMergeToRef RPC updating a single ref
can lead to an opaque fail that is being rescued at Gitaly.
So this commit adds an exclusive lease to the mergeability
check process with the key as the current MR ID.
Today, Pipelines for merge train run on `refs/merge`,
however, this causes a race condition that it can be
overwritten by CheckMergeabilityService.
This patch fixes the problem by generating `refs/train`
for those pipelines.
This MR introduces tracking of the `rebase_jid` for merge requests. As
with `merge_ongoing?`, `rebase_in_progress?` will now return true if a
rebase is proceeding in sidekiq.
After one release, we should remove the Gitaly-based lookup of rebases.
It is much better to track this kind of thing via the database.
When a user sets only_allow_merge_if_pipeline_succeeds, also named
as "Pipeline must succeed" project setting, we require the pipeline
to be present.
This solves race condition issues especially with external CI
providers when a build is triggered externally but no pipelines are
created in GitLab yet.
Document that a head pipeline is expected when using "Pipeline
must succeed" setting. Also explain limitations with the use of
only/except where there may not be any jobs created and the merge
request will not be allowed to be merged.
This couples the code that transitions the `MergeRequest#merge_status`
and refs/merge-requests/:iid/merge ref update.
In general, instead of directly telling `MergeToRefService` to update
the merge ref, we should rely on `MergeabilityCheckService` to keep
both the merge status and merge ref synced. Now, if the merge_status is
`can_be_merged` it means the merge-ref is also updated to the latest.
We've also updated the logic to be more systematic and less user-based.
We have one auto merge strategy today - Merge When Pipeline
Succeeds.
In order to add more strategies for Merge Train feature,
we abstract the architecture to be more extensible.
Removed arguments
Fix spec
This couples the code that transitions the `MergeRequest#merge_status`
and refs/merge-requests/:iid/merge ref update.
In general, instead of directly telling `MergeToRefService` to update
the merge ref, we should rely on `MergeabilityCheckService` to keep
both the merge status and merge ref synced. Now, if the merge_status is
`can_be_merged` it means the merge-ref is also updated to the latest.
We've also updated the logic to be more systematic and less user-based.
Starting with Rails 5, belongs_to now adds a presence validation to the
association, and so as of shoulda-matchers 4.0.0 the belong_to matcher
follows suit and tests that this validation is there by setting the
association to nil and asserting that there are validation errors. This
exposed an error with the `validate_branches` method: we need to check
the source and target project exist.
Prevents refspec as branch name, which would bypass branch protection
when used in conjunction with rebase.
HEAD seems to be a special case with lots of occurrence,
so it is considered valid for now.
Another special case is `refs/head/*`, which can be imported.
Due to the nature of pipelines for merge requests, deployments.ref can
be a merge request ref instead of a branch name.
We support the environment auto-stop hook for this case
Adds the groundwork needed in order to persist multi-line suggestions,
while providing the parsing strategy which will be reused for the
**Preview** as well.
This commit updates the pipeline block that appears on the merge request
page to include information that will be exposed by the API as part of
the post-merge pipeline feature.
Make it sane
Include merge ref head
Fix union
Improve a bit
Add spec
remove
add spec
Add changelog
fix coding offence
Apply suggestion to spec/models/merge_request_spec.rb
ok
ok
Fix
Fix spec
Fix spec
fix
Simplify the things
Memoize
OK
a
By preloading associations and batching issuable metadata lookups,
we can significantly cut the number of SQL queries needed to load
the Todos API endpoint.
On GitLab.com, my own tests showed my user's SQL queries went
from 365 to under 60 SQL queries.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/40378
Adds the ground work for writing into
the merge ref refs/merge-requests/:iid/merge the
merge result between source and target branches of
a MR, without further side-effects such as
mailing, MR updates and target branch changes.
Add project when creating milestone in specs
We validate milestone is from the same
project/parent group as issuable ->
we need to set project in specs correctly
Improve methods names and specs organization
Patch 1 that migrates issues/merge requests states from integer to string.
On this commit we are only adding the state_id column and syncing it with a backgroud migration.
On Patch 2 the code to use the new integer column will be deployed and the old column will be
removed.
In order to let users' sorting preferences transfer between devices, we
save the preference for issues and MRs (one preference for issues, one
for MRs) in the backend inside the UserPreference object
1. Avoid suggestions being applied on the same file
from reverting previous changes
2. Gracefully use and handle file changes error
when updating the file (though, it does not totally
solves the sync problem for multiple suggestion
applications at once)
This commit handles note diffs caching, which considerably improves
the performance on merge requests with lots of comments.
Important to note that the caching approach taken here is different
from `Gitlab::Diff::HighlightCache`. We do not reset the whole cache
when a new push is sent or anything else. That's because discussions
diffs are persisted and do not change.
It adds a base class for CompareTestReportsService
containing common code with CompareLicenseManagementReportsService
which is present in GitLab Enterprise Edition.
This reverts commit 793be43b35, reversing
changes made to 8d0b4872ba.
For projects not using any CI, enabling merge only when pipeline succeeds
caused merge requests to be in unmergeable state, which caused significant
confusion.
See https://gitlab.com/gitlab-org/gitlab-ce/issues/55144 for more details.
Previously for a given merge request, we would:
1. Create the array of commit SHAs involved in the push (A)
2. Request all merge request commits and map the SHA (B)
3. Reload the diff if there were any common commits between A and B
We can avoid additional database querying and overhead by
checking with the database whether the merge request contains any
of the commit SHAs.
Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/53213
This commit adds the module `FromUnion`, which provides the class method
`from_union`. This simplifies the process of selecting data from the
result of a UNION, and reduces the likelihood of making mistakes. As a
result, instead of this:
union = Gitlab::SQL::Union.new([foo, bar])
Foo.from("(#{union.to_sql}) #{Foo.table_name}")
We can now write this instead:
Foo.from_union([foo, bar])
This commit also includes some changes to make this new setup work
properly. For example, a bug in Rails 4
(https://github.com/rails/rails/issues/24193) would break the use of
`from("sub-query-here").includes(:relation)` in certain cases. There was
also a CI query which appeared to repeat a lot of conditions from an
outer query on an inner query, which isn't necessary.
Finally, we include a RuboCop cop to ensure developers use this new
module, instead of using Gitlab::SQL::Union directly.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/51307
There is still the edge case when 'no commits' changes to 'conflict'
would not trigger notification, which we ignore for now.
Calling can_be_merged? can cause exception (e.g. non-UTF8)
Ignore those by rescueing.
Remove unmergeable_reason as now only conflict is notified
Update spec
Old behavior of creating TODO when
“Merge When Pipeline Succeeds” service fails, is generalized to:
Create a TODO whenever MR became unmergeable
(and similar to notification, MR author and merge_user are both applicable)
First, transitions between can_be_merged & cannot_be_merged are removed,
as they are currently blocked in `check_if_can_be_merged`.
`can_be_merge` always returns to `unchecked` first,
before it can transition to `cannot_be_merged` (and vice versa).
We want to avoid repeated notification triggered by repeated transition
between `cannot_be_merged` & `unchecked`.
So we added `cannot_be_merged_recheck` state, similar to `unchecked`,
but as a mean to remember it’s from cannot_be_merged.
See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18042/#note_65945407
Since `unchecked` and `cannot_be_merged_recheck` both mean
“we are in the middle of checking if it is mergeable”,
quite often we need to see if merge_status is in either one of them,
so `check_state?` is added to achieve this.
Previously, this would issue a query for each unique `diff_refs_or_sha`
passed. This was because we didn't want to load other MR diffs into memory, as
they had some very large columns.
Now they are actually very small, and it's more efficient to just load them all
at once and do the finding in Ruby.
Previously, we kept them all in the cache. We don't need the highlight results
for older diffs - if someone does view that (which is rare), we can do the
highlighting on the fly.
On MySQL, at least, `Note#created_at` doesn't seem to store fractional seconds,
while `MergeRequest::Metrics#merged_at` does. This breaks the optimization
assumption that we only need to search for notes created *after* the MR has
been merged.
Unsynchronized system clocks also make this a dangerous assumption to make.
Adding a minute of leeway still optimizes away most notes, but allows both
cases to be handled more gracefully. If the system clocks are more than a
minute out, we'll still be broken, of course.
If we search for notes before the MR was merged, we have to load every commit
that was ever part of the MR, or mentioned in a push. In extreme cases, this can
be tens of thousands of commits to load, but we know they can't revert the merge
commit, because they are from before the MR was merged.
In the (rare) case that we don't have a `merged_at` value for the MR, we can
still search all notes.
This removes all usage of soft removals except for the "pending delete"
system implemented for projects. This in turn simplifies all the query
plans of the models that used soft removals. Since we don't really use
soft removals for anything useful there's no point in keeping it around.
This _does_ mean that hard removals of issues (which only admins can do
if I'm not mistaken) can influence the "iid" values, but that code is
broken to begin with. More on this (and how to fix it) can be found in
https://gitlab.com/gitlab-org/gitlab-ce/issues/31114.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/37447
When a project uses fast-forward merging strategy user has
to rebase MRs to target branch before it can be merged.
Now user can do rebase in UI by clicking 'Rebase' button
instead of doing rebase locally.
This feature was already present in EE, this is only backport
of the feature to CE. Couple of changes:
* removed rebase license check
* renamed migration (changed timestamp)
Closes#40301
The hook ordering influenced the diffs being generated as these used
values from before the update due to the memoization still being in
place. This commit reorders them and tests against this behaviour.
The Gitaly CommitService is being hammered by n + 1 calls, mostly when
finding commits. This leads to this gRPC being turned of on production:
https://gitlab.com/gitlab-org/gitaly/issues/514#note_48991378
Hunting down where it came from, most of them were due to
MergeRequest#show. To prove this, I set a script to request the
MergeRequest#show page 50 times. The GDK was being scraped by
Prometheus, where we have metrics on controller#action and their Gitaly
calls performed. On both occations I've restarted the full GDK so all
caches had to be rebuild.
Current master, 806a68a81f, needed 435 requests
After this commit, 154 requests
This throttles the number of UPDATE queries that can be triggered by
calling "touch" on a Note, Issue, or MergeRequest. For Note objects we
also take care of updating the associated "noteable" relation in a
smarter way than Rails does by default.
If a merge request was created with a branch name that also matched a tag name,
we'd generate a comparison to or from the tag respectively, rather than the
branch. Merging would still use the branch, of course.
To avoid this, ensure that when we get the branch heads, we prepend the
reference prefix for branches, which will ensure that we generate the correct
comparison.
The st_commits and st_diffs columns on merge_request_diffs historically held the
YAML-serialised data for a merge request diff, in a variety of formats.
Since 9.5, these have been migrated in the background to two new tables:
merge_request_diff_commits and merge_request_diff_files. That has the advantage
that we can actually query the data (for instance, to find out how many commits
we've stored), and that it can't be in a variety of formats, but must match the
new schema.
This is the final step of that journey, where we drop those columns and remove
all references to them. This is a breaking change to the importer, because we
can no longer import diffs created in the old format, and we cannot guarantee
the export will be in the new format unless it was generated after this commit.
Compared to the merge_request_diff association:
1. It's simpler to query. The query uses a foreign key to the
merge_request_diffs table, so no ordering is necessary.
2. It's faster for preloading. The merge_request_diff association has to load
every diff for the MRs in the set, then discard all but the most recent for
each. This association means that Rails can just query for N diffs from N
MRs.
3. It's more complicated to update. This is a bidirectional foreign key, so we
need to update two tables when adding a diff record. This also means we need
to handle this as a special case when importing a GitLab project.
There is some juggling with this association in the merge request model:
* `MergeRequest#latest_merge_request_diff` is _always_ the latest diff.
* `MergeRequest#merge_request_diff` reuses
`MergeRequest#latest_merge_request_diff` unless:
* Arguments are passed. These are typically to force-reload the association.
* It doesn't exist. That means we might be trying to implicitly create a
diff. This only seems to happen in specs.
* The association is already loaded. This is important for the reasons
explained in the comment, which I'll reiterate here: if we a) load a
non-latest diff, then b) get its `merge_request`, then c) get that MR's
`merge_request_diff`, we should get the diff we loaded in c), even though
that's not the latest diff.
Basically, `MergeRequest#merge_request_diff` is the latest diff in most cases,
but not quite all.
When we consider 'all' pipelines for MRs, we now mean:
1. The last 10,000 commits (unordered).
2. From the last 100 MR versions (newest first).
This seems to fix the MRs that time out on GitLab.com.
Use Commit#notes and Note.for_commit_id when possible to make sure we use all indexes available to us
Closes#34509
See merge request gitlab-org/gitlab-ce!15253
also, I refactored the MergeRequest#fetch_ref method to express
the side-effect that this method has.
MergeRequest#fetch_ref -> MergeRequest#fetch_ref!
Repository#fetch_source_branch -> Repository#fetch_source_branch!
Resolve "ActiveRecord::StatementInvalid: PG::QueryCanceled: ERROR: canceling statement due to statement timeout"
Closes#39054
See merge request gitlab-org/gitlab-ce!15063
For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very
slow.
What we can't do to fix this:
1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony.
2. Do the `uniq` in Ruby. That can still be very slow with hundreds of
thousands of commits.
3. Use a subquery. We haven't removed the `st_commits` column yet, but we will
soon.
Until 3 is available to us, we can just do 2, but also add a limit clause. There
is no ordering, so this may return different results, but our goal with these
MRs is just to get them to load, so it's not a huge deal.
In GitLab EE, a GitLab instance can be read-only (e.g. when it's a Geo
secondary node). But in GitLab CE it also might be useful to have the
"read-only" idea around. So port it back to GitLab CE.
Also having the principle of read-only in GitLab CE would hopefully
lead to less errors introduced, doing write operations when there
aren't allowed for read-only calls.
Closesgitlab-org/gitlab-ce#37534.
MergeRequest#create_merge_request_diff and MergeRequest#reload_diff are
the only places where we generate a new MR diff so that's where we
should fetch the ref.
This also ensures that the ref is not fetched when we call
merge_request.merge_request_diffs.create in
Github::Import#fetch_pull_requests.
Signed-off-by: Rémy Coutable <remy@rymai.me>
In this particular case the use of UNION ALL leads to a better query
plan compared to using 1 big query that uses an OR statement to combine
different data sources.
See https://gitlab.com/gitlab-org/gitlab-ce/issues/38508 for more
information.
This ensures the open issues/MR count caches are refreshed properly when
creating new issues or MRs. This MR also includes a change to the cache
keys to ensure all caches are rebuilt on the fly.
This particular problem was not caught in the test suite due to a null
cache being used, resulting in all calls that would use a cache using
the underlying data directly. In production the code would fail because
a newly saved record returns an empty hash in #changes meaning checks
such as `state_changed? || confidential_changed?` would return false for
new rows, thus never updating the counters.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/38061
This ensures the issues/MR cache of the sidebar is only updated when the
state or confidential flags changes, instead of changing this for every
update.
* upstream/master: (225 commits)
Add changelog entry
Backports EE 2756 logic to CE.
Make rubocop happy
Make profile settings dropdown consistent
Add filter by my reaction
Update spec initialization with it being a shared component
Update identicon path and selector
Renamed to `identicon` and make shared component
Merge branch 'master-i18n' into 'master'
Fix broken Frontend JS guide
Replace 'project/star.feature' spinach test with an rspec analog
Adds position fixed to right sidebar
Fixes the margin of the top buttons of the pipeline page
Remove commented out code
Better align fallback image emojis
Decrease Metrics/CyclomaticComplexity threshold to 15
Add changelog
Respect the default visibility level when creating a group
Further break with_repo_branch_commit into parts
Make sure inspect doesn't generate crazy string
...
Every project page displays a navigation menu that in turn displays the
number of open issues and merge requests. This means that for every
project page we run two COUNT(*) queries, each taking up roughly 30
milliseconds on GitLab.com. By caching these numbers and refreshing them
whenever necessary we can reduce loading times of all these pages by up
to roughly 60 milliseconds.
The number of open issues does not include confidential issues. This is
a trade-off to keep the code simple and to ensure refreshing the data
only needs 2 COUNT(*) queries instead of 3. A downside is that if a
project only has 5 confidential issues the counter will be set to 0.
Because we now have 3 similar counting service classes the code
previously used in Projects::ForksCountService has mostly been moved to
Projects::CountService, which in turn is reused by the various service
classes.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/36622
Having two states that essentially mean the same thing is very much like
having a boolean "true" and boolean "mostly-true": it's rather silly.
This commit merges the "reopened" state into the "opened" state while
taking care of system notes still showing messages along the lines of
"Alice reopened this issue".
A big benefit from having only two states (opened and closed) is that
indexing and querying becomes simpler and more performant. For example,
to get all the opened queries we no longer have to query both states:
SELECT *
FROM issues
WHERE project_id = 2
AND state IN ('opened', 'reopened');
Instead we can query a single state directly, which can be much faster:
SELECT *
FROM issues
WHERE project_id = 2
AND state = 'opened';
Further, only having two states makes indexing easier as we will only
ever filter (and thus scan an index) using a single value. Partial
indexes could help but aren't supported on MySQL, complicating the
development process and not being helpful for MySQL.
For merge requests created after 9.4, we have a `merge_request_diff_commits`
table we can get all the SHAs from very quickly. We just need to exclude these
when we load from the legacy format, by ignoring diffs with no serialised
commits.
Once these have been migrated in the background, every MR will see this
improvement.
This is an ID-less table with just three columns: an association to the merge
request diff the commit belongs to, the relative order of the commit within the
merge request diff, and the commit SHA itself.
Previously we stored much more information about the commits, so that we could
display them even when they were deleted from the repo. Since 8.0, we ensure
that those commits are kept around for as long as the target repo itself is, so
we don't need to duplicate that data in the database.
This is allowed for existing instances so we don't end up 76 offenses
right away, but for new code one should _only_ use this if they _have_
to remove non database data. Even then it's usually better to do this in
a service class as this gives you more control over how to remove the
data (e.g. in bulk).
This removes the need for relying on Rails' "dependent" option for data
removal, which is _incredibly_ slow (even when using :delete_all) when
deleting large amounts of data. This also ensures data consistency is
enforced on DB level and not on application level (something Rails is
really bad at).
This commit also includes various migrations to add foreign keys to
tables that eventually point to "projects" to ensure no rows get
orphaned upon removing a project.
Fix https://gitlab.com/gitlab-org/gitlab-ce/issues/27070
Deprecate "chat commands" in favor of "slash commands"
We looked for things like:
- `slash commmand`
- `slash_command`
- `slash-command`
- `SlashCommand`
I don't know why this happens exactly, but given an upstream and fork repository
from a customer, both of which required GC, resolving conflicts would corrupt
the fork so badly that it couldn't be cloned.
This isn't a perfect fix for that case, because the MR may still need to be
merged manually, but it does ensure that the repository is at least usable.
My best guess is that when we generate the index for the conflict
resolution (which we previously did in the target project), we obtain a
reference to an OID that doesn't exist in the source, even though we already
fetch the refs from the target into the source.
Explicitly setting the source project as the place to get the merge index from
seems to prevent repository corruption in this way.
The problem is that we often go via a diff object constructed from the diffs
stored in the DB. Those diffs, by definition, don't overflow, so we don't have
access to the 'correct' `real_size` - that is stored on the MR diff object
iself.
Rename column in the database
Rename fields related to import/export feature
Rename API endpoints
Rename documentation links
Rename the rest of occurrences in the code
Replace the images that contain the words "build succeeds" and docs referencing to them
Make sure pipeline is green and nothing is missing.
updated doc images
renamed only_allow_merge_if_build_succeeds in projects and fixed references
more updates
fix some spec failures
fix rubocop offences
fix v3 api spec
fix MR specs
fixed issues with partials
fix MR spec
fix alignment
add missing v3 to v4 doc
wip - refactor v3 endpoints
fix specs
fix a few typos
fix project specs
copy entities fully to V3
fix entity error
more fixes
fix failing specs
fixed missing entities in V3 API
remove comment
updated code based on feedback
typo
fix spec
Previously, we created an unmergeable todo when a merge request:
1. Had merge when pipeline succeeds set.
2. Became unmergeable.
However, when merge when pipeline succeeds fails due to unmergeability,
the flag isn't actually removed. And a merge request can become
unmergeable multiple times, as every time the target branch is updated
we need to re-check the mergeable status. This means that if the todo
was marked done, and the MR was checked again, a new todo would be
created for the same event.
Instead of checking this, we should create the todo from the service
responsible for merging when the pipeline succeeds. That way the todo is
guaranteed to only be created when we care about it.
This was wrong when there were over 100 files in the diff, because we
did not use the same diff options as subclasses of
`Gitlab::Diff::FileCollection::Base` when getting the raw diffs. (The
reason we don't use those classes directly is because they may perform
highlighting, which isn't needed for just counting the diffs.)
We don't need to create the intermediate FileCollection object with its
associated highlighting in order to count how many files changed. Both
the compare object and the MR diff object have a raw_diffs method that
is perfectly fine for this case.
Fix missing Note access checks in by moving Note#search to updated NoteFinder
Split from !2024 to partially solve https://gitlab.com/gitlab-org/gitlab-ce/issues/23867
## Which fixes are in this MR?
⚠️ - Potentially untested
💣 - No test coverage
🚥 - Test coverage of some sort exists (a test failed when error raised)
🚦 - Test coverage of return value (a test failed when nil used)
✅ - Permissions check tested
### Note lookup without access check
- [x] ✅ app/finders/notes_finder.rb:13 :download_code check
- [x] ✅ app/finders/notes_finder.rb:19 `SnippetsFinder`
- [x] ✅ app/models/note.rb:121 [`Issue#visible_to_user`]
- [x] ✅ lib/gitlab/project_search_results.rb:113
- This is the only use of `app/models/note.rb:121` above, but importantly has no access checks at all. This means it leaks MR comments and snippets when those features are `team-only` in addition to the issue comments which would be fixed by `app/models/note.rb:121`.
- It is only called from SearchController where `can?(current_user, :download_code, @project)` is checked, so commit comments are not leaked.
### Previous discussions
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_13_13 `: download_code` check on commit
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_19_19 `SnippetsFinder` should be used
- `SnippetsFinder` should check if the snippets feature is enabled -> https://gitlab.com/gitlab-org/gitlab-ce/issues/25223
### Acceptance criteria met?
- [x] Tests added for new code
- [x] TODO comments removed
- [x] Squashed and removed skipped tests
- [x] Changelog entry
- [ ] State Gitlab versions affected and issue severity in description
- [ ] Create technical debt issue for NotesFinder.
- Either split into `NotesFinder::ForTarget` and `NotesFinder::Search` or consider object per notable type such as `NotesFinder::OnIssue`. For the first option could create `NotesFinder::Base` which is either inherited from or which can be included in the other two.
- Avoid case statement anti-pattern in this finder with use of `NotesFinder::OnCommit` etc. Consider something on the finder for this? `Model.finder(user, project)`
- Move `inc_author` to the controller, and implement `related_notes` to replace `non_diff_notes`/`mr_and_commit_notes`
See merge request !2035
The target branch of a merge request has to be a branch in the project
for which the merge request is submitted. When a branch changes in a fork,
it does not make sense to reload diffs of merge requests in the upstream
project that use the same branch name as the target branch.
Please note that it does make sense to reload diffs when the source branch
changes.
When a merge request can only be merged when all discussions are
resolved. This feature allows to easily delegate those discussions to a
new issue, while marking them as resolved in the merge request.
The user is presented with a new issue, prepared with mentions of all
unresolved discussions, including the first unresolved note of the
discussion, time and link to the note.
When the issue is created, the discussions in the merge request will get
a system note directing the user to the newly created issue.
Flushing the events cache worked by updating a recent number of rows in
the "events" table. This has the result that on PostgreSQL a lot of dead
tuples are produced on a regular basis. This in turn means that
PostgreSQL will spend considerable amounts of time vacuuming this table.
This in turn can lead to an increase of database load.
For GitLab.com we measured the impact of not using events caching and
found no measurable increase in response timings. Meanwhile not flushing
the events cache lead to the "events" table having no more dead tuples
as now rows are only inserted into this table.
As a result of this we are hereby removing events caching as it does not
appear to help and only increases database load.
For more information see the following comment:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6578#note_18864037
Also allow merge request to be merged with skipped pipeline and the
"only allow merge when pipeline is green" feature enabled
Signed-off-by: Rémy Coutable <remy@rymai.me>
When reading conflicts:
1. Add a `type` field. `text` works as before, and has `sections`;
`text-editor` is a file with ambiguous conflict markers that can only
be resolved in an editor.
2. Add a `content_path` field pointing to a JSON representation of the
file's content for a single file.
3. Hitting `content_path` returns a similar datastructure to the `file`,
but without the `content_path` and `sections` fields, and with a
`content` field containing the full contents of the file (with
conflict markers).
When writing conflicts:
1. Instead of `sections` being at the top level, they are now in a
`files` array. This matches the read format better.
2. The `files` array contains file hashes, each of which must contain:
a. `new_path`
b. `old_path`
c. EITHER `sections` (which works as before) or `content` (with the
full content of the resolved file).
Now a merge request with a blank description will no longer produce a
merge commit message like this:
```
Merge branch 'foo' into 'master'
Bring the wonders of foo into the world
See merge request !7283
```
What an improvement! 🎉
Show all pipelines from all merge_request_diffs
This way we could also show pipelines from commits which
were discarded due to a force push.
Closes#21889
See merge request !6414
- Don't use `TableReferences` - using `.arel_table` is shorter!
- Move some database-related code to `Gitlab::Database`
- Remove the `MergeRequest#issues_closed` and
`Issue#closed_by_merge_requests` associations. They were either
shadowing or were too similar to existing methods. They are not being
used anywhere, so it's better to remove them to reduce confusion.
- Use Rails 3-style validations
- Index for `MergeRequest::Metrics#first_deployed_to_production_at`
- Only include `CycleAnalyticsHelpers::TestGeneration` for specs that
need it.
- Other minor refactorings.
- Move things common to `Issue` and `MergeRequest` into `Issuable`
- Move more database-specific functions into `Gitlab::Database`
- Indentation changes and other minor refactorings.
However, if MergeRequest#all_commits_sha would want to handle
non-persisted merge request, by judging its name, it should not
just give 1 SHA, but all of them.
But we don't really care all_commits_sha for non-persisted merge
request anyway. So I think we should just ignore that case.
Better to not implementing something than implementing it in a
wrong and confusing way.
1. Change multiple updates to a single `update_all`
2. Use cascading deletes
3. Extract an average function for the database median.
4. Move database median to `lib/gitlab/database`
5. Use `delete_all` instead of `destroy_all`
6. Minor refactoring
All the code that pre-calculates metrics for use in the cycle analytics
page.
- Ci::Pipeline -> build start/finish
- Ci::Pipeline#merge_requests
- Issue -> record default metrics after save
- MergeRequest -> record default metrics after save
- Deployment -> Update "first_deployed_to_production_at" for MR metrics
- Git Push -> Update "first commit mention" for issue metrics
- Merge request create/update/refresh -> Update "merge requests closing issues"
1. These changes bring down page load time for 100 issues from more than
a minute to about 1.5 seconds.
2. This entire commit is composed of these types of performance
enhancements:
- Cache relevant data in `IssueMetrics` wherever possible.
- Cache relevant data in `MergeRequestMetrics` wherever possible.
- Preload metrics
3. Given these improvements, we now only need to make 4 SQL calls:
- Load all issues
- Load all merge requests
- Load all metrics for the issues
- Load all metrics for the merge requests
4. A list of all the data points that are now being pre-calculated:
a. The first time an issue is mentioned in a commit
- In `GitPushService`, find all issues mentioned by the given commit
using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
flag for each of them.
- There seems to be a (pre-existing) bug here - files (and
therefore commits) created using the Web CI don't have
cross-references created, and issues are not closed even when
the commit title is "Fixes #xx".
b. The first time a merge request is deployed to production
When a `Deployment` is created, find all merge requests that
were merged in before the deployment, and set the
`first_deployed_to_production_at` flag for each of them.
c. The start / end time for a merge request pipeline
Hook into the `Pipeline` state machine. When the `status` moves to
`running`, find the merge requests whose tip commit matches the
pipeline, and record the `latest_build_started_at` time for each
of them. When the `status` moves to `success`, record the
`latest_build_finished_at` time.
d. The merge requests that close an issue
- This was a big cause of the performance problems we were having
with Cycle Analytics. We need to use `ReferenceExtractor` to make
this calculation, which is slow when we have to run it on a large
number of merge requests.
- When a merge request is created, updated, or refreshed, find the
issues it closes, and create an instance of
`MergeRequestsClosingIssues`, which acts as a join model between
merge requests and issues.
- If a `MergeRequestsClosingIssues` instance links a merge request
and an issue, that issue closes that merge request.
5. The `Queries` module was changed into a class, so we can cache the
results of `issues` and `merge_requests_closing_issues` across
various cycle analytics stages.
6. The code added in this commit is untested. Tests will be added in the
next commit.
Refactor ability.rb into Policies
## What does this MR do?
Factors out `ability.rb` into a new abstraction - the "policy" (stored in `app/policies`). A policy is a class named `#{class_name}Policy` (looked up automatically as needed) that implements `rules` as follows:
``` ruby
class ThingPolicy < BasePolicy
def rules
@user # this is a user to determine abilities for, optionally nil in the anonymous case
@subject # this is the subject of the ability, guaranteed to be an instance of `Thing`
can! :some_ability # grant the :some_ability permission
cannot! :some_ability # ensure that :some_ability is not allowed. this overrides any `can!` that is called before or after
delegate! @subject.other_thing # merge the abilities (can!) and prohibitions (cannot!) from `@subject.other_thing`
can? :some_ability # test whether, so far, :some_ability is allowed
end
def anonymous_rules
# optional. if not implemented `rules` is called where `@user` is nil. otherwise this method is called when `@user` is nil.
end
end
```
See merge request !5796
Add a download buttons for Build Artifacts
## What does this MR do?
This MR adds a download buttons for build artifacts of latest succesful pipeline in:
- dashboard of project,
- branches and tags views,
- and tree viewer
Implement #4255
## What are the relevant issue numbers?
Closes#4255, Closes#14419
## Screenshots
### Project main
### Branches
### Tags
### Source Tree
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] Download buttons
- [x] Models
- [x] Routes
- [x] Projects::ArtifactsController
- [x] API
- Tests
- Rails
- [x] Project#builds_for
- [x] branch name with slashes
- [x] only success builds
- [x] only latest builds
- [x] feature tests for download buttons
- API
- [x] branch name with slashes
- [x] only success builds
- [x] only latest builds
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !5142
Add test for closed MR without fork
Add view test visibility of Reopen and Close buttons
Fix controller tests and validation method
Fix missing space
Remove unused variables from test
closed_without_fork? method refactoring
Add information about missing fork
When closed MR without fork can't edit target branch
Tests for closed MR edit view
Fix indentation and rebase, refactoring
1. Record the `wip_flag_first_removed_at` and
`first_assigned_to_user_other_than_author` metrics for a merge
request. Use a `merge_request_metrics` table, similar to the one for
`issues`. Metrics are recorded `after_save`.
2. Move larger queries to a `CycleAnalytics::Queries` module.
GitLab Bot
Can Eldem
Jan Provaznik
Kerri Miller
Shinya Maeda
Nick Thomas
Igor
Eugenia Grieff
Oswaldo Ferreira
Andreas Brandl
Robert Speicher
George Koltsov
Dmitriy Zaporozhets
Mark Chao
Fabio Pitino
Douwe Maan
Yorick Peterse
GitLab Release Tools Bot
Stan Hu
Jasper Maes
Matija Čupić
Heinrich Lee Yu
Sean McGivern
Douglas Barbosa Alexandre
Francisco Javier López
Kamil Trzciński
Felipe Artur
Nathan Friend
Hiroyuki Sato
John Jarvis
Olivier Gonzalez
Jarka Košanová
Luke Duncalfe
Mario de la Ossa
Tiago Botelho
Gilbert Roulot
Akos Gyimesi
Alessio Caiazza
Chantal Rollison
Sean McGivern
Rémy Coutable
gfyoung
Felipe Artur
Robert Speicher
blackst0ne
lulalala
Bob Van Landuyt
Alejandro Rodríguez
🙈 jacopo beschi 🙉
Zeger-Jan van de Weg
Annabel Dunstone Gray
Eric Eastwood
micael.bergeron
Lin Jen-Shin
George Andrinopoulos
Douwe Maan
Andrew Newdigate
Toon Claes
micael.bergeron
Jacob Vosmaer
haseeb
Brian Neel
Gabriel Mazetto
James Lopez
Adam Niedzielski
Grzegorz Bizon
Toon Claes
Valery Sizov
Fatih Acet
mhasbini
Luke "Jared" Bennett
Kushal Pandya
James Edwards-Jones
Ruben Davila
Jan Christophersen
Dongqing Hu
twonegatives
Jarka Kadlecova
Mike Greiling
Gabriel Gizotti
Oswaldo Ferreira
Grzegorz Bizon
Rodolfo Santos
Guilherme Salazar
David Wagner
Z.J. van de Weg
Paco Guzman
Thomas Balthazar
Timothy Andrew
Katarzyna Kobierska
http://jneen.net/