We don't need to create the intermediate FileCollection object with its
associated highlighting in order to count how many files changed. Both
the compare object and the MR diff object have a raw_diffs method that
is perfectly fine for this case.
Fix missing Note access checks in by moving Note#search to updated NoteFinder
Split from !2024 to partially solve https://gitlab.com/gitlab-org/gitlab-ce/issues/23867
## Which fixes are in this MR?
⚠️ - Potentially untested
💣 - No test coverage
🚥 - Test coverage of some sort exists (a test failed when error raised)
🚦 - Test coverage of return value (a test failed when nil used)
✅ - Permissions check tested
### Note lookup without access check
- [x] ✅ app/finders/notes_finder.rb:13 :download_code check
- [x] ✅ app/finders/notes_finder.rb:19 `SnippetsFinder`
- [x] ✅ app/models/note.rb:121 [`Issue#visible_to_user`]
- [x] ✅ lib/gitlab/project_search_results.rb:113
- This is the only use of `app/models/note.rb:121` above, but importantly has no access checks at all. This means it leaks MR comments and snippets when those features are `team-only` in addition to the issue comments which would be fixed by `app/models/note.rb:121`.
- It is only called from SearchController where `can?(current_user, :download_code, @project)` is checked, so commit comments are not leaked.
### Previous discussions
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_13_13 `: download_code` check on commit
- [x] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b915c5267a63628b0bafd23d37792ae73ceae272_19_19 `SnippetsFinder` should be used
- `SnippetsFinder` should check if the snippets feature is enabled -> https://gitlab.com/gitlab-org/gitlab-ce/issues/25223
### Acceptance criteria met?
- [x] Tests added for new code
- [x] TODO comments removed
- [x] Squashed and removed skipped tests
- [x] Changelog entry
- [ ] State Gitlab versions affected and issue severity in description
- [ ] Create technical debt issue for NotesFinder.
- Either split into `NotesFinder::ForTarget` and `NotesFinder::Search` or consider object per notable type such as `NotesFinder::OnIssue`. For the first option could create `NotesFinder::Base` which is either inherited from or which can be included in the other two.
- Avoid case statement anti-pattern in this finder with use of `NotesFinder::OnCommit` etc. Consider something on the finder for this? `Model.finder(user, project)`
- Move `inc_author` to the controller, and implement `related_notes` to replace `non_diff_notes`/`mr_and_commit_notes`
See merge request !2035
The target branch of a merge request has to be a branch in the project
for which the merge request is submitted. When a branch changes in a fork,
it does not make sense to reload diffs of merge requests in the upstream
project that use the same branch name as the target branch.
Please note that it does make sense to reload diffs when the source branch
changes.
When a merge request can only be merged when all discussions are
resolved. This feature allows to easily delegate those discussions to a
new issue, while marking them as resolved in the merge request.
The user is presented with a new issue, prepared with mentions of all
unresolved discussions, including the first unresolved note of the
discussion, time and link to the note.
When the issue is created, the discussions in the merge request will get
a system note directing the user to the newly created issue.
Flushing the events cache worked by updating a recent number of rows in
the "events" table. This has the result that on PostgreSQL a lot of dead
tuples are produced on a regular basis. This in turn means that
PostgreSQL will spend considerable amounts of time vacuuming this table.
This in turn can lead to an increase of database load.
For GitLab.com we measured the impact of not using events caching and
found no measurable increase in response timings. Meanwhile not flushing
the events cache lead to the "events" table having no more dead tuples
as now rows are only inserted into this table.
As a result of this we are hereby removing events caching as it does not
appear to help and only increases database load.
For more information see the following comment:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6578#note_18864037
Also allow merge request to be merged with skipped pipeline and the
"only allow merge when pipeline is green" feature enabled
Signed-off-by: Rémy Coutable <remy@rymai.me>
When reading conflicts:
1. Add a `type` field. `text` works as before, and has `sections`;
`text-editor` is a file with ambiguous conflict markers that can only
be resolved in an editor.
2. Add a `content_path` field pointing to a JSON representation of the
file's content for a single file.
3. Hitting `content_path` returns a similar datastructure to the `file`,
but without the `content_path` and `sections` fields, and with a
`content` field containing the full contents of the file (with
conflict markers).
When writing conflicts:
1. Instead of `sections` being at the top level, they are now in a
`files` array. This matches the read format better.
2. The `files` array contains file hashes, each of which must contain:
a. `new_path`
b. `old_path`
c. EITHER `sections` (which works as before) or `content` (with the
full content of the resolved file).
Now a merge request with a blank description will no longer produce a
merge commit message like this:
```
Merge branch 'foo' into 'master'
Bring the wonders of foo into the world
See merge request !7283
```
What an improvement! 🎉
Show all pipelines from all merge_request_diffs
This way we could also show pipelines from commits which
were discarded due to a force push.
Closes#21889
See merge request !6414
- Don't use `TableReferences` - using `.arel_table` is shorter!
- Move some database-related code to `Gitlab::Database`
- Remove the `MergeRequest#issues_closed` and
`Issue#closed_by_merge_requests` associations. They were either
shadowing or were too similar to existing methods. They are not being
used anywhere, so it's better to remove them to reduce confusion.
- Use Rails 3-style validations
- Index for `MergeRequest::Metrics#first_deployed_to_production_at`
- Only include `CycleAnalyticsHelpers::TestGeneration` for specs that
need it.
- Other minor refactorings.
- Move things common to `Issue` and `MergeRequest` into `Issuable`
- Move more database-specific functions into `Gitlab::Database`
- Indentation changes and other minor refactorings.
However, if MergeRequest#all_commits_sha would want to handle
non-persisted merge request, by judging its name, it should not
just give 1 SHA, but all of them.
But we don't really care all_commits_sha for non-persisted merge
request anyway. So I think we should just ignore that case.
Better to not implementing something than implementing it in a
wrong and confusing way.
1. Change multiple updates to a single `update_all`
2. Use cascading deletes
3. Extract an average function for the database median.
4. Move database median to `lib/gitlab/database`
5. Use `delete_all` instead of `destroy_all`
6. Minor refactoring
All the code that pre-calculates metrics for use in the cycle analytics
page.
- Ci::Pipeline -> build start/finish
- Ci::Pipeline#merge_requests
- Issue -> record default metrics after save
- MergeRequest -> record default metrics after save
- Deployment -> Update "first_deployed_to_production_at" for MR metrics
- Git Push -> Update "first commit mention" for issue metrics
- Merge request create/update/refresh -> Update "merge requests closing issues"
1. These changes bring down page load time for 100 issues from more than
a minute to about 1.5 seconds.
2. This entire commit is composed of these types of performance
enhancements:
- Cache relevant data in `IssueMetrics` wherever possible.
- Cache relevant data in `MergeRequestMetrics` wherever possible.
- Preload metrics
3. Given these improvements, we now only need to make 4 SQL calls:
- Load all issues
- Load all merge requests
- Load all metrics for the issues
- Load all metrics for the merge requests
4. A list of all the data points that are now being pre-calculated:
a. The first time an issue is mentioned in a commit
- In `GitPushService`, find all issues mentioned by the given commit
using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
flag for each of them.
- There seems to be a (pre-existing) bug here - files (and
therefore commits) created using the Web CI don't have
cross-references created, and issues are not closed even when
the commit title is "Fixes #xx".
b. The first time a merge request is deployed to production
When a `Deployment` is created, find all merge requests that
were merged in before the deployment, and set the
`first_deployed_to_production_at` flag for each of them.
c. The start / end time for a merge request pipeline
Hook into the `Pipeline` state machine. When the `status` moves to
`running`, find the merge requests whose tip commit matches the
pipeline, and record the `latest_build_started_at` time for each
of them. When the `status` moves to `success`, record the
`latest_build_finished_at` time.
d. The merge requests that close an issue
- This was a big cause of the performance problems we were having
with Cycle Analytics. We need to use `ReferenceExtractor` to make
this calculation, which is slow when we have to run it on a large
number of merge requests.
- When a merge request is created, updated, or refreshed, find the
issues it closes, and create an instance of
`MergeRequestsClosingIssues`, which acts as a join model between
merge requests and issues.
- If a `MergeRequestsClosingIssues` instance links a merge request
and an issue, that issue closes that merge request.
5. The `Queries` module was changed into a class, so we can cache the
results of `issues` and `merge_requests_closing_issues` across
various cycle analytics stages.
6. The code added in this commit is untested. Tests will be added in the
next commit.
Refactor ability.rb into Policies
## What does this MR do?
Factors out `ability.rb` into a new abstraction - the "policy" (stored in `app/policies`). A policy is a class named `#{class_name}Policy` (looked up automatically as needed) that implements `rules` as follows:
``` ruby
class ThingPolicy < BasePolicy
def rules
@user # this is a user to determine abilities for, optionally nil in the anonymous case
@subject # this is the subject of the ability, guaranteed to be an instance of `Thing`
can! :some_ability # grant the :some_ability permission
cannot! :some_ability # ensure that :some_ability is not allowed. this overrides any `can!` that is called before or after
delegate! @subject.other_thing # merge the abilities (can!) and prohibitions (cannot!) from `@subject.other_thing`
can? :some_ability # test whether, so far, :some_ability is allowed
end
def anonymous_rules
# optional. if not implemented `rules` is called where `@user` is nil. otherwise this method is called when `@user` is nil.
end
end
```
See merge request !5796
Add a download buttons for Build Artifacts
## What does this MR do?
This MR adds a download buttons for build artifacts of latest succesful pipeline in:
- dashboard of project,
- branches and tags views,
- and tree viewer
Implement #4255
## What are the relevant issue numbers?
Closes#4255, Closes#14419
## Screenshots
### Project main
### Branches
### Tags
### Source Tree
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] Download buttons
- [x] Models
- [x] Routes
- [x] Projects::ArtifactsController
- [x] API
- Tests
- Rails
- [x] Project#builds_for
- [x] branch name with slashes
- [x] only success builds
- [x] only latest builds
- [x] feature tests for download buttons
- API
- [x] branch name with slashes
- [x] only success builds
- [x] only latest builds
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !5142
Add test for closed MR without fork
Add view test visibility of Reopen and Close buttons
Fix controller tests and validation method
Fix missing space
Remove unused variables from test
closed_without_fork? method refactoring
Add information about missing fork
When closed MR without fork can't edit target branch
Tests for closed MR edit view
Fix indentation and rebase, refactoring
1. Record the `wip_flag_first_removed_at` and
`first_assigned_to_user_other_than_author` metrics for a merge
request. Use a `merge_request_metrics` table, similar to the one for
`issues`. Metrics are recorded `after_save`.
2. Move larger queries to a `CycleAnalytics::Queries` module.
* upstream/master: (107 commits)
Fix external issue tracker "Issues" link leading to 404s
Fix CHANGELOG entries related to 8.11 release.
Fix changelog
Reduce contributions calendar data payload
Add lock_version to merge_requests table
Add hover color to emoji icon
Fix wrong Koding link
Capitalize mentioned issue timeline notes
Fix groups sort dropdown alignment
Use icon helper
Fix inline emoji text alignment
Adds response mime type to transaction metric action when it's not HTML
Moved two 8.11 changelog entries to 8.12
Fix markdown link in doc_styleguide.md
Display project icon from default branch
Reduce number of database queries on builds tab
Update CHANGELOG
Update Issue board documentation
Label list shows all issues (opened or closed) with that label
Update CHANGELOG
...
* upstream/master: (195 commits)
Fix expansion of discussions in diff
Improve performance of MR show page
Fix jumping between discussions on changes tab
Update doorkeeper to 4.2.0
Fix MR note discussion ID
Handle legacy sort order values
Refactor `find_for_git_client` and its related methods.
Remove right margin on Jump button icon
Fix bug causing “Jump to discussion” button not to show
Small refactor and syntax fixes.
Removed unnecessary service for user retrieval and improved API error message.
Added documentation and CHANGELOG item
Added checks for 2FA to the API `/sessions` endpoint and the Resource Owner Password Credentials flow.
Fix behavior around commands with optional arguments
Fix behavior of label_ids and add/remove_label_ids
Remove unneeded aliases
Do not expose projects on deployments
Incorporate feedback
Docs for API endpoints
Expose project for environments
...
* upstream/master: (359 commits)
Add new image to show the 'Reset template' button
Refactor description templates documentation
Remove index from pipeline toggles
Hide `Create new list button` on Issues and MRs pages
Remove params from build; general refactor
Style build container box; add check mark to active build
Display jobs as scrolling list in sidebar
Move stages and jobs to build sidebar
Removed vue assets
Move skipped tests to end of array
Remove unused data attributes
Update changelog
Style build dropdowns
Populate dropdowns with current build on pageload
Selecting stage updates builds dropdown
Add data attributes to builds
Change active state of list items; style dropdown items
Hide dropdown if all tests fit on one line; add counter to dropdown
Add overflow tests to dropdown
Order by build status
...
* upstream/master: (123 commits)
Limit the size of SVGs when viewing them as blobs
Add a spec for ProjectsFinder project_ids_relation option
Fix ProjectsFinder spec
Pass project IDs relation to ProjectsFinder instead of using a block
Speed up todos queries by limiting the projects set we join with
Used phantomjs variable
Ability to specify branches for pivotal tracker integration
Fix a memory leak caused by Banzai::Filter::SanitizationFilter
Update phantomjs link
Remove sleeping and replace escaped text.
Filters test fix
Fixed filtering tests
Removed screenshot command 💩
Updated failing tests
Used mirrored version on GitLab
Updated tests
Fix `U2fSpec` for PhantomJS versions > 2.
Fix file downloading
Install latest stable phantomjs
Use new PhantomJS version
...
Show deployment status on a MR view
## What are the relevant issue numbers?
Resolves#19571, one in the list of #19992
## Screenshots (if relevant)
external_url = nil
external_url != nil
Note, the timings are weird between merging and deploying, that is because I did it in the wrong order.
## Does this MR meet the acceptance criteria?
- [X] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
See merge request !5622
An MR can only be resolved in the UI if:
- It has conflicts.
- It has valid diff_refs (in other words, it supports new diff notes).
- It has no conflicts with one side missing.
- It has no conflicts in binary files.
- It has no conflicts in files too large to display.
- It has no conflicts containing invalid conflict markers.
DRY code + fix rubocop
Add more test cases
Append to changelog
DRY changes list
find_url service for merge_requests
use GET for getting merge request links
remove files
rename to get_url_service
reduce loop
add test case for cross project
refactor tiny thing
update changelog
- Preloading noteable we share the same noteable instance when more than one
discussion refers to the same noteable.
- Any other call to that object that is cached in that object will be for any
discussion.
- In those cases where merge_request_diff has all the sha stored to build a diff_refs get that
diff_refs using directly those sha instead accessing to the git repository to first get the
commits and later the sha.
* Add merge_request_diff validation to merge_request model
* Improve initialize of merge_request_diff object
* Rename some merge_request_diff methods for clarity
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
When importing a pull request from GitHub, the old and new branches may
no longer actually exist by those names, but we need to recreate the
merge request diff with the right source and target shas.
We use these `target_branch_sha` and `source_branch_sha` attributes to
force these to the intended values. But the reader methods were always
looking up to the target/source branch head instead of check if these
values was previously set.
1. Don't use case statements for dispatch anymore. This leads to a lot
of duplication, and makes the logic harder to follow.
2. Remove duplicated logic.
- For example, the `can_push_to_branch?` exists, but we also have a
different way of checking the same condition within `change_access_check`.
- This kind of duplication is removed, and the `can_push_to_branch?`
method is used in both places.
3. Move checks returning true/false to `UserAccess`.
- All public methods in `GitAccess` now return an instance of
`GitAccessStatus`. Previously, some methods would return
true/false as well, which was confusing.
- It makes sense for these kinds of checks to be at the level of a
user, so the `UserAccess` class was repurposed for this. The prior
`UserAccess.allowed?` classmethod is converted into an instance
method.
- All external uses of these checks have been migrated to use the
`UserAccess` class
4. Move the "change_access_check" into a separate class.
- Create the `GitAccess::ChangeAccessCheck` class to run these
checks, which are quite substantial.
- `ChangeAccessCheck` returns an instance of `GitAccessStatus` as
well.
5. Break out the boolean logic in `ChangeAccessCheck` into `if/else`
chains - this seems more readable.
6. I can understand that this might look like overkill for !4892, but I
think this is a good opportunity to clean it up.
- http://martinfowler.com/bliki/OpportunisticRefactoring.html
The issue was that `MergeRequest#mergeable?` returns false when the CI
state is not success and project.only_allow_merge_if_build_succeeds is
true. In this case `Projects::MergeRequestsController#merge` would
return the `:failed` status when enabling `merge_when_build_succeeds`,
thus leading to a weird state and the MR never automatically merged.
The fix is to disable the CI state check in the controller safeguard
that early return the `:failed` status.
Signed-off-by: Rémy Coutable <remy@rymai.me>
When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we
get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method
that will throw out out-of-bounds values.
Closes#18777
Enable the Rubocop DeprecatedClassMethods cop
This reports uses of `File.exists?` and `Dir.exists?`, which were both
deprecated in Ruby and will eventually be removed in favor of `.exist?`.
Also fixes all existing uses of the deprecated methods.
See merge request !4087
In 8278b763d9 the default behaviour of annotation
has changes, which was causing a lot of noise in diffs. We decided in #17382
that it is better to get rid of the whole annotate gem, and instead let people
look at schema.rb for the columns in a table.
Fixes: #17382
This reports uses of `File.exists?` and `Dir.exists?`, which were both
deprecated in Ruby and will eventually be removed in favor of
`.exist?`. Also fixes all existing uses of the deprecated methods.
Ci::Commit becomes a Pipeline object
1. Ci::Commit receives context: ref, :tag.
1. One Ci::Commit describes a one Pipeline
1. Pipeline is created from `.gitlab-ci.yml`
1. Pipeline is a ordered group of builds
1. We test MR against Pipeline
1. Pipelines have a separate view (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3703)
1. Pipeline can be triggered from UI (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3703)
1. Later we change `Trigger -> TriggerRequest -> Build` to `Trigger -> Pipeline` (future)
1. We add a Pipeline Hook that will be triggered on Pipeline status change (future)
1. We extend notifications to use `Pipeline Hook` to send summary on pipeline changes (future)
After merging that I'll prepare a separate MR that will unify naming, database columns, table names:
```
Ci::Commit -> Pipeline
Ci::Build -> Build
CommitStatus -> Job
GenericCommitStatus -> ExternalJob
ci_commits -> pipelines
ci_builds -> jobs
```
This MR implements first 5 points.
This is made to solve this issue https://gitlab.com/gitlab-org/gitlab-ce/issues/14149.
See merge request !3653
Remove fork link closes all merge requests opened on source project
Currently, if you:
- create a fork
- open a merge request on the source project
- remove the fork link
The created MR cannot be closed.
With this MR, all pending MR is closed when the fork link is removed.
See merge request !3189
Oswaldo Ferreira
Sean McGivern
Dongqing Hu
twonegatives
Jarka Kadlecova
Mike Greiling
James Lopez
Valery Sizov
Gabriel Gizotti
Sean McGivern
Rémy Coutable
Douwe Maan
Adam Niedzielski
Bob Van Landuyt
Oswaldo Ferreira
Grzegorz Bizon
Yorick Peterse
Douwe Maan
Rodolfo Santos
Guilherme Salazar
David Wagner
Nick Thomas
Douglas Barbosa Alexandre
Z.J. van de Weg
Paco Guzman
Felipe Artur
Thomas Balthazar
Robert Speicher
Kamil Trzcinski
Timothy Andrew
Lin Jen-Shin
Katarzyna Kobierska
Robert Speicher
http://jneen.net/
Dmitriy Zaporozhets
Connor Shea
Scott Le
Mathias Vestergaard
Patricio Cano
Stan Hu
Rui Anderson
Jacob Vosmaer (GitLab)
Jeroen Jacobs
Jeroen van Baarsen
Gabriel Mazetto