This adds counters for build artifacts and LFS objects, and moves
the preexisting repository_size and commit_count from the projects
table into a new project_statistics table.
The counters are displayed in the administration area for projects
and groups, and also available through the API for admins (on */all)
and normal users (on */owned)
The statistics are updated through ProjectCacheWorker, which can now
do more granular updates with the new :statistics argument.
It adds a brand new importer for Gitea!
This is a continuation of !6945 started by @bkc.
Gitea aims to be 100% GitHub-compatible but there's a few differences:
- Gitea is not an OAuth provider (yet): https://github.com/go-gitea/gitea/issues/27
- This means we cannot map Gitea users given an assignee ID => assignees are not set on imported issues and merge requests
- No releases API for now: https://github.com/go-gitea/gitea/issues/330
- API version is `v1` (GitHub is `v3`)
- The IID field for milestones is `id` compared to `number` in GitHub.
- Issues, PRs, milestones, labels don't have a `url` field (the importer now fallback to `''` in that case)
**Known issues:**
- Comments are not imported because comments JSON always have a blank `html_url`/`issue_url`/`pull_request_url`, so the IID cannot be extracted and the issuable cannot be found... :( This is tracked in https://github.com/go-gitea/gitea/issues/401, and solved by https://github.com/gogits/gogs/pull/3624 but this needs to be submitted / merged in Gitea.
This is noted in the documentation.
## Are there points in the code the reviewer needs to double check?
1. I've made `Import::GiteaController` inherit from `Import::GithubController` since both controllers should be identical in the long-term and their current differences are small.
1. I've added a base `IssuableFormatter` class from which `IssueFormatter` & `PullRequestFormatter` inherit
1. I've added shared examples for GitHub/Gitea importer classes
1. I've made `Gitlab::ImportSources` more robust and tested! 🎄
1. I've added routing specs for import routes! 🎄Closes#22348
See merge request !8116
Slack slash commands
## What does this MR do?
Implement Slack Slash Commands by utilizing generalized Mattermost presenter to fulfill Slack requirements.
## Why was this MR needed?
We want to expose Slack Slash Commands as a first-class service.
## What are the relevant issue numbers?
Supersedes !8007Closes#22182
See merge request !8126
This commit introduces the concept of deployment variables - variables
that are collected from deployment services and passed to CI runner
during a deployment build.
Deployment services specify the variables by overriding
"predefined_variables" method.
This commit also configures variables for KubernetesService
* add parent_id field to namespaces table to store relation with nested groups
* create routes table to keep information about full path of every group and project
* project/group lookup by full path from routes table
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
Replace issue access checks with use of IssuableFinder
Split from !2024 to partially solve https://gitlab.com/gitlab-org/gitlab-ce/issues/23867
## Which fixes are in this MR?
⚠️ - Potentially untested
💣 - No test coverage
🚥 - Test coverage of some sort exists (a test failed when error raised)
🚦 - Test coverage of return value (a test failed when nil used)
✅ - Permissions check tested
### Issue lookup with access check
Using `visible_to_user` likely makes these security issues too. See [Code smells](#code-smells).
- [x] 🚦 app/finders/notes_finder.rb:15 [`visible_to_user`]
- [x] 🚥 app/views/layouts/nav/_project.html.haml:73 [`visible_to_user`] [`.count`]
- [x] ✅ app/services/merge_requests/build_service.rb:84 [`issue.try(:confidential?)`]
- [x] ✅ lib/api/issues.rb:112 [`visible_to_user`]
- CHANGELOG: Prevented API returning issues set to 'Only team members' to everyone
- [x] ✅ lib/api/helpers.rb:126 [`can?(current_user, :read_issue, issue)`] Maybe here too?
- [x] ✅ lib/gitlab/search_results.rb:53 [`visible_to_user`]
### Previous discussions
- [ ] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#b2ff264eddf9819d7693c14ae213d941494fe2b3_128_126
- [ ] https://dev.gitlab.org/gitlab/gitlabhq/merge_requests/2024/diffs#7b6375270d22f880bdcb085e47b519b426a5c6c7_87_87
See merge request !2031
This refactors repository caching so it's possible to selectively
refresh certain caches, instead of just expiring and refreshing
everything.
To allow this the various methods that were cached (e.g. "tag_count" and
"readme") use a similar pattern that makes expiring and refreshing
their data much easier.
In this new setup caches are refreshed as follows:
1. After a commit (but before running ProjectCacheWorker) we expire some
basic caches such as the commit count and repository size.
2. ProjectCacheWorker will recalculate the commit count, repository
size, then refresh a specific set of caches based on the list of
files changed in a push payload.
This requires a bunch of changes to the various methods that may be
cached. For one, data should not be cached if a branch used or the
entire repository does not exist. To prevent all these methods from
handling this manually this is taken care of in
Repository#cache_method_output. Some methods still manually check for
the existence of a repository but this result is also cached.
With selective flushing implemented ProjectCacheWorker no longer uses an
exclusive lease for all of its work. Instead this worker only uses a
lease to limit the number of times the repository size is updated as
this is a fairly expensive operation.
This also updates _some_ specs to use these new methods, just to serve
as an example for others going forward, but by no means is this
exhaustive.
Original implementations at !5992 and !6012.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/20944
This would fix long standing failures running tests on
my development machine, which set `Gitlab.config.gitlab.host`
to another host because it's not my local computer. Now I
finally cannot withstand it and decided to fix them once and
for all.
Stop environment when branch is deleted
## What does this MR do?
This MR adds a environment teardown service, that is called when user deletes a branch. This most often happens when merge requests is merged.
## Does this MR meet the acceptance criteria?
- [x] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
## What are the relevant issue numbers?
Closes#23218
See merge request !7355
Defer saving project services to the database if there are no user changes
## What does this MR do?
It defers saving project services to the database as long as it is possible. It creates a project service when creating a project only if this project service has an active template. After that project services are saved on the first edit.
## Are there points in the code the reviewer needs to double check?
- tests that used `build_missing_services` before the change
- number of queries executed
## Why was this MR needed?
Motivation in #22281
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Fixes#22281
See merge request !6958
Douwe Maan
Douwe Maan
Semyon Pupkov
Kamil Trzcinski
Dmitriy Zaporozhets
Oswaldo Ferreira
Berna Castro
Robert Speicher
Minqi Pan
James Lopez
Markus Koller
Douglas Barbosa Alexandre
Rémy Coutable
Grzegorz Bizon
Adam Niedzielski
Felipe Artur
Oswaldo Ferreira
Ahmad Sherif
Sean McGivern
Yorick Peterse
Lin Jen-Shin
Z.J. van de Weg