Move CI job config entries from legacy to new config
## What does this MR do?
This MR extracts jobs configuration logic from legacy CI config processor to the new code.
## What are the relevant issue numbers?
#15060
## Does this MR meet the acceptance criteria?
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
See merge request !5087
1. Remove `Project#developers_can_push_to_protected_branch?` since it
isn't used anymore.
2. Remove `Project#developers_can_merge_to_protected_branch?` since it
isn't used anymore.
1. The crux of this change is in `UserAccess`, which looks through all
the access levels, asking each if the user has access to push/merge
for the current project.
2. Update the `protected_branches` factory to create access levels as
necessary.
3. Fix and augment `user_access` and `git_access` specs.
Implement #3243 New Issue by email
So we extend Gitlab::Email::Receiver for this new behaviour,
however we might want to split it into another class for better
testing it.
Another issue is that, currently it's using this to parse project
identifier:
Gitlab::IncomingEmail.key_from_address
Which is using:
Gitlab.config.incoming_email.address
for the receiver name. This is probably `reply` because it's used
for replying to a specific issue. We might want to introduce another
config for this, or just use `reply` instead of `incoming`.
I'll prefer to introduce a new config for this, or just change
`reply` to `incoming` because it would make sense for replying to
there, too.
The email template used in tests were copied and modified from:
`emails/valid_reply.eml` which I hope is ok.
/cc @DouweM #3243
See merge request !3363
This reduces the overhead of the method instrumentation code primarily
by reducing the number of method calls. There are also some other small
optimisations such as not casting timing values to Floats (there's no
particular need for this), using Symbols for method call metric names,
and reducing the number of Hash lookups for instrumented methods.
The exact impact depends on the code being executed. For example, for a
method that's only called once the difference won't be very noticeable.
However, for methods that are called many times the difference can be
more significant.
For example, the loading time of a large commit
(nrclark/dummy_project@81ebdea5df)
was reduced from around 19 seconds to around 15 seconds using these
changes.
* upstream/master: (45 commits)
Replace reject_blocked with reject_blocked! in callbacks.
Fix Project#to_param to keep invalid project suitable for use in URLs
Update CHANGELOG
Add feature specs for edit project settings
Fix renaming repository when name contains invalid chars under settings
Change requests_profiles resource constraint to catch virtually any file
Allow skipping users in autocomplete
Fix typo in CHANGELOG
Update CHANGELOG
Respective cache is now expired when creating a new branch
Update CHANGELOG
Unify HTML format in static error pages
Make error pages responsive design
Move color-logic into HipchatService#HipchatService
Depened on exact version of SimpleCov when patched
Refactor spam validation to a concern that can be easily reused and improve legibility in `SpamCheckService`
Refactor `SpamCheckService` to make it cleaner and clearer.
Submit all issues on public projects to Akismet if enabled.
Submit new issues created via the WebUI by non project members to Akismet for spam check.
Upgrade Bullet from 5.0.0 to 5.2.0.
...
* upstream/master: (620 commits)
Added '*.js.es6 gitlab-language=javascript' to .gitattributes
Fix CI status icon link underline
Update CHANGELOG after 8.10.1
Add CHANGELOG
Add es6 gem
Instrument Nokogiri parsing methods
Fix backup restore
Use project ID in repository cache to prevent stale data from persisting across projects
Add iid to MR API response
`WikiPage` should have a slug even when not persisted.
ES6ify all the things!
Make fork counter always clickable (!5463)
Revert "Merge branch '17073-tagscontroller-index-is-terrible-response-time-goes-up-to-5-…"
Fix CHANGELOG
Add spec for dashes in paths
Fix Error 500 when creating Wiki pages with hyphens or spaces
Add links to the real markdown.md file for all GFM examples
Remove magic comments from Ruby files (!5456)
Ignore invalid trusted proxies in X-Forwarded-For header
remove search_id for label dropdown filter
...
* master: (183 commits)
Add a spec for #20079.
Skip repository storage path valitaions on test environment
Use Pathname to make the repository storage path validations more robust
Update to gitlab_git 10.4.1 and take advantage of preserved Ref objects
Change nav link snippet controller
Reduce min width of pipeline table
Retrieve rendered HTML from cache in one request
Explain CI_PROJECT_NAMESPACE better
Bump vmstat version to fix issues reporting on FreeBSD
Fix sha icon positioning on safari
Don't drop in DropAndReaddHasExternalWikiInProjects
Mobile view for commit status
Fix ci icons getting cut off
Update CHANGELOG
Extract helper methods to clean up RepositoryArchiveCleanUpService spec
Use Dir.mktmpdir instead of FileUtils.mkdir_p in the spec
Fix firefox rendering of SVGs
Fix icons on commits page and builds page
Add new fork SVG to fix weird styling of other SVGs
Bug fixes
...
Add support for inline videos in issue, MR and notes (on issue, commit, MR, and MR diff)
## What does this MR do?
It adds support for inline videos in issue, MR and notes (on issue, commit, MR, and MR diff). Most of the work was done by @hayesr in !3508 but a few improvements were still missing.
## Why was this MR needed?
To be able to play uploaded videos in GitLab!
## What are the relevant issue numbers?
Closes#4142.
## Screenshots
### Video players
-----
-----
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
- [x] Test `VideoLinkFilter`
- [x] Test in `spec/features/markdown_spec.rb`
- [x] Improve `spec/uploaders/file_uploader_spec.rb`
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !5215
Added checks for migration downtime
This adds a set of checks that check/list which migrations require downtime (or not). It also comes with a CI task that fails should a migration not be tagged properly.
Fixes#14545
See merge request !4911
These new checks can be used to check if migrations require downtime or
not (as tagged by their authors). In CI this compares the current branch
with master so migrations added by merge requests are automatically
verified.
To check the migrations added since a Git reference simply run:
bundle exec rake gitlab:db:downtime_check[GIT_REF]
* master: (321 commits)
Fix the Sentry spam from CSP violations by disabling it.
Limit git rev-list output count to one in forced push check
Ensure Owners are included in the scope for authorized_projects
Fix alignment of icons on project page
Fix ci_status_helper_spec to look for new SVGs
use 2.0.5, actually (2.0.4 was a bad release)
upgrade rouge to 2.0.4
Fix help page paths to make sure shortcuts and the UI help page work.
fixes an issue cause by a bad merge
Vertically align status icon within table
Add new icons for every CI status
Add global style for running icon
Align running icon in merge request
Add new running icon; add a bunch of styles to get svg to match existing fa icons
Improve code design
Fix broken builds_for_ref
Move when tests before to make it no conflict with manual-actions
Use value of `yaml_variables` and `when` from config_processor if undefined
Add CHANGELOG entry
CHANGELOG item
...
Conflicts:
lib/ci/gitlab_ci_yaml_processor.rb
spec/lib/ci/gitlab_ci_yaml_processor_spec.rb
added spec for avatar saver
avatar saver!
added avatar restorer spec
fix spec
added avatar restorer class
fix export service
fix warnings, added changelog
fix spec
some refactoring based on feedback
fixed a few issues after testing i/e avatar
WIP - trying to replicate UTF-8 error
fix spec
fixing encoding issue and another spec, to do with MR diffs
fix issue and spec failure
Add changelog and bumped up I/E version
fix spec based on feedback - omitted target project
* master: (522 commits)
Fix CI yaml example
Align cancel and retry buttons
Remove deploy to production button
Fix a bug where the project's repository path was returned instead of the wiki path
Don't fail to highlight when Rouge doesn't have a lexer
Revert "Merge branch 'gl-dropdown-issuable-form' into 'master'"
Update tests
Don't fail when Ci::Pipeline doesn't have a project
Don't fail when a LegacyDiffNote didn't store the right diff
Update CHANGELOG
Use cattr_accessor instead duplicating code on NoteOnDiff concern
Fix mentioned users list on diff notes
Don't ask Heather to review documentation MR's
add project name and namespace to filename on project export
navbar_icon was renamed to custom_icon in:
use %(...) and %[...] in favor of %<...>
Fix spec Don't attempt to disable statement timeout on a MySQL DB
Disable statement timeout outside of transaction and during adding concurrent index
Disable PostgreSQL statement timeout during migrations
Add visibility icon
...
Upgrade to Rouge 2.0
## What does this MR do?
This MR is a refactor of the HTMLGitlab formatter. The original was copy-pasted from the legacy HTML formatter and adapted to suit our needs. In particular, this MR:
* Strips dead code and unused options
* Factors out the "wrapping" into the places that care about it - Bonzai and the view helpers
* Uses the Rouge 2.0 `token_lines` method to split tokens into lines, removing the custom code we used to do the same
* Uses the now-public `span` method instead of re-implementing it ourselves
* Removes options to not split into lines - the places where this feature wasn't being used are now using the plain `Formatters::HTML` instead of this class
See merge request !4691
* upstream/master:
navbar_icon was renamed to custom_icon in:
Fix spec Don't attempt to disable statement timeout on a MySQL DB
Disable statement timeout outside of transaction and during adding concurrent index
Disable PostgreSQL statement timeout during migrations
Add visibility icon
Remove previously introduced CSS that will not be used anymore
Make admin/groups view consistent with dashboard/groups
Fix New Group button spacing on mobile
Make CSS consistent again
Rename .group-controls to .controls
Remove unnecesary CSS class
Change bg color of collapsed diff to blue on hover
Reduce padding on collapsed diff message
updated changelog
fixed similar issue with gitlab.com importer because why not!
fix updated_at not preserved after import - for GitLab projects
Update permissons links to new page
CHANGELOG
Changed collapsed assignee tooltip to users name Fixes tooltip when updating the assignee
Refresh branch cache after `git gc`
Disable PostgreSQL statement timeout during migrations
Long-running migrations may take more than the timeout allowed by the database. Disable the session's statement timeout to ensure migrations don't get killed prematurely.
See merge request !5263
* upstream/master: (1547 commits)
Add margin between buttons if both retry and cancel are present
Add margin between labels; remove underline hover style on status button
udpated JS based on feedback
Use default cursor for table header of project files (!5165)
Fix duplicated entry in changelog [ci skip]
Improves left static sidebar behaviour
Include default callback URL (OAuth)
Cleanup feature proposal template
Simplify regex for string-based multi-word label surrounded in quotes
Revert "Merge branch '18193-developers-can-merge' into 'master'
"
Upgrade Rails from 4.2.6 to 4.2.7.
some JS magic to fix empty URL bug
formats my test properly
Update CHANGELOG
Doesn't match empty label references surrounded in quotes
Fix markdown rendering for label references that contains `.`
Fix markdown rendering for label references that begin with a digit
Fix markdown rendering for consecutive label references
Stub omniauth provider for GitLab
Update CHANGELOG
...
1. Don't use case statements for dispatch anymore. This leads to a lot
of duplication, and makes the logic harder to follow.
2. Remove duplicated logic.
- For example, the `can_push_to_branch?` exists, but we also have a
different way of checking the same condition within `change_access_check`.
- This kind of duplication is removed, and the `can_push_to_branch?`
method is used in both places.
3. Move checks returning true/false to `UserAccess`.
- All public methods in `GitAccess` now return an instance of
`GitAccessStatus`. Previously, some methods would return
true/false as well, which was confusing.
- It makes sense for these kinds of checks to be at the level of a
user, so the `UserAccess` class was repurposed for this. The prior
`UserAccess.allowed?` classmethod is converted into an instance
method.
- All external uses of these checks have been migrated to use the
`UserAccess` class
4. Move the "change_access_check" into a separate class.
- Create the `GitAccess::ChangeAccessCheck` class to run these
checks, which are quite substantial.
- `ChangeAccessCheck` returns an instance of `GitAccessStatus` as
well.
5. Break out the boolean logic in `ChangeAccessCheck` into `if/else`
chains - this seems more readable.
6. I can understand that this might look like overkill for !4892, but I
think this is a good opportunity to clean it up.
- http://martinfowler.com/bliki/OpportunisticRefactoring.html
1. When a merge request is being merged, save the merge commit SHA in
the `in_progress_merge_commit_sha` database column.
2. The `pre-receive` hook looks for any locked (in progress) merge
request with `in_progress_merge_commit_sha` matching the `newrev` it
is passed.
3. If it finds a matching MR, the merge is legitimate.
4. Update `git_access_spec` to test the behaviour we added here. Also
refactored this spec a bit to make it easier to add more contexts / conditions.
Fix: Infinity Bug in Commit Statistics
## What does this MR do?
It fixes a logic bug in the commits statistics: The code assumed that the amount of days involved in a commit range is equal to the difference between the first and last date. This is not true, though, as (from a human standpoint), a commit yesterday and a commit today involve two days, not one. Similarly, a fresh project with only commits made today already 'used' one day.
Since the number of involved days used to be zero for new projects, the result for commits per day quite often amounted to `Infinity`…
## Are there points in the code the reviewer needs to double check?
The test file. I hope it is up to the standards of GitLab.
## Why was this MR needed?
The bug occurres especially for new users with their first project while exploring GitLab.
## What are the relevant issue numbers?
This bug was reported as #1548.
## Screenshots (if relevant)
See merge request !4231
Allow specifying protected branches using wildcards
Closes#18627
# Tasks
- [ ] #18627!4665 Allow specifying protected branches using wildcards
- [x] Find existing usages of protected branches
- Protecting branches
- `ProtectedBranchesController` is used to mark a branch protected/unprotected
- `API::Branches` can be used to mark a branch protected/unprotected
- Enforcing branch protection
- `Gitlab::GitAccess` has helpers (`can_push_to_branch?`, `check`) that are used to deny pushes if a branch is protected
- Over SSH: `gitlab-shell` receives a push, and calls `/allowed` on the GitLab API, which calls `GitAccess.check`
- Over HTTP:
- `gitlab-workhorse` receives the request, and forwards it to rails
- Rails (in the `GitHttpController#git-recieve-pack`) runs basic checks (is the user logged in, not protected branch checks) and returns ok with `GL_ID` and `RepoPath`
- `gitlab-workhorse` looks at the response, and calls the relevant `gitlab-shell` action from `git-http/handlePostRPC`
- Rest of this flow is the same as the SSH flow above
- [x] Implementation
- [x] Backend
- [x] Change `project#protected_branch?` to look at wildcard protected branches
- [x] Change `project#developers_can_push_to_protected_branch?`
- [x] Change `project#open_branches`
- [x] Better error message when creating a disallowed branch from the Web UI
- [x] Frontend
- [x] Protected branches page should allow typing out a wildcard pattern
- [x] Add help text explaining the use of wildcards
- [x] Show matching branches for each protected branch
- [x] ~~On the index page~~
- [x] On a show page
- [x] Index?
- [x] Can't have the "last commit" column for wildcard protected branches
- [x] Fix / write tests
- [x] What happens if a hook is missing in dev?
- [x] Refactor
- [x] Test workflows
- Create a branch matching a wildcard pattern
- Push to a branch matching a wildcard pattern
- Force push to a branch matching a wildcard pattern
- Delete a branch matching a wildcard pattern
- [x] Test using Web UI
- [x] Test over SSH
- [x] Test over HTTP
- [x] Test as developer and master
- [x] Investigate performance
- [x] Test with a large number of protected branches / branches
- [x] Paginate list of protected branches
- [x] ~~Possibly rewrite `open_branches`~~
- [x] Add `iid`s to existing `ProtectedBranch`es
- [x] Add documentation
- [x] Add CHANGELOG entry
- [x] Add screenshots
- [x] Make sure [build](2f753e3ed2/builds) passes
- [x] Assign to endboss for review
- [x] Address @DouweM's comments
- [x] `protected_branch_params`
- [x] `exact_match` instead of `explicit_match`
- [x] When would self.name be blank?
- [x] Move `protected_branches.each` to a partial
- [x] Move `matching_branches.each` to a partial
- [x] If the branch is in @matching_branches, it's not been removed
- [x] move this regex to a method and memoize it
- [x] `commit_sha` directly for exact matches
- [x] Number of matches for wildcard matches, with a link
- [x] Wait for [build](43f9ce0e88/builds) to pass
- [x] Respond to @DouweM's comments
- [x] Don't use iid
- [x] Controller should use `@project.protected_branches.new`
- [x] move the memoization to `def wildcard_regex`
- [x] render with `collection: @protected_branches`
- [x] Wait for [build](f7beedf122/builds) to pass
- [x] Wait for @DouweM's review
- [x] Wait for @jschatz1's review
- [x] Respond to @jschatz1's comments
- [x] Use the new dropdown style
- [x] description should be moved to the description section without the styling
- [x] Protect button should be disabled when no branch is selected
- [x] Update screenshots
- [x] Merge conflicts
- [x] Make sure [build](20f3cfe8d5/builds) passes
- [ ] Revisit performance, possibly with staging/production data
- [ ] Get a dump of staging / run against staging live
- [ ] Get SSH access to staging
- [ ] Wait for review/merge
# Screenshots
## Creating wildcard protected branches
### Using the `GLDropdown` component
## Enforcing wildcard protected branches
### From the Web UI
### Over SSH
### Over HTTPS
## Listing matching branches
See merge request !4665
Move global ci entries handling from legacy to new config
## What does this MR do?
This MR moves responsibility of handling global CI config entries (like `image`, `services`), from legacy `GitlabCiYamlProcessor` to new CI Config
## Why was this MR needed?
This is the next iteration of CI configuration refactoring
## What are the relevant issue numbers?
#15060
## Does this MR meet the acceptance criteria?
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
See merge request !4820
Note: This feature was developed independently on master while this was
in review. I've removed the conflicting bits and left the relevant
additions, mainly a test for `Gitlab::Git::Hook`. The original commit
message follows:
1. `gitlab-shell` outputs errors to `stderr`, but we weren't using this
information, prior to this commit. Now we capture the `stderr`, and
display it in the flash message when branch creation fails.
2. This can be used to display better errors for other git operation
failures with small tweaks.
3. The return value of `Gitlab::Git::Hook#trigger` is changed from a
simple `true`/`false` to a tuple of `[status, errors]`. All usages
and tests have been updated to reflect this change.
4. This is only relevant to branch creation _from the Web UI_, since SSH
and HTTP pushes access `gitlab-shell` either directly or through
`gitlab-workhorse`.
5. A few minor changes need to be made on the `gitlab-shell` end. Right
now, the `stderr` message it outputs is prefixed by "GitLab: ", which
shows up in our flash message. This is better removed.
* master: (98 commits)
Enable Style/EmptyLines cop, remove redundant ones
Update CHANGELOG
Cache results from jQuery selectors to retrieve namespace name
Fix import button when import fail due the namespace already been taken
Fix snippets comments not displayed
Fix emoji paths in relative root configurations
Exclude requesters from Project#members, Group#members and User#members
Upgrade Thin from 1.6.1 to 1.7.0.
Many squashed commits
Cache autocomplete results
Upgrade Sidekiq from 4.1.2 to 4.1.4.
Upgrade seed-fu from 2.3.5 to 2.3.6
use has_many relationship with events
Support creating a todo on issuables via API
Expose target, filter by state as string
Add todos API documentation and changelog
Improve the request / withdraw access button
Metrics for Rouge::Plugins::Redcarpet and Rouge::Formatters::HTMLGitlab
Groundwork for Kerberos SPNEGO (EE feature)
Update CHANGELOG 8.9.5 for runners related fixes
...
Fix emoji paths in relative root configurations
## What does this MR do?
If a site specifies a relative URL root, emoji files would omit the path from the URL, leading to lots of 404s.
## Are there points in the code the reviewer needs to double check?
At first, I tried to use `ActionView::Helpers::AssetUrlHelper.asset_url` since this is what it's intended to do. But this helper function is extremely slow, and it took minutes to generate the URLs for the hundreds of links needed for each emoji.
## Why was this MR needed?
Because emojis were broken in relative URL installations
## What are the relevant issue numbers?
#15642
## Does this MR meet the acceptance criteria?
- [X] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
- [X] Added for this feature/bug
- [x] All builds are passing
- [X] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [X] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [X] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !5027
This stands as an alternative to using OAuth to access a user's Github
repositories. This is setup in such a way that it can be used without OAuth
configuration.
From a UI perspective, the how to import modal has been replaced by a full
page, which includes a form for posting a personal access token back to the
Import::GithubController.
If the user has logged in via GitHub, skip the Personal Access Token and go
directly to Github for an access token via OAuth.
Enable Style/UnneededCapitalW Rubocop cop
## What does this MR do?
This MR enables Rubocop cop that checks if `%W[]`, which supports interpolation, is needed.
## What are the relevant issue numbers?
#17478
See merge request !5010
If Redis were not running or USE_DB were set to false, the
application settings retrieval would fail completely. This
change only attempts to use the cache if the system actually
wants to connect to the DB and rescues any failures in talking to
Redis.
Closes#17557
Process.clock_gettime allows getting the real time in nanoseconds as
well as allowing one to get a monotonic timestamp. This offers greater
accuracy without the overhead of having to allocate a Time instance. In
general using Time.now/Time.new is about 2x slower than using
Process.clock_gettime(). For example:
require 'benchmark/ips'
Benchmark.ips do |bench|
bench.report 'Time.now' do
Time.now.to_f
end
bench.report 'clock_gettime' do
Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond)
end
bench.compare!
end
Running this benchmark gives:
Calculating -------------------------------------
Time.now 108.052k i/100ms
clock_gettime 125.984k i/100ms
-------------------------------------------------
Time.now 2.343M (± 7.1%) i/s - 11.670M
clock_gettime 4.979M (± 0.8%) i/s - 24.945M
Comparison:
clock_gettime: 4979393.8 i/s
Time.now: 2342986.8 i/s - 2.13x slower
Another benefit of using Process.clock_gettime() is that we can simplify
the code a bit since it can give timestamps in nanoseconds out of the
box.
* master: (352 commits)
Display last commit of deleted branch in push events (!4699)
add changelog
add missing attribute to attr_encrypted so it is fully backwards-compatible
Add "GitLab team members only" to diagram link
doc: note that .gitattributes uses default branch
use the conf lexer so we have highlighted comments
first draft of docs
support cgi style options, such as erb?parent=json
move the path alias to a more appropriate location
make #custom_language private
appease rubocop
add an alias for Snippet#path
appease rubocop
check the tag so that an instance will pass too
fix the spec, using project.change_head
Revert "bump the master sha for gitlab-test!9"
bump the master sha for gitlab-test!9
add custom highlighting via .gitattributes
Rename Licenses API to License Templates API
Check for conflict with wiki projects when creating a new project.
...
Improve validations and error handling in new CI config entries
## What does this MR do?
This MR improves validation in new CI config.
## Why was this MR needed?
With that it will be easier to handle errors during validation and post-processing.
## What are the relevant issue numbers?
This is a continuation of #15060
See merge request !4560
We assume that when someone adds a key for the configuration entry, but
does not provide a valid value, which causes entry to be `nil`, then
entry should be considered as the undefined one. We also assume this is
semantically correct, this is also backwards compatible with legacy CI
config processor.
See issue #18775 for more details.
Fix subsequent SAML sign ins
Fixes a bug when `auto_link_ldap_user` is `true` that causes SAML users to be unable to sign in a second time.
Fix the problem for https://gitlab.zendesk.com/agent/tickets/22546
See merge request !4718
Allow to fetch LFS from CI
## What does this MR do?
This adds support for fetching LFS object from CI jobs (mostly it's made for supporting GitLab CI).
## What is left?
- [x] Write tests covering a new authorization mechanism
cc @grzesiek @marin
See merge request !4465
* master: (345 commits)
use rails root join
fixed a couple of errors spotted in production
Fix RangeError exceptions when referring to issues or merge requests outside of max database values
Fix bug in `WikiLinkFilter`.
Small frontend code fixes and restore 8a2d88f commit
Warn about admin privilege to disable GitHub Webhooks
Listing GH Webhooks doesn't stop import process for non GH admin users
fixup! updated docs for api endpoint award emoji
Update CHANGELOG
Ensure Todos counters doesn't count Todos for projects pending delete
Add endpoints for award emoji on notes
Sort API endpoints and implement feedback
Add endpoints for Award Emoji
Fixed issue with assignee dropdown not selecting correctly
Removed update method Re-structured controller spec Renamed issuable param to issuable_id
Fix clibpoard buttons on "Check out branch" modal.
Track method call times/counts as a single metric
Cache todo counters (pending/done)
Fix a 'wrong number of arguments' error
Added missing mount point for Sidekiq Metrics API, after it got lost on rebase.
...
Previously we'd create a separate Metric instance for every method call
that would exceed the method call threshold. This is problematic because
it doesn't provide us with information to accurately get the _total_
execution time of a particular method. For example, if the method
"Foo#bar" was called 4 times with a runtime of ~10 milliseconds we'd end
up with 4 different Metric instances. If we were to then get the
average/95th percentile/etc of the timings this would be roughly 10
milliseconds. However, the _actual_ total time spent in this method
would be around 40 milliseconds.
To solve this problem we now create a single Metric instance per method.
This Metric instance contains the _total_ real/CPU time and the call
count for every instrumented method.
Allow customising of queries used for `update_column_in_batches`
This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:
1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain
In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.
See merge request !4680
* master: (189 commits)
Update CHANGELOG for !4659
Center the header logo for all Devise emails
Add previews for all customized Devise emails
Customize the Devise `unlock_instructions` email
Customize the Devise `reset_password_instructions` email
Customize the Devise `password_change` emails
Use gitlab-git 10.2.0
Use Git cached counters on project show page
Fix indentation scss-lint errors
Added title attribute to enties in tree view Closes#18353
Banzai::Filter::ExternalLinkFilter use XPath
Reduce queries in IssueReferenceFilter
Use gitlab_git 10.1.4
Fixed ordering in Project.find_with_namespace
Fix images in emails
Banzai::Filter::UploadLinkFilter use XPath
Turn Group#owners into a has_many association
Make project_id nullable
...
This follows a standard `ActiveModel` pattern of creating a custom
validators. We use `ActiveModel::EachValidator` here that reuses methods
provided by `LegacyValidationHelpers`.
We will remove `LegacyValidationHelpers` on some point in the future, at
the later stages of CI configuration refactoring. It may be possible
to rewrite custom validators to use format like:
`validates :config, array_of: String`
This ensures that whatever locks are acquired aren't held onto until the
end of the transaction (= after _all_ rows have been updated). Timing
wise there's also no difference between using a transaction and not
using one.
By passing a block to update_column_in_batches() one can now customize
the queries executed. This in turn can be used to only update a specific
set of rows instead of simply all the rows in the table.
* master: (1246 commits)
Update CHANGELOG
Update tests to make it work with Turbolinks approach
Use Turbolink instead of ajax
Reinitialize checkboxes to toggle event bindings
Turn off handlers before binding events
Removed console.log Uses outerWidth instead of width
Revert "Added API endpoint for Sidekiq Metrics"
Added API endpoint for Sidekiq Metrics
Added CHANGELOG entry for allocations Gem/name fix
Filter out classes without names in the sampler
Update the allocations Gem to 1.0.5
Put all sidebar icons in fixed width container
Instrument private/protected methods
Fix Ci::Build#artifacts_expire_in= when assigning invalid duration
Fix grammar and syntax
Update CI API docs
UI and copywriting improvements
Factorize members mails into a new Emails::Members module
Factorize access request routes into a new :access_requestable route concern
Factorize #request_access and #approve_access_request into a new AccessRequestActions controller concern
...
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
Generating the following tags
Grape#GET /projects/:id/archive
from Grape::Route objects like
{ :path => /:version/projects/:id/archive(.:format)
:version => “v3”,
:method => “GET” }
Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of
endpoints of the API, not with exact different requests
We can store this cache as an instance variable because
middleware are initialised only once
Add global entry with before script to new CI config
## What does this MR do?
This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.
See #15060
See merge request !4482
Improve Gitlab::Auth method names
Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.
See merge request !4589
* master: (59 commits)
Improved SVG sanitizer specs to include smoke tests for clean.
Refactored SVG sanitizer
Added SVG sanitizer fix to the changelog
Refactor SVG sanitizer and prevent `xlink:href` to refer to external resources
Fix SVG whitelisting to allow namespaced attributes
Fix Error 500 when using closes_issues API with an external issue tracker
Center layout navigation and remove icons
Fix preferences_spec test
Add back sidebar counters and username
Only create the backup directory if it is local
Fix safari logo loading animation safari bug
Revert side nav to full width; remove border under nav; remove tooltips on nav links; stop page content shifting with side nav; put project nav in container
Update media queries
Fix profile test
Fix logo at all screen widths, update sidebar text
Move tanuki icon to center of nav bar; keep nav closed by default; remove collapsed nav cookie
Remove unused MergeRequest#gitlab_merge_status method
Add CHANGELOG item for labels/milestones navigation change
Render issues link on issues subnav unless you visit merge request controller
Render only issues/mr in subnav depends on context
...
* master: (147 commits)
Minor MR comment fixes.
Update CHANGELOG for 8.8.4 and 8.8.5
Properly quote table name in Rake task for MySQL and PostgreSQL compatibility
Checks based on whether data is loaded not undefined
Checks for undefined when inserting autocomplete into textarea
Ignore frequent emojis in search.
Fixed tests
CHANGELOG
Improved the UX of issue & milestone date picker
Change date format to be non zero padded in order to fix failing test
Update method name for better understanding
Add tests for dates on tooltips
Fix local timeago on user dashboard
Update CHANGELOG
Toggling a task in a description with mentions doesn't creates a Todo
Update CHANGELOG
Fixed failing label subscribe test
Tests update
Updated subscribe icon
Fixed failing tests
...
Adjust the SAML control flow to allow LDAP identities to be added to an existing SAML user.
## What does this MR do?
It correctly lets an existing SAML user to add their LDAP identity automatically at login.
## Why was this MR needed?
A customer had issues with the `auto_link_ldap_user` feature. The flow was not working if there was an account with a SAML identity, but no LDAP identity. GitLab would pick up the correct LDAP person, but due to the order of the flow, that LDAP person was never associated with the user.
## What are the relevant issue numbers?
Fixes#17346
/cc @dblessing @balameb @stanhu
See merge request !4498
* master: (285 commits)
Bump recaptcha gem to 3.0.0 to remove deprecated stoken support
Load knapsack in Rakefile only when is bundled
Add License Finder information to contribution acceptance criteria.
Add LGPLv2 to license whiltelist
Instrument `RepositoryCheck::SingleRepositoryWorker` manually
Bump nokogiri to 1.6.8
Fix alignment of wiki top area
Update charcoal theme colors
Update nav link font size and spacing; fix hamburger icon
Fix control btn position
Remove todos count tests in nav
Test impersonation using img data attribute instead of username
Implement compact side nav
Fix knapsack for master
Align links and tabs
Add scrolling tabs to code subnav
Finish styling sub nav
Updated colors
Fixed failing tests
CHANGELOG item
...
Conflicts:
lib/gitlab/ci/config.rb
spec/lib/gitlab/ci/config_spec.rb
Add new GitLab CI configuration facade class
## What does this MR do?
This MR is a first iteration of major CI configuration refactoring. See #17139 and #15060 for more details.
## What are the relevant issue numbers?
Closes#17139
See merge request !4462
Rename Ci::Commit in specs
This is second iteration of renaming Ci::Commit occurences, this time I try to rename all `let's` and `factories` to match what type of object is created:
1. Rename factories: `ci_[...]_commit_[...]` to `ci_[...]_pipeline_[...]`,
2. Rename all `let's(:[ci_]commit)` to `let(:pipeline)`,
3. Rename all occurrences of `[ci_]commit` to `pipeline` if Pipeline object is created
cc @grzesiek @DouweM
See merge request !4463
Extend specs for builds badge
## What does this MR do?
This MR extends specs for builds badge.
## Why was this MR needed?
We added an edge case to specs, while trying to reproduce problem described in #17549
## What are the relevant issue numbers?
#17549
See merge request !4401
As for now, we keep this class inside a oryginal config processor class.
We will move implementation to this class and delegate to it from
current config processor.
After original gitlab ci yaml processor not longer has relevant
impelemntation we will replace it with new configuration class.
Separate reference gathering from rendering
This is a required step to allow batch processing when gathering references. This in turn would allow grabbing (for example) all mentioned users of an issue/merge request using a single query.
cc @rspeicher @DouweM
See merge request !3969
This class can be used to lazy-evaluate blocks of code the first time
they're called. This can be useful when a method performs a certain
heavy operation (e.g. a SQL query) that you only want to perform
whenever the result is used for the first time.
Added helper methods for database migrations
These helpers can ultimately be used to write migrations that don't
require downtime.
See #15464 for more information.
See merge request !3860
Rémy Coutable
James Lopez
Grzegorz Bizon
Paco Guzman
Lin Jen-Shin
tiagonbotelho
Stan Hu
Timothy Andrew
Douwe Maan
Yorick Peterse
Patricio Cano
Robert Speicher
Douwe Maan
Sean McGivern
Kamil Trzcinski
http://jneen.net/
Robert Speicher
Douglas Barbosa Alexandre
Timothy Andrew
Mathias Vestergaard
Tomasz Maczukin
Eric K Idema
Alejandro Rodríguez
Drew Blessing
ZJ van de Weg
Ilan Shamir
Jacob Vosmaer
Gabriel Mazetto
Jacob Schatz
Felipe Artur
DJ Mountney
Jonas Weber
Rubén Dávila
Fatih Acet